Ebot
Ebot, an Opensource Web Crawler built on top of a nosql database (apache couchdb, riak), AMQP database (rabbitmq), webmachine and mochiweb. Ebot is written in Erlang and it is a very scalable, distribuited and highly configurable web cawler. See wiki pages for more details
Install / Use
/learn @matteoredaelli/EbotREADME
EBOT (http://www.redaelli.org/matteo-blog/projects/ebot/) USAGE: see the wiki homepage (http://wiki.github.com/matteoredaelli/ebot/) and ebot_test.erl for details LICENSE: GPL v3+, see file LICENSE Copyright (c) 2010, 2011 - Matteo Redaelli at libero dot it
Tested with Linux (Ubuntu and Debian) and Erlang R13B03 and R14A
- DOWNLOAD "TAG" releases and not the latest sources that could be unstable...
