Web Crawler

This is a WebCrawler created with Beatufiul Soup and etc ! So we have some dependiceces for setup

$ git clone https://github.com/UtkucanBykl/WebCrawler.git
$ cd WebCrawler
$ pip install -r requirements.txt

And after check for the mongodb status if it is : True else:

$ sudo service mongod start

$ python main.py
$ cd WebCrawler

And visit the http://127.0.0.1:12345/

And crawle what you want.