Deploy in 115+ regions with the modern database for every enterprise.
MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
Start Free
Earn up to 16% annual interest with Nexo.
Let your crypto work for you
Put idle assets to work with competitive interest rates, borrow without selling, and trade with precision. All in one platform.
Geographic restrictions, eligibility, and terms apply.
PHPCrawl is a high configurable webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.
HarvestMan is a fully functional, multithreaded webcrawler cum offline-browser. It is highly customizable and supports as much as 55 plus options for controlling and customizing offline browsing. It is written entirely in the Python programming language.
The CMS-Bandits is a set of php scripts, with online html editor, calendar, search engine, rss reader, revision log, personal nickpage, comment system, webcrawler and even more.
Crawler.NET is a component-based distributed framework for web traversal intended for the .NET platform. It comprises of loosely coupled units each realizing a specific web crawler task. The main design goals are efficiency and flexibility.
Mygale is a news-gathering webcrawler, written in Python. It searches a number of well-known news sites for Python-related articles. Currently doesn't support searching for other topics, but this may change in the future.