Explore Projects
Discover 101 open source projects
Showing 21-40 of 101 projects
BruceDone/awesome-crawler
A comprehensive collection of web crawlers and scrapers in various programming languages.
bda-research/node-crawler
A NodeJS-based web crawler/spider for extracting data from websites using cheerio and jQuery.
SpiderClub/haipproxy
A high-availability distributed IP proxy pool powered by Scrapy and Redis for web crawling applications.
DropsDevopsOrg/ECommerceCrawlers
A collection of Python-based web crawlers for scraping data from various e-commerce and online platforms.
aui/font-spider
Smart webfont compression and format conversion tool for building fast and efficient web experiences.
wechatsync/Wechatsync
A TypeScript-based browser extension that allows syncing articles to multiple publishing platforms in one click.
tophubs/TopList
A Go-based web scraper that aggregates top headlines from major sites, useful for vibe coders building AI-powered apps.
201206030/novel-plus
This is a comprehensive content management system for novels, including features like recommendation, search, reading, and more.
dataabc/weibo-crawler
A Python-based web crawler that can scrape data, images, and videos from Weibo, a popular social media platform in China.
DedSecInside/TorBot
DedSecInside/TorBot is a dark web OSINT tool written in Python that crawls and extracts information from the Tor network.
Boris-code/feapder
A powerful Python-based web scraping framework with features like browser rendering and data deduplication.
elliotgao2/toapi
A Python library that makes it easy to create APIs for any website, simplifying web scraping and API development.
TheRook/subbrute
A Python tool to enumerate DNS records and subdomains for security testing and reconnaissance.
Gerapy/Gerapy
A distributed crawler management framework based on Scrapy, Scrapyd, Django, and Vue.js for web scraping.
my8100/scrapydweb
Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert for developers building with AI tools.
wnma3mz/wechat_articles_spider
A Python web scraper for extracting articles from WeChat official accounts
scrapy/scrapyd
A service daemon to run Scrapy spiders, a powerful web scraping library in Python.
5ime/video_spider
A PHP library for scraping and removing watermarks from various video platforms.
jaeles-project/gospider
Fast web spider written in Go for bug bounty
JAVClub/core
This repository is a web crawler for adult video streaming and torrent sites, not a developer tool.
Stay in the loop
Get weekly updates on trending AI coding tools and projects.