Explore Projects
Discover 101 open source projects
Showing 61-80 of 101 projects
yhangf/PythonCrawler
A collection of Python web crawling projects for developers interested in building web scrapers and spiders.
howie6879/ruia
An async Python 3.6+ web scraping micro-framework based on asyncio for building high-performance crawlers and spiders.
DanMcInerney/xsscrapy
An open-source web crawler and spider tool for detecting cross-site scripting (XSS) vulnerabilities.
librauee/Reptile
A comprehensive Python web scraping library covering a wide range of popular websites and platforms.
TheWebScrapingClub/webscraping-from-0-to-hero
A comprehensive resource for learning web scraping with Python, covering tools like Playwright, Scrapy, and Splash.
ShunCai/QZoneExport
A Chrome extension to backup and export data from QQ Zone, including posts, albums, and more.
Python3Spiders/WeiboSuperSpider
A powerful Python-based web crawler and toolkit for scraping Weibo data, including user profiles, comments, images, and more.
chriskite/anemone
Anemone is a Ruby web-spider framework for crawling and extracting data from websites.
srx-2000/spider_collection
A collection of Python web scraping scripts for various websites and platforms, including music, video, and real estate data.
ArchiveTeam/grab-site
A web crawler tool that outputs WARC files and provides a dashboard for managing crawls.
keenwon/antcolony
A Node.js-based web scraper and crawler for finding and downloading torrent files.
OreosLab/checkinpanel
A check-in panel for developers using AI tools, supporting various platforms and environments.
darbra/sperm
This is a collection of interesting reverse engineering articles worth checking out.
monperrus/crawler-user-agents
A Go library that provides a database of syntactic patterns of HTTP user-agents used by bots/crawlers/scrapers.
xisuo67/XHS-Spider
A web scraping and data collection tool for the Chinese social media platform Xiaohongshu (Little Red Book).
mvdbos/php-spider
A configurable and extensible PHP web spider for crawling and extracting data from websites.
kgspider/crawler
A JavaScript web scraper repository focused on reverse engineering and advanced crawling techniques.
alongubkin/spider
A now-inactive JavaScript library for creating simple, unsurprising web crawling and scraping tools.
erma0/douyin
A TypeScript-based web scraper for collecting public data from the Douyin (TikTok) platform.
okfn-brasil/querido-diario
An open-source project that provides access to Brazilian government gazettes, enabling civic tech and data science applications.
Stay in the loop
Get weekly updates on trending AI coding tools and projects.