Explore Projects
Discover 170 open source projects
Showing 21-40 of 170 projects
code4craft/webmagic
A scalable web crawler framework for Java developers to build custom web scrapers and data extraction tools.
ssssssss-team/spider-flow
A no-code web crawler platform that allows developers to define crawling workflows visually without writing code.
injetlee/Python
Python scripts for web scraping, automating common tasks like login, Excel manipulation, and WeChat operations.
faisalman/ua-parser-js
A powerful user-agent detection library for client-side and server-side web development.
guyueyingmu/avbook
An adult video management system with a web crawler, database, and magnet link library for Japanese adult videos.
shmilylty/OneForAll
OneForAll is a powerful subdomain collection tool for security researchers and bug bounty hunters.
apify/crawlee-python
Crawlee is a powerful web scraping and browser automation library for Python to build reliable crawlers.
TeamWiseFlow/wiseflow
A Python-based platform that uses LLMs to track and extract websites, RSS feeds, and social media for developers.
lorien/awesome-web-scraping
A comprehensive list of libraries, tools, and APIs for web scraping and data processing.
andeya/pholcus
Pholcus is a high-concurrency web crawler software written in Go for developers needing a powerful, distributed crawling solution.
BruceDone/awesome-crawler
A comprehensive collection of web crawlers and scrapers in various programming languages.
alirezamika/autoscraper
A powerful, lightweight web scraping library for Python that can automate data extraction from websites.
adithya-s-k/omniparse
A Python library for ingesting, parsing, and optimizing any data format for enhanced compatibility with GenAI frameworks.
bda-research/node-crawler
A NodeJS-based web crawler/spider for extracting data from websites using cheerio and jQuery.
autoscrape-labs/pydoll
Pydoll is a Python library for automating chromium-based browsers without a WebDriver, offering realistic interactions.
subzeroid/instagrapi
A fast and powerful Python library for interacting with the Instagram Private API, including features like automation and scraping.
MontFerret/ferret
Declarative web scraping library written in Go, providing a powerful DSL for extracting data from websites.
00-Evan/shattered-pixel-dungeon
Shattered Pixel Dungeon is an open-source traditional roguelike dungeon crawler for Android and iOS.
yujiosaka/headless-chrome-crawler
A powerful, distributed web crawler powered by Headless Chrome for scraping websites at scale.
firecrawl/firecrawl-mcp-server
Firecrawl MCP Server adds powerful web scraping and search capabilities to AI language models like Cursor and Claude.
Stay in the loop
Get weekly updates on trending AI coding tools and projects.