Explore Projects
Discover 96 open source projects
Showing 41-60 of 96 projects
striver-ing/wechat-spider
A WeChat spider that crawls articles, reading counts, likes, and comments from public accounts.
any4ai/AnyCrawl
AnyCrawl is a Node.js/TypeScript web scraper that extracts structured data from search engines and websites for use in AI/LLM applications.
xnl-h4ck3r/waymore
A Python-based tool that aggregates data from various sources to help with open-source intelligence and security research.
oxylabs/oxylabs-ai-studio-py
AI-powered web scraping and data gathering SDK for building intelligent agents and LLM apps
transitive-bullshit/awesome-puppeteer
A curated list of awesome resources for the Puppeteer headless Chrome/Chromium automation library.
lorien/grab
A powerful web scraping framework for Python that supports asynchronous crawling and flexible data extraction.
dw-dengwei/daily-arXiv-ai-enhanced
Daily arXiv paper crawler with AI summaries & GitHub Pages visualization for research discovery.
Barabama/FreeNodes
This is a Python script that crawls and automatically updates free proxy nodes for v2ray, clash, and other protocols.
spider-rs/spider
A powerful web scraping and crawling library for Rust developers
brightdata/brightdata-mcp
A powerful MCP server that provides an all-in-one solution for public web access and data extraction.
ReaJason/xhs
A Python-based library for scraping data from the Xiaohongshu (Little Red Book) e-commerce platform.
coleam00/mcp-crawl4ai-rag
A Python library for web crawling and RAG capabilities for AI agents and AI coding assistants.
shuaidaoya/FreeNodes
A tool that automatically crawls and shares free proxy nodes for various proxy software like v2ray and clash.
NateScarlet/holiday-cn
A Python tool for automatically scraping data on China's statutory holidays from government announcements.
hu17889/go_spider
A flexible and modular Go-based web crawler framework with a concurrent architecture.
watercrawl/WaterCrawl
A versatile TypeScript-based tool that transforms web content into LLM-ready data for AI/ML applications.
yhangf/PythonCrawler
A collection of Python web crawling projects for developers interested in building web scrapers and spiders.
coder-hxl/x-crawl
Flexible and AI-assisted Node.js crawler library for building web scrapers and crawlers.
howie6879/ruia
An async Python 3.6+ web scraping micro-framework based on asyncio for building high-performance crawlers and spiders.
thecodrr/fdir
Fast directory crawler and globbing library for NodeJS
Stay in the loop
Get weekly updates on trending AI coding tools and projects.