Explore Projects
Discover 255 open source projects
Showing 1-20 of 255 projects
firecrawl/firecrawl
Convert websites into LLM-ready data with API for scraping, crawling, and structured data extraction
scrapy/scrapy
Scrapy is a fast, high-level web crawling and scraping framework for Python developers.
soimort/you-get
Command-line media downloader for the Web
Mintplex-Labs/anything-llm
All-in-one AI app for local and remote LLM usage with RAG, agents, and MCP compatibility
NanmiCoder/MediaCrawler
Multi-platform social media crawler for content and comment scraping
dgtlmoon/changedetection.io
Website change detection and monitoring tool with notifications
feder-cr/Jobs_Applier_AI_Agent_AIHawk
AIHawk automates job applications using AI for tailored submissions.
gocolly/colly
Go Colly - Elegant Scraper and Crawler Framework for Golang
D4Vinci/Scrapling
Powerful, flexible Python library for effortless web scraping with AI-powered features.
jhao104/proxy_pool
Python proxy pool for web scraping with Redis storage
ScrapeGraphAI/Scrapegraph-ai
AI-powered web scraping library for extracting data from websites and documents
apify/crawlee
Web scraping and browser automation library for Node.js
soxoj/maigret
A comprehensive OSINT tool for collecting information on individuals from various online sources.
dzhng/deep-research
An AI-powered research assistant that performs iterative, deep research on any topic by combining search engines, web scraping, and large language models.
binux/pyspider
A powerful web crawler system in Python for building custom web scraping solutions.
Evil0ctal/Douyin_TikTok_Download_API
A high-performance async web scraping tool for extracting data from Douyin, TikTok, Bilibili and more.
twintproject/twint
An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API
Kr1s77/awesome-python-login-model
A Python script for simulating logins on various websites and scraping data with Selenium.
getmaxun/maxun
Turn websites into clean data pipelines & structured APIs in minutes with a low-code web scraping tool.
psf/requests-html
A Pythonic HTML parsing library that simplifies web scraping and interaction with HTTP resources.
Stay in the loop
Get weekly updates on trending AI coding tools and projects.