Explore Projects
Discover 170 open source projects
Showing 81-100 of 170 projects
oxylabs/oxylabs-ai-studio-py
AI-powered web scraping and data gathering SDK for building intelligent agents and LLM apps
xtuhcy/gecco
A lightweight web crawler built with Java for easy use
lorien/grab
A powerful web scraping framework for Python that supports asynchronous crawling and flexible data extraction.
dw-dengwei/daily-arXiv-ai-enhanced
Daily arXiv paper crawler with AI summaries & GitHub Pages visualization for research discovery.
friuns2/Leaked-GPTs
A collection of leaked GPT prompts and tools to bypass subscription limits and try out AI models.
Python3WebSpider/Python3WebSpider
A Python library for building web crawlers and spiders, suitable for vibe coders interested in web automation.
fhamborg/news-please
news-please is an integrated web crawler and information extractor for news that works out of the box.
decaywood/XueQiuSuperSpider
A Java-based stock information crawler for the XueQiu platform
JayBizzle/Crawler-Detect
A PHP library for detecting bots, crawlers, and spiders based on the user agent string.
lucasjinreal/weibo_terminater
A powerful Python-based web scraper for extracting data from Weibo, a popular Chinese social media platform.
sjdirect/abot
A cross-platform, fast, and flexible C# web crawler framework for developers building crawlers and spiders.
spider-rs/spider
A powerful web scraping and crawling library for Rust developers
simplecrawler/simplecrawler
Flexible event-driven web crawler for Node.js, useful for building custom web scraping solutions.
goclone-dev/goclone
A powerful Go-based website cloning tool that can clone websites to your computer within seconds.
zhegexiaohuozi/SeimiCrawler
A distributed, agile Java-based web crawler framework that can be used in SpringBoot applications.
rendora/rendora
Rendora is a dynamic server-side rendering solution using headless Chrome for SEO optimization.
BlankerL/DXY-COVID-19-Crawler
Real-time COVID-19 infection data crawler and API for developers tracking the pandemic.
Nekmo/dirhunt
A Python tool to find web directories without bruteforcing, useful for security researchers and penetration testers.
AAndyProgram/SCrawler
A media downloader for various social and adult platforms, including Twitter, Reddit, and OnlyFans.
Ehco1996/Python-crawler
This repository provides a systematic guide for learning how to build Python web crawlers.
Stay in the loop
Get weekly updates on trending AI coding tools and projects.