Explore Projects
Discover 255 open source projects
Showing 141-160 of 255 projects
howie6879/ruia
An async Python 3.6+ web scraping micro-framework based on asyncio for building high-performance crawlers and spiders.
librauee/Reptile
A comprehensive Python web scraping library covering a wide range of popular websites and platforms.
TheWebScrapingClub/webscraping-from-0-to-hero
A comprehensive resource for learning web scraping with Python, covering tools like Playwright, Scrapy, and Splash.
website-scraper/node-website-scraper
A Node.js library for scraping websites and downloading their contents locally.
Python3Spiders/WeiboSuperSpider
A powerful Python-based web crawler and toolkit for scraping Weibo data, including user profiles, comments, images, and more.
claffin/cloudproxy
A Python library that provisions proxy servers across cloud providers to improve web scraping success.
aivarsk/scrapy-proxies
A random proxy middleware for the Scrapy web scraping framework in Python.
Ge0rg3/requests-ip-rotator
A Python library to use AWS API Gateway's IP pool as a proxy for web scraping and security testing
apurvsinghgautam/dark-web-osint-tools
A collection of OSINT tools for exploring the dark web, including scraping, search, and data extraction capabilities.
scrapy/dirbot
A deprecated Python-based web scraping library for educational public web directories.
th3unkn0n/TeleGram-Scraper
A Python tool for scraping information from Telegram groups, including group member details.
megadose/OnionSearch
OnionSearch is a Python script that scrapes URLs from different .onion search engines for open-source intelligence.
justmarkham/DAT8
General Assembly's 2015 Data Science course covering topics like machine learning, data analysis, and data visualization.
saermart/DouyinLiveWebFetcher
A Python library for scraping real-time data from Douyin (TikTok) live streams, including comments and metadata.
probberechts/soccerdata
A Python library for scraping soccer data from various sources for sports analytics and data science.
paulpierre/informer
A Telegram bot for mass surveillance and data scraping, built with Python.
srx-2000/spider_collection
A collection of Python web scraping scripts for various websites and platforms, including music, video, and real estate data.
rushter/selectolax
A fast HTML5 parser with CSS selectors for Python, useful for web scraping and crawling tasks.
johntitus/node-horseman
Run PhantomJS, a headless web browser, from Node.js for web scraping, testing, and automation.
headzoo/surf
Stateful programmatic web browsing in Go, a powerful tool for automation and scraping.
Stay in the loop
Get weekly updates on trending AI coding tools and projects.