Explore Projects

Discover 170 open source projects

Active filters (1):
Search: crawler×
Clear all

Showing 141-160 of 170 projects

adyzng/jd-autobuy

A Python web scraper for automatically logging in and purchasing items on JD.com.

1.3K
Archived
Python
Backend & APIs
API Frameworks
Python
#crawler#scraper#jingdong

huaying/instagram-crawler

A Python library for scraping Instagram posts, profiles, and hashtag data without using the Instagram API.

1.3K
Archived
Python
Backend Frameworks
API Clients & Testing
#instagram#scraper#web-scraping

erma0/douyin

A TypeScript-based web scraper for collecting public data from the Douyin (TikTok) platform.

1.3K
Active
TypeScript
Backend Frameworks
Caching
TypeScript
#web-scraper#douyin#tiktok

0xInfection/XSRFProbe

A toolkit for auditing and exploiting Cross-Site Request Forgery (CSRF) vulnerabilities in web applications.

1.3K
Experimental
Python
Security Research
#audit#csrf#csrf-attacks

dwisiswant0/go-dork

A fast and efficient dork scanner written in Go for bug bounty and security research.

1.3K
Archived
Go
Penetration Testing
CLI Tools
#dork-scanning#bugbounty#infosec

kiddyuchina/Beanbun

A PHP-based, multi-process network crawler framework with good openness and extensibility.

1.3K
Archived
PHP
Workerman
#crawler#php#spider

rebrowser/rebrowser-patches

Collection of patches to avoid automation detection and captchas for web scraping and crawling tools.

1.3K
Experimental
JavaScript
Backend & APIs
CLI Tools
Puppeteer
#automation#web-scraping#headless

marukrap/RoguelikeDevResources

A curated list of resources for developing roguelike games, a popular genre of procedurally-generated dungeon crawlers.

1.3K
Archived
Roguelike
#game-development#roguelike#procedural-generation

InstaPy/instagram-profilecrawl

A Python script that quickly crawls and extracts information from Instagram profiles.

1.3K
Archived
Python
Backend Frameworks
CLI Tools
#automation#crawler#instagram

ityard/python-fxxk-spider

A Python crawler project for scraping free resources.

1.3K
Archived
Scrapy
#crawler#python#requests

0x676e67/rnet

An ergonomic Python HTTP client with TLS fingerprinting capabilities for web scraping and crawling.

1.2K
Active
Rust
API Clients & Testing
Backend Frameworks
Rust
#http#https#tls

Zeal-L/BiliBili-Manga-Downloader

A Python-based GUI tool for downloading manga from Bilibili, supporting search, multi-thread download, and local management.

1.2K
Experimental
Python
Backend Frameworks
Component Libraries (Python)
Python
#manga#download#bilibili

seveniruby/AppCrawler

An open-source app crawler tool built with Appium and Scala for automated app testing and exploration.

1.2K
Archived
Scala
CLI Tools
Cross-Platform
#appium#crawler#app-testing

the-robot/sqliv

A powerful SQL injection vulnerability scanner for web applications written in Python.

1.2K
Archived
Python
Penetration Testing
CLI Tools
Python
#sql-injection#vulnerability-scanner#web-security

gildas-lormeau/single-file-cli

CLI tool for saving a complete web page as a single HTML file, useful for web archiving and scraping.

1.2K
Stable
JavaScript
CLI Tools
CLI Tools
Node.js
#web-scraping#web-archiving#cli

kkyon/botflow

A Python framework for building data pipelines, web crawlers, and quantitative trading applications.

1.2K
Archived
Python
API Frameworks
ETL & Pipelines
Python
#data-pipeline#web-crawler#quantitative-trading

chenjiandongx/mzitu

This is a Python web crawler to scrape photos of beautiful women from a popular Chinese photo site.

1.2K
Archived
Python
Backend Frameworks
Web Crawlers
#web-scraping#crawler#image-downloader

yutto-dev/bilili

A Python-based downloader for Bilibili videos, bangumi, and danmaku that supports multithreading and subtitle downloads.

1.2K
Archived
Python
Backend Frameworks
CLI Tools
Python
#bilibili#crawler#downloader

timwhitez/crawlergo_x_XRAY

A Python tool that combines crawlergo dynamic crawler with the XRAY passive scanning functionality to provide comprehensive web application security testing.

1.2K
Archived
Python
API Frameworks
CLI Tools
#security-testing#web-scanning#passive-scanning

JustinBeckwith/linkinator

A TypeScript-based tool for finding and fixing broken links in websites, documentation, and local files.

1.2K
Active
TypeScript
Backend & APIs
Testing
Node.js
#broken-links#link-checker#seo
1...79

Stay in the loop

Get weekly updates on trending AI coding tools and projects.