ETL & Pipelines
Explore 310 open source projects in ETL & Pipelines
Showing 121-140 of 310 projects
ptwobrussell/Mining-the-Social-Web-2nd-Edition
An official compendium for the book 'Mining the Social Web' focused on web scraping and data analysis.
apache/gravitino
An open-source data catalog platform for building a high-performance, federated metadata lake.
microsoft/table-transformer
Deep learning model for extracting & analyzing table structures from PDFs and images with datasets.
ciur/papermerge
Self-hosted document management system with OCR for scanning and archiving papers digitally.
whylabs/whylogs
An open-source data logging library for machine learning models and data pipelines.
susanli2016/NLP-with-Python
A collection of Jupyter Notebooks demonstrating various NLP techniques and libraries like Scikit-Learn, NLTK, Spacy, and Gensim.
mars-project/mars
A unified framework for large-scale data computation that scales popular Python data tools like NumPy, Pandas, and Scikit-Learn.
datachain-ai/datachain
Comprehensive analytics, versioning, and ETL toolkit for multimodal data (video, audio, PDFs, images)
snakemake/snakemake
Snakemake is a workflow management system for reproducible and scalable data analysis.
jae-jae/QueryList
A progressive PHP crawler framework that allows developers to build elegant web scrapers and crawlers.
oxylabs/how-to-scrape-amazon-product-data
A Python-based web scraper for extracting Amazon product data like titles, ratings, prices, images, and descriptions.
justinzm/gopup
A Python data interface for various APIs, including economic and news data.
veb-101/Data-Science-Projects
A collection of data science projects in Python using Jupyter Notebook.
rilldata/rill
Rill is a tool for transforming data sets into powerful dashboards using SQL, enabling BI-as-code.
duckdb/ducklake
DuckLake is an integrated data lake and catalog format written in C++.
The-Japan-DataScientist-Society/100knocks-preprocess
A repository for the 100 Knocks of Data Science Preprocessing, focused on structured data processing.
EntilZha/PyFunctional
A Python library for creating data processing pipelines using functional programming principles.
taspinar/twitterscraper
A Python library for scraping tweets from Twitter, useful for data analysis and social media monitoring.
neilotoole/sq
sq is a Go-based data wrangling tool that supports a variety of data formats and databases.
timhutton/twitter-archive-parser
Python script to parse and export Twitter archive data in various formats.
Stay in the loop
Get weekly updates on trending AI coding tools and projects.