ETL & Pipelines
Explore 310 open source projects in ETL & Pipelines
Showing 181-200 of 310 projects
ptyadana/SQL-Data-Analysis-and-Visualization-Projects
This GitHub repository contains SQL data analysis and visualization projects using various tools and databases.
MarkPDFdown/markpdfdown
A high-quality PDF to Markdown conversion tool powered by large language model visual recognition.
jadianes/spark-py-notebooks
Apache Spark and Python tutorials for big data analysis and machine learning as Jupyter notebooks.
ClimbsRocks/auto_ml
Automated machine learning for analytics & production use cases powered by popular ML libraries.
jasonwei20/eda_nlp
Data augmentation for NLP using CNN and RNN, presented at EMNLP 2019
google/UIforETW
UIforETW is a C++ library for recording and managing ETW traces, providing a user interface for developers.
Hiflylabs/awesome-dbt
A curated list of awesome resources for the data transformation tool dbt, focused on analytics engineering.
Multiwoven/multiwoven
Open-source reverse ETL tool for data activation and customer data platform integration.
tansu-io/tansu
Apache Kafka-compatible broker with support for S3, PostgreSQL, SQLite, Apache Iceberg, and Delta Lake.
saermart/DouyinLiveWebFetcher
A Python library for scraping real-time data from Douyin (TikTok) live streams, including comments and metadata.
stripe-archive/mosql
A Ruby library that enables streaming replication from MongoDB to PostgreSQL databases.
cgarciae/pypeln
Concurrent data pipelines in Python for building efficient and scalable data processing workflows.
probberechts/soccerdata
A Python library for scraping soccer data from various sources for sports analytics and data science.
srx-2000/spider_collection
A collection of Python web scraping scripts for various websites and platforms, including music, video, and real estate data.
getdozer/dozer
Dozer is a real-time data movement tool that leverages CDC to move data between various sources and sinks.
re-data/re-data
A data quality and observability tool for monitoring and fixing data issues before they become problems.
lotus-data/lotus
A Python library that uses LLMs and embeddings to process datasets with up to 1000x speedups
ArchiveTeam/grab-site
A web crawler tool that outputs WARC files and provides a dashboard for managing crawls.
capitalone/DataProfiler
A Python library for extracting schema, statistics, and entities from datasets, useful for data profiling and privacy analysis.
hi-primus/optimus
Agile data preparation workflows made easy with popular Python data science libraries.
Stay in the loop
Get weekly updates on trending AI coding tools and projects.