ETL & Pipelines
Explore 310 open source projects in ETL & Pipelines
Showing 241-260 of 310 projects
Data-Learn/data-engineering
A comprehensive resource for developers to learn and get started with data engineering using Python.
petl-developers/petl
A Python library for extracting, transforming, and loading tabular data.
datazip-inc/olake
Fastest open-source data pipeline tool for replicating databases to data lakes in Apache Iceberg format.
ezrosent/frawk
An efficient awk-like language written in Rust for text processing and data manipulation tasks.
datavane/tis
A Java-based framework for building agile DataOps pipelines using tools like Flink, DataX, and Chunjun with a web UI.
wesm/msgvault
Archive, search, and analyze your entire email/chat history offline with DuckDB-powered analytics and AI queries.
DTStack/Taier
A big data development platform for submission, scheduling, operation and maintenance, and indicator information display.
GoogleCloudPlatform/DataflowTemplates
Provides pre-built Google Cloud Dataflow templates to simplify data processing tasks on the Google Cloud Platform.
mahmoudparsian/pyspark-tutorial
PySpark-Tutorial provides basic algorithms using PySpark for big data analytics and data processing.
rsvp/fecon235
Notebooks for financial economics, including analyses of Federal Reserve, GDP, inflation, and more.
networktocode/ntc-templates
A collection of TextFSM templates for parsing network device show commands, useful for network automation.
stanfordjournalism/search-script-scrape
A collection of 101 real-world web scraping exercises in Python 3 for data journalists.
Open-Source-Legal/OpenContracts
An enterprise-grade, API-first LLM workspace for unstructured document processing, with features like data extraction, redaction, and prompt engineering.
uber-archive/AthenaX
A scalable, SQL-based streaming analytics platform from Uber, built on top of Apache Flink.
lit26/finvizfinance
A Python library for financial analysis and data scraping from the Finviz platform.
sentinel-hub/eo-learn
An open-source Python framework for processing Earth observation data using machine learning.
drasi-project/drasi-platform
The Data Change Processing platform, a C# library for building CDC (change data capture) and change detection systems.
kevwan/go-stash
A high-performance, open-source data processing pipeline for ingesting Kafka data and sending it to Elasticsearch.
calogica/dbt-expectations
A port of Great Expectations to dbt test macros for data testing and validation in data engineering workflows.
skyplane-project/skyplane
Blazing fast, multi-cloud data transfer solution for developers looking to move data seamlessly across cloud providers.
Stay in the loop
Get weekly updates on trending AI coding tools and projects.