ETL & Pipelines
Explore 310 open source projects in ETL & Pipelines
Showing 141-160 of 310 projects
apache/hamilton
Hamilton is an open-source ETL framework that helps data scientists and engineers build modular, testable dataflows with lineage and metadata.
Lightning-AI/torchmetrics
Provides machine learning metrics for distributed, scalable PyTorch applications.
malloydata/malloy
Malloy is an open-source language for describing data relationships and transformations.
meltano/meltano
Meltano is a declarative, code-first data integration engine for building and scaling data and ML-powered products.
quarylabs/quary
Open-source BI platform for engineers to explore and model large-scale data pipelines.
rap2hpoutre/fast-excel
A fast and memory-efficient library for importing and exporting Excel files in Laravel applications.
instill-ai/instill-core
Instill Core is an open-source AI infrastructure tool for orchestrating data, models, and pipelines to build AI-powered applications.
man-group/ArcticDB
ArcticDB is a high-performance, serverless DataFrame database for the Python data science ecosystem.
supabase/etl
A real-time Postgres data replication and streaming library built in Rust for building CDC pipelines.
tensorflow/tfx
TFX is an end-to-end platform for deploying production ML pipelines.
reugn/go-streams
A lightweight stream processing library for Go developers that supports various streaming platforms.
timeplus-io/proton
Fast, single-binary C++ SQL ETL pipeline for stream processing, observability, analytics, and AI/ML.
minimaxir/facebook-page-post-scraper
A Python scraper for extracting data from Facebook Page posts for statistical analysis.
Jon-Becker/prediction-market-analysis
Framework for collecting and analyzing prediction market data with comprehensive Polymarket/Kalshi datasets.
0xemmkty/QuantMuse
Quantitative trading system with ML analysis, real-time data processing, and risk management
moj-analytical-services/splink
Fast, accurate, and scalable probabilistic data linkage with support for multiple SQL backends.
databricks/spark-deep-learning
Deep learning library for Apache Spark that provides high-level APIs and models for building machine learning pipelines.
apache/datafusion-ballista
Apache DataFusion Ballista is a distributed query engine for big data analysis, built with Rust and Arrow.
shancarter/mr-data-converter
A JavaScript library that converts CSV and tab-delimited data to web-friendly formats like JSON and XML.
bytewax/bytewax
Bytewax is a Python library for building scalable, fault-tolerant, and low-latency data processing pipelines.
Stay in the loop
Get weekly updates on trending AI coding tools and projects.