ETL & Pipelines
Explore 310 open source projects in ETL & Pipelines
Showing 261-280 of 310 projects
ptwobrussell/Mining-the-Social-Web
A library for mining data from social media platforms like Twitter, Facebook, and Reddit.
2ndQuadrant/pglogical
A high-performance logical replication extension for PostgreSQL that enables fast, cross-version database replication.
kkyon/botflow
A Python framework for building data pipelines, web crawlers, and quantitative trading applications.
sryza/spark-timeseries
A library for time series analysis on Apache Spark, enabling efficient large-scale time series processing.
langchain-ai/langchain-extract
A LangChain-based framework for extracting data from various sources using LLMs and APIs.
marsupialtail/quokka
A scalable, distributed ETL framework for building data lake analytics pipelines.
react-csv/react-csv
A React component library for generating CSV files on the fly from data arrays or objects.
predict-idlab/plotly-resampler
A Python library that helps visualize large time series data using the Plotly data visualization library.
lakehq/sail
LakeSail is a Rust-based computation framework that unifies batch processing, stream processing, and AI workloads.
pytroll/satpy
A Python package for processing earth-observing satellite data with support for common data formats and tools.
apache/incubator-xtable
Apache XTable is a cross-table converter for lakehouse table formats that facilitates interoperability across data processing systems and query engines.
ChawlaAvi/Daily-Dose-of-Data-Science
A collection of code snippets and tutorials for data science and data analysis in Python.
zinggAI/zingg
Scalable identity resolution, entity resolution, data mastering and deduplication using ML
thinh-vu/vnstock
A beginner-friendly Python toolkit for financial data extraction, analysis, and automation.
KEV0143/Parser-Chitai-Gorod
A high-speed, intelligent web scraper for the Chitai-Gorod book catalog, enabling structured data collection.
astronomer/astronomer-cosmos
Run your dbt Core or dbt Fusion projects as Apache Airflow DAGs and Task Groups with a few lines of code.
lensacom/sparkit-learn
A Python library that integrates Scikit-learn into the Apache Spark distributed computing framework.
abhishek-ch/around-dataengineering
A comprehensive knowledge hub for data engineering, machine learning, and MLOps tools and practices.
petewarden/dstk
A collection of open data sets and tools for data science and machine learning tasks.
opensemanticsearch/open-semantic-search
Open-source search and text analytics platform for exploring large document collections with semantic search and NLP
Stay in the loop
Get weekly updates on trending AI coding tools and projects.