ETL & Pipelines
Explore 310 open source projects in ETL & Pipelines
Showing 201-220 of 310 projects
paradigmxyz/cryo
cryo is a Rust library for extracting blockchain data to parquet, CSV, JSON, or Python dataframes.
aws-samples/aws-glue-samples
AWS Glue code samples for building data integration and ETL pipelines on AWS.
combust/mleap
MLeap is a library for deploying machine learning pipelines to production using Scala, Python, and Spark.
OBenner/data-engineering-interview-questions
This GitHub repository contains over 2,000 data engineering interview questions to help developers prepare.
meta-llama/synthetic-data-kit
Tool for generating high-quality synthetic datasets
emcf/thepipe
A Python library that helps developers extract structured data from tricky documents using vision-language models.
fossasia/event-collect
A Python-based scraper and converter for event websites to the Open Event format.
pvlib/pvlib-python
A Python library for simulating the performance of photovoltaic energy systems.
superlinked/superlinked
Superlinked is a Python framework for building high-performance search & recommendation apps with structured and unstructured data.
san089/goodreads_etl_pipeline
An end-to-end data pipeline for building a data lake, data warehouse, and analytics platform from GoodReads data.
pyjanitor-devs/pyjanitor
A Python library for cleaning and transforming data, inspired by the R package Janitor.
Factual/drake
A data workflow tool for data engineers and analysts, similar to 'Make for data'.
DataBrewery/cubes
A lightweight Python OLAP framework for multi-dimensional data analysis and reporting.
compose/transporter
Transporter is a powerful ETL tool that allows developers to sync data between various persistence engines.
bruin-data/bruin
A data platform that enables building data pipelines with SQL, Python, and ingesting from various sources.
NVIDIA-NeMo/Curator
Scalable data pre processing and curation toolkit for Large Language Models (LLMs)
tidyverse/tidyr
tidyr is an R package that provides a set of functions to tidy messy data into a format suitable for analysis.
AlexTheAnalyst/PortfolioProjects
This repository contains a collection of portfolio projects for a data analyst, not a developer discovery platform.
astronomer/dag-factory
Declaratively construct Apache Airflow DAGs with YAML configuration files, simplifying complex data pipeline management.
4lex4/scantailor-advanced
ScanTailor Advanced is a C++ library for processing scanned documents, including binarization, book scanning, and digitalization.
Stay in the loop
Get weekly updates on trending AI coding tools and projects.