ETL & Pipelines
Explore 310 open source projects in ETL & Pipelines
Showing 281-300 of 310 projects
NVIDIA-Merlin/NVTabular
NVTabular is a feature engineering and preprocessing library for tabular data used in recommender systems.
spring-attic/spring-cloud-dataflow
A microservices-based platform for cloud-native streaming and batch data processing on Cloud Foundry and Kubernetes.
lvgalvao/data-engineering-roadmap
Comprehensive roadmap for data engineering and AI development in Python
DataExpert-io/llm-driven-data-engineering
A public repository for exploring LLM-driven data engineering concepts and tools.
xillwillx/skiptracer
An open-source Python web scraping framework for OSINT and data collection tasks.
ucarGroup/DataLink
DataLink is a real-time and offline data exchange platform that supports synchronization between heterogeneous data sources.
scratchdata/scratchdata
A Swiss army knife for big data, enabling seamless integration with popular data warehousing solutions.
apache/amoro
Apache Amoro is an open-source Lakehouse management system built on big data formats like Flink, Hudi, and Iceberg.
Teradata/kylo
Kylo is an enterprise-grade data lake management platform built on big data technologies like Spark and Hadoop.
moby/datakit
Connect processes into powerful data pipelines with a simple git-like filesystem interface
patrickhulce/third-party-web
A comprehensive dataset on third-party entities and their impact on the web, useful for web performance analysis.
oeljeklaus-you/UserActionAnalyzePlatform
A big data platform for analyzing e-commerce user behavior using Hadoop, Spark, and Java.
apachecn/pyda-2e-zh
A Chinese translation of the book 'Python for Data Analysis' 2nd Edition, covering NumPy, Pandas, and other data analysis tools.
OpenGeoscience/geonotebook
A Jupyter notebook extension for geospatial visualization and analysis, focused on open geoscience.
mahmoudparsian/data-algorithms-book
This repository provides a comprehensive guide and implementations for data algorithms using MapReduce, Spark, Java, and Scala.
jf-tech/omniparser
A native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, and more.
rhiever/datacleaner
A Python tool that automatically cleans and preprocesses data for analysis and machine learning.
crazyhottommy/RNA-seq-analysis
This GitHub repository contains notes and code for analyzing RNA-seq data using Python and Snakemake.
metachris/pdfx
A Python library to extract text, metadata, and references from PDF files, including downloading referenced PDFs.
Mrkuhuo/data-warehouse-learning
Open-source data warehouse learning project with examples and code for building real-time and offline data pipelines.
Stay in the loop
Get weekly updates on trending AI coding tools and projects.