Explore Projects

Discover 5 open source projects

Active filters (1):

Search: kvcache×

Clear all

Showing 1-5 of 5 projects

kvcache-ai/ktransformers

A flexible framework for optimizing heterogeneous LLM inference and fine-tuning workflows.

16.7K

Active

Python

LLM Frameworks

React

#llm#inference#fine-tuning

kvcache-ai/Mooncake

Mooncake is a serving platform for Kimi, a leading LLM service provided by Moonshot AI, focused on disaggregation, inference, and RDMA.

4.9K

Active

C++

LLM Frameworks

API Frameworks

C++

#llm#inference#rdma

Zefan-Cai/KVCache-Factory

Unified compression methods for KV caching in autoregressive language models like GPT-3.

1.3K

Archived

Python

LLM Frameworks

Caching

Python

#kv-cache#kv-cache-compression#llm

uccl-project/uccl

Efficient communication library for GPUs, covering collectives, P2P, and EP for AI/ML workloads

1.2K

Active

C++

GPU Frameworks

API Frameworks

C++

#ai#gpu#hpc

Zefan-Cai/R-KV

A redundancy-aware KV cache compression library for improving reasoning model performance.

1.2K

Stable

Python

LLM Frameworks

Caching

Python

#kvcache#llm#reasoning-models

Stay in the loop

Get weekly updates on trending AI coding tools and projects.