Vybe Guide
Sign Up

Explore Projects

Discover 321 open source projects

Active filters (1):
Search: inferenceร—
Clear all

Showing 41-60 of 321 projects

NVIDIA/TensorRT

NVIDIA TensorRT is an SDK for high-performance deep learning inference on NVIDIA GPUs.

12.7K
Active
C++
Inference
#deep-learning#gpu-acceleration#inference

DayBreak-u/chineseocr_lite

A lightweight Chinese OCR library that supports vertical text recognition and NCNN/MNN/TNN inference with a small model size.

12.3K
Archived
C++
Computer Vision
PyTorch
#ocr#computer-vision#ncnn

bentoml/OpenLLM

Deploy open-source LLMs as OpenAI-compatible API endpoints using BentoML's model serving framework.

12.1K
Active
Python
AI Model Serving
Local Inference Engines
BentoML
#llm-inference#bentoml#model-serving

GeeeekExplorer/nano-vllm

A powerful Python library for working with large language models (LLMs) and natural language processing tasks.

12.0K
Stable
Python
LLM Frameworks
PyTorch
#llm#nlp#deep-learning

ubicloud/ubicloud

Open-source cloud platform with elastic compute, storage, databases, AI, and Kubernetes services.

11.9K
Active
Ruby
Managed Cloud Platforms
Ruby
#cloud#open-source#kubernetes

SparkAudio/Spark-TTS

Spark-TTS is an open-source Python library for high-quality text-to-speech inference.

10.9K
Experimental
Python
AI Voice & Speech
#text-to-speech#inference#open-source

aws/amazon-sagemaker-examples

A collection of Jupyter notebooks showcasing how to build and deploy machine learning models with Amazon SageMaker.

10.9K
Active
Jupyter Notebook
ML Ops
Jupyter Notebook
#machine-learning#deep-learning#data-science

huggingface/text-generation-inference

Large Language Model Text Generation Inference library for developers working with AI tools and models.

10.8K
Active
Python
LLM Frameworks
PyTorch
#nlp#transformer#bloom

mistralai/mistral-inference

Official inference library for Mistral models, a platform for building AI-powered applications.

10.7K
Stable
Jupyter Notebook
LLM Frameworks
React
#llm#llm-inference#mistralai

triton-inference-server/server

An optimized cloud and edge inference solution for deploying and running machine learning models.

10.4K
Active
Python
Inference
#machine-learning#deep-learning#gpu

Const-me/Whisper

High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model

10.2K
Archived
C++
Inference
#speech-recognition#whisper#asr

RunanywhereAI/runanywhere-sdks

Production ready AI toolkit for local AI inference

10.2K
Active
Kotlin
AI Coding Tools
#agent-framework#android#apple-intelligence

bigscience-workshop/petals

A distributed system for running large language models (LLMs) on personal devices, enabling faster fine-tuning and inference.

10.0K
Archived
Python
LLM Frameworks
PyTorch
#llm#distributed-computing#fine-tuning

openvinotoolkit/openvino

OpenVINO is an open-source toolkit for optimizing and deploying AI inference on a variety of hardware.

9.8K
Active
C++
Inference
#ai#computer-vision#deep-learning

deepseek-ai/3FS

A high-performance distributed file system designed for AI workloads like training and inference.

9.7K
Active
C++
Infrastructure
#distributed-file-system#ai-workloads#high-performance

pymc-devs/pymc

A powerful Bayesian modeling and probabilistic programming library for Python developers.

9.5K
Active
Python
LLM Frameworks
Python
#bayesian-inference#mcmc#probabilistic-programming

xorbitsai/inference

Unified, production-ready inference API to run open-source, speech, and multimodal models on cloud, on-prem, or your laptop.

9.1K
Active
Python
LLM Frameworks
Inference
PyTorch
#artificial-intelligence#llm#inference

pyro-ppl/pyro

A deep universal probabilistic programming library for Python and PyTorch, enabling Bayesian machine learning.

9.0K
Experimental
Python
LLM Frameworks
API Frameworks
PyTorch
#bayesian#probabilistic-modeling#variational-inference

oumi-ai/oumi

Easily fine-tune, evaluate and deploy open-source large language models like GPT-OSS and Llama.

8.9K
Active
Python
LLM Frameworks
Inference
Python
#llms#fine-tuning#evaluation

Tiiny-AI/PowerInfer

High-performance C++ library for fast local deployment of large language models (LLMs) like LLaMA.

8.8K
Active
C++
LLM Frameworks
API Frameworks
#llm#llm-inference#local-inference
124...17

Stay in the loop

Get weekly updates on trending AI coding tools and projects.