Inference Server Python Packages

inference-gpu

Turn any computer or edge device into a command center for your computer vision projects.

inference-cli

Turn any computer or edge device into a command center for your computer vision projects.

truss

The simplest way to serve AI/ML models in production

truss-transfer

The simplest way to serve AI/ML models in production

baseten-performance-client

The simplest way to serve AI/ML models in production

inference-sdk

Turn any computer or edge device into a command center for your computer vision projects.

inference

Turn any computer or edge device into a command center for your computer vision projects.

ramalama

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

inference-core

Turn any computer or edge device into a command center for your computer vision projects.

inference-cpu

Turn any computer or edge device into a command center for your computer vision projects.

friendli-client

[⛔️ DEPRECATED] Friendli: the fastest serving engine for generative AI

fastdeploy

Deploy DL/ ML inference pipelines with minimal extra code.

infergrid

Tenant-fair LLM inference orchestration on a single GPU. No Kubernetes.

baseten-inference-client

The simplest way to serve AI/ML models in production

geniusrise-vision

Huggingface bolts for geniusrise

kvwarden

Tenant-fair LLM inference orchestration on a single GPU. No Kubernetes.

fdclient

fastDeploy python client

pipeless-ai

An open-source computer vision framework to create and deploy computer vision applications that scale in minutes

pipeless-ai-cli

An open-source computer vision framework to build and deploy apps in minutes

geniusrise-audio

audio bolts for geniusrise

geniusrise-text

Text components powering LLMs & SLMs for geniusrise framework

pinferencia

Python + Inference - Model Deployment library in Python. Simplest model inference server ever.

periflow-client

[⛔️ DEPRECATED] Friendli: the fastest serving engine for generative AI

smart-reid

Turn any computer or edge device into a command center for your computer vision projects.