Inference Engine Python Packages

astroid

A common base representation of python source code for pylint and other projects

qai-hub-models

Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

onediff

an out-of-the-box acceleration library for diffusion models

experta

Expert Systems for Python

nocturnusai

Verified knowledge for AI agents. Compress context, extract and store facts, define rules, and ask questions — get deterministic answers with proof, not LLM guesses. Connect agents via MCP, Python SDK, TypeSc

aphrodite-engine

Large-scale LLM inference engine

qai-hub-models-cli

Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

onediffx

OneDiff: An out-of-the-box acceleration library for diffusion models.

krasis

Krasis is no longer distributed via PyPI. Install from GitHub: https://github.com/brontoguana/krasis

friendli-client

[⛔️ DEPRECATED] Friendli: the fastest serving engine for generative AI

nobodywho

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

neurobrix

Universal Deep Learning Inference Engine — execute any AI model without model-specific code

fedml

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

m5-infer

Extraordinary speed, extraordinary quality — an MLX-based inference engine for Apple Silicon.

exxa

Exa - Pytorch

zllm-zse

The inference engine the open-source world built for itself.

para-attn

https://wavespeed.ai/ Context parallel attention that accelerates DiT model inference with dynamic caching

kvcached

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

opencv-python-inference-engine

Wrapper package for OpenCV with Inference Engine python bindings.

yolomosaic

A Python library for visualizing YOLO detections and segmented instances on large orthomosaic images, with the ability to generate shapefiles for GIS integration

planer

Powerful Light Artificial NEuRon inference framework for CNN

quaynor

Embed local models in your app

ov-training-kit

Wrappers for scikit-learn, PyTorch and Tensorflow models with OpenVINO optimization

kimchima

The collections of tools for ML model development.