Ai Gateway Python Packages

litellm

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

litellm-proxy-extras

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

litellm-enterprise

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

portkey-ai

Build reliable, secure, and production-ready AI apps easily.

ai-stats-py-sdk

Open-source, OpenAI-compatible AI gateway with health-aware routing, failover, and live telemetry (cost, latency, errors). Built for production.

ollama-herd

Local AI load balancer for Ollama fleets — auto-discovery, smart routing, OpenAI-compatible API, zero config. Perfect for Mac Minis & Studios.

cost-katana

Cost Katana Python

freeride-gateway

One free AI endpoint, every free tier behind it. Local OpenAI-compatible gateway routing across OpenRouter, Groq, NVIDIA NIM, Cloudflare Workers AI, and HuggingFace with automatic failover.

freemygpt

FreeMyGPT — HTTP gateway that lets ChatGPT (and any LLM) drive any local MCP server via simple GET requests. Hybrid: MCP stdio backends by default, Codex CLI and other runtimes via bundled adapters.

sagify

LLMs and Machine Learning done easily

llmkit-sdk

Know what your AI agents cost. API gateway with budget enforcement, session tracking, and MCP tools.

rubeus

Build reliable, secure, and production-ready AI apps easily.

langgate

Lightweight AI inference gateway - local model registry & parameter transformer (Python SDK) - with optional Envoy proxy processor and FastAPI registry server deployment options.

amp-instrumentation

WSO2 AI Agent Manager is an open control plane designed for enterprises to deploy, manage, and govern AI agents at scale.

schmitech-orbit-client

The self-hosted AI gateway for production RAG across LLMs, databases, APIs, and files.

ferrolabsai

Official Python SDK for Ferro Labs AI Gateway — route LLM requests across 29 providers with a single OpenAI-compatible API

fastapi-ai-router

FastAPI middleware that turns your existing routes into a natural-language-callable surface using LLM function-calling. Drop-in. Zero new metadata.

kvfleet

Production-grade, KV-cache-aware intelligent routing for self-hosted and hybrid LLM fleets.

litellm-mcp

MCP server giving AI agents access to 100+ LLMs through LiteLLM

vllm-sr-sim

System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge

mcp-monkdb

An MCP server 🚀 for MonkDB, licensed under Apache 2.0 📜

nous-genai

Single-endpoint GenAI SDK (multi-provider, multimodal)

genai-calling

Single-endpoint GenAI SDK (multi-provider, multimodal)

litellm-ad

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]