Llmops Python Packages

litellm

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

mlflow-skinny

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

mlflow

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

mlflow-tracing

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

opentelemetry-instrumentation-vertexai

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

openinference-semantic-conventions

OpenTelemetry Instrumentation for AI Observability

openinference-instrumentation

OpenTelemetry Instrumentation for AI Observability

opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

opentelemetry-instrumentation-bedrock

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-cohere

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-llamaindex

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-ollama

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-qdrant

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-replicate

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-transformers

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-crewai

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-chromadb

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-haystack

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-weaviate

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-mistralai

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-pinecone

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-watsonx

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-milvus

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

opentelemetry-instrumentation-together

Open-source observability for your GenAI or LLM application, based on OpenTelemetry