PyPI Stats
  • Insights
  • PyPI
  • GitHub
  • Search
  • Compare
  • Advisories
  • Ecosystem
  • About
Home

Llm Observability Python Packages

Python packages with the GitHub topic llm-observability. Sorted by relevance, with stars and monthly downloads.
pydantic
logfire

AI observability platform for production LLM and agent systems.

25M 4K 230
comet-ml
opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

5.1M 19K 1K
JudgmentLabs
judgeval

The open source post-building layer for agents. Our environment data and evals power agent post-training (RL, SFT) and monitoring.

368K 1K 91
comet-ml
opik-optimizer

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

58K 19K 1K
agenta-ai
agenta

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

56K 4K 517
Helicone
helicone-helpers

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

10K 6K 560
memodb-io
acontext

Agent Skills as a Memory Layer

5K 3K 314
Mandark-droid
genai-otel-instrument

GenAI OpenTelemetry Auto-Instrumentation Library A comprehensive wrapper for automatic instrumentation of LLM/GenAI applications Supports all major LLM providers and MCP (Model Context Protocol) tool calls

5K 1 1
dunetrace
dunetrace

Real time anomaly detection layer for AI agents. Privacy-safe by design.

4K 38 3
sairintechnologycom
burnlens

Open-source LLM FinOps proxy — track OpenAI, Anthropic (Claude), and Google Gemini costs by feature, team, and customer. Zero code changes. pip install burnlens.

3K 1 0
helicone
helicone

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

3K 6K 560
helicone
helicone-async

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

2K 6K 560
syndicalt
pathlight

Visual debugging, execution traces, and observability for AI agents.

2K 15 3
BlazeUp-AI
observal-cli

Observal is an AI agent registry with first in class observabilty and eval framework

2K 839 79
comet-ml
comet-llm

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

1K 19K 1K
acailic
peaky-peek

Lightweight tracing SDK for AI agents. Capture decisions, tool calls, and LLM events with one context manager.

1K 5 0
acailic
peaky-peek-server

Local-first agent debugger with replay, failure memory, smart highlights, and drift detection.

1K 5 0
kums1234
otel-genai-graph

Project OpenTelemetry GenAI traces into a queryable graph (Neo4j or DuckDB) — agent delegation, cost attribution, blast radius.

1K 0 0
last9
l9gpu

GPU telemetry with workload attribution. One OTLP agent per node ties hardware metrics (NVIDIA, AMD, Intel Gaudi) to the K8s pod or Slurm job burning the GPU — so you know who's paying for that idle H100.

1K 10 2
smigolsmigol
llmkit-sdk

Know what your AI agents cost. API gateway with budget enforcement, session tracking, and MCP tools.

1K 10 3
HemantBK
chatbot-auditor

Quality auditor for AI chatbots. Analyzes your conversation logs to show where the bot is underperforming.

893 0 0
myscale
myscale-telemetry

Open-source observability for your LLM application.

670 55 7
wild-edge
wildedge-sdk

Python SDK for WildEdge

658 13 1
justinGrosvenor
alignmenter

Persona-aligned evaluation toolkit for auditing conversational AI authenticity, safety, and stability.

641 5 0
    • Data from PyPI, GitHub, ClickHouse, and BigQuery