Evals Python Packages

logfire

AI observability platform for production LLM and agent systems.

arize-phoenix

AI Observability & Evaluation

arize-phoenix-otel

AI Observability & Evaluation

arize-phoenix-client

AI Observability & Evaluation

arize-phoenix-evals

AI Observability & Evaluation

agentops

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

trulens-core

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-otel-semconv

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-feedback

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-dashboard

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-eval

Evaluation and Tracking for LLM Experiments and AI Agents

trulens

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-connectors-snowflake

Evaluation and Tracking for LLM Experiments and AI Agents

harbor-rewardkit

Harbor is a framework for running agent evaluations and creating and using RL environments.

shadow-diff

Behavior contracts for AI agents

trulens-providers-cortex

Evaluation and Tracking for LLM Experiments and AI Agents

evalica

Evalica, your favourite evaluation toolkit

trulens-providers-openai

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-providers-litellm

Evaluation and Tracking for LLM Experiments and AI Agents

cellin

build long-lived multimodal memory, dream over it, and retrieve context with transparent weighting

trulens-apps-langchain

Evaluation and Tracking for LLM Experiments and AI Agents

mcp-assert

Test your MCP server against the real protocol. Any language, any transport. No mocks, no imports, no language lock-in.

trulens-apps-llamaindex

Evaluation and Tracking for LLM Experiments and AI Agents

selectools

Production-ready Python framework for AI agents with built-in guardrails, audit logging, cost tracking, and hybrid RAG. Supports OpenAI, Anthropic, Gemini, Ollama. By NichevLabs.