Llm Evaluation Python Packages

mlflow-skinny

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

mlflow

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

mlflow-tracing

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

deepeval

The LLM Evaluation Framework

arize-phoenix

AI Observability & Evaluation

arize-phoenix-otel

AI Observability & Evaluation

arize-phoenix-client

AI Observability & Evaluation

arize-phoenix-evals

AI Observability & Evaluation

prompty

Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty is an asset class and format for LLM prompts designed to enhance observability, understandability, and portability for developers.

judgeval

The open source post-building layer for agents. Our environment data and evals power agent post-training (RL, SFT) and monitoring.

trulens-core

Evaluation and Tracking for LLM Experiments and AI Agents

garak

the LLM vulnerability scanner

opik-optimizer

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

trulens-otel-semconv

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-feedback

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-dashboard

Evaluation and Tracking for LLM Experiments and AI Agents

agenta

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

trulens-eval

Evaluation and Tracking for LLM Experiments and AI Agents

trulens

Evaluation and Tracking for LLM Experiments and AI Agents

giskard

🐢 Open-Source Evaluation & Testing library for LLM Agents

trulens-connectors-snowflake

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-providers-cortex

Evaluation and Tracking for LLM Experiments and AI Agents

lmms-eval

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks