Search Packages

agenta

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

giskard

🐢 Open-Source Evaluation & Testing library for LLM Agents

autorag

AutoRAG: An Open-Source Framework for Retrieval-Augmented Generation (RAG) Evaluation & Optimization with AutoML-Style Automation

rag-forge-observability

Production-grade RAG pipelines with evaluation baked in

rag-forge-core

Production-grade RAG pipelines with evaluation baked in

rag-forge-evaluator

Production-grade RAG pipelines with evaluation baked in

ragscore

⚡️ The "1-Minute RAG Audit" — Generate QA datasets & evaluate RAG systems in Colab, Jupyter, or CLI. Privacy-first, async, visual reports.

kb-arena

Benchmark 9 retrieval architectures on your documentation — find which KB architecture fits your data

llamator

Framework for testing vulnerabilities of GenAI systems.

open-rag-eval

A Python package for RAG Evaluation

rag-benchmarking

RAG Benchmarking — Framework-agnostic RAG/agentic-AI evaluation harness. Faithfulness, agentic metrics, EU AI Act Article 15 accuracy evidence. Apache 2.0.

rurage

RURAGE (Robust Universal RAG Evaluation) is a Python library developed to speed-up evaluation of RAG systems with Correctness, Faithfulness and Relevance axes using a variety of deterministic and model-based metrics.

smallevals

Small Language Models Evaluation Suite for RAG Systems

vero-eval

Open source framework for evaluating AI Agents

rail-score

DEPRECATED — use rail-score-sdk instead. This package redirects to rail-score-sdk.

syncreus-eval

Evaluate your LLM apps with one function call. Hallucination detection, RAG scoring, and agent evals for OpenAI, Anthropic, and more. 14 evaluators, pytest plugin, composite trust scores.

eval-rag

A comprehensive evaluation toolkit for assessing Retrieval-Augmented Generation (RAG) outputs using linguistic, semantic, and fairness metrics

ragaliq

LLM & RAG evaluation testing framework — hallucination detection, faithfulness metrics, answer relevance scoring, and retrieval pipeline testing with pytest integration