Search Packages

deepeval

The LLM Evaluation Framework

qa-metrics

An easy python package to run quick basic QA evaluations. This package includes standardized QA evaluation metrics and semantic evaluation metrics: Black-box and Open-Source large language model prompting and evaluation, exact match, F1 Score, PEDANT semantic match, transformer match. Our package also supports prompting OPENAI and Anthropic API.

parea-ai

Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)

rhesis-sdk

The testing platform for AI teams. Bring engineers, PMs, and domain experts together to generate tests, simulate (adversarial) conversations, and trace every failure to its root cause.

langfair

LangFair is a Python library for conducting use-case level LLM bias and fairness assessments

agentic-security

Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪

rhesis

The testing platform for AI teams. Bring engineers, PMs, and domain experts together to generate tests, simulate (adversarial) conversations, and trace every failure to its root cause.

fieldtest

LLM evaluation framework — define what correct, well-formed, and safe means before you measure

evalsense

Tools for systematic large language model evaluations

ccheck

MIT-licensed Framework for LLMs, RAGs, Chatbots testing. Configurable via YAML and integrable into CI pipelines for automated testing.

multinear

Multinear platform

vero-eval

Open source framework for evaluating AI Agents

llmevals

Eval

mseep-agentic-security

Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪

deepevals

The LLM Evaluation Framework

langalf

Agentic LLM vulnerability scanner

testllm

Deep eval provides evaluation platform to accelerate development of LLMs and Agents