Regression Testing Python Packages

shadow-diff

Behavior contracts for AI agents

pytest-accept

A pytest plugin for automatically updating doctest outputs

reframe-hpc

A powerful Python framework for writing and running portable regression tests and benchmarks for HPC systems.

mcp-assert

Test your MCP server against the real protocol. Any language, any transport. No mocks, no imports, no language lock-in.

evalview

Open-source testing and regression detection framework for AI agents. Golden baseline diffing, CI/CD integration, works with LangGraph, CrewAI, OpenAI, Anthropic Claude, HuggingFace, Ollama, and MCP.

pytest-mcp-assert

Test your MCP server against the real protocol. Any language, any transport. No mocks, no imports, no language lock-in.

pytest-ranking

A Pytest plugin for faster test failure detection

socx-cli

Unified command-line tool for EDA development teams to streamline common tasks and tools, and unify them under a single configurable CLI menu to increase accessibility and transparency of tools and scripts in collaborative development environments.

agentassay

Token-efficient stochastic testing for AI agents. 5-20x cost reduction. 10 framework adapters. Paper: arXiv:2603.02601

fhe-attack-replay

Unified attack-replay regression harness for FHE libraries (SEAL, OpenFHE, Lattigo, tfhe-rs).

modeldiffx

Model behavioral diffing - compare LLM outputs across versions, detect regressions.

touca

Continuous Regression Testing for Engineering Teams

webqa-agent

Autonomous web browser agent that audits performance, functionality & UX for engineers and vibe-coding creators. 网站自主评估测试 Agent，支持 GUI/CLI 一键完成性能、功能使用与交互体验的测试评估

conport

🚀 Continuous testing report generator written in Python

testdatatable

A shared data table store for use by testing applications

touca-fbs

Continuous Regression Testing for Engineering Teams

tlscanary

TLS/SSL Test Suite for Mozilla Firefox

nrtest

Numerical regression testing for computer simulations

toolcallcheck

Deterministic Python testing for tool-using agents. Mock MCP tools, assert exact tool calls and trajectories, verify headers, and run offline in CI.

pytest-unflakable

Official Unflakable Plugins for Python

locomotive

CI/CD load testing runner and regression analyzer for Locust

firewall-test

Framework for simulating traffic over firewall rulesets - for troubleshooting and automated CI regression-tests

regressormetricgraphplot

A simple package for comparing different Regression Models and Plotting with their most common evaluation metrics.

regrest

Regrest is a simple and powerful regression testing tool for Python. It automatically records function outputs on the first run and validates them on subsequent runs.