Bias Detection Python Packages

aif360

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

dataeval

Python library for analyzing data quality and its impact on model performance across classification and object-detection tasks.

dataeval-plots

Python library for analyzing data quality and its impact on model performance across classification and object-detection tasks.

model-diagnostics

Tools for diagnostics and assessment of (machine learning) models

langfair

LangFair is a Python library for conducting use-case level LLM bias and fairness assessments

trustlens

Open-source Python library for evaluating ML model reliability beyond accuracy — with calibration, failure, and fairness diagnostics for informed deployment decisions.

optipfair

Structured pruning and bias visualization for Large Language Models. Tools for LLM optimization and fairness analysis.

judicator

Who evaluates the evaluator? Judicator audits LLM-as-a-Judge systems for 7 documented bias types. Zero config. Works with any LLM.

ai-safety-tester

LLM security testing framework with CVE-style severity scoring and multi-model benchmarking

daml

Python library for analyzing data quality and its impact on model performance across classification and object-detection tasks.

wefe

The Word Embedding Fairness Evaluation Framework

rho-eval

Behavioral auditing toolkit for LLMs — audit any model across 8 dimensions (factual, toxicity, bias, sycophancy, reasoning, refusal, deception, over-refusal) using teacher-forced confidence probes.

seatbelt

Responsible AI auditing for LLMs and SLMs — deception, fairness, sociotechnical risk, regulatory compliance

fairsense-agentix

An agentic fairness and AI-risk analysis platform for detecting bias in text and images

knowledge-fidelity

Compress LLMs while auditing whether they still know truth vs myths. SVD compression + false-belief detection in one toolkit.

trusteval-ai

Enterprise LLM Evaluation & Responsible AI Framework — Benchmark bias, hallucination, PII leakage, and toxicity across Healthcare, BFSI, Retail & Legal industries. Supports OpenAI, Anthropic, Gemini & HuggingFace. Python SDK + CLI + Web Dashboard. 191 tests. Compliance-ready reports.

unbias-plus

Python package that finds biased language in text, explains why, suggests neutral wording, and rewrites the whole text. Use from CLI, API, or Python.

ai-fairness-toolkit

AI Fairness and Explainability Toolkit (AFET) is an open-source project aimed at providing tools and frameworks to assess, visualize, and mitigate bias in machine learning models. It supports multiple ML frameworks and offers a comprehensive suite of metrics and visualization components to enhance model transparency and fairness.

the-fairly-project

Bias detection Toolkit: Chrome Extension, Python Package, SOTA research paper docs.

aiscan

Scan your AI/ML models for problems before you put them into production.

phenoqc

PhenoQC is a lightweight, efficient, and user-friendly toolkit designed to perform comprehensive quality control (QC) on phenotypic datasets. It ensures that data adheres to standardized formats, maintains consistency, and is harmonized with recognized ontologies.

whis-ethical-ai

Ethical AI Validator detects bias and assesses fairness in AI models with statistical parity analysis, real-time monitoring, and automated GDPR/AI Act compliance reporting. Python 2.7+ compatible.

aif360-fork2

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

biasclear

Structural bias detection engine built on Persistent Influence Theory (PIT). Detect framing, anchoring, false consensus, and 30+ rhetorical distortion patterns.