Robustness Python Packages

recentrifuge

Recentrifuge: robust comparative analysis and contamination removal for metagenomics

holisticai

This is an open-source tool to assess and improve the trustworthiness of AI systems.

auto-lirpa

auto_LiRPA: An Automatic Linear Relaxation based Perturbation Analysis Library for Neural Networks and General Computational Graphs

adversarial-vision-challenge

NIPS Adversarial Vision Challenge

invarlock

Edit‑agnostic robustness evaluation reports for weight edits (InvarLock framework)

advertorch

A Toolbox for Adversarial Robustness Research

bound-propagation

Linear and interval bound propagation in Pytorch with easy-to-use API and GPU support.

deltatau-audit

Time Robustness Audit for RL agents — measures timing reliance, deployment robustness, and stress resilience

thetiscore

Service to examine data processing pipelines (e.g., machine learning or deep learning pipelines) for uncertainty consistency (calibration), fairness, and other safety-relevant aspects.

adaptor

ACL 2022: Adaptor: a library to easily adapt a language model to your own task, domain, or custom objective(s).

neuralsat

NeuralSAT: A DPLL(T) Framework for Verifying Deep Neural Networks

pylandscape

This project propose the loss landscape analysis as effective methodology to understand the robustness against natural perturbation of QNN.

linqit

Extend python lists operations using .NET's LINQ syntax for clean and fast coding.

deel-torchlip

PyTorch implementation for k-Lipschitz layers

imagenet-c

Corruption and Perturbation Robustness (ICLR 2019)

gaf-microbatch-pytorch

Implementation of Gradient Agreement Filtering, from Chaubard et al. of Stanford, but for single machine microbatches, in Pytorch

promptbench

A unified evaluation framework for large language models

byzfl

ByzFL Library

perceptual-advex

Code and data for the ICLR 2021 paper "Perceptual Adversarial Robustness: Defense Against Unseen Threat Models".

fiddler-auditor

Auditing large language models made easy.

thetis

Service to examine data processing pipelines (e.g., machine learning or deep learning pipelines) for uncertainty consistency (calibration), fairness, and other safety-relevant aspects.

graph-tiger

Python toolbox to evaluate graph vulnerability and robustness (CIKM 2021)

mercury-robust

mercury-robust is a framework to perform robust testing on ML models and datasets. It provides a collection of test that are easy to configure and helpful to guarantee robustness in your ML processes.

virny

An in-depth performance profiling library for machine learning models