Experimentation Python Packages

optimizely-sdk

Python SDK for Optimizely Feature Experimentation and Optimizely Full Stack (legacy).

gofeatureflag-python-provider

GO Feature Flag is a simple, complete and lightweight self-hosted cloud native feature flag solution 100% Open Source. 🎛️

rapidfireai

RapidFire AI: Rapid AI Customization from RAG to Fine-Tuning

moptipy

A package for metaheuristic optimization in Python.

ado-core

A framework for designing, executing and analysing experiment campaigns

yuclid

Orchestrating experiments on N-dimensional parameter spaces with custom metric collection + sliced-space visualization

uptrain

UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.

mltraq

Track and Collaborate on ML & AI Experiments.

pipelinex

PipelineX: Python package to build ML pipelines for experimentation with Kedro, MLflow, and more

ado-vllm-performance

A framework for designing, executing and analysing experiment campaigns

experiment-toolkit

Small, well-tested utilities for online controlled experiments: sample size, CUPED, sequential testing, delta-method variance.

moai-mdk

moai: Accelerated, Flexible, Modular, Reproducible, Insightful AI

feedx

Transparent, robust and trustworthy A/B experimentation for Shopping feeds.

gbstats

Stats engine for GrowthBook, the open source feature flagging and A/B testing platform.

ml-labs

No description available

expan

Open-source Python library for statistical analysis of randomised control trials (A/B tests)

blogstrap

An opinionated blogging platform

dead-salmon-brain

Dead Salmon Brain is a cluster computing system for analysing A/B experiments

agentrylab

Lightweight, hackable multi-agent orchestration lab (CLI + Python) with transcripts, checkpoints, budgets, and pluggable providers/tools.

dossier

A structured logging library for AI agents with session management and object unpacking

coii-sdk

agents team

hyperparameter-hunter

Easy hyperparameter optimization and automatic result saving across machine learning algorithms and libraries

ir-axioms

Axiomatic constraints for information retrieval and retrieval-augmented generation.

jovsatools

A collection of tools