Mechanistic Interpretability Python Packages

pyvene

Stanford NLP Python library for understanding and improving PyTorch models via interventions

llmoji

Making agents cuter

nnterp

Unified access to Large Language Model modules using NNsight

glassbox-mech-interp

Mechanistic interpretability + EU AI Act Annex IV compliance. 21/21 frameworks: ACDC edge-circuit discovery, multi-arch GQA/RMSNorm adapter (Llama-3/Mistral/Phi-3), cross-model comparison, causal scrubbing, DAS, Hessian bounds, BH FDR, folded LayerNorm, SAE polysemanticity, multi-corruption, held-out validation. Dual-licensed (MIT core + BSL 1.1 compliance engine).

openinterp

openinterp — Python SDK + CLI. FabricationGuard hallucination probe + ProbeBench leaderboard + Atlas search + Trace generation. pip install openinterp

carl-studio

Coherence-Aware Reinforcement Learning (CARL) - breakthrough LLM post-training and test-time training paradigm. carl builds the world's most advanced and intelligent agent systems that are a step change over current gen agents

steering-vectors

Steering vectors for transformer language models in Pytorch / Huggingface

mlx-lens

Mechanistic interpretability on Apple Silicon: steering vectors, residual capture, and SAE analysis for MLX models

reptimeline

Track how discrete representations evolve during neural network training — lifecycle events, phase transitions, ontology discovery, and causal verification

mechreward

Mechanistic interpretability as reward signal for RL training of LLMs

carl-core

Coherence-Aware Reinforcement Learning (CARL) - breakthrough LLM post-training and test-time training paradigm. carl builds the world's most advanced and intelligent agent systems that are a step change over current gen agents

neuro-scan

LLM Neuroanatomy Explorer — map what each transformer layer does

graphpatch

graphpatch is a library for activation patching on PyTorch neural network models.

mlxlmprobe

Visual probing and interpretability tool for MLX language models

codebook-features

Sparse and discrete interpretability tool for neural networks

glassbox-mcp

Open-source EU AI Act Annex IV compliance toolkit. Mechanistic interpretability + circuit discovery for transformers. One function call generates a court-ready evidence package

arrakis-mi

A mechanistic interpretability library for nerds.

reprobe

Phase-aware LLM activation steering and linear probing. A memory-efficient, practical implementation of Representation Engineering (RepE) for safety research.

todacomm

Topological Data Analysis Comparison of Multiple Models - A framework for characterizing transformer representations using persistent homology