Mixture Of Experts Python Packages

deepspeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

egobox

Efficient global optimization toolbox in Rust: bayesian optimization, mixture of gaussian processes, sampling methods

mixture-of-experts

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models

smt

Surrogate Modeling Toolbox

optillm

Optimizing inference proxy for LLMs

hivemind

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

krasis

Krasis is no longer distributed via PyPI. Install from GitHub: https://github.com/brontoguana/krasis

dissenter

Multi-LLM debate engine for complex questions — surface disagreement, synthesize decisions

terradev-cli

Cross-Cloud Compute Optimization Platform with Migration & Evaluation - v4.0.12

abliterix

Automated alignment adjustment for LLMs — direct steering, LoRA, and MoE expert-granular abliteration, optimized via multi-objective Optuna TPE.

peer-pytorch

Pytorch implementation of the PEER block from the paper, Mixture of A Million Experts, by Xu Owen He at Deepmind

st-moe-pytorch

Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

pytorch-mixtures

One-stop solutions for Mixture of Expert modules in PyTorch.

ternlang-jupyter

Ternlang is a ternary programming language (.tern), a runtime for XAI, MoE-LLMs and autonomous agents, shipped with Agentic CLI and in house SDK/IDE.

soft-moe-pytorch

Implementation of Soft MoE, proposed by Brain's Vision team, in Pytorch

mlx-flash

Run AI models too large for your Mac's memory — expert caching, speculative execution, and 15+ research techniques for MoE inference on Apple Silicon

neural-trees

sklearn-compatible PyTorch implementations of Soft Decision Trees, HMoE, and classifier comparison tests (5×2cv F-test). pip install neural-trees

switch-transformers

Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"

mixture-of-attention

Some personal experiments around routing tokens to different autoregressive attention, akin to mixture-of-experts

sinkhorn-router-pytorch

Sinkhorn Router - Pytorch

lmxlab

Transformer language models on Apple Silicon with MLX

mergoo

Impelementation of Leeroo LLM composer.

mixturelib

The implementation of mixtures for different tasks.

mlxlmprobe

Visual probing and interpretability tool for MLX language models