Parallelism Python Packages

tbb

oneAPI Threading Building Blocks (oneTBB)

cache-dit

A PyTorch-native inference engine with cache, parallelism, quantization for Diffusion Transformers.

tbb-devel

oneAPI Threading Building Blocks (oneTBB)

portal

Fast and reliable distributed systems in Python

paxml

Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimentation and parallelization, and has demonstrated industry leading model flop utilization rates.

central-mcp

Central station for multi project management using mcp and cli wrapper

octobot-channels

Python async multi-task communication library. Used by OctoBot projects.

tbb4py

oneAPI Threading Building Blocks (oneTBB)

async-channel

Python async multi-task communication library. Used by OctoBot projects.

detox

distributed tox (tox plugin to run testenvs in parallel)

riko

A Python stream processing engine modeled after Yahoo! Pipes

tabular-ml-toolkit

A helper library to jumpstart your machine learning project based on tabular or structured data.

lbaf

Analysis framework for exploring, testing, and comparing load balancing strategies

free-threading

Thread-first true parallelism for Python

torchgpipe

A GPipe implementation in PyTorch

syntax-sugar

A library adding some anti-Pythonic syntatic sugar to Python

asyncpal

Preemptive concurrency and parallelism for sporadic workloads

zerofun

Fast and reliable distributed systems in Python

bftool-pkg-sulcud

A custom worker pool to handle the distribution of function between threads and processes

sshp-bin

Parallel SSH Executor

prismalog

A high-performance Python logging library designed for multiprocessing environments. Features colored output, multiprocessing support, and flexible configuration. Benchmarks show up to 29K msgs/sec with minimal memory footprint. Zero external dependencies.

distrifuser

[CVPR 2024 Highlight] DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models

pyeventbus

Python Eventbus

fastiter

High-performance parallel iterators for Python 3.14+ free-threaded mode