Distributed Training Python Packages

timm

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

skypilot

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).

paddlepaddle

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

metaflow

Build, Manage and Deploy AI/ML Systems

skypilot-nightly

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).

torchx

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

torchx-nightly

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

tensorcircuit-nightly

The next-gen AI-native tensor-network-based quantum software framework

paddlepaddle-gpu

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

metaflow-stubs

Build, Manage and Deploy AI/ML Systems

paddlenlp

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

trainy-skypilot-nightly

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).

tool-helpers

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

fast-dataindex

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

hivemind

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

fedml

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

telescope-ui

Real-time visualization dashboard for LLM training runs

basetrainer

Pytorch分布式训练框架

adaptdl

Resource-adaptive cluster scheduler for deep learning training.

macfleet

Pool Apple Silicon Macs for distributed compute and ML training

codeflare-torchx

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

ddpw

A lightweight wrapper that bootstraps PyTorch's Distributed (Data) Parallel.

fast-tokenizer-python

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

redco

NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference