Transformer Python Packages

sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

faster-whisper

Faster Whisper transcription with CTranslate2

torchao

PyTorch native quantization and sparsity for training and inference

onnx2tf

A tool for converting ONNX files to LiteRT/TFLite/TensorFlow, PyTorch native code (nn.Module), TorchScript (.pt), state_dict (.pt), Exported Program (.pt2), and Dynamo ONNX. It also supports direct conversion from LiteRT to PyTorch.

lm-eval

A framework for few-shot evaluation of language models.

curated-transformers

🤖 A PyTorch library of curated Transformer models and their composable components

vllm-omni

A framework for efficient model inference with omni-modality models

mmdet

OpenMMLab Detection Toolbox and Benchmark

conformer

Implementation of the convolutional module from the Conformer paper, for use in Transformers

neuralforecast

Scalable and user friendly neural :brain: forecasting algorithms.

mnn

MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.

sglang-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

sgl-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

text-generation

Large Language Model Text Generation Inference

vllm-tpu

A high-throughput and memory-efficient inference and serving engine for LLMs

multilspy

multilspy is a lsp client library in Python intended to be used to build applications around language servers.

mmsegmentation

OpenMMLab Semantic Segmentation Toolbox and Benchmark.

pytorch-pretrained-bert

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

torchscale

Foundation Architecture for (M)LLMs

auto-gptq

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

linear-attention-transformer

Transformer based on a variant of attention that is linear complexity in respect to sequence length

linformer

Implementation of Linformer for Pytorch