Model Compression Python Packages

tensorflow-model-optimization

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

deepcache

[CVPR 2024] DeepCache: Accelerating Diffusion Models for Free

nni

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

torch-pruning

[CVPR 2023] DepGraph: Towards Any Structural Pruning; LLMs, Vision Foundation Models, etc.

tf-model-optimization-nightly

A suite of tools that users, both novice and advanced can use to optimize machine learning models for deployment and execution.

kxy

A Powerful Serverless Pre-Learning and Post-Learning Analysis Toolkit

micronet

A model compression and deploy lib.

fasterai

FasterAI: Prune and Distill your models with FastAI and PyTorch

picollm

On-device LLM Inference Powered by X-Bit Quantization

nni-daily

Neural Network Intelligence package

uni-layer

30+ layer contribution metrics from 7 theoretical categories for PyTorch model compression. Bridges for Torch-Pruning and PEFT/LoRA.

torch-optim

PyTorch models optimization by neural network pruning

aquvitae

The easiest Knowledge Distillation library for Light Weight DeepLearning

model-quantizer

A tool for quantizing large language models

octopus-ml

A collection of handy ML and data visualization and validation tools. Go ahead and train, evaluate and validate your ML models and data with minimal effort.

only-train-once

Only Train Once (OTO): Automatic One-Shot General DNN Training and Compression Framework

picollmdemo

picoLLM Inference Engine demos

prismkv

3-D Stacked-Plane KV Cache Quantizer — defensive prior art publication

kd-lib

A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.

musco-pytorch

MUSCO: Multi-Stage COmpression of Neural Networks

qsparse

Train neural networks with joint quantization and pruning on both weights and activations using any pytorch modules

archai

Platform for Neural Architecture Search

fused-turboquant

Fused Triton encode/decode kernels for TurboQuant KV cache compression, powered by Randomized Hadamard Transform.

bfas

Brute Force Architecture Search