PyPI Stats

compressed-tensors

Packages that depend on this package
24 dependents
Package Description Downloads/month
sgl-project sglang
SGLang is a high-performance serving framework for large language models and mul... 287.7M
vllm-project vllm
A high-throughput and memory-efficient inference and serving engine for LLMs 9.4M
vllm-project llmcompressor
Transformers-compatible library for applying various compression algorithms to L... 285K
vllm-project vllm-tpu
A high-throughput and memory-efficient inference and serving engine for LLMs 143K
MekayelAnik vllm-cpu
Wheels & Docker images for running vLLM on CPU-only systems, optimized for diffe... 31K
aphrodite-engine aphrodite-engine
Large-scale LLM inference engine 7K
vllm-project vllm-ascend
Community maintained hardware plugin for vLLM on Ascend 7K
sgl-project sglang-kt
SGLang is a high-performance serving framework for large language models and mul... 4K
MekayelAnik vllm-cpu-avx512bf16
Wheels & Docker images for running vLLM on CPU-only systems, optimized for diffe... 3K
MekayelAnik vllm-cpu-avx512vnni
vLLM CPU inference engine (AVX512 + VNNI optimized) 3K
MekayelAnik vllm-cpu-amxbf16
Wheels & Docker images for running vLLM on CPU-only systems, optimized for diffe... 3K
MekayelAnik vllm-cpu-avx512
vLLM CPU inference engine (AVX512 optimized) 2K
tokoro10g nori-captioner
Simple local web UI for captioning image/video datasets with optional local VLM ... 487
vllm-kunlun
vLLM Kunlun3 backend plugin 464
vllm-project vllm-hust
A high-throughput and memory-efficient inference and serving engine for LLMs 437
vllm-project wxy-test
A high-throughput and memory-efficient inference and serving engine for LLMs 375
vllm-project ai-dynamo-vllm
A high-throughput and memory-efficient inference and serving engine for LLMs 344
mmirage
Modular Multimodal Intelligent Reformatting and Augmentation Generation Engine -... 260
RomanMoz power-sglang-cuda124
SGLang fork for ppc64le with CUDA 12.4 and Torch Triton support 186
vllm-project vllm-emissary
A high-throughput and memory-efficient inference and serving engine for LLMs 132
vllm-project vllm-usf
A high-throughput and memory-efficient inference and serving engine for LLMs 115
vllm-project vllm-test-tpu
A high-throughput and memory-efficient inference and serving engine for LLMs 80
vllm-project vllm-fixed
A high-throughput and memory-efficient inference and serving engine for LLMs 42
sgl-project sglang-cpu
SGLang is a fast serving framework for large language models and vision language... 2