Search Packages

sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

sglang-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

sgl-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

vllm-tpu

A high-throughput and memory-efficient inference and serving engine for LLMs

tensorrt-llm

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

sglang-kt

SGLang is a high-performance serving framework for large language models and multimodal models.

pyptx

A Python DSL to write Nvidia PTX for Hopper and Blackwell in JAX and PyTorch

pygpukit

Minimal GPU runtime for Python - high-performance CUDA kernels, memory management, and LLM inference without heavy dependencies

penguin-burner

Nvidia ultimate undervolting companion on Linux. Now with a nice UI. Supports MSI Afterburner profile imports and LACT profile exports. Can automatically scan for the most optimal GPU VF curve and generate silent fan curves.

dblcsgen

DBLC Fast Structured Generation

vllm-hust

A high-throughput and memory-efficient inference and serving engine for LLMs

wxy-test

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-xft

A high-throughput and memory-efficient inference and serving engine for LLMs

ai-dynamo-vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-acc

A high-throughput and memory-efficient inference and serving engine for LLMs

nextai-vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-consul

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-npu

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-musa

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-rocm

A high-throughput and memory-efficient inference and serving engine for LLMs

flash-attention-triton

Cross-platform FlashAttention-2 Triton implementation for Turing+ GPUs with custom configuration mode

sgltmp

SGLang is yet another fast serving framework for large language models and vision language models.

vllm-emissary

A high-throughput and memory-efficient inference and serving engine for LLMs