Search Packages

sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

flashinfer-python

FlashInfer: Kernel Library for LLM Serving

flashinfer-cubin

FlashInfer: Kernel Library for LLM Serving

sglang-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

sgl-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

ms-swift

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-R1, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).

vllm-tpu

A high-throughput and memory-efficient inference and serving engine for LLMs

llamafactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

tensorrt-llm

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

sglang-kt

SGLang is a high-performance serving framework for large language models and multimodal models.

terradev-cli

Cross-Cloud Compute Optimization Platform with Migration & Evaluation - v4.0.12

awex

A high-performance RL training-inference weight synchronization framework, designed to enable second-level parameter updates from training to inference in RL workflows

abliterix

Automated alignment adjustment for LLMs — direct steering, LoRA, and MoE expert-granular abliteration, optimized via multi-objective Optuna TPE.

llmtuner

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

mlx-flash

Run AI models too large for your Mac's memory — expert caching, speculative execution, and 15+ research techniques for MoE inference on Apple Silicon

uccl

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

switch-transformers

Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"

plato-edge

Edge-optimized Cocapn fleet packages for ARM64 — pure Python, zero deps, <100KB

dblcsgen

DBLC Fast Structured Generation

vllm-hust

A high-throughput and memory-efficient inference and serving engine for LLMs

lazyllm-llamafactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

wxy-test

A high-throughput and memory-efficient inference and serving engine for LLMs

kmoe

Command-line manga downloader for kxx.moe / kzz.moe / koz.moe