Llama Python Packages

sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

strands-agents

A model-driven approach to building AI agents in just a few lines of code.

torchao

PyTorch native quantization and sparsity for training and inference

strands-agents-tools

A set of tools that gives agents powerful capabilities.

unsloth

Web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

unsloth-zoo

Web UI for training and running open models like Gemma 4, Qwen3.6, DeepSeek, gpt-oss locally.

aider-chat

aider is AI pair programming in your terminal

liger-kernel

Efficient Triton Kernels for LLM Training

curated-transformers

🤖 A PyTorch library of curated Transformer models and their composable components

gptcache

Semantic cache for LLMs. Fully integrated with LangChain and llama_index.

sglang-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

sgl-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

ms-swift

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-R1, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).

strands-agents-builder

An example agent demonstrating streaming, tool use, and interactivity from your terminal. This agent builder can help you to build your own agents and tools.

vllm-tpu

A high-throughput and memory-efficient inference and serving engine for LLMs

tensorzero

TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.

transformerlab

The open source research environment for AI researchers to seamlessly train, evaluate, and scale models from local hardware to GPU clusters.

liger-kernel-nightly

Efficient Triton Kernels for LLM Training

xinference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

paddlenlp

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

astrbot

AI Agent Assistant that integrates lots of IM platforms, LLMs, plugins and AI feature, and can be your openclaw alternative. ✨

llamafactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

shell-gpt

A command-line productivity tool powered by AI large language models like GPT-5, will help you accomplish your tasks faster and more efficiently.