Gguf Python Packages

auto-round

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

quantcpp

LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.

llmfit

Hundreds of models & providers. One command to find what runs on your hardware.

gguf-connector

gguf (GPT-Generated Unified Format) connector

auto-round-nightly

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

hf-mem

A CLI to estimate inference memory requirements for Hugging Face models, written in Python.

webscout

Webscout is the all-in-one search and AI toolkit you need. Discover insights with Yep.com, DuckDuckGo, and Phind; access cutting-edge AI models; transcribe YouTube videos; generate temporary emails and phone numbers; perform text-to-speech conversions; and much more!

jang

JANG — GGUF for MLX. YOU MUST USE JANG_Q RUNTIME. Adaptive Mixed-Precision Quantization + Runtime for Apple Silicon

soup-cli

Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done.

auto-round-lib

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

llama-cpp-py-sync

Auto-synced CFFI ABI python bindings for llama.cpp with prebuilt wheels (CPU/CUDA/Vulkan/Metal).

outetts

Interface for OuteTTS models.

llama-buddy

CLI wrapper for llama.cpp providing an ollama-like experience

llama-core

solo connector core built on llama.cpp

gguf-core

a simple way to interact llama with gguf

auto-round-hpu

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

onnxifier

Convert ANY IR to ONNX format

gguf-node

gguf node for comfyui

cgg

call GGUF model

ledgermind

​LedgerMind — an autonomous living memory for AI agents. It self-heals, resolves conflicts, distills experience into rules, and evolves without human intervention. SQLite + Git + reasoning layer. Perfect for multi-agent systems and on-device deployment.

openvino2onnx

Convert ANY IR to ONNX format

notolog

Notolog Markdown Editor

gguf-comfy

run flux1/sd3 model with beginner GPU (low cost) or even CPU

ovos-gguf-embeddings-plugin

A gguf embeddings plugin for OVOS