Llama Cpp Python Packages

cyllama

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

lilbee

Terminal-first local search and AI chat over your documents, code, and crawled websites. Semantic + hybrid search, vision OCR, auto-built wiki, browsable GGUF model catalog. Works as CLI, TUI, MCP server, REST API, or Python library. Offline by default, no sidecar services.

cyllama-rocm

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

cyllama-cuda12

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

cyllama-vulkan

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

cyllama-sycl

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

llama-cpp-py-sync

Auto-synced CFFI ABI python bindings for llama.cpp with prebuilt wheels (CPU/CUDA/Vulkan/Metal).

palacelite

轻量级本地 AI 记忆系统

egovault

Local-first personal data vault with LLM enrichment and RAG chat

llama-buddy

CLI wrapper for llama.cpp providing an ollama-like experience

edgevox

Offline voice agent framework for robots.

tightwad

Mixed-vendor GPU inference cluster manager with speculative decoding

sollol

Super Ollama Load Balancer - Performance-aware routing for distributed Ollama deployments with Ray, Dask, and adaptive metrics

mycellm

Distributed LLM inference across heterogeneous hardware. Pool GPUs into a P2P network with QUIC transport, Ed25519 identity, and an OpenAI-compatible API.

forge-guardrails

A Python framework for self-hosted LLM tool-calling and multi-step agentic workflows

inzen-bot

Multi-provider AI chatbot for the terminal. Chat with Claude, GPT-4, Ollama or llama.cpp without leaving your shell. Full conversation history. pip install inzen-bot

openai-agents-redis

Native OpenAI Agents SDK session management implementation using Redis as the persistence layer.

notolog

Notolog Markdown Editor

llama-server

LLaMA Server combines the power of LLaMA C++ with the beauty of Chatbot UI.

grammarflow

Powering Agent Chains by Constraining LLM Outputs

inferna

an early-stage experimental nanobind wrapper around llama.cpp

langport

A large language model serving platform.

corellm

Efficient LLM framework for local GGUF models with streaming and GUI

cognitive-cache

Optimal context window selection for LLM coding tools. Treats context as a constrained optimization problem, not retrieval. Beats RAG, grep, and LLM-triage baselines on real GitHub issues.