Search Packages

turboquant-vllm

TurboQuant KV cache compression plugin for vLLM — asymmetric K/V, 8 models validated, consumer GPUs

llm-autotune

Zero-config local LLM optimization for Ollama, LM Studio, and Apple Silicon MLX. Reduces TTFT by 40%, wall time for local agents by 46%, and RAM usage by 3x.

krasis

Krasis is no longer distributed via PyPI. Install from GitHub: https://github.com/brontoguana/krasis

sollol

Super Ollama Load Balancer - Performance-aware routing for distributed Ollama deployments with Ray, Dask, and adaptive metrics

contextpilot

Fast Long-Context Inference via Context Reuse

wildedge-sdk

Python SDK for WildEdge

torch-quant

BladeDISC is an end-to-end DynamIc Shape Compiler project for machine learning workloads.

rabbitllm

Run 70B+ LLMs on a single 4GB GPU — no quantization required. Layer-streaming inference for consumer hardware.

kito

Optimize layers structure of Keras model to reduce computation time

thinkrouter

Cut LLM reasoning-token costs by 60% with one line of code

llm-autobatch

Turn single LLM calls into fast micro-batches. Rust core, Python API.