Search Packages

petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

maxtext

MaxText is a simple, performant and scalable Jax LLM!

picollm

On-device LLM Inference Powered by X-Bit Quantization

groqchat

A terminal chatbot, powered by Groq Cloud API (Windows / macOS / Linux / Android / iOS)

easy-llms

Easy '1-line' calling of every LLM from OpenAI, MS Azure, AWS Bedrock, GCP Vertex, and Ollama

gchat

A terminal chatbot, powered by Groq Cloud API (Windows / macOS / Linux / Android / iOS)

picollmdemo

picoLLM Inference Engine demos

nollama

A terminal-based interface for interacting with large language models (LLMs)

githubaipy

Small python wrapper of Github Models API

h2ogpt

Private chat with local GPT with document, images, video, etc. 100% private, Apache 2.0. Supports oLLaMa, Mixtral, llama.cpp, and more. Demo: https://gpt.h2o.ai/ https://gpt-docs.h2o.ai/

mixtral-py

A Python module for running the Mixtral-8x7B language model with customisable precision and attention mechanisms.

test-petals

Easy way to efficiently run 100B+ language models without high-end GPUs