Llama2 Python Packages

openllm

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

ludwig

Low-code framework for building custom LLMs, neural networks, and other AI models

openllm-core

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

opencompass

OpenCompass is an LLM evaluation platform, supporting a wide range of models (Llama3, Mistral, InternLM2,GPT-4,LLaMa2, Qwen,GLM, Claude, etc) over 100+ datasets.

friendli-client

[⛔️ DEPRECATED] Friendli: the fastest serving engine for generative AI

google-jetstream

JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).

maxtext

MaxText is a simple, performant and scalable Jax LLM!

openllm-client

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

python-tgpt

AI Chat in Terminal + Package + REST-API

multillm

🚀 Invoke multiple large language models concurrently and the rank results. Add new models and ranking functions quickly.. ⭐

anyrobo

A local-first voice AI assistant framework. Create your own JARVIS.

llmgraph

Create knowledge graphs with LLMs

exxa

Exa - Pytorch

autollm

Ship RAG based LLM web apps in seconds.

nlp-models

Transformers based NLP models

llm-templates

Instruction/chat prompts creation library for text generation LLMs. It supports local and Hugging Face models.

bert-classifier

A repository for training transformer based models

llama-recipes

This is a compatibility package to keep projects build on llama-recipes compatible with the new name llama-cookbook. If you're updating your project or starting a new one please use llama-cookbook package

picollm

On-device LLM Inference Powered by X-Bit Quantization

gguf-modeldb

A quick and optimized solution to manage llama based gguf quantized models, download gguf files, retreive messege formatting, add more models from hf repos and more. It's super easy to use and comes prepacked with best preconfigured open source models: dolphin phi-2 2.7b, mistral 7b v0.2, mixtral 8x7b v0.1, solar 10.7b and zephyr 3b

llava-torch

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

llama-cookbook

Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model family and using them on various provider services

gguf-llama

Wrapper for simplified use of Llama2 GGUF quantized models.

llama2-wrapper

Use llama2-wrapper as your local llama2 backend for Generative Agents / Apps