Large Language Models Python Packages

ray

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

xgrammar

Fast, Flexible and Portable Structured Generation

onnx-ir

Efficient in-memory representation for ONNX, in Python

redisvl

Redis Vector Library (RedisVL) -- the AI-native Python client for Redis.

chronos-forecasting

Chronos: Pretrained Models for Time Series Forecasting

haystack-ai

Open-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.

gliner

Generalist and Lightweight Model for Named Entity Recognition (Extract any entity types from texts)

llm-sandbox

Lightweight and portable LLM sandbox runtime (code interpreter) Python library.

fla-core

🚀 Efficient implementations for emerging model architectures

flash-linear-attention

🚀 Efficient implementations for emerging model architectures

tokencost

Easy token price estimates for 400+ LLMs. TokenOps.

llm-guard

The Security Toolkit for LLM Interactions

megatron-core

Ongoing research training transformer models at scale

deeplake

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

langextract

A Python library for extracting structured information from unstructured text using LLMs with precise source grounding and interactive visualization.

agentscope

Build and run agents you can see, understand and trust.

lightrag-hku

[EMNLP2025] "LightRAG: Simple and Fast Retrieval-Augmented Generation"

reasoning-gym

[NeurIPS 2025 Spotlight] Reasoning Environments for Reinforcement Learning with Verifiable Rewards

camel-ai

🐫 CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents. https://www.camel-ai.org

langflow

Langflow is a powerful tool for building and deploying AI-powered agents and workflows.

langflow-base

Langflow is a powerful tool for building and deploying AI-powered agents and workflows.

multilspy

multilspy is a lsp client library in Python intended to be used to build applications around language servers.

tensorzero

TensorZero is an open-source LLMOps platform that unifies an LLM gateway, observability, evaluation, optimization, and experimentation.

upgini

Intelligent data search & enrichment for Machine Learning