Prompt Engineering Python Packages

mlflow-skinny

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

mlflow

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

mlflow-tracing

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

partial-json-parser

Parse partial JSON generated by LLM

opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

banks

LLM prompt language based on Jinja. Banks provides tools and functions to build prompts text and chat messages from generic blueprints. It allows attaching metadata to prompts to ease their management, and versioning is first-class citizen. Banks provides ways to store prompts on disk along with their metadata.

arize-phoenix

AI Observability & Evaluation

arize-phoenix-otel

AI Observability & Evaluation

outlines

Structured Outputs

arize-phoenix-client

AI Observability & Evaluation

arize-phoenix-evals

AI Observability & Evaluation

promptflow-core

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

promptflow-tracing

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

promptflow-devkit

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

judgeval

The open source post-building layer for agents. Our environment data and evals power agent post-training (RL, SFT) and monitoring.

llm-guard

The Security Toolkit for LLM Interactions

promptflow

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

promptflow-tools

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

promptflow-azure

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.

swarms

The Enterprise-Grade Production-Ready Multi-Agent Orchestration Framework. Website: https://swarms.ai

ell-ai

A language model programming library.

opik-optimizer

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

agenta

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

langchain-decorators

syntactic sugar 🍭 for langchain