Vision Language Model Python Packages

mlx-vlm

MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.

colpali-engine

The code used to train and run inference with the ColVision models, e.g. ColPali, ColQwen2, and ColSmol.

mlx-tune

Fine-tune LLMs on your Mac with Apple Silicon. SFT, DPO, GRPO, Vision, TTS, STT, Embedding, and OCR fine-tuning — natively on MLX. Unsloth-compatible API.

lmms-eval

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

x-anylabeling-cvhub

Effortless data labeling with AI support from Segment Anything and other awesome models.

vidore-benchmark

Vision Document Retrieval (ViDoRe): Benchmark. Evaluation code for the ColPali paper.

thepipe-api

Get clean data from tricky documents, powered by vision-language models ⚡

lica-gdb

GDB: GraphicDesignBench — benchmark suite for evaluating vision-language models on graphic design tasks

x-anylabeling

Effortless data labeling with AI support from Segment Anything and other awesome models.

nerif

LLM powered Python

mbodied

Seamlessly integrate state-of-the-art transformer models into robotics stacks

lica-gdb-helm

GDB: GraphicDesignBench - A real-world benchmark for evaluating AI on graphic design tasks

urban-worm

Workflow of reproducible multimodal inference for urban environment evaluation.

llava-torch

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

kani-vision

Kani extension for supporting vision-language models (VLMs). Comes with model-agnostic support for GPT-Vision and LLaVA.

kanoa

AI-powered interpretation of data science outputs with multi-backend support (Molmo, Gemini, Claude, OpenAI)

visionzip

Official repository for VisionZip (CVPR 2025)

unsloth-mlx

Fine-tune LLMs on your Mac with Apple Silicon. SFT, DPO, GRPO, Vision, TTS, STT, Embedding, and OCR fine-tuning — natively on MLX. Unsloth-compatible API.

kani-multimodal-core

Core shared libraries for multimodal Kani extensions.

ps3-torch

Scaling Vision Pre-Training to 4K Resolution

german-ocr

High-performance German document OCR - Local & Cloud with GPU/CPU support

gptparse

Document parser for RAG

openscenesense-ollama

Offline video analysis using Ollama models and local Whisper

litepali

Lightweight ColPali-based retrieval for cloud