Multi Modal Python Packages

modelscope

ModelScope: bring the notion of Model-as-a-Service to life.

agentscope

Build and run agents you can see, understand and trust.

docarray

Represent, send, store and search multimodal data

pyrhubarb

A Python framework for multi-modal document understanding with Amazon Bedrock

medmnist

[pip install medmnist] 18x Standardized Datasets for 2D and 3D Biomedical Image Classification

pyvalhalla

Open Source Routing Engine for OpenStreetMap

dalle-pytorch

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

byaldi

Use late-interaction multi-modal models such as ColPali in just a few lines of code.

cn-clip

Chinese version of CLIP which achieves Chinese cross-modal retrieval and representation generation.

transfusion-pytorch

Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

pyvalhalla-weekly

Open Source Routing Engine for OpenStreetMap

qwen

My personal implementation of the model from "Qwen-VL: A Frontier Large Vision-Language Model with Versatile Abilities", they haven't released model code yet sooo...

py-data-juicer

Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

deepke

[EMNLP 2022] An Open Toolkit for Knowledge Graph Extraction and Construction

brainles-preprocessing

preprocessing tools for multi-modal 3D brain imaging

avitai-artifex

A research-focused modular generative modeling library built on JAX/Flax NNX

vision-llama

Implementation of VisionLLaMA from the paper: "VisionLLaMA: A Unified LLaMA Interface for Vision Tasks" in PyTorch and Zeta

switch-transformers

Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"

move-dl

MOVE (Multi-Omics Variational autoEncoder) for integrating multi-omics data and identifying cross modal associations

mm-poe

Multiple Choice Reasoning via. Process of Elimination using Multi-Modal Models

mychatgpt

mychatgpt is a small and useful Python module that provides functions for interacting with OpenAI's GPT models to create conversational agents. This module allows users to have interactive conversations with the GPT models and keeps track of the conversation history in your Python Projects and Jupyter Notebooks.

tiny-gptv

Simple Implementation of TinyGPTV in super simple Zeta lego blocks

hsss

Implementation of a Hierarchical Mamba as described in the paper: "Hierarchical State Space Models for Continuous Sequence-to-Sequence Modeling"

kosmosx

Transformers at zeta scales