Large Scale Python Packages

camel-oasis

🏝️ OASIS: Open Agent Social Interaction Simulations with One Million Agents.

colossalai

Making large AI models cheaper, faster and more accessible

es-pandas

Read, write and update large scale pandas DataFrame with Elasticsearch

torchdr

TorchDR - PyTorch Dimensionality Reduction

colossalai-nightly

An integrated large-scale model training system with efficient parallelization techniques

parl

A high-performance distributed training framework for Reinforcement Learning

ms-mint-app

Metabolomics Integrator Interactive Web-Application (Mint)

brayns

Brayns Python API

facedb

A package designed for efficient face recognition across extensive photo collections, optimized for large-scale processing.

quick-batch

ultra simple command line tool for docker-scaling batch processing

ms-mint-app2

MS-MINT is a Python application for processing, analyzing, and visualizing large liquid chromatography-mass spectrometry (LC-MS) datasets

pyccea

A Python package of cooperative co-evolutionary algorithms for feature selection in high-dimensional data.

fleet-x

飞桨大模型开发套件，提供大语言模型、跨模态大模型、生物计算大模型等领域的全流程开发工具链。

custom-colossalai

Making large AI models cheaper, faster and more accessible

data-selection

Data Selection with Importance Resampling

redis-bulk-cleaner

Deletes keys from Redis database in bulk.

neorl

NeuroEvolution Optimization with Reinforcement Learning

ringer

Large-scale data structures hosted on the file-system

fleet-lightning

飞桨大模型开发套件，提供大语言模型、跨模态大模型、生物计算大模型等领域的全流程开发工具链。

gigl

Gigantic Graph Learning (GiGL) Framework: Large-scale training and inference for Graph Neural Networks

ls-mcmc

A light-weight library for large-scale Markov Chain Monte Carlo sampling

plsc

Paddle Large Scale Classification Tools，supports ArcFace, CosFace, PartialFC, Data Parallel + Model Parallel. Model includes ResNet, ViT, Swin, DeiT, CaiT, FaceViT, MoCo, MAE, ConvMAE, CAE.

xdeeprank

An eXtensible Package of Deep Learning based Ranking Models for Large-scale Industrial Recommender System with Tensorflow

paddle-fleet

Distributed Training Package Based on PaddlePaddle