Parallelization Python Packages

swifter

A package which efficiently applies any function to a pandas dataframe or series in the fastest available manner

gamma-pytools

Foundational tools for BCG X's data science packages.

bonobo

Extract Transform Load for Python 3.5+

parmap

Easy to use map and starmap python equivalents

sdaxen-python-utilities

Useful tools for common Python tasks.

sciris

Fast, flexible tools to simplify scientific Python

dprs

Directed percolation and related models implemented in Rust and wrapped in Python

covalent

Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.

parl

A high-performance distributed training framework for Reinforcement Learning

cova

Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.

aloisius

Create/Update/Delete AWS CloudFormation stacks in parallel

bluepyparallel

Provides an embarrassingly parallel tool with sql backend

django-collectfaster

Parallel file copying for Django's collectstatic.

covalent-awsbatch-plugin

Executor plugin interfacing Covalent with AWS Batch

qc-parallelizer

A module for optimally combining and distributing quantum circuits

covalent-ssh-plugin

Covalent SSH Plugin

covalent-awslambda-plugin

Executor plugin interfacing Covalent with AWS Lambda

covalent-ecs-plugin

Covalent ECS Plugin

covalent-slurm-plugin

Covalent Slurm Plugin

covalent-gcpbatch-plugin

Executor plugin interfacing Covalent with GCP Batch

covalent-braket-plugin

Covalent Braket Plugin

ml-impute

A package for synthetic data generation for imputation using single and multiple imputation methods.

pystream-pipeline

A simple and fast real-time data pipeline constructor package for Python

magda

Library for building Modular and Asynchronous Graphs with Directed and Acyclic edges (MAGDA)