Gym Python Packages

gymnasium

An API standard for single-agent reinforcement learning environments, with popular reference environments and related utilities (formerly Gym)

stable-baselines3

PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.

pettingzoo

An API standard for multi-agent reinforcement learning environments, with popular reference environments and related utilities

sb3-contrib

Contrib package for Stable-Baselines3 - Experimental reinforcement learning (RL) code

reasoning-gym

[NeurIPS 2025 Spotlight] Reasoning Environments for Reinforcement Learning with Verifiable Rewards

skrl

Modular Reinforcement Learning (RL) library (implemented in PyTorch, JAX, and NVIDIA Warp) with support for Gymnasium/Gym, NVIDIA Isaac Lab, MuJoCo Playground and other environments

minigrid

Simple and easily configurable grid world environments for reinforcement learning

gym-minigrid

Simple and easily configurable grid world environments for reinforcement learning

envpool

C++-based high-performance parallel environment execution engine (vectorized env) for general RL environments.

mo-gymnasium

Multi-objective Gymnasium environments for reinforcement learning

gem-llm

A Gym for Agentic LLMs

rl-zoo3

A training framework for Stable Baselines3 reinforcement learning agents, with hyperparameter optimization and pre-trained agents included.

lightzero

[NeurIPS 2023 Spotlight] LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision Scenarios (awesome MCTS)

stable-baselines

A fork of OpenAI Baselines, implementations of reinforcement learning algorithms

vmas

VMAS is a vectorized differentiable simulator designed for efficient Multi-Agent Reinforcement Learning benchmarking. It is comprised of a vectorized 2D physics engine written in PyTorch and a set of challenging multi-robot scenarios. Additional scenarios can be implemented through a simple and modular interface.

sumo-rl

Reinforcement Learning environments for Traffic Signal Control with SUMO. Compatible with Gymnasium, PettingZoo, and popular RL libraries.

gym-ignition

A toolkit for developing OpenAI Gym environments simulated with Ignition Gazebo.

scenario

Framework for developing OpenAI Gym robotics environments simulated with Ignition Gazebo

gym-sokoban

Sokoban environment for OpenAI Gym

train

A library to build and train reinforcement learning agents in OpenAI Gym environments.

pyrddlgym

A toolkit for auto-generation of OpenAI Gym environments from RDDL description files.

evox

Distributed GPU-Accelerated Framework for Evolutionary Computation. Comprehensive Library of Evolutionary Algorithms & Benchmark Problems.

gym-trading-env

A simple, easy, customizable Open IA Gym environments for trading.

rtgym

Easily implement custom Gymnasium environments for real-time applications