MARLlib

<div align="center">

<img src=docs/source/images/logo1.png width=75% />

</div> <h1 align="center"> MARLlib: A Multi-agent Reinforcement Learning Library </h1> <div align="center">

<img src=docs/source/images/allenv.gif width=99% />

</div>

 

| :exclamation: News | |:-----------------------------------------| | March 2023 :anchor:We are excited to announce that a major update has just been released. For detailed version information, please refer to the version info.| | May 2023 Exciting news! MARLlib now supports five more tasks: MATE, GoBigger, Overcooked-AI, MAPDN, and AirCombat. Give them a try!| | June 2023 OpenAI: Hide and Seek and SISL environments are incorporated into MARLlib.| | Aug 2023 :tada:MARLlib has been accepted for publication in JMLR.| | Sept 2023 Latest PettingZoo with Gymnasium are compatiable within MARLlib.| | Nov 2023 We are currently in the process of creating a hands-on MARL book and aim to release the draft by the end of 2023.|

Multi-agent Reinforcement Learning Library (MARLlib) is a MARL library that utilizes Ray and one of its toolkits RLlib. It offers a comprehensive platform for developing, training, and testing MARL algorithms across various tasks and environments.

Here's an example of how MARLlib can be used:

from marllib import marl

# prepare env
env = marl.make_env(environment_name="mpe", map_name="simple_spread", force_coop=True)

# initialize algorithm with appointed hyper-parameters
mappo = marl.algos.mappo(hyperparam_source='mpe')

# build agent model based on env + algorithms + user preference
model = marl.build_model(env, mappo, {"core_arch": "mlp", "encode_layer": "128-256"})

# start training
mappo.fit(env, model, stop={'timesteps_total': 1000000}, share_policy='group')

Why MARLlib?

Here we provide a table for the comparison of MARLlib and existing work.

| Library | Supported Env | Algorithm | Parameter Sharing | Model |:-------------:|:-------------:|:-------------:|:--------------:|:----------------:| | PyMARL | 1 cooperative | 5 | share | GRU | :x: | PyMARL2| 2 cooperative | 11 | share | MLP + GRU | :x: | MAPPO Benchmark | 4 cooperative | 1 | share + separate | MLP + GRU | :x: | | MAlib | 4 self-play | 10 | share + group + separate | MLP + LSTM | | EPyMARL| 4 cooperative | 9 | share + separate | GRU | :x: | | HARL| 8 cooperative | 9 | share + separate | MLP + CNN + GRU | :x: | | MARLlib | 17 no task mode restriction | 18 | share + group + separate + customizable | MLP + CNN + GRU + LSTM | |

| Library | Github Stars | Documentation | Issues Open | Activity | Last Update |:-------------:|:-------------:|:-------------:|:-------------:|:-------------:|:-------------:| | PyMARL | | :x: | | |
| PyMARL2| | :x: | | |
| MAPPO Benchmark| | :x: | | |
| MAlib | | | | |
| EPyMARL| | :x: | | |
| HARL*| | :x: | | |
| MARLlib | | | | |

* HARL is the latest MARL library that has been recently released:fire:. If cutting-edge MARL algorithms with state-of-the-art performance are your target, HARL is definitely worth a look!

key features

:beginner: MARLlib offers several key features that make it stand out:

• MARLlib unifies diverse algorithm pipelines with agent-level distributed dataflow, allowing researchers to develop, test, and evaluate MARL algorithms across different tasks and environments.
• MARLlib supports all task modes, including cooperative, col