<div align="center"> <img src="docs/assets/JobShopLabLogo.svg" alt="JobShopLab Logo" width="100"/> </div>

JobShopLab: A Scheduling Framework for Reinforcement Learning Under Real-World Constraints

<!-- <p align="center"> <a href="https://lbesson.mit-license.org/" alt="Backers on Open Collective"> <img src="https://badgen.net/github/license/micromatch/micromatch" /></a> <a href="https://proto-lab-ro.github.io/jobshoplab/" alt="Backers on Open Collective"> <img src="https://img.shields.io/readthedocs" /></a> </p> -->

JobShopLab is a flexible and modular framework designed to advance research and development in job shop scheduling using Reinforcement Learning (RL) techniques. It provides an adaptable Gym environment, enabling users to test and benchmark different scheduling algorithms under realistic constraints found in industrial settings.

<!--  --> <div align="center"> <img src="docs/assets/3d_sim.gif" alt="JobShopLab Logo" width="800"/> </div> The GIF above showcases our framework in action, visualizing an automated production process where an RL agent dynamically assigns task to machines and vehicles. Colored blocks represent orders moving through different production stations, while AI makes real-time decisions to optimize order allocation and scheduling.

Features

• Modular Gym Environment: A customizable and extensible framework for testing diverse scheduling strategies and problem specifications.
• Reinforcement Learning Ready: Seamless integration with RL algorithms via the standard Gym Interface.
• Real-World Constraints: Incorporates transport logistics, buffer management, machine buffer configuration, machine breakdowns, setup times, and stochastic processing conditions.
• Multi-Objective Optimization: Supports scheduling based on multiple objectives, such as makespan, energy efficiency, machine utilization, and lead time.
• Pip Installable: Easy installation from the repository, ensuring quick setup and integration into existing projects.

Installation

JobShopLab requires Python 3.12 or higher.

To install JobShopLab, clone the repository and install it in editable mode using pip.

cd <desired_dir>

# ssh
git clone git@github.com:proto-lab-ro/jobshoplab.git

# or https
git clone https://github.com/proto-lab-ro/jobshoplab.git

# install python module in editable mode
pip install -e <repo_dir>

Replace <desired_dir> with your target directory and <repo_dir> with the path to your local clone of the JobShopLab repository.

Getting Started

After installation, you can initialize and interact with JobShopLab in your Python scripts as follows:

from pathlib import Path

from jobshoplab import JobShopLabEnv, load_config

# Load a pre-defined configuration
config = load_config(config_path=Path("./data/config/getting_started_config.yaml"))

# Create the environment
env = JobShopLabEnv(config=config)

# Run with random actions until done
done = False
while not done:
    action = env.action_space.sample()
    obs, reward, truncated, terminated, info = env.step(action)
    done = truncated or terminated

# Visualize the final schedule
env.render()

We also provide a Getting Started Jupyter Notebook that walks you through the framework setup, environment interaction, and running basic reinforcement learning experiments. You can find this notebook in the repository under jupyter/getting_started.ipynb.

Framework Overview

JobShopLab extends the classical Job Shop Scheduling Problem (JSSP) by considering real-world production constraints and enabling RL-based optimization. It provides a state-machine-based simulation model that includes:

• Machines: Modeled with setup times, breakdowns, stochastic processing, and configurable prebuffer/postbuffer settings.
• Transport Units: Handling job movements between machines with delays and constraints.
• Buffers: Limited storage capacity impacting scheduling decisions, with machine-specific buffer configurations.

Experiments

To validate the framework, we trained an RL agent and compared its scheduling performance against traditional Priority Dispatch Rules (PDRs). Using the PPO algorithm from Stable Baselines3, the agent learned to optimize makespan efficiently. Compared to heuristic methods like Shortest Processing Time (SPT) and Most Work Remaining (MWKR), the RL-based approach achieves superior scheduling performance out of the box, in the standard academic cases and also with significantly increased complexity due to additional constraints such as buffer and transport constraints.

Academic instances found in Literature. Definitions can be found in data/jssp_instances/*.yaml

| Instance | mwkr | spt | lb | rl | | -------- | ---- | ---- | ---- | ---- | | ft06 | | | 55 | 55 | | ft10 | | | 930 | 981 | | la16 | 1238 | 1588 | 945 | 978 | | ta01 | 1786 | 1872 | 1231 | 1231 | | ta41 | 2632 | 3067 | 2005 | 2496 |

Note on RL Results: Results from individual hyperparameter optimizations

Extendet real-world constrains

Academic instances with arbitrary transport times between machines definitions can be found in data/instances/jssptransport/*.yaml

| Instance | fifo| mwkr | spt | rl| |---------|---------|---------|---------|---------| | ft06-t | 83 | 85 | 85 | 76 | | ft10-t | 1463 | 1286 | 1167 | 1061 | | ft20-t | 1714 | 1763 | 1790 | 1567 | | la01-t | 1200 | 1221 | 936 | 771 | | la02-t | 973 | 993 | 1023 | 782 | | la03-t | 971 | 1098 | 945 | 789 | | la04-t | 1122 | 1113 | 1113 | 828 | | la05-t | 784 | 851 | 776 | 642 | | la06-t | 1405 | 1366 | 1240 | 1112 | | la07-t | 1419 | 1332 | 1356 | 1127 | | la08-t | 1363 | 1337 | 1357 | 1143 | | la09-t | 1379 | 1403 | 1332 | 1120 | | la10-t | 1295 | 1205 | 1226 | 1044 | | la11-t | 1534 | 1689 | 1675 | 1409 | | la12-t | 1552 | 1638 | 1447 | 1240 | | la13-t | 1782 | 1599 | 1689 | 1400 | | la14-t | 2036 | 1954 | 1883 | 1710 | | la15-t | 1719 | 1752 | 1631 | 1512 | | la16-t | 1502 | 1634 | 1338 | 1186 | | la17-t | 1307 | 1318 | 1318 | 999 | | la18-t | 1226 | 1235 | 1261 | 1079 | | la19-t | 1147 | 1409 | 1129 | 1075 | | la20-t | 1476 | 1359 | 1195 | 1123 | | la21-t | 1675 | 1655 | 1575 | 1355 | | la22-t | 1637 | 1920 | 1499 | 1341 | | la23-t | 1617 | 1504 | 1450 | 1325 | | la24-t | 1774 | 1616 | 1620 | 1277 |

Note on RL Results: Training was performed with one set of hyperparameter over all instances

Testing

JobShopLab uses pytest for testing. The test suite includes unit tests, integration tests, and end-to-end tests.

To run the test suite:

# Run all tests
pytest

# Run tests with coverage report
./scripts/get_test_coverage.sh

# Run specific test categories
pytest tests/unit_tests/
pytest tests/integration_tests/

Contributing

We welcome contributions to JobShopLab! If you have ideas for improvements or bug fixes, feel free to submit an issue, or pull request on our repository.

How to Contribute

1. Fork the repository.
2. Create a new branch for your feature or fix.
3. Implement your changes and ensure they are well-documented.
4. Submit a pull request with a detailed explanation of your modifications.

Acknowledgment

This work has been supported by the European Union’s Horizon Europe research and innovation programme under the following Chips JU projects.

• A-IQ Ready (Grant Agreement No. 101096658) – HORIZON-KDT-JU-2021-2-RIA
• Cynergy4MIE (Grant Agreement No. 101140226) – HORIZON-KDT-JU-2023-2-RIA-Topic-1

The views and opinions expressed are those of the authors and do not necessarily reflect those of the European Union or the granting authority. Neither the European Union nor the granting authority can be held responsible for them.