A Q-Learning algorithm for flexible job shop scheduling in a real-world manufacturing scenario

Jessica Coto Palacio, Yailen Martinez Jimenez, Leander Schietgat, Bart Van Doninck, Ann Nowe

Research output: Chapter in Book/Report/Conference proceedingConference paper

20 Downloads (Pure)

Abstract

In this work we propose a Reinforcement Learning approach for a real-world flexible job shop scheduling scenario, where a two-armed robot and a human operator share two workstations in order to assemble light switches. The approach returns a schedule of predefined assembly actions, taking into account given constraints specific to the scenario, such as which actions should precede others, which actions should occur together, or which actions should occur separately. In order to avoid collisions, the work area is divided into zones and all these constraints are taken into account by the algorithm in order to propose a schedule that optimizes the time it takes to prepare the customer orders. The approach that is taken is a reinforcement learning approach. There both arms and the operator are considered to be agents, and they learn from experience who should do which job best, taking into account the above mentioned constraints, as well as the estimated timings to execute the tasks. These estimated timings can be online updated, as well as their uncertainty. The experiments show that the proposed algorithm outperforms the currently used algorithm, which is a constraint solver, in terms of computation time and it is more user friendly with respect to expressing constraints. It also allows to take into account the uncertainty on the execution times of the tasks, and to add slack at critical timeslots, without unnecessarily increasing the makespan. Moreover, the algorithm allows to recompute an updated solution dynamically and efficiently when it turns out an action cannot be performed within the proposed time due to unexpected events. While we show the benefits in a scenario with a two-armed robot collaborating with a human for assembling light switches, the approach is relevant for many assembly tasks in the manufacturing industry where machines and human operators work together.
Original languageEnglish
Title of host publication9th CIRP Conference on Assembly Technology and Systems
PublisherElsevier
Pages227
Number of pages232
Publication statusPublished - Mar 2022

Fingerprint

Dive into the research topics of 'A Q-Learning algorithm for flexible job shop scheduling in a real-world manufacturing scenario'. Together they form a unique fingerprint.

Cite this