A Q-Learning algorithm for flexible job shop scheduling in a real-world manufacturing scenario

Jessica Coto Palacio, Yailen Martinez Jimenez, Leander Schietgat, Bart Van Doninck, Ann Nowe

Onderzoeksoutput: Conference paper

1 Citaat (Scopus)
47 Downloads (Pure)

Samenvatting

In this work we propose a Reinforcement Learning approach for a real-world flexible job shop scheduling scenario, where a two-armed robot and a human operator share two workstations in order to assemble light switches. The approach returns a schedule of predefined assembly actions, taking into account given constraints specific to the scenario, such as which actions should precede others, which actions should occur together, or which actions should occur separately. In order to avoid collisions, the work area is divided into zones and all these constraints are taken into account by the algorithm in order to propose a schedule that optimizes the time it takes to prepare the customer orders. The approach that is taken is a reinforcement learning approach. There both arms and the operator are considered to be agents, and they learn from experience who should do which job best, taking into account the above mentioned constraints, as well as the estimated timings to execute the tasks. These estimated timings can be online updated, as well as their uncertainty. The experiments show that the proposed algorithm outperforms the currently used algorithm, which is a constraint solver, in terms of computation time and it is more user friendly with respect to expressing constraints. It also allows to take into account the uncertainty on the execution times of the tasks, and to add slack at critical timeslots, without unnecessarily increasing the makespan. Moreover, the algorithm allows to recompute an updated solution dynamically and efficiently when it turns out an action cannot be performed within the proposed time due to unexpected events. While we show the benefits in a scenario with a two-armed robot collaborating with a human for assembling light switches, the approach is relevant for many assembly tasks in the manufacturing industry where machines and human operators work together.
Originele taal-2English
Titel9th CIRP Conference on Assembly Technology and Systems
UitgeverijElsevier
Pagina's227-232
Aantal pagina's6
Volume106
DOI's
StatusPublished - mrt 2022

Publicatie series

NaamProcedia CIRP
UitgeverijElsevier
ISSN van geprinte versie2212-8271

Vingerafdruk

Duik in de onderzoeksthema's van 'A Q-Learning algorithm for flexible job shop scheduling in a real-world manufacturing scenario'. Samen vormen ze een unieke vingerafdruk.

Citeer dit