There is an increased interest in solving complex constrained problems where part of the input is not given as facts, but received as raw sensor data such as images or speech. We will use ‘visual sudoku’ as a prototype problem, where the given cell digits are handwritten and provided as an image thereof. In this case, one first has to train and use a classifier to label the images, so that the labels can be used for solving the problem. In this paper, we explore the hybridisation of classifying the images with the reasoning of a constraint solver. We show that pure constraint reasoning on predictions does not give satisfactory results. Instead, we explore the possibilities of a tighter integration, by exposing the probabilistic estimates of the classifier to the constraint solver. This allows joint inference on these probabilistic estimates, where we use the
solver to find the maximum likelihood solution. We explore the trade-off between the power of the classifier and the power of the constraint reasoning, as well as further integration through the additional use of structural knowledge. Furthermore, we investigate the effect of calibration of the probabilistic estimates on the reasoning. Our results show that such hybrid approaches vastly outperform a separate approach, which encourages a further integration of prediction (probabilities) and constraint solving.
Originele taal-2English
TitelHybrid Classification and Reasoning for Image-based Constraint Solving
RedacteurenEmmanuel Hebrard, Nysret Musliu
Plaats van productieCham
Aantal pagina's16
Volume 12296
ISBN van elektronische versie978-3-030-58942-4
ISBN van geprinte versie978-3-030-58941-7
StatusPublished - 19 sep 2020
Evenement17th International Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research - Online
Duur: 21 sep 202024 sep 2020

Publicatie series

NaamLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12296 LNCS
ISSN van geprinte versie0302-9743
ISSN van elektronische versie1611-3349


Conference17th International Conference on the Integration of Constraint Programming, Artificial Intelligence, and Operations Research
Internet adres

Bibliografische nota

Funding Information:
Acknowledgements. This research received funding from the Flemish Government under the “Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen” programme.

Publisher Copyright:
© 2020, Springer Nature Switzerland AG.

Copyright 2022 Elsevier B.V., All rights reserved.


Duik in de onderzoeksthema's van 'Hybrid Classification and Reasoning for Image-based Constraint Solving'. Samen vormen ze een unieke vingerafdruk.

Citeer dit