Representation Learning with Information Theory to Detect COVID-19 and its Severity

Abel Diaz Berenguer; Tanmoy Mukherjee; Yifei Da; Matías Nicolás Bossa Bossa; Maryna Kvasnytsia; Jef Vandemeulebroucke; Nikos Deligiannis; Hichem Sahli

doi:10.1007/978-3-031-25082-8_41

Representation Learning with Information Theory to Detect COVID-19 and its Severity

Abel Diaz Berenguer, Tanmoy Mukherjee, Yifei Da, Matías Nicolás Bossa Bossa, Maryna Kvasnytsia, Jef Vandemeulebroucke, Nikos Deligiannis, Hichem Sahli

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Research › peer-review

1 Citation (Scopus)

Abstract

Successful data representation is a fundamental factor in machine learning based medical imaging analysis. Deep Learning (DL) has taken an essential role in robust representation learning. However, the inability of deep models to generalize to unseen data can quickly overfit intricate patterns. Thereby, the importance of implementing strategies to aid deep models in discovering useful priors from data to learn their intrinsic properties. Our model, which we call a dual role network (DRN), uses a dependency maximization approach based on Least Squared Mutual Information (LSMI). LSMI leverages dependency measures to ensure representation invariance and local smoothness. While prior works have used information theory dependency measures like mutual information,
these are known to be computationally expensive due to the density estimation step. In contrast, our proposed DRN with LSMI formulation does not require the density estimation step and can be used as an alternative to approximate mutual information. Experiments on the CT based COVID-19 Detection and COVID-19 Severity Detection Challenges of the 2nd COV19D competition [24] demonstrate the effectiveness of our method compared to the baseline method of such competition.

Original language	English
Title of host publication	Lecture Notes in Computer Science
Subtitle of host publication	Computer Vision – ECCV 2022 Workshops
Editors	Leonid Karlinsky, Tomer Michaeli, Ko Nishino
Publisher	Springer, Cham
Pages	605-620
Number of pages	16
Volume	13807
ISBN (Electronic)	978-3-031-25082-8
ISBN (Print)	978-3-031-25081-1
DOIs	https://doi.org/10.1007/978-3-031-25082-8_41
Publication status	Published - 23 Feb 2023

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	13807 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Bibliographical note

Funding Information:
Acknowledgement. We want to thank the organizers of the 2nd COV19D Competition occurring in the ECCV 2022 Workshop: AI-enabled Medical Image Analysis - Digital Pathology & Radiology/COVID19 for providing access to extensive and high-quality data to benchmark our model. This research has been partially financed by the European Union under the Horizon 2020 Research and Innovation programme under grant agreement 101016131 (ICOVID).

Publisher Copyright:
© 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Keywords

Representation learning
mutual information
COVID-19 detection

Access to Document

10.1007/978-3-031-25082-8_41

Cite this

Diaz Berenguer, A., Mukherjee, T., Da, Y., Bossa Bossa, M. N., Kvasnytsia, M., Vandemeulebroucke, J., Deligiannis, N., & Sahli, H. (2023). Representation Learning with Information Theory to Detect COVID-19 and its Severity. In L. Karlinsky, T. Michaeli, & K. Nishino (Eds.), Lecture Notes in Computer Science: Computer Vision – ECCV 2022 Workshops (Vol. 13807, pp. 605-620). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13807 LNCS). Springer, Cham. https://doi.org/10.1007/978-3-031-25082-8_41

Diaz Berenguer, Abel ; Mukherjee, Tanmoy ; Da, Yifei ; Bossa Bossa, Matías Nicolás ; Kvasnytsia, Maryna ; Vandemeulebroucke, Jef ; Deligiannis, Nikos ; Sahli, Hichem. / Representation Learning with Information Theory to Detect COVID-19 and its Severity. Lecture Notes in Computer Science: Computer Vision – ECCV 2022 Workshops. editor / Leonid Karlinsky ; Tomer Michaeli ; Ko Nishino. Vol. 13807 Springer, Cham, 2023. pp. 605-620 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inbook{32ee10bccde244d4ae5bca9ae9bbdfba,

title = "Representation Learning with Information Theory to Detect COVID-19 and its Severity",

abstract = "Successful data representation is a fundamental factor in machine learning based medical imaging analysis. Deep Learning (DL) has taken an essential role in robust representation learning. However, the inability of deep models to generalize to unseen data can quickly overfit intricate patterns. Thereby, the importance of implementing strategies to aid deep models in discovering useful priors from data to learn their intrinsic properties. Our model, which we call a dual role network (DRN), uses a dependency maximization approach based on Least Squared Mutual Information (LSMI). LSMI leverages dependency measures to ensure representation invariance and local smoothness. While prior works have used information theory dependency measures like mutual information,these are known to be computationally expensive due to the density estimation step. In contrast, our proposed DRN with LSMI formulation does not require the density estimation step and can be used as an alternative to approximate mutual information. Experiments on the CT based COVID-19 Detection and COVID-19 Severity Detection Challenges of the 2nd COV19D competition [24] demonstrate the effectiveness of our method compared to the baseline method of such competition.",

keywords = "Representation learning, mutual information, COVID-19 detection",

author = "{Diaz Berenguer}, Abel and Tanmoy Mukherjee and Yifei Da and {Bossa Bossa}, {Mat{\'i}as Nicol{\'a}s} and Maryna Kvasnytsia and Jef Vandemeulebroucke and Nikos Deligiannis and Hichem Sahli",

note = "Funding Information: Acknowledgement. We want to thank the organizers of the 2nd COV19D Competition occurring in the ECCV 2022 Workshop: AI-enabled Medical Image Analysis - Digital Pathology & Radiology/COVID19 for providing access to extensive and high-quality data to benchmark our model. This research has been partially financed by the European Union under the Horizon 2020 Research and Innovation programme under grant agreement 101016131 (ICOVID). Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.",

year = "2023",

month = feb,

day = "23",

doi = "10.1007/978-3-031-25082-8_41",

language = "English",

isbn = "978-3-031-25081-1",

volume = "13807",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer, Cham",

pages = "605--620",

editor = "Leonid Karlinsky and Tomer Michaeli and Ko Nishino",

booktitle = "Lecture Notes in Computer Science",

}

Diaz Berenguer, A , Mukherjee, T , Da, Y , Bossa Bossa, MN , Kvasnytsia, M , Vandemeulebroucke, J , Deligiannis, N & Sahli, H 2023, Representation Learning with Information Theory to Detect COVID-19 and its Severity. in L Karlinsky, T Michaeli & K Nishino (eds), Lecture Notes in Computer Science: Computer Vision – ECCV 2022 Workshops. vol. 13807, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 13807 LNCS, Springer, Cham, pp. 605-620. https://doi.org/10.1007/978-3-031-25082-8_41

Representation Learning with Information Theory to Detect COVID-19 and its Severity. / Diaz Berenguer, Abel ; Mukherjee, Tanmoy ; Da, Yifei ; Bossa Bossa, Matías Nicolás ; Kvasnytsia, Maryna ; Vandemeulebroucke, Jef ; Deligiannis, Nikos ; Sahli, Hichem.

Lecture Notes in Computer Science: Computer Vision – ECCV 2022 Workshops. ed. / Leonid Karlinsky; Tomer Michaeli; Ko Nishino. Vol. 13807 Springer, Cham, 2023. p. 605-620 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 13807 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Research › peer-review

TY - CHAP

T1 - Representation Learning with Information Theory to Detect COVID-19 and its Severity

AU - Diaz Berenguer, Abel

AU - Mukherjee, Tanmoy

AU - Da, Yifei

AU - Bossa Bossa, Matías Nicolás

AU - Kvasnytsia, Maryna

AU - Vandemeulebroucke, Jef

AU - Deligiannis, Nikos

AU - Sahli, Hichem

N1 - Funding Information: Acknowledgement. We want to thank the organizers of the 2nd COV19D Competition occurring in the ECCV 2022 Workshop: AI-enabled Medical Image Analysis - Digital Pathology & Radiology/COVID19 for providing access to extensive and high-quality data to benchmark our model. This research has been partially financed by the European Union under the Horizon 2020 Research and Innovation programme under grant agreement 101016131 (ICOVID). Publisher Copyright: © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

PY - 2023/2/23

Y1 - 2023/2/23

N2 - Successful data representation is a fundamental factor in machine learning based medical imaging analysis. Deep Learning (DL) has taken an essential role in robust representation learning. However, the inability of deep models to generalize to unseen data can quickly overfit intricate patterns. Thereby, the importance of implementing strategies to aid deep models in discovering useful priors from data to learn their intrinsic properties. Our model, which we call a dual role network (DRN), uses a dependency maximization approach based on Least Squared Mutual Information (LSMI). LSMI leverages dependency measures to ensure representation invariance and local smoothness. While prior works have used information theory dependency measures like mutual information,these are known to be computationally expensive due to the density estimation step. In contrast, our proposed DRN with LSMI formulation does not require the density estimation step and can be used as an alternative to approximate mutual information. Experiments on the CT based COVID-19 Detection and COVID-19 Severity Detection Challenges of the 2nd COV19D competition [24] demonstrate the effectiveness of our method compared to the baseline method of such competition.

AB - Successful data representation is a fundamental factor in machine learning based medical imaging analysis. Deep Learning (DL) has taken an essential role in robust representation learning. However, the inability of deep models to generalize to unseen data can quickly overfit intricate patterns. Thereby, the importance of implementing strategies to aid deep models in discovering useful priors from data to learn their intrinsic properties. Our model, which we call a dual role network (DRN), uses a dependency maximization approach based on Least Squared Mutual Information (LSMI). LSMI leverages dependency measures to ensure representation invariance and local smoothness. While prior works have used information theory dependency measures like mutual information,these are known to be computationally expensive due to the density estimation step. In contrast, our proposed DRN with LSMI formulation does not require the density estimation step and can be used as an alternative to approximate mutual information. Experiments on the CT based COVID-19 Detection and COVID-19 Severity Detection Challenges of the 2nd COV19D competition [24] demonstrate the effectiveness of our method compared to the baseline method of such competition.

KW - Representation learning

KW - mutual information

KW - COVID-19 detection

UR - http://www.scopus.com/inward/record.url?scp=85150997814&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-25082-8_41

DO - 10.1007/978-3-031-25082-8_41

M3 - Chapter

SN - 978-3-031-25081-1

VL - 13807

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 605

EP - 620

BT - Lecture Notes in Computer Science

A2 - Karlinsky, Leonid

A2 - Michaeli, Tomer

A2 - Nishino, Ko

PB - Springer, Cham

ER -

Diaz Berenguer A , Mukherjee T , Da Y , Bossa Bossa MN , Kvasnytsia M , Vandemeulebroucke J et al. Representation Learning with Information Theory to Detect COVID-19 and its Severity. In Karlinsky L, Michaeli T, Nishino K, editors, Lecture Notes in Computer Science: Computer Vision – ECCV 2022 Workshops. Vol. 13807. Springer, Cham. 2023. p. 605-620. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-031-25082-8_41

Representation Learning with Information Theory to Detect COVID-19 and its Severity

Abstract

Publication series

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

VLIR409: (COVID-19 PROMPT) Toward PrecisiOn Medicine for the Prediction of Treatment response to Covid-19 in Cuba

EUAR46: H2020: icovid: AI-based chest CT analysis enabling rapid COVID diagnosis and prognosis

Semi-supervised medical image classification via distance correlation minimization and graph attention regularization

Representation Learning with Information Theory for COVID-19 Detection

Explainable-by-design Semi-Supervised Representation Learning for COVID-19 Diagnosis from CT Imaging