Learned Multimodal Convolutional Sparse Coding for Guided Image Super-Resolution

Iman Marivani; Evangelia Tsiligianni; Bruno Cornelis; Nikolaos Deligiannis

doi:10.1109/ICIP.2019.8803313

Learned Multimodal Convolutional Sparse Coding for Guided Image Super-Resolution

Iman Marivani, Evangelia Tsiligianni, Bruno Cornelis, Nikolaos Deligiannis

Electronics and Informatics

Research output: Chapter in Book/Report/Conference proceeding › Conference paper

10 Citations (Scopus)

Abstract

The success of deep learning in various tasks, including solving inverse problems, has triggered the need for designing deep neural networks that incorporate domain knowledge. In this paper, we design a multimodal deep learning architecture for guided image super-resolution, which refers to the problem of super-resolving a low-resolution image with the aid of a high-resolution image of another modality. The proposed architecture is based on a novel deep learning model, obtained by unfolding a proximal method that solves the problem of convolutional sparse coding with side information. We applied the proposed architecture to super-resolve near-infrared images using RGB images as side information. Experimental results report average PSNR gains of up to 2.85 dB against state-of-the-art multimodal deep learning and sparse coding models.

Original language	English
Title of host publication	2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings
Publisher	IEEE
Pages	2891-2895
Number of pages	5
ISBN (Electronic)	9781538662496
DOIs	https://doi.org/10.1109/ICIP.2019.8803313
Publication status	Published - Sep 2019
Event	IEEE International Conference on Image Processing 2019 - Taiwan, Taipei, Taiwan, Province of China Duration: 22 Sep 2019 → 25 Sep 2019

Publication series

Name	Proceedings - International Conference on Image Processing, ICIP
Volume	2019-September
ISSN (Print)	1522-4880

Conference

Conference	IEEE International Conference on Image Processing 2019
Abbreviated title	ICIP
Country/Territory	Taiwan, Province of China
City	Taipei
Period	22/09/19 → 25/09/19

Keywords

Guided image super-resolution
convolutional sparse coding
multimodal deep neural networks

Access to Document

10.1109/ICIP.2019.8803313

Cite this

Marivani, I., Tsiligianni, E., Cornelis, B., & Deligiannis, N. (2019). Learned Multimodal Convolutional Sparse Coding for Guided Image Super-Resolution. In 2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings (pp. 2891-2895). [8803313] (Proceedings - International Conference on Image Processing, ICIP; Vol. 2019-September). IEEE. https://doi.org/10.1109/ICIP.2019.8803313

@inproceedings{683dee8f240e4765ba5b8101b9dece60,

title = "Learned Multimodal Convolutional Sparse Coding for Guided Image Super-Resolution",

abstract = "The success of deep learning in various tasks, including solving inverse problems, has triggered the need for designing deep neural networks that incorporate domain knowledge. In this paper, we design a multimodal deep learning architecture for guided image super-resolution, which refers to the problem of super-resolving a low-resolution image with the aid of a high-resolution image of another modality. The proposed architecture is based on a novel deep learning model, obtained by unfolding a proximal method that solves the problem of convolutional sparse coding with side information. We applied the proposed architecture to super-resolve near-infrared images using RGB images as side information. Experimental results report average PSNR gains of up to 2.85 dB against state-of-the-art multimodal deep learning and sparse coding models.",

keywords = "Guided image super-resolution, convolutional sparse coding, multimodal deep neural networks",

author = "Iman Marivani and Evangelia Tsiligianni and Bruno Cornelis and Nikolaos Deligiannis",

year = "2019",

month = sep,

doi = "10.1109/ICIP.2019.8803313",

language = "English",

series = "Proceedings - International Conference on Image Processing, ICIP",

publisher = "IEEE",

pages = "2891--2895",

booktitle = "2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings",

note = "IEEE International Conference on Image Processing 2019, ICIP ; Conference date: 22-09-2019 Through 25-09-2019",

}

Marivani, I, Tsiligianni, E , Cornelis, B & Deligiannis, N 2019, Learned Multimodal Convolutional Sparse Coding for Guided Image Super-Resolution. in 2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings., 8803313, Proceedings - International Conference on Image Processing, ICIP, vol. 2019-September, IEEE, pp. 2891-2895, IEEE International Conference on Image Processing 2019, Taipei, Taiwan, Province of China, 22/09/19. https://doi.org/10.1109/ICIP.2019.8803313

Learned Multimodal Convolutional Sparse Coding for Guided Image Super-Resolution. / Marivani, Iman; Tsiligianni, Evangelia ; Cornelis, Bruno ; Deligiannis, Nikolaos.

2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings. IEEE, 2019. p. 2891-2895 8803313 (Proceedings - International Conference on Image Processing, ICIP; Vol. 2019-September).

Research output: Chapter in Book/Report/Conference proceeding › Conference paper

TY - GEN

T1 - Learned Multimodal Convolutional Sparse Coding for Guided Image Super-Resolution

AU - Marivani, Iman

AU - Tsiligianni, Evangelia

AU - Cornelis, Bruno

AU - Deligiannis, Nikolaos

PY - 2019/9

Y1 - 2019/9

N2 - The success of deep learning in various tasks, including solving inverse problems, has triggered the need for designing deep neural networks that incorporate domain knowledge. In this paper, we design a multimodal deep learning architecture for guided image super-resolution, which refers to the problem of super-resolving a low-resolution image with the aid of a high-resolution image of another modality. The proposed architecture is based on a novel deep learning model, obtained by unfolding a proximal method that solves the problem of convolutional sparse coding with side information. We applied the proposed architecture to super-resolve near-infrared images using RGB images as side information. Experimental results report average PSNR gains of up to 2.85 dB against state-of-the-art multimodal deep learning and sparse coding models.

AB - The success of deep learning in various tasks, including solving inverse problems, has triggered the need for designing deep neural networks that incorporate domain knowledge. In this paper, we design a multimodal deep learning architecture for guided image super-resolution, which refers to the problem of super-resolving a low-resolution image with the aid of a high-resolution image of another modality. The proposed architecture is based on a novel deep learning model, obtained by unfolding a proximal method that solves the problem of convolutional sparse coding with side information. We applied the proposed architecture to super-resolve near-infrared images using RGB images as side information. Experimental results report average PSNR gains of up to 2.85 dB against state-of-the-art multimodal deep learning and sparse coding models.

KW - Guided image super-resolution

KW - convolutional sparse coding

KW - multimodal deep neural networks

UR - http://www.scopus.com/inward/record.url?scp=85076798089&partnerID=8YFLogxK

U2 - 10.1109/ICIP.2019.8803313

DO - 10.1109/ICIP.2019.8803313

M3 - Conference paper

T3 - Proceedings - International Conference on Image Processing, ICIP

SP - 2891

EP - 2895

BT - 2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings

PB - IEEE

T2 - IEEE International Conference on Image Processing 2019

Y2 - 22 September 2019 through 25 September 2019

ER -

Learned Multimodal Convolutional Sparse Coding for Guided Image Super-Resolution

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

SRP11: Strategic Research Programme: Processing of large scale multi-dimensional, multi-spectral, multi-sensorial and distributed data (M³D²)

Cite this

Learned Multimodal Convolutional Sparse Coding for Guided Image Super-Resolution

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Projects

SRP11: Strategic Research Programme: Processing of large scale multi-dimensional, multi-spectral, multi-sensorial and distributed data (M³D²)

Cite this