Projects per year
Abstract
Compact convolutional neural networks (CNNs) have witnessed exceptional improvements in performance in recent years. However, they still fail to provide the same predictive power as CNNs with a large number of parameters. The diverse and even abundant features captured by the layers is an important characteristic of these successful CNNs. However, differences in this characteristic between large CNNs and their compact counterparts have rarely been investigated. In compact CNNs, due to the limited number of parameters, abundant features are unlikely to be obtained, and feature diversity becomes an essential characteristic. Diverse features present in the activation maps derived from a data point during model inference may indicate the presence of a set of unique descriptors necessary to distinguish between objects of different classes. In contrast, data points with low feature diversity may not provide a sufficient amount of unique descriptors to make a valid prediction; we refer to them as random predictions. Random predictions can negatively impact the optimization process and harm the final performance. This paper proposes addressing the problem raised by random predictions by reshaping the standard cross-entropy to make it biased toward data points with a limited number of unique descriptive features. Our novel Bias Loss focuses the training on a set of valuable data points and prevents the vast number of samples with poor learning features from misleading the optimization process. Furthermore, to show the importance of diversity, we present a family of SkipNet models whose architectures are brought to boost the number of unique descriptors in the last layers. Our Skipnet-M can achieve 1% higher classification accuracy than MobileNetV3 Large.
Original language | English |
---|---|
Title of host publication | Proceedings - 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021 |
Place of Publication | IEEE |
Publisher | IEEE |
Pages | 6536-6546 |
Number of pages | 11 |
Volume | 18 |
ISBN (Electronic) | 978-1-6654-2812-5 |
ISBN (Print) | 978-1-6654-2813-2 |
DOIs | |
Publication status | Published - 2021 |
Event | 18th IEEE/CVF International Conference on Computer Vision - virtual, online, Montreal, Canada Duration: 11 Oct 2021 → 17 Oct 2021 |
Publication series
Name | Proceedings of the IEEE International Conference on Computer Vision |
---|---|
ISSN (Print) | 1550-5499 |
Conference
Conference | 18th IEEE/CVF International Conference on Computer Vision |
---|---|
Abbreviated title | ICCV 2021 |
Country/Territory | Canada |
City | Montreal |
Period | 11/10/21 → 17/10/21 |
Fingerprint
Dive into the research topics of 'Bias Loss for Mobile Neural Networks'. Together they form a unique fingerprint.Projects
- 1 Finished
-
SRP11: Strategic Research Programme: Processing of large scale multi-dimensional, multi-spectral, multi-sensorial and distributed data (M³D²)
Schelkens, P., Deligiannis, N., Jansen, B., Kuijk, M., Munteanu, A., Sahli, H., Steenhaut, K., Stiens, J., Schelkens, P., Cornelis, J. P., Kuijk, M., Munteanu, A., Sahli, H., Stiens, J. & Vounckx, R.
1/11/12 → 31/12/23
Project: Fundamental