Samenvatting
There is a substantial demand for deep learning methods that can work with limited, high-dimensional, and noisy datasets. Nonetheless, current research mostly neglects this area, especially in the absence of prior expert knowledge or knowledge transfer. In this work, we bridge this gap by studying the performance of deep learning methods on the true data distribution in a limited, high-dimensional, and noisy data setting. To this end, we conduct a systematic evaluation that reduces the available training data while retaining the challenging properties mentioned above.
Furthermore, we extensively search the space of hyperparameters and compare state-of-the-art architectures and models build and trained from scratch to advocate for the use of multi-objective tuning strategies. Our experiments highlight the lack of performative deep learning models in current literature and investigate the impact of training hyperparameters. We analyze the complexity of the models and demonstrate the advantage of choosing models tuned under multi-objective criteria in lower data regimes to reduce the likelihood to overfit. Lastly, we demonstrate the importance of selecting a proper inductive bias given a limited-sized dataset. Given our results, we conclude that tuning models using a multi-objective criterion results in simpler yet competitive models when reducing the number of data points.
Furthermore, we extensively search the space of hyperparameters and compare state-of-the-art architectures and models build and trained from scratch to advocate for the use of multi-objective tuning strategies. Our experiments highlight the lack of performative deep learning models in current literature and investigate the impact of training hyperparameters. We analyze the complexity of the models and demonstrate the advantage of choosing models tuned under multi-objective criteria in lower data regimes to reduce the likelihood to overfit. Lastly, we demonstrate the importance of selecting a proper inductive bias given a limited-sized dataset. Given our results, we conclude that tuning models using a multi-objective criterion results in simpler yet competitive models when reducing the number of data points.
Originele taal-2 | English |
---|---|
Status | Published - 18 okt 2024 |
Evenement | Multi-Objective Decision Making Workshop at ECAI 2024 - Schools of Philology and Communication Sciences, Santiago de Compostela, Spain Duur: 20 okt 2024 → 20 okt 2024 https://modem2024.vub.ac.be |
Workshop
Workshop | Multi-Objective Decision Making Workshop at ECAI 2024 |
---|---|
Verkorte titel | MODeM 2024 |
Land/Regio | Spain |
Stad | Santiago de Compostela |
Periode | 20/10/24 → 20/10/24 |
Internet adres |