Spurious Valleys and Clustering Behavior of Neural Networks

Onderzoeksoutput: Conference paperResearch

Samenvatting

Neural networks constitute a class of functions that are typically non-surjective, with high-dimensional fibers and complicated image. We prove two main results concerning the geometry of the loss landscape of a neural network. First, we provide an explicit effective bound on the sizes of the hidden layers so that the loss landscape has no spurious valleys, which guarantees the success of gradient descent methods. Second, we present a novel method for analyzing whether a given neural network architecture with monomial activation function can represent a target function of interest. The core of our analysis method is the study of a specific set of error values, and its behavior depending on different training datasets.
Originele taal-2English
TitelProceedings of the 40th International Conference on Machine Learning
RedacteurenAndreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, Jonathan Scarlett
UitgeverijProceedings of Machine Learning Research (PMLR)
Pagina's28079-28099
Aantal pagina's21
Volume202
StatusPublished - jul 2023
EvenementFortieth International Conference on Machine Learning - Hawaiʻi Convention Center, 1801 Kalākaua Avenue , Honolulu, United States
Duur: 23 jul 202329 jul 2023
Congresnummer: 40
https://icml.cc/Conferences/2023

Publicatie series

NaamProceedings of Machine Learning Research
ISSN van elektronische versie2640-3498

Conference

ConferenceFortieth International Conference on Machine Learning
Verkorte titelICML 2023
Land/RegioUnited States
StadHonolulu
Periode23/07/2329/07/23
Internet adres

Bibliografische nota

Publisher Copyright:
© 2023 Proceedings of Machine Learning Research. All rights reserved.

Vingerafdruk

Duik in de onderzoeksthema's van 'Spurious Valleys and Clustering Behavior of Neural Networks'. Samen vormen ze een unieke vingerafdruk.

Citeer dit