Spurious Valleys and Clustering Behavior of Neural Networks

Research output: Chapter in Book/Report/Conference proceedingConference paper


Neural networks constitute a class of functions that are typically non-surjective, with high-dimensional fibers and complicated image. We prove two main results concerning the geometry of the loss landscape of a neural network. First, we provide an explicit effective bound on the sizes of the hidden layers so that the loss landscape has no spurious valleys, which guarantees the success of gradient descent methods. Second, we present a novel method for analyzing whether a given neural network architecture with monomial activation function can represent a target function of interest. The core of our analysis method is the study of a specific set of error values, and its behavior depending on different training datasets.
Original languageEnglish
Title of host publicationProceedings of the 40th International Conference on Machine Learning
EditorsAndreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, Jonathan Scarlett
PublisherProceedings of Machine Learning Research (PMLR)
Number of pages21
Publication statusPublished - Jul 2023
EventFortieth International Conference on Machine Learning - Hawaiʻi Convention Center, 1801 Kalākaua Avenue , Honolulu, United States
Duration: 23 Jul 202329 Jul 2023
Conference number: 40

Publication series

NameProceedings of Machine Learning Research
ISSN (Print)2640-3498


ConferenceFortieth International Conference on Machine Learning
Abbreviated titleICML 2023
Country/TerritoryUnited States
Internet address


Dive into the research topics of 'Spurious Valleys and Clustering Behavior of Neural Networks'. Together they form a unique fingerprint.

Cite this