Questioning the question: Exploring how physical degrees of freedom are retrieved with neural networks

When studying a physical system, it is crucial to identify the degrees of freedom that characterize that system. Recently, specific neural networks have been designed to retrieve these underlying degrees of freedom automatically. Indeed, fed with data from a physical system, a variational autoencoder can learn a latent representation of that system that directly corresponds to its underlying degrees of freedom. However, the understanding of these neural networks is limited on two fronts. First, very little is known about the impact of the question vector, a key parameter in designing performant autoencoders. Second, there is the mystery of why the correct degrees of freedom are found in the latent representation, not an arbitrary function of these parameters. Both gaps in our understanding are addressed in this paper. To study the first question on the optimal design of the question vector, we investigate physical systems characterized by analytical expressions with a limited set of degrees of freedom. We empirically show how the type of question influences the learned latent representation. We find that the stochasticity of a random question is fundamental in learning physically meaningful representations. Furthermore, the dimensionality of the question vector should not be too large. To address the second question, we make use of a symmetry argument. We show that the learning of the degrees of freedom in the latent space is related to the symmetry group of the input data. This result holds for linear and nonlinear transformations of the degrees of freedom. In this way, in this paper, we contribute to the research on automated systems for discovery and knowledge creation.