Beyond Mitchell: Multi-Objective Machine Learning -- minimal entropy, energy and error

In this paper the need for a truly multi-objective view on machine learning processes is put forward. Though methodologies exist that take into account multiple objectives like model complexity or generalization, they eventually serve to achieve higher prediction accuracies only --- albeit with good generalization properties. The fundamental conjecture of this paper, yet to be experimentally validated, is that the machine learning strategies that aim to optimize the structural and energetic properties of models, are the processes that lead
to hierarchy building and abstraction respectively, reflected ultimately in the internal representations. In this sense, this paper attempts to give impetus to create a holistic vision in which the impact of different algorithms on abstraction and hierarchy can be investigated, especially in the context of Deep Learning.