On the use of Neural Networks and Support Vector Machines for the identification of nonlinear state space models

This poster discusses the application of regression methods from the machine learning community to the identification of nonlinear state space models. In this approach, linear modeling techniques are used to capture system dynamics, and the remaining nonlinear terms are identified separately. This combines the best of two worlds. On one hand the linear identification theory is very well suited to model dynamic systems, while on the other hand the methods from the machine learning community are very powerful tools to model multidimensional static nonlinear functions. By combining these two methodologies, an approximated static version of the problem is solved, so that one does not have to deal with the recursion in the state equation. A two step initialization procedure is proposed. First the dynamics of the system are modeled with the Best Linear Approximation, resulting in a linear state space model, then a simplified problem is obtained by estimating the nonlinear states. This can be done by solving a Least Squares problem or, alternatively, by means of Kalman filtering. Several possibilities can then be considered to estimate the nonlinearities in the model. This work focuses on the use of Neural Networks and Support Vector Machines as regression tools, and analyses advantages and drawbacks of the two methods. Finally, all parameters of the initialized nonlinear model are optimized, e.g. by means of a Levenberg-Marquardt algorithm.