Structure and motion analysis: variational Methods and Related PDE models

Structure and motion analysis plays a very important role in image processing and computer vision and has been applied successfully in many areas, such as computer interface, robotics, medical industry, etc. Two main aspects are addressed in this work: 3D structure from motion analysis and 2D dense motion estimation.Variational techniques and PDEs, as well as optimization approaches are investigated. An integration framework of three different visual modules is proposed for 3D structure and motion estimation, namely, (i) line drawing interpretations, (ii) 2D motion constraint described by the normal flow of line segment, and (iii) 2D feature correspondences. Their integration is made via an overall objective function to be optimized. A general incremental rigidity scheme is developed for implementation. The performance on image sequence of polyhedral scene shows its feasibility and advantages. Alternatively, a variational approach to efficiently estimate 3D line orientation from motion is proposed. This approach simplifies the structure from motion estimation dramatically. The associated vector-valued reaction diffusion model corresponds to the separate scalar valued diffusion processes, one for each component of the 3D line orientation, which are coupled to each other through the reaction term. The L-curve technique is proposed here to select a proper regularization parameter and hence produces better estimation results. For accurate 2D dense motion estimation, i.e., optical flow, the differential techniques are investigated with the focus on different types of PDE based diffusion models. Following the analysis of nonlinear isotropic diffusion model for image filtering by Catt\'{e} et al., we propose the fundamentaltheoretical analysis of the nonlinear isotropic diffusion model for optical flow estimation, consisting of a system of two coupled reaction diffusion equations. We prove, for a non increasing, non negative, differentiable and bounded diffusion function, the existence, stability and uniqueness of its solution based on the Schauder's fixed point theorem. In order to preserve discontinuities in optical flow field at both magnitude and orientation, a hybrid diffusion model is proposed, driven by both the flow and image through the nonlinear isotropic diffusion term and the linear anisotropic diffusion term, respectively. Following similar analysis as for the nonlinear isotropic diffusion, we use the Schauder's fixed point theorem to prove that there exists a solution to the hybrid diffusion model, it is stable and unique. Two numerical schemes are proposed to implement the hybrid diffusion model, namely the explicit and the semi-implicit schemes. Their properties, such as the efficiency and accuracy, are verified from experiments on both synthetic and real image sequences. On the other hand, as efficient solvers, multigrid methods are used in our work for efficient optical flow estimation. Linear multigrid methods are proposed to improve the numerical solution of nonlinear isotropic diffusion models through the multi-resolution representation. Applying the multigrid strategydirectly to the nonlinear isotropic diffusion model, we propose a nonlinear multigrid diffusion model. Two schemes are developed: the standard nonlinear multigrid method and the full multigrid and full approximation storage (FMG-FAS) method. Their efficiency, fast convergence, and accuracy compared with classical methods are validated in the experimental results. Furthermore, the nonlinear multigrid framework can be extended to other diffusion models, such as the proposed hybrid diffusion model.