Towards an operational approach for superresolution enhancement of multi-angle remote sensing imagery

The dissertation focuses on superresolution (SR) enhancement of remote sensing imagery. By making full advantage of the complementary information provided by the subtle sub-pixel shifts between a set of low resolution (LR) images of the same scene, SR has proved to be an efficient way of increasing the spatial resolution of target remote sensing imagery since it was first applied to multi-temporal LANDSAT 4 remote sensing data back in 1984. New opportunities for SR have occurred as an increasing number of recently available sensors such as Multispectral Thermal Imager (MTI), IKONOS, Quickbird, WorldView-2, Multi-angle Imaging SpectroRadiometer (MISR), the Along Track Scanning Radiometers (ATSR-1, ATSR-2, AATSR), and the Compact High Resolution Imaging Spectrometer onboard the Project for On-board Autonomy (CHRIS/Proba) are equipped with multi-angle capabilities providing multiple LR images suitable for SR image enhancement. The main objective of this research is to develop practical and operationally applicable multi-temporal and multi-angle SR algorithms and procedures, while investigating the potential of SR to bridge the gap between high resolution (HR) airborne imagery and LR spaceborne imagery. In order to perform SR image reconstruction the geometric disparity between LR images must be described and modeled. Indeed, subpixel image registration is the key for any successful SR algorithm. Therefore, the first part of the research focuses on geometric registration of multi-angle remote sensing imagery. The reason why multi-angle rather than multi-temporal remote sensing image registration is addressed, is because geometric registration in this situation is more challenging, and the developed multi-angle registration technique can be easily extended to the multi-temporal case. The main difficulty of multi-angle geometric registration lies in the fact that 1) images captured at large view angles are susceptible to resolution change and blurring; and 2) local geometric distortion caused by topographic effects and/or platform instability may be important. Hence many rigid geometric models such as translational, rotational, affine or projective transform models, popular in the SR literature, cannot be used as such. We found that the thin plate spline (TPS) model can successfully account for the geometric disparity among LR images; however the estimation of the TPS model relies on a sufficient amount of control points (CPs). In order to identify enough CPs for model estimation, we propose a two-step non-rigid automatic registration scheme that integrates the merits of both area-based and feature-based methods. In order to assure the high quality of detected CPs, a rigorous screening procedure that combines robust statistics theory and a priori knowledge of the imaging system is adopted to eliminate outliers as well as CPs with large random errors. The proposed registration scheme has been tested using CHRIS/Proba data. The second part of the research focuses on non-uniform interpolation, which is among the most intuitive SR algorithms. When pixels of LR images are projected onto a HR grid via geometric registration, HR image pixels can be obtained by intelligent fusion of surrounding LR pixels. This simple non-uniform interpolation scheme has been used for a long time by the remote sensing community. The most promising non-uniform interpolation technique in the context of SR is kernel regression, which is based on the assumption that the local image patch is approximated by an N-term Taylor series. However, the performance of kernel regression based SR can greatly decrease when applied to multi-angle remote sensin