Coding light field images

Student thesis: Master's Thesis

Abstract

In the last few years, plenoptic image modalities have been emerging at a high speed. These novel modalities, such as light field, point cloud and holographic imaging, open the door to new applications and functionalities. It is widely assumed that they have the potential to revolutionize the way that digital photographic content is produced and consumed. However, with their rich information content also comes a huge amount of data that needs to be stored and transmitted. The need for efficient compression is now timely and more important than ever. This thesis aims at the coding of a large acquisition of high resolution views from a wide-aperture light field. More specifically, the views are captured along a regular, linear, horizontal trajectory with a small baseline. The thesis targets lossy compression solutions with the highest quality for a given bitrate, in addition to random access and backward compatibility with existing coding standards. The light field images are studied in the epipolar- plane image (EPI) domain, which reveals the regular structure that is contained in the light field acquisition. Two methods are proposed that exploit this redundancy in order to compress the data. First, EPI analysis is used to extract the disparity of the EPI lines. In this way, we are able to condense the collection of EPI lines to a small set of line descriptors. The results show satisfying compression and reconstruction. However, this method lacks desirable capabilities such as scalability and backward compatibility. The second solution looks at JPEG 2000 Part 10, or JP3D, to encode the 3D light fields. Experiments are done to investigate how manipulation of the data can improve compression. We observe that rectification and prediction are effective ways to optimize the volumes for coding. Especially the combination of these two steps, creating a rectified difference volume, yields significant improvement in terms of quality versus bitrate. A second series of experiments comparing the performance of JP3D with HEVC, shows that JP3D consistently outperforms the video codec for the rectified difference volume. Consequently, the solution that we propose for the 3D light field coding problem, consists of encoding a reference view and the associated disparity map with JPEG or JPEG 2000, in addition to a residual volume encoded with JP3D. This volume is the rectified version of the difference volume from a depth-based view prediction regularized by wavelet interpolation. This coding solution meets backward compatibility and other requirements and can be used, starting from today, to efficiently code 3D light fields, until a more optimal or all-encompassing coding standard is made available.
Date of Award2017
Original languageEnglish
Awarding Institution
  • Vrije Universiteit Brussel

Keywords

  • 3D light fields
  • compression
  • epipolar-plane image
  • JP3D
  • view prediction

Cite this

'