Light Field Super-Resolution Based on Viewpoint Image and EPI Feature Fusion

‍ ‍Light field （LF） has a wide range of applications since it can capture spatial and angular information of light rays simultaneously. However， the resolution is limited by the hardware of the imaging device and the trade-off between spatial and angular resolution. The low spatial resolution greatly affects the quality and applications of LF images. In this paper， by making full use of the characteristics of LF to enhance the image details， we proposed an end-to-end LF super-resolution method based on the feature fusion of viewpoint image （VI） and epipolar plane image （EPI）， which was capable of super-resolving all VIs at the same time. Specifically， in this method， the low-resolution LF images were stacked and arranged in the horizontal and vertical EPI directions. With the property that EPI information was contained in 3D VI stacks， a 3D decreasing convolutional network with dual-branch structure was used to process the input 4D LF data， which was able to extract and fuse the features of the VI and EPI information simultaneously and fully explore the texture information and geometric consistency of the LF. Experimental results on both real-world and synthetic LF datasets showed that the proposed method not only has better visual performance， but also maintains better geometric consistency in subjective quality than other state-of-the-art methods. It also had fewer model parameters and faster execution speed.