视差引导的可变形卷积光场图像超分辨重建方法

Disparity-Guided Deformable Convolution for Light Field Image Super-Resolution

  • 摘要: 光场图像超分辨重建旨在通过利用光场图像的视角域互补信息提升光场图像分辨率,恢复图像细节并改善图像质量。当前光场图像获取设备主要为微透镜相机(例如Lytro相机、RayTrix相机)和阵列相机。其中,微透镜相机记录的不同视角图像的视差最大值通常小于1像素,阵列相机记录的不同视角图像的视差最大值通常大于1像素。现有光场图像超分辨重建方法大多基于微透镜相机设计,在应用于阵列相机记录的大视差光场图像时会因为视角间互补信息利用不充分导致明显的性能下降。受启发于光场视差估计与可变形卷积网络的启发,本文设计了视差引导的可变形卷积光场图像超分辨重建方法,用于获取大视差光场图像视角域互补信息。所提方法首先对光场各子视角图像进行视差估计,基于视差图生成可变形卷积偏移量,随后实现跨视角特征对齐与互补信息结合,最后通过多级蒸馏机制完成特征融合与超分辨重建。本文在领域通用的5个公开数据集上对算法的有效性进行了验证。实验结果表明,所提方法可实现领先的超分辨重建性能,并且在针对大视差具备较好的鲁棒性。

     

    Abstract: ‍ ‍Light field image super-resolution reconstruction aims to enhance the resolution of light field images by utilizing complementary information across different views. This process focuses on restoring image details and improving overall image quality. The primary devices for capturing light field images are microlens cameras (such as Lytro and RayTrix cameras) and camera arrays. In microlens cameras, the maximum disparity between the images recorded from different viewpoints is typically less than one pixel. In contrast, for camera arrays, this maximum disparity can exceed one pixel. Most existing super-resolution reconstruction methods for light field images are tailored for microlens cameras; however, when applied to large disparity images (e.g., those captured by array cameras), these methods often experience significant performance degradation due to inadequate utilization of complementary information among different views. Inspired by light field disparity estimation techniques and deformable convolutional networks, this paper presents a disparity-guided deformable convolution method for light field image super-resolution. This method captures complementary information in the view domain of light field images with large disparities. The proposed approach begins by estimating the disparity of each sub-view image within the light field. It then generates deformable convolution offsets based on the disparity map, facilitating inter-view feature alignment and the combination of complementary information. Ultimately, the method achieves feature fusion and super-resolution reconstruction through a multi-level distillation mechanism. The effectiveness of the algorithm is validated on five widely used public light field datasets. Experimental results demonstrate that the proposed method achieves state-of-the-art performance in super-resolution reconstruction and is robust against large disparities.

     

/

返回文章
返回