基于事件及考虑像素级模糊程度的图像去模糊
Event-Based Deblurring via Pixel-Wise Image Blurry Degree
-
摘要: 在图像去模糊任务中,现有的端到端深度学习方法通常使用共享的卷积核来处理图像的整体空间位置,即使用的卷积核在整个图像的所有位置上都是相同的,不会根据具体位置的不同而改变。这意味着这些方法在处理图像时,无论图像中某个区域的模糊程度如何,都使用相同的卷积核进行处理。然而,在某些复杂的模糊场景中,使用共享卷积核可能无法很好地处理图像的非均匀模糊情况。为此,本文提出了一种创新方法,利用像素级模糊程度来增强端到端图像去模糊的效果。具体来说,本文设计并训练了一个去模糊网络(Deblurring Network, DeblurNet),能够从输入图像和曝光时间内的事件数据中精确估计模糊程度图。随后,本文通过基于模糊程度的特征调制(Degree-based Feature Modulation, DFM)技术,依据模糊程度图自适应调节DeblurNet的特征。DeblurNet是一个端到端卷积神经网络,专门用于复原模糊图像的清晰度,通过动态卷积核来处理不同模糊程度的区域。这一策略实现了对非均匀模糊的空间可变卷积,从而有效地去除图像中的非均匀模糊。本文在合成数据集和真实事件数据集上进行了大量实验,并使用公开方法作为DeblurNet的基线。结果表明,提出的方法能够在合成和真实数据集上持续提升现有方法的性能,展示出较好的泛化能力。Abstract: For image deblurring, current end-to-end deep learning methods typically employ shared convolution kernels to process all spatial locations across the entire image. Thus, the convolution kernel used remains the same for all positions in the image and does not adapt based on specific locations. This implies that such methods apply the same convolution kernel to all areas, regardless of the varying levels of blurriness in different regions. However, in some complex blurred scenarios, the use of shared convolution kernels may fail to effectively handle non-uniform blurs across an image. To address this, this paper proposes an innovative approach that leverages pixel-level blur degrees to enhance the performance of end-to-end image deblurring. Specifically, a network named DegreeNet is designed and trained to accurately estimate the blur degree map from input images and event data captured during the exposure time. Subsequently, through Degree-based Feature Modulation (DFM), the blur degree map adaptively modulates the features in DeblurNet, an end-to-end convolutional neural network specifically designed to restore clarity in blurred images, with dynamic convolution kernels to address regions with varying blur levels. This strategy enables spatially adaptive convolution for non-uniform blur removal, effectively eliminating non-uniform blur in images. Extensive experiments on synthetic datasets and real-world event datasets were conducted using public methods as baselines for DeblurNet. The experimental results indicate that the proposed method consistently enhances the performance of these existing methods on both synthetic and real datasets, demonstrating strong generalization capabilities.