物理可见性引导的高斯泼溅稀疏视角合成方法
VisGS: Physical-Visibility Aware Sparse-View Synthesis Using Gaussian Splatting
-
摘要: 新视角合成是三维(Three-Dimensional,3D)重建领域一项重要的任务,旨在从一组给定的输入图像中,生成任意新视角位置的物体或场景。最近,新的场景表示方法3D高斯泼溅(3D Gaussian Splatting, 3DGS)在包括新视角合成在内的许多3D视觉任务中取得显著的成效,但其性能在输入稀疏视角时会迅速下降,这主要源于模型在训练过程中产生的过拟合问题导致模型对新视角的泛化能力下降,其根源是模型复杂度与数据量不匹配。已有方法采用随机剪枝以降低复杂度,但缺乏可靠的几何引导,导致模型效果不稳定。另一些方法引入外部深度先验作为监督,但往往伴随噪声与细节损失。针对上述问题,本文提出一种基于物理可见性的高斯泼溅稀疏视角合成框架(Physical-visibility aware Gaussian splatting for sparse-view synthesis, VisGS)。该方法设计了一个关键的物理可见性剪枝模块以解决现有剪枝策略不稳定的问题,根据场景中每个高斯点的物理可见性分数,显式且稳定地决定该点保留与否,从而实现对冗余或错误高斯的高效精准剪枝。其次,本文设计了一个物理可见性感知的扰动机制,通过对高不确定性区域添加扰动,迫使模型更好地关注到这些关键区域,从而增强模型在复杂可见性情况下的鲁棒性。在LLFF和DTU等基准数据集上的实验结果表明,所提出的方法在少样本训练条件下,可以保证剪枝区域的正确性和稳定性,显著提高了新视角合成的质量。Abstract: In the field of three-dimensional (3D) reconstruction, novel view synthesis plays a fundamental role that aims to generate images of a scene from unseen viewpoints given a set of input images. Recently, 3D Gaussian Splatting (3DGS) has achieved remarkable success in this task, exhibiting superior performance in different 3D applications. However, under sparse-view settings, 3DGS faces significant challenges, as its performance deteriorates due to overfitting during model training, thereby limiting its ability to generalize to unseen viewpoints. Prior studies suggest that this overfitting arises from a mismatch between model complexity and the available volume of input data. Existing approaches address this issue by reducing model complexity through random pruning, while others incorporate external depth priors as supervisory signals. However, these methods often lack principled guidance during the pruning process, leading to high variability and inconsistent improvements in synthesis performance. To overcome this limitation, we propose VisGS, a sparse-view novel view synthesis framework built upon Gaussian splatting that explicitly integrates physical visibility constraints. Specifically, we develop a physical-visibility-based pruning module that overcomes the instability of existing pruning strategies. This module deterministically evaluates whether each Gaussian primitive should be retained based on its physical visibility within the scene, thereby enabling precise and efficient elimination of redundant or erroneous primitives. Moreover, we propose a visibility-aware perturbation mechanism that injects structured perturbations into regions of high uncertainty, encouraging the model to better capture ambiguous structures and enhancing robustness under complex visibility conditions. Experimental results on benchmark datasets, including LLFF and DTU, demonstrated that, under few-shot settings, the proposed method improves the accuracy and stability of pruned regions, mitigates the uncertainty inherent in random pruning, and substantially improves the quality of novel view synthesis.
下载: