基于上下文建模的学习式孪生网络立体视频压缩方法
Contextual Siamese Learning Framework for Efficient Stereo Video Compression
-
摘要: 随着自动驾驶和虚拟现实等智能应用的快速发展,立体视频作为一种同时包含外观信息与深度线索的多维信号形式,被广泛应用于环境感知与沉浸式交互等场景。然而,立体视频在时间维和视点维上均存在大量冗余,其数据规模随分辨率和帧率的提升迅速增长,对存储与传输系统提出了更高要求。因此,如何实现立体视频的高效压缩,已成为视频编码与多媒体信号处理领域中的重要研究问题。与单视点视频相比,立体视频不仅具有复杂的时序相关性,还引入了显著的视点间冗余,使得冗余建模与高效压缩更具挑战性。现有立体视频压缩方法多基于传统多视点编码框架或端到端学习式编码器,通过显式残差编码与视差预测来减少冗余,但在概率建模层面仍难以充分挖掘多源上下文信息的统计相关性,其编码结构通常依赖多阶段残差与预测模块的级联,且对左右视点之间的条件依赖关系缺乏显式的结构化表达,从而限制了模型表达能力。针对上述问题,本文提出一种基于上下文建模的学习式孪生网络立体视频压缩方法,构建结构清晰的双视点编码框架,在特征空间中联合建模时间上下文与视点上下文信息,并通过条件编码网络对待编码特征的概率分布进行自适应估计,在无需显式残差计算的情况下有效减少时序与视点冗余。实验结果表明,在Cityscapes和KITTI 2015立体视频数据集上,所提出的方法在率失真性能上优于传统立体视频编解码器及代表性的学习式方法,显著提升了端到端立体视频的压缩效率。Abstract: With the rapid advancement of intelligent applications such as autonomous driving and virtual reality, stereo video has been widely used for environmental perception and immersive interaction because it simultaneously captures appearance information and depth cues. However, stereo video contains substantial redundancy in both the temporal and inter-view dimensions, and its data volume increases significantly with higher resolutions and frame rates, placing considerable demands on storage and transmission systems. Consequently, efficient stereo video compression has become an important research topic in video coding and multimedia signal processing. Compared with monocular video, stereo video involves not only more complex temporal dependencies but also significant inter-view redundancy, making redundancy modeling and efficient compression more challenging. Existing stereo video compression methods, including traditional multiview coding frameworks and end-to-end learned codecs, typically reduce redundancy through explicit residual coding and disparity prediction. However, at the probabilistic modeling level, these approaches often fail to fully exploit the statistical dependencies among multiple sources of contextual information. Their coding architectures generally rely on cascaded, multi-stage prediction and residual modules and lack an explicit and structured representation of the conditional dependencies between the left and right views, which limits model expressiveness. To address these limitations, this paper proposes a contextual Siamese learning framework for stereo video compression. The proposed dual-view coding architecture jointly models temporal and inter-view context in the feature space, while a conditional coding network adaptively estimates the probability distribution of the features to be encoded. This design effectively reduces temporal and inter-view redundancy without explicit residual computation. Experimental results on the Cityscapes and KITTI 2015 stereo video datasets demonstrate that the proposed method outperforms conventional stereo video codecs and representative learning-based approaches in rate-distortion performance, significantly improving the compression efficiency of end-to-end stereo video coding.
下载: