方乐缘,旷洋,刘强,等. 基于时差提示SAM的遥感变化检测[J]. 信号处理,2024,40(3): 417-427. DOI: 10.16798/j.issn.1003-0530.2024.03.001.
引用本文: 方乐缘,旷洋,刘强,等. 基于时差提示SAM的遥感变化检测[J]. 信号处理,2024,40(3): 417-427. DOI: 10.16798/j.issn.1003-0530.2024.03.001.
FANG Leyuan, KUANG Yang, LIU Qiang, et al. Temporal difference prompted SAM for remote sensing change detection[J]. Journal of Signal Processing, 2024,40(3):417-427. DOI: 10.16798/j.issn.1003-0530.2024.03.001.
Citation: FANG Leyuan, KUANG Yang, LIU Qiang, et al. Temporal difference prompted SAM for remote sensing change detection[J]. Journal of Signal Processing, 2024,40(3):417-427. DOI: 10.16798/j.issn.1003-0530.2024.03.001.

基于时差提示SAM的遥感变化检测

Temporal Difference Prompted SAM for Remote Sensing Change Detection

  • 摘要: 遥感变化检测作为观测和分析地表变化的重要手段,以双时相图像为输入,旨在预测变化发生的“位置”。近期出现的基础模型,例如SAM模型(Segment Anything Model),展现出强大的通用性和泛化能力,有望为变化检测任务提供更为有效的解决方案。然而,由于遥感图像的特殊成像特性,它们在许多遥感应用中的直接使用往往效果不佳。此外,SAM模型最初被设计用于分割单时相图像,其通过人工添加点或框的提示来实现,但这种过于直观的方法在处理双时相图像输入时并不适用。为了应对上述挑战,本文提出了一种基于时差提示SAM的遥感变化检测方法(TDPS),充分发挥 SAM 模型强大的视觉识别能力,以改进对遥感图像的变化检测。具体而言,本文首先在 SAM 骨干网络中引入了低阶可微调参数,以减轻自然图像到遥感图像上的域偏移。其次,本文设计了时相差异提示生成器,通过将双时相图像的特征与查询嵌入一起优化,得到适用于变化检测任务的提示向量。最后,大量实验证明了本文方法的有效性,在两个常用的变化检测数据集LEVIR-CD和WHU-CD上取得了最先进的性能,F1 指标相比于最先进方法分别提升了1.4%和2.5%。

     

    Abstract: ‍ ‍Remote sensing change detection, as a crucial means to observe and analyze surface alterations, utilizes bi-temporal images as input to predict the “locations” of changes. The recent emergence of foundation models, such as the segment anything model (SAM), has demonstrated powerful universality and generalization capabilities, offering more effective solutions for change detection tasks. However, their direct use in many remote sensing applications is often unsatisfactory due to the special imaging properties of remote sensing images. Additionally, SAM is initially designed for segmenting single-temporal images, relying on the manual addition of point or box prompts, which proves overly intuitive when handling bi-temporal image inputs. To address these challenges, this paper proposes temporal difference prompted SAM for remote sensing change detection (TDPS), exploiting the robust visual recognition capabilities of the SAM to improve change detection in remote sensing images. Specifically, we first introduce low-rank fine-tuning parameters into the SAM backbone network to mitigate domain shifts from natural images to remote sensing images. Second, a temporal difference prompt generator is designed to optimize features of bi-temporal images together with query embeddings, yielding prompt vectors suitable for change detection tasks. Finally, extensive experiments demonstrate the effectiveness of the proposed approach, achieving state-of-the-art performance on two commonly used change detection datasets, LEVIR-CD and WHU-CD, with F1 scores improved by 1.4% and 2.5%, respectively, compared to state-of-the-art methods.

     

/

返回文章
返回