光谱信息扩展的单目标跟踪技术研究进展
Recent Advances in Single Object Tracking Methods with Extended Spectral Information
-
摘要: 基于可见光图像的单目标跟踪方法经过多年发展,取得了卓越的成效。然而,在低照度、杂乱背景、相似目标干扰等场景时,这类方法的目标表征具有明显的限制。与此相比,热红外图像和多光谱图像提供了热辐射和光谱特性,为目标跟踪技术提供了光谱信息的扩展。基于可见光-红外RGB-T(RGB-Thermal)和多光谱图像的单目标跟踪技术已经取得了一定的进展,然而目前仍缺乏一个全面的综述文献。因此,本文对光谱信息扩展的单目标跟踪技术进行了全面的回顾与分析。首先,本文梳理并总结了光谱信息扩展的目标跟踪框架。在此基础上,详细讨论了基于RGB-T图像的目标跟踪改进方法,包括特征融合策略的设计、先验挑战属性的利用和高质量特征的生成。同时,从多光谱跟踪方法的传统模式、谱段选择策略的构建、多光谱特征的高效表达和适用于多种成像设备的模型设计四个方面,全面阐述了基于多光谱图像的目标跟踪方法的改进策略和方法优势。此外,本文汇总了目前RGB-T和多光谱图像的单目标跟踪数据集和评价指标,对比了不同跟踪算法在这些数据集上的性能表现。最后,探讨了光谱信息扩展的单目标跟踪技术的未来发展趋势,强调了构建大规模数据集、发展多光谱数据的预训练模型、充分表达光谱维度信息、理解红外及高光谱图像数据分布和大模型与目标跟踪技术结合作为未来研究的热点和难点,旨在为该领域的研究人员提供参考和启示。Abstract: Single object tracking technology is a fundamental research problem in the field of computer vision, with extensive applications across various domains. Single object tracking methods based on RGB images achieved remarkable success following years of meticulous development. However, these methods exhibit substantial limitations and challenges when deployed in low-light environments, cluttered backgrounds, and scenarios with interference from similar targets. By contrast, infrared thermal imaging and hyperspectral imaging enhance object tracking technologies by providing thermal radiation characteristics and spectral features, thereby extending the spectral information of target representation. Despite the great progress in single object tracking methods with extended spectral information based on RGB-T(RGB-Thermal) and hyperspectral images, a comprehensive review of the literature remains lacking. To bridge this gap, this paper aims to provide a thorough review of single object tracking technologies with extended spectral information. First, this paper presents a survey of the domestic and international development of single object tracking technologies enhanced by extended spectral features, reviews the frameworks based on RGB images, and summarizes those based on RGB-T and hyperspectral images. The frameworks are categorized into those based on traditional, Siamese-based, and Transformer-based tracking methods. With the advancements of computational capabilities and the development of deep learning, deep learning-based tracking methods are increasingly demonstrating exceptional performance. Further, this paper provides a detailed discussion on the methods for improving object tracking based on RGB-T images, including the design of feature fusion strategies, utilization of prior challenge attributes in video sequences, and the generation of high-quality feature representation. Moreover, this paper extensively describes the structure, improvement objectives, and methodological advantages of object tracking methods based on hyperspectral images, covering aspects such as traditional hyperspectral tracking methods, the construction of spectral band selection strategies, efficient expression of hyperspectral features, and the design of universal models suitable for various hyperspectral imaging devices. Furthermore, the current datasets and evaluation metrics are compiled based on RGB-T and hyperspectral imaging, the performance of various tracking algorithms is compared for these datasets, and the advantages and disadvantages of the different methods are summarized. Among these, novel RGB-T tracking methods based on generative models achieve impressive tracking results. Finally, this study explores the future trends of single object tracking methods with extended spectral information. The conclusions are summarized as follows. Single object tracking methods based on RGB-T and hyperspectral images attract significant interest among researchers and are poised to be crucial in the future of object tracking technologies. The available RGB-T and hyperspectral tracking datasets are relatively limited in volume and diversity, which restricts the training and evaluation of algorithms; hence, constructing large-scale datasets is fundamental for advancing tracking methodologies. Additionally, thoroughly extracting the spectral characteristics of objects, comprehensively understanding the distribution of infrared and hyperspectral image data, and effectively integrating large models with single object tracking are formidable challenges and prominent topics for future research. This paper provides references and insights for future studies.