Non-Contact Heart Rate Detection Algorithm Based on Transformer and CNN Feature Fusion

‍ ‍Arrhythmia is a common risk factor for cardiovascular diseases， and how to give early warning of arrhythmia has become a research hotspot. The traditional heart rate detection is contact type， which is very inconvenient. Remote photoplethysmography （rPPG） is proposed to achieve non-contact measurement， which has great application value in many scenarios. However， the environment and subject motion easily affect non-contact measurement methods. In response to these problems， this paper proposed a non-contact heart rate detection model （TC-Net） based on Transformer global expression and CNN local feature fusion. This paper built a parallel Transformer network and a TC-Net model of the CNN network. This model included two branches， the CNN branch was used to extract the local area features of the rPPG signal， and the Transformer was used to remove the global expression of the rPPG signal. Since the feature lengths of global expression features and local features were different， this paper proposed a feature interaction module， which used convolutional layers and up-and-down sampling modules to align the feature lengths of the two branches. Then， TC-Net fused the local features and the global expression through the feature interaction module to obtain the final feature vector for detection. Finally， comparative validation was carried out on one of the most widely used benchmarks in remote heart rate measurement evaluation， the MAHNOB-HCI dataset. The indicators had reached the best level. Therefore， the non-contact heart rate detection model TC-Net proposed in this paper is based on Transformer global expression and CNN local feature fusion， which has high accuracy and robustness. In addition， TC-Net is only built with a few simple convolution layers and attention layers， and the model complexity is low， which is convenient for subsequent practical applications.