Millimeter Wave Radar Based on Spatial-Temporal Transformer 3D Human Posture Reconstruction

‍ ‍Deep learning technology facilitates the accurate extraction of human motion features and reconstruction of 3D poses by using millimeter wave （mm Wave） radar signals. However， current mm Wave radar human posture reconstruction frequently adopts a single-stage strategy， which involves directly mapping radar images to 3D joint coordinates. Implementation of this cross-domain hierarchical mapping task creates challenges for the network in terms of reconstruction accuracy， depth-information expression， and pose coherence To address this problem， this paper proposes a 3D human pose reconstruction model using multi-stage mm Wave radar， termed the spatial-temporal pose reconstruction transformer （STPRT）， which improves reconstruction accuracy using a two-stage strategy. First， a parallel multi-resolution subnetwork is constructed to extract multi-scale 2D joint information and spatial position features from horizontal and vertical radar images and fuse them， after which the fully connected layer generates 2D human pose coordinates. Second， the spatial-temporal Transformer encodes the high-dimensional spatial features of the 2D joint coordinates in each frame using the spatial attention module. The temporal attention module captures the temporal evolution of pose features in the sequence frames， enhances the depth perception and spatial accuracy between poses， and improves the mapping process from the 2-3D pose. In addition， the exponential moving average （EMA） strategy is employed during the training process to adjust the gradient descent， thereby improving overall mapping accuracy and consistency. Verification using the mm Wave radar public dataset RFSkeleton3D demonstrate that， compared with existing mm-Pose and RF-based pose machine （RPM） models， the proposed model reduces the average joint position error to 7.3 cm and decreases the number of parameters.