Micro-expression Recognition Based on Optical Flow Method and Pseudo Three-dimensional Residual Network

‍ ‍‍ ‍Micro-expression is a dynamically changing facial expression with complex temporal and spatial characteristics， which brings great difficulties to its recognition. A micro-expression recognition method based on optical flow method and pseudo three-dimensional residual network （P3D ResNet） was proposed. By means of optical flow， the movement information of micro-expression is modeled， which provides the network with key information and enriches the spatial dimension of data. The temporal and spatial characteristics of micro-expression were further studied by psedu-3D residual network. Firstly， the three mainstream micro-expression datasets were fused， and the fused datasets were preprocessed. Then， the TVL1 optical flow method was used to extract the optical flow feature sequence representing the micro-expression motion information， and the obtained optical flow feature sequence was connected with the micro-expression gray image sequence by channel to form a new three-channel micro-expression image sequence. Finally， the obtained micro-expression data was enhanced and fed into the psedu-3D residual network. Meanwhile， the spatial and temporal features of micro-expression are extracted to realize the recognition of micro-expression. Among them， P3D ResNet used two-dimensional convolution filters to extract spatial features of micro-expression in the framework of residual network， and one-dimensional convolution filters to extract temporal features of micro-expression to simulate three-dimensional convolution filters. Experiments on fused dataset showed that the performance of method in this paper had a significant improvement over the benchmark method， UF1 and UAR had increased by 14.71%， 14.58%， respectively. The recognition performances of the method proposed in this paper are better than the existing advanced methods on the fused datasets and three independent datasets， which proves the advancement and robustness of the micro-expression recognition method in this paper.