Abstract: Aiming at the problem of abnormal behavior detection, an AP clustering sparse representation video anomaly detection method based on spatiotemporal depth features is proposed.In view of the uneven distribution of large amount of background information and effective information in the video sequence, the optical flow method combined with non-uniform cell segmentation is used to extract the moving targets of the video and obtain the spatial temporal interest cuboid with different spatial sizes.The three-dimensional convolutional neural network is used to extract the spatial-temporal deep features of different spatial temporal interest cuboids to describe the original video sequence in three dimensions. Aiming at the big data problem of deep learning method, we apply the AP clustering method in dictionary learning, and the representative feature of the training sample is add to the dictionary, which greatly reduces the dictionary dimension and reduces the memory requirement of sparse representation. The test samples are clustered by AP and only the representative cluster centers are detected. The time for computation will be reduced and the detection effect is not sensitive to threshold. The experimental results show that the proposed method is superior to the existing detection methods.