Abstract: Security inspection is the first guarantee for the safety of people’s lives and property. Intelligent security inspection is an inevitable trend for the future development of the security inspection industry. On the consequences of the complex background, scales of prohibited items and blind with each other, traditional object detection algorithms always fail to get an excellent performance. Based on the object detection architecture of SSD neural network, we proposed a novel Asymmetrical Convolution Multi-View Neural Network(ACMNet), which gain a good result on multi-scale prohibited item detection in X-ray security image. Three modules have been added to the ACMNet: Asymmetrical Tiny Convolution Module (ATM), Dilated Convolution Multi-View Module(DCM), Fusion strategy of multi-scale feature map(MF). The detailed features learned by ATM module are going to identify small-scale prohibited item,the DCM module provides local and global information between different objects context to solve blind problems and MF strategy is performed to fuse feature maps of different scales between high and low level features to improve the accuracy of prohibited items under background interference in security images. In the simulation experiments, using the open source dataset and self-built data set in the field of X-ray security inspection, ACMNet has achieved satisfactory results in terms of accuracy.