Abstract: ‍ ‍Automatic Modulation Classification （AMC） plays an important role in the field of communication. Aiming at the insufficient feature extraction ability of single traditional convolutional neural networks in signal classification problems， we exploit an end-to-end dilated convolutional neural network using multi-dimensional features to classify modulated signals. This method not only uses the original sampling signal， but also uses the instantaneous amplitude and phase information of the input signal； after the original IQ （In-phase and Quadrature） data is input into the neural network， the network first preprocesses the input IQ signal through the built-in data preprocessing module to extract the original signal Amplitude and phase information， and then the two kinds of feature information are respectively extracted through two parallel Convolutional Neural Network （CNN） structures. We exploit the dilated convolution to make the convolution network has a larger receptive field， and it also better combines contextual information. Meanwhile expansion convolution is combined with residual network structure. The output feature vectors are matrix-multiplied to achieve the purpose of fusion. The whole recognition process is based on end-to-end. The data preprocessing module is embedded in the neural network， so the neural network completes the data preprocessing. Thanks to the end-to-end structure， we only need to send the original IQ data directly to the neural network； The simulation experiment results show that compared with the convolutional neural network with a single structure，the recognition accuracy of the model proposed in this paper has been greatly improved on RML2016.10a dataset，and the highest recognition accuracy can reach 85%.At the same time， the model also has a certain ability to classify 16QAM and 64QAM signals that cannot be recognized by single branch structure.