Abstract: Facial Expression Recognition (FER) has always been an important field in pattern recognition. Because facial recognition can be easily affected by light, attitude and individual differences, facial recognition in the wild still did not obtain considerable progress. To achieve better facial recognition performance, a method of transferring face recognition net into facial net was proposed based on fine-tuning face recognition net. Furthermore, Softmax-MSE loss function and Double Activate Layer (DAL) structure were proposed to improve the discriminative ability of the model. The experiments were performed on FER 2013 dataset and SFEW 2.0 dataset and obtained overall classification accuracy of 61.59% and 47.23% respectively, which has achieved state-of-the-art performance.