Abstract: For the residual noise problem faced by existing speech enhancement algorithms, an optimal speech enhancement algorithm was presented, which was combined with the masking properties of human auditory system. The Algorithm consisted of two stages, and the MMSE-LSA spectral estimation method was applied to the noisy speech for noise reduction in the first stage. After the first stage processing, the signal-to-noise ratio of the noisy speech was improved. Then, for the residual noise of the enhancement speech signal processed by the first stage, the masking properties of human auditory system was used in the second stage. A perceptual enhancement filter was designed based on the human auditory perception, which could effectively remove the residual noise. Experimental results under different kinds of background noise and different signal-to-noise ratios demonstrate that the residual noise is significantly reduced and the subjective perception quality of the enhanced speech signal is increased.