Abstract: To improve the robustness of the blind source separation algorithm in the presence of reverberation and noise, a blind source separation method for convolutional mixed signals under underdetermined conditions is proposed in this work. Due to the high reverberation, the mixing model presents convolution property even in the time-frequency (TF) domain. By using the characteristics of room impulse response, the instantaneous model in the TF domain is extended to a convolution model that is more suitable for high reverberation environments, Based on the convolution model in the TF domain, a joint optimization problem of blind source separation is developed, and the resulting optimization problem is solved via an alternating optimization framework where the alternating direction method of multipliers (ADMM) is devised. The results of simulation experiments show that the performance of the convolution model in the reverberation environment outperforms the instantaneous model, which verifies the effectiveness of the convolution model in the TF domain.