Abstract: In the fields of face recognition, intra-class difference features caused by expression, illumination and occlusion was sharable among other classes. Thus we could get intra-class difference features from face database where each kind of image is sufficient. By simple image subtracting, ESRC got an extra database to handle the single sample face recognition problem and got a good recognition rate. However, the intra-class difference features obtained by this method didn’t include all the difference between testing and training samples.To address this issue, we used JSM to obtain the intra-class difference information. A series of related signal could be represented as a combination of common features and discriminative features by JSM. Therefore, we could get discriminative features, namely intra-class difference features from sufficient samples. Finally training samples in SRC consisted of the single sample and the intra-class difference features as well. This algorithm gets a better result on AR face database and provides an effective solution for single sample face recognition problem.