Abstract: In super resolution through neighbor embedding (NE), the training and reconstruction process were performed in the feature space, and so the feature selection is very important. In addition, most NE-based methods use the dictionary without test, which make the choice of neighbors blindly. Considering the importance of feature selection and to avoid the blindness of neighbor selection, we proposed a novel super resolution through neighbor embedding approach. Firstly, estimate the global image with field of experts. Secondly, recover the residual image through NE-based method. In the process of residual image reconstruction, we firstly divide the image into patches and extract the feature with linear filter. Then, the training images are divided into two groups. One group is used to train the low and high resolution reconstruction dictionary, and the other is used to test the reconstruction dictionary and obtain the neighbor selection dictionary. Finally, we select the neighborhood size for the input image patches and reconstruct them with reconstruction dictionary. The experiment results show that, compared with other NE-based methods, our method can recover more high frequency details and sharp edge. What’s more, the recover images have better subjective and objective quality.