Abstract: The samples in each class set can be supposed to distribute on a same low-dimensional manifold of the high-dimensional observation space. With regard to how to take advantage of this manifold structure for the effective classification of the multiple observation sets, label propagation classification algorithm of multiple observation sets based on L1-Graph representation is proposed in this paper. Based on sparse representation to construct L1-Graph and obtains a similarity matrix between samples as the first step. All observation images belong to a same class is restricted that to obtain a label matrix of special structure on the basis of semi-supervised label propagation algorithm. Lastly, transform the computation of the optimization label matrix to an optimization problem of discrete object function and obtains the class of the test samples. Experiments on the USPS handwritten digit database, ETH-80 object recognition database and Cropped Yale face recognition database show that the proposed method is valid and efficient.