Abstract: In order to overcome the harbor background interference and prevent a huge number of annotations from large view optical remote sensing images, this paper proposed a structured sparse representation method based on small sample set to realize the inshore ship detection. Here, the constructed structured sparse representation dictionary contained three parts which include inshore ship sub-dictionary part, harbor background sub-dictionary part and error matrix part. By iterated structured sparse representation dictionary training process, discriminative sparse coding can be generated for inshore ship detection according to ship confident value calculation. First, the multi-directional near-shore vessel target sample and the complex background information samples are subjected to HOG feature extraction and PCA analysis to initialize the atom. Then the dictionary is trained by K-SVD and LASSO algorithm. The error matrix is introduced into the dictionary to represent the intra-class difference of the sample, which enhances the discriminative ability and system robustness of sparse coding. Finally, the confidence calculation method for ship target area extraction is proposed, and the generated structured sparse coding is analyzed to realize the rapid inshore ship extraction. Experiments are carried out on different sizes dictionary models and structured sparse expression models before and after the introduction of error matrices. The experiment results show the effectiveness of proposed structured sparse representation, and achieve better detection performance than state of the art method.