Abstract: This paper proposes a convolution kernel initialization method based on sample image local pattern clustering, which can be used to initialize the convolution kernel in Convolutional Neural Network training. In convolutional neural networks, the main role of convolution kernels can be seen as the use of matched filtering to extract local patterns in an image and use it as a feature for subsequent image object recognition. To this end, a part of typical sample images are selected in the image training set, and subgraphs of the same size as the convolution kernel are extracted from these images as image local pattern vector sets. Firstly, the topological characteristics of the local pattern subgraph set are applied to the rough classification. Then, for each subclass after the rough classification, the potential local pattern subgraph is obtained by using the potential function clustering method to form the candidate subgraph pattern set. They are trained as the initial convolution kernel of CNN. The experimental results show that the proposed method can significantly accelerate the convergence speed of the initial CNN network training, and also improve the network recognition accuracy after the final training.