Abstract: In order to not only maintain image features, but also to accelerate diffusion rate, an adaptive diffusion model is proposed in this paper. In this model, the flat areas, line edges and corners of damaged image can be distinguished according to eigenvalues of the average gradient matrix. Meanwhile, diffusion direction of image information is controlled by average gradient matrix and its eigenvectors. Thus, this model can adaptively select diffusion rate and its direction in different regions so as to maintain the structural characteristics of damaged image. The experimental results demonstrate that this model can speed up the diffusion rate while maintaining the image structure. When it is applied in image inpainting, this model can accelerate the inpainting while the inpainted image is in line with human visual effect.