Abstract: With the rapid development of digital imaging technology， images have become some of the important carriers of information in daily life and learning. However， owing to the inevitable data transmission errors， improper storage， or obscured important information， the image information may be incomplete， which will significantly affects people’s understanding and analysis. Recently， several progressive image inpainting networks， which can restore gradually corrupted images from coarse to fine， have achieved remarkable progress. However， these progressive methods may cause error propagation from lower progressive levels to higher levels. Aiming at the limitations， this paper proposes an image inpainting algorithm based on multi-stage reconstruction collaborative optimization （MSNet） for better visual consistency and texture continuity. In MSNet， a three-stage inpainting structure is designed to collaboratively optimize progressive inpainting contents. Specifically， the first stage is for the coarse inpainting， and then the structure and details are polished in the second parallel inpainting stage. In the second stage， we propose a parallel image content refinement module （PCRM）， in which the structure information and local details are independently refined by a two-branch parallel structure： self-attention based U-Net and enhanced residual branches. Here， self-attention based U-Net tends to extract the structure of image features and restore them through the multi-head attention mechanism. An enhanced residual structure adds a module for enhancing detail information by gradient optimization， which focuses on the recovery of important details and improves the discrimination of important content and background information. After PCRM， a structure-detail fusion structure is presented in the third stage of MSNet to embed details into structural information and reduce the visual discontinuity， which can fuse the multiple parallel reconstructions generated by second stage and obtain accuracy inpainting results with better spatial representation. Extensive experimental results demonstrate that the image inpainting results of proposed MSNet can obtain more accurate image restoration results with better visual quality.