Enhancing Deep Shadow Features for Any-to-Any Relighting

‍ ‍Any-to-any relighting is to relight the source image with the illumination implicitly given in the guide image. Existing any-to-any relighting methods adopt an end-to-end learning way， resulting in a high coupling between shadow features and color temperature features， which further affects the accuracy of shadow generation. To this end， this paper proposes an any-to-any relighting method based on deep shadow features enhancement. The key to this method is to design an additional shadow decoder to directly generate the corresponding shadow image from the implicit representations. At the same time， to make full use of the learned shadow features， we introduce a feature fusion module based on the attention mechanism to realize the adaptive fusion of relighting features and shadow features. In addition， we experimentally found that using a polynomial kernel function to map the source image to high-dimensional features and then taking them as network input can further improve the performance. Experiments on the VIDIT dataset demonstrate the effectiveness of the proposed method.