A Novel Method to Facial Beauty Prediction Based on Self-supervised Learning

‍ ‍Facial beauty prediction is a cutting-edge research topic on how to let computers judge the beauty of faces. With the continuous progress of deep learning， certain effects have been achieved. However， facial beauty prediction based on deep learning requires a lot of training data and expensive facial beauty annotation. Therefore， how to achieve better results with few number of labeled samples remains to be studied deeply. Unlabeled face images are very easy to obtain， such as intercepting face images from videos or crawling face images from the network. Obtaining universal face representations from low-cost unlabeled face images can improve facial beauty prediction under the condition of few number. Self-supervised learning can use unlabeled data to learn better features in the upstream task， thus reducing the dependence on labeled data in the downstream task. For this reason， we present to apply self-supervised learning in facial beauty prediction in this paper， which can make full use of data samples and reduce the dependence of the model on labeled data. Self-supervised learning learns general representations from a large number of unlabeled images through upstream tasks. By using self-supervised learning， upstream tasks can extract more abundant general features， migrate the general features extracted from upstream tasks to downstream specific tasks， and improve the feature expression ability of downstream specific tasks. The self-supervised method in this paper includes two stages： the unlabeled pretraining stage and the labeled weight fine-tuning stage. Among them， the pretraining stage is divided into two steps： intra-batch object recognition and multi-view feature clustering. That is learning different points （negative pairs） of different samples through object recognition within the batch， learning the similarity （positive pairs） of sample individuals through multi-view feature clustering. Object recognition within a batch is to set the unlabeled face image of each batch as a unique hot label， so as to identify the difference between each face image in the batch. Multi-view feature clustering firstly enhances the face image for many times， then encodes the data augmented face image to obtain the facial attribute features， and finally aggregates the face image features enhanced by different data through self-supervised constraints. The weight of the facial attribute extracted in the pretraining stage is transferred to the downstream facial beauty prediction task， and then fine-tuned through the labeled facial beauty data， so as to obtain the final facial beauty prediction model. We experiment with the Large-Scale Asian Facial Beauty Database （LSAFBD） and SCUT-FBP5500 database. The method presented in this paper is better than the monitoring method based on Resnet18 under the condition of few number and higher than the accuracy of the conventional self-supervised method. Experimental results based on LSAFBD and SCUT-FBP5500 databases show that when only 1/32 of the original training set is used， accuracy is improved by 14.7% and 6.35% respectively compared with supervised learning. Compared with the traditional self-supervised learning method， it also has a certain improvement under the condition that only 1/2、1/4、1/8、1/16 and 1/32 of the original training set are used. A linear evaluation experiment was carried out to evaluate the effectiveness of the self-supervised learning method. Experimental results show that our method also has a high level. On LSAFBD and SCUT-FBP5500 databases， the self-supervised learning method presented can make full use of data samples， reduce the dependence of the model on labeled data to a certain extent， and improve the prediction accuracy.