Abstract: Recently deep neural networks have achieved great success in various multiple source digital image analysis and interpretation tasks. They have been gradually deployed in many applications such as smart surveillance, medical image analysis and autonomous driving. However, they are vulnerable to adversarial attacks. One of the most effective method for adversarial robustness enhancement is to retrain deep neural network using adversarial examples which maximize the loss function of the deep model. Yet it requires semantic annotation information to generate adversarial attacks and often perform poorly on the original data set. This paper proposes a self-supervised contrastive learning framework to adversarially train a robust neural network without labeled data. We aim to maximize the representation similarity between a random augmentation of an image and its instance level adversarial perturbation. Specifically, it relies on two neural networks that interact and learn from each other. The proposed method can be used to improve the adversarial robustness of pre-trained models, and can also be used to enhance the two stage robustness. We validate the proposed method on two remote sensing scene classification benchmark data sets.