Abstract: ‍ ‍In the secure communication scenario where the unmanned aerial vehicle （UAV） served multiple ground mobile users and there was an eavesdropper eavesdropping information， in order to maximize the secrecy rate， this paper proposed a new deep reinforcement learning algorithm to optimize the 3D trajectory of the UAV. This algorithm was named correct trajectory - deep deterministic policy gradient （CT-DDPG）. CT-DDPG algorithm used multiple deep neural networks to interact with the environment， and modified the activation function value of the output layer to replace the traditional method of using multiple activation functions to simplify the structure of the deep neural network. At the same time， the flight trajectory of the UAV was modified so that the UAV was always in the best position to maximize the secrecy rate. Compared with other reinforcement learning algorithms， this algorithm had short training time and can update the position of UAV in real time. The simulation results show that the proposed algorithm can converge quickly and complete the flight mission while ensuring the secure communication of UAV.