Abstract: The combination of UAV technology and computer vision technology has a wide range of requirements in the civil and military fields. However， the current algorithms can not adapt to the special conditions of UAV， such as rotation of view angle， obstacle occlusion， target scale change and so on. According to the practical difficulties and challenges， a multi-target detection and tracking algorithm based on deep learning is proposed.The main work is as follows： in the aspect of detection， the detection network based on darknet53 is trained as the detector through the public data set and a large amount of data actually collected； in the aspect of tracking， the car Reid data set is used to train a residual network to extract the appearance information of the target， the Kalman filter is used to extract the motion information of the target， and a fusion formula is used to integrate the two information Finally， the tracking result is obtained by Hungarian matching algorithm. Experiments are carried out on uav123 data set and actual data set respectively， and the conclusion is that the algorithm can detect and track stably under the conditions of rotation of view angle， change of target scale and occlusion of obstacles.