Abstract: As a key enabling technology for sixth-generation （6G） communication networks， integrated sensing and communication （ISAC） facilitates both wireless communication and environmental sensing by sharing hardware architectures and signal processing mechanisms. This integration enhances spectrum efficiency while reducing hardware costs. Concurrently， unmanned aerial vehicle （UAV）， functioning as intelligent nodes in three-dimensional space， have gained widespread application in many fields， such as military reconnaissance， logistics delivery， and disaster response， owing to their flexibility， wide coverage， and cost-effectiveness. Investigating ISAC networks integrated with UAVs is essential for enhancing spectrum efficiency and optimizing the utilization of low-altitude resources. However， the broadcast nature of wireless communication presents significant challenges to the secure transmission of sensitive information in ISAC networks. This underscores the need to enhance information security through physical layer security techniques. This study focuses on the physical layer secure transmission in a UAV swarm uplink scenario， wherein a ground ISAC base station transmits confidential information to the UAV swarm while sensing multiple ground targets. Multiple eavesdropping UAV positioned near the UAV swarm attempt to intercept the confidential information. To enhance the physical layer security performance of the ground ISAC base station tasks， a joint optimization problem is formulated involving the transmit beamforming strategy of the ISAC base station and the trajectory planning of the UAV swarm. A deep reinforcement learning （DRL） based algorithm is proposed to solve this optimization problem. First， a sum average secrecy rate maximization problem is formulated under sensing performance constraints and subsequently modeled as a Markov decision process （MDP）. The security performance of the ISAC network is then improved through joint optimization of the decision variables via carefully designed action and policy networks. Simulation results demonstrate that the proposed method enhances the average secrecy rate by 185.3% compared to a benchmark algorithm， thereby validating its effectiveness in both trajectory planning and beamforming design.