Abstract: ‍ ‍Due to the flexible， swift， and low-cost features of unmanned aerial vehicle （UAV） networking， aerial base stations are considered as a promising technology in future wireless communications. UAV clusters can complete complex tasks through coordination and cooperation， which has great research and practical value， while efficient communication between UAVs is a big challenge currently facing. In order to cost less transmission power as much as possible under the premise of meeting the communication rate between the UAVs， this paper proposes an intelligent decision-making algorithm for cluster scheme and power control based on deep reinforcement learning. First， this paper designs three UAV cluster solutions to provide seamless wireless coverage for the ground users； then， this paper proposes a deep Q-network （DQN） based cluster scheme and power control algorithm， and uses a deep neural network to output the decision-making UAV cluster scheme and transmission power under different conditions， and studied the importance sampling technique to improve training efficiency. The simulation results demonstrate that the proposed deep reinforcement learning algorithm can correctly select the UAV cluster scheme and transmission power， and use less transmission power to meet the communication rate requirements between the UAVs compared with the Deep Learning Without Reinforcement Learning （DL-WO-RL） algorithm. Moreover， importance sampling technique can shorten the convergence time of the DQN algorithm.