Hybrid Deep Reinforcement Learning for Dynamic Node Selection in Underwater Sensor Networks

A key challenge when performing target tracking tasks using underwater sensor networks is the dynamic selection of optimal sensor nodes to reduce resource consumption while maintaining tracking accuracy. Reinforcement learning offers a data-driven， self-learning， and intelligent decision-making solution for this node selection problem. However， traditional online learning methods face difficulties in simultaneously ensuring training efficiency and policy robustness owing to issues such as a scarcity of real data and insufficient environmental adaptability. To address this challenge， this study proposes a dynamic node selection method for underwater sensor networks using a hybrid reinforcement learning framework. The method achieves data augmentation and environmental adaptation by combining offline virtual training with online real interaction in a hybrid decision-making mechanism. First， the posterior Cramér-Rao bound for target tracking is derived based on the target motion model and sensor observation characteristics， and an optimization model aimed at minimizing the number of active nodes is established under this constraint. Subsequently， a Hybrid Deep Q-Network is constructed to solve the sensor node selection problem. Virtual states and observation data are generated during real measurement intervals by leveraging prior knowledge such as the target motion model and measurement noise， thereby effectively expanding the training sample set and balancing training efficiency with policy adaptability. Experimental results show that the proposed method outperforms both pure online reinforcement learning and traditional heuristic optimization algorithms in tracking accuracy， with a significantly accelerated convergence speed and substantially reduced decision time. This method can effectively meet the real-time node selection requirements of dynamic underwater sensor networks with a low computational overhead.