Application of Q-Learning algorithm in channel selection for opportunistic spectrum access

Considering the problem of channel selection for opportunistic spectrum access (OSA), a QLearning based channel selection scheme was proposed for OSA in this paper. A secondary user detected the channels licensed to some primary users periodically before it decided whether to transmit in the OSA system. Under imperfect sensing circumstances, the construction of channel selection model of the secondary user and the designation of an appropriate reward function play a significant role in the continuous interaction and learning between the agent and unknown environment, thus selecting the channel with the maximum cumulative reward. During the learning stage, a Boltzmann learning rule using simulated annealing ideas was employed to realize the tradeoff between channel exploration and exploitation. As the simulation results show, the proposed algorithm can get access to suitable channel, and raise the average system capacity and throughput of the secondary user effectively in the absence of prior knowledge on the channel environment.