Abstract: ‍ The evolution of B5G and 6G， along with associated wireless technologies， has not only demanded higher communication rates but has also facilitated various industrial applications such as vehicle to everything， smart manufacturing， and the industrial Internet of things， all of which rely on reliable wireless communication and accurate sensing capabilities. However， the proliferation of base stations operating in the same geographic area for next-generation communication systems has led to increased challenges related to interference and power attenuation at cell boundaries. In response， a distributed access point based cell-free integrated sensing and communication （ISAC） system has emerged as a promising solution. This system overcomes the limitations of co-location design and fosters effective ISAC functions. On one hand， the cell-free system focuses on achieving a user-centered communication service architecture. Each distributed base station serves nearby communication users and automatically switches based on movement， eliminating cell boundaries while ensuring large-scale continuous coverage and high-quality connections for network users. On the other hand， hardware and wireless resources can be effectively shared， enabling traditional communication infrastructures to incorporate sensing capabilities at minimal cost. The base station gathers target state information in the environment through signal analysis， ultimately achieving collaborative gains in communication and perception functions through mutual assistance， enhancing spectrum efficiency while reducing communication overhead. However， achieving communication signal-based sensing functionality still faces challenges， particularly in implementing directional beamforming due to hardware limitations. Traditional beamforming methods incur significant signaling overhead as the codebook and codewords increase， relying on specific environmental assumptions. To meet the performance requirements of cell-free ISAC systems， a new beamforming scanning algorithm needs to be designed. This paper presents an intelligent method for joint detection beamforming codebook selection in cell-free ISAC systems operating in the millimeter-wave frequency band. Firstly， preprocessed information from receiver access points is utilized to calculate path loss and target estimation information， while statistical metrics are constructed to monitor the presence of targets. Subsequently， multiple transmissions of beam codebook selections are conducted， and feedback information from echo signals is recorded. Reinforcement learning algorithms are then employed to explore the mapping between optimal code words and strong feedback information， resulting in an efficient beam codebook exploration strategy. Continuous adjustments to communication environments and target characteristics allow the learning-based beam scanning model to eliminate dependence on environmental prior knowledge， which significantly enhances the model's generalization performance. Numerical experiments validate the effectiveness of the proposed algorithm， demonstrating a substantial reduction in the number of explorations required for identifying optimal transmission and reception beam pairs， particularly when compared to traditional beam scanning algorithms. This efficiency is particularly pronounced in scenarios involving large-scale beam combinations. Furthermore， the proposed algorithm exhibits robustness even under low signal-to-noise ratio conditions， consistently identifying optimal transmission and reception beam pairs with remarkable efficiency， requiring only a minimal number of attempts.