Probabilistic Constellation Codebook Optimization for OFDM-Based ISAC for Low-Slow-Small Target Detection

Reusing communication signals for environmental sensing is an effective approach to reducing hardware complexity and improving spectrum efficiency in integrated sensing and communications （ISAC） systems. Among various candidate waveforms， orthogonal frequency division multiplexing （OFDM） is widely considered a key solution to communication-centric ISAC owing to its mature engineering implementation and favorable spectral characteristics. However， practical communication systems commonly employ non-constant modulus quadrature amplitude modulation （QAM）， which introduces symbol randomness. In sensing processing based on matched filtering （MF） and reciprocal filtering （RF）， this randomness affords elevated sidelobe levels， increased background fluctuations， and degraded parameter estimation accuracy. These effects become particularly severe for low signal-to-noise ratios and significantly limit the detection performance for low-altitude， slow-moving， and small targets. To address these challenges， herein we develop a comprehensive sensing model for OFDM-based ISAC signals， and for the first time， establish the intrinsic mapping between channel-domain and parameter-domain estimation， systematically revealing the effects of modulation symbol randomness on sensing performance under different time-frequency filtering structures. Based on this analysis， a probabilistic constellation shaping （PCS） optimization framework is proposed for MF and RF. By optimizing the input probability distribution of constellation points， the proposed method maximizes mutual information under sensing and power constraints. Therefore， communication and sensing performance can be flexibly balanced without modifying the OFDM physical layer parameters or constellation set. The simulation results show that the proposed approach significantly enhances sensing accuracy with only minor communication rate loss. Field experiments on low-altitude， slow-moving UAV targets further validate its effectiveness in realistic environments.