Abstract: To tackle the problem of detecting the presence of time-frequency overlapped co-channel multi-signal, an improved algorithm based on fractal box dimension is proposed. The box dimension of instantaneous amplitude is used as detection statistics, which of noise is proved to be a constant 1.415 by theoretical deduction, while which of co-channel multi-signal is turned out equaling to 1 approximately. According to the result, the theoretical threshold can be set. It is signal if the received signal's box dimension of instantaneous amplitude is less than the threshold, otherwise it is noise. The co-channel multi-signal is overlapped by several independent signals, of which the modulation types include MASK, MPSK, MQAM and MFSK. Simulation results prove that the method is effective and accurate in the additional Gaussian noise environments. When SNR is more than -2dB, the detection probability reaches 100% while false-alarm probability is very low. Additionally, it is robust to modulation types, modulation parameters and source number, easy to compute and can be used as real-time processing. Compared to the existing methods, the threshold proposed in this paper is more accurate, and the performance is improved greatly.