Abstract: TACAN is a wireless navigation system which can provide both range and azimuth measurement. Combining the measurements above with elevation measurement, a target’s three-dimensional position can be easily fixed. Despite the advantages above, the relatively large measurement error of TACAN system results in an inaccurate estimation of position. In an attempt to solve this problem, a filter scheme based on converted measurement modal is manipulated to realize accurate state estimation with nonlinear measurements. In according with the feature of TACAN measurements, the statistic error characteristic of converted measurement is firstly derived in the plane consists of range and elevation measurements. Based on the derivation above, the three-dimensional statistic error characteristic is deduced, which presents the final form of converted measurement modal for TACAN navigation system. A Kalman filter using proposed converted measurement modal is manipulated to estimate target state in TACAN navigation system and satisfied tracking performance is achieved. Comparative study with some state-of-the-art algorithms verifies the superiority of proposed algorithm because both the mean and covariance are conditioned strictly on measurement. The algorithm proposed in this paper is more accurate, credible and computational efficient over others for different tracking scenarios, which certifies its comprehensiveness and robustness.