Abstract: Millimeter wave transmission combining with orbital angular momentum (OAM) can reduce the loss of beam power, improve the purity of the received signal, and thus increase the system capacity. During the transmission process, the antennas at the transmitter and receiver must be parallel and coaxial, otherwise the accuracy of mode detection at the receiver can be seriously influenced. In this paper, firstly the OAM millimeter wave communication system model is described, and then the channel modeling of the system is carried out based on the energy migration characteristics of OAM modes, including theoretical derivation of the power distribution between different OAM modes and construction of the channel transmission matrix in the case of the off-axis misalignment between the transmitter and receiver. Based on the transfer matrix, the system capacity is calculated and analyzed. The simulation results show that the capacity of OAM millimeter-wave communication system decreases and then keeps constant with the increase of lateral offset distance. And when the mode group with smaller modes or larger interval is selected by the system, the performance is less affected by the offset and the system capacity is larger. The results lay a theoretical foundation for the improvement of the system performance in the off-axis misalignment case.