Abstract: ‍ ‍This paper investigates the downlink transmission design of a reconfigurable intelligent surface （RIS）-aided millimeter wave massive multiple input multiple output （MIMO） communication system based on unsupervised learning. First， considering imperfect channel state information （CSI） at the transmitter， we derive a closed-form upper bound for the average spectral efficiency of the RIS-aided mmWave downlink transmission system. Furthermore， taking into account the impact of hardware impairment， discrete Fourier transform （DFT） codebook-based hybrid precoding is adopted at the transmitter， and each element of the RIS can only take limited discrete phase shift values. With these conditions， we propose an unsupervised learning based method to jointly design the hybrid precoder at the base station， the phase shift of each element on the RIS， and the digital combiner at the user equipment， under the goal of maximizing the derived system average spectral efficiency upper bound. This method takes a novel two-stage network model to generate the phase shift of each element on the RIS and the hybrid precoder at the base station respectively， while the digital combiner at the user equipment is generated by minimum mean squared error （MMSE） metric. An efficient phased training approach is also developed for the proposed two-stage network model. In this approach， the two networks generating the phase shift of each element on the RIS and the hybrid precoder at the base station are trained separately at the first two stages. After that， the two networks are trained jointly at the third stage. Simulation results demonstrate that the proposed method is more robust to imperfect CSI than the traditional iterative algorithms. When the degree of channel estimation error increases， the performance of the proposed method decreases by only about 1%， while the traditional iterative algorithm decreases by about 40%. At the same time， the proposed method is more robust to the transmission environment， and can maintain a performance improvement of at least 30% compared with traditional iterative algorithms. In addition， the proposed method has a hundred-fold improvement over the traditional iterative algorithm in the computation time cost.