Abstract: Chinese is a syllabic language and its pronouncing process bears the characteristic of “rugby”. Aiming at these, a model is proposed to describe dynamic Chinese visemes with inner-syllabic and inter-syllabic modelings respectively. As for a syllable, lip sub-movements model based on initials and finals are used, which first extracts the lip feature parameters of initials and finals and get the simplified viseme model by categorizing the mouth shapes according to the parameters, and then computes the mouth shape likelihood between lip sub-movements and the pronouncing process of syllables. As for inter syllables, weighting function of vowel impact grading is used to simulate the effect of co-articulation. Experimental results show that comparing to the Chinese visemes described by phoneme or triphone model, the method promotes the animation efficiency and makes more reasonable and natural Chinese lip animation.