Abstract: A novel approach using bayesian network is proposed for local semantic modeling of natural scenes. Directions of region’s neighborhood and adjacent region’s semantics are involved in the structure of the bayesian network. Image representation is formed by the local sematic descriptors for categorization of scenes. Parameters of the bayesian network are learned using the training set with manual annotation. For test images, the probability of the regions’ semantic is infered by the bayesian network based on the lowlevel features as well as the semantics of adjacent regions. The final annotation result of whole image regions is approached by iterations through th network. Images are represented through the frequency of occurrence of the local semantic objects. Experiment conducted on natural scenes’ dataset demonstrate the effectiveness and effciency of the proposed approach for local semantic modeling and categorization of natural scenes.