Abstract:  Harbor recognition is an important aspect for remote sensing ocean application research. Accurate and fast harbor detection has great significance for improving the ability of automatic interpretation for remote sensing images. Man-made Object is composed of some components according to certain structure, besides perceptual organization of human vision can acquire related organization and structure of images in the case of ignoring prior knowledge of image content. Therefore perceptual organization is helpful for recognizing man-made object, this paper introduce perceptual organization principle into harbor detection. This paper proposes a kind of harbor detection method based on perceptual organization by establishing group of jetty areas, group of lines between jetty butts and harbor organization mechanism based on production rules. Compared with an existing harbor detection algorithm that has inaccurate harbor contour result and inefficiency, our experiments show that novel harbor detection method has an accurate result, a whole contour and faster operation.