Abstract: It is challenging issues to extract the moving foreground objects from background robustly in visual surveillance system. In this paper, we present a novel texture-based cluster-like algorithm to detect motion with codebook and Gaussian local binary patterns (GLBP), which may get texture background model on-line. Firstly, a codebook model like pixel cluster is constructed. Distribution of background pixels is presented by pixel cluster using computing the color and brightness distortion between codebook and current pixel. Our algorithm updates the codebook model both in initial step and detection step to deal with changes of background pixels. A single Gaussian model of pixel-wise is used to build the pixel’s value change model on-line. Gaussian local binary patterns background model is constructed on-line by applying the correlation and texture of spatially proximal pixels. Finally current image is segmented into two parts, foreground and background by fusing the three features: codebook model, single Gaussian background model and Gaussian local binary patterns. Experiments show that our proposed algorithm achieves robust performance in natural videos.