Abstract: In the research of object tracking and detection, the shadow of objects usually is thought as part of the moving foreground, it will cause the merger of moving targets, geometric distortion and even loss of target. If the shadow of moving objects could be removed we could truly get the gravity center of objects and recognize the action of the object correctly. In this paper we propose a shadow removal approach in YCbCr color space. Firstly, we used background subtraction method to get the candidate target area with a shadow. Second, we did background subtraction in the YCbCr space, because the YCbCr information of Shadow and target objects were very different, So you could get the precise target area after shadow removal through threshold determination, it would improve accuracy and robustness of a specific object recognition. Experimental results show that the method could remove the shadow while retaining the information of foreground objects as much as possible.