A system and a method are disclosed for an adaptive discriminative generative model with a probabilistic interpretation. As applied to visual tracking, the discriminative generative model separates the target object from the background more accurately and efficiently than conventional methods. A computationally efficient algorithm constantly updates the discriminative model over time. The discriminative generative model adapts to accommodate dynamic appearance variations of the target and background. Experiments show that the discriminative generative model effectively tracks target objects undergoing large pose and lighting changes.