A system and method for local deformable motion analysis accurately tracks the motion of an object such that local motion of an object is isolated from global motion of an object. The object is viewed in an image sequence and image regions are sampled to identify object image regions and background image regions. The motion of at least one of the identified background image regions is estimated to identify those background image regions affected by global motion. Motion from multiple background image regions are combined to measure the global motion in that image frame. The measured global motion in the object image regions are compensated to measure local motion of the object and the local motion of the object is tracked.