In a video processing system scene shifts are detected and the need for adjusting the brightness between pairs of adjacent frames of a video motion picture is assessed. The need for brightness adjustment is assessed and a possible scene shift is detected by identifying homogeneous pixels in at least one frame of a pair of adjacent frames of the motion picture. The brightness of one of the adjacent frames is then adjusted and a scene shift is detected based on the ratio of the number of homogeneous pixels undergoing a substantial change in brightness relative to the total number of homogeneous pixels identified. When a need for a brightness adjustment is determined the brightness of one of the frames is adjusted before estimating optical flow between the frames.