A pre-processing system and method is used to compensate for global brightness changes in consecutive video frames. The system and method uses a brightness change model to identify the global brightness changes and to discard the brightness in a source frame so that the brightness changes between consecutive reconstructed video frames is kept to a minimum. The Hough Transform is used as one example of the model to accumulate votes for different values of brightness change model parameters. Compensation for any detected brightness changes in the source frame is performed with filtering of the model parameters when necessary, to help avoid any artifacts in the compensated source frame. Different application sequences of the brightness change model allows for multiple levels of processing delays.