A system and method for detecting and tracking humans, such as pedestrians, in low visibility conditions or otherwise. A night vision camera periodically captures a an infrared image of a road from a single perspective. A pedestrian detection module determines a position of a pedestrian in the frame by processing the captured image. The pedestrian detection module includes a support vector machine to compare information derived from the night vision camera to a training database. A pedestrian tracking module estimates pedestrian movement of the detected pedestrian from in subsequent frames by applying filters. The tracking module uses Kalman filtering to estimate pedestrian movement at periodic times and mean-shifting to adjust the estimation. An output display module interleaves detection frames and tracking frames in generating output video for the display.