Described herein are a number of approaches for implementing a video encoder with hardware-enabled rewind functionality. In several embodiments, rewind functionality can be implemented in hardware, in a manner which allows the transform engine of the encoder to reprocess video data, without requesting data from other stages in the encoder. Such rewind functionality is useful in implementing some video standards in a pipeline architecture, such as the H.264 standard. In one embodiment, a method of encoding video data is described, which involves obtaining a first portion of video data from a first location in a buffer, and performing an encoding operation on it. The second portion of video data is obtained from a second location in the buffer, and encoding operations begin on the second portion. The first portion of video data can be retrieved from the first location, in order to reprocess the first portion if necessary.