A system for executing video encoding operations. The system includes a video encoder for encoding an incoming video stream into a plurality of macro blocks. A motion estimation engine is coupled to the video encoder for controlling the encoding of the macro blocks. A video rate control processor is coupled to the video encoder and coupled to the motion estimation engine. The video rate control processor receives a plurality of parameters from the video encoder that indicate an encoding complexity for a macro block and a video frame of the video stream and, upon receiving an indication from the motion estimation engine, computes a quantization parameter for the macro block. The quantization parameter is dynamically adjusted for the video stream to achieve a target bit rate.