An apparatus and method for combining digital information with at video stream and for using the digital information to modify or augment video frames in the video stream is disclosed. The apparatus for decoding a video stream comprises a video receiver configured to receive a video stream, the video stream including a plurality of video frames. A video processor is configured to receive auxiliary data corresponding to the video stream, the auxiliary data including the information indicative of at least one video frame of the plurality of video frames. The video processor is further configured to modify the video frame in accordance with the auxiliary data. The video processor uses the received auxiliary data to identify a portion of the at least one video frame, the portion being modified in the act of modifying the video frame, other portions of the at least one video frame not being so modified. The video processor applies a filter to the portion of the at least one video frame.