A variable length decoder (VLD) for decoding MPEG-1 and -2 syntax compliant video bit streams. The VLD includes a micro-sequencer and VLD command decode/execution unit for controlling the MPEG decoding process using a novel instruction set. The instruction set includes a set of commands for decoding the video data and a set of flow control instructions. A rotator/barrel shifter is provided for making a predetermined number of encoded bits from the video bit stream available to the VLD and a variable length table decoder for variable length decoding using the MPEG standard variable length code (VLC) tables. The variable length table decoder shares a prefix pattern matching scheme across all of the VLC tables and organizes the variable length codes into a series of subtables. Each subtable corresponds to one of the unique prefix patterns. Variable length codes are decoded by identifying a leading pattern in the video data bit stream and, in parallel, accessing the subtable corresponding to that leading pattern. Run-length and amplitude level DCT coefficient symbols are stored in compressed form, and decoded as needed by an inverse transform unit. Motion vectors are also stored until needed by a motion compensation unit.