Systems and methods for representing sequential motion patterns are described. In one aspect, video frames are converted into a sequence of energy redistribution measurements. One or more motion filters are then applied to the ER measurements to generate one or more temporal sequences of motion patterns, the number of temporal sequences being a function of the number of motion filters.