A system and method is disclosed for indexing and summarizing music videos. A music video summary controller in a video playback device receives a multimedia data stream that contains a plurality of music videos. The music video summary controller (1) segments and identifies the music videos, and (2) creates a music video summary file for each music video. The music video summary file contains information such as title of the song and the name of the artist. The music video summary file also contains an identifying audio segment and video segment of the song. A user can access a list of the music video summary files and select which music video to play on the video playback device by selecting its corresponding music video summary file.