High quality replay of a streaming multimedia video segment on demand is provided. The multimedia segment may be transmitted as a stream by a server system and received by a client system over a fixed bandwidth communications path. The client system receives a base layer of the stream, decodes the base layer and displays the decoded base layer at a first quality level. The client system stores at least a portion of the base layer in a cache. When a request for replay service for the multimedia segment is received from a user, for high quality replay, the client system may receive at least one enhancement layer of the stream corresponding to the stored portion of the base layer. The client system may then read the stored portion of the base layer from the cache, decode the stored portion of the base layer and the at least one enhancement layer, and display the decoded base and enhancement layers at a second quality level, higher than the first quality level.