A system and process for controlling the coding bit rate of streaming media data is presented where a server streams data that exhibits one of a number of coding bit rates supported by the server. Initially, the server chooses the coding bit rate. However, after this startup period, the client provides coding bit rate requests. The server transmits the streaming media data at the most appropriate supported coding bit rate closest to the rate requested. The coding bit rates requested are those estimated to provide a high quality playback of the streaming data while still keeping a decoder buffer of the client filled to a desired level. A leaky bucket model is incorporated so that the changes in buffer duration due to natural variation in the instantaneous coding bit rate are not mistaken for changes in buffer duration due to network congestion.