A voice speed control apparatus comprising: an utterance/non-utterance judging unit judging whether a processing target segment in inputted a voice signal is an utterance segment or a non-utterance segment; a non-utterance continuation length acquiring unit acquiring a non-utterance continuation length representing a length of the voice signal judged continuously to be the non-utterance; a determining unit determining a reproducing speed of the processing target segment in the voice signal in accordance with the non-utterance continuation length so that the reproducing speed gets higher as the non-utterance continuation length gets larger and so that an increase in reproducing speed is restrained to a greater degree as the non-utterance continuation length becomes smaller; and a changing unit changing the reproducing speed of the voice signal, corresponding to the reproducing speed.