A speech recognition system plays prompts to a user in order to obtain information from the user. If the user begins to speak, the prompt should stop. However, the system may receive sounds other than speech from the user while playing a prompt, in which case the prompt should continue. The system temporarily stops a prompt when it detects a sound or when it preliminarily determines that a detected sound may be a target sound (such as words from the user). The system then determines whether the received sound is a target sound or some other sound (such as coughing or a door shutting). If the received sound is not determined to be a target sound, then the prompt is resumed. The prompt can be resumed at any appropriate point, such as the point where it was stopped, a prior phrase boundary, or the beginning of the prompt.