A crawler that is either based on an interactive mode of operation or includes an interactive mode along with one or more other modes, such as automatic or manual. Similar to an automatic mode crawler, the crawler traverses web sites, web content and links. However, if the crawler encounters a structure that requires human interaction, such as a form, a radio button selector, a drop down selector, a human verification test, etc., the crawler pauses and prompts a user to take action.