A microprocessor has a data stream prefetch unit for processing a data stream prefetch instruction. The instruction specifies a data stream and a speculative stream hit policy indicator. If a load instruction hits in the data stream, then if the load is non-speculative the stream prefetch unit prefetches a portion of the data stream from system memory into cache memory; however, if the load is speculative the stream prefetch unit selectively prefetches a portion of the data stream from the system memory into the cache memory based on the value of the policy indicator. The load instruction is speculative if it is not guaranteed to complete execution, such as if it follows a predicted branch instruction whose outcome has not yet been finally determined to be correct. In one embodiment, the stream prefetch unit performs a similar function for store instructions that hit in the data stream.