One embodiment of the present invention provides a system that facilitates multi-versioning loops to facilitate modulo scheduling. Upon receiving a computer program, the system analyzes the code to locate loops within the program. When a loop is located, the system examines the loop termination condition to determine if it is based on a "not-equal-to" condition that makes it hard to determine beforehand whether the loop will terminate. If the loop termination condition is based on a "not-equal-to" condition, the system creates multiple versions of the loop, at least one of which will terminate and can be modulo scheduled, and at least one of which might be an infinite loop and consequently cannot be modulo scheduled.