An adaptive mutation operator for a genetic algorithm that programmatically mutates individuals in a constrained optimization for a modeled system is discussed. The mutation operator takes into account linear and bound constraints in generating new mutated individuals. The mutation operator generates random mutation direction vectors and random initial step sizes. A mutated individual is generated and moved along a randomly chosen mutation direction vector a distance equal to the initial step size. The generated mutated individual is compared to the linear and bound constraints. In the event the generated mutated individual is located in an infeasible region, the illustrative embodiment of the present invention automatically adjusts the step size to a smaller value and generates another mutated individual along the chosen mutation direction vector. The process iterates until the generated individual is within the feasible region. The number of available valid mutation directions increases as the step size decreases.