An approach for solving a problem involves modifying difficulty measures of training cases used to develop candidate solutions. A set of candidate solutions and a set of training cases are provided. Each of the training cases is associated with a difficulty measure. A candidate solution is operated on a particular training case and a performance measure for the candidate solution is obtained. A credibility rating for the candidate solution may be determined. The credibility rating is indicative of a degree to which the performance measure represents a good estimator of the difficulty measure of the particular training case. The difficulty measure of the particular training case may be modified based on the performance measure and the credibility rating of the candidate solution. The difficulty measure of the selected training case may be modified based on the performance measures.