A method for generating representations of visual characteristics of images is presented. The method includes receiving search criteria. The criteria include images to be searched, query images and expected result sets, and a retrieval metric. The method identifies objects within each image and selectively generates a representation of visual characteristics of each image using descriptors from an inventory of descriptors in accordance with the retrieval metric. The method compares the representations of the query image to representations of the images to be searched and determines a search result. The search result is compared to the expected result. If the results do not match, the generating, comparing and determining steps are re-executed with reselected descriptors based on the search result and the retrieval metric. The re-execution continues in a trial-and-error approach until acceptable search results are achieved. When achieved, the method encodes the process for generating the representations.