Computing devices are tested by providing a suite of test programs on a server for execution by a plurality of identical computing devices that are coupled to the server. A respective unique identifier is assigned to each of the plurality of the computing devices, for use in communicating with the server. The test programs are downloaded from the server for execution by the computing devices coupled thereto, so that at different computing devices execute different test programs substantially simultaneously. The server receives messages from the computing devices with respect to execution of the test programs, each of the messages containing the respective unique identifier, and controls the execution of the test programs in the suite based on the messages. The server dynamically distributes the test programs to a changing population of the computing devices, optimizing the distribution so as to minimize the time to complete the suite.