A distributed parallel computing system actively takes advantage of problem partitioning to balance the computing load among computing resources continually during processing. Variable problem partitions (VPPs) are initially defined as groups of original problem cells (OPCs). VPPs may be redefined and redistributed during execution, if necessary, to optimize performance based on the actual computing agent parameters and costs observed or reported through self-tests. For example, a good rule for efficient execution of a computing problem may be that the time required to perform a computation sequence (iteration) of all OPCs in a VPP should be comparable to the time required to share results via edge OPCs at the VPP collection perimeters. The rules that yield cost-efficient execution may be saved and re-used to generate initial partitionings for subsequent computing problem execution runs.