In a computer implemented system and method for analyzing problem solving abilities, analytic models are produced to quantify how students construct, modify and retain problem solving strategies as they learn to solve science problems online. Item response theory modeling is used to provide continually refined estimates of problem solving ability as students solve a series of simulations. In parallel, student's strategies are modeled by self-organizing artificial neural network analysis, using the actions that students take during problem solving as the classifying inputs. This results in strategy maps detailing the qualitative and quantitative differences among problem solving approaches. The results are used to provide reports of strategic problem solving competency for a group of students so that teachers can modify teaching strategies to overcome noted deficiencies.