Transition Matrix Analysis Analyzing Students Use of Cognitive Resources in Physics

Conceptual surveys of multiple choice format have been developed to test the effect of pedagogical interventions on students understanding of physics knowledge. Predominantly, they are administered in pre and posttest settings and analyzed to obtain a performance gain. However, focusing on the correct answers to each question alone ignores the incorrect options which could inform us the stability and coherency of the knowledge structure of students. According to the resource model framework, each specific answer from students could reflect a specific cognitive resource being activated and implemented in the context at hand. Consequently, conceptual surveys could demonstrate how instruction could affect the activation of different cognitive resources of students in a variety of contexts when administered before and after instruction. Guided by the resources framework, we propose a transition matrix analysis to analyze data collected through conceptual surveys to investigate how instruction affects the consistency of cognitive resources activation by students. To provide proof of concept, we demonstrated how to utilize this method by analyzing students responses to a subset of questions from the DIRECT survey in a classroom study.