Tolerating Approximate Answers about Student Learning
Presented by Derek Briggs at the Oxford University Centre for Educational Assessment on May 24, 2018.
The statistician John Tukey once wrote “Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise.” Some prime examples of vague questions in education, for which only approximate answers are likely available: What are students learning? How much are they learning? Are they learning enough? These questions, which require considerable unpacking, stand in contrast to more precise questions that can be answered with greater confidence: How reliable are the scores on this test? Is a test score high enough to infer mastery of the content domain? Is the score predictive of success on other related tests? These latter questions, while not wrong per se, represent the sorts of things that psychometricians think that people should care about, rather than the sorts of questions they actually care about. There is good reason to debate the appropriate role for psychometrics to play in contexts where there is a desire to make inferences about student growth. In this talk, I use recent and ongoing research on learning progressions in mathematics and science to illustrate how measurement provides a valuable frame of reference in our attempts to answer questions about student learning. Yet I also emphasize the danger of overselling measurement as an outcome when theories of learning are nascent and scoreable items are in short supply.