Date of Award
Master of Science (MS)
School of Computing
Dr. Jacob Sorber, Committee Chair
Dr. Kelly Caine
Dr. Bart Knijnenburg
Novel technologies and teaching methods are being integrated into post-secondary classrooms at unprecedented rates. Educators must be certain implemented changes provide equitable alternatives to traditional pedagogical practices, before migrating to modern paradigms. However, changes in classroom practices each introduce unique psychological inﬂuences into the classroom, which may inﬂuence traditional metrics of pedagogical success. Evaluation methods for assessing classroom changes should evolve with pedagogy, to accurately measure its effects. In this paper, we share our experience exploring the equity of replacing traditional paper-and-pencil testing with digital examinations, to demonstrate the complexity of modern educational assessment. We observed a variety of variables, many of which often go unmeasured in pedagogical evaluations, inﬂuenced our metrics of success, and altered our initial conclusions. Further, our analysis suggests classroom changes may inﬂuence demographic groups differently, and these effects may be hidden, if success is only measured within the aggregate student body. Evaluations that do not explicitly consider and identify these effects may not be able to observe them. Our experience indicates that evaluations that do not account for demographics are incomplete, at best. More importantly, our results suggest these evaluations may draw, and support, invalid conclusions. By describing the difﬁculties we encountered in evaluating the equity of digital examinations, we invite educators and education researchers to move beyond simplistic evaluations, consider the underlying factors inﬂuencing traditional metrics of success, and adopt a more nuanced view of pedagogical evaluation practices.
Storer, Kevin M., "Nuanced Views of Pedagogical Evaluation" (2017). All Theses. 2758.