Date of Award


Document Type


Degree Name

Master of Science (MS)


School of Computing

Committee Member

Dr. Jacob Sorber, Committee Chair

Committee Member

Dr. Kelly Caine

Committee Member

Dr. Bart Knijnenburg


Novel technologies and teaching methods are being integrated into post-secondary classrooms at unprecedented rates. Educators must be certain implemented changes provide equitable alternatives to traditional pedagogical practices, before migrating to modern paradigms. However, changes in classroom practices each introduce unique psychological influences into the classroom, which may influence traditional metrics of pedagogical success. Evaluation methods for assessing classroom changes should evolve with pedagogy, to accurately measure its effects. In this paper, we share our experience exploring the equity of replacing traditional paper-and-pencil testing with digital examinations, to demonstrate the complexity of modern educational assessment. We observed a variety of variables, many of which often go unmeasured in pedagogical evaluations, influenced our metrics of success, and altered our initial conclusions. Further, our analysis suggests classroom changes may influence demographic groups differently, and these effects may be hidden, if success is only measured within the aggregate student body. Evaluations that do not explicitly consider and identify these effects may not be able to observe them. Our experience indicates that evaluations that do not account for demographics are incomplete, at best. More importantly, our results suggest these evaluations may draw, and support, invalid conclusions. By describing the difficulties we encountered in evaluating the equity of digital examinations, we invite educators and education researchers to move beyond simplistic evaluations, consider the underlying factors influencing traditional metrics of success, and adopt a more nuanced view of pedagogical evaluation practices.



To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.