Date of Award

8-2017

Document Type

Thesis

Degree Name

Master of Science (MS)

Department

School of Computing

Committee Member

Dr. Jacob Sorber, Committee Chair

Committee Member

Dr. Kelly Caine

Committee Member

Dr. Bart Knijnenburg

Abstract

Novel technologies and teaching methods are being integrated into post-secondary classrooms at unprecedented rates. Educators must be certain implemented changes provide equitable alternatives to traditional pedagogical practices, before migrating to modern paradigms. However, changes in classroom practices each introduce unique psychological influences into the classroom, which may influence traditional metrics of pedagogical success. Evaluation methods for assessing classroom changes should evolve with pedagogy, to accurately measure its effects. In this paper, we share our experience exploring the equity of replacing traditional paper-and-pencil testing with digital examinations, to demonstrate the complexity of modern educational assessment. We observed a variety of variables, many of which often go unmeasured in pedagogical evaluations, influenced our metrics of success, and altered our initial conclusions. Further, our analysis suggests classroom changes may influence demographic groups differently, and these effects may be hidden, if success is only measured within the aggregate student body. Evaluations that do not explicitly consider and identify these effects may not be able to observe them. Our experience indicates that evaluations that do not account for demographics are incomplete, at best. More importantly, our results suggest these evaluations may draw, and support, invalid conclusions. By describing the difficulties we encountered in evaluating the equity of digital examinations, we invite educators and education researchers to move beyond simplistic evaluations, consider the underlying factors influencing traditional metrics of success, and adopt a more nuanced view of pedagogical evaluation practices.

Share

COinS