Beyond Teaching Quality: A Multilevel Analysis of Bias, Context, And Measurement in Student Evaluations of Teaching
Brittny A. James · container not stated · 2026
Not cached yet. This record was fetched live for this request and is stored after the lookup. Reopening the page should serve the cached copy.
· OpenAlex · dissertation · 2026
Abstract
This study examines Student Evaluations of Teaching (SET) as organizational performance metrics by evaluating the psychometric properties of a 16-item SET instrument and variation in evaluation outcomes across instructor and course characteristics. Using course section–level data (N = 88,749) from a large public research university spanning Fall 2020 through Summer 2024, the study is organized into two papers. Paper 1 applies Principal Component Analysis and reliability analysis to assess the instrument's structure and internal consistency.
Results support a two-factor solution reflecting Instructional Quality and Course Delivery and Student Engagement and Learning Effort, with strong reliability (α = .977). Paper 2 examines variation in SET scores using t-tests, ANOVA, and hierarchical linear mixed-effects models with instructor-level random intercepts (ICC = .344). Race-based disparities were robust across all model specifications, persisting after full contextual adjustment. Sex effects were small and attenuated to non-significance once departmental context was controlled. A significant Sex × Course Level interaction indicated that a marginal female advantage at the undergraduate level attenuated at the graduate level.
A significant Sex × Race interaction revealed that underrepresented female instructors face a compounded disadvantage beyond independent sex and race effects. These findings have direct implications for the equitable use of SET data in faculty evaluation systems.
Open the resolved locator · 10.57709/387
Metadata licence: OpenAlex metadata (CC0)