go-v.orgdoi

Beyond Teaching Quality: A Multilevel Analysis of Bias, Context, And Measurement in Student Evaluations of Teaching

Brittny A. James · container not stated · 2026

  • DOI 10.57709/387
  • Publisher OpenAlex
  • Type dissertation
  • Issued 2026

Not cached yet. This record was fetched live for this request and is stored after the lookup. Reopening the page should serve the cached copy.

· OpenAlex · dissertation · 2026

Abstract

This study examines Student Evaluations of Teaching (SET) as organizational performance metrics by evaluating the psychometric properties of a 16-item SET instrument and variation in evaluation outcomes across instructor and course characteristics. Using course section–level data (N = 88,749) from a large public research university spanning Fall 2020 through Summer 2024, the study is organized into two papers. Paper 1 applies Principal Component Analysis and reliability analysis to assess the instrument's structure and internal consistency.

Results support a two-factor solution reflecting Instructional Quality and Course Delivery and Student Engagement and Learning Effort, with strong reliability (α = .977). Paper 2 examines variation in SET scores using t-tests, ANOVA, and hierarchical linear mixed-effects models with instructor-level random intercepts (ICC = .344). Race-based disparities were robust across all model specifications, persisting after full contextual adjustment. Sex effects were small and attenuated to non-significance once departmental context was controlled. A significant Sex × Course Level interaction indicated that a marginal female advantage at the undergraduate level attenuated at the graduate level.

A significant Sex × Race interaction revealed that underrepresented female instructors face a compounded disadvantage beyond independent sex and race effects. These findings have direct implications for the equitable use of SET data in faculty evaluation systems.

Open the resolved locator · 10.57709/387

Metadata licence: OpenAlex metadata (CC0)

Registry
OpenAlex DOI index + API
Fetched
2026-11-20 17:33 UTC
Cache state
not cached yet (bulk-index+openalex)
Locator
https://hdl.handle.net/20.500.14694/16581
Metadata licence
OpenAlex metadata (CC0)
Scope
Metadata as returned by the resolver; abstracts are never generated here.