Revalidating “Mini Surgical Theater Educational Environment Measure (mini-STEEM)”: Is it a Content Valid Instrument?

Abstract

Introduction: Operating Room (OR) based learning experience has traditionally remained challenging owing to complex medical, psychosocial, educational, and administrative factors, and so has its measurement. Recently, mini-STEEM, an abbreviated form of Surgical Theater Educational Experience Measure (STEEM), has been employed to evaluate medical students’ OR-based learning experience. This study aims to assess its content and construct validity in light of an updated extensive literature review.

Methods: Extensive literature review was done to identify and evaluate various factors affecting students' OR-based learning within the OR-setting. Expert validation of the items was done in light of constructs identified through literature review by 8 experts who had experience both in surgical teaching and medical education. The item content validity index (I-CVI) was assessed. Thematic analysis of the qualitative responses was done to understand the dynamics of validity scores. Factor analysis was performed to see the factor loadings for the items and their model fitness.

Results: Against previously identified 26 constructs, mini-STEEM items were expert-validated for clarity, relevance, and comprehension. The item content validity index was suboptimal and did not cover all of the dimensions and constructs identified through a previous literature review. Reliability was checked through Cronbach’s alpha (0.779) after piloting through a cohort of students (n=156). Qualitative analysis revealed that mini-STEEM does not cover all of the domains essential for student learning within the OR setting, and does not have adequate representative items to address all elements. Exploratory factor analysis (EFA) showed reasonable loadings; however, the model fitness through confirmatory factor analysis (CFA) was not possible considering low or no representation of most of the themes.

Conclusion: In light of evolving literature evidence, the content validity of mini-STEEM to assess the quality of medical students' OR-based learning experience remains low, despite having good reliability and construct loadings.