Validation of a Surgical Objective Structured Clinical Examination (S-OSCE) Using Convergent, Divergent, And Trainee-Based Assessments of Fidelity

Adele Orovec, Alex Bishop, Stephanie A. Scott, Dave Wilson, C. Glen Richardson, William Oxner, R. Andrew Glennie

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)

Abstract

Objective: Describe the validation of a surgical objective structured clinical examination (S-OSCE) for the purpose of competency assessment based on the Royal College of Canada's Can-MEDS framework. Design: A surgical OSCE was developed to evaluate the management of common orthopedic surgical problems. The scores derived from this S-OSCE were compared to Ottawa Surgical Competency Operating Room Evaluation (O-SCORE), a validated entrustability assessment, to establish convergent validity. The S-OSCE scores were compared to Orthopedic In-Training Examination (OITE) scores to evaluate divergent validity. Resident evaluations of the clinical encounter with a standardized patient and the operative procedure were scored on a 10-point Likert scale for fidelity. Setting: A tertiary level academic teaching hospital. Participants: 21 postgraduate year 2 to 5 trainees of a 5-year Canadian orthopedic residency program creating 160 operative case performances for review. Results: There were 5 S-OSCE days, over a 4-year period (2016-2019) encompassing a variety of surgical procedures. Performance on the S-OSCE correlated strongly with the O-SCORE (Pearson correlation coefficient 0.88), and a linear regression analysis correlated moderately with year of training (R² = 0.5345). The Pearson correlation coefficient between the S-OSCE and OITE scores was 0.57. There was a significant increase in the average OITE score after the introduction of the surgical OSCE. Resident fidelity ratings were available from 16 residents encompassing 8 different surgical cases. The average score for the overall simulation (8.0±1.6) was significantly higher than the cadaveric surgical simulation (6.5 ± 0.8) (p < 0.001) Conclusions: The S-OSCE scores correlate strongly with an established form of assessment demonstrating convergent validity. The correlation between the S-OSCE and OITE scores was less, demonstrating divergent validity. Although residents rank the overall simulation highly, the fidelity of the cadaveric simulation may need improvement. Administration of a surgical OSCE can be used to evaluate preoperative and intraoperative decision making and complement other forms of assessment.

Original languageEnglish
Pages (from-to)1000-1008
Number of pages9
JournalJournal of Surgical Education
Volume79
Issue number4
DOIs
Publication statusPublished - Jul 1 2022

Bibliographical note

Publisher Copyright:
© 2022 Association of Program Directors in Surgery

ASJC Scopus Subject Areas

  • Surgery
  • Education

PubMed: MeSH publication types

  • Journal Article

Fingerprint

Dive into the research topics of 'Validation of a Surgical Objective Structured Clinical Examination (S-OSCE) Using Convergent, Divergent, And Trainee-Based Assessments of Fidelity'. Together they form a unique fingerprint.

Cite this

Orovec, A., Bishop, A., Scott, S. A., Wilson, D., Richardson, C. G., Oxner, W., & Glennie, R. A. (2022). Validation of a Surgical Objective Structured Clinical Examination (S-OSCE) Using Convergent, Divergent, And Trainee-Based Assessments of Fidelity. Journal of Surgical Education, 79(4), 1000-1008. https://doi.org/10.1016/j.jsurg.2022.01.014