Developing and Assessing the Feasibility of Implementing a Surgical Objective Structured Clinical Skills Examination (S-OSCE)

Objective: To1 describe the development and evaluate the feasibility of a surgical objective structured clinical examination (OSCE) for the purpose of competency assessment based on the Royal College of Canada's CanMEDS framework. Design: A unique surgical OSCE was developed to evaluate the clinical and surgical management of common orthopaedic problems using simulated patients and cadaveric specimens. Cases were graded by degree of difficulty (less complex, complex, more complex) Developing an assessment tool with significant resource utilization and good correlation with traditional methods is challenging. The feasibility of an OSCE that evaluates independent clinical and surgical decision making was evaluated. In addition, as part of establishing construct validity, correlation of OSCE scores with previously validated O-scores was performed. Setting: A tertiary level academic teaching hospital. Participants: Thirty-four Postgraduate year 3-5 trainees of a 5-year Canadian orthopedic residency program creating 96 operative case performances available for final review. Results: The development of the OSCE cases involved a multistep process with attending surgeons, residents and a surgical education consultant. There were 4 different OSCE days, over a 3 year period (2016-2018) encompassing a variety of less complex and more complex procedures. Performance on the OSCE correlated strongly with the (O-SCORE, 0.89) and a linear regression analysis correlated moderately with year of training (r² = 0.5737). The feasibility analysis demonstrated good financial practicality with solid programmatic integration. Conclusions: The unique surgical OSCE scores correlate strongly with an established entrustability scale. Administering this OSCE to evaluate preoperative and intraoperative decision making to complement other forms of assessment is feasible. The financial burden to training programs is modest in comparison to the insight gained by both residents and faculty.