Beyond “Measure Twice”: New Performance Standards in Surgery
The elegantly designed study by Szas et al3 incorporates 2 validated assessment tools for trainees’ skills in 3 Canadian surgical residency programs, implementing the tools in 3 sites. By testing 36 residents in 101 operations, with novel and appropriately selected analysis the authors were able to set performance standards and examine the distribution of attainment of the participating cohort of surgical trainees. Their conclusion that it took, on average, fewer cases to meet standards in nontechnical than technical skills may surprise some surgeons and educators; trainees achieved proficiency in nontechnical skills after 35 cases and technical skills after 47 cases. Demonstrating that these concepts are related but distinct, 16 trainees were able to achieve the nontechnical standard, but not the technical standard, whereas 5 trainees reached the technical standard but not the nontechnical standard. It may be concerning that only 15 of 36 trainees achieved both standards, but heartening to learn that no trainee failed in both endeavors.
This article adds to the literature in 3 distinct and important ways: (1) By assessing technical and nontechnical skills together, it demonstrates that both skill sets are essential for a comprehensive understanding of surgical competence; (2) By quantifying standards of nontechnical skills, it enhances the objectivity and reproducibility of observational assessment tools, which are limited to an extent by their inherent subjectivity; and (3) By establishing measurable competence standards, it moves surgical education away from the “hidden curriculum,” and toward transparent milestones of expected achievement in both technical and nontechnical skills.
As the authors point out, they have achieved this in 1 operation, and in only 3 sites. By doing so, they, however, present a blueprint for formative assessment of the essential skills for surgeons in training. The same process could be conducted to establish standards for a number of index operations, and in time an open-source national database of learning curves in technical and nontechnical skills could be created.
Each surgeon has a unique learning curve for a given procedure. Only through continual measurement of technical and nontechnical skills through residency and beyond, can individual surgeons understand their progression to, and maintenance of competence. An occasional assessment is not sufficient, as evidence from generalizability studies show that 5 assessments of the same surgeon conducting the same procedure are required to minimize error from the rater, trainee, and context.4 Technologies such as simulation5 and video,6 which are gaining traction as modalities to support surgical education, could be incorporated to lessen the challenge of conducting these assessments at scale.
After reading this article, it is difficult to avoid speculating how surgical faculty would fare in similar assessments, and considering a future educational landscape in which demonstrating competence may be a prerequisite for surgical assessors. Setting performance standards that faculty and residents can both get behind will help everyone demonstrate progression toward proficiency and independent practice in a transparent and reproducible manner.