Predicting Performance of First-Year Residents: Correlations Between Structured Interview, Licensure Exam, and Competency Scores in a Multi-Institutional Study


    loading  Checking for direct PDF access through Ovid

Abstract

PurposeTo determine whether scores on structured interview (SI) questions designed to measure noncognitive competencies in physicians (1) predict subsequent first-year resident performance on Accreditation Council for Graduate Medical Education (ACGME) milestones and (2) add incremental validity over United States Medical Licensing Examination (USMLE) Step 1 and Step 2 Clinical Knowledge scores in predicting performance.MethodThe authors developed 18 behavioral description questions to measure key noncognitive competencies (e.g., teamwork). In 2013–2015, 14 programs (13 residency, 1 fellowship) from 6 institutions used subsets of these questions in their selection processes. The authors conducted analyses to determine the validity of SI and USMLE scores in predicting first-year resident milestone performance in the ACGME’s core competency domains and overall.ResultsSI scores predicted midyear and year-end overall performance (r = 0.18 and 0.19, respectively, P < .05) and year-end performance on patient care, interpersonal and communication skills, and professionalism competencies (r = 0.23, r = 0.22, and r = 0.20, respectively, P < .05). SI scores contributed incremental validity over USMLE scores in predicting year-end performance on patient care (ΔR = 0.05), interpersonal and communication skills (ΔR = 0.09), and professionalism (ΔR = 0.09; all P < .05). USMLE scores contributed incremental validity over SI scores in predicting year-end performance overall and on patient care and medical knowledge.ConclusionsSI scores predict first-year resident year-end performance in the interpersonal and communication skills, patient care, and professionalism competency domains. Future research should investigate whether SIs predict a range of clinically relevant outcomes.

    loading  Loading Related Articles