Predicting Performance of First-Year Residents: Correlations Between Structured Interview, Licensure Exam, and Competency Scores in a Multi-Institutional Study

    loading  Checking for direct PDF access through Ovid


Purpose:To determine whether scores on structured interview (SI) questions designed to measure noncognitive competencies in physicians (1) predict subsequent first-year resident performance on Accreditation Council for Graduate Medical Education (ACGME) milestones and (2) add incremental validity over United States Medical Licensing Examination (USMLE) Step 1 and Step 2 Clinical Knowledge (CK) scores in predicting performance.Method:The authors developed 18 behavioral description questions to measure key noncognitive competencies (e.g., teamwork). In 2013–2015, 14 programs (13 residency, 1 fellowship) from 6 institutions used subsets of these questions in their selection processes. The authors conducted analyses to determine the validity of SIs and USMLE scores in predicting first-year resident milestone performance in the ACGME’s core competency domains and overall.Results:SI scores predicted mid-year and year-end overall performance (r = .18 and .19, respectively, P < .05) and year-end performance on patient care, interpersonal and communication skills, and professionalism competencies (r = .23, r = .22, and r = .20, respectively, p < .05). SI scores contributed incremental validity over USMLE scores in predicting year-end performance on patient care (ΔR= .05), interpersonal and communication skills (ΔR= .09), and professionalism (ΔR= .09; all P < .05). USMLE scores contributed incremental validity over SI scores in predicting year-end performance overall and on patient care and medical knowledge.Conclusions:SI scores predict first-year resident year-end performance in the interpersonal and communication skills, patient care, and professionalism competency domains. Future research should investigate whether SIs predict a range of clinically relevant outcomes.

    loading  Loading Related Articles