AbstractProblem Statement and Background.
Examinees can make three types of errors on the short-menu questions in the Clinical Reasoning Skills component of the Medical Council of Canada’s Qualifying Examination Part I: (1) failing to select any correct responses, (2) selecting too many responses, or (3) selecting a response that is inappropriate or harmful to the patient. This study compared the information provided by equal and differential weighting of these errors.Method.
The item response theory nominal model was applied to fit examinees’ response patterns on the 1998 test.Results.
Differential error weighting resulted in improved model fit and increased test information for examinees in the lower half of the achievement continuum.Conclusion.
Differential error weighting appears promising. The pass score is near the lower end of the achievement continuum; therefore, this approach may improve the accuracy of pass-fail decisions.