Establishing Extrapolation Evidence for the MyTIPreport Feedback Application Using a Contemporary Validation Approach

    loading  Checking for direct PDF access through Ovid

Abstract

BACKGROUND:

OB/GYN residencies use myTIPreport to improve resident feedback. Validation is viewed as a process with construct validity established from different levels of evidence. In Kane's validity framework, extrapolation focuses on links between assessment outcomes and other measures of performance.

METHODS:

Single-institution, prospective cohort study of OB/GYN residents (n=24). Using the ACGME case log system and myTIPreport application, associations of PGY level, number of previous hysteroscopies (HSC), total vaginal hysterectomies (TVH), and total abdominal hysterectomies (TAH) with myTIPreport ratings of each procedure when performed in the operating room were assessed using Spearman's test correlation. A pilot study of inexpensive, low technology simulation models was initiated to assess extrapolation evidence for myTIPreport in the simulation setting (n=8).

RESULTS:

Associations (P<.01) existed between PGY level and myTIPreport scores in the OR (r=0.612 for HSC, r=0.763 for TAH, r=0.864 for TVH). Associations (P<.01) existed between number of procedures performed and myTIPreport scores in the OR (r=0.587 for HSC, r=0.681 for TAH, r=0.869 for TVH). An association (r=0.852, P=.031) existed between number of TVHs performed in the OR and myTIPreport scores during simulation, but not for TAH (P=.199) nor HSC (P=.154) due to the small sample size.

DISCUSSION:

The association between myTIPreport TVH performance with PGY level and number of TVHs was the strongest, supporting extrapolation evidence. The associations were weaker for TAH and HSC in both the OR and simulation settings secondary to higher ratings for lower PGY levels and fewer cases.

Related Topics

    loading  Loading Related Articles