PTH-145 Validation of direct observation of procedural skills (DOPS) assessments for paediatric gastroscopy

    loading  Checking for direct PDF access through Ovid

Abstract

Introduction

Direct observation of procedural skills (DOPS) are tools designed by the Joint Advisory Group (JAG) to assess competence in endoscopy. These were expanded in July 2016 (new DOPS) to include those specific to paediatric gastroscopy (OGD). However, paediatric OGD DOPS assessments have not been validated.

Methods

We performed a prospective UK-wide analysis of formative paediatric OGD DOPS submitted to the JETS e-Portfolio over one-year (August 2016–2017). Scores were averaged across procedural domains (pre-procedural, procedural, post-procedural and endoscopic non-technical skills – ENTS), and according to skillset (cognitive or technical), and compared with the overall performance score using Spearman’s correlation. Overall scores were also compared by trainee grade.

Results

157 DOPS assessments were completed by 20 unique trainers for 17 trainees. Overall performance score comprised: 1: Maximal supervision (4.5%), 2: Significant supervision (14.0%), 3: Minimal supervision (24.8%) and 4: Competent (56.7%). By domain, overall competence correlated most with mean scores for the ‘Insertion and Withdrawal’ domain (rho: 0.884, p<0.001), Management (rho 0.834, p<0.001), Visualisation (rho 0.819, p<0.001), ENTS (0.773, p<0.001), ‘Post-procedural’ (rho 0.611, p<0.001) and pre-procedural (rho 0.575, p<0.001). By skillset, overall score correlated most with performance in ‘Technical’ (rho 0.860, p<0.001), followed by ENTS and ‘Cognitive’ domains (rho 0.742, p<0.001) domains compared to ENTS. In terms of DOPS items, overall competence score correlated most with ‘Management of Complications’ (rho 0.852, p<0.001) and ‘Angulation and Tip Control’ (rho 0.834, p<0.001), and least with ‘Confirms Consent’ (rho 0.396, p<0.001) and ‘Equipment Check’ (rho 0.528, p<0.001). Overall scores correlated with seniority of training (p<0.001) [figure 1], demonstrating construct validity.

Conclusions

Competencies in paediatric OGD, as assessed within DOPS, vary in their correlation with overall competence. Strong correlations were reported for the ‘Insertion and Withdrawal’ domain, ‘Management of Complications’ items, and ‘’Technical’ skillsets. As assessors are completing the new DOPS in a consistent manner, with scores improving by seniority, this provides validity evidence for the new paediatric OGD DOPS.

Related Topics

    loading  Loading Related Articles