Advertisement

The opportunities and cautions of cumulative sum analysis in assessing learning curves in pediatric urology

Published:September 26, 2022DOI:https://doi.org/10.1016/j.jpurol.2022.08.027
      In this issue of the Journal of Pediatric Urology, Stern et al. present on an experienced laparoscopic surgeon's experience in adopting the robotic-assisted platform for pediatric pyeloplasty [

      Stern N, Li Y, Wang P, Dave S. A cumulative sum (CUSUM) analysis studying operative times and complications for a surgeon transitioning from laparoscopic to robot-assisted pediatric pyeloplasty: defining proficiency and competency. J Pediatr Urol 2022;18:822–9.

      ]. This manuscript can be read at multiple levels. First, the authors nicely demonstrate both overall and task-specific learning curves for surgeons adapting the robotic technique. While the robotic platform has already saturated many marketplaces, this is relevant work for those areas where the surgical robot still represents an emerging opportunity. A second, and perhaps more generalizable, lens through which to view this work is as a road-map to cumulative sum analysis (CUSUM). CUSUM is a version of statistical process (or quality) control that may a novel concept to many in pediatric urology. In essence, this data analytic technique, seeks to monitor iterative trends over time and compare these outputs to a chosen standard. Specifically, CUSUM evaluates the cumulative sum of differences between the expected standard, herein described as the mean operative time to that point [
      • Novoa N.M.
      • Varela G.
      Monitoring surgical quality: the cumulative sum (CUSUM) approach.
      ]. This manuscript provides an opportunity to highlight both the power and limitations of the CUSUM analytic technique. First, one notes that CUSUM is not immediately intuitive, and I would encourage the reader to take the time in reviewing the authors' elegant description of the methodology in order to fully appreciate the manuscript. Owing to the underlying mathematics, CUSUM is markedly useful to measure small changes over time. To this end, CUSUM is an attractive tool to monitor operative learning curves in general, as one expects iterative and often smaller progressive improvements from case to case. The authors retroactively were able to define “cut-points” in proficiency, which could theoretically be used in association with this methodology to define surgical proficiencies in training progression and/or credentialing. Visually, one can see in Figure 1 how the learning curve decreases steadily, though not sequentially, after the first case (blue line). However, the CUSUM curve (black dots) demonstrates a much more visually dramatic series of breakpoints. One point of caution with the mathematics and reference value (operative time mean) is the tendency to convert outcomes that are progressing in a linear fashion into a more distinctly quadratic term, especially if the early experience tends to be greater than the overall sample mean, as is typical in operative times early in the learning curve [
      • Woodall W.H.
      • Rakovich G.
      • Steiner S.H.
      An overview and critique of the use of cumulative sum methods with surgical learning curve data.
      ]. Put another way, any individual surgeon is likely to demonstrate a similar progression on a CUSUM plot, from learning to proficiency to mastery, based on use of their own means. A true comparison of mastery, however, would be benchmarking against a well-accepted standard. This comment is not to critique the surgeon-experience in the article, as they have progressed quite rapidly to commendable times, but rather to caution against over-interpretation of CUSUM in assessing learning curves more broadly. Alternatively, creating curves that benchmark against published complications, as has been reported in hypospadias [
      • Parikh A.M.
      • Park A.M.
      • Sumfest J.
      Cumulative summation (CUSUM) charts in the monitoring of hypospadias outcomes: a tool for quality improvement initiative.
      ] and replicated in this manuscript, may provide a more broadly comparable use for CUSUM plots.
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Pediatric Urology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

      1. Stern N, Li Y, Wang P, Dave S. A cumulative sum (CUSUM) analysis studying operative times and complications for a surgeon transitioning from laparoscopic to robot-assisted pediatric pyeloplasty: defining proficiency and competency. J Pediatr Urol 2022;18:822–9.

        • Novoa N.M.
        • Varela G.
        Monitoring surgical quality: the cumulative sum (CUSUM) approach.
        Mediastinum. 2020; 4: 4
        • Woodall W.H.
        • Rakovich G.
        • Steiner S.H.
        An overview and critique of the use of cumulative sum methods with surgical learning curve data.
        Stat Med. 2021; 40: 1400-1413
        • Parikh A.M.
        • Park A.M.
        • Sumfest J.
        Cumulative summation (CUSUM) charts in the monitoring of hypospadias outcomes: a tool for quality improvement initiative.
        J Pediatr Urol. 2014; 10: 306-311

      Linked Article