In this issue of the Journal of Pediatric Urology, Stern et al. present on an experienced
laparoscopic surgeon's experience in adopting the robotic-assisted platform for pediatric
Stern N, Li Y, Wang P, Dave S. A cumulative sum (CUSUM) analysis studying operative
times and complications for a surgeon transitioning from laparoscopic to robot-assisted
pediatric pyeloplasty: defining proficiency and competency. J Pediatr Urol 2022;18:822–9.
]. This manuscript can be read at multiple levels. First, the authors nicely demonstrate
both overall and task-specific learning curves for surgeons adapting the robotic technique.
While the robotic platform has already saturated many marketplaces, this is relevant
work for those areas where the surgical robot still represents an emerging opportunity.
A second, and perhaps more generalizable, lens through which to view this work is
as a road-map to cumulative sum analysis (CUSUM). CUSUM is a version of statistical
process (or quality) control that may a novel concept to many in pediatric urology.
In essence, this data analytic technique, seeks to monitor iterative trends over time
and compare these outputs to a chosen standard. Specifically, CUSUM evaluates the
cumulative sum of differences between the expected standard, herein described as the
mean operative time to that point [
Monitoring surgical quality: the cumulative sum (CUSUM) approach.
]. This manuscript provides an opportunity to highlight both the power and limitations
of the CUSUM analytic technique. First, one notes that CUSUM is not immediately intuitive,
and I would encourage the reader to take the time in reviewing the authors' elegant
description of the methodology in order to fully appreciate the manuscript. Owing
to the underlying mathematics, CUSUM is markedly useful to measure small changes over
time. To this end, CUSUM is an attractive tool to monitor operative learning curves
in general, as one expects iterative and often smaller progressive improvements from
case to case. The authors retroactively were able to define “cut-points” in proficiency,
which could theoretically be used in association with this methodology to define surgical
proficiencies in training progression and/or credentialing. Visually, one can see
in Figure 1 how the learning curve decreases steadily, though not sequentially, after
the first case (blue line). However, the CUSUM curve (black dots) demonstrates a much
more visually dramatic series of breakpoints. One point of caution with the mathematics
and reference value (operative time mean) is the tendency to convert outcomes that
are progressing in a linear fashion into a more distinctly quadratic term, especially
if the early experience tends to be greater than the overall sample mean, as is typical
in operative times early in the learning curve [
- Woodall W.H.
- Rakovich G.
- Steiner S.H.
An overview and critique of the use of cumulative sum methods with surgical learning
]. Put another way, any individual surgeon is likely to demonstrate a similar progression
on a CUSUM plot, from learning to proficiency to mastery, based on use of their own
means. A true comparison of mastery, however, would be benchmarking against a well-accepted
standard. This comment is not to critique the surgeon-experience in the article, as
they have progressed quite rapidly to commendable times, but rather to caution against
over-interpretation of CUSUM in assessing learning curves more broadly. Alternatively,
creating curves that benchmark against published complications, as has been reported
in hypospadias [
- Parikh A.M.
- Park A.M.
- Sumfest J.
Cumulative summation (CUSUM) charts in the monitoring of hypospadias outcomes: a tool
for quality improvement initiative.
] and replicated in this manuscript, may provide a more broadly comparable use for