Research Article| Volume 18, ISSUE 6, P822-829, December 2022

A cumulative sum (CUSUM) analysis studying operative times and complications for a surgeon transitioning from laparoscopic to robot-assisted pediatric pyeloplasty: Defining proficiency and competency



      The transition from laparoscopic to robot-assisted procedures leads to potential increase in operative times and health care costs. Cumulative sum (CUSUM) analysis can objectively study the learning curve to detect significant changes in operative timing and monitor complication rates.


      The objective of this study is to investigate the total and step-specific times for pediatric robot-assisted pyeloplasty (RAP) to investigate the learning curve of a single surgeon transitioning from laparoscopic to RAP.

      Study design

      This prospective cohort study included 50 consecutive RAP procedures performed since the inception of our robotic program from June 2013 to January 2019. The CUSUM of RAP total operative time (OT) was calculated to determine the breakpoints between learning phases using piecewise linear regression. Cumulative-observed-minus-expected failure chart with 80% and 95% reassurance boundary lines was constructed using 5% acceptable and 10% unacceptable complication rates. Step-specific operative times were prospectively recorded by an independent observer for port placement, dissection and hitch stitch placement, pelvis dismemberment and spatulation, suturing and port removal.


      Piecewise linear regression for OT identified breakpoints at case 13 and 29 suggesting transition at these points between Learning to Proficiency, and Proficiency to Competency. The overall mean OT was 142.2 ± 46.0 min. There was a significant difference in the mean OT between Learning (203.9 ± 35.3 min, the initial 13 cases), Proficiency (159.2 ± 18.6 min, the middle 16 cases), and Competency (126.6 ± 19.7 min, the last 21 cases) phases (p < 0.001). The complication rate for RAP stabilized around the acceptable level of 5% up to case 41 before finalizing at 8% overall. The step-specific analysis suggested that suturing entered the Competency phase at case 27, with a 50% decrease in suturing time from Learning to Proficiency and Competency.


      Our study suggests that by case 30 a surgeon transitioning to RAP can achieve a significant decrease in OT. Complication rates remained within acceptable limits throughout, indicating that RAP can be safely adopted, even in low volume RAP centres. Suturing competency seems to be a significant advantage of the robotic platform as suggested by early significant decrease in suturing times noted between the Learning and Proficiency phases.


      Summary Figure 1
      Graphical AbstractPiecewise linear regression of CUSUM (black dots) of robot assisted pyeloplasty (RAP) operative times (blue) with breakpoints at case 12, 95% CI [11.6, 13.2] and case 29, 95% CI [27.5, 29.8], and an R2 value of 0.9875.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Pediatric Urology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Doerffel K.
        • Herfurth G.
        • Liebich V.
        • Wendlandt E.
        The shape of CUSUM? an indicator for tendencies in a time series.
        Fresenius’ J Anal Chem. 1991; 341: 519-523
        • Bissell A.F.
        Cusum techniques for quality control.
        Applied Statistics. 1969; 18: 1
        • Grigg O.A.
        • Farewell V.T.
        • Spiegelhalter D.J.
        Use of risk-adjusted CUSUM and RSPRT charts for monitoring in medical contexts.
        Stat Methods Med Res. Apr. 2003; 12: 147-170
        • Rogers C.A.
        • Reeves B.C.
        • Caputo M.
        • Ganesh J.S.
        • Bonser R.S.
        • Angelini G.D.
        Control chart methods for monitoring cardiac surgical performance and their interpretation.
        J Thorac Cardiovasc Surg. Dec. 2004; 128: 811-819
        • Williams A.K.
        • Chalasani V.
        • Martínez C.H.
        • Osbourne E.
        • Stitt L.
        • Izawa J.I.
        • et al.
        Cumulative summation graphs are a useful tool for monitoring positive surgical margin rates in robot-assisted radical prostatectomy.
        BJU Int. May 2011; 107: 1648-1652
        • Mackenzie H.
        • Miskovic D.
        • Ni M.
        • Parvaiz A.
        • Acheson A.G.
        • Jenkins J.T.
        • et al.
        Clinical and educational proficiency gain of supervised laparoscopic colorectal surgical trainees.
        Surg Endosc. Aug. 2013; 27: 2704-2711
        • Park J.S.
        • Ahn H.K.
        • Na J.
        • Lee H.H.
        • Yoon Y.E.
        • Yoon M.G.
        • et al.
        Cumulative sum analysis of the learning curve for video-assisted minilaparotomy donor nephrectomy in healthy kidney donors.
        Medicine. 2018; 97: e0560, Apr
        • Broering D.C.
        • Berardi G.
        • el Sheikh Y.
        • Spagnoli A.
        • Troisi R.I.
        Learning curve under proctorship of pure laparoscopic living donor left lateral sectionectomy for pediatric transplantation.
        Ann Surg. Mar. 2020; 271: 542-548
        • Williams S.M.
        • Parry B.R.
        • Schlup M.M.
        Quality control: an application of the cusum.
        BMJ. May 1992; 304: 1359-1361
        • Cundy T.P.
        • Gattas N.E.
        • White A.D.
        • Najmaldin A.S.
        Learning curve evaluation using cumulative summation analysis—a clinical example of pediatric robot-assisted laparoscopic pyeloplasty.
        J Pediatr Surg. Aug. 2015; 50: 1368-1373
        • Bütter A.
        • Merritt N.
        • Dave S.
        Establishing a pediatric robotic surgery program in Canada.
        Journal of Robotic Surgery. Jun. 2017; 11: 207-210
        • Stern N.
        • Wang P.
        • Dave S.
        Instituting robotic pediatric urologic surgery in the Canadian healthcare system: evaluating the feasibility and outcomes of robot-assisted pyeloplasty and ureteric reimplantation.
        Canadian Urological Association Journal. Sep. 2020; 15
        • Dindo D.
        • Demartines N.
        • Clavien P.-A.
        Classification of surgical complications.
        Ann Surg. Aug. 2004; 240: 205-213
        • Bokhari M.B.
        • Patel C.B.
        • Ramos-Valadez D.I.
        • Ragupathi M.
        • Haas E.M.
        Learning curve for robotic-assisted laparoscopic colorectal surgery.
        Surg Endosc. Mar. 2011; 25: 855-860
        • Muggeo V.M.R.
        Estimating regression models with unknown break-points.
        Stat Med. 2003; 22: 3055-3071, Oct
        • Kurtz M.P.
        • Leow J.J.
        • Varda B.K.
        • Logvinenko T.
        • Yu R.N.
        • Nelson C.P.
        • et al.
        Robotic versus open pediatric ureteral reimplantation: costs and complications from a nationwide sample.
        J Pediatr Urol. 2016; 12: 408.e1-408.e6
        • Minnillo B.J.
        • Cruz J.A.S.
        • Sayao R.H.
        • Passerotti C.C.
        • Houck C.S.
        • Meier P.M.
        • et al.
        Long-term experience and outcomes of robotic assisted laparoscopic pyeloplasty in children and young adults.
        J Urol. 2011; 185: 1455-1460, Apr
        • Riachy E.
        • Cost N.G.
        • Defoor W.R.
        • Reddy P.P.
        • Minevich E.A.
        • Noh P.H.
        Pediatric standard and robot-assisted laparoscopic pyeloplasty: a comparative single institution study.
        J Urol. Jan. 2013; 189: 283-287
        • Chan G.
        • Pautler S.E.
        Use of cumulative summation (CUSUM) as a tool for early feedback and monitoring in robot-assisted radical prostatectomy outcomes and performance.
        Canadian Urological Association Journal. Jul. 2018; 13
        • Ohwaki K.
        • Endo F.
        • Shimbo M.
        • Hattori K.
        The use of cumulative sum analysis to derive institutional and surgeon-specific learning curves for robot-assisted radical prostatectomy.
        J Endourol. Sep. 2020; 34: 969-973
        • Langley S.
        • Hill A.
        • Hall B.
        Royal Australasian College of Surgeons Surgical Audit Guide.
        5th ed. 2021
        • Varda B.K.
        • Wang Y.
        • Chung B.I.
        • Lee R.S.
        • Kurtz M.P.
        • Nelson C.P.
        • et al.
        Has the robot caught up? National trends in utilization, perioperative outcomes, and cost for open, laparoscopic, and robotic pediatric pyeloplasty in the United States from 2003 to 2015.
        J Pediatr Urol. Aug. 2018; 14 (336.e1-336336.e8)
        • McCulloch P.
        • Altman D.G.
        • Campbell W.B.
        • Flum D.R.
        • Glasziou P.
        • Marshall J.C.
        • et al.
        No surgical innovation without evaluation: the IDEAL recommendations.
        Lancet. Sep. 2009; 374: 1105-1112
        • Abboudi H.
        • Khan M.S.
        • Guru K.A.
        • Froghi S.
        • Win G.
        • Van Poppel H.
        • et al.
        Learning curves for urological procedures: a systematic review.
        BJU Int. Oct. 2014; 114: 617-629
        • Kassite I.
        • Braik K.
        • Villemagne T.
        • Lardy H.
        • Binet A.
        The learning curve of robot-assisted laparoscopic pyeloplasty in children: a multi-outcome approach.
        J Pediatr Urol. Dec. 2018; 14: 570.e1-570.e10

      Linked Article

      • The opportunities and cautions of cumulative sum analysis in assessing learning curves in pediatric urology
        Journal of Pediatric UrologyVol. 18Issue 6
        • Preview
          In this issue of the Journal of Pediatric Urology, Stern et al. present on an experienced laparoscopic surgeon's experience in adopting the robotic-assisted platform for pediatric pyeloplasty [1]. This manuscript can be read at multiple levels. First, the authors nicely demonstrate both overall and task-specific learning curves for surgeons adapting the robotic technique. While the robotic platform has already saturated many marketplaces, this is relevant work for those areas where the surgical robot still represents an emerging opportunity.
        • Full-Text
        • PDF