Reliability of grading of vesicoureteral reflux and other findings on voiding cystourethrography

Published:September 06, 2016DOI:



      Voiding cystourethrography (VCUG) is the modality of choice to diagnose vesicoureteral reflux (VUR). Although grading of VUR is essential for prognosis and clinical decision-making, the inter-observer reliability for grading has been shown to vary substantially. The Randomized Intervention for Children with VesicoUreteral Reflux (RIVUR) trial provides a large cohort of children with VUR to better understand the reliability of VCUG findings.


      To determine the inter-observer consistency of the grade of VUR and other VCUG findings in a large cohort of children with VUR.

      Study design

      The RIVUR trial is a randomized controlled trial of antimicrobial prophylaxis in children with VUR diagnosed after UTI. Each enrollment VCUG was read by a local clinical (i.e. non-reference) radiologist, and independently by two blinded RIVUR reference radiologists. Reference radiologists' disagreements were adjudicated for trial purposes. The grade of VUR and other VCUG findings were extracted from the local clinical radiologist's report. The unit of analysis included individual ureters and individual participants. We compared the three interpretations for grading of VUR and other VCUG findings to determine the inter-observer reliability.


      Six-hundred and two non-reference radiology reports from 90 institutions were reviewed and yielded the grade of VUR for 560 left and 524 right ureters. All three radiologists agreed on VUR grade in only 59% of ureters; two of three agreed on 39% of ureters; and all three disagreed on 2% of ureters (Table). Agreement was better (≥92%) for other VCUG findings (e.g. bladder shape “normal”). The non-reference radiologists' grade of VUR differed from the reference radiologists' adjudicated grade by exactly one grade level in 19% of ureters, and by two or more grade levels in 2.2% of ureters. When the participant was the unit of analysis, all three radiologists agreed on the grade of VUR in both ureters in just 43% of cases.


      Our study shows considerable and clinically relevant variability in grading VUR by VCUG. This variability was consistent when comparing non-reference to the adjudicated reference radiologists' assessment and the reference radiologists to each other. This study was limited to children with a history of UTI and grade I–IV VUR and may not be generalizable to all children who have a VCUG.


      TableStudy summary
      No. of VCUG reports analyzed 602
      Gender of participants
       Male 49
       Female 553
      Age in months at time of VCUG (median) [IQR] 11 [5,30]
      No. of ureters analyzed 1081
      Reflux grade agreement
       Between non-reference and each reference radiologist (three-way)
      All three agree 638/1081 (59%)
      Two agree, one disagree 417/1081 (39%)
      All three disagree 27 (2%)
       Between non-reference and adjudicated reference radiologists' score (two-way)
      Agree 805 (75%)
      Disagree 275 (25%)
      Kappa (95% CI) 0.66 (0.62–0.69)


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Pediatric Urology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Lebowitz R.L.
        • Olbing H.
        • Parkkulainen K.V.
        • Smellie J.M.
        • Tamminen-Mobius T.E.
        International system of radiographic grading of vesicoureteric reflux. International Reflux Study in Children.
        Pediatr Radiol. 1985; 15: 105-109
        • Garin E.H.
        • Olavarria F.
        • Garcia Nieto V.
        • Valenciano B.
        • Campos A.
        • Young L.
        Clinical significance of primary vesicoureteral reflux and urinary antibiotic prophylaxis after acute pyelonephritis: a multicenter, randomized, controlled study.
        Pediatrics. 2006; 117: 626-632
        • Ardissino G.
        • Avolio L.
        • Dacco V.
        • Testa S.
        • Marra G.
        • Viganò S.
        • et al.
        Long-term outcome of vesicoureteral reflux associated chronic renal failure in children. Data from the ItalKid Project.
        J Urol. 2004; 172: 305-310
        • Roussey-Kesler G.
        • Gadjos V.
        • Idres N.
        • Horen B.
        • Ichay L.
        • Leclair M.D.
        • et al.
        Antibiotic prophylaxis for the prevention of recurrent urinary tract infection in children with low grade vesicoureteral reflux: results from a prospective randomized study.
        J Urol. 2008; 179 (discussion 679): 674-679
        • Peters C.A.
        • Skoog S.J.
        • Arant Jr., B.S.
        • Copp H.L.
        • Elder J.S.
        • Hudson R.G.
        • et al.
        Summary of the AUA guideline on management of primary vesicoureteral reflux in children.
        J Urol. 2010; 184: 1134-1144
        • Kronemer K.A.
        • Don S.
        • Luker G.D.
        • Hildebolt C.
        Soft-copy versus hard-copy interpretation of voiding cystourethrography in neonates, infants, and children.
        AJR Am J Roentgenol. 1999; 172: 791-793
        • Greenfield S.P.
        • Carpenter M.A.
        • Chesney R.W.
        • Zerin J.M.
        • Chow J.
        The RIVUR voiding cystourethrogram pilot study: experience with radiologic reading concordance.
        J Urol. 2012; 188: 1608-1612
        • Metcalfe C.B.
        • Macneily A.E.
        • Afshar K.
        Reliability assessment of international grading system for vesicoureteral reflux.
        J Urol. 2012; 188: 1490-1492
        • Carpenter M.A.
        • Hoberman A.
        • Mattoo T.K.
        • Mathews R.
        • Keren R.
        • Chesney R.W.
        • et al.
        The RIVUR trial: profile and baseline clinical associations of children with vesicoureteral reflux.
        Pediatrics. 2013; 132: e34-e45
        • Cohen J.
        A coefficient of agreement for nominal scales.
        Ed Psych Msmt. 1960; 20: 37-46
        • Craig J.C.
        • Irwig L.M.
        • Christie J.
        • Lam A.
        • Onikul E.
        • Knight J.F.
        • et al.
        Variation in the diagnosis of vesicoureteric reflux using micturating cystourethrography.
        Pediatr Nephrol. 1997; 11: 455-459
        • O'Neil B.B.
        • Cartwright P.C.
        • Maves C.
        • Hoeg K.
        • Presson A.P.
        • Wallis M.C.
        Reliability of voiding cystourethrogram for the grading of vesicoureteral reflux.
        J Pediatr Urol. 2014; 10: 107-111
        • Feinstein A.R.
        • Cicchetti D.V.
        High agreement but low kappa: I. The problems of two paradoxes.
        J Clin Epidemiol. 1990; 43: 543-549
        • Estrada Jr., C.R.
        • Passerotti C.C.
        • Graham D.A.
        • Peters C.A.
        • Bauer S.B.
        • Diamond D.A.
        • et al.
        Nomograms for predicting annual resolution rate of primary vesicoureteral reflux: results from 2,462 children.
        J Urol. 2009; 182: 1535-1541
        • Dias C.S.
        • Silva J.M.
        • Diniz J.S.
        • Lima E.M.
        • Marciano R.C.
        • Lana L.G.
        • et al.
        Risk factors for recurrent urinary tract infections in a cohort of patients with primary vesicoureteral reflux.
        Pediatr Infect Dis J. 2010; 29: 139-144
        • Peters C.
        • Rushton H.G.
        Vesicoureteral reflux associated renal damage: congenital reflux nephropathy and acquired renal scarring.
        J Urol. 2010; 184: 265-273