Research Article| Volume 16, ISSUE 2, P207-217, April 2020

Assessing the methodological and reporting quality of clinical systematic reviews and meta-analyses in paediatric urology: can practices on contemporary highest levels of evidence be built?

Published:December 07, 2019DOI:



      Systematic reviews and meta-analyses provide a comprehensive summary of research studies and are used to assess clinical evidence, form policy and construct guidelines. This is pertinent to childhood surgery with issues of consent and condition prevalence. The aims of this study were to evaluate the methodological and reporting quality of these reviews and to identify how these reviews might guide clinical practice amongst those conditions most commonly encountered and managed by practicing paediatric urologists.


      A systematic search of the English literature was performed to identify systematic reviews and meta-analyses focusing on clinical paediatric urology (1/1/1992-1/12/2018) to include common paediatric urological conditions managed by paediatric urology residents/fellows. To these reviews, Assessing the Methodological Quality of Systematic Reviews (AMSTAR)-2 and Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) scores were applied. Univariate linear regression and descriptive statistical methods were performed.


      From an initial literature review of 1723 articles, 227 were included in the analysis. Inter-reviewer agreement was high amongst 3 independent reviewers (κ = 0.92). Eighty-four percent of systematic reviews and meta-analyses were published since 2009 following publication of the PRISMA guidelines. The overall impact factor was 3.38 (0.83–17.58), with adherence to AMSTAR-2 criteria 48.46% and PRISMA criteria 70.1%. From a methodological perspective, 15% of reviews were of moderate quality, 65% were of low quality and 20% reviews were of critically low quality, with none found to have good quality reporting.


      Summary Figure
      Graphical AbstractMethodological quality of existing paediatric urology peer-reviewed literature based on average AMSTAR-2 ratings of manuscripts


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Pediatric Urology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Macrae D.
        Conducting clinical trials in paediatrics.
        Crit Care Med. 2009; 37: S136-S139
        • Eaton S.
        An introduction to meta-analysis in paediatric surgery.
        J Pediatr Surg. 2013; 48: 281-287
        • Shea B.J.
        • Reeves B.C.
        • Wells G.
        • Thuku M.
        • Hamel C.
        • Moran J.
        • et al.
        AMSTAR-2: a critical appraisal tool for systematic reviews that include randomized or non-randomized studies of healthcare interventions, or both.
        BMJ. 2017; 358: J4008
        • Moher D.
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
        Ann Intern Med. 2009; 151: 264-269
        • Jadad A.R.
        • Cook D.J.
        • Jones A.
        • Klassen T.P.
        • Tugwell P.
        • Moher M.
        • et al.
        Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals.
        J Am Med Assoc. 1998; 280: 278-280
        • Xia L.
        • Xu J.
        • Guzzo T.J.
        Reporting and methodological quality of meta-analyses in urological literature.
        Peer J. 2017; 5: e3129
        • Braga L.H.
        • Pemberton J.
        • DeMaria J.
        • Lorenzo A.J.
        Methodological concerns and quality appraisal of contemporary systematic reviews and meta-analyses in pediatric urology.
        J Urol. 2011; 186: 266-272
        • Borawski K.M.
        • Norris R.D.
        • Fesperman S.F.
        • Vieweg J.
        • Preminger G.M.
        • Dahm P.
        Levels of evidence in the urological literature.
        J Urol. 2007; 178: 1429-1433
        • Egger M.
        • Davey Smith G.
        • Altman D.G.
        Systematic reviews in health care.
        BMJ Books, London2001
        • Scales Jr., C.D.
        • Norris R.D.
        • Peterson B.L.
        • Preminger G.M.
        • Dahm P.
        Clinical research and statistical methods in the urological literature.
        J Urol. 2005; 174: 1374-1379
        • Assmann S.F.
        • Pocock S.J.
        • Enos L.E.
        • Kasten L.E.
        Subgroup analysis and other subgroup (mis)uses of baseline data in clinical trials.
        Lancet. 2000; 355: 1064-1069
        • Guyatt G.
        • Jaeschke R.
        • Heddle N.
        • Cook D.
        • Shannon H.
        • Walter S.
        Basic statistics for clinicians: 1. Hypothesis testing.
        CMAJ (Can Med Assoc J). 1995; 152: 27-32
        • Afshar K.
        • Jafari A.S.
        • Seth A.
        • Lee J.K.
        • MacNeily A.E.
        Publications by the American Academy of Pediatrics Section on Urology: the quality of research design and statistical methodology.
        J Urol. 2009; 182: 1906-1910
        • MacDonald S.L.
        • Canfield S.E.
        • Fesperman S.F.
        • Dahm P.
        Assessment of the methodological quality of systematic reviews published in the urological literature from 1988 to 2008.
        J Urol. 2010; 184: 648-653
        • Greenland S.
        • O'Rourke K.
        On the bias produced by quality scores in meta-analysis, and a hierarchal view of proposed solutions.
        Biostatistics. 2001; 2: 463-471
        • Juni P.
        • Witschi A.
        • Bloch R.
        • Egger M.
        The hazards of scoring the quality of clinical trials for meta-analysis.
        J Am Med Assoc. 1999; 282: 1054-1060
        • Liu P.
        • Qiu Y.
        • Qian Y.
        • Chen X.
        • Wang Y.
        • Cui J.
        • et al.
        Quality of meta-analyses in major leading gastroenterology and hepatology journals: a systematic review.
        J Gastroenterol Hepatol. 2017; 32: 39-44
        • Moher D.
        • Liberati A.
        • Tetzlaff J.
        • Altman D.G.
        • PRISMA Group
        Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
        PLoS Med. 2009; 6e1000097
        • Ostlie D.J.
        • Peter S.D.S.
        The current state of evidence-based pediatric surgery.
        J Pediatr Surg. 2010; 45: 1940-1946
        • Schulz K.F.
        • Altman D.G.
        • Moher D.
        CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials.
        PLoS Med. 2010; 7e1000251
        • Blakely M.L.
        • Kao L.S.
        • Tsao K.
        • Huang E.Y.
        • Tsai A.
        • Tanaka S.
        • et al.
        Adherence of randomized trials within children’s surgical specialties published during 2000 to 2009 to standard reporting guidelines.
        J Am Coll Surg. 2013; 217: 394-399
        • Cullis P.S.
        • Gudlaugsdottir K.
        • Andrews J.
        A systematic review of the quality of conduct and reporting of systematic reviews and meta-analyses in paediatric surgery.
        PLoS One. 2017; 12e0175213
        • Salim A.
        • Mullassery D.
        • Losty P.D.
        Quality of systematic reviews and meta-analyses studies published in paediatric surgery.
        Br J Surg. 2017; 52: 1732-1735