Advertisement
Letter to the Editor| Volume 10, ISSUE 1, P200-201, February 2014

Reputation rankings for pediatric urology moderately reflect academic productivity

  • Francis Deng
    Correspondence
    Tel.: +1 703 509 6157; fax: +1 888 706 5895.
    Affiliations
    Washington University School of Medicine, Campus Box 8077, 660 S. Euclid Ave, St. Louis, MO 63110-1093, USA
    Search for articles by this author
Published:September 17, 2013DOI:https://doi.org/10.1016/j.jpurol.2013.07.021
Reputation rankings for pediatric urology moderately reflect academic productivity
Previous ArticleResponse to Nasir et al.
Next ArticleResponse to “Reputation rankings for pediatric urology moderately”
      Lloyd et al. [

      Lloyd JC, Madden-Fuentes RJ, Nelson CP, Kokorowski PJ, Wiener JS, Ross SS, et al. The publication ranking score for pediatric urology: quantifying thought leadership within the subspecialty. J Pediatr Urol 2013 Apr 24. pii: S1477-5131(13)00075-2. http://dx.doi.org/10.1016/j.jpurol.2013.03.012. [Epub ahead of print].

      ] presented the Publication Ranking Score (PRS) as an objective measure of hospital “process” quality that differs significantly from US News and World Report (USNWR) hospital rankings. This contrast is important, the authors suggested, because USNWR rankings may “lean heavily on institutional reputation.” The USNWR reputation survey, however, accounts for only 25% of the overall ranking. In this context, Joseph [

      Joseph DB. Editorial comment on “A publication ranking score for pediatric urology: quantifying thought leadership within the subspecialty”. J Pediatr Urol 2013 May 23. pii: S1477-5131(13)00106-X. http://dx.doi.org/10.1016/j.jpurol.2013.04.012. [Epub ahead of print].

      ] suggested that “the PRS ranking would be better compared to the process domain (reputation) and not the overall final institutional ranking”. Such an analysis is presented below, revealing that the PRS is significantly associated with reputation in the specialty of pediatric urology.
      I extracted the published PRS data (impact factor-normalized number of publications) for the top 50 pediatric urology hospitals from 2012 [

      Lloyd JC, Madden-Fuentes RJ, Nelson CP, Kokorowski PJ, Wiener JS, Ross SS, et al. The publication ranking score for pediatric urology: quantifying thought leadership within the subspecialty. J Pediatr Urol 2013 Apr 24. pii: S1477-5131(13)00075-2. http://dx.doi.org/10.1016/j.jpurol.2013.03.012. [Epub ahead of print].

      ] and identified individual institutions based on their USNWR rank and number of faculty. The scores were linked with mean USNWR reputation scores in pediatric urology from 2011 to 2013 (all the years when USNWR published data for the top 50 children's hospitals in each specialty) [

      Olmsted MG, Geisen E, Murphy J, Bell D, Morley M, Williams J, et al. Best children's hospitals 2013e14 methodology. RTI Int Retrieved Sept 7, 2013, from: http://www.usnews.com/pubfiles/BCH_Methodology_2013-14.pdf.

      ,

      Olmsted MG, Geisen E, Murphy J, Williams J, Pitts A, Bell D, et al. Best children's hospitals 2012e13 methodology. RTI Int Retrieved Sept 7, 2013, from: http://www.rti.org/pubs/bchmethod_2012-13.pdf.

      ,

      Olmsted MG, McFarlane E, Murphy J, Severance J, Pitts A, Bell D, et al. Best children's hospitals 2011e12 methodology. RTI Int Retrieved Sept 7, 2013, from: http://www.rti.org/pubs/abchmethod_2011-12.pdf.

      ]. For years when a hospital was not in the top 50, its reputation score was set to zero. Descriptive statistics and Spearman's correlation coefficients were calculated using Microsoft Excel 2010/XLSTAT-Pro 2013.4.04 (Addinsoft, New York, NY). Percentage agreement was determined by the number of top n hospitals ranked by PRS that were also in the top n ranked on the basis of USNWR reputation score.
      Institutional rank based on PRS differed from that based on USNWR reputation score by a mean ±7.8 positions (Table 1), which is somewhat less than the average ±12.0 position change between overall USNWR ranking and PRS ranking that Lloyd et al. [

      Lloyd JC, Madden-Fuentes RJ, Nelson CP, Kokorowski PJ, Wiener JS, Ross SS, et al. The publication ranking score for pediatric urology: quantifying thought leadership within the subspecialty. J Pediatr Urol 2013 Apr 24. pii: S1477-5131(13)00075-2. http://dx.doi.org/10.1016/j.jpurol.2013.03.012. [Epub ahead of print].

      ] reported. The majority of rankings based on reputation score agreed with PRS rankings. Compared to PRS, rankings based on reputation similarly classified the top hospital, six of the top 10 hospitals, and 16 of the top 20 hospitals. Across all 50 hospitals, there was a highly statistically significant association of moderate size between PRS and reputation score (ρ = 0.576, p < 0.0001).
      Table 1Publication Ranking Score (PRS) (2006–2011) of the US News and World Report (UNSWR) 2012 top 50 pediatric urology hospitals re-ranked by USNWR reputation score (2011–2013).
      PRS (no. of publications adjusted by impact factor)PRS rankREP (USNWR specialist reputation score average 2011–2013; in %)REP rankChange in ranking (PRS–REP)
      505.3185.810
      308.3223.49−7
      265.6311.110−7
      237.646.513−9
      210.8526.06−1
      207.8679.324
      196.778.812−5
      176.386.314−6
      17690.637−28
      171.61039.446
      159.41133.156
      151.9124.719−7
      147.11324.885
      143.3144.818−4
      141.6154.420−5
      138.7161.427−11
      137.91710.5116
      116.5180.343−25
      113.8192.924−5
      108.52059.6317
      106.7210.344−23
      90.5221.625−3
      90.1233.5221
      86.8240.735−11
      84.9255.0178
      74.8261.229−3
      72.3270.539−12
      65.6283.4235
      60.1290.932−3
      57.8300.047−17
      57.5311.0301
      533224.9725
      52.9330.146−13
      50.3341.2286
      44.5351.4269
      41.3365.21620
      33.7370.047−10
      33.6380.442−4
      33.3390.047−8
      32.2405.81525
      28.7410.4401
      21.4421.03012
      19.6430.93310
      18.3440.047−3
      13.2450.2450
      12.3464.42026
      10.4470.73512
      8.5480.63810
      2.3490.4409
      1.7500.83416
      Other than the focus on reputation, several differences between the current analysis and that of Lloyd et al. merit mention.
      First, while multiple years of data were included in the PRS, the authors compared only 1 year of USNWR rankings. Noise from year-to-year changes in ranking methodology or data error may obscure real relationships. To address this concern, I averaged 3 years of reputation data.
      Second, the authors did not describe how they determined PRS rankings. They reported ties at ranks 47 and 49 despite differences in impact factor-adjusted publication numbers, which may affect the rank-based statistics calculated. For transparency, I recalculated PRS rankings based on those raw scores alone.
      Third, the authors computed Spearman's rank correlations on the USNWR top 10 subgroup (in addition to the entire top 50), even though a sample size of 10 is underpowered to detect associations of moderate size using this test [
      • Bonett D.G.
      • Wright T.A.
      Sample size requirements for estimating Pearson, Kendall and Spearman correlations.
      Psychometrika. 2000; 65: 23-28
      ]. Unsurprisingly, they failed to find a statistically significant correlation between USNWR rank and PRS ranks in this group.
      Finally, though the authors found a highly statistically significant association between the overall ranks of institutions in the USNWR top 50 and PRS ranks (p = 0.0004), they write it off as “little correlation”. To the contrary, a Spearman's ρ of 0.48 is considered of moderate size. The current analysis only further reaffirms the presence of an association between PRS and existing quality metrics.
      These findings suggest that reputation captures an aspect of hospital quality that overlaps significantly with academic productivity. The existence of collinearity may hold implications for ranking publishers and policy-makers looking for adjunctive measures of hospital process quality.

      Funding

      None.

      Conflict of interest

      None.

      References

      1. Lloyd JC, Madden-Fuentes RJ, Nelson CP, Kokorowski PJ, Wiener JS, Ross SS, et al. The publication ranking score for pediatric urology: quantifying thought leadership within the subspecialty. J Pediatr Urol 2013 Apr 24. pii: S1477-5131(13)00075-2. http://dx.doi.org/10.1016/j.jpurol.2013.03.012. [Epub ahead of print].

        View in Article

      2. Joseph DB. Editorial comment on “A publication ranking score for pediatric urology: quantifying thought leadership within the subspecialty”. J Pediatr Urol 2013 May 23. pii: S1477-5131(13)00106-X. http://dx.doi.org/10.1016/j.jpurol.2013.04.012. [Epub ahead of print].

        View in Article

      3. Olmsted MG, Geisen E, Murphy J, Bell D, Morley M, Williams J, et al. Best children's hospitals 2013e14 methodology. RTI Int Retrieved Sept 7, 2013, from: http://www.usnews.com/pubfiles/BCH_Methodology_2013-14.pdf.

        View in Article

      4. Olmsted MG, Geisen E, Murphy J, Williams J, Pitts A, Bell D, et al. Best children's hospitals 2012e13 methodology. RTI Int Retrieved Sept 7, 2013, from: http://www.rti.org/pubs/bchmethod_2012-13.pdf.

        View in Article

      5. Olmsted MG, McFarlane E, Murphy J, Severance J, Pitts A, Bell D, et al. Best children's hospitals 2011e12 methodology. RTI Int Retrieved Sept 7, 2013, from: http://www.rti.org/pubs/abchmethod_2011-12.pdf.

        View in Article
        • Bonett D.G.
        • Wright T.A.
        Sample size requirements for estimating Pearson, Kendall and Spearman correlations.
        Psychometrika. 2000; 65: 23-28
        View in Article

      Article info

      Publication history

      Published online: September 17, 2013

      Identification

      DOI: https://doi.org/10.1016/j.jpurol.2013.07.021

      Copyright

      © 2013 Journal of Pediatric Urology Company. Published by Elsevier Inc. All rights reserved.

      ScienceDirect

      Access this article on ScienceDirect

      Linked Article

      We use cookies to help provide and enhance our service and tailor content. To update your cookie settings, please visit the for this site.
      Copyright © 2023 Elsevier Inc. except certain content provided by third parties. The content on this site is intended for healthcare professionals.


      RELX