Critical review and quality-assessment of cost analyses in radiotherapy: How reliable are the data?

Purpose/objective: Health economic evaluations (HEE) are increasingly having an impact on policymakers, although the results greatly depend on the quality of the methodology used and on transparent reporting. The two main objectives of this study were to evaluate the quality of cost analyses of external beam radiotherapy (EBRT) and to assess the comprehensiveness and relevance of cost criteria defined in three validated quality-assessment instruments.

Materials and methods: The selection of articles was based on a previous systematic literature review of EBRT-costing studies retrieved from January 2004 to January 2015 (Period 1) in MEDLINE, Embase, and NHS-EED databases and completed in a second time period from January 2015 to November 2018 (Period 2). Three validated instruments to assess the methodology quality with the CHEC and the QHES, and the methodology with the CHEERS checklists were used. The quality was evaluated by both quantitative and qualitative analyses. The scoring robustness was examined with the Kendall coefficient of concordance and inter-class correlation coefficients.

Results: In total, twenty-three articles were selected. The main geographic areas of cost analyses were Canada (n = 5), France (n = 4), and the USA (n = 4). The most commonly studied pathologies and technologies were prostate (n = 7) and head and neck cancer (n = 5) and IMRT (n = 8) and IGRT (n = 2), respectively. The mean instrument scores demonstrated a fair degree of methodological quality, with 69.7% for the CHEC, 73.6% for the QHES, as well as for the reporting quality, with 59.4% for CHEERS for Period 1 (74.4%, 71.5%, and 66.1%, respectively, for Period 2). An additional qualitative analysis per criterion revealed that certain items, essential for understanding the costing methodology and the results (e.g., the time horizon, discount rate, sensitivity analysis) were often only partially completed. Statistical analysis confirmed that the reviewers' scoring was consistent. The instruments identified the same top three articles, albeit with a degree of variation in the ranking.

Conclusion: Qualitative and quantitative assessment of cost analyses in EBRT exhibits a fair level of study quality in terms of the methodology and reporting transparency. The impact of cost calculations on the final HEE result appears to be underestimated, and increased transparency of the data sources and the methodologies is needed.