Abstract
Introduction
The American Society of Clinical Oncology Value Framework (ASCO-VF) and European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS) have been developed to quantify clinical benefit and establish value of anticancer therapies. While ASCO-VF is indifferent to the cancer type, ESMO-MCBS does not endorse their framework for scoring hematologic malignancies. The reason for this exclusion and whether ESMO-MCBS can be applied in the hematologic setting is still unclear. The purpose of this study was to determine whether measurement characteristics of ESMO-MCBS are similar to ASCO-VF when evaluating hematologic malignancies.
Methods
Phase III randomized controlled trials (RCTs) of hematological malignancy drugs approved by the US Food and Drug Administration, European Medicines Agency and Health Canada between 2006 to 2017 were identified and scored using ASCO-VF (version 2) and ESMO-MCBS (version 1.1) by two independent reviewers. We assessed the convergent construct validity of the two frameworks to determine the degree to which these two theoretically similar measures assess the same construct of clinical benefit. Spearman correlation coefficients were calculated for the preliminary framework scores (using only the survival efficacy components of the scores) and the final framework scores (following adjustment of the preliminary scores with toxicity and quality of life (QoL) data). Correlation coefficients were also calculated for each subtype of blood cancer separately. The inter-rater reliability of the frameworks was assessed using intra-class correlation coefficients (ICC).
Results
In total, 48 studies were included and scored using the ASCO-VF (median score: 34.38; IQR: 22.26 - 51.69) and ESMO-MCBS (median score: 3; IQR: 2.75 - 4). Out of the 48 RCTs, 16 (33%) evaluated novel anticancer treatments for leukemia, 23 (48%) for myeloma, 8 (17%) for lymphoma and 1 (2%) for myelodysplastic syndromes. Spearman correlation coefficients between both frameworks for preliminary and final scores were 0.01 (95% CI, -0.23 to 0.25) and -0.02 (95% CI, -0.28 to 0.24), respectively. When stratified by indication, the correlation coefficients for the preliminary scores were -0.01 (95% CI, -0.50 to 0.49), -0.06 (95% CI, -0.46 to 0.29), and 0.58 (95% CI, -0.21 to 0.91) for leukemia, myeloma, and lymphoma, respectively. The correlation coefficients for the final scores by cancer indications were -0.09 (95% CI, -0.56 to 0.42), 0.15 (95% CI, -0.22 to 0.49), and -0.29 (95% CI, -0.86 to 0.52) for leukemia, myeloma, and lymphoma, respectively (Table 1). For both preliminary and final scores, ASCO-VF showed excellent and ESMO-MCBS showed good inter-rater reliability (Table 2).
Conclusions
Our results suggest divergent validity of ESMO-MCBS and ASCO-VF when assessing hematologic malignancies, indicating that these frameworks likely measure different constructs of clinical benefit in this setting. Furthermore, the ASCO-VF demonstrated improved inter-rater reliability compared to ESMO-MCBS. It remains unclear which framework is correctly assessing the clinical benefit of hematologic drugs. To better understand if one framework is more accurate in its evaluation, future research that compares these frameworks to other established measures of clinical benefit (such as quality adjusted life years) in the hematologic setting is warranted. Due to the lack of construct validity and ESMO's endorsement of their framework solely for solid cancers, the ASCO-VF should currently be considered for use in the hematologic setting.
No relevant conflicts of interest to declare.
This icon denotes a clinically relevant abstract
This feature is available to Subscribers Only
Sign In or Create an Account Close Modal