Validation of the Hematopoietic Cell Transplantation-Specific Comorbidity Index: a prospective, multicenter GITMO study

Allogeneic hematopoietic stem cell transplantation (HSCT) is an established therapy for malignant and nonmalignant hematologic disorders. In recent years, novel approaches, such as the reduced-intensity conditioning regimens, have expanded the use of HSCT also to elderly patients or to patients otherwise ineligible for conventional transplants. HSCT still remains associated with a significant mortality and morbidity, although the Seattle team has recently observed a substantial reduction of nonrelapse mortality (NRM) and overall mortality in the last years.¹  Careful assessment of risks and benefits before transplantation remains however an essential issue. Three major factors influence NRM and overall survival (OS) after HSCT: the patient's disease, the type of transplant procedure and donor, and the patient's risk profile, which includes age, performance status, and presence of comorbidities. In an attempt to improve quantification of the patient's risk profile, Sorror et al recently proposed the Hematopoietic Cell Transplantation-Specific Comorbidity Index (HCT-CI) developed from a single-center retrospective analysis and internal validation.²  The HCT-CI demonstrated capturing more pretransplant comorbidities than the previously used Charlson Comorbidity Index and to provide better assessment of NRM, defining 3 risk groups, with HCT-CI score of 0 (low-risk), 1-2 (intermediate risk) and ≥ 3 (high-risk), respectively, showing linear predictions of NRM and OS.

On this premise, the HCT-CI score has been included as an eligibility criterion in some clinical trials, but it has never been externally validated by a large multicenter longitudinal study. Furthermore, it is not known how the clinical usefulness of the HCT-CI applies to all the different malignant hematologic diseases or whether its use should be preferably restricted to selected disorders. In this study, we prospectively collected comorbidity data to compute the HCT-CI score in a consecutive series of patients undergoing allogeneic BM transplantation in Italy. The primary aim of the study was to externally validate the HCT-CI in terms of calibration and discrimination in a multicenter, prospective study setting. As a secondary aim, we evaluated the usefulness of the HCT-CI in different patient subgroups.

All Italian centers performing HSCTs in adult patients and belonging to the national BM transplantation network (Gruppo Italiano Trapianto di Midollo Osseo [GITMO]) were asked to participate in this prospective, multicenter observational study. As it is mandatory for any GITMO center to enter information for all their consecutive transplants into the European Bone Marrow Transplantation (EBMT) database (Promise, Project Manager Internet Service; http://www.ebmt.org), we first updated the Minimum Essential Data section of the Promise database to include questions specifically assessing the presence/absence of all the comorbidities required to calculate the HCT-CI score. Permission to perform this study was obtained from the GITMO Clinical Studies Board. All patients provided informed consent in accordance with the Declaration of Helsinki for the analysis of their clinical data.

Per protocol, we considered eligible for analysis only those first transplants performed from January 1, 2008 to February 1, 2011 on patients > 18 years for malignant or nonmalignant hematologic disease, and using peripheral blood stem cells or BM as cell source (thus excluding cord blood cells). This time frame was chosen because, from preliminary analysis of previous enrollment data, it was expected to yield at least about 200 cases for each of 4 prespecified diagnoses (acute leukemias, non-Hodgkin lymphomas, multiple myeloma, and myelodysplastic syndromes [MDSs]).

According to the EBMT criteria,³  we considered as myeloablative any regimen with a total busulfan dose > 8 mg/kg, or cyclophosphamide dose > 120 mg/kg (or > 60 mg/kg if in combination with other drugs), or melphalan dose > 140 mg/m² or total body irradiation dose > 6 Gy; reduced intensity/nonmyeloablative conditioning regimens were all others regimens with dosages below the aforementioned limits. According to the original paper of Sorror et al, we defined acute leukemia in first complete remission, chronic myeloid leukemia in chronic phase, and MDS-refractory anemia as low risk diseases; high risk diseases were all other diagnoses.² 

NRM was defined as death from nonrelapse causes; OS was defined as the time from transplantation to death for any cause. Data were censored at time of death or last available follow-up, as available from the mandatory EBMT update from each GITMO Center.

Multiple imputation was used to account for sporadic missing values in covariates other than those affecting the HCT-CI, to allow multivariate analyses be carried in the whole dataset, using the Stata mi impute procedure.⁴  Competing risks analysis was used to calculate the 12 and 24 months NRM cumulative incidence, using the Gray test to test differences between HCT-CI score groups.^5,6  OS was estimated using the Kaplan-Meier method; hazard ratios were computed between subgroups using Cox regression for OS and competing risk regression for NRM, both stratified for center. Competing risk regression was used to compute NRM cumulative incidence rates, considering nontransplant mortality as the competing event. The predictive ability of the HCT-CI score was assessed using time-dependent receiver-operator curves analysis.⁷  All computations were performed using Stata and the procedures cmprsk and survival ROC of the R statistical package.^8,9 

A total of 44 of 46 (95.6%) of all GITMO centers performing HSCTs in adult patients agreed to participate in the study. From 3318 HSCTs performed during the considered time frame by the enrolling centers, 1937 were available for analysis. The reasons for exclusion from the study were as follows: comorbidities not reported (n = 1167), second or more transplants (n = 111), incomplete follow-up data (n = 55), nonhematologic diseases (n = 31), and lost to follow-up (n = 17). There were no material differences in terms of OS and NRM between the 1937 evaluated patients and the 1167 who were excluded because of failure to report comorbidities (24-month OS, 53.9%; 95% CI, 50.8%-56.8% vs 56.6%; 95% CI, 52.5%-60.5%; P = .23, respectively, in the evaluated/excluded groups; 24-month NRM, 23.8%; 95% CI, 21.3%-26.5% vs 26.1%; 95% CI, 23.3%-30.6%; P = .15 in the evaluated/excluded groups).

Table 1 reports the main characteristics and the prevalence of comorbidities in the cohort, 1119 patients (58%) being classified as low risk (HCT-CI score = 0), 441 (23%) as intermediate risk (HCT-CI score = 1-2), and 377 (19%) as high risk (HCT-CI score ≥ 3).

The patient's follow-up totaled 1681 patient-years, with a median time of 10.1 months from transplantation (range, 0.03-38.77). During follow-up, 666 deaths were observed in the cohort (332 NRM, 334 disease-related deaths). At multivariate analysis, HCT-CI score, age above 50 years, high-risk disease, and unrelated donor were all associated with increased NRM and decreased OS (Table 2). HCT-CI score and high-risk disease were the strongest predictors of both NRM and OS, as apparent from multivariate analysis. Table 3 and Figure 1 report the OS and NRM for the low-, intermediate-, and high-risk HCT-CI score groups.

To validate the predictive ability of the HCT-CI score, we compared its performance in our dataset in terms of calibration and discrimination.^10,11  Two-year NRM cumulative incidences, accounting for competing risks, and OS were used to compare the calibration of the HCT-CI model, and are presented in Figures 1 and 2 and stratified according to different clinical conditions in Table 4. Figure 3 presents NRM and OS probabilities stratified for reduced intensity or myeloablative conditioning regimens and HCT-CI, after Cox (OS) or competing-risk regression (NRM) adjustment for all the variables presented in Table 2.

Increasing HCT-CI score confirmed to be associated with higher 1- and 2-year probability of NRM and with lower OS, with a double NRM, and overall mortality risk in those patients having a score ≥ 3. However, while the Sorror's 2-year predicted NRM nearly overlapped those observed in our cohort for low- and intermediate-risk HCT-CI scores, the HCT-CI predicted 2-year NRM did overestimate the observed 2-year NRM for the high-risk category (41% vs 27.3% respectively; Figure 4).

In terms of discriminatory capability, the HCT-CI score showed a c-value of 0.60 and 0.54 for NRM and OS, respectively. Prespecified subgroup analysis disclosed significant differences in NRM prediction by the HCT-CI score, being higher in patients undergoing transplantation for lymphoma, MDS, and acute myeloid leukemia (AML) in first remission (Table 4). In contrast, in patients with multiple myeloma, the predictive ability was lower, with no clear NRM gradient between the 3 HCT-CI score groups and the lowest observed c-value.

Finally, we assessed the accuracy of data collected for HCT-CI scoring on a random set of 244 patients. These audits were made at the participating sites by researchers independent from the original abstractors, who recalculated from the patient charts the HCT-CI score completely blinded from the previous results. Discrepant results were observed in 26 of 244 patients, resulting in a shift of the HCT-CI score group in 9.8% of the audited sample: 13 of 26 had an increased HCT-CI score, 11 of 26 a decreased score, and in 2 cases the score remained unchanged (but with different individual comorbidities). However, there were no differences between the original and recalculated mean HCT-CI score (1.58 vs 1.60; P = .66), and there were no differences in terms of NRM predictive ability between the original and recalculated HCT-CI, (c statistic 0.699 vs 0.700; P = .911; both adjusted for age, sex, high-risk disease, donor type, and stem cell source).

In this study, we primarily aimed at externally validate the HCT-CI, a widely used prognostic index originally proposed in 2005²  whose usefulness has been subsequently reported only in studies based on limited, retrospective and mostly single-center patient series.^12-46  To this aim, we prospectively enrolled a wide cohort of unselected patients consecutively undergoing allogeneic BM transplantation in Italy and evaluated the NRM and OS predictive ability of the HCT-CI, applying the same selection criteria originally used by Sorror et al.²  To our knowledge, this is the first study aimed to validate the HCT-CI in a prospective large cohort analysis using a multicenter national registry.

As a first result, we were able to confirm that the HCT-CI is an independent predictor of both NRM and OS. However, whereas we observed a very good linear correlation between the predicted and the observed NRM in patients having a HCT-CI score of 0 and 1-2, the observed NRMs were considerably lower in patients having a HCT-CI score greater than 2. This finding therefore reduces the overall predictive value of the HCT-CI score, as indicated by a lower discriminatory c-value in our study (0.60 vs 0.65 reported by Sorror et al² ) and could be partly explained by an overfitting of the original regression model, a well-known statistical artifact that justifies the need for validation in external cohorts.¹¹  The reliability of the HCT-CI scoring is another important issue, particularly in multicenter studies, in which exposure misclassification could be a consequence both of intra- and between-center variability, and reduces the degree of association between HCT-CI and NRM. To assess the degree of intra-center data accuracy, we performed an audit on ∼ 12% of patient charts, finding only minimal changes in HCT-CI score that did not affect the final results of our study. Between-center variability could not be formally evaluated in our study, but we tackle this issue by stratifying our analyses for centers. Finally, another possible explanation of the different predictive capability of the HCT-CI score could well be the higher percentage of patients transplanted for MDS and chronic myeloid leukemia in the Sorror et al cohort.²  In these 2 disease subgroups, our analysis disclosed that the NRM is remarkably high in those patients having a HCT-CI score ≥ 3. Therefore, an additional explanation of the reduced observed NRM in patients with an HCT-CI score ≥ 3 could therefore lie in the different sample composition of our cohort.

Additional major differences exist between our study population and the population of the original study of Sorror et al.²  First, we analyzed only adult patients. Second, our cohort had a different composition in terms of a higher percentage of high risk diseases (69% vs 41%) and of unrelated donors (50% vs 42%), more infections (11% vs 4%) but less psychiatric (3% vs 9%), mild hepatic (5% vs 16%), and mild pulmonary (10% vs 24%) comorbidities. Our cohort showed a higher proportion of patients having an HCT-CI score of 0. It is however worthwhile to note that, in our cohort, the baseline 2-year NRM risk in patients with a HCT-CI score of 0 was 14.7%, a value that nearly overlaps the Sorror et al-predicted NRM (14%).²  Therefore, despite the unavoidable differences in patient composition and the different (prospective) design of our study, the comparability of the baseline estimates in the 2 studies strengthens the validity of our findings and the generalizability of the HCT-CI.

Prespecified subgroup analyses showed that, in patients having a HCT-CI score equal to 0, the NRM was similar for myeloablative and reduced-intensity/nonmyeloablative conditioning regimens, whereas in patients having a HCT-CI score ≥ 1 the use of reduced-intensity/nonmyeloablative conditioning regimens was correlated with lower NRM rates during the first year. After the first year, however, the myeloablative group reaches a plateau, whereas in the reduced-intensity/nonmyeloablative conditioning regimen group, NRM continues to increase. The advantage of related donor compared with unrelated donor is evident across all the HCT-CI risk groups, but in the setting of transplant from related donor the absence (HCT-CI score = 0) or presence (HCT-CI score ≥ 1) of comorbidities shows a significant impact on NRM.

As a secondary finding, we demonstrated that the predictive ability of HCT-CI was higher in patients having lymphoma, as already reported,^21,26,27  or MDS, as already reported,^{14,15,18,28,29}  and in those receiving peripheral blood stem cells as stem cell source. On the other hand, the predictive value was much lower in patients having acute leukemias. However, because the decision to offer elective HSCT in patients with AML in first complete remission without high-risk characteristics is controversial, we analyzed separately the 413 AML patients who received HSCT in first remission. In these patients, the 2-year NRM was 9.1%, 11.4%, and 19.4% for the HCT-CI score groups 0, 1-2, and ≥ 3 respectively. Although these findings should be taken very cautiously given the lack of randomization to HSCT in our cohort, they nonetheless suggest that the NRM risk in selected patients with no or few comorbidities could be very limited, and below the reported rate of death attributable to disease relapse, even in patients with AML at low risk of relapse (estimated to be ∼ 20%).⁴⁷  The same reasoning may be applied to patients with lymphoma or myelodysplastic syndrome because the NRM risk is increased more than 2-fold in patients having an HCT-CI ≥ 3 compared with those having a score equal to 0. Therefore, our study further supports the need for appropriately designed studies investigating HSCT transplant in those patients having a low HCT-CI score predicting a low NRM risk.

In an additional subgroup analysis, we evaluated the predictive role of HCT-CI in patients undergoing reduced-intensity or myeloablative conditioning regimens. Given the observational nature of our study, we used a multivariate approach to weight the reciprocal contributions of HCT-CI and conditioning regimens to OS and transplant-related mortality, adjusted for high-risk disease, age, and donor type. In this analysis, HCT-CI was a determinant of both OS and NRM, whereas type of conditioning was related to OS but not with NRM, a finding that seems in keeping with a previously published report in a smaller series.¹⁹  Our results support the hypothesis that HCT-CI score groups and myeloablative conditioning have an apparently additive effect in predicting survival: for instance, 2-year survival is similarly reduced by an HCT-CI score 1-2 in patients receiving reduced-intensity conditioning or by an HCT-CI score 0 in patients receiving myeloablative conditioning.

A possible limitation of the present study is the unavailability of the HCT-CI in a relevant fraction of all potentially eligible patients, given that 1167 of 3318 (35%) were excluded from analysis because, despite that the study formally started on January 1, 2008, some GITMO centers started to prospectively enter comorbidity data into the MEDAB database with some delay. This explains why several patients (potentially eligible for study because they received BMT in the per-protocol time-frame) were considered as missing. However, we did not observe any difference both in terms of OS and NRM between those patients in whom comorbidities were reported and those without, supporting the validity of our study and the absence of a relevant selection bias.

In conclusion, in the largest recent cohort of unselected patients undergoing HSCT so far described, we were able to confirm the clinical utility of the HCT-CI score, although with a slightly reduced discriminant capability. Furthermore, our findings suggest that the HCT-CI could have the potential to identify patients at low NRM risk that could benefit from a more intensive, transplant-based treatment strategy in selected disease subgroups. This latter finding needs to be further explored by appropriately designed randomized clinical trials.

The authors thank Caterina Micò, Irene Donnini, Alessandra Sperotto, and Carlo Borghero for the audit reevaluation of patients' charts. The authors also thank colleagues from the following institutions (GITMO centers) in Italy that contributed to the study: Department of Hematology; Nuovo Ospedale Torrette, Ancona (P. Leoni); Division of Hematology, Ospedale S.G. Moscati, Ascoli Piceno (P. Galieni); Division of Hematology, Ospedale “S. S. Antonio e Biagio” Alessandria (A. Levis); Division of Hematology, University of Bari, Bari (G. Specchia); Division of Hematology, Ospedali Riuniti, Bergamo (A. Rambaldi); Institute of Hematology and Clinical Oncology “L. A. Seragnoli,” Ospedale “S. Orsola-Malpighi,” University of Bologna, Bologna (G. Bandini, M. Baccarani); Department of Hematology, Ospedale Regionale, Bolzano (M. Casini, S. Cortelazzo); Bone Marrow Transplant Center, Spedali Civili, Brescia (D. Russo); Division of Hematology and Bone Marrow Transplant Center, Ospedale Oncologico “A. Businco,” Cagliari (E. Angelucci, D. Baronciani); Bone Marrow Transplantation Unit, Ospedale “R. Binaghi,” University of Cagliari, Cagliari (G. La Nasa); Division of Hematology and Bone Marrow Transplantation, Ospedale “Ferrarotto,” Catania (G. Milone); Division of Hematology, Ospedale “S. Croce e Carlo,” Cuneo (N. Mordini); Department of Hematology, Ospedale “Careggi,” University of Florence, Firenze (A. Bosi, S. Guidi); Division of Hematology, Ospedale “S. Martino,” Genova (A. Bacigalupo, M. T. Van Lint); Hematology-Bone Marrow Transplantation Unit, Istituto Nazionale dei Tumori, University of Milano, Milano (P. Corradini); Istituto Europeo di Oncologia, Milano (G. Martinelli); Division of Hematology Ospedale “Cà Granda” Niguarda, Milano (E. Morra, P. Marenco); Department of Hematology, Fondazione Istituti di Ricovero e Cura a Carattere Scientifico (IRCCS) Ospedale Maggiore Policlinico, Mangiagalli e Regina Elena, Milano (G. Lambretenghi Deliliers, F. Onida); Hematology and Bone Marrow Transplantation Unit, S. Raffaele Scientific Institute, Milano (F. Ciceri, J. Peccatori); Transplantation Unit Department of Oncology-Hematology, IRCCS Clinica Humanitas, Rozzano (L. Castagna); Department of Oncology and Hematology University of Modena and Reggio Emilia, Modena (F. Narni); Division of Hematology and Transplant Unit, Ospedale “S. Gerardo,” University of Milano-Bicocca, Monza (P. Pioltelli); Division of Hematology, University of Napoli “Federico II” Medical School, Napoli (C. Selleri); Division of Hematology and Transplant Unit, Ospedale “V. Cervello,” Palermo (R. Scimè); Department of Oncology, Hematology Unit, Ospedale “La Maddalena,” Palermo (M. Musso); Division of Hematology, University of Pavia, Fondazione IRCCS Policlinico “S. Matteo,” Pavia (E. P. Alessandrino); Hematology and Transplant Center, Ospedale “S. Salvatore,” Pesaro (G. Visani); Department of Hematology, Ospedale Civile, Pescara (P. Di Bartolomeo); Oncology and Hematology Department, Ospedale “Guglielmo da Saliceto,” Piacenza (D. Vallisa, L. Cavanna); Division of Hematology, Univeristy of Pisa, Pisa (M. Petrini, F. Papineschi); Transplant Unit “A. Neri,” Ospedale “Bianchi-Melacrino-Morelli,” Reggio Calabria (P. Iacopino, G. Messina); Hematology Unit, Arcispedale “S. Maria Nuova,” Reggio Emilia (F. Merli, L. Gugliotta); Division of Hematology, Department of Cellular Biotechnologies and Hematology, University “La Sapienza” (A. P. Iori, R. Foà); Hematology and Stem Cell Transplantation Unit Ospedale “S. Camillo,” Roma (A. Locasciulli, I. Majolino); Division of Hematology, University “Cattolica S. Cuore,” Roma (G. Leone, S. Sica); Hemato-Oncology Transplant Unit, University “Tor Vergata,” Transplant Network, Roma (W. Arcese); Unit of Hematology and Bone Marrow Transplantation, IRCCS, “Casa Sollievo della Sofferenza,” S. Giovanni Rotondo (A. M. Carella, N. Cascavilla); Division of Hematology and Bone Marrow Unit, Azienda Ospedaliera Universitaria Senese “S. Maria alle Scotte,” Siena (G. Marotta); Institute of Hematology, Ospedale “San Giusepppe Moscati,” Taranto (P. Mazza); Division of Hematology, Ospedale “S. Giovanni Battista,” Torino (M. Falda, B. Bruno); Division of Hematology, Ospedale “C. Panico,” Tricase (V. Pavone); Division of Hematology and Bone Marrow Transplantation, University of Udine, Udine (R. Fanin, F. Patriarca); Division of Hematology and Bone Marrow Unit, Policlinico “G.B. Rossi,” Verona (G. Pizzolo, F. Benedetti); and Department of Hematology, Ospedale “S. Bortolo,” Vicenza (F. Rodeghiero, R. Raimondi). R.R. thanks Michela Trentin for her support in data collection.

This work was supported in part by Fondazione Progetto Ematologia (Hematology Project Foundation, Vicenza, Italy) and Associazione Vicentina per le Leucemie, i Linfomi e il Mieloma/Associazione Italiana Leucemie, Vicenza, Italy.

Contribution: R.R., A. Bosi, and R.F. designed the research; R.R. and R.O. collected data; R.R., A.T., F.R., R.F., A.R., A. Bosi, and A. Bacigalupo analyzed and interpreted data; R.C. performed statistical analysis; and R.R and A.T. wrote the manuscript.

Conflict-of-interest disclosure: The authors declare no competing financial interests.

Correspondence: Roberto Raimondi, Hematology Department, S. Bortolo Hospital, Via Rodolfi 37, 36100 Vicenza, Italy; e-mail: raimondi@hemato.ven.it.