Creating an automated contemporaneous cohort in sickle cell anemia to predict survival after disease-modifying therapy

Myeloablative gene therapy and gene editing expand curative options for children and adults with sickle cell anemia (SCA). With the initiation of the novel cellular therapies, the Food and Drug Administration (FDA) requires a contemporaneous group receiving disease-modifying therapy to compare key clinical outcomes, including mortality. Developing a contemporaneous cohort of rare diseases is costly, inefficient, and requires many person-hours to construct and maintain it. Furthermore, the FDA requires a cellular therapy treatment group to have a follow up for at least 15 years after completion of gene therapy or gene editing trials.¹ More efficient strategies are required to efficiently and effectively develop contemporaneous comparison groups to comply with FDA requirements for novel treatments of rare diseases.

The widespread use of electronic health record systems creates new opportunities for the secondary use of big data from patient medical records. Options for rare disease clinical history typically require years of following up from a single center or a consortium of multiple centers. Clinical data warehouses of electronic health record data are available at many health care systems across the United States. These data warehouses contain clinical information from multiple years of medical records. Using clinical and laboratory data in a warehouse for creating a cohort of a rare genetic disease is emerging as a reasonable alternative to determine the clinical history of a rare disease.^2-6 Our objective was to close the knowledge gap of constructing an automated contemporaneous cohort in children and adults with SCA to evaluate the impact of disease-modifying treatment on mortality.

In keeping with the overall goal of the Cure Sickle Cell Initiative, a collaborative patient-focused research effort designed to accelerate the advancement of genetic-based cures for sickle cell disease (SCD), we used the clinical data warehouse of electronic health records at the Vanderbilt University Medical Center (VUMC). We tested the hypothesis that an automated contemporaneous cohort of children and adults with SCA could be developed to predict the mortality across different disease-modifying treatment options.

We conducted an observational study using VUMC’s electronic health record data. The VUMC institutional review board approved this study. It was conducted in accordance with the Declaration of Helsinki.

We developed an automated cohort using algorithms to identify individuals with SCA, defined as having either hemoglobin SS (HbSS) or HbS β thalassemia⁰ phenotype, in the electronic health record to create our cohort. These algorithms were adapted from other published algorithms.^5,7,8 The algorithm is described in the supplemental Appendix (supplemental Table 3; supplemental Figures 1-3; supplemental Appendix Methods). Individuals with SCA were identified from the VUMC Research Derivative, a fully identified clinical data warehouse drawn from the VUMC electronic health record. Individuals were included if there was an opportunity to prescribe a disease-modifying therapy such as hydroxyurea therapy or blood transfusion. Individuals in the cohort had SCA and consistent medical care within our system, defined as being seen for more than 0.5 years, and not having an interval greater than 3.0 years between consecutive visits (Figure 1). The full details of the automated cohort creation are listed in the supplemental Appendix (supplemental Table 10; supplemental Figures 1-3; supplemental Appendix Methods). Descriptions of the data extraction process for key data elements can be found in supplemental Table 6. Data were obtained from VUMC between 1 January 2004 and 30 April 2021, which included a cohort of more than 3 million individuals evaluated at the medical center during that period.

We also used another method to develop a cohort of individuals followed at VUMC to compare clinical features and overall mortality rates with the fully automated cohort. We refer to this cohort as the hybrid cohort because the construction included a low-level automated component that can be done without advanced computer programmer skills and a manual review of the electronic health record compared with the fully automated cohort. The essential covariates of the automated component of the hybrid cohort, deemed likely to be generalizable to most electronic health record systems, included age, sex, follow-up time, the gap between visits, and prescriptions for hydroxyurea. The key covariates of the manual chart review included the presence and type of SCA, disease-modifying treatment type, such as chronic transfusion and transplant, and mortality.

Our primary outcome measure to test our hypothesis was mortality. To determine the patient's vital status (alive or dead) for the automated cohort, we used structured fields for vital status and dates of death information automatically extracted from our data warehouse when available. These data fields were derived from multiple sources, including electronic health records; the social security death index, which contains only the fact of death and was available on 31 December 2016 in the data warehouse; or an external death data source, Datavant, an organization that augments the social security administration death master files with information from newspapers, funeral homes, and memorials to construct an individual-level database of >80% of US deaths annually.⁹ When a deceased person did not have a death date available from automated methods, we used the last visit date as a proxy for the date of death.

For the hybrid cohort, the vital status and date of death included only those recorded in the electronic health record. When the fact of death was available without a date of death, the last day of follow up in the clinic was substituted as the date of death. Further descriptions of the automated and manual data collection methods are described in Appendix supplemental Table 10.

We estimated the number of person-hours in the program based on the personnel allocated to the grant that worked on the project from 2019 to 2021. The research personnel included 2 research coordinators and 2 data analysts with research computer programming skills to collect information from the data warehouse. The team also included 1 senior project manager and 2 physicians.

From our data warehouse, we obtained demographic variables, insurance status, laboratory results, dates of the first and last hematology encounters within our health care system, medications, procedures such as transfusions and hematopoietic stem cell transplants (HSCTs), and the date of death. The demographic variables included the date of birth and sex. Laboratory results from Hb fractionation analyses (eg, isoelectric focusing or high-performance liquid chromatography) were used in automated algorithms to determine the sickle cell type based on standard cutoffs determined by previous literature.^10,11 Medications, regular red blood cell transfusions, and HSCTs were obtained using computerized algorithms. A participant was considered to have hydroxyurea therapy if they had 1 year of prescriptions for hydroxyurea in our electronic health record, which was defined as being followed for 1 year after initiating hydroxyurea prescriptions. A participant was considered to receive regular blood transfusion therapy if the participant had at least 9 transfusions in 12 months for 2 years.^12,13 If 2 or more transfusions occurred within 13 days, they were counted as only 1 transfusion. The coding of the type of disease-modifying therapy was mutually exclusive and hierarchical with treatment priorities: HSCT > blood transfusion > hydroxyurea therapy; for example, a participant who had been on hydroxyurea and chronic transfusion is coded as transfusion. All adults receiving HSCT had myeloablative-matched related donor or reduced intensity–related haploidentical HSCT.¹⁴ Total follow-up time was calculated from the first hematology encounter or first disease-modifying treatment (ie, hydroxyurea prescription, chronic transfusion, or transplant) until death or the last hematology visit.

Data were summarized as counts and percentages for categorical variables, and depending on the data distribution, as either means with standard deviations or medians and interquartile range for continuous variables. Cumulative incidence curves were generated using the Kaplan–Meier method. To identify associations with mortality, we constructed a baseline multivariable Cox regression model with time-varying effects for treatment type. Mortality was adjusted for age at the start date, sex, Medicaid status, and the type of treatment in our models. A patient was classified as either having Medicaid insurance for none of the visits (0%), all visits (100%), or some visits (1%-99%). The proportional hazards assumption was assessed, and Martingale and deviance residuals were used to assess model assumptions and the effect of outlier cases. P < .05 was considered significant for all analyses. Statistical Package for the Social Sciences version 27 (IBM Corp, Armonk, NY) was used to perform all analyses. All primary analyses were completed with the automated cohort, in which all data were obtained automatically using computer programming scripts. Additional sensitivity analyses were done with a hybrid cohort, which included automated and manually checked data and are presented in the supplemental Appendix.

More than 3 million unique participants were represented in the VUMC data warehouse during the sampling interval of 1 January 2004 and 30 April 2021. Our automated methods identified 988 participants with SCD (HbSS, HbSC, and other SCD compound heterozygotes). Our algorithms identified an SCD phenotype for 883 of these 988 participants, and 560 were designated to have SCA (Figure 1), demonstrating a 63% proportion of all adults with SCD. We removed 92 participants without visit dates during the assessment period or follow up of fewer than 0.50 years and 49 participants with a gap in the care of 3 or more years. Our final cohort of 419 participants with SCA had a median age of 10.2 years at the first visit, 49.9% were male, and had a median follow up of 7.4 years. A total of 25.8% (108 of 419), 37.5% (157 of 419), and 36.8% (154 of 419) had Medicaid insurance for none of the visits, all of the visits, and some of the visits, respectively (Table 1). The incidence rate of death was 0.15 per 100 patient-years for children younger than 18 years (95% confidence interval [CI], 0.03-0.44) and 2.89 per 100 patient-years for adults older than 18 years (95% CI, 2.01-3.88). Among adults, those 18 to 36 years had an incidence rate of death of 2.36 per 100 patient-years and those older than 36 years (95% CI, 1.57-3.42) had an incidence rate of death of 4.72 per 100 patient-years (95% CI, 2.70-7.67). A majority received a hydroxyurea prescription (58.0%) for at least a year, significantly fewer individuals received at least 1 year of regular blood transfusions for at least 2 years (13.3%), and only a small percentage of individuals received an HSCT (5.3%). Baseline laboratory values for the automated cohort are presented in supplemental Table 1.

Hydroxyurea therapy, transfusions, and transplant had significantly lower mortality than no disease-modifying treatment (P ≤ .001, P = .011, and P = .007, respectively). None of the 3 treatments had significantly different survival compared with the other 2. A Kaplan–Meier curve was constructed to evaluate the different treatments (Figure 2). A Cox regression model was then constructed using sex, age at the first visit, Medicaid insurance status throughout the sampling frame (0%, 1%-99%, and 100%, respectively), and time-varying treatment type, with the same categories for treatment and with no disease-modifying treatment used as reference categories in the model (Table 2). Age was associated with increased mortality (hazard ratio [HR], 1.09; P < .001). Compared with those on no disease-modifying treatment, participants prescribed with hydroxyurea therapy for at least a year had a decreased hazard of mortality (HR, 0.38; P = .016). Medicaid insurance status was associated with an increased hazard of mortality (Medicaid 1%-99%: HR, 3.05; P = .038 and Medicaid 100%: HR, 7.44; P ≤ .001) compared with the reference category of never having Medicaid insurance. No statistical difference was observed between those treated with regular transfusions and those who did not receive disease-modifying therapy (HR, 0.71; P = .440). Adults receiving HSCT had the highest survival (100%), but they were the lowest number of individuals (n = 22), and the regression coefficients could not be reliably estimated because there were no deaths.

We identified 432 individuals with SCA based on a manual electronic health record review. Of the 432 patients, 383 (81.8%) were also in the automated cohort and 49 were only in the hybrid cohort. There were 36 patients who were only in the automated cohort and not the hybrid cohort. For the primary outcome measure, mortality, the automated cohort had 47 deaths, whereas the hybrid cohort only had 29 deaths. The median survival age in the automated and hybrid cohorts, using only those who entered as adults, was 49.3 years (95% CI, 42.9-54.9) and 46.1 years (95% CI, 43.0-58.2), respectively. The demographic and laboratory features of the automated cohort were similar to those of the hybrid cohort (supplemental Tables 2-4). However, the follow-up time, the proportion of deaths, and the incidence rate of death differed on when comparing the automated with the hybrid cohort (supplemental Table 2).

The automated approach required ∼6500 hours (Table 3). Two programmers worked efficiently at ∼35% effort for more than 2 years (4000 hours) and 1 supervisor at 40% effort for more than 2 years (2500 hours). Two experienced data extractors worked almost full-time for more than 2 years to pull the manual record in more than 1000 individuals with SCD for an estimated ∼5350 hours. The data extractors were supervised by a project manager and 2 experienced clinicians for ∼200 hours for more than 2 years.

Myeloablative gene therapy and gene editing clinical trials have expanded curative therapy options for individuals with SCA; however, the FDA mandates contemporaneous comparison groups to demonstrate the relative benefits and harms of these novel therapies. Traditionally, creating contemporaneous cohorts has been done through manually curated SCA registries; however, these registries are expensive to initiate and even more challenging to maintain. As a proof of principle, we demonstrated the feasibility that the medical record data warehouse of a large single medical center could be used to create a contemporaneous cohort of children and adults with SCA. Furthermore, this cohort can be used to assess the mortality rate as a comparator group to experimental cellular therapies. In our cohort, those with HSCTs had the highest survival, although only ∼5% of the total SCA cohort received this curative therapy. Individuals receiving at least 1 year of hydroxyurea therapy, ∼60% of the cohort, had a higher survival than individuals receiving no disease-modifying treatment, ∼23% of the cohort. Importantly, our study cohort of individuals with SCA consisted of ∼63% of the expected proportion of individuals with SCA in large cohort studies.^15,16 

Our automated approach estimates similar overall and median survival as previous hybrid or manual data extraction approaches from other medical centers and our own. Our results for the median age of survival in children and adults with SCA are similar to those of prior studies.^17,18 In one of those studies, the median age of survival (49.3 years) of our automated cohort was similar to that of a recent manually constructed cohort (49.2 years).¹⁷ Also, the overall survival to 18 years of our cohort was 98%, similar to the expected mortality rates in other large cohort studies conducted in high-income settings.^19-21 

Our study showed a similar survival advantage as other studies for adults with SCA prescribed with hydroxyurea compared with those not prescribed with hydroxyurea.^22-25 Previously, using a manual chart review approach, our VUMC team showed that hydroxyurea therapy was not associated with improved survival in adults with SCA;¹⁷ however, those analyses differed from our current approach. Specific differences between the 2 cohorts, which may account for these discordant results may be related to the approach for classifying individuals as being treated with hydroxyurea and the mutually exclusive hierarchal modeling strategy. Our results are similar to observational studies with a median follow up of at least 10 years showing improved survival in children¹⁹ and adults^22,24,26 treated with hydroxyurea compared with individuals not treated with hydroxyurea.

An important clinical finding in our regression analysis is the high HR for death in individuals receiving Medicaid insurance compared with adults who always had another insurance type (eg, Medicare, other dual plans, self-pay, or private insurance). Others have shown that the presence of Medicaid and lower socioeconomic status are associated with mortality in adults with SCA.^27,28 Together, these studies emphasize the importance of screening adults with Medicaid coverage to assess other social determinants of health that may put these individuals at risk for earlier mortality.

Our approach to creating an automated contemporaneous cohort had strengths and weaknesses and lessons learned (Table 3). First, we were able to pull the clinical information over 17 years from a large health care system which includes multiple hospitals. Other individuals in small or large health care systems may be able to replicate or extend our tools to mimic our approach for assessment of mortality rate of their institution in their health care system. Second, our work builds on the extensive prior data of automated cohorts in SCD.^5,7,29-32 However, to our knowledge, the integration of disease-modifying therapies and mortality has not been included in other automated cohort studies. Our tools also utilized more information from the electronic health records than existing approaches from administrative data sets which primarily use billing codes. Third, the initial personal-hours investment does not require significant future annual personnel expenditure to estimate the vital status of the cohort. The most significant challenge for establishing an automated cohort is the substantial initial personnel time, informatics expertise, and required SCD clinical expertise. However, after the contemporaneous cohort has been established, the results can be updated annually at a fraction of the original cost. Because of the comprehensive and efficient approach of the automated cohort, we have already elected to use our SCD automated cohort as a contemporaneous comparison group for individuals with SCD who received curative therapy (5U01HL156620).

The automated and hybrid approach significantly differs in capturing mortality in the cohort. The automated cohort used our data warehouse, which obtains mortality data from our electronic health record and external sources, and found 18 more deaths than the hybrid approach during the same period. The lack of mortality data in the electronic health records was a major limitation in estimating the mortality rate. Otherwise, our hybrid cohort had similar patient characteristics and laboratory values as the automated cohort (supplemental Tables 2-4).

Several limitations occurred in our study to determine the mortality rate in the automated and hybrid cohorts. First, we used prescriptions of hydroxyurea as a marker for hydroxyurea therapy without confirming adherence to therapy. However, prescription history is a reasonable approach for disease-modifying treatment assignment and is similar to previous large cohorts showing hydroxyurea therapy to be associated with a survival benefit.^19,22,24,26 Second, we could not account for hydroxyurea prescriptions outside of VUMC. However, our clinical practice has not significantly changed during the sampling frame, and relatively few adults and fewer children have hydroxyurea prescribed outside of a VUMC provider. Despite these 2 limitations, the preponderance of evidence indicates that in long-term follow-up cohort studies, hydroxyurea therapy decreases mortality as compared with no disease-modifying treatment in adults with SCD. Another limitation in our analysis plan is the fragmentation of care, in which individuals have irregular follow-up visits. Because of the inability to adequately treat and observe the effects of treatment on our primary outcome of mortality, we excluded people with gaps of greater than 3 years. Finally, we could not evaluate the impact of the recent FDA-approved drugs, L-glutamine, crizanlizumab, and voxelotor on mortality. However, over the next decade, our automated cohort will be poised to evaluate the effects of these medications on mortality.

The unique strengths of our study include:

1. Developed for the first time, a contemporaneous cohort of children and adults with SCD who received sustained treatment with disease-modifying therapy from a data warehouse of electronic health records.

2. Unlike previous studies, our approach provided mortality estimates that considered the impact of disease-modifying therapy to estimate mortality. The importance of this strategy cannot be underscored.

3. For the first time, we provided the methodology required to create an SCD electronic cohort from a data warehouse of individuals with SCD.

4. We showed, for the first time, the internal resources required, including the personnel and the necessary time to develop a contemporaneous cohort of children and adults with SCD.

Together, these advantages provide a new opportunity to develop contemporaneous cohorts to evaluate new therapies.

In a large health care system of >3 million individuals, we showed an automated contemporaneous cohort of children and adults with SCA could be generated with a data warehouse to estimate mortality. Members of the SCD and HSCT VUMC teams will use the results of these studies to advise families on the relative benefits and risks of present FDA-approved disease-modifying therapies and emerging strategies to cure SCD in children and adults. For the SCD community, in this article, we also provided computer algorithms to replicate, expand, or create their health care systems contemporaneous SCD cohort to compare to novel curative therapies being evaluated.

This work was partly funded by National Institutes of Health (NIH) Agreement OT3HL147810 as part of the Cure Sickle Cell Initiative. The Cure Sickle Cell Initiative is a collaborative, patient-focused research effort designed to accelerate the advancement of genetic-based cures for SCD. The initiative is funded by the National Heart, Lung, and Blood Institute, NIH. This work was, in part, supported by the National Center for Advancing Translational Sciences, Clinical and Translational Science Awards (grant UL1 TR002243). In addition, this work was supported by National Heart, Lung, and Blood Institute, NIH grant K23HL141447 (R.M.C.).

The views and conclusions contained in this article are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the NIH. Its contents are solely the authors' responsibility and do not necessarily represent the official views of the National Center for Advancing Translational Sciences or the NIH.

Contribution: M.R.D., M.R., K. Wuichet, and R.M.C. designed the study; K. Wuichet, D.L.G., B.H., M.C., J.H., X.N., M.R.D., and R.M.C. collected the data; M.R. performed the analyses; M.R.D., K. Wuichet, M.R., and R.M.C. interpreted the results; R.M.C. and K. Wuichet wrote the manuscript; and all authors reviewed and edited the manuscript.

Conflict-of-interest disclosure: The authors declare no competing financial interests.

Correspondence: Michael R. DeBaun, Department of Pediatrics, Vanderbilt University Medical Center, 2525 West End Ave, Suite 750, Nashville, TN 37203; email: m.debaun@vumc.org.