• COVID-19 does not exacerbate the already high risk of venous thromboembolism among hospitalized patients with sickle cell disease.

Venous thromboembolism (VTE) is a life-threatening complication observed among patients with sickle cell disease (SCD) and also among those with severe COVID-19 infection. Although prior studies show that patients with SCD are at risk of severe COVID-19 illness, it remains unclear if COVID-19 infection further increases VTE risk for this population. We hypothesized that patients with SCD hospitalized for COVID-19 would have higher VTE rates than those hospitalized for other causes. Using electronic health record data from a multisite research network, TriNetX, we identified 2 groups of patients with SCD hospitalized during 2020: (1) with COVID-19 and (2) without COVID-19. We compared VTE rates using risk ratios estimated based on adjusted Poisson regression model with log link and robust error variances. Of the 281 SCD patients hospitalized with COVID-19 and 4873 SCD patients hospitalized without COVID-19 , 35 (12.46%) and 418 (8.58%) had incident VTE within 6 months of the index hospitalization respectively. After adjusting for differences in baseline characteristics, no significant differences in VTE rates within 6 months were found between the 2 groups (adjusted relative risk, 1.06 [95% confidence interval, 0.79-1.41]). These data suggest that hospitalization with COVID-19 does not further increase VTE risk in patients with SCD.

Venous thromboembolism (VTE) is the third leading cause of vascular mortality that contributes substantially to annual health care utilization and long-term morbidity.1,2  Individuals with sickle hemoglobinopathy (ie, that are either heterozygous or homozygous for a single nucleotide substitution of valine for glutamic acid at the sixth position of the β-globin gene) are associated with an increased risk of VTE.3,4  Epidemiologic studies indicate a 2 to 4 times higher risk of VTE among patients with sickle cell disease (SCD) compared with patients with other conditions such as asthma that are similar in gender, age, and hospitalization frequency. This is particularly concerning as VTE in patients with SCD is independently associated with death.5,6 

VTE is also a life-threatening complication in patients infected with the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus that causes COVID-19.7-11  The rates of VTE are especially high among COVID-19 patients with sepsis requiring admission to the intensive care unit.12  Sepsis-associated coagulopathy in patients with COVID-19 results from widespread inflammation that provokes both microvascular and macrovascular thrombosis and leads to multiorgan dysfunction.13  Prior studies indicate that patients with SCD are at greater risk for severe COVID-19 illness.14,15  After adjusting for clinical characteristics, our prior analysis suggested that there were no significant differences in VTE outcomes within 14 days of COVID-19 between Black individuals with SCD and those without SCD.14  However, little is known about the additive risk of VTE due to COVID-19 within the SCD population who have a preexisting thrombosis risk due to a hypercoagulable state.16  We hypothesized that patients with SCD hospitalized for COVID-19 would have higher VTE rates compared with those hospitalized for other indications. To test this hypothesis, we used electronic health record data and made population-level comparisons of VTE rates among patients with SCD hospitalized for COVID-19 vs those hospitalized but did not have COVID-19.

We conducted a retrospective cohort study using data from TriNetX, a research network of electronic health record data from >40 health care organizations across the United States.17  Overall, our study population included 70.4% cases from health care organizations in the South, 17.8% cases from Midwest, 7% from Northeast, and 1.51% from the West. All ages were included in the study. We identified 2 groups: (1) patients with SCD hospitalized for COVID-19 or for another diagnosis but tested positive for SARS-CoV-2 (for the sake of brevity, referred as patients with SCD hospitalized with COVID-19) during calendar year 2020 and (2) patients with SCD hospitalized without COVID-19 during 2020. SCD patients were identified using a previously validated algorithm18,19  that was slightly modified to exclude subjects with International Classification of Diseases (ICD) diagnosis code of sickle cell trait to enhance specificity. All types of SCD as specified by ICD codes were included in the study population (D57.0*, D57.1*, D57.2*, D57.4*, D57.8*). A table of laboratory values of the cohort identified supports that these were patients with SCD (supplemental Table 1). COVID-19 cases were identified using the diagnosis codes for COVID-19 infection or a positive SARS-CoV-2 diagnostic test. We considered cases to be hospitalized with COVID-19 if hospitalization occurred within 14 days of COVID-19 diagnosis or a positive result on COVID-19 testing. The comparator group of patients with SCD hospitalized for other indications excluded patients who had COVID-19 at any point in time. The study was exempt from institutional review board approval. It was conducted in accordance with the Declaration of Helsinki.

Index hospitalization

The date of COVID-19 diagnosis and the first hospitalization in year 2020 for patients with SCD hospitalized with COVID-19 and patients with SCD hospitalized without COVID-19, respectively.

Outcome

VTE was identified using ICD version 10 codes (supplemental Table 2) within 1 month, 3 months, and 6 months of the index event. This included pulmonary embolism (PE) and deep vein thrombosis (DVT). The ICD version 10 codes are sensitive to identify thromboembolic events.20,21  The ICD codes have also previously been used by other published studies to identify VTE outcomes among individuals with SCD.6,22 

Statistical analysis

We compared demographic and clinical characteristics between the 2 groups using t tests or the χ2 test. Poisson regression models with log link and robust error variance were used to estimate the relative risk (RR) and 95% confidence intervals (CI). The outcomes of VTE incidence within 1 month, 3 months, and 6 months were modeled separately. The models included baseline characteristics, which were significantly different between the 2 groups as covariates. A P value of <.05 was considered statistically significant. As a sensitivity analysis, we restricted our cohort to those identified as having sickle cell anemia (hemoglobin SS or Sβ0) and repeated the analysis described above.

Our study included 281 SCD patients with COVID-19 and 4873 SCD patients without COVID-19 hospitalized during year 2020. Patients with SCD hospitalized with COVID-19 were older, and a higher proportion of them had a history of hypertension, acute or chronic kidney disease, obesity, and prior DVT/PE (Table 1).

Table 1.

Characteristics of patients with sickle cell disease hospitalized with and without COVID-19

CharacteristicTotal
(N = 5154)
SCD COVID (N = 281)SCD, NO COVID (N = 4873)P value
Age, mean (SD) 28.2 (17.6) 31.3 (18.0) 28.0 (17.6) .0021* 
<18 y of age, % 32.0% 22.4% 32.5% .004* 
Females, % 54.2 51.2 54.3 .3146 
SCD subtype Hemoglobin SS or Sβ0, % 77.2 77.9 77.1 .3318 
≥3 hospitalization in past 3 y, % 59.1 62.8 58.8 .2511 
Prior history of comorbidities, % 
Asthma 26.5 26.3 26.5 .9594 
Hypertension 26.2 35.2 25.7 .0004* 
Heart failure 10.9 14.2 10.7 .0620 
Ischemic heart disease 5.6 7.8 5.5 .0959 
Diabetes mellitus 8.7 9.6 8.7 .5835 
Acute/chronic kidney disease 20.0 29.2 19.5 <.0001* 
Liver disease 10.6 13.5 10.4 .1008 
Obesity 13.5 20.6 13.1 .0003* 
Chronic obstructive pulmonary disease 4.4 6.4 4.2 .0852 
Cerebral infarction, % 9.1 9.3 9.1 .9456 
Prior DVT and/or pulmonary embolism 15.8 22.8 15.3 .0009* 
ICU admissions after index event 
Within 30 d of index event, % 4.9 9.6 4.7 <.001* 
Within 90 d of index event, % 5.7 11.0 5.3 <.001* 
Within 180 d of index event, % 6.8 12.8 6.5 <.001* 
CharacteristicTotal
(N = 5154)
SCD COVID (N = 281)SCD, NO COVID (N = 4873)P value
Age, mean (SD) 28.2 (17.6) 31.3 (18.0) 28.0 (17.6) .0021* 
<18 y of age, % 32.0% 22.4% 32.5% .004* 
Females, % 54.2 51.2 54.3 .3146 
SCD subtype Hemoglobin SS or Sβ0, % 77.2 77.9 77.1 .3318 
≥3 hospitalization in past 3 y, % 59.1 62.8 58.8 .2511 
Prior history of comorbidities, % 
Asthma 26.5 26.3 26.5 .9594 
Hypertension 26.2 35.2 25.7 .0004* 
Heart failure 10.9 14.2 10.7 .0620 
Ischemic heart disease 5.6 7.8 5.5 .0959 
Diabetes mellitus 8.7 9.6 8.7 .5835 
Acute/chronic kidney disease 20.0 29.2 19.5 <.0001* 
Liver disease 10.6 13.5 10.4 .1008 
Obesity 13.5 20.6 13.1 .0003* 
Chronic obstructive pulmonary disease 4.4 6.4 4.2 .0852 
Cerebral infarction, % 9.1 9.3 9.1 .9456 
Prior DVT and/or pulmonary embolism 15.8 22.8 15.3 .0009* 
ICU admissions after index event 
Within 30 d of index event, % 4.9 9.6 4.7 <.001* 
Within 90 d of index event, % 5.7 11.0 5.3 <.001* 
Within 180 d of index event, % 6.8 12.8 6.5 <.001* 

ICU, intensive care unit; SD, standard deviation.

*

P-values <0.05 were considered statistically significant. 

Of the SCD patients hospitalized with COVID-19, 20 (7.12%), 28 (9.96%), and 35 (12.46%) had an incident VTE within 1, 3, and 6 months of COVID-19 diagnosis, respectively. In the comparator group, 257 (5.27%), 332 (6.81%), and 418 (8.58%) had an incident VTE within 1, 3, and 6 months of index hospitalization, respectively. There were 13 (4.6%) in the COVID-19 group and 182 (3.7%) in the comparator group who experienced VTE during their index hospitalization. Unadjusted analysis reflected no significant differences in VTE incidence at 1 month post–index event between the 2 groups (RR, 1.35; 95% CI, 0.87-2.01; P = .1806), whereas VTE incidence at 3 months (RR, 1.46; 95% CI, 1.01-2.11; P = 0.0420) and 6 months (RR, 1.45; 95% CI, 1.05-2.00; P = 0.0237) after hospitalization was significantly higher among patients with SCD hospitalized for COVID-19. However, after adjusting for age, history of hypertension, acute or chronic kidney disease, obesity, and prior VTE/PE history, there were no significant differences in VTE risk between the “SCD COVID-19” and the “SCD, no COVID-19” groups (Table 2). The model estimates showed that age, prior history of VTE/PE, and prior history of acute/chronic kidney disease were significantly associated with VTE risk at all time points. The history of prior VTE/PE had a large effect size, which is consistent with the observation of a high VTE recurrence rate in patients with SCD.22  Similar estimates were observed when restricting our study population to those classified as having sickle cell anemia.

Table 2.

Model parameters and estimates for outcome of VTE incidence at 1 mo, 3 mo, and 6 mo post–index event

Table 2a. Outcome: VTE incidence at 1 mo
ParameterEstimateStandard errorP-valueAdjusted RR (95% CI)
Intercept −4.7962 0.1626 <.0001* — 
Patients with SCD hospitalized with COVID (yes vs no) −0.0556 0.1964 0.7773 0.95 (0.64-1.39) 
Age 0.0081 0.0035 0.0218* 1.01 (1.00-1.02) 
Prior history of hypertension (yes vs no) −0.1192 0.1268 0.3473 0.89 (0.69-1.14) 
Prior history of obesity (yes vs no) 0.2317 0.1221 0.0578 1.26 (0.99-1.60) 
Prior history of VTE/pulmonary embolism (yes vs no) 3.0830 0.1700 <.0001* 21.82 (15.64-30.45) 
Prior history of acute/chronic kidney disease (yes vs no) 0.2592 0.1279 0.0428* 1.30 (1.01-1.67) 
Table 2a. Outcome: VTE incidence at 1 mo
ParameterEstimateStandard errorP-valueAdjusted RR (95% CI)
Intercept −4.7962 0.1626 <.0001* — 
Patients with SCD hospitalized with COVID (yes vs no) −0.0556 0.1964 0.7773 0.95 (0.64-1.39) 
Age 0.0081 0.0035 0.0218* 1.01 (1.00-1.02) 
Prior history of hypertension (yes vs no) −0.1192 0.1268 0.3473 0.89 (0.69-1.14) 
Prior history of obesity (yes vs no) 0.2317 0.1221 0.0578 1.26 (0.99-1.60) 
Prior history of VTE/pulmonary embolism (yes vs no) 3.0830 0.1700 <.0001* 21.82 (15.64-30.45) 
Prior history of acute/chronic kidney disease (yes vs no) 0.2592 0.1279 0.0428* 1.30 (1.01-1.67) 
Table 2b. Outcome: VTE incidence at 1 mo
ParameterEstimateStandard errorP-valueAdjusted RR (95% CI)
Intercept −4.3676 0.1265 <.0001* — 
Patients with SCD hospitalized with COVID (yes vs no) 0.0478 0.1655 0.7726 1.05 (0.76-1.45) 
Age 0.0106 0.0029 0.0003* 1.01 (1.01-1.02) 
Prior history of hypertension (yes vs no) −0.1395 0.1106 0.2071 0.87 (0.70-1.08) 
Prior history of obesity (yes vs no) 0.2409 0.1036 0.0200 1.27 (1.04-1.56) 
Prior history of VTE/pulmonary embolism (yes vs no) 2.7530 0.1359 <.0001* 15.69 (12.02-20.48) 
Prior history of acute/chronic kidney disease (yes vs no) 0.2577 0.1097 0.0188* 1.29 (1.04-1.60) 
Table 2b. Outcome: VTE incidence at 1 mo
ParameterEstimateStandard errorP-valueAdjusted RR (95% CI)
Intercept −4.3676 0.1265 <.0001* — 
Patients with SCD hospitalized with COVID (yes vs no) 0.0478 0.1655 0.7726 1.05 (0.76-1.45) 
Age 0.0106 0.0029 0.0003* 1.01 (1.01-1.02) 
Prior history of hypertension (yes vs no) −0.1395 0.1106 0.2071 0.87 (0.70-1.08) 
Prior history of obesity (yes vs no) 0.2409 0.1036 0.0200 1.27 (1.04-1.56) 
Prior history of VTE/pulmonary embolism (yes vs no) 2.7530 0.1359 <.0001* 15.69 (12.02-20.48) 
Prior history of acute/chronic kidney disease (yes vs no) 0.2577 0.1097 0.0188* 1.29 (1.04-1.60) 
Table 2c. Outcome: VTE incidence at 6 mo
ParameterEstimateStandard errorP-valueAdjusted RR (95% CI)
Intercept −3.9230 0.1029 <.0001*  
Patients with SCD hospitalized with COVID (yes vs no) 0.0572 0.1481 0.6995 1.06 (0.79-1.41) 
Age 0.0093 0.0025 0.0002* 1.01 (1.00-1.01) 
Prior history of hypertension (yes vs no) −0.1334 0.0936 0.1540 0.88 (0.73-1.05) 
Prior history of obesity (yes vs no) 0.1975 0.0896 0.0275* 1.22 (1.02-1.45) 
Prior history of VTE/pulmonary embolism (yes vs no) 2.5353 0.1124 <.0001* 12.62 (10.12-15.73) 
Prior history of acute/chronic kidney disease (yes vs no) 0.2858 0.0943 0.0024* 1.33 (1.11-1.60) 
Table 2c. Outcome: VTE incidence at 6 mo
ParameterEstimateStandard errorP-valueAdjusted RR (95% CI)
Intercept −3.9230 0.1029 <.0001*  
Patients with SCD hospitalized with COVID (yes vs no) 0.0572 0.1481 0.6995 1.06 (0.79-1.41) 
Age 0.0093 0.0025 0.0002* 1.01 (1.00-1.01) 
Prior history of hypertension (yes vs no) −0.1334 0.0936 0.1540 0.88 (0.73-1.05) 
Prior history of obesity (yes vs no) 0.1975 0.0896 0.0275* 1.22 (1.02-1.45) 
Prior history of VTE/pulmonary embolism (yes vs no) 2.5353 0.1124 <.0001* 12.62 (10.12-15.73) 
Prior history of acute/chronic kidney disease (yes vs no) 0.2858 0.0943 0.0024* 1.33 (1.11-1.60) 
*

P-values <0.05 were considered statistically significant.

To the best of our knowledge, this is the first study to evaluate VTE risk following COVID-19 in patients with SCD. Overall, these data suggest that for patients with SCD hospitalized with COVID-19 infection, anticoagulation management decisions should be based on individual risk factors, including age, obesity, prior history of acute/chronic kidney disease, and, most importantly, a prior history of VTE. Interestingly, a substantial proportion of VTE cases in our study population occurred in July (20%), November (22.8%) and December (20%) of 2020, when widespread use of “intermediate-dosed” anticoagulation therapy was general clinical practice to prevent VTE in patients with COVID-19.23  However, the lack of information on both dosage and duration of anticoagulation therapy administration limited our ability to determine the impact of this exposure on study outcomes. Updated clinical anticoagulation management guidelines for individuals hospitalized for COVID 19 suggest that in certain subgroups, therapeutic anticoagulation may improve clinical outcomes.24  Thus, additional studies, especially those conducted in patients with SCD affected with COVID-19, could help guide prophylactic and therapeutic anticoagulation management.

The strengths of this study lie in the inclusion of data from multiple health care organizations, thereby representing hospital settings within the United States that permit some degree of generalizability. However, certain limitations exist. These include the possibility of missing incident VTE in patients seeking treatment at a different health care institution not part of the TriNetX network. However, >70% of our study population had clinical encounters in 2021 at institutions within the TriNetX network, suggesting that health care fragmentation and failure to detect new VTE cases was minimal. Importantly, the overall ICD codes have a high sensitivity to identify acute VTE outcomes among hospitalized patients.20,21  Coding is different for acute vs chronic venous thrombosis. However, the lack of patient identifiers in this administrative database precluded access to and review of imaging studies for VTE diagnosis, and therefore, we cannot differentiate between prior residual VTE and new onset recurrent VTE,

In conclusion, we did not find significant differences in VTE rates between SCD patients hospitalized with COVID-19 and those hospitalized without COVID-19. These data provide support for the application of current general clinical anticoagulation guidelines for COVID-19 in the general population to patients with SCD. Future prospective studies controlled for anticoagulant therapy exposure may provide more direct evidence to guide antithrombotic management in this unique population.

Contribution: A. Singh and A.S.S. designed research; A. Singh analyzed data; A.M.B. and T.W. provided critical content expertise; and A. Singh, A.M.B., and A.S.S. contributed to writing of the manuscript.

Conflict-of-interest disclosure: The authors declare no competing financial interests.

Correspondence: Ashima Singh, Medical College of Wisconsin, 8701 W Watertown Plank Road, Suite 3050, Milwaukee, WI 53226; e-mail: ashimasingh@mcw.edu.

1.
Nicholson
M
,
Chan
N
,
Bhagirath
V
,
Ginsberg
J
.
Prevention of venous thromboembolism in 2020 and beyond
.
J Clin Med.
2020
;
9
(
8
):
2467
.
2.
Beckman
MG
,
Hooper
WC
,
Critchley
SE
,
Ortel
TL
.
Venous thromboembolism: a public health concern
.
Am J Prev Med.
2010
;
38
(
4 Suppl
):
S495
-
S501
.
3.
Folsom
AR
,
Tang
W
,
Roetker
NS
, et al
.
Prospective study of sickle cell trait and venous thromboembolism incidence
.
J Thromb Haemost.
2015
;
13
(
1
):
2
-
9
.
4.
Little
I
,
Vinogradova
Y
,
Orton
E
,
Kai
J
,
Qureshi
N
.
Venous thromboembolism in adults screened for sickle cell trait: a population-based cohort study with nested case-control analysis
.
BMJ Open.
2017
;
7
(
3
):
e012665
.
5.
Naik
RP
,
Streiff
MB
,
Haywood
C
Jr
,
Nelson
JA
,
Lanzkron
S
.
Venous thromboembolism in adults with sickle cell disease: a serious and under-recognized complication
.
Am J Med.
2013
;
126
(
5
):
443
-
449
.
6.
Brunson
A
,
Lei
A
,
Rosenberg
AS
,
White
RH
,
Keegan
T
,
Wun
T
.
Increased incidence of VTE in sickle cell disease patients: risk factors, recurrence and impact on mortality
.
Br J Haematol.
2017
;
178
(
2
):
319
-
326
.
7.
Malas
MB
,
Naazie
IN
,
Elsayed
N
,
Mathlouthi
A
,
Marmor
R
,
Clary
B
.
Thromboembolism risk of COVID-19 is high and associated with a higher risk of mortality: a systematic review and meta-analysis
.
EClinicalMedicine.
2020
;
29
:
100639
.
8.
Kasinathan
S
,
Mohammad Ashraf
H
,
Minkowitz
S
,
Adeyinka
A
,
Bailey-Correa
K
.
COVID-19 infection and acute pulmonary embolism in an adolescent female with sickle cell disease
.
Cureus.
2020
;
12
(
12
):
e12348
.
9.
Mahajan
P
,
Dass
B
,
Radhakrishnan
N
,
McCullough
PA
.
COVID-19-associated systemic thromboembolism: a case report and review of the literature
.
Cardiorenal Med.
2020
;
10
(
6
):
462
-
469
.
10.
Rout
A
,
Chan
A
.
A case of COVID-19 complicated by venous thromboembolism despite being on prophylactic anticoagulation
.
J Med Cases.
2020
;
11
(
7
):
189
-
191
.
11.
Salam
S
,
Mallat
J
,
Elkambergy
H
.
Acute high-risk pulmonary embolism requiring thrombolytic therapy in a COVID-19 pneumonia patient despite intermediate dosing deep vein thromboprophylaxis
.
Respir Med Case Rep.
2020
;
31
:
101263
.
12.
Tan
BK
,
Mainbourg
S
,
Friggeri
A
, et al
.
Arterial and venous thromboembolism in COVID-19: a study-level meta-analysis
.
Thorax.
2021
:
76
(
10
):
970
-
979
.
13.
Iba
T
,
Levy
JH
,
Connors
JM
,
Warkentin
TE
,
Thachil
J
,
Levi
M
.
The unique characteristics of COVID-19 coagulopathy
.
Crit Care.
2020
;
24
(
1
):
360
.
14.
Singh
A
,
Brandow
AM
,
Panepinto
JA
.
COVID-19 in individuals with sickle cell disease/trait compared with other Black individuals
[published correction appears in Blood Adv. 2022;6(8):2535].
Blood Adv.
2021
;
5
(
7
):
1915
-
1921
.
15.
Panepinto
JA
,
Brandow
A
,
Mucalo
L
, et al
.
Coronavirus disease among persons with sickle cell disease, United States, March 20-May 21, 2020
.
Emerg Infect Dis.
2020
;
26
(
10
):
2473
-
2476
.
16.
Shet
AS
,
Lizarralde-Iragorri
MA
,
Naik
RP
.
The molecular basis for the prothrombotic state in sickle cell disease
.
Haematologica.
2020
;
105
(
10
):
2368
-
2379
.
17.
TriNetX
. Real-World Data. https://trinetx.com/real-world-data/. Accessed 8 December 2021.
18.
Singh
A
,
Mora
J
,
Panepinto
JA
.
Identification of patients with hemoglobin SS/Sβ0 thalassemia disease and pain crises within electronic health records
.
Blood Adv.
2018
;
2
(
11
):
1172
-
1179
.
19.
Michalik
DE
,
Taylor
BW
,
Panepinto
JA
.
Identification and validation of a sickle cell disease cohort within electronic health records
.
Acad Pediatr.
2017
;
17
(
3
):
283
-
287
.
20.
Fang
MC
,
Fan
D
,
Sung
SH
, et al
.
Validity of using inpatient and outpatient administrative codes to identify acute venous thromboembolism: the CVRN VTE study
.
Med Care.
2017
;
55
(
12
):
e137
-
e143
.
21.
Lawrence
K
,
Joos
C
,
Jones
AE
,
Johnson
SA
,
Witt
DM
.
Assessing the accuracy of ICD-10 codes for identifying acute thromboembolic events among patients receiving anticoagulation therapy
.
J Thromb Thrombolysis.
2019
;
48
(
2
):
181
-
186
.
22.
Brunson
A
,
Keegan
T
,
Mahajan
A
,
White
R
,
Wun
T
.
High incidence of venous thromboembolism recurrence in patients with sickle cell disease
.
Am J Hematol.
2019
;
94
(
8
):
862
-
870
.
23.
Connors
JM
,
Levy
JH
.
COVID-19 and its implications for thrombosis and anticoagulation
.
Blood.
2020
;
135
(
23
):
2033
-
2040
.
24.
National Institutes of Health
. Coronavirus Disease 2019 (COVID-19) Treatment Guidelines. https://www.covid19treatmentguidelines.nih.gov/. Accessed 18 January 2022.

Author notes

Contact the corresponding author for data sharing: ashimasingh@mcw.edu.

The full-text version of this article contains a data supplement.

Supplemental data