Validation of the Khorana score for predicting venous thromboembolism in 40 218 patients with cancer initiating chemotherapy

Cancer incidence is rising, and venous thromboembolism (VTE) remains a common and potentially fatal complication among patients with cancer.^1,2  Cancer-associated VTE poses a burden on health care systems, disrupts cancer treatment, and contributes to emotional distress and physical discomfort.^3,4  Patients with VTE also carry a substantial risk of recurrence.⁵ 

Primary prophylaxis with low-molecular-weight heparin reduces the risk of VTE, but the associated bleeding risk and burden of daily injections have hampered its implementation into routine clinical practice.⁶  Risk stratification tools have therefore been developed aiming to identify individuals at high risk of VTE, among whom the risk-benefit ratio is favorable.⁷  The most widely recommended is the Khorana score, originally designed to assess the risk of VTE in patients initiating chemotherapy.⁸  It is a simple, point-based risk score attributing points to certain cancer types, routine laboratory test values, and body mass index (see supplemental Table 1 for details). The Khorana score is widely recommended by guidelines and has been used to define 2 randomized trial populations investigating the role of direct oral anticoagulants vs placebo for primary prevention of VTE in ambulatory cancer patients.^9,10  Both studies demonstrated a reduction in the incidence of VTE but with a higher risk of major bleeding. Accordingly, the net clinical benefit of anticoagulation for primary VTE prophylaxis continues to be debated, and the optimal treatment decision threshold for anticoagulation when using the Khorana score is not well defined.^11,12  Some guidelines recommend using a Khorana score of ≥3 as a potential indication for anticoagulation,¹³  while most recommend a score threshold of ≥2 as used in the randomized trials.^9,10,14-16  Indeed, cumulative incidences of VTE vary widely for similar score levels when compared across validation studies, cancer types, and cancer therapies, indicating a need for further validation or refinement of the Khorana score.¹⁷  Also, some previous studies have ignored competing risk of death, which may lead to an overestimation of VTE risk and subsequently overoptimistic claims about the possible absolute risk reduction in a real-world setting.^9,10,18-20 

We aimed to assess the clinical potential of using the Khorana score to stratify an unselected sample of Danish ambulatory cancer patients initiating chemotherapy according to the risk of incident VTE, both overall and by cancer subtypes. We also aimed to quantify the impact of ignoring competing risk of death when estimating absolute risk.

This is a register-based cohort study of Danish ambulatory cancer patients initiating chemotherapy. Administrative nationwide health registries were used to characterize patients and identify outcomes.²¹  Information was obtained using a unique personal identification number given to all Danish residents at birth or immigration. This study included data from 1) the Civil Registration System, storing information about age, sex, vital and migration status,²¹  2) the National Patient Register containing information on discharge diagnosis for hospitalized patients since 1977,²²  3) the National Prescription Registry, which holds information on all reimbursed prescriptions from Danish pharmacies since 1995,²³  and 4) the Register of Laboratory Results for Research database, which contains laboratory values for most regions in Denmark.²⁴  Codes used to define the study population, covariates, and outcomes are available in supplemental Table 2.

The study population consisted of ambulatory patients initiating chemotherapy for cancer at Danish hematology or oncology departments in Denmark from 2010 through mid-2018. In accordance with the original Khorana derivation cohort, patients were followed from the day of initiation of chemotherapy.⁸  Patients who had claimed a prescription for any anticoagulant drug within 1 year or who had an existing potential indication for anticoagulation (previous atrial fibrillation, VTE, or mechanical heart valves) were excluded. Chemotherapy was defined using Danish examination and procedure codes registered in the National Patient Register, codes which have a positive predictive value of 95%.^25,26  The type of cancer was defined by the specific cancer diagnosis registered in combination with the procedure code for chemotherapy. If patients had more than 1 cancer diagnosis in relation to the procedure code for chemotherapy, preference was given to the most high-risk diagnosis as defined by the Khorana score. If patients had no cancer diagnosis prior to chemotherapy, inclusion in the study population was allowed only if they were registered with a cancer diagnosis within 30 days after chemotherapy. Patients with primary brain cancer and myeloma were also excluded since they were not included in the original Khorana study.⁸  Not all Danish regions reported laboratory values to the Register of Laboratory Results for Research database throughout the entire study period, and patients with no laboratory values available were therefore excluded.²⁴ 

The Khorana score is a guideline-recommended point-based risk score used to estimate the risk of incident VTE in ambulatory cancer patients.⁸  Points are attributed according to cancer type (stomach or pancreas [2 points], lung, lymphoma, gynecologic, bladder, or testicular cancer [1 point]), platelet count ≥350 × 10⁹/L (1 point), hemoglobin level <10 g/dL (equivalent to <6.21 mmol/L) or using erythrocyte growth factors (1 point), leukocyte count >11 × 10⁹/L (1 point), and body mass index ≥35 kg/m² (1 point). Laboratory values were pretreatment values obtained prior to medical cancer treatment. Information on cancer type was obtained using the National Patient Register. Body mass index ≥35 kg/m² was defined using the International Classification of Diseases (ICD) 10 codes. Data on height and weight were also extracted from the regional VARiS MedOncology database, which stores information about ambulatory cancer patients treated at the North Region Denmark center for oncology based at Aalborg University Hospital. This was done to evaluate the magnitude of underestimation of patients with BMI ≥35 kg/m² using ICD codes alone.²⁷  Platelet, hemoglobin, and leukocyte count were extracted from the Register of Laboratory Results for Research database.²⁴  Information on erythropoiesis-stimulating agents was extracted by treatment codes in the National Patient Registry. Information on anticoagulation use was obtained from the National Prescription Registry.

Patients were followed until the occurrence of VTE (pulmonary embolism or deep vein thrombosis, both proximal and distal, as ICD codes do not permit a clear distinction), emigration, death, or end-of-follow-up, whichever occurred first. VTE was defined as a primary or secondary in- or outpatient diagnosis given in combination with a relevant imaging procedure, ensuring a positive predictive value of approximately 91%.²⁸  Information on death and emigration was obtained from the Civil Registration System. Patients were followed for a maximum period of 6 months in the primary analysis to align with guideline recommendations and follow-up from randomized clinical trials.^9,10,29  As a secondary sensitivity outcome, we defined a broad definition of VTE comprising pulmonary embolism, deep vein thrombosis, thrombophlebitis, and thrombosis located in the retinal, cerebral, portal, caval, or renal vein, and without requiring an imaging procedure in combination with the diagnosis.

The Khorana score was evaluated in 3 different versions: 1) according to the original categorization of low risk (score 0), intermediate risk (score 1 to 2) and high risk (score ≥3),⁸  2) by individual score levels, and 3) according to the threshold used to define the populations in the AVERT and CASSINI trials (score ≥2), which also reflects the treatment threshold recommended by most guidelines.^9,10,29  Baseline characteristics were presented overall and stratified according to Khorana score levels, with proportions for categorical values and median and interquartile range for continuous variables. The number of outcome events was reported overall and by Khorana score levels and categories. The 6-month cumulative incidence of VTE was calculated using the cumulative incidence function, which takes into account the competing risk of death.³⁰  Incidence curves are presented graphically both overall and by Khorana risk score levels.

The associations between the individual score components and combinations thereof with the outcomes were evaluated using a competing risk model taking into account competing risk of death, the Fine and Gray method, and presented as subdistribution hazard ratios.³¹ 

The performance of the Khorana score was assessed across a priori defined cancer types since concerns have been raised about the usefulness in some types.³²  These included gastrointestinal tract, hepatobiliary or pancreatic, lung, breast, urologic, gynecologic, and hematologic cancer.

The cumulative incidence of VTE was also estimated using the Kaplan-Meier function to assess the degree of overestimation of risk compared with the more appropriate cumulative incidence function considering death as a competing event.³³ 

In another sensitivity analysis, patients were censored if diagnosed with atrial fibrillation or if claiming a prescription for an anticoagulant drug during follow-up. A sensitivity analysis restricting the latest study inclusion to mid-2016 was also performed. This was done to evaluate whether low-molecular-weight heparin used from 2017 (where primary outpatient thromboprophylaxis was first mentioned in Danish guidelines) had contributed to artificially low risk estimates. Many patients were excluded due to missing laboratory values, and the overall risk of VTE was reported to evaluate whether those excluded were systematically different from the study population.

Discrimination was evaluated by C-statistics specifically for competing risk data.³⁴  Analyses were performed using Stata version 16 and R statistical software version 3.5.2.

We identified 40 218 patients initiating chemotherapy from 2010 through July 2018 who met the study inclusion criteria (see Figure 1). Baseline characteristics are presented in Table 1. The median age was 65 years, and the majority were females (55.4%). When categorized according to the original Khorana classification, 35.4% were considered low risk (Khorana score 0), 53.6% intermediate risk (Khorana score 1 to 2), and 10.9% as high risk (Khorana score ≥3). Among the cancer types attributed points in the Khorana score, the predominant type was lung cancer (16.3%) and the least prevalent testicular cancer (0.9%). Most patients had a cancer type not assigned points in the Khorana score (59.3%). The proportions of solid and hematologic cancers were 88.0% and 12.0%, respectively.

In the unselected cohort of patients who have undergone medical cancer treatment in the North Region of Denmark and therefore registered in the VARiS MedOncology database, 5.7% of patients had a body mass index >35 kg/m². Using ICD codes alone, we identified 1.2%, indicating that approximately 4.5% of the cohort may have had their Khorana score underestimated by 1 point.

Cumulative incidences of VTE are presented in Table 2 and Figure 2. During 6 months of follow-up, 1000 VTE events occurred. In the same period, 5016 patients died, corresponding to a 6-month mortality of 12.5%. Across Khorana score levels and cancer types, 6-month mortality ranged from 1.4% among those with breast cancer and low Khorana scores to 39.8% among patients with lung cancer and Khorana score ≥3. No patients were lost to follow-up due to emigration.

The overall cumulative incidence of VTE considering competing risk of death was 2.5% (95% CI, 2.3-2.6) and increased according to Khorana score levels and categories.

Among patients eligible for anticoagulation according to guidelines (Khorana score ≥2), the 6-month cumulative incidence of VTE was 3.6% (95% CI, 3.3-3.9). Of 1000 VTE events, 47.1% occurred among patients with Khorana scores ≥2.

The 6-month C-statistic for the Khorana score was 0.60 (95% CI, 0.58-0.62), meaning the probability was 60% that the baseline Khorana score level was higher for patients experiencing VTE than for those who did not experience the event (ie, who were event-free and either died within or survived the 6 months).³⁵ 

Cumulative incidence below and above the guideline-recommended treatment threshold of Khorana score ≥2 stratified by cancer type are presented in Figure 3. The capacity of this score threshold for separating patients into risk strata was limited for patients with hepato-biliary or pancreatic cancer, lung cancer, and gynecologic cancer. Among patients with gastrointestinal, urologic, and hematologic cancer, a clear visual separation of patients into lower and higher risk categories was evident. Patients with breast cancer with a Khorana score ≥2 were too few to present stratified cumulative incidence curves. For the entire breast cancer group, the 6-month risk was 0.7% (95% CI, 0.5-0.9), and the subdistribution hazard ratio for those with Khorana score ≥2 vs 0 to 1 was 2.39 (0.86-6.67). Among the remaining patients with Khorana scores ≥2, the 6-month risk was highest in patients with urologic cancer (5.8% [95% CI, 3.9-8.2]) and lowest in patients with breast cancer (1.5% [95% CI, 0.5-3.5]). Cumulative incidences by cancer subtype stratified by the original Khorana categorization are available in supplemental Table 3.

Associations between individual components of the Khorana score and the risk of VTE are presented in Table 3. A higher Khorana score level was associated with an increasingly higher risk of VTE. After adjustment for other Khorana score components, all individual components except for high platelet count were numerically associated with a higher risk of VTE, although confidence intervals were wide for some components.

Estimated cumulative incidence using the Kaplan-Meier function resulted in a relative overestimation compared with the competing risk model (see Figure 4). In the overall population, the overestimation was negligible in the low Khorana score levels, but up to 19.1% in the Khorana score level with the highest competing mortality risk. When stratified by cancer type, the largest overestimation by Kaplan-Meier was seen among patients with urological cancer and Khorana scores ≥3, which was 23.0% compared with estimates considering competing risk of death. The cumulative incidence with the Kaplan-Meier estimator among patients with Khorana score ≥2 was 4.0% vs 3.6% in the competing risk analysis.

Implementation of a broad definition of VTE resulted in higher cumulative incidences compared with the strict definition (see supplemental Table 4 for details).

Censoring patients diagnosed with atrial fibrillation or claiming a prescription for an anticoagulant drug during follow-up yielded similar 6-month VTE risk estimates as in the main analysis, with risks of 1.5% for low-risk, 2.7% for intermediate-risk, and 4.0% for high-risk patients.

Risk estimates were also similar when limiting the observation period to before 2017, with risks of 1.4%, 2.7%, and 4.1% for low-, intermediate-, and high-risk categories, respectively.

The overall 6-month risk of VTE was identical among those included in the study population and those excluded due to missing laboratory values (both 2.5%).

In a large, unselected population of ambulatory cancer patients, the Khorana score was able to risk-stratify patients according to the 6-month risk of incident VTE, but only for patients with certain cancer subtypes. In general, absolute risks of VTE were lower than reported in previous validation studies and randomized trials. We also observed that the choice of statistical methodology had a substantial impact on estimated risk.

There is mounting evidence supporting that anticoagulation, either low-molecular-weight heparin or direct oral anticoagulants, has the capacity to prevent VTE in ambulatory cancer patients.^6,9,10,36  The benefit of primary prophylaxis differs for patients with moderate- (2) or high-risk (≥3) Khorana scores, with the highest benefit for highest risk patients.³⁷  The effectiveness and safety are consistent regardless of whether patients have newly diagnosed or recurrent cancer and for those with and without metastatic disease.^38,39  However, due to uncertainties about the net clinical benefit, mainly related to bleeding risk, guideline recommendations remain vague, and primary thromboprophylaxis is therefore rarely used in clinical practice.^40-43 

The Khorana score has been validated previously. However, the present study was larger than previous validation studies reporting 6-month incidence combined, allowing for stratified analyses with consistent methodology across cancer subgroups.^17,44  A comprehensive meta-analysis of Khorana validation studies demonstrated that the weighted 6-month incidence of VTE in cancer patients was 11% among patients with Khorana scores ≥3, which was substantially higher than the 4.1% in the present study.¹⁷  Importantly, risk estimates of the included studies ranged from 0% to 44.4%, showing that the Khorana score is far from a well-calibrated risk score.^17,45  Of note, the meta-analysis did not report how the cumulative incidences were calculated, including whether competing risk models were used. Several studies have depicted or reported cumulative incidence by Khorana levels using the Kaplan-Meier function, including the pivotal randomized trials,^9,10,18-20  but failing to account for competing risk of death may overestimate incidence.⁴⁶  In the present study, we demonstrated that inappropriate use of the Kaplan-Meier method overestimated risk estimates up to 23% among those with the highest competing mortality risk. Overestimation of risk also leads to lower numbers needed to treat and consequently overoptimistic claims of the expected absolute benefit of treatment.⁴⁷  For example, assuming a 50% relative risk reduction among patients with Khorana score ≥2 as approximately reported in the AVERT trial, the number needed to treat to prevent 1 VTE event in the present study would be 50 based on the Kaplan-Meier estimate and 56 based on the cumulative incidence function. Nevertheless, the reported cumulative incidences from the present study should also be interpreted with caution, as also discussed below in the “Strengths and limitations” section.

The C-statistic for the Khorana score in this study was poor at 0.60, but it must be interpreted in the context of its limitations as a measure of overall risk score performance when the purpose of the score is guiding treatment decisions in relation to a given score threshold.⁴⁸  In this case, calibration (ie, how well the observed risk estimates are consistent with estimates from previous studies) is of higher clinical relevance. Absolute risk estimates in the present study were markedly lower than in previous studies. In the AVERT trial, the reported 6-month risk of VTE in the placebo arm was 10.2%, and the corresponding estimate in CASSINI was 8.8%.^9,10  Even when applying a broad outcome definition and calculating absolute risks using the Kaplan-Meier function, the corresponding absolute risks in this study were substantially lower than in the trial populations. Of note, inclusion criteria in the present study and the randomized trials were not identical (eg, we excluded patients with a history of VTE who may carry higher risks of future VTE).⁴⁹  Methodologic limitations associated with using administrative registries for outcome identification may also have contributed to the difference in risk estimates compared with the randomized trials. Mortality was lower in the present study than in the trials, likely reflecting different distributions of patients receiving chemotherapy with curative and palliative intent. However, guidelines for thromboprophylaxis do not consider the indication for chemotherapy, and VTE risk has also been reported to be substantial for patients undergoing curatively intended treatment.⁵⁰ 

Importantly, the net clinical benefit and number needed to treat to prevent 1 event depend on the absolute risk of VTE.^6,36,37  It has been estimated that patients with a 6-month cumulative incidence <8% are unlikely to benefit from primary prophylaxis based on post hoc data from the AVERT trial.¹⁸  In the present study, only 0.2% of patients with Khorana scores ≥5 had 6-month risks exceeding this threshold.

While the reductionist design of the Khorana score supports its applicability into clinical practice, it also ignores other potential prognostic factors for incident VTE in cancer patients (eg, cancer stage, type of anticancer treatment, history of other cancer, and biomarkers).^7,51  Ultimately, the complexity of a risk score should not hamper clinical implementation. The results from this and previous studies do not support the use of a universal Khorana score threshold for all cancer patients since it provided no clinically relevant stratification in several cancer subgroups (see Figure 3). Other studies also found a limited clinical value of the Khorana score in patients with lung cancer, indicating the need for considering other risk markers for this subgroup.^52,53  Indeed, refined risk stratification to guide primary prevention of VTE remains an unmet need among cancer patients. Cancer-specific Khorana score thresholds or development of cancer-specific risk scores are needed for improving individualized risk assessment while also considering patient preferences.^41,54  Several alternative risk scores already exist but are not recommended in guidelines.⁷  Future studies with a focus on risk stratification should ensure consistent and appropriate methodology to optimize opportunities for between-study comparisons and to avoid overestimation of risk when ignoring competing risk of death. Comparative effectiveness studies of anticoagulation vs placebo, whether randomized or observational, would benefit from estimating the net clinical benefit (eg, using the number needed to treat for net effect measure).⁵⁵  Also, bleeding risk likely differs markedly across cancer types, but no formal bleeding risk assessment tools are recommended in current guidelines, underlining another important evidence gap.^29,55 

The large sample size minimized random variation and allowed for stratified analyses across several cancer types.

The number of patients with body mass index >35 was underestimated based on ICD-10 identification alone, but the resulting minor misclassification of Khorana score levels is unlikely to have impacted the results substantially.²⁷  The validity of a diagnosis of VTE given in combination with a relevant imaging procedure is high (>90%) and unlikely to be associated with Khorana score levels. Information bias is therefore not a likely explanation of the observed differences in risk across Khorana levels. We also applied different methodology and outcome definitions to explore the potential scope of variation in risk estimates. The sensitivity of ICD codes for identifying VTE is not known, and some events may not have been coded by the treating physicians, thus contributing to underestimation of risk.

We aimed to assess VTE risk in a population free from anticoagulation. Nonetheless, some patients could theoretically have been treated with low-molecular-weight heparin, which is not traceable in registries since it is handed out directly from the hospital departments. This would cause artificially low risk estimates. However, we excluded all patients with a recent history of a claimed prescription of an anticoagulant drug as well as with potential comorbidity, which could trigger an indication for low-molecular-weight heparin. An option for primary outpatient thromboprophylaxis with low-molecular-weight heparin was introduced in Danish guidelines in 2017 but is not used in clinical practice.^40,42,56  Restricting the study period to before 2017 also revealed similar risk estimates, indicating that the use of low-molecular-weight heparin in the most recent years did not contribute to lowering risk estimates.

Patients were followed in administrative registries with virtually complete follow-up. Some deaths could be due to undiagnosed pulmonary embolism. Previous studies have reported a 5% to 10% prevalence of pulmonary embolism in autopsied cancer patients.^57,58  Such deaths due to VTE would have been misclassified as competing risk mortality instead of counting as a VTE event, potentially leading to underestimation of the incidence of VTE.

We were unable to describe bleeding risk, another key determinant of the benefit of anticoagulation, since primary thromboprophylaxis in ambulatory patients is not routinely used in Denmark.

The Khorana score was able to risk-stratify ambulatory cancer patients initiating chemotherapy according to the risk of VTE, but only for selected cancer types, challenging the universal application of the Khorana score for guiding decisions on primary thromboprophylaxis. Reported risk estimates varied substantially depending on the choice of methodology. Also, absolute risks of VTE for patients eligible for anticoagulation in guidelines were lower than observed in previous studies. Thus, the absolute benefit of implementing routine primary thromboprophylaxis in an unselected cancer population may be smaller than anticipated by randomized trial data.

The authors wish to thank Annette Juul Madsen for assisting with data extraction from the VARiS MedOncology database.

The study was supported by North Denmark Region’s Fund for Health Sciences Research.

Contribution: T.F.O. conceived the study idea, analyzed the data, and wrote the paper; F.S. also analyzed the data; F.S., P.B.N., A.G.O., I.L.G., M.T.S., I.E.A., and T.B.L. contributed with substantial contributions to the study design, interpretation of data for the work, and revised the paper critically for important intellectual content; A.K.V., G.P., and S.N. contributed to the interpretation of data and revised the manuscript critically for important intellectual content; and all authors gave final approval of the version to be published.

Conflict-of-interest disclosure: P.B.N. Personal fees from Boehringer Ingelheim, grants and personal fees from Daiichi-Sankoy, grants from BMS/Pfizer, and grants and personal fees from Bayer outside the submitted work. G.P. Research grant support from Bristol Myers Squibb/Pfizer Alliance, Janssen, Boston Scientific Corporation, Bayer, Amgen, and Portola. S.N. Speaker bureau for Leo Pharma, BMS/Pfizer, and Bayer, and grant support from Leo Pharma. I.E.A. speaking fees from Pfizer and Bayer. T.B.L. Consulting fees from BMS/Pfizer, Bayer, and MSD, speaker bureau for BMS/Pfizer, Bayer, and MSD, and grant support from Bayer and Daiichi-Sankoy. F.S. Consulting fees from Bayer. All other authors declare no competing financial interests.

Correspondence: Thure Filskov Overvad, Aalborg Hospital Science and Innovation Centre, Søndre Skovvej 15, 9000 Aalborg, Denmark; e-mail: t.overvad@rn.dk.