Key Points
Novel mixed methods were used to select risk factors for venous thromboembolism and bleeding RAMs in medical inpatients.
Risk factors were identified that require further research to confirm or refute their importance in RAMs.
Abstract
Risk assessment models (RAMs) for venous thromboembolism (VTE) and bleeding in hospitalized medical patients inform appropriate use of thromboprophylaxis. Our aim was to use a novel approach for selecting risk factors for VTE and bleeding to be included in RAMs. First, we used the results of a systematic review of all candidate factors. Second, we used the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach to assess the certainty of the evidence for the identified factors. Third, we using a structured approach to select factors to develop the RAMs, by building on clinical and methodological expertise. The expert panel made judgments on whether to include, potentially include, or exclude risk factors, according to domains of the GRADE approach and the Delphi method. The VTE RAM included age >60 years, previous VTE, acute infections, immobility, acute paresis, active malignancy, critical illness, and known thrombophilia. The bleeding RAM included age ≥65 years, renal failure, thrombocytopenia, active gastroduodenal ulcers, hepatic disease, recent bleeding, and critical illness. We identified acute infection as a factor that was not considered in widely used RAMs. Also, we identified factors that require further research to confirm or refute their importance in a VTE RAM (eg, D-dimer). We excluded autoimmune disease which is included in the IMPROVE (International Medical Prevention Registry on Venous Thromboembolism) bleeding RAM. Our results also suggest that sex, malignancy, and use of central venous catheters (factors in the IMPROVE bleeding RAM) require further research. In conclusion, our study presents a novel approach to systematically identifying and assessing risk factors to be included or further explored during RAM development.
Introduction
Venous thromboembolism (VTE), comprising deep venous thrombosis (DVT) and pulmonary embolism (PE), is a common cause of morbidity and mortality in hospitalized medical patients. The risk of VTE with hospitalization may increase more than eightfold.1,2 Patient- and disease-specific risk factors and their interaction modulate the magnitude and duration of the VTE risk in hospitalized medical patients.2 Patients receive thromboprophylaxis to mitigate this risk, but inappropriate use of VTE prophylaxis in low-risk patients may not meaningfully reduce VTE rates and may cause bleeding.2 Knowing a patient’s risk of VTE or bleeding would aid health care providers in selecting the appropriate prevention and management options to optimize patient care.3,4 Risk assessment models (RAMs) can help with stratification of an individual patients’ risk of developing a VTE or bleeding event and the choice of preventive measures.
The American Society of Hematology (ASH) guidelines for management of venous thromboembolism: prophylaxis for hospitalized and nonhospitalized medical patients described 15 existing RAMs, and the authors called for more research to improve and validate them.5-7 Derived from various studies, most of the identified RAMs were not developed based on a systematic review. This means that, although they are derived from large cohort studies, unmeasured potential risk factors in a specific cohort would have no possibility of being included in a RAM, whereas they could be captured as a candidate risk factor in a systematic review. We identified only 1 systematic review conducted 11 years ago that assessed VTE as an outcome in medical patients, but it did not include a meta-analysis or weighted statistical analysis of the prognostic factors.8
There are several additional reasons for improving or validating currently existing RAMs. First, RAMs of single studies often rely on statistical significance, and in this setting, random variation or lack of power may or may not lead to the statistical significance of a risk factor in a prediction model. Second, statistical methods should be complemented by clinical expertise, to identify risk factors that are meaningful for health professionals. For example, if a RAM provides inaccurate or poorly calibrated estimates of VTE risk (ie, it over- or underpredicts by ignoring clinical context), it may mislead health care professionals. Third, the lack of using standardized definitions for risk factors causes confusion across RAMs. For example, Ye et al1 examined the various definitions of immobility used in previous studies and observed inconsistencies, which makes reproduction and validation of previous studies challenging.1 Fourth, RAMs should be able to predict specific events accurately and still be relatively easy to use. Fifth, there is no universal consensus on use of a specific RAM in hospitalized medical patients, in part because of the reasons just mentioned.9
Therefore, we used a novel approach to support the development of new RAMs and inform the updating of widely used RAMs for VTE and bleeding in hospitalized acutely, critically, and chronically ill medical patients. We first conducted a systematic review of all relevant risk factors in hospitalized medical patients.10 In tandem, we used extensive clinical and methodological expertise to assess the certainty in the identified risk factors and select them by using a structured approach that requires clinical expertise.
Methods
Ethics
After the review of the project, the Hamilton Integrated Research Ethics Board waived the need for ethics approval for this study.
Study design
We conducted a study that combined systematic review methods and an assessment of the certainty of the evidence according to Grading of Recommendations Assessment, Development, and Evaluation (GRADE). This work then informed a structured Delphi-based expert judgment to include, potentially include, or exclude risk factors for VTE and bleeding in hospitalized medical patients, by using the GRADE Evidence to Decision (EtD) criteria.11 The process is described in Figure 1.
Participants
The expert panel included clinicians and researchers with expertise in management of VTE and bleeding in hospitalized medical patients, and in the development, validation, and implementation of RAMs for clinical practice. Panel members participated in a face-to-face panel meeting, responded to surveys and questionnaires, and provided feedback on reports. They completed declaration-of-interest forms to ensure transparency on potentially existing conflicts with regard to existing RAMs and other factors.
The research team selected members of the expert panel by using purposive sampling and the following criteria:
The research team compiled the evidence for presentation, drafted the questions for the Delphi process, analyzed the responses, and summarized the results.Systematic review
Before this study, we conducted a systematic review to identify all potential risk factors for VTE and bleeding in hospitalized medical patients, which we have described in detail elsewhere.10 In brief, we searched Medline and EMBASE from inception to May 2018. We considered prognostic factors and RAM studies that identified potential prognostic factors for the outcomes VTE and bleeding in hospitalized adult acutely, critically, or chronically ill medical patients. We defined VTE as any symptomatic or asymptomatic DVT or PE within 90 days after discharge. Bleeding included major or nonmajor but clinically significant bleeding within 90 days after discharge.13,14 Reviewers extracted data in duplicate and independently and assessed the certainty of the evidence by using the GRADE approach.15 The results of the systematic review were used for this study.10
Delphi process
Delphi round 1.
We asked the expert panel by e-mail to provide input (eg, identify gaps) on the list of risk factors that we identified through the systematic review. Expert panelists responded confidentially and independently so as not to influence other panel members. Subsequent Delphi rounds aimed to build on this by requesting that the experts make clinical and methodological judgements based on the available evidence.15,16
EtD frameworks.
For the second and third round of the Delphi process, we used the GRADEpro Guideline Development Tool to facilitate the process.17 The tool includes standardized tables and frameworks:
Evidence profiles. During the third round of the Delphi process, we used these tables, which provide synthesized evidence for each risk factor based on a meta-analysis and an assessment of the certainty of the evidence rated as high, moderate, low, or very low. We based these ratings on considerations of risk of bias, indirectness, inconsistency, and imprecision.15
GRADE EtD frameworks. During the second and third rounds of the Delphi process, we used the following criteria from the EtDs to facilitate the panel’s decision-making process18 : benefits and harms, resource requirements, equity, acceptability, and feasibility. These criteria allowed the expert panel to make judgements about the risk factors that were both evidence-based and clinically relevant. Although this approach followed good practice in RAM development that suggests attaining high predictability while remaining relatively simple and applicable in clinical settings, it is novel, as it uses a structured approach based on EtD criteria.18,19
Delphi round 2.
We held a face-to-face panel meeting to discuss the systematic review findings and the approach to judge which risk factors should be included in the RAM for VTE and bleeding. We presented the results of the systematic review and the results of the first round of the Delphi process. The results included individual and pooled estimates of the associations and the corresponding confidence intervals for each identified risk factor using forest plots for each meta-analysis. After reviewing the results and discussing resource requirements, equity, feasibility, and acceptability, the panel was asked to categorize the risk factors into 3 sets: included, potentially included, and excluded. We defined the included risk factors as those that should be included in a RAM. We defined the potentially included factors as candidates for consideration in a RAM. The excluded risk factors were those that should not be considered for a RAM. Reasons for exclusion included potential interaction with other factors or no association with the outcome.
The research team and the expert panel noted inconsistencies in the definitions of the risk factors across studies and agreed that standardizing the definitions is critical for deciding which ones to include in a RAM and for future research, data collection purposes, and most important, clinical relevance. We standardized the definitions of the included and potentially included risk factors by reviewing the definitions of the original study as detailed in supplemental Tables 6 and 7. We then obtained input from the panel to draft the suggested definitions.
Delphi round 3.
We conducted a Web-based anonymous survey through SurveyMonkey.20 We asked questions after presenting results of the previous round, the assessment of the certainty of the evidence, and a descriptive summary of findings for each risk factor from the systematic review. In this round, we asked the panel members to formally judge the effect estimate of the meta-analysis, the resource requirements, and the impact on equity, acceptability, and feasibility of each risk factor. Based on these criteria, the panel judged whether these risk factors should be included, potentially included, or excluded.
We determined a priori that we would make final judgments based on simple majority votes. However, when votes were spread across all 3 categories and both the included and excluded category had more than 1 vote each, we determined that the risk factor should be potentially included.
To harmonize the definitions of risk factors, we shared draft definitions with the expert panel based on the literature we identified. We asked them to review the information and provide feedback, which we incorporated in final definitions of the risk factors (supplemental Table 8).
Final risk RAMs.
The risk of VTE or bleeding for each risk factor was presented as an odds ratio (OR) with the relative 95% confidence interval (derived from the meta-analysis). To develop the VTE and bleeding RAMs, we log transformed the ORs into β coefficients and determined the linear predictor (Y) for VTE or bleeding.21 The final RAMs, presented as regression models, are given as21, where LP is the linear predictor Y of the outcome VTE or bleeding that is derived from the logistic regression model, where β0 is the intercept, β1 is the β coefficient for the first risk factor, and X1 is the first risk factor, and so on.21 To determine the contribution of each risk factor to the overall risk of VTE or bleeding, we summed the β coefficients, divided each by the total, and multiplied by 100 (Tables 2 and 3). We did not evaluate nonlinear logistic regression models.
We then presented a case scenario where we computed the patient-specific risk or predicted probability of VTE and bleeding based on the formula P = odds/(odds + 1), where odds is the exponent of the linear predictor (Y) (odds = ey), and Y is the linear predictor that was derived from the final RAM. If the risk factor in the RAM is present, X is given a value of 1 and if absent, a value of 0. In the case scenario, we applied a calculated β0 corresponding to a VTE or bleeding risk of 0% because of the assumption that we would identify all risk factors. Absence of any risk factor would therefore approach a risk of 0%.
Results
Response rate and participant characteristics
We included 9 physicians who were both clinical and methodological experts in the field (M.C., M.K.G., H.J.S., F.A.S., A.C.S., M.B.S., S.C.W., N.A.Z., and L.M.), one of whom was a biostatistician (L.M.)22,23 (supplemental Table 1). For each of the 3 rounds of the Delphi process, we achieved a 100% response rate.
Systematic review findings
We identified 17 eligible studies, 14 of which reported on VTE and described 29 candidate prognostic factors,24-37 and 3 studies that reported on bleeding and described 17 candidate factors.38-40 Supplemental Tables 2 and 3 provide the evidence profiles for VTE- and bleeding-related prognostic factors. A detailed description of the results of the systematic review and corresponding forest plots of the meta-analyses are published elsewhere.10
VTE model risk factor selection
Delphi round 1.
Listed in Table 1 are the 29 potential risk factors identified from the systematic review and the additional 3 factors suggested by the expert panel, for a total of 32. The risk factors were all evaluated at the time of admission, except for immobility, acute paresis, and critical illness, which were assessed both at admission and during the hospital stay.
Source . | Risk factors . | |
---|---|---|
VTE | ||
Included studies in the systematic review | 1. Age >60 y | 16. Known history of thrombophilia |
2. Sex | 17. Thrombocytosis | |
3. Previous VTE | 18. Central venous catheter use | |
4. Acute infection | 19. Varicose veins | |
5. Respiratory failure | 20. Leg edema | |
6. Renal failure | 21. Tobacco | |
7. Immobility* | 22. Coronary artery disease | |
8. Acute paresis* | 23. Heart failure | |
9. Severe stroke | 24. Fever | |
10. Malignancy | 25. Elevated heart rate | |
11. Critical illness* | 26. Leukocytosis | |
12. D-dimer | 27. Low Barthel index score | |
13. Autoimmune disease | 28. Elevated C-reactive protein | |
14. Obesity | 29. Fibrinogen levels | |
15. Hormone use | ||
Panel input | 1. Recent surgery <30 d before admission | |
2. Recent long bone fractures | ||
3. Recent air travel | ||
Bleeding | ||
Included studies in the systematic review | 1. Age ≥65 y | 10. CVC use |
2. Sex | 11. Active gastroduodenal ulcers in past 3 mo | |
3. Renal failure | 12. Hepatic disease | |
4. Malignancy | 13. Antithrombotic medication use† | |
5. Critical illness | 14. Rehospitalization† | |
6. Autoimmune disease | 15. Blood dyscrasias | |
7. Obesity | 16. Recent bleeding | |
8. Hormone use | 17. Low hemoglobin | |
9. Thrombocytopenia | ||
Panel input | None |
Source . | Risk factors . | |
---|---|---|
VTE | ||
Included studies in the systematic review | 1. Age >60 y | 16. Known history of thrombophilia |
2. Sex | 17. Thrombocytosis | |
3. Previous VTE | 18. Central venous catheter use | |
4. Acute infection | 19. Varicose veins | |
5. Respiratory failure | 20. Leg edema | |
6. Renal failure | 21. Tobacco | |
7. Immobility* | 22. Coronary artery disease | |
8. Acute paresis* | 23. Heart failure | |
9. Severe stroke | 24. Fever | |
10. Malignancy | 25. Elevated heart rate | |
11. Critical illness* | 26. Leukocytosis | |
12. D-dimer | 27. Low Barthel index score | |
13. Autoimmune disease | 28. Elevated C-reactive protein | |
14. Obesity | 29. Fibrinogen levels | |
15. Hormone use | ||
Panel input | 1. Recent surgery <30 d before admission | |
2. Recent long bone fractures | ||
3. Recent air travel | ||
Bleeding | ||
Included studies in the systematic review | 1. Age ≥65 y | 10. CVC use |
2. Sex | 11. Active gastroduodenal ulcers in past 3 mo | |
3. Renal failure | 12. Hepatic disease | |
4. Malignancy | 13. Antithrombotic medication use† | |
5. Critical illness | 14. Rehospitalization† | |
6. Autoimmune disease | 15. Blood dyscrasias | |
7. Obesity | 16. Recent bleeding | |
8. Hormone use | 17. Low hemoglobin | |
9. Thrombocytopenia | ||
Panel input | None |
Immobility, acute paresis, and critical illness are risk factors that were assessed, both at admission and during the hospital stay.
Antithrombotic medication use and rehospitalization are risk factors that were assessed after admission.
Delphi round 2.
The panel judged 9 of the 32 factors to be included: age, previous VTE, acute infections, immobility, acute paresis, malignancy, critical illness, D-dimer level, and known thrombophilia. They potentially included the following 14 factors: sex, respiratory failure, severe stroke, autoimmune disease, obesity, thrombocytosis, central venous catheter (CVC) use, leg edema, fever, heart rate, leukocytosis, recent long bone fractures, recent travel, and history of heart failure (acute heart failure was excluded). The expert panel excluded renal failure, hormone use, tobacco, and coronary artery disease, as the results showed little to no association with the outcome VTE. They excluded varicose veins, low Barthel index score, elevated C-reactive protein, and fibrinogen levels, because they perceived these risk factors to be nonspecific and not routinely measured in patients admitted to the hospital. They also excluded recent surgery <30 days before admission, so that the focus would be on medical patients.
Delphi round 3.
Based on our voting criteria, the panel agreed to include the following 8 risk factors for the VTE RAM: age >60 years (by consensus), previous VTE (by consensus), acute infections, immobility, acute paresis, active malignancy (by consensus, a history of malignancy was excluded), critical illness, and known history of thrombophilia. The final round of our approach eliminated sex, elevated heart rate, and recent air travel from the list of potentially included risk factors (Table 2). The judgements for these factors are detailed in supplemental Table 3. The panel also agreed on definitions for these risk factors as described in supplemental Table 8.
Bleeding model risk factor selection
Delphi round 1.
A total of 17 risk factors were candidates, based on the systematic review (Table 1). The expert panel agreed and did not suggest any additional factors. The risk factors were all evaluated at the time of admission, except for use of antithrombotic medication and rehospitalization, which were assessed after the index admission.
Delphi round 2.
During the face-to-face meeting, the panel judged 8 of the 17 factors to be included in the RAM: age ≥65 years, critical illness, thrombocytopenia, active gastroduodenal ulcer in the past 3 months, hepatic disease, recent bleeding, blood dyscrasias, and use of antithrombotic medication. They agreed that 5 of the 17 factors were candidates for inclusion: sex, renal failure, malignancy, autoimmune disease, and CVC use. The expert panel excluded hormone use, because the results showed little to no association with the outcome bleeding. They excluded obesity because of the low certainty of evidence that it is a risk factor. They also excluded low hemoglobin and rehospitalization because of the very low certainty of evidence and the lack of specificity with the outcome bleeding.
Delphi round 3.
The panel determined that the following risk factors should be included in the bleeding RAM: age ≥65 years, renal failure, thrombocytopenia (by consensus), active gastroduodenal ulcer in the past 3 months, hepatic disease, recent bleeding, use of antithrombotic medication, and critical illness. However, we opted not to include antithrombotic medication use as a risk factor in the bleeding RAMs, because our aim was to develop a RAM that would assist health care professionals in identifying medical patients at admission who may be at increased risk of bleeding caused by anticoagulants, to appropriately weigh benefits and harms before starting treatment. Sex, CVC use, blood dyscrasias and malignancy were rated as potentially included (Table 2) but autoimmune disease was excluded. The judgments for these factors are detailed in supplemental Table 5. Supplemental Table 8 presents the agreed upon definitions for these risk factors.
VTE |
Respiratory failure |
Severe stroke |
Elevated D-dimer |
Autoimmune disease |
Obesity |
Thrombosis |
Central venous catheter use |
Leg edema |
History of heart failure |
Fever |
Leukocytosis |
Recent long bone fracture |
Bleeding |
Sex |
Central venous catheter use |
Blood dyscrasias |
Malignancy |
VTE |
Respiratory failure |
Severe stroke |
Elevated D-dimer |
Autoimmune disease |
Obesity |
Thrombosis |
Central venous catheter use |
Leg edema |
History of heart failure |
Fever |
Leukocytosis |
Recent long bone fracture |
Bleeding |
Sex |
Central venous catheter use |
Blood dyscrasias |
Malignancy |
Final RAMs.
Risk factors . | OR (95% CI) from systematic review . | β coefficients (log OR) . | SE . | Contribution to overall risk, % . |
---|---|---|---|---|
VTE | ||||
Age >60 y | 1.34 (1.17-1.55) | 0.29 | 0.07 | 3.6 |
Previous VTE | 6.08 (3.71-9.97) | 1.81 | 0.25 | 22.7 |
Acute infections | 1.48 (1.16-1.89) | 0.39 | 0.12 | 4.9 |
Immobility | 3.17 (2.18-4.62) | 1.15 | 0.19 | 14.4 |
Acute paresis | 2.97 (1.20-7.36) | 1.09 | 0.46 | 13.6 |
Active malignancy | 2.65 (1.79-3.91) | 0.98 | 0.20 | 12.3 |
Critical illness | 1.65 (1.39-1.95) | 0.50 | 0.09 | 6.3 |
Known history of thrombophilia | 5.88 (2.80-12.35) | 1.77 | 0.38 | 22.2 |
Total | 7.98 | |||
Bleeding | ||||
Age ≥65 y | 1.95 (1.59-2.38) | 0.67 | 0.10 | 12.3 |
Renal failure | 1.43 (1.06-1.93) | 0.36 | 0.15 | 6.6 |
Thrombocytopenia | 1.79 (0.97-3.29) | 0.58 | 0.31 | 10.7 |
Active gastroduodenal ulcers | 2.74 (1.42-5.26) | 1.01 | 0.34 | 18.6 |
Hepatic disease | 1.53 (1.09-2.15) | 0.43 | 0.17 | 7.9 |
Recent bleeding | 5.15 (2.45-10.81) | 1.64 | 0.38 | 30.2 |
Critical illness | 2.10 (1.42-3.11) | 0.74 | 0.20 | 13.7 |
Total | 5.43 |
Risk factors . | OR (95% CI) from systematic review . | β coefficients (log OR) . | SE . | Contribution to overall risk, % . |
---|---|---|---|---|
VTE | ||||
Age >60 y | 1.34 (1.17-1.55) | 0.29 | 0.07 | 3.6 |
Previous VTE | 6.08 (3.71-9.97) | 1.81 | 0.25 | 22.7 |
Acute infections | 1.48 (1.16-1.89) | 0.39 | 0.12 | 4.9 |
Immobility | 3.17 (2.18-4.62) | 1.15 | 0.19 | 14.4 |
Acute paresis | 2.97 (1.20-7.36) | 1.09 | 0.46 | 13.6 |
Active malignancy | 2.65 (1.79-3.91) | 0.98 | 0.20 | 12.3 |
Critical illness | 1.65 (1.39-1.95) | 0.50 | 0.09 | 6.3 |
Known history of thrombophilia | 5.88 (2.80-12.35) | 1.77 | 0.38 | 22.2 |
Total | 7.98 | |||
Bleeding | ||||
Age ≥65 y | 1.95 (1.59-2.38) | 0.67 | 0.10 | 12.3 |
Renal failure | 1.43 (1.06-1.93) | 0.36 | 0.15 | 6.6 |
Thrombocytopenia | 1.79 (0.97-3.29) | 0.58 | 0.31 | 10.7 |
Active gastroduodenal ulcers | 2.74 (1.42-5.26) | 1.01 | 0.34 | 18.6 |
Hepatic disease | 1.53 (1.09-2.15) | 0.43 | 0.17 | 7.9 |
Recent bleeding | 5.15 (2.45-10.81) | 1.64 | 0.38 | 30.2 |
Critical illness | 2.10 (1.42-3.11) | 0.74 | 0.20 | 13.7 |
Total | 5.43 |
SE, standard error.
The risk factors with the largest contributions to the overall VTE risk (Table 3) are previous VTE (22.6%), known thrombophilia (22.2%), immobility (14.5%), and acute paresis (13.6%), whereas those with the largest contributions to the overall bleeding risk (Table 3) are recent bleeding (30.2%), active gastroduodenal ulcers in past 3 months (18.6%), and critical illness (13.7%).
Case scenario.
This case scenario suggests how these RAMs can be used in clinical practice. Consider a young, acutely ill medical patient (XAge >60 = 0) who is admitted to the hospital for an acute infection (XAcute infection = 1). The patient has no history of VTE (XPrevious VTE = 0), is mobile (XImmobility = 0), has no acute paresis (XParesis = 0), no active malignancy (XActive Malignancy = 0), no critical illness (XCritical illness = 0), and no known history of thrombophilia (XKnown history of thrombophilia = 0). Based on the VTE RAM, this patient has a linear predictor (y) of −5.41.
The following formula was used to calculate the odds:
Odds = ey = e−5.41 = 0.0045,
In terms of the bleeding risk profile, the patient (XAge ≥65 = 0) reported having known thrombocytopenia (XThrombocytopenia = 1) and an active gastroduodenal ulcer in the past 3 months (XActive gastroduodenal ulcers = 1), but had none of the other risk factors included in the bleeding RAM; that is, the patient had no recent bleeding episode (XRecent bleeding = 0), impaired renal function (XRenal failure = 0), hepatic disease (XHepatic disease = 0), or any critical illness (XCritical illness = 0). Based on the developed bleeding RAM, this patient has a linear predictor (y) of −4.01 which corresponds to a bleeding probability of ∼1.8% based on the formulas.
One recommendation in the ASH guidelines assessed the effect of any parenteral anticoagulation (unfractionated heparin, low-molecular-weight heparin, or fondaparinux) compared with none.7 Based on the results of the meta-analyses, the relative risk was 0.58 for combined symptomatic PE and DVT and 1.48 for major bleeding.7 Based on the predicted probabilities in the case scenario and on the effects of parenteral anticoagulation on VTE and bleeding, if the patient were prescribed thromboprophylaxis, the absolute risk of VTE would be reduced by ∼0.2%, whereas the absolute risk of bleeding would increase by ∼0.9%, amounting to an absolute risk for VTE of 0.26% and for bleeding of 2.66%.
These risk estimates are useful for implementing the corresponding ASH recommendations regarding acutely or critically ill medical patients: mechanical VTE prophylaxis compared with a combination of pharmacological and mechanical or pharmacological VTE prophylaxis alone.7 Given the bleeding risk and if the patient places a relatively high value on avoiding bleeding complications, the harms would outweigh the benefits. Interpreting the conditional recommendation, “In acutely or critically ill medical patients, the ASH guideline panel suggests using pharmacological VTE prophylaxis over mechanical VTE prophylaxis (conditional recommendation, very low certainty in the evidence of effects.),”7 suggests not using pharmacological thromboprophylaxis for this patient. Thus, the following ASH recommendation should be used for this patient: “In acutely or critically ill medical patients who do not receive pharmacological VTE prophylaxis, the ASH guideline panel suggests [emphasis in the original] using mechanical VTE prophylaxis over no VTE prophylaxis (conditional recommendation, moderate certainty in the evidence of effects).”7
Discussion
Summary of findings
We used a novel approach to systematically identify and assess risk factors to support the development of a RAM and inform the update of widely used RAMs for VTE and bleeding in hospitalized acutely, critically, or chronically ill medical patients. First, we conducted a systematic review of all relevant risk factors in hospitalized medical patients.10 Second, we assessed the certainty of the evidence in identified risk factors. Third, we selected the factors to include in the RAMs, using an innovative structured approach based on GRADE that required extensive clinical and methodological expertise. The expert panel made judgments on whether to include, potentially include, or exclude identified risk factors from the final RAMs using the Delphi method based on GRADE criteria. This novel approach allowed us to identify risk factors that should be included in a RAM and factors that require further exploration. These findings support the development or update of a RAM that can accurately predict specific events while remaining relatively simple and applicable to use in clinical settings. If a RAM provides inaccurate over- or underestimates of future event occurrences, it may lead to mismanagement of patient care and health care resources. On the other hand, if a model has high predictability power but is difficult to apply, time consuming, costly, or less relevant, it will not be commonly used. However, these RAMs should be externally validated and should be assessed in prospective impact studies.
Our 8-factor VTE RAM was developed based on risk factors assessed at admission and during hospital stay and includes age >60 years, previous VTE, acute infections, immobility, acute paresis, active malignancy, critical illness, and known history of thrombophilia. Our 7-factor bleeding RAM was developed based on risk factors assessed only at admission and includes age ≥65 years, renal failure, thrombocytopenia, active gastroduodenal ulcer in the past 3 months, hepatic diseases, recent bleeding, and critical illness. The potentially included risk factors (Table 2) require further study to confirm or refute their importance for the respective RAMs.
Comparison with other RAMs
We developed RAMs for VTE and bleeding in hospitalized medical patients that were similar but not identical to some widely used RAMs in current practice, such as the IMPROVE (International Medical Prevention Registry on Venous Thromboembolism) VTE RAM, IMPROVE Bleeding RAM, Intermountain RAM, MITH (Medical Inpatients and Thrombosis) RAM, and PADUA VTE RAM.33,35,38,41,42 Compared with the IMPROVE VTE RAM, acute infection is an additional important risk factor.33 We identified 12 additional candidate risk factors, 5 of which were considered in other VTE RAMs, including CVC use in the Intermountain RAM,42 respiratory failure and heart failure in the PADUA and MITH VTE RAMs, severe stroke in the PADUA VTE RAM, and thrombocytosis and leukocytosis in the MITH RAM.35,41 Compared with the PADUA model, we found that coronary artery disease does not predict VTE risk, and obesity requires further research.41 As for risk of bleeding, our RAM suggests that autoimmune disease did not predict bleeding risk because of conflicting results in the included studies.38 Also, we did not include sex, malignancy, and CVC use, which are included in the IMPROVE Bleeding RAM, because we judged that these were candidate risk factors that require further exploration.38
Strengths
Our study is based on rigorous methods that are innovative for several reasons. First, the systematic review conducted by our group and the input from the expert panel in comprehensively identifying all potential risk factors is a limitation of cohort studies. Second, assessing the certainty of the evidence based on a structured framework allows for an expression of the confidence in the predictive ability of the factors. Third, the expert panel made judgements on the inclusion of risk factors in the RAMs using GRADE criteria by accounting for their resource requirements, impact on equity, acceptability, and feasibility, all of which are relevant in clinical practice. Fourth, we are not aware of prior use of this approach to developing RAMs. Fifth, we standardized the definitions of the included and potentially included risk factors to decrease variability in methods of measurement across settings. Standardized definitions will provide more clarity to health care professionals, including researchers, when evaluating and weighing patients’ risks of VTE and bleeding and subsequent management options. Our work also strongly suggests the need to standardize definitions of risk factors if we are to make further progress in this area.
Limitations
Potential limitations of the systematic review findings are the inconsistency and variability across eligibility criteria in the included studies and variability in study design, study type, sample size, and definitions of the risk factors. Other limitations included the inconsistency in the diagnostic approaches used across studies and the contamination of the study population with nonmedical hospitalized patients for some of the risk factors. We were unable to conduct a meta-regression to adjust for study level characteristics, because the number of studies was too small for this analysis. Also, we did not conduct an external validation which is an essential next step. However, validation is a continuous process, and our approach should be viewed as a method of validating the content of current widely used RAMs. For example, our VTE RAM validated the findings in the IMPROVE VTE RAM, highlighted the need to include acute infection as a factor, and provided a list of candidate factors that require further evaluation.
Implications for practice
The findings of this study can aid in RAM development or updating. Our findings can also help health care professionals in evaluating the risk of VTE from index admission until discharge and the risk of bleeding on admission in hospitalized medical patients for optimal patient management. Ideally, this can be achieved by integrating the RAMs in clinical decision aids to assist with deriving individual-based recommendations from published population-based guideline recommendations for shared decision making.
Implications for future research
Our developed RAMs should be tested in an external validation study using individual patient data sets. Such validation is essential before conducting an impact analysis that would allow the RAMs to be adopted in routine clinical practice. Also, the potentially included risk factors should be explored in further research. We standardized the definitions of the risk factors to help researchers build more uniform datasets and registries. Decreasing variability will facilitate the reproduction and validation of studies across settings.
After review of the work and discussions with the expert panel, we noted that a risk assessment for VTE and bleeding conducted only on admission is insufficient and will not account for a change in risk factors throughout hospitalization. For example, transferring a patient from a medical ward to the intensive care unit or development of acute renal failure or acute infection may change the risk level. Therefore, developing a system for dynamic risk assessment of hospitalized medical patients from admission to discharge is important. The shortening of hospital lengths of stay, the lack of routine postdischarge thromboprophylaxis, and the recent availability of thromboprophylactic agents that can be used for extended thromboprophylaxis in hospitalized medical patients makes testing our proposed VTE RAM for dynamic use especially important.
Conclusion
We developed a novel structured approach for selecting risk factors for VTE and bleeding in hospitalized medical patients that is evidence based, clinically meaningful, and relevant. Our findings support the development of new RAMs and the update of widely used RAMs. Also, our findings inform external validation and prospective impact assessment studies that may be undertaken to evaluate the performance of these RAMs in assessing VTE and bleeding risk for this population. These findings may assist decision makers in weighing the risk of VTE with that of bleeding, to appropriately select VTE prevention strategies and optimize patient care for different patient risk groups.
Original data are available by e-mail request to the corresponding author.
Acknowledgments
The authors thank ASH for providing the space to hold the panel workshop and for logistical support with the preparations.
This work was supported by a subcontract (200-2016-92458) from the US Centers for Disease Control and Prevention (CDC) through Karna LLC.
The findings and conclusions of this report are those of the authors and do not necessarily represent the official position of the CDC.
Authorship
Contribution: A.J.D., L.M., M.R., and H.J.S. analyzed the data; A.J.D. and H.J.S. drafted the manuscript; and all authors interpreted the results, critically revised the manuscript, and approved of the final version.
Conflict-of-interest disclosure: A.J.D., I.E.-I., A.I., E.A.A., and H.J.S. are members of the GRADE working group. H.J.S. is cochair of the panel that developed the American Society of Hematology 2018 guidelines for management of venous thromboembolism: prophylaxis for hospitalized and nonhospitalized medical patients (ASH 2018 VTE guidelines) and obtained grant funding from the CDC for this study. A.C.S. has received remuneration for consulting work for Bayer, Janssen, and Portola; has obtained research support grants from Boehringer Ingelheim, Janssen, and the Centers for Medicare and Medicaid Services; and reports an intellectual conflict as the leader of the group that developed and validated the IMPROVE VTE tool for VTE risk assessment in hospitalized medical patients. M.B.S. has received remuneration for consulting work for Bayer, Janssen, Pfizer, and Portola and research support grants from Boehringer-Ingelheim, Janssen, Portola, and Roche. M.C. is a former member of the board of directors (2013-2017) of the American Heart Association and chaired the panel that developed the ASH 2018 VTE guidelines. F.A.S. and N.A.Z. participated as panel members for the ASH 2018 VTE guidelines. N.A.Z. reported receiving honoraria in 2017 from ASH for the Highlights of the ASH 2017 meeting (Dallas, New York, and Latin America). S.C.W. served as cochair of the panel for Antithrombotic Therapy for VTE Disease. N.A.Z. and M.C. report intellectual conflicts as leaders of the group that derived and validated the MITH RAM for VTE risk assessment in hospitalized medical patients. The remaining authors declare no competing financial interests.
Correspondence: Holger J. Schünemann, Department of Health Research Methods, Evidence, and Impact, McMaster University, 1280 Main St West, Hamilton, ON L8N 3Z5, Canada; e-mail: schuneh@mcmaster.ca.
References
Author notes
The full-text version of this article contains a data supplement.