Incomplete harms reporting in systematic reviews of direct oral factor Xa inhibitors Open Access

TO THE EDITOR:

Atrial fibrillation (AF) affects nearly 6 million Americans and could climb to 16 million by 2050.¹ Venous thromboembolism adds further thrombotic burden, and both conditions are now commonly managed with direct oral factor Xa (FXa) inhibitors, such as apixaban and rivaroxaban, whose compelling efficacy and freedom from routine laboratory monitoring have driven them to the forefront of anticoagulant care.² Clinical guidelines now prioritize these agents over warfarin in both AF and venous thromboembolism algorithms, accelerating their uptake worldwide.² Prescription volumes have surged accordingly, reflecting confidence in their ease of use and robust trial evidence. Nevertheless, pharmacovigilance databases continue to record thousands of serious bleeding events each year, underscoring the need for rigorous postmarketing safety synthesis.³ These agents also account for the largest share of emergency-department visits for medication harm, chiefly bleeding, in the United States.³ Systematic reviews (SRs) underpin guideline recommendations, but meta-research in other fields shows they often underreport adverse events while fully detailing efficacy.⁴ We therefore evaluated, how comprehensively SRs of apixaban and rivaroxaban report harms, in a Preferred Reporting of Systematic Reviews and Meta-Analyses 2020 (PRISMA-2020)–compliant assessment, to determine whether current evidence syntheses give clinicians and patients a balanced view of bleeding risk.

A PROSPERO (International Prospective Register of Systematic Reviews)–registered protocol guided this PRISMA-2020–compliant cross-sectional study. A medical librarian searched MEDLINE, EMBASE, Epistemonikos, and the Cochrane Database of Systematic Reviews from database inception through 2022; full strategies and yields appear in supplemental Material (Open Science Framework).⁵ We additionally hand-searched reference lists and relevant conference proceedings to capture gray literature, ensuring maximal sensitivity. Duplicate citations were removed with EndNote X9’s automated routine followed by manual verification. Two calibrated reviewers independently screened records in Rayyan, extracted data with a piloted Google Form and resolved disagreements with a third reviewer. Harms terminology followed the PRISMA-Harms checklist⁶ (supplemental Figure 2); completeness was appraised with the 12-item Mahady checklist⁴ and the Loke adverse-effects framework⁷; methodological quality with AMSTAR-2.⁸ Overlap of primary trials between SRs was quantified with the corrected covered area (CCA) index,⁹ and χ² or Fisher exact tests (Stata 16.1) explored associations with P < .05.

Our search retrieved 3781 records; 376 duplicates were removed, 3293 titles/abstracts were excluded, and 70 SRs met eligibility (Figure 1).

Sixty-nine (98.6%) SRs identified harms as a primary outcome; 38 (54.3%) applied a severity scale; and 43 (61.4%) concluded that FXa inhibitors were favorable (Table 1). The most frequently studied indications were for the prevention of deep vein thrombosis/phycoerythrin (31/70, 44.3%), and risk reduction of stroke and embolism in patients with nonvalvular AF (28/70, 40.0%). The median number of randomized trials per SR was 8 (interquartile range, 5-13). English-language restriction was present in 51 (72.9%) reviews, potentially introducing language bias. Funding sources were disclosed in only 18 (25.7%) SRs, 9 of which reported industry sponsorship.

Using the AMSTAR-2 appraisal tool, 69 of 70 (98.6%) SRs were found to be of “critically low” reporting quality and 1 (1.4%) SR was rated as “low” reporting quality.

Forty-seven SRs (67.1%) fulfilled >50% of Mahady items (supplemental Table 2). Although nearly every review mentioned harms in the title/abstract (98.6%) and defined them in methods (90.0%), only 7 (10.0%) reported treatment discontinuations, and 10 (14.3%) embedded harms terms in their search strategy. Fewer than one-quarter (20.0%) suggested future research to clarify safety signals.

Prespecification of harms occurred in 67 SRs (95.7%), yet only 5 (7.1%) publicly posted a protocol addressing safety outcomes, and 10 (14.3%) detailed methods for handling missing harms data (supplemental Table 3).

Across the 70 SRs, 242 unique randomized trials were identified; overall CCA was 2.4%. Twelve SR dyads exhibited high overlap (CCA ≥ 50%) but overlapped on <60% of harms, indicating selective extraction (supplemental Table 4). This discrepancy shows that reviews drawing on nearly identical primary evidence can still diverge markedly in the safety outcomes they present. Completeness of harms reporting was unrelated to authors’ claims of PRISMA adherence (P = .61).

Harms reporting in SRs of apixaban and rivaroxaban remains strikingly incomplete: one-third met ≤50% of recommended items, 90% omitted treatment discontinuations, the clearest patient-centered signal of intolerability, and 46% failed to apply validated bleeding scales, undermining risk–benefit appraisal in older or polypharmacy populations. Reviews published after 2020 fared no better than earlier work, suggesting slow uptake of updated guidance, whereas teams with PROSPERO protocols achieved modestly higher Mahady scores, underscoring the value of prospective planning. Similar deficiencies have been observed in other therapeutic areas: Mahady et al, found that one-third of gastroenterology SRs omitted harms altogether, highlighting the cross-disciplinary nature of the problem.⁴ Our own sample showed that treatment discontinuations were the single most neglected element, an omission that conceals real-world intolerability and the sudden thromboembolic risk that follows abrupt cessation of anticoagulation. Nearly half of reviews likewise failed to grade bleeding severity, leaving clinicians uncertain whether reported events represented nuisance bruising or life-threatening hemorrhage. Strengths of our study include a preregistered protocol, coverage of 4 databases, masked duplicate screening/extraction, and combined use of Mahady and Loke frameworks; limitations include potential misclassification of older reviews by AMSTAR-2, focus on 2 FXa inhibitors, and cross-sectional design. Ultimately, to remedy these gaps, journals and guideline developers should require a completed PRISMA-Harms or Mahady checklist, mandate reporting of treatment discontinuations and severity-graded bleeding end points, and promote harmonized terminology (eg, MedDRA) in line with the CONSORT-Harms extension.¹⁰ Such measures would curb selective extraction, preserve clinically relevant safety signals, and provide clinicians and patients the balanced evidence needed to judge whether the hemorrhagic risks of direct oral FXa inhibitors are justified by their proven thromboembolic benefits.

Our cross-sectional analysis was conducted based on the guidelines established by PRISMA. No human subjects were involved in our study, and thus, institutional review board oversight was not required.

Acknowledgments: The authors are grateful to Courtney Kennedy who assisted in the development of our search strategy and to The Oklahoma State University medical library for their procurement of relevant literature. The authors are grateful for the corrected covered area guidance and code provided by Riaz Qureshi.

Contribution: All authors helped design and conduct the study, collected data for analysis, performed the data analysis, prepared the manuscript, and approved the final manuscript.

Conflict-of-interest disclosure: M.V. reports funding from the National Institute on Drug Abuse, the National Institute on Alcohol Abuse and Alcoholism, the US Office of Research Integrity, Oklahoma Center for Advancement of Science and Technology, and internal grants from Oklahoma State University Center for Health Sciences, all outside of the present work. The remaining authors declare no competing financial interests.

Correspondence: Adam Khan, Department of Medical Student Research, Oklahoma State University Center for Health Sciences, 1111 W 17th St, Tulsa, OK 74107; email: adam.khan@okstate.edu.