Systematic review of outcome reporting in studies of SCD and pregnancy: marked heterogeneity hinders meaningful data synthesis

TO THE EDITOR:

Sickle cell disease (SCD) is the most common hereditary hemoglobinopathy worldwide. Pregnancy in patients with SCD has been associated with increased rates of maternal and fetal morbidity and mortality.^1-3 Although pregnancy in this population is becoming more common, evidence-based care is hampered by small sample sizes, scarcity of prospective investigations, methodological limitations of available studies,^4,5 and absence or heterogeneity of outcome definitions. This nonuniform outcome reporting subverts meaningful data synthesis; positioning research standardization as the sensible way forward,^6-11 with development of common data elements (CDEs).¹² However, before embarking on CDE creation, a review of literature is required to delineate the current state of outcome reporting in the area of study. To date, no such data were available for SCD in pregnancy. We conducted a systematic review to assess outcome reporting in studies addressing pregnancy outcomes in patients with SCD (International Prospective Register of Systematic Reviews registration number: CRD42021153841). The search strategy (supplemental Appendix 1) was developed and conducted by an information specialist and structured according to the Peer Review of Electronic Search Strategies 2015 guidelines, including publications in English from January 1998 to September 2019. The review was conducted and reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.¹³ Inclusion criteria consisted of publications, with maternal, pregnancy-related, or neonatal outcomes in patients with SCD during pregnancy or the puerperium. Two reviewers (A.K.M. and E.A.) independently screened titles and abstracts, reviewed full texts, and abstracted data of included articles.

Although a study of low methodological quality may alter the interpretation of a systematic review assessing a clinical conundrum, it does not impact the types of outcomes investigated.¹⁴ Because the objective of this study was to compile all contemporaneous outcomes regardless of study quality, as with other similar published investigations,^14-16 no methodological quality assessment was undertaken. Published outcomes were classified according to an established taxonomy,^17,18 mapping outcomes to 38 domains inside 5 core areas.

Following removal of duplicates, 2167 records underwent title and abstract screening. Of those, 186 full texts were assessed for eligibility and 86 were included in the analysis (supplemental Figure 1). Supplemental Table 1 details the characteristics of included studies (references included). These originated equally from Asia, Europe, and South America, (18/86; 21% each), followed by North America (16/86; 19%) and Africa (16/86; 17%). One study (1.2%) included multiple low-resourced countries worldwide, and in fact most (52/86; 61%) were conducted in low-resource settings. Most investigations were retrospective cohorts (63/86; 73%), followed by descriptive case series (7/86; 8%), and case control studies (7/86; 8%). One prospective, nonrandomized control trial (1%) and 2 pending randomized control trials (2%), which have published their clinical protocols including outcome description were also identified.

Overall, 784 distinct outcomes were documented and collated into 133 outcomes (supplemental Table 2). Details of the reported outcomes, including the variable definitions where available, are summarized in supplemental Table 3. Figure 1 depicts the reported outcomes stratified by core areas.¹⁷ Most outcomes (82%) were focused on the “physiological/clinical” core area, followed by “resource use” (11%) and death (5%). None of the studies included outcomes dealing with “life impact” (ie, physical or mental functioning, global quality of life, and compliance or satisfaction).¹⁷ Specific outcomes varied widely across studies. supplemental Figure 3 depicts the association between the proportion of studies and the proportion of outcomes being reported. Only 10% of all documented outcomes were reported in 70% of the included publications. Most of the individual outcomes were reported in less than a tenth of all studies. Not a single outcome appeared across all studies.

Of the 133 reported outcomes, 37 (28%) were documented in at least 10% of studies and are presented in Figure 2. Explicit definitions were lacking in the majority, and substantial heterogeneity was present within definitions suggested for what in essence represented the “same” outcome (supplemental Table 3). For example, hypertensive disorders (reported in 74% of studies) was defined in only 13% of the studies, and definitions varied in terms of (1) systolic and/or diastolic blood pressure criteria, (2) specification of gestational age needed for diagnosis, (3) definition of proteinuria (ie, dipstick, 24 hour urine protein collection), and (4) severity classifications, which were often vague. Similarly, “crisis” (reported in 54% and defined in only 7% of studies) included definitions that were often vague, lacked an objective component, and most often encompassed pain of sufficient severity to require narcotic analgesia or admission for IV treatment. “Vaso-occlusive pain/crisis” more specifically (reported in 33% of studies and defined in 7%) subsumed definitions encompassing “any sickle-related pain” without limitation to a specific body system, or ones restricting pain to “bone or joint painful episode,” “one extremity,” or “bone or abdomen.” Only 1 definition used the visual analog scale. Likewise, “anemia” (reported in 29% of studies and defined in 13%) contained definitions that varied in (1) thresholds (hemoglobin <11, <10.5, <8, <7.5, and <7 g/dL) and (2) time frames (during gestation, before or following transfusion, before birth, or in the puerperium; and not specified in most). This ubiquity of arbitrarily chosen cutoffs, highlights the exigency of concurrent reporting of original values. As an example, reporting of hemoglobin concentration should be required for studies addressing anemia.

Inclusion of composite outcomes further complicate data harmonization efforts. For instance, reporting solely on hypertensive disorders may include any combination of pregnancy-induced hypertension, preeclampsia, eclampsia, or HELLP syndrome (hemolysis, elevated liver enzymes, and low platelets), each with different prognostic implications. Without a rigorous definition and stipulated CDEs, the constitution of the composite outcome is often impossible to ascertain and impractical to meta-analyze.^3,10,18-20 Consequently, reports of composite outcomes should be accompanied by a detailed account of all outcomes that comprise them, including clear definitions and distinct summary measures.

Worth noting is the considerable inequity among reported core areas of the established taxonomy,¹⁷ with most publications (82%) focusing primarily on the physiological/clinical core areas, at the expense of life impact, resource use, adverse events, and death. Although publication of outcomes in the physiological/clinical domain is relatively straightforward from the clinical and research perspective, overlooking the other core areas can risk neglecting endpoints that may be just as important from the perspective of patients, health care providers, or health care systems, detracts from a comprehensive understanding of the condition, and conceivably underestimates the true disease burden. With the under-reporting of the “resource use” core area for instance, incomplete understanding of the essentials required for optimal care may, and likely does, result in insufficient allocation of resources relative the true magnitude of need.

Our systematic review is the first to comprehensively document all outcomes and definitions reported in studies addressing SCD in pregnancy and the puerperium. We have identified (1) marked heterogeneity in reported outcomes, (2) lack of outcome definition in the majority of cases, (3) substantial variation in available outcome definitions, and (4) preferential focus on physiological/clinical areas, at the expense of other domains, such as resource use and life impact. Our findings draw attention to a critical barrier to the acquisition of a robust evidence base for this traditionally understudied population and provide a foundation for future research harmonization efforts. Furthermore, although formulation of CDEs and definitions for the variables of interest remains central to research standardization, supplemental reporting of raw values can provide an auxiliary avenue to enable data harmonization, thereby promoting cross-study comparisons and meta-analysis, especially where clinically debatable outcome definitions are under consideration.

Contribution: A.K.M. created the concept for the project and designed the methodology of the systematic review, performed the literature review, and wrote the manuscript; M.J.R. performed literature search and article review and reviewed the manuscript; E.A. performed literature search and article review, analyzed and organized data, and wrote the manuscript; and N.S., K.H.M.K., and R.W. assisted in analyzing and reviewing the manuscript for submission.

Conflict-of-interest disclosure: The authors declare no competing financial interests.

Correspondence: Ann Kinga Malinowski, Division of Maternal-Fetal Medicine, Department of Obstetrics and Gynaecology, Mount Sinai Hospital, 700 University Ave, Suite 3-909, Toronto, ON M5G 1Z5, Canada; e-mail: ann.malinowski@sinaihealth.ca.