A case of apples and pears?

To the editor:

Stasi et al report a valuable systematic review and meta-analysis to conclude that treating Helicobacter pylori is worthwhile in immune thrombocytopenic purpura (ITP).¹  However, we have several reservations about the methodology of this review, which may have led to misleading conclusions.

First, the search strategy was incomplete. Including textwords as well as MeSH headings into the search strategy reveals 38 extra possible studies for inclusion.²  The Cochrane Handbook³  also recommends searching conference abstracts as these may provide studies in prepublication and unpublished studies, which can have significant positive and negative affects on conclusions.⁴ 

Second, the exclusion criteria of this review may have led to bias. No reason is given why studies with fewer than 15 subjects were excluded. Much relevant research has been undertaken in Japan; the exclusion of foreign language studies may have excluded relevant papers, which tend to be negative.⁵  Excluding studies that only diagnosed H pylori infection serologically may also have excluded useful higher quality studies. The quoted reference⁶  describes enzyme-linked immunosorbent assay (ELISA) tests with good sensitivity and specificity. If the authors were concerned about these they could have been examined in a subgroup analysis. The authors thereby excluded a randomized trial,⁷  which may be less biased than observational studies. More information could have been provided on quality assessment and its impact on the review's conclusions.

The authors' measures of publication bias do not demonstrate the absence of bias, as the Egger test should not be used when heterogeneity (I² = 86%), is present.⁸  Visual inspection of the funnel plot does not show the normal inverted V that would be consistent with no publication bias.

Third, we consider that combining studies in a meta-analysis was inappropriate in this area, like combining apples and pears. It would have been more helpful to detail the individual studies especially the control groups, previous treatments and severity of ITP. Egger et al state that there is a danger that meta-analysis of observational data produce very precise but spurious results and the statistical combination of data should therefore not be a prominent component of systematic reviews of observational studies.⁹  The large variation in the study results implies heterogeneity, suggesting either confounding factors, bias, or biologic factors (eg, ethnicity, strain of bacterium), all of which suggest meta-analysis is not appropriate. There seems to have been little attempt to investigate this heterogeneity. Combining observational data by the DerSimonian-Laird method may introduce bias because it increases the weight given to larger studies; in controlled trials larger studies are more precise, but in observational studies they may be more biased.

In summary, we believe the incomplete search strategy and inclusion of several sources of bias compromise the systematic review. The heterogeneity of results obtained implies that meta-analysis should not have been performed. We agree with the authors that further randomized controlled trials are required in this important area.