Discordant pathologic diagnoses of myelodysplastic neoplasms and their implications for registries and therapies

The myelodysplastic neoplasms (MDS) are a heterogeneous collection of clonal, neoplastic hematopoietic disorders with widely variable prognoses and a propensity to evolve to acute myeloid leukemia (AML). MDS is diagnosed in 3.6 per 100 000 people in the United States yearly, translating to ∼20 000 new cases.¹ Median survival ranges from approximately a decade to <1 year² for patients with lower- vs higher-risk MDS (per the revised International Prognostic Scoring System [IPSS-R]^2,3), respectively. Treatment is determined based on disease severity, predominating cytopenia, and patient goals, and can vary from supportive care to hematopoietic cell transplantation.^4,5 Thus, making a precise diagnosis, and distinguishing MDS from benign mimics or other myeloid malignancies, is critical.

The most widely accepted pathologic classification for myeloid neoplasms and acute leukemia, enumerated by the World Health Organization (WHO), has traditionally been based on bone marrow morphology⁶ although it has recently undergone a revision, identifying subtypes by genetic abnormalities.⁷ Despite the strict guidelines governing MDS diagnoses, accurate pathologic assessment presents challenges because of interobserver variability in interpreting morphology, blast percentage, and difficulties in quantifying dysplasia. This may also affect the accuracy of population-based data from cancer registries such as the Surveillance, Epidemiology, and End Results program.⁸ Although definitions of MDS are evolving with incorporation of molecular mutations, mutation testing methods are not uniform. Previous single-center studies have shown disagreements between initial and subsequent MDS bone marrow reviews in up to one-quarter of patient samples.⁹ 

The National Heart, Lung, and Blood Institute (NHLBI) National MDS Natural History Study (NCT02775383) is an ongoing prospective cohort study conducted across 144 sites in the United States and Israel intended to establish a data and biospecimen repository to advance the understanding of MDS.¹⁰ Patients are enrolled at the time that they are suspected of having MDS or have been diagnosed with de novo or therapy-related MDS within the past 12 months and are having a bone marrow biopsy performed as part of routine care. A diagnosis of MDS, related malignancies, or other causes of cytopenias is made by a local pathologist, using WHO 2016 classifications, and then confirmed or refuted by study-affiliated central pathologists who review bone marrow specimens, and clinical and laboratory data. Thus, the study provides a unique opportunity to rigorously compare initial and subsequent bone marrow reviews, with additional tertiary reviews to adjudicate disagreements.

In this study, we compared initial, local MDS diagnoses with central, adjudicated review to quantify rates and degrees of clinically meaningful differences between broad disease categories. We further determined whether diagnostic misclassification affected the therapy received.

The NHLBI MDS Natural History Study protocol is approved through the National Cancer Institute central institutional review board, and informed consent was obtained from all patients before participation. Patients who were eligible were selected from those in the ongoing study with enrollment dates from June 2016 to 27 March 2020, when enrollment was paused because of the COVID-19 pandemic. At enrollment, patients either underwent a bone marrow diagnostic workup for suspicion of MDS, or had been diagnosed with de novo or therapy-related MDS within the past 12-months, had not received hematopoietic growth factors in the past 6-months, had not received any other therapy for MDS, and were undergoing a bone marrow diagnostic workup. Participants were excluded from the study if they had a solid tumor or hematologic malignancy (except for in situ cancers of the skin [basal or squamous cell], uterine cervix, bladder, breast, or prostate) or had received radiotherapy or any nonhormonal treatment for cancer within 2 years of enrollment; had an established hereditary bone marrow failure syndrome; or had aplastic anemia, paroxysmal nocturnal hemoglobinuria, amegakaryocytic thrombocytopenic purpura, or isolated large granular lymphocyte leukemia, as previously described.¹⁰ The decision to perform diagnostic assays (eg, flow cytometry) was at the discretion of the local provider. Similarly, evaluation of hereditary bone marrow failure was done by the enrolling investigator based on the participant’s clinical history. Cases were excluded if the local site diagnosis was not obtainable, or if a patient withdrew before central review. Several assessments were performed at follow-up visits including, but not limited to, laboratory evaluations, clinical assessment, patient reported outcomes assessments, and treatment regimens. Patient flow is described in the study CONSORT diagram (https://thenationalmdsstudy.net/system/files/MDS_Consort_0.pdf) and full details of patient eligibility criteria can be found in the study protocol (https://thenationalmdsstudy.net/system/files/20220908_MDS_NHLBI_PROTOCOL.pdf).

The complete histopathology review process for a participant enrolled in this study and the data and samples available at each review stage are illustrated in Figure 1. In brief, local site staff submitted to the central laboratory the local pathologist’s diagnosis (referred to as the "original local diagnosis”) based on the WHO 2016 classification,⁷ in addition to other source data, which can include targeted clinical history, molecular or cytology assays, unstained bone marrow biopsies, aspirates, and peripheral blood samples. Upon receipt, bone marrow slides and peripheral blood smears were stained by the central laboratory, and the central pathologist (L.Z. and L.M.) conducted a review to determine diagnosis (referred to as the “central diagnosis”). The central pathologist also validated the diagnosis in the local pathology report (referred to as the “validated local diagnosis”) prepared by the local pathologist against the original local diagnosis submitted by site staff. In general, the materials available to both the local and central pathologist were the same except that the central pathologist also had access to the local pathology report. In the original study design, the local pathology report was made available to the central pathologist only after the initial diagnosis was made, at which point the central pathologist was permitted to change the diagnosis. Central pathologists were unblinded to local pathology reports for the entire central review process for any reviews conducted on or after 29 July 2019. The study did not have any requirements regarding the certification or qualifications of locally employed pathologists. We used a large number of sites to obtain a representative snapshot of centers, as well as the range of experience that exists among pathologists, across the country. Because this is an ongoing study, the protocol, CONSORT diagram, along with other supporting documents/information can be found on the study website (https://thenationalmdsstudy.net/mds-study-information).

If the central diagnosis and validated local diagnosis agreed in the broad disease diagnosis (eg, both identified a case as MDS, regardless of MDS subtype), then the consensus diagnosis was considered and referred to as the “final study diagnosis.” If there was discordance between the central diagnosis and validated local diagnosis (eg, a local diagnosis was benign cytopenia and a central diagnosis was MDS), an independent, tertiary review was performed by 2 board-certified hematopathologists to adjudicate the discrepancy, and whose diagnosis was then considered the final study diagnosis. Tertiary referral academic centers are the primary practices for the central and tertiary pathologists whereas the local pathology practices are more varied (mix of academic and community centers).

Initial comparisons between local and central pathologic diagnoses relied on the diagnosis as coded on electronic reporting forms from a local site, rather than the local pathology report. In cases of disagreement identified using electronic reporting forms, the actual local pathology report was assessed to determine whether discrepancies between the local pathologist diagnosis and the central pathologist diagnosis persisted. Hence, discrepancies in diagnosis between local and central reviews were categorized as events in which pathologists disagreed on diagnosis (disagreement between the final study diagnosis and the validated local diagnosis), or events in which pathologists agreed on diagnosis but a data entry error at the local site miscoded the local pathologist’s diagnosis (disagreement between the final study diagnosis and the original local diagnosis). This type of data entry error, in which the original local diagnosis as entered on the study case report form differed from the local pathologist’s determination as recorded on the pathology report, is referred to as a “site miscoding error.”

Local site and central pathologists classified patients into 1 of the following disease groups: MDS, MDS/myeloproliferative neoplasm (MPN) overlap, idiopathic cytopenia of undetermined significance (ICUS), AML with <30% blasts without core binding factor or acute promyelocytic leukemia, or an alternative diagnosis (defined as participants with AML with ≥30% blasts, nonmyeloid malignancies, or other cytopenia or cancers). Only patients with a final study diagnosis of MDS, MDS/MPN, ICUS, and AML with <30% blasts are enrolled into a longitudinal cohort for follow-up approximately every 6 months, whereas patients with alternative diagnoses are assigned into a cross-sectional cohort with no further follow-up expected. Because of the study’s intent to not affect individual care decisions, local sites were informed only of the assignment into the longitudinal or cross-sectional cohort and not of the final study diagnosis. Patients were expected to complete follow-up visits until the local site was informed of the cohort assignment.

Data were also collected on therapies administered to patients during follow-up. In general, these data were collected only for patients assigned to the longitudinal cohort. However, therapy data may have been collected for some cross-sectional participants with alternative diagnoses whose cohort assignment was pending at the time of the follow-up visit. A panel consisting of National MDS Natural History Study steering committee members with significant clinical experience treating patients with MDS (E.J.G., A.E.D., T.A.B., W.S., and M.A.S.) was formed to assess the appropriateness of therapies administered to study participants for whom there was discordance in the validated local and final study diagnosis. The appropriateness of therapies was evaluated based primarily on the patient’s final study diagnosis but also based upon other pathology, laboratory, and clinical data. Pairs of reviewers from the panel were randomly assigned to each case and independently made determinations of therapy appropriateness; a third reviewer from the panel adjudicated assessments that differed between the paired reviewers.

Frequencies and proportions of discrete demographic factors, including sex, race, and ethnicity, and mean (standard deviations) of continuous demographic factors were summarized by the final study assignment categories. Proportions of cases classified into each disease category were tabulated by local and central review assignments. Concordance rates between local and central assignment were summarized for all cases and by disease category. κ statistics and 2-sided 95% confidence intervals (CIs) were used to assess the level of agreement. Proportions were compared using Fisher exact tests. A 2-sided significance level of .05 was applied for all statistical tests to denote significance. All statistical analyses were conducted using SAS version 9.4.¹¹ 

A total of 918 participants with MDS or suspected MDS who underwent local and central pathology review were included in the analysis. The number of participants as a percentage of the total enrolled in the NHLBI MDS Natural History Study, by year, was: 2016, 100% (75/75); 2017, 99% (253/256); 2018, 97% (270/279); 2019, 74% (252/342); 2020, 24% (68/281). The median age was 72 years (range, 20-95) and the majority of participants were male (n = 563, 61%), White (n = 829, 90%), and enrolled from Midwestern states (n = 490, 53%), with Wisconsin contributing the highest percentage of participants (n = 180, 20%). Mean (±standard deviation) blood counts before diagnosis were hemoglobin of 10.8 (±2.4) g/dL, platelet count of 180.5 (±154.9) × 10⁹/L, and absolute neutrophil count of 3.1 (±3.6) × 10⁹/L. Among participants included in the longitudinal cohort (n = 375), the median follow-up time was 2.3 years. There have been n = 188 deaths in this group to date. The vast majority of participants (94%, n = 864) were enrolled at the time of diagnosis, whereas 6% (n = 54) were enrolled within 12 months of a previous diagnosis of MDS. These 2 groups were similar in terms of their demographics and final study diagnoses (supplemental Table 1). As expected, those enrolled with a previous diagnosis of MDS were more likely to have a final study diagnosis of MDS.

Based on diagnosis data entered locally by the sites, 264 patients (29%) were diagnosed as having MDS, 15 (2%) MDS/MPN overlap, 62 (7%) ICUS, 0 (0%) AML with <30%, and 577 (63%) other (including 46 [5%] other AML [>30% blasts], 49 [5%] other malignancies, and a remaining group of 482 [53%] with other diagnoses; Table 1). After final central pathology review, this distribution shifted, with 266 (29%) diagnosed as MDS, 45 (5%) MDS/MPN overlap, 49 (5%) ICUS, 15 (2%) AML with <30%, and 543 (59%) other (including 38 (4%) other AML, 93 (10%) other malignancies, and a remaining group of 412 (45%) with other diagnoses). Categorization and final diagnoses of the “other” group in the MDS Natural History Study can be found elsewhere¹² and is also included as supplemental Table 2.

Among patients classified as having MDS based on local data entry (Table 2), MDS unclassifiable and MDS with multilineage dysplasia were the most common WHO classifications⁷ (24% and 18%, respectively). After final central pathology review, these classifications remained similar except that patient bone marrows that were classified as MDS unclassifiable decreased to only 6%, whereas MDS with multilineage dysplasia increased to 27% (P < .001). Fifty-one percent of MDS participants were categorized as having very low– or low-risk IPSS-R disease, compared with 23% as intermediate-risk disease and 26% as high- or very high–risk disease. The overall IPSS-R cytogenetic risk category distribution is provided in supplemental Table 3.

Central pathologists verified the original local diagnosis only 67% of the time (n = 615, Table 3), with a κ statistic of 0.54 (95% CI, 0.49-0.59), indicating a moderate level of agreement.^13,14 For MDS specifically, central pathologists agreed with the original local diagnosis 73% of the time, which was greater than the concordance observed for MDS/MPN (20%), ICUS (41%), AML with <30% (0%), and other nonmyeloid malignancies (28%) cases. Nearly 50% (n = 147) of the 303 discordant cases were misclassified locally as having other cytopenia or cancers. Cytogenetic analysis was performed in 86% (789/918) of the study participants. For each case, local and central pathologists had access to identical cytogenetic information, thus, central pathologists did not have an advantage when classifying cases.

Of MDS participants, ∼56% were categorized with a lower-risk IPSS-R score compared with 20% and 25% with an intermediate and higher risk score, respectively. Agreement between central and local IPSS-R scores was high overall at 85.4% (152/178), with agreement of 87.1% (74/85) for very low/low scores, 75.6% (31/41) for intermediate scores, and 90.4% (47/52) for very high/high scores; the associated κ statistic of 0.83 (95% CI, 0.76-0.89) indicates almost perfect agreement.

Site miscoding errors, in which the local diagnosis as entered on the study case report form differed from the local pathologist’s determination as recorded on the pathology report, accounted for over half (53%, 162/303) of the local misdiagnoses. After reclassifying the local diagnosis based on local pathology review (ie, correcting for site miscoding errors), the agreement between local and central diagnoses improved from 67% to 85%. The κ statistic of 0.78 (95% CI, 0.74-0.81, Table 4) indicates better agreement after correction of site miscoding errors. For MDS diagnoses specifically, the local and central pathologically confirmed concordance improved to 79%, and to >62% within each of the MDS/MPN, ICUS, AML with <30% and other nonmyeloid-malignancies groups.

Figure 1 shows the information to available to pathologists at the time of local, central, and tertiary review. Genetic data generated by a site was also available centrally, so that local and central pathologists had access to the same information. Of all cases, ∼10% (88/918) had genetic data available at diagnosis. Local and central diagnostic agreement was very similar between those with and without genetics available (with genetics, κ = 0.73 [95% CI, 0.61-0.85]; without genetics, κ = 0.78 [95% CI, 0.74-0.81]; supplemental Tables 4 and 5). Of the 15% true misdiagnosed cases (ie, after correcting for site miscoding errors), 12% had molecular genetic data at the time of diagnosis. By MDS subtype, the rate of agreement between validated local diagnosis and central diagnosis was slightly lower (70%; supplemental Table 6). The overall κ of 0.65 (95% CI, 0.58-0.72) still indicates substantial agreement in diagnosis across MDS subtypes.

Therapy reviews were done for 97% of patients (363/375) with a final study diagnosis of MDS, MDS/MPN, ICUS, or AML with <30% blasts, and a small number of patients with other diagnoses (other AML, other malignancy, or other: 10%, 55/543). Across these 418 patients, therapies used to treat MDS and related myeloid malignancies were reported in 37% (n = 154, Table 5), including 43% of patients with MDS, 49% with MDS/MPN, and 86% with AML with <30% blasts. Antineoplastic agents were the most common regimens, administered in 83% of all participants and 84% of participants with MDS. Hypomethylating agents such as azacitidine or decitabine were among the most commonly prescribed antineoplastic agents, given to 66% of patients overall and 72% of participants with MDS. Treatment rates were significantly lower for cases with discordant diagnoses (25%) compared with those with diagnoses on which local and central pathologists agreed (40%) after accounting for site-entered coding errors (24/95 vs 130/323; P = .008).

A total of 7% (7/95) of the treated patients with local and central discordance in diagnosis received a total of 14 inappropriate therapies based on review by the centralized therapy review committee (Table 6). In the centrally diagnosed MDS disease group, 3 (6%; locally misdiagnosed as MDS/MPN overlap, other malignancy, or other) received inappropriate therapies consisting of immunosuppressants, corticosteroids, and nonhypomethylating antineoplastic agents. Immunoglobulins, corticosteroids, and nonhypomethylating antineoplastic agents were also considered inappropriate recommendations for 3 cases centrally diagnosed as MDS/MPN (locally misdiagnosed as ICUS or other) or AML with <30% (locally misdiagnosed as MDS). Azacitidine was also considered an inappropriate recommendation for a centrally diagnosed ICUS case (locally misdiagnosed as other).

MDS can be challenging to distinguish from other related diagnoses, such as other myeloid malignancies or precursor conditions associated with cytopenias. Skilled pathologists collaborating with experienced clinicians are critical to ensuring that patients receive the correct diagnosis and appropriate therapy. In this study, we examined a well-characterized cohort of patients enrolled into the NHLBI MDS Natural History Study to determine differences in diagnosis between initial bone marrow biopsy review and central adjudicated review; whether differences were because of database coding errors or specimen interpretation; and how diagnostic misclassification affected administered therapy.

We found that in one-third of cases central pathologists changed local pathologist diagnoses in a clinically meaningful way (eg, from a nonmalignant process to a malignant process, or to a process that substantively could affect treatment or prognosis). Interestingly, simple miscoding errors to a central database represented >50% of the initially apparent misdiagnoses, which when corrected resulted in a true overall misdiagnosis rate of 15% for the entire cohort, one-fifth for all MDS and oligoblastic AML diagnoses, and more than one-third for related myeloid malignancies.

The true impact of misdiagnoses is reflected in both patients being provided with misinformation, which could affect their planning and understanding of prognosis; and, more seriously, in them receiving the wrong or delayed appropriate therapy. We found that 7% of patients for whom there was a discordance in diagnosis received inappropriate therapy, and, in 1 case, appropriate therapy was not received until ∼18 months after the initial misdiagnosis.

These findings have broad implications for the accuracy of clinical trials of new drugs or novel drug combinations in MDS and related myeloid malignancies. Given the relatively high percentage of patients misdiagnosed based on local pathologist findings or data entry errors, central review of bone marrow aspirates and biopsies, or at the very least local pathologist reports, should be standard, particularly for trials at later stages of drug development.

Hematopathologic classifications are increasingly complex with the introduction of the WHO 2022 and International Consensus Classification 2022.^7,15 Here, we have shown, even before the current era with 2 systems, the challenges in reporting MDS diagnoses accurately. Given the complexities of these diagnoses, we suggest seeking an academic hematopathology confirming consultation when feasible and/or repeat bone marrow biopsy/aspirate with a second opinion to refine or clarify diagnoses and ensure that patients receive the most appropriate therapy. Research personnel should receive training specific to MDS and related conditions to minimize errors in database population.

These findings help to explain discrepancies in regional and national database reporting of MDS incidence rates, subtypes, and outcomes. For example, 1 such study¹⁶ that included data from the North American Association of Central Cancer Registries and the Surveillance, Epidemiology, and End Results program on almost 25 000 patients with MDS reported MDS–not otherwise specified as occurring in 56% of those patients. Although the error rate for MDS coding within such registries is unknown, it could be as high as that observed in this study. Extrapolating our findings, we might expect that 21% of the total reported patients with MDS might not have an MDS diagnosis at all, and 75% of the patients with MDS–not otherwise specified classified would have refined WHO diagnoses of other specific MDS subtypes. Such measurement error may limit the interpretation of population-based epidemiologic studies of MDS epidemiology, etiology, and survival.

This study does have limitations. Although this was a study conducted in over 140 sites across the United States, applicability to any individual site’s diagnostic accuracy and international generalizability may be limited. Sites participating in the current study may be different from other sites, for example, in terms of resources, pathologist experience, pathologist certifications, experience of data-entry site staff, and volumes of patients with MDS, which could impact the accuracy of local MDS diagnosis and applicability of findings on a population level. Similarly, differences among participating sites, and between local and central laboratories (eg, variability in stain quality) may have contributed to discordant diagnoses among pathologists. Pathologic diagnoses were based on WHO 2016 criteria; although these have since been revised and could affect discrepancies in diagnosis within MDS, we report discrepancies between diagnoses of MDS and other conditions, both malignant and nonmalignant.

Unfortunately, mutational testing is not as ubiquitous as preferable in patients ultimately shown to have myeloid malignancies or to be at risk of these conditions. Our analysis focused on data available at baseline, and is a real-world snapshot into available data, which was identical locally and centrally. Mutational testing was subsequently performed centrally to bolster the study as a national resource, with some patients being reclassified as having Clonal Cytenia of Undetermined Significance, although these data were not available to central pathologists for purposes of these analyses. This study was designed to capture patients with suspected MDS or related conditions, and this occurred >50% of the time. That this percentage rate was not higher despite protocol guidance regarding ruling out other causes of cytopenias may reflect the enrollment of patients with mild cytopenias and the broader prevalence of older adults with cytopenias because of other etiologies in the United States population as a whole.

In conclusion, misdiagnosis of patients with suspected MDS occurred at an appreciable rate, as did miscoding errors of MDS diagnoses, and this resulted in mistreatment of patients in a minority of cases. The accuracy of MDS diagnoses and prognosis can be dependent upon strong collaboration between clinicians and skilled pathologists. Second-opinion diagnoses sought by patients or medical providers may be critical to limiting the frequency at which misinformation is given and improve the course of recommended therapy.

On behalf of the investigators of the National Heart, Lung, and Blood Institute National Myelodysplastic Syndromes (MDS) Natural History Study.

The National MDS Natural History study has been supported by US Federal Government contracts HHSN268201400003I and HHSN268201400002I from the National Heart, Lung, and Blood Institute, and additional funding by the National Cancer Institute (NCI) to NCI Community Oncology Research Program and the participating clinical centers.

The content is solely the responsibility of the authors and does not represent the policy of the National Institutes of Health or the Department of Health and Human Services.

Contribution: E.J.G. and M.A.S. designed the study and wrote the manuscript; D.H. and M.O. performed the statistical analyses; and all authors contributed to study design, interpretation of the data, preparation of the manuscript, and approved its content.

Conflict-of-interest disclosure: D.E.R. reports membership on a board or advisory committee for NanoString Technologies, Inc. R.B. is an employee of Aptose Biosciences; holds stock in Aptose Biosciences Private Company; consults for Bristol Myers Squibb (BMS), Astex, and AbbVie; holds membership on a board or advisory committee of Gilead and Epizyme; and reports consultancy and research funding from Takeda. T.A.B. holds stock in BMS Private Company; reports membership on a board or advisory committee of BMS, Cardinal Health, and Eli Lilly; and holds stock in Heron Therapeutics Private Company. E.P. received research funding from BMS, Kura, and Incyte, and reports honoraria from Taiho and Blueprint. A.E.D. reports consultancy for and membership on a board or advisory committee of Taiho, Novartis, and BMS, and membership on a board or advisory committee of Takeda. M.A.S. reports membership on a board or advisory committee of Novartis, Kurome, and BMS. The remaining authors declare no competing financial interests.

Correspondence: Mikkael A. Sekeres, Division of Hematology, Department of Medicine, Sylvester Comprehensive Cancer Center, University of Miami, 1120 NW 14th St, Suite 610M, Miami, FL 33156; e-mail: msekeres@med.miami.edu.