Automated Disease Classification Using Whole Genome Sequencing (WGS) and Whole Transcriptome Sequencing (WTS) Data with Transparent Artificial Intelligence (AI)

Background: Currently, hematologic neoplasms are diagnosed using a combination of methods, which require complex equipment and highly skilled clinical laboratory scientists and technicians - scarce resources.

WGS and WTS could streamline this process and become a singular method. Interpretation of WGS and WTS data in a diagnostic setting is extremely challenging due to the breadth of data and its high-dimensional data types. AI will be mandatory to identify clinically meaningful genetic patterns and produce unbiased diagnosis.

Aim: Compute leukemia diagnosis using AI methods with WGS and WTS data only, depicting relevant features for a decision and thus making its results comprehensible and transparent to humans.

Methods: To train the model we used our cohort of 4,689 samples both with WGS (100x coverage, 2x151bp) and WTS (50 mio reads/sample, 2x101bp), along with our independent final routine diagnosis based on gold standard techniques (GST) and following WHO guidelines. Single nucleotide variants (SNV), structural variants (SV) and copy number alterations (CNA) from WGS data using a tumor w/o normal pipeline and gene fusions (GF) and gene expression (GE) from WTS were extracted. The cohort comprised of 30 different neoplasms and was severely imbalanced (n: 20 - 773).

To test its performance another independent cohort which was not used during model creation (n=202, 22 entities) was selected.

Results: We trained an ensemble of multi-class classifier using SageMaker (AWS, Seattle, WA) based on LGBM implementation of gradient boosted decision trees (Ke et al, 2017) in a one vs. rest architecture (1vRA). The model accuracy reached 85% overall on a 5-fold cross-validation (Fig 1a). Since neighboring disease types such as MGUS/MM, MDS-EB-2/AML/CMML are in some cases difficult to classify correctly using GST, we trained entity-specific classifiers operating independently. Rather than forcing a single predicted class to be predicted as overwhelmingly likely, this architecture accounts for ambiguous entities. In addition to reflecting biological similarity, the 1vRA resulted in improved probability calibration, so that cases with ambiguous leukemias are more easily identifiable by the distribution of predicted class probabilities and flagged for a human. Expected calibration error was only 3.8% for the 1vRA with entity-specific components, compared to 8.7% for a single LGBM model.

Typically, AI methods are black boxes, making it hard for a medical professional to understand predictions, which results in low confidence and acceptance of such systems. Thus, we particularly focused on the transparency aspect of the model. We employed the SHAP library (Lundberg et al, 2017) to retrace the models output and gain insight into the features (i.e. which SNV, SV, GE etc.) predominantly driving classification results (e.g. LPL case Fig 1b). Fig. 1c illustrates the application of SHAP at the global cohort level for two individuals wrongly diagnosed with CML compared to CML correctly predicted cohort. By using a decision plot, we can observe which features are the most important contributors to the model's prediction. Fig 1c shows that predictions for CML are primarily driven by the BCR-ABL1 features, as expected.

In our independent test cohort the following entities reached a very high concordance, such as AML (16/21) AUL (11/12), BCP-ALL (10/10), CML (13/13), HZL (8/8), MGUS (7/7), Multiple Myeloma (9/11), PNH (10/10), T-ALL (6/7). Other clear cut entities with correct high level predictions include BPDCN, FL, LPL, PPBL, NK-cell, HCL-variant and HGBL.

In other entities such as T-NHL results were more heterogeneous, but this was also expressed in the probability scores given by the model. The first choice had a probability score of ~50%, exposing the correct diagnosis as the second likeliest one with ~40%. Test cohort included cases with mixed diagnostic characteristics, e.g. MDS/MPN-RS-T (4/11 correct, 4 predicted as MDS, and 3 as MPN).

Conclusion: We present an AI tool to interpret WGS and WTS data aiming to predict the final diagnosis without any human input and high concordance to today's WHO classification. Due to the high data dimensionality of WGS and WTS data, an impossible feat for a human. The tool is exposed via a web application and visualizations make the automated decisions transparent and verifiable through humans paving a way for better adoption of WGS and WTS into a clinical routine setting.

Figure 1

View largeDownload slide

Figure 1

View largeDownload slide

Close modal