Leveraging a Large Language Learning Model to Improve Health Equity Content in the First-Year Medical School Classical Hematology Curriculum

Background

Disparities in hematology care affect people living with sickle cell disease, hemophilia in females (XX), and Duffy null antigen status, to name a few. Integrating health equity content into the first-year medical student (MS1) preclinical curriculum is crucial for training physicians to provide equitable, comprehensive care. The preclinical years present a unique opportunity wherein essential topics and a framework for clinical reasoning are taught prior to rotating on medical services. This makes it prime for incorporating health equity content that will later inform medical care.

The Yale Health Equity Thread within the Yale School of Medicine (YSM) has defined several domains of health equity, including (1) race & ethnicity, (2) sex & gender, and (3) sexual orientation & gender identity. YSM faculty were previously queried regarding barriers to incorporating health equity content into the MS1 curricula and noted time to develop content, an already packed curriculum, and lack of expertise with this content as barriers. The hematology MS1 course includes lectures and small group workshops that center on clinical cases. Here we examine the use of a large language model (LLM), Humata.ai to identify gaps in health equity domains and address barriers to incorporating health equity content into the MS1 hematology workshop.

Methods

During the 2023- 2024 academic year, 11 instructors led six small group workshops on Bleeding Disorders and Thrombosis. The workshop was provided to Humata.ai. Humata.ai was prompted to identify gaps in the three health equity domains. An audit summary was generated for each clinical case in the workshop and presented to instructors for review. Instructors were then invited to complete a survey and free text questions to assess their desire to incorporate health equity content into the course, identify barriers, and rate the utility of the audit in improving health equity content into the workshops.

Results

There were 7 (64%) survey responses. All respondents expressed a desire to include sex & gender, and race & ethnicity content. There were no reported barriers to incorporating sex & gender or race & ethnicity content. For sexual orientation & gender identity, 72% said “yes” and 28% “maybe” regarding desire to incorporate content. Barriers to incorporating sexual orientation & gender identity content included unfamiliarity with the topic (28%) and lack of time to update course material (28%).

Sex & gender content: 43% of respondents selected “yes” and 57% “maybe” to the Humata.ai audit being helpful with the content. 100% plan to incorporate the suggestions into the small group workshops.

Sexual orientation & gender identity: 57% of respondents selected “yes” and 43%

“maybe” to the Humata.ai audit being helpful with the content. 71% plan to incorporate the suggestions, while 43% “maybe.”

Race & ethnicity: 29% of respondents selected “yes”, 57% “maybe” and 14% “no.” 57% plan to incorporate the suggestions, and 43% “maybe.”

Qualitative responses:

“The sexual orientation and gender identity prompts were the most helpful followed by the sex and gender-based prompts.”

“I think that the audits could be particularly helpful in terms of discussing gender-specific patient presentations and inclusion of teaching material for transgender and non-binary individuals. Although admittedly, I would have probably discussed many of the suggestions if prompted to without the assistance of the AI audit.”

“Discussion of gender affirming hormone therapy and thrombosis [was helpful].”

Conclusion:

Our study demonstrates the potential of leveraging LLMs to enhance inclusion of health equity content into the MS1 curricula. Despite the small number of respondents, the responses signal a strong preference to incorporate health equity content. Barriers to sexual orientation & gender identity inclusion were due to unfamiliarity with the topic and time constraints. Humata.ai audit was helpful in addressing sex & gender-based prompts, but not so much race & ethnicity. The use of LLM to audit course material might decrease barriers to incorporating such content by prompting instructors to consider gaps in curricula. Future steps include ongoing survey of workshop leaders for the upcoming academic year and expanding the audit to include lectures and other workshops and team-based learning activities.

Van Doren:Daiichi Sankyo: Ended employment in the past 24 months, Speakers Bureau; Pfizer/GBT: Ended employment in the past 24 months, Speakers Bureau; Sobi: Ended employment in the past 24 months, Speakers Bureau; Sanofi: Ended employment in the past 24 months, Speakers Bureau; Pharmacosmos, Inc: Consultancy, Honoraria.