International validation of a health-related quality-of-life questionnaire for Hodgkin lymphoma: the EORTC QLQ-HL27

Hodgkin lymphoma (HL) has seen significant advances in treatment, making it 1 of the most curable cancers.¹ To date, the overall 5-year relative survival ranges from 92% for early stage HL to 82% for advanced stage HL, based on data from patients diagnosed between 2011 and 2017.^2,3 The prevalence of HL has increased dramatically, with ∼220 000 patients in the United States currently living with a diagnosis of HL³; in Europe the 5-year prevalence was estimated at 78 000 in 2020.⁴ The incidence is higher in patients aged 15 to 40 years and in those aged >55 years. However, despite the encouraging advances in the treatment of HL, the development of second malignancies after cancer therapy remains a major concern, contributing to both morbidity and mortality in survivors.⁵ Patients diagnosed with HL have a threefold to more than fivefold increased risk of developing secondary malignancies.^5-7 As a result, the current treatment approach seeks to strike a balance between achieving high cure rates and minimizing the risk of treatment-related adverse events and late effects.^1,5 

With improved survival, there has been increased attention to health-related quality of life (HRQoL) of patients and survivors with HL, albeit still to a limited extent. Literature shows that patients with HL report a variety of problems such as functional, neurosensory and cardiopulmonary impairments; fatigue; anxiety; and worries about new symptoms and recurrence of disease.^8-17 Furthermore, for younger people, the aggressive cancer treatment often interrupts developmental milestones, such as obtaining education, establishing relationships, and getting or maintaining a job.¹⁸ Those experiencing symptoms report substantially lower HRQoL than those not experiencing symptoms.¹⁹ 

International recommendations for various hematologic diseases are now increasingly focusing on the assessment of HRQoL.^20,21 These assessments, including functional aspects or symptom burden, provide unique information that may help to facilitate clinical decision making in the setting of hematologic malignancies.²² In 2018, an international expert panel on hematologic malignancies has voiced concern about the limited data in the HRQoL area and advocated for urgent efforts to raise standards of patient-reported outcomes (PROs) in research and practice.⁵ Because there were no HL-specific questionnaires to capture disease-specific symptoms and functional health issues relevant for patients with HL, the European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Group (QLG) previously developed the EORTC Quality of Life Questionnaire (QLQ) HL 27 (QLQ-HL27),²³ to be used in conjunction with the EORTC QLG core questionnaire (QLQ-C30). The QLQ-HL27 consists of 27 items, of which 24 items contribute to 4 multi-item subscales, that is (1) symptom burden because of disease and/or treatment; (2) physical condition/fatigue; (3) emotional impact; and (4) worries about health and functioning; with 3 single conditional items (ie, problems at work or study, worry about working/education, and concerns about the ability to have children).

The aim of this study was to validate the QLQ-HL27 by testing its scale structure, and to evaluate its acceptability and reliability in an international sample of patients with HL.

This study was performed according to the EORTC QLG guidelines for module development.²⁴ In brief, this module development process consists of 4 phases: phase 1, generation of relevant QoL issues; phase 2, conversion of the QoL issues into a set of items; phase 3, pretesting the item list or preliminary module questionnaire; and phase 4, large-scale international field-testing. Phase 1 to 3 have been published previously²³ (Figure 1). This article presents the phase 4 results.

Patients were eligible for inclusion if they were aged ≥18 years at diagnosis, had a confirmed diagnosis of HL,²⁵ and were sufficiently proficient in the local language. Patients with severe dementia or patients with psychotic disorders or severe mental disorders that cause abnormal thinking and perceptions and therefore are not able to complete a questionnaire (according to the physician judgment), were excluded.

Patients were recruited in 12 countries according to the EORTC QLG guidelines. Patients were recruited in daily clinical practice via their local hematologist or (research) nurse. The target sample size of 324 patients with HL was determined by the number of items in the questionnaire × 10 (the accepted “rule of thumb”²⁴). This number was increased by 25% to account for missing data. We aimed to recruit ±70% of patients under, or after, first-line treatment; and ±30% of patients under, or after, second-line treatment. The number of target samples for test–retest and responsiveness-to-change analysis (RCA) was 125 and 75, respectively.

Ethical approval from each participating center was obtained and all patients provided written informed consent. The protocol was approved by the EORTC QLG. The study was coordinated from The Netherlands and collaborators met at the biannual meetings of the EORTC QLG to discuss the project.

Patients completed the EORTC QLQ-C30 (version 3.0),²⁶ EORTC QLQ-HL27,²³ and a debriefing questionnaire. The questionnaire was completed at any time from diagnosis onward, including after treatment; this was defined as the baseline questionnaire. A subset of patients who were clinically stable (ie, those who completed the questionnaire at least 3 months after completion of treatment) were eligible to complete the questionnaire 2 weeks later for a second time for test–retest analysis. For RCA, another subset of patients, who were expected to experience a change in clinical status (ie, on vs after treatment) were eligible to complete the questionnaires again between 3 and 5 months after termination of treatment. EORTC translation guidelines were used to produce questionnaires in the EORTC standard languages and all relevant languages for participating countries.²⁷ The Computer-based Health Evaluation System (CHES²⁸) was used for data collection. Patients had the possibility to complete questionnaires using paper-based versions, electronically at the hospital, or using a remote patient portal configured for this study.

The QLQ-HL27 consists of 27 items, contributing to 4 multi-item subscales and 3 conditional items as determined in phase 1 to 3: symptom burden because of disease and/or treatment (6 items [q1: muscle weakness; q2: aches in muscles/joints; q3; aches in bones; q3: problems with sense of taste; q5: vulnerable veins; and q6: itching of skin]), physical condition/fatigue (4 items [q7: shortness of breath on exertion; q8: lack of energy; q9: felt drowsy; and q10: sudden tiredness]), emotional impact (6 items [q11: mood changes, q12: lack of confidence in body; q13: restless or agitated; q14: dissatisfied how body functions; q15: lacked self-confidence; and q16: difficulty accepting disease]), and worries about health and functioning (8 items [q17: worry about picking up an infection; q18: worry about health in future; q19: worry about recurrence; q20: worry about becoming chronically ill; q21: worry about becoming dependent; q22: worry about getting another type of cancer; q23: worry about treatment causing future health problems; and q24: worry about damage to heart and blood vessels]). The 3 conditional items, which patients complete only if relevant/applicable to them, are about problems at work or study (q25); worry about working/education (q26); and concerns about the ability to have children (q27).

Items are rated using a 4-point response scale (“not at all,” “a little,” “quite a bit,” and ‘‘very much’’) and the reference time frame for all items is the preceding week.²³ The scoring approach for the QLQ-HL27 is identical to that of the EORTC QLQ-C30, that is, calculating the mean of the items of a specific multi-item scale or using the single conditional item score and then converting it into a standardized scale ranging from 0 to 100. A higher score for all the multi-item scales and items represents a higher level of symptomatology or problems.

The EORTC QLQ-C30 comprises 30 items of 5 functional scales (physical, role, cognitive, emotional, and social), 3 symptom scales (fatigue, nausea and vomiting, and pain), 5 single-item symptoms (dyspnea, insomnia, appetite loss, constipation, and diarrhea), 1 item assessing financial impact, and 2 items to rate global health status and QoL.²⁶ For the functioning scales and the global health status/QoL scale, a higher score indicates better health. For the symptom scales/items, a higher score indicates higher symptom severity.²⁶ 

The EORTC QLG phase 4 debriefing questionnaire was used to assess how much time patients took to complete the QLQ-HL27 questionnaire; whether they needed help to complete it; whether any of the items were confusing, difficult to answer, or upsetting; and whether patients had any further comments or suggestions.²⁴ 

Sociodemographic (age, sex, living arrangement, educational level, and employment status) and clinical (treatment type, treatment line, time since last treatment, time since diagnosis, and stage of disease at diagnosis) data were recorded when patients completed the baseline questionnaire and again at the second assessment for patients completing it for RCA, by the medical staff, and based on patients’ hospital records. In addition, a modified version of the Charlson Comorbidity Index (with the addition of high blood pressure)^29,30 and the Eastern Cooperative Oncology group (ECOG) performance status³¹ were collected.

To confirm the hypothesized 4 scales for the QLQ-HL27 (resulting from phase 3), a 4-correlated factor model for confirmatory factor analysis with the lavaan package³² and semTools package in R³³ was used. Diagonally weighted least squares estimation was used with the mean- and variance-adjustment procedure. A mean- and variance-adjusted scaled χ² was calculated for each model. This is the standard (normal theory) χ² statistic divided by a scaling correction to better approximate a χ² under nonnormality. A comparative fit index and the Tucker-Lewis Index with values of ≥0.97 indicate a good fit, and values between 0.95 and 0.97 an acceptable fit. Root mean square error of approximation values of <0.05 indicate a good fit and values, between 0.05 and 0.08 an acceptable fit.³⁴ Because these goodness-of-fit statistics are derived from the models using the χ² test, they too are scaled and become robust to nonnormality. All standardized factor loadings were required to be >0.4 and statistically significant.³⁵ Cronbach α coefficient was calculated for the multi-item scales, with a value of ≥0.70 regarded as adequate reliability.³⁶ 

The test–retest reliability of scales was calculated with the intraclass correlation coefficient (ICC; 2-way random, single measure)^37,38 to test whether scales remained stable in a group in which they were expect to be. An ICC of ≥0.8 was considered good, and ≥0.9 was considered excellent.³⁹ 

Correlations between the scales of the QLQ-HL27 and QLQ-C30 functioning, fatigue, and pain scales based upon discussion with the team were examined using Pearson product-moment correlations to assess convergent validity and to evaluate whether similar constructs, measured by different questionnaires, correlate with each other. It was expected that the symptom burden scale and the physical condition/fatigue scale of the QLQ-HL27 would be correlated with functioning, fatigue, and pain of the QLQ-C30, that is, Pearson r of >0.40.⁴⁰ Furthermore, we expected an r of >0.40 for the emotional impact and worry about health and functioning scale of the QLQ-HL27 and the emotional functioning scale of the QLQ-C30.

Analysis of variance was used to examine known-groups validity, that is, the ability of the QLQ-HL27 to distinguish between groups known to be clinically different. We used ECOG performance score (0 vs 1-3) “on” vs “after” treatment at baseline as grouping variables.

To evaluate whether the questionnaire can pick up change among patients who changed in clinical status and completed the QLQ-HL27 again at least 3 months after end of treatment, RCA were performed by comparing their scores “on” vs “after” treatment. Paired t tests were used to test for the statistical significance of changes in scores at the 2 assessments.

Prevalence of symptoms and worries on the QLQ-HL27 was based on the number of patients who answered, “a little,” “quite a bit,” or “very much” on a certain item.

Analyses were performed using R⁴¹ version 4.0.5, and SAS version 9.4. (SAS Institute, Cary, NC, 1999). A P value of <.05 was considered statistically significant. Effect sizes (ESs) were calculated using Cohen d statistic, with an ES of 0.2 considered small, 0.5, moderate; and 0.8, large.⁴² 

From 2018 to 2021, a prospective sample of 381 patients was enrolled from 12 countries. Fewer patients than planned were recruited in the “under, or after, second-line treatment” group. Mean age of patients was 41 years; 56% were male; 43% were on treatment during baseline questionnaire completion; and mean time since diagnosis was 2.9 years. Additional patient characteristics are reported in Table 1. With respect to RCA and test–retest, 98 (target 75) and 126 (target 125) patients, respectively, were recruited.

Most patients (n = 355; 93%) completed the QLQ-HL27 without missing any item. Item 5 (“Have you had vulnerable veins [eg, when having blood taken or receiving treatment]?”) was most frequently not completed (n = 14, 4%). The 3 conditional questions (items 25, 26, and 27) were completed by 83% (n = 316), 84% (n = 320), and 81% (n = 308), respectively.

The debriefing questionnaire was completed by 377 patients (99%). Completion of the QLQ-HL27 took patients on average 10 minutes, and 88% completed it in ≤15 minutes. Assistance was provided to 15% of patients (n = 58), primarily with practical support (eg, reading and/or writing). Overall, 4% to 5% of patients found at least 1 item confusing or upsetting; mostly, this involved the items on worrying, recurrence, ability to work, and concerns about the ability to have children.

Standardized factor loadings for the original 4-factor model for the QLQ-HL27 were all statistically significant and >0.4 (supplement 1). The models showed acceptable fit, and correlations between the factors ranged between 0.46 and 0.78, with the highest correlations between the symptom burden and physical condition/fatigue factors. Cronbach α value was acceptable to good for all scales of the QLQ-HL27 (supplement 1).

Test–retest reliability revealed no significant differences in responses over time. The ICCs were good, to excellent, for all scales and single conditional items of the QLQ-HL27 (Table 2).

Correlations between the scales of the QLQ-HL27 and the functioning, global health status/QoL, fatigue, and pain scale of the QLQ-C30 are shown in supplement 2. The scales that were predicted to be conceptually related correlated substantially with each other (r > .4).

Patients who were on treatment during completion of the baseline questionnaire had statistically significantly higher mean scores on the symptom burden (P < .01; ES = 0.40), physical condition/fatigue (P < .01; ES = 0.57), and emotional impact scales (P = .04; ES = 0.21) compared with scores from those who were no longer treatment (Table 3). Patients with an ECOG score of ≥1 had statistically significantly higher mean scores on symptom burden (P < .01; ES = 0.50) and physical condition/fatigue (P < .01; ES = 0.60) compared with patients with an ECOG score of 0. Differences in scores on the other scales and single conditional items were not statistically significant.

Patients who had a change in their clinical status (ie, from “on treatment” to “after treatment, ≥3 months”) showed statistically significantly lower scores on symptom burden (P = .02; ES = 0.26) and physical condition/fatigue (P < .01; ES = 0.40). Scores on the emotional impact and worries about health and functioning, and on the 3 conditional single items remained relatively stable (Table 4).

All items of the QLQ-HL27 ranged in score from “not at all” to “very much.” Symptom percentages on the items related to the symptom burden and physical fatigue scales were higher for patients on treatment compared with scores from those who were no longer on treatment at time of the baseline questionnaire (Figure 2A-B). Nevertheless, most symptoms were still frequently reported by those after treatment, for example, muscle weakness (29% a little, and 11% quite a bit/very much), aches in muscles and joints (34% a little, and 20% quite a bit/very much), and lack of energy (42% a little, and 19% quite a bit/very much). The prevalence of problems belonging to the emotional impact scale were also higher for patients on treatment compared with those who were no longer on treatment, except for lacking self-confidence (Figure 2C). The prevalence of worries about health and functioning were similar between patients on treatment and those who were no longer treatment (Figure 2D).

This study represents the final phase of the EORTC module development process and examined the reliability, validity, and psychometric properties of the QLQ-HL27 in an international heterogeneous sample of 381 patients with HL across 12 countries and languages. The QLQ-HL27 comprises 4 scales, that is, symptom burden, physical fatigue, emotional impact, and worries about health and functioning, and can be used among all patients with HL. The questionnaire provides a patient-reported measure of HRQoL in line with the views expressed by patients during the international development process.²³ On average, patients completed the QLQ-HL27 in 10 minutes and it was viewed as easy to complete. The originally hypothesized 4-factor model exhibited acceptable model-data fit, test–retest reliability was good to excellent, and convergent validity was demonstrated. Moreover, the QLQ-HL27 showed to be able to discriminate between groups that are perceived as different with respect to symptom intensity (eg, ECOG performance status, or on vs after treatment).

Responsiveness of the QLQ-HL27 was shown for the symptom burden and physical fatigue scales in patients whose clinical status changed over time, whereby patients reported better scores when they were no longer on treatment compared with when they were on treatment. Scores of the emotional impact and worries about health and functioning scales did not improve over time in the period from treatment until 3 to 5 months after treatment. In a recent, large, multicenter longitudinal German study among 4215 patients with HL using the EORTC QLQ-C30, it was observed that in the first year after treatment, survivors of all stages reported substantial improvements in all HRQoL parameters compared with their HRQoL levels during treatment.¹² The HRQoL of survivors included in that study reached sustained levels within 2 years after treatment and remained stable after that; whereby cognitive, emotional, role, and social functioning, as well as fatigue, dyspnea, and financial problems, were severely and persistently affected.¹² A systematic review also observed that among patients with HL, several aspects of HRQoL including emotional effects can be persistent among patients with lymphoma.⁴³ Our results are also similar to what we observed in the RCA of the validation of other EORTC measures for patients with chronic lymphocytic leukemia (CLL; ie, QLQ-CLL17)⁴⁴ and non-Hodgkin lymphoma (NHL; ie, QLQ-NHL-HG29 and QLQ-NHL-LG20).

For highly curable diseases, such as HL, greater emphasis is now placed on selection of initial treatments with effort to reduce the dose and the field of radiotherapy, to maximally avoid late effects and long-term symptoms and to maintain functioning and HRQoL. Treatments are typically considered to be the most important contributor to the development of late effects. In addition, it appears that impaired HRQoL before treatment is also a significant predictor of impaired/deteriorated HRQoL during survivorship.¹² Increased awareness of this association is therefore needed, and symptom monitoring, starting at diagnosis, can help clinicians to identify patients at risk^45,46 for long-term problems after cancer treatment. Timely interventions or referral to other health care providers can then be provided for those in need; for example, referral to psychosocial-oncology staff for supportive care for patients with severe worries, has been shown to be effective.⁴⁷ 

The recruited patients during both development and validation phases of the EORTC questionnaire development were representative of the HL population with respect to sociodemographic, clinical, and treatment characteristics. Patients included in this study followed the common HL regimes. This has greatly increased the accuracy of the QLQ-HL27 in capturing the most relevant HRQoL aspects for patients with HL currently seen in routine practice. The QLQ-HL27 can therefore be used in clinical trials to evaluate treatment effects on HRQoL. It can also be used in daily clinical practice to help clinicians identify specific symptoms or worries that require further examination and discussion with the patient. To account for possible expected future treatment–related symptoms, the QLQ-HL27 can be complemented with an item(s) from the EORTC Item Library when applicable to the specific treatment. The Item Library is a repository of >950 unique items and is available in many languages and was developed to facilitate flexible and timely measurement of symptoms.⁴⁸ The QLQ-HL27 and scoring algorithm for generating the QLQ-HL27 scale scores is available via the EORTC QLG’s website.⁴⁹ Separate items may be used to calculate the prevalence of issues, although use of the multi-item scales enlarges the reliability.

A limitation of the study was that although the total sample was large, the country samples were too small to perform a country-specific psychometric evaluation (eg, DIF analyses). Furthermore, fewer patients than planned were recruited in the “under or after second-line treatment” group. Overall, study strengths are the crosscultural validation and systematic development according to established EORTC guidelines, and the apparent clinical validity and applicability across treatment phases.

In conclusion, this large-scale international study supports the validity and clinical utility of a newly developed PRO measure for patients with HL. The use of this disease-specific questionnaire (EORTC QLQ-HL27), to be used in conjunction with the EORTC QLQ-C30, makes it feasible to assess most relevant symptoms and functional health issues of patients with HL. Implementation of this specific HL questionnaire in research and practice is expected to further increase quality of PRO research and to generate clinically relevant data that can be used to better inform treatment decision making.

The authors thank Filipa Alves da Costa, Bayan Inserat, Monika Sztankay, Lucia Neppl, Signe Rud Reinhold, Mirella Lentini, Kostas Stamatopoulos, Maria Papaioannou, Alkistis-Kyra Panteliadou, Maria Gomes Silva, and Ana Miranda for their valuable contribution to this study.

The project was awarded an EORTC QLG grant for module development, phase 4.

Contribution: S.O., F.E., C.K., D.P., A.B., K.C., and L.v.d.P.-F. designed the research study; S.O., F.E., O.S., F.C.B., C.d.J., D.D., J.L., S. Malak, D.P., C.W.S., G.C., S. Molica, L.G., E.N., A.B., E.N., A.X., W.A., A.M., A.C.F., K.C., C.C., and C.K. recruited patients and performed the research; S.O. and J.A.v.R. analyzed the data; S.O. wrote the manuscript; and all authors critically revised the manuscript and approved the submitted and final version of the manuscript.

Conflict-of-interest disclosure: F.E. reports consultancy for AbbVie, Incyte, and Syros, outside the submitted work. J.L. reports a role on the advisory board of Amgen; and reports consultancy for Evaluation Software Development, all outside the submitted work. The remaining authors declare no competing financial interests.

Correspondence: Simone Oerlemans, Netherlands Comprehensive Cancer Organisation Research and Development, Godebaldkwartier 419, Utrecht 3511 DT, The Netherlands; e-mail: s.oerlemans@iknl.nl.