Patient-reported outcomes (PROs) are any outcome evaluated directly by the patient himself and based on the patient's perception of a disease and its treatment(s). PROs are direct outcome measures that can be used as clinical meaningful endpoints to characterize treatment benefit. They provide unique and important information about the effect of treatment from a patient's view. However, PROs will only be considered adequate if the assessment is well-defined and reliable. In 2009, the FDA has issued a guidance, which defines good measurement principles to consider for PRO measures intended to give evidence of treatment benefit in drug development. In hematologic clinical trials, when applied rigorously, they may be used to evaluate overall treatment effectiveness, treatment toxicity, and quality of patient's well-being at short-term and long-term after treatment from a patient's perspective. In situations in which multiple treatment options exist with similar survival outcome or if a new therapeutic strategy needs to be evaluated, the inclusion of PROs as an endpoint can provide additional data and help in clinical decision making. Given the diversity of the hematological field, the approach to measurement needs to be tailored for each specific situation. The importance of PROs in hematologic diseases has been highlighted in a number of international recommendations. In addition, new perspectives in the regulatory field will enhance the inclusion of PRO endpoints in clinical trials in hematology, allowing the voice of the patients with hematologic diseases to be taken into greater consideration in the development of new drugs.

Learning Objectives
  • Understand the general principles around the assessment of patient-reported outcomes in drug development

  • Understand the importance of the patient's perspective in drug development for hematology and the most recent advances in assessment and regulatory issues

Our purpose with this paper was to give an overview of the assessment of patient-reported outcomes (PROs) in the context of drug development in hematology. First we present the general principles around the assessment of PROs in a drug development context. In a second part, we present how this general framework applies specifically to the field of hematology. Finally, we introduce some recent developments that will likely drive the field forward in the near future.

As defined by the US Food and Drug Administration (FDA), a patient-reported outcome “is any report of the status of a patient's health condition that comes directly from the patient, without interpretation of the patient's response by a clinician or anyone else.”1  For the European Medicines Agency (EMA), “any outcome evaluated directly by the patient himself and based on patient's perception of a disease and its treatment(s) is called patient-reported outcome (PRO). The term PRO is proposed as an umbrella term to cover both single-dimension and multi-dimension measures of symptoms, health-related quality-of-life (HRQL), health status, adherence to treatment, satisfaction with treatment, etc.”2 

Treatment benefit is demonstrated by evidence that the treatment has a positive impact on a meaningful concept of interest.1,3,4  On the regulatory standpoint, clinically meaningful endpoints for the characterization of treatment benefits are direct measures of how a patient feels, functions, or survives, and are expected to predict the effect of therapy.5  They represent or characterize the clinical outcome of interest, either objective (eg, survival, disease exacerbation, clinical event) or subjective (eg, symptoms, health-related quality-of-life). Indirect outcome measures or surrogate endpoints, such as physical signs, observations, or laboratory tests/biomarkers, are substitute for direct outcome measures, and therefore do not confer full evidence on clinical benefit to the patient. Treatment benefit can be demonstrated as either a comparative advantage in how patients survive, feel, or function, or a comparative reduction in treatment-related toxicity. Thus, well-defined PROs are direct outcome measures, and can be used as clinical meaningful endpoints to characterize treatment benefit.

As a result of the FDA PRO Harmonization meetings in the early 2000s, a consensus on the value of the patient's perspective and the rationale for its inclusion in drug development emerged and was structured around 4 key points6  i.e., the patient's perspective is:

  1. A unique indicator for assessing disease impact: as such, PROs may be the sole indicator of disease activity (eg, in migraine, rhinitis, etc), a key indicator (eg, in asthma, psoriasis, etc) or may supplement other indicators (eg, FEV1 and daily functioning).

  2. Essential for evaluating treatment efficacy: in clinical trials, patients' report is the sole source of data on the frequency and severity of symptoms and side effects, and the impact of treatment on functioning and well-being.

  3. Useful for interpreting clinical outcomes: PRO data from clinical trials contribute to the comprehensive evaluation of the benefits of a new treatment.

  4. A key element in treatment decision making: a number of specialty groups and organizations recommend the use of PROs in clinical trials and have published guidelines for selecting outcome measures specific to the unique characteristics and evaluation needs of the underlying disease.

However, a PRO measure will only be considered adequate to characterize a treatment benefit if the assessment is well-defined and reliable.7  The FDA guidance on the use of PRO measures in medical product development to support labeling claims,1  which was issued in 2009, defines good measurement principles to consider for well-defined and reliable PRO measures intended to provide evidence of treatment benefit. In brief, the evaluation of a PRO in a clinical research setting should address the following considerations:

The PRO instrument should be clearly defined (conceptual framework, number of items, scoring rules, etc) and should be underpinned by rigorous measurement science supporting primarily its content validity but also the other measurement properties (reliability, validity, ability to detect change); if used in an international context, its translation or cultural adaptation should be the outcome of a robust methodology.

The objectives, design and analyses of the clinical trial(s) in which the PRO is an endpoint should be appropriately specified; the same clinical trial design principles that apply to other endpoint measures also apply to PROs. More specifically, the position of the PRO endpoint in the endpoint model (and hierarchy) should be carefully defined: the role the PRO endpoint in the clinical trial (ie, primary, key secondary, or supportive) is key in the evaluation of the instrument. Furthermore, the data analysis of PRO endpoints in a clinical trial should be cautious regarding the management of multiple, possibly composite, endpoints, and of missing data and should provide ways to support the meaningful interpretation of results (eg, responder analysis, cumulative distribution functions).

In addition to the FDA PRO guidance, a roadmap to Patient-Focused Outcome Measurement in Clinical Trials8  was issued by the FDA to illustrate how one might embark upon a sound, orderly, instrument selection or development pathway that is in alignment with the objectives of the drug development program and the clinical trial context of use (Figure 1). It shows that proper assessment of an outcome in a clinical trial should be underpinned by sound understanding of the disease (Column 1) and a precise and specific conceptualization of the expected treatment benefit (Column 2).

General issues

Hematology covers a wide spectrum of blood disorders which can be classified in 2 categories: non-neoplastic and neoplastic.9  The wide variety of disorders involves various treatment strategies with very different therapeutic implications (in terms of duration of treatment or side effects); each of which with a different impact on patients' lives. The evaluation of the patient's perspective in clinical research in hematology, whatever the disorder is, is certainly important and should follow the same rules as those described in the FDA PRO guidance1  and the Roadmap.8  Hence the approach needs to be tailored for each specific situation, and there is no “one-size-fits-all' solution given the diversity of the hematologic field. Nonetheless applying rigorously the approaches described above will undoubtedly allow the proper demonstration of treatment benefits based on PRO endpoints in hematology.

Novik et al indicate that PROs are often used in hematology to complement primary outcomes such as survival, clinician assessments and physiologic or laboratory-based measures.10  In many situations, the patient's perspective brings different information from the clinician or biological perspectives, and according to the very definition of treatment benefit, it should not be overlooked. For example, PROs were shown to be complementary to survival, clinical response, and adverse effects in characterizing treatment benefit in multiple myeloma.11,12  In situations in which multiple treatment options exist with similar survival outcome or if a new therapeutic strategy needs to be evaluated, the inclusion of PROs or health-related quality-of-life as an endpoint can provide additional data and help in clinical decision making. In palliative care, PRO assessment may be the sole indicator of treatment assignment or change.

The importance of PRO evaluation is stated in a number of international recommendations for various hematologic diseases, namely, for hemophilia,13  immune thrombocytopenia,14,15  myelodysplastic syndromes,16  chronic lymphocytic leukemia,17  acute leukemia,18  non-Hodgkin's lymphoma,19  Hodgkin's lymphoma,20,21  and multiple myeloma.22  It is beyond the scope of this paper to describe comprehensively the PRO evaluation in all hematologic disorders; however, in 2012, the European Hematology Association (EHA) Scientific Working Group on Quality and Life and Symptoms has been working with clinicians and researchers from 17 countries and 9 patients' organizations on Guidelines on PROs in hematology.10  The guidelines provide recommendations on PRO assessments in clinical studies for patients with hematologic diseases such as leukemias, lymphomas, multiple myeloma, myelodysplastic syndromes, hemophilia, Von Willebrand disease, immune thrombocytopenia, and anemia of chronic disease. They also present the evaluation of quality-of-life and symptoms in patients undergoing bone marrow transplantation, in patients receiving anticoagulant therapy, in long-term blood cancer survivors, and in children and adolescents with hematologic malignancies. These EHA guidelines are an excellent starting point in the endeavor of assessing a PRO endpoint in a clinical trial. In contrast, on the regulatory side, very little guidance has been published by the FDA and the EMA in benign hematology. There is a greater focus on PROs in malignancies.23-25 

A success story: the approval of Jakafi

On November 2011, Jakafi (ruxolitinib) was approved by FDA for the treatment of intermediate or high-risk myelofibrosis (MF) including primary MF, post-polycythemia vera MF (PPV-MF), and postessential thrombocythemia MF (PET-MF). Prior to this date, there were no approved products for this indication, and therefore there were no regulatory precedent on the endpoints required in this orphan disease. The regulatory endpoints that provided the basis for a full FDA approval of ruxolitinib were: (1) a decrease in splenomegaly, and (2) an improvement in MF-related symptoms as measured by a novel PRO instrument, the modified Myelofibrosis Symptom Assessment Form (MFSAF) version 2.0 diary.

Several meetings with the FDA were held in 2008 and in 2009 to discuss endpoint strategy. After several rounds of discussion and no agreement on 2 previous Special Protocol Assessments (SPAs), the FDA advised to use objectively measured spleen size reduction as a primary endpoint, and a clinically relevant benefit, such as symptom improvement measured daily, as a secondary endpoint, to support registration. Symptom improvement was measured using a revised version of the MFSAF published by Mesa et al.26  The changes made to the MFSAF followed the requirements of the FDA PRO guidance1 : qualitative patient interviews and cognitive testing were performed and demonstrated the content validity of the new version of the MFSAF, ie, the modified MFSAF version 2.0 diary. The instrument was composed of 7 questions: 6 questions asking about the severity of symptoms such as night sweats, itchiness, abdominal discomfort, pain under the ribs, early satiety and bone or muscle pain, and 1 question asking about inactivity. Scores ranged from 0 to 10 with 0 representing “absent” and 10 representing “worst imaginable”. The 6 symptomatic questions were scored in a composite referred to as the Total Symptom Score (TSS). However, the instrument was not demonstrated as “fit for purpose” yet, as required by the PRO guidance. To achieve this, evidence on measurement properties, as well as information to support meaningful interpretation of the results of the TSS, were generated using blinded data at 1 month of the 6 month randomized, placebo-controlled phase III trial and also the unblended data obtained at the end of the study.

The phase III trial was a double-blind, randomized, placebo-controlled study in 309 patients who were refractory to or were not candidates for available therapy. Patients were dosed with Jakafi or matching placebo. The primary efficacy endpoint was the proportion of patients achieving ≥35% reduction from baseline in spleen volume at Week 24 as measured by MRI or CT. Secondary endpoints included duration of a 35% or greater reduction in spleen volume and proportion of patients with a 50% or greater reduction in TTS from baseline to week 24 as measured by the MFSAF v2.0 diary. Electronic handheld diaries were used to collect the patient-reported symptom data. At baseline, the mean TSS was 18.0 in the ruxolitinib group and 16.5 in the placebo group. A higher proportion of patients in the ruxolitinib group (45.9%) had a 50% or greater reduction in TSS than in the placebo group (5.3%, P < 0.001), with a median time to response of <4 weeks.

It was decided to include the TSS in the final product label because the endpoint was alpha-controlled, the data collected was complete, the treatment effect size was clinically meaningful, and all of the MF symptoms were driving the TSS (not just 1 or 2 symptoms). Full approval was obtained with FDA, indicating that, without the demonstration of clinical benefit using the data produced by the modified MFSAF v2.0 diary, it would have been hard to justify anything but accelerated approval.

Challenges

Some of the main challenges in the evaluation of PROs in hematology include the following: (1) the design of oncology trials is not always optimized for PROs (ie, single arm or open-label design), (2) difficulty in collecting data from patients that have advanced or progressive disease due to failing health or cognitive challenges, and (3) high levels of missing data.23,27  Challenges also include assessment of PRO endpoints in rare diseases, such as hemophilia28  or Willebrand's disease, where the limited number of available patients make the conduct of a comprehensive PRO research agenda difficult. Similarly, assessment of PROs in pediatric context require particular care.

Another challenge is represented by the globalization of clinical research. In 2007, over 60% of pivotal studies submitted to the Center for Drug Evaluation and Research (CDER) contained data from 1 or more international study sites.29  From October 2007 to September 2008, clinical trials for medical products were conducted at nearly 6500 international sites. This situation has implications for PRO measures, which need to be appropriately translated and cross-culturally adapted to be used globally. A methodology (ie, linguistic validation) has been developed30-32  which is advised to be followed for all PRO measures to ensure conceptual equivalence between the original and the translations. Several PRO measures used in hematology have been translated using this process (eg, the HaemoQOL,33  or the QLQ-C30 and QLQ-MY2034 ).

The Prescription Drug User Fee Act (PDUFA) was first authorized in 1992 and was conceived to overcome the “drug lag” of the 1980s by providing the FDA with increased staffing and resources, and by encouraging medical innovation. PDUFA was reauthorized (as PDUFA V) as part of the Food and Drug Administration Safety and Innovation Act, enacted in July 2012, and went into effect on October 1, 2012. The FDA's Patient-Focused Drug Development initiative is a commitment under PDUFA V that aims to more systematically gather patients' perspectives on their condition and available therapies to treat their condition. This initiative clearly demonstrates that the central position of the patients in the drug development is now acknowledged by regulatory agencies, in particular the FDA. One of the goals of this initiative is to increase the use of PROs in clinical trials. Twenty public meetings over the course of PDUFA V were planned, each focused on a specific disease area. At the moment, 2 meetings were held in hematology, ie, in sickle cell disease, on February 2, 201435 , and in hemophilia A, hemophilia B, von Willebrand disease, and other heritable bleeding disorders, on September 22, 2014.36  These meetings gave FDA and the pharmaceutical industry a unique opportunity to hear directly from patients and caregivers about their experiences with their disease and its treatments. Discussion focused on 2 key topics: (1) the effects of the disease that matter most to patients, and (2) patients' perspectives on treatments and on clinical trial participation. The input generated through these meetings enhanced the FDA understanding of the burden of the disease on patients and their experiences with the treatment and their perspectives for the future. The outcome of these meetings will be used to enhance drug development programs in each disease area, by informing the discussion between the regulators and sponsors that submit products for marketing approval in the US. This input may also be of value to the drug development process more broadly. For example, the report may be useful to drug developers as they explore potential areas of unmet needs. It could also point to the potential need for development and qualification of new outcome measures in clinical trials.

In a recent presentation at the FDA Clinical Outcome Assessments (COA) workshop,37  organizational changes at the Office of Hematology and Oncology Products (OHOP) were introduced with the declared objective to advance PRO evaluation in hematology and oncology and facilitate the review of PROs in the dossiers submitted to OHOP. Over the last 2 years, interactions with the Study Endpoints and Labeling Development (SEALD) team were increased, with the organization of OHOP-SEALD working groups with monthly meetings. OHOP PRO and labeling expertise has been improved with an Office level dedication, PRO leads, and associate directors of labeling in each OHOP clinical division providing PRO and labeling expertise and interacting with SEALD to improve consistency between OHOP Divisions, and finally overall OHOP educational opportunities for reviewers with monthly hematology oncology case series. The goal is to provide more detailed, consistent, and proactive PRO advice to sponsors.

Patient-reported outcomes provide unique and important information about the effect of a treatment from a patient's view in hematology, as in other fields. In hematologic clinical trials, they may be used to evaluate overall treatment effectiveness, treatment toxicity, and quality of patient's well-being at short-term and long-term after treatment from a patient's perspective. The inclusion of PRO endpoints in clinical trials in hematology will certainly continue to grow in the future allowing the voice of the patients with hematologic diseases to be taken into greater consideration in the development of new drugs, which it is always important to keep in mind, aims to address patients' needs.

Catherine Acquadro, Mapi Research Trust, 27 rue de la Villette, 69003 Lyon, France; Phone: 04-72-13-65-75; Fax: 04-72-13-66-82; e-mail: cacquadro@mapigroup.com.

1
US Department of Health, Food and Drug Administration (2009)
Patient-reported outcome measures: use in medical product development to support labeling claims
Federal Register
2009
Accessed April 23, 2015, vol. 
74
 
35
(pg. 
65132
-
65133
2
European Medicines Agency
Reflection paper on the regulatory guidance for the use of health-related quality of life (HRQL) measures in the evaluation of medicinal products
Accessed April 23, 2015 
3
Papadopoulos
 
EJ
Establishing evidence of treatment benefit: focus on outcome assessment
Accessed April 23, 2015 
4
Papadopoulos
 
EJ
An approach to outcome measure development: a regulatory perspective
Accessed April 24, 2015 
5
Temple
 
R
Are surrogate markers adequate to assess cardiovascular disease drugs?
JAMA
1999
, vol. 
282
 
8
(pg. 
790
-
795
)
6
Acquadro
 
C
Berzon
 
R
Dubois
 
D
et al. 
Incorporating the patient's perspective into drug development and communication: an ad hoc task force report of the Patient-Reported Outcomes (PRO) Harmonization Group meeting at the Food and Drug Administration, February 16, 2001
Value Health
2000
, vol. 
5
 (pg. 
522
-
531
)
7
CFR: Code of Federal Regulations Title 21. 21CFR 314.126
Accessed April 25, 2015 
8
US Department of Health and Human Services, Food and Drug Administration
Roadmap to Patient-Focused Outcome Measurement in Clinical Trials
Accessed April 28, 2015 
9
Swerdlow
 
SH
Campo
 
E
Harris
 
NL
et al. 
WHO Classification of Tumours of Haematopoietic and Lymphoid Tissues
2008
Ed 4th
Lyon, France
IARC Press
10
Novik
 
A
Salek
 
S
Ionova
 
T
Guidelines Patient-Reported outcomes in Hematology
2012
Accessed April 20, 2015
The Netherlands
The Hague
 
11
Viala
 
M
Bhakar
 
AL
de la Loge
 
C
et al. 
Patient-reported outcomes helped predict survival in multiple myeloma using partial least squares analysis
J Clin Epidemiol
2007
, vol. 
60
 
7
(pg. 
670
-
679
)
12
Delforge
 
M
Dhawan
 
R
Robinson
 
D
et al. 
Health-related quality of life in elderly, newly diagnosed multiple myeloma patients treated with VMP vs. MP: results from the VISTA trial
Eur J Haematol
2012
, vol. 
89
 
1
(pg. 
16
-
27
)
13
Guidelines for the management of hemophilia
2012
Ed 2d
Montreal, Quebec
World Federation of Hemophilia
14
Rodeghiero
 
F
Stasi
 
R
Gernsheimer
 
T
et al. 
Standardization of terminology, definitions and outcome criteria in immune thrombocytopenic purpura (ITP) of adults and children: report from an international working group
Blood
2009
, vol. 
113
 
11
(pg. 
2386
-
2393
)
15
Kirsch
 
M
Klaassen
 
RJ
De Geest
 
S
Matzdorff
 
A
Ionova
 
T
Dobbels
 
F
Understanding the importance of using patient-reported outcome measures in patients with immune thrombocytopenia
Semin Hematol
2013
, vol. 
50
 
Suppl 1
(pg. 
39
-
42
)
16
Tefferi
 
A
Barosi
 
G
Mesa
 
RA
et al. 
International working group (IWG) consensus criteria for treatment response in myelofibrosis with myeloid metaplasia, for the IWG for myelofibrosis research and treatment (IWG-MRT)
Blood
2006
, vol. 
108
 
5
(pg. 
1497
-
1503
)
17
Hallek
 
M
Cheson
 
BD
Catovsky
 
D
et al. 
Guidelines for the diagnosis and treatment of chronic lymphocytic leukemia: a report from the International Workshop on Chronic Lymphocytic Leukemia updating the National Cancer Institute-Working Group 1996 guidelines
Blood
2008
, vol. 
111
 
12
(pg. 
5446
-
5456
)
18
Appelbaum
 
FR
Goldman
 
L
Ausiello
 
D
The acute leukemias
Cecil Medicine
2007
Philadelphia, PA
Saunders
 
Ch 194. Ed 23
19
Mauch
 
PM
Armitage
 
JON
Harris
 
R
et al. 
Non-Hodgkin's Lymphomas
2004
Philadelphia, PA
Lippincott Williams & Wilkins
20
Hoppe
 
R
Mauch
 
PM
Armitage
 
JON
Hodgkin Lymphoma
2007
Philadelphia, PA
Lippincott Williams & Wilkins
21
Engert
 
A
Horning
 
SJ
Hodgkin lymphoma: a comprehensive update on diagnostics and clinics
2011
Berlin
Springer
22
Morgan
 
GP
Krishnan
 
B
Jenner
 
M
Davies
 
FE
Advances in oral therapy for multiple myeloma
Lancet Oncol
2006
, vol. 
7
 
4
(pg. 
316
-
325
)
23
European Medicines Agency
Oncology Working Party. Guideline on the evaluation of anticancer medicinal products in man
Accessed April 25, 2015 
24
European Medicines Agency
Oncology Working Party. Reflection Paper on the use of patient reported outcome (PRO) measures in oncology studies
Accessed April 25, 2015 
25
US Department of Health and Human Services, Food and Drug Administration
Clinical trials endpoints for the approval of cancer drugs and biologics
Accessed April 25, 2015 
26
Mesa
 
RA
Schwager
 
S
Radia
 
D
et al. 
The Myelofibrosis Symptom Assessment Form (MFSAF): an evidence-based brief inventory to measure quality of life and symptomatic response to treatment in myelofibrosis
Leuk Res
2009
, vol. 
33
 
9
(pg. 
1199
-
1203
)
27
Basch
 
E
Geoghegan
 
C
Coons
 
SJ
et al. 
Patient-reported outcomes in cancer drug development and US regulatory review: perspectives from industry, the Food and Drug Administration, and the patient
JAMA Oncol
2015
, vol. 
1
 
3
(pg. 
375
-
379
)
28
Santagostino
 
E
Lentz
 
SR
Busk
 
AK
Regnault
 
A
Iorio
 
A
Assessment of the impact of treatment on quality of life of patients with haemophilia A at different ages: insights from two clinical trials on turoctocog alfa
Haemophilia
2014
, vol. 
20
 
4
(pg. 
527
-
534
)
29
Ayalew
 
K
FDA perspective on international clinical trials
Accessed April 27, 2015 
30
Acquadro
 
C
Conway
 
K
Giroudet
 
C
Mear
 
I
Linguistic Validation Manual for Health Outcome Assessments
2012
Ed 2
Lyon, France
MAPI Institute
31
Wild
 
D
Grove
 
A
Martin
 
ML
et al. 
ISPOR principles of good practice: the cross-cultural adaptation process for patient reported
Value Health
2005
, vol. 
8
 
2
(pg. 
94
-
110
)
32
Wild
 
D
Eremenco
 
S
Mear
 
I
et al. 
Multinational trials: recommendations on the translations required, approaches to using the same language in different countries, and the approaches to support pooling the data: the ISPOR Patient Reported Outcomes Translation & Linguistic Validation Good Research Practices Task Force Report
Value Health
2009
, vol. 
12
 (pg. 
430
-
440
)
33
von Mackensen
 
S
Campos
 
IG
Acquadro
 
C
Strandberg-Larsen
 
M
Cross-cultural adaptation and linguistic validation of age-group-specific haemophilia patient-reported outcome (PRO) instruments for patients and parents
Haemophilia
2013
, vol. 
19
 
2
(pg. 
73
-
83
)
34
Cocks
 
K
Cohen
 
D
Wisløff
 
F
et al. 
An international field study of the reliability and validity of a disease-specific questionnaire module (the QLQ-MY20) in assessing the quality of life of patients with multiple myeloma
Eur J Cancer
2007
, vol. 
43
 
11
(pg. 
1670
-
1678
)
35
US Department of Health and Human Services, Food and Drug Administration
The Voice of the Patient: Sickle Cell Report
Accessed April 26, 2015 
36
US Department of Health and Human Services, Food and Drug Administration
Patient-Focused Drug Development for Hemophilia A, Hemophilia B, von Willebrand Disease and other Heritable Bleeding Disorders
Accessed April 30, 2015 
37
US Department of Health and Human Services, Food and Drug Administration
PDUFA V Clinical Outcome Assessments Public Workshop
2015
4
1
Accessed April 26, 2015 

Competing Interests

Conflict-of-interest disclosures: C.A. declares no competing financial interests; A.R. is employed by Mapi.

Author notes

Off-label drug use: None disclosed.