Published on in Vol 22, No 9 (2020): September

Preprints (earlier versions) of this paper are available at, first published .
Exploring the Health-Related Quality of Life of Patients Treated With Immune Checkpoint Inhibitors: Social Media Study

Exploring the Health-Related Quality of Life of Patients Treated With Immune Checkpoint Inhibitors: Social Media Study

Exploring the Health-Related Quality of Life of Patients Treated With Immune Checkpoint Inhibitors: Social Media Study

Original Paper

1Bristol-Myers Squibb, Rueil-Malmaison, France

2Kap Code, Paris, France

3Bristol-Myers Squibb, Stockport, United Kingdom

4Paris-Sud University, Paris, France

5Paris-Descartes Universitiy, Paris, France

6AP-HP, Paris, France

7INSERM U1178, Paris, France

8Bordeaux University, Bordeaux, France

9Inserm U1219, Bordeaux, France

10UFR Pharmacy, Grenoble Alpes University, Grenoble, France

11Bristol-Myers Squibb, London, United Kingdom

Corresponding Author:

Paméla Voillot, MSc

Kap Code

28 rue d'Enghien

Paris, 75010


Phone: 33 624058742


Background: Immune checkpoint inhibitors (ICIs) are increasingly used to treat several types of tumors. Impact of this emerging therapy on patients’ health-related quality of life (HRQoL) is usually collected in clinical trials through standard questionnaires. However, this might not fully reflect HRQoL of patients under real-world conditions. In parallel, users’ narratives from social media represent a potential new source of research concerning HRQoL.

Objective: The aim of this study is to assess and compare coverage of ICI-treated patients’ HRQoL domains and subdomains in standard questionnaires from clinical trials and in real-world setting from social media posts.

Methods: A retrospective study was carried out by collecting social media posts in French language written by internet users mentioning their experiences with ICIs between January 2011 and August 2018. Automatic and manual extractions were implemented to create a corpus where domains and subdomains of HRQoL were classified. These annotations were compared with domains covered by 2 standard HRQoL questionnaires, the EORTC QLQ-C30 and the FACT-G.

Results: We identified 150 users who described their own experience with ICI (89/150, 59.3%) or that of their relative (61/150, 40.7%), with 137 users (91.3%) reporting at least one HRQoL domain in their social media posts. A total of 8 domains and 42 subdomains of HRQoL were identified: Global health (1 subdomain; 115 patients), Symptoms (13; 76), Emotional state (10; 49), Role (7; 22), Physical activity (4; 13), Professional situation (3; 9), Cognitive state (2; 2), and Social state (2; 2). The QLQ-C30 showed a wider global coverage of social media HRQoL subdomains than the FACT-G, 45% (19/42) and 29% (12/42), respectively. For both QLQ-C30 and FACT-G questionnaires, coverage rates were particularly suboptimal for Symptoms (68/123, 55.3% and 72/123, 58.5%, respectively), Emotional state (7/49, 14% and 24/49, 49%, respectively), and Role (17/22, 77% and 15/22, 68%, respectively).

Conclusions: Many patients with cancer are using social media to share their experiences with immunotherapy. Collecting and analyzing their spontaneous narratives are helpful to capture and understand their HRQoL in real-world setting. New measures of HRQoL are needed to provide more in-depth evaluation of Symptoms, Emotional state, and Role among patients with cancer treated with immunotherapy.

J Med Internet Res 2020;22(9):e19694



Health-related quality of life (HRQoL) is a complex subjective concept pertaining to multiple domains including physical, emotional, social, professional, and functional well-being [1,2]. The number of cancer cases is continuing to rise across Europe with approximately 8% of people currently living with cancer [3] and with the side effects of cancer treatment (eg, hair loss, pain, fatigue, nausea) [4]. Patients and their families also often experience psychological distress as a result of a cancer diagnosis and its treatment, including stress, anxiety, and depression [5]. Therefore, HRQoL is particularly important to understand and to be quantified among people living with cancer or caring for someone with cancer. Several cancer-related HRQoL self-administered questionnaires, such as the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30 (EORTC QLQ-C30) [6] and the Functional Assessment of Cancer Therapy - General (FACT-G) [7], have been used and validated in oncology patient populations, but are limited in that they are required to be complete at a predefined time.

Recently, the internet has provided an opportunity for introducing new sources of clinically relevant information related to patients’ perception of illness and its burden [8]. Specifically, patients and their relatives are increasingly using forums, blogs, and social media to obtain health-related information and support [9]. Indeed, the information generated online represents an alternative way to understand patients’ and relatives’ health state compared with self-administered questionnaires. These patient-generated health data are produced spontaneously—thus not limited to medical consultations, for instance—mostly anonymously, and may better correspond to patients’ and relatives’ feelings than close-ended questions. Moreover, text mining techniques applied to analyze social media data can be used with relative ease [10] and have opened up new opportunities to bridge the gap between qualitative and quantitative data [11].

In parallel to the development of new strategies for collecting information on HRQoL of patients with cancer, treatment with immunotherapy has emerged as an innovative curative approach that, instead of destroying cancer cells directly, stimulates the immune system making it capable of identifying and selectively attacking cancer cells. The body develops an internal defense mechanism which can lead to reduced side effects and improved HRQoL [12]. Recent studies have assessed HRQoL of patients undergoing immunotherapy using standard questionnaires, but exclusively only in clinical trial settings [13-17]. In addition to prolonged survival, HRQoL results showed that immune checkpoint inhibitors (ICIs), such as nivolumab, maintained or improved baseline HRQoL levels in patients with advanced melanoma [18], advanced renal cell carcinoma [19], or advanced squamous non-small-cell lung cancer [20]. However, HRQoL of patients treated with ICI remains largely unknown under real-world conditions and it is not known whether the EORTC QLQ-C30 and the FACT-G questionnaires capture the full range of HRQoL domains and experiences relevant to these specific patients. The use of social media data offers a novel opportunity to explore this.

This study assessed and compared conceptual coverage of ICI-treated patients’ HRQoL between standard questionnaires and users’ ICI-related experiences described in social media. We hypothesized that, given the evolving and dynamic nature of the HRQoL concept, new HRQoL subdomains would emerge from social media posts going beyond the coverage of existing questionnaires, especially in a population of patients treated with new drugs such as ICI.

Study Design and Population

This was a retrospective study using a text mining approach to retrieve information from social media posts written by French language internet users between January 1, 2011, and August 31, 2018. The start date of January 2011 was selected because it corresponds to the date of marketing authorization for selling the first available immuno-oncology treatment in France. Included posts (comprising forum posts or comments on videos or photos) had to mention past or current patients’ experience with ICI (ie, ipilimumab, nivolumab, pembrolizumab, atezolizumab, or those available through early access schemes or clinical trials, such as durvalumab, tremelimumab, and avelumab). Posts referring to treatments other than ICI were excluded. Posts could be authored by patients themselves or by their relatives, here interchangeably referred to as patients. Posts were retrieved from the following social media: 12 generic French medical web forums; 4 cancer-specialized French medical web forums; and 3 generic social media (Facebook, YouTube, and Instagram). These social media were screened from the Detec’t database [21]. Description of forums is provided in Multimedia Appendix 1. All posts had to be publicly available and include at least one of the predetermined keywords with their synonyms (see the list in Multimedia Appendix 2). Ambiguous posts or duplicates (ie, similar posts posted by the same user in different social media) were excluded through a manual review.

Data Extraction

Automatic and manual extraction methods were utilized, depending on the source of the posts. For generic French medical web forums, posts were automatically extracted using the Detec’t WebCrawler [22]. Collection of the posts was performed according to the HTML structure of each forum. All posts containing one of the predefined keywords were automatically retrieved from discussion threads and deidentified (signature and quote withdrawal). The deidentification of posts was performed by using an in-house algorithm based on regular expression to automatically identify specific sequences of characters (proper names, phone numbers, postal codes, mail addresses, etc). For specialized French medical web forums, posts containing predefined keywords were identified using a Google operator searching these keywords in selected forum websites (Site: +URL). Identified discussion threads were manually explored, and posts containing one of the predefined keywords were manually retrieved and deidentified. Finally, for the 3 generic social media, posts were identified and retrieved by manually searching the predefined keywords using the social media embedded search fields. These posts were also manually deidentified. Data from these 3 collection methods were then grouped in a unique data set (the analysis corpus), which went through several steps of cleaning (preprocessing by removing French accents, unnecessary spaces, and punctuation and lowercasing all words; removal of stop words; and stemming, based on Porter’s algorithm [23]) and formatting (transformation of the corpus into a matrix; creation of tokens and measure of their frequency; exclusion of hardly used, ambiguous, and misspelled words; and document-term matrix weighting; Figure 1).

Figure 1. Diagram of the steps for creating the analysis corpus.
View this figure

Study Variables

The analysis corpus contained information on data source (name of the forum or social media), post characteristics (URL of the page or discussion where the post was published; date of the publication; pseudonym or alias of the user; keyword associated with the post leading to its extraction), patient characteristics (age, gender, type of cancer), and user status among “patient,” “relative,” and “unspecified.” Regarding content of retrieved posts, for each patient “associated HRQoL domains” and “associated subdomains” were collected manually. “Associated HRQoL domains” were grouped considering the classification of domains provided in existing measures (ie, the EORTC QLQ-C30 and the FACT-G). For each domain, “associated subdomains” mentioned by patients were also collected even if not included in standard measures in order to allow new subdomains to emerge.

Comparison With HRQoL Standard Questionnaires

The EORTC QLQ-C30 [6] and the FACT-G [7] are among the most widely used questionnaires to capture HRQoL of patients with cancer in research and clinical settings [24,25]. Compared with other questionnaires, they are not limited to a specific cancer type [26,27] and cover the highest number of HRQoL domains across all cancer-specific questionnaires [28]. Both questionnaires were first developed in 1993 [29] and are validated in the French language [30,31].

The EORTC QLQ-C30 is a 30-item questionnaire composed of multi-item scales and single items, and comprises 5 functional scales (physical activity, role, emotional state, cognitive state, and social state), 3 symptoms scales (fatigue, nausea and vomiting, and pain), and a global health status and HRQoL scale. The remaining single items assess additional symptoms commonly reported by patients with cancer: dyspnea, insomnia, lack of appetite, constipation, diarrhea, and financial difficulties. The FACT-G is a 27-item questionnaire divided into 4 well-being subscales: physical well-being, social/family well-being, emotional well-being, and functional well-being [32]. Coverage of domains identified in social media and related subdomains measured by the EORTC QLQ-C30 and the FACT-G was manually assessed independently by 2 operators and compared through a concept mapping approach [33].

Data Analysis

Each post corresponded to a statistical unit. Frequentist analysis was performed on extracted posts to characterize the whole analysis corpus through the following indicators: number of posts, number of patients, occurrence of HRQoL domain(s), keywords (including the number of extracted posts), data source, and users’ characteristics. A Venn diagram was generated through the CRAN package “VennDiagram.” The list of identified subdomains, including the number of patients and posts, was presented in a descriptive format using Microsoft Excel. For each subdomain, coverage by one or several items of the 2 questionnaires was assessed. For each domain, coverage rates were calculated by dividing the sum of subdomain occurrences covered by questionnaires by the total number of occurrences in the social media. The diagram indicating the coverage rates of each HRQoL domain retrieved from social media posts through the standard questionnaires EORTC QLQ-C30 and FACT-G was generated through Microsoft Excel.

Description of the Population and Posts

The final analysis corpus included 267 social media posts meeting the inclusion criteria, with a maximum of 11 posts from 1 patient and a median of 2 posts per patient. Through the manual extraction, we identified 150 patients (posters) who described their personal experience with ICI (89/150, 59.3%) or that of their relative (61/150, 40.7%). A majority of patients were women (82/150, 54.7%) and gender was undetermined for only 8/150 patients (5.3%). The type of related cancer was identified for 123/150 patients (82.0%): the most frequent cancers were lung cancer and melanoma (Table 1). The majority of posts were retrieved from 1 cancer-specific patient forum (La ligue contre le cancer, 78 posts by 43 patients) and 1 generic medical forum (Doctissimo, 76 posts by 43 patients). The most frequently identified keyword was “immunotherapy” (72/150, 48.0%) followed by “nivolumab” (31/150, 20.7%) and “ipilimumab” (20/150, 13.3%).

Table 1. Patients characteristics.
CharacteristicsPatients mentioning their quality of life (N=150)
Gender, n (%)

Women82 (54.7)

Men60 (40.0)

Undetermined8 (5.3)
Type of cancer, n (%)

Lung cancer46 (30.7)

Melanoma41 (27.3)

Others36 (24.0)

Undetermined27 (18.0)
Type of immunotherapy, n (%)

Nivolumab31 (20.7)

Ipilimumab20 (13.3)

Pembrolizumab16 (10.7)

Others11 (7.3)

Undetermined72 (48.0)
Posts number, n267

Per patient, median (min-max)2 (1-11)
Social media, n (%)

Doctissimo43 (28.7)

La ligue contre le cancer43 (28.7)

Facebook16 (10.7)

Others48 (32.0)

Domains and Subdomains of HRQoL

Of the study population, 91.3% (137/150) mentioned at least one HRQoL domain in their posts. We identified 8 HRQoL domains, with 83.9% (115/137) patients mentioning Global health, 55.5% (76/137) mentioning Symptoms, 35.8% (49/137) mentioning Emotional state, 16.1% (22/137) mentioning Role, 9.5% (13/137) mentioning Physical activity, 6.6% (9/137) mentioning Professional situation, 1.5% (2/137) mentioning Social state, and 1.5% (2/137) mentioning Cognitive state. A Venn diagram illustrates the distribution of the HRQoL domains and the relationship among them (Figure 2). Verbatim examples for each HRQoL domains are presented in Table 2.

A total of 42 subdomains of HRQoL were mentioned in the social media posts and were distributed as follows: 1 for Global health, 13 for Symptoms, 4 for Physical activity, 10 for Emotional state, 7 for Role, 3 for Professional situation, 2 for Cognitive state, and 2 for Social state. Table 2 provides the list and occurrences of each social media subdomain. Among 115 patients mentioning their Global health, a majority (63/115, 54.8%) considered it to be poor, while 27.8% (32/115) and 17.4% (20/115) perceived it to be stable and good, respectively. As for Symptoms, 76 patients mentioned symptoms 123 times, which were grouped into 13 subdomains. The most addressed subdomain of Symptoms was Fatigue/Tiredness (34/123, 27.6%). A total of 49 patients mentioned at least one subdomain concerning Emotional state. With 15 occurrences, Hope was the most recurrent subdomain (15/49, 31%). The most frequent subdomains of Role were normal life/reduced activities and pace of life (12/22, 55%). Only 13 patients mentioned Physical activity and the most reported subdomains pertaining to this larger domain were minimal or no physical activity/maintained activity (5/13, 38%) and difficulty walking/eating (5/13, 38%). Professional situation was mentioned by 9 patients with the most frequent subdomain being sick leave (6/9, 67%). Two subdomains were mentioned for the Cognitive state and the Social state, respectively.

Table 2 also synthetizes coverage of each social media subdomain by items of the QLQ-C30 and the FACT-G questionnaires. The QLQ-C30 showed a wider coverage of social media subdomains than the FACT-G (45% [19/42] versus 29% [12/42], respectively).

Figure 2. Venn diagram showing the distribution of patients according to their posts mentioning HRQoL domains. HRQoL: health-related quality of life.
View this figure
Table 2. Verbatim examples of posts related to HRQoLa of patients with cancer.
HRQoL domain and specific subdomainVerbatim example (translation from French)
Global health

Deteriorated“Hello, we started immunotherapy 3 weeks ago, we didn’t combine it with [Drug], his health is deteriorating more and more. Yesterday he was re-hospitalized...I don’t think his condition could get any worse...I strongly hope he gets better, that he can eat and move...”

Weight loss“When I was diagnosed with melanoma in May 2010, the [hospital] immediately referred me to the [protocol] ([Drug] treatment). (...) it must be said that I lost 9 kilos after the 1st injection, then 2 kilos per month, until we stopped this treatment, in [month] 2011, for a total weight loss of 28 kilos. Since the beginning of the injections I had to stop working.”
Emotional state

Optimism“I have lung cancer and metastasis (...). I have had immuno with [drug] for 69 weeks, I am quite well and the mass sometimes decreases slightly and sometimes I have almost no nodule on the lungs. I have almost no reactions as I do not lose my hair, no nausea, just a little bit. I am at stage 4, but I trust my treatment and I have a lot of hope.”
Exhaustion“I have been fighting lung cancer with bone metastases for more than three years. Since the beginning of July, I have been trying immunotherapy. If people have had this treatment, I would like to know what the side effects were. As for me, I am very tired, have no appetite and my moral is deteriorating.”

Physical activity

Minimal or no physical activity/maintained activity“Cancer was diagnosed in [month] 2017. An operation was not possible. In [month] after the immunotherapy the saturation level was very low and the oxygen allowed maintaining a very minimal physical activity... a few steps...”

Normal life“I have been receiving this treatment for 1 year and a half, I don\'t know if it will cure me but thanks to [Drug] I have been in remission for more than a year. Relatively non-toxic, it allows to lead a normal life in parallel.”

Reduced activities and pace of life“Hello. I have melanoma stage 4; for 3 years, interferon, [Drug], [Drug] currently every 15 days. I am tired, so exhausted that I can no longer take care of my house, but I have the rage to live.”
Professional situation

Maintained work activity“In short, of course I\'m afraid of dying, but that\'s out of the question. This type of cancer is treatable if taken in time. And I was in stage 4 Clark. Immunotherapy is not always cool, but I still work, I manage the house, the children and I am still smiling.”
Cognitive state

Concentration disorders“Here I am, dean of this post: a year under [drug]. And no trace of melanoma! (...) If I had to relive it I would stop working earlier because I have the impression that [drug] has mostly prevented me from recovering, from sleeping well, with a cumulative effect on the end where I ended the year on the kneecaps wondering if I was not depressed so much I could not concentrate on anything.”
Social state

No longer participates in family parties“On the first scan, increased lung metastases...but no new ones, so that\'s something. (...) We won’t do much for the holidays. Until recently we used to go to my in-laws, there were a lot of us, but since my mother-in-law died and since I\'m no longer in Olympic shape, we haven’t been moving around!”

aHRQoL: health-related quality of life.

Coverage of HRQoL Domains in Social Media by the QLQ-C30 and FACT-G Questionnaires

As shown in Table 3, Global health was entirely covered by both the QLQ-C30 and the FACT-G. Physical activity, Professional situation, Cognitive state, and Social state were also fully covered by the QLQ-C30. For Symptoms, the EORTC QLQ-C30 covered a majority (68/123, 55.3%) of the subdomains identified in social media posts, and so did the FACT-G (72/123, 58.5%). Coverage was lower for the Emotional state domain: 14% (7/49) by the EORTC QLQ-C30 and 49% (24/49) by the FACT-G. Finally, the EORTC QLQ-C30 covered 77% (17/22) of subdomains for Role, whereas the FACT-G covered 68% (15/22). Of these domains, the FACT-G fully covered only the Professional situation, whereas it covered 46% (6/13) of the Physical activity, with no coverage of neither Cognitive state nor Social state.

Table 3. HRQoLa subdomains in social media posts and their coverage by EORTC QLQ-C30b and FACT-Gc questionnaires.
HRQoL domains and subdomains in social media (number of occurrences in posts)EORTC QLQ-C30 coverage (question number)FACT-G coverage (question number)
Global health

Deteriorated/stable/better (115)✓ (Q29)✓ (GF7)

Fatigue/Tiredness (34)✓ (Q10, Q12, Q18)✓ (GP1)
None/Unspecified adverse events (23)X✓ (GP5)
Fever (8)XX
Pain/Joint pain/Leg pain/Liver pain (15)✓ (Q9, Q19)✓ (GP4)
Cough (7)XX
Loss of appetite/Weight loss (11)✓ (Q13)X
Rash/itch (6)XX
Respiratory trouble/Breathlessness (6)✓ (Q18)X
Headache (3)XX
Thyroid disorders (4)XX
Heavy legs (2)XX
Hair loss (2)XX
Diarrhea (2)✓ (Q17)X
Emotional state

Optimism/Hope (15)X✓ (GE3)
Exhaustion (7)XX
Distress (6)XX
Good mental health/Morale (5)X✓ (GE1, GE2)
Fear (4)✓ (Q22)✓ (GE5, GE6)
Depression (3)✓ (Q24)X
Psychological disorders (3)XX
Stable health (3)XX
Isolate oneself (2)XX
Emotional exhaustion due to side effects (1)XX

Normal life/Reduced activities and pace of life (12)✓ (Q6, Q7)✓ (GF6, GF3)
Time constraints regarding medical care (3)XX
Financial problems (2)✓ (Q28)X
Maintained activities (2)✓ (Q6, Q7)✓ (GF6)
Home care constraints (1)XX
Improved HRQoL (1)✓ (Q30)✓ (GF7)
Ability to drive again (1)XX
Physical activity

Minimal or no physical activity/maintained activity (6)✓ (Q1, Q2, Q3, Q4)✓ (GP1, GP7)
Difficulty walking/eating (5)✓ (Q2, Q3, Q5)X
Difficulty climbing stairs (1)✓ (Q2, Q3, Q5)X
Difficulty getting into bed alone (1)✓ (Q2, Q3, Q5)X
Professional situation

Sick leave (6)✓ (Q6)✓ (GF1, GF2)
Maintained work activity (2)✓ (Q6)✓ (GF1, GF2)
Reduction in working time (1)✓ (Q6)✓ (GF1, GF2)
Cognitive state

Concentration problems (1)✓ (Q20)X
Memory problems (1)✓ (Q25)X
Social state

Requiring family assistance for medical care (1)✓ (Q26)X
Inability to participate in family celebrations (1)✓ (Q26)X

aHRQoL: health-related quality of life.

bEORTC QLQ-C39: European Organization for Research and Treatment of Cancer QLQ-C30.

cFACT-G: Functional Assessment of Cancer Therapy - General.

Specific subdomains which were not covered by both the EORTC QLQ-C30 and the FACT-G were fever, cough, rash/itch, headache, thyroid disorders, heavy legs, and hair loss for the Symptoms domain; exhaustion, distress, psychological disorders, stable health, emotional exhaustion due to side effects, and isolation for the Emotional state domain; and time constraints regarding medical care and ability to drive again for the Role domain (Figure 3).

Figure 3. Diagram indicating the coverage rates of each HRQoL domains retrieved in social media by the standard questionnaires EORTC QLQ-C30 and FACT-G. FACT-G: Functional Assessment of Cancer Therapy - General; QLQ-C30: Quality of Life Questionnaire Core 30.
View this figure

Principal Findings

This novel approach of using social media identified that some of the content posted by patients with cancer and caregivers with experience of ICI overlaps with concepts captured in the 2 most frequently used HRQoL questionnaires. However, the main findings also included the fact that there are a large number of concepts which are not captured in these 2 HRQoL questionnaires. These results confirmed our hypothesis by underlining the emergence of new subdomains of HRQoL in patients treated with immunotherapy. In particular, we observed that retrieved social media posts frequently addressed specific subdomains of the HRQoL domains of Symptoms, Emotional state, and Role, which are not fully covered by the EORTC QLQ-C30 and the FACT-G. Reasons for not including these subdomains in standard questionnaires are varied. First, questionnaires such as the EORTC QLQ-C30 and the FACT-G are designed to be short, effective, and time-saving, thus reducing burden on patients completing these questionnaires. As a consequence, they are limited and do not cover the whole range of the issues impacting patients’ HRQoL. Second, these questionnaires were developed more than 25 years ago, which may explain why the domains might not fully capture the impact of recently developed therapies, such as ICI. The concept of HRQoL is evolving and several users have already raised the problem of “partial covering” [33], which relates to the complexity of measuring this broad ranging concept through a robust methodology [34]. Third, subdomains retrieved by our study might be specific to immunotherapy and, therefore, not measured by generic standard questionnaires. Finally, accurately quantifying an individual’s HRQoL is, per se, a debated question, because standardized questionnaires might restrict a patient’s choice and limit their spontaneity, thus not using a patient-centered approach [35].

We observed a remarkable usage in the occurrences of the keyword “immunotherapy” which likely results from the growing availability of ICI, the steadily increased use of ICI in the French health care system, and, in parallel, the growing proportion of social media users.

Comparison With the Literature on HRQoL and Immunotherapy

The conceptual and psychometric measurement properties of the EORTC QLQ-C30 and the FACT-G have not yet been systematically examined in ICI-treated patient populations and the results of this study cast some doubt on the content validity of these measures in ICI-treated patients. Indeed, content validity is the extent to which an instrument measures the important aspects of concepts most significant and relevant to a patient’s condition and its treatment [36]. Because there are subdomains which do not appear in the 2 questionnaires that participants completed, we can assume that the 2 questionnaires lack content validity for this specific patient population. This does underline the need for new or adapted patient-reported outcomes in patients treated with immunotherapy.

Existing studies using these questionnaires in patients with cancer treated with immunotherapy are still often limited to research settings [17,18,37]. For instance, the work by Long and colleagues [18] has demonstrated that the use of nivolumab maintained baseline HRQoL levels to provide long-term quality of survival benefit among 418 patients with advanced melanoma. Cella and colleagues [19] have confirmed the association between nivolumab treatment and HRQoL improvement using the FACT-G among 847 patients with advanced renal cell carcinoma.

Although HRQoL has already been assessed with standard questionnaires in a number of clinical trials of immunotherapy, covered domains were pre-established and limited. Instead, in our study various new spontaneous subdomains emerged, such as fever, time constraints of treatment, difficulty in driving, and isolation. Furthermore, certain subdomains covered by the questionnaires might be inadequately designed for patients treated with ICI. For example, in the FACT-G, hope is collected in a negative way (ie, “I am losing hope in the fight against my illness”), whereas social media users mainly referred to hope in an optimistic way (ie, “regaining hope”).

Limitations of the Study

This study was not without limitations. First, selection bias was a major limitation because analyses were restricted to selected data sources and available contents. The population under study was composed of social media users who might not necessarily reflect the characteristics of all patients with cancer receiving immunotherapy. However, because social media are increasingly used by patients [38], especially in France [39], retrieved posts should pertain to an important section of the French population. We also collected data from relatives who provided immunotherapy-related experiences of patients with cancer who were not active on social media. However, given the small number of patients and relatives in our sample, we were not able to distinguish their posts within the analysis corpus. Our results should then be interpreted considering this further limitation. HRQoL self-assessed by patients might be different from the evaluation provided by relatives, as shown in previous research [40]. Similarly, the limited size of our analysis corpus did not allow an analysis per type of cancer. Melanoma, for instance, has long been treated with immunotherapy [41], which means that HRQoL of patients with melanoma might be different from HRQoL related to other cancers.

Second, an extraction bias is also possible because we only considered posts containing predefining keywords. If users expressed their experiences with immunotherapy and consequent impact on HRQoL by using other nonspecific words, their posts were not included in the final analysis corpus. To mitigate this bias, the set of keywords was as comprehensive as possible.

Third, because this study was based on secondary use of data published in social media, it was impossible to get additional data and information from patients (only identified by their pseudonym). The analysis was then restricted to what users mentioned, which can lead to missing data or incomplete capture of the patients’ full experience. In particular, subdomains emerging from social media were spontaneously addressed by users versus items from standard questionnaires. The fact that some items of the EORTC QLQ-C30 or the FACT-G were not mentioned in social media posts does not mean that patients were not concerned by them. For this, our conclusions should be considered with a certain degree of caution.

Fourth, because immunotherapy is a fairly recent therapeutic approach, few posts could be identified. A larger analysis corpus is needed to obtain more robust results and to validate our initial findings. As demonstrated in this study, the number of posts concerning immunotherapy is increasing year after year and new studies will benefit from this expanding analysis corpus. In particular, posts should be compared across countries where the EORTC QLQ-C30 and the FACT-G are usually administered to capture HRQoL. Exploration of potential cross-country differences in subdomains mentioned in social media would be noteworthy.

Fifth, social media represent an ideal place where patients can freely and spontaneously discuss their experiences with their therapy, thus providing valuable information on their HRQoL. However, this observation should be interpreted cautiously, because social media data may include a higher frequency of erroneous information, and patients posting on social media forums may not be representative of the wider patient population [10].

Finally, biases related to semantic analyses must be considered. Given the low number of posts within our analysis corpus, we were obliged to retrieve and code the mentions manually and could not apply automated analysis, for example, using topic modeling.

Implications and Future Research

We were able to include users’ subjective narratives in the evaluation of the impact of ICI on patients’ HRQoL. The results of our study suggest that commonly used measures such as the EORTC QLQ-C30 and the FACT-G may require updating to improve their coverage and applicability of HRQoL domains under real-world conditions. The challenge in measuring HRQoL lies in its uniqueness to individuals [35] and questionnaires such as the EORTC QLQ-C30 and the FACT-G might not take account of this by imposing standardized models of HRQoL. For this reason, as already demonstrated in studies concerning other diseases than cancer [42-46], posts in online forums and social media should be integrated in the assessment of patients’ HRQoL, because they can help either detect adverse events or characterize patient experience in a more individualized and spontaneous way.

In summary, this study suggested to explore further specific HRQoL domains related to patients treated with ICI to potentially enrich existing standard questionnaires with new items that are more relevant for these patients in their daily confrontation with disease and treatment.


Patients with cancer and their relatives are using social media to share their experiences with immunotherapy and its impact on HRQoL, particularly with regard to Global health and Symptoms. Emotional state and Role are also increasingly referenced in online forums and social media. Collecting and analyzing these spontaneous narratives can be helpful to capture how immunotherapy affects patients’ HRQoL in a more individualized way, thus obtaining information on more facets of life that are important for patients. While standard questionnaires can provide objective scores, which are easily interpretable from a clinical and research point of view, mining social media posts might better inform health care professions and patients of the impact of immunotherapy on patients’ HRQoL under real-world conditions. Future research is required to corroborate our findings and propose new individualized measures covering HRQoL more in depth than existing standard questionnaires.


The medical writing support was provided by Ilaria Montagni-Duchemin of the microenterprise Scientific Medical Writer IMD. The study was funded by Bristol-Myers Squibb.

Conflicts of Interest

F-EC, BB, LM, HL, and A-FG have disclosed that they are employees of Bristol-Myers Squibb. VG was a recipient of funding for studentships from Bristol-Myers Squibb. PV, PF, CF, and SS have disclosed that they are employees of Kap Code, a CRO, and are contracted with Bristol-Myers Squibb to carry out this study. BF and CT are contracted with Bristol-Myers Squibb to participate in this study.

Multimedia Appendix 1

Description of the websites used to retrieve the posts under study.

DOCX File , 15 KB

Multimedia Appendix 2

List of predefined keywords for posts inclusion.

DOCX File , 13 KB

  1. Kaplan RM. Health status and health policy: Quality of life in health care evaluation and resource allocation. Qual Life Res 1994 Dec;3(6):457-458. [CrossRef]
  2. Noble H. Quality of life and health related quality of life – is there a difference? Evidence-Based Nursing. London: BMJ; 2014 Jan 27.   URL: https:/​/blogs.​​ebn/​2014/​01/​27/​quality-of-life-and-health-related-quality-of-life-is-there-a-difference/​ [accessed 2020-08-28]
  3. Ferlay J, Colombet M, Soerjomataram I, Dyba T, Randi G, Bettio M, et al. Cancer incidence and mortality patterns in Europe: Estimates for 40 countries and 25 major cancers in 2018. Eur J Cancer 2018 Nov;103:356-387. [CrossRef] [Medline]
  4. International Agency for Research on Cancer. Cancer Tomorrow. Geneva: World Health Organization; 2019.
  5. Heydarnejad MS, Hassanpour DA, Solati DK. Factors affecting quality of life in cancer patients undergoing chemotherapy. Afr Health Sci 2011 Jun;11(2):266-270 [FREE Full text] [Medline]
  6. Aaronson NK, Ahmedzai S, Bergman B, Bullinger M, Cull A, Duez NJ, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 1993 Mar 03;85(5):365-376. [CrossRef] [Medline]
  7. Cella DF, Tulsky DS, Gray G, Sarafian B, Linn E, Bonomi A, et al. The Functional Assessment of Cancer Therapy scale: development and validation of the general measure. J Clin Oncol 1993 Mar;11(3):570-579. [CrossRef] [Medline]
  8. Tapi Nzali MD, Bringay S, Lavergne C, Mollevi C, Opitz T. What Patients Can Tell Us: Topic Analysis for Social Media on Breast Cancer. JMIR Med Inform 2017 Jul 31;5(3):e23 [FREE Full text] [CrossRef] [Medline]
  9. Brigo F, Igwe SC, Nardone R, Orioli A, Otte WM. Cancer information disparities on the internet: An infodemiological study. Journal of Cancer Policy 2016 Jun;8:33-37. [CrossRef]
  10. McDonald L, Malcolm B, Ramagopalan S, Syrad H. Real-world data and the patient perspective: the PROmise of social media? BMC Med 2019 Jan 16;17(1):11 [FREE Full text] [CrossRef] [Medline]
  11. Wiedemann G. Opening up to big data: Computer-assisted analysis of textual data in social sciences. Forum: Qualitative Social Research (Forum Qualitative Sozialforschung) 2013;14(2):Art13 [FREE Full text] [CrossRef]
  12. Hoos A, Britten CM. The immuno-oncology framework: Enabling a new era of cancer therapy. Oncoimmunology 2012 May 01;1(3):334-339 [FREE Full text] [CrossRef] [Medline]
  13. Ramirez RA, Lu J, Thomas KEH. Quality of life for non-small cell lung cancer patients in the age of immunotherapy. Transl Lung Cancer Res 2018 Apr;7(Suppl 2):S149-S152 [FREE Full text] [CrossRef] [Medline]
  14. Brahmer JR, Rodríguez-Abreu D, Robinson AG, Hui R, Csőszi T, Fülöp A, et al. Health-related quality-of-life results for pembrolizumab versus chemotherapy in advanced, PD-L1-positive NSCLC (KEYNOTE-024): a multicentre, international, randomised, open-label phase 3 trial. The Lancet Oncology 2017 Dec;18(12):1600-1609. [CrossRef]
  15. Cohen EEW, Bell RB, Bifulco CB, Burtness B, Gillison ML, Harrington KJ, et al. The Society for Immunotherapy of Cancer consensus statement on immunotherapy for the treatment of squamous cell carcinoma of the head and neck (HNSCC). J Immunother Cancer 2019 Jul 15;7(1):184 [FREE Full text] [CrossRef] [Medline]
  16. Iivanainen S, Alanko T, Peltola K, Konkola T, Ekström J, Virtanen H, et al. ePROs in the follow-up of cancer patients treated with immune checkpoint inhibitors: a retrospective study. J Cancer Res Clin Oncol 2019 Mar 21;145(3):765-774 [FREE Full text] [CrossRef] [Medline]
  17. Anagnostou V, Yarchoan M, Hansen AR, Wang H, Verde F, Sharon E, et al. Immuno-oncology Trial Endpoints: Capturing Clinically Meaningful Activity. Clin Cancer Res 2017 Aug 31;23(17):4959-4969. [CrossRef]
  18. Long G, Atkinson V, Ascierto P, Robert C, Hassel J, Rutkowski P, et al. Effect of nivolumab on health-related quality of life in patients with treatment-naïve advanced melanoma: results from the phase III CheckMate 066 study. Ann Oncol 2016 Oct;27(10):1940-1946 [FREE Full text] [CrossRef] [Medline]
  19. Cella D, Grünwald V, Escudier B, Hammers HJ, George S, Nathan P, et al. Patient-reported outcomes of patients with advanced renal cell carcinoma treated with nivolumab plus ipilimumab versus sunitinib (CheckMate 214): a randomised, phase 3 trial. The Lancet Oncology 2019 Feb;20(2):297-310. [CrossRef]
  20. Reck M, Taylor F, Penrod JR, DeRosa M, Morrissey L, Dastani H, et al. Impact of Nivolumab versus Docetaxel on Health-Related Quality of Life and Symptoms in Patients with Advanced Squamous Non-Small Cell Lung Cancer: Results from the CheckMate 017 Study. J Thorac Oncol 2018 Feb;13(2):194-204 [FREE Full text] [CrossRef] [Medline]
  21. Kürzinger ML, Schück S, Texier N, Abdellaoui R, Faviez C, Pouget J, et al. Web-Based Signal Detection Using Medical Forums Data in France: Comparative Analysis. J Med Internet Res 2018 Nov 20;20(11):e10466 [FREE Full text] [CrossRef] [Medline]
  22. Abdellaoui R, Foulquié P, Texier N, Faviez C, Burgun A, Schück S. Detection of Cases of Noncompliance to Drug Treatment in Patient Forum Posts: Topic Model Approach. J Med Internet Res 2018 Mar 14;20(3):e85 [FREE Full text] [CrossRef] [Medline]
  23. Ramasubramanian C, Ramya R. Effective Pre-Processing Activities in Text Mining using Improved Porter’s Stemming Algorithm. Int J Adv Res Computer and Commun Eng 2013;2(12):4536-4538 [FREE Full text]
  24. Luckett T, King M, Butow P, Oguchi M, Rankin N, Price M, et al. Choosing between the EORTC QLQ-C30 and FACT-G for measuring health-related quality of life in cancer clinical research: issues, evidence and recommendations. Ann Oncol 2011 Oct;22(10):2179-2190 [FREE Full text] [CrossRef] [Medline]
  25. Kuijpers W, Giesinger JM, Zabernigg A, Young T, Friend E, Tomaszewska IM, et al. Patients' and health professionals' understanding of and preferences for graphical presentation styles for individual-level EORTC QLQ-C30 scores. Qual Life Res 2016 Mar 9;25(3):595-604 [FREE Full text] [CrossRef] [Medline]
  26. Chopra I, Kamal KM. A systematic review of quality of life instruments in long-term breast cancer survivors. Health Qual Life Outcomes 2012 Jan 31;10(1):14 [FREE Full text] [CrossRef] [Medline]
  27. Wheelwright S, Darlington A, Hopkinson JB, Fitzsimmons D, White A, Johnson CD. A systematic review of health-related quality of life instruments in patients with cancer cachexia. Support Care Cancer 2013 Sep 25;21(9):2625-2636. [CrossRef] [Medline]
  28. Ferrans CE. Differences in what quality-of-life instruments measure. J Natl Cancer Inst Monogr 2007 Oct 18;2007(37):22-26. [CrossRef] [Medline]
  29. Fayers P, Bottomley A. Quality of life research within the EORTC—the EORTC QLQ-C30. Eur J Cancer 2002 Mar;38:125-133. [CrossRef]
  30. Conroy T, Mercier M, Bonneterre J, Luporsi E, Lefebvre J, Lapeyre M, et al. French version of FACT-G: validation and comparison with other cancer-specific instruments. Eur J Cancer 2004 Oct;40(15):2243-2252. [CrossRef]
  31. Uwer L, Rotonda C, Guillemin F, Miny J, Kaminsky M, Mercier M, et al. Responsiveness of EORTC QLQ-C30, QLQ-CR38 and FACT-C quality of life questionnaires in patients with colorectal cancer. Health Qual Life Outcomes 2011 Aug 22;9(1):70 [FREE Full text] [CrossRef] [Medline]
  32. Webster K, Cella D, Yost K. The Functional Assessment of Chronic Illness Therapy (FACIT) Measurement System: properties, applications, and interpretation. Health Qual Life Outcomes 2003 Dec 16;1:79 [FREE Full text] [CrossRef] [Medline]
  33. van Nieuwenhuizen C, Schene AH, Koeter MWJ, Huxley PJ. The Lancashire Quality of Life Profile: modification and psychometric evaluation. Soc Psychiatry Psychiatr Epidemiol 2001 Jan 12;36(1):36-44. [CrossRef] [Medline]
  34. The Whoqol Group. Development of the World Health Organization WHOQOL-BREF Quality of Life Assessment. Psychol. Med 1998 May 01;28(3):551-558. [CrossRef]
  35. Carr AJ, Higginson IJ. Are quality of life measures patient centred? BMJ 2001 Jul 02;322(7298):1357-1360 [FREE Full text] [CrossRef] [Medline]
  36. Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, et al. Content validity--establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2--assessing respondent understanding. Value Health 2011 Dec;14(8):978-988 [FREE Full text] [CrossRef] [Medline]
  37. Cella D, Grünwald V, Nathan P, Doan J, Dastani H, Taylor F, et al. Quality of life in patients with advanced renal cell carcinoma given nivolumab versus everolimus in CheckMate 025: a randomised, open-label, phase 3 trial. The Lancet Oncology 2016 Jul;17(7):994-1003. [CrossRef]
  38. De Martino I, D'Apolito R, McLawhorn AS, Fehring KA, Sculco PK, Gasparini G. Social media for patients: benefits and drawbacks. Curr Rev Musculoskelet Med 2017 Mar 21;10(1):141-145 [FREE Full text] [CrossRef] [Medline]
  39. 39 J. Les usages de l’Internet santé : vers une parentalité connectée ? La prévention toujours en re-création. À l’école la PMI 2016:155-186 [FREE Full text] [CrossRef]
  40. Minaya P, Baumstarck K, Berbis J, Goncalves A, Barlesi F, Michel G, et al. The CareGiver Oncology Quality of Life questionnaire (CarGOQoL): development and validation of an instrument to measure the quality of life of the caregivers of patients with cancer. Eur J Cancer 2012 Apr;48(6):904-911. [CrossRef] [Medline]
  41. Lugowska I, Teterycz P, Rutkowski P. Immunotherapy of melanoma. Contemp Oncol (Pozn) 2018 Mar;22(1A):61-67 [FREE Full text] [CrossRef] [Medline]
  42. Mao JJ, Chung A, Benton A, Hill S, Ungar L, Leonard CE, et al. Online discussion of drug side effects and discontinuation among breast cancer survivors. Pharmacoepidemiol Drug Saf 2013 Mar 16;22(3):256-262 [FREE Full text] [CrossRef] [Medline]
  43. Schück S, Foulquié P, Mebarki A, Voillot P, Texier N, Faviez C. Détection automatique du mésusage des neuroleptiques dans le trouble anxieux et la démence à partir des réseaux sociaux. Revue d'Épidémiologie et de Santé Publique 2018 Jun;66:S211-S212. [CrossRef]
  44. Chen X, Faviez C, Schuck S, Lillo-Le-Louët A, Texier N, Dahamna B, et al. Mining Patients' Narratives in Social Media for Pharmacovigilance: Adverse Effects and Misuse of Methylphenidate. Front Pharmacol 2018;9:541 [FREE Full text] [CrossRef] [Medline]
  45. Foulquié P, Voillot P, Faviez C, Mebarki A, Chen X, Texier N, et al. Étude de l’usage du méthylphénidate sur les réseaux sociaux. Revue d'Épidémiologie et de Santé Publique 2018 Jun;66:S225. [CrossRef]
  46. Schück S, Voillot P, Foulquié P, Faviez C, Mebarki A, Texier N, et al. Que nous apportent les réseaux sociaux quant à la crise sanitaire du Levothyrox ® d’août 2017 ? Revue d'Épidémiologie et de Santé Publique 2018 Jun;66:S225. [CrossRef]

EORTC: European Organization for Research and Treatment of Cancer
FACT-G: Functional Assessment of Cancer Therapy - General
HRQoL: health-related quality of life
ICI: immune checkpoint inhibitors
QLQ-C30: Quality of Life Questionnaire Core 30

Edited by G Eysenbach; submitted 28.04.20; peer-reviewed by A Benetoli, M El Tantawi; comments to author 04.07.20; revised version received 10.07.20; accepted 26.07.20; published 11.09.20


©François-Emery Cotté, Paméla Voillot, Bryan Bennett, Bruno Falissard, Christophe Tzourio, Pierre Foulquié, Anne-Françoise Gaudin, Hervé Lemasson, Valentine Grumberg, Laura McDonald, Carole Faviez, Stéphane Schück. Originally published in the Journal of Medical Internet Research (, 11.09.2020.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.