This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
The most frequent malignant disease in women is breast cancer. In the metastatic setting, quality of life is the primary therapeutic goal, and systematic treatment has only a limited effect on survival rates; therefore, the concept of the health-related quality of life (HRQoL) and measurement of patient-reported outcomes (PROs) are gaining more and more importance in the therapy setting of diseases such as breast cancer. One of the frequently used questionnaires for measuring the HRQoL in patients with breast cancer is the Functional Assessment of Cancer Therapy-Breast (FACT-B). Currently, paper-based surveys still predominate, as only a few reliable and validated electronic-based questionnaires are available. ePRO tools for the FACT-B questionnaire with proven reliability are missing so far.
The aim of this study was to analyze the reliability of tablet-based measurement of FACT-B in the German language in adjuvant (curative) and metastatic breast cancer patients.
Paper- and tablet-based questionnaires were completed by a total of 106 female adjuvant and metastatic breast cancer patients. All patients were required to complete the electronically based (ePRO) and paper-based version of the FACT-B. A frequency analysis was performed to determine descriptive sociodemographic characteristics. Both dimensions of reliability (parallel forms reliability using Wilcoxon test and test of internal consistency using Spearman ρ) and agreement rates for single items, Kendall tau for each subscale, and total score were analyzed.
High correlations were shown for both dimensions of reliability (parallel forms reliability and internal consistency) in the patients’ response behavior between paper-based and electronically based questionnaires. Regarding the reliability test of parallel forms, no significant differences were found in 35 of 37 single items, while significant correlations in the test for consistency were found in all 37 single items, in all 5 sum individual item subscale scores, as well as in total FACT-B score.
The ePRO version of the FACT-B questionnaire is reliable for patients with breast cancer in both adjuvant and metastatic settings, showing highly significant correlations with the paper-based version in almost all questions all subscales and the total score.
The most frequent malignant disease in women is breast cancer; indeed, about 70,000 new cases of breast cancer are diagnosed in Germany every year. Therapeutic options have been improved, resulting in an overall 5-year survival rate of patients with early-stage disease of >90% [
Since systematic palliative treatment has only a limited effect on survival rates, the concept of the health-related quality of life (HRQoL) and measurement of patient-reported outcomes (PROs) are gaining more and more importance in the therapy setting of progressive diseases, such as breast cancer, especially in the adjuvant or metastatic setting [
Collecting and analyzing pencil and paper-based data are difficult tasks without possibilities for direct response or interaction [
The aim of the study was to analyze the reliability of a tablet-based ePRO app for FACT-B in German for measuring the HRQoL in adjuvant and metastatic breast cancer patients in comparison with the validated paper-based version of FACT-B. It was planned to determine whether differences exist in response behavior between the validated paper-based PRO version of FACT-B and a new ePRO version, whether the answers between paper-based and ePRO questionnaire differ in a relevant way, and whether the patients’ response behavior is influenced by the mode of answering (paper- or tablet-based). In order to achieve these aims, patients were asked to complete both the paper- and tablet-based version of the FACT-B questionnaire.
From July 2015 to May 2016, paper-based and tablet-based PRO questionnaires were completed by a total of 106 female adjuvant and metastatic breast cancer patients treated consecutively at the Department of Women’s Health (Tübingen, Germany) and the National Cancer Center (Heidelberg, Germany). Patients were recruited as a part of the electronic Patient-Reported Outcomes and Compliance Analysis (ePROCOM) and Patient Engagement Pilotstudie Mammakarzinom-individualisierte und Ressourcen-effiziente Patient Reported Outcomes Erfassung durch Digitale Therapieunterstützungssysteme (PEPPER) studies. The aims of ePROCOM were to evaluate the general patient acceptance and practicability of a Web-based app for a PRO questionnaire for patients with adjuvant or metastatic breast cancer. Patients were asked to participate to compare their response behavior in paper-based and Web-based questionnaires and analyze the reliability of the ePRO versions of the questionnaires European Organisation for Research and Treatment of Cancer (EORTC) quality of life questionnaire (QLQ)-C30, as published previously [
The data collection was performed in 4 parts. The first part focused on patientsʼ socioeconomic variables. The second part comprised the FACT-B questionnaire, consisting of 37 questions with responses required on a 5-point Likert scale (from 0,
For measuring ePRO, we used the “Patient-informiert- interaktiv-Arzt (PiiA),” that is “patient interactively informs doctor” Web-based app, which allows patients to answer the relevant questions on a tablet. The PiiA portal is a Web-based solution for capturing PROs, which our working group has developed. Patients receive anonymized user credentials and are asked to complete the FACT-B questionnaire.
All statistical analyses were conducted using SPSS Statistics (IBM, version 24). First, a frequency analysis was performed to determine the descriptive sociodemographic characteristics of patients. After that, we analyzed both dimensions of reliability (parallel forms reliability and test of internal consistency) and examined the disparity of responses and the rate of consistency between paper- and tablet-based responses. Both types of reliability were calculated for the 37 single items as well as for scores of the 5 dimensions, including the subscales for Physical Well-Being (PWB), Social/Family Well-Being (SWB), Emotional Well-Being (EWB), Functional Well-Being (FWB), and Breast Cancer Subscale (BCS), and the FACT-B total score in accordance with the FACT-B guidelines [
Screenshot of the Patient-informiert-interaktiv-Arzt app’s FACT-B (Functional Assessment of Cancer Therapy-Breast) questionnaire for the dimension “physical well-being” (German). Source: Authors work, licensed under fair use.
As such an analysis is considered an explorative study, all reported
Overall, 106 eligible female patients with breast cancer were recruited who completed questions from FACT-B both in a paper-based format and electronically using a tablet. Originally, 153 patients were assessed for eligibility, of which 47 were excluded during recruiting, allocation, and data analysis, as shown in the Consolidated Standards of Reporting Trials flow diagram (
In all, 53 patients were assigned to tablet-based filling followed by paper-based filling in the same session (arm A), while the same number of patients completed the paper-based version followed by the tablet-based questionnaire (arm B). Both the paper- and tablet-based questionnaires were completed by patients consecutively during the same ambulance visit. Patients who had not completed more than half of the FACT-B questions in either paper- or tablet-based format were excluded (arm A, 1 patient; arm B, 2 patients). We did not find any significant differences between the 2 arms in the response behavior, sociodemographic status, or therapy setting; therefore, the 2 arms were considered together. Beforehand, all single items of the 2 arms were compared. Ten patients (arm A) and 16 patients (arm B), respectively, produced missing data in some questions (more often in the tablet-based questionnaire) [
Consolidated Standards of Reporting Trials flow diagram.
Sociodemographic characteristics of the patients (n=106).
Sociodemographic variables | Descriptive analyses | |
Mean (SD) | 51 (11.31) | |
Median (range, minimum-maximum) | 52 (54, 30-84) | |
Median | 3 | |
Interquartile range (25%-75% quantiles) | 2 (3-5) |
aLevel of education: 1=lowest; 5=highest.
Education level and therapy setting of the patients (n=106).
Variable | Frequency, n (%) | 95% CI | |
No qualification | 1 (0.9) | 0-6 | |
Main or secondary school graduation | 43 (40.5) | 32-50 | |
Advanced technical graduation | 19 (17.9) | 10-26 | |
High school diplomaa | 33 (31.1) | 22-40 | |
Not specified | 10 (9.4) | 2-15 | |
Metastatic | 30 (28.3) | 19-35 | |
Adjuvant setting | 76 (71.6) | 61-83 |
aHigh school diploma indicates “Abitur.”
In addition, slight differences were noted between paper-based PRO and ePRO in the individual item scores of the 5 dimensions and the total FACT-B score (
The total score is slightly higher in ePRO (mean difference: 1.73; median difference: 0.63), but without statistically significant differences.
Parallel forms reliability (Wilcoxon test) in single items.
Single items | Paper-based patient-reported outcomes | Electronic patient-reported outcomes | |||||
Mean (SD) | Median (Interquartile range) | Mean (SD) | Median (Interquartile range) | ||||
GP1 | 1.68 (1.22) | 2 (2) | 1.59 (1.21) | 1 (1) | .22 | ||
GP2 | 0.54 (0.86) | 0 (1) | 0.60 (0.88) | 0 (1) | .26 | ||
GP3 | 1.34 (1.21) | 1 (2) | 1.29 (1.18) | 1 (2) | .72 | ||
GP4 | 0.98 (1.01) | 1 (1.25) | 1.03 (0.99) | 1 (2) | |||
GP5 | 1.50 (1.17) | 1 (1) | 1.58 (1.09) | 1 (1) | .08 | ||
GP6 | 1.22 (1.12) | 1 (2) | 1.23 (1.05) | 1 (2) | .24 | ||
GP7 | 0.77 (1.09) | 0 (1) | 0.73 (1.03) | 0 (1) | >.99 | ||
GS1 | 3.20 (0.94) | 3 (1) | 3.16 (1.04) | 3 (1) | .82 | ||
GS2 | 3.61 (0.73) | 4 (1) | 3.5 (0.86) | 4 (1) | |||
GS3 | 3.18 (1.09) | 4 (1) | 3.14 (1.1) | 3 (1) | >.99 | ||
GS4 | 3.33 (0.75) | 3 (1) | 3.28 (0.81) | 3 (1) | .39 | ||
GS5 | 3.37 (0.82) | 4 (1) | 3.33 (0.77) | 3 (1) | .11 | ||
GS6 | 3.65 (0.77) | 4 (.0) | 3.63 (0.82) | 4 (1) | .83 | ||
GS7 | 1.98 (1.66) | 2 (2.5) | 2.07 (1.11) | 2 (2) | .72 | ||
GE1 | 1.32 (1.09) | 1 (1) | 1.22 (1.01) | 1 (1) | .40 | ||
GE2 | 2.68 (1.16) | 3 (2) | 2.91 (0.98) | 3 (2) | .05 | ||
GE3 | 0.60 (1.17) | 0 (1) | 0.44 (0.77) | 0 (1) | .11 | ||
GE4 | 1.19 (1.03) | 1 (2) | 1.12 (0.99) | 1 (2) | .83 | ||
GE5 | 1.22 (1.14) | 1 (2) | 1.18 (1.05) | 1 (1.5) | .70 | ||
GE6 | 1.42 (1.26) | 1 (1) | 1.32 (1.06) | 1 (1) | .59 | ||
GF1 | 2.12 (1.22) | 1 (1) | 2.21 (1.20) | 1 (1) | .23 | ||
GF2 | 2.30 (1.20) | 2 (2) | 2.32 (1.12) | 2 (2) | .34 | ||
GF3 | 2.50 (1.10) | 3 (1) | 2.51 (1.11) | 3 (1) | .81 | ||
GF4 | 2.58 (1.04) | 3 (1) | 2.55 (1.01) | 3 (1) | .39 | ||
GF5 | 2.40 (1.18) | 3 (1) | 2.41 (1.15) | 3 (1) | .25 | ||
GF6 | 2.53 (1.15) | 3 (1) | 2.65 (1.05) | 3 (1) | .20 | ||
GF7 | 2.19 (1.15) | 2 (1) | 2.17 (1.07) | 2 (1.5) | .81 | ||
B1 | 0.75 (0.93) | 0.25 (1) | 0.72 (0.90) | 0.5 (1) | .49 | ||
B2 | 0.58 (1.10) | 0 (1) | 0.52 (0.99) | 0 (1) | .79 | ||
B3 | 0.68 (1.03) | 0 (1) | 0.63 (0.91) | 0 (1) | >.99 | ||
B4 | 1.73 (1.04) | 2 (1) | 1.69 (1.01) | 2 (1) | .81 | ||
B5 | 1.47 (1.41) | 1 (2) | 1.47 (1.38) | 1 (2) | .49 | ||
B6 | 1.88 (1.41) | 2 (2) | 1.82 (1.45) | 1.5 (2) | .06 | ||
B7 | 2.02 (1.38) | 2 (2) | 2.14 (1.33) | 2 (2) | .33 | ||
B8 | 1.23 (1.34) | 1 (2) | 1.13 (1.37) | 1 (2) | .11 | ||
B9 | 2.55 (1.18) | 3 (1) | 2.62 (1.17) | 3 (2) | .58 | ||
P2 | 1.28 (1.11) | 1 (2) | 1.13 (1.05) | 1 (1.75) | .18 |
aStatistically significant difference.
Parallel forms reliability (Wilcoxon test) for scoring values of 5 Functional Assessment of Cancer Therapy-Breast (FACT-B) dimensions (subscales).
FACT-Ba dimensions | FACT-B questions | Paper-based patient-reported outcomes | Electronic patient-reported outcomes | |||
Mean (SD) | Median (Interquartile range) | Mean (SD) | Median (Interquartile range) | |||
Physical Well-Being Sum individual item scores | GP1-GP7 | 19.97 (6.11) | 21.0 (9.0) | 19.89 (5.88) | 20.0 (8.25) | 0.05 |
Social/Family Well-Being Sum individual item scores | GS1-GS7 | 22.88 (3.93) | 24.0 (5.0) | 22.34 (4.60) | 23.33 (5.0) | 0.25 |
Emotinal Well-Being Sum individual item scores | GE1-GE6 | 16.89 (4.84) | 18.0 (6.0) | 17.73 (4.68) | 18.0 (6.0) | . |
Functional Well-Being Sum individual item scores | GF1-GF7 | 16.73 (6.29) | 18.0 (9.0) | 16.75 (6.10) | 18.0 (9.45) | 0.69 |
Breast Cancer Subscale Sum individual item scores | B1-B9; P2 | 26.35 (6.28) | 28.0 (9.0) | 28.56 (7.11) | 30.0 (11.33) | <. |
FACT-B total score | 102.66 (22.0) | 106.33 (28.81) | 104.39 (22.47) | 107 (30.31) | 0.05 |
aFACT-B: Functional Assessment of Cancer Therapy-Breast
aStatistically significant difference.
Distribution of total scores (Boxplot). PRO: patient-reported outcome.
Test of internal consistency in single items: results of correlation (Spearman
Dimensions | Spearman ρ | Agreement (%) | ||
GP1 | 0.869 | <.001 | 72.0 | |
GP2 | 0.836 | <.001 | 86.9 | |
GP3 | 0.866 | <.001 | 70.1 | |
GP4 | 0.836 | <.001 | 75.4 | |
GP5 | 0.837 | <.001 | 68.3 | |
GP6 | 0.842 | <.001 | 71.5 | |
GP7 | 0.889 | <.001 | 86.4 | |
GS1 | 0.782 | <.001 | 77.8 | |
GS2 | 0.880 | <.001 | 85.4 | |
GS3 | 0.908 | <.001 | 87.8 | |
GS4 | 0.782 | <.001 | 76.3 | |
GS5 | 0.829 | <.001 | 81.3 | |
GS6 | 0.931 | <.001 | 91.8 | |
GS7 | 0.747 | <.001 | 77.0 | |
GE1 | 0.753 | <.001 | 71.0 | |
GE2 | 0.525 | <.001 | 66.6 | |
GE3 | 0.796 | <.001 | 82.3 | |
GE4 | 0.868 | <.001 | 78.0 | |
GE5 | 0.931 | <.001 | 85.8 | |
GE6 | 0.733 | <.001 | 65.7 | |
GF1 | 0.881 | <.001 | 74.0 | |
GF2 | 0.770 | <.001 | 69.7 | |
GF3 | 0.889 | <.001 | 81.0 | |
GF4 | 0.821 | <.001 | 71.5 | |
GF5 | 0.934 | <.001 | 87.2 | |
GF6 | 0.910 | <.001 | 82.4 | |
GF7 | 0.897 | <.001 | 75.2 | |
B1 | 0.858 | <.001 | 86.4 | |
B2 | 0.925 | <.001 | 87.0 | |
B3 | 0.904 | <.001 | 87.9 | |
B4 | 0.777 | <.001 | 80.7 | |
B5 | 0.900 | <.001 | 80.6 | |
B6 | 0.949 | <.001 | 83.6 | |
B7 | 0.843 | <.001 | 69.5 | |
B8 | 0.945 | <.001 | 86.0 | |
B9 | 0.684 | <.001 | 70.1 | |
P2 | 0.850 | <.001 | 73.8 |
aStatistically highly significant correlations.
Test of internal consistency in the individual subscale item scores and the total score: Kendall tau analysis.
Dimensions | Kendall tau (95% CI) | |
Physical Well-Being Sum individual item scores | 0.784 (0.723-0.835) | <. |
Social/Family Well-Being Sum individual item scores | 0.648 (0.545-0.749) | <. |
Emotional Well-Being Sum individual item scores | 0.737 (0.638-0.820) | <. |
Functional Well-Being Sum individual item scores | 0.797 (0.731-0.858) | <. |
Breast Cancer Subscale Sum individual item scores | 0.638 (0.536-0.724) | <. |
Total score | 0.801 (0.741-0.852) | <. |
aStatistically significant correlations.
Correlation between electronic patient-reported outcome (ePRO) and paper-based total Functional Assessment of Cancer Therapy-Breast scores.
In both dimensions of reliability (parallel forms reliability and internal consistency), we found high correlations when comparing single items in the patients’ response behavior between paper-based PRO and ePRO in the FACT-B questionnaire. In the test of parallel forms reliability, we only found statistically significant differences in all but 2 questions. In the test of consistency, moderate to high correlations were found in all 37 single items and all sum individual item subscale scores. Based on the empirical results, the PiiA tool’s ePRO version of FACT-B seems to be reliable for measuring the HRQoL in patients with metastatic and adjuvant breast cancer. According to these results, we would not expect that future use of the PiiA tool in the
Although ePRO apps are being used increasingly frequently, paper-based surveys of PRO still predominate in clinical research because reliable, electronically validated questionnaires are lacking. One of the most commonly used questionnaires for measuring the HRQoL, especially in patients with breast cancer, is the FACT-B, for which there is a reliable, paper-based format in many languages but no reliable, electronically based version exists in German. Using the electronically based version of FACT-B and other PRO without verifying the reliability could endanger the significance of ePRO surveys. Thus, a corresponding analysis in relation to differences and correlations between the paper-based version of FACT-B and newly developed Web-based tools are of great importance. It can be assumed that many aspects (ie, sociodemographic aspects, technical skills, health condition, and, perhaps, design specifics of the ePRO tool) may influence both patients’ willingness to use the tool and their response behavior, which underlines the need for reliability analyses [
Despite positive results, some limitations of the study design and methodological implementation should be mentioned, which could reduce the representativeness of the data. In 2 of 37 questions, we found statistically significant differences by the Wilcoxon test (parallel forms reliability). This can be a random observation based on numerous tests performed. A further explanation could be that patients were less concentrated owing to time pressure. Patients (both in arms A and B) were required to complete both paper-based and ePRO during an outpatient hospital visit. Patients were surveyed while they were receiving chemotherapy and were not permitted to take the questionnaire home to complete it; this also explains increasing missing values, especially in the last quarter of both questionnaire versions; possibly the length of the survey (paper-based and ePRO FACT-B and EORTC QLQ C-30, socioeconomic data, and evaluation of the tool) was too extensive for some patients. Possibly, the burden of disease and the therapy were potentially affecting the ability of some patients to complete both the paper- and tablet-based version of the questionnaire during an outpatient visit. Hence, a possible limiting factor was an inadequate screening as to whether all patients were able to cope with the psychological burden of participating in the study, as it is known that psycho-oncological distress is a commonly associated burden that could potentially influence the willingness to use ePRO and as a result ePRO’s reliability [
Although ePRO is being used more and more often, questionnaires with proven reliability and validity are lacking. FACT-B is one of the most commonly used questionnaires worldwide for measuring the HRQoL in patients with breast cancer, but a reliable ePRO version is also missing here. One of the strengths of this study is that the reliability of a new tool for the ePRO measurement in patients with breast cancer was analyzed, while other studies often assign paper-based versions to a tablet-based format without verifying the reliability. The reliability of the PiiA tool could be ascertained for the questionnaires FACT-B (this paper) and EORTC QLQ C-30 [
Electronically based PRO is constantly being adopted in clinical research and clinical routine, which underlines the need for reliable questionnaires. The evaluated electronic version of the FACT-B is reliable for patients with breast cancer in an adjuvant or metastatic setting because high correlations were found in almost all questions, all subscales, and the total score. Thus, this study concludes that the validated paper-based questionnaire of FACT-B and the ePRO tool are equal.
CONSORT-EHEALTH checklist (V 1.6.1).
Breast Cancer Subscale
electronic Patient-Reported Outcomes and Compliance Analysis
European Organisation for Research and Treatment of Cancer
Emotional Well-Being
Functional Assessment of Cancer Therapy-Breast
Functional Well-Being
health-related quality of life
Patient Engagement Pilotstudie Mammakarzinom - individualisierte und Ressourcen-effiziente Patient Reported Outcomes Erfassung durch Digitale Therapieunterstützungssysteme
Patient-informiert-interaktiv-Arzt
patient-reported outcomes
Physical Well-Being
quality of life questionnaire
Social/Family Well-Being
This study was supported by the Klaus Tschira Stiftung, gemeinnützige GmbH.
PG has received honoraria from Novartis and financial support for symposia from Novartis, Roche, and PharmaMar.