This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Electronic applications are increasingly being used in hospitals for numerous purposes.
Our aim was to assess differences in the characteristics of patients who choose paper versus electronic questionnaires and to evaluate the data quality of both approaches.
Between October 2012 and June 2013, 136 patients participated in a study on diagnosis-induced stress and anxiety. Patients were asked to fill out questionnaires at six different moments during the diagnostic phase. They were given the opportunity to fill out the questionnaires on paper or electronically (a combination of tablet and Web-based questionnaires). Demographic characteristics and completeness of returned data were compared between groups.
Nearly two-thirds of patients (88/136, 64.7%) chose to fill out the questionnaires on paper, and just over a third (48/136, 35.3%) preferred the electronic option. Patients choosing electronic questionnaires were significantly younger (mean 47.3 years vs mean 53.5 in the paper group,
Younger patients and patients with a higher level of education have a preference towards filling out questionnaires electronically. In the hospital, a tablet is an excellent medium for patients to fill out questionnaires with very little missing information. However, for filling out questionnaires at home, paper questionnaires resulted in a better response than Web-based questionnaires.
With the evolution of modern technology, electronic applications are increasingly being used in hospitals. Web-based applications and touchscreen devices are finding their way into hospitals for numerous purposes. These electronic applications can be useful for research purposes, for collecting patient-reported outcomes, and questionnaires [
Obtaining high response rates without missing information is important for research purposes, as non-responders can bias study results [
The aim of our study was to assess differences in demographic characteristics of patients choosing paper versus electronic questionnaires and to evaluate data quality and completeness of data of both approaches.
This study was performed in the University Medical Center Utrecht, the Netherlands (approximate caseload of 180 newly diagnosed breast cancer patients per year). In 2011, same-day diagnosis for breast cancer was introduced with the aim to provide a definitive diagnosis within one day in over 80% of patients. Reducing the time of uncertainty about a diagnosis could potentially reduce anxiety and stress. All patients suspected of breast cancer visited the outpatient breast clinic and underwent physical examination, diagnostic imaging (mammography and ultrasound) with a histological biopsy if indicated, and received a final diagnosis at the end of the day after a multidisciplinary meeting.
Between mid-October 2012 and June 2013, all patients referred to the same-day diagnosis out-patient breast clinic were eligible to participate in the study. Approval for this study was obtained from the local ethics committee, and all patients signed written informed consent. All patients were asked to fill out the 6-item State Trait Anxiety Inventory (STAI) [
Outcome measures included differences in demographic characteristics between patients choosing paper or electronic questionnaires and data quality, focusing on age, reason for referral, breast cancer history, level of education, and baseline anxiety. Data quality was assessed by focusing on missing information, defined as a questionnaire containing at least one unanswered question. To assess if a breast cancer diagnosis affected the quality of data, subgroup analysis including only patients with a benign diagnosis was performed.
Demographics, history of breast disease, and diagnostic findings were described as proportions and means with standard deviation. Differences in demographic characteristics, reported anxiety score, and completeness of reported data between the electronic and the paper group were compared by means of chi-square test and independent samples
Overview of the six measuring moments during the diagnostic phase to evaluate level of stress and anxiety.
Screenshot of the electronic questionnaire (measuring moment 2).
Screenshot of the paper questionnaire (measuring moment 2).
Of 321 patients referred to our out-patient breast clinic, 136 patients (42.4%) agreed to participate in the study. All patients were offered the choice of paper or electronic questionnaires.
The mean age was 51.3 years (range 18-85 years) and 35.3% (48/136) patients chose to fill out the questionnaires electronically (
There was significantly more missing information (ie, questionnaires containing at least one unanswered question) in the paper group during the diagnostic day (measuring moments 1-3) compared to the electronic group (28/88 vs 1/48,
In the week after the diagnostic day (measuring moments 4-6), missing information was significantly more prevalent in the electronic group (41/48, 85%) compared to the paper group (38/88, 43%) (
Demographic characteristics of patients undergoing 1-day diagnosis for suspected breast cancer, comparing patients choosing paper questionnaires (n=88) to those choosing electronic questionnaires (n=48).
Characteristics | Paper, n (%) | Electronic, n (%) |
|
|
Female | 86 (98) | 47 (98) | .94b | |
Age in years, mean (SD), range | 53.5 (14.1), 22-85 | 47.3 (11.4), 18-66 | .01c | |
|
||||
|
Screening | 27 (31) | 16 (37) | .75b |
|
Palpable lesion/symptoms | 61 (69) | 32 (67) |
|
Positive family history of breast cancer – yes | 34 (39) | 13 (27) | .18b | |
Previous breast disease/complaints – yes | 28 (32) | 19 (40) | .36b | |
Previous breast cancer diagnosis – yes | 4 (5) | 3 (6) | .67 b | |
|
||||
|
I | 18 (21) | 9 (19) | .54b |
|
II | 34 (39) | 22 (46) |
|
|
III | 9 (10) | 9 (19) |
|
|
IV | 15 (17) | 5 (10) |
|
|
V | 10 (11) | 3 (6) |
|
|
VI | 1 (1) | 0 (0) |
|
|
No imaging performed | 1 (1) | 0 (0) |
|
|
||||
|
No | 51 (58) | 29 (60) | .62b |
|
Yes, histology | 32 (36) | 18 (38) |
|
|
Yes, cytology | 5 (6) | 1 (2) |
|
Cancer – yes | 16 (18) | 7 (15) | .59b | |
|
||||
|
Low-moderatee | 21 (26) | 1 (3) | .004b |
|
Moderate-highf | 27 (33) | 10 (29) |
|
|
Highg | 33 (41) | 24 (69) |
|
|
Missing | 7 | 13 |
|
Mean baseline anxiety score, STAI (SD) | 46.4 (13.3) | 43.8 (13.5) | .30c |
a
bCalculated by chi-square test.
cCalculated by independent samples
dLevel of education is based on the Dutch educational system
eLow-moderate education includes primary education/low pre-vocational/secondary general education.
fModerate-high education includes secondary vocational/higher general and pre-university education.
gHigh education includes higher vocational education/university.
Differences in proportion of patients with incompletely filled out questionnaires between patients opting for paper questionnaires (n=88) and patients choosing electronic questionnaires (n=48).
|
Paper, n (%) | Electronic, n (%) |
|
Baseline – moment 1 – STAI 1 | 25 (28) | 0 (0) |
|
Measuring moment 2 – STAI 2 | 24 (27) | 1 (2) |
|
Measuring moment 3 – STAI 3 | 25 (28) | 1 (2) |
|
Measuring moment 4 – STAI 4 | 28 (32) | 27 (56) |
|
Measuring moment 5 – STAI 5 | 31 (35) | 34 (71) |
|
Measuring moment 6 – STAI 6 | 35 (40) | 33 (69) |
|
Measuring moment 1-3 – in hospitala | 28 (32) | 1 (2) | <.001 |
Measuring moment 4-6 – at homeb | 38 (43) | 41 (85) | <.001 |
aIncludes all patients with at least 1 incomplete questionnaire in measuring moment 1, 2, or 3.
bIncludes all patients with at least 1 incomplete questionnaire in measuring moment 4, 5, or 6.
The use of tablets and Web-based questionnaires for collection of patient-reported data has many potential advantages over the use of paper questionnaires. Still, the present study shows that a majority of patients preferred paper over electronic questionnaires. Younger patients and those with a higher level of education were more likely to opt for electronic questionnaires. When using tablets (during the diagnostic day in the hospital), more complete information was collected compared to using paper questionnaires. These data suggests that tablets are superior to paper questionnaires. However, the use of Web-based questionnaires resulted in less complete data collection than paper questionnaires. This might be due to the study design where patients could fill out the electronic questionnaires only on a specific day.
A major advantage of filling out electronic questionnaires is that information is immediately saved. Other advantages of the use of tablets include the possibility of automatically reminding the patient to fill out the questionnaires, and providing information and entertainment. We did not electronically remind patients by email to fill out the questionnaires. Considering the high percentage of incompletely filled out Web-based questionnaires (85%), we would definitively incorporate this in a future study. We did use an automated message when not all questions were answered. This resulted in completely filled out questionnaires in the electronic group, which could possibly lead to more complete data.
Possible drawbacks of using Web-based questionnaires are high non-response rates, impaired reliability and validity, and safety or confidentiality issues [
Completeness of data collection was very high in the tablet group, with only 1 of the 48 patients not filling out all questions at the first three measuring moments. Missing information was highest in the Web-based group, where many patients (41/48) did not fill out all questions at the last three measuring moments. One likely reason for the high rate of missing information in this group was that patients could fill out the questionnaires only on the correct day (ie, exactly 1, 3, or 7 days after the patient’s visit). Our aim was to measure patients’ anxiety at specific moments in time, and we limited the possibility of filling out the questionnaires to the correct day only. Patients in the paper group, however, were able to fill out the questionnaires at any given time. This led to more missing information in the Web-based group, and results on missing information need to be interpreted with this information in mind. However, limiting patients to one specific moment to fill out the questionnaires might lead to more accurate measurements of patients’ anxiety at that specific moment. For higher response in the Web-based group, automated email reminders could be useful.
A possible limitation of this study was that we included only breast cancer patients and consequently, 98% were female. These results are therefore not generalizable to other populations. There could be reduced missing data in the electronic group when other groups are included in a similar study (eg, men, young adults).
Younger patients and patients with a higher level of education have a preference towards filling out questionnaires electronically. In the hospital, a tablet is an excellent medium for patients to fill out questionnaires with very little missing information. However, for filling out questionnaires at home, paper questionnaires result in a better response compared to Web-based questionnaires.
Breast Imaging Reporting and Data System
hypertext markup language
State Trait Anxiety Inventory
The authors thank Peter Sommer and Pieter Ruijssenaars from BeagleBoxx for their cooperation and providing the software and hardware for measuring patient impact of same-day diagnosis.
None declared.