This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Searching the internet for cancer-related information helps patients with cancer satisfy their unmet information needs and empowers them to play a more active role in the management of their disease. However, to benefit from the search, patients need a sufficient level of skill to search, select, appraise, and apply web-based health information.
We aim to study the operational, navigational, information, and evaluation skills and problems of patients with cancer performing cancer-related search tasks using the internet.
A total of 21 patients with cancer were recruited during their stay at the rehabilitation clinic for oncological rehabilitation. Participants performed eight cancer-related search tasks using the internet. The participants were asked to think aloud while performing the tasks, and the screen activities were recorded. The types and frequencies of performance problems were identified and coded into categories following an inductive coding process. In addition, the performance and strategic characteristics of task execution were summarized descriptively.
All participants experienced problems or difficulties in executing the tasks, and a substantial percentage of tasks (57/142, 40.1%) could not be completed successfully. The participants’ performance problems were coded into four categories, namely operating the computer and web browser, navigating and orientating, using search strategies, and evaluating the relevance and reliability of web-based information. The most frequent problems occurred in the third and fourth categories. A total of 90% (19/21) of participants used nontask-related search terms or nonspecific search terms. A total of 95% (20/21) of participants did not control for the source or topicality of the information found. In addition, none of the participants verified the information on 1 website with that on another website for each task.
A substantial group of patients with cancer did not have the necessary skills to benefit from cancer-related internet searches. Future interventions are needed to support patients in the development of sufficient internet-searching skills, focusing particularly on information and evaluation skills.
Searching the internet for cancer-related information enables patients with cancer to satisfy their unmet information needs and empowers them to play a larger role in the management of their disease. Unmet information needs are one of the most frequently reported unmet supportive care needs of patients with cancer (6%-93%) during the treatment and posttreatment phases [
The percentage of patients with cancer who use the internet to search for cancer-related information is high. In 1 Swedish, 1 American, and 1 Dutch sample, 63%-75% of the participants used the internet to search for cancer-related information or general health information [
Patients with cancer have various reasons for searching cancer-related web-based information. They use the internet to develop questions to discuss with their physician, verify information given by their physician, or seek alternative treatments [
Searching the internet for cancer-related information is positively associated with patient-reported outcomes and socioeconomic characteristics of patients with cancer. Patients with cancer who search the internet for cancer-related information are more involved in medical decision-making [
Nevertheless, to benefit from cancer-related internet searching, cancer-related web-based information must be reliable, and patients with cancer need a sufficient level of skills to search, select, appraise, and apply web-based health information [
Thus far, research on internet searching skills has focused on general healthy populations [
Therefore, the primary goal of this study is to gain insight into the operational, navigational, information, and evaluation skills and problems of patients with cancer performing cancer-related search tasks using the internet.
A performance test was conducted to obtain in-depth insight into the operational, navigational, information, and evaluation skills and problems of patients with cancer using the internet to search for cancer-related information on the web. Three qualitative methods of data collection were used: (1) the think-aloud method [
The report of this study followed the recommendations of the Standards for Reporting Qualitative Research, consisting of 21 items that aimed to improve the transparency of all aspects of qualitative research by providing clear standards for reporting qualitative research (
The study protocol for this qualitative study is freely available at the Open Science Framework [
The participants were recruited during the first week of their 3-week stay at a rehabilitation clinic for oncological inpatient rehabilitation. Recruitment was conducted by the medical director (GE) of the clinic who approached the participants during the patient consultations. Patients were included if they had been diagnosed with any type of cancer and if they had sufficient oral and written proficiency in the German language. An appointment for the performance test was scheduled within the following week, and the participants received informed consent forms. Informed consent included information about the study goal, potential risks and benefits of the study, the voluntary nature of participation, and the type and duration of data storage. The participants were instructed to sign the informed consent before data collection. All appointments occurred at the rehabilitation clinic and were conducted by the same researcher (LLD). The sample size in this study was based on the concept of theoretical saturation [
Each appointment started with a short questionnaire to collect the following data: (1) the participants’ socioeconomic characteristics (age, gender, education, and marital status); (2) their medical characteristics (cancer type, time since cancer diagnosis, and self-perceived health status measured using the second-to-last items of the German version of the EORTC QLQ C-30 [European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-30] [
Performance tests were started when all the items were completed. The patients executed 8 cancer-related internet search tasks (
Imagine that you have noticed the following effects on your physical and mental well-being during your cancer treatment: listlessness, physical and mental exhaustion that does not improve even by sleep or rest. Search the internet for the symptom’s name.
Please search the internet for treatments or methods of treatment for chronic or persistent “fatigue” (this task always came after task 1).
Please search the internet for various providers who offer psycho-oncological counseling in the Hamburg area (postcode: 22529).
Formulate a disease-related question you have had in the past and show how you would approach this on the internet.
Please search the patient guidelines of the German Cancer Society for your specific type of cancer.
Search for the information sheet of the Cancer Information Service “Cancer on the internet: Surf safely.”
With the help of information from the internet, please name 5 possible side effects or symptoms of the specific cancer therapy (eg, chemotherapy and radiotherapy) that you received.
With the help of information from the internet, please name possible ways that you could change your diet to promote your well-being or reduce side effects.
The performance tests were recorded using
Each performance test was conducted using the same hardware (laptop, mouse, and keyboard) with identical settings. The laptop was connected to an active internet connection and was programmed with the 3 most popular web browsers (Internet Explorer, Mozilla Firefox, and Google Chrome). The participants were instructed to choose the web browser with which they had the most experience. All web browsers began with a blank page. To prevent the participants from being influenced by previous participants’ search activities, the web browser was reset after each participant by removing the web browser history and cookies using CCleaner version 5.44 (Avast Software). If the participants were unable to perform the task, help was offered by the researcher present. The participants received €15 (US $17.70) for participating in the study.
Statistical analyses were conducted using SPSS Statistics (version 25, IBM SPSS Inc). The participants’ sociodemographic characteristics, medical characteristics, and general and cancer-related internet usage were summarized descriptively.
Video and audio data, as well as the researcher’s real-time notes, were analyzed to (1) identify participants’ performance problems, (2) evaluate the participants’ performance, and (3) identify performance and strategic characteristics of task execution.
To identify participants’ performance problems, the researchers followed an inductive coding process [
The evaluation of the participants’ performance per task and the difficulty of the tasks were based on two variables: (1) could the participants complete their task
The execution and strategic characteristics of task execution were described by six variables: (1) the used web browser (Internet Explorer, Mozilla Firefox, or Google Chrome), (2) the starting point (eg, a specific website or a search engine), (3) the position of the opened website in the Google search listings from top to bottom (the position of the opened website in the search listings is an indicator of whether participants look beyond the first search results), (4) the number of words per search query (the use of a single search term was considered too unspecific; it is more important for the successful completion of a task to use task-related search terms than a large number of search terms), (5) the number of times a search query needed to be adjusted (a higher number of adjustments per task indicated a higher difficulty of the task), and (6) the name of the opened websites (Do the most often opened websites have a good content ranking and a commercial interest?) [
The influence of the participants’ education (>10 years of school education vs ≤10 years of school education), age (above vs below median), self-perceived internet skills (above vs below median), internet experience (above vs below median), and time since cancer diagnosis (above vs below median) on the participants’ average number of problems per task and percentage of successfully completed tasks were analyzed using 2-tailed
The study was conducted in accordance with the Code of Ethics of the Declaration of Helsinki and was surveyed by the Ethics Committee of the Medical Association (Hamburg, Germany). Written informed consent was obtained from all the participants before participation.
Slightly more women (12/22, 55%) than men participated in the study (
Medical and sociodemographic characteristics of the participants (N=22).
Participant characteristics | Values | ||
Age (years), mean (SD; range) | 56.8 (12; 25-81) | ||
Gender (female), n (%) | 12 (55) | ||
|
|||
|
Living alone | 6 (27) | |
|
Living with a partner | 16 (73) | |
|
|||
|
University degree | 6 (27) | |
|
13 years of school education | 3 (14) | |
|
10 years of school education | 9 (41) | |
|
9 years of school education | 4 (18) | |
|
|||
|
Breast cancer | 6 (27) | |
|
Colon cancer | 4 (18) | |
|
Prostate cancer | 3 (13) | |
|
Lung cancer | 2 (9) | |
|
Kidney cancer | 2 (9) | |
|
Other | 8 (32) | |
Time since cancer diagnosis (months), median (range) | 6 (1-207) | ||
Self-perceived health status, mean (SD) | 4.5 (1.0) |
aMultiple selection.
The participants’ mean internet experience was 15 years (
General and cancer-related internet usage of participants (N=22).
Participant characteristics | Values | |||
Internet experience (years), mean (SD; range) | 15.4 (7.7; 0-30) | |||
|
||||
|
No utilization | 1 (5) | ||
|
0-30 | 5 (23) | ||
|
30-60 | 7 (32) | ||
|
60-120 | 6 (27) | ||
|
>120 | 3 (15) | ||
|
||||
|
Obtaining general information about my cancer (ie, treatment information) | 14 (64) | ||
|
Communication with relatives or friends | 14 (64) | ||
|
Search for treatment options | 12 (55) | ||
|
Search for health care professionals | 8 (36) | ||
|
Verifying information received from health care professionals | 7 (32) | ||
|
Contact health care professionals (ie, oncologist) | 4 (18) | ||
|
Contact pharmacist | 3 (14) | ||
|
Contact other patients | 3 (14) | ||
|
Search for alternative treatment options | 3 (14) | ||
|
Search scientific data (ie, Google Scholar) | 3 (14) | ||
|
||||
|
Never | 2 (9) | ||
|
Rarely | 12 (54) | ||
|
More than once a month | 3 (14) | ||
|
More than once a week | 4 (18) | ||
|
Daily | 1 (5) | ||
|
||||
|
Value, range | 1-5 | ||
|
|
|||
|
|
Ergonomic skills | 3.5 (1.4) | |
|
|
Navigation skills | 3.2 (1.3) | |
|
|
Evaluating information reliability skills | 3.5 (0.7) | |
|
|
Determining information relevance skills | 3.5 (0.8) |
aMultiple selection.
Performance tests of the 21 participants were included in the analysis. The performance of participant 22 was excluded because the participant could not execute the tasks due to stress. On average, the participants executed 6.8 tasks. Participants 1 and 21 performed only 2 tasks. A total of 57% (12/21) of participants executed all 8 tasks. All data on task execution and performance problems are available in Figshare [
None of the participants used medical websites as a starting point. All search tasks were started using the Google search engine. On average, the participants successfully completed 59.9% (85/142) of all the tasks. Task F (Search the information sheet of the Cancer Information Service) had the highest rate of successful completions (14/18, 78%) and took participants, on average, the shortest time to execute (
Performance and strategic characteristics of the task execution (n=21).
Tasks | Aa (n=19) | Bb (n=17) | Cc (n=17) | Dd (n=17) | Ee (n=19) | Ff (n=18) | Gg (n=17) | Hh (n=18) | |||||||||
|
|||||||||||||||||
|
Completed independently | 12 (63) | 10 (59) | 10 (59) | 11 (65) | 6 (32) | 14 (78) | 12 (71) | 10 (55) | ||||||||
|
Completed with help | 1 (5) | 2 (12) | 0 (0) | 0 (0) | 0 (0) | 1 (6) | 0 (0) | 1 (6) | ||||||||
|
Not completed | 6 (32) | 5 (29) | 7 (41) | 6 (35) | 13 (68) | 3 (16) | 5 (29) | 7 (39) | ||||||||
|
|||||||||||||||||
|
Value, mean (SD; range) | 195 |
220 |
253 |
323 |
209 |
177 |
228 |
235 |
||||||||
|
|||||||||||||||||
|
Google Chrome | 16 (84) | 15 (87) | 13 (76) | 13 (76) | 16 (84) | 14 (78) | 14 (82) | 13 (72) | ||||||||
|
Mozilla Firefox | 3 (16) | 2 (13) | 4 (24) | 4 (24) | 3 (16) | 3 (16) | 3 (18) | 3 (17) | ||||||||
|
Internet Explorer | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 1 (6) | 0 (0) | 2 (11) | ||||||||
|
|||||||||||||||||
|
First search result | 11 (41) | 11 (42) | 6 (27) | 9 (36) | 15 (63) | 12 (57) | 7 (33) | 5 (19) | ||||||||
|
Second to fifth search result | 13 (48) | 14 (54) | 9 (41) | 8 (32) | 9 (37) | 9 (43) | 10 (48) | 15 (58) | ||||||||
|
Sixth to tenth search result | 3 (11) | 1 (4) | 7 (32) | 8 (32) | 0 (0) | 0 (0) | 4 (19) | 2 (8) | ||||||||
|
Second page of Google |
0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 4 (15) | ||||||||
|
|||||||||||||||||
|
Value, mean (SD) | 5.6 (4.7) | 3.3 (2.2) | 4.8 (3.5) | 3.4 (2.3) | 2.8 (0.6) | 3.8 (1.6) | 3.4 (1.2) | 3.9 (2.8) | ||||||||
|
Single search term, n (%) | 0 (0) | 2 (8) | 1 (5) | 2 (8) | 0 (0) | 2 (10) | 0 (0) | 0 (0) | ||||||||
|
|||||||||||||||||
|
Amount | 6 (22) | 2 (8) | 5 (23) | 6 (24) | 4 (17) | 6 (29) | 6 (29) | 4 (15) |
aIdentify the symptom fatigue.
bSearch treatment methods for cancer-related fatigue.
cSearch service providers who offer psycho-oncological counseling.
dRetrieve previously searched disease information.
eSearch the patient guideline for your cancer type.
fSearch the information sheet of the Cancer Information Service.
gSearch for five symptoms of the treatment you received.
hSearch for options to change your diet.
iPosition of opened websites in the Google search listings from top to bottom.
jIn some cases, participants opened more than one website. The position in the Google search listings of each website is listed here.
The participants’ performance problems were coded into four categories concerning internet searching skills (
Performance problems and number of participants experiencing those problems for each task.
Tasks | Aa |
Bb |
Cc |
Dd |
Ee |
Ff |
Gg |
Hh |
Totali |
|
|
||||||||||
|
Operating the keyboard | 5 (26) | 4 (24) | 3 (18) | 3 (18) | 5 (26) | 4 (22) | 4 (24) | 4 (22) | 7 (33) |
|
Controlling the mouse | 6 (32) | 2 (12) | 2 (12) | 2 (12) | 5 (26) | 5 (28) | 3 (18) | 6 (33) | 8 (38) |
|
Using the scroll bar | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 2 (11) | 1 (6) | 0 (0) | 0 (0) | 3 (14) |
|
Operating the web browser | 2 (11) | 1 (6) | 3 (18) | 3 (18) | 0 (0) | 1 (6) | 3 (18) | 3 (17) | 11 (48) |
|
Reading difficulties | 0 (0) | 1 (6) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 1 (6) | 0 (0) | 2 (10) |
|
Participants with >1 problem per task | 5 (26) | 0 (0) | 2 (12) | 3 (18) | 3 (16) | 3 (18) | 2 (12) | 4 (22) | 6 (29) |
|
||||||||||
|
Keeping orientation on a website | 0 (0) | 1 (6) | 0 (0) | 0 (0) | 1 (6) | 1 (6) | 1 (6) | 2 (11) | 5 (29) |
|
Using and understanding a PDF file | 1 (5) | 0 (0) | 0 (0) | 0 (0) | 3 (11) | 0 (0) | 0 (0) | 0 (0) | 3 (14) |
|
Using dropdown lists | 0 (0) | 1 (6) | 0 (0) | 0 (0) | 1 (6) | 0 (0) | 0 (0) | 0 (0) | 2 (10) |
|
Orientation in the Google search engine | 1 (5) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 1 (5) |
|
Participants with >1 problem per task | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 0 (0) | 1 (6) | 1 (6) | 1 (6) | 2 (10) |
|
||||||||||
|
Too broad search query | 0 (0) | 2 (12) | 1 (6) | 2 (12) | 0 (0) | 2 (12) | 0 (0) | 0 (0) | 5 (24) |
|
Nonspecific or nontask-related search query | 9 (47) | 5 (29) | 9 (53) | 1 (6) | 5 (26) | 4 (22) | 4 (24) | 2 (11) | 19 (90) |
|
Spelling and grammatical errors in search query | 2 (11) | 1 (6) | 5 (29) | 2 (12) | 5 (26) | 4 (22) | 2 (12) | 0 (0) | 11 (52) |
|
Adjusting the search query | 1 (5) | 0 (0) | 1 (6) | 0 (0) | 2 (11) | 0 (0) | 1 (6) | 1 (6) | 6 (29) |
|
Selection of task-related search results | 0 (0) | 0 (0) | 3 (18) | 2 (12) | 0 (0) | 4 (22) | 4 (24) | 3 (16) | 11 (52) |
|
Selecting physician instead of patient guidelines | N/Aj,k | N/Ak | N/Ak | N/Ak | 8 (42) | N/Ak | N/Ak | N/Ak | N/Ak |
|
Participants with >1 problem per task | 3 (16) | 0 (0) | 5 (29) | 0 (0) | 5 (26) | 2 (12) | 1 (6) | 1 (6) | 9 (43) |
|
||||||||||
|
Controlling the source of information | 19 (100) | 17 (100) | 17 (100) | 17 (100) | 18 (95) | 18 (100) | 17 (100) | 17 (94) | 21 (100) |
|
Searching in commercial websites | 1 (5) | 0 (0) | 2 (12) | 2 (12) | 0 (0) | 0 (0) | 3 (18) | 6 (33) | 8 (38) |
|
Verifying the information | 17 (89) | 15 (88) | 16 (94) | 16 (94) | N/Al | N/Al | 14 (82) | 17 (94) | 21 (100) |
|
Scanning a website for relevant information | 2 (11) | 1 (6) | 0 (0) | 0 (0) | 0 (0) | 1 (6) | 0 (0) | 5 (28) | 6 (29) |
|
Participants with >1 problem per task | 17 (89) | 15 (88) | 15 (88) | 16 (94) | 0 (0) | 1 (6) | 15 (88) | 18 (100) | 21 (100) |
|
||||||||||
|
Understanding the task | 4 (21) | 2 (12) | 0 (0) | 0 (0) | 1 (5) | 2 (12) | 3 (18) | 0 (0) | 7 (33) |
|
Forgetting the task | 0 (0) | 2 (12) | 1 (6) | 1 (6) | 3 (16) | 3 (17) | 0 (0) | 0 (0) | 9 (43) |
|
Keeping focus | 2 (11) | 1 (6) | 1 (6) | 3 (18) | 0 (0) | 1 (6) | 1 (6) | 0 (0) | 5 (24) |
aIdentify the symptom fatigue.
bSearch treatment methods for cancer-related fatigue.
cSearch service providers who offer psycho-oncological counseling.
dRetrieve previously searched disease information.
eSearch the patient guideline for your cancer type.
fSearch for the information sheet of the Cancer Information Service.
gSearch for five symptoms of the treatment you received.
hSearch for options to change your diet.
iNumber of participants who experienced this problem during the execution of at least one task.
jN/A: not applicable.
kThe problem was task-specific (task 5).
lVerifying the information was not applicable for this task because participants were instructed to search for a particular website.
A total of 62% (13/21) of participants had at least 1 problem using computer hardware (keyboard and mouse) or problems using basic web browser functions.
In total, 33% (7/21) of participants had problems using the keyboard, especially locating keys while typing search terms or searching for the
Overall, the operational problems were not task-specific. Most of the operating problems (94/122, 77%) were experienced by the same 6 participants. After completing several tasks, the present researcher observed that the participants became increasingly frustrated with the recurring operational problems.
In addition, 2 behaviors were observed but were not coded as operational problems. A total of 81% (17/21) of participants closed the web browser after every individual task and reopened the browser for the next task. Furthermore, 71% (15/21) of participants used a single tab for all searches.
A total of 33% (7/21) of participants experienced at least one problem with navigation and orientation in web browsers and on websites. Problems often occurred when the websites had complex structures, such as different graphical control elements (ie, dropdown lists or anchor links).
A total of 24% (5/21) of participants had problems maintaining their orientation on a website. They lost their orientation for various reasons. Twice, the participants felt confused by the browser’s starting page (eg, “This is not where I wanted to be” [Participant 8]). Two other times, the participants did not understand the function of the anchor link at the top of the website, which would have helped them jump to the relevant information in the text. A total of 14% (3/21) of participants could not distinguish an opened PDF file from a website. In total, 10% (2/21) of participants did not find relevant information on websites because they could not use the websites’ dropdown lists. One participant lost orientation due to the Google option
Notably, most of the orientation and navigation problems (94/122, 77%) were experienced by the same group of 6 participants who encountered most of the operational problems.
A total of 95% (20/21) of participants experienced at least 1 problem using search strategies. Most of the problems occurred in the first stage when the participants formulated the search terms.
A total of 24% (5/21) of participants used only single search terms that were too broad to successfully complete the tasks. In total, 90% (19/21) of participants used nontask-related search terms or nonspecific search terms. For example, Participant 5 used the search terms
The use of nontask-related, nonspecific, or grammatically incorrect search terms made the participants adjust their search terms or select a nontask-related webpage from Google search listings. A total of 29% (6/21) of participants experienced problems adjusting the search terms. For example, Participant 9 adjusted the original search query from
To complete task 5, the participants had to find the patient guidelines (PDF file) of their specific cancer type. A total of 38% (8/21) of participants selected clinical practice guidelines instead of the patient guidelines. Both types of guidelines can be found on the same website.
All participants experienced at least 1 problem while evaluating their relevance and reliability. None of the participants controlled the source or topicality of the information, except for Participant 19. Furthermore, none of the participants verified the information on 1 website with that on another website for each task. Most of the participants only opened a second website when they were not satisfied with the information on the first website. A total of 19% (4/21) participants made critical comments regarding the reliability of commercial websites; for example:
The first search result is an advertisement. Therefore, I will not consider that webpage.
Nevertheless, 38% (8/21) of participants selected and searched the providers’ websites with a commercial interest. In total, 4 of these participants even opened websites marked as ads by Google. A total of 29% (6/21) of participants did not scan the selected websites for relevant information to complete the task. They read the websites’ headings and then completed the task because they were convinced that the information they were looking for could be found on the website.
Notably, 71% (15/21) of participants made comments regarding the following: (1) the reliability of certain websites; for example:
I got offers. Here, from yelp.com. I have problems with opening this website because for me that is dubious information.
(2) the reliability of certain types of websites; for example:
What I would not read are patient forums. Where some laymen write what they did...I would rely on medical tips.
or (3) the internet as a source of cancer-related information; for example:
The internet in general is way too superficial. I read a book to gather information about that.
A total of 33% (7/21) of participants experienced problems understanding the tasks (“I don’t know what I am supposed to search” [Participant 14]; task 1), usually (10/12, 83%) resulting in nontask-related search queries. A total of 43% (9/21) of participants forgot the task during execution and had to reread it. A total of 24% (5/21) of participants were distracted by other nontask-related information. For example, Participant 3 became distracted by a brochure about
Participants 1 and 21 were not included in this analysis because they executed fewer than 3 tasks. The participants who were younger (mean 2.7, SD 1.0) and had higher self-perceived internet skills (mean 2.9, SD 1.1), on average, encountered significantly fewer (
This study examined the level of operational, navigational, information, and evaluation skills of a sample of patients with cancer performing 8 cancer-related search tasks using the internet. The results indicate that a substantial group of patients with cancer did not have the necessary operational, navigational, information, and evaluation skills to benefit from cancer-related internet searches. A total of 29% (6/21) of participants had major problems with the operation of the hardware, operation of the computer and web browser, and with navigation and orientation in web browsers and on websites. These participants produced three-fourths (94/122, 77%) of the operational and navigational problems. These problems caused great frustration among the participants and often resulted in tasks not being completed successfully. A total of 6 participants completed only 29% (12/42) of their tasks successfully. Although the operational and navigational skills of most participants (15/21, 71%) seemed to be sufficient for searching the internet, the information and especially the evaluation skills were much lower. Many participants struggled with formulating a task-related search query (19/21, 90%), selecting a task-related search result (11/21, 52%) of a provider without a commercial interest (8/21, 38%), and browsing the website to find the answer to the task (6/21, 29%). Strikingly, only 19% (4/21) of participants verified the information on 1 website with that on another website, and only 5% (1/21) of participant informed himself about the provider of the website. The remaining participants seemed to take no interest in the source or topicality of the information. These findings are alarming because previous research has shown that the quality of cancer-related web-based information varies widely [
Our results are consistent with those of previous studies that used performance tests to analyze the internet-searching skills of healthy participants [
The rate of search queries with single search terms was the main difference between the search strategies used in 2002 [
An exploratory analysis of our data indicated that younger age, higher self-perceived internet skills, more internet experience, and higher education were associated with encountering fewer performance problems and completing a higher percentage of tasks successfully. In addition, more time since diagnosis was associated with fewer performance problems and a slightly higher percentage of successfully completed tasks. The results of our exploratory analysis should be interpreted carefully because our sample size (n=19) was too small to make assumptions about the population of patients with cancer. However, previous research on internet-searching skills using performance tests confirmed that younger age, a higher level of education, more internet experience, and higher self-perceived internet skills are associated with more successful task completion [
This study had several limitations. First, the participants performed the tasks in an artificial research setting under experimental conditions. They may have felt more nervous than if they had been in a natural setting. For example, Participant 22 did not start the performance test because of stress. In addition, the participants may have felt time pressure to perform the tasks and may have focused less on evaluating the reliability or topicality of the websites. We tried to minimize the pressure to perform and time pressure by explicitly reminding participants to take their time and that there were no right or wrong answers. Second, some participants had to use unfamiliar hardware as they accessed the internet exclusively with their smartphones or tablets [
A substantial group of patients with cancer did not have the necessary skills to benefit from cancer-related internet searches. The problems included operating the hardware, navigation and orientation in web browsers and on websites, and in particular formulating a task-related search query and critically evaluating and verifying web-based content. Given the high number of participants with higher education and relatively high internet experience, the need for future interventions or programs to increase the internet-searching skills of patients with cancer may be underestimated in this study. Additional important performance problems may be identified in future studies that concentrate on older, low-educated patients with little internet experience.
Standards for Reporting Qualitative Research checklist.
Names of websites and the number of times they were opened.
Completed tasks and number of encountered problems per assignment related to education level, age, self-perceived internet skills, internet experience, and time since diagnosis.
European Organization for Research and Treatment of Cancer Quality of Life Questionnaire-30
This study was financed with own funds.
LLD was responsible for conceptualization, data curation, formal analysis, investigation, methodology, project administration, resources, validation, visualization, writing—original draft and review and editing. HS was responsible for the investigation, methodology, supervision, validation, and writing—review and editing. CB was responsible for conceptualization, investigation, methodology, supervision, validation, and writing—review and editing. GE was responsible for the recruitment of the sample, project administration, and writing—review and editing. All authors reviewed the final manuscript.
None declared.