To design a prototype of an electronic tool to address gaps in chemotherapy-related symptom management, we used a user-centered participatory design methodology [17]. A Web-based tool was preferred to allow use on any internet-enabled device to mitigate logistical and cost concerns. Potential end users, including patients, their caregivers, and health care providers (HCPs; medical oncologists, primary care physicians, and oncology nurses and pharmacists) were involved in all steps of prototype conceptualization and development. The study was approved by the University Health Network and Trillium Health Partners ethics boards. Informed consent was obtained from all participants; all participants received an honorarium for their time.

A convenience sampling approach was utilized whereby HCPs were invited to participate directly by a study team member. Patients and their caregivers were invited to participate by their treating medical oncologist or through an email sent out by the Cancer Care Ontario Patient and Family Advisory Council. Patient participants were required to have received chemotherapy for any cancer type with any intent within the previous 2 years. To ensure a wide sampling of views, each participant undertook only one study activity (ethnographic field study, focus group, or prototype testing). All participants completed a baseline questionnaire to assess their level of interaction with information technology. The patient questionnaire consisted of 11 demographic questions and 10 information technology questions; the HCP questionnaire consisted of 11 demographic questions and 11 information technology questions. For questions estimating hours of computer and internet usage per day, the lower range number was used for analysis.

To understand and gather insights into the context in which care is provided, we utilized the event-focused ethnographic field study methodology of Bloomberg et al [17] consisting of field observation and interviews and focus groups with potential end users. To understand the tasks, workflows, information requirements and usage patterns, communication and decision-making mechanisms, and use of supporting systems involved in toxicity management, field observations were undertaken. Field observation consisted of silent first-hand observation [18] of interactions between HCPs and patients during routine clinic visits in the ambulatory clinical setting with contemporaneous notes taken by 2 2 human factors specialists (ASY and LP). Following each field observation session, separate semistructured interviews were held with providers and patients to gather further information on gaps and facilitators. Interviews were guided by a script but left room to clarify any observed issues to gain deeper insights. Contemporaneous notes were taken, and interviews were audio recorded. Saturation of patient insights, defined as exhaustion of new feedback and themes [19], was not reached following the first round of ethnography; so additional patients were recruited and studied until saturation was reached, as confirmed by sampling of 2 additional patients. A total of 5 study team members confirmed saturation by consensus.

To encourage open discussion of views on issues and to minimize the risk of power imbalance, separate HCP and patient and caregiver focus groups were held and moderated by 2 human factors specialists (ASY and LP). Each participant attended a 1- to 2-hour focus group. Focus groups utilized open-ended questions and probes to generate discussion about gaps in the current health system related to symptom management as well as the content and functionality of a toxicity management tool. Focus groups were audio recorded and contemporaneous notes were taken.

Findings from demographics and preferences questionnaires were summarized using descriptive statistics. Qualitative data gathered during the ethnographic field studies and focus groups were thematically analyzed using the affinity diagramming method of Holtzblatt and Beyer [20] during ideation sessions.

Preparation for the affinity diagramming method consisted of a coding exercise on all the gathered data. Moreover, each of the 2 human factors specialists (ASY and LP) reviewed all data from all sessions generating codes denoting relevant keywords, phrases, and quotes from participant data. Each source of data (field observation and interviews and focus groups) was analyzed separately, which afforded methodological triangulation [21].

These codes were used in the ideation sessions, which were attended by 2 human factors study members (ASY and LP) and 3 HCP study members (RP, MK, and MP). On the basis of the principles of a Team Interpretation session [20], the 5 study members discussed each piece of coded data, sharing insights from the clinical and human factors perspectives. Through discussion of coded data, natural groupings began to form, which developed into themes that encapsulated each natural grouping. For example, one quote from an HCP, “Patients are overwhelmed. My guess is that 95% of the information is instantly forgotten.” was coded as Overwhelmed and grouped under the patient-specific theme of Anxiety and feeling overwhelmed.

Consensus on the themes was reached by all 5 members, and theme saturation was achieved when no new themes emerged for any presented piece of coded data [19]. This method allowed for effective identification of interrelated factors and existing issues related to remote management of chemotherapy-related toxicities; these themes informed the design of the prototype Web-based toxicity management tool.

On the basis of the different functional and information requirements gathered during the needs assessment, distinct interactive prototype interfaces were designed for both the patient and HCP roles. Usability testing was facilitated by human factors specialists and comprised hour-long cognitive walk-throughs, where participants performed realistic tasks while thinking aloud [22]. Depending on participant preference, usability testing was conducted either in a private room in the clinical area or at the human factors usability laboratories. The human factors specialists (ASY and LP) explained the purpose and objectives of the usability testing to participants and presented the low fidelity prototype (screen shots without any of the interactive functionality) with scenarios for the participant to complete using the prototype. Sessions were observed, videotaped, and audio taped, and contemporaneous notes were taken. Participants were debriefed at the conclusion of testing to ascertain if any further issues arose during testing.

Target recruitment for usability testing was 4 to 5 participants in the patient and caregiver and clinician groups, respectively, to identify up to 80% of usability problems as recommended in the literature [11]. The tool, comprising both a patient-facing and clinician-facing interface, was refined over 2 rounds of usability testing, resulting in a higher fidelity interactive prototype.

The data from the patient and caregiver and clinician groups were analyzed separately. The data from the video recordings from usability testing also underwent thematic analysis using affinity diagramming methodology [20] with 2 human factors specialists (ASY and LP) reaching consensus on the issues on both interfaces discovered through usability testing. The 2 human factors specialists jointly rank ordered the issues using a weighted decision matrix [23], assigning weights and ratings to various criteria including Ease of design change, Severity of usability issue, Benefit to patient, and Benefit to HCP, for each of the identified issues. The issues were iterated upon in the prototype starting from the highest overall rank until all issues were addressed.

Between December 2014 and November 2015, 49 patients, caregivers, and HCPs participated in the study. Overall, 8 patients and 8 HCPs participated in ethnographic field studies; 6 patients, 1 caregiver, and 4 HCPs participated in focus groups; and 11 patients and 11 HCPs participated in usability testing (Figure 1). Baseline characteristics of study participants are shown in Table 1; as only 1 caregiver participated, findings were combined with patient responses. Patient and caregiver participants were primarily female (58% [15/26]) with a median age of 55 years, had a diagnosis or cared for someone with a diagnosis of breast cancer (31% [8/26]), had at least a college education (81% [21/26]), and spoke English as their first language (92% [24/26]). HCPs were primarily female (80% [16/20]) with a median age of 50 years, had been in the medical profession on average for 20 years, spoke English as their first language (85% [17/20]), and had a hospital-based practice (90% [18/20]).

Computer and internet use were very common among HCPs, patients, and caregivers (Table 2) with the majority of participants using computers at work (80% [37/46]) and at home (93% [43/46]) and having access to internet at home (98% [45/46]). HCPs reported using electronic devices approximately twice as much as patients (6.5 vs 3.5 hours per day). Most participants reported being comfortable or very comfortable using computers, smartphones and tablets, internet, email, and instant messaging. Only 50% (23/46) of patients and HCPs felt comfortable using social media with 50% (13/26) of patients and 15% (3/20) of HCPs either not using or feeling not at all comfortable with social media platforms.

Thematic analysis of data collected from the ethnographic field studies and focus groups revealed themes of the recognition of gaps in the current health system, the existence of barriers to accessing care, and the need for timely support in decision making, which were common to both patients and HCPs (Figure 2). Additional themes were specific to either patients and caregivers or to HCPs. Patients and caregivers recognized that health systems often do not match their needs and that many psychological symptoms are not well addressed, especially feelings of being overwhelmed and anxious. HCPs identified the need for clear lines of accountability for any decisions and advice given to patients through a Web-based tool.

The design of an interactive prototype with separate patient and HCP interfaces, bridges, was informed by the findings of the user needs assessment. The patient interface includes functionality for toxicity reporting, self-management advice based on the reported toxicities, an appointment calendar, educational materials, and options for HCP communication (Figure 3). The HCP interface includes functionality for viewing patient visit history and voicemails left by the patient, trending of toxicity reporting data, and a standardized treatment guide for handling common chemotherapy-related toxicities.

Iterative rounds of usability testing were undertaken with 22 participants (11 patients and 11 HCPs) to further refine the prototype and evaluate end-user needs. During usability testing, patients noted that it is often left to them to self-organize their care including integration of complex scheduling information and medication administration (Textbox 1). Many patients desired access to an up-to-date Web-based calendar from the hospital or a calendar they could synchronize with their own device. Patients were very clear that they wanted access to information specific to their cancer type and treatment regimen rather than general information which may not apply to them. They had concerns about whether they would actually use the tool if they were feeling unwell and highlighted that the ability for a caregiver to fill it out for them would be useful. Patients also provided insight into the fact that care teams are frequently comprised of many members that change regularly.

Issues noted by HCPs included concerns of how a Web-based tool would be integrated into existing workflows and procedures, especially existing electronic medical records, to avoid entering the same information into multiple HIT systems. The context within which patient-reported toxicities occurred was felt to be very important, so the ability to add notes within the app was desired. Issues of confidentiality and privacy were highlighted as critical and requiring clarification, particularly with regard to patient consent around communication between HCPs and caregivers. The ability to prioritize patient symptoms and other issues in order of urgency was also felt to be important to ensure that the most serious issues were addressed first. The use of a treatment guide for symptom management was seen as a useful tool to ensure HCPs give consistent advice to patients.

In our study, patients, caregivers, and HCPs engaged in designing an interactive Web-based prototype of a chemotherapy-related toxicity management tool. Issues within the existing systems identified by participants included the need for HCP-supported decision making and self-management strategies for patients, access to credible and timely information, improved communication (both between patients and HCPs, and among HCPs), and integration of the tool within existing workflows to prevent redundancy and confidentiality issues. Understanding local context including the required features and functionalities and potential implementation issues particular to the local context is essential for long-term success. Our study identified a number of additional functionalities and implementation challenges that add to this growing body of literature such as concerns from providers regarding accountability within the tool.

Functionalities of existing tools for patients undergoing cancer treatment that have been developed and pilot tested range from symptom tracking alone [24-30], symptom tracking in conjunction with self-care support [31-36], and symptom tracking with active symptom monitoring [12,15,37-47]. Our findings confirm previous work, some of which was published before and some during the conduct of our study, such as high levels of patient satisfaction with remote symptom monitoring systems and the need for ease of use of the tool. When using any tool designed to improve the quality of patient care, the local context is a critical factor that must be addressed, as it will determine the success of implementation. Our study also identified additional functionalities and implementation challenges that add to this growing body of literature including the need to streamline etool integration into patient self-care and current clinical practice workflows before wide-scale adoption and accountability for provider actions within the etool necessitating clearly defined roles and responsibilities of users. This complements findings from a previous study by Mooney et al [48] utilizing a telephone-based remote monitoring system for cancer patients that highlighted the need to explicitly address who is going to address alerts, as assuming providers will add these duties on to their existing responsibilities does not work.

The evidence for positive impact of etools for symptom monitoring during cancer treatment is growing. A recent study by Denis et al [46] of Web-based follow-up of self-reported symptoms in lung cancer patients following initial treatment reported improved overall survival. Basch et al [15,16] found that remote monitoring of patient symptoms during chemotherapy resulted in significantly improved quality of life, fewer ED visits and hospitalizations, and was associated with prolonged survival. Both interventions included health team alerting functionality for severe symptoms. Likewise, Chumbler et al [49] found a significant reduction in clinic visits and chemotherapy-related hospitalizations using a Cancer Care Coordination Home Telehealth approach.

Our findings need to be evaluated in the context of study limitations. Our participants were from 2 large urban centers in southern Ontario and, thus, may not represent end-user needs from other jurisdictions. Selection bias may be present as HCPs who volunteered to participate may have been inherently more interested in using HIT solutions. Likewise, patients who contacted the study coordinator directly may represent a more motivated population than a random sample and may not be representative of the views of the overall population. HCP and patient participants were mostly female (HCPs: 16 vs 3; patients/caregivers: 15 vs 10), so findings may not adequately represent the views of their male counterparts. Our patient and caregiver participants had a median age of 55 and reported having at least a college education (81% [21/26]), English as their first language (92% [24/26]), and comfort using the internet and technology. As such, additional work is needed to better understand the needs of end users who are older, non-English speaking, less educated, or less familiar with technology. The design of our interactive prototype is a first step in building an effective electronic tool for patient care. Additional studies are needed to explore whether the tool impacts patient outcomes; although, the recently published trials from the United States [15] and France [46] provide important information regarding the benefits of such systems for patients in controlled settings. Data on wide-scale adoption of etools in this setting are not yet available.

The method of affinity diagramming aims to bring together issues and insights from various stakeholders, from which overarching themes emerge. Although the data were collected using a number of approaches to understand the complexities and subtleties of toxicity management within the Canadian context, it could be argued that the resulting artifact from affinity diagraming affords only a thin description of toxicity management, that is, a limitation of the method is that the resulting affinity diagram is only a brief summary of the themes that emerged, lacking rich context. However, it should be noted that the human factors study members who conducted the ethnographic field study and participated in ideation sessions were also those who designed the prototype. Thus, although the themes from affinity diagramming informed the design of the prototypes, the human factors study members were also able to draw from the contextual data that they experienced firsthand. Although the number of participants required for usability testing has been well-established [11], there is little consensus on the number of participants required for interviews and focus groups [21,50,51]. Given this limitation, the authors instead strived for saturation of the collected data and resulting themes [21] and triangulation using multiple sources of data (field observation, interviews, and focus groups) [51] to arrive at a comprehensive understanding of toxicity management from various stakeholders.

We have shown that using a human factors design approach for a Web-based application to support management of chemotherapy-related toxicities has the potential to address gaps in cancer care. As these gaps were identified within the local context of care, the design, iterated upon through prototyping and usability testing, seeks to address these needs directly. Our study highlighted that operationalizing a Web-based tool has significant system implications, including assigning responsibility for monitoring the tool to appropriate HCPs and the need to embed the tool into existing workflows and systems. Integration of new etools into self-care by patients and practice is a key issue that needs to be addressed for wide-scale adoption.