We conducted a 15-month multicentered, nonblinded, pragmatic pilot cluster-randomized controlled trial (cRCT), called the My Team of Care study, allocating participants to receive access to Loop as the intervention arm or to usual care as the control arm. The unit of randomization was at the level of the physician, and the unit of analysis was at the level of the individual patient.

See Multimedia Appendix 1 for the CONSORT EHEALTH checklist [29].

The study took place at the Temmy Latner Center for Palliative Care at Mount Sinai Hospital and Princess Margaret Cancer Center in Toronto, Ontario, Canada, from January 2015 to April 2016. The Temmy Latner Center is the largest home-based palliative care program in Canada, consisting of 23 palliative care physicians. The Princess Margaret Cancer Center is a University of Toronto‒affiliated research hospital with 46 medical oncologists and 42 radiation oncologists. At both study sites, patients generally access their physician through visits and telephone messages; some health care providers are contactable via email.

Participants consisted of eligible patients plus their principal cancer physician (oncologist or palliative care physician), and if interested, their family caregiver (informal, unpaid). For the intervention arm, additional health care providers as identified by the patient were also invited to participate as members of the circle of care to use Loop.

Eligible patients were aged 18 or older; had stage IV cancer or stage III cancer with poor prognosis as determined by their oncologist (a survival prognosis of ˃3 months but ˂2 years); Eastern Cooperative Oncology Group (ECOG) performance status score of 0, 1, or 2, as assessed by their oncologist or palliative care physician at time of enrollment; English literacy and language competency to provide informed consent and complete questionnaires; and patients or caregivers had access to a computer and Internet. Exclusion criteria were currently receiving or a candidate for hormone therapy for breast or prostate cancer (given the impact on prognosis); impaired mental status assessed with the Bedside Confusion Scale [30] (score of ≥2 suggesting cognitive impairment); or participation in another study precluding participation in this study.

Participants randomly allocated to the intervention arm received access to Loop. Loop is a secure online communication tool for team-based clinical collaboration that enables patients and caregivers to communicate asynchronously with multiple members of the health care team involved in providing their direct care (ie, not individuals hired for the purpose of research), as well as for health care providers including physicians, nurses, and allied health professionals to communicate with each other.

The development of Loop followed a user-centered design approach [31] with substantial end-user and stakeholder involvement (including caregivers, health care providers from several specialties, and patients with different conditions). As described by Kurahashi et al [26], this process included initial needs assessments, ethnographic observational studies, and affinity diagramming leading to the development of a prototype. This was followed by simulation activities, usability testing in laboratory and real-world settings (ie, home, clinics, hospitals, offices), and piloting with patients and clinical teams.

Loop was developed with an intuitive interface to allow for use ideally without prior training. A patient profile and space are created that can be viewed by the patient, health care providers, and caregivers on a computer or mobile phone after logging in with an email address and password (Figures 1 and 2). Each patient’s Loop is a secure space partitioned from other Loops and can be joined only if involved in the patient’s care and authenticated by a study administrator. Health care providers may be a part of multiple patients’ Loops, but patients cannot access the Loops of other patients. On the main page, individuals can write and post text-based messages. All messages posted by any member of a patient Loop can be read and responded to by members of that given Loop. All entries remain on the patient space, allowing for previous posts to be viewed. The messages are threaded in conversations and can be searched using various filters. In addition to posting messages, users may label posts with user-defined “tags” and an “Attention To” feature that specifies individuals to be alerted to a post by a generic email. No updates to the system were made during the trial.

In order to ensure that there would be at least one health care provider on each team, medical/radiation oncologists or palliative care physicians were recruited first (“initiating physician”), randomized, and patients from their practices were then approached in clinic prior to appointments or over the phone. Physicians at the two study sites were notified of the study through announcements at educational rounds, and physicians who expressed interest in participating were followed up with directly.

All participants provided consent and were asked to complete a baseline demographics form and an internally developed 11-item survey on participants’ access, comfort, and usage of computers and Internet [32]. Initiating physicians and patients in the intervention arm were then invited to register on Loop. Registration involved completion of a form requesting participant name, address, and, in the case of health care providers, professional license number. Once registered on the tool, an intervention patient was considered active in the study. Through the tool, patients could invite family caregivers and additional health care providers to join their Loop. Study administrators contacted, explained the study, obtained consent from all additional members of a patient Loop prior to registration, and posted an introductory message welcoming participants. Study administrators were part of patient Loops only for the purpose of providing assistance during registration and with using the tool, as requested. When a patient was no longer part of the study, Loops remained open for 2 weeks to allow for message exchange records to be exported and saved.

Use of the tool and the type of communication that could occur on the tool were not prescribed. The intervention protocol did not specify intent to replace existing care practices or methods of communication; the intervention was additive. As Loop was not meant to be used for urgent communication, this was reinforced during the consent process and in the Loop terms of use.

Recruitment of initiating physicians and their patients was conducted similarly in the control arm.

Usage of the intervention was evaluated from message exchange transcripts and audit data from Loop. Data included time to registration on Loop from consent date, number of participants who registered on Loop, number of messages exchanged, number of times additional features (Attention To, Tagging) were used, and number of views and posts by participants.

We conducted a content analysis of Loop messages to summarize the content of messages on patient Loops. Two coders independently reviewed messages exchanged in each patient Loop and assigned categories thematically that emerged from the data (see Multimedia Appendix 2 for coding framework and definitions) [33]. Categories were assigned to messages and any responses or follow-up posts. Categories were assigned only once per Loop and not quantified. If multiple categories were perceived in a single post, then each was included once as a category identified in that particular Loop. Messages posted by administrators to welcome team members were excluded.

The primary feasibility outcomes were participant recruitment rate and implementation fidelity defined as the proportion of participants who were randomized, completed the baseline demographics and computer usage questionnaire, and if in the intervention arm, registered on Loop, with ≥70% completion indicating feasibility success. This threshold was selected as an adequate threshold to justify further study and has been suggested previously in the literature [34].

Secondary outcomes were measured using standardized instruments to assess the impact of Loop. Mean difference over the course of the study (from baseline to months 1, 2, and 3) for each instrument was calculated. Patient-reported continuity and coordination of care was measured with the 8-item Picker Ambulatory Cancer Care Survey (Picker) Continuity and Coordination subscale questionnaire. The Picker scale is scored by summing absolute positive responses, divided by the total number of responses (scores range from 0-100, with higher scores being better), and a minimal clinically important difference of 10 points has been previously found to be significant [35,36]. The Palliative care Outcomes Scale (POS) was used to assess patient-reported quality of care and well-being. The POS is a 12-item self-administered questionnaire (total scores range from 0-40, with higher scores being worse); a difference of one point on each item is considered clinically meaningful [37-39]. The Edmonton Symptom Assessment Scale (ESAS) is a 9-item, patient-reported questionnaire of symptom intensity (pain, tiredness, nausea, depression, anxiety, drowsiness, appetite, well-being, shortness of breath) with each item rated from 0 (worst) to 10 (best). Individual symptoms are summed for the Total Symptom Distress score (ranging from 0-90, with higher scores indicating worse symptom distress) [40]. Health care utilization was measured as number of visits to the emergency department and number of hospitalizations and was self-reported at each monthly assessment using the Ambulatory and Home Care Record [41].

Data were collected monthly for 3 months from baseline (four time points). Questionnaires were distributed electronically using online surveys emailed to study participants via Research Electronic Data Capture version 6.16.7 [42], a data management system hosted at the Applied Health Research Center of St. Michael’s Hospital, Toronto, Ontario. Patients who did not respond within 1 week were followed up via a reminder email or telephone call and were considered lost to follow-up if not reachable after four contacts. We piloted outcome assessments and survey administration prior to the study. We also collected qualitative interview data at assessments, which will be reported separately.

This study was designed as a cRCT with initiating physicians recruited first and randomized in order to minimize contamination between study arms. Participating patients were allocated to the study arm to which their initiating physician had been randomized. Randomization was done by a statistical team independent of the study using a computer-generated randomization sequence consisting of permuted blocks of varying size, and assigned initiating physicians in a 1:1 ratio to the intervention and control arms. It was not possible to blind patients completely to study arm, but control patients provided consent without being informed of the existence of another arm. This was done to minimize bias of control patients basing their decision to participate on study arm assignment. Control patients were informed that they were taking part in a study on patient-physician communication to improve health care delivery and care management. Investigators and initiating physicians were aware of study arm assignment.

A formal sample size calculation was not computed, as this was a pilot study with primary feasibility outcomes. We set a target sample size of 20-25 patients per study arm, which has been previously justified as sufficient for pilot evaluations [34,43]

The primary analysis was intention-to-treat with available cases. We did not make adjustments for missing data but secondarily report comparison of data for complete cases (participants who completed outcome assessments at all time points). Descriptive statistics were used to describe each study arm. Analysis compared mean change scores and unadjusted differences in mean change scores on the preliminary effectiveness outcomes between study arms. Statistical tests of difference were not conducted since the study was not powered to undertake hypothesis testing.

All participants provided written, informed consent to participate. Research Ethics Boards of the University Health Network, Mount Sinai Hospital, University of Toronto, and the Community Care Access Centers of Toronto, Ontario, approved the study.

In the intervention arm, 83% (20/24) of patients who consented, registered on Loop (Table 3). In terms of health care provider load, the mean number of patient Loops per initiating physician was 1.6 (range 0-7). Registration on Loop required that the baseline questionnaire be completed beforehand. The time from consent to registration on Loop varied considerably, with the mean time to registration being 39 days. Some patients experienced disease worsening between consent and registration, and one patient delayed taking the step to register for 156 days due to personal circumstances. Over the study period, the majority (85%, 17/20) of Loops had message exchanges, with 45% (13/20) having more than six messages exchanged. During the study, there were 358 logins by all participants: 43 on the mobile version and 315 on the desktop version. Patients viewed their Loops more often relative to their number of posts (a difference of 14.3) compared to initiating physicians viewing and posting to their patients’ Loops (a difference of 3.9).

Content analysis of messages revealed that of Loops with messages exchanged, messages regarding medical care management, reporting of symptoms, and appointment coordination predominated (these categories were identified in 50%, 45%, and 45% of the 20 patient Loops, respectively), while only 10% of the Loops had messages that were prescription-related queries (see Figure 4). No urgent messages were exchanged during the study.

For the primary outcomes, the mean number of patients recruited per initiating physician was similar between study arms (in intervention arm 2.4 [range 0-7] in intervention vs control arm 2.7 [range 0-7]) (Table 4). With respect to implementation fidelity, 88% (21/24) of control patients who consented completed the baseline questionnaire and 75% (18/24) of intervention patients who consented completed the baseline questionnaire, along with registering on the tool. Regarding patient retention, in the intervention arm, 3 patients withdrew due to declining health, 1 patient withdrew because they were no longer interested, and 1 patient died. In the control arm, 1 patient withdrew due to declining health, and 5 patients died. Instrument and item completion were approximately proportional to patient retention. Of patients completing the baseline outcome assessment, 52% (11/21) in the intervention arm and 52% (11/21) in the control arm completed Month 3 outcome assessments.

Results described are based on available cases. Mean change scores and unadjusted difference in change scores between study arms for preliminary effectiveness outcomes are presented in Table 5. At Month 3, there was an increase in Picker scale scores (in intervention arm +10.2 [SD 31.5] vs control arm -1.1 [SD 30.3]), a negligible change in POS (in intervention arm +0.8 [SD 4.4] vs control arm +0.5 [SD 5.4]), and an increase in ESAS Total Symptom Distress score (in intervention arm +2.3 [SD 10.7] vs control arm +3.4 [SD 8.7]). The number of patients with emergency room visits self-reported at baseline was 3 in the intervention arm and 1 in the control arm. At the third month, no visits were reported in the intervention arm and 3 were reported in the control arm. Similar numbers were observed for number of patients with hospitalizations. On complete case analysis, the Picker scale showed a between-arm difference of +18.5 [47.4] in favor of the intervention arm (Table 6).

In this pilot cRCT evaluating an online communication tool for clinical collaboration, trial feasibility conditions and implementation goals were met. The study was not powered to observe changes in outcomes between study arms, but we did observe an increase in continuity of care scores in the intervention arm at last follow-up, which was maintained on complete case analysis. Regarding the assembly of teams, though each patient identified at least one additional health care provider, only 65% of patient Loops had an additional health care provider register on the tool. Loop was primarily used for medical care management, symptom-related discussions, and appointment coordination.

As a population with complex care needs [45], the advanced cancer population served as an exemplar patient population in which to evaluate Loop but also proved challenging from a participation standpoint. Although the proportion of eligible patients who consented in this study was slightly higher than two previous studies conducted at the same institution with the same population (38% here vs 10%) [36,46], a number of patients withdrew due to ill health or died over the course of the study. This was expected given the uncertainty in prognosis in this population. Instrument completion rates reflected patient retention rates, indicating that questionnaire administration was feasible despite the nature of the patient population.

Loop was designed to connect patients and caregivers to their team of health care providers in a virtual space where communication might be facilitated outside of appointments and across care settings [26]. While we did not assess differences in measures of effect for statistical significance, preliminary Picker scale results appear to support potential for this tool to improve continuity of care in future studies of adequate statistical power. Contextualized with the content of messages, the findings of this study may suggest that there were important needs that could be dealt with between appointments by using the tool, contributing to increased perceptions of continuity of care.

The care of patients with complex needs requires a redefining of the relationship between health care providers and patients to a team-based model of care that engages the patient [47]. These patients often have interdependent issues and thus require collaborative approaches to care (negotiated decision making between individuals in a synergistic manner around shared goals) over coordination between providers (alignment of functioning among independent individuals to address common needs) [48]. Here, the greater number of posts by patients over health care providers and the patient-driven content of messages (eg, Updates, Appointment Coordination) are suggestive that in this study, coordination tasks were addressed to some extent but collaboration did not occur. Given these results, we recognize that Loop in isolation did not produce collaboration, and further consideration into building relationships among these teams is required [49].

We further found that assembling the team was difficult in this study, with few health care providers from outside academic practices, who were identified by the patient, agreeing to join. Other studies have found that barriers to health care provider participation and uptake of studies of eHealth tools include lack of provider compensation and perceived worry about the burden of patient overuse [50]. Although this increased burden has not been observed thus far [51], better strategies to improve integration into clinical workflow need to be examined, especially for physicians with large patient rosters. Implementation of incentive schemes, akin to what has been done in the province of Ontario, Canada, for electronic consultations [52] may also improve uptake of eHealth tools, like Loop, into practice.

In this study, Loop was intentionally provided to teams without training. We observed that participants were able to understand and use the core functionality of Loop, that is, to post and read messages. We further observed that patients viewed their Loop more often than they posted compared to health care providers, who posted nearly as often as they viewed a Loop. This could be interpreted as showing that patients were more proactive tool users, while health care providers are more likely to wait for notifications before logging in.

While many tools for patient-physician communication exist (frequently as secure email or part of patient portals) [19,53], few have considered the potential value of team-based communication, which is crucial for complex care scenarios or situations requiring ongoing care. At least four studies have evaluated variations of tools connecting patients and caregivers with multiple health care providers. Gulmans et al [25] found that patient groups who used their tool more often tended to have a larger care network (number of professionals registered per patient). Furthermore, Ralston et al [23], in evaluating secure messaging as part of a portal, found that messaging increased proportionally with patient morbidity, which reinforces the suggestion that messaging is of more value in complex care. While our study was too small to examine such associations, these findings support the increased value that eHealth communication tools may have as complexity of team and illness increase.

In a study by Hsiao et al [22], as has been noted elsewhere [54,55], participants felt that text-based communication may diminish the therapeutic relationship gained from in-person visits or unstructured voice-based contact (such as telephone). This suggests that such forms of communication should supplement, but not replace, appointments or calls.

The ZWIP tool, by Robben et al [24], allows for patient-provider and between-provider communication. Evaluation in frail elderly patients found that use by both patients and providers depended on provider use of the tool. Health care providers considered implementation strategies (such as training to use the tool) “very necessary” to make the most use of ZWIP. This finding may reflect the need for guided implementation to facilitate integration into clinical workflow and to improve the use of Loop.

The results of this study should be interpreted within the context of the study’s strengths and limitations. As a pilot study, we aimed for a sample size that was adequate to determine feasibility of implementation. However, this sample size limited the ability to test the effectiveness of the intervention. All health care providers described themselves as very comfortable with computers and worked in fully computerized practices; however, this may not be true of every medical practice, limiting study generalizability. Similarly, the complexity of clinical cases and nature of physician practice may be different elsewhere. While patients with advanced cancer are a prototypical population of patients with complex care needs and have involvement of multiple providers, similar results may not be reflected in other populations. The cRCT design, involving recruitment and randomization of initiating physicians (clusters) sequentially, and their patients prospectively, may have led to selection bias because of differential recruitment rates by provider and differences in their clinical subspecialty. We also observed that more patients died in the control arm than in the intervention arm possibly indicating unmeasured confounding. As use of the tool was voluntary, there is also a risk of confounding by indication, with patients who have more issues needing to use the tool more often, or functionally limited patients using the tool less often.

The results of this study will inform the next phase of research, which aims to (1) understand the conditions that affect tool adoption and assembly of teams, (2) understand the relationship between use and outcomes such as continuity and quality of care, and (3) examine the contexts and target populations where the benefits of a tool like Loop may be best realized and where the effort to assemble the care team is justified. We anticipate that by making adjustments to the implementation approach [56] through use of site champions, consideration of strategies to foster team collaboration as a co-intervention with Loop, and considering an initiating physician (the “index provider”) from those additional provider disciplines that were less represented, we may address issues with team assembly and optimize collaboration on Loop. We also expect that with a longer duration of follow-up, as could be done in other patient populations, participants’ comfort with using Loop may improve, and the content of communication may become more oriented towards care planning and decision making, over coordination.

In this study, we found that it was feasible to implement Loop in clinical practice and that the tool may have the potential to improve continuity of care. We observed that Loop messages reflected message categories of medical care management, symptom reporting, and appointment coordination, among others. Usage of the tool suggests that some coordination tasks were improved but further strategies to build collaboration among team members may be needed. As an ongoing goal of eHealth development, the integration of the dynamic components of care (communication and collaboration) with the static repositories of medical records would enable a more seamless provision of health care. However challenging this may be in the current environment of multiple electronic health records across organizations, studying collaborative tools like Loop advances this goal.