Reducing Symptom Distress in Patients With Advanced Cancer Using an e-Alert System for Caregivers: Pooled Analysis of Two Randomized Clinical Trials

Background Symptom distress in patients toward the end of life can change rapidly. Family caregivers have the potential to help patients manage those symptoms, as well as their own stress, if they are equipped with the proper resources. Electronic health (eHealth) systems may be able to provide those resources. Very sick patients may not be able to use such systems themselves to report their symptoms but family caregivers could. Objective The aim of this paper was to assess the effects on cancer patient symptom distress of an eHealth system that alerts clinicians to significant changes in the patient’s symptoms, as reported by a family caregiver. Methods A pooled analysis from two randomized clinical trials (NCT00214162 and NCT00365963) compared outcomes at 12 months for two unblinded groups: a control group (Comprehensive Health Enhancement Support System [CHESS]-Only) that gave caregivers access to CHESS, an online support system, and an experimental group (CHESS+CR [Clinician Report]), which also had CHESS but with a CR that automatically alerted clinicians if symptoms exceeded a predetermined threshold of severity. Participants were dyads (n=235) of patients with advanced lung, breast, or prostate cancer and their respective family caregivers from 5 oncology clinics in the United States of America. The proportion of improved patient threshold symptoms was compared between groups using area-under-the-curve analysis and binomial proportion tests. The proportion of threshold symptoms out of all reported symptoms was also examined. Results When severe caregiver-reported symptoms were shared with clinicians, the symptoms were more likely to be subsequently reported as improved than when the symptoms were not shared with clinicians (P<.001). Fewer symptom reports were completed in the group of caregivers whose reports went to clinicians than in the CHESS-Only group (P<.001), perhaps because caregivers, knowing their reports might be sent to a doctor, feared they might be bothering the clinician. Conclusions This study suggests that an eHealth system designed for caregivers that alerts clinicians to worrisome changes in patient health status may lead to reduced patient distress. Trial Registration Clinicaltrials.gov NCT00214162; https://clinicaltrials.gov/ct2/show/NCT00214162 (Archived by WebCite at http://www.webcitation.org/6nmgdGfuD) and Clinicaltrials.gov NCT00365963; https://clinicaltrials.gov/ct2/show/NCT00365963 (Archived by WebCite at http://www.webcitation.org/6nmh0U8VP)


1a-i) Identify the mode of delivery in the title
Identify the mode of delivery. Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title. Avoid ambiguous terms like "online", "virtual", "interactive". Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g. email), use "computer-based" or "electronic" only if ooine products are used. Use "virtual" only in the context of "virtual reality" (3-D worlds). Use "online" only in the context of "online support groups". Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms. 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1a-i? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study The title says the the target group was adults with advanced cancer.
1b) ABSTRACT: Structured summary of trial design, methods, results, and conclusions NPT extension: Description of experimental treatment, comparator, care providers, centers, and blinding status.

1b-i) Key features/functionalities/components of the intervention and comparator in the METHODS section of the ABSTRACT
Mention key features/functionalities/components of the intervention and comparator in the abstract. If possible, also mention theories and principles used for designing the site. Keep in mind the needs of systematic reviewers and indexers by including important synonyms. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1b-i? * Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "a control group (Comprehensive Health Enhancement Support System [CHESS]-Only) that gave caregivers access to CHESS, an online support system and an experimental group (CHESS+CR [Clinician Report]), which also had CHESS but with a CR that automatically alerted clinicians if symptoms exceeded a predetermined threshold of severity.

1b-ii) Level of human involvement in the METHODS section of the ABSTRACT
Clarify the level of human involvement in the abstract, e.g., use phrases like "fully automated" vs. "therapist/nurse/care provider/physician-assisted" (mention number and expertise of providers involved, if any). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it)

2a-i) Problem and the type of system/solution
Describe the problem and the type of system/solution that is object of the study: intended as stand-alone intervention vs. incorporated in broader health care program? Intended for a particular patient population? Goals of the intervention, e.g., being more cost-effective to other interventions, replace or complement other solutions? (Note: Details about the intervention are provided in "Methods" under 5) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 2a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Managing Patient Symptom Distress As advanced cancer treatments enable patients to live longer, managing patient symptoms becomes even more important for patients, informal (family or friend) caregivers, and clinicians [1][2][3][4][5]. In some cases, the side effects of advanced treatment (eg, pain and cognitive limitations) can create problems that challenge the fabric of the family [6,7] and sometimes even lead to conflict [6,7] between the family and the clinical team [8]. Some of the problem, from a patient and caregiver perspective, revolves around when and how to get the attention of the clinical team without "bothering" them [9]. . . . Research has shown the importance of timely alerts to providers regarding disconcerting changes in patient conditions. Recent developments in electronic symptom collection systems [12,13] offer promise for more timely and accurate information, greater patient acceptance, and reduced cost compared with paper-based systems [14]. Studies of such systems have shown moderate to significant improvement of patient symptoms and quality of life [15][16][17][18][19], and even survival [20]. A key issue is when and how to reach clinicians effectively, given how busy they are.
The CHESS caregiver system studied in this analysis contained a symptom-reporting system, the Clinician Report (CR), which delivered to the clinical team information about worrisome changes in symptoms collected from informal caregivers of advanced-stage cancer patients [24,30,31]. . . .Specifically, the CR contacted the clinical team whenever a threshold symptom was reported-that is, when a caregiver rated at least one of 10 patient symptoms at ≥7 on a 0 to 10 severity scale. The alert was intended to quickly bring clinician attention to severe Specifically, the study addresses this question: Does the CHESS system with the CR reduce symptom distress in patients more than CHESS without the CR?"

3b-i) Bug Vxes, Downtimes, Content Changes
Bug Uxes, Downtimes, Content Changes: ehealth systems are often dynamic systems. A description of changes to methods therefore also includes important changes made on the intervention or comparator during the trial (e.g., major bug Uxes or changes in the functionality or content) (5-iii) and other "unexpected events" that may have inpuenced study design such as staff changes, system failures/downtimes, etc. [2].

Does your paper address subitem 3b-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study We did not have important changes to the intervention during the trials.

4a-i) Computer / Internet literacy
Computer / Internet literacy is often an implicit "de facto" eligibility criterion -this should be explicitly clariUed.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Caregiver Internet comfort" was assessed at baseline and is reported in Table 1.

4a-ii) Open vs. closed, web-based vs. face-to-face assessments:
Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. ooine), e.g., from an open access website or from a clinic, and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasianonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email conUrmation, phone calls) were used to detect/prevent these.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4a-ii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study This is a pooled analysis from two randomized trials. Details about recruitment are provided in previous publications, to which we provide references.

4a-iii) Information giving during recruitment
Information given during recruitment. Specify how participants were briefed for recruitment and in the https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4a-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study This is a pooled analysis from two randomized trials. Details about recruitment are provided in previous publications, to which we provide references.
4b) Settings and locations where the data were collected Does your paper address CONSORT subitem 4b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Recruitment sites were 5 cancer centers in the Northeastern, Midwestern, and Southwestern United States."

4b-i) Report if outcomes were (self-)assessed through online questionnaires
Clearly report if outcomes were (self-)assessed through online questionnaires (as common in web-based trials) or otherwise.

Does your paper address subitem 4b-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Sites used their own letterhead as appropriate. Participants who were recruited to the study were already being treated at the institutions involved.
5) The interventions for each group with sutcient details to allow replication, including how and when they were actually administered

5-i) Mention names, credential, a;liations of the developers, sponsors, and owners
Mention names, credential, atliations of the developers, sponsors, and owners [6] (if authors/evaluators are owners or developer of the software, this needs to be declared in a "Conpict of interest" section or mentioned elsewhere in the manuscript).
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "The CHESS System With Reports to Clinicians The Comprehensive Health Enhancement Support System (CHESS) refers to extensively tested information and communication technologies for coping with cancer and other serious illnesses."

5-ii) Describe the history/development process
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results.

Does your paper address subitem 5-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study

"The CHESS System With Reports to Clinicians
The CHESS "Coping with Cancer" website [29] was designed instead for caregivers to (1) provide them with well-organized cancer, caregiving, and bereavement information, (2) serve as a channel to communicate with and receive support from other caregivers, experts, clinicians, and their social networks, (3) act as a coach by gathering information from caregivers and providing feedback based on algorithms (decision rules), and (4) provide tools (eg, a program to organize support from family and friends) to improve the caregiving experience. The relationship between theory and CHESS was previously reported [30]. The CHESS caregiver system studied in this analysis contained a symptom-reporting system, the Clinician Report (CR), which delivered to the clinical team information about worrisome changes in symptoms collected from informal caregivers of advanced-stage cancer patients [24,30,31]. Specifically, the CR contacted the clinical team whenever a threshold symptom was reported-that is, when a caregiver rated at least one of 10 patient symptoms at ≥7 on a 0 to 10 severity scale. The alert was intended to quickly bring clinician attention to severe symptoms, potentially leading to timelier symptom management."

5-iii) Revisions and updating
Revisions and updating. Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial. Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b).

Does your paper address subitem 5-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study The intervention was not changed during the trials.

5-iv) Quality assurance methods
Provide information on quality assurance methods to ensure accuracy and quality of information provided [1], if applicable.

Does your paper address subitem 5-iv?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study These details are not provided in this paper, but in previous publications about the two RCTs. From the paper: "Details about recruitment, randomization, and procedures were previously reported [24,29]."

5-v) Ensure replicability by publishing the source code, and/or providing screenshots/screencapture video, and/or providing ^owcharts of the algorithms used
Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing powcharts of the algorithms used. Replicability (i.e., other researchers should in principle be able to replicate the study) is a hallmark of scientiUc reporting.

Does your paper address subitem 5-v?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Data collection for the two RCTs ended in 2009. We acknowledge this limitation in the Discussion. While we do not believe this is an important limitation related to the significance of the results, specifics such as source code and screenshots would be outdated now.

5-vi) Digital preservation
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org, and/or publishing the source code or screenshots/videos alongside the article). As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.

Does your paper address subitem 5-vi?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study The applications are archived at the Center for Health Enhancement Systems Studies and available from the corresponding author.

5-vii) Access
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of speciUc group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi).

2 3 4 5
subitem not at all important essential Does your paper address subitem 5-vii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US "All caregivers who needed a computer were mailed a laptop with Internet access and a user manual. Participants who already had a computer with Internet access were reimbursed for Internet access during the study period." subitem not at all important essential Does your paper address subitem 5-viii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study

5-viii) Mode of delivery, features/functionalities/components of the intervention and comparator, and the theoretical framework
Please see the answer to item 5-ii.

5-ix) Describe use parameters
Describe use parameters (e.g., intended "doses" and optimal timing for use). Clarify what instructions or recommendations were given to the user, e.g., regarding timing, frequency, heaviness of use, if any, or was the intervention used ad libitum. Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Upon initial log-in and every 7 days after, caregivers were prompted when logging into CHESS to complete a check-in with questions about patient symptom status from a modified Edmonton Symptom Assessment Scale (ESAS) [33]."

5-x) Clarify the level of human involvement
Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability).

Does your paper address subitem 5-x?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Technical support was available by telephone. CHESS staff provided training on using CHESS via telephone or in clinic. Those in the CHESS+CR group were told that symptoms reported as "high" would trigger an email to the clinical team. Participants were not told the threshold, but when they gave a rating of ≥7, they were encouraged to call the clinic and notified on the website that the clinical team would be alerted. "

5-xi) Report any prompts/reminders used
Report any prompts/reminders used: Clarify if there were prompts (letters, emails, phone calls, SMS) to use the application, what triggered them, frequency etc. It may be necessary to distinguish between the level of prompts/reminders required for the trial, and the level of prompts/reminders for a routine application outside of a RCT setting (discuss under item 21 -generalizability). Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Please see the answer to 5-ix.

5-xii) Describe any co-interventions (incl. training/support)
Describe any co-interventions (incl. training/support): Clearly state any interventions that are provided in addition to the targeted eHealth intervention, as ehealth intervention may not be designed as stand-alone intervention. This includes training sessions and support [1]. It may be necessary to distinguish between the level of training required for the trial, and the level of training for a routine application outside of a RCT setting (discuss under item 21 -generalizability.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-xii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Please see the answer to 5-x.
6a) Completely deUned pre-speciUed primary and secondary outcome measures, including how and when they were assessed Does your paper address CONSORT subitem 6a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US "We examined two outcomes: (1) the proportion of improved caregiverreported, severe ("threshold") symptoms that patients had out of all threshold symptoms and (2) the proportion of caregiver-reported threshold symptoms that patients had out of the total symptoms reported on. Specifically, the study addresses this question: Does the CHESS system with the CR reduce symptom distress in patients more than CHESS without the CR?" 6a-i) Online questionnaires: describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed If outcomes were obtained through online questionnaires, describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed [9]. (activity, drowsiness, and well-being) with three common cancer symptoms (fatigue, constipation, and diarrhea). The modified ESAS contained 10 items. This analysis focuses on individual ratings for each of the 10 symptoms rather than on a single scale score calculated across symptoms."

6a-ii) Describe whether and how "use" (including intensity of use/dosage) was deVned/measured/monitored
Describe whether and how "use" (including intensity of use/dosage) was deUned/measured/monitored (logins, logUle analysis, etc.). Use/adoption metrics are important process outcomes that should be reported in any ehealth trial.

Does your paper address subitem 6a-ii?
Copy and paste relevant sections from manuscript text "Upon initial log-in and every 7 days after, caregivers were prompted when logging into CHESS to complete a check-in with questions about patient symptom status from a modified Edmonton Symptom Assessment Scale (ESAS) [33]. " 6a-iii) Describe whether, how, and when qualitative feedback from participants was obtained Describe whether, how, and when qualitative feedback from participants was obtained (e.g., through emails, feedback forms, interviews, focus groups).

Does your paper address subitem 6a-iii?
Copy and paste relevant sections from manuscript text "The effects of the CR may be explained in various ways, as clinicians explained in qualitative interviews [31], which include the following: 1. The CR could help clinicians better prepare to address patient symptoms and caregiver concerns in clinic visits. 2. The CR may boost caregiver efficacy in discussing symptoms with clinicians.
3. The CR may deepen caregiver involvement because caregivers can monitor patient symptoms and report their concerns directly to the clinical team. 4. The CR may enable earlier intervention because the severe symptom distress that triggers alerts might otherwise be unreported and therefore not attended to." 6b) Any changes to trial outcomes after the trial commenced, with reasons Does your paper address CONSORT subitem 6b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study No changes are reported.
7a) How sample size was determined https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US NPT: When applicable, details of whether and how the clustering by care provides or centers was addressed

7a-i) Describe whether and how expected attrition was taken into account when calculating the sample size
Describe whether and how expected attrition was taken into account when calculating the sample size.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 7a-i?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Details about the determination of sample size are not reported in this pooled analysis.

7b) When applicable, explanation of any interim analyses and stopping guidelines
Does your paper address CONSORT subitem 7b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Not applicable.
8a) Method used to generate the random allocation sequence NPT: When applicable, how care providers were allocated to each trial group Does your paper address CONSORT subitem 8a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "After caregivers and patients completed the consent form and pretest, a random number generator at the University of Wisconsin randomly assigned dyads to CHESS-Only or CHESS+CR (1:1 ratio)." 8b) Type of randomisation; details of any restriction (such as blocking and block size) Does your paper address CONSORT subitem 8b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Randomization was blocked by dyad relationship (spouse or partner vs nonspouse or partner) and race (white vs non-white)." 9) Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned Does your paper address CONSORT subitem 9? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US This detail is not described in this pooled analysis of two RCTs. "Details about recruitment, randomization, and procedures were previously reported [24,29]." 10) Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions Does your paper address CONSORT subitem 10? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study These details are not provided in this pooled analysis. "Details about recruitment, randomization, and procedures were previously reported [24,29]." 11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how NPT: Whether or not administering co-interventions were blinded to group assignment

11a-i) Specify who was blinded, and who wasn't
Specify who was blinded, and who wasn't. Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering co-interventions (if any).

2 3 4 5 subitem not at all important essential
Does your paper address subitem 11a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US not in the ms, or briepy explain why the item is not applicable/relevant for your study Participants were not blinded, as stated in the abstract.
11a-ii) Discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator" Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator".

Does your paper address subitem 11a-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Details about recruitment, randomization, and procedures were previously reported [24,29]." 11b) If relevant, description of the similarity of interventions (this item is usually not relevant for ehealth trials as it refers to similarity of a placebo or sham intervention to a active medication/intervention) Does your paper address CONSORT subitem 11b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US "In the CHESS-Only group, the information reported at check-in was intended for caregivers and not sent to clinicians. In the CHESS+CR group, CHESS summarized the caregiver-provided information and made it available (with patient permission) to the clinical team [31]." 12a) Statistical methods used to compare groups for primary and secondary outcomes NPT: When applicable, details of whether and how the clustering by care providers or centers was addressed Does your paper address CONSORT subitem 12a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "To compare the two outcomes of interest between groups over time (ie, improved patient threshold symptoms and proportion of patient threshold symptoms), the area under the curve (AUC) was calculated based on group-aggregated values. The AUC per group was calculated using the trapezoidal rule in NCSS 2007 [35]. Group differences in AUC were assessed by converting group AUC into a relative proportion per group and then conducting a proportional difference test (StatXact 5,Cytel) [36]. In addition, aggregated symptom reports were averaged across the 12 months. Differences in these averaged improved threshold symptoms and proportions of threshold symptoms were tested using the same method. The standardized statistics, P value and 95% CIs, were calculated based on methods outlined by Miettinen and Nurminen [37] and Chan and Zhang [38]. To test for potential response bias after group assignment, the proportion of patient threshold symptoms reported by caregivers at pretest was compared with the proportion at the first check-in using the multiple-sample McNemar test [39]. All tests were conducted at alpha=.05 level."

12a-i) Imputation techniques to deal with attrition / missing values
Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials. Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4]).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 12a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US "Patients and caregivers who completed fewer than two check-ins were excluded in the analysis because they supplied no data for comparison." 12b) Methods for additional analyses, such as subgroup analyses and adjusted analyses Does your paper address CONSORT subitem 12b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Examination of Possible Response Bias Caregivers in the CHESS+CR group reported a lower proportion of threshold symptoms at each 2-month period (Table 3). It may be that they felt they might be bothering the doctor or that word would get back to the patient that the caregiver thought the patient was doing poorly. To examine whether caregivers ranked symptoms lower to avoid triggering the alert to the clinician, we compared symptom ratings at pretest with those at the first online check-in, assuming any reluctance to alert the clinician would not have been present on pretest because pretest data were not sent to clinicians and patients knew this.

Does your paper address subitem X26-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US "The trials were approved by the institutional review boards at each recruitment site."

x26-ii) Outline informed consent procedures
Outline informed consent procedures e.g., if consent was obtained ooine or online (how? Checkbox, etc.?), and what information was provided (see 4a-ii). See [6] for some items to be included in informed consent documents.

Does your paper address subitem X26-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Details about recruitment, randomization, and procedures were previously reported [24,29]."

X26-iii) Safety and security procedures
Safety and security procedures, incl. privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline) 1 2 3 4 5 subitem not at all important essential

Does your paper address subitem X26-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US "Details about recruitment, randomization, and procedures were previously reported [24,29]."

RESULTS
13a) For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome NPT: The number of care providers or centers performing the intervention in each group and the number of patients treated by each care provider in each center Does your paper address CONSORT subitem 13a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study See Figure 1. 13b) For each group, losses and exclusions after randomisation, together with reasons Does your paper address CONSORT subitem 13b? (NOTE: Preferably, this is shown in a CONSORT ow diagram) * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US See Figure 1.

13b-i) Attrition diagram
Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other Ugures or tables demonstrating usage/dose/engagement. 1 2 3 4 5 subitem not at all important essential

Does your paper address subitem 13b-i?
Copy and paste relevant sections from the manuscript or cite the Ugure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "The attrition rate in this study (33.6%, 36/107) in the CHESS-Only group and 38.1% (42/110) in the CHESS+CR group) is comparable with other clinical trials of patients with advanced cancer. A review of 18 interventional supportive and palliative oncology trials found an attrition rate of 44% at study end [32]." 14a) Dates deUning the periods of recruitment and followup Does your paper address CONSORT subitem 14a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study In general: "Between September 2004 and April 2007, 235 dyads of patients with advanced-stage cancer and their primary informal caregivers were recruited to one of two randomized clinical trials of CHESS." For each RCT: ""Details about recruitment, randomization, and procedures were previously reported [24,29]." https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US

14a-i) Indicate if critical "secular events" fell into the study period
Indicate if critical "secular events" fell into the study period, e.g., signiUcant changes in Internet resources available or "changes in computer hardware or Internet delivery resources" 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 14a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study No secular events are reported.
14b) Why the trial ended or was stopped (early) Does your paper address CONSORT subitem 14b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Not applicable.

15) A table showing baseline demographic and clinical characteristics for each group
NPT: When applicable, a description of care providers (case volume, qualiUcation, expertise, etc.) and centers (volume) in each group Does your paper address CONSORT subitem 15? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US See Table 1.

15-i) Report demographics associated with digital divide issues
In ehealth trials it is particularly important to report demographics associated with digital divide issues, such as age, education, gender, social-economic status, computer/Internet/ehealth literacy of the participants, if known.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 15-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study See Table 1. 16) For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups

16-i) Report multiple "denominators" and provide deVnitions
Report multiple "denominators" and provide deUnitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at speciUc pre-deUned time points of interest (in absolute and relative numbers per group). Always clearly deUne "use" of the intervention.

17a-i) Presentation of process outcomes such as metrics of use and intensity of use
In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational deUnitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is deUned (e.g., timeout after idle time) [1] (report under item 6a).

Does your paper address subitem 17a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "In the 12-month period, the proportion of patient symptoms reported by caregivers at least twice did not differ by randomization group (CHESS-Only, 71/107, 66.4% vs CHESS+CR, 68/110, 61.8%, standardized difference Z=0.696, P=.49, 95% CI −.082 to .171). We also examined data about participant use of the system to see whether different participants used the system differently (ie, by gender, race, age, living situation, education, employment status, or income) and found no statistically significant difference." 17b) For binary outcomes, presentation of both absolute and relative effect sizes is recommended Does your paper address CONSORT subitem 17b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US Not applicable.
18) Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-speciUed from exploratory Does your paper address CONSORT subitem 18? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Examination of Possible Response Bias Caregivers in the CHESS+CR group reported a lower proportion of threshold symptoms at each 2-month period (Table 3). It may be that they felt they might be bothering the doctor or that word would get back to the patient that the caregiver thought the patient was doing poorly. To examine whether caregivers ranked symptoms lower to avoid triggering the alert to the clinician, we compared symptom ratings at pretest with those at the first online check-in, assuming any reluctance to alert the clinician would not have been present on pretest because pretest data were not sent to clinicians and patients knew this. A multiple-sample McNemar test [39] was used to test for differences between the CHESS-Only and CHESS+CR caregiver groups on changes in reporting threshold symptoms from pretest to first online check-in. A total of 1329 symptoms were rated by 135 caregivers (70 CHESS-Only, 65 CHESS+CR) at pretest and first online check-in and were used in this analysis. At pretest, no significant difference was found between randomized groups in the number of patient threshold symptoms reported (Z=0.189, P=.85, 95% CI −0.03 to 0.04), but the difference in reporting threshold symptoms from pretest to first online check-in was statistically significant (Z=6.910, P<.001, 95% CI 6.50 to 7.29), with CHESS+CR caregivers reporting a lower proportion of threshold symptoms at first check-in than at pretest ( Table 5), suggesting that CHESS+CR caregivers may have had a response bias toward lower ratings when they knew a clinician might be alerted."

18-i) Subgroup analysis of comparing only users
A subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii).

Does your paper address subitem 18-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US women are reporters overall versus 49.4% (131/266) of men; χ21 (N=266)=5.2, P=.02. In the CHESS+CR group, 64.4% (75/117) of women versus 45.5% (53/117) of men were reporters, χ21 (N=117)=4.02, P=.045; no significant differences were found in the CHESS-Only group. We also looked at 35 outcomes, such as caregiving burden and patient quality of life at each survey time frame, and did not find a consistent difference between caregivers who reported symptoms and those who did not.

19) All important harms or unintended effects in each group
(for speciUc guidance see CONSORT for harms) Does your paper address CONSORT subitem 19? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Caregivers in the CHESS+CR group reported a lower proportion of threshold symptoms at each 2-month period (Table 3). It may be that they felt they might be bothering the doctor or that word would get back to the patient that the caregiver thought the patient was doing poorly."

19-i) Include privacy breaches, technical problems
Include privacy breaches, technical problems. This does not only include physical "harm" to participants, but also incidents such as perceived or real privacy breaches [1], technical problems, and other unexpected/unintended incidents. "Unintended effects" also includes unintended positive effects [2].

Does your paper address subitem 19-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US None reported.

19-ii) Include qualitative feedback from participants or observations from staff/researchers
Include qualitative feedback from participants or observations from staff/researchers, if available, on strengths and shortcomings of the application, especially if they point to unintended/unexpected effects or uses. This includes (if available) reasons for why people did or did not use the application as intended by the developers.

Does your paper address subitem 19-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Please see the response to 6a-iii.

22) Interpretation consistent with results, balancing beneUts and harms, and considering other relevant evidence
NPT: In addition, take into account the choice of the comparator, lack of or partial blinding, and unequal expertise of care providers or centers in each group

22-i) Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use)
Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use). https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 22-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study Research question: "Does the CHESS system wth the CR reduce symptom distress in patients more than CHESS without the CR?" Answer: "Our results show that for symptoms causing severe (≥7 on a 0 to 10 scale) distress, patients whose caregivers had access to CHESS+CR, and therefore had an alert sent to their clinicians, had a greater proportion of symptom improvements than those with CHESS-Only, whose clinicians did not receive alerts or have access to ratings."

22-ii) Highlight unanswered new questions, suggest future research
Highlight unanswered new questions, suggest future research.

Does your paper address subitem 22-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "The overall assessed symptom rate in the CHESS+CR group dropped significantly more than in the CHESS-Only group. A response bias may have occurred if caregivers avoided using the check-in or rated symptoms lower because they feared bothering the clinician [41] or upsetting the patient. The lower overall assessed symptom rate in the CHESS+CR group may also be explained by clinician response." 20) Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses

20-i) Typical limitations in ehealth trials
Typical limitations in ehealth trials: Participants in ehealth trials are rarely blinded. Ehealth trials often look at a multiplicity of outcomes, increasing risk for a Type I error. Discuss biases due to non-use of the intervention/usability issues, biases through informed consent procedures, unexpected events. https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 20-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study

"Limitations
The study has limitations. Data collection was completed in 2009. . . . Each institution had a small number of clinicians, and they were not randomized. The effect of the CR on symptom improvement could potentially be greater than demonstrated here because clinicians seeing patients in both groups may have improved their symptom management with control-group patients as a result of changes they made with the intervention group. . . . Furthermore, all outcome variables were self-reported. . . . a distinct weakness of the paper is the lack of qualitative evidence from caregivers and patients. . . . Although the study suggests the potential of CR-like systems to enhance patient care and speed recovery from distressing symptoms, further research with different patient populations would help validate and improve the generalizability of these findings."

21) Generalisability (external validity, applicability) of the trial Undings
NPT: External validity of the trial Undings according to the intervention, comparators, patients, and care providers or centers involved in the trial

21-i) Generalizability to other populations
Generalizability to other populations: In particular, discuss generalizability to a general Internet population, outside of a RCT setting, and general patient population, including applicability of the study results for other organizations 1 2 3 4 5 subitem not at all important essential

Does your paper address subitem 21-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "Although the study suggests the potential of CR-like systems to enhance patient care and speed recovery from distressing symptoms, further research with different patient populations would help validate and improve the generalizability of these findings."

21-ii)
Discuss if there were elements in the RCT that would be different in a routine application https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US setting Discuss if there were elements in the RCT that would be different in a routine application setting (e.g., prompts/reminders, more human involvement, training sessions or other co-interventions) and what impact the omission of these elements could have on use, adoption, or outcomes if the intervention is applied outside of a RCT setting.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 21-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study "In addition, widespread use of such a system poses important challenges -cost, risk aversion, clinician time, and interoperability with the electronic health record (EHR)."

23) Registration number and name of trial registry
Does your paper address CONSORT subitem 23? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study " (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study

X27-i) State the relation of the study team towards the system being evaluated
In addition to the usual declaration of interests (Unancial or otherwise), also state the relation of the study team towards the system being evaluated, i.e., state if the authors/evaluators are distinct from or identical with the developers/sponsors of the intervention.

Does your paper address subitem X27-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briepy explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/formResponse?hl=en_US None declared.

About the CONSORT EHEALTH checklist
As a result of using this checklist, did you make changes in your manuscript? * yes, major changes yes, minor changes no What were the most important changes you made as a result of using this checklist?
How much time did you spend on going through the checklist INCLUDING making changes in your manuscript * 4 hours without making changes in the manuscript As a result of using this checklist, do you think your manuscript has improved? * yes no Other: Would you like to become involved in the CONSORT EHEALTH group?