Assessing Real-Time Moderation for Developing Adaptive Mobile Health Interventions for Medical Interns: Micro-Randomized Trial

Background Individuals in stressful work environments often experience mental health issues, such as depression. Reducing depression rates is difficult because of persistently stressful work environments and inadequate time or resources to access traditional mental health care services. Mobile health (mHealth) interventions provide an opportunity to deliver real-time interventions in the real world. In addition, the delivery times of interventions can be based on real-time data collected with a mobile device. To date, data and analyses informing the timing of delivery of mHealth interventions are generally lacking. Objective This study aimed to investigate when to provide mHealth interventions to individuals in stressful work environments to improve their behavior and mental health. The mHealth interventions targeted 3 categories of behavior: mood, activity, and sleep. The interventions aimed to improve 3 different outcomes: weekly mood (assessed through a daily survey), weekly step count, and weekly sleep time. We explored when these interventions were most effective, based on previous mood, step, and sleep scores. Methods We conducted a 6-month micro-randomized trial on 1565 medical interns. Medical internship, during the first year of physician residency training, is highly stressful, resulting in depression rates several folds higher than those of the general population. Every week, interns were randomly assigned to receive push notifications related to a particular category (mood, activity, sleep, or no notifications). Every day, we collected interns’ daily mood valence, sleep, and step data. We assessed the causal effect moderation by the previous week’s mood, steps, and sleep. Specifically, we examined changes in the effect of notifications containing mood, activity, and sleep messages based on the previous week’s mood, step, and sleep scores. Moderation was assessed with a weighted and centered least-squares estimator. Results We found that the previous week’s mood negatively moderated the effect of notifications on the current week’s mood with an estimated moderation of −0.052 (P=.001). That is, notifications had a better impact on mood when the studied interns had a low mood in the previous week. Similarly, we found that the previous week’s step count negatively moderated the effect of activity notifications on the current week’s step count, with an estimated moderation of −0.039 (P=.01) and that the previous week’s sleep negatively moderated the effect of sleep notifications on the current week’s sleep with an estimated moderation of −0.075 (P<.001). For all three of these moderators, we estimated that the treatment effect was positive (beneficial) when the moderator was low, and negative (harmful) when the moderator was high. Conclusions These findings suggest that an individual’s current state meaningfully influences their receptivity to mHealth interventions for mental health. Timing interventions to match an individual’s state may be critical to maximizing the efficacy of interventions. Trial Registration ClinicalTrials.gov NCT03972293; http://clinicaltrials.gov/ct2/show/NCT03972293

Manuscript tracking number * If this is a JMIR submission, please provide the manuscript tracking number under "other" (The ms tracking number can be found in the submission acknowledgement email, or when you login as author in JMIR. If the paper is already published in JMIR, then the ms tracking number is the fourdigit number at the end of the DOI, to be found at the bottom of each published article in JMIR) 1a) Does your paper address CONSORT item 1a? * I.e does the title contain the phrase "Randomized Controlled Trial"? (if not, explain the reason under "other") It contains "Micro-randomized Trial" as this was a micro-randomized trial, not a ran 1a-i) Identify the mode of delivery in the title Identify the mode of delivery. Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title. Avoid ambiguous terms like "online", "virtual", "interactive". Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g. email), use "computer-based" or "electronic" only if orine products are used. Use "virtual" only in the context of "virtual reality" (3-D worlds). Use "online" only in the context of "online support groups". Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms.
subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 1a-i? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Our title states "Mobile Health Interventions" 1a-ii) Non-web-based components or important co-interventions in title Mention non-web-based components or important co-interventions in title, if any (e.g., "with telephone support").
Does your paper address subitem 1a-ii?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study There were none.
1a-iii) Primary condition or target group in the title Mention primary condition or target group in the title, if any (e.g., "for children with Type I Diabetes") Example: A Web-based and Mobile Intervention with Telephone Support for Children with Type I Diabetes: Randomized Controlled Trial 1b) ABSTRACT: Structured summary of trial design, methods, results, and conclusions NPT extension: Description of experimental treatment, comparator, care providers, centers, and blinding status.
subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 1a-iii? * Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Title says "Medical Interns"

1b-i) Key features/functionalities/components of the intervention and comparator in the METHODS section of the ABSTRACT
Mention key features/functionalities/components of the intervention and comparator in the abstract. If possible, also mention theories and principles used for designing the site. Keep in mind the needs of systematic reviewers and indexers by including important synonyms. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 1b-i? * Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes, "The study investigated when to provide mHealth interventions to individuals in stressful work environments in order to improve their behavior and mental health. The mHealth interventions targeted three categories of behavior: mood, activity, and sleep" "Every week, interns were randomly assigned to receive push notiUcations of a particular category (mood, activity, sleep, or no notiUcations). " 1b-ii) Level of human involvement in the METHODS section of the ABSTRACT Clarify the level of human involvement in the abstract, e.g., use phrases like "fully automated" vs. "therapist/nurse/care provider/physician-assisted" (mention number and expertise of providers involved, if any). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) Does your paper address subitem 1b-ii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study There was no human involvement.
subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 1b-iii) Open vs. closed, web-based (self-assessment) vs. face-toface assessments in the METHODS section of the ABSTRACT Mention how participants were recruited (online vs. orine), e.g., from an open access website or from a clinic or a closed online user group (closed usergroup trial), and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment). Clearly say if outcomes were self-assessed through questionnaires (as common in web-based trials). Note: In traditional orine trials, an open trial (open-label trial) is a type of clinical trial in which both the researchers and participants know which treatment is being administered. To avoid confusion, use "blinded" or "unblinded" to indicated the level of blinding instead of "open", as "open" in web-based trials usually refers to "open access" (i.e. participants can self-enrol). (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) Does your paper address subitem 1b-iii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes, partially. We didn not mention blinding and recruitment as this was too much for the short abstract. "The interventions aimed to improve three different outcomes, weekly mood (assessed via a daily survey), weekly step count, and weekly sleep time." "Every day in the study, we collected interns' daily mood valence, sleep, and step data. " 1b-iv) RESULTS section in abstract must contain use data INTRODUCTION 2a) In INTRODUCTION: ScientiUc background and explanation of rationale Does your paper address subitem 1b-iv?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes.
"We conducted a 6-month micro-randomized trial (MRT) on 1,565 medical interns." 1b-v) CONCLUSIONS/DISCUSSION in abstract for negative trials Conclusions/Discussions in abstract for negative trials: Discuss the primary outcome -if the trial is negative (primary outcome not changed), and the intervention was not used, discuss whether negative results are attributable to lack of uptake and discuss reasons. (Note: Only report in the abstract what the main paper is reporting. If this information is missing from the main body of text, consider adding it) Does your paper address subitem 1b-v?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Does your paper address subitem 3b-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study There were no bug Uxes, downtimes, or content changes after commencement.
subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 4a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes, under the Participants subsection. "Medical doctors starting their year-long internship in summer of 2018 were eligible to participate in the study. Interns were on-boarded before the start of their internship (between April 2018-June 2018), in which they were instructed to download the study app, were provided Fitbits, completed a baseline survey, and were able to begin entering mood scores. " 4a-i) Computer / Internet literacy Computer / Internet literacy is often an implicit "de facto" eligibility criterion -this should be explicitly clariUed.
Does your paper address subitem 4a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study This was not an issue for our study. Participants did need iPhones. This is described under recruitment. "The study app and study participation were restricted to interns using an iPhone, the phone brand used by a majority of interns. " subitem not at all important 1 2 3 4 5 essential 4a-ii) Open vs. closed, web-based vs. face-to-face assessments: Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. orine), e.g., from an open access website or from a clinic, and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant. In online-only trials, clarify if participants were quasi-anonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email conUrmation, phone calls) were used to detect/prevent these.
Does your paper address subitem 4a-ii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes, mentioned under recruitment "Participants were recruited through emails, which were sent to future interns from 47 different recruitment institutions between April 1, 2018 and June 25, 2018. The recruitment institutions comprised both medical schools, where emails were sent to all graduates, and residency locations, where emails were sent to all incoming interns. 5,233 future interns received the initial email inviting them to participate in the study. 2,134 (41%) interns downloaded the study app, completed the consent form, and Ulled out the baseline survey sometime before June 25, 2018. The study app and study participation were restricted to interns using an iPhone, the phone brand used by a majority of interns. The 2,134 interns received a Fitbit Charge 2. Of the 2,134, 1,565 interns (73%) were randomly selected to participate in the MRT (see Supplement Section 1 for an explanation of this initial randomization). These 1,565 interns were randomized according to subitem not at all important 1 2 3 4 5 essential 4b) Settings and locations where the data were collected 4a-iii) Information giving during recruitment Information given during recruitment. Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results.
Does your paper address subitem 4a-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes, mentioned under recruitment "Participants were recruited through emails, which were sent to future interns from 47 different recruitment institutions between April 1, 2018 and June 25, 2018. The recruitment institutions comprised both medical schools, where emails were sent to all graduates, and residency locations, where emails were sent to all incoming interns. 5,233 future interns received the initial email inviting them to participate in the study. 2,134 (41%) interns downloaded the study app, completed the consent form, and Ulled out the baseline survey sometime before June 25, 2018. The study app and study participation were restricted to interns using an iPhone, the phone brand used by a majority of interns. The 2,134 interns received a Fitbit Charge 2. Of the 2,134, 1,565 interns (73%) were randomly selected to participate in the MRT (see Supplement Section 1 for an explanation of this initial randomization). These 1,565 interns were randomized according to Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes, here "Since the primary aim of the study is focused on understanding the effects of interventions on intern mental health, we employ a daily EMA to measure mood valence (see Figure 1-ii). Daily mood is one of two cardinal symptoms of depression [24]. This daily mood EMA is used widely to track mood in depressed patients [25]. There are more widely used measurements of mental health other than mood valence (such as the Patient Health Questionnaire-9, PHQ-9 [26]). However, these questionnaires are more time intensive and their assessment may cause higher non-response rates. Participants are prompted to enter their daily mood every day at a user-speciUed time between 5 pm and 10 pm.
In addition to collecting EMA data, the app aggregates and displays visual summaries of interns' historical data, including step and sleep counts (collected through the Fitbit) and mood EMA data" 4b-i) Report if outcomes were (self-)assessed through online questionnaires Clearly report if outcomes were (self-)assessed through online questionnaires (as common in webbased trials) or otherwise.
subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 4b-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes, here "Since the primary aim of the study is focused on understanding the effects of interventions on intern mental health, we employ a daily EMA to measure mood valence (see Figure 1-ii). Daily mood is one of two cardinal symptoms of depression [24]. This daily mood EMA is used widely to track mood in depressed patients [25]. There are more widely used measurements of mental health other 5) The interventions for each group with sujcient details to allow replication, including how and when they were actually administered subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 5-i) Mention names, credential, ajliations of the developers, sponsors, and owners Mention names, credential, ajliations of the developers, sponsors, and owners [6] (if authors/evaluators are owners or developer of the software, this needs to be declared in a "Consict of interest" section or mentioned elsewhere in the manuscript).
Does your paper address subitem 5-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study No, his was not an issue for us.

5-ii) Describe the history/development process
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results.
Does your paper address subitem 5-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study No, this was not done as the development process was not unique.

5-iii) Revisions and updating
Revisions and updating. Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial. Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b).
Does your paper address subitem 5-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study No, there was no revising or updating once the trial began.

5-iv) Quality assurance methods
Provide information on quality assurance methods to ensure accuracy and quality of information provided [1], if applicable.
Does your paper address subitem 5-iv?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study This was not applicable.
subitem not at all important 1 2 3 4 5 essential subitem not at all important 1 2 3 4 5 essential 5-v) Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing sowcharts of the algorithms used Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing sowcharts of the algorithms used. Replicability (i.e., other researchers should in principle be able to replicate the study) is a hallmark of scientiUc reporting.
Does your paper address subitem 5-v?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Link to the code for the analysis is provided on the Urst author's website. This is mentioned in the paper.

5-vi) Digital preservation
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org, and/or publishing the source code or screenshots/videos alongside the article). As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.
Does your paper address subitem 5-vi?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study There is not url to the application. Links to the website where the app info can be found are given in the paper.
subitem not at all important 1 2 3 4 5 essential 5-vii) Access Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of speciUc group. If known, describe how participants obtained "access to the platform and Internet" [1]. To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi).
Does your paper address subitem 5-vii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Since the primary aim of the study is focused on understanding the effects of interventions on intern mental health, we employ a daily EMA to measure mood valence (see Figure 1-ii). Daily mood is one of two cardinal symptoms of depression [24]. This daily mood EMA is used widely to track mood in depressed patients [25]. There are more widely used measurements of mental health other than mood valence (such as the Patient Health Questionnaire-9, PHQ-9 [26]).
However, these questionnaires are more time intensive and their assessment may cause higher non-response rates. Participants are prompted to enter their daily mood every day at a user-speciUed time between 5 pm and 10 pm.
In addition to collecting EMA data, the app aggregates and displays visual summaries of interns' historical data, including step and sleep counts (collected through the Fitbit) and mood EMA data. See Figure 1- Does your paper address subitem 5-ix?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study The goal of the study was to discover timing, so optimal timing and doses were not known prior.

5-x) Clarify the level of human involvement
Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered". It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21generalizability).
Does your paper address subitem 5-x?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study No, the human involvement was minimal.

5-xi) Report any prompts/reminders used
Report any prompts/reminders used: Clarify if there were prompts (letters, emails, phone calls, SMS) to use the application, what triggered them, frequency etc. It may be necessary to distinguish between the level of prompts/reminders required for the trial, and the level of prompts/reminders for a routine application outside of a RCT setting (discuss under item 21 -generalizability).
subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 5-xi? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Not applicable.

5-xii) Describe any co-interventions (incl. training/support)
Describe any co-interventions (incl. training/support): Clearly state any interventions that are provided in addition to the targeted eHealth intervention, as ehealth intervention may not be designed as stand-alone intervention. This includes training sessions and support [1]. It may be necessary to distinguish between the level of training required for the trial, and the level of training for a routine application outside of a RCT setting (discuss under item 21 -generalizability. Does your paper address subitem 5-xii? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study 6a) Completely deUned pre-speciUed primary and secondary outcome measures, including how and when they were assessed Yes, done here "interventions on intern mental health, we employ a daily EMA to measure mood valence (see Figure 1-ii). Daily mood is one of two cardinal symptoms of depression [24]. This daily mood EMA is used widely to track mood in depressed patients [25]. There are more widely used measurements of mental health other than mood valence (such as the Patient Health Questionnaire-9, PHQ-9 [26]). However, these questionnaires are more time intensive and their assessment may cause higher non-response rates. Participants are prompted to enter their daily mood every day at a user-speciUed time between 5 pm and 10 pm.
In addition to collecting EMA data, the app aggregates and displays visual summaries of interns' historical data, including step and sleep counts (collected through the Fitbit) and mood EMA data. See Figure 1- Does your paper address CONSORT subitem 6b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Not applicable.
subitem not at all important 1 2 3 4 5 essential 7b) When applicable, explanation of any interim analyses and stopping guidelines 8a) Method used to generate the random allocation sequence NPT: When applicable, how care providers were allocated to each trial group 7a-i) Describe whether and how expected attrition was taken into account when calculating the sample size Describe whether and how expected attrition was taken into account when calculating the sample size.
Does your paper address subitem 7a-i?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Not applicable. We aimed to recruit as many interns as possible for this exploratory trial.
Does your paper address CONSORT subitem 7b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study 8b) Type of randomisation; details of any restriction (such as blocking and block size) 9) Mechanism used to implement the random allocation sequence (such as sequentially numbered containers), describing any steps taken to conceal the sequence until interventions were assigned Does your paper address CONSORT subitem 8a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study This was done automatically with the app.
Does your paper address CONSORT subitem 8b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study The randomization is described thoroughly. There is no blocking.
Does your paper address CONSORT subitem 9? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study "In order to determine the best time to deliver notiUcations of different categories, we ran an MRT. The MRT design is pictured in Figure 2. The MRT design and protocol were approved by University of Michigan IRB (UM IRB Protocol #HUM00033029).
The main randomization was the weekly randomization to a speciUc notiUcation category (mood, activity, sleep) or to no-notiUcation. Thus, we were able to compare how a week of a certain notiUcation category changed intern behavior when compared to a week of no notiUcations.
The randomization-and the ensuing analysis of effects-occurred at the weeklylevel for two reasons. For one, the notiUcations are not intended to change the interns' behavior in the next few hours, but over the next few days. Randomizing and analyzing effects at the weekly-level, as opposed to daily-or minute-level, permitted discovery of longer-term effects. Secondly, as interns are quite busy, they may not have signiUcant behavior change after receiving a single notiUcation. Instead interns received several notiUcations of the same category and had a consistent reminder about improving that category.
Given a week when a user was randomized to receiving notiUcations, every day they were further randomized (with 50% probability) to receive a notiUcation. Hence, for a mood notiUcation week the user received, on average, 3.5 mood notiUcations that week. The purpose of this randomization is to balance delivering enough notiUcations to be noticeable and cause behavior change, but not too often that it leads to treatment fatigue [9]. Treatment fatigue is pervasive in mHealth [7] and for individuals with heavy workloads [9].
Another way to prevent treatment fatigue is through increased variability in notiUcations and the order they are received [34]. For each notiUcation category, the notiUcations alternated between life insights and tips. Also, given a type and category, each notiUcation was drawn randomly, without replacement, from a corresponding bucket of notiUcations. The bucket reUlled once it was completely emptied. Alternating between life insights and tips increased the day to day variability of the notiUcation framing. Drawing notiUcations without replacement ensured that users are not receiving repeats of the same notiUcation. Under this scheme, on average, a user did not receive a repeated notiUcation for 16 weeks." 10) Who generated the random allocation sequence, who enrolled participants, and who assigned participants to interventions 11a) If done, who was blinded after assignment to interventions (for example, participants, care providers, those assessing outcomes) and how NPT: Whether or not administering co-interventions were blinded to group assignment subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 10? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study This was done through the Sen Lab, which is mentioned in the paper.
11a-i) Specify who was blinded, and who wasn't Specify who was blinded, and who wasn't. Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering co-interventions (if any).
Does your paper address subitem 11a-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Not applicable for the MRT because all participants received treatment at certain times, so it was not possible to blind them.
subitem not at all important 1 2 3 4 5 essential 11b) If relevant, description of the similarity of interventions (this item is usually not relevant for ehealth trials as it refers to similarity of a placebo or sham intervention to a active medication/intervention) 12a) Statistical methods used to compare groups for primary and secondary outcomes NPT: When applicable, details of whether and how the clustering by care providers or centers was addressed 11a-ii) Discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator" Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator".
Does your paper address subitem 11a-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Participants did not know. All participants received the intervention of interest during some weeks of the study.
Does your paper address CONSORT subitem 11b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Not applicable.
subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 12a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes "To estimate the coejcients, we use a multi-categorical extension of the weighted and centered least squares estimator described in [35]. The estimation method provides asymptotically unbiased estimates of the causal effect moderation of interest. The method also protects against potential misspeciUcation of terms not interacted with treatment (\alpha_0X_t+\alpha_1M_t). The method assesses the uncertainty of the coejcient estimates using robust standard error estimation-the "sandwich" estimator [36]-to account for correlation between outcomes over time" 12a-i) Imputation techniques to deal with attrition / missing values Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials. Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4] subitem not at all important 1 2 3 4 5 essential Does your paper address subitem X26-ii? Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes "Participants were recruited through emails, which were sent to future interns from 47 different recruitment institutions between April 1, 2018 and June 25, 2018. The recruitment institutions comprised both medical schools, where emails were sent to all graduates, and residency locations, where emails were sent to all incoming interns. 5,233 future interns received the initial email inviting them to participate in the study. 2,134 (41%) interns downloaded the study app, completed the consent form, and Ulled out the baseline survey sometime before June 25, 2018. The study app and study participation were restricted to interns using an iPhone, the phone brand used by a majority of interns. The 2,134 interns received a Fitbit Charge 2. Of the 2,134, 1,565 interns (73%) were randomly selected to participate in the MRT (see Supplement Section 1 for an explanation of this initial randomization). These 1,565 interns were randomized according to Figure 2. Interns were incentivized to participate in the study by receiving the Fitbit wearable and up to $125, distributed Uve times throughout the year ($25 each time) based on continued participation."

X26-iii) Safety and security procedures
Safety and security procedures, incl. privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline) Does your paper address subitem X26-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study This was not done.

RESULTS
13a) For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome NPT: The number of care providers or centers performing the intervention in each group and the number of patients treated by each care provider in each center 13b) For each group, losses and exclusions after randomisation, together with reasons Does your paper address CONSORT subitem 13a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes, "Participants were recruited through emails, which were sent to future interns from 47 different recruitment institutions between April 1, 2018 and June 25, 2018. The recruitment institutions comprised both medical schools, where emails were sent to all graduates, and residency locations, where emails were sent to all incoming interns. 5,233 future interns received the initial email inviting them to participate in the study. 2,134 (41%) interns downloaded the study app, completed the consent form, and Ulled out the baseline survey sometime before June 25, 2018. The study app and study participation were restricted to interns using an iPhone, the phone brand used by a majority of interns. The 2,134 interns received a Fitbit Charge 2. Of the 2,134, 1,565 interns (73%) were randomly selected to participate in the MRT (see Supplement Section 1 for an explanation of this initial randomization). These 1,565 interns were randomized according to subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 13b? (NOTE: Preferably, this is shown in a CONSORT sow diagram) * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes "Participants were recruited through emails, which were sent to future interns from 47 different recruitment institutions between April 1, 2018 and June 25, 2018. The recruitment institutions comprised both medical schools, where emails were sent to all graduates, and residency locations, where emails were sent to all incoming interns. 5,233 future interns received the initial email inviting them to participate in the study. 2,134 (41%) interns downloaded the study app, completed the consent form, and Ulled out the baseline survey sometime before June 25, 2018. The study app and study participation were restricted to interns using an iPhone, the phone brand used by a majority of interns. The 2,134 interns received a Fitbit Charge 2. Of the 2,134, 1,565 interns (73%) were randomly selected to participate in the MRT (see Supplement Section 1 for an explanation of this initial randomization). These 1,565 interns were randomized according to Figure 2. Interns were incentivized to participate in the study by receiving the Fitbit wearable and up to $125, distributed Uve times throughout the year ($25 each time) based on continued participation."

13b-i) Attrition diagram
Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other Ugures or tables demonstrating usage/dose/engagement. 14a) Dates deUning the periods of recruitment and follow-up Does your paper address subitem 13b-i?
Copy and paste relevant sections from the manuscript or cite the Ugure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study No, the attrition diagram was unnecessary and didn't provide anything necessary. Everything was described here: "Participants were recruited through emails, which were sent to future interns from 47 different recruitment institutions between April 1, 2018 and June 25, 2018. The recruitment institutions comprised both medical schools, where emails were sent to all graduates, and residency locations, where emails were sent to all incoming interns. 5,233 future interns received the initial email inviting them to participate in the study. 2,134 (41%) interns downloaded the study app, completed the consent form, and Ulled out the baseline survey sometime before June 25, 2018. The study app and study participation were restricted to interns using an iPhone, the phone brand used by a majority of interns. The 2,134 interns received a Fitbit Charge 2. Of the 2,134, 1,565 interns (73%) were randomly selected to participate in the MRT (see Supplement Section 1 for an explanation of this initial randomization). These 1,565 interns were randomized according to Figure 2. Interns were incentivized to participate in the study by receiving the Fitbit wearable and up to $125, distributed Uve times throughout the year ($25 each time) based on continued participation." subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 14a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes "Participants were recruited through emails, which were sent to future interns from 47 different recruitment institutions between April 1, 2018 and June 25, 2018. The recruitment institutions comprised both medical schools, where emails were sent to all graduates, and residency locations, where emails were sent to all incoming interns. 5,233 future interns received the initial email inviting them to participate in the study. 2,134 (41%) interns downloaded the study app, completed the consent form, and Ulled out the baseline survey sometime before June 25, 2018. The study app and study participation were restricted to interns using an iPhone, the phone brand used by a majority of interns. The 2,134 interns received a Fitbit Charge 2. Of the 2,134, 1,565 interns (73%) were randomly selected to participate in the MRT (see Supplement Section 1 for an explanation of this initial randomization). These 1,565 interns were randomized according to Figure 2. Interns were incentivized to participate in the study by receiving the Fitbit wearable and up to $125, distributed Uve times throughout the year ($25 each time) based on continued participation." 14a-i) Indicate if critical "secular events" fell into the study period Indicate if critical "secular events" fell into the study period, e.g., signiUcant changes in Internet resources available or "changes in computer hardware or Internet delivery resources" Does your paper address subitem 14a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study 14b) Why the trial ended or was stopped (early) 15) A Does your paper address CONSORT subitem 15? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes, but not in a table as it was quite brief: "Of the 1,565 interns in the MRT, 56% were female, 49% had previously experienced an episode of depression. The interns represented 321 different residency locations and 42 specialties. The study interns' baseline information closely resembled the known characteristics of the general medical intern population [19]. Throughout the trial, we measured intern mood valence, steps, and nightly sleep. Summaries of the weekly-level averages of those data can be found in Table 2."

15-i) Report demographics associated with digital divide issues
In ehealth trials it is particularly important to report demographics associated with digital divide issues, such as age, education, gender, social-economic status, computer/Internet/ehealth literacy of the participants, if known.
16) For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 15-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes, but not in a table as it was quite brief: "Of the 1,565 interns in the MRT, 56% were female, 49% had previously experienced an episode of depression. The interns represented 321 different residency locations and 42 specialties. The study interns' baseline information closely resembled the known characteristics of the general medical intern population [19]. Throughout the trial, we measured intern mood valence, steps, and nightly sleep. Summaries of the weekly-level averages of those data can be found in Table 2." 16-i) Report multiple "denominators" and provide deUnitions Report multiple "denominators" and provide deUnitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at speciUc pre-deUned time points of interest (in absolute and relative numbers per group). Always clearly deUne "use" of the intervention.
subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 16-i? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes, "Participants were recruited through emails, which were sent to future interns from 47 different recruitment institutions between April 1, 2018 and June 25, 2018. The recruitment institutions comprised both medical schools, where emails were sent to all graduates, and residency locations, where emails were sent to all incoming interns. 5,233 future interns received the initial email inviting them to participate in the study. 2,134 (41%) interns downloaded the study app, completed the consent form, and Ulled out the baseline survey sometime before June 25, 2018. The study app and study participation were restricted to interns using an iPhone, the phone brand used by a majority of interns. The 2,134 interns received a Fitbit Charge 2. Of the 2,134, 1,565 interns (73%) were randomly selected to participate in the MRT (see Supplement Section 1 for an explanation of this initial randomization). These 1,565 interns were randomized according to Figure 2. Interns were incentivized to participate in the study by receiving the Fitbit wearable and up to $125, distributed Uve times throughout the year ($25 each time) based on continued participation"

16-ii) Primary analysis should be intent-to-treat
Primary analysis should be intent-to-treat, secondary analyses could include comparing only "users", with the appropriate caveats that this is no longer a randomized sample (see 18-i).
17a) For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% conUdence interval) Does your paper address subitem 16-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes, no users were dropped from the study. Intend to treat was used.
"Participants were recruited through emails, which were sent to future interns from 47 different recruitment institutions between April 1, 2018 and June 25, 2018. The recruitment institutions comprised both medical schools, where emails were sent to all graduates, and residency locations, where emails were sent to all incoming interns. 5,233 future interns received the initial email inviting them to participate in the study. 2,134 (41%) interns downloaded the study app, completed the consent form, and Ulled out the baseline survey sometime before June 25, 2018. The study app and study participation were restricted to interns using an iPhone, the phone brand used by a majority of interns. The 2,134 interns received a Fitbit Charge 2. Of the 2,134, 1,565 interns (73%) were randomly selected to participate in the MRT (see Supplement Section 1 for an explanation of this initial randomization). These 1,565 interns were randomized according to Does your paper address CONSORT subitem 17a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes "Primary Aim We conclude that previous week's average daily self-reported mood valence is a statistically signiUcant negative moderator of the effect of notiUcations on average daily self-reported mood valence. The estimate for the moderation is average daily self-reported mood valence. The estimate for the moderation is -0.052 (SE = 0.014, P = .001). Figure 3 plots the estimated treatment effect at various values of the moderator. Figure 3 shows that the effect of notiUcations (compared to no notiUcations) was positive for weeks when previous mood was low, but negative for weeks when previous mood was high. For example, when previous week's average daily mood was 3, we estimate that a week of notiUcations increased an intern's average daily mood by 0.19 (effect size = 0.14). However, when previous week's average daily mood was 9, we estimate that a week of notiUcations decreased an intern's average daily mood by 0.12 (effect size = -0.08).

Exploratory Sub-Aim
For each notiUcation category, we plotted the estimated treatment effect at various values of the moderator. Essentially, we broke apart the moderation effect in Figure 3 into the 3 categories of notiUcations. The result is shown in Figure 4. We included the line for general notiUcations from Figure 3 for reference. Figure 4 demonstrates that the moderation by previous week's average daily mood was similar for all 3 notiUcation categories.

Secondary Aim 1
We conclude that previous week's average daily step count is a statistically signiUcant negative moderator of the effect of activity notiUcations on average daily steps. The estimate for the moderation is -0.039 (SE = 0.015, P = .01). Figure 5 plots the estimated treatment effect at various values of the moderator. In Figure 5, for interpretability, we re-transformed the moderation effect back In Figure 5, for interpretability, we re-transformed the moderation effect back from the analysis scale (square root) to the original scale. We see from Figure 5 that the effect of activity notiUcations (compared to no notiUcations) was positive for weeks when previous steps were low, but negative for weeks when previous steps were high. For example, when previous week's average daily step count was 5,625, we estimate that a week of activity notiUcations increased an intern's average daily step count by 165 steps (effect size = 0.05). However, when previous week's average daily step count was 12,100, we estimate that a week of activity notiUcations decreased an intern's average daily step count by 60 steps (effect size = -0.02).

Secondary Aim 2
We conclude that previous week's average daily sleep is a statistically signiUcant negative moderator of the effect of sleep notiUcations on average daily sleep. The estimate for the moderation is -0.075 (SE = 0.018, P < .001). Figure 6 plots the estimated treatment effect at various values of the moderator. Again, we re-transformed the moderation effect back from analysis scale (square root) to the original scale. Also, for interpretability, the x-axis is on the hourly scale, while the y-axis is on the minute scale. We see from Figure 6 that the effect of sleep notiUcations (compared to no notiUcations) was positive for weeks when previous sleep was low, but negative for weeks when previous sleep was high. For example, when previous week's average daily sleep was 5 hours, we estimate that a week of sleep notiUcations increased an intern's average daily sleep by 8 minutes (effect size = 0.11). However, when previous week's average daily sleep was 8 hours, we estimate that a week of sleep notiUcations decreased an intern's average daily sleep by 5 minutes (effect size = -0.07).

Additional Analyses
Section 1 of the Supplement contains detailed results on additional analyses, including analysis of non-moderated main effects, changes in effects over time, effects of life insights and tips, effects on long-term PHQ-9 scores, and analysis of baseline moderators. There is evidence of a negative effect of (general) subitem not at all important 1 2 3 4 5 essential of baseline moderators. There is evidence of a negative effect of (general) notiUcations on mood. There is also evidence of a positive effect of activity notiUcations on step counts and sleep notiUcations on sleep duration. All of these effect sizes, however, are small. There is not strong evidence that these effects change over time. There is minor evidence that tips perform better than life insights on improving step count and sleep duration. We did not see any effects on long-term mental health outcomes. We saw some evidence of nonlinear moderation for the primary and secondary aims. The non-linear moderator analysis suggested that, when the moderators are high, the treatment effect on sleep hours and step count is close to 0 (as opposed to negative). Finally, we found that baseline variables, such as gender and depression history, were weak moderators of notiUcation effects, demonstrating the value of personalizing intervention delivery on real-time data.
" 17a-i) Presentation of process outcomes such as metrics of use and intensity of use In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational deUnitions is critical. This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length". These must be accompanied by a technical description how a metric like a "session" is deUned (e.g., timeout after idle time) [1] (report under item 6a).
Does your paper address subitem 17a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Not applicable. Use was not a concern. The supplement contains information on missing data. 17b) For binary outcomes, presentation of both absolute and relative effect sizes is recommended 18) Results of any other analyses performed, including subgroup analyses and adjusted analyses, distinguishing pre-speciUed from exploratory subitem not at all important 1 2 3 4 5 essential Does your paper address CONSORT subitem 17b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Not applicable.
Does your paper address CONSORT subitem 18? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Found in supplement under additional analyses.

18-i) Subgroup analysis of comparing only users
A subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii).
Does your paper address subitem 18-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Not applicable. Does your paper address subitem 19-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Yes "There were also a couple technological errors that occurred throughout the trial.
There were 8 days (out of the 182 total days) where, due to server issues, no notiUcations were sent to any subject. Also, the weekly randomization to a notiUcation category occurred without replacement, as opposed to with replacement as originally intended." subitem not at all important 1 2 3 4 5 essential DISCUSSION 22) Interpretation consistent with results, balancing beneUts and harms, and considering other relevant evidence NPT: In addition, take into account the choice of the comparator, lack of or partial blinding, and unequal expertise of care providers or centers in each group subitem not at all important 1 2 3 4 5 essential 19-ii) Include qualitative feedback from participants or observations from staff/researchers Include qualitative feedback from participants or observations from staff/researchers, if available, on strengths and shortcomings of the application, especially if they point to unintended/unexpected effects or uses. This includes (if available) reasons for why people did or did not use the application as intended by the developers.
Does your paper address subitem 19-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or briesy explain why the item is not applicable/relevant for your study Not applicable.
22-i) Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use) Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use).