Effect Evaluation of a Web-Based Coaching Intervention to Support Implementation of Sex Education Among Secondary School Teachers: Randomized Controlled Trial

Background The quality of implementation is important to ensure the effectiveness of behavioral change interventions in practice. Implementing such programs with completeness and adherence is not an automatic process and may require additional support. In school settings, the support teachers receive during implementation is often limited and appears to fall short when attempting to preserve completeness and adherence in program delivery. With the aim to improve completeness and adherence of teachers’ delivery of a sexual health promoting intervention (“Long Live Love” [LLL]) in secondary education, a Web-based e-coach was developed (“lesgevenindeliefde.nl”or“teachinglove.nl”). The effectiveness of the e-coach, as part of a broader implementation strategy, in influencing teachers’ implementation was evaluated. Objective This study aimed to report on the effect evaluation to determine the effect of the Web-based e-coach on teacher implementation of a school-based sex education program called LLL and on its determinants. Methods A cluster randomized controlled trial (e-coaching vs waiting list control) was conducted with a baseline assessment (T0) and follow-up (T1) 2 weeks after completing the LLL program. A total of 43 schools with 83 teachers participated in the study. In the follow-up, 38 schools participated, 23 in the e-coaching condition with 41 teachers and 15 in the control condition with 26 teachers. Multilevel regression analysis was used to evaluate the effect of the e-coaching website on implementation behavior, namely, completeness and adherence to LLL implementation, and on its determinants. Results The e-coaching intervention was not found to have an effect on teachers’ implementation behavior; teachers assigned to the experimental e-coaching website did not score higher on completeness (P=.60) or adherence (P=.67) as compared with teachers in the control condition. When comparing the 30 teachers who made actual use of the e-coaching website with the 37 teachers who did not, no significant differences were found either (P≥.54). In addition, there was no effect of e-coaching on the determinants of teacher implementation behavior (t67-75≤0.69; P≥.22). Conclusions E-coaching was not found to be effective in enhancing completeness of and adherence to LLL by teachers. The lack of effect may be attributed to the intervention content, the limited use, or the study design itself. The e-coaching intervention may not have adequately addressed adherence and completeness of LLL to bring about behavioral change. Furthermore, the e-coaching intervention was not or insufficiently used by teachers. A possible biased sample of motivated, able teachers may have agreed to participate in the study, and a possible “ceiling effect” may have been present because of the high implementation grade. This, however, does not imply that Web-based coaching in itself is an ineffective strategy to promote adherence and completeness of program implementation. A process evaluation is required as follow-up. Trial Registration International Standard Randomised Controlled Trial Number ISRCTN11754581; http://www.isrctn.com/ISRCTN11754581 (Archived by WebCite at http://www.webcitation.org/70C5TUOOh)

1b) ABSTRACT: Structured summary of trial design, methods, results, and conclusions NPT extension: Description of experimental treatment, comparator, care providers, centers, and blinding status.1b-i) Key features/functionalities/components of the intervention and comparator in the METHODS section of the ABSTRACT Mention key features/functionalities/components of the intervention and comparator in the abstract.If possible, also mention theories and principles used for designing the site.Keep in mind the needs of systematic reviewers and indexers by including important synonyms.(Note: Only report in the abstract what the main paper is reporting.If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1b-i?* Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "A cluster randomized controlled trial (ecoaching vs waiting list control) was conducted with a baseline assessment (T0) and follow up (T1) two weeks after completing the LLL program".
1b-ii) Level of human involvement in the METHODS section of the ABSTRACT Clarify the level of human involvement in the abstract, e.g., use phrases like "fully automated" vs. "therapist/nurse/care provider/physician-assisted" (mention number and expertise of providers involved, if any).(Note: Only report in the abstract what the main paper is reporting.If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1b-ii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study There was no human involvement in this study.The method section furthermore clearly explains that teachers were responsible for using the website, 1b-iii) Open vs. closed, web-based (self-assessment) vs. face-to-face assessments in the METHODS section of the ABSTRACT Mention how participants were recruited (online vs. of ine), e.g., from an open access website or from a clinic or a closed online user group (closed usergroup trial), and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment).Clearly say if outcomes were self-assessed through questionnaires (as common in web-based trials).Note: In traditional of ine trials, an open trial (open-label trial) is a type of clinical trial in which both the researchers and participants know which treatment is being administered.To avoid confusion, use "blinded" or "unblinded" to indicated the level of blinding instead of "open", as "open" in web-based trials usually refers to "open access" (i.e.participants can self-enrol).(Note: Only report in the abstract what the main paper is reporting.If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1b-iii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Participants were recruited form the national database of all secondary schools in the Netherlands"after stratification acoording to region and education level".Teachers were not informed of the existence of the e coaching vs waitinglist control group.
1b-iv) RESULTS section in abstract must contain use data Report number of participants enrolled/assessed in each group, the use/uptake of the intervention (e.g., attrition/adherence metrics, use over time, number of logins etc.), in addition to primary/secondary outcomes.(Note: Only report in the abstract what the main paper is reporting.If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1b-iv?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study effect on teachers' implementation behavior; teachers assigned to the experimental ecoaching website did not score higher on completeness (P= .60;Regression weight = 2.12 (3.99); 95% C.I. = 10.26,6.02; Cohen's d = .17)or fidelity (P= .67;Regression weight = 0.09 (0.21); 95% C.I. = 0.33, 0.51; Cohen's d = .14)as compared to teachers in the control condition.When comparing the 30 teachers who made actual use of the ecoaching website with the 37 teachers who did not, no significant differences were found either (P's ≥.54).There was also no effect of ecoaching on the determinants of teacher implementation behavior (t's;≤ 0.69; P's ≥ .22;.33 > Cohen's d > .06)." 1b-v) CONCLUSIONS/DISCUSSION in abstract for negative trials Conclusions/Discussions in abstract for negative trials: Discuss the primary outcome -if the trial is negative (primary outcome not changed), and the intervention was not used, discuss whether negative results are attributable to lack of uptake and discuss reasons.(Note: Only report in the abstract what the main paper is reporting.If this information is missing from the main body of text, consider adding it) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 1b-v?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The primary outcome is summarized and explanation is attributed to intervention content, limited use and study design.A process evaluation is then suggested and conducted to further explain these results.

INTRODUCTION 2a)
In INTRODUCTION: Scienti̾c background and explanation of rationale 2a-i) Problem and the type of system/solution Describe the problem and the type of system/solution that is object of the study: intended as stand-alone intervention vs. incorporated in broader health care program?Intended for a particular patient population?Goals of the intervention, e.g., being more cost-effective to other interventions, replace or complement other solutions?(Note: Details about the intervention are provided in "Methods" under 5) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 2a-i?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The problem is clearly explained, namely that teacher implementation of school based sexeducation programs is not optimal and support is needed in every phase of the implementation process.The available support appears to fall short, therefore this ecoaching intervention was developed to support teachers in complete and correct implementation of a schoolbased program called Long Live Love.The Ecoach is part of a broader implementation strategy: "Correct implementation is important for the effectiveness of an intervention.An intervention that is implemented completely and 2a-ii) Scienti c background, rationale: What is known about the (type of) system Scienti̾c background, rationale: What is known about the (type of) system that is the object of the study (be sure to discuss the use of similar systems for other conditions/diagnoses, if appropiate), motivation for the study, i.e. what are the reasons for and what is the context for this speci̾c study, from which stakeholder viewpoint is the study performed, potential impact of ̾ndings [2].Brie y justify the choice of the comparator.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 2a-ii?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study enough [15].It remains important to provide teachers with more personal assistance and ongoing support and consultation during the process of putting an innovation into practice [6,16,17,14].Currently, this support is limited to providing practical support in the form of teacher manuals with practical information on the content of the lessons and on how to deliver such lessons.However, more indepth coaching focusing on determinants of implementation such as selfefficacy and social support to enhance completeness and fidelity is lacking [18,19,11,2026]".
Bug ̾xes, Downtimes, Content Changes: ehealth systems are often dynamic systems.A description of changes to methods therefore also includes important changes made on the intervention or comparator during the trial (e.g., major bug ̾xes or changes in the functionality or content) (5-iii) and other "unexpected events" that may have in uenced study design such as staff changes, system failures/downtimes, etc. [2].
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 3b-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study One component of the ecoaching website, the selfreflection tool was out of order for most of the study and was most likely the reason why most teachers did not use it, however this only came to light during the process evaluation study, which is why it is not mentioned in this effect paper.

4a) Eligibility criteria for participants
Does your paper address CONSORT subitem 4a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "Only teachers who taught SRH were contacted.The schools with teachers who accepted the invitation (N = 45) were randomly assigned to either the control or the intervention (ecoach) group"

4a-i) Computer / Internet literacy
Computer / Internet literacy is often an implicit "de facto" eligibility criterion -this should be explicitly clari̾ed.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study This was not relevant to mention in this study as all teachers in the Netherlands are computer literate, largely due to their profession.
4a-ii) Open vs. closed, web-based vs. face-to-face assessments: Open vs. closed, web-based vs. face-to-face assessments: Mention how participants were recruited (online vs. of ine), e.g., from an open access website or from a clinic, and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment), i.e., to what degree got the study team to know the participant.In online-only trials, clarify if participants were quasianonymous and whether having multiple identities was possible or whether technical or logistical measures (e.g., cookies, email con̾rmation, phone calls) were used to detect/prevent these.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4a-ii?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "Teachers within these schools were invited, by email and telephone, to use the revised LLL program and to participate in a survey study on their experience and implementation of Sexual and Reproductive Health education (SRH) and LLL.....Teachers in the intervention and control group who consented to participation first received the baseline survey (T0) by post."

4a-iii) Information giving during recruitment
Information given during recruitment.Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 4a-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Does your paper address subitem 4b-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Teachers were aware that this study was being conducted by STI AIDS Nethelrands and Maastriht University in the email communication with the teachers and as logo's on the surveys.
5) The interventions for each group with suf̾cient details to allow replication, including how and when they were actually administered 5-i) Mention names, credential, af liations of the developers, sponsors, and owners Mention names, credential, af̾liations of the developers, sponsors, and owners [6] (if authors/evaluators are owners or developer of the software, this needs to be declared in a "Con ict of interest" section or mentioned elsewhere in the manuscript).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not relevant in this case as the intervention developer is not an author or evaluator of this study.

5-ii) Describe the history/development process
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The development of the intervention is described extensively in a published article and is referred to in the text."For a more detailed description of the ecoach see Schutte et al., 2016 [29]."

5-iii) Revisions and updating
Revisions and updating.Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial.
Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study The intervention was evaluated more or less directly after it was complete, thus no changes were made to the intervention during the evaluation process.

5-iv) Quality assurance methods
Provide information on quality assurance methods to ensure accuracy and quality of information provided [1], if applicable.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-iv?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study This is not mentioned in the ms but next to the selfreported data, several classroom observations during teacher implementation were conducted to get an impression of how teachers in both the control and intervention group were implementing the Long Live Love program.Does your paper address subitem 5-v?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study A flow chart is provided in the paper.

5-vi) Digital preservation
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org,and/or publishing the source code or screenshots/videos alongside the article).As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-vi?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study

5-vii) Access
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of speci̾c group.If known, describe how participants obtained "access to the platform and Internet" [1].To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi

5-x) Clarify the level of human involvement
Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered".It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21 -generalizability).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-x?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "At the same time, teachers in both groups were offered a training from the MHS in their region prior to implementing the revised LLL program.Separate trainings were delivered to teachers in the ecoach intervention group vs. the control group with teachers in the intervention group receiving additional information during the training about the ecoaching website and being stimulated to use it during the implementation of LLL....Additionally, teachers in the intervention group were given access to the ecoaching website with a personal user name and password, and an edition of the LLL teacher manual which contained references to the website."

5-xi) Report any prompts/reminders used
Report any prompts/reminders used: Clarify if there were prompts (letters, emails, phone calls, SMS) to use the application, what triggered them, frequency etc.It may be necessary to distinguish between the level of prompts/reminders required for the trial, and the level of prompts/reminders for a routine application outside of a RCT setting (discuss under item 21 -generalizability).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 5-xi?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Does your paper address subitem 6a-i?
Copy and paste relevant sections from manuscript text Not applicable to this study: Questionnaires were sent and received by post, not online.
6a-ii) Describe whether and how "use" (including intensity of use/dosage) was de ned/measured/monitored Describe whether and how "use" (including intensity of use/dosage) was de̾ned/measured/monitored (logins, log̾le analysis, etc.).Use/adoption metrics are important process outcomes that should be reported in any ehealth trial.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 6a-ii?
Copy and paste relevant sections from manuscript text Use of ecoaching was measured as a binary variable: "Did you visit the 'Lesgeven in de Liefde' website for teachers during your use of the new LLL program?" (0 = no, never, 1 = yes) 6a-iii) Describe whether, how, and when qualitative feedback from participants was obtained Describe whether, how, and when qualitative feedback from participants was obtained (e.g., through emails, feedback forms, interviews, focus groups).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 6a-iii?
Copy and paste relevant sections from manuscript text 7a) How sample size was determined NPT: When applicable, details of whether and how the clustering by care provides or centers was addressed 7a-i) Describe whether and how expected attrition was taken into account when calculating the sample size Describe whether and how expected attrition was taken into account when calculating the sample size.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 7a-i?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study A power analysis was conducted: Assuming fidelity as primary outcome measure, an effect size d = 0.60, a 5% alpha and a power of 90%, a total of 58 teachers per condition was required (29 schools with 2 teachers per school).Considering the nesting of teachers within schools and the randomization of schools, with the possibility of dropout, 70 teachers per condition or 140 teachers in total was required.
Does your paper address subitem 11a-i?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study evaluation".
The MHS trainers were aware of which teachers were in the intervention group and which not as they administered 2 separate trainings to these groups: "Separate trainings were delivered to teachers in the ecoach intervention group vs. the control group with teachers in the intervention group receiving additional information during the training about the e coaching website and being stimulated to use it during the implementation of LLL" 11a-ii) Discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator" Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator".
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 11a-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Teachers were blinded to the condition they were in: "Teachers were not informed about the existence of these two groups...The teachers in the control group were not exposed to or informed about the website, until after the end of the ecoach evaluation" Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "Given the nested structure of the design and the data (partly repeated) measurements nested within teachers nested within schools) multilevel regression analyses were used to evaluate the effects of ecoaching on teachers' (determinants of) implementation of LLL.Two levels were defined in the multilevel analysis: 1) school, 2) teacher.
The model included the predictors group (1 for intervention group (e coach), 0 for control group) for the outcomes of implementation behavior (completeness and fidelity) and group, time of measurement (baseline and posttest) and the interaction time x group for the determinants".

12a-i) Imputation techniques to deal with attrition / missing values
Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials.Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4]).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 12a-i?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study A seperate analysis was run to test for differences between teachers who actually used the intervention versus those who did not: "When comparing the 30 teachers who made actual use of the e coaching website with the 37 teachers who did not, still no significant differences were found in completeness (P = .54;

x26-ii) Outline informed consent procedures
Outline informed consent procedures e.g., if consent was obtained of ine or online (how?Checkbox, etc.?), and what information was provided (see 4a-ii).See [6] for some items to be included in informed consent documents.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem X26-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Information was provided online and offline (email, telephone) and consent was obtained in this manner as well: "Teachers within these schools were invited, by email and telephone, to use the revised LLL program and to participate in a survey study on their experience and implementation of Sexual and Reproductive Health education (SRH) and LLL...Teachers in the intervention and control group who consented to participation first received the baseline survey (T0) by post."

X26-iii) Safety and security procedures
Safety and security procedures, incl.privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline) 1 2 3 4 5 subitem not at all important essential Does your paper address subitem X26-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Teachers did not remain anonymous due to linking the basline measurement to the followup.Confidentiality was preserved.Teachers had the option to contact their local MHS or the evluators if there were any problems accessing the website.

RESULTS
13a) For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome NPT: The number of care providers or centers performing the intervention in each group and the number of patients treated by each care provider in each center Does your paper address CONSORT subitem 13a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study This is described in the results section and included a flow diagram.Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study This is described in the results section and included a consort flow diagram.

13b-i) Attrition diagram
Strongly recommended: An attrition diagram (e.g., proportion of participants still logging in or using the intervention/comparator in each group plotted over time, similar to a survival curve) or other ̾gures or tables demonstrating usage/dose/engagement.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 13b-i?
Copy and paste relevant sections from the manuscript or cite the ̾gure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study This is described in the results section and included a flow diagram.
14a) Dates de̾ning the periods of recruitment and followup Does your paper address CONSORT subitem 14a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study completing the LLL program....Teachers had two weeks to complete and return the survey.Nonresponders got a reminder by email and eventually by telephone three days after the deadline and were given another two weeks to return the survey....One week before expected completion of the LLL program, all teachers were reminded by email and telephone about the upcoming posttest questionnaire (T1).Within two weeks after completing the implementation of the LLL program, the T1 survey was sent to all teachers.Reminders were sent by email and eventually by telephone to nonresponders." 14a-i) Indicate if critical "secular events" fell into the study period Indicate if critical "secular events" fell into the study period, e.g., signi̾cant changes in Internet resources available or "changes in computer hardware or Internet delivery resources" 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 14a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not applicable to this study.No secular events occured in the study period.
14b) Why the trial ended or was stopped (early) Does your paper address CONSORT subitem 14b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not applicable to this study.The study was completed.Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Described in the text and see Table 2.

15-i) Report demographics associated with digital divide issues
In ehealth trials it is particularly important to report demographics associated with digital divide issues, such as age, education, gender, social-economic status, computer/Internet/ehealth literacy of the participants, if known.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 15-i?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Described in the text and see Table 2.
16) For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups 16-i) Report multiple "denominators" and provide de nitions Report multiple "denominators" and provide de̾nitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at speci̾c pre-de̾ned time points of interest (in absolute and relative numbers per group).Always clearly de̾ne "use" of the intervention.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 16-i?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Primary analysis should be intent-to-treat, secondary analyses could include comparing only "users", with the appropriate caveats that this is no longer a randomized sample (see 18-i).
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 16-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "The ecoaching intervention was not found to have an effect on teachers' implementation behavior; teachers assigned to the experimental ecoaching website did not score higher on completeness (P= .60;Regression weight = 2.12 (3.99); 95% C.I. = 10.26,6.02; Cohen's d = .17)or fidelity (P= .67;Regression weight = 0.09 (0.21); 95% C.I. = 0.33, 0.51; Cohen's d = .14)as compared to teachers in the control condition.When comparing the 30 teachers who made actual use of the ecoaching website with the 37 teachers who did not, no significant differences were found either (P's ≥.54).There was also no 17a) For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% con̾dence interval) Does your paper address CONSORT subitem 17a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study experimental ecoaching website did not score higher on completeness (P= .60;Regression weight = 2.12 (3.99); 95% C.I. = 10.26,6.02; Cohen's d = .17)or fidelity (P= .67;Regression weight = 0.09 (0.21); 95% C.I. = 0.33, 0.51; Cohen's d = .14)as compared to teachers in the control condition.When comparing the 30 teachers who made actual use of the ecoaching website with the 37 teachers who did not, no significant differences were found either (P's ≥.54).There was also no effect of ecoaching on the determinants of teacher implementation behavior (t's;≤ 0.69; P's ≥ .22;.33 > Cohen's d > .06)." 17a-i) Presentation of process outcomes such as metrics of use and intensity of use In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational de̾nitions is critical.This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length".These must be accompanied by a technical description how a metric like a "session" is de̾ned (e.g., timeout after idle time) [1] (report under item 6a).Does your paper address subitem 17a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Use of the website is more extensively discussed in the process evaluation paper.Use of the website was however presented briefly in this paper: "Based on the survey, it turned out that of the 41 teachers in the intervention group, 30 actually visited the website (75%).When comparing the 30 teachers who made actual use of the ecoaching website with the 37 teachers who did not, still no significant differences were found in completeness (P = .54;Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study prior to implementation.Despite ecoaching being systematically developed, and with the input of experienced teachers, ecoaching was not found to be effective in changing teachers' implementation behaviour or its determinants.In general, teachers implemented the new LLL program moderately during the pilot study.It is often difficult to prove the effectiveness of interventions or implementation strategy, even if they are solidly grounded in theory and evidence [18]." 22-ii) Highlight unanswered new questions, suggest future research Highlight unanswered new questions, suggest future research.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 22-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study present due to the implementation grade of teachers participating in the study already being high, making it difficult to improve using the e coaching intervention.Finally, the teachers who agreed to participate in the study may have been a biased sample of motivated, experienced teachers who were already capable of delivering LLL successfully....Secondly, for an intervention to have an effect, it is important that the intervention is used and positively perceived.By not being used or insufficiently used by teachers, ecoaching is unlikely to have an effect [36,37,38]....Lastly, the intervention itself may have been suboptimal." 21) Generalisability (external validity, applicability) of the trial ̾ndings NPT: External validity of the trial ̾ndings according to the intervention, comparators, patients, and care providers or centers involved in the trial

21-i) Generalizability to other populations
Generalizability to other populations: In particular, discuss generalizability to a general Internet population, outside of a RCT setting, and general patient population, including applicability of the study results for other organizations 1 2 3 4 5 subitem not at all important essential Does your paper address subitem 21-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "Our study was an RCT design and included a baseline measure and post test yet the number of participants that completed both surveys was relatively low, despite the dropout rate being low.This could have resulted in a lack of power to find significant effects if present and affected the generalizability of the results".
21-ii) Discuss if there were elements in the RCT that would be different in a routine application setting Discuss if there were elements in the RCT that would be different in a routine application setting (e.g., prompts/reminders, more human involvement, training sessions or other co-interventions) and what impact the omission of these elements could have on use, adoption, or outcomes if the intervention is applied outside of a RCT setting.
1 2 3 4 5 subitem not at all important essential Does your paper address subitem 21-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Screen shots and a demo video of the website were presented in the previous published article on the development of the ecoaching website Schutte L, van den Borne M, Kok G, Meijer S, Mevissen F. Innovatively supporting teachers' implementation of schoolbased sex education: Developing a webbased coaching intervention from problem to solution.Journal of Medical Internet Research 2016;18(7):e136.PMID: 27405241 Several factors might explain the lack of effect of ecoaching on implementation of LLL.....Motives for teachers' use or nonuse of e coaching need to be further explored as well as means to increase use of the website by teachers.Taking contextual factors as well as individual factors into consideration remains important when stimulating implementation [6,18,28]....Means of stimulating teachers to use the website need to be explored."20) Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses 20-i) Typical limitations in ehealth trialsTypical limitations in ehealth trials: Participants in ehealth trials are rarely blinded.Ehealth trials often look at a multiplicity of outcomes, increasing risk for a Type I error.Discuss biases due to non-use of the intervention/usability issues, biases through informed consent procedures, unexpected events.
Ensure replicability by publishing the source code, and/or providing screenshots/screencapture video, and/or providing owcharts of the algorithms used Ensure replicability by publishing the source code, and/or providing screenshots/screen-capture video, and/or providing owcharts of the algorithms used.Replicability (i.e., other researchers should in principle be able to replicate the study) is a hallmark of scienti̾c reporting.
11/26/2016 CONSORTEHEALTH (V 1.6.1)Submission/Publication Form https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL23O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lN… 14/38 5-v) Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study "Additionally, teachers in the intervention group were given access to the ecoaching website with a personal user name and password, and an edition of the LLL teacher manual which contained references to the website."5-viii)Mode of delivery, features/functionalities/components of the intervention and comparator, and the theoretical framework Describe mode of delivery, features/functionalities/components of the intervention and comparator, and the The Ecoach could be used by teachers prior to and during deliverance of the LLL program.For a more detailed description of the e coach seeSchutte et al., 2016 [29].Teachers in the intervention ecoaching group were stimulated to use the website through references to it in the teacher manual and by motivating them to use the website during the MHS training but teachers were given the liberty to further use the website as desired or needed."At the same time, teachers in both groups were offered a training from the MHS in their region prior to implementing the revised LLL program.Separate trainings were delivered to teachers in the ecoach intervention group vs. the control group with teachers in the intervention group https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL23O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lN… 15/38 Does your paper address subitem 5-vii?* theoretical framework [6] used to design them (instructional strategy [1], behaviour change techniques, persuasive features, etc., see e.g.,[7, 8]for terminology).This includes an in-depth description of the content (including where it is coming from and who developed it) [1]," whether [and how] it is tailored to individual circumstances and allows users to track their progress and receive feedback"[6].This also includes a description of communication delivery channels and -if computer-mediated communication is a component -whether communication was synchronous or asynchronous[6].It also includes information on presentation strategies [1], including page design principles, average amount of text on pages, presence of hyperlinks to other resources, etc.[1].*Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study on homonegative remarks or behavior.The content of the ecoach was guided by theories on implementation behavior[10,28], and based on a needs assessment among the target group[29].Intervention objectives were psychosocial determinants, such as awareness, teachers' personal benefit, social support, (anticipated) student responses and selfefficacy.Does your paper address subitem 5-ix?Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL23O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lN… 16/38not in the ms, or brie y explain why the item is not applicable/relevant for your study Halfway during the pilot implementation, an email reminded teachers in the intervention group to use the ecoaching website".5-xii)Describeanyco-interventions(incl.training/support)Describeanyco-interventions(incl.training/support):Clearlystateanyinterventions that are provided in addition to the targeted eHealth intervention, as ehealth intervention may not be designed as stand-alone intervention.This includes training sessions and support [1].It may be necessary to distinguish between the level of training required for the trial, and the level of training for a routine application outside of a RCT setting (discuss under item 21 -generalizability.Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study receiving additional information during the training about the ecoaching website and being stimulated to use it during the implementation of LLL.In the end, 58.2% (N=39; n=24 from the intervention group; n= 15 control group) of the participating teachers in the survey received a training from 14 different MHS's.The remaining teachers did not receive training, either because they refused the training as they felt there was no need or because the MHS in their region was not offering the training.""Copyandpasterelevantsectionsfromthe manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Fidelity was measured as the extent to which the LLL program was implemented according to the guidelines as prescribed in the teacher manual, with scores ranging from 1 ("I reviewed the program and only delivered a few lesson suggestions according to the teacher manual), to 5 ("I delivered all lesson suggestions for the LLL program exactly according to the teacher manual")[35].All measures, including number of items, response scales, reliability and exemplary items are presented in Table1."Ifoutcomes were obtained through online questionnaires, describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed [9].
https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL23O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lN… 17/38 "6a-i) Online questionnaires: describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL23O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lN… 18/38 Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not applicable to this study.No changes to trial outcomes after the trial commenced.
https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL23O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lN… 19/38 Not relevant for this paper.Additional analyses are conducted in the process evaluation paper to better understand differences between users and nonusers of the website.
Does your paper address CONSORT subitem 12b? *Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL23O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lN… 24/38Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Trial registration number: All procedures in the study were approved by the authorized Ethical Review Committee of Psychology & Neuroscience (ERCPN) at Maastricht University.Registration of this trial was not required.
For each group, losses and exclusions after randomisation, together with reasons Does your paper address CONSORT subitem 13b? (NOTE: Preferably, this is shown in a CONSORT ow diagram) * https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL23O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lN… 26/38 13b) who participated in this study however is demonstrated in the flow diagram and explained in the text and tables.e.g."A total of 43 schools with 83 teachers participated in the study.In the followup 38 schools participated, 23 in the ecoaching condition with 41 teachers, 15 in the control condition with 26 teachers.Multilevel regression analysis was used to evaluate the effect of the ecoaching website, on (determinants of) completeness and fidelity of LLL implementation.The ecoaching intervention was not found to have an effect on teachers' implementation behavior; teachers assigned to the 16-ii) Primary analysis should be intent-to-treat https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL23O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lN… 29/38 participants Subgroup analysis of comparing only usersA subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii).One component of the ecoaching website, the selfreflection tool was out of order for most of the study.This is mentioned and explained and relevant to the process evaluation paper, following this effect paper but has no relevance to this particular effect paper.Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study This is described in more detail in the process evaluation paper following this effect paper.
*Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Not applicable to this paper.No binary outcomes.https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL23O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lN… 31/3818-i)Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL23O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lN… 32/38 https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL23O8O9hrL5Sw/viewform?hl=en_US&formkey=dGlKd2Z2Q1lN… 33/38