Effects of Information Architecture on the Effectiveness and User Experience of Web-Based Patient Education in Middle-Aged and Older Adults: Online Randomized Experiment

Background Web-based patient education is increasingly offered to improve patients’ ability to learn, remember, and apply health information. Efficient organization, display, and structural design, that is, information architecture (IA), can support patients’ ability to independently use web-based patient education. However, the role of IA in the context of web-based patient education has not been examined systematically. Objective To support intervention designers in making informed choices that enhance patients’ learning, this paper describes a randomized experiment on the effects of IA on the effectiveness, use, and user experience of a patient education website and examines the theoretical mechanisms that explain these effects. Methods Middle-aged and older adults with self-reported hip or knee joint complaints were recruited to use and evaluate 1 of 3 patient education websites containing information on total joint replacement surgery. Each website contained the same textual content based on an existing leaflet but differed in the employed IA design (tunnel, hierarchical, or matrix design). Participants rated the websites on satisfaction, engagement, control, relevance, trust, and novelty and completed an objective knowledge test. Analyses of variance and structural equation modeling were used to examine the effects of IA and construct a theoretical model. Results We included 215 participants in our analysis. IA did not affect knowledge gain (P=.36) or overall satisfaction (P=.07) directly. However, tunnel (mean 3.22, SD 0.67) and matrix (mean 3.17, SD 0.69) architectures were found to provide more emotional support compared with hierarchical architectures (mean 2.86, SD 0.60; P=.002). Furthermore, increased perceptions of personal relevance in the tunnel IA (β=.18) were found to improve satisfaction (β=.17) indirectly. Increased perceptions of active control in the matrix IA (β=.11) also improved satisfaction (β=.27) indirectly. The final model of the IA effects explained 74.3% of the variance in satisfaction and 6.8% of the variance in knowledge and achieved excellent fit (χ217,215=14.7; P=.62; root mean square error of approximation=0.000; 95% CI [0.000-0.053]; comparative fit index=1.00; standardized root mean square residual=0.044). Conclusions IA has small but notable effects on users’ experiences with web-based health education interventions. Web-based patient education designers can employ tunnel IA designs to guide users through sequentially ordered content or matrix IA to offer users more control over navigation. Both improve user satisfaction by increasing user perceptions of relevance (tunnel) and active control (matrix). Although additional research is needed, hierarchical IA designs are currently not recommended, as hierarchical content is perceived as less supportive, engaging, and relevant, which may diminish the use and, in turn, the effect of the educational intervention.

I.e does the title contain the phrase "Randomized Controlled Trial"?(if not, explain the reason under "other") Not a RCT.The study is identi ed as a randomized experiment in the 1a-i) Identify the mode of delivery in the title Identify the mode of delivery.Preferably use "web-based" and/or "mobile" and/or "electronic game" in the title.Avoid ambiguous terms like "online", "virtual", "interactive".Use "Internet-based" only if Intervention includes non-web-based Internet components (e.g.email), use "computer-based" or "electronic" only if o ine products are used.Use "virtual" only in the context of "virtual reality" (3-D worlds).Use "online" only in the context of "online support groups".Complement or substitute product names with broader terms for the class of products (such as "mobile" or "smart phone" instead of "iphone"), especially if the application runs on different platforms.
Does your paper address subitem 1a-i?* Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: "web-based patient education" subitem not at all important 1 2 3 4 5 essential Does your paper address subitem 1a-ii?
Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: no non-web-based components 1a-iii) Primary condition or target group in the title Mention primary condition or target group in the title, if any (e.g., "for children with Type I Diabetes") Example: A Web-based and Mobile Intervention with Telephone Support for Children with Type I Diabetes: Randomized Controlled Trial Does your paper address subitem 1a-iii?* Copy and paste relevant sections from manuscript title (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: "middle-aged and older adults" 1b-i) Key features/functionalities/components of the intervention and comparator in the METHODS section of the ABSTRACT Mention key features/functionalities/components of the intervention and comparator in the abstract.If possible, also mention theories and principles used for designing the site.Keep in mind the needs of systematic reviewers and indexers by including important synonyms.Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: "information architecture" and "(tunnel, hierarchical, or matrix design)" 1b-ii) Level of human involvement in the METHODS section of the ABSTRACT Clarify the level of human involvement in the abstract, e.g., use phrases like "fully automated" vs. "therapist/nurse/care provider/physician-assisted" (mention number and expertise of providers involved, if any).(Note: Only report in the abstract what the main paper is reporting.If this information is missing from the main body of text, consider adding it) Does your paper address subitem 1b-ii?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer 1b-iii) Open vs. closed, web-based (self-assessment) vs. face-toface assessments in the METHODS section of the ABSTRACT Mention how participants were recruited (online vs. o ine), e.g., from an open access website or from a clinic or a closed online user group (closed usergroup trial), and clarify if this was a purely web-based trial, or there were face-to-face components (as part of the intervention or for assessment).Clearly say if outcomes were self-assessed through questionnaires (as common in web-based trials).Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer 1b-iv) RESULTS section in abstract must contain use data Does your paper address subitem 1b-iv?
Copy and paste relevant sections from the manuscript abstract (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer 1b-v) CONCLUSIONS/DISCUSSION in abstract for negative trials Conclusions/Discussions in abstract for negative trials: Discuss the primary outcome -if the trial is negative (primary outcome not changed), and the intervention was not used, discuss whether negative results are attributable to lack of uptake and discuss reasons.Information given during recruitment.Specify how participants were briefed for recruitment and in the informed consent procedures (e.g., publish the informed consent documentation as appendix, see also item X26), as this information may have an effect on user self-selection, user expectation and may also bias results.
Does your paper address subitem 4a-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer Does your paper address CONSORT subitem 4b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: "The complete experiment was conducted online via survey software (Qualtrics).Each eligible participant was provided a hyperlink to the survey.After providing consent, participants lled out questionnaires regarding their communication preferences and skills, health, anxiety, and coping behaviour, which were used to determine the patient pro le [39].Participants also stated the extent to which they already felt knowledgeable about TJR surgery (part A).
In part B participants were randomly assigned to one of three experimental conditions using Qualtrics' built-in randomizer.The allocation sequence and assignments were concealed from all participants, the researchers, and the consultant hired for patient recruitment until all data was collected.Participants were initially asked to focus on either the website's design or content.After reviewing the website's design, participants reported satisfaction and user perceptions.They were then asked to view the website a second time whilst focusing on content.Then they lled out a knowledge test designed for the purpose of this study.The order of focus (design vs. content) was counterbalanced.Finally, participants shared their socio-demographic information and received a code for reimbursement (part C). " Does your paper address subitem 4b-i?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: "The complete experiment was conducted online via survey software (Qualtrics).Each eligible participant was provided a hyperlink to the survey.After providing consent, participants lled out questionnaires regarding their communication preferences and skills, health, anxiety, and coping behaviour, which were used to determine the patient pro le [39].Participants also stated the extent to which they already felt knowledgeable about TJR surgery (part A).
In part B participants were randomly assigned to one of three experimental conditions using Qualtrics' built-in randomizer.The allocation sequence and assignments were concealed from all participants, the researchers, and the consultant hired for patient recruitment until all data was collected.Participants were initially asked to focus on either the website's design or content.After reviewing the website's design, participants reported satisfaction and user perceptions.They were then asked to view the website a second time whilst focusing on content.Then they lled out a knowledge test designed for the purpose of this study.The order of focus (design vs. content) was counterbalanced.Finally, participants shared their socio-demographic information and received a code for reimbursement (part C). " Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer 5-i) Mention names, credential, a liations of the developers, sponsors, and owners Mention names, credential, a liations of the developers, sponsors, and owners [6] (if authors/evaluators are owners or developer of the software, this needs to be declared in a "Con ict of interest" section or mentioned elsewhere in the manuscript).
Does your paper address subitem 5-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer

5-ii) Describe the history/development process
Describe the history/development process of the application and previous formative evaluations (e.g., focus groups, usability testing), as these will have an impact on adoption/use rates and help with interpreting results.
Does your paper address subitem 5-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: in dedicated sections "design process" and "pilot usability study" Revisions and updating.Clearly mention the date and/or version number of the application/intervention (and comparator, if applicable) evaluated, or describe whether the intervention underwent major changes during the evaluation process, or whether the development and/or content was "frozen" during the trial.Describe dynamic components such as news feeds or changing content which may have an impact on the replicability of the intervention (for unexpected events see item 3b).
Does your paper address subitem 5-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer

5-iv) Quality assurance methods
Provide information on quality assurance methods to ensure accuracy and quality of information provided [1], if applicable.
Does your paper address subitem 5-iv?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: paper includings screenshots and o ine websites on request

5-vi) Digital preservation
Digital preservation: Provide the URL of the application, but as the intervention is likely to change or disappear over the course of the years; also make sure the intervention is archived (Internet Archive, webcitation.org,and/or publishing the source code or screenshots/videos alongside the article).As pages behind login screens cannot be archived, consider creating demo pages which are accessible without login.
Does your paper address subitem 5-vi?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: o ine copies of websites are archived and available on request

5-vii) Access
Access: Describe how participants accessed the application, in what setting/context, if they had to pay (or were paid) or not, whether they had to be a member of speci c group.If known, describe how participants obtained "access to the platform and Internet" [1].To ensure access for editors/reviewers/readers, consider to provide a "backdoor" login account or demo mode for reviewers/readers to explore the application (also important for archiving purposes, see vi).
Does your paper address subitem 5-vii?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: eligible participants accessed the application in their spare time on their chosen device."Participants received monetary reimbursement (€15) for their participation." Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer

5-x) Clarify the level of human involvement
Clarify the level of human involvement (care providers or health professionals, also technical assistance) in the e-intervention or as co-intervention (detail number and expertise of professionals involved, if any, as well as "type of assistance offered, the timing and frequency of the support, how it is initiated, and the medium by which the assistance is delivered".It may be necessary to distinguish between the level of human involvement required for the trial, and the level of human involvement required for a routine application outside of a RCT setting (discuss under item 21generalizability).
Does your paper address subitem 5-x?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer

5-xi) Report any prompts/reminders used
Report any prompts/reminders used: Clarify if there were prompts (letters, emails, phone calls, SMS) to use the application, what triggered them, frequency etc.It may be necessary to distinguish between the level of prompts/reminders required for the trial, and the level of prompts/reminders for a routine application outside of a RCT setting (discuss under item 21 -generalizability).
Does your paper address subitem 5-xi?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: "Eligible participants who had not started or completed the survey after three weeks were reminded via email once."Copy and paste relevant sections from manuscript text Your answer 6a-ii) Describe whether and how "use" (including intensity of use/dosage) was de ned/measured/monitored Describe whether and how "use" (including intensity of use/dosage) was de ned/measured/monitored (logins, log le analysis, etc.).Use/adoption metrics are important process outcomes that should be reported in any ehealth trial.
Does your paper address subitem 6a-ii?
Copy and paste relevant sections from manuscript text Your answer 6a-iii) Describe whether, how, and when qualitative feedback from participants was obtained Describe whether, how, and when qualitative feedback from participants was obtained (e.g., through emails, feedback forms, interviews, focus groups).

Does your paper address subitem 6a-iii?
Copy and paste relevant sections from manuscript text Yes: "we collected short qualitative feedback forms on the perceived (dis)advantages of the website."Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: "participants were randomly assigned to one of three experimental conditions using Qualtrics' built-in randomizer.The allocation sequence and assignments were concealed from all participants, the researchers, and the consultant hired for patient recruitment until all data was collected." Does your paper address CONSORT subitem 8b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No: not applicable (no restrictions) Does your paper address CONSORT subitem 9? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: "participants were randomly assigned to one of three experimental conditions using Qualtrics' built-in randomizer.The allocation sequence and assignments were concealed from all participants, the researchers, and the consultant hired for patient recruitment until all data was collected." Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: "participants were randomly assigned to one of three experimental conditions using Qualtrics' built-in randomizer.The allocation sequence and assignments were concealed from all participants, the researchers, and the consultant hired for patient recruitment until all data was collected." 11a-i) Specify who was blinded, and who wasn't Specify who was blinded, and who wasn't.Usually, in web-based trials it is not possible to blind the participants [1, 3] (this should be clearly acknowledged), but it may be possible to blind outcome assessors, those doing data analysis or those administering co-interventions (if any).

Does your paper address subitem 11a-i? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: "The allocation sequence and assignments were concealed from all participants, the researchers, and the consultant hired for patient recruitment until all data was collected."12a) Statistical methods used to compare groups for primary and secondary outcomes NPT: When applicable, details of whether and how the clustering by care providers or centers was addressed 11a-ii) Discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator" Informed consent procedures (4a-ii) can create biases and certain expectations -discuss e.g., whether participants knew which intervention was the "intervention of interest" and which one was the "comparator".
Does paper address subitem 11a-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer Does your paper address CONSORT subitem 11b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: we discuss that the textual and graphical content in the three experimental conditions is the same and provide screenshots to show this.

Does your paper address CONSORT subitem 12a? *
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: statistical methods are discussed in detail in section 'statistical methods' Imputation techniques to deal with attrition / missing values: Not all participants will use the intervention/comparator as intended and attrition is typically high in ehealth trials.Specify how participants who did not use the application or dropped out from the trial were treated in the statistical analysis (a complete case analysis is strongly discouraged, and simple imputation techniques such as LOCF may also be problematic [4]).
Does your paper address subitem 12a-i?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: missing data is reported in the participant ow diagram and each data table.As there was almost no missing data, no data was imputed.We address the limitations of the cases in which we had complete missing outcome data in the discussion: "Of particular concern is that 6% of the sample dropped out after viewing the allocated website, as they might have done so based on their (negative) response to the website.This could make the study susceptible to type I errors [64,65].This problem could not be remedied by intention-to-treat analysis due to the design of the experiment in which dropped-out participants had complete missing outcome data [65].Therefore, we checked whether dropout was associated with allocation to a speci c website, which was not the case.This made it unlikely that participants stopped because they were discontent with the allocated website." Does your paper address CONSORT subitem 12b? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: statistical methods are discussed in detail in section 'statistical methods' Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer

x26-ii) Outline informed consent procedures
Outline informed consent procedures e.g., if consent was obtained o ine or online (how?Checkbox, etc.?), and what information was provided (see 4a-ii).See [6] for some items to be included in informed consent documents.

Does your paper address subitem X26-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer X26-iii) Safety and security procedures Safety and security procedures, incl.privacy considerations, and any steps taken to reduce the likelihood or detection of harm (e.g., education and training, availability of a hotline) https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 31/43

RESULTS
13a) For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analysed for the primary outcome NPT: The number of providers or centers performing the intervention in each group and the number of patients treated by each care provider in each center 13b) For each group, losses and exclusions after randomisation, together with reasons Does your paper address subitem X26-iii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer Does your paper address CONSORT subitem 13a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: this information is included in the participant ow diagram.
Does your paper address CONSORT subitem 13b? (NOTE: Preferably, this is shown in a CONSORT ow diagram) * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: this information is included in the participant ow diagram.Does your paper address subitem 13b-i?
Copy and paste relevant sections from the manuscript or cite the gure number if applicable (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer Does your paper address CONSORT subitem 14a? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: recruitment, experiment, and data collection were all conducted in July 2018 (Methods section) 14a-i) Indicate if critical "secular events" fell into the study period Indicate if critical "secular events" fell into the study period, e.g., signi cant changes in Internet resources available or "changes in computer hardware or Internet delivery resources" Does your paper address subitem 14a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No: not applicable (experiment was not stopped early) Does your paper address CONSORT subitem 15? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: this information is provided in Does your paper address subitem 15-i?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: this information is provided in ) For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% con dence interval) 16-i) Report multiple "denominators" and provide de nitions Report multiple "denominators" and provide de nitions: Report N's (and effect sizes) "across a range of study participation [and use] thresholds" [1], e.g., N exposed, N consented, N used more than x times, N used more than y weeks, N participants "used" the intervention/comparator at speci c pre-de ned time points of interest (in absolute and relative numbers per group).Always clearly de ne "use" of the intervention.
Does your paper address subitem 16-i?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: each analysis reports the applicable sample size (N) and effect size.Use is de ned as a visit, that is expressed in seconds.

16-ii) Primary analysis should be intent-to-treat
Primary analysis should be intent-to-treat, secondary analyses could include comparing only "users", with the appropriate caveats that this is no longer a randomized sample (see 18-i).
Does your paper address subitem 16-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study 17a-i) Presentation of process outcomes such as metrics of use and intensity of use In addition to primary/secondary (clinical) outcomes, the presentation of process outcomes such as metrics of use and intensity of use (dose, exposure) and their operational de nitions is critical.This does not only refer to metrics of attrition (13-b) (often a binary variable), but also to more continuous exposure metrics such as "average session length".These must be accompanied by a technical description how a metric like a "session" is de ned (e.g., timeout after idle time) [1] (report under item 6a).
your paper address subitem 17a-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Does your paper address CONSORT subitem 18? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes, the de nition of the subgroups is provided in the introduction, section "does one IA design t all?" and Table 1 "Description of communicative preferences and capabilities of three patient pro les".Analysis plan regarding subgroups is described: "The concept of tailored IAs was explored in a two-way ANOVA with condition and pro le as the independent variables.". Results are reported in section "Tailored information architectures: interactions with patient pro le".

18-i) Subgroup analysis of comparing only users
A subgroup analysis of comparing only users is not uncommon in ehealth trials, but if done, it must be stressed that this is a self-selected sample and no longer an unbiased sample from a randomized trial (see 16-iii).
Does your paper address subitem 18-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use).
Does your paper address subitem 22-i?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: the rst two paragraphs repeat the study questions, aims, and summarizes the answers.

22-ii) Highlight unanswered new questions, suggest future research
Highlight unanswered new questions, suggest future research.
Does your paper address subitem 22-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer Does your paper address subitem 20-i?* Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: several limitations are discussed in the section 'strengths and limitations'.
These include typical ehealth trial limitations such as self-selection of participants.

21-i) Generalizability to other populations
Generalizability to other populations: In particular, discuss generalizability to a general Internet population, outside of a RCT setting, and general patient population, including applicability of the study results for other organizations Does your paper address subitem 21-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer Does your paper address subitem 21-ii?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer Does your paper address CONSORT subitem 23? * Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No: not applicable (no RCT).
Does your paper address CONSORT subitem 24? * Cite a Multimedia Appendix, other reference, or copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No: not applicable (no trial protocol available, because not a RCT) In addition to the usual declaration of interests ( nancial or otherwise), also state the relation of the study team towards the system being evaluated, i.e., state if the authors/evaluators are distinct from or identical with the developers/sponsors of the intervention.
Does your paper address subitem X27-i?
Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Your answer STOP -Save this form as PDF before you click submit To generate a record that you lled in this form, we recommend to generate a PDF of this page (on a Mac, simply select "print" and then select "print as PDF") before you submit it.
When you submit your (revised) paper to JMIR, please upload the PDF as supplementary le.
Don't worry if some text in the textboxes is cut off, as we still have the complete information in our database.Thank you!
your paper address CONSORT item 1a? * (Note: Only report in the abstract what the main paper is reporting.If this information is missing from the main body of text, consider adding it) https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 10address subitem 1b-i?* Note: In traditional o ine trials, an open trial (open-label trial) is a type of clinical trial in which both the researchers and participants know which treatment is being administered.To avoid confusion, use "blinded" or "unblinded" to indicated the level of blinding instead of "open", as "open" in web-based trials usually refers to "open access" (i.e.participants can self-enrol).(Note: Only report in the abstract what the main paper is reporting.If this information is missing from the main body of text, consider adding it) https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dGl… 11address subitem 1b-iii?
4b-ii) Report how institutional a liations are displayedReport how institutional a liations are displayed to potential participants [on ehealth media], as a liations with prestigious hospitals or universities may affect volunteer rates, use, and reactions with regards to an intervention.(Nota required item -describe only if this may bias results) https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 18/43 5) The interventions for each group with su cient details to allow replication, including how and when they were actually address subitem 4b-ii?
(V 1.6.1)-Submission/Publication Form https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 39(external validity, applicability) of the trial ndings NPT: External validity of the trial ndings according to the intervention, comparators, patients, and care providers or centers involved in the trial subitem ) Typical limitations in ehealth trials Typical limitations in ehealth trials: Participants in ehealth trials are rarely blinded.Ehealth trials often look at a multiplicity of outcomes, increasing risk for a Type I error.Discuss biases due to non-use of the intervention/usability issues, biases through informed consent procedures, unexpected events.
(V 1.6.1)-Submission/Publication Form https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 41/43 25) Sources of funding and other support (such as supply of drugs), role of funders X27) Con icts of Interest (not a CONSORT item) address CONSORT subitem 25? *Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: "This work is part of the research programme Tailored healthcare through customer pro ling (Project 314-99-118), which is nanced by the Netherlands Organisation for Scienti c Research (NWO) and Zimmer Biomet Inc.We thank the participants who took part in the (pilot) study and our consortium partners for their constructive feedback on our work.""Part of the funding of this project is provided by Zimmer Biomet Inc (see also Acknowledgments).This sponsor has had no role in study design of this protocol, data collection, analysis and interpretation, or writing of the report.In the case that this partner wants to apply for a patent based on research ndings, publication can be postponed by a maximum of three months.No party has the right to prohibit publication of these ndings.The authors have full access to study data."X27-i) State the relation of the study team towards the system being evaluated

Final
step: Click submit !Click submit so we have your answers in our database!Never submit passwords through Google Forms.This content is neither created nor endorsed by Google.Report Abuse -Terms of Service Would you like to become involved in the CONSORT EHEALTH group?This would involve for example becoming involved in participating in a workshop and writing an "Explanation and Elaboration" document other comments or questions on CONSORT EHEALTH Your answer SUBMIT Forms Report number of participants enrolled/assessed in each group, the use/uptake of the intervention (e.g., attrition/adherence metrics, use over time, number of logins etc.), in addition to primary/secondary outcomes.(Note: Only report in the abstract what the main paper is reporting.If this information is missing from the main body of text, consider adding it) Mode of delivery, features/functionalities/components of the intervention and comparator, and the theoretical framework Describe mode of delivery, features/functionalities/components of the intervention and comparator, and the theoretical framework [6] used to design them (instructional strategy [1], behaviour change techniques, persuasive features, etc., see e.g.,[7, 8]for terminology).This includes an in-depth description of the content (including where it is coming from and who developed it) [1]," whether [and how] it is tailored to individual circumstances and allows users to Describe use parameters (e.g., intended "doses" and optimal timing for use).Clarify what instructions or recommendations were given to the user, e.g., regarding timing, frequency, heaviness of use, if any, or was the intervention used ad libitum.
https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 21/43Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: content is based on existing education: "All websites contained the same textual content based on an existing patient education lea et titled 'Instructions after an outpatient Total Hip Prosthesis (THP)' [Instructies na een Totale Heup Prothese (THP) in dagbehandeling] used by the local hospital (Reinier de Graaf Gasthuis, the Netherlands).The lea et addressed practical concerns before and after outpatient THR surgery including preparation for surgery, pain, medication, and physiotherapy.".The design features of information architecture which differ between experimental conditions are discussed with examples and screenshots.5-ix) Describe use parameters https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 22/43 essential Does your paper address subitem 5-ix?
Describe any co-interventions (incl.training/support):Clearlystateanyinterventionsthat are provided in addition to the targeted eHealth intervention, as ehealth intervention may not be designed as stand-alone intervention.This includes training sessions and support[1].It may be necessary to distinguish between the level of training required for the trial, and the level of training for a routine application outside of a RCT setting (discuss under item 21 -generalizability.Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study No: not applicable to the experiment (independent single use of website) Online questionnaires: describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed If outcomes were obtained through online questionnaires, describe if they were validated for online use and apply CHERRIES items to describe how the questionnaires were designed/deployed[9].
https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 23/43 essential 5-xii) Describe any co-interventions (incl.training/support)Copyand paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: section measurements describes outcomes measures, validation of the questionnaires used and example items.Timing of measurements is discussed in section 'Participants and procedure' 6a-i) https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 24/43

Table 2
For each group, number of participants (denominator) included in each analysis and whether the analysis was by original assigned groups https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 34/43 16) Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Yes: Effect sizes are included for each analyses (primary and secondary) in the text: "F(2,212)=6.376,P=.002, η2=.057." and in the data tables.
https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 35/43 Include privacy breaches, technical problems.This does not only include physical "harm" to participants, but also incidents such as perceived or real privacy breaches [1], technical problems, and other unexpected/unintended incidents."Unintendedeffects"alsoincludesunintended positive effects[2].Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Include qualitative feedback from participants or observations from staff/researchers, if available, on strengths and shortcomings of the application, especially if they point to unintended/unexpected effects or uses.This includes (if available) reasons for why people did or did not use the application as intended by the developers.Copy and paste relevant sections from the manuscript (include quotes in quotation marks "like this" to indicate direct quotes from your manuscript), or elaborate on this item by providing additional information not in the ms, or brie y explain why the item is not applicable/relevant for your study Restate study questions and summarize the answers suggested by the data, starting with primary outcomes and process outcomes (use) https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 37/43 NPT: In addition, take into account the choice of the comparator, lack of or partial blinding, and unequal expertise of care providers or centers in each group 19-i) Include privacy breaches, technical problems CONSORT-EHEALTH (V 1.6.1)-Submission/Publication Form https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 38/43 Registration number and name of trial registry 24) Where the full trial protocol can be accessed, if available 21-ii) Discuss if there were elements in the RCT that would be different in a routine application setting Discuss if there were elements in the RCT that would be different in a routine application setting (e.g., prompts/reminders, more human involvement, sessions or other co-interventions) and what impact the omission of these elements could have on use, adoption, or outcomes if the intervention is applied outside of a RCT setting.
16-7-2019 CONSORT-EHEALTH (V 1.6.1)-Submission/Publication Form https://docs.google.com/forms/d/e/1FAIpQLSfZBSUp1bwOc_OimqcS64RdfIAFvmrTSkZQL2-3O8O9hrL5Sw/viewform?hl=en_US&formkey=dG… 40/43 As a result of using this checklist, did you make changes in your manuscript?* What were the most important changes you made as a result of using this checklist?