This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Web-based patient education is increasingly offered to improve patients’ ability to learn, remember, and apply health information. Efficient organization, display, and structural design, that is, information architecture (IA), can support patients’ ability to independently use web-based patient education. However, the role of IA in the context of web-based patient education has not been examined systematically.
To support intervention designers in making informed choices that enhance patients’ learning, this paper describes a randomized experiment on the effects of IA on the effectiveness, use, and user experience of a patient education website and examines the theoretical mechanisms that explain these effects.
Middle-aged and older adults with self-reported hip or knee joint complaints were recruited to use and evaluate 1 of 3 patient education websites containing information on total joint replacement surgery. Each website contained the same textual content based on an existing leaflet but differed in the employed IA design (tunnel, hierarchical, or matrix design). Participants rated the websites on satisfaction, engagement, control, relevance, trust, and novelty and completed an objective knowledge test. Analyses of variance and structural equation modeling were used to examine the effects of IA and construct a theoretical model.
We included 215 participants in our analysis. IA did not affect knowledge gain (
IA has small but notable effects on users’ experiences with web-based health education interventions. Web-based patient education designers can employ tunnel IA designs to guide users through sequentially ordered content or matrix IA to offer users more control over navigation. Both improve user satisfaction by increasing user perceptions of relevance (tunnel) and active control (matrix). Although additional research is needed, hierarchical IA designs are currently not recommended, as hierarchical content is perceived as less supportive, engaging, and relevant, which may diminish the use and, in turn, the effect of the educational intervention.
Verbal and written patient education methods are often supplemented with web-based education to improve patients’ ability to learn, remember, and apply health information. Such improvements are needed because patients’ recall of traditional education is generally poor [
There are many options to engage patients with web-based education, ranging from animations and interactive exercises to tailored health advice [
IA concerns “the structural design of a shared information environment” [
The tunnel IA design is the most common IA in health interventions: 90%-100% of interventions for chronic illness or mental health support include some form of tunneling [
Many scholars have condemned the
Conceptual model of information architecture (IA). Solid arrows represent expected effects related to matrix IA design, dashed arrows represent expected effects related to hierarchical IA design, and dotted arrows represent expected effects related to tunnel IA design.
First, we hypothesize that IA design affects user engagement. User engagement is defined as “a quality of user experience characterized by the depth of an actor’s investment when interacting with a digital system” [
Our expectations regarding IA design as a determinant of engagement are twofold. First, tunnel IA designs (in comparison with hierarchical and matrix IA) are thought to increase behavioral engagement because the sequential, predefined setup allows researchers to persuasively guide users through the web-based process, resulting in extended use [
As stated earlier, tunnel IA designs have been found to decrease user perceptions of control [
Perceived personal relevance refers to the extent to which people feel that information is relevant to themselves and their situation [
Perceived trust is a belief that influences whether a patient is willing to engage with health education [
Finally, we considered perceived novelty as a potential explanatory variable. As the tunnel IA design is the norm in health interventions, other IA designs may offer more novel ways to access health information. Novelty in the context of interfaces can “act as a curiosity generating mechanism that arouses the imaginations of users and captures their interest in a site” [
A final consideration in examining the effects of IA is the role of individual preferences and capabilities. Many recommendations regarding IA design take user characteristics into account. For example, Lynch and Horton [
This study used a previously defined set of user profiles of patients [
Description of communicative preferences and capabilities of three total joint replacement patient profilesa.
Managing profile | Optimistic profile | Modest profile |
High preference for open communication | Moderate preference for open communication | Moderate preference for open communication |
High preference for emotionally supportive communication | Low preference for emotionally supportive communication | Moderate preference for emotionally supportive communication |
High critical communication capabilities | Moderate critical communication capabilities | Low critical communication skills |
High personal communication capabilities | Moderate personal communication capabilities | Low personal communication skills |
High self-efficacy for health information | High self-efficacy for health information | Low self-efficacy for health information |
aPatient profiles are based on Groeneveld et al [
The aims of this study are threefold: (1) to test the effects of IA in the context of a TJR surgery patient education website on knowledge acquisition and satisfaction with web-based education; (2) to test possible working mechanisms of IAs, including user engagement, perceived user control, perceived personal relevance, perceived trust, and perceived novelty; and (3) to explore the potential of tailored IAs.
In July 2018, we conducted a between-subjects experiment comparing the knowledge and satisfaction gained from a patient education website with three different IA designs. Ethics approval for this study was obtained from the Human Research Ethics Committee Delft University of Technology. Participants provided written consent and signed a data processing agreement formulated in concordance with the General Data Protection Regulation.
Participants were recruited using a Dutch web-based consumer research service (respondenten.nl B.V.). Middle-aged to older adults (40-80 years) with self-reported chronic hip or knee joint complaints (including arthrosis, wear and tear, chronic inflammation, birth deficits, or unknown causes) were eligible for participation. To detect a small-to-medium effect (
The complete experiment was conducted on the web via survey software (Qualtrics). Each eligible participant was provided a hyperlink to the survey. After providing consent, participants filled out questionnaires regarding their communication preferences and skills, health, anxiety, and coping behavior, which were used to determine the patient profile [
The three websites were designed between March and June 2018 by a design agency (Panton B.V.) specializing in the design of products, services, and processes for health care under the supervision of the first author. The lead designer provided literature on IA [
Prototypes of the three websites were pilot tested with 7 patients (age range 46-77 years) scheduled for TJR surgery and 7 informal caregivers (age range 42-76 years) in June 2018. The pilot test focused specifically on usability of the websites rather than effectiveness in terms of knowledge acquisition. Interested patients present at the clinic for scheduled group-based patient education were shown the prototypes after they provided written consent. They first freely explored the websites while mentioning aloud any (positive or negative) aspects that stood out. Then, they were asked to find information about the first checkup after surgery. This assignment was used to identify usability issues and software bugs [
All websites contained the same textual content based on an existing patient education leaflet titled
The tunnel IA website design had a chronological sequential ordering of topics presented as a timeline, starting with
Annotated screenshots of tunnel, hierarchical, and matrix information architecture (IA) design of a Dutch patient education website to prepare patients for total joint replacement surgery. Tunnel IA: (A) next/previous buttons, (B) grayed-out text (not yet accessible), and (C) next/previous buttons. Hierarchical IA: (D) table of contents, (E) major grouping by recovery phase, and (F) return to main menu. Matrix IA: (G) topic matrix and (H) hyperlink. All screenshots depict the same content about pain and swelling (pijn en zwelling).
The primary outcomes of interest are knowledge acquisition and website satisfaction. Satisfaction with web-based education captures both the attitude of patients toward website functioning (eg, satisfaction with comprehensibility and with emotional support derived from the website) as well as their affective attitude (eg, satisfaction with website attractiveness) [
A total of 5 multiple-choice (MC) questions and 3 open questions about (self-)care after TJR surgery were used to assess knowledge acquisition. The questions were based on the information provided on the websites and included, for example:
Satisfaction with patient education was measured using the Website Satisfaction Scale [
We included 5 constructs to explore the theoretical mechanisms through which (tailored) IAs may influence knowledge acquisition and satisfaction. The first is user engagement, as measured through the UES-SF [
We conducted chi-square (χ2) and analyses of variance (ANOVA) tests to check whether background characteristics were evenly distributed over experimental conditions. To test the main effect of IA, 2 ANOVA tests were conducted with satisfaction and knowledge gain as dependent variables. Follow-up pairwise
To construct a conceptual model of how IA influences satisfaction and knowledge acquisition, we used structural equation modeling. User perceptions of engagement, personal relevance, active control, trust, and novelty (hereafter, mediating variables) were regressed on IA. Satisfaction and knowledge were regressed on IA and the mediating variables. To improve the parsimony and fit of the model, we removed nonsignificant paths. As our hypotheses suggest that IA design may influence perceived control and subsequently user engagement, and ultimately satisfaction and knowledge, we also constructed a separate serial mediation model for this hypothesis specifically. Model chi-square (χ2), comparative fit index (CFI), standardized root mean square residual (SRMR), and root mean square error of approximation (RMSEA) were used to determine model fit. A model was considered to have a good fit when
We enrolled 235 participants, of which, 215 participants were included in the analysis (
Participant recruitment and follow-up diagram. IA: information architecture.
Participant characteristics (N=215).
Variable | Value | |
Age (years), mean (SD)a | 57.18 (7.70) | |
|
||
|
Female | 155 (72.1) |
|
Male | 60 (27.9) |
|
||
|
Primary education | 3 (1.4) |
|
Lower secondary education | 95 (44.2) |
|
Higher secondary education | 36 (16.7) |
|
Tertiary education | 81 (37.7) |
|
||
|
Employed | 83 (38.6) |
|
Self-employed | 35 (16.3) |
|
Retired | 37 (17.2) |
|
Beneficiary | 29 (13.5) |
|
Other or none | 31 (14.4) |
|
||
|
Married or long-term relationship | 132 (61.4) |
|
Divorced | 41 (19.1) |
|
Never married | 35 (16.3) |
|
Widowed | 5 (2.3) |
|
Other | 2 (0.9) |
|
||
|
Partner | 124 (57.7) |
|
Friend | 75 (34.9) |
|
Child | 52 (24.2) |
|
Neighbor | 36 (16.7) |
|
Family member | 34 (15.8) |
|
Colleague | 7 (3.3) |
|
Group (church or sports) | 4 (1.9) |
|
Other | 2 (0.9) |
|
No support | 25 (11.6) |
Internet use in hours per day, mean (SD)c | 3.17 (2.14) | |
|
||
|
Personal computer or laptop | 194 (90.7) |
|
Phone | 175 (81.8) |
|
Tablet | 88 (41.1) |
Self-reported previous knowledge of hip replacement surgery, mean (SD)d | 1.85 (0.92) | |
|
||
|
Optimistic | 90 (41.9) |
|
Modest | 72 (33.5) |
|
Managing | 53 (24.7) |
aData were missing for 1 participant.
bParticipants could select multiple answers.
cData were missing for 10 participants.
dData were missing for 2 participants.
All three websites received predominantly positive feedback via the open qualitative feedback forms; participants appreciated that they were
Knowledge acquisition, satisfaction, and user perceptions of patient education website by information architecture.
Outcome, mean (SD) | Tunnel IAa (n=75) | Matrix IA (n=71) | Hierarchical IA (n=69) | (η2)b | ||
|
3.69 (0.52) | 3.65 (0.52) | 3.50 (0.48) | .07 | N/Ac | |
|
Attractiveness | 3.73 (0.61) | 3.68 (0.65) | 3.61 (0.61) | .50 | N/A |
|
Comprehension | 4.24 (0.56) | 4.21 (0.59) | 4.17 (0.71) | .79 | N/A |
|
Emotional support | 3.22 (0.67) | 3.17 (0.69) | 2.86 (0.60) | .002d | .057 |
Knowledge acquisition | 51.64 (19.55) | 48.02 (19.75) | 47.3 (19.63) | .36 | N/A | |
|
3.71 (0.55) | 3.65 (0.55) | 3.48 (0.57) | .047e | .028 | |
|
Focused attention | 3.16 (0.75) | 3.00 (0.70) | 2.85 (0.79) | .04f | .030 |
|
Esthetic appeal | 3.76 (0.68) | 3.75 (0.68) | 3.52 (0.76) | .08 | N/A |
|
Reward | 3.81 (0.62) | 3.78 (0.57) | 3.58 (0.68) | .06 | N/A |
|
Perceived usability | 4.08 (0.68) | 4.05 (0.78) | 3.98 (0.78) | .67 | N/A |
Perceived active control | 3.84 (0.67) | 3.95 (0.65) | 3.63 (0.74) | .02g | .035 | |
Perceived personal relevance | 3.08 (0.86) | 2.73 (0.83) | 2.64 (0.86) | .005h | .050 | |
Perceived trustworthiness | 3.94 (0.56) | 3.92 (0.57) | 3.78 (0.59) | .21 | N/A | |
Perceived novelty | 3.43 (0.75) | 3.46 (0.73) | 3.10 (0.76) | .007i | .046 | |
Time spent in minutes:seconds | 5:53 (4:24) | 5:18 (4:15) | 4:59 (4:09) | .44 | N/A |
aIA: information architecture.
bEffect size is only provided for significant differences.
cN/A: not applicable.
dHierarchical IA was significantly different from both tunnel IA (
eHierarchical IA was significantly different from tunnel IA (
fHierarchical IA was significantly different from tunnel IA (
gHierarchical IA was significantly different from matrix IA (
hTunnel IA was significantly different from both hierarchical IA (
iHierarchical IA was significantly different from both tunnel IA (
Main effects of information architecture.
The ANOVA tests demonstrated that the tunnel and matrix designs performed significantly better than the hierarchical IA design. To explain why tunnel and matrix IAs perform better compared with hierarchical IAs, we selected the hierarchical IA as the reference category in the mediation model.
The first mediation model (Model 1) specified that the effect of IA on knowledge and satisfaction would be mediated by user perceptions of engagement, active control, personal relevance, trust, and novelty. Specification of complete mediation results in a fully saturated regression model with zero degrees of freedom, as the number of observations is equal to the number of parameters [
Overall, models 2 to 4 all achieved similarly good fit (
Pathways included in mediation models 1, 2, 3, and 4.
Outcome and predictor or mediator | Path estimate (Model 1) | Model 2 | Model 3 | Model 4 | ||
|
||||||
|
Tunnel IAa | 0.190 | .02 | ✓b | ✓ | —c |
|
Matrix IA | 0.139 | .08 | ✓ | — | — |
|
||||||
|
Tunnel IA | 0.142 | .07 | ✓ | — | — |
|
Matrix IA | 0.215 | .006 | ✓ | ✓ | ✓ |
|
||||||
|
Tunnel IA | 0.243 | .002 | ✓ | ✓ | ✓ |
|
Matrix IA | 0.048 | .54 | — | — | — |
|
||||||
|
Tunnel IA | 0.133 | .09 | ✓ | — | — |
|
Matrix IA | 0.109 | .17 | — | — | — |
|
||||||
|
Tunnel IA | 0.208 | .007 | ✓ | — | — |
|
Matrix IA | 0.225 | .004 | ✓ | ✓ | — |
|
||||||
|
User engagement | 0.226 | .045 | ✓ | ✓ | ✓ |
|
Perceived active control | 0.006 | .96 | — | — | — |
|
Perceived personal relevance | 0.089 | .22 | — | — | — |
|
Trust | −0.007 | .93 | — | — | — |
|
Perceived novelty | −0.006 | .95 | — | — | — |
|
||||||
|
User engagement | 0.382 | <.001 | ✓ | ✓ | ✓ |
|
Perceived active control | 0.273 | <.001 | ✓ | ✓ | ✓ |
|
Perceived personal relevance | 0.169 | <.001 | ✓ | ✓ | ✓ |
|
Trust | 0.227 | <.001 | ✓ | ✓ | ✓ |
|
Perceived novelty | 0.026 | .60 | — | — | — |
|
||||||
|
Tunnel IA design | 0.042 | .59 | — | — | — |
|
Matrix IA design | −0.018 | .82 | — | — | — |
|
||||||
|
Tunnel IA design | −0.011 | .80 | — | — | — |
|
Matrix IA design | −0.017 | .68 | — | — | — |
|
||||||
|
User engagement×matrix IA | 0.031 | .19 | — | — | — |
|
Perceived novelty×matrix IA | −0.001 | .95 | — | — | — |
|
Trust×matrix IA | −0.001 | .93 | — | — | — |
|
Perceived personal relevance×matrix IA | 0.004 | .58 | — | — | — |
|
Perceived active control×matrix IA | 0.001 | .96 | — | — | — |
|
User engagement×tunnel IA | 0.043 | .12 | — | — | — |
|
Perceived novelty×tunnel IA | −0.001 | .95 | — | — | — |
|
Trust×tunnel IA | −0.001 | .93 | — | — | — |
|
Perceived personal relevance×tunnel IA | 0.022 | .25 | — | — | — |
|
Perceived active control×tunnel IA | 0.001 | .96 | — | — | — |
|
||||||
|
User engagement×tunnel IA | 0.073 | .02 | ✓ | ✓ | — |
|
Perceived active control×tunnel IA | 0.039 | .09 | ✓ | — | — |
|
Perceived personal relevance×tunnel IA | 0.041 | .01 | ✓ | ✓ | ✓ |
|
Trust×tunnel IA | 0.030 | .11 | — | — | — |
|
Perceived novelty×tunnel IA | 0.005 | .61 | — | — | — |
|
User engagement×matrix IA | 0.053 | .09 | ✓ | — | — |
|
Perceived active control×matrix IA | 0.059 | .02 | ✓ | ✓ | ✓ |
|
Perceived personal relevance×matrix IA | 0.008 | .54 | — | — | — |
|
Trust×matrix IA | 0.025 | .18 | — | — | — |
|
Perceived novelty×matrix IA | 0.006 | .61 | — | — | — |
aIA: information architecture.
bPathways indicated with a check mark were included in the model formulation.
cPathways indicated with an em dash were excluded in the model formulation.
Fit statistics of mediation models 2, 3, and 4.
Model | Chi-square (df) | χ2 divided by df | CFIa | SRMRb | RMSEAc | 95% CI | |
Model 2 | 4.7 (9) | .86 | 0.522 | 1 | 0.027 | 0.000 | 0.000-0.041 |
Model 3 | 10.8 (13) | .63 | 0.833 | 1 | 0.042 | 0.000 | 0.000-0.057 |
Model 4 | 14.7 (17) | .62 | 0.864 | 1 | 0.044 | 0.000 | 0.000-0.053 |
aCFI: comparative fit index.
bSRMR: standardized root mean square residual.
cRMSEA: root mean square error of approximation.
Structural equation model of the effects of information architecture.
The model explains the effect of IA as follows: compared with hierarchical IAs, health information presented in a tunnel IA is perceived as more personally relevant (β=.18). This subsequently increases user satisfaction (β=.17). Matrix IAs, in comparison with hierarchical IAs, significantly increase the active control users perceive to have over the health information (β=.11), which also increases satisfaction (β=.27). Furthermore, the model shows that next to user perceptions of personal relevance and active control, user engagement and perceived trust in the health information affect users’ satisfaction with a patient education website. Although we hypothesized that perceived novelty would also be affected by IA and affect satisfaction and knowledge in turn, this was not the case. Finally, we already established that IA design did not directly affect knowledge acquisition. The model demonstrated that IA also did not indirectly influence knowledge, as none of the tested mediation pathways were significant. Knowledge acquisition was influenced by user engagement (β=.26), but user engagement itself was unaffected by IA.
The serial mediation model, including perceived control and user engagement, confirmed that IA design did not significantly predict satisfaction (
Serial mediation model of matrix information architecture effects on satisfaction via active control and engagement. IA: information architecture.
Interaction effects between IA and patient profile indicated that some IA designs were preferred more by users with specific profiles (
Interaction effects between information architecture and patient profile.
The aim of this study is to investigate how the organization, display, and structural design of a website, that is, IA, influences patients’ experience with web-based patient education and the satisfaction and knowledge derived from the educational content. We wanted to understand whether user perceptions of engagement, control, personal relevance, trust, and novelty could explain how IA affects satisfaction and knowledge. Furthermore, we examined whether a user’s profile affected which IA design was most effective or enjoyable to explore the potential of tailored IA design. Research on IA in the context of web-based health education has been sparse [
This study compared three IA designs: tunnel, hierarchical, and matrix design. We found that in comparison with hierarchical IAs, tunnel and matrix IAs slightly improve user satisfaction. This effect may be explained by increased user perceptions of personal relevance in the tunnel IA and increased perceptions of control in the matrix IA. Contrary to our hypotheses and earlier findings [
Our finding that tunnel IA design specifically affects satisfaction with emotional support is consistent with research showing that tunneled education improved the emotional well-being of patients with type 2 diabetes and chronic low back pain [
The results of IA design on user engagement were mixed; the matrix IA achieved the highest subjective (ie, self-reported) engagement, but the tunnel IA was used the longest (albeit, not significantly longer). This reflects the dichotomous nature of engagement raised in the introduction, where engagement is thought to include both a subjective component of immersion and a behavioral component of use [
Finally, this study focused on three simple IA designs for experimental clarity. Hybrid IAs that combine design elements from different IAs could mitigate the disadvantages associated with nonhybrid IAs. As users were most satisfied with matrix and tunnel IAs, hybrid matrix-tunnel designs should be explored further specifically. This study also identified that a large proportion of older adults with self-reported joint complaints use mobile phones (82%) and tablet devices (41%). As web-based IA designs cannot be ported to smartphones [
A secondary objective of this study is to explore the potential of tailored IAs. We found that participants with the highest information needs (so-called
In any case, the incongruence between anticipated and actual match of patient profile and IA design indicates that translating stated preferences to a tailored design is complex. Although the knowledge base on what works for whom is growing slowly, it may be more beneficial in the meantime to offer users a choice of IAs rather than dictating one design. Studies that explored the benefit of tailoring the mode of health information (eg, text, illustrations, audio-visual material) have successfully used
This study was conducted among adults who had self-reported joint complaints and may have viewed web-based education differently than patients scheduled for TJR surgery. However, previous studies have successfully tested health education websites in similar populations [
Another limitation was self-selection, as participants were able to determine whether they wanted to join or leave the study. Between invitation for participation and inclusion in the study, 37% of participants were lost to follow-up. Of particular concern is that 6% of the sample dropped out after viewing the allocated website, as they might have done so based on their (negative) response to the website. This could make the study susceptible to type I errors [
The strengths of this study include the experimental design. Although randomized experiments of website features known as
Overall, our findings indicate that IA has small but notable effects on users’ experiences with web-based health education interventions, at least in the context of orthopedic patient education. Tunnel IA design, in which users are guided through sequentially ordered content, improves perceptions of personal relevance and, in turn, user satisfaction. This design may be specifically appropriate for patients with high information needs. In contrast, providing users with more control over the way they progress through a web-based health intervention via a matrix IA design has positive effects on user perceptions of active control, which also contributes to higher satisfaction. Although additional research on IA design in different target groups and interventions is needed, hierarchical IA designs are not recommended at the moment, as hierarchical content is perceived as less supportive, engaging, and relevant, which may diminish the use and, in turn, the effect of the educational intervention.
Dutch translation of the User Engagement Scale-Short Form: validity, questionnaire items, and instructions for scoring.
Perceived advantages and disadvantages of tunnel, hierarchical, and matrix information architecture designs (translated from Dutch).
CONSORT-eHEALTH checklist (V 1.6.1).
analyses of variance
comparative fit index
information architecture
multiple choice
root mean square error of approximation
recommender system
standardized root mean square residual
total hip prosthesis
total joint replacement
User Engagement Scale-Short Form
This work is part of the research program Tailored health care through customer profiling (Project 314-99-118), which is financed by the Netherlands Organisation for Scientific Research (NWO) and Zimmer Biomet Inc. The authors thank the participants who took part in the pilot study and the consortium partners and reviewers for their constructive feedback on this work.
Part of the funding of this project is provided by Zimmer Biomet Inc (refer to the Acknowledgments section). This sponsor had no role in the study design of this protocol, data collection, analysis and interpretation, or writing of the report. In the case that this partner wants to apply for a patent based on research findings, publication can be postponed for a maximum of 3 months. No party has the right to prohibit the publication of these findings. The authors have full access to the study data.