Adolescent/parent dyads were recruited from 2 large rheumatology clinics in university-affiliated pediatric tertiary care centers. Dyads were eligible to participate if (1) the adolescents were 12 to 18 years of age, (2) the adolescent had been diagnosed with JIA by a rheumatologist, and (3) both adolescent and parent were able to speak and read English or French as their primary language. Adolescents were excluded if they had major cognitive impairments or comorbid medical or psychiatric illnesses that may have impacted their ability to understand and use the Internet program. Eligible patients listed on the participating hospitals’ rheumatology database registries were sent a study information letter inviting them to participate. This method was supplemented by convenience sampling of patients attending a regularly scheduled clinic visit to achieve a heterogeneous sample in terms of age and disease severity. The study was approved by research ethics boards at both sites.

A qualitative usability testing approach with semistructured, audio taped interviews and observation by a trained observer was undertaken, with iterative cycles to determine the usability of the Internet intervention and to further refine the prototype [16,17]. This design approach was based on the concept of a ‘‘hermeneutical circle’’ as described by Snodgrass and Coyne [19], which is an iterative process of implementing a design, learning and understanding from discussion and feedback, and making subsequent design refinements. This iterative rapid design development approach concentrated on testing the user-interface with a focus on user performance and satisfaction.

After giving informed consent, participants were asked to complete a general information questionnaire that included questions about demographic characteristics and level of comfort with computers and the Internet. A health information questionnaire was completed by a research assistant to gather additional demographic and disease-related data from the patient’s chart (duration of illness, disease subtype and the rheumatologist’s global assessment of the patient’s disease severity rated on 10-cm visual analogue scale) [20]. Adolescents and their parents then participated in separate 30- to 45-minute semistructured usability testing interviews in a quiet room with computer and Internet access. Participants were given a brief explanation of the online program by the research assistant. The sessions were audio taped and participants were asked to “think aloud” as they worked through a “live” version of the program. Participants were guided to move through a standardized list of features in each of the 12 modules, which included content, graphics, video clips, interactive components, animations, and the discussion forum, while commenting on any difficulties they had. It was anticipated that participants would be able to move through all of these tasks during the usability interview (ie, within 40 minutes). The research assistant also recorded the length of the sessions and made field notes about any problems encountered during the sessions (eg, tracking participant navigation errors and program errors) as well as ease of use and learning. At the end of the session, participants were asked a series of questions using a semistructured interview guide to ascertain user satisfaction and suggestions for improving the program (See Multimedia Appendix 1). Interviews were conducted in two iterative cycles, one cycle per site. Following the first iterative cycle with English-speaking participants, changes were made to the user interface screens based on problems identified from the content analysis of the taped interviews and field notes. The revised user interface was then evaluated in a second iterative cycle with French-speaking participants.

The Internet program is a 12-module interactive multi-component treatment protocol that consists of JIA-specific education (eg, common problems associated with treatment and disease), self-management strategies (eg, how to deal with pain and stress), and social support (eg, monitored discussion boards and narratives in the form of written stories and video clips of adolescents with JIA). There are two modules specifically for parents to help them promote healthy teens’ behaviors. The program content was developed by a team of experts from across Canada, written at a grade 6 to7 reading level, and geared to the needs identified by adolescents and their parents [14]. The Teens Taking Charge: Managing Arthritis Online treatment program is composed of 310 content pages (approximately 156459 words), flash files (animations), images, videos, forums (separate adolescent and parent discussion boards, surveys, and interactive forms (eg, weekly knowledge quizzes). The entire program was translated into French using forward and backward translation as well as being reviewed by Francophone medical experts. The video clips of teens for the French site were recorded with Francophone adolescents and their parents in Montreal to ensure they were culturally competent.

In the website architecture, four applications are integrated together to provide the experience of a single website to the users. These applications include (1) the content application for both English and French, which is delivered by the AboutKidsHealth.ca content management system; (2) the forums application, which provides a framework for creating and managing discussion forums for the users; (3) the survey application, which provides support for creating, managing, and gathering survey questions and responses; and (4) the custom application, which provides management of interactive questions. Each of the applications has its own database model where data are managed and stored. Security and restriction of the program are based on the ASP.NET membership model and ensure that only authenticated users with the allowed permission (ie, role) are able to view the allowed content and to participate in the private forums. The membership model works across several applications and accommodates a single sign-on process so users do not have to sign in to every application and thus can move across the applications seamlessly. Users get the best experience of the Teens Taking Charge: Managing Arthritis Online treatment program with Internet Explorer 6/7 and Flash player 9+ on a Windows XP/Vista platform. The given computer requirements ensure that users are able to view the animation and video streams within the site and as well view the embedded interactive forms, which are delivered via an AJAX interface that captures the responses to a database. The AJAX interface reduces the number of clicks and pages to keep the user experience intuitive. Please see Multimedia Appendix 2 for an overview of the Teens Taking Charge: Managing Arthritis Online progam.

The quantitative data from the questionnaires was analyzed using SAS version 9.1.3 [21] to determine measures of central tendency and the distribution of values. Interviews were audio taped and transcribed verbatim. The French interviews were transcribed directly into English by two bilingual transcriptionists. Field notes taken during the interviews were also transcribed and included in the analytic process. Simple content analysis was performed after each iterative cycle, with categories emerging form the usability research questions (ie, learnability, efficiency, errors, and satisfaction) and frequencies calculated [22]. Initially, the English (iterative cycle 1) and French (iterative cycle 2) interviews were analyzed separately. Minor modifications to the website were made based on the first iterative cycle of testing and were subsequently evaluated in the second cycle of testing. No further modifications to the prototype were made based on the second iterative cycle of testing. The data from these two cycles were similar and provided a strong source of triangulation for the developing themes. To ensure anonymity, all participants were identified by pseudonyms, and quotes were identified by participant type (adolescent = A; parent = P) [23].

Patients meeting the study criteria who attended clinics from August 22 through October 2, 2008, were invited to participate in this study at the two sites. Of the 20 eligible English-speaking dyads, 11 (55%) agreed to participate, and of the 23 French-speaking dyads, 8 (39%) agreed to participate. The mean age of the sample of adolescents was 15.7 years (SD 1.5), and 14 (74%) were female. Of the 19 adolescents, 15 (79 %) were in high school. The mean duration of illness was 6.99 years (SD 5.03, range 0.3 - 15.2 years). The most common disease onset subtype was polyarticular arthritis with 7 of the 19 adolescents (37%) having this diagnosis; the average disease severity rating was 0.92 out of 10 (SD 1.5, range 0 - 4.9). Of the participating adolescents, 100% had a computer at home, and 100% were comfortable using a computer. These findings are in accordance with recent data that 88% of 15-year-old Canadian students have at least one computer at home and are comfortable using the computer/Internet [24]. Thirteen of the 19 parents/caregivers were female (63%), 10 of the 19 parents/caregivers (53%) were aged 40 to 49 years, and 9 (47%) responded that their highest level of education was high school. The numbers, means, and standard deviations for demographic and disease-related characteristics of the adolescent sample by usability testing phase are shown in Table 1. Participants’ comfort level with and use of computers and the Internet are described in Table 2.

User performance was measured in terms of (1) ease at which the user could navigate through the site, (2) ease of learning in terms of how fast a user who has never seen the user interface before could learn it sufficiently well to accomplish basic tasks, and (3) the frequency and severity of errors. Overall the participants rated the Teens Taking Charge: Managing Arthritis Online health portal as easy to use. All of the adolescents were able to navigate through the site with little to no guidance, whereas 5 of the 19 parents (27%) required several minutes to orientate themselves to the site before confidently navigating the interface. In terms of ease of learning, all participants were able to accomplish the standardized set of basic tasks in 35 minutes or less, as was predicted.

During the usability testing, three types of errors were observed: navigation errors, presentation errors, and control usage problems. Navigation errors were defined as failures to locate functions, excessive keystrokes, or failures to follow recommended screen flow. There were minimal navigation errors with 10% of adolescents and 21% of parents experiencing one. All navigation errors encountered were easily recovered from and were considered to be minor. No changes were made based on these findings. Presentation errors were defined as failures to locate and properly act upon desired information or selection errors due to labeling ambiguities. These were found to be concentrated in the medication module. Presentation errors were encountered by 58% of parents and 26% of adolescents on the homepage of the medication module, which were coded as fatal errors. Users found the labeling on the medication homepage by classification of medication type to be unclear and were unable to navigate to the information they wanted about their/their child’s particular arthritis medication(s). This page was changed by adding a list of generic medications grouped by their drug classification with direct links to each medication discussed in that section to enhance user performance (See Figures 1 and 2 for before and after screen shots of medication homepage). Following this change, this presentation error did not occur in iterative cycle 2 testing with French-speaking participants. Control usage errors were defined as improper tool bar or entry field usage. In iterative cycle 1, 42% of adolescents and 26% of parents were unable to identify controls to stop, start, or advance the functional features on the “stress and thinking animation” and video clips at the start of each module, which were coded as fatal errors. These were changed and retested in iterative cycle 2 with no further control usage errors arising. See Figure 3 for an example of the “stress and thinking” animation controls following usability testing. There were no differences between French and English participants in the types of errors found.

In terms of satisfaction with the Teens Taking Charge: Managing Arthritis Online health portal, adolescents and their parents perceived that their satisfaction with the site was based on the overall design aesthetics of the user interface, content, functionality and features, and opportunities for social support. There were minor differences within these categories by participant (adolescent versus parent/caregiver) and language.

The overall design aesthetics of the website were felt to be a critical factor from the perspective of the end users that helped to enhance engagement and motivation to use the site. Four subthemes were identified under the broad theme of design aesthetics including layout, navigation, visual assets, and visual appeal. Differences were found between both adolescents and parents and French and English participants. Adolescents wanted the large amounts of text “chunked up” and broken up with visual assets (graphics, animations, illustrations), while parents did not mind having large amounts of text to scroll through. Similarly, 42% of parents indicated that they would like a larger font; however, only 15% of adolescents cited this as a problem. Furthermore, 16% of both French adolescents and parents suggested adding labels to the medical diagrams, where this was not an issue with English participants. Finally, all the participants felt that the most important information on each page should be at the top of the page, otherwise teens will not scroll down to find it. See Table 3 for a summary of issues and changes made to the design aesthetics.

All of the participants commented positively on the content of the program including the text, images, and multimedia components. Four subthemes were identified under the broad theme of content including completeness, understandability, quality and credibility, and relevance. For the most part, participants were satisfied with the completeness of content; however, 21% of adolescents and 32% of parents suggested additional content. The most common suggestion for additional content was for videos or photos to show teens how to do the various exercises. However, due to budgetary and time constraints we were not able to add these. Additionally, 89% of adolescents and 95% of parents felt that the content was relevant, and 74% of adolescents and 79% of parents rated the content as trustworthy. The site was created with a focus on understandability and all text was developed at a grade 6 to7 reading level. Participants expressed appreciation for this consideration and generally found the information, language level, and explanation of medical terminology (in a glossary of words) to be helpful in furthering their understanding of topics that were either unclear, new to them, or both. French grammar errors were found by 20% of French adolescents and 5% of French parents. These errors were corrected following usability testing. See Table 4 for a summary of comments and changes made with regard to website content.

Functionality and features refers to the adaptive and interactive features of the website. These included a glossary of terms, relaxation audio clips, printable PDF information forms for teachers, click-through animations (ie, how we feel pain, how antiinflammatory medications work, stress, and thinking), videos of teens with arthritis, interactive questions (weekly knowledge quizzes), a journal for keeping track of symptoms and weekly goals, and a discussion board with an “Ask an Expert” feature where participants could send an email question to a designated health care professional. Participants unanimously felt that these adaptive and interactive features allowed for an increased level of personalization of the website content to meet the individual needs of the users and were all good features of the website. They also commented that these features would help to enhance motivation and engagement with the program.

Sociability refers to the ability of the Internet program to support social interactions among participants. The site featured 2 discussion boards, one for teens and one for parents, video clips of teens talking about ways they have learned to manage their arthritis, in addition to written stories of hope submitted by teens with JIA. All of the participants commented that these features of the site would help to ease feelings of hopelessness and particularly isolation. However, while almost all teens (95%) indicated that they would like to use the discussion forum to chat with other teens; less than one-half (42%) of parents felt they would use a parent discussion forum. Many teens commented that they did not know another teen with arthritis and that the interactive features (discussion board, stories of hope, video clips of teens with JIA) made them feel supported and “not alone” in their illness.

Overall, the program was well received, and 84% of all participants expressed a desire to use the program in the future. Most participants felt the program would have been especially helpful when they were first diagnosed with JIA. Adolescents and caregivers also liked that the site was geared toward the promotion of self-management skills to reduce symptoms, improve quality of life, and promote transition to adult health care services.