Published on 12.07.17 in Vol 19, No 7 (2017): July
Self-Directed Telehealth Parent-Mediated Intervention for Children With Autism Spectrum Disorder: Examination of the Potential Reach and Utilization in Community Settings
Background: There is a significant need for strategies to increase access to evidence-based interventions for children with autism spectrum disorder (ASD). One novel approach is to train parents to use evidence-based interventions for their child with ASD via telehealth. Pilot work examining the efficacy of one such program, ImPACT Online, demonstrated a high rate of parent program engagement, low attrition, and associated gains in parent learning and child social communication.
Objective: The objective of this study was to conduct an open trial of ImPACT Online to better understand its dissemination potential.
Methods: We examined the reach and representativeness of families who registered (n=36) compared to families who were referred (n=139) to the open trial for one referral site. We then compared the demographics of all families who enrolled in the open trial (n=112) to families who enrolled in one of two controlled trials of the same program (n=50). We also examined metrics of program engagement for the open and controlled trials, the relationship between program engagement and changes in parents’ intervention knowledge, and program evaluation for the participants in the open trial.
Results: In total, 25.8% (36/139) of the parents who were given information about the program at their child’s diagnostic feedback session registered with the program. The parents who enrolled in the open (OT) and controlled trials (CT), respectively, were similar in gender (OT: 84.8% (95/112); CT: 88% (44/50), female), marital status (OT: 80.4% (90/112) ; CT: 69.6% (32/46), married), education (OT: 58.0% (65/112); CT: 54.0% (27/50), college degree or higher), and employment status (OT: 58.0% (65/112); CT: 65.3% (32/49), employed outside the home). The child participants were similar in terms of gender (OT: 83.0% (93/112); CT: 76.0% (38/50), male) and race and ethnicity (OT: 38.4% (43/112); CT: 24.0% (12/50), minority). However, the mean chronological age of the child participants in the open trial group was significantly higher (Mean=60.0 months) than in the controlled trial group (Mean=43.0 months), with t160=5.22, P<.001. Parents in the open trial engaged with the program at a significantly lower rate than the controlled trial, F3,81=21.14, P<.001. Program engagement was significantly associated with gains in parent intervention knowledge across both the groups, beta=.41, t=2.43, P=.02. Participants in the open access trial evaluated the program highly, but several barriers were noted.
Conclusions: These data suggest that additional strategies may need to be developed to support families in using telehealth-based parent-mediated intervention in community settings.
J Med Internet Res 2017;19(7):e248
Autism spectrum disorder (ASD) is a chronic and pervasive neurodevelopmental disorder characterized by deficits in social communication and the presence of restricted and repetitive behaviors . Individuals with ASD often require intensive and comprehensive intervention across the life span [ ]. There has been a dramatic increase in the number of individuals diagnosed with ASD over the last two decades, with prevalence rates reaching 1 in 88 [ ]. However, there has not been corresponding growth in the availability of evidence-based services, contributing to high levels of unmet service needs for individuals with ASD and their families [ ]. These issues highlight the need for systematic research focused on developing and improving strategies for dissemination and implementation of evidence-based ASD services.
Parent-mediated intervention is one cost-effective and ecologically valid way to increase access to evidence-based ASD intervention. Numerous studies have established that parents can be successfully taught to use evidence-based strategies to improve their child’s social-communicative functioning . Additional benefits of parent-mediated intervention include increase in generalization and maintenance of child skill, a reduction in parent stress, and an increase in family leisure time [ , ]. Yet, there continue to be barriers involved with the dissemination of parent-mediated intervention, including a shortage of trained professionals, limited financial resources and transportation, lack of child care, geographic isolation, lengthy waitlists, and extensive time commitments [ ]. Thus, it is essential to consider the adaptation of evidence-based, parent-mediated interventions for non-traditional service delivery methods [ ]. Telehealth and technology-based applications have the potential to augment or even replace traditional service models to increase access to evidence-based services [ ]. Self-directed telehealth programs can provide a cost-effective means for intervention to be accessed from anywhere at any time [ ]. They can deliver highly standardized instruction with fidelity, while also supporting individualized learning [ ]. The use of self-directed telehealth programs to provide instruction in evidence-based interventions has been explored across health-related disciplines, disorders, and treatment approaches with promising outcomes [ ]. Taken together, these data suggest that telehealth applications may serve as a promising alternative service-delivery model to increase the reach of evidence-based ASD practices, including parent-mediated intervention [ ].
Although there has been growing interest in using telehealth to deliver parent-mediated intervention for children with ASD, empirical evaluations of such programs are limited. Several recent studies have demonstrated the initial efficacy of evidence-based, parent-mediated autism interventions when delivered by computer or over the internet, with or without therapist assistance [- ]. One such program, ImPACT Online, is an interactive website that teaches parents to promote their child’s social communication within the context of play and daily routines [ , , ]. The content was modified from Project ImPACT, an evidence-based social communication curriculum for young children with ASD [ ]. ImPACT Online can be delivered as a self-directed intervention or with therapist assistance via videoconferencing. A recent pilot randomized controlled efficacy trial comparing the two formats demonstrated positive effects on parent learning and child social communication skills for both groups, although there was an added benefit of therapist assistance on some parent and child outcomes [ , ]. Although engagement with the program website was significantly higher among parents in the therapist-assisted group, parents in the self-directed group engaged with the program at a relatively high rate, with 69% completing the program [ ]. There was also a significant positive relationship between program completion and improvements in parent intervention knowledge and fidelity, independent of therapist assistance, suggesting that the use of the self-directed website contributed to parent learning.
Self-directed telehealth interventions do not require a trained professional and can typically be administered at a much reduced cost. Thus, they have a strong dissemination potential  and may provide an effective method for increasing access to intervention for underserved families of children with ASD. At the same time, much is unknown regarding the potential impact such interventions may have should they prove efficacious in larger trials. For example, it is currently unclear whether families who participate in controlled trials of telehealth parent-mediated interventions are representative of families who would use these interventions in the community. In addition, little is known about the reach of such programs in the community or the representativeness of the community families who choose to enroll. Research on self-directed telehealth programs for other conditions suggests that they may have a more limited reach than anticipated. For example, only 5% of health plan participants recruited by personal letter enrolled in a treatment trial of a Web-based weight loss program [ ]. Further, those who enrolled in a Web-based weight loss program through their workplace were more likely to have higher household incomes, education, and health literacy than those who did not enroll, indicating disparities in access [ ].
In addition, it is not clear whether the high rates of program engagement observed in our pilot trial of ImPACT Online would be observed in participants who openly enroll in the program. Attrition rates during open trials of self-directed telehealth interventions are often much higher than those found in controlled trials of the same intervention, in which potential participants are closely screened, monitored, and “pushed” to continue engaging with the program by the research staff . For example, Christensen et al [ ] found that less than 1% of individuals who enrolled in an open trial of MoodGym, a self-directed CBT-based telehealth program for depression, completed all five modules, compared with nearly 23% in their controlled trial; this represents a 450-fold increase in attrition among community participants.
Highly efficacious programs that have a limited reach or low quality implementation are not likely to have a significant impact on public health . Therefore, it is important to evaluate these factors, alongside more traditional evaluations of program efficacy, to understand the potential impact that telehealth-based, parent-mediated intervention can have on access to care for underserved families of children with ASD [ ]. This information will be critical to understanding the likelihood that this type of intervention will be able to reach the families for whom it is designed as well as identifying the required program modifications and supports necessary to facilitate its use and ultimate impact.
This study conducted an open trial of ImPACT Online to better understand the dissemination potential of a self-directed, telehealth-based, parent-mediated intervention for families of children with ASD. Specifically, we examined (1) the reach and representativeness of a subset of families who enrolled in the open trial, (2) the demographics of families who enrolled in the open trial compared to families who enrolled in one of our two controlled trials of the same program, 3) metrics of program engagement for the open and controlled trials, (4) the relationship between program engagement and changes in parents’ intervention knowledge, and (5) program evaluation for the open trial.
Participants in the open trial group (n=112) were parents of a child with ASD between the ages of 27 and 152 months who self-enrolled through the ImPACT Online website. To be included in the analyses, the participant had to identify as the child’s mother, father, or step-parent, live in the United States, be proficient in English, and complete the intake questionnaires. In addition, the child had to score above the cut-off on the Modified Checklist for Autism in Toddlers (M-CHAT; under 36 months)  or the Social Communication Questionnaire (SCQ; 36 months or older) [ ]. An additional 101 registrants were excluded because they or their child did not meet one or more of these inclusion criteria.
Participants in the controlled trial group (n=50) were parents of a child with ASD between the ages of 19 and 73 months who had enrolled in the pilot (n=27) or full-scale efficacy trial (n=23, recruitment ongoing) of ImPACT Online. To participate, the participant had to identify as the child’s mother, father, or step-parent, live within 3 hours of the research lab, and be proficient in English. Inclusion criteria for the child included being between the ages of 18 and 95 months at intake, meeting criteria for ASD on the Autism Diagnostic Observation Schedule-2nd Edition (ADOS-2) , and having no known medical cause of ASD. presents participant characteristics by group.
|Open trial||Controlled trial||Test statistics||P value|
|Gender: female, n (%)||95 (84.8%)||44 (88.0%)||χ21 (N=162)=0.3||.59|
|Education level: college degree, n (%)||65 (58.0%)||27 (54.0%)||χ21 (N=162)=0.2||.63|
|Marital status: married, n (%)||90 (80.4%)||32 (69.6%)||χ21 (N=158)=2.2||.14|
|Employment status: employed, n (%)||65 (58.0%)||32 (65.3%)||χ21 (N=161)=0.8||.39|
|Internet literacy, mean (SD)||36.7 (3.5)||37.6 (3.9)||t156=-1.49||.14|
|Gender: male, n (%)||93 (83.0%)||38 (76.0%)||χ21 (N=162)=1.1||.29|
|Race/Ethnicity: minority, n (%)||43 (38.4%)||12 (24.0%)||χ21 (N=162)=3.2||.07|
|Chronological age in months, mean (SD)||60.0 (21.1)||43.0 (13.7)||t160=5.22||<.001|
|Mean intervention in hours / week (SD)||6.9 (9.2)||10.7 (10.1)||t97=-1.95||.05|
Open Trial Group
Information about the open trial was disseminated via flyers given to families by professionals at community organizations serving children with ASD, websites providing information about ASD, and an Internet search. Community organizations who expressed interest were given recruitment materials with a unique site code which participants entered at the time of program registration to track referrals. Recruitment materials provided a link to the ImPACT Online website. The website described the content of the program, system requirements, research requirements, and allowed visitors to view a brief video demonstration of the program. Participants had to indicate their consent to participate in research before registering. After registration, participants were asked to complete several short pre-treatment questionnaires and were then given access to the program. 6 months after registration, participants were sent an email asking them to complete the post-treatment questionnaires. Non-responders were sent a follow-up email one week, and two weeks later, until a total of three emails were sent. Participants received a US $25 gift certificate for the completion of the post-treatment questionnaires. Throughout their participation, participants in the open trial received no individual contact with research staff.
Controlled Trial Group
Information about the pilot and subsequent full-scale efficacy trials was disseminated via community providers to families within 3 hours of the research lab. Interested parents were directed to contact the research lab to learn more about the specific study. Consent and intake assessments were conducted in person at the research lab and in the family’s home. All participants were provided a computer and high speed internet, if needed.
After intake assessments, pilot study participants were randomly assigned to a self-directed (n=13) or therapist-assisted group (n=14). Participants in the full-scale study were randomly assigned to a self-directed (n=6), therapist-assisted (n=8), or informational control group (n=9). Participants in the self-directed groups for both studies received periodic individual phone and email contact from project staff regarding research-related tasks throughout their participation, but they received no support in learning the intervention. Participants in the therapist-assisted groups for both studies received access to the ImPACT Online website in addition to twice-weekly coaching via videoconferencing to help them learn the intervention. Participants in the informational control group received access to an informational website and monthly support calls. Participants completed post-treatment questionnaires and assessments after roughly 6 months, and again at a 3-month follow-up. Participants received a US $25 gift certificate for the completion of assessments at pre- and post-treatment, and follow-up.
ImPACT Online is a secure, password-protected, interactive Web application. The content was adapted from Project ImPACT , a naturalistic, developmental behavioral intervention (NDBI) [ ] that teaches parents to promote their child’s social communication during play and daily routines. The instructional content was presented in 12, self-directed lessons, each of which took approximately 75 minutes to complete. Each lesson contained a narrated slideshow (average length=27.2 minutes), written manual, self-check questions, video-based exercises, homework plan, and reflection questions. Parents were advised to practice the intervention with their child for one week following each lesson. Parents could also access supplemental material outside of the lessons, including a video library that contained longer video examples of the intervention, a moderated forum, and additional program handouts and informational resources. Parents received automated weekly emails that provided tips for implementing the intervention techniques along with a link to the program to encourage program use.
One community organization, a university-based diagnostic center, tracked the number of referrals made to the program (recruitment flyers distributed during the diagnostic feedback session). The organization also tracked the age of the child in months, the child’s gender, and the family’s zip code. Zip codes were used to derive a median household income using US Census data for the referral group. This information was then compared with the same data provided by families who enrolled in the program using the referral site’s unique access code to calculate the potential reach of the program and the representativeness of the participants.
Participants were asked to provide information on their gender, education level, employment status, marital status, and zip code. They were also asked to provide information on their child’s gender, age, diagnostic status, the number of hours per week, and the type of intervention their child received.
Participants completed a brief, modified version of the Computer-Email-Web Fluency Scale  at pre-treatment to assess their level of comfort using the computer and Internet. Items such as (1) how frequently do you conduct a search using an Internet search engine, and (2) how frequently do you send or receive email, were rated on a 5-point scale from never (1) to daily (5). Total scores on the measure could range from 8 to 42, with higher scores indicative of greater comfort with using the Internet.
Program engagement was measured via ImPACT Online’s electronic tracking of user behavior. Four metrics of program engagement were calculated: (1) average number of logins to the site, (2) average number of minutes spent on the site across the intervention period, (3) percent of learning activities visited at least once across the 12 lessons (out of a possible 71), and (4) program completion defined by having visited 75% or more of the learning activities. We also examined the percent of participants completing each lesson to determine whether there was a drop-off in program engagement associated with a particular lesson. Completion was defined as having clicked on 75% or more of the learning activities for a lesson at least once.
Parent Intervention Knowledge
Participants completed the ImPACT Knowledge Quiz, a 20-item, multiple-choice quiz, that assesses comprehension of the key elements of ImPACT at intake and approximately 6 months later to measure changes in their intervention knowledge.
Participants in the open trial were asked to complete several measures to evaluate their experience with the program and barriers to program engagement 6 months after registering with the site. The Behavioral Intervention Rating Scale (BIRS) is a well-validated measure that asks individuals to endorse items that assess the acceptability of a treatment’s procedures and its perceived effectiveness on a 6-point scale, ranging from 1 (highly disagree) to 3 (neutral) to 6 (highly agree) . The BIRS was modified to better reflect the goals of the current intervention (acquisition of social-communication skills). Cronbach alpha for the BIRS was .97. The Website Usability Scale is a 10-item questionnaire developed for this project [ ]. Participants rate the ease of use of the website and the perceived helpfulness of each program component on a 6-point scale, with higher scores indicating greater usability. Cronbach alpha for the website usability scale was .96. The Barriers to Treatment Participation Scale is a 44-item, well-validated measure of common barriers to participation in child outpatient treatment [ ] . Questions are answered on a scale of 1 (“Never a problem”) to 5 (“Very often a problem”), with higher scores indicative of a greater number of perceived barriers. Items were modified to reflect potential barriers associated with the use of a self-directed, telehealth-based parent training intervention (as opposed to clinician-led, clinic-based therapy), resulting in the removal of 17 items assessing therapist- and clinic-related barriers, and the addition of 5 items assessing technology-related barriers. The modified scale contained 32 items and had a Cronbach alpha of .87. Participants were also asked to respond to 3 open-ended questions: (1) Please indicate any benefits of this program, (2) Please indicate any limitations of this program, and (3) Please provide any recommendations for improving this program.
The data were examined for normality by inspecting the skewness coefficient. Number of logins and the amount of time spent on the site were both positively skewed; a square root transformation was used to normalize their distributions. Intervention knowledge at post-treatment was negatively skewed; an arcsine transformation was used to normalize intervention knowledge at pre and post-treatment. All other data were normally distributed.
We first examined the reach of ImPACT Online for one referral site by comparing the number of parents who registered for ImPACT Online with the number of parents who were provided a referral from a professional at the time of diagnosis at that site (n=139). A total of 36 parents (25.8%) who were given information about ImPACT Online at the time of their child’s diagnosis registered with the site. Child gender, age, and median household incomes per year were compared for parents who registered (participant sample) and the referral sample. No significant differences were found between the two groups regarding gender distribution (Referral sample: 73.8% male; Participant sample: 71.4% male), χ21 (N = 161) =.1, P=.78, child age in months (Referral sample: mean=41.8, SD=12.5; Participant sample: mean=41.7, SD=11.6), t34=−.03, P=.98, or median income (Referral sample: mean=$48,379, SD=$13,685; Participant sample: mean=$52,528, SD=$16,730), t35=1.49, P=.15).
Next, we compared the demographics of families who enrolled in the open trial (OT; n=112) with the controlled trial studies (CT; n=50), regardless of whether they had completed the 6-month follow-up period or their group assignment in the lab-based trials. As shown in, parent participants in both groups were primarily female (OT: 84.8% (95/112); CT: 88% (44/50)), married (OT: 80.4% (90/112); CT: 69.6% (32/46)), slightly over half in each group had a college degree or higher (OT: 58.0% (65/112); CT: 54.0% (27/50)), and were employed full or part time (OT: 58.0% (65/112); CT: 65.3% (32/49)). Scores on the CEWFS ranged from 25 to 42, with average scores toward the upper end of the scale for both groups (OT: mean=36.7, SD=3.5; CT: mean=37.6, SD=3.9). Chi-square analyses and independent sample t tests revealed no significant group differences on any of these parent demographic variables.
The majority of child participants in both groups were male (OT: 83.0% (93/112); CT: 76.0% (38/50)), and rates did not significantly differ between groups. The children’s mean chronological age was significantly higher in the open trial (mean=60.0, SD=21.1) than the controlled trial (mean=43.0, SD=13.7), t160=5.22, P<.001, likely due to the age restriction of the controlled trials. Children in the open trial were marginally more likely to be a racial or ethnic minority than the lab-based trials (OT: 38.4% (43/112); CT: 24.0% (12/50)), χ21, (n=162) =3.2, P=.07, which could reflect the general demographics of the recruitment area for the controlled trials.
Participants in both groups reported that their children received a variety of different intervention services (eg, special education, ABA, speech therapy, occupational therapy, play groups), with intensities ranging from 0 to 41 hours per week. However, children in the open trial (mean=6.9, SD=9.2) received marginally fewer hours per week of intervention than children in the controlled trials (mean=10.7, SD=10.1), t97=−1.95, P=.05.
Next, we examined program engagement metrics for the subset of participants in the open trial who had completed the 6-month follow-up period (n=94). A number of participants who enrolled in the open trial did not access the program after completing the pre-treatment assessments. Thus, we only included data from participants who accessed the first lesson (n=68, 61% of sample). In our previous work, therapist assistance was shown to enhance parent engagement in the program . Thus, in order for the data to be comparable across studies, only the participants in the controlled trials who were assigned to the self-directed group and had completed the follow-up were included (n=17).
We examined group differences on the first three metrics of program engagement using MANOVA (Multivariate analysis of variance). There was a significant effect of trial type on program engagement, F3, 81=21.14, P<.001; Wilk’s Λ=.53, ƞ=.47. As observed in, the participants in the open trial engaged significantly less with the program on all of these metrics than the controlled trials. In addition, only 12% of participants in the open trial completed the program compared with 88% of participants the controlled trials, which represents a significant group difference, χ21 (N=85)=40.3, P<.001.
|Open trial||Controlled trial |
|Test statistic||P value|
|Number of logins, mean (SD)||9.9 (8.5)||29.8 (13.3)||t83=−6.95b||<.001|
|Amount of time on site, mean (SD)||188.1 (197.1)||701.3 (371.0)||t83=−7.09b||<.001|
|% Learning activities completed, mean (SD)||26.3 (27.2)||88.6 (26.0)||t83=−8.55||<.001|
|% Participants completing program||12%||88%||χ21 (N=85)=40.3||<.001|
banalysis used transformed data.
To better understand the rate of attrition, we examined the percent of participants who completed each lesson for the two groups. As observed in, attrition in the open trial group occurred early in the program, with majority of participants discontinuing the program by the second lesson. In contrast, participants in the controlled trials maintained a high rate of engagement across the lessons.
We examined the degree to which program engagement was predictive of gains in parent intervention knowledge. We included those participants in the open trial (n=26, 38%) and the self-directed groups of controlled trials (n=16, 94%) who had completed the post-treatment questionnaire. For this analysis, we used a hierarchical linear regression, with parents’ intervention knowledge at the 6-month follow-up as the dependent variable. Parents’ pre-treatment intervention knowledge and trial type were entered in the first step. Program engagement, defined as percent of learning activities completed, was entered in the second step. After controlling for pre-treatment intervention knowledge and trial type, program engagement was a significant predictor of post-treatment intervention knowledge, beta=.41, t=2.43, P=.02, and explained additional variance in post-treatment intervention knowledge (R2 change=.10, F=5.88, P=.02). This finding suggests that regardless of the trial type, participants who completed more of the program made greater improvements in their intervention knowledge from pre- to post-treatment.
Finally, we examined the program evaluation measures for the participants in the open trial (n=26). Participants reported the program as highly acceptability on the BIRS (mean=4.6, SD=1.0) and the website as highly usable (mean=4.9, SD=1.0). Parents also reported low levels of perceived barriers on the modified BTPS (mean=1.9, SD=.5). To understand the most common perceived barriers to using this program better, we examined the means for individual items on the modified BTPS. The three barrier items with the highest means (means ≥2.5) were: (1) “I did not have the time to practice (complete the homework)” (mean=3.0, SD=1.2); (2) “During the course of treatment, I experienced a lot of stress” (mean=2.9, SD=1.4); and (3) “Crises at home made it hard for me to complete the program” (mean=2.8, SD=1.3).
The open-ended responses were analyzed by compiling responses and identifying emergent themes using inductive content analysis. Two authors worked independently to develop codes based on the written responses. They then met to discuss initial codes and develop consensus regarding final codes. Any conflicts were discussed until consensus was reached. The most frequent themes identified in response to the program’s benefits, limitations, and recommendations are presented in.
In this study, we conducted an open trial of the self-directed format of ImPACT Online to understand its dissemination and implementation potential better. Our reach data indicate that only a quarter of parents who were provided information about the program at the time of their child’s diagnosis registered with the site. This finding is somewhat sobering given that parents in our pilot study reported that they would have liked to have received information about the program at the time of their child’s initial ASD diagnosis . The fact that such a low percentage of families enrolled in the program may suggest that the diagnostic feedback session may not be the optimal time to provide a referral to this type of program. Parents of children with ASD consistently report that the diagnostic process is extremely stressful [ ] and, while in retrospect, immediate access to parent-mediated intervention may sound attractive, at the time it may be perceived as too overwhelming. At the same time, the reach of telehealth interventions in general is poorly understood [ ] and studies of eHealth interventions for other conditions, such as weight loss, have found much lower reach for their interventions, even among patients at high-risk for negative health outcomes [ ]. So perhaps, the enrollment rate that we observed is, in fact, promising; however, without additional research in this area, it is difficult to know.
Our reach data were obtained from a specialty diagnostic clinic that provided a referral to the program at the time of diagnosis. It is unclear what the reach of this program would be in other communities or means of dissemination, such as advertisements, web-links, personal invitation via mail or email, or personal recommendation provided by an intervention provider. Different methods of dissemination may be expected to lead to different levels of enrollment as well as ongoing engagement with the program. Additional research that can examine differences in referral methods and the optimal times to refer parents to such programs would be helpful.
Although the data we were able to collect about families before they enrolled in the open trial were very limited, what we did collect suggested that those families who enrolled in the program were not significantly different from the referral sample in terms of median household income, child gender, or age. Although limited by sample size, this finding provides some indication that the families who enrolled in the open trial were fairly representative of the referral group. In addition, our demographic information suggests that, for the most part, participants who enrolled in our controlled trials were very similar to families who enrolled in the open trial. Although these data are, again, limited due to sample size constraints, they provide some confidence that the data being generated from our ongoing efficacy study may well generalize to community users. At the same time, there were few families with very low socioeconomic status in any of our samples, and thus, participation was not representative of the full range of families who could potentially benefit from this type of intervention. Therefore, future research should focus on increasing enrollment of parents from low socioeconomic status backgrounds into both open and controlled trials of telehealth-based, parent-mediated intervention.
Our program engagement data suggested much lower rates of engagement among our open trial participants than the participants in our controlled trials. This finding is not surprising given the previous research on telehealth-based interventions for other conditions. Participants in our controlled trials received a significant amount of personal contact with the research team throughout their participation, which may have affected their motivation to engage with the program. Although program engagement was much lower among participants in the open trial than the controlled trial, it was still higher than many open trials of similar interventions [- ]. This may suggest a higher level of motivation among parents of children with ASD than individuals who are pursuing health promotion or mental health services for themselves.
It has been proposed that program completion may not be necessary to achieve significant improvements, in part because some individuals who discontinue eHealth interventions do so because they improve . Inspection of the percent of participants completing each lesson suggests that there was an early decline in lesson completion among participants in the open trial. This is particularly problematic given that the first three lessons are focused on background information and the primary intervention content is not presented until lesson four, at which point fewer than 30% of participants completed the lesson. Thus, it appears unlikely that early attrition is due to improvements in this case. Although our data were limited due to a high percent of participants lost to follow-up (dropout attrition), we also found an association between the amount of program content accessed and increase in parent intervention knowledge, suggesting that that program engagement is an important factor in learning. This finding suggests a possible need to retool the content of the program to introduce intervention content earlier or for the use of additional strategies, particularly during the first few lessons, to boost engagement in real world contexts.
Our program evaluation data for the open trial participants suggested high levels of the treatment acceptability and website usability and low levels of perceived barriers. Our qualitative data complemented these findings, suggesting that parents identified a number of benefits of the program, including the accessibility of the program, and the ease of learning the intervention, the acceptability of the intervention strategies, and improvement in their child’s social communication skills. These themes have also been identified among participants in our controlled trial .
The most consistently reported barriers on the modified BTPS and open-ended questions were related to competing stressors at home and difficulty finding time to complete the program. These barriers might be expected to adversely impact parent participation in all forms child treatment, not just self-directed, telehealth-based parent-mediated intervention . Our qualitative data also indicated a need for therapist support, simplification of some of the training components, and a desire for greater online community support. Therapist support was also identified as important theme among participants in our controlled trial [ ]. However, simplification of training components and increased online community support were not reported in our controlled trial, suggesting that these factors may need to be considered in more detail, especially as self-directed programs are moved into community use.
The literature suggests a number of program design factors that may impact engagement in self-directed telehealth interventions, such as ImPACT Online . First, the narrated slideshows, which were used to present the intervention content, were 27.2 minutes long, on an average. Guo and colleagues [ ] have suggested that videos in an online learning environment should be no longer than 6 minutes. Although participants could stop and resume the videos at any point, breaking up the narrated slideshows may make the videos more engaging or easier to access in shorter increments. Likewise, ImPACT Online includes 12 lessons which are designed to be completed over the course of 3-6 months. This represents a significant time investment. It has been suggested that shorter, simpler interventions are better suited to online delivery than the more complex ones that require hours of online work [ ]. Research identifying active ingredients of the ImPACT intervention could inform the development of a “leaner” intervention with fewer components which may be less prone to attrition.
Second, the Pew Research Center states that people are increasingly turning to their mobile phones to access the Internet . Thus, making the website mobile compliant or developing an app component for those with a mobile phone would allow parents to access the lessons in any location. In addition, the use of personalized feedback or tailored messaging could be used to provide more individualized messages and content to the parent [ ]. Another possibility would be to add some elements of gamification. Gamification uses elements of games in non-game contexts to support and engage individuals to complete tasks within a learning environment. These typically have been found to improve user engagement, learning, and ease of use [ ]. There are myriad ways that this can be achieved, including giving points for users interacting with the content, achieving badges, and being included on a team. Past research has suggested that gamification can increase knowledge in health domains, self-efficacy, and motivation which can improve a myriad of health outcomes [ ].
Finally, online communities may help improve peoples’ use of health content and interventions . ImPACT Online included a moderated asynchronous forum; however, it was rarely used. It has been suggested that this type of online community may require a minimum threshold of activity to support ongoing site utilization [ ]. Indeed, this was a reported limitation of the program. Currently, there are a multitude of online support groups available online, especially for parents of children with special needs [ ]. Thus, it may be more practical to utilize currently available platforms. For instance, a closed and secret Facebook group is easier to develop, maintain and engage parents [ ]. In addition to the parents being able to provide social support and exchange information, it is another way moderators can redirect parents back into the ImPACT Online website and content.
Future research that can evaluate which strategies are most likely to increase parent engagement in ImPACT Online and similar interventions is needed. In addition, as we did not collect data on other aspects of parent learning such as procedural fidelity, parent well-being, or child gains, future research should investigate how this program impacts important parent and child outcomes in real world settings.
This project was supported by Idea Development Award #W81XWH-10-1-0586 from the Department of Defense’s Autism Research Program, grant #R40MC277040100 from the Maternal and Child Health Bureau of the Health Resources & Services Administration, and a RAIND grant from Michigan State University. We would like to thank the parents and children who partipated in this work.
Conflicts of Interest
- American Psychiatric Association , editor. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Arlington, VA: American Psychiatric Association; 2013.
- Maglione MA, Gans D, Das L, Timbie J, Kasari C, Technical Expert Panel, HRSA Autism Intervention Research – Behavioral (AIR-B) Network. Nonmedical interventions for children with ASD: recommended guidelines and further research needs. Pediatrics 2012 Nov;130(Suppl 2):S169-S178 [FREE Full text] [CrossRef] [Medline]
- Baio J. Prevalence of autism spectrum disorders: autism and developmental disabilities monitoring network, 14 Sites, United States, 2008. MMWR Surveill Summ 2012 Mar 30;61(3):1-19.
- Bethell CD, Lindly O, Kogan M, Zuckerman K. Health Care Quality for Children: The Position of Autism Spectrum Disorder. In: Patel VB, Preedy VR, Martin CR, editors. Comprehensive Guide to Autism. New York: Springer Science+Business Media; 2014:139-165.
- Oono IP, Honey EJ, McConachie H. Parent-mediated early intervention for young children with autism spectrum disorders (ASD). Cochrane Database Syst Rev 2013 Apr 30(4):CD009774. [CrossRef] [Medline]
- Koegel RL, Schreibman L, Britten KR, Burke JC, O'Neill RE. Comparison of parent-training to direct child treatment. In: Koegel RL, Rincover A, Egel AL, editors. Educating and Understanding Autistic Children. San Diego, CA: College-Hill Press; 1982:260-279.
- Tonge B, Brereton A, Kiomall M, Mackinnon A, King N, Rinehart N. Effects on parental mental health of an education and skills training program for parents of young children with autism: a randomized controlled trial. J Am Acad Child Adolesc Psychiatry 2006 May;45(5):561-569. [CrossRef] [Medline]
- Taylor TK, Webster-Stratton C, Feil EG, Broadbent B, Widdop CS, Severson HH. Computer-based intervention with coaching: an example using the Incredible Years program. Cogn Behav Ther 2008;37(4):233-246 [FREE Full text] [CrossRef] [Medline]
- Feil EG, Baggett KM, Davis B, Sheeber L, Landry S, Carta JJ, et al. Expanding the reach of preventive interventions: development of an Internet-based training for parents of infants. Child Maltreat 2008 Nov;13(4):334-346 [FREE Full text] [CrossRef] [Medline]
- Baggett KM, Davis B, Feil EG, Sheeber LB, Landry SH, Carta JJ, et al. Technologies for expanding the reach of evidence-based interventions: preliminary results for promoting social-emotional development in early childhood. Topics Early Child Spec Educ 2010 Feb 01;29(4):226-238 [FREE Full text] [CrossRef] [Medline]
- Hollon SD, Muñoz RF, Barlow DH, Beardslee WR, Bell CC, Bernal G, et al. Psychosocial intervention development for the prevention and treatment of depression: promoting innovation and increasing access. Biol Psychiatry 2002 Sep 15;52(6):610-630. [Medline]
- Webb TL, Joseph J, Yardley L, Michie S. Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res 2010 Feb 17;12(1):e4 [FREE Full text] [CrossRef] [Medline]
- Knutsen J, Wolfe A, Burke BL, Hepburn S, Lindgren S, Coury D. A systematic review of telemedicine in autism spectrum disorders. Rev J Autism Dev Disord 2016 Dec;3(4):330-344. [CrossRef]
- Ingersoll B, Wainer AL, Berger NI, Pickard KE, Bonter N. Comparison of a self-directed and therapist-assisted telehealth parent-mediated intervention for children with ASD: a pilot RCT. J Autism Dev Disord 2016 Jul;46(7):2275-2284. [CrossRef] [Medline]
- Vismara LA, McCormick CEB, Wagner AL, Monlux K, Nadhan A, Young GS. Telehealth parent training in the early start Denver model: results from a randomized controlled study. Focus Autism Other Dev Disabl 2016 May 26:1088357616651064. [CrossRef]
- Nefdt N, Koegel R, Singer G, Gerber M. The use of a self-directed learning program to provide introductory training in pivotal response treatment to parents of children with autism. J Posit Behav Interv 2009 May 07;12(1):23-32. [CrossRef]
- Wainer AL, Ingersoll BR. Increasing access to an ASD imitation intervention via a telehealth parent training program. J Autism Dev Disord 2015 Dec;45(12):3877-3890. [CrossRef] [Medline]
- Pickard KE, Wainer AL, Bailey KM, Ingersoll BR. A mixed-method evaluation of the feasibility and acceptability of a telehealth-based parent-mediated intervention for children with autism spectrum disorder. Autism 2016 Oct;20(7):845-855. [CrossRef] [Medline]
- Ingersoll B, Berger NI. Parent engagement with a telehealth-based parent-mediated intervention program for children with autism spectrum disorders: predictors of program use and parent outcomes. J Med Internet Res 2015 Oct 06;17(10):e227 [FREE Full text] [CrossRef] [Medline]
- Ingersoll B, Dvortcsak A. Teaching Social Communication to Children with Autism: A Practitioner's Guide to Parent Training and a Manual for Parents. New York: Guilford Press; 2010.
- Bennett GG, Glasgow RE. The delivery of public health interventions via the Internet: actualizing their potential. Annu Rev Public Health 2009;30:273-292. [CrossRef] [Medline]
- Glasgow RE, Nelson CC, Kearney KA, Reid R, Ritzwoller DP, Strecher VJ, et al. Reach, engagement, and retention in an Internet-based weight loss program in a multi-site randomized controlled trial. J Med Internet Res 2007 May 09;9(2):e11 [FREE Full text] [CrossRef] [Medline]
- You W, Almeida FA, Zoellner JM, Hill JL, Pinard CA, Allen KC, et al. Who participates in internet-based worksite weight loss programs? BMC Public Health 2011 Sep 20;11:709 [FREE Full text] [CrossRef] [Medline]
- Christensen H, Griffiths KM, Korten AE, Brittliffe K, Groves C. A comparison of changes in anxiety and depression symptoms of spontaneous users and trial participants of a cognitive behavior therapy website. J Med Internet Res 2004 Dec 22;6(4):e46 [FREE Full text] [CrossRef] [Medline]
- Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999 Sep;89(9):1322-1327. [Medline]
- Robins DL, Fein D, Barton ML, Green JA. The modified checklist for autism in toddlers: an initial study investigating the early detection of autism and pervasive developmental disorders. J Autism Dev Disord 2001 Apr;31(2):131-144. [Medline]
- Berument SK, Rutter M, Lord C, Pickles A, Bailey A. Autism screening questionnaire: diagnostic validity. Br J Psychiatry 1999 Nov;175:444-451. [Medline]
- Lord C, Rutter M, DiLavore PC, Risi S, Gotham K, Bishop SL, et al. Autism Diagnostic Observation Schedule, Second Edition: ADOS-2. Los Angeles, CA: Western Psychological Services; 2012.
- Schreibman L, Dawson G, Stahmer AC, Landa R, Rogers SJ, McGee GG, et al. Naturalistic developmental behavioral interventions: empirically validated treatments for autism spectrum disorder. J Autism Dev Disord 2015 Aug;45(8):2411-2428 [FREE Full text] [CrossRef] [Medline]
- Bunz U. The computer-email-web (CEW) fluency scale-development and validation. Int J Hum Comput Interact 2004;17(4):479-506.
- Elliott S, Treuting M. The behavior intervention rating scale: development and validation of a pretreatment acceptability and effectiveness measure. J Sch Psychol 1991;29(1):43-51.
- Kazdin AE, Holland L, Crowley M, Breton S. Barriers to treatment participation scale: evaluation and validation in the context of child outpatient treatment. J Child Psychol Psychiatry 1997 Nov;38(8):1051-1062. [Medline]
- Siklos S, Kerns KA. Assessing the diagnostic experiences of a small sample of parents of children with autism spectrum disorders. Res Dev Disabil 2007;28(1):9-22. [CrossRef] [Medline]
- Glasgow RE. eHealth evaluation and dissemination research. Am J Prev Med 2007 May;32(5 Suppl):S119-S126. [CrossRef] [Medline]
- Christensen H, Mackinnon A. The law of attrition revisited. J Med Internet Res 2006 Sep 29;8(3):e20; author reply e21 [FREE Full text] [CrossRef] [Medline]
- Farvolden P, Denisoff E, Selby P, Bagby RM, Rudy L. Usage and longitudinal effectiveness of a Web-based self-help cognitive behavioral therapy program for panic disorder. J Med Internet Res 2005 Mar 26;7(1):e7 [FREE Full text] [CrossRef] [Medline]
- Wanner M, Martin-Diener E, Bauer G, Braun-Fahrländer C, Martin BW. Comparison of trial participants and open access users of a web-based physical activity intervention regarding adherence, attrition, and repeated participation. J Med Internet Res 2010 Feb 10;12(1):e3 [FREE Full text] [CrossRef] [Medline]
- Eysenbach G. The law of attrition. J Med Internet Res 2005 Mar 31;7(1):e11 [FREE Full text] [CrossRef] [Medline]
- Smith T, Linnemeyer R, Scalise D, Hamilton J. Barriers to outpatient mental health treatment for children and adolescents: parental perspectives. J Fam Psychother 2013;24(2):73-92. [CrossRef]
- Kelders SM, Kok RN, Ossebaard HC, Van Gemert-Pijnen JE. Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res 2012 Nov 14;14(6):e152 [FREE Full text] [CrossRef] [Medline]
- Guo PJ, Kim J, Rubin R. How video production affects student engagement: An empirical study of MOOC videos. 2014 Presented at: ACM Conference on Learning at Scale; March 04-05, 2014; Atlanta, Georgia, USA p. 41-50 URL: http://dl.acm.org/citation.cfm?id=2566239
- Christensen H, Griffiths K, Groves C, Korten A. Free range users and one hit wonders: community users of an Internet-based cognitive behaviour therapy program. Aust N Z J Psychiatry 2006 Jan;40(1):59-62. [CrossRef] [Medline]
- Greenwood S, Perrin A, Duggan M. Pew Internet. 2016 Nov 11. Social media update 2016 URL: http://www.pewinternet.org/2016/11/11/social-media-update-2016/ [accessed 2017-06-05] [WebCite Cache]
- Wangberg SC, Bergmo TS, Johnsen JA. Adherence in Internet-based interventions. Patient Prefer Adherence 2008 Feb 02;2:57-65 [FREE Full text] [Medline]
- Cugelman B. Gamification: what it is and why it matters to digital health behavior change developers. JMIR Serious Games 2013 Dec 12;1(1):e3 [FREE Full text] [CrossRef] [Medline]
- Theng YL, Lee JW, Patinadan PV, Foo SS. The use of videogames, gamification, and virtual environments in the self-management of diabetes: a systematic review of evidence. Games Health J 2015 Oct;4(5):352-361. [CrossRef] [Medline]
- An LC, Schillo BA, Saul JE, Wendling AH, Klatt CM, Berg CJ, et al. Utilization of smoking cessation informational, interactive, and online community resources as predictors of abstinence: cohort study. J Med Internet Res 2008 Dec 20;10(5):e55 [FREE Full text] [CrossRef] [Medline]
- Stoddard JL, Augustson EM, Moser RP. Effect of adding a virtual community (bulletin board) to smokefree.gov: randomized controlled trial. J Med Internet Res 2008 Dec 19;10(5):e53 [FREE Full text] [CrossRef] [Medline]
- Ammari T, Schoenebeck S, Morris M. Accessing social support and overcoming judgment on social media among parents of children with special needs. In: Proceedings of ICWSM 2014. 2014 Jun 01 Presented at: ICWSM; 2014 Jun 1; Ann Arbor, MI URL: http://www-personal.umich.edu/~tawfiqam/Ammari_SpecialNeedsParents_ICWSM14.pdf
- Holtz B, Smock A, Reyes-Gastelum D. Connected motherhood: social support for moms and moms-to-be on Facebook. Telemed J E Health 2015 May;21(5):415-421. [CrossRef] [Medline]
Edited by G Eysenbach; submitted 10.02.17; peer-reviewed by B Bie, Y Mejova; comments to author 12.04.17; revised version received 04.05.17; accepted 05.05.17; published 12.07.17
©Brooke Ingersoll, Katherine Shannon, Natalie Berger, Katherine Pickard, Bree Holtz. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 12.07.2017.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.