Background: There is a lack of effectiveness studies when digital parent training programs are implemented in real-world practice. The efficacy of the internet-based and telephone-assisted Finnish Strongest Families Smart Website (SFSW) parent training intervention on the disruptive behavior of 4-year-old children was studied in a randomized controlled trial setting in Southwest Finland between 2011 and 2013. After that, the intervention was implemented nationwide in child health clinics from 2015 onwards.
Objective: The main aim of this study was to compare the treatment characteristics and effectiveness of the SFSW parent training intervention between the families who received the intervention when it was implemented as a normal practice in child health clinics and the families who received the same intervention during the randomized controlled trial.
Methods: The implementation group comprised 600 families who were recruited in the SFSW intervention between January 2015 and May 2017 in real-world implementation. The RCT intervention group comprised 232 families who were recruited between October 2011 and November 2013. The same demographic and child and parent measures were collected from both study groups and were compared using linear mixed-effect models for repeated measurements. The child psychopathology and functioning level were measured using the Child Behavior Checklist (CBCL) version 1.5-5 for preschool children, the Inventory of Callous-Unemotional Traits (ICU), and a modified version of the Barkley Home Situations Questionnaire. Parenting skills were measured using the 31-item Parenting Scale and the shorter 21-item Depression, Anxiety and Stress Scale (DASS-21). The estimated child and parent outcomes were adjusted for CBCL externalizing scores at baseline, maternal education, duration of the behavior problems, and paternal age. The baseline measurements of each outcome were used as covariates.
Results: The implementation group was more likely to complete the intervention than the RCT intervention group (514/600, 85.7% vs 176/232, 75.9%, respectively; P<.001). There were no significant differences between the implementation and RCT intervention groups with regard to child measures, including CBCL externalizing score (–0.2, 95% CI –1.3 to 1.6; P=.83), total score (–0.7, 95% CI –3.0 to 4.5; P=.70), internalizing score (–0.3, 95% CI –1.0 to 1.6; P=.64), and ICU total score (–0.4, 95% Cl –1.9 to 1.2; P=.64). No significant difference was detected in the Parenting Scale total score (0.0, 95% Cl –0.1 to 0.1; P=.50), while DASS-21 total score differed nearly significantly (2.5, 95% Cl 0.0-5.1; P=.05), indicating better improvement in the implementation group.
Conclusions: The internet-based and telephone-assisted SFSW parent training intervention was effectively implemented in real-world settings. These findings have implications for addressing the unmet needs of children with disruptive behavior problems. Our initiative could also provide a quick socially distanced solution for the considerable mental health impact of the COVID-19 pandemic.
Trial Registration: ClinicalTrials.gov NCT01750996; https://clinicaltrials.gov/ct2/show/NCT01750996
International Registered Report Identifier (IRRID): RR2-10.1186/1471-2458-13-985
There is mounting evidence from randomized controlled trials (RCTs) that parents can be trained to tackle and reduce children’s disruptive behavior and improve their parenting skills [- ]. These findings are of upmost importance to public health professionals because children who exhibit disruptive behavior face increased risks of adult psychiatric disorders, substance use, crime, suicide, and other adversities [ , ]. Sufficiently strong evidence has been published on the efficacy of parent training to suggest that psychosocial services for children should include evidence-based parent training programs [ , ]. The need for services to tackle childhood disruptive behavior is enormous, but only a minority of families receive them [ ]. There are challenges to implementing traditional face-to-face group-based parent training programs in real-world settings. One issue is the large number of barriers such as high cost, poor access, inconvenience, and low fidelity [ , ]. Another is keeping the content of the intervention consistent with the original evidence-based treatment [ ].
Digitally assisted interventions are becoming more common, as they can overcome the barriers associated with conventional programs [, ]. They are also likely to become increasingly popular, as child mental health services struggle to deal with the considerable increase in demand for their services as a result of the COVID-19 pandemic. This unprecedented global health emergency is expected to have major ongoing effects on child mental health owing to factors such as quarantine measures, social distancing, and school closures [ ]. The pandemic started at a time when resources were already under pressure, and these are expected to be further affected by manpower shortages and a global recession that puts even greater pressure on health budgets. Digitally assisted interventions are cost-effective solutions that require fewer personnel and can reach geographically remote areas that would otherwise be outside of the reach of specialist services.
RCT studies have shown that remote and digitally assisted parent training programs have worked well in clinical settings [, ]. We previously reported 12-month and 24-month follow-up studies of the first RCT on the Strongest Families Smart Website (SFSW). This RCT used a population-based sample and provided an internet-based parent training intervention with weekly telephone coaching [ - ]. The development of the SFSW intervention was based on the social learning and cognitive behavioral theories as well as positive parenting practices [ - ]. The target population was 4-year-old children who displayed high levels of disruptive behavior when they were screened during annual health checkups at child health clinics across Southwest Finland. The RCT showed that the children and parents who received the SFSW parent training program derived significant benefits from the initiative. The children displayed significant reductions in their disruptive behavior and other psychiatric symptom domains at their 24-month follow-up assessments. They also demonstrated the same improvements when they were compared with an education control group. The education control group received access to a static website that provided parents with information on how to tackle behavior problems and 1 phone call with a coach. Improved parenting skills were maintained in the intervention group at the 24-month follow-up assessment [ ].
There has been growing interest in implementation research during the past 2 decades. Dissemination refers to how knowledge of new practices is actively and passively extended, and implementation refers to how new practices are incorporated into real-world environments. The term implementation gap is used to refer to the difference between our knowledge of what works and how it works [, ]. Unfortunately, the strong effects that are observed in controlled RCT settings can weaken or become ambiguous when they are implemented in real-world settings [ ]. Meta-analyses have shown that effective implementation has been associated with better outcomes, and the magnitude of the mean effect sizes was considerably higher when programs were carefully implemented and when fidelity was confirmed [ ]. Successfully converting psychosocial interventions from experimental environments to real-world practice requires a solid framework and a structured implementation plan [ ]. Research on evidence-based parent training programs after the RCT stage has often focused on examining the characteristics of an optimal implementation environment rather than maintaining the effectiveness of the intervention. We are not aware of any previous reports on the effectiveness of implementing digital interventions for disruptive behavior so that they can form part of the routine care that children below school age can receive.
This was the first study to report the effectiveness of the SFSW internet-based and telephone-assisted parent training program for preschool children when it was implemented in real-world settings. The intervention was put into practice after the population-based screening was used to identify children with disruptive behavior problems during routine visits to Finnish child health clinics at the age of 4 years. The primary aim was to report the changes in the children’s psychopathology and functioning level and any improvement in their families’ parenting skills. The children and their parents were followed up 6 months after the SFSW intervention was nationally implemented in Finnish primary care child health clinics. We compared the treatment characteristics and effectiveness between the families who received the SFSW intervention in these real-world settings from January 2015 to May 2017 and the families who received the intervention during the RCT from October 2011 to November 2013. Finally, we verified the findings by carrying out the following additional analyses. The first analysis excluded families who did not complete the parent training program. The second analysis excluded the Turku study site from the implementation study group because it was the only site that participated in both the RCT intervention and the implementation phases. In the third analysis, we compared the implementation and the RCT education control group. Our hypothesis was that the effectiveness of the SFSW intervention would be maintained if the protocol used in our previous RCT and the structured implementation plan were strictly adhered to.
This study was a longitudinal comparison of 2 parallel groups. The implementation group comprised 600 families who received the SFSW internet-based and telephone-assisted parent training program in the real-world setting between January 2015 and May 2017. The implementation phase covered 95 child health clinics in 12 administrative regions across Finland. The RCT intervention group comprised 232 families who had been recruited by 42 child health clinics in 7 administrative regions in Southwest Finland between October 2011 and November 2013. The administrative regions in both the RCT and implementation studies contained both urban and rural areas. Turku was the only region that participated in both studies.
There were both differences and similarities between the implementation and the RCT intervention studies. First, the implementation group received the intervention when it was integrated as a normal practice of the child health clinics, and therefore, all families who met the inclusion criteria were eligible to enter. In the implementation phase, both participants and the health care workers received information that the SFSW parent training intervention has been evaluated as an intervention with strong documented effects by the Finnish national evaluation and classification system for evidence-based interventions . This evaluation was partly based on the results of our previous RCT study [ , ]. In contrast, in the RCT, the intervention was not integrated as a normal practice of the child health clinics. Only those families who were randomized to the intervention group received the intervention. Second, in the implementation phase, an implementation plan, including decision supporting and administration component, was followed. This was important because the implementation phase included increasing number of communities in the whole Finland while the RCT was conducted in a predetermined area of Southwest Finland. Third, the most important similarity was that the content of the SFSW intervention was maintained in the implementation group as identical as possible with that of the original RCT intervention. In both groups, the same psychopathology and parenting measures were collected at baseline and 6-month follow-up. Data on children’s daily activities were collected only for the implementation group immediately after the intervention and at the 6-month follow-up. The timeline of the RCT and implementation studies is shown in Figure S1 of . The study protocol of the RCT has previously been published [ ] and registered at ClinicalTrials.gov (NCT01750996).
This study focused on the 6-month follow-up assessments of children who displayed a high level of disruptive behavior when they were screened at 4 years of age during routine child health clinic visits. The screening procedure in the implementation study followed the same principles that were used in the RCT study. It was integrated into the standard 4-year-old child health checkups carried out by the child health clinics in the participating administrative regions . All children living in Finland are invited to annual health assessments before they start school at 7 years of age, and attendance rates are just under 100% [ ].
In the implementation group, the first 600 eligible parents who agreed to take part in the program received the SFSW parent training intervention. Initially, 8866 children were screened for highly disruptive behavior and 1099 (12.4%) met the screening criteria. The implementation group equated to 6.8% (600/8866) of the initial population-based sample and 54.6% (600/1099) of those who were eligible to take part. The reference group consisted of 232 families who were randomized to receive the intervention during the previous RCT study [, ]. Information was obtained from 427 (71.2%) of the 600 families in the implementation group at the 6-month follow-up assessments compared to 184 (79.3%) of the 232 families in the RCT intervention group. shows the flowchart of the implementation and RCT intervention groups. The families were typically recruited within 1 month of the child’s fourth birthday. They received a study information pack and were asked to bring the completed health questionnaire to the clinic.
Inclusion and Exclusion Criteria
The screening measures and enrollment criteria were identical for the implementation and RCT studies . Population-based screening for behavior problems was conducted for all children at the age of 4 years by using the conduct scale of the Strengths and Difficulties Questionnaire [ - ]. The parents were asked if their child had mild, moderate, or severe problems through a single question: “Overall, do you think that your child has difficulties in one or more of the following areas: emotions, behavior, or being able to get on with other people?” About 16.5% of the children (16.7% [1477/8866] and 16.3% [758/4656] in the implementation and RCT groups, respectively) who were screened, scored 5 or more out of 10 corresponding to the 80th percentile cutoff point and reported that the child had difficulties. This indicated a high level of behavior problems. The other inclusion criteria were that the parents perceived that child had at least minor difficulties in emotions, behavior, or social interactions. To participate in the study, the family had to live in an administrative region participating in the study, at least one parent had to speak native Finnish or Swedish, and they needed access to a telephone, computer, and internet connection. We excluded children who were unable to speak in full sentences, had hearing or vision impairments, or were receiving or had received behavior treatment. The exclusion criteria also included children who had been diagnosed with autism, Down syndrome, fetal alcohol syndrome, an intellectual disability, a severe mental disorder such as psychosis or depression, or who had a genetic diagnosis of mental retardation. We also excluded parents whose children did not live with them because they were subject to child protection services owing to child custody, abuse, or neglect issues. Details of the inclusion and exclusion criteria have previously been reported [ ].
The participants in the implementation and RCT intervention study groups received the SFSW parent training program, which combines an interactive website with weekly telephone coaching [, ]. One parent was identified for each child and they filled in the web-based questionnaire. However, they were also encouraged to get the child’s other parent involved in the program as much as possible. The program was guided by coaches who were professionals of health care and social services, that is, public health nurses, public nurses, or social workers from the child services. Of note, in the RCT study, there were 6 coaches, and during the implementation study, there were 10 coaches, 6 of whom had not participated in the RCT. The coaches had weekly phone calls with the parents, which were sometimes organized using texts or emails, and they monitored their progress on the website. The intervention consisted of 11 weekly themes that were explored during the interactive web-based program and the associated telephone coaching sessions. After the baseline survey, the coaches called the parents and they agreed to personalize goals tailored to individual behavior problems demonstrated by the child. The program aimed to reduce the problems identified by the parents by teaching them positive and practical parenting skills. During the first 7 weeks, the parent learned positive and practical problem-solving skills and were encouraged to develop an understanding of their child’s emotional development. The primary aim was to reorient the parent so that they noticed the child’s positive, not negative, behavior and reacted with a positive response. The second aim was to apply the skills in everyday situations, to plan daily activities in advance, and to use the methods they were taught to reinforce positive behavior. The final weekly themes focused on reinforcing their new skills and developing sustained positive parenting. The parents practiced the acquired skills with their child, independent of the coach’s support, and learned how to sustain the skills once the program had finished. The content and the conceptual framework of the weekly themes are depicted in . Each internet-based session comprised an introduction to the weekly theme, session content, video exercises, troubleshooting tips, and a review of what the parent had learnt. Instructional videos and audio clips illustrated the practical applications of the parents’ new skills. The coaches gave the parents feedback about their progress in applying the new skills and encouraged them throughout the program. They only proceeded to the next weekly theme when the parents had mastered the skill-related questions in the current one. This typically took 1 or 2 weeks. The children did not have access to the website or take part in the coaching calls. We are not aware of any potential adverse effects of the parent training in this study or in previous studies [ ].
|Session||Training components||Key training elements||Parental goals||Coaching elements||Parental action|
|Introduction to the program||Telephone coaching||Set up the parents for success||Reorient the parents to “How to break the negative circle”||Working alliance Identifying behavior problems Goal setting Present the first weekly theme||Actively start to notice the good|
|Notice the good||Web-based material (text, videos, audio clips) Telephone coaching||Positive and active parenting||Boost self-esteem of the child and parents and change the parents’ views of the child||Working alliance Evaluate the goal setting by modeling, practice such as role play, feedback, support||Notice good behavior often Positive verbal interaction and body language|
|Spread attention around||Web-based material (text, videos, audio clips) Telephone coaching||Positive, impartial parenting||Strengthen child’s empathy skills||Same as above||Learn to spread attention actively Praise the child for interacting positively with others|
|Ignore whining and complaining||Web-based material (text, videos, audio clips) Telephone coaching||Positive, self-controlled parenting||Teaches parents self-regulation||Same as above||Use positive thinking to stay calm and in control of the situations|
|Prepare for changes||Web-based material (text, videos, audio clips) Telephone coaching||Positive, proactive parenting||Reinforce good daily routines||Same as above||Warn that behavior must change Use positive “when you do this, then this will happen” statements|
|Plan ahead at home||Web-based material (text, videos, audio clips) Telephone coaching||Positive, proactive parenting||Reinforce child’s active role and involve them in planning||Same as above||Listens to the child’s ideas, plans daily situations at home|
|Reinforce by rewarding||Web-based material (text, videos) Telephone coaching||Positive, active parenting||Involve the child in planning and reinforce good daily routines||Same as above||Understand realistic goal setting and how to use praises and rewards|
|Plan ahead outside the home||Web-based material (text, videos) Telephone coaching||Positive, proactive parenting||Reinforce child’s active role and involve them in planning||Same as above||Listen to the child’s ideas Plan situations outside the home|
|Cooperate with day care||Web-based material (text, videos) Telephone coaching||Positive cooperation and communication between parent and day care||Help child to manage and succeed||Same as above||Set realistic goals and rewards Cooperate|
|Plan how to use time-out||Web-based material (text, videos) Telephone coaching||Positive, self-controlled parenting||Teach self-regulation and consistency||Reassure and use positive skills How to use time-out||Learn to be consequent Plan how to manage difficult situations|
|Revise: Problem-solving and future application of skills||Web-based material (text, videos) Telephone coaching||Positive daily parenting in future||Remind parents of positive proactive parenting skills||Ensure that parent is using all the skills and stays on track||Understand how using skills helps to prevent setbacks|
Quality Assurance and Implementation Plan
To ensure the integrity of the intervention and the accuracy of the data, several quality assurance measures were in effect during the implementation phase. These were similar to the quality assurance measures during the RCT study [, ]. The implementation plan is summarized below and has been previously described in detail [ ]. The implementation plan was driven by 3 core components [ , ]. First was recruitment, staff selection, and training. Once the coaches were recruited, they received intensive training on the SFSW program and were supervised and regularly monitored to make sure they adhered to the protocol. Together with supervision and staff performance evaluation, this provided systematic quality assurance [ ]. The second core component was ongoing supervision and staff performance evaluation. The coaches took part in systematic weekly supervision meetings and group case conferences, where they reviewed and discussed the families they were coaching. Coaches with previous experience of the SFSW program acted as supervisors. After each telephone call, the coaches assessed their own performance on a scale of 4-10. The supervisor received a message from the digital platform about self-assessments that scored 6 or more and discussed the content of the call with the coach. To ensure the fidelity of the data, about 10% of the phone calls was audited by the coach supervisors with the parent’s permission and evaluated for competency. Additional training and monitoring of future calls were provided, if indicated. The coaches were required to report any adverse effects such as safety issues, abuse, or neglect to the supervisors, and the case was reported to the child protective services. Of note, 3 cases were reported during the implementation study and none during the RCT. The third core component was the decision supporting and administration. The development, delivery, and implementation process of the digital SFSW parent training intervention were centralized at the Research Center for Child Psychiatry at the University of Turku. The research group and the assisting staff of the Research Center introduced the SFSW intervention and the implementation process to the directors of child and family health services of the primary health care of each administrative region. A jointly funded research contract was signed by both parties. The research group maintained contact with the directors across the study region by organizing regular meetings and providing them with user-friendly monthly progress reports, which included the number of families who had been screened and enrolled. Training was offered to the team leaders of the child health clinics and public health nurses in order to integrate the intervention into primary health care. Moreover, local and national media were involved to increase public awareness of the SFSW intervention.
The outcome measures were the same in the implementation and RCT studies [, ]. The main measurement tool used to measure disruptive behavior was the 24-item Child Behavior Checklist 1.5-5 (CBCL/1.5-5) version for preschool children [ ]. The CBCL/1.5-5 asks parents to rate emotional, behavioral, and social problems and has an additional section where they can provide extra information. It yields total scores and syndrome scales for the following items: emotionally reactive, anxious/depressed, somatic complaints, withdrawn, sleep problems, attention problems, and aggressive behavior. The first 4 syndromes yield the internalizing score, while the last 2 yield the externalizing score. The CBCL/1.5-5 also includes 5 subscores from the Diagnostic and Statistical Manual of Mental Disorders, fifth edition: affective, anxiety, pervasive developmental problems, attention-deficit/hyperactivity disorder, and oppositional disorder [ ]. A large cross-cultural study from 24 countries, including Finland, reported good psychometric properties and good internal consistency for the CBCL preschool version (Cronbach alphas for total, externalizing, and internalizing scores: .94, .88, and .84, respectively) [ , ]. We used the Inventory of Callous-Unemotional Traits (ICU) to measure child psychopathy traits. The instrument consists of 24 items and has been reported to have good psychometric properties for 4-year-old children [ , ]. Cronbach alphas of .93, .81, .88, and .86 have previously been reported for total score, callousness, uncaring, and unemotional scores, respectively, for 4-year-old children [ ].
Daily activities were only assessed for the implementation study. Parents were asked to rate the impact of the child´s behavior during daily transitions, including getting dressed, getting ready for day care, during the evening meal, and getting ready for bed. It also covered social interactions, including playing with siblings and other children during a car or bicycle ride and in public places such as the supermarket. A Cronbach alpha of .64 was calculated using our implementation data. The questionnaire was adapted from the Barkley Home Situations Questionnaire, which asks the parent to rate whether the child’s behavior causes problems during specified daily routines .
The Parenting Scale, which is a 30-item questionnaire, was used to measure parenting skills [, ]. Cronbach alphas of .78, .66, .68, and .50 were calculated for total score, laxness, overreactivity, and hostility, respectively, by using our implementation data. We evaluated the parents’ stress, anxiety, and depression symptoms with the shorter 21-item Depression, Anxiety and Stress Scale (DASS-21) [ ]. The internal consistency of DASS-21 has been reported as 0.93, 0.88, 0.82, and 0.90 for total scale and DASS-21, respectively, in a large study that represented a nonclinical sample [ ].
The analyses compared the 600 families in the real-world implementation group to the 232 families in the RCT intervention group. Categorical demographic variables, including the child, parent, and family characteristics, are presented as numbers and percentages. Continuous demographic variables, including the parents’ age and duration of child’s behavioral problems, are presented as means and standard deviations. We explored any differences at baseline between the 2 groups by using Pearson chi-square test or Fisher exact test for the categorical variables and the two-tailed Student t test for the continuous variables. The primary and secondary outcome variables were analyzed with a linear mixed-effect model for repeated measurements. The within factor was time, namely, baseline and 6-months follow-up, and the RCT intervention group and the implementation group provided the between factor. The covariates in the statistical models were CBCL externalizing scores at baseline, maternal education, duration of behavior problems, and the baseline measurement of each outcome. The statistical model used to analyze the CBCL externalizing score consisted of the group and time main effects, the group-by-time interaction effect, and the following covariates: the CBCL externalizing score at baseline, maternal education, and duration of behavior problems. Meanwhile, the statistical model used to analyze all the secondary outcome variables, namely, the CBCL total and other CBCL subscores, ICU, the Parenting Scale, and DASS-21 consisted of the group and time main effects and the group-by-time interaction effect. It also included the following covariates: the specific secondary variable to be analyzed at baseline, the CBCL externalizing score at baseline, maternal education, and the duration of behavior problems.
The sensitivity analyses comprised the families who had completed the parent training program as well as the treatment comparisons. Turku was excluded from analysis, as it was the only site that had taken part in both the implementation and RCT intervention studies. As the study subjects in the implementation group were recruited from January 2015 to May 2017 and in the RCT intervention group from October 2011 to November 2013, we also tested the effect of the recruitment year on the CBCL externalizing score at baseline. The model included the effects of recruitment year, maternal education, and duration of behavior problems. The effect of the recruitment year was insignificant (P=.17). An additional analysis also compared the implementation to the RCT education control group. The model included the CBCL externalizing score at baseline, maternal education, duration of behavior problems, paternal age, and the baseline measurements of each outcome as covariates. A P value <.05 was considered to be statistically significant. The statistical analyses were performed using SAS 9.4 (SAS Institute).
Ethical approval for the implementation study was received from the University of Turku (approval number: 18/2018). The parents provided written informed consent for both the implementation and the RCT studies.
The number of families who discontinued the program was 86 (14.3%) of the 600 families in the implementation group compared to 56 (24.1%) of the 232 families in the RCT intervention group. This meant that the odds ratio was 1.9 with a 95% CI of 1.3 to 2.8 (P<.001), as seen in. The 6-month follow-up assessment was completed by 71.2% (427/600) of the parents in the implementation group and 79.3% (184/232) of the parents in the RCT intervention group (P<.001), as seen in . shows that there were no differences between the implementation group and the RCT intervention group when it came to the parent, family, and child characteristics and the factors related to parent training program. However, the mothers in the implementation group had higher educational levels than the mothers in the RCT intervention group (P=.046) and the children experienced a longer duration of behavior problems (P=.004). The mean duration of the telephone coaching calls was 37 minutes in both the implementation and the RCT intervention groups. The total duration of telephone coaching plus the average time spent on the program website was 13.8 hours in the implementation group and 14.1 hours in the RCT intervention group (P=.49).
|Demographics||Implementation group (n=600)||Randomized controlled trial |
|Parent and family characteristics|
|Family structurea, n (%)||.54|
|Two biological parents||489 (81.6)||191 (83.5)|
|Single biological parent||82 (13.7)||24 (10.4)|
|Biological parent and foster parent||19 (3.2)||9 (3.9)|
|Other||9 (1.5)||5 (2.2)|
|Age (years), mean (SD)|
|Maternal||30.3 (4.8)||30.5 (5.4)||.68|
|Paternal||32.7 (5.7)||33.2 (5.9)||.28|
|Maternal educationb, n (%)||.046|
|Elementary school or less||15 (2.5)||13 (5.7)|
|Secondary education||204 (34.2)||85 (37)|
|College or university degree||378 (63.3)||132 (57.4)|
|Paternal educationc, n (%)||.28|
|Elementary school or less||27 (4.8)||16 (7.4)|
|Secondary education||280 (50.1)||99 (45.8)|
|College or university degree||252 (45.1)||101 (46.8)|
|Child characteristics, n (%)|
|Female||238 (39.7)||90 (38.8)|
|Male||362 (60.3)||142 (61.2)|
|Day care outside homed||.29|
|Yes||476 (79.9)||192 (83.1)|
|No||120 (20.1)||39 (16.9)|
|Minor||301 (50.2)||129 (55.6)|
|Definite||252 (42)||92 (39.7)|
|Severe||47 (7.8)||11 (4.7)|
|Duration of problemse||.004|
|<6 months||193 (33)||102 (45.1)|
|6-12 months||155 (26.5)||44 (19.5)|
|>12 months||237 (40.5)||80 (35.4)|
|Program characteristics, mean (SD)|
|Total number of calls||10.4 (2.5)||10.1 (3.3)||.20|
|Duration of calls for the 11 themes (min)||37.3 (11.0)||37.3 (13.5)||.96|
|Duration of website access per theme (min)||45.3 (19.3)||47.8 (19.9)||.12|
|Total duration of calls (h)||6.5 (2.4)||6.4 (3.3)||.65|
|Total duration of website access (h)||7.3 (2.8)||7.5 (3.2)||.56|
|Total duration of program (h)||13.8 (4.3)||14.1 (5.4)||.49|
aMissing observations: implementation group (n=1); randomized controlled trial group (n=2).
bMissing observations: implementation group (n=3); randomized controlled trial group (n=2). Pairwise comparisons: elementary school or less versus secondary education (P=.06); elementary school or less versus college or university degree (P=.02); secondary education versus college or university degree (P=.28).
cMissing observations: implementation group (n=41); randomized controlled trial group (n=1).
dMissing observations: implementation group (n=4); randomized controlled trial group (n=1).
eMissing observations: implementation group (n=15); randomized controlled trial group (n=6). Pairwise comparisons: <6 months versus 6-12 months (P=.003); <6 months versus >12 months (P=.01); 6-12 months versus >12 months (P=.42).
In the implementation group, there were significant improvements from the baseline to the 6-month follow-up assessment in the primary outcome, which was the CBCL externalizing score. The same was true for the secondary outcomes: CBCL total and internalizing scores and the total scores of the ICU, Parenting Scale, and DASS-21 (). The sensitivity analysis, which included the participants who completed the whole program (Table S1 of ), yielded similar estimates of the improvements in all the outcomes. shows the mean scores of the primary outcome, CBCL externalizing score, and the secondary outcomes at baseline and 6 months in the implementation and the RCT intervention groups. There were no significant differences between the 2 groups in the CBCL externalizing, total, or internalizing scores. In addition, no significant differences were seen in the total scores of the Parenting Scale or ICU. The estimated difference of 2.5 (95% CI 0.0-5.1) points in DASS-21 nearly reached statistical significance (P=.05), indicating better improvement in the implementation group when it was compared to that of the RCT intervention group. Of note, the improvement in DASS-21 showed significantly better improvement in the implementation group (estimated difference 1.1, 95% Cl 0.1-2.2; P=.04). When only the participants who completed the whole parent training program in the implementation group were compared to those in the RCT intervention group, the results remained similar (Table S2 of ).
The additional analyses compared the changes in primary and secondary outcomes between the implementation and the RCT education control groups, as shown in Table S3 of. There were significant differences between the groups in CBCL externalizing, total, and internalizing scores, as well as the total scores of the Parenting Scale and DASS-21. However, the total ICU score did not reach statistical significance (P=.27). As the city of Turku participated in both the implementation study and the RCT study, we repeated the analyses by excluding the participants living in Turku from the implementation group. This did not show any significant differences in any of the symptom scores between the study groups (Table S4 of ). Changes in daily activities from the baseline assessment to posttreatment and the 6-month follow-up assessment are shown in the Table S5 of . This information was only obtained from the implementation group; therefore, comparisons with the RCT intervention group could not be made. There were significant improvements in all measurements for social interactions and daily transitions from baseline to posttreatment and to the 6-month follow-up. The data for daily activities were obtained from 83% (498/600) of the participants in posttreatment and 66.5% (399/600) of the participants in the follow-up.
|Variable||Baseline (n=600), meana (SE)||After 6 months (n=600), meana (SE)||Mean changeb (SE)||95% CI||Pc value|
|Child Behavior Checklist externalizing score||21.1 (0.5)||14.8 (0.5)||6.2 (0.4)||5.5 to 7.0||<.001|
|Child Behavior Checklist Total score||48.8 (1.2)||33.6 (1.3)||15.2 (1.0)||13.3 to 17.2||<.001|
|Child Behavior Checklist Internalizing score||12.1 (0.4)||8.5 (0.5)||3.6 (0.4)||2.9 to 4.3||<.001|
|Aggression||18.0 (0.4)||12.5 (0.4)||5.5 (0.3)||4.9 to 6.1||<.001|
|Attention||3.1 (0.1)||2.4 (0.1)||0.7 (0.1)||0.6 to 1.0||<.001|
|Sleep||4.0 (0.2)||2.5 (0.2)||1.5 (0.1)||1.2 to 1.7||<.001|
|Withdrawn||2.4 (0.1)||1.6 (0.1)||0.8 (0.1)||0.6 to 1.0||<.001|
|Somatic||2.9 (0.1)||2.0 (0.2)||0.8 (0.1)||0.6 to 1.1||<.001|
|Anxious||2.9 (0.1)||2.0 (0.1)||0.8 (0.1)||0.6 to 1.0||<.001|
|Emotional||3.9 (0.2)||2.8 (0.2)||1.2 (0.1)||0.9 to 1.4||<.001|
|Diagnostic and Statistical Manual of Mental Disorders, fifth edition subscores|
|Affective problems||3.3 (0.1)||2.0 (0.2)||1.3 (0.1)||1.1 to 1.5||<.001|
|Anxiety problems||4.2 (0.2)||2.9 (0.2)||1.4 (0.1)||1.1 to 1.6||<.001|
|PDDd problems||4.7 (0.2)||3.3 (0.2)||1.4 (0.2)||1.1 to 1.7||<.001|
|ADHDe problems||6.0 (0.2)||4.5 (0.2)||1.6 (0.1)||1.3 to 1.8||<.001|
|ODDf problems||6.5 (0.2)||4.6 (0.2)||1.9 (0.1)||1.6 to 2.1||<.001|
|Inventory of Callous-Unemotional Traits|
|Total||24.6 (0.5)||20.6 (0.5)||4.0 (0.4)||3.2 to 4.7||<.001|
|Callousness||8.3 (0.2)||6.2 (0.2)||2.2 (0.2)||1.8 to 2.5||<.001|
|Uncaring||13.2 (0.2)||11.6 (0.3)||1.6 (0.2)||1.3 to 2.0||<.001|
|Unemotional||3.1 (0.1)||2.9 (0.1)||0.2 (0.1)||–0.1 to 0.4||.30|
|Total||3.2 (0.0)||2.7 (0.0)||0.6 (0.0)||0.5 to 0.6||<.001|
|Laxness||2.7 (0.0)||2.2 (0.0)||0.4 (0.1)||0.4 to 0.5||<.001|
|Overreactivity||3.9 (0.1)||3.1 (0.1)||0.8 (0.0)||0.7 to 0.9||<.001|
|Hostility||1.9 (0.0)||1.6 (0.1)||0.3 (0.1)||0.3 to 0.4||<.001|
|21-item Depression, Anxiety and Stress Scale short form|
|Total||18.5 (1.1)||12.1 (1.1)||6.4 (0.8)||4.9 to 7.9||<.001|
|Depression||5.2 (0.5)||3.1 (0.5)||2.1 (0.3)||1.4 to 2.7||<.001|
|Anxiety||2.4 (0.3)||1.4 (0.3)||1.1 (0.2)||0.7 to 1.4||<.001|
|Stress||11.0 (0.5)||7.7 (0.5)||3.3 (0.4)||2.6 to 4.0||<.001|
bChange from baseline to 6 months after providing informed consent.
cAdjusted with maternal education and duration of problems.
dPDD: pervasive developmental disorder.
eADHD: attention-deficit/hyperactivity disorder.
fODD: oppositional defiant disorder.
|Variable||Mean (SE) change from baseline to 6 months||Implementation versus RCT intervention, mean (95% CI)||Pc value|
|Implementation group (n=600), meana (SE)||RCTb intervention (n=232), meana (SE)|
|Child Behavior Checklist externalizing score||6.3 (0.4)||6.1 (0.6)||–0.2 (–1.3 to 1.6)||.83|
|Child Behavior Checklist total score||15.3 (1.0)||14.6 (1.6)||–0.7 (–3.0 to 4.5)||.70|
|Child Behavior Checklist internalizing score||3.7 (0.4)||3.4 (0.6)||–0.3 (–1.0 to 1.6)||.64|
|Aggression||5.5 (0.3)||5.5 (0.5)||–0.0 (–1.2 to 1.3)||.95|
|Attention||0.7 (0.1)||0.6 (0.1)||–0.1 (–0.2 to 0.4)||.53|
|Sleep||1.5 (0.1)||1.5 (0.2)||–0.0 (–0.5 to 0.5)||1.0|
|Withdrawn||0.8 (0.1)||0.5 (0.2)||–0.3 (–0.0 to 0.7)||.08|
|Somatic||0.8 (0.1)||0.6 (0.2)||–0.2 (–0.2 to 0.7)||.29|
|Anxious||0.9 (0.1)||1.0 (0.2)||–0.1 (0.5 to 0.3)||.62|
|Emotional||1.2 (0.1)||1.3 (0.2)||–0.1 (–0.7 to 0.4)||.58|
|Diagnostic and Statistical Manual of Mental Disorders, fifth edition subscores|
|Affective problems||1.3 (0.1)||1.3 (0.2)||0.0 (–0.4 to 0.5)||.95|
|Anxiety problems||1.4 (0.1)||1.5 (0.2)||–0.1 (–0.6 to 0.4)||.69|
|PDDd problems||1.4 (0.2)||1.2 (0.3)||0.2 (–0.3 to 0.8)||.41|
|ADHDe problems||1.6 (0.1)||1.2 (0.2)||3.5 (–0.2 to 0.9)||.17|
|ODDf problems||1.9 (0.1)||2.2 (0.2)||–0.3 (–0.7– 0.2)||.26|
|Inventory of Callous-Unemotional Traits|
|Total||4.0 (0.4)||4.3 (0.7)||–0.4 (–1.9 to 1.2)||.64|
|Callousness||2.0 (0.2)||2.1 (0.3)||–0.1 (–0.7– 0.8)||.83|
|Uncaring||1.6 (0.2)||1.9 (0.3)||–0.2 (–1.0 to 0.5)||.53|
|Unemotional||0.2 (0.1)||0.3 (0.2)||–0.1 (–0.6 to 0.3)||.44|
|Total||0.6 (0.0)||0.5 (0.0)||0.0 (–0.1 to 0.1)||.50|
|Laxness||0.4 (0.0)||0.4 (0.1)||0.0 (–0.1 to 0.2)||.79|
|Overreactivity||0.8 (0.1)||0.6 (0.1)||0.2 (–0.0 to 0.4)||.07|
|Hostility||0.3 (0.0)||0.3 (0.1)||0.0 (–0.1 to 0.2)||.85|
|21-item Depression, Anxiety and Stress Scale short form|
|Total||6.4 (0.7)||3.9 (1.1)||2.5 (0.0 to 5.1)||.05|
|Depression||2.1 (0.3)||1.0 (0.5)||1.1 (0.1 to 2.2)||.036|
|Anxiety||1.0 (0.2)||0.8 (0.3)||0.3 (–0.4 to 0.1)||.44|
|Stress||3.3. (0.4)||2.2 (0.6)||1.1 (–0.2 to 2.4)||.09|
bRCT: randomized controlled trial.
cAdjusted with maternal education and duration of problems.
dPDD: pervasive developmental disorder.
eADHD: attention-deficit/hyperactivity disorder.
fODD: oppositional defiant disorder.
This was the first population-based study to evaluate the effectiveness of an internet-based and telephone-assisted parent training intervention for children with behavior problems when it was implemented in real-world practice. The children’s psychiatric problems improved, including externalizing and internalizing problems and callousness. The findings were remarkable from the perspective of the children’s social development, as the program had significant effects on daily transitions and activities such as getting dressed, dining behavior, activities outside the home, and interactions with other people. Parents reported that their parenting skills had improved and they demonstrated less distress in dealing with their children at the 6-month follow-up. Most importantly, this study shows that the improvements that had been achieved were similar to those reported for the intervention group in the RCT. There was no difference in the changes in the children’s psychiatric problems or parenting skills when the implementation and RCT groups were compared. Furthermore, when changes between the implementation and RCT education control groups were compared, the implementation group showed significantly better improvements in the children’s externalizing and internalizing problems as well as in parenting skills and parents’ distress. In addition to the effectiveness of the treatment, the ability to engage and retain parents in the program is one of the keys to successful parent training interventions [- ]. Previously, we reported high parental satisfaction levels in both the RCT and implementation groups [ ]. High satisfaction levels and the quality of relationships between parents and professionals have been associated with greater improvements in the effectiveness of interventions [ , ]. The dropout rate in our RCT study was 24%, while previous studies on digital parenting interventions report usually 30%-50% dropout rates [ , - ]. In general, high dropout rates in digital interventions have been especially associated with nonguided interventions [ , - ]. The reasons for the exceptionally low dropout rate in the implementation phase (14%) are likely to be multiple. One possible explanation is that in the implementation phase, the SFSW intervention that was offered had gained research-based evidence and the benefits of it were known and communicated to the professionals in the primary health care, especially in child health clinics, and to parents and largely in the media. Thus, the public and the professionals were aware of the intervention and its benefits. It is very likely that this convinced both health nurses at the child health clinics who motivate the parents in engaging in the program and the concerned parents tackling with their child’s challenges.
In order to successfully implement interventions, we need to know whether they work and why they work . Success can be related to how appropriate the background theory is, the context where the intervention takes place, practical issues such as how easy it is to attend sessions, and specific intervention practices such as practicing specific parenting skills [ ]. Our SFSW intervention fulfilled these criteria well. It was based on the social learning theory and the cognitive behavioral theory as well as principles of positive parenting, which provided a sound theoretical framework for the intervention. The context of the program was well-defined, including a clear definition of the population that the program was aimed at, and there were clear inclusion and exclusion criteria. The program also had a clear structure, including a description of the core components, which was practiced through modeling, practice, feedback, and support. It has previously been emphasized that a solid framework and a structured implementation plan are needed to successfully make the transition from evidence-based psychosocial interventions to real-world clinical practice [ ]. We systematically followed a structured plan during the implementation process [ ]. The SFSW program contained the core implementation drivers that facilitated the process when intervention was implemented in the primary health care. The same quality assurance measures were in place during the RCT and implementation phase. These were based on the centralized delivery of the intervention, which used a digital platform and ongoing training, monitoring, and supervision of the program coaches. It is important to note that the primary health care staff were also provided with ongoing training. In addition, the program was effectively administered by including regular meetings with the directors of the child and family services and providing them with user-friendly reports. Media coverage raised awareness, and this made it easier to recruit families and increased the perceived value of the program [ ].
Several practical features of the program may have paved the way for positive outcomes during the real-world implementation. First, the program was much easier for the parents than face-to-face interventions because they did not need to leave home or work or make childcare arrangements. Second, the telephone coaching provided immediate problem-solving, which may have been more rewarding for the parents than communicating using emails or text messages. A recent meta-analysis showed that digital interventions that included support and guidance, such as telephone calls, had larger effect sizes on mental health outcomes than smartphone interventions without any personal support . Third, the coaches were well-trained and formed good relationships with the parents [ ], which is central to the success of any intervention [ ].
There were some limitations in our study. First, although the parental and child outcomes were measured using well-validated questionnaires, they were rated by the same person, namely, the parent. One parent was identified for each child, but they were also encouraged to get the child’s other parent involved in the program as much as possible. Further details on the level of parental involvement could have added to the richness of the data, but there were practical limitations to collecting this. To reduce the possibility of the common rater variance, observations by other informants such as day care personnel could have validated our findings. Second, we have discussed mechanisms that could have been responsible for the positive outcomes. However, there is very little empirical evidence on whether the effects of the intervention resulted from the internet sessions, the personal telephone coaching, parental motivation, or a combination of those factors. Further studies need to examine factors that explain these positive outcomes. Personalized medicine is increasingly being used to move away from one-size-fits-all interventions to those that are more tailored to individual needs. This approach could yield useful information on the mechanisms underlying interventions and enable more accurate targeting.
The target group, content, and effectiveness of the intervention were maintained when the implementation group results were compared with the findings of the RCT intervention. Internet-based telephone-assisted parent training interventions may have advantages over traditional group-based treatment approaches when the goal is to identify children at risk in the community at an early stage. This new approach can provide effective parent training for a large number of families, including many who would not normally participate in clinic-based services. Referring families who need parent training to clinical services often results in substantial delays and they need other support while they are waiting. Digitally delivered interventions move child mental health treatment outside traditional clinics and into people’s homes and schools, increasing access and reducing stigma. In addition, they can be increased to help more families, and parents are more likely to stay with the program until the end. There is a global shortage of skilled staff who can address child mental health problems in low- and high-income countries and even in countries with public health care [, ]. This could become an even greater issue when demand inevitably increases because of the impact of the COVID-19 pandemic on children and the effects of the expected global recession on health care budgets. Our study highlights the positive findings that were demonstrated when our internet-based training and phone coaching initiative provided support for the parents of children with behavior problems, who were identified using population-based screening at primary health care. This initiative made the successful transition from an RCT to real-world settings, and our findings may have potential global implications for addressing the unmet needs of children with mental health issues if the findings are repeated in other sociocultural contexts.
We wish to thank the family coaches and Project Engineer Atte Sinokki from the Department of Child Psychiatry, University of Turku, as well as the public health directors and health nurses at the participating municipal child health clinics who made this study possible. This research was funded by the Academy of Finland Flagship Program Inequalities, Interventions, and a New Welfare State (decision 320162), the Academy of Finland Health from Cohorts and Biobanks Program (decision 308552), and the European Research Council under the European Union’s Horizon 2020 research and innovation program (grant 101020767) (author AS).
Conflicts of Interest
AS is the founder and director of Digifamilies, which provides evidence-based treatments to Finnish public health services. The Strongest Families Institute (SFI) is a not-for-profit organization that delivers services to Canadian families. PJM is the cofounder and Chair of SFI Board of Directors. The other authors have no conflicts of interest to declare.
Timeline of the randomized controlled trial and implementation studies.DOCX File , 13 KB
Change from baseline to 6 months in child psychopathology, parenting skills and parents’ stress in the Implementation group for participants who completed the program.DOCX File , 31 KB
Mean changes from baseline to 6 months in child psychopathology, parenting skills, and parents’ stress in the implementation and the randomized controlled trial intervention groups for participants who completed the program.DOCX File , 23 KB
Mean changes from baseline to 6 months in child psychopathology, parenting skills, and parents’ stress in the implementation and randomized controlled trial educational control groups.DOCX File , 23 KB
Change from baseline to 6 months in child psychopathology, parenting skills, and parents’ stress in the implementation and randomized controlled trial intervention groups. The city of Turku is excluded from the implementation data.DOCX File , 25 KB
Change from baseline to posttreatment and 6 months in daily activities and social interactions in the implementation group (n=600).DOCX File , 26 KB
- Dretzke J, Davenport C, Frew E, Barlow J, Stewart-Brown S, Bayliss S, et al. The clinical effectiveness of different parenting programmes for children with conduct problems: a systematic review of randomised controlled trials. Child Adolesc Psychiatry Ment Health 2009 Mar 04;3(1):7 [FREE Full text] [CrossRef] [Medline]
- Michelson D, Davenport C, Dretzke J, Barlow J, Day C. Do evidence-based interventions work when tested in the "real world?" A systematic review and meta-analysis of parent management training for the treatment of child disruptive behavior. Clin Child Fam Psychol Rev 2013 Mar;16(1):18-34. [CrossRef] [Medline]
- Baumel A, Pawar A, Kane JM, Correll CU. Digital Parent Training for Children with Disruptive Behaviors: Systematic Review and Meta-Analysis of Randomized Trials. J Child Adolesc Psychopharmacol 2016 Oct;26(8):740-749. [CrossRef] [Medline]
- Caspi A, Moffitt TE, Newman DL, Silva PA. Behavioral observations at age 3 years predict adult psychiatric disorders. Longitudinal evidence from a birth cohort. Arch Gen Psychiatry 1996 Nov;53(11):1033-1039. [CrossRef] [Medline]
- Sourander A, Jensen P, Davies M, Niemelä S, Elonheimo H, Ristkari T, et al. Who is at greatest risk of adverse long-term outcomes? The Finnish From a Boy to a Man study. J Am Acad Child Adolesc Psychiatry 2007 Sep;46(9):1148-1161. [CrossRef] [Medline]
- Perrin EC, Leslie LK, Boat T. Parenting as Primary Prevention. JAMA Pediatr 2016 Jul 01;170(7):637-638. [CrossRef] [Medline]
- Weber L, Kamp-Becker I, Christiansen H, Mingebach T. Treatment of child externalizing behavior problems: a comprehensive review and meta-meta-analysis on effects of parent-based interventions on parental characteristics. Eur Child Adolesc Psychiatry 2019 Aug;28(8):1025-1036. [CrossRef] [Medline]
- Jensen PS, Goldman E, Offord D, Costello EJ, Friedman R, Huff B, et al. Overlooked and underserved: "action signs" for identifying children with unmet mental health needs. Pediatrics 2011 Nov;128(5):970-979 [FREE Full text] [CrossRef] [Medline]
- Breitenstein SM, Gross D, Christophersen R. Digital delivery methods of parenting training interventions: a systematic review. Worldviews Evid Based Nurs 2014 Jun;11(3):168-176. [CrossRef] [Medline]
- Mastnak W. Psychopathological problems related to the COVID-19 pandemic and possible prevention with music therapy. Acta Paediatr 2020 Aug;109(8):1516-1518 [FREE Full text] [CrossRef] [Medline]
- McGrath PJ, Lingley-Pottie P, Thurston C, MacLean C, Cunningham C, Waschbusch DA, et al. Telephone-based mental health interventions for child disruptive behavior or anxiety disorders: randomized trials and overall analysis. J Am Acad Child Adolesc Psychiatry 2011 Nov;50(11):1162-1172. [CrossRef] [Medline]
- Sanders MR, Baker S, Turner KMT. A randomized controlled trial evaluating the efficacy of Triple P Online with parents of children with early-onset conduct problems. Behav Res Ther 2012 Nov;50(11):675-684. [CrossRef] [Medline]
- Sourander A, McGrath PJ, Ristkari T, Cunningham C, Huttunen J, Lingley-Pottie P, et al. Internet-Assisted Parent Training Intervention for Disruptive Behavior in 4-Year-Old Children: A Randomized Clinical Trial. JAMA Psychiatry 2016 Apr;73(4):378-387. [CrossRef] [Medline]
- Sourander A, McGrath PJ, Ristkari T, Cunningham C, Huttunen J, Hinkka-Yli-Salomäki S, et al. Two-Year Follow-Up of Internet and Telephone Assisted Parent Training for Disruptive Behavior at Age 4. J Am Acad Child Adolesc Psychiatry 2018 Sep;57(9):658-668.e1. [CrossRef] [Medline]
- Ristkari T, Kurki M, Suominen A, Gilbert S, Sinokki A, Kinnunen M, et al. Web-Based Parent Training Intervention With Telephone Coaching for Disruptive Behavior in 4-Year-Old Children in Real-World Practice: Implementation Study. J Med Internet Res 2019 Apr 11;21(4):e11446 [FREE Full text] [CrossRef] [Medline]
- Bandura A. Social Learning Theory, 1st edition. Morristown, New York: General Learning Press; 1971.
- Reinecke MA, Dattilio FM, Freeman A, editors. Cognitive Therapy with Children and Adolescents: A Casebook for Clinical Practice, 2nd ed. New York City, New York: Guilford Press; 2003.
- Piquero AR, Jennings WG, Diamond B, Farrington DP, Tremblay RE, Welsh BC, et al. A meta-analysis update on the effects of early family/parent training programs on antisocial behavior and delinquency. J Exp Criminol 2016 Apr 13;12(2):229-248. [CrossRef]
- Fixsen DL, Blase KA, Naoom SF, Wallace F. Core Implementation Components. Research on Social Work Practice 2009 May 27;19(5):531-540. [CrossRef]
- May CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci 2016 Oct 19;11(1):141 [FREE Full text] [CrossRef] [Medline]
- Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol 2008 Jun;41(3-4):327-350. [CrossRef] [Medline]
- Vis C, Mol M, Kleiboer A, Bührmann L, Finch T, Smit J, et al. Improving Implementation of eMental Health for Mood Disorders in Routine Practice: Systematic Review of Barriers and Facilitating Factors. JMIR Ment Health 2018 Mar 16;5(1):e20 [FREE Full text] [CrossRef] [Medline]
- Kasvun tuki (early interventions) resource. ITLA Children's Foundation. 2016 Jun 23. URL: https://kasvuntuki.fi/en/interventions/voimaperheet/ [accessed 2021-02-15]
- McGrath PJ, Sourander A, Lingley-Pottie P, Ristkari T, Cunningham C, Huttunen J, et al. Remote population-based intervention for disruptive behavior at age four: study protocol for a randomized trial of Internet-assisted parent training (Strongest Families Finland-Canada). BMC Public Health 2013 Oct 21;13:985 [FREE Full text] [CrossRef] [Medline]
- Child welfare clinic. National Institute for Health and Welfare. URL: https://thl.fi/fi/web/lapset-nuoret-ja-perheet/peruspalvelut/aitiys_ja_lastenneuvola/lastenneuvola [accessed 2021-02-15]
- Goodman R. The Strengths and Difficulties Questionnaire: a research note. J Child Psychol Psychiatry 1997 Jul;38(5):581-586. [CrossRef] [Medline]
- Koskelainen M, Sourander A, Kaljonen A. The Strengths and Difficulties Questionnaire among Finnish school-aged children and adolescents. Eur Child Adolesc Psychiatry 2000 Dec;9(4):277-284. [CrossRef] [Medline]
- Borg A, Kaukonen P, Salmelin R, Joukamaa M, Tamminen T. Reliability of the strengths and difficulties questionnaire among Finnish 4-9-year-old children. Nord J Psychiatry 2012 Dec;66(6):403-413. [CrossRef] [Medline]
- Ristkari T, Mishina K, Lehtola M, Sourander A, Kurki M. Public health nurses' experiences of assessing disruptive behaviour in children and supporting the use of an Internet-based parent training programme. Scand J Caring Sci 2020 Jun;34(2):420-427 [FREE Full text] [CrossRef] [Medline]
- Barlow J, Smailagic N, Huband N, Roloff V, Bennett C. Group-based parent training programmes for improving parental psychosocial health. Cochrane Database Syst Rev 2012 Jun 13(6):CD002020. [CrossRef] [Medline]
- Achenbach TM, Rescorla LA. Manual for the ASEBA Preschool Forms & Profiles. Burlington, VT: AESBA; 2000.
- American Psychiatric Association. Anxiety disorders. In: Diagnostic and Statistical Manual of Mental Disorders, 5th ed. Washington DC: American Psychiatric Publishing; May 22, 2013.
- Sourander A. Emotional and behavioural problems in a sample of Finnish three-year-olds. Eur Child Adolesc Psychiatry 2001 Jun;10(2):98-104. [CrossRef] [Medline]
- Rescorla LA, Achenbach TM, Ivanova MY, Harder VS, Otten L, Bilenberg N, et al. International comparisons of behavioral and emotional problems in preschool children: parents' reports from 24 societies. J Clin Child Adolesc Psychol 2011;40(3):456-467 [FREE Full text] [CrossRef] [Medline]
- Kimonis ER, Fanti KA, Anastassiou-Hadjicharalambous X, Mertan B, Goulter N, Katsimicha E. Can Callous-Unemotional Traits be Reliably Measured in Preschoolers? J Abnorm Child Psychol 2016 May 7;44(4):625-638 [FREE Full text] [CrossRef] [Medline]
- Ezpeleta L, de la Osa N, Granero R, Penelo E, Domènech JM. Inventory of callous-unemotional traits in a community sample of preschoolers. J Clin Child Adolesc Psychol 2013;42(1):91-105. [CrossRef] [Medline]
- Barkley RA, Murphy KR. Home situations questionnaire. In: Attention-Deficit Hyperactivity Disorder: A Clinical Workbook, 2nd Edition. New York City, New York: Guilford Press; 1998.
- Arnold DS, O'Leary SG, Wolff LS, Acker MM. The Parenting Scale: A measure of dysfunctional parenting in discipline situations. Psychological Assessment 1993;5(2):137-144. [CrossRef]
- Rhoades KA, O'Leary SG. Factor structure and validity of the parenting scale. J Clin Child Adolesc Psychol 2007;36(2):137-146. [CrossRef] [Medline]
- Lovibond SH, Lovibond PF. Manual for the Depression Anxiety Stress Scales, 2nd ed. Sydney, NSW: Psychology Foundation of Australia; 1995.
- Henry JD, Crawford JR. The short-form version of the Depression Anxiety Stress Scales (DASS-21): construct validity and normative data in a large non-clinical sample. Br J Clin Psychol 2005 Jun;44(Pt 2):227-239. [CrossRef] [Medline]
- Chacko A, Jensen SA, Lowry LS, Cornwell M, Chimklis A, Chan E, et al. Engagement in Behavioral Parent Training: Review of the Literature and Implications for Practice. Clin Child Fam Psychol Rev 2016 Sep;19(3):204-215. [CrossRef] [Medline]
- Dadds MR, Sicouri G, Piotrowska PJ, Collins DA, Hawes DJ, Moul C, et al. Keeping Parents Involved: Predicting Attrition in a Self-Directed, Online Program for Childhood Conduct Problems. J Clin Child Adolesc Psychol 2019;48(6):881-893. [CrossRef] [Medline]
- Ozbek A, Gencer O, Mustan AT. Which parents dropout from an evidence-based parenting programme (Triple-P) at CAMHS? Comparison of programme-completing and dropout parents. Clin Child Psychol Psychiatry 2019 Jan;24(1):144-157. [CrossRef] [Medline]
- Kazdin AE, Whitley MK. Pretreatment social relations, therapeutic alliance, and improvements in parenting practices in parent management training. J Consult Clin Psychol 2006 Apr;74(2):346-355. [CrossRef] [Medline]
- Schmidt F, Chomycz S, Houlding C, Kruse A, Franks J. The Association Between Therapeutic Alliance and Treatment Outcomes in a Group Triple P Intervention. J Child Fam Stud 2013 Jul 3;23(8):1337-1350. [CrossRef]
- van der Zanden RAP, Speetjens P, Arntz KS, Onrust SA. Online group course for parents with mental illness: development and pilot study. J Med Internet Res 2010 Dec 19;12(5):e50 [FREE Full text] [CrossRef] [Medline]
- Enebrink P, Högström J, Forster M, Ghaderi A. Internet-based parent management training: a randomized controlled study. Behav Res Ther 2012 Apr;50(4):240-249. [CrossRef] [Medline]
- Kable JA, Coles CD, Strickland D, Taddeo E. Comparing the Effectiveness of On-Line versus In-Person Caregiver Education and Training for Behavioral Regulation in Families of Children with FASD. Int J Ment Health Addict 2012 Dec;10(6):791-803 [FREE Full text] [CrossRef] [Medline]
- Hall CM, Bierman KL. Technology-assisted Interventions for Parents of Young Children: Emerging Practices, Current Research, and Future Directions. Early Child Res Q 2015;33:21-32 [FREE Full text] [CrossRef] [Medline]
- Baumel A, Muench F, Edan S, Kane JM. Objective User Engagement With Mental Health Apps: Systematic Search and Panel-Based Usage Analysis. J Med Internet Res 2019 Sep 25;21(9):e14567 [FREE Full text] [CrossRef] [Medline]
- Torous J, Lipschitz J, Ng M, Firth J. Dropout rates in clinical trials of smartphone apps for depressive symptoms: A systematic review and meta-analysis. J Affect Disord 2020 Feb 15;263:413-419. [CrossRef] [Medline]
- Karyotaki E, Efthimiou O, Miguel C, Bermpohl FMG, Furukawa TA, Cuijpers P, Individual Patient Data Meta-Analyses for Depression (IPDMA-DE) Collaboration, et al. Internet-Based Cognitive Behavioral Therapy for Depression: A Systematic Review and Individual Patient Data Network Meta-analysis. JAMA Psychiatry 2021 Apr 01;78(4):361-371 [FREE Full text] [CrossRef] [Medline]
- Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev 2012 Jun;32(4):329-342. [CrossRef] [Medline]
- Blasé K, Fixsen DL. Core intervention components: identifying and operationalizing what makes programs work. ASPE: Office of the Assistant Secretary for Planning and Evaluation. 2013 Jan 13. URL: https://aspe.hhs.gov/report/core-intervention-components-identifying-and-operationalizing-what-makes-programs-work [accessed 2021-02-15]
- Linardon J, Cuijpers P, Carlbring P, Messer M, Fuller-Tyszkiewicz M. The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry 2019 Oct;18(3):325-336 [FREE Full text] [CrossRef] [Medline]
- Flückiger C, Del Re AC, Wampold BE, Horvath AO. The alliance in adult psychotherapy: A meta-analytic synthesis. Psychotherapy (Chic) 2018 Dec;55(4):316-340. [CrossRef] [Medline]
- Juengsiragulwit D. Opportunities and obstacles in child and adolescent mental health services in low- and middle-income countries: a review of the literature. WHO South East Asia J Public Health 2015;4(2):110-122 [FREE Full text] [CrossRef] [Medline]
- Measuring progress: Commitments to support and expand the mental health workforce in England. British Medical Association. URL: https://www.bma.org.uk/collective-voice/policy-and-research/education-training-and-workforce/mental-health-workforce-report [accessed 2021-02-15]
|CBCL: Child Behavior Checklist|
|DASS-21: 21-item Depression, Anxiety and Stress Scale|
|ICU: Inventory of Callous-Unemotional Traits|
|RCT: randomized controlled trial|
|SFSW: Strongest Families Smart Website|
Edited by A Mavragani; submitted 03.03.21; peer-reviewed by Y Perry, A Woolard; comments to author 20.04.21; revised version received 31.08.21; accepted 15.12.21; published 04.04.22Copyright
©Andre Sourander, Terja Ristkari, Marjo Kurki, Sonja Gilbert, Susanna Hinkka-Yli-Salomäki, Malin Kinnunen, Laura Pulkki-Råback, Patrick J McGrath. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 04.04.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.