The Curtin University ethics committee approved this study and all participants provided signed consent. Participants were recruited via the Curtin University Psychology Clinic and were either self-referred or referred from community health agencies via telephone, letter, fax, or email. Participant recruitment began in January 2010 and ended in April 2011. The recruitment ended because no more time was available for this activity during the course of the degree. We recruited 29 participants but 3 did not meet the inclusion criteria. Inclusion criteria consisted of a primary diagnosis of a Diagnostic and Statistical Manual of Mental Disorders, 4th edition text revision (DSM-IV-TR) [35] Axis-I disorder, aged 18-65 years old, and living in Perth, Western Australia. Exclusion criteria included a DSM-IV-TR [35] diagnosis of anorexia, psychosis (past or present), or a personality disorder as the primary diagnosis, as well as any self-harm or suicidal behaviors currently receiving psychotherapy and/or involvement in legal proceedings. Figure 1 displays the participant flow over the course of the study. All assessments, treatment, and data collection were conducted at the university clinic.

The sample included 26 participants, aged 18-59 years (mean 30 years, SD 11 years). The videoconferencing condition included 6 males and 8 females; the in-person condition included 5 males and 7 females. An in-person clinical interview along with the Structured Clinical Interview for the DSM (SCID) [36,37] was used to screen the participants. The Interference of Severity Scale adapted from the Anxiety Disorders Interview Schedule [38] was used to determine which disorder should be the primary focus of treatment. The average number of presenting disorders in both conditions was 3, and there were only 2 clients who presented with only 1 diagnosis. The number of clients in each condition who met the criteria for various Axis-I and Axis-II disorders is displayed in Tables 1 and 2. No data were collected regarding clients’ prior experience with computers and videoconferencing technology. Six participants reported that they were on antidepressants before starting the study. A record of medication usage by participants was not taken, but they were asked not to change their medication dosage during the course of the study.

For each client, an individualized CBT formulation was devised. Manualized CBT interventions were used as a guide to planning and implementing treatment for specific disorders such as panic/agoraphobia [39], depression/dysthymia [3], hypochondriasis [40], generalized anxiety [41], and obsessive-compulsive disorder [42]. Other conditions were treated with standard CBT techniques such as psychoeducation, symptom monitoring, cognitive restructuring, and exposure exercises relevant to their presenting symptoms. The interventions were focused on addressing symptoms of the primary Axis-I diagnosis. Axis-II conditions were noted in the assessment sessions but were not targeted during the course of treatment.

The Depression Anxiety and Stress Scale (DASS) [43] was used to measure global clinical symptoms. The Quality of Life Enjoyment and Satisfaction scale (QLES) [44] was used to measure changes in quality of life as a result of treatment. Given that a mixed diagnostic cohort was used, disorder specific measures for the primary presenting condition were also administered. These included the Beck Depression Inventory-II [45], Obsessive-Compulsive Inventory [46], Health Anxiety Questionnaire [47], Penn State Worry Questionnaire [48], and the Anxiety Sensitivity Index [49]. The DASS, QLES, and disorder-specific measures were administered before treatment, after treatment, and 6 weeks after treatment had ended. The Working Alliance Inventory Short Form [50] was used to measure the strength of the working alliance from both the clients’ and therapists’ perspectives. Client satisfaction was measured using the shortened Client Satisfaction Questionnaire [51]. The Telehealth Satisfaction Questionnaire [52] was used to measure client satisfaction specifically with the technology. Both the Working Alliance Inventory Short Form and the Client Satisfaction Questionnaire were administered at posttreatment to all clients; the Telehealth Satisfaction Questionnaire was administered at posttreatment only to clients allocated to the videoconferencing condition. All questionnaires were completed by hand.

Three Mac computers were used during the course of this study. All three computers were connected via ethernet (100 Mbps/sec) within the same building. The program used to conduct the videoconferencing was iChat, versions 4.0.9 to 5.0.3. When the study was conducted, the interactions via iChat were encrypted, but this feature was removed in Mac OS X 10.7. Therefore iChat may no longer be suitable for future telepsychology research.

The initial assessment consisted of a clinical interview, the SCID, and administration of the DASS and QLES. The therapist (primary author) who conducted the screening, diagnosis, and treatment was a provisionally registered (trainee) clinical psychologist doctoral student. Following the initial assessment, clients were randomly allocated to receive treatment either in-person or via videoconference. Simple random allocation was used to determine assignment to condition. This was achieved by generating a randomized list of binary numbers [53]. The generated list was calculated on the basis of 200 potential participants, two groups, and one repeat of randomization. Clients in the in-person condition began “treatment as usual”. Clients in the videoconferencing condition were instructed to walk into the treatment room at the clinic during their allotted time and sit in front of the computer. After the initial diagnosis session, clients allocated to the videoconferencing condition had no other in-person contact with the treating therapist. Materials, such as new homework diaries, were placed in the treatment room before the client arrived. Participants in the videoconferencing condition shared the content of their homework diaries orally; therefore, they did not need to hand in material to the therapist. All of the client sessions were recorded so that the fidelity and credibility of the therapists’ diagnosis and treatments could be monitored. The clinical research supervisor (the second author, a registered clinical psychologist) provided weekly supervision for a minimum of 1 hour to the practicing psychologist (primary author) throughout the duration of data collection. During supervision, videotaped sessions of all clients in the study were observed and checked for adherence to CBT protocols and that the majority of the session time adhered to the protocol. Clients who took part in the study were offered 12 weekly 1-hour sessions and an additional follow-up session 6 weeks after the 12th session. A fee for clinical services was not charged, and no reimbursement for their time was provided. In-session exposure activities were limited to tasks that could be accomplished within the treatment room. Sessions times were arranged directly with the therapist; therefore, there was no need for participants in either condition to interact with support staff such as a receptionist. Participants were aware that the therapist was located within the same building. The CONSORT-EHEALTH guidelines for improving and standardizing the reporting of Web-based and mobile health interventions were used as a guide throughout this research [54].

The research design included one nominal fixed effect (condition: in-person versus videoconference), one ordinal fixed effect (time: pre, post, follow-up), one nominal random effect (participant), and four scale outcomes (depression, anxiety, stress, and quality of life). The data generated by this design were analyzed with a multilevel mixed effects linear regression model [55] as implemented through the Generalized Linear Mixed Models procedure in the SPSS software, version 19 [56]. The analysis was “multilevel” in the sense that it was conducted within the context of a hierarchical data structure in which time was nested within participant. In order to optimize the likelihood of convergence, a separate analysis was run for each of the four outcomes. With a predicted large effect size of d=0.5, an alpha level of .05, desired power of 0.8, and a correlation of 0.6 between repeated measures, the estimated total sample size using G-Power [57] was 26 (13 participants per condition). The original intent was to have sufficient power to detect a small difference, which would have required 115 participants. However, this target became impractical given the time and resource constraints.

Of the clients eligible to take part in the study, 81% (21/26) completed the full course of treatment, and 65% (18/26) completed the follow-up data. Figure 1 shows the participant flow over the course of the study. In the videoconferencing condition, 2 of the participants completed only 10 sessions: the first, because they moved overseas, and the second stated that they had done all the change they felt capable of doing at that time. In the videoconferencing condition, 2 of the participants were no longer eligible for inclusion in the study at follow-up: the first, because they required ongoing treatment after the posttreatment data had been collected and the second decided to begin medication in the final week of treatment. In the in-person condition, 1 participant dropped out after nine sessions and no reason was given. Also in the in-person condition, 1 participant was no longer eligible to be included in the study at follow-up because they commenced additional ongoing psychotherapy treatment after the initial 12-weeks of treatment in the study. The analyses were conducted twice: once with only the participants that completed all 12 sessions and once with all the participants that completed eight or more sessions. The results of the two analyses provided the same findings; therefore, participants who completed eight or more sessions were included in the final analyses.

There was a small nonsignificant difference between the mean ages of participants in the in-person condition (mean 29.67, SD 9.31) and the videoconferencing condition (mean 31.93, SD 13.33) (t ₂₄=–0.49, P=.63, two-tailed, 95% CI -11.73 to 7.21, d=–0.19). A Pearson chi-square test of contingencies indicated that gender was equally distributed across conditions χ² ₁=0.004 (N=26), P=.95, and the association between gender and condition was very small with phi=.01. The difference between conditions at pretest was not significant on the DASS depression subscale (t ₂₃=0.05, P=.96, two-tailed, 95% CI -8.28 to 8.56, d=0.02), the anxiety subscale (t ₂₃=0.37, P=.71, two-tailed, 95% CI -5.98 to 8.57, d=0.15), the stress subscale (t ₂₃=0.19, P=.85, two-tailed, 95% CI -8.36 to 6.98, d=–0.08), or on the QLES (t ₂₄=0.28, P=.78, two-tailed, 95% CI -0.47 to 0.62, d=0.10).

In order to reduce the chances of a Type-1 error, the Bonferroni correction was applied throughout the four analyses making the alpha level .0125. The descriptive statistics for the in-person and videoconferencing conditions are reported in Table 3. The interaction between time and condition was not significant for all three DASS subscales (Depression F _2,58=1.77, P=.18; Anxiety F _2,58=0.36, P=.7; and Stress F _2,58=4.19, P=.02), and the QLES (F _2,62=0.82, P=.45); therefore, the main effects can be interpreted without qualification. There was a significant main effect for time on all three DASS subscales (Depression F _2,58=14.47, P<.001; Anxiety F _2,58=9.34, P<.001; and Stress F _2,58=23.70, P<.001) and the QLES (F _2,62=10.64, P<.001). The effect sizes were large throughout (d=1.41, 1.14, 1.81, and 1.17 respectively). In contrast, there was no significant main effect for condition on any of the DASS subscales (Depression F _1,58=1.98, P=.16; Anxiety F _1,58=0.69, P=.41 and Stress F _1,58=2.11, P=.15) or the QLES (F _1,62=0.25, P=.62). The effect sizes ranged from small to medium (d=0.37, 0.22, 0.38, and 0.13 respectively). Figures 2-5 depict the upper and lower bound of the 95% confidence intervals around the mean scores at pre, post, and 6 weeks following treatment in both conditions for the three DASS subscales and the QLES.

The secondary analyses were administered only at posttest. There were no significant differences between conditions in client ratings of the Working Alliance Inventory Short Form (t ₂₁=–0.63, P=.53, one-tailed, d=–0.26), or in therapist ratings (t ₂₂=0.53, P=.60, one-tailed, d=0.23), and client ratings of the Client Satisfaction Questionnaire (t ₂₂=–0.29, P=.77, one-tailed, d=–0.12). Clients in the videoconference-based condition who completed 10 or more sessions were included in the Telehealth Satisfaction Questionnaire analysis (n=13). Scores ranged from 61.4 to 100 (maximum score is 100), with a mean of 90.44 (SD 11.11).

Reliable change and clinical significance calculations [58] were conducted on the DASS subscales, the QLES, and disorder specific measures. Normative data were used to calculate the cutoff criteria [3,43-47,59-64]. Tables 4-6 display the proportion of clients in each condition that met the respective criteria for the three DASS subscales and the QLES respectively.

Fisher’s Exact test [65] was used to compare the proportion of participants meeting the criteria for reliable change and clinical significance in each condition. With regards to reliable change, Fisher’s Exact was not significant from pre- to posttreatment or from pretreatment to follow-up for the depression (P=.41, P=.26) and anxiety (P=.60, P=.99) DASS subscales, and the QLES (P=.65, P=.99). Fisher’s Exact P, however, was significant for the stress DASS subscale from both pre- to posttreatment (P=.03) and from pretreatment to follow-up (P=.03) in favor of the videoconferencing condition. With regards to clinically significant change, the difference between conditions was not significant from pre- to posttreatment or from pretreatment to follow-up for the depression (P=.67, P=.30), anxiety (P=.99, P=.99), and stress (P=.19, P=.13) DASS subscales or the QLES (P=.99, P=.62). Table 7 displays the number of clients that upon discharge met the criteria for both reliable change (improved) and clinically significant change (recovered) for the various disorder specific measures administered in each condition. However, not all clients presented with severe symptoms at pretreatment. The proportion of clients whose symptoms were identified in the moderate to severe range at pretreatment are displayed in Table 8.

The findings of this controlled trial indicate that CBT was effective in significantly reducing symptoms of depression, anxiety, and stress and increasing quality of life in both in-person and videoconferencing conditions. Furthermore, outcomes for the videoconferencing group were comparable to the in-person group, with no significant differences being observed between the two conditions. This finding is consistent with prior CBT meta-analysis research [66,67] and prior telepsychology research [18,25-27,68-71]. Although the sample was not large enough to permit a test of noninferiority, the 95% confidence intervals around the mean scores across time and condition suggest CBT treatment administered via videoconference is not inferior to treatment provided in-person. This finding is congruent with prior noninferiority analyses [28,70]. It is also important to note that the clinical outcomes at posttreatment and follow-up were better, albeit not significantly better, in the videoconferencing condition than in the in-person condition. Therefore, if this study had a greater power to detect an effect, it is unlikely this study would have shown videoconference-based CBT to be inferior to in-person.

Client and therapist ratings of the working alliance were high in both the in-person and videoconferencing conditions. The difference between conditions was not significant, and the effect size was trivial. These findings suggest that CBT treatment via videoconference does not compromise the working alliance, which is congruent with prior telepsychology research [26,72]. The results also indicated that client satisfaction was high and did not significantly differ between conditions, which again is congruent with prior research [19,20,73].

The inclusion of a mixed diagnostic client cohort with comorbid diagnosis (as measured in a structured clinical interview) is a strength of this study. A client group with a range of disorders and comorbidities helps in generalizing the findings to real-world clinical practice populations and thus provides empirical support for the effectiveness of CBT-oriented telepsychology. Second, the same therapist treated all clients, and thus this variable was held constant. By including only 1 therapist in the study, the difference between conditions cannot be attributed to differences between therapists. A further strength of this study is that it was conducted on a high-speed local network. As a result of conducting the study on a system that is presently faster than the average Australian Internet speed, it may be a longer period of time before the conclusions of this study become outdated. Also, all of the participants in both conditions received treatment in the same building; consequently, this study was able to demonstrate that satisfaction remained high even when travel was not a contributing factor in the ratings of service satisfaction. No technological problems occurred during the study, and clients did not report any complaints about the technology.

Although this study has successfully extended previous telepsychology research [25-27,30,69,70], there are some limitations. First, because only 1 therapist was included in this study, it remains unclear as to whether or not the findings generalize to other therapists. Second, the diagnosis of clients in both conditions was conducted in-person by the same therapist who conducted the subsequent treatment. It is possible that rapport and a therapeutic alliance began in this initial in-person diagnosis session and influenced clinical outcomes. Third, the sample size was not large enough to conduct an analysis of noninferiority. The limited small sample size also weakens the generalizability of the findings. Fourth, there was a high risk of bias in the study given that the principal investigator was also responsible for treatment and assessing participants. However, clinical supervision was provided regarding both diagnosis and treatment throughout the study, and sessions were recorded and watched by the supervising clinician to temper this potential bias. Fifth, the diagnosis of participants was not evenly distributed across conditions, and as a result, the two groups might not have been equally matched. Sixth, as is the case in most research studies, the participants did not have to pay for the services, and this may have led them to provide socially desirable answers on the questionnaires. A replication study involving a fee for service would be able to clarify if this is an issue. Seventh, the average age of participants was 30; therefore, it is likely that most participants were already familiar with computer technology, which may have influenced their acceptance of the media.

Despite being a strength of the study, the inclusion of a mixed diagnostic cohort combined with a small sample size means there was a large degree of variability across the clients, which may have reduced the power of the study. Also, Bonferroni corrections were applied to reduce the risk of a Type-I error, further reducing the power of the study to detect significant interactions between time and condition. While providing promising results, a larger study is still needed to more thoroughly address the primary research question.

Not all outcomes in this study were favorable. Only 10% (1/10) of clients in the in-person condition and 33% (2/13) of participants in the videoconference condition were identified as meeting the criteria for a clinically significant reduction in anxiety. These results could be because 33% (4/12) and 36% (5/14) (in-person and videoconference respectively) of participants were identified on the DASS as having normal-to-mild levels of anxiety before beginning treatment. Alternatively, the therapist may not have been as competent at treating symptoms of anxiety compared to depression for which 43% (3/7) and 77% (7/9) of participants (in-person and videoconference respectively) were identified as meeting the criteria for clinically significant change. The presence of comorbid personality disorders may also have hampered treatment outcomes. In either case, a larger trial involving multiple therapists, systematic monitoring of treatment protocol adherence and a single diagnosis may help to overcome these issues in future research.

Some psychotherapy procedures via videoconference required creative modification. In-session exposure exercises via videoconference had to be limited to what could be conducted within the clinic room. For example a client working through exposure-response prevention activities pertaining to germs had to find items they found troubling that were in their treatment room as opposed to in-person sessions that could have the freedom to move outside of the treatment room. Another change was that cognitive restructuring worksheets had to be conducted on a digital Microsoft Word document on the clients’ computer but controlled from the therapist’s computer. These examples show that treatment via videoconference may alter how some interactions are conducted but does not prevent essential treatment components from being provided.

It is important to consider the findings of this study within the context of the social justice, financial, and practical issues facing remote psychological services in Australia. In Australia, there is a disparity in access to psychological services between people who live in cities and those who live in rural and remote areas [74]. Rural and remote populations have higher rates of accidents, suicide, and exposure to violence [75] as well as increased rates of risk factors known to play a role in the emergence of psychological dysfunction, including poor physical health, obesity, smoking, drug abuse, high blood pressure, and poor nutrition [76]. As for the indigenous populations in rural and remote communities, the state of affairs is considerably worse where the rates of mortality and morbidity are approximately 2-4 times higher than for nonindigenous Australians [77]. People in rural and remote Australia are in the highest need of specialized mental health services; however, due to the lack of specialized clinicians working in these areas, the accessibility of those services frequently range from little to none [76]. Hence, there is a substantial demand for videoconference-based psychological services. However, there is often insufficient technological infrastructure to provide reliable high-speed broadband Internet communications, particularly in remote areas where there can be less than one person per square kilometer [78]. It is hoped that with the rollout of the National Broadband Network in Australia more rural and remote areas will gain access to high-quality videoconferencing technology. Prior research [17,79] has indicated that providing remote services via videoconference can be cost effective under some circumstances, but further research is needed to determine if it is also cost effective for small isolated indigenous communities. Further research is needed to identify the logistical challenges of providing videoconference-based CBT to rural and remote communities and to determine which method of service delivery is best suited to the needs of specific communities. Telepsychiatry services through medicare in Australia have increased from 15 consultations in 2002/3 to 2555 in 2010/11 [80]. This indicates that when available, both service providers and clients are willing to use the technology. Hopefully, this trend will extend to telepsychology services in the future as access to the technology increases.

In conclusion, this study provides support for the effectiveness of CBT via telepsychology for an adult client cohort and preliminary support of noninferiority. The results of this controlled trial provide important evidence to justify the greater use of videoconferencing to bridge the gap in service provision to populations who would otherwise not receive effective psychological treatments. Future research should continue to involve mixed diagnostic client cohorts with comorbid diagnoses and be directed at conducting large multisite randomized controlled trials that employ noninferiority design and analysis procedures.