This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Training mental health professionals to deliver evidence-based therapy (EBT) is now required by most academic accreditation bodies, and evaluating the effectiveness of such training is imperative. However, shortages of time, money, and trained EBT clinician teachers make these challenges daunting. New technologies may help. The authors have developed the first empirically evaluated comprehensive Internet therapist training program for interpersonal psychotherapy (IPT).
The aim of this study was to examine whether (1) the training protocol would increase clinicians’ knowledge of IPT concepts and skills and (2) clinicians would deem the training feasible as measured by satisfaction and utility ratings.
A total of 26 clinicians enrolled in the training, consisting of (1) a Web-based tutorial on IPT concepts and techniques; (2) live remote training via videoconference, with trainees practicing IPT techniques in a role-play using a case vignette; and (3) a Web-based portal for therapists posttraining use to help facilitate implementation of IPT and maintain adherence over time.
Trainees’ knowledge of IPT concepts and skills improved significantly (
Results support the efficacy and feasibility of this technology in training clinicians in EBTs and warrant further empirical evaluation.
The importance of preparing mental health professionals to deliver evidence-based therapy (EBT) is now well established [
Interpersonal psychotherapy (IPT) is one of the oldest and best-studied EBTs [
With the increasing emphasis on EBTs comes an increasing focus on how to implement and evaluate psychotherapy training. Accreditation bodies in psychiatry (Accreditation Council for Graduate Medical Education) and psychology (American Psychological Association) require not only training in EBTs but an evidence-based approach to evaluating the effectiveness of such training, that is, whether clinicians demonstrate competence in administering the treatment [
New technologies may help [
In this pilot study, we developed the first comprehensive Internet training program for IPT to be empirically evaluated. This three-part, interactive therapist training protocol focuses on IPT for major depression and consists of (1) a Web-based tutorial on IPT concepts and techniques; (2) live remote training via videoconference, with trainees practicing IPT techniques in a role-play using a case vignette; and (3) a Web-based portal for therapists posttraining use to help facilitate integration of IPT into their clinical practice and maintain adherence and quality over time. The goal of the study was to examine the following hypotheses: (1) the training protocol would increase clinicians’ knowledge of IPT concepts and skills from baseline and (2) clinicians would deem the training feasible as measured by satisfaction and utility ratings.
Before training, trainees took a 38-item pretest on their knowledge of IPT concepts and principles. Following the pretest, they received a username and password to access the Web-based tutorial and completed it at their own pace. Trainees could email the instructors with questions about the material. After completing the Web-based tutorial, trainees took a posttest of IPT knowledge and a user satisfaction questionnaire. Trainees then received a 45-60 min live applied training session conducted via videoconference with an experienced IPT trainer (JDL, JCM, or KLB). During this session, the trainer portrayed a standardized depressed patient, whereas the trainee role-played as therapist (see below). After completing the video session, trainees completed a satisfaction questionnaire and received a link to the IPT posttraining website. The posttraining website was designed to facilitate implementation and adherence following training and to guide the clinician in structuring sessions with their first IPT patients.
The training components paralleled components of IPT. In IPT, the patient and IPT therapist together define a central interpersonal problem focusing on one of four categories: grief, role transition, role dispute, or interpersonal deficits [
Learning objectives by module: interpersonal psychotherapy (IPT) tutorial.
Module | Learning goal |
Welcome and overview | Describe the goals of the tutorial |
Principles of IPTa for depression | Describe the theoretical roots for IPT |
Describe the IPT theory for the cause and treatment of depression | |
Describe the role of the IPT therapist |
|
Describe the three phases of IPT treatment, the interpersonal inventory, and developing a case formulation | |
3: The four IPT problem areas: grief | Explain the difference between normal grief and abnormal grief (complicated bereavement) |
Describe the therapeutic goals in treating depression resulting from abnormal grief | |
Describe questions to use in order to assess the presence of abnormal grief | |
Describe how to facilitate the grieving process | |
The four IPT problem areas: role transition | Describe a role transition |
Describe how a role transition may result in depression | |
Identify when a role transition is an issue for a patient | |
Explain the treatment goals in treating depression resulting from a role transition | |
The four IPT problem areas: role dispute | Describe the nature of role disputes |
Identify when a role dispute is an issue for a patient | |
Identify the three stages in role disputes | |
Describe the therapeutic goals in treating patients presenting depression resulting from a role dispute | |
The four IPT problem areas: interpersonal deficits | Describe when interpersonal deficits are the focus of treatment |
Describe the treatment goals in IPT in treating depression resulting from interpersonal deficits | |
Describe how patients with interpersonal deficits differ from patients presenting with depression resulting from the other three problem areas | |
Mechanisms of change in IPT | Describe the four ways in which IPT achieves therapeutic goals |
aIPT: interpersonal psychotherapy.
Because multi-modal learning and high levels of interactivity enhance learning [
After completing the Web-based tutorial and Web-based posttest, trainees completed a supervised clinical training session using videoconferencing. During the applied session, the trainer portrayed a standardized depressed patient, whereas the trainee role-played the IPT therapist. The goal was to offer trainees practice in applying the skills learned during the tutorial. The trainer provided feedback and suggestions in real time as appropriate. Trainees also had the opportunity to ask questions about the IPT approach. The role play was designed to portray the patient’s second or third session, to provide the trainee an opportunity to practice developing an interpersonal formulation with the patient, and achieving agreement with the patient on the focal IPT problem area. We did not role play initial sessions because these typically focus on history gathering, which we finessed by providing summary information on the patient’s history up front to the trainee. Starting a session with “How have things been since we last met?” is a pattern that begins with session 2. A crucial point in IPT treatment is the therapist’s formulation of a focal problem area in session 2 or 3 and getting the patient’s agreement on it; this then organizes the remainder of the treatment [
A limitation of many professional training programs is lack of carryover to practice [
To assess trainees’ gains in knowledge of the concepts, principles, and techniques of IPT, the authors developed a 38-item pre- and posttest covering the core concepts in the tutorial. The text contained a combination of multiple-choice and true-false questions in proportions mandated by continuing education guidelines from the American Psychological Association and the National Association of Social Work. Testing served dual functions of assessing and reinforcing learning. Trainees received rationales for the correct answers after completing the posttest to reinforce learning. Posttests were given after completion of each module. The test had good internal consistency reliability (coefficient alpha =.79).
We evaluated user satisfaction from two perspectives: technical implementation, and clinical content. User satisfaction with technical aspects of the training tutorial was assessed using the System Usability Scale (SUS) [
Descriptive statistics assessed trainee satisfaction with the clinical content of the training components. Trainees rated each training component along six dimensions using a 4-point scale (strongly agree, agree, disagree, and strongly disagree). Trainees also rated global satisfaction and had opportunity for open-ended feedback. Scale items were developed in prior studies on user satisfaction with Web-based training [
A two-tailed paired
This study has been approved by the Allendale Institutional Review Board. All procedures performed in this study involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments. Informed consent was obtained from all individual participants included in the study. This article does not contain any studies with animals performed by any of the authors.
Clinicians were recruited through advertisements in professional journals from National Association of Social Work and the American Psychological Association and through an announcement on the International Society for Interpersonal Psychotherapy (ISIPT) listserv. A total of 35 clinicians inquired about the study and were offered free participation. Of these, 26 (74%, 26/35) enrolled in the study and started the tutorial, and 22 (62%, 22/35) completed it. Furthermore, 18 (51%, 18/35) trainees participated in the live applied training session after completing the tutorial. The Allendale Institutional Review Board approved the study protocol, and all participants signed informed consent statements.
Of the 26 community clinicians starting the Web-based tutorial, 23 (88%) were female, 22 (85%) white, 1 (4%) African American, 4 (15%) Hispanic, and 3 (11%) other or mixed racial categories. Participants came from 15 US states and one each from Mexico, Brazil, Canada, and the United Kingdom. Mean age was 41.6 years (standard deviation [SD]=11.5, range: 26-63 years), and mean years of clinical experience was 10.5 (SD=7.6, range: 1-26 years). Eleven (42%, 11/26) were social workers, 11 (42%, 11/26) psychologists, 2 (8%, 2/26) marriage and family therapists, 1 (4%, 1/26) a psychiatrist, and 1 (4%, 1/26) a psychiatric nurse. Additionally, 24 (92%, 24/26) were actively conducting psychotherapy with clients. Only 3 (11%) reported having received any prior formal training in IPT: one through a continuing education workshop, one as part of undergraduate coursework, and one in graduate coursework. Three participants (14%, 3/26) reported having used some IPT techniques in their practice before participating in the study.
The mean number of correct answers on the 38-item IPT concepts and skills quiz improved significantly from 16.5 (SD 4.6) on pretest to 27.5 (SD 4.0) on posttest,
The SUS evaluated user satisfaction with the technical features of the Web-based tutorial. Mean SUS score was 90.6 (SD 11.4). This corresponds to a mean rating between “excellent” and “best imaginable.”
System Usability Scale ratings of user satisfaction with technical features of Web-based tutorial.
User satisfaction with the tutorial was also evaluated by whether clinicians felt the learning objectives of each module were met. Nineteen learning objectives were identified a priori for the seven modules (
Global ratings of how much the trainee learned (scale range from 1=very little to 5=a great deal) was also obtained, as required by continuing education accreditation agencies. The mean rating of trainees learning from the Web-based tutorial was 3.91.
Ratings of user satisfaction: clinical content of Web-based tutorial.
Ratings of user satisfaction: applied training via videoconference.
User satisfaction with the applied training session via videoconference appears in
Ratings of user satisfaction: posttraining Web portal.
The mean time it took trainees to complete the Web-based tutorial was 3.3 hours (SD=0.8, range 2.3-5.0 hours). The average module was 29.1 min (SD 18.3). The tutorial was completed over a mean of 27.4 days (SD=22.9, range 1-66 days). The mean duration for participation in the entire training protocol (start of Web-based tutorial to evaluation of Web portal) was 91 days (SD 18.5).
This pilot study provides evidence to support the efficacy and feasibility of this technologically advanced, three-part therapist training intervention on IPT for major depression. Results supported both our hypotheses: the tutorial increased trainee knowledge of IPT, and trainees reported high levels of satisfaction with the three training components. User satisfaction has critical importance: if trainees do not like a training program, find it too difficult to use, or not useful, they will not complete it. In our study, trainees described high satisfaction with both technical aspects and clinical content of the training components and had a completion rate of 85% for the Web-based tutorial and 69% for the live applied training. This ranks somewhat higher than average compared with other Web-based trainings [
If successfully disseminated, this intervention may assist academic training programs in solving the dual challenges they face in expanding training curricula to include EBT’s and ensuring the training is effective. IPT in particular is an EBT in which most practitioners do not receive training, despite its strong empirical standing. As no treatment is universally effective, expanding training options to include multiple EBTs helps produce more well-rounded clinicians and provides more treatment options for patients, some of whom may prefer or respond better to IPT than other EBTs. IPT has been far less disseminated than CBT (which has comparable supportive evidence) and psychodynamic therapy (which has far less empirical support). IPT uses the medical model of illness which makes it very compatible with clients treated with a combination of psychotherapy and medication.
From a practical standpoint, the model this study used augments traditional approaches to training rather than replacing them [
The technology may also enhance posttraining supervision in several ways. Videoconferencing facilitates access to training supervisors. Access to a training supervisor following training has been found an essential ingredient of posttraining success superior to other forms of posttraining options such as peer consultation [
Results from this study are consistent with other studies on the use of these technologies for training clinicians on other EBTs such as CBT [
Limitations of this pilot study include lack of randomization and a control group, such as comparison to current standard methods of clinical instruction. The small sample size limits generalizability of results. In addition, since we did not collect patient data, it is impossible to know how the training program affected clinical practice. Although trainees received no feedback following pretesting on conceptual knowledge, there is the potential for practice effects. In the absence of a control group, it is difficult to determine the extent to which improvement in scores on the tutorial was due to training as opposed to induction bias. However, some studies have found pretests increase learning by orienting learners to subsequent information [
Future studies could include a larger sample size, as well a cohort of recent graduates from academic internships who are interested in additional psychotherapy training following graduation. Such training could count toward continuing education credits [
Example of Web-based tutorial multimedia content.
Example of clinical vignettes and challenge questions.
Example of pre-session checklist from IPT Web portal.
Example of post-session checklist from IPT Web portal.
cognitive behavioral therapy
evidence-based therapy
interpersonal psychotherapy
System Usability Scale
The authors would like to thank Richard DeVouno for developing the eLearning component of the Web-based tutorial. This study was funded with federal funds from the National Institute of Mental Health, National Institutes of Health, Department of Health and Human Services, under Small Business Innovation Research (SBIR) grant number R43MH106169.
Drs Kobak, Lipsitz, and Markowitz have intellectual property rights and a proprietary interest in the Web-based training program described in this project. Dr. Markowitz receives salary support from the National Institute of Mental Health, the Earl Mack Foundation, and the New York State Psychiatric Institute; minor book royalties from American Psychiatric Publishing, Basic Books, and Oxford University Press; and an editorial stipend from Elsevier Press.