This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
In the interests of patient health outcomes, it is important for medical students to develop clinical communication skills. We previously proposed a telehealth communication skills training platform (EQClinic) with automated nonverbal behavior feedback for medical students, and it was able to improve medical students’ awareness of their nonverbal communication.
This study aimed to evaluate the effectiveness of EQClinic to improve clinical communication skills of medical students.
We conducted a 2-group randomized crossover trial between February and June 2016. Participants were second-year medical students enrolled in a clinical communication skills course at an Australian university. Students were randomly allocated to complete online EQClinic training during weeks 1–5 (group A) or to complete EQClinic training during weeks 8–11 (group B). EQClinic delivered an automated visual presentation of students’ nonverbal behavior coupled with human feedback from a standardized patient (SP). All students were offered two opportunities to complete face-to-face consultations with SPs. The two face-to-face consultations were conducted in weeks 6–7 and 12–13 for both groups, and were rated by tutors who were blinded to group allocation. Student-Patient Observed Communication Assessment (SOCA) was collected by blinded assessors (n=28) at 2 time points and also by an SP (n=83). Tutor-rated clinical communications skill in face-to-face consultations was the primary outcome and was assessed with the SOCA. We used t tests to examine the students’ performance during face-to-face consultations pre- and postexposure to EQClinic.
We randomly allocated 268 medical students to the 2 groups (group A: n=133; group B: n=135). SOCA communication skills measures (score range 4–16) from the first face-to-face consultation were significantly higher for students in group A who had completed EQClinic training and reviewed the nonverbal behavior feedback, compared with group B, who had completed only the course curriculum components (
The EQClinic is a useful tool for medical students’ clinical communication skills training that can be applied to university settings to improve students clinical communication skills development.
There is good evidence that effective patient-clinician communication can positively influence patient health outcomes [
The traditional method for clinical communication skills training is to provide students with feedback on video-recorded face-to-face consultations [
However, like WebEncounter, most clinical communication skills training systems tend to limit training to verbal communication skills and overlook important nonverbal communication behaviors. This is problematic, given that nonverbal communication is the major communication channel between individuals [
We have previously described a platform called EQClinic [
Therefore, the goal of this study was to conduct a randomized crossover trial of the EQClinic incorporated into a university medical school curriculum (
Participants were second-year undergraduate medical students from an Australian medical school. All students were enrolled in a communication skills training course provided by the medical school. Prior to this study, they were not offered any training about teleconsultations in the medical school. This study was approved by the University of New South Wales Research Ethics Committee (HC Reference Number: HC16048). Students were asked to sign an online consent form when they first accessed EQClinic. No content or methodological modifications were made after study commencement.
The same 5 surveys previously reported by our group were used in the present study [
EQClinic comprises five components: an online training component, a personal calendar, a real-time interaction component, a nonverbal behavior detector, and a feedback generator. In the following sections, we briefly describe each of these.
EQClinic provides training videos and documents for students and SPs to familiarize themselves with the platform. EQClinic also provides students and SPs with an automated personal calendar. SPs can offer their availability on the calendar for students to make a booking. All appointments are confirmed using the automated messaging system without need for human resources.
Once the appointment has been confirmed, videoconferencing enables a student and an SP to have a teleconsultation. The application works on most Web browsers of a personal computer or Android tablet. During the recorded consultation, the SP can record positive and negative moments using a “thumbs” tool and comment box.
To facilitate learning through reflection, online assessments were included for students. The SPs evaluated student performance immediately after the teleconsultation, during which time the students conduct a self-assessment using the same form. Students could immediately review the SP’s rating.
Using audio processing and computer vision techniques, EQClinic automatically analyses the video recordings and detects the following nonverbal behaviors: head movements (nodding, head shaking, and head tilting), facial expressions (smiling and frowning), body movements (body leaning, hand gestures, and overall body movements), voice properties (volume and pitch), and speech patterns (turn taking and speaking ratio changes).
Feedback information includes computer-generated nonverbal behavior feedback (NVBF) and comment feedback from the SP. EQClinic visualizes students’ nonverbal behavior using two types of feedback reports: single-feature and combined-feature reports. The single-feature feedback report illustrates each form of nonverbal behavior separately. The combined-feature feedback report displays multiple kinds of nonverbal behavior on one page. The comment feedback provides students a report that contains all the comments from the SP and tutor.
The administrator of this course randomly allocated a cohort of 268 students to group A (n=133) or group B (n=135) (see
Flowchart of student participation in the EQClinic medical communication training program.
All participating SPs and students completed training via the training component of EQClinic. In the SPs’ online training component, training videos demonstrated how to book appointments, conduct consultations with students, provide comments, and evaluate the student’s performance. The patient scenario was also included in training and detailed the main symptoms of the SP and other historical information. All SPs were required to complete this online training. Following training, the SPs listed their availability for consultations on their EQClinic calendar.
Students were requested, by email, to complete one teleconsultation with an SP through EQClinic. The email described the details of the study and asked them to log in to EQClinic to complete the training module. It also informed them that, once they finished the training, they could request a consultation time from the slots available on their personal calendar. The SPs and students were allowed to have the teleconsultation anywhere as long as there was (1) a Web browser on a personal computer or an Android tablet with an external or built-in camera and microphone, (2) a good Internet connection, and (3) good lighting.
EQClinic teleconsultation comprised three sections: interviewing, assessing, and reviewing (see
Students were emailed to ask them to return to the system 24 hours after the consultation to review different kinds of feedback, which included the video recording, comments from the SP, and automated NVBF. Students also completed the postinterview questionnaire.
Workflow for the EQClinic consultation. SOCA: Student-Patient Observed Communication Assessment; SP: standardized patient.
Face-to-face consultations were conducted in consultation rooms of a university-based clinical skills center. A trained tutor was present in the room to observe and assess the performance of the student during the consultation with the SP. The tutors were blinded to condition allocation (group A or group B). The tutor completed a SOCA form to assess the student, and the SP did not provide any evaluation and feedback for the student on this occasion. The students were asked to review the tutor’s assessment and complete the Reflection Questionnaire. The scenario design and length of face-to-face consultations were the same as those for the teleconsultations.
Mean group medical communication skills (measured by Student-Patient Observed Communication Assessment score) assessment results (part 1: weeks 1–7).
Component | Weeks 1–5 (TCa) | Weeks 6–7 (F2FCb) | ||||||
Group A (n=127) | Group A (n=59) | Group A (NVBFc) (n=33) | Group B (n=107) | |||||
Mean | SD | Mean | SD | Mean | SD | Mean | SD | |
Total score | 11.59 | 2.67 | 13.02 | 1.49 | 13.21 | 1.45 | 12.58 | 1.61 |
Providing structure | 2.88 | 0.72 | 3.17 | 0.50 | 3.27 | 0.45 | 3.12 | 0.53 |
Gathering information | 2.92 | 0.72 | 3.25 | 0.58 | 3.15 | 0.51 | 3.07 | 0.56 |
Building rapport | 2.95 | 0.73 | 3.34 | 0.60 | 3.39 | 0.66 | 3.24 | 0.56 |
Understanding patient’s needs | 2.83 | 0.80 | 3.25 | 0.58 | 3.39 | 0.56 | 3.14 | 0.61 |
aTC: teleconsultation.
bF2FC: face-to-face consultation.
cNVBF: students who had a face-to-face consultation and reviewed the nonverbal behavior feedback.
Mean group medical communication skills (measured by Student-Patient Observed Communication Assessment score) assessment results (part 2: weeks 8–13).
Component | Weeks 8–11 (TCa) | Weeks 12–13 (F2FCb) | ||||||||||
Group B (n=130) | Group A (n=109) | Group A (ConAc) (n=53) | Group B (n=35) | Group B (ConBd) (n=30) | Group B (NVBFe) (n=13) | |||||||
Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | Mean | SD | |
Total score | 13.13 | 2.31 | 13.28 | 1.54 | 13.28 | 1.46 | 13.43 | 1.63 | 13.53 | 1.52 | 13.62 | 1.64 |
Providing structure | 3.31 | 0.67 | 3.25 | 0.49 | 3.23 | 0.46 | 3.31 | 0.52 | 3.37 | 0.48 | 3.38 | 0.49 |
Gathering information | 3.34 | 0.72 | 3.26 | 0.55 | 3.30 | 0.53 | 3.51 | 0.55 | 3.53 | 0.50 | 3.46 | 0.50 |
Building rapport | 3.16 | 0.73 | 3.46 | 0.58 | 3.47 | 0.54 | 3.34 | 0.58 | 3.37 | 0.55 | 3.46 | 0.63 |
Understanding patient’s needs | 3.32 | 0.68 | 3.31 | 0.57 | 3.28 | 0.59 | 3.26 | 0.44 | 3.27 | 0.44 | 3.31 | 0.46 |
aTC: teleconsultation.
bF2FC: face-to-face consultation.
cConA: group A students who participated in two consultations (one face-to-face consultation, one teleconsultation) before week 12.
dConB: group B students who participated in two consultations (one face-to-face consultation, one teleconsultation) before week 12.
eNVBF: students who had a face-to-face consultation and reviewed the nonverbal behavior feedback.
Following group B exposure to the EQClinic, the mean total SOCA scores from the second face-to-face consultation did not differ between the groups (group A: mean 13.28; group B: mean 13.53,
We used paired-samples
We incorporated EQClinic into a medical communication skills teaching curriculum to provide students with additional practice opportunities with SPs. Importantly, students could review their nonverbal communication behaviors. We examined the effects of EQClinic on medical students’ learning of communication skills evaluated via the students’ assessment (SOCA) scores. Results showed that students who completed a teleconsultation using EQClinic and reviewed the NVBF achieved higher SOCA scores in the first face-to-face consultation. In addition, students accomplished higher SOCA scores in their second face-to-face consultation if they completed a teleconsultation between the two face-to-face consultations.
Overall, adherence to the program was somewhat less than anticipated, with only 30% of student completing all components of the study. Dropout increased as the semester progressed. However, given the requirements of the undergraduate course, and the tendency for increased workload as the semester progresses, this result is unsurprising.
The results of the first face-to-face consultation show that the students who completed a teleconsultation and reviewed the NVBF component scored significantly higher in their face-to-face consultation than did students who did not interact with SPs on EQClinic. These results are promising. The difference in performance between the 2 groups seems to indicate that having EQClinic practice coupled with reviewing feedback improved medical communication skills in group A. As noted above, group B students achieved lower mean overall SOCA score in the first face-to-face consultation. However, overall, group B students showed significant improvement from their first to second face-to-face consultation. However, whether students reviewed their NVBF did not influence results for this group.
These findings are interesting because they suggest improvement in communication skills assessment after reviewing nonverbal feedback. While the need for medical communication skills training is widely accepted within the medical teaching community [
Furthermore, that we showed no significant difference between group scores in the second face-to-face assessment seems to indicate that the timing of exposure to EQClinic within a teaching curriculum did not influence students’ learning results. In our study, group A was exposed to EQClinic at the beginning of the course; whereas group B was exposed in the middle of their course. We showed that at the commencement of the curriculum, when students did not have significant knowledge of clinical communication skills, exposure to EQClinic yielded a measurable bump in their clinical communication skills. For medical educators this seems to indicate that EQClinic could be incorporated at any period during the teaching curriculum.
We also showed that group B performed significantly better than group A on the SP-rated EQClinic teleconsultations. This difference could be explained in several ways. The first way relates to timing of EQClinic exposure, with group B completing the teleconsultation later in the semester than group A. Second, completing face-to-face consultations before being exposed to EQClinic, experience, and feedback garnered from the face-to-face consultation may have improved student performance in the teleconsultation. The third possibility is that the SPs who assessed students via the EQClinic increased their ratings across the semester. However, SP ratings neither contributed to student assessment nor were a central feature of the EQClinic.
Telehealth studies involving medical students and interns in urban, rural, and remote areas indicated that this medium was a useful learning tool [
However, based on our findings, it remains unclear which of the learning components were most useful to enhancing students’ learning. Moreover, although a single exposure to the EQClinic led to a measurable improvement in students’ medical communication skills scores, future studies will benefit from an examination of the appropriate “dose” of EQClinic. This will help determine the necessary exposure needed to provide sustained improvement and generalizable communication skills training. Finally, the growth of collected student data by EQClinic will aid the refinement of rules and models using machine learning algorithms to indicate to students what nonverbal behavior is associated with positive or negative responses and feedback from SPs in their clinical teleconsultations.
There are several limitations to our study that should be considered when interpreting these findings. First, the absence of baseline measures limited our ability to observe change over time. Second, all the consultations conducted in this study were limited to a history-taking scenario. In reality, clinicians encounter many different scenarios. For example, when breaking bad news to patients, the clinician has to handle difficulties related to emotions. In addition, all the students in this study were second-year medical students who had limited knowledge about communication skills. Future studies may explore whether EQClinic is also useful for senior medical students and professionals. A third limitation is the relatively low proportion of students (30%) who completed all components of the study. While the sample was still appropriate for the statistical tests conducted, future investigations will benefit from exploring in greater detail the reason for student nonparticipation.
This study provided evidence that furnishing medical students with opportunities to conduct teleconsultations with SPs improved medical communication skills. In particular, offering enhanced and quantified feedback information facilitates their reflection and enhances their learning of clinical communication skills. Importantly, this study demonstrated that EQClinic was a useful and practical communication skills learning tool that is well suited to medical students within university settings.
CONSORT-EHEALTH Checklist V1.6 [
nonverbal behavior feedback
Student-Patient Observed Communication Assessment
standardized patient
This project was funded by the Brain and Mind Centre at The University of Sydney Australia and the Australian Government. RAC is supported by the Australian Research Council.
None declared.