This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Mental health policy makers encourage the development of electronic decision aids to increase patient participation in medical decision making. Evidence is needed to determine whether these decision aids are helpful in clinical practice and whether they lead to increased patient involvement and better outcomes.
This study reports the outcome of a randomized controlled trial and process evaluation of a Web-based intervention to facilitate shared decision making for people with psychotic disorders.
The study was carried out in a Dutch mental health institution. Patients were recruited from 2 outpatient teams for patients with psychosis (N=250). Patients in the intervention condition (n=124) were provided an account to access a Web-based information and decision tool aimed to support patients in acquiring an overview of their needs and appropriate treatment options provided by their mental health care organization. Patients were given the opportunity to use the Web-based tool either on their own (at their home computer or at a computer of the service) or with the support of an assistant. Patients in the control group received care as usual (n=126). Half of the patients in the sample were patients experiencing a first episode of psychosis; the other half were patients with a chronic psychosis. Primary outcome was patient-perceived involvement in medical decision making, measured with the Combined Outcome Measure for Risk Communication and Treatment Decision-making Effectiveness (COMRADE). Process evaluation consisted of questionnaire-based surveys, open interviews, and researcher observation.
In all, 73 patients completed the follow-up measurement and were included in the final analysis (response rate 29.2%). More than one-third (48/124, 38.7%) of the patients who were provided access to the Web-based decision aid used it, and most used its full functionality. No differences were found between the intervention and control conditions on perceived involvement in medical decision making (COMRADE satisfaction with communication: F1,68=0.422,
The development of electronic decision aids to facilitate shared medical decision making is encouraged and many people with a psychotic disorder can work with them. This holds for both first-episode patients and long-term care patients, although the latter group might need more assistance. However, results of this paper could not support the assumption that the use of electronic decision aids increases patient involvement in medical decision making. This may be because of weak implementation of the study protocol and a low response rate.
Dutch Trial Register (NTR) trial number: 10340; http://www.trialregister.nl/trialreg/admin/rctsearch.asp?Term=10340 (Archived by WebCite at http://www.webcitation.org/6Jj5umAeS).
Shared decision making in mental health care has been dubbed an ethical imperative [
Drake and Deegan [
More evidence is needed to determine whether electronic decision aids are helpful in clinical practice and can lead to increased patient involvement and better outcomes. In addition, more information is needed about what proportion of patients are willing and able to work with Web-based decision aids and in what form (with or without assistance, using their own computer or a clinic computer). This paper reports on a randomized controlled trial and process evaluation of a Web-based intervention to facilitate shared decision making, with or without assistance, for people with psychotic disorders. Our aim was to investigate this intervention in a naturalistic setting, meaning that all eligible patients were included to be able to determine how many of them would actually use the decision aid.
Informed consent was obtained by research nurses. Patients were provided with an information brochure and they received a phone number and email address of a research assistant who they could contact for further information. The national Dutch medical ethical committee for mental health care (Medisch-ethische Toetsingscommissie instellingen Geestelijke Gezondheidszorg; METiGG) assessed the study protocol and judged that the study could be conducted without the committee’s approval. The trial was registered at the Dutch Trial register (NTR trial number: 10340).
The study was carried out in a Dutch mental health institution (Friesland Mental Health Care Service, city of Leeuwarden) with a catchment area of approximately 650,000 inhabitants. Data were collected from June 2011 to July 2012. The trial was completed when all patients provided their last measurement. Patients were recruited from 2 outpatient teams for psychosis: the early intervention for psychosis team (a multidisciplinary team for the treatment of patients with a first episode of psychosis) and a rehabilitation team (a multidisciplinary team for patients with chronic schizophrenia). We used broad inclusion criteria. Participants had to meet criteria for a nonaffective psychosis (brief psychotic disorder, schizophreniform disorder, schizoaffective disorder, schizophrenia, or psychotic disorder not otherwise specified) as defined by the
To calculate the sample size, we used the SPSS SamplePower software program (IBM Corp, Armonk, NY, USA). Given an alpha of .05, a power of .80, and an effect size of .50 (based on results of a comparable study [
We conducted an open-label, 2-group, parallel, randomized controlled trial with approximately the same number of patients in each group. Patients were allocated to either an intervention group that was offered a Web-based tool to support shared decision making or a control group that received care as usual. Randomization of patients was conducted by using the online Research Randomizer [
Patients in the control condition received care as usual, as described in the local disease management program for the treatment of people with psychosis. Treatment modules were initially chosen by a clinician in accordance with a treatment path that a patient entered based on the staging of the disorder (first episode or stabilizing/rehabilitation phase), clinician-rated scores on the Health of the Nation Outcome Scale (HoNOS), and patient-rated scores on the Camberwell Assessment of Need Short Appraisal Schedule (CANSAS-P). During a treatment plan meeting, clinicians informed patients about the indicated treatment modules and also discussed alternatives. A final decision was made in a process of shared decision making (which was not further specified in the disease management program).
Patients in the intervention condition received care as described in the local disease management program for the treatment of people with psychosis plus they were offered the opportunity to make use of the Web-based information and decision tool (see
In addition to this selection of modules, patients also had the opportunity to view all available treatment modules irrespective of the questionnaire outcomes. The information about the available modules in the catalog included an overview of its content and duration; a description of problems/symptoms the treatment module is usually indicated for; names, functions, and pictures of clinicians involved; a short story by a patient who tells his/her experience with the treatment module (see
Patients using the Web-based tool were asked to look through the treatment modules and choose the modules of their preference by ticking the appropriate checkboxes. Patients could print the checkbox form and take it with them to their treatment plan evaluation session to discuss with their clinician.
Patients were informed about the Web-based decision aid by research nurses during a biyearly appointment for Routine Outcome Monitoring (ROM), and they were offered an information brochure. Patients were given the opportunity to use the decision aid either on their own (at their home computer, or at a computer of the service) or with support of an assistant. Furthermore, an assistant was available by phone for help on 3 days each week. Patients received a log-in account by email or on paper from an assistant. No further instructions were given about the optimal timing of frequency regarding the use of the decision aid.
Screenshot of the first webpage with a questionnaire (in Dutch) about care needs.
Screenshot of the second webpage including a digital catalog with descriptions of treatment modules.
Screenshot of the second webpage with a short patient story.
After randomization, baseline measurement took place during a biyearly face-to-face ROM session for all participating patients. Participating clinicians were asked to complete an attitude questionnaire around the same time. Up to 6 weeks after the ROM session, patients in the intervention condition had the opportunity to make use of the Web-based tool. Approximately 6 weeks after ROM, a meeting was planned between the patient and a key clinician in which ROM results were evaluated and a new treatment plan was created or an existing one was adjusted. Patients were sent a final questionnaire by mail. Upon returning the questionnaire to our research center, they received a gift certificate worth €7.50. We deviated from the procedure described in the original research protocol in 1 important aspect: we conducted 1 follow-up measurement instead of 2 because a second follow-up meeting appeared to be not feasible within the time limits.
Self-reported quality of life was measured with the Manchester Short Assessment of Quality of Life (MANSA) [
Psychosocial functioning was measured with the HoNOS [
Symptom severity was measured with the Positive and Negative Syndrome Scale (PANSS) [
Patients’ preference to participate in medical decision making was measured by the decision-making preference subscale of the Autonomy Preference Index (API) [
The primary outcome measure was patient-perceived involvement in medical decisions measured with the patient-rated Combined Outcome Measure for Risk Communication and Treatment Decision-making Effectiveness (COMRADE) [
We used the patient-rated Client Satisfaction Questionnaire (CSQ) [
Descriptive statistics were used to investigate client characteristics. Baseline measures of both conditions were compared using unpaired
The intervention described previously can be considered a complex intervention because it consists of several components (use of new technology, implementation in regular care, evaluation) and is highly dependent on the context in which it is delivered. Complex interventions are interventions that contain various interacting components of which the whole is more than the sum of its parts [
The process evaluation of this study consisted of
Open interviews with a sample of 15 patients who did and did not receive the allocated intervention. An interview guide was created in accordance with the guidelines provided by Hennink et al [
Researcher observation of clinicians discussing implementation of the intervention during clinical meetings, which were recorded in a notebook by a research assistant. Themes of interest were identified by the research team and further discussed with the clinical teams when necessary.
A questionnaire-based survey among clinicians consisting of 3 parts: (1) investigating their attitude toward shared decision making and the use of a Web-based decision aid (based on Punter [
This process evaluation provided data to shed light on how well the intervention was implemented, to what extent the trial outcomes were related to the quality of the implementation and the setting in which it was implemented, and what processes might have mediated these relations.
In the process evaluation, we collected data to answer 5 questions about potential problems related to implementation and context.
The first question was: Could the outcomes be affected by a negative attitude of clinicians toward shared decision making or the Web-based decision aid? In a questionnaire-based survey, clinicians’ attitudes were investigated. On a 5-point Likert scale ranging from completely disagree to completely agree, clinicians agreed or completely agreed with 4 statements about shared decision making in general, and 9 statements about the use of a decision aid in decision-making processes. The mean total score on this scale was 3.52 (SD 0.49), meaning that most clinicians showed a positive attitude toward shared decision making and the use of decision aids.
The second question was: Do clinicians think there are too many hampering factors to realize a process of shared decision making? In addition, 18 clinicians reported that in processes of shared decision making, the following factors were often or almost always experienced as hampering decision making: patients receive contradictory advice from multiple clinicians (12/18, 67%), patients have difficulty accepting their diagnosis (12/18, 67%), and patients are indecisive (10/17, 59%). The following factors were reported as never or sometimes hampering: patients want to participate to a greater degree than the clinician prefers (15/18, 83%), patients have other interfering health problems (15/18, 83%), lack of time (14/18, 78%), cultural differences (14/18, 78%), patients bring in too much information to discuss (13/18, 72%), patients ask for a treatment that is not evidence-based (12/17, 71%), clinician has too little information to make a decision (12/17, 71%), patients do not understand the information (12/18, 67%), patients are too anxious or worried to listen to what the clinician has to say to them (11/18, 61%), and patients refuse treatment that could benefit them (10/18, 56%).
The third question was: Could the outcomes be affected by the clinicians’ judgment about patients’ capabilities and interests? Clinicians were asked to what extent they considered patients to be capable and interested in shared decision making. Of the 128 patient observations, clinicians rated most patients as being able to understand the arguments presented, being capable of making reasonable decisions, and being interested in the topics discussed as well as in participating in medical decision making. Patients who were rated by their clinicians as not capable of making decisions (score 1-3) had a significantly lower score than patients rated as capable of making decisions on both subscales of the COMRADE (COMRADE satisfaction with communication:
The fourth question was: Could any problems be observed with fulfillment of the study protocol? Through researcher observation, several recurring themes were identified during clinical meetings in which the trial was discussed. Case managers sometimes were hesitant and felt troubled to invite intervention patients to make use of the decision tool. First, they were doubtful whether patients were able to handle either the computer program or participation in a research trial. Second, they were not sure that patients would benefit from the decision aid because not all treatment options included in the decision aid were actually offered by their organization (eg, music therapy was listed among the treatment options, but no music therapy was currently offered because of absence of a music therapist). In addition, various clinicians reported that they were unsure when to discuss outcomes of the decision aid with their patients because not all conducted a formal treatment evaluation session with their patients following their ROM assessment. Some only discussed ROM results within the clinical team and not directly with patients.
The fifth question was: Did patients experience any problems with the intervention that was not covered in the satisfaction questionnaire? Open interviews among patients who chose to use or not use the website provided some additional details on the process. First, all patients were initially informed about the decision aid by an information booklet and in a meeting with a research nurse, but most of them received additional explanation from their case manager. Some framed the decision aid predominantly within a research context (“by using the decision aid, you contribute to research”), whereas others described it as an attempt to improve services (“using the decision aid might help you reflect on the treatment you want”). This might have affected patients’ expectations of the intervention. Moreover, interviews revealed discrepancies between the policy of the local disease management program and patients’ experiences in clinical practice. Most of the interviewed patients could not remember their ROM results being discussed with them and some could not remember whether a treatment plan was created.
A total of 250 patients (n=124 intervention vs n=126 control) were included in the trial of whom 73 completed the follow-up measurement and were included in the final analysis (response rate 29.2%). Of these 73 patients, 40 were in the intervention and 33 in the control condition. Of the 40 patients in the intervention condition who completed the follow-up measurement, 30 used the decision aid. A detailed overview of the flow of participants is presented in
Percentage of clinicians (completely) agreeing with statements about shared decision making and decision aids (n=19).
Item | Agree or completely agree, n (%) |
A decision aid will cause patients to ask more questions than they would otherwise have asked | 16 (84) |
A decision aid will cause patients to be more involved in decision making about treatmenta | 15 (83) |
All eligible patients should be invited to use the decision aid | 15 (79) |
Knowing risks and benefits, most patients want to decide how acceptable treatment is to them | 13 (68) |
Patients using a decision aid will be much better informed | 13 (68) |
Patients should see a decision aid before a treatment decision is made | 12 (63) |
Patients usually want to be an equal partner with physicians in making important treatment decisions | 10 (53) |
With a decision aid, I will be able to reduce time spent educating patients about treatmenta | 7 (39) |
Most patients prefer the clinicians to take responsibility for their medical problems | 4 (21) |
Using a decision aid will reduce the risk of malpractice | 4 (21) |
A decision aid will eliminate the need for third-party utilization as second opinion | 3 (16) |
A decision aid may cause some patients to make the wrong choice | 3 (16) |
The majority of patients do not wish to be involved in decision making about their treatment | 1 (5) |
an=18.
Participant flow diagram.
Demographic variables and baseline data of patients included in the analysis are presented in
The patients who dropped out of the study and did not complete the follow-up measurement were slightly younger (
Demographic variables and baseline data of study participants.
Variable | Intervention (n=40) | Control (n=33) |
|
|
Age (years), mean (SD) | 37 (12.35) | 40 (13.47) | .35 | |
Sex (female), n (%) | 13 (33) | 21 (64) | .01 | |
Education (≥ 12 years), n | 10 (n=12) | 10 (n=12) | .99 | |
Job or study, n (%) | 13 (33; n=39) | 16 (48) | .23 | |
Partner, n (%) | 9 (23; n=39) | 18 (55) | .01 | |
Use of antipsychotics, n (%) | 29 (73) | 22 (67) | .60 | |
|
|
|
|
|
|
GAF | 61.8 (9.08) | 57.4 (10.91) | .06 |
|
MANSA | 60.7 (9.50) | 62.3 (13.26) | .58 |
|
HoNOS | 7.7 (4.75) | 8.4 (4.32) | .53 |
|
PANSS total score | 13.3 (5.24) | 15.4 (5.51) | .13 |
|
API | 55.7 (12.72) | 52.7 (12.96) | .38 |
Number of patients from the first episode of psychosis team within condition, n (%) | 16 (40) | 13 (39) | .99 |
aUsing Fisher exact test or
bGAF: Global Assessment of Functioning; MANSA: Manchester Short Assessment of Quality of Life; HoNOS: Health of the Nation Outcome Scales; PANSS: Positive and Negative Syndrome Scale; API: Autonomy Preference Index.
Intention-to-treat analyses showed that patients in the intervention condition did not differ from patients in the control condition in their perceived involvement in medical decision making (COMRADE) after they had used the Web-based decision aid (COMRADE satisfaction with communication:
Per protocol analyses also showed that patients in the intervention condition who received the allocated intervention and completed the follow-up measure (n=30) did not differ regarding their perceived involvement in medical decision making and in satisfaction with care from patients in the control condition (n=33) (COMRADE satisfaction with communication:
In an additional analysis, patients in the intervention condition who received the allocated intervention (n=30) were compared to patients in the intervention condition who did not receive the allocated intervention (n=10). No differences were found for patients’ perceived involvement in medical decision making (COMRADE satisfaction with communication:
Primary outcome data of patients’ perceived involvement in medical decision making at the end of the study using the Combined Outcome Measure for Risk Communication and Treatment Decision-making Effectiveness (COMRADE) test.
COMRADE subscalea | Intervention, mean (SD) | Control, mean (SD) |
|
|
Satisfaction with communication (n=73) | 38.25 (1.06 | 37.19 (1.165) | 0.422 (1,68) | .52 |
Confidence in decision (n=70) | 38.78 (1.17) | 38.72 (1.307) | 0.086 (1,67) | .77 |
aGroup differences were analyzed using a general linear model with age and partner status as covariates.
Of the 48 patients who used the Web-based decision aid, 12 used their own computer, 12 used the computer at the clinic, and 6 used a computer elsewhere. Furthermore, 13 used the decision aid independently, 16 received assistance from a professional (often their case manager), and 1 received assistance from someone else. First-episode patients used their own computer and used the decision aid without assistance more often than chronic patients did. Of the 48 patients who used the website, 34 (71%) used full functionality of the Web-based decision aid, meaning that patients completed the care needs assessment (first webpage of the website) and looked through the digital catalog with descriptions of treatment modules (second webpage of the website). More than half of them were long-term care patients (27/48, 56%).
In the intervention condition, 29 of 48 patients who used the decision aid (60%) completed questions about their satisfaction with the decision aid. They agreed or completely agreed with the following statements: “I have been well informed about the treatment options offered by Friesland Mental Health Care Service by the decision aid” (22/29, 76%), “The advice presented by the decision aid has helped me to reflect on what I want” (22/29, 76%), “The decision aid was easy to use” (20/28, 71%), “I would recommend the decision aid to others” (20/27, 74%) and “The decision aid helped me to get a clearer view on what my problem areas or points of interest are” (17/28, 61%). Patients were divided on whether the decision aid helped them to better prepare the evaluation meeting with their clinicians, 44% (12/27) said it did help; 56% (15/27) were neutral or said it did not help. Means and standard deviations can be found in
Secondary outcome data of patients’ satisfaction with the Web-based decision aid.
Question | Mean (SD)a |
I have been well informed about the treatment options offered by the GGZ Friesland by the decision aid (n=29) | 3.93 (0.84) |
The advice presented by the decision aid has helped me to reflect on what I want (n=29) | 3.86 (0.79) |
As a consequence of using the decision aid, I was better prepared for the evaluation meeting with my clinician (n=27) | 3.33 (0.78) |
The decision aid helped me to get a clearer view on what my problem areas or points of interest are (n=28) | 3.61 (0.92) |
The decision aid was easy to use (n=28) | 3.79 (1.07) |
I would recommend the decision aid to others (n=27) | 3.89 (0.75) |
aScores ranged from 0 (completely disagree) to 5 (completely agree).
In this study, we report on a clinical trial and process evaluation of a Web-based intervention to facilitate shared decision making for people with psychotic disorders.
To be able to explore potential implementation issues and contextual problems within the trial, we conducted a process evaluation. This evaluation showed that no significant problems could be observed in the attitude and beliefs of clinicians. Participating clinicians had an overall positive attitude toward shared decision making. They reported that their patients were generally interested in and capable of participating in medical decision making, they considered patient decision aids be to potentially helpful, and they judged relatively few factors to be hampering in a shared decision-making process. However, problems were observed in the implementation of the intervention. Not all patients in the intervention group were actually offered the possibility to use the decision aid and, more importantly, ROM and treatment evaluation meetings in which the treatment plan was to be discussed in a process of shared decision making did not always take place. Moreover, interviews indicate that the Web-based intervention might have been framed differently to different patients, which may have shaped their expectations and affected their evaluation. An interesting finding in the process evaluation was that patients who perceived their involvement in medical decision making as low were judged by clinicians to be less capable of participating in decision making. This could imply that patients participate less because they are less capable. Nevertheless, we cannot rule out that patients participate less because clinicians consider them less capable and, therefore, provide less opportunities for patients to participate in decision making.
The findings of our trial show that more than one-third of the patients who were provided access to the Web-based decision aid chose to use it and most used full functionality of the decision aid whether they were first-episode patients or long-term patients. Users and nonusers did not differ in demographic variables. At least one-quarter of the patients used their own computer and a similar proportion used the decision aid without assistance. Most of these were first-episode patients. On average, users of the decision aid reported to be rather satisfied with the system. Nevertheless, primary outcome results could not support the assumption that the use of electronic decision aids increases patient involvement in medical decision making, neither in intention-to-treat analyses nor in per protocol analyses. In addition, we did not find a difference in self-reported satisfaction with care between patients who had the opportunity to use the decision aid versus those who did not.
Our outcomes are in-line with the study by Woltmann et al [
Given the problems observed in the process evaluation, the intervention designed for our study appeared not to fit in optimally with the routine practice of the participating clinical care teams. Therefore, the lack of significant effects on our outcome measures cannot be solely attributed to failures intrinsic to the intervention. Future studies might benefit from a stronger integration of shared decision-making interventions in clinical practice by training clinical teams in using (output) from decision aids. A comprehensive overview of the working flow of patients and clinicians is crucial to realize this integration. Given the low response rate and moderate participation rate in this study, it may also be desirable to investigate efficacy of decision aids in a less naturalistic setting in which participating patients are selected more strictly and required to use the decision aid before performing a naturalistic study. In addition, special attention should be paid to the selection of outcome measures used to assess the shared decision-making process. Instruments focusing on satisfaction might suffer from ceiling effects, and instruments such as the COMRADE may be too broad and indirect to detect changes in the decision-making process. A better alternative is to record conversations between clinicians and patients and observe what is actually happening within that conversation. A promising instrument for this may be the recently developed Mappin’SDM [
The main limitation of this study is the weak implementation of the study protocol; as a result, it is difficult to draw firm conclusions about the study’s outcomes. We tried to prevent this by preparing the participating teams before the start of the trial and keeping closely in touch during the trial (eg, being present at clinical meetings, functioning as helpdesk, sending individual emails to participating clinicians as reminders of specific actions). Another important limitation is the large numbers of dropouts before the follow-up measurement, even though patients were offered a small gift for returning their completed questionnaire.
Our study also has strengths. Most importantly, it affirms previous findings that many people with a severe mental illness can work with electronic decision aids, either with or without assistance, at the clinic or at home. Furthermore, our study provides insight in variation among the population concerning interest in and use of electronic decision aids. Our results suggest that part of the population is not able or does not feel the need to work with these decision aids. Based on our results, the ratio of users versus nonusers could be 50-50. Another strength is that we collected detailed information about allocation and reception of the intervention with varying illness durations, and we included a process evaluation that allowed us to perform a critical analysis on the trial results.
The development of electronic decision aids to facilitate shared medical decision making is encouraged and many people with a psychotic disorder can work with them. This holds for both first-episode patients and long-term care patients, although the latter group might need more assistance. However, effects of decision aids on patient participation in medical decision making have not been consistently demonstrated.
Video of the Web-based decision aid.
CONSORT-EHEALTH Checklist V1.6.2 [
Autonomy Preference Index
Camberwell Assessment of Need Short Appraisal Schedule–Self-Report Version
Combined Outcome Measure for Risk Communication and Treatment Decision-making Effectiveness
Client Satisfaction Questionnaire
Diagnostic and Statistical Manual of Mental Disorders (Fourth Edition, Text Revision)
Global Assessment of Functioning
Health of the Nation Outcome Scales
Manchester Short Assessment of Quality of Life
Medisch-ethische Toetsingscommissie instellingen Geestelijke Gezondheidszorg (Dutch medical ethical committee for mental health care)
NederlandsTrial Register (Dutch trial register)
Positive and Negative Syndrome Scale
Routine Outcome Monitoring
This study was supported by the Netherlands Organisation for Health Research and Development (ZonMw); Fonds Psychische Gezondheid; ICT regie; and the Dutch Ministry of Health, Welfare and Sport under the name WEGWEIS (grant number 300020011).
None declared.