This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Patient information and education, such as decision aids, are gradually moving toward online, computer-based environments. Considerable research has been conducted to guide content and presentation of decision aids. However, given the relatively new shift to computer-based support, little attention has been given to how multimedia and interactivity can improve upon paper-based decision aids.
The first objective of this review was to summarize published literature into a proposed classification of features that have been integrated into computer-based decision aids. Building on this classification, the second objective was to assess whether integration of specific features was associated with higher-quality decision making.
Relevant studies were located by searching MEDLINE, Embase, CINAHL, and CENTRAL databases. The review identified studies that evaluated computer-based decision aids for adults faced with preference-sensitive medical decisions and reported quality of decision-making outcomes. A thematic synthesis was conducted to develop the classification of features. Subsequently, meta-analyses were conducted based on standardized mean differences (SMD) from randomized controlled trials (RCTs) that reported knowledge or decisional conflict. Further subgroup analyses compared pooled SMDs for decision aids that incorporated a specific feature to other computer-based decision aids that did not incorporate the feature, to assess whether specific features improved quality of decision making.
Of 3541 unique publications, 58 studies met the target criteria and were included in the thematic synthesis. The synthesis identified six features: content control, tailoring, patient narratives, explicit values clarification, feedback, and social support. A subset of 26 RCTs from the thematic synthesis was used to conduct the meta-analyses. As expected, computer-based decision aids performed better than usual care or alternative aids; however, some features performed better than others. Integration of content control improved quality of decision making (SMD 0.59 vs 0.23 for knowledge; SMD 0.39 vs 0.29 for decisional conflict). In contrast, tailoring reduced quality of decision making (SMD 0.40 vs 0.71 for knowledge; SMD 0.25 vs 0.52 for decisional conflict). Similarly, patient narratives also reduced quality of decision making (SMD 0.43 vs 0.65 for knowledge; SMD 0.17 vs 0.46 for decisional conflict). Results were varied for different types of explicit values clarification, feedback, and social support.
Integration of media rich or interactive features into computer-based decision aids can improve quality of preference-sensitive decision making. However, this is an emerging field with limited evidence to guide use. The systematic review and thematic synthesis identified features that have been integrated into available computer-based decision aids, in an effort to facilitate reporting of these features and to promote integration of such features into decision aids. The meta-analyses and associated subgroup analyses provide preliminary evidence to support integration of specific features into future decision aids. Further research can focus on clarifying independent contributions of specific features through experimental designs and refining the designs of features to improve effectiveness.
Over the past decade, health care has shifted from paper-based practice to electronic health records [
Considerable research has been conducted to guide content and presentation of decision aids [
Theory suggests that integration of media rich or interactive features into computer-based decision aids can have a positive impact on quality of decision making by engaging patients in decision making beyond traditional static approaches [
Four databases (MEDLINE, Embase, CINAHL, and CENTRAL) were searched for all relevant studies published from 1946-2013. Three main concepts of decision support, the patient, and computer were mapped to the most relevant controlled vocabulary using Medical Subject Headings (MeSH), and free-text terms were added where necessary. Full search strategies are outlined in
The review identified studies that evaluated computer-based decision aids for adults faced with a preference-sensitive medical decision (ie, treatment, risk management, screening, or prevention) and that reported at least one quality of decision-making outcome (ie, knowledge, decisional conflict [
Screening of articles was completed in two stages. Articles were first screened for relevance based on the information provided in the title and abstract and were then evaluated for inclusion based on the full text. Two reviewers independently screened articles at each stage (AS and DK). Disagreements were resolved by discussion and consensus between the 2 reviewers. Overall kappa score was calculated to assess interrater reliability [
One reviewer completed data abstraction (AS), which focused on citation information, study design, decision context, interventions, controls (eg, usual care or alternative aids), components being tested, and quality of decision-making outcomes. If an article included in the review cited a development paper or webpage, then information from these sources was used to supplement data abstracted from the article.
All studies identified for the systematic review were included in the thematic analysis. Data abstracted from the articles was used to create a proposed classification of features that have been integrated into computer-based decision aids to date. The classification was developed based on logical groupings and informed by themes from decision aid literature [
Selection of groupings was completed by 1 reviewer (AS), guided by steps outlined by Thomas & Harden for conducting thematic analysis: (1) line-by-line coding of articles to record components, (2) development of descriptive themes, and (3) creation of analytical themes [
Only randomized controlled trials (RCTs) that reported knowledge or decisional conflict were included in the quantitative synthesis. Decisional certainty, satisfaction with decision making, and decisional self-efficacy were not included due to the lower number of studies that reported these outcomes. The standardized mean difference (SMD; ie, Cohen’s
The overall effect of computer-based decision aids was estimated by pooling the SMD of each study using Review Manager (version 5.3). Studies were pooled using inverse variance weighting and random effects models with 95% confidence intervals. Heterogeneity of pooled SMDs was assessed based on I2 statistics [
Subgroup analyses were conducted to test whether specific features (or types of components) could explain some of the heterogeneity in the overall effect. Subgroup analyses compared pooled SMDs for decision aids that incorporated a specific feature to other computer-based decision aids that did not incorporate the feature to assess whether specific features were associated with improvements in quality of decision making. The Review Manager test for subgroup differences was used to assess statistical significance.
The search identified 3541 eligible articles. The title and abstract screen retained 135 articles. Full text screening identified 58 studies that met the target criteria and were included in the thematic synthesis. The overall kappa score for screening was 0.60, reflecting moderate interrater agreement [
Data abstracted from articles are presented in
Studies assessed quality of decision making by measuring knowledge (36/58, 62%), decisional conflict (30/58, 52%), decisional certainty (21/58, 36%), satisfaction with decision making (16/58, 28%), and decisional self-efficacy (7/58, 12%). Studies compared computer-based decision aid performance to usual care (18/58, 31%), alternative aids (29/58, 50%), or based on pre-assessments (14/58, 24%).
Modified PRISMA study selection flowchart.
The thematic analysis identified six main features that have been integrated into computer-based decision aids: content control, tailoring, patient narratives, explicit values clarification, feedback, and social support. A proposed classification for these features and types of components is presented in
Proposed classification of features that have been integrated into computer-based decision aids (58 studies).
Features | Types of components | Examples of components |
Content control: Patient has control over access to information | Navigation | Menu bar, search function, television-like interface, touchscreen, help menu |
Clarity of information | Glossary, information summaries, supplementary risk diagrams, metaphors, narration | |
Optional information | “Learn more” sections for detailed information about topics of interest | |
Access to external resources | Reference lists, links to summaries of recent studies or clinical practice guidelines, developer contact information | |
Tailoring: Patient receives personalized information | Demographics | Patient age, gender, race/ethnicity, family history, health literacy |
Clinical condition | Specific diagnosis, stage of disease, comorbidities, current symptoms, current medications, past treatment experience, eligibility for interventions | |
Values, preferences, and beliefs | Language, preferred role in decision making, stage of decision making, preference for colloquial vs technical terms, beliefs around efficacy of screening or treatment | |
Knowledge deficits | Focus on information that is unclear or incorrect based on knowledge pre-tests | |
Patient narratives: Patient reflects on experiences of others | Patient stories (focus on personal experiences) | Video of patient sharing personal experience |
Behavior modeling (focus on process of deliberation) | Video of patient weighing options, video vignettes of common concerns around decision making | |
Explicit values clarification: Patient examines personal values and preferences | Decision points | Strategically placed questions to determine whether patient is prepared to move forward to next section of decision aid |
Notebook | Memory aid used to store issues of concern, “bookmarks” for important sections | |
Weighting exercises | Simple yes/no questions, feeling thermometer, balance scale, selecting initial treatment decision | |
Trade-off exercises | Simple rank order exercises, adaptive conjoint analysis-based tools | |
Social matching | “Soap opera” episodes with questions to determine which character embodies patient’s values and preferences | |
Personal reflection | Patient considers perspectives of others affected by the decision (eg, partners, family members, or others) | |
Feedback: Patient receives important information around decision making based on interactions with aid | Decision aid progress | Program tracks information that has been covered, and suggests important information that has not been accessed |
Knowledge | Self-evaluations provide feedback on comprehension of evidence presented | |
Summary of preferences | Bar graphs depicting relative importance of personal values and preferences | |
Optimal choice | Patient values and preferences are incorporated into an algorithm to determine the most suitable option | |
Decisional consistency | Alerts patient if initial treatment decision is not consistent with optimal choice | |
Summary of decision aid activity (usually printed) | Plan of action based on initial treatment decision, personal risk summaries | |
Social support: Patient encouraged to involve others in decision-making | Community support | Celebrity endorsement, video of patient celebration after completing treatment, links to support groups |
Integration of family | Modules specific to others affected by the decision, information on how to communicate with partner | |
Facilitation of shared decision making | Video of physician describing options and outcomes, video of physician encouraging patient to adhere to chosen option, video coaching to overcome physician communication barriers, recommended questions for physician consultations, copy of decision aid summary placed in patient chart, physician-specific modules |
The majority of studies included in the thematic analysis provided content control (42/58, 72%). Two-thirds tailored information to the patient (38/58, 66%), and almost half incorporated patient narratives (28/58, 48%). Over half of the studies provided explicit values clarification (31/58, 53%), feedback (36/58, 62%), or social support (32/58, 55%). One third of the studies incorporated five (13/58, 22%) or all six (10/58, 17%) of these features.
Eighteen studies were included in the meta-analysis to assess whether or not use of computer-based decision aids improved knowledge. The studies included were published between 2001 and 2013. Most computer-based decision aids performed significantly better than usual care or alternative aid controls (14/18, 78%); the performance of the remaining decision aids was not significantly different from controls. Overall, computer-based decision aids were associated with significant improvements in knowledge with a pooled SMD of 0.54 (95% CI 0.36-0.71;
We included 21 studies in the meta-analysis to assess whether or not use of computer-based decision aids improved decisional conflict. The studies included were published between 2002 and 2013. Most computer-based decision aids performed significantly better than usual care or alternative aid controls (13/21, 62%); the performance of the remaining decision aids was not significantly different from controls. Overall, computer-based decision aids were associated with significant improvements in decisional conflict with a pooled SMD of 0.35 (95% CI 0.23-0.48;
Although computer-based decision aids performed significantly better than usual care or alternative aids, there was a high level of heterogeneity in study-level SMDs. The I2 statistics were 84% and 75% for knowledge and decisional conflict, respectively.
Forest plot of SMDs for improvements in knowledge (18 studies).
Forest plot of SMDs for improvements in decisional conflict (21 studies).
The six features and associated types identified through the thematic analysis were used to inform subgroup analyses. The results are presented in
Overall, integration of content control was positively associated with quality of decision making, although the association was only significant for knowledge (
Conversely, tailoring was negatively associated with knowledge and decisional conflict (
Similarly, patient narratives reduced quality of decision making; however, the association was significant only for decisional conflict (
Explicit values clarification reduced knowledge (
Overall, providing feedback was negatively associated with knowledge and decisional conflict (
Number of studies and pooled SMDs for improvements in knowledge comparing decision aids including each feature to decision aids not including the feature (18 studies).
Feature and types of components | Studies, n | Feature included, |
Studies, n | Reference (no feature), |
|
|
Overall: Any feature | 18 | 0.54 (0.36-0.71) | 0 | — | — | |
|
15 | 0.59 (0.39-0.79) | 3 | 0.23 (0.05-0.41) | .008 | |
|
Navigation | 7 | 0.47 (0.19-0.76) | 11 | 0.59 (0.34-0.83) | .56 |
|
Clarity of information | 13 | 0.65 (0.44-0.87) | 5 | 0.24 (-0.05-0.54) | .03 |
|
Optional information | 7 | 0.76 (0.42-1.09) | 11 | 0.38 (0.21-0.54) | .05 |
|
Access to external resources | 6 | 0.63 (0.15-1.10) | 12 | 0.51 (0.32-0.70) | .65 |
|
10 | 0.40 (0.18-0.62) | 8 | 0.71 (0.44-0.99) | .08 | |
|
Demographics | 9 | 0.38 (0.15-0.62) | 9 | 0.71 (0.45-0.96) | .07 |
|
Clinical condition | 8 | 0.36 (0.11-0.61) | 10 | 0.69 (0.46-0.93) | .06 |
|
Values, preferences, and beliefs | 3 | 0.31 (0.00-0.62) | 15 | 0.59 (0.38-0.79) | .14 |
|
Knowledge deficits | 0 | — | 18 | 0.54 (0.36-0.71) | — |
|
8 | 0.43 (0.19-0.68) | 10 | 0.65 (0.37-0.93) | .26 | |
|
Patient stories | 7 | 0.47 (0.20-0.75) | 11 | 0.59 (0.34-0.83) | .54 |
|
Behavior modeling | 3 | 0.39 (0.11-0.67) | 15 | 0.57 (0.36-0.78) | .32 |
|
11 | 0.48 (0.30-0.65) | 7 | 0.67 (0.23-1.12) | .42 | |
|
Decision points | 0 | — | 18 | 0.54 (0.36-0.71) | — |
|
Notebook | 3 | 0.59 (0.35-0.84) | 15 | 0.53 (0.33-0.73) | .68 |
|
Weighting exercises | 8 | 0.41 (0.24-0.58) | 10 | 0.65 (0.34-0.95) | .18 |
|
Trade-off exercises | 3 | 0.58 (0.12-1.04) | 15 | 0.53 (0.33-0.72) | .84 |
|
Social matching | 1 | 0.43 (0.18-0.68) | 17 | 0.55 (0.36-0.73) | .47 |
|
Personal reflection | 1 | 0.43 (0.18-0.68) | 17 | 0.55 (0.36-0.73) | .47 |
|
8 | 0.46 (0.27-0.64) | 10 | 0.60 (0.31-0.89) | .40 | |
|
Decision aid progress | 0 | — | 18 | 0.54 (0.36-0.71) | — |
|
Knowledge | 2 | 0.60 (0.12-1.08) | 16 | 0.53 (0.35-0.72) | .80 |
|
Summary of preferences | 0 | — | 18 | 0.54 (0.36-0.71) | — |
|
Optimal choice | 3 | 0.42 (0.11-0.73) | 15 | 0.57 (0.36-0.78) | .44 |
|
Decisional consistency | 2 | 0.17 (0.03-0.31) | 16 | 0.60 (0.40-0.79) | <.001 |
|
Summary of decision aid activity | 6 | 0.44 (0.23-0.65) | 12 | 0.60 (0.34-0.86) | .35 |
|
10 | 0.58 (0.32-0.84) | 8 | 0.50 (0.23-0.76) | .67 | |
|
Community support | 4 | 0.91 (0.34-1.48) | 14 | 0.45 (0.27-0.63) | .14 |
|
Integration of family | 3 | 0.50 (0.29-0.72) | 15 | 0.54 (0.34-0.74) | .82 |
|
Facilitation of shared decision making | 6 | 0.44 (0.13-0.75) | 12 | 0.59 (0.36-0.82) | .45 |
aReview Manager test for subgroup differences.
Number of studies and pooled SMDs for improvements in decisional conflict comparing decision aids including each feature to decision aids not including the feature (21 studies).
Feature and types of components | Studies, n | Feature included, pooled SMD (95% CI) | Studies, n | Reference (no feature), |
|
|
Overall: Any feature | 21 | 0.35 (0.23-0.48) | 0 | — | — | |
|
14 | 0.39 (0.23-0.56) | 7 | 0.29 (0.08-0.49) | .42 | |
|
Navigation | 8 | 0.22 (0.10-0.34) | 13 | 0.42 (0.23-0.60) | .08 |
|
Clarity of information | 12 | 0.46 (0.28-0.65) | 9 | 0.23 (0.07-0.40) | .07 |
|
Optional information | 6 | 0.44 (0.20-0.68) | 15 | 0.32 (0.17-0.47) | .42 |
|
Access to external resources | 5 | 0.72 (0.12-1.33) | 16 | 0.28 (0.18-0.37) | .15 |
|
12 | 0.25 (0.13-0.37) | 9 | 0.52 (0.26-0.79) | .07 | |
|
Demographics | 10 | 0.29 (0.16-0.42) | 11 | 0.43 (0.20-0.65) | .31 |
|
Clinical condition | 10 | 0.26 (0.12-0.40) | 11 | 0.46 (0.23-0.68) | .14 |
|
Values, preferences, and beliefs | 7 | 0.18 (0.07-0.30) | 14 | 0.44 (0.27-0.61) | .02 |
|
Knowledge deficits | 0 | — | 21 | 0.35 (0.23-0.48) | — |
|
8 | 0.17 (0.08-0.26) | 13 | 0.46 (0.28-0.65) | .005 | |
|
Patient stories | 5 | 0.20 (0.03-0.38) | 16 | 0.39 (0.24-0.54) | .11 |
|
Behavior modeling | 4 | 0.16 (0.05-0.27) | 17 | 0.41 (0.25-0.56) | .01 |
|
13 | 0.36 (0.20-0.51) | 8 | 0.36 (0.14-0.58) | .97 | |
|
Decision points | 0 | — | 21 | 0.35 (0.23-0.48) | — |
|
Notebook | 4 | 0.48 (-0.02 to 0.98) | 17 | 0.32 (0.20-0.44) | .56 |
|
Weighting exercises | 9 | 0.35 (0.16-0.53) | 12 | 0.36 (0.19-0.54) | .89 |
|
Trade-off exercises | 3 | 0.48 (-0.08 to 1.04) | 18 | 0.33 (0.20-0.45) | .60 |
|
Social matching | 1 | 0.33 (-0.02 to 0.68) | 20 | 0.36 (0.23-0.49) | .89 |
|
Personal reflection | 0 | — | 21 | 0.35 (0.23-0.48) | — |
|
11 | 0.32 (0.16-0.49) | 10 | 0.39 (0.19-0.58) | .63 | |
|
Decision aid progress | 1 | 0.62 (0.09-1.15) | 20 | 0.35 (0.22-0.47) | .32 |
|
Knowledge | 1 | 1.23 (0.27-2.19) | 20 | 0.34 (0.22-0.46) | .07 |
|
Summary of preferences | 1 | 0.37 (0.04-0.70) | 20 | 0.35 (0.23-0.48) | .93 |
|
Optimal choice | 4 | 0.45 (0.09-0.81) | 17 | 0.33 (0.19-0.46) | .54 |
|
Decisional consistency | 2 | 0.24 (0.02-0.45) | 19 | 0.37 (0.23-0.51) | .31 |
|
Summary of decision aid activity | 9 | 0.32 (0.15-0.50) | 12 | 0.39 (0.20-0.57) | .62 |
|
11 | 0.38 (0.19-0.57) | 10 | 0.34 (0.17-0.51) | .75 | |
|
Community support | 4 | 0.50 (-0.08 to 1.07) | 17 | 0.33 (0.21-0.45) | .58 |
|
Integration of family | 2 | 0.64 (-0.30 to 1.58) | 19 | 0.35 (0.22-0.47) | .54 |
|
Facilitation of shared decision making | 8 | 0.29 (0.13-0.45) | 13 | 0.38 (0.21-0.56) | .46 |
aReview Manager test for subgroup differences.
Social support improved knowledge (
This review summarizes published literature into a proposed classification of features that have been integrated into computer-based decision aids. The thematic synthesis identified six main features of content control, tailoring, patient narratives, explicit values clarification, feedback, and social support. Building on this classification, meta-analyses with tests for subgroup differences were conducted to evaluate whether specific features improved quality of decision making. Overall, decision aids that integrated these features performed significantly better than usual care or alternative aids. The exploratory subgroup analyses rank-ordered the features. Overall, content control performed better than other features. Conversely, tailoring and patient narratives performed worse compared to other features. Results were varied for different types of explicit values clarification, feedback, and social support.
The proposed features classification is the first of its kind for decision aids. It serves two purposes: to provide the first step towards improving reporting of features that are integrated into computer-based decision aids and to promote use of such features in future decision aids. Currently, reporting standards for interventions are specific about the overarching goal of replicability; however, they offer little guidance around how to reach this goal. For example, the Consolidated Standards of Reporting Trials (CONSORT) statement simply states that authors should report sufficient information to ensure replicability, including detail around how and when interventions were administered [
As expected, computer-based decision aids were associated with significant improvements in knowledge and decisional conflict compared to usual care or alternative aids [
Overall, content control improved quality of decision making. All types of content control performed better than other features, with the exception of navigation. Content control is intended to provide patients with control over order, detail, and type of evidence presented [
Interestingly, navigation reduced quality of decision making compared to other features. Given that navigation is a foundational piece of computer-based interventions, this may represent a reporting bias. As a result of journal space limitations, navigation may have been underreported in exchange for reporting novel or impressive decision aid components. This relates back to the need for a classification to ensure that all features are reported; otherwise, important features may be overlooked as a result of biased evaluations.
Tailoring reduced quality of decision making, with all subgroups performing worse than other features. In general, tailoring is intended to translate evidence into patient-specific information to improve engagement. The effects of tailoring can be split into two categories: (1) effects on calculation of risk estimates, and (2) effects on presentation of information.
Tailoring can be used to frame evidence in terms of patient demographics or clinical condition to present only viable treatment options with more accurate estimates of associated risks and benefits. Ideally, this should provide a better understanding of personal situations and lead to high-quality decision making. However, evidence around the benefits of tailoring risk estimates is varied [
Tailoring can also be used to present evidence in terms of patient preferences or to address knowledge deficits, in an effort to facilitate understanding or to correct misinformation. However, this form of tailoring may limit the amount or type of evidence that is presented. For example, decision aids can be tailored to information-seeking style (ie, high or low levels of detail) [
Similarly, patient narratives reduced quality of decision making. Patient narratives are intended to provide insight into patient experiences and bring attention to important evidence to consider throughout the decision-making process. In addition, information presented through patient narratives is processed differently than written information and can improve understanding and retention of evidence [
In this study, both patient stories and behavior modeling scenarios reduced quality of decision making. Shaffer and Zikmund-Fisher have developed a taxonomy for patient narratives outlining dimensions that are expected to impact decision making: (1) purpose of the narrative, (2) content of the narrative, and (3) evaluative valence (ie, tone of the narrative) [
The negative effects of tailoring and patient narratives on quality of decision making in decision aids were unexpected, considering the positive impact of tailoring and patient narratives when employed in behavior change interventions [
Specific types of explicit values clarification had a positive effect on quality of decision making. Explicit values clarification methods are intended to guide patients through specific tasks to identify personal values and preferences [
Specific types of feedback were also associated with improvements in quality of decision making. Feedback is intended to provide the patient with important information around decision making based on interactions with the decision aid. Progress through the decision aid and knowledge feedback both improved quality of decision making. Both are intended to ensure that the patient is well informed by confirming that all necessary information is reviewed by the patient and to correct misinformation, respectively. Summary of preferences, optimal choice, and decisional consistency are types of feedback that are specific to explicit values clarification methods. Summary of preferences provides feedback around how patients personally value risks and benefits integral to decision making. Optimal choice builds on summary of preferences, by suggesting which option is best based on patients’ values and preferences, which had a positive effect on decisional conflict. Similarly, research has shown that providing implications of stated values (ie, optimal choice) may have a positive effect on decision making [
Specific types of social support improved quality of decision making. This feature is intended to reinforce that the patient is not alone in their experiences or decision making. Social support is a recurring theme throughout patient needs assessments for medical care [
Based on the study findings, content control should be integrated into decision aids to allow patients to select the order, level of detail, and type of information presented. This approach allows the patient to directly access topics of interest, view alternative presentations of information for clarity, and access optional information or external resources. However, to ensure balanced representation of all options, it is important to integrate safeguards to ensure that the patient reviews all necessary evidence (ie, not “optional” information) prior to making a final decision.
Tailoring, as currently developed and presented, should be used with caution, as it may reduce quality of decision making. Ineffective tailoring may have resulted from superficial or non-transparent tailoring, which patients did not believe reflected their true risk. Allowing patients to “self-tailor” through content control may be a viable option until effective strategies for tailoring information are established.
Patient narratives should also be used with caution, as they may reduce quality of decision making. Patient narratives may unintentionally present unbalanced or biased information, which may undermine statistical data presented in the decision aid or encourage patients to focus on factors that are not in line with personal values or preferences. Further research should focus on identifying types of narrative content and presentation that facilitate quality decision making.
Further research is also needed in the areas of feedback, explicit values clarification, and social support to guide future integration. There was substantial heterogeneity in effects between types of components within each of these features, which may reflect artificial grouping of components. In addition, small sample sizes limited appropriate assessments, with many components having been tested only in one decision aid, which limited guidance for integration of these features into decision aids.
Studies included in the meta-analyses had a high level of heterogeneity with regard to patient populations, decision context, characteristics of the interventions, and components being tested, as well as choice of usual care or alternative aid controls. Studies were selected for inclusion based on testing a computer-based decision aid intervention, evaluating quality of decision making by measuring either knowledge or decisional conflict, and using an RCT design. For each subgroup analysis, decision aids that incorporated a specific feature (or type of component) were compared to decision aids that did not incorporate the feature. Effectively, this approach compared groupings of studies that tested various complex decision aids against very different control groups. Therefore, results from this study should be interpreted as “hypothesis-generating” and should be considered preliminary evidence to guide future work in this area.
Small numbers of studies incorporated certain types of components, which reduced the power to detect significant subgroup differences but also increased the probability of false positives. Nevertheless, subgroup analyses were conducted for all features and types of components for completion, and the number of studies in each subgroup was considered when interpreting the results.
Similarly, the study did not adjust for numerous comparisons generated through the subgroup analyses, which also increased the probability of false positives. However, given the exploratory nature of the study, such adjustments may not be necessary, since findings will require further research to establish independent contributions of each feature [
Given that the majority of decision aids incorporated multiple features, conducting subgroup analyses limited the capacity to disentangle the effects of specific features or to assess whether specific bundles of features were more effective for improving quality of decision making. Ideally, conducting a meta-regression, similar to the analysis described by Michie et al, would address these shortcomings [
Integration of media rich or interactive features into computer-based decision aids can improve quality of preference-sensitive decision making beyond traditional static approaches. However, this is an emerging field with limited evidence to guide implementation. The systematic review and thematic synthesis identified features used in available computer-based decision aids, in an effort to facilitate reporting of these features and to promote integration of such features into decision aids. The meta-analyses and associated subgroup analyses provide preliminary evidence to support integration of specific features into future decision aids. Further research can focus on clarifying independent contributions of specific features through experimental designs and refining the designs of features to improve effectiveness.
Systematic review search strategies for MEDLINE, Embase, CINAHL, and CENTRAL.
Information about studies included in the systematic review.
Consolidated Standards of Reporting Trials
Medical Subject Headings
Preferred Reporting Items for Systematic Reviews and Meta-Analyses
randomized controlled trial
standardized mean difference
AS is supported by the Canadian Institutes of Health Research (CIHR) through the Frederick Banting and Charles Best Canada Graduate Scholarship Doctoral Research Award and by Knowledge Translation Canada through the Strategic Training Initiative in Health Research (STIHR) Fellowship.
The authors would like to thank Genevieve Gore for assistance in developing the search strategies for the systematic review. The authors would also like to thank investigators who provided additional information required to calculate SMDs for the meta-analyses and tests for subgroup differences.
AS and RT were responsible for study conception and design; AS and DK acquired data; AS, AM, and RT analyzed and interpreted data; AS drafted the paper; and DK, AM, RT made critical revisions. All authors approved the final manuscript.
None declared.