Published on 26.03.18 in Vol 20, No 3 (2018): March
Effect of Harm Anchors in Visual Displays of Test Results on Patient Perceptions of Urgency About Near-Normal Values: Experimental Study
Background: Patient-facing displays of laboratory test results typically provide patients with one reference point (the “standard range”).
Objective: To test the effect of including an additional harm anchor reference point in visual displays of laboratory test results, which indicates how far outside of the standard range values would need to be in order to suggest substantial patient risk.
Methods: Using a demographically diverse, online sample, we compared the reactions of 1618 adults in the United States who viewed visual line displays that included both standard range and harm anchor reference points (“Many doctors are not concerned until here”) to displays that included either (1) only a standard range, (2) standard range plus evaluative categories (eg, “borderline high”), or (3) a color gradient showing degree of deviation from the standard range.
Results: Providing the harm anchor reference point significantly reduced perceived urgency of close-to-normal alanine aminotransferase and creatinine results (P values <.001) but not generally for platelet count results. Notably, display type did not significantly alter perceptions of more extreme results in potentially harmful ranges. Harm anchors also substantially reduced the number of participants who wanted to contact their doctor urgently or go to the hospital about these test results.
Conclusions: Presenting patients with evaluative cues regarding when test results become clinically concerning can reduce the perceived urgency of out-of-range results that do not require immediate clinical action.
J Med Internet Res 2018;20(3):e98
Patients can increasingly view their laboratory test results directly via patient portals of electronic health record (EHR) systems . While patients value such information to enable self-management and support informed patient-provider interactions [ - ], access to test results does not guarantee that patients can understand or use that information to improve their health or their care [ ]. In particular, most current EHR patient portals present test results to patients in tables [ ], which is a format that is difficult for many patients to interpret, especially those with lower numeracy or literacy skills [ ]. In addition, EHR portals typically only provide patients with one reference point (the standard range) to aid in interpreting such data. Patients who receive out-of-range test values may have little idea how alarmed they should, or should not, feel [ , , , ]. Consequently, patients may call their doctor for an urgent appointment or even go to the hospital for something that would be more appropriately managed through regular follow-up visits [ ].
In order to improve patient understanding of laboratory test results, our research team used user-centered design principles to develop several visual number line formats for presenting these types of data . In an experimental test, displaying test results in these number line formats instead of tables increased user sensitivity to test result variations [ ]. We also showed that many people interpret all results outside of the standard range as equally urgent, even though many slightly out-of-range results are not, in fact, clinically concerning.
The core problem that patients face is one of information evaluability [, ]. Many patients lack training and experience with most laboratory tests, so they cannot necessarily map a particular test result to its meaning (ie, how good or bad it is, or how much risk it represents). As long as most patients lack meaningful reference points beyond the standard range, they will struggle to discriminate between different types of non-normal test results.
Here, we present results from additional data collected at the same time as the previous study in which we tested a visual display format that added a second reference point to indicate how far outside of the standard range a test value needs to be to become clinically concerning. Our objective was to determine whether providing such “harm anchors” would reduce the perceived urgency of near-normal values without significantly altering perceptions of extreme values, thereby increasing overall sensitivity to test result variations.
Participants were a demographically diverse, stratified random sample of adults in the United States recruited during May 2016 from a panel of Internet users, and the survey was administered by Survey Sampling International. The design, sampling process, data management procedures, and outcome measures received exempt status approval from the University of Michigan Health Sciences and Behavioral Sciences Institutional Review Board.
Participants were asked to imagine that they were using an online EHR portal to view laboratory test results and then saw three specific test results (platelet count, alanine aminotransferase [ALT], and serum creatinine). Each test result was initially shown as slightly outside of the standard range and then as a more extreme test result. All participants viewed a platelet count of 135 x 109/L and then 25 x 109/L, an ALT value of 80 U/L and then 360 U/L, and a creatinine value of 2.2 mg/dL and then 3.4 mg/dL.
In a previous paper, we compared participant reactions to three visual number line graph displays (): simple line displays that show the standard range and endpoints, but have no other visual reference points; blocks line displays that included color-coded evaluative categories (eg, “borderline high”); and gradient line displays that used a color gradient to indicate the extent of deviation from the standard range [ ]. The range of values shown and the evaluative labels and categories displayed were selected as plausible values based on input from several clinician members of our research team.
In this analysis, we compared the results from those three previously published conditions to data collected at the same time from an additional (randomized) group of participants. These participants received displays that included an added harm anchor (ie, a threshold line outside of the standard range labeled, “many doctors are not concerned until here”) but were otherwise identical to the simple line displays (see, bottom image). This language was developed through several iterations of pretesting with patients. Based on consultations with multiple clinician collaborators, we selected the anchor levels shown (platelets=100 x 109/L; ALT=160 U/L; creatinine=3.0 mg/dL) as plausible approximations of the point at which nonnormal values require more urgent attention for most patients.
The primary outcome measure was respondents’ subjective sense of urgency to the displayed test results. We averaged respondents’ responses to two questions: “How alarming does this result feel to you?” and, “How urgent of an issue is this result?” (both measured on 6-point Likert scales; 1=not at all, 6=very). The resulting measure of subjective urgency showed high reliability for all tests and test results (Cronbach alpha=.91 to .95). We measured behavioral intentions by asking whether respondents would initiate a new contact with a health professional (eg, by calling their doctor for an urgent appointment or going to a hospital) versus either waiting until their next regular appointment or doing nothing (for question details, see Zikmund-Fisher, et al ). To measure display format preferences, we used respondents’ average responses on a set of four questions (Cronbach alpha=.87) that asked how well the images described the results, how helpful they were, whether respondents would trust the images, and whether respondents would like to see results presented in these formats (all measured using 5-point Likert scales) [ ]. We report one-way analyses of variance (ANOVAs) with Bonferroni corrections to compare ratings of perceived urgency and user preferences, and chi-squared tests to compare willingness to wait. All analyses were performed using STATA 14 [ ]. All tests of significance were two-sided and used alpha=.05.
A total of 1618 adult participants (aged 19 to 89 years) completed the survey and were randomized to the experimental conditions examined here. Seefor respondent characteristics.
As shown in, providing the harm anchor reference point on the visual display significantly reduced perceived urgency of the close-to-normal ALT and creatinine results compared to all three other display formats (all P values <.001) without significantly altering perceptions of more extreme results in potentially harmful ranges. For platelet count results, however, we observed only a mildly significant difference between the harm anchor display and the blocks display. Use of the harm anchor labels also substantially reduced the number of participants who wanted to contact their doctor urgently or go to the hospital when shown near-normal ALT or creatinine test results ( ).
Overall, there were no significant differences in participants’ preferences among the four display types (Harm Anchor mean=3.77 vs Simple Line mean=3.62, Blocks Line mean=3.76, Gradient Line mean=3.68; ANOVA F(3, 1615)=2.33, P=.07).
|Age, mean (range)||48.8 (19,89)|
|Gender, n (%)|
|Race, n (%)|
|Native American||13 (0.8)|
|Other / Multi-race||69 (4.3)|
|Bachelor’s degree or higher education, n (%)||810 (50.1)|
|Current health, n (%)|
|Very good||605 (37.4)|
|Test result||Harm Anchor||Simple Line||Blocks Line||Gradient Line|
|Rating||P valuea||Rating||P valuea||Rating||P valuea|
|Platelets=135 x 109/L||3.66||3.72||>.99||3.94||.016||3.73||>.99|
|Platelets=135 x 109/L||5.09||5.26||.32||5.20||>.99||5.30||.10|
aP values reported are for comparisons to the harm anchor condition.
|Test result||Harm Anchor||Simple Line||Blocks Line||Gradient Line||Overall test|
|Chi square statistic||P value|
|Platelets=135 x 109/L||44.2%||50.0%||53.3%||51.6%||χ2(3)=7.61||.06|
As hypothesized, presenting patients with cues regarding the values at which particular test results become clinically concerning reduced respondents’ perceptions of urgency about certain types of out-of-range results that were not of immediate clinical concern. Our results suggest that including harm anchors in test result communications could, in certain circumstances, provide important benefits to patients by increasing the evaluability of variations among out-of-range results.
As in our previous comparison of visual line displays versus tabular displays , the effect size we observed varied substantially across the three tests presented. A possible explanation for this finding is that it is a function of the relative size of the standard reference range, as compared to the range of values shown. The largest effect of harm anchors was observed for ALT tests, which have a narrow reference range within a large range of possible values. By contrast, we observed minimal to no effect of harm anchors on displays of platelet counts; a test for which deviations of 50 to 100 (x 109/L) outside of the standard range (only 20%-40% of the width of the standard range) represent significant changes in patient risk. Thus, harm anchors may be most useful when communicating with patients about unfamiliar tests that can have wide ranges of potential variation.
Operationalization of this idea, however, will require overcoming several challenges. First, harm anchors, by definition, represent clinical judgment, and different clinicians may reasonably disagree regarding the point at which harm threshold should be set . Second, even if harm thresholds could be agreed upon, the point at which a patient (or clinician) should view a test result as requiring urgent action should logically vary based on patient characteristics or medical context (eg, initial diagnosis vs long term management). We also acknowledge the primary limitations of this study: the use of a hypothetical scenario and the testing of the harm anchor concept within the constraints of a particular visual display design. Inclusion of harm anchor information within other types of visual displays or tables might result in different findings than those observed here.
Nonetheless, our results demonstrate that designing displays to inform patients regarding what is dangerous, as opposed to what is considered usual or normal, might offer practical benefits. We suggest that harm anchors or other risk-related reference points should be considered when designing patient-facing displays of health data in order to increase the interpretability of such communications. These types of displays should be most useful in situations where relatively unfamiliar laboratory tests are being conducted for monitoring purposes (eg, monitoring liver or kidney function while on extended medication regimens). These situations are likely to result in mild deviations in test result values that, while important to monitor, are not immediately concerning to clinicians. Enabling patients to know that these mild deviations are not urgent will reduce patient worry and might also minimize unnecessary patient requests for urgent appointments when routine follow-up would be sufficient.
This work was previously presented at the Annual Meeting of the Society for Medical Decision Making, Vancouver, BC, Canada; October 24, 2016.
Funding for this research was provided entirely by a grant from the United States Agency for Healthcare Research and Quality to BJZ-F (R01 HS021681). The funding agreement assured the authors’ independence in: designing the study; the collection, analysis, and reporting of the data; and the decision to submit the article for publication. HOW receives salary support from a Research Scholar Junior 1 Career Development Award by the Fonds de Recherche du Québec-Santé.
The authors acknowledge the assistance of Sandeep Vijan, MD and Kenneth Langa, MD in determining the appropriate display ranges and categorization schemas for each of the types of test results used in this study. In addition, we acknowledge the graphic design efforts of Patrick Winkel in creating the images used in this study, as well as the work of Grace Bienek in developing earlier versions of these visual designs. Lastly, we are grateful for the guidance and inspiration provided by the patient members of our research team: Margaret Newton, Stephanie Burke, and James Piazza.
Conflicts of Interest
- American Hospital Association. Washington, DC: American Hospital Association; 2016. Individuals' ability to electronically access their hospital medical records, perform key tasks is growing URL: http://www.aha.org/research/reports/tw/16jul-tw-healthIT.pdf [accessed 2016-10-29] [WebCite Cache]
- Cimino JJ, Patel VL, Kushniruk AW. The patient clinical information system (PatCIS): technical solutions for and experience with giving patients access to their electronic medical records. Int J Med Inform 2002 Dec 18;68(1-3):113-127. [Medline]
- Keselman A, Slaughter L, Smith CA, Kim H, Divita G, Browne A, et al. Towards consumer-friendly PHRs: patients' experience with reviewing their health records. AMIA Annu Symp Proc 2007:399-403 [FREE Full text] [Medline]
- Bhavnani V, Fisher B, Winfield M, Seed P. How patients use access to their electronic GP record--a quantitative study. Fam Pract 2011 Apr;28(2):188-194 [FREE Full text] [CrossRef] [Medline]
- Pillemer F, Price RA, Paone S, Martich GD, Albert S, Haidari L, et al. Direct release of test results to patients increases patient engagement and utilization of care. PLoS One 2016;11(6):e0154743 [FREE Full text] [CrossRef] [Medline]
- O'Kane M, Freedman D, Zikmund-Fisher BJ. Can patients use test results effectively if they have direct access? BMJ 2015 Feb 11;350:h673. [Medline]
- Krist AH, Woolf SH. A vision for patient-centered health information systems. JAMA 2011 Jan 19;305(3):300-301. [CrossRef] [Medline]
- Zikmund-Fisher BJ, Exe NL, Witteman HO. Numeracy and literacy independently predict patients' ability to identify out-of-range test results. J Med Internet Res 2014;16(8):e187 [FREE Full text] [CrossRef] [Medline]
- Kelman A, Robinson CO, Cochin E, Ahluwalia NJ, Braverman J, Chiauzzi E, et al. Communicating laboratory test results for rheumatoid factor: what do patients and physicians want? Patient Prefer Adherence 2016 Dec;10:2501-2517 [FREE Full text] [CrossRef] [Medline]
- Giardina TD, Baldwin J, Nystrom DT, Sittig D, Singh H. Patient perceptions of receiving test results via online portals: a mixed-methods study. J Am Med Inform Assoc 2017:in press (forthcoming). [CrossRef] [Medline]
- Solomon J, Scherer A, Exe N, Witteman H, Fagerlin A, Zikmund-Fisher B. Is this good or bad? Redesigning visual displays of medical test results in patient portals to provide context and meaning. In: CHI'16 Extended Abstracts. 2016 Presented at: Human Factors in Computing Systems 2016; May 7-12, 2016; San Jose, CA p. 2314-2320. [CrossRef]
- Zikmund-Fisher BJ, Scherer AM, Witteman HO, Solomon JB, Exe NL, Tarini BA, et al. Graphics help patients distinguish between urgent and non-urgent deviations in laboratory test results. J Am Med Inform Assoc 2017 May 01;24(3):520-528 [FREE Full text] [CrossRef] [Medline]
- Hsee CK, Loewenstein GF, Blount S, Bazerman MH. Preference reversals between joint and separate evaluations of options: A review and theoretical analysis. Psychol Bull 1999;125(5):576-590. [CrossRef]
- Hsee CK, Zhang J. General evaluability theory. Perspect Psychol Sci 2010 Jul;5(4):343-355. [CrossRef] [Medline]
- Stata Corporation. Stata Statistical Software. College Station, TX: Stata Corporation; 2015.
- Piva E, Plebani M. Interpretative reports and critical values. Clin Chim Acta 2009 Jun;404(1):52-58. [CrossRef] [Medline]
|ALT: alanine aminotransferase|
|ANOVA: analysis of variance|
|EHR: electronic health record|
Edited by G Eysenbach; submitted 06.09.17; peer-reviewed by W Cha, T Risling; comments to author 02.12.17; revised version received 04.01.18; accepted 10.01.18; published 26.03.18
©Brian J Zikmund-Fisher, Aaron M Scherer, Holly O Witteman, Jacob B Solomon, Nicole L Exe, Angela Fagerlin. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 26.03.2018.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.