Three demographically diverse internet samples of patients with hypertension reviewed several brief vignettes describing fictitious patients with hypertension through Web survey; each vignette included a graph of the patients’ BP data. All 3 studies used a within-subjects design, where all participants reviewed all vignettes, presented in random order, and provided judgments about the degree of hypertension control for every patient/vignette. The Web surveys are described below, and the results are reported in accordance with the Checklist for Reporting Results of Internet E-Surveys (ie, CHERRIES checklist) [14]. The Institutional Review Board at the University of Missouri approved all studies. The 3 studies had very similar methodologies; therefore, we will describe their methods and results together. All samples were recruited by Qualtrics, a survey company that maintains an opt-in demographically diverse internet panel that participates; details about participant recruitment are provided below.

Each vignette described a patient who was being treated for hypertension and included a graph of the patient’s BP data over the past 2 years. Figure 1 presents an example of the data visualization tool. In Study 1, 9 vignettes systematically varied in the mean systolic BP (SBP; 130, 145, and 160 mm HG) and BP SD (5, 15, and 25) depicted in the graph; the slope was held constant (Multimedia Appendix 1). The mean SBP was chosen to represent clinical cases that included examples of controlled, uncontrolled, and borderline hypertension according to the 2014 Evidence-Based Guideline for the Management of High Blood Pressure in Adults from the panel members appointed to the Eighth Joint National Committee [15]. The SDs were chosen to represent small, moderate, and large mean variability according to published SBP values [16].

In Study 2, 6 vignettes differed in the SBP mean (130, 145, and 160 mm HG) and pattern of change over time (ie, trend or slope of the data; increasing or decreasing), while holding the SD constant (15 mm HG; Multimedia Appendix 2). In Study 3, we used 10 vignettes that differed in their mean SBP (130 and 145 mm HG), presence of outliers (no outliers, 1 outlier, or 2 outliers), and positioning of those outliers (above or below the mean; Multimedia Appendix 3).

Primary outcomes were (1) perceived BP control; (2) need for medication change; (3) subjective risk of heart attack; and (4) subjective risk of stroke for each vignette. Perceived BP control and need for medication change were assessed using agree-disagree Likert scales, while the subjective likelihood of heart attack and stroke were measured using unlikely-likely Likert scales. In addition, we asked participants to estimate the proportion of SBP values out of range for each vignette: “What percentage of the patient’s systolic blood pressure values (ie, top number) would you estimate to be out of the goal range?”

After evaluating all vignettes, participants completed the Subjective Numeracy Scale (SNS) [17] and a Single-Item Literacy Screener for health literacy (SILS) [18]. In addition, participants provided demographic information (age, gender, race/ethnicity, education, and income) and responded to 2 additional items about how often they monitor and graph their own BP; no identifying information was collected or stored.

All data were collected by open Web surveys developed using Qualtrics survey development software [19]. Informed consent, task instructions, practice task, vignettes, and study outcomes were all presented through a Web survey. The research team generated vignettes, BP displays, and outcomes for the studies. Of note, no adaptive items were used in the Web surveys. Responses to primary outcomes described above were recorded through slider bar. Responses to the SNS, SILS, and demographic items were all recorded using radio buttons on the Web survey, except age, which was recorded as free text. Responses to all items were required. Furthermore, back buttons on the browsers were disabled during the survey so participants could not go back and review previous webpages or change their responses on previous pages.

In all 3 Web surveys, participants completed one practice vignette that included directions for the practice task and 5 practice items; this section of the Web surveys utilized 3 webpages. Each vignette and its corresponding 5 questionnaire items (hypertension control, need for a medication change, 10-year heart attack risk, 10-year stroke risk, and percentage of SBP values out of goal range) added 2 additional pages to each Web survey (per vignette). The SILS and SNS scales added 9 items, and the demographic variables added 7 items to each Web study. Furthermore, we provided a free textbox at the end of the study for participants to share any additional feedback on the data visualizations presented in the Web studies.

In Study 1, the Web survey included 9 vignettes; therefore, participants completed 68 total items across 27 total webpages. In Study 2, the Web survey included 6 vignettes; participants completed 53 total items across 21 total webpages. In Study 3, the Web survey included 10 vignettes; participants completed 73 total items across 29 total webpages. The presentation order of the vignettes and their associated outcome measures was randomized within subjects. In addition, the team pilot-tested each Web survey themselves for functionality and usability before the surveys were deployed.

Qualtrics identified participants with hypertension through a single self-reported measure: “Has your doctor ever diagnosed you with hypertension, also known as high blood pressure?”; similar self-report items have been used to identify patients with hypertension in other epidemiological studies [20,21]. Informed consent was received from all participants after they were apprised about the purpose of the survey and the maximum completion time for the survey (30 minutes); participants were also provided with the name and contact information for the Project Director. Data for all 3 studies were collected between May 2016 and July 2016.

We used Internet Protocol addresses and Qualtrics identification numbers to determine unique site visitors. The survey completion rate was 86% (51/59) for Study 1, 79% (50/63) for Study 2, and 66% (55/83) for Study 3. All completed questionnaires were used in the analyses.

We planned to recruit 50 patients with hypertension for each of the 3 Web studies. The sample size was determined a priori using G-Power [22,23] with the following data characteristics: 80% power to detect a significant effect (f=0.25) at an alpha of .05, with a minimum correlation of .50 between repeated measures. All outcomes were treated as continuous variables. We examined the effects of data variations on primary and secondary outcomes by conducting a series of multivariate analysis of variance tests for repeated measures. Predictors included the level of the mean SBP (all studies), SBP SD (Study 1), presence and direction of slope (Study 2), and the number and direction of outliers (Study 3). All tests were conducted in SPSS version 24 and R version 3.5 and were considered statistically significant when P<.05.

Across the 3 experiments, a total of 156 patients with hypertension participated in this research. Participants were majority female (102/156, 65.4%) and white (116/156, 74.4%), with a mean age of 47.30 (range 19-79) years; Table 1 summarizes additional participants’ characteristics. In accordance with the sex and gender reporting guidelines [24], we present the data for all 3 studies disaggregated by gender in Multimedia Appendix 4.

In contrast, when the patient had BP that was objectively well controlled (mean 130 mm HG), the variability of the BP data had a significant impact on judgments about hypertension control and related health risks. For example, when the variability was low (SD 5 mm HG), BP was considered reasonably well controlled with little need for a medication change, and there was a much lower perceived risk of heart attack and stroke. In contrast, when variability in BP data was high (SD 25 mm HG), the perception of BP control was similar to those patients who had much higher mean (160 mm HG) BP values. This is noteworthy because evidence suggests that BP mean, much more than variability, is predictive of outcomes that matter to patients [25] (ie, heart attack and stroke). The effects of home BP variability on cardiovascular events and mortality are based on post-hoc analyses of 2 studies that looked at multiple indices of home BP variability [26]. The SD of BP is highly dependent on the mean BP, and it is unclear if indices of BP variability independent of the mean BP incrementally predict cardiovascular mortality or total mortality beyond mean SBP [27-29]. In addition, BP variability has unclear prognostic significance as varying methods or indices have been used to quantify the BP variability in all studies with no current standard or optimal indices available to quantify the BP variability.

Patients in Study 1 were also asked to recall the percentage of SBP values that were “out of range” (ie, exceeded 140 mm HG) in the graph. The recall was largely inaccurate but varied significantly by the SBP mean and SD (F_4,200=8.73; P<.001; generalized η²=0.04; Figure 2). With low variability (SD 5), participants overestimated the percent of SBP values out of range when hypertension was controlled (mean 130) and underestimated the percent of SBP values out of range when hypertension was borderline (mean 145) or uncontrolled (mean 160). The estimates of SBP values out of range were more accurate when there was moderate to high variability in the BP data. Figure 2 shows a jellyfish plot of error in SBP recall. Dots above the dotted line represent participant overestimation, and dots below represent underestimation. For each vignette, we show the following: (1) the mean (large black dot) and 95% CI (black line) in the center; (2) a dot plot of all data points on the left; and (3) a kernel-density plot of the distribution of answers on the right.

In Study 1, we observed a significant interaction between the SBP mean and SBP SD on perceived BP control (F_4,200=16.94; P<.001; generalized η²=0.08), need for medication change (F_4,200=16.19; P<.001; generalized η²=0.08), heart attack risk (F_4,200=8.88; P<.001; generalized η²=0.04), and stroke risk (F_4,200=11.70; P<.001; generalized η²=0.05; see Tables 2 and 3). When the mean SBP was high (160 mm HG), variability in the BP data did not impact the perception of BP control. Across all 3 SDs, participants reported that hypertension was not well controlled, patients should change their medication, and patients had an elevated risk of heart attack and stroke.

In Study 2, we observed that patient ratings of hypertension control, need for medication changes, and 10-year risk of heart attack and stroke were significantly affected by trends visible in the BP data (see Tables 4 and 5) . Participants judged a positive data trend (depicting SBP values increasing over time) to be significantly less well controlled (F_1,49=107.53; P<.001; generalized η²=0.45), in greater need of medication change (F_1,49=129.07; P<.001; generalized η²=0.51), and at greater 10-year risk for a heart attack (F_1,49=112.79; P<.001; generalized η²=0.47) and stroke (F_1,49=111.52; P<.001; generalized η²=0.47) than a negative data trend (depicting SBP values decreasing over time) with the same SBP mean. Consistent with Study 1, participants’ recall of the percentage of SBP measurements out of range was largely inaccurate (Figure 3). Although the graphs depicting an increasing and decreasing trend had the same number of out-of-range measurements, participants recalled seeing a greater percentage of out-of-range values when the SBP was increasing over time than when the slope was decreasing (F_1,49=33.04; P<.001; generalized η²=0.16).

Another way of examining the effect of variability in judgments about BP control is to consider the presence of extreme values (ie, outliers) independent of the overall measurement variance. In Study 3, we compared judgments about uniform data patterns (ie, no discernable outliers) with those with either 1 or 2 outliers. In addition, we systematically varied whether those outliers were above or below the mean BP of the depicted time period. The presence and number of outliers significantly affected judgments of hypertension control (F_{4, 216}=17.98; P<.001; generalized η²=0.14), need for medication change (F_4,216=13.38; P<.001; generalized η²=0.11), and perceived risk of heart attack (F_4,216=12.85; P<.001; generalized η²=0.10), and stroke (F_4,216=13.81; P<.001; generalized η²=0.11; Tables 6 and 7). When any extreme values were present, participants judged the patient to have hypertension that was significantly less well controlled, to be in greater need of medication change, and to be at greater 10-year risk for a heart attack or stroke than when the data had a more uniform distribution. Furthermore, 2 outliers (whether above or below the mean) were considered significantly more concerning than a single outlier.

Finally, recall for the percent of SBP values out of range was also significantly impacted by the presence and magnitude of extreme values (F_4,216=5.54; P<.001; generalized η²=0.04; Figure 4). When hypertension was controlled (mean 130), participants overestimated the percent of SBP values out of range with a uniform distribution and outlier(s) above the mean but more accurately recalled the percent of SBP values out of range when the outlier(s) were below the mean. In contrast, with borderline hypertension control (mean 145), recall for percent of SBP values out of range was fairly accurate with a uniform distribution and when there was only one outlier above the mean. However, participants consistently underestimated the percent of SBP values out of range when there was a single outlier below the mean or 2 outliers in either direction.

To inform the development of a physician-patient shared data visualization tool for BP values, we conducted 3 vignette-based Web studies to understand better how patients interpret the variability in visualizations of BP data. In all studies, we observed that patients with hypertension consistently judged variations in BP data as meaningful indicators of hypertension control. In Study 1, increased variability in SBP data was associated with increasingly negative judgments about hypertension control, even when the mean SBP remained constant and within goal range. In Study 2, we demonstrated that the pattern of change in SBP values over time also significantly influenced judgments about hypertension control. While graphs depicting an increase in the SBP over time were appropriately judged to be of greater concern than graphs showing a decrease in the SBP over time, recall for the number of out-of-range BP values was inaccurate. When the SBP values increased over time, patients remembered more out-of-range values than when the slope decreased, even though the number of out-of-range values did not differ in our study. Finally, the presence of outliers (Study 3) also significantly impacted patients’ judgments about hypertension control, the need for a medication change, and the risks associated with uncontrolled hypertension. Observing even 1 or 2 outliers in BP data leads patients to inappropriately determine that hypertension is not well controlled.

The use of data visualization in electronic medical records has the potential to transform clinical encounters. While the technology to develop these tools is available, little is known about how these data displays will influence patients’ judgments about hypertension control and subsequent decisions regarding treatment. These 3 studies have demonstrated that judgments about hypertension control are strongly and inappropriately influenced by the presence of outliers and variability in the data. Outliers and variability mask mean BP values and the presence of data trends—important predictors of BP-related health outcomes (eg, heart attack or stroke) [25]. These findings are important for the development of interventions to promote shared decision making in primary care, which must direct attention to clinically meaningful information, that is, mean BP and trends rather than variability in BP or outliers [25]. It should be stressed that these conclusions about the potential benefits of visualization techniques that minimize the impact of outliers and variability would apply only to measurements obtained from well-calibrated BP measurement devices. These recommendations should not be applied to devices where variability is derived from inaccurate measurement.

There are several limitations to these studies that potentially constrain their generalizability. One limitation is the use of internet patient samples. When patients are making decisions about treatment for hypertension, they are typically made in conjunction with their physician during a clinic visit. In addition, while our sample of patients is more demographically diverse than typical internet samples, it is not representative of the population of patients with hypertension. Furthermore, we focused only on patients in these studies; therefore, future work should examine the effect of data visualization on physicians’ judgments about hypertension control, as well as the effect on shared patient-physician decisions. It is possible that physicians will perform similarly to patients because we are examining judgments that stem from common perceptual and cognitive processes [30,31]. On the other hand, physicians may have greater knowledge about the relative importance of the BP mean and variability than patients, which could alter their judgments.

Health information technologies provide an opportunity for patients to become more engaged in decision making about hypertension control. We are endeavoring to design a data visualization tool for BP that can be jointly used by physicians and patients in this decision-making process. This tool aims to make the limited time shared in the exam room more efficient and effective. Defining how data elements, such as trends, variability, and outliers, support or detract from an understanding of the data will aid in the design of data visualizations that highlight meaningful characteristics of the data; this may, in turn, result in shared decisions that are better informed. Areas for future study include understanding how these parameters influence physician judgments about hypertension control and how information acquisition from data visualizations is affected by numeracy, health literacy, and graph literacy of patients and physicians.