A secondary analysis of a cross-sectional wellness study of US physicians from all subspecialties was performed. The original survey collected data between October 12, 2017, and March 15, 2018. The sample was assembled using the American Medical Association Physician Masterfile, a nearly complete record of all US physicians, independent of American Medical Association membership. Participation involved voluntary completion of an anonymous electronic survey. Full details of the sampling strategy, recruitment, data collection, and assessment for response bias have previously been reported [32]. Briefly, 30,456 physicians from the Masterfile were invited to participate (Figure 2). Of these, 5197 (17.1%) physicians participated in the study [32]. A random group of 1250 of the participants was invited to complete a subsurvey evaluating their EHR’s usability. Responders who were retired from clinical practice were excluded from the analysis. To evaluate for response bias, an intensive follow-up survey was conducted in a sample of nonresponders. There were 248 (52.1% of 476 invited) participants in the follow-up survey. The Stanford University and Mayo Clinic institutional review boards reviewed and approved the study protocol.

Study measures included items pertaining to participants’ demographic characteristics (age, gender, relationship status), medical specialty, hours worked per week, number of nights on call per week, practice setting, symptoms of burnout, and perception of their EHR’s usability and daily clinical workload.

Physician-perceived EHR usability was measured using the SUS, an industry standard for a quick, reliable measurement of technology usability [33-35]. The SUS includes 10 items, each on a 5-point Likert scale from Strongly Disagree to Strongly Agree that is scored from 0 to 4, with each even-numbered question reverse coded. The items are summed and then multiplied by 2.5 to normalize scores from 0 to 100, with higher scores indicating higher usability. Consistent with convention [34,36], the language of the SUS was modified such that references to “the system” being evaluated were changed to “my EHR.”

Provider task load was measured using 4 items (the mental demand, physical demand, temporal demand, and effort required subscales) from the NASA-TLX [28]. The rationale for exclusion of the frustration and performance NASA-TLX items from our measurement of PTL is that on principal component analysis with oblimin rotation and Kaiser normalization to determine underlying patterns between the 6 items of the NASA-TLX and the emotional exhaustion and depersonalization scales of the Maslach Burnout Inventory, frustration and performance clustered as one component along with emotional exhaustion and depersonalization scores, suggesting that these domains of the NASA-TLX are measures of work-related distress and would be expected to be collinear with burnout measures. Inclusion of the mental demand, physical demand, temporal demand, and effort required NASA-TLX items is consistent with previous reports assessing workload of physicians and nurses [27,37]. Respondents were prompted to “reflect on a day [they] performed clinical work during the last 1-2 weeks that is representative of a typical current clinical work day” and rate their perception of each subscale demand type on a scale of 0 to 100 (with 100 being the highest level of demand) [38,39]. The 4 scores were summed for a composite score ranging from 0 to 400 [40].

Burnout was measured using the validated criterion standard, the Maslach Burnout Inventory [41-44]. Respondents with a high score on the emotional exhaustion (≥27) or depersonalization subscale (≥10) were considered to have at least 1 symptom of burnout [41,45-47].

Standard descriptive statistics were used to characterize the physician sample that responded to the EHR usability subsurvey and the PTL items of the survey. Associations between variables were evaluated using the Wilcoxon rank sum test (continuous variables) or χ² test (categorical variables), as appropriate. Univariable linear regression was used to examine the association between EHR usability and PTL. On preliminary analysis controlling for medical specialty, specialties with smaller numbers of participants in the subsurvey had considerable variability. To control for this variation, specialties with fewer than 20 participants were grouped together in a pooled category of “Other” specialties. The Other category included these specialties (number of respondents in parentheses): neurosurgery (9), ophthalmology (8), otolaryngology (10), other (18), physical medicine and rehabilitation (15), preventive medicine & occupational medicine (4), radiation oncology (5), and urology (7). Two multivariable linear regression analyses were performed to investigate whether the relationship between SUS score and burnout is mediated by PTL. Demographic and professional factors included in the multivariable regression analyses were age, gender, relationship status, hours worked per week, medical specialty, nights on call, and practice setting. All tests were 2-sided, with a type I error level of .05. Analyses were completed using R statistical software (R Foundation for Statistical Computing, Vienna, Austria, Version 3.6.0) with the exception of the mediation analyses, which were conducted using the PROCESS version 3 macro for SPSS (IBM Corporation) [48].

There were 5197 responders to the full survey. A randomly selected group of 1250 of these responders received the EHR usability subsurvey. Among these responders, the 848 individuals (67.8%) who responded to all SUS and PTL items were included in the present analysis. The demographic characteristics (age, gender, and medical specialty) of the respondents included in this analysis were generally similar to the full survey respondents and US physicians (Table 1).

Physician-perceived EHR usability and PTL scores were similar to those previously reported in the primary analyses of this survey [7,27]. For this analysis, the mean SUS score was 46.1 (SD 22.1; range: 0-100; IQR 30-62.5), and the mean composite PTL score was 262.5 (SD 71.7; range: 0-400; IQR 215-315) with mean subscale scores of 70.9 (SD 20.8) for mental demand, 47.8 (SD 27.3) for physical demand, 68.3 (SD 24.9) for time demand, and 75.6 (SD 21.1) for effort required. On univariate analysis, each 1-point increase in SUS score (indicating greater EHR usability) was associated with a 0.55-point decrease in PTL score (P<.001; Figure 3). The relationship between perceived EHR usability and composite PTL scores is shown for the 17 specialty discipline categories in Figure 4.

On multivariable analysis adjusting for age, gender, relationship status, medical specialty, practice setting, hours worked per week, and number of nights on call per week, EHR SUS scores were associated with PTL. Each 1-point increase in SUS score was associated with a 0.57-point decrease in composite PTL score (P<.001; Table 2). Gender; age; practicing emergency medicine, anesthesiology, or psychiatry; hours worked per week; and number of nights on call per week were also associated with composite PTL scores in this model. Being female, practicing emergency medicine or anesthesiology, more hours worked per week, and more nights on call per week were all associated with higher PTL as a measure of workload, whereas being older and practicing psychiatry were both associated with lower workload.

In the first ordinary least squares regression model of the mediation analysis (Figure 5), higher SUS was significantly related to lower PTL scores (β=−.537, 95% CI −0.755 to −0.319; SE 0.111; P<.001). In the second logistic regression model, which included SUS and PTL as predictor variables of burnout, both SUS and PTL were significantly associated with burnout (OR 0.978, 95% CI 0.972 to 0.985 and OR 1.009, 95% CI 1.007 to 1.011, respectively). The bootstrap confidence intervals derived from 5000 samples indicated that the indirect effect of PTL on the association between SUS and burnout was significant (effect=−0.005, 95% CI −0.007 to −0.003). From this result, higher SUS was associated with lower PTL, which was associated with lower odds of burnout.

The findings of this national study indicate that physician-perceived EHR usability scores are strongly associated with physician daily task load. After adjusting for multiple personal and professional characteristics, individual physicians’ composite PTL scores were 0.55 points lower for each 1 point more favorable EHR usability, as measured by the SUS. The mediation analysis supports the proposed conceptual framework in which task load acts as a mediator between EHR usability and professional burnout. Despite the strong statistically significant associations found, the amount of variability in PTL as a measure of workload explained by EHR usability was small (R²=0.03). This finding indicates that factors other than EHR usability appear to be the primary drivers of physician workload as assessed by the mental demand, physical demand, temporal demand, and effort required subscales of the NASA-TLX. In the primary analysis of this survey, age, gender, medical specialty, hours worked per week, number of nights on call per week, and practice setting were all associated with PTL as a measure of workload [27]. To put these findings in context, a 10-point lower PTL was associated with 30% lower odds of burnout in the primary analysis [27]. An 18-point higher average SUS score (associated with a 10-point lower PTL in this analysis) would give the EHR a grade of D (instead of F) and be above the score of Microsoft Excel (Microsoft Corporation, 2009) [7].

Exploring this relationship and variation in EHR usability and PTL as a measure of workload by specialty reveals that some specialties with higher task load rated their EHRs more favorably (eg, anesthesiology and dermatology) while other specialties associated with lower task load (eg, orthopedic surgery and family medicine) rated their EHRs less favorably. This finding suggests the relationship between EHR usability and task load may not be due to physicians in specialties with higher task load rating their EHR less favorably. For example, anesthesiologists may benefit from the EHR’s ability to provide certain information rapidly in the operating room, yet still have higher task load due to factors in the anesthesiology clinical context that are not EHR-related. However, only three specialties (emergency medicine, anesthesiology, and psychiatry) remained predictive of task load after controlling for EHR usability, gender, age, hours worked per week, and number of nights on call per week. The analysis is likely underpowered to explain specific differences between individual specialties. On balance, SUS appears to have only a small influence on task load, although the effect of SUS on PTL varies by specialty.

It is also notable that in this analysis, lower EHR usability and higher workload were both associated with the odds of burnout after controlling for multiple personal and professional characteristics, suggesting that these are distinct domains that both represent potential improvement targets to reduce physician burnout. A study of 46 participants’ perception of the usability and task load of three popular websites found no association between usability and task load as assessed by the SUS and NASA-TLX [49]. Important differences between that study and the present study were that SUS scores were only in the high range (compared to the EHR) and that they measured both domains for individual tasks as opposed to in aggregate as we have here. The primary analysis of this national survey of 5197 physicians found that age, gender, relationship status, hours worked per week, and practicing certain medical specialties were all associated with the odds of burnout [32]. In this analysis, gender was not significantly predictive of the odds of burnout. Given this finding and the growing literature on the association of gender on physician burnout [32,50,51], future work should explore the interactions between these variables and further evaluate how EHR usability and task load vary by personal and professional characteristics. For example, a mixed-methods assessment of 25 intensive care physicians running simulated cases found gender-based differences in perceived EHR workload stress, satisfaction, and usability as assessed by the NASA-TLX and SUS [52]. Modest redesign of computerized reminders was associated with improved usability and decreased mental workload for 16 nurses in a simulated environment as assessed by the NASA-TLX [53]. A randomized crossover trial of 7 pediatric surgeons reported improved SUS scores and lower cognitive workload scores when order sets were systematically developed [54]. Future work could explore the relationship between EHR usability, specific task load, and professional burnout across and between different medical specialty disciplines and EHR vendor products as well as further differentiate the contribution of administrative burden to these areas by specialty. Policy makers should also explore the potential savings to the health care system that might be realized by improving EHR usability and how removing required administrative tasks that are not intrinsic to the practice of medicine could decrease task load and burnout.

This study is subject to several limitations. First, although the association between SUS and PTL scores is statistically significant, cross-sectional data cannot prove causation or the potential direction of effect. Second, with all survey research, response bias and representativeness of the study sample are potential limitations. An assessment for response bias was employed in this study, including evaluation of a random sample of nonresponders to the initial survey who participated in an incentivized follow-up survey. Although women were more likely to participate in the EHR usability subsurvey, participants were generally representative of US physicians with respect to age, years in practice, and prevalence of burnout. Third, self-reported data is subject to limitations [55,56]. Physician perceptions of usability and workload may not accurately reflect reality [20]. For example, one dimension of usability is error tolerance, a system’s ability to prevent or recover from an error; due to poor usability, EHR users may not be aware of errors they make in the EHR [11,12]. In addition, individual respondents could have a tendency to provide similar responses to different questions, which could contribute to a common method bias [57]. Fourth, although the SUS and NASA-TLX are considered the industry standards to assess technology usability and task load, they are intended to assess a specific task within a single system, not an entire class of software or an entire day of clinical work. Instead, this analysis provides a global assessment of EHR usability and provider task load and, therefore, cannot account for differences between specific tasks or vendors’ products and may be subject to recall bias. Fifth, physician respondents could conflate EHR usability issues with regulatory and clerical demands that manifest in the EHR but may be unrelated to EHR user interface design [21,22]. If that is the case, the association of clerical burden with task load and burnout could be stronger than that of EHR usability [7,16].

Despite these limitations, as the first national study exploring the relationship between EHR usability and workload in practice across medical specialties and practice settings, this analysis adds an important dimension to existing knowledge about factors associated with PTL and physician burnout. Our findings are consistent with multiple smaller studies of task load in both simulated [29,30] and real world settings [58,59] suggesting that more usable EHR interfaces are associated with lower cognitive load. Technology changes physicians’ information gathering and reasoning strategies, thereby shaping cognitive behavior [60]. Cognitive load theory should be taken into consideration to design EHR interfaces and workflows that meet users’ needs and are free from unnecessary extraneous cognitive load, such that the EHR becomes part of a physician work environment more closely resembling a “manageable cockpit” that is streamlined, ergonomic, and safe [10,61,62].

As assessed by US physicians using standardized metrics of technology and workload, a strong association was observed between EHR usability and task load, with more favorable usability associated with lower task load. Although EHR usability was significantly related to workload, the effect size was small, indicating that factors other than the EHR appear to be the primary drivers of workload. Both outcomes were also associated with the odds of burnout. Efforts to address physician burnout should attend to both improving EHR usability and addressing other drivers of task load, particularly extraneous load. Improving EHR usability while decreasing extraneous task load has the potential to allow practicing physicians more available working memory for medical decision making and patient communication. Specific areas to target could include consolidating the display of related information, reducing redundancy of information, and increasing standardization [63,64].