This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
There has been a lack of understanding on what types of specific clinical information are most valuable for doctors to access through mobile-based electronic medical records (m-EMRs) and when they access such information. Furthermore, it has not been clearly discussed why the value of such information is high.
The goal of this study was to investigate the types of clinical information that are most valuable to doctors to access through an m-EMR and when such information is accessed.
Since 2010, an m-EMR has been used in a tertiary hospital in Seoul, South Korea. The usage logs of the m-EMR by doctors were gathered from March to December 2015. Descriptive analyses were conducted to explore the overall usage patterns of the m-EMR. To assess the value of the clinical information provided, the usage patterns of both the m-EMR and a hospital information system (HIS) were compared on an hourly basis. The peak usage times of the m-EMR were defined as continuous intervals having normalized usage values that are greater than 0.5. The usage logs were processed as an indicator representing specific clinical information using factor analysis. Random intercept logistic regression was used to explore the type of clinical information that is frequently accessed during the peak usage times.
A total of 524,929 usage logs from 653 doctors (229 professors, 161 fellows, and 263 residents; mean age: 37.55 years; males: 415 [63.6%]) were analyzed. The highest average number of m-EMR usage logs (897) was by medical residents, whereas the lowest (292) was by surgical residents. The usage amount for three menus, namely inpatient list (47,096), lab results (38,508), and investigation list (25,336), accounted for 60.1% of the peak time usage. The HIS was used most frequently during regular hours (9:00 AM to 5:00 PM). The peak usage time of the m-EMR was early in the morning (6:00 AM to 10:00 AM), and the use of the m-EMR from early evening (5:00 PM) to midnight was higher than during regular business hours. Four factors representing the types of clinical information were extracted through factor analysis. Factors related to patient investigation status and patient conditions were associated with the peak usage times of the m-EMR (
Access to information regarding patient investigation status and patient conditions is crucial for decision making during morning activities, including ward rounds. The m-EMRs allow doctors to maintain the continuity of their clinical information regardless of the time and location constraints. Thus, m-EMRs will best evolve in a manner that enhances the accessibility of clinical information helpful to the decision-making process under such constraints.
Clinical work that takes place in various locations (ie, wards or clinics) and involves various treatment tasks (ie, diagnosis or operation) requires doctors to move a lot [
One research stream examined the behavioral patterns related to the adoption and use of m-EMRs, including personality traits and social norms [
Typically, clinical work is carried out through a daily process, which is organized based on hospital conditions [
These attempts may provide fundamental solutions for increasing the use of m-EMRs in large hospitals by identifying the most valuable clinical information accessed through such records. Additionally, these discussions may provide knowledge in research areas investigating the value of m-EMR usage in terms of information flow efficiency. Therefore, as a first attempt to shed light on the issues mentioned above, this study aimed to explore an empirical resolution on what type of clinical information is most valuable for doctors to access through an m-EMR based on their actual usage logs and when such information is accessed. In addition, this study aimed to discuss the importance of such information.
A tertiary hospital in Seoul, South Korea, with more than 2700 beds and approximately 912,300 admissions each year developed an m-EMR app in 2010. The main purpose of this m-EMR app is to allow medical personnel to read patient information without issuing treatment orders [
The app comprises four default menus and several submenus. The default menus provide patient lists, and doctors can choose one of the following menus: inpatient list, operation patient list, consult patient list, and emergency patient list. The submenus allow doctors to access patient details such as laboratory test results, medical records, and medication orders. The structure of information accessed through the m-EMR app is shown in
Structure of information accessed through the hospital’s mobile-based electronic medical records app. Usage logs from 12 menus (gray-shaded menus) providing 22 types of information were used in this study. PACS means picture archiving and communication system.
This research was approved by the institutional review board (IRB No. 2016-0287). To determine what type of clinical information is most valuable for doctors to access through an m-EMR and when such information is accessed, a two-step empirical analysis was conducted. First, the usage patterns of both the m-EMR and the hospital information system (HIS) on an hourly basis were explored. Comparing the usage patterns for both types of systems can provide an explanation on when access to clinical information through m-EMRs is valuable. Furthermore, it can provide a basis to explain why certain clinical information read through an m-EMR is more valuable than when read using the HIS.
Second, the types of clinical information accessed most frequently during m-EMR peak usage times were investigated. The usage concentration of a particular type of information within a specific time interval indicated that its value was high at that time [
When evaluating clinical information, it might be inappropriate to analyze the m-EMR usage logs at a very raw level (ie, usage count of each menu). Although some menus are used frequently, they may serve as intermediary channels to reach submenus that access detailed information. Thus, it is important to mine the raw usage logs so that usage patterns become representational clinical information. Data preprocessing and factor analysis were applied to extract representational clinical information. Finally, a random intercept logistic regression was employed to determine the association between usage peak intervals and representational clinical information.
For the study data, usage counts (population data) of the m-EMR and the utilization rate of the HIS central processing unit (CPU) were used. The CPU usage rate represents the amount of time that the CPU processes tasks in a specific time interval [
Flowchart of data preprocessing and analysis; m-EMR: mobile-based electronic medical records, HIS: hospital information system, CPU: central processing unit.
The structure of the m-EMR was designed to display some lower-level information (ie, lab result values) simultaneously using upper-level information (ie, lab results) (
Owing to their default status, the four patient list menus are likely to be used regardless of intent. Thus, the usage amount of these menus should be treated differently from that of the other submenus, even though these menus provide the function of a patient list check. To address this issue, logs used primarily to check patient lists (the four patient list menus) were separated from logs used to access detailed patient information. Specifically, if the log remained in the four default menus (ie, there were no usage traces after these default menus had been used) during one usage session, it was considered that the doctor simply identified the patient lists during that session. However, if there were traces indicating that the submenus were used after the four default menus had been used, it was considered that the doctor accessed detailed information. Thus, the four patient list menus could each have had two purposes (four menus × two purposes). Therefore, 16 variables representing the usage logs of the menus were included in this study (four patient list menus assumed to be default menus used to access submenus, designated by the subscript “default”; four patient list menus assumed to be used to check patient lists; and eight submenus). R version 3.3.2 (The R Project for Statistical Computing) was used for data preprocessing.
First, the general usage statistics of the medical and surgical departments were reviewed to determine whether m-EMR use differed according to the user characteristics and tasks. Second, the usages of the m-EMR and the HIS CPU over time were compared. The units of the two usage logs are different because the m-EMR usage level is based on the usage counts, whereas the HIS CPU usage level is based on the CPU utilization rate. Thus, the normalized values of the HIS and m-EMR usage over time were compared. Third, the peak usage intervals of the m-EMR were defined. The usage counts (number of times the m-EMR was accessed) per hour were normalized, and a continuous interval with normalized values that are greater than 0.5 (ie, the median of the normalized values) was defined as a peak interval. Details of the usage per menu during the peak usage interval were then examined at the raw-data level.
In a hierarchical app design, higher-level menus serve as links to the submenus while providing particular information [
To generate indicators of how relevant a usage session is to specific clinical information (ie, representational clinical information), a factor analysis was applied [
To analyze what type of clinical information is accessed frequently during peak m-EMR usage intervals, a random intercept logistic regression was applied. The random intercept model is often used to address individual heterogeneity when data are observed repeatedly [
The dependent variable (1=peak usage time, 0=outside the peak usage time) indicates whether a usage session belonged to the usage peak interval of the m-EMR. For the independent variables, the scores from the results of the factor analysis were used. In addition, the model controlled whether the m-EMR was used on a weekday or holiday, and for the demographics, that is, age, gender, and six positions (residents, fellows, and professors from medical departments and residents, fellows, and professors from surgical departments). The model was implemented using STATA version 14 (StataCorp LLC).
Equation for random intercept logistic regression.
A total of 524,929 usage logs for 12 menus, which provide 22 types of information, were stored during the study period (March to December 2015). The overall user characteristics and usage statistics are listed in
The HIS CPU usage rate for one week of November 2016 was used in this study. The usage patterns of both the HIS and the m-EMR based on the time of day were significantly different (
The peak usage interval for the m-EMR was defined as 6:00 am to 10:00 am.
Usage statistics of the m-EMR menus at peak usage intervals.
Usage count | Time | Total | |||
6-7 am (n=357) | 7-8 am (n=460) | 8-9 am (n=474) | 9-10 am (n=429) | (6-10 am) | |
Inpatient list | 10,059 | 15,207 | 13,681 | 8149 | 47,096 |
Lab results | 5810 | 10,051 | 12,818 | 9829 | 38,508 |
Investigation list | 3668 | 7156 | 8636 | 5876 | 25,336 |
Doctor note | 6083 | 5587 | 4193 | 2088 | 17,951 |
Nurse note | 7654 | 5655 | 2581 | 1196 | 17,086 |
Investigation other than lab results | 2169 | 5285 | 5134 | 2339 | 14927 |
PACS (picture archiving and communication system) view | 1639 | 2661 | 2586 | 1324 | 8210 |
Order view | 1073 | 2352 | 1430 | 724 | 5579 |
Consult patient list | 1379 | 1718 | 1168 | 506 | 4771 |
Emergency patient list | 816 | 1042 | 937 | 538 | 3333 |
Operation patient list | 219 | 856 | 323 | 257 | 1655 |
Medication history | 15 | 54 | 54 | 28 | 151 |
Difference in peak times between the m-EMR (mobile-based electronic medical records) and HIS (hospital information system). The graph of the m-EMR shows the normalized values over time, based on the m-EMR usage log. The graph of the HIS indicates the normalized values over time, based on the HIS CPU utilization rate. Each unit on the x-axis represents the hour (ie, 9 indicates the hour between 9:00 AM and 10:00 AM.).
A total of five factors with 13 variables were extracted under the conditions that the eigenvalues were greater than 1 and that the communality value for all variables was greater than 0.4 (
Factor 1 (F1): investigation status. This indicates a session in which a doctor accesses the investigation status and is defined based on a positive association with the variables of investigations (
Factor 2 (F2): emergency patient information. This indicates a session in which a doctor accesses emergency patient information and is defined based on a positive association with the Emergency patient listdefault and Doctor note variables.
Factor 3 (F3): patient conditions. This indicates a session in which a doctor accesses previous patient conditions and is defined based on a positive association with the Nurse note and Order view variables.
Factor 4 (F4): identification of patients in the emergency room (ER) or ward. This indicates a session in which a doctor identifies a patient in the ER or ward and is defined based on a positive association with the Emergency patient list and Inpatient list variables.
Factor 5 (F5): miscellaneous. This indicates a session in which the information access does not show a clear pattern. These sessions are associated with default menus and are indications that the doctor is accessing patient details through the submenus. However, because no usage patterns of the submenus can be determined, sessions associated with this factor are considered as miscellaneous.
None of the factors have a strong relationship (ie, factor loading with an absolute value greater than 0.4) with the Inpatient listdefault variable. This indicates a lack of correlation between Inpatient listdefault and other menu uses during a single usage session.
Results of factor analysis.
Variables | Factor | Communalitye | ||||
F1 | F2 | F3 | F4 | F5 | ||
Investigation other than lab results | .050 | −.022 | .204 | .096 | .603 | |
PACS (picture archiving and communication system) view | −.017 | −.120 | .126 | .060 | .549 | |
Investigation list | .016 | .078 | −.119 | −.087 | .693 | |
Lab results | −.173 | .120 | −.281 | −.199 | .460 | |
Emergency patient listdefaultb | −.003 | −.220 | .021 | .011 | .808 | |
Doctor note | −.041 | .376 | .044 | −.079 | .800 | |
Nurse note | −.140 | .075 | .044 | −.030 | .649 | |
Order view | .109 | −.147 | .100 | .107 | .544 | |
Emergency patient list | .200 | .064 | .099 | −.027 | .529 | |
Inpatient list | −.023 | −.030 | .067 | −.053 | .552 | |
Inpatient listdefault | .227 | .347 | .066 | −.375 | .103 | .516 |
Operation patient listdefault | .070 | .053 | −.136 | −.004 | .541 | |
Consult patient listdefault | −.049 | −.099 | .264 | −.100 | .544 | |
Result of adequacy tests for factor analysis | Bartlett testc: |
|||||
Keiser–Meyer–Olkin testd: 0.663 |
aFactor loadings with absolute values greater than 0.4 are in italics.
bThe “default” subscript indicates a menu likely used as the default screen.
cBartlett test evaluates the presence of a common component.
dThe Keiser–Meyer–Olkin test evaluates the appropriateness of the size of observations and number of variables used in the factor analysis.
eCommunality indicates how much the extracted factors account for each variable.
The results of a random intercept logistic regression indicate that F1 (investigation status) and F3 (patient conditions) are positively associated with peak usage intervals (
The control variable, Weekday, is statistically significant (
Diagram of associations between factors (only factors with loading values greater than 0.4 are listed); PACS: picture archiving and communication system.
Variable | Coefficient | Standard error | ||||||
F1 (investigation status) | .038 | 0.011 | .001 | |||||
F2 (emergency patient information) | −.226 | 0.017 | <.001 | |||||
F3 (patient conditions) | .210 | 0.013 | <.001 | |||||
F4 (identification of patients in the emergency room or ward) | −.109 | 0.013 | <.001 | |||||
F5 (miscellaneous) | −.126 | 0.014 | <.001 | |||||
Weekday | .566 | 0.023 | <.001 | |||||
Fellows (general medical departments) | .667 | 0.126 | <.001 | |||||
Fellows (surgical departments) | .417 | 0.146 | .01 | |||||
Professors (general medical departments) | .503 | 0.153 | <.001 | |||||
Professors (surgical departments) | .440 | 0.166 | .01 | |||||
Residents (general medical departments) | .302 | 0.111 | .01 | |||||
Age | −.008 | 0.006 | .22 | |||||
Gender | .0240 | 0.073 | .75 | |||||
Cons | −1.445 | 0.216 | <.001 |
aThe rank of residents from surgical departments was used as the baseline position to control the doctor position characteristics. The dependent variable indicates whether the usage session belongs to the peak interval or lies outside the usage peak interval (1=peak usage, 0=outside the peak usage). The number of observations is 56,756 (usage sessions), and the number of doctors is 653.
This study aimed to explore what types of clinical information accessed through an m-EMR are most valuable for doctors and when they access such information and to discuss how valuable such clinical information actually is. In large hospitals with complex treatment processes, patient care necessarily entails significant doctor movement. In such an environment, continuous awareness of the patient information through a desktop PC may not be efficient for doctors. Thus, several previous studies have demonstrated the utility of using mobile devices in relation to information flow efficiency during the treatment process [
The analysis conducted in this study demonstrates the unique value of an m-EMR system, which is distinct from a PC-based system in terms of information transaction. Interestingly, the m-EMR appears to be used frequently at times when the HIS is rarely used. Specifically, the HIS is heavily used during regular business hours (9:00 am to 6:00 pm), whereas the use of the m-EMR peaks early in the morning (6:00 am to 10:00 am). The m-EMR usage peak corresponds to morning rounds or the time just before routine work begins [
Furthermore, the use of the m-EMR is higher from early evening (5:00 pm) to midnight than during regular business hours. The high usage rate of the m-EMR during this time may indicate that doctors outside the hospital access patient information through the system. Owing to the continuity of patient care, doctors should check their patient information after work or share their opinions with colleagues who are on the night shift [
The results of this study indicate that an analysis of raw-level usage logs might lead to distorted results when exploring m-EMR usage patterns. Owing to the nature of the m-EMR structure, some menus can often be used regardless of intent. For instance, the inpatient list as one of the default menus is most frequently used during the peak usage interval at the raw-data level. There are two purposes for using this menu. First, the menu can be used as a simple patient checklist to review a list of patients under the doctor's responsibility or a list of newly admitted patients. Second, the menu can be unintentionally used owing to the default state of the menu. Considering the entire analysis, most doctors in this study might have set the inpatient list as their default screen. Specifically, the results of a descriptive analysis show that the use of the inpatient list was overwhelming, in contrast to the low use of other candidate default menus (ie, consult, emergency, and operation patient lists). Given that doctors have to use the default menu before using other submenus of the m-EMR app, its high utilization may indicate that the inpatient list menu is used most frequently as the default menu. Moreover, the results of a factor analysis indicate that there is no clear usage pattern after the Inpatient listdefault has been used. These results suggest that the inpatient list is used frequently as the default screen regardless of the doctor's intention. In addition, the investigation list is a gate menu located at the middle level for grouping the investigation results of patients rather than providing specific clinical information. Although the usage of these menus is high (ie, the first and third most frequently used menus), their usage amount may not be crucial in assessing the value of specific clinical information accessed through an m-EMR. These facts emphasize the importance of data science skills when examining the usage features of m-EMRs. Several advanced data mining techniques can be useful to investigate the usage characteristics of m-EMRs in more detail. For instance, process and sequential mining techniques may provide a better explanation on how doctors use m-EMRs by identifying and visualizing the sequence of usage patterns [
This study found four patterns of representational clinical information access (ie, investigation status, patient conditions, emergency patient information, and identification of patients in the ER or ward) when using an m-EMR. These differentiated usage patterns might indicate that specific information was accessed in an m-EMR usage session according to the treatment context. In other words, it might indicate that the m-EMR was used for unique purposes during each usage session. According to a regression analysis, the investigation status and patient conditions are positively associated with the times of peak usage, which correspond to the morning rounds or the time just before the rounds begin. Previous studies showed that important decisions in a treatment environment are made during the ward rounds [
The results of this study suggest that information obtained by a doctor through an m-EMR varies depending on the doctor’s department or task. A descriptive analysis shows that the overall usage of the m-EMR by doctors in general medical departments is higher than that of doctors in surgical departments. These results can be explained in terms of the intrinsic differences between the medical and surgical departments. Although both groups of doctors have the common goal of treating their patients, their tasks and working environments are different [
This research has several limitations. First, the research was conducted using log data from an m-EMR app used in only a single hospital. It is likely that each hospital has a unique m-EMR system and different schedules for its ward rounds. Therefore, other research environments might yield different results from those of this study. However, the value of an m-EMR in terms of information access is expected to also be demonstrable in other research environments. Second, it is acknowledged that more data are required to enable much better research. The data collection period for the m-EMR usage in this study differed from that for the HIS CPU usage rate. However, considering that the medical staff do not significantly change the way they use the HIS during their work processes, an analysis using log data from the m-EMR app and the HIS during the same period is expected to yield results similar to those of this study. In addition, information on personal and organizational tendencies regarding the use of m-EMRs was not included in this study. Previous studies have shown that personal and organizational characteristics have significant impacts on information technology usage in hospitals [
The most prominent feature of an m-EMR is location-independence in terms of information accessibility. Thus, m-EMRs can be best designed to facilitate access to information when doctors are under time and location constraints. Particularly during the early morning when access to clinical information through a desktop PC is highly limited, doctors can read information regarding a patient’s status using an m-EMR. In this regard, m-EMRs will best evolve in such a way that patient information essential for decision making during ward rounds is easily accessed and effectively presented.
Further research is required to gain a deeper understanding of m-EMR usage. The requirements for information acquisition through an m-EMR may vary according to the characteristics of different medical tasks. In addition, clinical information can be presented in various ways, depending on the design of particular m-EMRs. Thus, there may be research opportunities in exploring representational clinical information in other medical environments or using other m-EMR designs. Additionally, further research may aim to investigate the association between specific doctor groups and preferences for the types of information accessed through an m-EMR.
Service Structure and Contents of the mobile electronic medical record.
Overall usage statistics of the m-EMR based on doctor position.
central processing unit
emergency room
hospital information system
mobile electronic medical records
picture archiving and communication system
personal computer
The authors would like to thank the Medical Information Office of Asan Medical Center for providing log data on its mobile electronic medical records and for supporting the data analysis and interpretation. This research has been approved by the institutional review board (IRB No. 2016-0287).
None declared.