This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
User engagement is a key performance variable for eHealth websites. However, most existing studies on user engagement either focus on a single website or depend on survey data. To date, we still lack an overview of user engagement on multiple eHealth websites derived from objective data. Therefore, it is relevant to provide a holistic view of user engagement on multiple eHealth websites based on cross-site clickstream data.
This study aims to describe the patterns of user engagement on eHealth websites and investigate how platforms, channels, sex, and income influence user engagement on eHealth websites.
The data used in this study were the clickstream data of 1095 mobile users, which were obtained from a large telecom company in Shanghai, China. The observation period covered 8 months (January 2017 to August 2017). Descriptive statistics, two-tailed
The medical category accounted for most of the market share of eHealth website visits (134,009/184,826, 72.51%), followed by the lifestyle category (46,870/184,826, 25.36%). The e-pharmacy category had the smallest market share, accounting for only 2.14% (3947/184,826) of the total visits. eHealth websites were characterized by very low visit penetration and relatively high user penetration. The distribution of engagement intensity followed a power law distribution. Visits to eHealth websites were highly concentrated. User engagement was generally high on weekdays but low on weekends. Furthermore, user engagement gradually increased from morning to noon. After noon, user engagement declined until it reached its lowest level at midnight. Lifestyle websites, followed by medical websites, had the highest customer loyalty. e-Pharmacy websites had the lowest customer loyalty. Popular eHealth websites, such as medical websites, can effectively provide referral traffic for lifestyle and e-pharmacy websites. However, the opposite is also true. Android users were more engaged in eHealth websites than iOS users. The engagement volume of app users was 4.85 times that of browser users, and the engagement intensity of app users was 4.22 times that of browser users. Male users had a higher engagement intensity than female users. Income negatively moderated the influence that platforms (Android vs iOS) had on user engagement. Low-income Android users were the most engaged in eHealth websites. Conversely, low-income iOS users were the least engaged in eHealth websites.
Clickstream data provide a new way to derive an overview of user engagement patterns on eHealth websites and investigate the influence that various factors (eg, platform, channel, sex, and income) have on engagement behavior. Compared with self-reported data from a questionnaire, cross-site clickstream data are more objective, accurate, and appropriate for pattern discovery. Many user engagement patterns and findings regarding the influential factors revealed by cross-site clickstream data have not been previously reported.
Providing and delivering web-based services is a major trend in the digital transformation of health services [
User engagement is a key variable for eHealth websites [
Although many previous studies have investigated engagement patterns [
User engagement on eHealth websites has received considerable attention in recent years. A review of the literature suggests two main research streams investigating engagement patterns and engagement interventions. The first research stream is descriptive in nature. The areas investigated include diabetes management [
The second research stream focuses on designing interventions to improve user engagement. System design [
The review results listed above indicate that extant studies on user engagement in eHealth only focus on a single website. A higher level of analysis that provides a complete picture of how users engage in different types of eHealth websites is still lacking. Although the meta-analysis allows multiple websites to be considered together, existing review studies on this topic still focus on a single category [
To bridge this research gap, we provide an analysis of user engagement on all eHealth sites with cross-site clickstream data in this study. Following the two research streams on user engagement [
First, we are interested in investigating user engagement patterns on all eHealth websites. More specifically, we will provide a framework for understanding the engagement patterns on eHealth websites. The framework includes the taxonomy of eHealth websites, market share, penetration, engagement intensity, engagement variety, day and hour trends, customer loyalty, and cross-site engagement. The taxonomy of eHealth websites is necessary because there are too many individual eHealth websites that cannot be covered in a single study. In addition, working on specific websites makes it difficult to reach a conclusion with general significance. Market share and penetration are included because they can jointly describe the market status quo and potential for that type of eHealth website (eg, a small market share with a high penetration usually means a great potential). Intensity, variety, time trend, loyalty, and cross-site behavior are included because they describe different aspects of user engagement. Therefore, the first research question (RQ) is as follows: What are the overall patterns of user engagement on multiple eHealth websites on smartphones (RQ1)?
Second, we are interested in identifying the factors that may influence user engagement on all eHealth websites. Following the Person, Environment, and Technology framework [
This study has several practical implications. First, our clickstream data analysis indicates that the visit penetration for eHealth websites is very low, and users usually concentrate only on one or two websites. However, eHealth websites are also characterized by relatively high user penetration. Therefore, eHealth websites should have great market potential. One possible way to increase user engagement on more eHealth websites is to provide cross-site recommendations. Medical websites are ideal sources for effectively providing referral traffic for lifestyle and e-pharmacy websites. However, managers must be cautious that the opposite may not be true. Understanding the asymmetric nature of cross-site browsing can help managers improve the effects of cross-site recommendations.
Second, the findings of this research show that Android users are more engaged in eHealth websites than iOS users, partly because more health apps are available on the Android platform, and the Android platform has a higher percentage of free apps than the iOS platform. Therefore, managers of the iOS platform should encourage developers to develop more health apps (especially free apps or apps with in-app purchase features) in the future.
Third, the results of this study suggest that app users are, on average, 4.5 times more engaged than browser users. Therefore, all eHealth websites should provide apps for both Android and iOS platforms. The managers of eHealth websites should also encourage users to download their apps and urge users to access their websites from apps instead of browsers.
The data used in this study are the access log data of 1095 4G users from a large telecom company in Shanghai, China. The observation period was 8 months (January 2017-August 2017). After removing confidential information (eg, telephone numbers), we obtained users’ internet access records on smartphones. Each access record contains the encrypted user ID, access time, mobile platform (mobile operating system), and URL visited. User demographic information such as encrypted user ID, sex, age, and monthly expenditures on mobile phones was also included in the data set.
The eHealth websites investigated in this study can be classified into the following three categories: medical, lifestyle, and e-pharmacy [
The eHealth websites investigated in this study (in China; n=373).
Category and website | Domain name | Visits, n (%) | Visitors, n (%) | |
|
||||
|
Good Doctor | haodf.com | 44,202 (23.9) | 56 (15.1) |
|
WeDoctor | guahao.com | 37,734 (20.4) | 38 (10.2) |
|
39 Health Net | 39.net | 23,007 (12.5) | 42 (11.4) |
|
Ask Doctor Quickly | 120ask.com | 15,613 (8.5) | 70 (18.9) |
|
Seeking Medical Advice | xywy.com | 11,769 (6.4) | 69 (18.6) |
|
Chunyu Doctor | chunyuyisheng.com | 1684 (0.9) | 9 (2.3) |
|
||||
|
Mint Health | boohee.com | 41,462 (22.4) | 8 (2.1) |
|
Health Preserving | cndzys.com | 4307 (2.3) | 17 (4.6) |
|
So-Young | soyoung.com | 1101 (0.6) | 34 (9) |
|
||||
|
Kang Aiduo Pharmacy | 360kad.com | 2639 (1.4) | 11 (3) |
|
Jianke Pharmacy | jianke.com | 1308 (0.7) | 17 (4.7) |
The engagement patterns investigated in this study include market share, penetration, engagement intensity, engagement variety, day and hour trends, customer loyalty, and cross-site engagement.
Market share is the percentage of the market that a single category controls based on the number of visits. The proportion of medical websites is relatively large and accounts for 72.51% (134,009/184,826) of the total, whereas the proportion of e-pharmacy websites is very small and accounts for only 2.14% (3947/184,826) of the total. The proportion is lifestyle websites is 25.36% (46,870/184,826).
This finding indicates that the greatest demand for eHealth websites is to obtain health knowledge and medical advice such as that on prevention, diagnosis, prognosis, and treatment plans. Lifestyle websites also received considerable market share, suggesting that the idea of health management is currently pervasive in China. However, the proportion of visits to e-pharmacy websites was relatively small. A possible reason for this is that the purchase of drugs is a low-frequency demand. Another possible reason is that e-pharmacies are not yet included in the scope of medical insurance in most areas of China. Lack of trust in e-pharmacy websites is also a reason.
eHealth behavior penetration measures how user behaviors on eHealth websites compare with those of all web behaviors. We focus on two types of user behaviors (
The results in
eHealth behavior penetration among three categories (n=373).
Category | Visit penetration, n (%) | User penetration, n (%) |
Medical | 134,009 (0.082) | 124 (33.5) |
Lifestyle | 46,870 (0.029) | 47 (12.7) |
e-Pharmacy | 3947 (0.002) | 23 (6.1) |
Engagement intensity is the number of visits to eHealth websites per session. In this study, we defined the length of a session as a day. Therefore, we measured engagement intensity as the number of visits to eHealth websites within a day. The engagement intensity patterns according to category are shown in
Visit intensity per day of the three categories of eHealth websites.
The results of the Kolmogorov-Smirnov test (D=0.025;
Engagement variety measures the extent to which users visit different types of eHealth websites. In this study, engagement variety was measured by the number of distinct eHealth websites over 3 months (
The distribution of engagement variety (n=373).
Number of websites accessed | Visitors, n (%) |
1 | 238 (63.8) |
2 | 76 (20.4) |
3 | 45 (12.1) |
4 | 7 (1.9) |
5 | 5 (1.3) |
6 | 2 (0.5) |
7 | 0 (0) |
This finding suggests that eHealth websites are highly isolated. Users have great inertia and pay attention to only one or two websites. For example, low engagement variety may be attributed to the fact that increasingly more eHealth websites provide one-stop services where users can meet almost all their health needs on one site. The low visit variety also suggests that the links among eHealth websites are insufficient. As a result, users from one website may not be aware of other websites for quite a long time.
We were interested in user engagement patterns at the week and day levels. For both the week and day levels, we observed the trends of the three key engagement variables (ie, engagement volume, user volume, and engagement intensity) over time. The engagement volume was measured by the number of visits. The user volume was measured by the number of unique users. The engagement intensity was measured by the number of visits per user. All the measures for engagement volume, user volume, and engagement intensity were based on 373 users who visited the websites listed in
The fluctuation of the engagement volume in a week.
The fluctuation of the user volume in a week.
The fluctuation of the engagement intensity in a week.
For medical websites, there was more engagement from Monday to Wednesday, with the highest engagement volume and intensity seen on Monday. From Thursday to Saturday, the engagement volume and intensity decreased gradually until Sunday. The user volume was the highest on Tuesday, but the lowest on Sunday. In addition, the user volume of medical websites fluctuated more than that of the other two categories of websites.
For lifestyle websites, the engagement volume and intensity increased from Sunday to Thursday and then gradually decreased until Saturday. Engagement volume and intensity were the highest on Thursday and lowest on Saturday. However, the user volume on Thursday was the lowest in the week.
For e-pharmacy websites, the engagement volume, user volume, and engagement intensity were all the lowest compared with those of medical and lifestyle websites. The engagement volume and intensity were the highest on Friday but lowest on Saturday.
As shown in
The engagement trends at the day level are shown in
The fluctuation of the engagement volume in a day.
The fluctuation of the user volume in a day.
The fluctuation of the engagement intensity in a day.
For medical websites, user engagement peaked at noon. The highest engagement volume appeared at 12 noon, and the highest user volume appeared at 11 AM. However, peak engagement intensity occurred between 5 AM and 6 AM. One possible explanation is that users who encounter health problems at night will search for health information on the web during this period.
For lifestyle websites, the highest engagement was in the evening. For example, peak engagement volume and intensity occurred at 8 PM. This is because the use of lifestyle websites (eg, yoga exercise) usually takes a long time, and the ideal time is right after work. However, the largest number of users were engaged in lifestyle websites at 6 PM. Other peaks in engagement volume and intensity occurred at 7 AM, 1 PM, and 4 PM.
For e-pharmacy websites, user engagement fluctuated throughout the day. One special case is that the engagement intensity reaches its peak at 1 AM. e-Pharmacies are the most intensively used eHealth sites, and they have an engagement intensity that is even higher than those of the remaining two categories (ie, medical and lifestyle). One possible explanation for this phenomenon is that offline drug stores are closed at this time, and e-pharmacies are the only choice.
Customer loyalty is a measure of a customer’s likelihood of engaging in repeat business with a company or brand. In this study, customer loyalty was measured using the following variables:
Total visits: the total number of visits within the observation period.
Visit days: the number of days visited within the observation period.
Average daily visits: the average number of visits per day.
Recency: the number of days since the last visit (in this study, recency was measured based on the difference between the last visit date and the end of the observation period).
A radar chart was used to present customers’ loyalty to the three categories of eHealth websites (
Customer loyalty.
The results in
A user may visit several eHealth websites simultaneously, a phenomenon known as cross-site visits [
At the user level, suppose that the number of users who visit lifestyle websites is x and the number of users who also visit medical websites is y. The cross-site engagement of lifestyle websites with medical websites is y/x. Cross-site engagement at the user level is shown in
Cross-site engagement on the user level.
Category | Medical, n/N (%) | Lifestyle, n/N (%) | e-Pharmacy, n/N (%) |
Medical | 124/124 (100) | 31/124 (25) | 21/124 (17) |
Lifestyle | 31/47 (66) | 47/47 (100) | 11/47 (23) |
e-Pharmacy | 21/23 (92) | 11/23 (48) | 23/23 (100) |
At the visit level, suppose that the number of users who visit the lifestyle websites is w, and the corresponding number of visits is u. On the same day, the number of visits to medical websites by these users is v, and the cross-site engagement of lifestyle websites to medical websites is v/u. Cross-site engagement at the visit level is shown in
Cross-site engagement on the visit level (n=373).
Category | Medical, n/N (%) | Lifestyle, n/N (%) | e-Pharmacy, n/N (%) |
Medical | 134,009/134,009 (100) | 44,223/134,009 (33) | 3868/134,009 (3) |
Lifestyle | 44,223/46,870 (94) | 46,870/46,870 (100) | 2812/46,870 (6) |
e-Pharmacy | 3868/3947 (98) | 2812/3947 (71) | 3947/3947 (100) |
In this section, we investigate how the platform, channel, sex, and income influence user engagement (ie, engagement volume and engagement intensity) on eHealth websites. More specifically, we first investigate their influence independently and then investigate their interaction effects. Engagement volume is measured by the number of visits, and engagement intensity is measured by the number of visits per session (in this study, a session is defined as 1 day).
The platform refers to the operating system of the mobile phone used to visit eHealth websites. In this study, we focus on two platforms, iOS and Android, because they possess 97% of the global mobile market share. There are many differences between iOS and Android that may lead to different engagement behaviors on eHealth websites. Android has the greatest global market share at approximately two-thirds and has more app downloads than iOS. Sensor Tower reports that the Google Play Store experienced approximately 75.7 billion first-time app installs worldwide in 2018 [
iOS and Android also have different user groups. Owing to its broad price range and lower entry-level price point, Android has the largest global share in lower-income areas and developing nations [
The results of the two-tailed
Comparison of user engagement between platformsa.
Platform engagement | Value, mean (SD) | Cohen |
|||||||
|
2.26 (371) | .02 | 0.23 | ||||||
|
Android | 3.98 (2.08) |
|
|
|
||||
|
iOS | 3.48 (2.22) |
|
|
|
||||
|
32.10 (371) | <.001 | 1.39 | ||||||
|
Android | 3.29 (1.90) |
|
|
|
||||
|
iOS | 1.18 (1.00) |
|
|
|
aBox-Cox transformation was applied to engagement volume and engagement intensity.
One possible explanation for the difference is that there are more health apps and fewer charges on Android. This makes it easier for Android users to find free health apps to satisfy their needs. In addition, Android users are more introverted and more proficient in information technology [
The channel refers to the method through which a mobile user interacts with an eHealth website. In this study, we focused on two types of channels: mobile browsers and mobile apps. A browser can be found on any mobile phone, regardless of the operating system. Accessing an eHealth website through a browser is convenient because users do not need to download or install an app before the visit. However, it is essential to remember that network access, quality, and speed are all factors that can affect mobile web experience. Compared with a browser, an app has several advantages. For example, mobile apps offer greater personalization and operational efficiency, along with multiple other exclusive features. A well-designed mobile app can perform actions much quicker than a mobile website. In contrast to websites that generally use web servers, apps usually store their data locally on mobile devices. For this reason, data retrieval is quicker on mobile apps. Apps can further save users’ time by storing their preferences and using them to take proactive actions on their behalf. In addition, mobile apps can access and use built-in device features such as cameras, GPS, and location. Leveraging device capabilities leads to an enhanced, more convenient user experience.
We performed a
Comparison of user engagement between channelsa.
User engagement and channel | Value, mean (SD) | Cohen |
|||
|
15.51 (371) | <.001 | 1.44 | ||
|
App | 3.78 (2.13) |
|
|
|
|
Browser | 0.78 (0.76) |
|
|
|
|
21.51 (371) | <.001 | 1.09 | ||
|
App | 2.49 (1.76) |
|
|
|
|
Browser | 0.59 (0.57) |
|
|
|
aBox-Cox transformation was applied to engagement volume and engagement intensity.
The literature suggests that male and female users exhibit sizable differences in web-based engagement behaviors [
Comparison of user engagement between sexesa.
User engagement and sexes | Value, mean (SD) | Cohen |
|||||||
|
0.82 (371) | .41 | 0.10 | ||||||
|
Female | 4.06 (2.46) |
|
|
|
||||
|
Male | 3.83 (2.15) |
|
|
|
||||
|
−8.36 (371) | <.001 | 0.38 | ||||||
|
Female | 2.14 (1.58) |
|
|
|
||||
|
Male | 2.84 (1.93) |
|
|
|
aBox-Cox transformation was applied to engagement volume and engagement intensity.
The results in
The literature also suggests that high-income and low-income users exhibit some differences in engagement behavior [
Comparison of user engagement between low- and high-income usersa.
User engagement and types of users | Value, mean (SD) | Cohen |
|||||||
|
0.94 (371) | .35 | 0.11 | ||||||
|
Low-income users | 4.03 (2.34) |
|
|
|
||||
|
High-income users | 3.79 (2.17) |
|
|
|
||||
|
−1.07 (371) | .29 | 0.04 | ||||||
|
Low-income users | 2.57 (1.78) |
|
|
|
||||
|
High-income users | 2.65 (1.85) |
|
|
|
aBox-Cox transformation was applied to engagement volume and engagement intensity.
The results in
Interactions may exist among the four factors identified earlier. Therefore, an analysis of variance was conducted to test the potential interaction effects, and the results are shown in
The analysis of variance resultsa.
Factor | |||
|
|||
|
|
|
|
|
Channel×income | 0.03 (1) | .86 |
|
Platform×sex | 1.36 (1) | .24 |
|
Channel×sex | 0.09 (1) | .76 |
|
|||
|
|
|
|
|
Channel×income | 0.28 (1) | .60 |
|
Platform×sex | 0.12 (1) | .73 |
|
Channel×sex | 0.18 (1) | .67 |
aBox-Cox transformation was applied to engagement volume and engagement intensity.
bItalicization denotes significance (
The details of the interaction between the platform and income are shown in
The interaction between the platform and incomea.
Income and platform | Engagement volume | Engagement intensity | |
|
|||
|
Android | 4.55 | 2.86 |
|
iOS | 3.22 | 1.36 |
|
|||
|
Android | 3.97 | 2.41 |
|
iOS | 3.98 | 1.63 |
aBox-Cox transformation was applied to engagement volume and engagement intensity.
Several major findings were obtained in this study. First, the market share analysis indicates that the medical category accounts for the largest market share of eHealth website visits (134,009/184,826, 72.51%), followed by the lifestyle category (46,870/184,826, 25.36%). The e-pharmacy category had the smallest market share, accounting for only 2.14% (3947/184,826) of the total visits.
Second, eHealth websites are characterized by very low visit penetration but relatively high user penetration. This means that although eHealth websites are associated with a low usage frequency, they are closely related to everyone and have great market potential.
Third, the distribution of engagement intensity follows a power law distribution. A large number of eHealth needs involve only a small number of visits, whereas a very small number of complex eHealth needs must be realized through a large number of visits.
Fourth, visits to eHealth websites were highly concentrated. Most users (238/373, 63.8%) visited only one eHealth website within 3 months. On average, each user visits 1.5 eHealth websites. Fewer than 40% of users visit multiple eHealth websites.
Fifth, there are day and hour trends in eHealth website engagement patterns. User engagement is generally high on weekdays but low on weekends. In addition, user engagement increases gradually from morning to noon. After noon, user engagement declines until it reaches its lowest level at midnight.
Sixth, customer loyalty also differed significantly among the categories. Lifestyle websites, followed by medical websites, had the highest customer loyalty. e-Pharmacy websites had the lowest customer loyalty.
Seventh, cross-site browsing among categories was not symmetrical. For example, 66% (31/47, 66%) of lifestyle website users visited medical websites, whereas only 25% (31/124, 25%) of medical website users visited lifestyle websites. The asymmetric nature indicates that popular eHealth websites, such as medical websites, can effectively provide referral traffic for lifestyle and e-pharmacy websites. However, the opposite is not true.
Eighth, Android users are more engaged than iOS users on eHealth websites. This is because users can find more health apps that cost less on the Android platform. Another possible explanation is that Android users are more introverted or more proficient in information technology.
Ninth, app users are much more engaged than browser users. The engagement volume of app users is 4.85 times that of browser users, and the engagement intensity of app users is 4.22 times that of browser users. Such a sizable engagement gap can be explained by the great advantage of apps over browsers.
Tenth, male users had greater engagement intensity than female users. The engagement gap between male and female users can be explained by the fact that male users are more proficient in information technology skills.
Finally, income negatively moderates the influence of the platform (Android vs iOS) on user engagement. The advantage of Android users over iOS users regarding engagement volume and engagement intensity is more salient among low-income users. Low-income Android users are the users most engaged on eHealth websites. Conversely, low-income iOS users are those who are least engaged on eHealth websites.
This study also has some limitations. First, the sample size used in this study was not very large. Only 373 users from Shanghai, China, were included in the data set. More users should be incorporated in future analyses. Second, the income variable used in this study was measured using a proxy. It is measured by the monthly telecom expenditure. Although monthly expenditures should be associated with user income, their relationship is not deterministic. Better approaches, such as surveys, can be used to measure user income in future studies.
In this study, we provide an overview of user engagement behavior on eHealth websites based on cross-site clickstream data. More specifically, we conducted an analysis to determine the market shares of different categories of eHealth websites, penetration of eHealth behavior, engagement intensity, engagement variety, day and hour trends, customer loyalty, and cross-site engagement behavior. Furthermore, we investigated the factors that influence user engagement on eHealth websites. The results indicate that the platform (Android vs iOS), channel (browser vs app), and sex (female vs male) have significant influences on engagement behavior. In addition, income (high vs low) negatively moderates the influence of platforms on engagement behavior.
Future research may focus on how the configuration of eHealth website resources may influence user engagement. Each eHealth website may have some health care resources (eg, health information, e-consultation, provider rating, and web-based registration). According to the resource orchestration theory, the role of one resource is not independent. Instead, its effect depends on the presence of other resources. How the configuration of resources may influence user engagement is an important RQ for the managers of eHealth websites. A configurational approach (eg, fuzzy set qualitative comparative analysis) can be used in the future to investigate the best resource composition pattern for eHealth websites.
research question
This research was supported by the Humanity and Social Science Youth Foundation of the Ministry of Education of China (grant 18YJC630068) and the National Natural Science Foundation of China (grants 71971082 and 71471064).
JL and JY conceived the research and design of the study protocol. KY, XB, and XL performed the study and collected the data. All authors (JL, KY, XB, XL, and JY) contributed to drafting the manuscript.
None declared.