Background: Multimorbidity represents a global health challenge, which requires a more global understanding of multimorbidity patterns and trends. However, the majority of studies completed to date have often relied on self-reported conditions, and a simultaneous assessment of the entire spectrum of chronic disease co-occurrence, especially in developing regions, has not yet been performed.
Objective: We attempted to provide a multidimensional approach to understand the full spectrum of chronic disease co-occurrence among general inpatients in southwest China, in order to investigate multimorbidity patterns and temporal trends, and assess their age and sex differences.
Methods: We conducted a retrospective cohort analysis based on 8.8 million hospital discharge records of about 5.0 million individuals of all ages from 2015 to 2019 in a megacity in southwest China. We examined all chronic diagnoses using the ICD-10 (International Classification of Diseases, 10th revision) codes at 3 digits and focused on chronic diseases with ≥1% prevalence for each of the age and sex strata, which resulted in a total of 149 and 145 chronic diseases in males and females, respectively. We constructed multimorbidity networks in the general population based on sex and age, and used the cosine index to measure the co-occurrence of chronic diseases. Then, we divided the networks into communities and assessed their temporal trends.
Results: The results showed complex interactions among chronic diseases, with more intensive connections among males and inpatients ≥40 years old. A total of 9 chronic diseases were simultaneously classified as central diseases, hubs, and bursts in the multimorbidity networks. Among them, 5 diseases were common to both males and females, including hypertension, chronic ischemic heart disease, cerebral infarction, other cerebrovascular diseases, and atherosclerosis. The earliest leaps (degree leaps ≥6) appeared at a disorder of glycoprotein metabolism that happened at 25-29 years in males, about 15 years earlier than in females. The number of chronic diseases in the community increased over time, but the new entrants did not replace the root of the community.
Conclusions: Our multimorbidity network analysis identified specific differences in the co-occurrence of chronic diagnoses by sex and age, which could help in the design of clinical interventions for inpatient multimorbidity.
With the recent improvements in clinical interventions, advances in public health, lifestyle changes, and environmental exposures, multimorbidity has been a growing global health challenge [- ]. Although multimorbidity is widely considered as the norm, not the exception, it still has an inconsistent definition and heterogeneity in methodology, which makes it difficult to gauge its prevalence and pattern in the general population [ - ]. In light of the increased mortality, lower quality of life, and higher utilization of health care services associated with multimorbidity [ - ], a global understanding of the multimorbidity pattern and trend is needed. Although a variety of studies have investigated the patterns of multimorbidity [ - ], most of them were conducted using cross-sectional surveys, which were generally limited either by their small number of self-reported conditions or by a small sample size. Therefore, a multidimensional approach is still needed to understand the full spectrum of multimorbidity networks, time trends, and patterns in age and sex, particularly in developing countries or regions [ ].
With the enhancement of the storage capacity and accessibility of electronic information systems, digitized clinical record keeping has made routinely collected administrative data of unprecedented depth and variability available to researchers. This provides an opportunity for the application of network analysis to extract conceptual insights from large and messy data sets [- ]. Although notable studies are few and mostly carried out in developed countries, they have provided promising findings in human phenotypic multimorbidity networks. For instance, based on the disease history of more than 30 million patients collected from hospital claims, correlations for more than 10,000 comorbid disease pairs were calculated and visualized in a phenotypic disease network [ ]. Differences in identified multimorbidity across sex and racial groups were identified through macro analyses at the organ level [ , ]. Additionally, a study in Taiwan constructed an epidemiological disease network and examined its temporal pattern [ ]. However, it is difficult to investigate the true extent of multimorbidity associations from these studies because of the differential definition of multimorbidity at the cross-sectional level or over a lifetime period, the difference in the measurement of associations, and the study settings that were mainly dominated by developed countries or regions.
To address these gaps, we performed a retrospective study based on all inpatients living in a megacity in southwest China. We applied the standardized definition and classification system of multimorbidity . Our major aim is to provide a multidimensional approach to understand the complex comorbid relationships among the full spectrum of chronic diseases in general multimorbidity inpatients in southwest China. Furthermore, this study aimed to assess age and sex differences in the multimorbidity pattern and investigate highly correlated communities and their temporal trends.
The workflow of this study is shown in. First, we assessed the quality of the data set and confirmed the study population. Then, the cosine index was selected to construct sex- and age-specific multimorbidity networks. Next, we identified central diseases and bursts, and examined their differences across sex and age. Finally, we divided the networks into communities and assessed their temporal trends. Below, we provide more details on each step of the analysis.
This study was approved by the Ethics Committee of the Health Information Center of Sichuan Province. The data were analyzed anonymously to maintain the privacy of inpatient data. As a study of previously collected administrative data, this work was exempt from informed consent requirements.
Data Source and Study Design
In this retrospective cohort analysis, we used the regional database of longitudinal clinical data for inpatients, which was provided by the Health Information Center of Sichuan Province. This regional database includes the anonymized hospital discharge reports (HDRs) collected from all of the 534 secondary hospitals and 144 tertiary hospitals in Sichuan Province; therefore, each inpatient’s longitudinal clinical data were available. Each HDR contained information on the anonymized identity, age, sex, residential address, visit and discharge dates, principal discharge diagnosis, and up to 15 secondary diagnoses. All diseases were specified according to the ICD-10 (International Classification of Diseases, 10th revision) codes at 3 digits.
The eligibility criteria included inpatients who were residents of Chengdu and alive for the entire study period. A total of 5.2 million individuals (about 31.5% of Chengdu’s population) with 9.2 million HDRs from 2015 to 2019 were included. As we were interested in diseases (ICD-10: A00-R99), hospitalizations in which the patients were marked only for general symptoms  (226,193 cases in total) were removed. According to the sex-specific diagnoses [ , ], 2329 male inpatients and 31 female inpatients were further removed due to conflicts between diagnoses and sex. Finally, the data preprocessing resulted in a total of 8.8 million hospitalizations corresponding to about 5.0 million individuals of all ages, and the sample was large enough to estimate age- and sex-specific multimorbidity patterns.
Chronic Diseases and Multimorbidity Definition
In 2018, the Academy of Medical Sciences recommended the adoption of a uniform definition and reporting system for multimorbidity , which identified multimorbidity as the co-existence of two or more chronic conditions (a physical noncommunicable disease, a mental health condition, or an infectious disease of long duration). Since chronic conditions would not be expected to go away in a single hospitalization period, we considered a 5-year period [ ] rather than a single hospitalization for the definition of multimorbidity. The Chronic Condition Indicator [ ], developed as part of the Healthcare Cost and Utilization Project, was used to differentiate between acute and chronic ICD-10 codes at 3 digits [ ]. A total of 489 and 505 chronic disease codes were separately retained in males and females, respectively.
In order to generate more consistent and reliable estimates, we focused on chronic diseases with ≥1% prevalence for each of the following age strata: <7, 7-14, 15-19, 20-24, 25-29, 30-34, 35-39, 40-44, 45-49, 50-54, 55-59, 60-64, 65-69, 70-74, 75-79, and 80+ years, and for both males and females , resulting in a total of 149 and 145 chronic diseases, respectively, which were further used in downstream analyses ( ).
Multimorbidity Network Generation and Network Properties Calculation
A multimorbidity network developed from inpatients contains a set of nodes that are connected through edges. The node represents a chronic disease (ICD-10 codes at 3 digits), such that the node size is proportional to the disease prevalence and its color identifies the ICD-10 category.
The edge in the multimorbidity network denotes the comorbid strength between co-existence diseases. Typically, the higher the comorbid strength of a disease pair, the lower the probability of co-existence by chance alone [, , ]. The relative risk (RR; calculated in Equation 1) or Pearson correlation coefficient (ϕ, calculated in Equation 2) was often used to quantify the comorbid strengths of disease pairs [ , , ]. These 2 measures are not entirely independent of each other, as they are both affected by the sample size and have intrinsic bias [ ]. As we were interested in tightly interconnected disease pairs, mutually exclusive disease pairs with negative comorbid strengths (RR < 1 or ϕ < 0) were excluded. Since the Salton cosine index (SCI; calculated in Equation 4) is immune to the sample size and only considers the co-occurrences and the prevalence of multimorbidity [ ], we selected it to construct and compare the multimorbidity networks with a widely varied sample size in each of the sex- and age-specific groups.
where nab denotes the number of co-occurrences of diseases a and b, na and nb represent the number of occurrences of diseases a and b, respectively, and Ntotal is the total number of inpatients in the sex/age-specific group.
Generally, a cutoff for the SCI is defined by assessing the relationship between the Pearson correlation and SCI, where the number of significantly correlated diseases is equal in both networks . For each sex- and age-specific stratum, the steps to find a cutoff for the SCI are as follows: Step 1, Calculate the Pearson correlation coefficient (ϕ, calculated in Equation 2) and select the statistically significant correlations at α=.01 (calculated in Equation 3); Step 2, Find the minimum number of disease pairs , where p, the maximum number of edges possible among n nodes detected in step 1, is equal to n(n-1)/2; Step 3, Find the number of pairs (q), where nab ≥ nab_minimum; Step 4, Find the SCI cutoff ( B), where the number of pairs is equal to q, detected in step 3. The above steps were used to create networks for males and females in each of the 16 age groups and then merge the same edges in different age groups into the general networks for males and females.
The Kolmogorov-Smirnov test was applied to investigate whether the degree distribution follows a power law. The structural properties can be measured using several network indices, such as the density, diameter, average path length, degree, weighted degree, closeness centrality, and betweenness centrality . The closeness centrality measures the shortest distance of the disease away from other chronic diseases. Hence, the higher the closeness centrality of a disease, the higher the risk of co-occurrence with different diseases in fewer number of steps. The betweenness centrality denotes the number of shortest paths through a disease. Then, the higher the betweenness centrality of a disease, the higher the likelihood to form bridges between other diseases.
The Central Diseases, Hubs, and Bursts
In order to distinguish the node centrality in the network, the PageRank algorithm  was applied, which considers the edge weights. The higher the PageRank value, the more “central” the disease [ ]. The parameters were set as commonly assumed, where epsilon = 0.001 and probability = 0.85. Since no established guideline exists for how many nodes are central and since the number of nodes hugely differed among all the age groups in our study, we defined the central diseases as the nodes with the top 10 percentile of the PageRank value across the 16 age strata of males and females.
The connectivity of disease a is defined as the sum of all weights of all edges attached to it, which quantifies how strongly a disease is connected to the others. Diseases with larger connectivity are more likely to have a “system-wide” impact on the network. In the study, diseases with the top 10 percentile connectivity values for each of the 16 age strata of males and females have been referred to as the corresponding hubs.
In order to find the nodes with a vastly increased number of edges across age groups (defined as bursts) and to explore the sex difference in the age where the first large leaps occurred, we separately constructed male and female age-based trajectories of degree (k) for each node. The nodes with degree leaps ≥6 in the consecutive age groups with such leaps appearing at least one time in the subsequent stratum were defined as bursts. These bursts play essential roles in increasing the multimorbidity burden. Therefore, detecting the age of the first large leaps can help to understand the progression of multimorbidity.
Community Detection and Temporal Trends of Communities
Community detection separates the nodes of a generic undirected network into communities, such that connections within communities are stronger than those between them [, ]. In order to identify distinct clusters of co-occurring diseases, we applied the Louvain method, a heuristic method based on modularity optimization [ ]. Modularity Q is widely used to compare the partition quality and as an objective function to be optimized [ ]. Furthermore, the community detection algorithm used in our analysis considers the weight of the links. Eigenvector centrality measures the influence of a node in a network [ ]. The node with the largest eigenvector centrality in the community was therefore considered as the community root.
Observing how communities change over time can also provide valuable information about the network . We applied the same methodology year by year and compared the results across time. As a result, we obtained the temporal trends of the multimorbidity networks. The Pearson correlation coefficient was used to measure the correlation for communities received in consecutive years.
All statistical analyses, network constructions, and visualizations were conducted in R software (version 3.5.1; R Development Core Team).
Chronic Diseases and the Prevalence of Multimorbidity
About 5.0 million unique inpatients (representing about 30.3% of the overall Chengdu population) were enrolled in the study, among which 36.8% (a total of 1,843,050 unique inpatients, about 11.2% of the Chengdu population) had two or more chronic diseases (). Demographically, the 1.8 million unique inpatients with multimorbidity consisted of inpatients of all ages with a higher percentage of females (52.1%). Generally, males had a statistically higher percentage of multimorbidity compared with females, except for the age group of 70-79 years (as shown in ). In addition, males in the middle-age (30-34 years and 45-64 years) and older elderly (80+ years) age groups had a large number of chronic diseases compared with females.
Properties of Age- and Sex-Specific Multimorbidity Networks
The phenotypic multimorbidity network analysis identified the network’s global structure and uncovered chronic diseases with a closer co-occurrence (). The RR and Pearson correlation coefficient used to measure disease co-occurrence are not entirely independent of each other ( A). Therefore, the SCI was used to measure the strength of comorbid diseases, and the cutoff of the SCI was determined by assessing the relationship between the Pearson correlation coefficient and SCI ( B). The cumulative distribution of the number of edges by nodes (degree (k) distribution) presented exponential decays ( C). Both the multimorbidity networks of males and females were scale-free as the distribution followed a power law (Kolmogorov-Smirnov test, P=.18 in the male network and P=.09 in the female network). The number of nodes and edges for the multimorbidity networks across age strata and by sex ranged from 22 to 74 and 18 to 579, respectively ( D and 3E). For patients above the age of 30 years, the number of edges we found was more significant in the male multimorbidity network. The number of edges became smaller in the lower age groups, but stronger disease connections were identified ( F). lists the topological properties of each network. Generally, the multimorbidity networks in the younger age groups (≤40 years old) were sparser, except for females <7 years old. The maximum diameter and average path length of the female multimorbidity network were 10 and 4, respectively, which were larger than those of the male network (8 and 2.8, respectively). The average closeness centrality in the multimorbidity network of middle-aged (30-54 years) males was significantly higher than that for females (Wilcoxon test, both P<.05).
|Network in each age group (years)||Density||Diameter||Average path length||Average degree||Avg.w degreea||Avg.clos centralityb||Avg.bet centralityc|
|Male multimorbidity network|
|Female multimorbidity network|
aAvg.w degree: average weighted degree.
bAvg.clos centrality: average closeness centrality.
cAvg.bet centrality: average betweenness centrality.
dThe values in the multimorbidity network in males were statistically higher than those in females (P<.05).
Age- and Sex-Specific Differences in Central Diseases, Hubs, and Bursts
The female and male multimorbidity networks are visualized inA and 4B, respectively. According to the frequency and comorbidity strength, the top 20 comorbid disease pairs involved 13 diseases, among which 11 diseases (E11, E78, I10, I11, I25, I27, I50, I63, I67, I70, and J44) were common to both males and females, and 2 diseases were sex specific (hyperplasia of the prostate [N40] in males and spondylosis [M47] in females). The most comorbid disease pair was essential hypertension (I10) with cerebral infarction (I63), which occurred in males older than 30 years and females older than 40 years. Notably, there existed a few disease pairs that exhibited strong comorbid strengths but only occurred in a typical age group, for instance, congenital malformation co-existence in children <7 years old. Based on the PageRank algorithm, 23 and 26 chronic diseases were identified as central diseases in the female and male multimorbidity networks. Among these diseases, 14 chronic diseases were common to both males and females, and comprised critical diseases across different ages, such as heart failure (I50), essential hypertension (I10), glycoprotein metabolism disorders (E77), and lipoprotein metabolism disorders (E78) ( C). Interestingly, depressive episodes (F32) and other anxiety disorders (F41) represented the central diseases among females aged 7-14 years and 25-29 years, respectively. A total of 26 unique diseases were hubs in the multimorbidity networks, including 19 hubs common to both males and females, 1 female-specific hub (spondylosis at 50-59 years), and 6 male-specific hubs ( D). Furthermore, for each burst, which had at least 2 degree leaps ≥6 through consecutive age groups, connectivity trajectories across age groups are presented in E and 4F. A total of 7 burst nodes were common to both males and females, among which essential hypertension (I10) first occurred in men aged 30 to 34 years. Among the 4 male-specific burst nodes, the earliest leaps were glycoprotein metabolism disorders (E77), which happened at 25-29 years. Remarkably, 9 diseases were classified as not only central diseases, but also hubs and bursts. Among them, 5 were common to both males and females, including essential hypertension (I10), chronic ischemic heart disease (I25), cerebral infarction (I63), other cerebrovascular diseases (I67), and atherosclerosis (I70). Therefore, particular diseases act as both bursts for increasing the network complex and as hubs for having a “system-wide” impact on the network, and some of these act as central diseases for playing the most important role in the network.
Disease progression by age was evaluated by analyzing the connectivity trajectories of each disease (). Males had a higher connectivity, except for the youngest age groups (≤14 years old). In contrast, females had a steeper slope, particularly those aged 55+ years. The central diseases in both males and females showed a higher connectivity compared with noncentral diseases, and the connectivity difference between central diseases and noncentral diseases increased with age among males and females older than 35 years. A similar pattern also appeared in the hubs, but its connectivity difference was more conspicuous at a younger age compared with that in central diseases, which is consistent with the network topology. Subsequently, the bursts in males and females had a higher connectivity after the initial degree leaps ≥6 occurred (males older than 35 years and females older than 40 years).
Temporal Trends of Communities
The community structures showed little variation across time, while the community root tended to be stable throughout time (). The number of chronic diseases in the community increased over time, consistently in both males and females, and the new entrants did not replace the community root. For instance, the number of diseases in the community, among which the disorder of glycoprotein metabolism (E77) or other aplastic anemias (D61) was identified as the root, increased from 8 to 23 in the female community and from 13 to 25 in the male community. In addition, the community common to both males and females was defined as having the same root within the community, where many diseases were common to both sex groups and few diseases were sex specific. For instance, in the community with chronic renal failure (N18) as the root, both the female and male communities included the same diseases, such as secondary hypertension (I15), chronic nephritic syndrome (N03), nephrotic syndrome (N04), and disorders resulting from impaired renal tubular function (N25), while the male community also included vitamin D deficiency (E55) and the female community also included gout (M10) and systemic lupus erythematosus (M32). The clustering of mental health disorders, including depressive episode (F32), other anxiety disorders (F41), and somatoform disorders (F45), differed by sex. For example, the male community included only mental health disorders, while the female community included mental health disorders and various physical diseases. As for sex-specific diseases, the majority of female-specific diseases were in a separate community, with female infertility (N97) or endometriosis (N80) as the root. As for male-specific diseases, hyperplasia of the prostate (N40) was consistent throughout time, and its eigenvector centrality was even higher than that of atherosclerosis, heart failure, or cerebral infarction.
We constructed multimorbidity networks among multimorbidity inpatients (about 11.2% of the Chengdu population) consisting of all ages, which established the connections between chronic diseases in the general hospitalized population from a megacity with 16.5 million residents in southwest China. Multimorbidity affected people of all ages, and their complex interactions were more intensive among males and inpatients ≥40 years old. Notably, in the female multimorbidity network, mental health disorders co-occurred with various mental and physical diseases (eg, metabolic disorders, cardiovascular diseases, and neurodegeneration diseases), among which the co-existence of a depressive episode with other anxiety disorders was detected in 14 age groups of 7-79 years. Moreover, disease connectivity leaps, central diseases, and highly interlinked communities were detected. To the best of our knowledge, this is the first regional study within a developing country that applied regional hospital discharge records rather than self-reported survey data to provide an overview of the prevalence of multimorbidity, obtain multimorbidity patterns, and assess the sex and age differences. Our results demonstrate that the application of network-based algorithms to routinely collected health care data might provide a way to better screen and identify the complex interactions among chronic diseases.
Multimorbidity Affects People of All Ages
Multimorbidity affects people of all ages, even children who are inpatients (≤14 years old), where 5% of these children have at least two chronic diseases. The findings were in accordance with what was reported in previous studies based on the general population because the prevalence differed by age range, sex, ethnicity, socioeconomic status, and lifestyle, cultural, and health-seeking behaviors [, , - ]. Our results showed that 36.8% of the inpatients living in Chengdu during the 2015-2019 time period had at least two chronic diseases, which is lower than that in the Netherlands (multimorbidity prevalence of 57%) [ ], Spain (multimorbidity prevalence of 43.2%) [ ], and Canada (multimorbidity prevalence of 53.3%) [ ], but higher than that in the United Kingdom (multimorbidity prevalence of 19%) [ ], Scotland (multimorbidity prevalence of 31.1%) [ ], Singapore (multimorbidity prevalence of 26.2%) [ ], Italy (multimorbidity prevalence of 15.3%) [ ], and Denmark (multimorbidity prevalence of 21.6%) [ ]. A scoping review found a wide range in multimorbidity prevalence in the general population as reported in studies using a large data set, from 15.3% to 68.4% [ ]. The reported multimorbidity prevalence is still highly varied due to inconsistent measurements of chronic conditions and multimorbidity [ , ]. Additionally, a study using claims data in Beijing reported that the prevalence of multimorbidity was 51.6% and 81.3% for middle-aged adults (45-59 years) and older adults (≥60 years), respectively [ ], which were higher than the respective prevalences of 41.7% and 75.2% in our study. One explanation might be related to the differential study design, since the study in Beijing used both outpatient and inpatient clinical diagnoses for the measurement of multimorbidity. However, the study in Beijing only used 13 most frequently mentioned diseases to measure multimorbidity and the study population was restricted to people who had been employed, which would limit its generalizability to a general population. Therefore, estimating the prevalence of multimorbidity based on a regional database is essential for the design of health care strategies. To the best of our knowledge, this is the first regional study within a developing country to provide an overview of the prevalence of multimorbidity using regional hospital discharge records rather than self-reported survey data. The prevalence of multimorbidity increased with age, which is in line with the findings of previous epidemiological studies that the prevalence of multimorbidity may be increasing, at least in part, because of population aging [ , , , ].
Age and Sex Differences in Multimorbidity Patterns
We identified the multimorbidity patterns in age- and sex-specific inpatient groups, which were comparable with previous studies in developed countries or regions [, , , ]. For instance, Ioakeim-Skoufa et al [ ] found associations of respiratory disorders with circulatory diseases, and depression and anxiety with chronic musculoskeletal diseases. In our study, we identified the most frequent and strongest comorbid disease pairs, such as the associations of circulatory diseases with endocrine diseases, diseases of the musculoskeletal system, and respiratory disorders, which co-occurred more frequently than expected by random chance. Our data set consisted of hospitalizations of all ages, and thus contained information about the diseases that are common and specific among age and sex groups. For example, we identified some disease pairs that occurred throughout life (eg, heart failure co-occurring with complications of heart disease and lipoprotein metabolism disorders co-occurring with diabetes mellitus) and some pairs with a stronger comorbid strength but only occurrence among a typical age group (eg, congenital malformation co-existence in children <7 years old). Intuitively, chronic diseases would be expected to co-occur in an individual if their resilience or vulnerability was altered or if they shared a common pattern of influence [ - ]. Thus, as in previous studies assessing the disease trajectory in patients with depression [ ] and type 2 diabetes [ ], and the general population [ ], regional databases collecting HDRs spanning a sufficient time period (generally 10+ years as in the above mentioned studies) will support further studies to explore the potential causal directions among complex correlations. In accordance with previous studies [ , ], we identified a difference in the associations of mental disorders with physical diseases among different sex groups, and generally stronger associations were found in females than in males. This sex difference within mental health multimorbidity may be related to differences in the patient care–seeking behaviors between males and females, as indicated in a previous study that reported on how social factors can discourage males from seeking mental health care [ ]. Our findings support the development of interdisciplinary and multidisciplinary treatment strategies for patients with depression or anxiety [ , , ], since they frequently had physical diseases, such as metabolic disorders, Alzheimer disease, epilepsy, hypertension, chronic ischemic heart disease, heart failure, cerebral infraction, atherosclerosis, gastroesophageal reflux disease, decubitus ulcer, and spondylosis. The data-driven discovery of diseases co-occurring might be useful to generate potential hypotheses for coexisting diseases (eg, sharing the same gene, having common risk factors, and displaying a consistent temporal progression trend [ , ]) and their differences in age and sex (eg, physical, hormonal, and even genetic differences by sex [ , ] and disease progression with age [ ]). Additionally, the data-driven discovery of diseases co-occurring, especially based on a complete population with a high-quality health care database, might have impacts on disease management [ ].
We found lifelong comorbid disease pairs among multimorbidity inpatients, for example, lipoprotein metabolism disorders (E78) co-occurring with diabetes mellitus (E11) in both males and females ≥15 years old, heart failure (I50) co-occurring with complications of heart disease (I51) in females ≥15 years old, and a depressive episode (F32) co-occurring with other anxiety disorders (F41) in females aged 7-79 years. Congenital malformations, generally the earliest diagnosed diseases in life (ie, pre- or perinatally), had a higher prevalence in multimorbid girls (<7 years) than in boys (<7 years) in our study, especially congenital malformations of the circulatory system (including congenital malformations of the cardiac septa, Q21; congenital malformations of the pulmonary and tricuspid valves, Q22; congenital malformations of the aortic and mitral valves, Q23; other congenital malformations of the heart, Q24; and congenital malformations of the great arteries, Q25). The prevalence of congenital heart disease in girls was higher than in boys , and treatment or progression of congenital heart disease could cause complications of heart disease and heart failure [ , ], which may support the finding in our study that heart failure co-occurring with complications of heart disease was earlier in females than in males. Chronic diseases with the earliest connectivity leaps begin at 25-29 years in males, about 15 years earlier than in females. These findings indicate the need to appropriately handle multimorbidity among youth or middle-aged patients [ , ]. Attention and activities are required to prevent such people from entering the multimorbidity category, especially for lipoprotein metabolism disorders, diabetes mellitus, hypertension, heart failure, spondylosis, chronic renal failure, fibrosis and cirrhosis of the liver, and gout. In addition, appropriate guidelines and flexible care management support systems are required across a broader age range.
Within each multimorbidity network, we identified the central disease, which played the most important role in the network (eg, having a large number of comorbid diseases and thus increasing the scale of network, having a relatively smaller number of comorbid diseases but exhibiting stronger comorbid strengths, and playing the role of connectivity to connect those unconnected diseases). In the biomedical scenario, central diseases may be interpreted as those that are more likely to appear in patients with multimorbidity or lead to multimorbidity. Therefore, common causal genes and molecular processes or signaling pathways may be shared among central diseases and their neighbors [, ]. We found that circulatory diseases and metabolic diseases were the most important diseases among almost all age groups; thus, clinical studies of the identified central diseases may be helpful to improve prevention strategies and health care policies [ ]. Notably, younger females should receive more attention on 2 mental health disorders, depressive episodes and other anxiety disorders, which significantly increased the scale of their multimorbidity network. The observations of central diseases could have important implications on the design of health care prevention, such that measurements targeting a specific factor may benefit many related diseases.
Furthermore, we observed that some communities remained stable across time, while others became more extensive due to the occurrence of more diseases. Few studies have observed the temporal trends of networks or communities [, ]. Jiang et al found that the network structure, connectivity, and module structure varied across time [ ]. The work of van Oostrom et al [ ] showed that the prevalence of chronic diseases in the general practice registration over the period between 2004 and 2011 increased from 34.9% to 41.8%, and this increase could be only partially explained by the aging of the population. In our study, the community of mental health disorders in the male multimorbidity network consisted of depressive episodes, other anxiety disorders, and somatoform disorders, which seemed independent with physical diseases, while the community involving females additionally included various physical diseases, such as disorders of lipoprotein metabolism and other lipidemias (E78), essential hypertension (I10), hypertensive heart disease (I11), cerebral infarction (I63), atherosclerosis (I70), gastroesophageal reflux disease (K21), and spondylosis (M47). The clustering difference in mental health disorders according to sex might be related with the higher underdiagnosis rate of mental health disorders in males [ ] and provide evidence for differential strategies in diagnosis and treatment for males and females. For instance, when 2 diseases are discordant in terms of their pathogenesis (eg, depression co-occurring with cerebrovascular disease in females), they may require separate time-intensive treatment plans [ , - ]. Additionally, within the community, both concordant and discordant diseases in terms of their pathogenesis were included (eg, a female community with atherosclerosis as the root and including diabetes mellitus, hypertension, cerebrovascular diseases, mental health disorders, and spondylosis), which might lead to very different management needs and treatment strategies [ , - ]. Communities can describe the interconnections among chronic diseases, with more tight connections between those in the same community. In a further study, it will be of interest to examine the direction of these interconnections or to explore their common risk factors for priority management.
Strengths and Limitations
The main strengths of this study can be summarized as follows. First, this is the first regional study in developing countries based on a large-scale data set (8.8 million hospital discharge records) to examine multimorbidity patterns and trends, and their differences across age and sex. Additionally, a network-based approach is applied to extract conceptual insights from routinely collected hospital discharge records. The use of this method can be extended to other health care data sets. Lastly, the use of routinely collected administrative data at a regional level is advantageous because the data are uniformity distributed and unbiased, which provides an opportunity to identify the co-occurrence of rare clinical diseases.
This study has some limitations. First, the main limitation of this study is the unavailability of individual-level socioeconomic status, lifestyle, and clinical variables. These factors would play essential roles in understanding the differences among multimorbidity patterns . This limitation is common among studies that use routinely collected health care data sets. Second, the data set that was used did not contain information on the outpatients who were seeking solely outpatient care. Hence, it is vital to interpret our findings in the context of the inpatient population in a developing country. Third, we excluded individuals who were not alive during the study period to obtain a more homogenous study population, which may underestimate the diseases that have high mortality rates. However, it was demonstrated by a previous study that this exclusion criterion did not drastically impact the results [ ].
In this paper, we performed a network-based analysis of 8.8 million hospital discharge records and identified age and sex differences in multimorbidity patterns and the evolution of multimorbidity over time. This longitudinal study provides the first evidence from a developing country that multimorbidity affects people of all ages and their complex interactions are more intensive among males and inpatients ≥40 years old. Mental health disorders were comorbid with more various mental and physical diseases in females than in males. The lifelong comorbid disease pairs, disease connectivity leaps, central diseases, highly interlinked communities, and age- and sex-specific comorbidity patterns detected in the study might provide suggestions for enhancing integrated management in multimorbidity patients. Meanwhile, the network-based approach applied in our study could investigate all the multimorbidity connections at the population level, which could be used within health care data sets in other settings.
This work was supported by the Key Research and Development Project of Sichuan Province, China (grants 2018SZ0114 and 2019YFS0271).
Conflicts of Interest
Diseases with prevalence ≥1% in males and females.DOCX File , 33 KB
Multimorbidity prevalence in each age strata and by sex.DOCX File , 17 KB
- Forman DE, Maurer MS, Boyd C, Brindis R, Salive ME, Horne FM, et al. Multimorbidity in Older Adults With Cardiovascular Disease. J Am Coll Cardiol 2018 May 15;71(19):2149-2161 [FREE Full text] [CrossRef] [Medline]
- Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. The Lancet 2012 Jul 7;380(9836):37-43 [FREE Full text] [CrossRef] [Medline]
- Calderón-Larrañaga A, Vetrano DL, Ferrucci L, Mercer SW, Marengoni A, Onder G, et al. Multimorbidity and functional impairment-bidirectional interplay, synergistic effects and common pathways. J Intern Med 2019 Mar;285(3):255-271 [FREE Full text] [CrossRef] [Medline]
- Xu X, Mishra GD, Jones M. Evidence on multimorbidity from definition to intervention: An overview of systematic reviews. Ageing Res Rev 2017 Aug;37:53-68. [CrossRef] [Medline]
- Griffith LE, Gilsing A, Mangin D, Patterson C, van den Heuvel E, Sohel N, et al. Multimorbidity Frameworks Impact Prevalence and Relationships with Patient-Important Outcomes. J Am Geriatr Soc 2019 Aug;67(8):1632-1640. [CrossRef] [Medline]
- Nicholson K, Terry AL, Fortin M, Williamson T, Bauer M, Thind A. Prevalence, characteristics, and patterns of patients with multimorbidity in primary care: a retrospective cohort analysis in Canada. Br J Gen Pract 2019 Sep;69(686):e647-e656 [FREE Full text] [CrossRef] [Medline]
- Multimorbidity: Understanding the challenge. Richmond Group of Charities. 2018. URL: https://richmondgroupofcharities.org.uk/sites/default/files/multimorbidity_-_understanding_the_challenge.pdf [accessed 2020-10-12]
- Cassell A, Edwards D, Harshfield A, Rhodes K, Brimicombe J, Payne R, et al. The epidemiology of multimorbidity in primary care: a retrospective cohort study. Br J Gen Pract 2018 Apr;68(669):e245-e251 [FREE Full text] [CrossRef] [Medline]
- Payne RA, Abel GA, Guthrie B, Mercer SW. The effect of physical multimorbidity, mental health conditions and socioeconomic deprivation on unplanned admissions to hospital: a retrospective cohort study. CMAJ 2013 Mar 19;185(5):E221-E228 [FREE Full text] [CrossRef] [Medline]
- Payne RA, Mendonca SC, Elliott MN, Saunders CL, Edwards DA, Marshall M, et al. Development and validation of the Cambridge Multimorbidity Score. CMAJ 2020 Feb 03;192(5):E107-E114 [FREE Full text] [CrossRef] [Medline]
- Randall DA, Lujic S, Havard A, Eades SJ, Jorm L. Multimorbidity among Aboriginal people in New South Wales contributes significantly to their higher mortality. Med J Aust 2018 Jun 02;209(1):19-23. [CrossRef] [Medline]
- Khorrami Z, Rezapour M, Etemad K, Yarahmadi S, Khodakarim S, Mahdavi Hezaveh A, et al. The patterns of Non-communicable disease Multimorbidity in Iran: A Multilevel Analysis. Sci Rep 2020 Feb 20;10(1):3034 [FREE Full text] [CrossRef] [Medline]
- Yao S, Cao G, Han L, Chen Z, Huang Z, Gong P, et al. Prevalence and Patterns of Multimorbidity in a Nationally Representative Sample of Older Chinese: Results From the China Health and Retirement Longitudinal Study. J Gerontol A Biol Sci Med Sci 2020 Sep 25;75(10):1974-1980. [CrossRef] [Medline]
- Zemedikun DT, Gray LJ, Khunti K, Davies MJ, Dhalwani NN. Patterns of Multimorbidity in Middle-Aged and Older Adults: An Analysis of the UK Biobank Data. Mayo Clin Proc 2018 Jul;93(7):857-866. [CrossRef] [Medline]
- Roso-Llorach A, Violán C, Foguet-Boreu Q, Rodriguez-Blanco T, Pons-Vigués M, Pujol-Ribera E, et al. Comparative analysis of methods for identifying multimorbidity patterns: a study of 'real-world' data. BMJ Open 2018 Mar 22;8(3):e018986 [FREE Full text] [CrossRef] [Medline]
- Ioakeim-Skoufa I, Poblador-Plou B, Carmona-Pírez J, Díez-Manglano J, Navickas R, Gimeno-Feliu LA, et al. Multimorbidity Patterns in the General Population: Results from the EpiChron Cohort Study. Int J Environ Res Public Health 2020 Jun 14;17(12):4242 [FREE Full text] [CrossRef] [Medline]
- Fabbri E, Zoli M, Gonzalez-Freire M, Salive ME, Studenski SA, Ferrucci L. Aging and Multimorbidity: New Tasks, Priorities, and Frontiers for Integrated Gerontological and Clinical Research. J Am Med Dir Assoc 2015 Aug 01;16(8):640-647 [FREE Full text] [CrossRef] [Medline]
- Brunson JC, Laubenbacher RC. Applications of network analysis to routinely collected health care data: a systematic review. J Am Med Inform Assoc 2018 Feb 01;25(2):210-221 [FREE Full text] [CrossRef] [Medline]
- Hidalgo CA, Blumm N, Barabási AL, Christakis NA. A dynamic network approach for the study of human phenotypes. PLoS Comput Biol 2009 Apr;5(4):e1000353 [FREE Full text] [CrossRef] [Medline]
- Amell A, Roso-Llorach A, Palomero L, Cuadras D, Galván-Femenía I, Serra-Musach J, et al. Disease networks identify specific conditions and pleiotropy influencing multimorbidity in the general population. Sci Rep 2018 Oct 29;8(1):15970 [FREE Full text] [CrossRef] [Medline]
- Kalgotra P, Sharda R, Croff JM. Examining health disparities by gender: A multimorbidity network analysis of electronic medical record. Int J Med Inform 2017 Dec;108:22-28. [CrossRef] [Medline]
- Kalgotra P, Sharda R, Croff JM. Examining multimorbidity differences across racial groups: a network analysis of electronic medical records. Sci Rep 2020 Aug 11;10(1):13538 [FREE Full text] [CrossRef] [Medline]
- Jiang Y, Ma S, Shia B, Lee T. An Epidemiological Human Disease Network Derived from Disease Co-occurrence in Taiwan. Sci Rep 2018 Mar 14;8(1):4557 [FREE Full text] [CrossRef] [Medline]
- Westergaard D, Moseley P, Sørup FKH, Baldi P, Brunak S. Population-wide analysis of differences in disease progression patterns in men and women. Nat Commun 2019 Feb 08;10(1):666 [FREE Full text] [CrossRef] [Medline]
- Multimorbidity: a priority for global health research. The Academy of Medical Sciences. 2018. URL: https://acmedsci.ac.uk/file-download/99630838 [accessed 2020-10-12]
- Chronic Condition Indicators for ICD-10-CM (beta version). Healthcare Cost and Utilization Project (HCUP). URL: https://www.hcup-us.ahrq.gov/toolssoftware/chronic_icd10/chronic_icd10.jsp [accessed 2020-10-12]
- Menche J, Sharma A, Kitsak M, Ghiassian SD, Vidal M, Loscalzo J, et al. Disease networks. Uncovering disease-disease relationships through the incomplete interactome. Science 2015 Feb 20;347(6224):1257601 [FREE Full text] [CrossRef] [Medline]
- Ahlgren P, Jarneving B, Rousseau R. Requirements for a cocitation similarity measure, with special reference to Pearson's correlation coefficient. J. Am. Soc. Inf. Sci 2003 Apr;54(6):550-560. [CrossRef]
- Egghe L, Leydesdorff L. The relation between Pearson's correlation coefficient r and Salton's cosine measure. J. Am. Soc. Inf. Sci 2009 May;60(5):1027-1036. [CrossRef]
- Freeman LC. Centrality in social networks conceptual clarification. Social Networks 1978 Jan;1(3):215-239. [CrossRef]
- Brin S, Page L. The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems 1998 Apr;30(1-7):107-117. [CrossRef]
- Martin T, Zhang X, Newman MEJ. Localization and centrality in networks. Phys Rev E Stat Nonlin Soft Matter Phys 2014 Nov;90(5-1):052808. [CrossRef] [Medline]
- Verma A, Bang L, Miller JE, Zhang Y, Lee MTM, Zhang Y, DiscovEHR Collaboration. Human-Disease Phenotype Map Derived from PheWAS across 38,682 Individuals. Am J Hum Genet 2019 Jan 03;104(1):55-64 [FREE Full text] [CrossRef] [Medline]
- Fortunato S, Barthélemy M. Resolution limit in community detection. Proc Natl Acad Sci U S A 2007 Jan 02;104(1):36-41 [FREE Full text] [CrossRef] [Medline]
- Blondel VD, Guillaume J, Lambiotte R, Lefebvre E. Fast unfolding of communities in large networks. J. Stat. Mech 2008 Oct 09;2008(10):P10008. [CrossRef]
- Hongsong C. Networks, Crowds, and Markets: Reasoning about a Highly Connected World (Easley, D. and Kleinberg, J.; 2010) [Book Review]. IEEE Technol. Soc. Mag 2013;32(3):10-30. [CrossRef]
- Bähler C, Huber CA, Brüngger B, Reich O. Multimorbidity, health care utilization and costs in an elderly community-dwelling population: a claims data based observational study. BMC Health Serv Res 2015 Jan 22;15:23 [FREE Full text] [CrossRef] [Medline]
- Kostev K, Jacob L. Multimorbidity and polypharmacy among elderly people followed in general practices in Germany. Eur J Intern Med 2018 Sep;55:66-68. [CrossRef] [Medline]
- Wong A, Boshuizen HC, Schellevis FG, Kommer GJ, Polder JJ. Longitudinal administrative data can be used to examine multimorbidity, provided false discoveries are controlled for. J Clin Epidemiol 2011 Oct;64(10):1109-1117. [CrossRef] [Medline]
- Violán C, Foguet-Boreu Q, Hermosilla-Pérez E, Valderas JM, Bolíbar B, Fàbregas-Escurriola M, et al. Comparison of the information provided by electronic health records data and a population health survey to estimate prevalence of selected health conditions and multimorbidity. BMC Public Health 2013 Mar 21;13:251 [FREE Full text] [CrossRef] [Medline]
- McLean G, Gunn J, Wyke S, Guthrie B, Watt GCM, Blane DN, et al. The influence of socioeconomic deprivation on multimorbidity at different ages: a cross-sectional study. Br J Gen Pract 2014 Jul;64(624):e440-e447 [FREE Full text] [CrossRef] [Medline]
- Low LL, Kwan YH, Ko MSM, Yeam CT, Lee VSY, Tan WB, et al. Epidemiologic Characteristics of Multimorbidity and Sociodemographic Factors Associated With Multimorbidity in a Rapidly Aging Asian Country. JAMA Netw Open 2019 Nov 01;2(11):e1915245 [FREE Full text] [CrossRef] [Medline]
- Lenzi J, Avaldi VM, Rucci P, Pieri G, Fantini MP. Burden of multimorbidity in relation to age, gender and immigrant status: a cross-sectional study based on administrative data. BMJ Open 2016 Dec 21;6(12):e012812 [FREE Full text] [CrossRef] [Medline]
- Schiøtz ML, Stockmarr A, Høst D, Glümer C, Frølich A. Social disparities in the prevalence of multimorbidity - A register-based population study. BMC Public Health 2017 May 10;17(1):422 [FREE Full text] [CrossRef] [Medline]
- Chua YP, Xie Y, Lee PSS, Lee ES. Definitions and Prevalence of Multimorbidity in Large Database Studies: A Scoping Review. Int J Environ Res Public Health 2021 Feb 09;18(4):1673 [FREE Full text] [CrossRef] [Medline]
- Nicholson K, Almirall J, Fortin M. The measurement of multimorbidity. Health Psychol 2019 Sep;38(9):783-790. [CrossRef] [Medline]
- Wang X, Yao S, Wang M, Cao G, Chen Z, Huang Z, et al. Multimorbidity among Two Million Adults in China. Int J Environ Res Public Health 2020 May 13;17(10):3395 [FREE Full text] [CrossRef] [Medline]
- Bhavnani SK, Dang B, Penton R, Visweswaran S, Bassler KE, Chen T, et al. How High-Risk Comorbidities Co-Occur in Readmitted Patients With Hip Fracture: Big Data Visual Analytical Approach. JMIR Med Inform 2020 Oct 26;8(10):e13567 [FREE Full text] [CrossRef] [Medline]
- George M, Smith A, Sabesan S, Ranmuthugala G. Physical Comorbidities and Their Relationship with Cancer Treatment and Its Outcomes in Older Adult Populations: Systematic Review. JMIR Cancer 2021 Oct 13;7(4):e26425 [FREE Full text] [CrossRef] [Medline]
- Valderas JM, Starfield B, Sibbald B, Salisbury C, Roland M. Defining comorbidity: implications for understanding health and health services. Ann Fam Med 2009;7(4):357-363 [FREE Full text] [CrossRef] [Medline]
- Han X, Hou C, Yang H, Chen W, Ying Z, Hu Y, et al. Disease trajectories and mortality among individuals diagnosed with depression: a community-based cohort study in UK Biobank. Mol Psychiatry 2021 Nov;26(11):6736-6746 [FREE Full text] [CrossRef] [Medline]
- Khan A, Uddin S, Srinivasan U. Comorbidity network for chronic disease: A novel approach to understand type 2 diabetes progression. Int J Med Inform 2018 Jul;115:1-9. [CrossRef] [Medline]
- Steffen A, Nübel J, Jacobi F, Bätzing J, Holstiege J. Mental and somatic comorbidity of depression: a comprehensive cross-sectional analysis of 202 diagnosis groups using German nationwide ambulatory claims data. BMC Psychiatry 2020 Mar 30;20(1):142 [FREE Full text] [CrossRef] [Medline]
- Corrigan P. How stigma interferes with mental health care. Am Psychol 2004 Oct;59(7):614-625. [CrossRef] [Medline]
- Ivbijaro GO, Enum Y, Khan AA, Lam SS, Gabzdyl A. Collaborative care: models for treatment of patients with complex medical-psychiatric conditions. Curr Psychiatry Rep 2014 Nov;16(11):506 [FREE Full text] [CrossRef] [Medline]
- Smith SM, Wallace E, O'Dowd T, Fortin M. Interventions for improving outcomes in patients with multimorbidity in primary care and community settings. Cochrane Database Syst Rev 2021 Jan 15;1:CD006560 [FREE Full text] [CrossRef] [Medline]
- Jensen AB, Moseley PL, Oprea TI, Ellesøe SG, Eriksson R, Schmock H, et al. Temporal disease trajectories condensed from population-wide registry data covering 6.2 million patients. Nat Commun 2014 Jun 24;5:4022 [FREE Full text] [CrossRef] [Medline]
- Haug N, Deischinger C, Gyimesi M, Kautzky-Willer A, Thurner S, Klimek P. High-risk multimorbidity patterns on the road to cardiovascular mortality. BMC Med 2020 Mar 10;18(1):44 [FREE Full text] [CrossRef] [Medline]
- Regitz-Zagrosek V, Kararigas G. Mechanistic Pathways of Sex Differences in Cardiovascular Disease. Physiol Rev 2017 Jan;97(1):1-37 [FREE Full text] [CrossRef] [Medline]
- Gerdts E, Regitz-Zagrosek V. Sex differences in cardiometabolic disorders. Nat Med 2019 Nov;25(11):1657-1666. [CrossRef] [Medline]
- Chun H, Yue Y, Wang Y, Dawa Z, Zhen P, La Q, et al. High prevalence of congenital heart disease at high altitudes in Tibet. Eur J Prev Cardiol 2019 May;26(7):756-759. [CrossRef] [Medline]
- Hinton RB, Ware SM. Heart Failure in Pediatric Patients With Congenital Heart Disease. Circ Res 2017 Mar 17;120(6):978-994. [CrossRef]
- Garcia AM, Beatty J, Nakano SJ. Heart failure in single right ventricle congenital heart disease: physiological and molecular considerations. Am J Physiol Heart Circ Physiol 2020 Apr 01;318(4):H947-H965 [FREE Full text] [CrossRef] [Medline]
- Taylor AW, Price K, Gill TK, Adams R, Pilkington R, Carrangis N, et al. Multimorbidity - not just an older person's issue. Results from an Australian biomedical study. BMC Public Health 2010 Nov 22;10:718 [FREE Full text] [CrossRef] [Medline]
- De Las Rivas J, Fontanillo C. Protein-protein interaction networks: unraveling the wiring of molecular machines within the cell. Brief Funct Genomics 2012 Nov;11(6):489-496. [CrossRef] [Medline]
- van Oostrom SH, Gijsen R, Stirbu I, Korevaar JC, Schellevis FG, Picavet HSJ, et al. Time Trends in Prevalence of Chronic Diseases and Multimorbidity Not Only due to Aging: Data from General Practices and Health Surveys. PLoS One 2016;11(8):e0160264 [FREE Full text] [CrossRef] [Medline]
- Yu W, Singh SS, Calhoun S, Zhang H, Zhao X, Yang F. Generalized anxiety disorder in urban China: Prevalence, awareness, and disease burden. J Affect Disord 2018 Jul;234:89-96. [CrossRef] [Medline]
- Zulman DM, Asch SM, Martins SB, Kerr EA, Hoffman BB, Goldstein MK. Quality of care for patients with multiple chronic conditions: the role of comorbidity interrelatedness. J Gen Intern Med 2014 Mar;29(3):529-537 [FREE Full text] [CrossRef] [Medline]
- Piette JD, Kerr EA. The impact of comorbid chronic conditions on diabetes care. Diabetes Care 2006 Mar;29(3):725-731. [CrossRef] [Medline]
- Mercer SW, Gunn J, Bower P, Wyke S, Guthrie B. Managing patients with mental and physical multimorbidity. BMJ 2012 Sep 03;345:e5559. [CrossRef] [Medline]
- Soley-Bori M, Bisquera A, Ashworth M, Wang Y, Durbaba S, Dodhia H, et al. Identifying multimorbidity clusters with the highest primary care use: 15 years of evidence from a multi-ethnic metropolitan population. Br J Gen Pract 2021 Oct 20:BJGP.2021.0325 [FREE Full text] [CrossRef] [Medline]
|HDR: hospital discharge report|
|ICD-10: International Classification of Diseases, 10th revision|
|RR: relative risk|
|SCI: Salton cosine index|
Edited by T Leung; submitted 13.01.21; peer-reviewed by K Nicholson, Y Zhu; comments to author 06.04.21; revised version received 06.05.21; accepted 12.01.22; published 25.02.22Copyright
©Liya Wang, Hang Qiu, Li Luo, Li Zhou. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 25.02.2022.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.