The overall design of this project is illustrated in Figure 1 to provide context for this paper. The work reported here focuses on public perspectives, the methods used, and the results. The first three steps in Figure 1 have been completed and published [9,18]. The section on the far-right side of the figure, regarding quantitative validation, illustrates directions we plan to explore in future studies.

An online survey (Multimedia Appendix 1) was administered through Ohio University in the summer of 2017, through ResearchMatch [19] in the summer of 2018, and through Clemson University in the summer of 2020, providing three samples. The inclusion criterion for participation in the survey was being 18 years of age or older. The participants were allowed to share the survey’s URL link, and all respondents acknowledged informed consent.

The survey included seven demographic questions and rating items related to the importance of the 29 health indicators. Definitions of these health indicators were provided within the survey (Multimedia Appendix 2). In the survey, the 29 health indicators were separated into five subscales [1,2,17]:

After removing invalid data, the final sample yielded 362 responses from Ohio University, 694 from ResearchMatch, and 97 from Clemson University (Multimedia Appendix 3). Survey items used by Ohio University and Clemson University were rated on a scale of 0 to 10, where 0 refers to “not at all important” and 10 refers to “extremely important.” Survey items in the ResearchMatch sample were measured using a scale of 0 to 100, where 0 refers to “not at all important” and 100 refers to “extremely important.” Therefore, as part of the data cleaning process, the data from ResearchMatch were converted to a scale of 0 to 10 (Multimedia Appendix 4 contains the codebook). In the Ohio University survey, there were five health indicators—blood sugar, blood triglycerides, HDL, LDL, and total cholesterol—for which a scale of 0 to 11, instead of 0 to 10, was used. Due to this error, data for these five indicators were removed from the Ohio University data set. As a result, the total sample size of these five indicators was 791 respondents, whereas the total sample size of the other indicators was 1153 respondents.

The internal reliability of the survey instruments was calculated for the overall set and the three institutional subsets using Cronbach α [20].

Data analyses included rating the 29 health indicators based on their perceived importance. After aggregating data from the three samples with descriptive statistics, a three-step analysis was conducted. The first step of the analysis involved determining whether the three samples had unanimous agreement on the importance of each indicator. A one-way analysis of variance (ANOVA) with a post hoc test was conducted using SPSS software (version 27; IBM Corp) for each indicator to examine any group mean difference, where the independent variables were the three samples and the dependent variables were the individual survey responses. A Levene test was used to test the homogeneity of variance for each indicator before running an ANOVA. The indicators with no group mean differences across samples were retained for the following analysis step.

The second step of the analysis examined the mean differences between the retained indicators via a one-way ANOVA, where the independent variables were the identified indicators and the dependent variables were the individual survey responses. Any indicators with no significant mean differences were grouped together because they could not be ranked.

The third step of the analysis ranked the groups of indicators into various levels after grouping the indicators with no mean differences. A one-way ANOVA with a post hoc test was conducted to examine the mean differences between the levels of indicators, where the independent variables were the levels of indicators and the dependent variables were the individual survey responses. Any significant mean difference between any two levels of indicators indicated the ranking order of the two levels.

This study was approved by the Institutional Review Boards of Ohio University (17-X-142) and Clemson University (IRB2019-441).

The primary purpose of this study was to identify the public perspectives on the importance of the 29 selected health indicators, whether they agreed with each other, and, if so, what were the importance rankings of the health indicators that agreed with each other. Figure 2 summarizes the analytic strategies and the main results of each step. The following paragraphs elaborate on the detailed results for each step.

Descriptive statistics for the 29 health indicators are reported in Table 1. The descriptive statistics for the demographic information for all respondents are reported in Multimedia Appendix 5. Descriptive analyses show that drug and substance abuse, smoking and tobacco use, alcohol abuse, major depression, and diet and nutrition were found to be the five most important health indicators, as rated by the study participants. Additionally, race and ethnicity, possession of a high school diploma, engagement in life, unemployment status, and sun protection were the five least important health indicators. Self-rated health status, the most-used health indicator to assess an individual’s health status, was ranked in the 20th position.

A Levene test was conducted to test the homogeneity of variance for each indicator before running an ANOVA. This resulted in nine health indicators with homogenous variance: blood sugar level, HDL cholesterol, LDL cholesterol, total cholesterol, immunization and vaccination, insurance coverage, cancer screening detection, air quality index greater than 100, and self-rated health status (Multimedia Appendix 6). A total of 20 health indicators were found to have heterogeneous variance. These included the following indicators: blood triglycerides, alcohol abuse, BMI, diet and nutrition, drug or substance abuse, family history of cancer, physical inactivity, smoking and tobacco use, sun protection, personal care needs, hypertension screening, HIV testing, high school diploma as a health indicator, supply of dentists, engagement in life, health literacy rate, major depression, having a sense of purpose in one’s life, race and ethnicity, and unemployment (Multimedia Appendix 7).

For the nine indicators with homogeneous variance, a one-way ANOVA was used. A one-way ANOVA Welch test was used for the 20 indicators with heterogeneous variance. As a result, 13 indicators were found to have no statistically significant mean differences among the three samples. This indicates that survey participants generally agreed on the relative level of importance of these indicators (Table 2). The 13 indicators were blood sugar level, blood triglycerides, HDL cholesterol, LDL cholesterol, total cholesterol, personal care needs, HIV testing, self-rated health status, supply of dentists, health literacy rate, immunization and vaccination, cancer screening detection, and air quality index greater than 100. The means and SDs of their ratings are presented in Table 2. These 13 indicators were retained for the second step of the analysis. Significant mean differences were found among the other 16 indicators, which indicates that survey participants disagreed on their level of importance (Multimedia Appendix 8 contains the post hoc results).

In the second step of the analysis, a one-way ANOVA was run for the 13 retained indicators, where the independent variables were the 13 indicators and the dependent variables were the individual survey responses. The indicators with no mean differences were grouped into the same level (Table 3) because they were rated as equally important and could not be ranked within a level. As a result, seven levels were formed (Table 3). Level 1 to level 7 rankings were organized based on the mean importance of the health indicators from high to low within and between levels. Level 1 included blood sugar level and immunization and vaccination; level 2 included LDL cholesterol; level 3 included HDL cholesterol, blood triglycerides, cancer screening detection, and total cholesterol; level 4 included health literacy rate; level 5 included personal care needs and air quality index greater than 100; level 6 included self-rated health status and HIV testing; and level 7 included the supply of dentists.

In the third step of the analysis, a one-way ANOVA was used to rank the seven levels of indicators, where the independent variables were the seven levels and the dependent variables were the individual survey responses. There were seven indicators in levels 1 to 3: blood sugar level, immunization and vaccination, LDL cholesterol, HDL cholesterol, blood triglycerides, cancer screening detection, and total cholesterol. These indicators were found to be significantly more important to the survey participants than the six indicators ranked in levels 4 to 7: health literacy rate, personal care needs, air quality index greater than 100, self-rated health status, HIV testing, and supply of dentists (Table 4).

Among the more important indicators, the two indicators in level 1 (ie, blood sugar level and immunization and vaccination) were rated as significantly more important than the four indicators in level 3 (ie, HDL cholesterol, blood triglycerides, cancer screening detection, and total cholesterol). Therefore, based on the surveys and our analysis results, among these 13 agreeable health indicators, blood sugar level, and immunization and vaccination were the most important, and the perspectives of the participants were agreed upon across all three samples.

Furthermore, among the less important indicators, the indicator assigned to level 4 (ie, health literacy rate) was found to be significantly more important than the two indicators in level 6 (ie, self-rated health status and HIV testing) and the indicator assigned to level 7 (ie, supply of dentists). Additionally, the two indicators assigned to level 5 (ie, air quality index >100 and personal care needs) were found to be significantly more important than the indicator assigned to level 7 (ie, supply of dentists). Therefore, the survey and analysis results showed that the supply of dentists was the least important among the 13 health indicators, with agreed-upon perspectives across the three samples. Additionally, the inferential statistical test results among the levels provided more confidence in ranking the seven levels from the most important (ie, blood sugar level and immunization and vaccination) to the least important (ie, supply of dentists). The statistical significance test results among the levels provided evidence for prioritizing the 13 health indicators.

The 29 items from the survey instruments showed good levels of internal reliability (Cronbach α=.912), as did each of the three subsets related to institutions where the survey was administered (Table 5). Instruments with Cronbach α values equal to or higher than .7 are generally considered to be reliable [20].

Among all three samples, the ranking of the importance of 13 out of 29 (45%) health indicators showed agreement (Table 3). However, these health indicators were not necessarily more important than the other 16; instead, participants were observed to have perceived importance more consistently among these 13 health indicators. When we compared the 13 health indicators (Table 3) and their corresponding rankings in Table 1, we noticed that the 13 health indicators were placed between the 6th and 24th rankings in Table 1. This indicated more agreement among participants regarding the middle-ranked health indicators than the higher- or lower-ranked ones. The perspectives were more heterogeneous for the higher- or lower-ranked health indicators. Noticeably, the currently widely used standard individual health indicator, self-rated health status, was ranked 20th based on the results of the descriptive statistics. These results indicate a need for new and improved health indicators.

Among the 13 health indicators found in the seven levels, all levels were not significantly different from their immediate next level (Table 4); that is, there were no significant differences between levels 1 and 2 (ie, between n and n + 1). There were, however, significant differences between level 1 and levels 3 to 7 (ie, between n and any level higher than n + 1). These results pertain to the further prioritization of health indicators.

Given descriptive statistics and inferential test results, our findings among the 13 health indicators can reasonably be generalized to some extent to a broader population beyond our survey respondents. We do not claim the complete generalizability of our results mainly because our respondents were not perfectly representative of the composition of the American population. However, we believe that the 13 health indicators and their importance rankings within and among levels can provide substantial and useful evidence when such indicators need to be prioritized.

Cronbach α is one of the more cited statistics for informing internal consistency for the items of an instrument. If Cronbach α is greater than .7, the instrument is reliable [20]. The Cronbach α for the entire survey among the three samples was between .893 and .925. This indicates that we developed a reliable survey instrument. When examining the subscales, only the health care category, which included vaccination and immunization, insurance coverage, and personal care need, was below .7. The items in this category are among the most discussed topics in health care in the United States. Understandably, the reliability is lower since the respondents have relatively less consistent perspectives regarding these items.

This study provides a more comprehensive understanding of the indicators affecting an individual’s health status, particularly as compared to self-rated health status, the most commonly used health status measurement [2]. Although there are advantages associated with using a single health indicator during clinical encounters, we believe that the multidimensional measurement of an individual’s health status may be more objective and can provide additional insights into the individual’s health status, particularly if we are concerned with improving and maximizing the preventive health care services offered. Obtaining these public perspectives is the first step toward a more accurate and effective measurement of individual health status.

This work can be potentially used in two ways: (1) to develop a more comprehensive and objective measurement of an individual’s health status and (2) to develop a health index for an individual. Additionally, these results can be used to prioritize various health indicators (eg, to distinguish between core and secondary indicators). They can also be referenced by designers and developers for EHR systems, personal health record (PHR) systems, or other data capture and analysis applications to determine what health indicators to include in the systems. Furthermore, these results can contribute to developing a health index, which can be used to stratify healthy research participants to make them more comparable. This would be analogous to the Charlson Comorbidity Index [21] or propensity scores [22], which are broadly used in clinical epidemiology data analytics, both of which, however, are disease oriented. Although the health indicators reported here are not in a formula format, this will be a focus for future research. These results set the foundation for further weighting, prioritizing, and validating health indicators via additional data resources.

Additionally, these measurements can track overall health status, measure the outcomes of preventive services, or aggregate data to examine community health. Although having more data points provides increased accuracy and specificity for health indicators embedded within an EHR or PHR, it is important to consider clinician burnout [23] when using technology. Therefore, it is necessary to be mindful of the impacts that creating more data capture requirements or expectations of clinical users may have. In this regard, prioritizing health indicators is a necessary step.

Over the years, other systems have been developed to assess various health risks and associations. The Johns Hopkins Adjusted Clinical Group system, developed and maintained by Johns Hopkins University for over 30 years, is a global tool used in population health analytics [24]. This system is focused on chronic conditions and comorbidities, and its goal is, therefore, fundamentally different from ours, which is to measure individual health status, versus disease status, more accurately. Another system, the Committee on Quality Measures for the Healthy People Leading Health Indicators [3], focuses more on quality measures with an aim to align the measurements within a framework of assessment, improvement, and accountability. The focus, however, is on monitoring and reporting at the population level, not necessarily individual health [3].

There are other health-related surveys broadly used worldwide. For example, the 36-Item Short Form Health Survey, developed by the RAND Corporation [25], measures quality-of-life and health outcomes. Similarly, compared with the related but smaller 12-Item Short Form Health Survey [26], our health indicators provide a more comprehensive measurement beyond physical and mental health. The 9-item Patient Health Questionnaire [27] is a validated tool that measures depression severity. However, we were looking for more objective indicators to measure an individual’s physical and mental health status in our work.

Our health indicators have good but not all-inclusive coverage. The Institute of Medicine’s Committee on the Recommended Social and Behavioral Domains and Measures for Electronic Health Records identified measures across the individual and neighborhood levels that involve sociodemographic, psychological, and behavioral data [4,5]. Among the 17 domains identified by the committee [4,5], 10 were included in our 29 health indicators. Healthy People 2030 [28] proposed 22 leading health indicators for different age groups, of which 16 are included in our health indicators.

The main limitation of this study is that it is only the first step in determining the importance of these health indicators and, notably, the results are subjective, as they are based on public perspectives. Further validation of these results via additional objective measures, such as health care expenditure by disease category [29] and the burden of illness estimates for specific disease categories [30], is needed to support these findings. In this study, for each health indicator, the sample size of valid responses ranged from 791 to 1153. We recognize that larger sample sizes may generate more conclusive and generalizable results. Therefore, our results about the 13 health indicators, even though they are inferential statistics, should be treated as preliminary baseline results; future research may be needed to validate these findings in other settings.

Another limitation concerns the survey respondents. Females comprised the majority of survey respondents, making up 72.1%, 77.7%, and 69.0% of the samples from Ohio University, ResearchMatch, and Clemson University, respectively. We noticed a similar phenomenon in other studies conducted via ResearchMatch. While we are pleased with the relatively large sample size, responses may reflect the perspectives of well-educated females more than those of other groups. For example, survey respondents with a college-level education and beyond represented 54.6%, 82.2%, and 74.0% of the respondents from Ohio University, ResearchMatch, and Clemson University, respectively.

In addition to the distribution imbalance in gender and educational background among our respondents, we also noticed that race and ethnicity groups (Multimedia Appendix 5) were not perfectly representative of the composition of the American population. The breakdown by racial groups among respondents of our surveys was as follows: White American, 87.3%; African American, 3.3%; Hispanic and Latino American, 2.2%; Asian American, 1.6%; Native American, 0.4%; and two or more races, 2%. We recognize that our data set’s gender and ethnicity imbalances are limitations of our current convenience sampling method. In the future, a stratified random-sampling method based on census-based population demographical data might provide more representative results and be a better option. This is a critical point that should be considered when using the results from this study.

We foresee several potential directions in which to continue this project. Our primary goal for future research is to validate the results obtained from the three completed surveys. This can be accomplished in several ways. Because we wish to measure individual health status accurately over time, the use of longitudinal data would be ideal. One data source is a citizen science project initiated by the National Institutes of Health, the All of Us [31] research program. Another source is the UK Biobank initiated in the United Kingdom [32], but the most ideal source would be well-documented longitudinal data of a group of individuals that include not only their EHR data but also other data that correlate with our health indicators. Such ideal data sources would allow for examining the corresponding health indicators and validation of the importance of health indicators via EHR records and additional health-related data. In this way, public perspectives will be considered along with more concrete quantitative evidence to ensure more confidence in prioritizing health indicators and using them for various purposes.

Additionally, to mitigate the effect of the current imbalances seen in respondents regarding gender, race and ethnicity, and other factors, we could explore the possibility of stratified random sampling to proactively select more representative participants. The respondent pool can be more proportionally representative of the composition of the American population. As a potential future project, we may also explore possible correlations between demographic variables and rating results.

Well-designed health indicators are critical tools needed to accurately measure individual health status. They enable the determination of effective preventive services and verify their outcomes. Obtaining the public’s perspective on specific health indicators is the first step toward prioritizing them for analytical and clinical use. This study found that the top five–rated health indicators were drug and substance abuse, smoking and tobacco use, alcohol abuse, major depression, and diet and nutrition. Our respondents, however, had heterogeneous views on the top- and bottom-rated health indicators. The middle 13 health indicators were rated more homogeneously among all the respondents. These 13 health indicators were separated into seven levels based on their perceived importance, providing further evidence that was used to prioritize these health indicators. Levels 1 to 7 were organized based on the mean importance of health indicators from high to low within and between each level. Level 1 included blood sugar level and immunization and vaccination; level 2 included LDL cholesterol; level 3 included HDL cholesterol, blood triglycerides, cancer screening detection, and total cholesterol; level 4 included health literacy rate; level 5 included personal care needs and air quality index greater than 100; level 6 included self-rated health status and HIV testing; and level 7 included the supply of dentists. The results of this study can provide evidence to EHR or PHR system designers and developers, which they can then use to select health indicators to incorporate into their systems.