The use of digital strategies to address public health priorities, such as HIV, has increased tremendously in the United States in the last decade [1-4]. To inform implementation strategies, many survey instruments have been developed to measure contextual attitudes about health technology use across patient populations. The National Cancer Institute’s Health Information National Trends Survey (HINTS) is a biennial nationally representative survey to assess the impact of the health information environment. It includes items useful to describe individual perceptions about the privacy and security of electronic medical records in a national sample [5]. Instrument development has been applied to assess attitudes about health technology use in specific populations, such as among older adults [6]. Psychometric constructs assessed within instruments are sometimes the basis of path models developed to measure behavioral intentions among consumers using health information technology, such as the Health Information Technology Acceptance Model [7]. This study focuses on developing an instrument for a neglected area of study, patient portal use for the prevention and care of sexually transmitted infections (STIs), including HIV.

Patient portals provide patients with secure web-based access to health information, such as laboratory results and prescription medications. The web-based personal health information that a consumer accesses when using their patient portal is a type of health information technology (Health IT), referred to as personal health records (PHRs) [8,9]. Health IT is a broad concept that encompasses an array of technologies applied in health systems, such as PHRs, electronic health records (EHRs), or e-prescribing [10]. Logging on to a patient portal, users may message doctors, locate health education content, and a variety of options, depending on the Health IT platform. Certification for Health IT products is established by standards, implementation specifications, and certification criteria adopted by the US Secretary of Health and Human Services [11]. Data security and privacy measures, such as encrypting authentication credentials, are outlined within standards [12]. This is important because privacy concerns and mistrust are known barriers to PHR adoption [13,14]. Nevertheless, among US sexual minority men (SMM), patient portals are highly acceptable for delivery of comprehensive sexual health services that support HIV preventative behaviors, such as disclosing STI PHRs with sex partners [15,16].

In a 2016 market of approximately 186 certified vendors, 92% of US hospitals contracted either Cerner, MEDITECH, Epic Systems, CPSI, McKesson, and MEDHOST to supply certified Health IT [17]. To be certified, EHR vendors must include a patient portal module and use Fast Healthcare Interoperability Resources specifications to allow communication across vendor platforms [12,18]. Examples of patient portals are Epic’s MyChart, Veterans Administration’s My HealtheVet, Geisinger’s MyGeisinger, and Kareo’s Kareo EHR Patient Portal. There are over 300 unique Health IT products with 2015 certification that allow patients (ie, consumers) to view, transmit, and download patient medical data to a third party [19]. PHR systems may also be standalone and untethered to a health system’s EHR system [9].

As a result of federal incentive programs, namely, the Merit-Based Incentive Payment System programs Advancing Care Information, which have replaced Meaningful Use, patient portal access in health care settings has grown exponentially in the last decade and provides an excellent opportunity to deliver important health communications to patients [20-22]. By 2015, 95% of hospitals and 63% of office-based physicians had IT solutions allowing patients to view web-based health records [22,23]. Patient medical records are often perceived as highly private and only to be viewed by the patient and authorized representatives, which are based on important ethical and societal concerns of privacy [24-26]. However, portals also allow patients to interpersonally use information in PHRs outside of health care settings, such as sharing laboratory results or prescriptions with family, friends, and partners when promoting health behaviors [27]. Furthermore, few empirical studies have measured the preferences for and potential health benefits of the patient portal for sexual health and well-being on both individual and interpersonal levels.

To date, little information is available on patients’ perceptions of patient portals and their potential use for the prevention and care of STIs, including HIV [28-30]. STIs are substantial public health problems, and they may be particularly well suited for prevention and management with patient portals. With billions of dollars in US health care costs, an estimated 20 million new infections each year, and as major causes of preventable disease, STIs are serious and growing public health threats [31]. In 2018, there were 2,457,118 combined cases of chlamydia, gonorrhea, and syphilis alone in the United States, continuing the trend of record-breaking numbers within the last decade [32]. The increasing prevalence of multidrug-resistant gonorrhea and the rise in syphilis infections also signal the urgent need for more effective prevention strategies [33,34].

Inefficient access and adherence to HIV medication among people living with HIV and pre-exposure prophylaxis (PrEP) among some SMM risk groups contribute to the burden of new HIV infections. In fact, persons with HIV who attain virologic suppression cannot transmit HIV to others, as regular PrEP use can effectively prevent the acquisition of HIV infection [35,36]. Adherence requires compliance with outpatient provider visits, filling in prescriptions, and taking antiretroviral therapy (ART) or PrEP over time; however, nonadherence can increase HIV disease progression and risk of disease transmission. Patient portals may effectively be able to create an easy-to-use system for organizing health information and be helpful in engaging SMM in care on all fronts of HIV infection.

Given the stigma about same-sex sexuality and talking about HIV and STIs, portals can also bridge communication gaps with providers and sex partners. According to the Centers for Disease Control and Prevention, gay, bisexual, and other SMM are disproportionately impacted by syphilis, HIV, and other STIs [37,38]. Factors such as patient-provider communication barriers and anticipated health care stigma experienced among SMM are known to hinder the use of STI preventive services such as testing [39-41]. In addition, controlling HIV is harder among persons with mental health issues, resulting in lower rates of HIV virologic suppression [42,43]. In a call to action for comprehensive HIV services for SMM, Beyrer et al [44] recommend health providers to provide integrated mental health services for SMM. SMM today have demonstrated high acceptability of medical and public health practice supported by mobile devices or mobile health (mHealth) in HIV interventions; however, these solutions have largely operated outside of the Health IT aegis of EHR and integrated PHR systems [45]. Previous studies examining perceptions of patient portal use for HIV care and STI prevention have demonstrated high acceptability and concern about potential breaches of privacy [28,30,46].

With this changing environment, new studies continue to burgeon modeling the implementation of health innovations for behavioral interventions focused on SMM. However, little data are available on the attributes and expectations of patient portals among SMM broadly across the United States. Theoretical frameworks, such as the Diffusion of Innovation theory and the extended unified theory of adoption and use of technology, are often used in behavioral sciences to inform implementation of new health technologies across consumer populations [6,47,48]. The theories posit that attributes of health technology and performance expectations are drivers of behavioral intentions to use health technology, and further behavioral intentions and habits determine behaviors or patterns of use. At the foundation of optimizing the design and uptake of new health technologies is scaling consumer perceptions about technology. Therefore, the overall goal of this study is to evaluate the psychometrics and adaptation of an instrument for measuring attitudes about using patient portals for STI prevention and care. Within-group comparisons of instrument scale scores are examined for categorical variables, including age, US region, last STI test, STI diagnosis, anticipated health care stigma, and mental health status.

Data were obtained from the 2018 American Men’s Internet Survey (AMIS). AMIS is an annual cross-sectional web-based survey of US residents who are aged at least 15 years; are cisgender male; and are gay or bisexual or have ever had sex with a man [49]. The study was conducted in compliance with federal regulations governing the protection of human subjects, and the study protocol was reviewed and approved by the Institutional Review Board at Emory University.

A randomized subset of 4647 AMIS participants was presented with survey items focusing on using patient portal services for STI prevention and care. Only survey participants reporting the ability to view their web-based health records were included in the analysis. Of these 2566 participants with patient portal access, the participants with do not know and refuse to answer responses on patient portal items were excluded from the analysis, resulting in a final analytic sample of 1375 men.

The Patient Portal Sexual Health Instrument (PPSHI) is a scale designed to evaluate perceptions of using patient portals to promote STI prevention and care behaviors. Items were originally adapted from a study among coed students at a historically Black university characterizing the perceived role of patient portals in supporting STI prevention behaviors—the Electronic Sexual Health Information Notification and Education (eSHINE) Study. In total, the eSHINE Study-Patient Portal Instrument (eSHINE-PPI) consisted of 19 items representing 4 subscales: (1) sexual health engagement (4 items), (2) informational resource compatibility (3 items), (3) valuation of services (5 items), and (4) PHR impact (7 items). A complete list of eSHINE-PPI items, factor loadings corresponding to unique subscales, and reliability coefficients can be found in Multimedia Appendix 1 [29,50]. To explore a more parsimonious instrument, we adapted 12 items from 3 eSHINE-PPI subscales to a nationwide survey of SMM. We focused on eliminating the eSHINE-PPI subscale items with the lowest factor loadings. Responses were also reduced from 7-point Likert scales to 4-point ordinal scales. We then adapted the eSHINE-PPI instrument to the AMIS-2018 survey.

The lowest loading item on the sexual health engagement subscale, “I plan to manage my medical records with PHRs in the future,” was eliminated. The final adapted subscale (3 items) measured perceived attributes of using (vs not using) patient portals, specifically that (1) it is a more convenient way to manage my sexual health records, (2) it encourages people to be more aware of their sexual health, and (3) it will help people like me make better sexual health decisions. Responses were coded as follows: 0=strongly disagree, 1=disagree, 2=agree, and 3=strongly agree. Possible scores ranged from 0 to 9.

The following 4 items were eliminated from the PHR impact subscale: (1) PHRs make it easier for people to routinely have check-in conversations with partners about STI prevention, (2) Partners using PHRs will start talking about STI prevention earlier in a relationship, (3) I would have more discussions with partners about STI testing if PHRs were more commonly used, and (4) Using PHRs with a partner builds trust. The final adapted subscale (3 items) measured agreement with beliefs that sharing STI PHRs with partners will (1) improve communication on HIV and other STIs, (2) improve confidence in the testing information a partner share, and (3) improve control over my sexual health and decision making. These responses were coded as follows: 0=definitely not, 1=probably not, 2=probably, and 3=definitely. Possible scores ranged from 0 to 9.

The item In addition to electronic sexually transmitted disease (STD) results, which services are important for PHRs to include: Access to all of your medical records was eliminated from the valuation of services subscale. Given the increase in telehealth success, the Services to communicate with your doctor or health professionals item was modified to specify video chat services to communicate with health care providers [51,52]. In response to recent decade increases in the use of interactive games to improve HIV prevention behaviors and the use of home test kits for STI screening, we added 2 items assessing valuation for games to promote sexual health and the ability to order home test kits for HIV and STDs to the valuation of services subscale [45,53-58]. For the final adapted subscale (6 items), participants were asked to rate the value of 6 patient portal features: (1) games to promote sexual health, (2) ability to order home test kits for STIs, (3) counseling and resources for people with STIs, (4) telehealth services, (5) ability to locate STI testing centers, and (6) tips or tools for managing sexual health. Responses on patient portal functionality items were coded as follows: 0=no value, 1=low value, 2=moderate value, and 4=high value. Possible scores ranged from 0 to 18.

The Informational Resource Compatibility Subscale (3 items) was not adapted to AMIS. Possible scores for the total AMIS-PPSHI ranged from 0 to 36.

First, we conducted a principal component analysis (PCA) of the 12 items to estimate an appropriate number of factors to retain. The retention of factors was determined by a parallel analysis and examining eigenvalues greater than 1.0 [59]. We used the exploratory factor analysis (EFA) to estimate the factor structure and loadings. Item loadings were examined for each factor using cutoff values ≥0.50. Sampling adequacy was indicated by a Kaiser-Meyer-Olkin (KMO) score greater than 0.80 [60].

We estimated the overall scale and subscale internal reliability with Cronbach α for the new AMIS-PPSHI. Confirmatory factor analysis (CFA) was used to evaluate measures of fit, including the Tucker-Lewis index (TLI), the comparative fit index (CFI), the root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR). Thresholds for fit statistics were values close to 0.95 for TLI and CFI, values close to 0.08 for SRMR, and values close to 0.06 for RMSEA [61]. For standardized factor loadings, we used a cutoff of 0.50 [62].

We calculated the total AMIS-PPSHI scale scores for comparisons within group categories, including age, STI diagnosis history, recency of testing, serious mental illness, and anticipated health care stigma. The Kessler 6-item (K6) scale for psychological distress was used to determine the presence of serious mental illness (score 13 or greater) [63]. Anticipated health care stigma was defined as whether the participant was ever afraid to seek health care services because of worrying that someone may learn they have sex with men. Two-sample t tests were used to compare binary group differences in scale scores using a P value of .05 as the criterion for statistical significance. Analysis of variance (ANOVA) was used to compare group differences in scale scores for variables with more than 2 groups, also using a P value of .05 as the criterion for statistical significance.

The sample consisted of 1375 US SMM with a median age of 34 years and an IQR of 25 to 50 years. The demographic characteristics of the study sample are presented in Table 1. Table 2 presents the items included in the questionnaire with calculated mean and mode scores. The KMO score (0.8492) established sampling adequacy. We used a Promax rotation because the factor correlations exceeded 0.32 [64]. On the basis of eigenvalues above 1.0 and parallel analysis, we estimated a 3-factor solution for factor analysis. Factor analysis yielded 3 distinct factors using the 0.50 cutoff and no cross-loading above 0.15.

Table 3 displays the 12 items and their respective loadings across the 3 factors. Items 1-3 loaded on a factor named sexual health engagement and awareness, reflecting the perceived attributes of using patient portals to engage in sexual health care. Items 4-6 loaded on a factor named enhancing dyadic communication, reflecting the perceived attributes of using STI PHRs to share testing history with partners. Items 7-12 loaded on a third factor named managing sexual health care, reflecting the desired functionality of patient portals. Inter-factor correlation were 0.42 between the sexual health engagement and awareness and enhancing dyadic communication factors. Sexual health engagement and awareness and managing sexual health care had a correlation of 0.29. The estimated correlation between managing sexual health care and enhancing dyadic communication was 0.42. The CFA results (Table 4) suggested good fit indices for the 3-factor model (RMSEA=0.061, CFI=0.964, TLI=0.953, SRMR=0.041, and coefficient of determination =0.996).

Table 5 presents factor scores for AMIS-PPSHI total and Table 6 factor scores for AMIS-PPSHI subscales by group categories using the sum of scores for variables with factor loadings above a cutoff of 0.50 [65]. The mean (M) and standard deviation (SD) AMIS-PPSHI total score was mean 28.94 (SD 5.14). Mean PPSHI subscale scores were as follows: sexual health engagement and awareness mean 7.46 (SD 1.70), enhancing dyadic communication mean 7.57 (SD 1.71), and managing sexual health care mean 13.90 (SD 3.22). By region, AMIS-PPSHI total scores were moderately higher than average in the South and West compared with the Northeast and Midwest regions. Mean AMIS-PPSHI scores decreased with increasing age category, most notably in the enhancing dyadic communication scale (F_3,1371=10.87; P<.001). Participants who were tested 12 months before the study had slightly higher mean scores on sexual health engagement and awareness, mean 7.52 (SD 1.66) versus mean 7.18 (SD 1.83), and enhancing dyadic communication, mean 7.62 (SD 1.69) versus mean 7.38 (SD 1.82). The largest difference in the AMIS-PPSHI score were in comparisons of participants according to their history of HIV or recent STI. The overall highest scores were among people without HIV and without recent STI. Participants living with HIV have the lowest overall AMIS-PPSHI mean scores, mean 27.57 (SD 5.89), primarily because of enhancing dyadic communication scores, mean 6.85 (SD 2.13). There are no significant differences in scores by anticipated health care stigma nor serious mental illness. However, scores were marginally higher among participants with a Kessler 6-item psychological distress scale (K6) score≥13.

The goal of this study is to develop a brief instrument for measuring attitudes focused on using patient portals for STI prevention and care among a nationwide sample of SMM. HIV and other STIs are costly and have a high burden on SMM. Patient portals could be used to address risky sexual behaviors; however, past studies have not looked at service design and consumer adoption models for patient-facing IT solutions. Therefore, we created a brief instrument to enable the measurement of attitudes toward using patient portals for STI prevention and care among SMM with access to a patient portal. The instrument was adapted from earlier scientific work, is short, and may be added to health questionnaires focused on sexual health–related technology use. Furthermore, we demonstrated that AMIS-PPSHI might be adapted to include novel consumer-oriented features as technology evolves.

The resulting instrument consists of 12 items and 3 subscales measuring constructs of (1) sexual health engagement and awareness, (2) enhancing dyadic communication, and (3) managing sexual health care. Constructs cover unique aspects of patient portal use. First, portals should communicate personalized sexual health information to the user. Second, interpersonal use of patient portals may occur outside of health settings, particularly in events that share test histories with sexual partners. Third, patient portals should empower patients to engage with an array of sexual health care management services, such as testing and telehealth.

PPSHI and its 3 subscales had a strong overall internal consistency. As expected, younger participants are more receptive to technology use. Scores have an indirect relationship with age; SMM aged 15-24 years had the highest scores. Scores stratified by STI diagnosis reveal interesting dynamics, most notably in the enhancing dyadic communication subscale. Scores on the enhancing dyadic communication subscale are the lowest among participants with HIV, an indicator of the highly stigmatized nature of HIV. Interpersonal use of patient portals with sexual partners may likely be lower for participants with HIV or other chronic STIs. Interventions are needed to reduce this stigma and to strengthen self-efficacy for discussing with partners the topics pertinent to sexual health and wellness.

Similar scores by anticipated health care stigma may be an indicator of acceptability of patient portals among participants who may be less likely to receive HIV care services [39,41]. Slightly higher AMIS-PPSHI scores among participants with mental health illness support the acceptability of the patient portal use among patients with mental health disorders [66]. Together with earlier studies, findings support patient portals as a promising avenue to plan interventions around increasing health engagement among marginalized groups, including persons with HIV [28,30,46]. The overall high mean sample score on AMIS-PPSHI may also indicate that patient portal interventions may extend to other areas of health care engagement, such as achieving hepatitis A and B vaccination and screening recommendations for SMM [67-69]. Messages delivered through patient portals have been demonstrated to increase herpes zoster vaccination in adults [70]. Thus, the current global climate of hepatitis A outbreak among SMM reflects missed opportunities to leverage patient portals to deliver hepatitis A and B vaccination screening messages to SMM [71-75]. The application of machine learning algorithms to identify PrEP candidates using EHR data may also be applied to identify candidates for hepatitis A and B vaccination; however, research is needed to develop efficacious algorithms [76].

The strength of AMIS-PPSHI is that it is based on empirical research, and it is very timely to the growing technology-based STI prevention models. The scale is shortened in item numbers and response options and still holds a strong internal consistency. The factor analysis is based on both an acceptable sample size of participants and the number of observed variables. Overall, the instrument performed statistically well in psychometric analysis. The age distribution of the sample offers some comparisons and extrapolation across groups of adolescent, young, and adult SMM. However, neither AMIS-PPSHI nor its subscales have been validated as constructs related to patterns of patient portal use for sexual health and wellness. Future validation studies may explore the relationship of constructs with patterns of patient portal use specific to sexual health and wellness. Clinical researchers may test the enhancing dyadic communication subscale as a determinant of an individual’s likelihood to use PHRs for sharing STI test histories with main and nonmain partners [16]. The subscale may then be applied in clinical settings with decision analytics to identify patients for interventions that are less likely to disclose STI PHRs with partners. More data are needed on patterns of patient portal use for sexual health services such as viewing electronic STI test results, viewing health information on STIs, and ordering medications to prevent and treat STIs.

African American or Black SMM are notably underrepresented in the study sample. Future studies are needed to apply PPSHI across a broad nationwide sample of Black SMM and youth—the race or ethnic group most overburdened by HIV and other STI incidence and prevalence [34,38]. Additional validation studies are needed among other priority populations for STI prevention in the United States and other countries with burgeoning mHealth environments. A further limitation is that the instrument does not include items focused on the use of patient portals to report sexual health–related behaviors and outcomes. Ecological momentary assessment and patient-reported outcome measures are mechanisms that can feed data into patient portal systems and inform decision support algorithms for the user or health care providers [77-79]. Given the adaptability of PPSHI, items may be added to patient-reported outcomes to assess perceptions about reporting personal data to the patient portal for sexual health.

In summary, we suggest that PPSHI and its components could predict behavioral intentions and patterns for patient portal use for health behaviors related to STI prevention and care among SMM. PPSHI is feasibly adaptable to questionnaires and may have useful applications in electronic patient intake surveys. Short surveys on patient intake forms assessing risk behaviors have been used to inform clinical decision support algorithms, prompting providers to encourage STI screening for patients [80]. Assessing PPSHI constructs in patients may similarly be useful in informing provider messaging within decision support algorithms, for example, encouraging patients to share STI PHRs with partners.