Using a randomized controlled trial design (ClinicalTrials.gov: NCT00142077), we evaluated an electronic PCHR system to modify knowledge, beliefs, and behavior around influenza. Participants were recruited from eight Hewlett Packard Corporation work sites in the northeastern United States in the fall of 2005. Employees at the research sites were recruited with two emails sent to their work email address by the company’s human resources department. The emails contained study information and invited potential participants to complete a brief set of questions to assess eligibility. Eligible volunteers were 18 years of age or older, comfortable reading and writing in English, part-time or full-time employees of the company and had reliable Internet access at work, school, or home and used email at least once every 2 days. In addition, participants could not have a history of a severe reaction to influenza vaccine or severe allergy to chicken eggs, since both of these conditions contraindicate use of the influenza vaccine. Enrollment was initially planned for October 2005; however, just prior to the original recruitment period, several Hewlett Packard work sites were closed and employees were relocated or laid off at several other sites. Therefore, the study began in November 2005 and our recruitment pool was smaller than anticipated. All participants electronically provided informed consent prior to study initiation. The study was approved by the Committee on Clinical Investigation at Children’s Hospital Boston, Boston, MA, USA.

Assignment to the intervention and control groups was performed at the level of the corporation work site in order to prevent employees at the same site from sharing information about the trial, including recommendations provided in the health messages. Prior to study initiation, we created two groups with four sites in each such that the number of employees in each arm was evenly distributed. The two groups were then randomly assigned to the intervention and control arms by a person unfamiliar with the details of the work sites. Participants in the study were informed that the study was to evaluate health promotion using a PCHR with an electronic messaging system and were masked as to whether they were in the intervention or control groups.

We used a PCHR system called PING [9-11] (new versions are called Indivo [12]), which is built to open standards on a flexible XML data model and is accessible over the Web. PING is designed to enable patients to own complete, secure copies of their medical record and to integrate information over time and across sites of care [5,11]. In this investigation, we tested the survey, decision support, and health messaging features of PING, and records did not include any health information beyond the data provided by subjects for this study. Enrolled subjects completed online health risk assessment surveys, the responses to which drove the decision support system to generate and send tailored health messages for participants in the intervention group. These messages were sent to participants’ PING record inbox, and participants were simultaneously notified with a standard, plain-text email instructing them to visit and log on to their PING record to review the message (Multimedia Appendix 1: PING Record Welcome Screen and Inbox).

Participants in the intervention and control groups completed three types of survey. The first was a baseline survey that was posted in their PING record immediately after registration was completed (Multimedia Appendix 2: Enrollment Survey). This survey collected demographic data; information on medical history; health-related behaviors; influenza risk factors; knowledge, beliefs, and behavior around influenza; and information related to Internet use. Information was also collected on household members, including their age; gender; attendance at work, school, or daycare; and behaviors and risk factors related to influenza. The baseline survey administered to the control group contained additional questions addressing routine health and knowledge and behaviors regarding cardiovascular health and sun protection.

The second survey was a biweekly survey consisting of a brief set of questions that was administered approximately every 2 weeks (Multimedia Appendix 3: Biweekly Survey). A total of seven of these surveys were administered between December 1, 2005, and March 1, 2006. Information was collected on recent respiratory illnesses in participants and household members, including duration of symptoms, missed work or school days, medication use, and health care utilization, and an update was obtained on their influenza vaccine status. Biweekly surveys for the control group included additional questions on routine health care use and recent gastrointestinal or other illness.

The third survey was an exit survey administered at the end of the study, 2 weeks after the last biweekly survey. It contained the same questions on influenza knowledge, beliefs, and behavior as the baseline survey, as well as questions to evaluate the electronic interface of the application, its usability, the content of the questions, and the overall utility of the PCHR-based program to participants. The survey administered to the control group additionally contained the same questions on knowledge and behaviors regarding cardiovascular health and sun protection administered in the baseline survey.

Participants in the intervention group received different types of influenza-related health messages throughout the study period. Some of these were personalized based on the information provided in the baseline and biweekly surveys and were posted in the record after a participant completed one of these surveys. Messages were tailored to include advice for all household members, to identify individuals at high risk for influenza-related complications, and to provide information on respiratory illnesses if a participant or household member became ill with a respiratory infection. The health messages were also tailored based on the home addresses of participants to advise them of influenza activity in their area. Other messages contained general information and were provided on a weekly or monthly basis. The content of the health messages was regularly monitored throughout the study period to ensure that proper messages were being generated and transmitted to participants.

There were five types of health message:

Figure 1

Sample health message: vaccine reminder (In this example, Alice and Bob are household members of the study participant.)

The primary outcome was change in knowledge, beliefs, and behavior surrounding influenza prevention. Change in knowledge was assessed using a set of nine questions in the baseline and exit surveys addressing influenza transmission and prevention, influenza illness, and the influenza vaccine. Change in beliefs was measured with a set of six questions on influenza illness and vaccine, administered in the baseline and exit surveys. Measurements of behavior change consisted of the rate of influenza vaccination, the rate of work attendance despite a respiratory illness, and responses to two questions in the baseline and exit surveys on hand hygiene and cough etiquette.

Secondary outcomes included the rate of influenza vaccination among household members and changes in knowledge and behavior regarding cardiovascular health and sun protection, measured using nine questions in the baseline and exit surveys administered to the control group. Finally, the usability and utility of the PCHR-based program were assessed by means of 12 questions in the exit survey, as well as survey completion rates and mean days to survey completion.

Logistic regression models were used to analyze the changes in responses for the questions on knowledge and behavior surrounding influenza. The models controlled for baseline responses in the initial survey. A variable for participant work sites was tested in the models to control for clustering. This variable was not significant in any of the analyses and was excluded from the final models. Immunization rates were compared using chi-square analysis. Rates for missed work during an illness were examined using the SAS v9.1 (SAS Institute, Inc, Cary, NC, USA) PROC GENMOD procedure in order to control for correlated responses from participants with more than one illness. The knowledge and behavior questions on cardiovascular risk and sun protection for the control group were analyzed with the McNemar test. Assessment of the usability and utility of the program was performed through examination of the responses to the questions on user experience and calculation of completion rates and mean days to survey completion.

Participant flow is shown in Figure 3. We recruited participants during a 4-week period between November 10 and December 7, 2005. Of the 3540 employees at the eight work sites, 144 employees registered for the study and 125 completed the baseline survey. Of these, 119 (95%) completed between one and seven biweekly surveys, and 99 (79%) completed the exit survey. The baseline characteristics of the intervention and control groups are shown in Table 1. Only the gender distribution differed between the two groups.

Figure 3

Flow diagram of study participation

Of the 123 participants who completed the baseline survey in time to be included in the study, the average number of days to complete the survey was 1.8 days (range 0-25 days) and 1.4 days (range 0-20 days) among intervention and control group participants, respectively. Among the 119 participants who completed at least one biweekly survey, the mean time to completion among intervention subjects (N = 67) was 3.3 days (range 0-24 days) and among control subjects (N = 52), 3.1 days (range 0-15 days). The mean number of completed biweekly surveys was 6.6 (range 1-7) for the intervention group and 6.7 (range 1-7) for the control group. A total of 80% (99/123) of participants completed the exit survey, with mean times to completion of 6.3 days (range 0-27 days) for the intervention group (N = 56) and 7.6 days (range 0-23 days) for the control group (N = 43). Completion rates were 80% (56/70) and 81% (43/53) among the intervention and control groups, respectively.

Among the participants who completed the exit survey, 78% (77/99) rated the PCHR as “extremely” or “very” easy to use and 84% (83/99) indicated survey questions were “extremely” or “very” clear. When asked about specific parts of the messaging system, the aspects deemed most useful by participants were messages with information on prevention of influenza illness, general information on influenza illness, and messages indicating influenza activity in participant's’ geographic area. Overall, 73% (72/99) responded that they would be “extremely” or “very” likely to participate again in the use of a PCHR-based health promotion program such as this one. In terms of privacy concerns for providing information electronically, 57% (56/99) were “not at all” or “a little” concerned, and an additional 25% (25/99) were “moderately” concerned. A total of 62% (61/99) indicated that they would have been willing to provide additional health-related information. Participants found the biweekly surveys to be brief, with 51% (50/99) responding that completion took less than 5 minutes.

Among participants in the intervention group, 54% (30/56) rated the messaging system as “extremely” or “very” useful in providing information about influenza, 13% (7/56) indicated that the messaging system was “extremely” or “very” important in their decision about whether to obtain the influenza vaccine for themselves, and 20% (11/56) responded the same regarding its importance in the immunization of household members.