Computer-based health-risk assessments are electronic surveys which can be completed by patients privately, for example during their waiting time in a clinic, generating a risk report for the clinician and a recommendation sheet for the patient at the point of care. Despite increasing popularity of such computer-based health-risk assessments, patient attitudes toward such tools are rarely evaluated by reliable and valid scales. The lack of psychometric appraisal of appropriate scales is an obstacle to advancing the field.
This study evaluated the psychometric properties of a 14-item Computerized Lifestyle Assessment Scale (CLAS).
Out of 212 female patients receiving the study information at a family practice clinic, 202 completed a paper questionnaire, for a response rate of 97.6%. After 2 weeks, 52 patients completed the scale a second time.
Principal component analysis revealed that CLAS is a multidimensional scale consisting of four subscales (factors): (1) Benefits: patient-perceived benefits toward the quality of medical consultation and means of achieving them, (2) Privacy-Barrier: concerns about information privacy, (3) Interaction-Barrier: concerns about potential interference in their interaction with the physician, and (4) Interest: patient interest in computer-assisted health assessments. Each subscale had good internal consistency reliability ranging from .50 (2-item scale) to .85 (6-item scale). The study also provided evidence of scale stability over time with intraclass correlation coefficients of .91, .82, .86, and .67 for the four subscales, respectively. Construct validity was supported by concurrent hypotheses testing.
The CLAS is a promising approach for evaluating patients’ attitudes toward computer-based health-risk assessments.
The use of computer interactive technology in health care settings is on the rise. Many studies report using patient-administered computer programs for health-risk assessments [
Many practical implications have also been recognized for computer-based health-risk assessments. At the organizational level, the advantages include speed and efficiency, accountability, quality improvement, and cost containment [
However, user attitudes toward interactive computer technology are important when considering applications. In 1986, Nickell and Pinto developed a computer attitude scale for the general population [
In our review of the literature on computer-based health-risk assessments, two scales were identified as potentially applicable to general patient populations. The first scale was developed by Lucas in 1977 and tested among patients visiting specialized clinics in hospital settings [
Addressing some of these concerns, Skinner developed a short 14-item Computerized Lifestyle Assessment Scale (CLAS) in 1993 [
Lack of psychometric appraisal of scales may impede research and innovation to advance the field. Recent studies have begun to report patients’ general reactions to the use of computer interactive technology. In 2000, Dugaw et al reported patients’ overall acceptance of computerized medical history taking in an emergency department, with limited description of the measurement [
Considering the potential of CLAS, this study evaluated its psychometric properties as part of a larger research program on computer-based screening for lifestyle risks, including partner abuse, among female patients. Using standard procedures [
The study was conducted at a multidisciplinary family practice clinic affiliated with a teaching hospital in Toronto, ON, Canada. The study was approved by the hospital research ethics board as part of a research project on prevalence of partner abuse; details are provided elsewhere [
All adult female patients with an appointment were eligible to participate if they were at least 18 years of age, could speak and read English, and could provide informed consent. The study participants were recruited in 15 days over a period of three consecutive weeks in February of 2004. On recruitment days, all adult female patients with appointments were given a brief letter of invitation by the clinic receptionist at the time of arrival. These potential participants were then approached in the waiting area by a recruiter to confirm their eligibility and inquire about their interest in the study. Willing participants were taken to a separate room in the clinic, unaccompanied by family or friends, where they completed the survey after giving informed consent. At this time (T1), participants were also asked to consent to a subsequent contact after 2 weeks (T2) to administer the CLAS a second time. Participants sealed the survey in the provided envelope before returning it to the recruiter. Then, participants received health brochures (domestic violence, cancer, and heart health) with telephone numbers for domestic violence counsellors and the assaulted women’s helpline.
The survey included the CLAS, which is a 14-item scale that covers patients’ positive and negative perceptions about computer-based health-risk assessments [
We would like to know your opinion about a computer survey of patients. This survey is completed by patients on a computer before seeing their family doctor. The computer survey asks questions about lifestyle and health risks such as smoking, stress, conflict in relationships, and safety. The questions appear on the computer screen one by one. The patient answers by touching one of the options on the computer screen using a non-ink pen. Patients do not type or use any computer parts but only touch the screen to give answers. This computer survey uses simple day-to-day language of 5th grade reading level. The computer system prints (1) a summary of patient health risks for the doctor to review, and (2) an information sheet for the patients about their reported health risks.
The aim was to recruit a sample of 200 participants. As CLAS included 14 items, a sample of 200 was expected to generate an adequate subject-to-variable ratio of 14:1 to derive latent constructs. For factor analytical approaches, Gorsuch (1983) and others recommend a subject-to-variable ratio of five when the communalities are high and there are many variables for each factor [
The CLAS items [
Prior to reliability and validity analysis, we examined the latent structure of the scale. The latent constructs of the CLAS were examined by employing principal component analysis (PCA) [
The scale reliability was estimated by both internal consistency and test-retest reliability of the subscales. To examine homogeneity of items or internal consistency, item-total correlation [
After factors were derived and reliability established, construct validitywas investigated. For this analysis, we tested hypotheses that were based on existing literature. Further details on the hypotheses are presented in the Results section under construct validity. The hypotheses were tested by using Pearson product moment (
Among 361 women approached, 212 eligible women received the study details in privacy, 207 provided written consent (response rate 97.6%), and 202 returned the completed surveys. Participants had a mean age of 45.3 years (range 19 to 86) and 36% were immigrants, with the top two groups from Europe and Asia (
Sociodemographic characteristics (N = 202)
Variable | No. | % |
Age (years), mean (SD) | 201 | 45.3 (15.4) |
Current marital status | 202 | |
Married or common law or intimate | 74.9 | |
Separated or divorced or widowed | 13.9 | |
Single, not in relationship | 11.4 | |
Country of birth: Canada | 129 | 63.9 |
If immigrant: years lived in Canada | 71 | |
Less than 10 years | 23.9 | |
11 to 20 years | 16.9 | |
More than 20 years | 59.2 | |
If immigrant: country of birth | 72 | |
Europe | 36.1 | |
East or South East Asia or South Asia | 29.1 | |
West Indies, Latin America, or Caribbean | 20.8 | |
Middle East or West Asia | 6.9 | |
Africa | 5.6 | |
Highest education | 201 | |
Less than high school | 3.0 | |
High school, some or complete | 19.9 | |
University or higher, some or complete | 77.1 | |
Current employment | 201 | |
Full-time or part-time | 64.2 | |
Unemployed | 13.9 | |
Retired or on disability | 21.9 | |
Household annual income (Can $) | 181 | |
Less than 20,000 | 15.5 | |
20,001 to 40,000 | 19.9 | |
40,001 to 60,000 | 20.5 | |
More than 60,000 | 44.2 | |
Access to computer at home or work | 200 | 87.0 |
Use of computer in the last month | 200 | |
Every day or two to three times a week | 81.5 | |
Once a week or once a month | 6.5 | |
Not at all | 12.0 | |
English ability,* mean (SD) | 201 | 4.5 (0.87) |
Survey comfort level,† mean (SD) | 199 | 4.0 (1.2) |
*Scale of 1 to 5: 1 = poor, 2 = fair, 3 = good, 4 = very good, 5 = excellent.
†Scale of 1 to 5: 1 = very uncomfortable, 2 = uncomfortable, 3 = not sure, 4 = comfortable, 5 = very comfortable.
Item summary statistics and Pearson correlations
Item† | % Miss* | Mean‡ | SD | Skewness | Kurtosis | Item Correlation | |||||||||||||
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | ||||||
1. Routine | 1.0 | 3.93 | 0.86 | -0.74 | 0.62 | 1 | |||||||||||||
2. Lifestyle | 0.5 | 3.67 | 0.94 | -0.73 | 0.39 | .58 | 1 | ||||||||||||
3. Save time | 2.0 | 3.73 | 0.92 | -0.64 | 0.55 | .62 | .47 | 1 | |||||||||||
4. Better assess |
1.0 | 3.28 | 0.84 | -0.07 | 0.17 | .54 | .43 | .50 | 1 | ||||||||||
5. Comfortable | 1.0 | 3.92 | 0.85 | -0.86 | 0.93 | .60 | .52 | .41 | .34 | 1 | |||||||||
6. Trusted | 2.0 | 3.36 | 0.94 | -0.33 | 0.02 | .47 | .33 | .37 | .42 | .59 | 1 | ||||||||
7. Confidentiality | 0.5 | 3.30 | 1.14 | -0.17 | -1.06 | -.08 | -.05 | -.80 | -.06 | -.24 | -.29 | 1 | |||||||
8. Certain information | 0.5 | 3.39 | 1.12 | -0.24 | -0.78 | -.14 | -.04 | -.15 | -.04 | -.22 | -.26 | .47 | 1 | ||||||
9. Mistakes | 1.0 | 2.63 | 0.84 | 0.51 | 0.27 | -.18 | -.16 | -.14 | -.27 | -.39 | -.41 | .41 | .38 | 1 | |||||
10. Less time | 2.0 | 3.35 | 0.99 | -0.34 | -0.41 | -.06 | .02 | .16 | -.02 | -.19 | -.13 | .25 | .17 | .26 | 1 | ||||
11. Personal touch | 1.0 | 3.38 | 1.19 | -0.14 | -1.12 | -.32 | -.24 | -.20 | -.30 | -.34 | -.32 | .28 | .40 | .43 | .40 | 1 | |||
12. Another doctor§ | 0.5 | 2.13 | 0.98 | 0.98 | 0.92 | -.35 | -.28 | -.30 | -.33 | -.47 | -.27 | .29 | .26 | .42 | .32 | .50 | 1 | ||
13. Answer honestly§ | 1.5 | 4.42 | 0.68 | -1.36 | 3.36 | .40 | .19 | .32 | .23 | .50 | .28 | -.07 | -.19 | -.20 | -.12 | -.20 | -.35 | 1 | |
14. No pat info§ | 4.5 | 4.26 | 0.70 | -0.97 | 2.15 | .35 | .19 | .19 | .20 | .25 | .23 | -.04 | -.03 | -.06 | -.06 | -.06 | -.12 | .34 | 1 |
*% Miss, % missing response.
†Full item statements are provided in
‡Scale of 1 (strongly disagree) to 5 (strongly agree).
§Skewed items.
The item means and standard deviations were acceptable, while three items were skewed (
On conducting the PCA, the first 10 eigenvalues were 4.7, 2.1, 1.1, 1.0, .85, .76, .69, .53, .50, and .44. Four factors emerged with eigenvalues greater than or equal to one, accounting for 63.7% of the total variance. Based on the scree plot, either a three-factor or four-factor solution was indicated. We considered three-, four-, and five-factor solutions, and the four-factor solution yielded the most interpretable results. A summary of the PCA with varimax rotation is presented in
Summary of principal component analysis with varimax rotation
Item | Factor Loadings |
|
|||
Benefits | Privacy |
Interaction |
Interest | ||
1. Computers will help doctors with routine lifestyle questions | .79 | .74 | |||
2. The computer is a good way to ask lifestyle questions | .79 | .65 | |||
3. It would save doctors time. | .78 | .65 | |||
4. Doctors will make better assessments with such computer systems | .74 | .60 | |||
5. I would feel comfortable answering questions on a computer | .58 | .61 | |||
6. Computers can be trusted | .54 | (.41)† | .51 | ||
7. I would worry about confidentiality | .82 | .69 | |||
8. I do not want certain information about me on the computer | .81 | .69 | |||
9. Too many mistakes will be made with computer | .60 | (.39)† | .55 | ||
10. Doctors would spend less time with patients | .81 | .71 | |||
11. There will be loss of personal touch of a doctor | .69 | .67 | |||
12. I would find another doctor | .63 | .57 | |||
13. I would want to read patient information sheet | .80 | .67 | |||
14. I would answer honestly | .74 | .62 |
*
†Item shared loading between factors above the critical value.
Variances accounted for by the four identified factors (Benefits, Privacy-Barrier, Interaction-Barrier, and Interest) after the rotation were 33.6%, 15.0%, 8.0%, and 7.2%, respectively. The item “Computers can be trusted” in the first factor (Benefits) shared loading (.41) with the second factor (Privacy-Barrier) above the critical value of .38. Also, the item “Too many mistakes will be made with computer” in the second factor (Privacy-Barrier) shared loading (.39) with the third factor (Interaction-Barrier) above the critical value.
The Benefits factor consisted of six items with factor loadings ranging from .79 to .54. The items loading on this factor cover perceived benefits toward the quality of medical consultation and means of achieving the benefits. The Privacy-Barrier factor consisted of three items dealing with patient concerns about privacy, with loadings ranging from .82 to .60. The Interaction-Barrier factor consisted of three items covering patient concerns about interference in the interaction with the physician, with loadings ranging from .81 to .63. Although the Interest factor consisted of only two items, both items had strong factor weightings (ie, .80 and .79). The stability of this factor was also apparent during execution of the five-factor solution. Both items of this factor continued to load together while the fifth factor consisted of one item pulled from the Interaction-Barrier factor.
To estimate internal consistency reliability, we considered the following criteria for each subscale: (1) an item-total correlation of at least .3 for all items, (2) no increase in the Cronbach alpha coefficient if an item was deleted, and (3) general acceptability of the item means and standard deviations. All three criteria were met for the subscales (
Internal consistency of the subscales
Item | Mean (SD)* | Corrected |
Cronbach Alpha |
|
|||
1. Computers will help doctors with routine lifestyle questions | 3.9 (0.87) | .77 | .80 |
2. The computer is a good way to ask lifestyle questions | 3.7 (0.94) | .62 | .83 |
3. It would save doctors time. | 3.7 (0.91) | .61 | .83 |
4. Doctors will make better assessments with such computer systems | 3.3 (0.85) | .59 | .83 |
5. I would feel comfortable answering questions on a computer | 3.9 (0.85) | .66 | .82 |
6. Computers can be trusted | 3.4 (0.95) | .57 | .84 |
|
|||
7. I would worry about confidentiality | 3.3 (1.1) | .53 | .54 |
8. I do not want certain information about me on the computer | 3.4 (1.1) | .52 | .55 |
9. Too many mistakes will be made with computer | 2.6 (0.84) | .46 | .64 |
|
|||
10. Doctors would spend less time with patients | 3.4 (0.99) | .42 | .66 |
11. There will be loss of personal touch of a doctor | 3.4 (1.2) | .56 | .47 |
12. I would find another doctor | 2.1 (0.97) | .49 | .57 |
|
|||
13. I would want to read patient information sheet | 4.3 (0.70) | .34 | – |
14. I would answer honestly | 4.4 (0.63) | .34 | – |
*Scale 1 to 5: strongly disagree, agree, not sure, agree, strongly agree.
†Adjusted reliability coefficient, adjusted to compare to scales with six items.
The Cronbach alpha coefficients for the four subscales Benefits, Privacy-Barrier, Interaction-Barrier, and Interest were .85, .70, .67, and .50, respectively. There was no increase in Cronbach alpha if items were deleted from the first three subscales. This analysis did not apply to the Interest subscale as it had two items only. The item-total correlation for the subscales Benefits, Privacy-Barrier, and Interaction-Barrier ranged from .77 to .57, .53 to .46, and .52 to .44, respectively. We also calculated the reliability coefficients adjusted for the length of subscale [
Scale reliability over time was assessed with the test-retest data (n = 52). At T2, 52 patients were successfully reached out of 145 T1 participants who consented to the second contact. The reduced participation at T2 was due to (1) the study requirement that the second administration of the CLAS occur within 2 weeks of the first administration, and (2) the fact that many patients were difficult to reach because they had provided telephone numbers at work. The T2 participants were similar to the other T1 participants (n = 150) on sociodemographic characteristics, including age, country of birth, number of years lived in Canada, education level, employment status, income, English language abilities, access to computers, computer use in the last month, relationship status, experiences of intimate partner violence, number of visits to family practice, and perceived health. However, the T2 participants were less likely to be employed than participants who consented but could not be reached for second contact (
To evaluate validity of the derived constructs, several hypotheses were formulated based on a literature review. We hypothesized that the Benefits factor would be positively associated with participants’ frequent use of computers as greater familiarity with computers is likely to increase peoples’ comfort and perceptions of the benefits [
The hypotheses were tested by correlation analyses. The Benefits factor was positively associated with poorer self-perceived health and intimate partner victimization (
As hypothesized, the Privacy-Barrier and Interaction-Barrier factors had positive significant associations with participants’ non-Canadian-born status (
The CLAS has demonstrated good preliminary psychometric properties and shows promise as a tool for assessing patient attitudes toward computer-based health-risk assessments. Each of the four latent constructs or derived subscales of the CLAS had good internal consistency that exceeded the recommended threshold of 0.7 [
The use of a psychometrically validated scale is an essential element in facilitating clinical and policy decisions about the application of computer-based health-risk assessments. This is of particular importance for sensitive health risks and conditions where superiority of computer-based risk assessments over personal interviews is already well documented with respect to patient disclosure of socially sensitive information. These health risks and conditions include behaviors related to sex, alcohol, drugs, HIV, and violence [
The findings also highlight the complex nature of human behavior. Study participants perceived barriers in two distinct ways: barriers regarding privacy and barriers regarding interaction with physicians. At the implementation level, this underscores the need to measure both domains to understand and thereafter address effective use of computer-based health-risk assessments. At the theoretical level, this distinction is novel to the original conception of the scale. Possibly, patient attitudes have taken specific forms with the increasing use of computers. Recent studies reveal that use of the Internet for health information influences the way people relate to physicians, make medical decisions, and access health services [
Our post hoc analyses indicate that study participants who were immigrants or had lower socioeconomic status perceived more barriers. This raises two critical questions: (1) Is this an extension of the “digital divide?”, and (2) What does it mean for implementation? The term “digital divide” stems from research and refers to “decreased access to information technologies, particularly the Internet, for racial and ethnic minorities, person with disabilities, rural populations, and those with low socioeconomic status” [
Several limitations of this study should be noted. The CLAS predominantly measures the decision-making aspect of human behavior, though it has relevance for research on explaining and changing behavior regarding computerized assessments. Future studies should explore other aspects such as patient self-efficacy and cue-to-action. The construct of Interest would also benefit from further conceptual development. Further, our analysis of the construct validity is post hoc in nature. Many of the correlations were not strong even when significant. This is possibly due to our convenience-based use of a larger survey to select variables which in turn had a more distal than proximal relationship with the CLAS constructs. Although we found support for most of our hypothesized relationships, the Benefits subscale was not associated with the participants’ use of computers, contrary to our hypothesis. The study sample was relatively more educated than the average general population, and 87% of the participants had access to computers at home or work; almost a similar proportion reported using the computer every day or at least two to three times a week. Perhaps frequent use of computers makes people think critically about their advantages and disadvantages, leading to a cautious assessment of their benefits. On other side, it is also possible that computers have now become part of our everyday life and their benefits are taken for granted, reducing the level of perceived benefits seen a few years ago. Future research with larger samples should examine this further and establish the construct validity with a priori selection of variables. Also, it will be important to conduct a classic multitrait-multimethod study in which the four constructs on the CLAS are assessed via different methods (eg, peer ratings, behavior observations). This type of study will provide evidence for both convergent and discriminate aspects of the CLAS construct validity.
Caution is warranted regarding the generalizability of our study findings. We evaluated psychometric properties of the CLAS with female patients only. A future study involving both men and women is needed to ensure its applicability to all patients visiting primary health care settings. Further, patients were recruited from a single site. However, the collaborating clinic had several physicians and served a large number of diverse patients with estimated annual visits of 50,000. The study obtained a high response rate and, reassuringly, the participants were similar to females residing in Toronto in terms of immigration and marital status [
This study of patients in a family practice setting advances our understanding of the properties, applicability, and generalizability of the CLAS. This is an important improvement over previous assessments of other scales that relied on samples of convenience or were not specific to patient populations. Furthermore, the phrasing of items in the CLAS is expected to allow people from different ethnocultural backgrounds to reply in a meaningful way, unlike some other existing scales. At the same time, future research with a heterogeneous sample is needed to enhance its generalizability by gender and socioeconomic status while examining the utility for low and high users of computers. In conclusion, this study is a step toward facilitating research and interventions for promoting patient acceptance of computer interactive technology.
The study contributed to doctoral and fellowship training of Farah Ahmad, funded by the Canadian Institutes of Health Research (No. 17744), the Institute of Gender and Health, and Ontario Women’s Health Council. The authors wish to thank Wendy Levinson, thesis/fellowship supervisor, and Donna E. Stewart, thesis advisor, for their conceptual contributions. All in-kind support provided by the Centre for Research on Inner City Health, The Keenan Research Centre in the Li Ka Shing Knowledge Institute of St. Michael's Hospital, is much appreciated.
None declared.
Computerized Lifestyle Assessment Scale
intraclass correlation
principal component analysis