This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
The current “Millennial Generation” of college students majoring in the health professions has unprecedented access to the Internet. Although some research has been initiated among medical professionals to investigate the cognitive basis for health information searches on the Internet, little is known about Internet search practices among health and medical professional students.
To systematically identify health professional college student perspectives of personal eHealth search practices.
Q methodology was used to examine subjective perspectives regarding personal eHealth search practices among allied health students majoring in a health education degree program. Thirteen (n = 13) undergraduate students were interviewed about their attitudes and experiences conducting eHealth searches. From the interviews, 36 statements were used in a structured ranking task to identify clusters and determine which specific perceptions of eHealth search practices discriminated students into different groups. Scores on an objective measure of eHealth literacy were used to help categorize participant perspectives.
Q-technique factor analysis of the rankings identified 3 clusters of respondents with differing views on eHealth searches that generally coincided with participants’ objective eHealth literacy scores. The proficient resourceful students (pattern/structure coefficient range 0.56-0.80) described themselves as using multiple resources to obtain eHealth information, as opposed to simply relying on Internet search engines. The intermediate reluctant students (pattern/structure coefficient range 0.75-0.90) reported engaging only Internet search engines to locate eHealth information, citing undeveloped evaluation skills when considering sources of information located on the Internet. Both groups of advanced students reported not knowing how to use Boolean operators to conduct Internet health searches. The basic hubristic students (pattern/structure coefficient range 0.54-0.76) described themselves as independent procrastinators when searching for eHealth information. Interestingly, basic hubristic students represented the only cluster of participants to describe themselves as (1) having received instruction on using the Internet to conduct eHealth searches, and (2) possessing relative confidence when completing a search task.
Subjective perspectives of eHealth search practices differed among students possessing different levels of eHealth literacy. These multiple perspectives present both challenges and opportunities for empowering college students in the health professions to use the Internet to obtain and appraise evidence-based health information using the Internet.
The Internet continues to be widely used to facilitate research and learning for health and medical information. Eight out of 10 Internet users look online for health information, making it the third most popular Web activity next to checking email and using search engines [
Obtaining health information using the Internet involves a variety of competencies that health information seekers generally lack [
Increasingly, health and medical professionals must use at least basic eHealth literacy skills to perform their job-related responsibilities [
College students who are professionally trained in the health and medical professions should be taught the knowledge and skills necessary to conduct advanced eHealth information searches on the Internet. These search tasks are complemented by critical appraisals of both the information content and source [
Hanik and Stellefson [
In light of these preliminary research findings, it is important to better understand how personal eHealth search practices are perceived among health and medical professional students. These insights may provide a context for determining the types of characteristics that predict and explain eHealth literacy achievement within this population. The purpose of the current research study was to systematically identify health professional college student perspectives of eHealth search practices. The current study addressed three research questions in hopes of achieving this research aim:
1. How many clusters of health professional college students exist, given information about perceptions of personal eHealth search practices?
2. Which college students belong to the eHealth search clusters that emerge?
3. Which specific perceptions of personal eHealth search practices provide the basis for differentiating the clusters that emerge?
To systematically identify health professional college students’ perspectives of their own eHealth search practices, Q methodology [
In Q-method research, participants are the independent variables and the text-based statements they are asked to evaluate are the subject of analysis. Participants are asked to systematically order (or “Q sort”) text-based statements presented to them according to how those statements fit into their own belief system regarding how they believe themselves to be. After participants sort the text-based statements presented to them, the Q method seeks to identify patterns embedded within the Q sorts completed by different participants. Any existing patterns suggest intersubjective orderings of beliefs shared among participants, thus revealing social perspectives [
In the context of the current study, it was hoped that the Q method would help detect any qualitative patterns within undergraduate health professional students asked to consider beliefs about their own personal eHealth search practices. Specifically, the researchers were interested in whether the intersubjective orderings of eHealth search beliefs were common among participants possessing distinct levels of eHealth literacy (eg, basic, intermediate, and proficient). To facilitate this analysis, the Q study protocol was split into three sequential steps: (1) development of the concourse, (2) facilitating the Q sort procedure, and (3) interpreting data from the Q sorts.
In Q methodology, a “concourse” is the list of statements that sufficiently represents the “universe of viewpoints” about a topic [
The 12 statements informed by ACT were written on index cards, color coded, and numbered and each student was given corresponding index cards to write open-ended responses to each statement. For example, each student was asked to respond to the statement, “List the source you use most when you search for health information on the Internet.” After all participants responded to each statement, 504 unique statements (42 students × 12 statements) were generated. Repetitive responses were removed, and a literature review [
After the final concourse was developed, a subset of 36 representative statements (known as a “Q sample”) was selected to provide a miniature depiction of the larger concourse. This practice is suggested when using Fisher’s experimental design principles in Q methodology [
Statements used for participant Q sorts.
Statement # | Statement content |
1 | I use Web sources that are easy to cite. |
2 | I rely on search engines (eg Google, Bing) to find health information for research projects. |
3 | I have been taught how to find reliable health information on the Internet. |
4 | I have had assignments that required me to evaluate online health information sources. |
5 | I use up-to-date information for assignments that require me to find health information online. |
6 | I use the library databases (eg, EBSCO or CSA) when I search for information. |
7 | I seek help from library staff for difficult Web searches. |
8 | I get feedback from professors regarding the quality of Web resources I use for homework assignments. |
9 | I check the ending of Web addresses (eg, .com, .gov, or .edu) when I search for information online. |
10 | I consider the source of online information when I find useful information for my research projects. |
11 | I usually have at least one assignment per semester that requires me to conduct a Web search for health information. |
12 | I brainstorm to help figure out the health information that is important for my project. |
13 | I know how to critically evaluate online health information sources. |
14 | I evaluate online health information that I use for projects such as research assignments. |
15 | I finish research projects, such as papers, at least one week before their due dates. |
16 | I look for up-to-date online health information when I conduct Web searches. |
17 | I go to the library when I start a research project. |
18 | I can figure out how to find information that is unfamiliar to me. |
19 | I know where to find reliable online health information. |
20 | When I am assigned to complete a research paper, I do not hand in the first draft as the final product. |
21 | I use search engines (eg, Google or Bing) when I search for health information online. |
22 | I get flustered looking for health information I know little or nothing about. |
23 | I find it difficult understanding new health information found on the Internet. |
24 | I do not know where to find reliable online health information. |
25 | I know how to use Boolean operators. |
26 | I know what is meant by “peer review.” |
27 | I am confident in my ability to find reliable health information online. |
28 | I use health information that I can easily understand. |
29 | I know what Boolean operators are. |
30 | I have difficulty finding information when I use library databases such as EBSCO or CSA. |
31 | I evaluate health information sources when conducting health information searches on the Internet. |
32 | I know what a primary source is. |
33 | I go to my professor for help to make sure I use quality health information for research projects. |
34 | I follow references back to the original source when I find online research studies/reports that are useful for research assignments. |
35 | I can find useful sources of health information using the library database. |
36 | I use refined search parameters to narrow my online health information searches. |
These representative statements were then rank-ordered by the study participants in what is known as a Q sort task. To complete the Q sort, participants were instructed to order the statements according to which statements described them the most and which described them the least when considering their attitudes and experiences conducting eHealth searches. This encouraged participants to sort the cards such that the completed sort would have the shape of a triangle. Columns at both extremes of the triangle possess one card and each column incrementally closer to the center possesses an additional card, with the middlemost column containing 6 cards (thus resembling a quasi-normal distribution). Each participant’s Q sort consisted of 11 columns. The leftmost column was assigned a score of –5 (least descriptive) and the rightmost column was assigned a score of +5 (most descriptive).
In order to make the overwhelming task of rank-ordering 36 statements more manageable, participants were instructed to read all the statements first to get an impression of the range of statements they were asked to evaluate, and then they were asked to sort the cards into 3 distinct piles: one pile for statement cards that described them the least, one pile for cards that did not describe them at all, and one pile for statement cards that described them the most. Participants were then told to take the cards that were least descriptive and order them according to the pattern depicted in
Once the ranking task was completed, each card was assigned a score based on the column position it occupied (see
Final distribution of Q sort procedure (Q sort table).
To recruit participants to complete the Q sort procedure, personalized emails were sent to a convenience sample of 20 undergraduate health education students who had recently participated in the aforementioned study assessing eHealth literacy among college students [
In Q-method research, factor interpretation frequently involves considering relevant independent variables to determine characteristics that may be shared among clusters of participants [
Data from participants’ Q sorts of the 36 statements were analyzed using Q-technique exploratory factor analysis (EFA) [
The final step in the analysis involved an effort to determine which (if any) of the 36 statements provided a basis for differentiating the clusters of students identified (ie, the factors). Factor scores were computed for each statement and plotted for each retained factor to determine the extent to which each cluster of students agreed or disagreed with how descriptive each statement was regarding their own perspectives on conducting eHealth searches. Factor scores less than –1.0 and more than +1.0 were more than one standard deviation from the factor score mean, so these were the statements of least or most agreement among the individuals defining the factors [
A total of 13 students agreed to participate in the Q study following recruitment (response rate = 65%). All participants were female and the majority (8/13, 77%) were third- and fourth-year students (ie, juniors and seniors) majoring in health education with an emphasis in the allied health professions. This number of participants was judged to be sufficient given that Q-method research requires the number of participants be small relative to the number of ranked variables [
Demographic characteristics of participants.
Characteristic | n (%) | |
|
||
Female | 13 (100) | |
|
||
Freshman | 1 (8) | |
Sophomore | 4 (31) | |
Junior | 2 (15) | |
Senior | 6 (46) | |
|
||
Allied health | 12 (92) | |
Community health | 1 (8) |
The Q-technique EFA of the 36 statements yielded 3 factors representing 3 salient perspectives among study participants. The 3-factor structure suggests there were 3 types of health education students with regard to eHealth search practices. The varimax-rotated factor pattern coefficients (ie, the correlations between each participant with each of the 3 factors) suggested that the 3 factors were reasonably independent of one another.
Factor pattern/structure coefficients for participants.
Participanta | Proficient resourceful | Intermediate reluctant | Basic hubristic |
P1 | 0.56b | 0.24 | 0.46 |
P2 | 0.67b | 0.12 | 0.25 |
P3 | 0.73b | 0.26 | 0.47 |
P4 | 0.80b | 0.31 | -0.26 |
I3 | 0.71b | 0.24 | 0.36 |
I4 | 0.78b | -0.01 | 0.38 |
I5 | 0.60b | 0.35 | 0.36 |
I1 | 0.11 | 0.89b | 0.09 |
I2 | 0.34 | 0.85b | 0.05 |
B3 | 0.20 | 0.75b | 0.39 |
B1 | 0.23 | 0.19 | 0.54b |
B2 | 0.08 | 0.40 | 0.74b |
B4 | 0.37 | -0.13 | 0.76b |
a P = proficient group, I = intermediate group, B = basic group
b Pattern/structure coefficients above 0.50
The first factor was correlated with all participants who were proficient achievers on the RRSA-h and with3 participants who were intermediate achievers. The second factor was highly correlated with two participants from the intermediate group and one participant from the basic group. The third factor was correlated with the 3 remaining participants in the basic group. Thus, after analyzing quantitative performance on the RRSA-h in relation to findings from the Q-technique EFA, it was determined that perspectives of personal eHealth search practices did, in fact, differ among health professional students of basic, intermediate, and proficient eHealth literacy. More than half of the students (7/13, 54%) clustered on the proficient factor, while 3 students clustered on each of the 2 other factors. The authors determined that the magnitude of the differences between the primary and cross loadings for each participant across each factor were large enough (≥ 10% difference) to consider each participant as a defining individual for the factor with their largest pattern/structure coefficient.
The factor scores [
Salient statements for retained factors less than –1 and greater than +1.a
Statement # | Proficient resourceful | Intermediate reluctant | Basic hubristic | |
2 | -1.96 | 1.34 | ||
3 | 1.99 | |||
6 | 1.85 | |||
7 | 1.12 | -1.38 | -1.11 | |
8 | -1.07 | |||
9 | 1.37 | |||
11 | 1.21 | |||
12 | 1.01 | -1.50 | ||
13 | -1.23 | |||
15 | -1.64 | |||
16 | 1.15 | |||
17 | -1.37 | |||
18 | 1.24 | |||
19 | 1.83 | |||
20 | 1.53 | 2.04 | -1.79 | |
21 | -1.09 | 2.31 | ||
22 | -1.49 | |||
23 | -1.08 | |||
24 | -1.25 | -1.04 | ||
25 | -1.66 | -1.56 | ||
26 | 1.62 | |||
27 | 1.64 | |||
28 | -1.29 | 1.87 | ||
29 | -1.75 | -1.52 | ||
32 | 1.28 | |||
34 | -1.16 |
a Factor scores between –1.0 and +1.0 were removed from table
The proficient resourceful students (PRS) described themselves as relying on multiple resources to obtain eHealth information (Statements 2, 6, and 12), as opposed to simply relying on Internet search engines to conduct Web searches (Statements 2 and 21). They also indicated that they worked well with research partners (including library staff members) to brainstorm ideas for research projects and seek further assistance to conduct difficult Internet searches (Statements 7 and 12). Conversely, intermediate reluctant students (IRS) reported relying solely on Internet search engines when conducting eHealth searches (Statements 2 and 21). The IRS group also described themselves as working more independently with less reliance on using library resources or instructors to obtain assistance when searching (Statements 7, 8, and 17).
The PRS group described being able to search for up-to-date, even unfamiliar, health information on the Internet (Statements 16, 22, and 24), whereas IRS tended to limit the breadth of their eHealth searches, tending not to seek out original documents from the reference sections of books and manuscripts (Statement 34). Furthermore, IRS perceived themselves as lacking critical skills for evaluating sources of eHealth information (Statement 13). Both types of students reported not knowing what Boolean operators were or how to use them to effectuate eHealth searches (Statements 25 and 29).
Basic hubristic students (BHS), like their IRS counterparts, preferred to search for eHealth information independently (Statements 7 and 12). They also reported being procrastinators who were more likely to identify with submitting a first draft as a final research product (Statements 15 and 20). However, BHS were the only participants to strongly identify with (1) receiving some instruction and practical experience conducting health research on the Internet (Statements 3 and 11), and (2) possessing confidence when attempting to locate and recognize reliable eHealth information, even when researching an unfamiliar topic (Statements 18, 19, 23, 24, 26, and 27).
Q methodology was chosen as a robust qualitative technique to measure the subjective perspectives of eHealth search practices among undergraduate students enrolled in a health professional degree program. This study applied a Q sort technique to identify clusters of students representing different levels of eHealth literacy. Each cluster was described in terms of perceived skill level, confidence in ability to conduct eHealth searches, and past educational experiences. Three distinct viewpoints were revealed concerning perceptions of eHealth search practices among different “types” of students. These three viewpoints were found to share some common elements, especially when considering participants’ personal eHealth literacy (ie, basic, intermediate, or proficient). In addition, the specific similarities and differences between student clusters are useful to consult when determining which component eHealth search skills are typical among different types of undergraduate health professional students. The following discussion makes use of the distinctive statements identified above to shed light on how results from this Q-method study might be used to suggest implications for the instruction of different “types” of college students majoring in the health professions.
The PRS described themselves as students who relied on multiple resources to obtain eHealth information, as opposed to simply relying on Internet search engines. They worked well with library personnel to brainstorm ideas for research projects and sought guidance on how to conduct difficult searches. A previous study on eHealth search tendencies among college students noted that students are apt to seek digital health information from multiple, complementary sources of information [
The IRS were more reliant on Internet search engines to conduct eHealth searches. College students have reported using Internet search engines almost exclusively to locate online health information [
The IRS also perceived they lacked evaluation skills when considering sources of eHealth information. Previous work has shown perceived ability to evaluate eHealth information to correspond with actual evaluation ability among college students in the health professions [
Interestingly, neither PRS nor IRS reported receiving formalized training on how to search for quality health information on the Internet. There are a variety of competencies associated with obtaining eHealth information, including the knowledge, skills, abilities, and other attributes (KSAOs) necessary to conduct basic and advanced information searches, apply Boolean operators to limit search results, and understand (sometimes ambiguous) eHealth terminology. These KSAOs may be limited, even among high-performing students. Previous research has shown that college students are not equally capable of accessing health information online [
Navigating through health information obtained on a mobile device can present the user with unique Internet search and retrieval obstacles that are separate and distinct from searches of the Internet on a desktop CPU. Novel coursework in media literacy can assist in training health professional students living in the digital age to access and use health information available in the new age of mHealth applications. More practical continuing education and learning experiences should be provided to both instructors and students alike to ensure that mobile and digital technologies are included as a subtopic of eHealth literacy. It is important that attention is given to supporting instructional programs using mHealth applications within public health interventions.
The BHS, like the IRS, preferred to work independently when searching for eHealth information. Students who possess inferior skills searching for and evaluating eHealth information should be encouraged to seek out consultation during the eHealth information-seeking process. Other research indicates that simply observing the thought processes and search tendencies of higher-level students could indirectly result in better learning outcomes [
The BHS were the only participants to describe themselves as receiving instruction on how to conduct Internet searches for health information. Furthermore, the BHS described themselves as possessing confidence in their ability to find and recognize reliable eHealth information even when researching an unfamiliar topic. This is one major distinguishing characteristic that separated low performers from the more advanced student clusters. Speculation in previous research [
Practical applications for training allied health students using the dimensions of eHealth literacy.
Dimension | Definition | Practical applications for training |
Media | Skills to apply cognitive process and critical thinking to media messages | Provide opportunities for students to gather and assess health information from a variety of media sources. |
The authors suggest instructors of courses utilize the media literacy lesson plans created as part of student reporting labs at PBS (http://www.studentreportinglabs.com/lesson-plans). | ||
Information | Skills to know where to go to find the appropriate information and how to use the information once collected | Information literacy skills should be incorporated very early into the curricula. The authors suggest readers review Kingsley et al [ |
Computer | Skills to be able to use computers to solve problems | Provide online or hybrid computer literacy training that requires students to become more comfortable with using computers. |
Provide assignments that require allied health students to conduct Internet searches and validate the information found. | ||
Scientific | Skills to understand the political and sociological dimensions of science | Require a research-training component as part of all allied health degree programs. |
Health | Skills to understand health information and make appropriate health decisions | Incorporate training within allied health classes on how students use valid and reliable health information from sources to make health decisions. |
Incorporate the free, online Health Resources and Services Administration (HRSA) training (http://www.hrsa.gov/publichealth/healthliteracy/index.html). |
Participation in the current study was limited to a convenience sample of female respondents. It is important to note that this limitation was reflective of the disproportionate number of female students enrolled in this particular health education major, and the literature has indicated that female students are more likely than male students to use the Internet to locate health information [
There was also an uneven number of first- and second-year students versus third- and fourth-year students. This potentially skewed the results considering that the more senior students likely possessed more experience searching for health information on the Internet. Because of the small, nonrandom samples characteristic in Q methodology, findings may not be broadly applicable to various groups of undergraduate health professionals. The current Q study can only be considered representative of the continuum of perspectives that may exist about eHealth search practices among undergraduate female health education students attending a large, research-oriented university. Eliciting subjective perspectives of personal eHealth search practices at multiple types of college and universities, representing schools of diverse backgrounds (eg, different research and teaching institutions) with varied allied health and medical specialty areas (eg, nursing, physical therapy, public health, and physician assistants), may very well result in different perspectives emerging. To fully develop population validity for the variety of students in the health and medical professions, future studies should examine the link between health profession major area of emphasis and perceptions of eHealth search practices.
Although the Q technique has strengths, such as enabling comparisons across subjective topics [
As well, study participants had already completed the RRSA-h assessment and also received feedback on their performance before completing their individual Q sorts. This represented a testing threat to the internal validity of results from this study because feedback on the RRSA-h may have altered students’ perceptions of personal eHealth search skills along with their perceived need for instruction and assistance when searching for eHealth information. Finally, the reliability of the Q sorts could not be verified by a test–retest procedure due to time limitations inherent within the research protocol.
The context where eHealth literacy is applied and understood is dynamic and evolving [
atomic components of thought theory
basic hubristic students
exploratory factor analysis
grade point average
intermediate reluctant students
knowledge, skills, abilities, and other attributes
proficient resourceful students
Research Readiness Self-Assessment-Health
skills, abilities, and other attributes
None declared.