The Criteria People Use in Relevance Decisions on Health Information: An Analysis of User Eye Movements When Browsing a Health Discussion Forum

Background People are increasingly accessing health-related social media sites, such as health discussion forums, to post and read user-generated health information. It is important to know what criteria people use when deciding the relevance of information found on health social media websites, in different situations. Objective The study attempted to identify the relevance criteria that people use when browsing a health discussion forum, in 3 types of use contexts: when seeking information for their own health issue, when seeking for other people’s health issue, and when browsing without a particular health issue in mind. Methods A total of 58 study participants were self-assigned to 1 of the 3 use contexts or information needs and were asked to browse a health discussion forum, HealthBoards.com. In the analysis, browsing a discussion forum was divided into 2 stages: scanning a set of post surrogates (mainly post titles) in the summary result screen and reading a detailed post content (including comments by other users). An eye tracker system was used to capture participants’ eye movement behavior and the text they skim over and focus (ie, fixate) on during browsing. By analyzing the text that people’s eyes fixated on, the types of health information used in the relevance judgment were determined. Post-experiment interviews elicited participants’ comments on the relevance of the information and criteria used. Results It was found that participants seeking health information for their own health issue focused significantly more on the poster’s symptoms, personal history of the disease, and description of the disease (P=.01, .001, and .02). Participants seeking for other people’s health issue focused significantly more on cause of disease, disease terminology, and description of treatments and procedures (P=.01, .01, and .02). In contrast, participants browsing with no particular issue in mind focused significantly more on general health topics, hot topics, and rare health issues (P=.01, .01, and .01). Conclusion Users browsing for their own health issues used mainly case-based relevance criteria to relate the poster's health situation to their own. Participants seeking for others’ issues used mostly general knowledge–based criteria, whereas users with no particular issue in mind used general interest– and curiosity-based criteria.

investigate online and offline health information users. I took age, job and nationality as the basic demographic categories in this study, while putting all other categories not mentioned in the literature in the others category. The others category comprise mostly infrequent types of information such as race and location. The anonymity of online discussion forums allow users to share and exchange information without knowing personal details. In the postings, age and country were sometimes mentioned to give readers a general profile and context for interpreting the shared information.
Patient symptoms include symptom description, subjective feeling of having the symptom (condition), and patient history of disease. The categories of symptom description and patient history of disease came from previous literature. Fox and Duggan [1] found that American Internet users searched for specific disease conditions. They reported that users of an online Facebook group for diabetes posted their concerns of negative events to find out if their experience matched with those of other users. There is one category of health information not mentioned in previous literature: subjective feeling of (having) the problem/condition. During the open coding process, I found that some posts contained this category of health information-description of problem or condition based on the writer's opinion. It sometimes starts with "I feel like…" or "I think it should be …". It is different from symptom description in that symptom description is the direct description of the patient's problem or condition that has been confirmed by a health professional to be a symptom of the disease. The subjective feeling of having a symptom is a speculation of the user. Hence, this category was added in the sub-section of patient symptom.
Emotion support is another aspect of user related health information. It includes attitude to the problem, emotional status of having/knowing the problem, and other people's attitude and support. These categories all come from previous literature, but were reorganized in this study. Chuang and Yang [24] defined emotional support into the following types: relationship, physical affection, confidentiality, sympathy, listening, understanding and empathy, encouragement and prayers. However, these types are too detailed to be useful in this study. Moreover, confidentiality, listening, understanding and empathy can be categorized into other's attitude since they represent other's viewpoints. Hence, I used other's attitude and support to represent all these detailed categories. However, these categories did not take into consideration the emotional aspect of the post writer. Hence, in this study, attitude to the problem, and emotional status of having/knowing the problem were added to represent the poster's own emotions.
Patient experience of drug and treatment were not frequently found in previous literature. In prior studies, drug and treatment information usually referred to formal or authoritative drug and treatment information. In this study, patient experience of drug and treatment focuses on the user's self report of taking the drug and treatment, including self-reported feelings and effect. The self-reported feelings and drug effects may not match the effects found in clinical trials and listed on the drug labels. The category of patient experience of drug/treatment includes the following subcategories: perceived side effect, interaction with another health problem of patient, dosage used, description of used procedure, and caution or reminder from patient.

Factual/general knowledge-based health information
Factual/general knowledge-based health information refers to health information that is based on medical or scientific facts, or proven and well-known factual knowledge. This type of health information comes from medical reports, surveys, medical journals and experiments, and has higher credibility than user selfreported health information. I subdivided it into two sub-categories: etiology and drug information.
Etiology is subdivided into the cause of disease and the formal/professional description of the disease. These two categories come from the literature. Hudges and Cohen (2011) found that there are professionally reviewed health information on some websites including formal descriptions of diseases and (formal) symptoms of diseases. They also found that the medical terms sometimes were used in the description of disease and drug effects. Hence, I added terminology in this category.
Drug information refers to the formal expression of drug names, effects, side effects, drug interaction, contra-indication, etc. These categories come from prior literature of online drug information research. Kishomoto and Fukushima [22] found that Japanese health information users searched online for drug information covering the topics of side-effect, action mechanism, effect on body, effectiveness, generic drugs, drug interaction, drug dosage, and services of pharmacies. In this study, I reorganized them into the following subcategories: efficacy, indication, contra-indication, interaction with other drugs, and pharmacology. The action mechanism was replaced by pharmacology. The drug dosage has been included in the category of patient experiences of drug.

General-awareness based health information
There is another group of health information that is neither related to user's experience of health problem, nor to medical factual knowledge. I used generalawareness based health information to represent this broad category of health information. This is subdivided into: general health issue, and curiosity based health issue.
General health issue is related to public awareness and concern. During the open coding process, I found that 3 participants who had no particular health issue browsed for topics of diet, weight control, and mental health issue. Fox and Duggan [1] found that many US Internet users searched for health information about health and fitness. Hence, I used this term to represent these types of health information. It is subdivided into common health issue, pollution, smoking, and hot topic of general interest. A common health issue is not really a particular health condition, but a health area that the public is interested in, such as weight control and diet control. Pollution and smoking are well-known health-related issues. Hot topic of general interest refers to health issues that are prevalent or of major social concern, and not limited to particular diseases or topics publicized in the mass media. It includes drug safety and mental disorders such as depression.
Besides general health issues of public interest, I added curiosity-based health information in this category, subdivided into: rare issue, interesting story, famous people and counter-intuitive information. The details of the coding scheme divided into three broad categories of health information are listed in Table 1.