We searched the top 3 most used search engines: Microsoft Bing [44], Google [45], and Yahoo [46-48] using the keywords “delirium” and misspelled “delerium” [49] to ensure comprehensive coverage of websites addressing the topic. “Delirium” is the primary and widely recognized term in both clinical and public domains, allowing us to retrieve the most relevant content. Including the misspelling “delerium” accounted for potential variations in user input, helping to capture additional website content that might not be optimized for the correct spelling. We disabled location identifiers, conducted the searches using a newly launched incognito or private window, and cleared cookies before each search to ensure search results were not influenced by precise geographic location or search history. We collected the top 200 search results from each search engine to capture the most relevant and widely accessed websites while maintaining consistency across engines. Search engines, such as Microsoft Bing, Google, and Yahoo, use sophisticated algorithms to rank search results based on factors such as relevance to keywords searched, content quality, and user engagement. In this study, while acknowledging that search results are typically personalized and may vary for each user based on factors such as location, search history, and preferences, for the purposes of analysis, we assumed that all users received the same search results. After removing duplicates, we excluded websites if they met any of the following exclusion criteria: (1) does not provide information on delirium, (2) written in a language other than English, (3) retrieved URL linked to a media source, (eg, YouTube channel or podcast), (4) retrieved URL targeted researchers or health care providers (eg, research articles), or (5) retrieved URL linked to a website that produced an error message, or content not available without a subscription. Despite disabling location identifiers, it is possible that search engine algorithms used IP addresses or language settings and identified the delirium association that is most closely associated with our location (ie, the American Delirium Society). As such, we also evaluated the website of the delirium association (European Delirium Association), which was not identified by the search (Figure 1).

A team of patients, families, researchers, and health care providers (herein referred to as reviewers) evaluated delirium website content. The quality of website content was determined using the validated DISCERN tool (Multimedia Appendix 1) and the JAMA (Journal of the American Medical Association) benchmark criteria (Multimedia Appendix 2). DISCERN is a standardized set of criteria developed to evaluate the transparency, quality, and reliability of health information written for the public [50,51]. The DISCERN instrument has 16 questions. Each question can be scored from 1 (definite no) to 5 (definite yes). The total score can range from 16-80, wherein scores can be interpreted as excellent (63-75 points), good (51-62 points), fair (39-50 points), poor (27-38 points), and very poor (16-26 points) quality [52]. Two reviewers independently and in duplicate evaluated each website using the DISCERN instrument. DISCERN scores that differed by 5 points or more between the two reviewers were scored again by a third reviewer. The mean of all reviewers was taken as the final DISCERN score for each website. JAMA benchmark criteria include 4 standards of credible information sources: authorship (affiliations and credentials), attribution (references, sources, and copyright), disclosure (conflicts of interests), and currency (when content is posted and updated) [53]. Scores range from 0 to 4, with a score of 4 indicating the website is a credible source. Two reviewers independently and in duplicate assigned scores for each item in the JAMA benchmark criteria to indicate if it was present (score=1) or absent (score=0). Disagreements in scoring were resolved through discussion or the inclusion of a third reviewer.

There are no published guidelines for evaluating delirium education materials. To evaluate the quality of content from each website, a working group of patient partners (past ICU patients and families who are members of our research team), delirium researchers, and clinicians identified seven key points that patients, families, and the public would want to know about delirium: (1) the definition of delirium (must align with definition from the DSM-5 (Diagnostic and Statistical Manual of Mental Disorders Fifth Edition]) [54], the American Delirium Society, European Delirium Association, or Australasian Delirium Association [55-57]; Multimedia Appendix 3); (2) delirium risk factors [5]; (3) short and long-term outcomes of delirium [5]; (4) signs and symptoms of delirium [5,58]; (5) information to differentiate between delirium and dementia [59]; (6) delirium prevalence; and (7) strategies to prevent and manage delirium [13,15]. The DISCERN criteria evaluate treatment choices, so only items 1-6 (of 7) were evaluated. Two reviewers evaluated the websites independently and in duplicate to determine if the site provided complete and accurate information in these 6 areas (Multimedia Appendix 3). Each item was scored to be present (score=2), somewhat present (ie, incorrect or incomplete information, score=1), or absent (score=0). Scores could range from 0 to 12, with a higher score indicating the website had more sufficient delirium content. Disagreements in scoring were resolved through discussion or the inclusion of a third reviewer.

The level of difficulty of the reading material of each website was scored using the Simple Measure of Gobbledygook (SMOG) formula [60], the Flesch Reading Ease score [61], and the Flesch Kincaid grade level [62]. These scores are credible tools that are widely used to evaluate readability [63]. The included websites were evaluated if they were written at the American Medical Association and National Institutes of Health recommendation of a grade 6 reading level [64,65].

A standardized data extraction template was created and piloted to ensure reviewers understood how to evaluate the quality, content, and readability of each website. The template included detailed column headers to ensure each website was scored the same way (eg, guidelines for rating each DISCERN question). We also collected the following variables for each website: URL or website address, date the website was created, country, free text description of the website, website creators, and patient population or clinical setting targeted. We mapped the included websites onto the patient engagement framework that described patient engagement as inform (provision of education), activate (prompts action), or collaborate (encouraging interaction with health care providers) [3]. To ensure consistency in evaluations, all reviewers underwent comprehensive training. All reviewers read the DISCERN handbook and the study protocol, which described how to evaluate the quality, content, and readability of each website. All reviewers met to go through the evaluation criteria and to evaluate one website together. We then conducted a calibration exercise wherein all reviewers independently evaluated the same 5 randomly selected websites. We met to discuss discrepancies and reached a consensus on the overall quality of the 5 websites. The remaining websites were evaluated independently and in duplicate. Quality checks were conducted at regular intervals to verify that the criteria were being interpreted and applied consistently. Reviewers met regularly to discuss progress and address any uncertainties or discrepancies identified by the quality checks.

Website scores were summarized descriptively using counts and percentages, mean (SD), or median (IQR). The Kruskal-Wallis test was used to evaluate website scores between website categories (eg, foundation or advocacy group, government, hospital, and academic institution). Statistical analyses were performed in Stata/MP (version 14.2; StataCorp LLC). A P value of <.05 was considered significant. We compiled the top 10 websites based on the highest weighted combined quality score. Each score-delirium-related content, readability, DISCERN, and JAMA benchmark was normalized using observed (readability only) or possible score ranges. Readability was reverse-scaled because a lower score is better. We weighted readability, delirium-related content, and DISCERN at 30% each, and JAMA benchmark criteria (website content transparency) at 10%. These weightings, determined by the working group, were used to calculate a final weighted score, where a higher score indicated higher quality.

As the study did not involve human participants or human biological materials, the study was deemed exempt from the ethics board approval process. This study relied on publicly available data and therefore, ethics approval was not needed for this study as per the University of Calgary Conjoint Health Research Ethics Board guidelines [66].

On November 21, 2024, we searched Bing, Google, and Yahoo and removed duplicates and URLs that did not meet inclusion criteria; we identified 106 websites targeted toward patients, families, and the public. This included private or public hospitals (eg, Mayo Clinic: 21/106, 20%), commercial websites (eg, Merck: 20/106, 19%), and websites affiliated with a foundation or advocacy group (eg, American Delirium Society: 16/106, 15%). A description of each website is in Multimedia Appendix 4. Over half of the included URLs (62/106, 58%) used all 3 engagement strategies: inform (eg, delirium information), activate (eg, how to prevent or manage delirium), and collaborate (eg, when or how to talk with a health care provider about delirium). A total of 17 websites (16%) used inform strategies only, and 22 websites (21%) used both inform and activate engagement strategies. Of the 82 (77%) websites that reported a date when the website was posted or updated, the median time since the last update was 3 (IQR 2-5) years.

The median DISCERN score was 42 (IQR 33-50), representing fair quality. The quality ranged from 20 (very poor quality) to 78 (excellent quality). Many websites did not include a list of the sources used to compile the information on the website, refer to areas of uncertainty (eg, delirium may be missed or difficult to identify, no guideline-recommended pharmacological treatments for delirium), nor describe risks of each treatment (eg, antipsychotics if used for severe agitation). Several websites shared information that was not backed by a current evidence synthesis [5] or available guidelines [13,15]. This included claims that antipsychotics were the first line of treatment for delirium without discussion of their risk of prolonging or exacerbating delirium symptoms. Many websites mention antipsychotics as a means to manage the symptoms of delirium, such as agitation, which is a safety concern (supported by recent guidelines) [13,15]. DISCERN scores were significantly different between website categories (ie, academic, commercial, foundation or advocacy group, etc).

The median JAMA score was 1 (IQR 1-3), with 17/106 websites (16%) scoring 0, which indicates most websites lacked transparency. In particular, most websites did not list authors or contributors, their affiliations, and relevant credentials (67/106, 63%) nor included references and sources for all content (65/106, 61%). JAMA scores were not significantly different between website categories (ie, academic, commercial, foundation or advocacy group, etc). The median scores for delirium website transparency can be seen in Table 1.

The median score for delirium-related content was 8 (IQR 6-9). Nearly half of the websites included a complete definition of delirium (77/106, 73%). Website evaluators rated several websites (26/106, 25%) to have a somewhat correct definition of delirium (eg, missing items from the DSM-5 or delirium society definitions of delirium). A majority of the websites (77/106, 73%) included predisposing (eg, age and dementia) and precipitating delirium risk factors (eg, surgery and dehydration) that are consistent with available evidence [5]. Websites that reported somewhat correct delirium risk factors (23/106, 22%) reported incomplete or inaccurate risk factors. These websites either focused solely on predisposing or precipitating factors, overlooked key groups like infants and young children, or included risk factors not widely associated with delirium in the literature (eg, male sex). Most websites (83/106, 78%) included the signs and symptoms of delirium. Those websites that somewhat described the signs and symptoms of delirium (19/106, 18%) often missed a hallmark of delirium (eg, inattention, acute onset, fluctuating course, or disorganized thinking). Many websites did not include short and long-term outcomes associated with delirium (54/106, 51%). This included not describing delirium as a risk factor for dementia or the association between delirium and risk for long-term cognitive decline. Many websites did not describe the prevalence of delirium (41/106, 39%) or vaguely described its prevalence (eg, “common;” 31/106, 29%). Half of the websites stated that there was a difference between dementia and delirium (54/106, 51%). Out of these 54 websites, 43 (80%) described in detail how dementia and delirium differed. Table 2 shows the mean scores for delirium content.

The median SMOG readability score was a grade 15 (IQR 13-17, range 10-22) level. The median Flesch Kincaid Reading Level score was a grade 10 (IQR 9-12, range 5-21) level. The median Flesch Reading Ease score was 47 (IQR 38-56; range 0-80.7), indicating the websites are difficult to read for most of the population. Of the 106 websites, 5 (5%) were written at a reading level equal to or lower than grade 6.

The median for the weighted quality score for websites was 56.1 (IQR 49.3-65.5, range 28.8-84.1). Based on the normalized and weighted evaluation criteria, the top 10 websites for delirium for patient and family education are summarized in Table 3. The Mayo Clinic website, which appeared as the top result for Google and Yahoo “delirium” searches, ranked 15th overall based on weighted scores for quality, content, and readability, with an overall weighted score similar to the top 10 websites (69.8). In contrast, one of the websites listed among the top 10 search results ranked 83rd overall due to incomplete and low-quality information such as inaccurate claims (eg, counseling as a method to address disorientation, male sex as a risk factor, and a section on “confusion.”). The Wikipedia entry on delirium was in the first 10 results of Microsoft Bing (3rd), Google (5th), and Yahoo (4th). However, it had an overall weighted score of 54.8 due to its poor readability. Weighted quality scores were not significantly different between website categories (ie, academic, commercial, foundation or advocacy group, etc).

Delirium websites are one source of delirium education to prepare families to partner with delirium prevention, detection, and management. This study reports on the quality, content, and readability of websites with delirium information for patients, families, and the public. Overall, our findings suggest the quality of delirium-related website content is fair, with many websites lacking credibility and transparency. The American Medical Association and National Institutes of Health recommend that patient education materials be written at a grade 6 reading level [64,65], but nearly all websites (100/106, 95%) were written at higher than a grade 6 reading level. Furthermore, websites that families may encounter when looking for information about delirium may include incomplete information about delirium, overlook key groups such as infants or young children, or provide outdated or inaccurate information. This study identified the 10 best websites that patients, families, and clinicians can refer to, to find information about delirium.

It is crucial that high-quality and easily understandable websites on delirium are available to patients, families, and the public for several reasons. First, delirium is a common and serious medical condition that can be confusing and frightening for patients and families [4,10-13,77,78]. This may prompt families to search for delirium on the internet. Clear and concise information will help families to better understand delirium, its causes, signs, and prevention or management options. Second, countries and organizations promote engaging families with patient care [16,23,79]. To be active participants in delirium care, families must be provided with delirium education. Accessible health information can empower patients and families to participate in delirium prevention, detection, and management [20,80] and seek medical attention when necessary. Like any family engagement intervention, not all families may want to participate in delirium care, and they have reported that one of their preferential ways to receive information is through the internet. As such, websites should offer information that informs (providing information on delirium), activates (encouraging families to prevent and manage delirium), and collaborates (preparing families to discuss delirium with health care teams) with families to cater to a broad audience with varying information needs.

To improve the quality of delirium websites, it is essential to identify common deficiencies among them. A majority of the included websites did not provide proper citations for the information used in the website content. Including sources on websites benefits readers by fostering credibility, transparency, and accountability of the website information and enables readers to access additional delirium resources. Most websites (100/106, 95%) were written above the recommended grade 6 level. It is important that health information is presented in a manner that is accessible to its target so that the information is not misinterpreted and to enable the reader to make informed decisions about their health. Other studies evaluating the quality of health information on websites also report that health information is not written at an appropriate reading level [81-85]. While website creators can modify their content to meet the grade 6 recommendation, the best practice would be to codevelop websites with patients, families, and the public. Websites should also include the following delirium-related content: (1) the definition of delirium (from the DSM-5 [54] or from delirium societies or associations) [55-57], (2) delirium risk factors [5], (3) short and long-term outcomes of delirium [5], (4) signs and symptoms of delirium [54,58], (5) information to differentiate between delirium and dementia [59], (6) delirium prevalence [1,2,4-7], and (7) strategies to prevent and manage delirium [13,15]. With more people using the internet to access health information, it is imperative that website developers follow the above guidance to ensure websites include the highest quality and readable delirium information for patients, families, and the public.

This study has several implications for practice. First, there is a need for families to be integrated as partners in delirium care [86]. An important first step is to provide families with delirium information to prepare them to participate in delirium prevention, detection, and management. However, staff often lack the time to provide comprehensive delirium education to families due to heavy workloads and clinical responsibilities. Health care providers can leverage this curated list of websites to supplement delirium information provided to patients and families. By directing families to these websites, they can ensure families are consulting the highest quality and most reliable delirium information currently available on the internet. Second, it is clear that higher-quality delirium websites with plain language are needed. This can be accomplished by hospital organizations or delirium societies creating their own delirium education materials. Hospitals can leverage the set of criteria (ie, DISCERN, JAMA benchmark, readability measures, and delirium content) when adapting or creating delirium-related materials on their websites. Furthermore, there remains a need for policymakers to prioritize the importance of digital health literacy. This might include initiatives to enhance how people navigate health information on websites. This might also include advocating for high-quality educational resources.

There are several strengths and limitations that should be considered in this study. To our knowledge, this is the first study to identify and evaluate websites on delirium that could be used for patient and family education. Delirium websites were identified using the top 3 most used search engines (Bing, Google, and Yahoo). Finally, all study activities included patient and family partners. Despite these strengths, there are several limitations that should be considered. First, the search strategy consisted of only 2 search terms: delirium and the most commonly misspelled form of delirium (delerium). It is possible that patients, families, and the public may use other terms to search for delirium information, and, as such, some websites may have been missed. Second, the search was performed in Canada. Despite disabling location services, which limited the extent to which the search engines can use our location, other aspects of our location may have been inferred based on our IP address or language settings. For this reason, and that our team was proficient in English, we only included websites written in English. As such, the results of this study may not be generalizable to people who do not speak English. To attain a more comprehensive understanding of delirium-related websites in languages other than English, further research is warranted. This should include other prominent search engines with significant market shares (eg, Yandex, Baidu, Petal Search, and DuckDuckGo) and collaborative efforts with researchers from diverse linguistic backgrounds and countries. Third, the analysis of the websites was limited to what was reported, which may not have comprehensively captured the development of the website. This may include if websites used credible sources or delirium experts to compile the websites but did not cite the sources or identify the delirium experts. Finally, the tool that was used to evaluate delirium content has not been externally validated, as it was developed specifically for this study. Finally, while DISCERN, JAMA benchmark, delirium content, and readability tools are robust for assessing quality and transparency, they focus on structural elements rather than usability or accessibility for patients and families. Websites with high scores often had dense information or busy layouts that may overwhelm patients or families seeking concise and accessible information. These metrics do not account for patient or family experience, engagement, or cultural relevance, which are critical for effective patient and family education.

This study suggests delirium websites for the public are of fair quality. Inadequacies in evaluated websites, such as lack of transparency, incomplete delirium information, and poor readability, should be addressed when updating current or creating new delirium websites aimed at patients, families, or the public. Following the outlined standards for quality, delirium content, and readability will ensure high-quality, transparent, and accessible delirium information for patients, families, and the public.