Digital new media have become indispensable tools for health information and network support, notably among individuals experiencing chronic and terminal health conditions, such as many forms of cancer [1,2]. New media platforms, represented by social media and artificial intelligence (AI) chatbots, offer collaborative and participatory environments that serve as valuable resources for patients with cancers, survivors of cancer, and caregivers of patients [3,4]. These platforms facilitate the exchange of information [5], provide opportunities to seek external support [6], and share personal experiences of cancer [7]. The rise of AI chatbots has attracted considerable interest in their use for health information [8], with studies indicating that they can provide higher-quality information and more empathetic responses than physicians [9]. Further, among the US adults who report being on the internet almost constantly, specifically on social media [10], 72% report leveraging their preferred platform, and increasingly generative AI, to learn about health conditions diagnosed in themselves and their friends and family [11].

Research shows that turning to web-based outlets can improve psychosocial outcomes across various acute and chronic conditions [12]. Access to high-quality information further enhances health literacy, decision-making, health behaviors, outcomes, and health care experiences [13-17]. Conversely, low-quality information can promote trust in unproven, ineffective, and harmful treatments, as well as fake news, conspiracy theories, and misleading testimonials about “natural” or alternative cures [18]. Kington et al [19] described “high-quality information” as science-based and aligned with the best available evidence. Huang et al [20] further identified 15 key attributes of information quality, including accuracy, objectivity, accessibility, completeness, and clarity.

As cancer remains a leading cause of death globally, with rising incidence rates [21], related discussions are widespread on social media and AI chatbots [22,23], making the topic a target for commercial and political interests [24,25]. Substantial financial incentives drive the promotion of treatments [26], and patients, often desperate for hope, are particularly susceptible to misleading information circulated in digital spaces [18]. Limitations in models, databases, and algorithms pose challenges to the accuracy and credibility of the information provided by AI chatbots [27,28]. Survivors also face cognitive challenges such as fatigue, memory deficits, and reduced executive function [29,30], which may impair their ability to critically evaluate the information quality they received [31]. Consequently, low-quality information may lure patients into submitting to costly and harmful treatments, undermining the potential benefits of digital support networks [32].

To date, systematic, scoping, and other literature reviews generally report inconclusive and nondefinitive findings on the quality of cancer-related information on social media [22,33,34]. Most cross-sectional studies indicate considerable variability in information quality across platforms, content formats (ie, text-based or video-based), and cancer types [35]. This inconsistency may stem from differences in cancer types studied, platforms analyzed, and quality assessment approaches used. A critical gap remains the breadth of literature on this topic that directly accounts for (1) a wide variety of cancer types, (2) a multitude of platforms, and (3) differences in findings over time. Social media, in particular, has evolved considerably in the past two decades, giving rise to AI chatbots that now represent a new source of health information. Accounting for factors such as users’ shift in medium preference (text-based vs video-based), and platform changes (Twitter to X, and at the time of writing, the potential closure of TikTok) may reveal temporal trends that implicate improvement or worsening of information quality over time.

This paper presents a systematic review of empirical studies published between 2014 and 2023 that address cancer-related information on new media. It explores the growing reliance on these platforms for cancer health information, trends in platform use, and myriad tools used in the included studies to determine the information quality. Specifically, we addressed the following research questions (RQs): (RQ1) What are the key characteristics of studies evaluating the quality of cancer-related information on social media and AI chatbots, and how have these characteristics evolved? (RQ2) What factors influence the conclusion of quality assessments, and how do they vary across platforms, assessment tools, and cancer types? (RQ3) What patterns emerge in the assessment findings of new media cancer-related information quality?

Findings from this review contribute to further supporting our understanding of new media’s role as a source of cancer-related health information. By adopting a longitudinal perspective, the study identifies trends in quality assessment studies using data from social media and AI chatbots and offers evidence-based recommendations for future research on this vital topic.

This review was reported according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines (Multimedia Appendix 1). Our protocol was retrospectively registered on PROSPERO (CRD420251058032) on May 23, 2025.

As a review study of published studies, our research is considered nonhuman participant research and is exempt from review by an Institutional Review Board.

Between January and September 2024, we conducted a systematic literature search on four electronic databases (PubMed, Web of Science, Scopus, and Medline) for studies. Full strategies for the databases are available in Table S1 in Multimedia Appendix 2. We only included studies that underwent peer review and publication between 2014 and 2023 and presented empirical qualitative, quantitative, or computational findings in English. We also reviewed the reference lists of included studies to identify any additional publications that met the inclusion criteria but were not retrieved in the original search. Three researchers conducted independent examinations of the selection criteria, search terms, and the records returned by the search. Disagreements were resolved by discussion among the three researchers until a consensus was reached.

We initially included studies if they met three broad categories of inclusion: (1) they contained a synthesis of cancer-related new media data, (2) they used an information quality assessment to rate the quality of data along varying criteria, and (3) they were published in English between 2014 and 2023. Cancer types were defined according to the International Classification of Diseases for Oncology. Likewise, we considered three broad categories of new media for inclusion: (1) text and graph-based media, such as Facebook, Instagram, Pinterest, Reddit, Twitter (now X), WeChat, and Weibo; (2) video-based media such as YouTube, TikTok, and Xigua; and (3) generative AI-based media, such as ChatGPT, Chatsonic, Microsoft Bing AI, and Perplexity.

We excluded studies that did not present original findings, unpublished manuscripts from preprint repositories such as arXiv, conference abstracts, narratives, literature reviews, and editorials. Studies were also excluded if they discussed the quality of information but did not elaborate on how it was assessed, for example, without conducting a comprehensive and systematic data collection and analysis, without referring to any quality criteria.

Two researchers independently assessed the quality and risk of bias of the included studies using the Joanna Briggs Institute Critical Appraisal Checklist for Analytical Cross-Sectional Studies [36] and the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) statement checklist [37]. Studies evaluated by the Joanna Briggs Institute were considered to have a yes or no risk of bias or to have some concerns in study design, conduct, and analysis. The STROBE checklist assessed and illustrated the quality of reporting in key sections of studies. Additionally, we recorded whether each study reported an ethical approval process, including compliance with human and animal rights standards, informed consent procedures, and data privacy protections.

Two researchers independently rated the certainty of evidence of outcome using GRADE (Grading of Recommendations Assessment, Development and Evaluation) via the GRADEpro Guideline Development Tool online platform, including assessment domains of risk of bias, inconsistency, indirectness, imprecision, and publication bias.

Disagreements between the two raters were discussed until a consensus was reached.

We extracted the following data from the included studies: (1) authors, (2) year of publication, (3) journal name, (4) study design (ie, quantitative, qualitative, or computational), (5) sample size, (6) sample genders, (7) study location, (8) authors’ identities (individual or constitute, nonmedical professional or medical professional), (7) social media platform, (8) search strategies, (9) cancer types, (10) the language of the original data, (11) the profession of the raters, (12) contents of the posts, and (13) study results.

We extracted the information assessment tools used in the included studies, such as the DISCERN [38] tool, along with the outcomes measured by each. Despite the variation in tools, most studies assessed at least one of the following: (1) overall quality: a holistic assessment of the posts; (2) technical criteria: an assessment of the layout and functionality of the data; (3) readability and understandability: an assessment of reading and comprehension level; (4) accuracy and misinformation: an evaluation of scientific validity of the content; (5) completeness and coverage: the scope of topics; (6) actionability and usefulness: an assessment of the practical nature of the content; (7) harmfulness: whether the content can cause or promote harm, and (8) commercial bias: evidence of the direct influence of commercial interests. We categorized the 75 studies based on the tools and metrics used to evaluate the quality of information and draw comparisons across studies. Data were extracted by one researcher and independently verified by a second researcher. All analyses were performed using Stata/SE (version 18.0; StataCorp LLC) and RStudio (version 4.3; Posit PBC).

This review summarizes the body of literature on the quality of cancer-related information on new media, published from 2014 to 2023. Our initial search identified 7841 studies, including (1) 3001 records from PubMed, (2) 2032 records from Web of Science, (3) 1293 records from Scopus, and (4) 1515 records from Medline. After screening papers by title and abstract, 140 potentially eligible papers remained for full-text review. Through full-text review, we excluded 65 additional studies because they lacked analytic rigor or did not include quality of information as a component of results. Our final sample size comprised 75 individual studies. Figure 1 depicts our filtering process according to the PRISMA guidelines.

We present our results below in relation to each of our RQs.

(RQ1) What are the key characteristics of studies evaluating the quality of cancer-related information on social media and AI chatbots, and how have these characteristics evolved?

Table 1 summarizes characteristics of included studies, including binary classifications of whether certain features were present or not. More than 80% of studies followed similar practices in the search process, including documenting their search period (70/75, 93%), search tool (69/75, 92%), query terms (73/75, 97%), and only considering one search tool (62/75, 83%), single platform (67/75, 89%). A total of 20 studies assessed content in languages other than English, including Chinese (n=6), German (n=5), Arabic (n=3), Japanese (n=2), French (n=2), and Spanish (n=2), and several other languages were examined in single studies, such as Korean, Italian, Turkish, Swedish, and Danish. During the rating process, 73 (97%) studies used a nonblinded rating process, 66 (88%) studies documented the number of raters, and 69 (92%) studies relied on more than one rater. However, across studies that reported content distributor information, 14 (19%) studies defined their rating criteria a priori, and 42 (56%) studies included a rater with a medical background. In the reporting process, more than 60% of studies reported “views,” “like,” or “unlike,” and “comment” behaviors, while a few tracked posts sharing behavior (11/75, 15%). About 70% of studies reported the posters’ identity, but few provided information on their demographics (17/75, 23%). Among the included studies, 67 (89%) reported the contents or topics discussed in the posts.

A majority of studies demonstrated strong methodological quality, with 74 (98%) studies appropriately describing subjects and setting, 74 (98%) using standard condition measurement, all using valid outcome measurements, and using appropriate statistical analyses. However, we identified a considerable risk of bias in the handling of confounding factors; only 43 (57%) studies identified potential confounders and implemented strategies to address them (Figure 2A). Most studies reported key elements, while 22 (29%) studies did not include other analyses (eg, analysis of subgroup and interactions and sensitivity analysis), and 23 (31%) studies did not list funding sources (Figure 2B). Of the included studies, 34 (45%) reported ethical approval consideration, whereas 41 (55%) did not mention ethical approval. Among those that did report ethical approval, most stated that it was not required as the research only involved publicly available data, excluded human or animal participants, and removed identifiable information (eg, user IDs, links, and contact details), as is common with social media research. Only 3 studies obtained formal approval from research ethics committees [44-46]. Tables S2-S152 in Multimedia Appendix 2 [5,44-117] present details of the risk of bias and quality evaluation for individual studies.

Funnel plot and Egger test revealed significant asymmetry (bias coefficient –5.67, 95% CI –9.63 to –1.71; P=.006) only for studies evaluating misinformation, suggesting that studies could be missing in the literature that reported null or negative findings of misinformation or small study effects. No bias was detected for other indicators (Figures S1-S11 in Multimedia Appendix 2).

Since all included studies were observational in design, they were initially rated as low certainty according to the GRADE framework. Subsequently, the certainty of evidence for all evaluated outcomes was downgraded to very low due to one or more of the following reasons: high inconsistency (I²>75% in most outcomes), risk of indirectness (due to heterogeneous platforms or measurement definitions and tools), and, in some cases, imprecision (wide CIs and small sample sizes). These limitations significantly reduce confidence in the pooled estimates based on estimates. Details for each assessment and the reason for downgrading are in Table S153 in Multimedia Appendix 2.

(RQ2) What factors influence the conclusion of quality assessments, and how do they vary across platforms, assessment tools, and cancer types?

The ordinal logistic regression analysis identified several factors associated with reporting higher quality conclusions (Table 2). Video-based media (OR 0.02, 95% CI 0.01-0.12), studies on rare cancers (OR 0.32, 95% CI 0.16-0.65), and studies on combined cancer types (OR 0.04, 95% CI 0.01-0.14) were less likely to yield high-quality conclusions than text-based media and studies on common cancers.

During the search process, studies using multiple search tools (OR 0.30, 95% CI 0.13-0.73), mentioning search terms (OR 0.13, 95% CI 0.02-0.81), reporting initial hits (OR 0.14, 95% CI 0.07-0.28), sourcing content in languages other than English (OR 0.35, 95% CI 0.16-0.76), and analyzing multiple media platforms (OR 0.06, 95% CI 0.02-0.27) were less likely to report higher quality conclusions.

During the rating process, studies disclosing the number of raters (OR 0.02, 95% CI 0.00-0.14) were less likely to report high-quality conclusions. In contrast, studies including process graphs (OR 3.06, 95% CI 1.61-5.79) and using literature-based assessments (OR 2.93, 95% CI 1.04-8.20) were more likely to report higher quality conclusions.

During the report process, studies reporting engagement metrics such as likes or dislikes (OR 3.35, 95% CI 1.20-9.38), forwards or shares (OR 6.17, 95% CI 1.61-23.65), and mentioning content or topics (OR 3.77, 95% CI 1.43-9.94), were associated with higher odds of reporting high-quality conclusions.

(RQ3) What patterns emerge in the assessment findings of new media cancer-related information quality?

This systematic review and meta-analysis offer robust insight into new media-driven information quality studies pertaining to cancer. Our findings showcase a complex and diverse body of literature varying across multiple domains, including time, platform, quality assessment types, and findings therein. Several important patterns merited further discussion.

First, evidence of temporal changes highlights the evolution of media platforms over time. The studies evidenced clear changes in used platforms, reflecting their availability and popularity at the time. For example, early in the study period, we principally identified studies analyzing YouTube videos. Prior to 2021, there was also greater use of text-based media, including Twitter, Reddit, Pinterest, Facebook, and others. However, between 2021 and 2023, most studies shifted solely to video-based platforms. The shift in emphasis from text and image to video-based social media largely aligns with US patterns in that a majority of the user-generated social media content created daily since 2021 consists of videos [123,124]. This period also coincides with changes to data access policies on platforms such as Twitter (now X), which closed its application programming interface in 2022, limiting data collection efforts for research [125]. Another important point is that the second and third most popular social media platforms in the United States (Facebook and Instagram, respectively) [123] represented a small proportion of the total number of studies reviewed (6 studies on Facebook and 2 studies on Instagram). This represents a critical gap that warrants further investigation. The other temporal pattern is that the first cancer-related information quality assessment of generative AI chatbot content occurred in 2023 [117]. In the coming years, more studies are likely to expand in this area. Future research should compare the quality of information of multiple AI platforms and models—especially in the areas of response consistency and alignment with scientific best practices.

Given the proliferation of video-based social media over time, it is also critical to contextualize information quality patterns by cancer types. Our findings generally highlight that video-based cancer content is engaged but at notable risk for low-quality information. Indeed, our analysis presents high engagement (eg, views, likes, and comments) for video-based platforms, which may be due to their general accessibility, visual appeal, or targeted algorithmic marketing. This finding might suggest that cancer-related content on video-based platforms may have greater reach and visibility compared with text-based content, and empirical evidence supports this [64,126]. However, our analysis shows that information quality on video platforms is significantly lower than on text-based platforms (OR 0.02, 95% CI 0.01-0.12; P<.001), underscoring challenges in content moderation and infoveillance. Similarly, newly emerging AI chatbots exhibit potential to deliver cancer-related information of higher quality, as evidenced by higher DISCERN scores than video-based media [114,116,117]. However, limitations remain, particularly concerning understandability, actionability, and occasional misinformation [115-117], which may be driven by hallucinations. Future AI-development efforts should focus on ensuring more comprehensive and representative information, while improving the clarity and factual accuracy of AI-generated responses.

Second, this study highlights shifts in research priorities and interests, as well as changes in cancer types and the quality of information associated with them. Between 2014 and 2018, increased attention was given to certain cancer types, primarily breast, prostate, skin, and colorectal cancers. These cancers are all highly prevalent in the United States, and disparities in health care seeking, treatment, and outcomes affect each [127], which may explain their broad emphasis during this period. Studies of these cancers remained prevalent throughout the data collection period. However, between 2018 and 2023, research interest expanded to include rarer and less media-publicized cancers such as gastric, thyroid, and spinal cancers. This shift might reflect increasing interest in underrepresented cancers and the need to address gaps in communication and information quality across a wider range of cancer types; however, more research here is needed.

Our findings likewise demonstrated that information quality varied across the studied cancer types. Specifically, research focusing on rare cancers was of significantly lower quality compared to studies on common, highly prevalent cancers (OR 0.32, 95% CI 0.16-0.65). Indeed, studies of more common cancers such as skin, lung, breast, colorectal, prostate, liver, and thyroid consistently achieved the highest overall quality, understandability, completeness, and usefulness scores [67,68,71,72,78,83,100,104,107,116,117]. This conclusion may be explained by the relatively late initiation and limited volume of research focused on the quality of information regarding rare cancers.

Third, addressing the quality of cancer-related information on media platforms presents unique challenges and opportunities. Prior research attributes lower information quality on new media to several factors, including algorithmic amplification of popular content regardless of the source [128,129], brevity of text and short-form limiting content depth [130], and increased content from nonexpert creators with no moderation [131]. The literature highlights various recommendations for addressing these issues, including improving architecture for detecting and eliminating incorrect content [132], increasing mobile health (mHealth) and digital health media campaigns to promote health literacy [133], encouraging more medical entities to have a social media presence [133], and auditing risks and functionalities for AI-generated materials [134]. The success of these strategies is mixed, suggesting further work is needed to address the confounding issue of the reach and visibility of cancer information relative to information quality risk. Regardless, with the evident growth of health content on new media, timely attention to these issues remains a public health priority.

Findings from this review offer four tangible recommendations for future research. First, while it is clear that new media information quality assessments are growing and popular fields, there is limited consensus on which information quality assessment is needed relative to the task. Based on our findings, we strongly encourage future researchers conducting such assessments to consider robust and validated tools such as DISCERN, PEMAT U/A, GQS, and the JAMA-BC criteria. Although application of these tools can be time-intensive, each provides an evaluation of health content in several domains simultaneously, ensuring thorough analysis of the data. Further, consistently using these tools for information quality assessments will allow for robust and streamlined meta-analyses, which would allow for even greater insights than cross-sectional designs alone. However, we also acknowledge that even when these tools are consistently applied, variations in study objectives, media platforms, and evolving content may still pose challenges in achieving standardized assessments across different research contexts.

Second, when conducting mHealth or digital health interventions, we advise researchers to launch them through video-based apps, such as YouTube and TikTok. Comparing studies consistently revealed greater engagement with data from these sources than text-based sources, regardless of cancer type or account attributes. This directly suggests that the broad access and ease-of-use of these videos may make health-related content more readily accessible to others, regardless of barriers to care. Prior research strongly recommends using innovative strategies such as narrative storytelling or personal testimonials to boost engagement. Regardless of the strategies, it is clear from our findings that the shift away from text-based media is not a trend but part of the shifting social media landscape. At the same time, researchers should also remain abreast of new media trends and adapt accordingly.

Third, mHealth or digital health interventions should be produced by or in partnership with specialists, clinicians, and hospitals. Studies considering content posted by professional entities consistently had better information quality across inventories, cancers, and platforms. Although in some cases, content by medical professionals was not as well-engaged as content from other nonmedical accounts, these cases may reflect content posted on text-based platforms where engagement was consistently lower. Specialists with direct expertise in the topic as video protagonists—including celebrities with lived cancer experience— may help extend the credibility of the content and transcend trust barriers for certain topics, including vaccinations and routine screenings.

Fourth, our results are only starting to show the emergence of AI as a health promotion tool. Professionals in the health and medical fields should continue researching generative AI content for information quality.

Fifth, our results equally highlight the need for licensed care providers to consider the impact of new media on patient health-related decision-making. To our knowledge, few providers, including nurse practitioners, physicians, or insurance representatives, currently inquire about how much new media patients consume during consultations. However, given that our findings indicate a wide array of information quality, coupled with profuse and growing use of new media in the United States and abroad, it may be imperative for providers to inquire about use in controlled settings.

First, while we believe we offered a thorough review of the new media cancer information literature, we were not able to consider other facets that may lead to varying information quality, such as the professional level of raters, content language, nation of origin, and different criteria in the searching and rating process. As this review focused only on English-language-based studies, similar reviews focusing on studies in other languages (such as Spanish, Arabic, and Chinese) and media data (such as LinkedIn, Snapchat, and other chatbots) are needed. Second, the inconsistent nomenclature of certain information quality and measurement used may be driving high levels of heterogeneity in our meta-analysis. For example, the measure of “actionability” was sometimes called “usefulness,” depending on the study. This inconsistent nomenclature made it difficult to categorize study findings, which may have resulted in a marginally biased result. Further, even in cases where studies used the same quality assessment, the diversity of platforms, cancer types, and scope of work may have also contributed to higher levels of heterogeneity. Subgroup analyses to explore the sources of heterogeneity were limited by the small number of records included in those subgroups and the poor quality of reporting in the available data. As such, pooled findings should be interpreted as indicative rather than definitive. Third, this study is minimally representative of generative AI as an information source. Given its growing popularity and strong body of literature considering generative AI as a health communication tool, we strongly recommend further systematic reviews specifically including information quality assessments using multiple generative AI tools. Meanwhile, we found a limited number of published studies specifically addressing certain platforms, such as Facebook and Instagram, during our search period. Finally, this review was prone to publication bias among studies on misinformation and other risks of bias.

High-quality cancer information on new media contributed to informed cancer-related decision-making. However, the quality of this information remained at a moderate to moderate-low level, particularly in terms of overall quality, transparency, understandability, accuracy, actionability, harmfulness, and commercial bias. Our findings indicated that poster identity, medium (text-based, video-based, and AI media), and assessments of highly prevalent cancers were associated with higher information quality. Gaps in information quality, which were evident across all studies considered in our review, remain a critical and timely public health issue.