The research was designed as an observational cross-sectional study. Its sample consisted of 125 PCI-related websites intended for the general population in 5 languages (25 for each included language)—English, German, Hungarian, Romanian, and Russian. The Google Search engine was used to identify eligible websites, using “stent” as query term (used as such in English, German, and Romanian; “sztent” in Hungarian; and “стент” in Russian). The query terms were selected based on their popularity as shown by Google Trends, a tool that analyzes the frequency of top search queries in Google Search across various regions and languages. The links returned by the Google Search engine were screened according to a set of preestablished inclusion and exclusion criteria. To be included, a website had to address the subject of coronary stenting, presenting the information in the desired language and a minimum of 300 words. The information had to be targeted at internet users without medical education. Pages addressing subjects other than PCI, sponsored pages appearing in the top hierarchical positions in the results list, and infected or inaccessible pages were excluded. Websites consisting exclusively of audio or video content and websites allowing access only after registration or payment of a fee were also excluded. Similarly, web pages that presented the topic of interest in the form of news or comments on forums and social networks—in other words, pieces of information not meant to thoroughly present the subject of PCI—were not included in the sample. Websites deemed fit for inclusion were consecutively analyzed following a rigorous protocol, briefly illustrated in Figure 1.

The Google searches were performed in April 2019 for English, Hungarian, and Romanian language websites, while the Russian language inquiry took place in June 2019. German language websites were subsequently added to the study in November 2020. A total number of 83, 39, 121, 167, and 94 websites in English, German, Hungarian, Romanian, and Russian language, respectively, were screened until the acquisition of the 25 eligible links for each of the subsamples.

Firstly, the examination was aimed at the websites’ general characteristics and their compliance with 12 general credibility criteria derived from the eEurope 2002 core quality criteria for health-related websites supported by the Commission of the European Communities [8] (Figure 1). Next, the selected pages underwent an exhaustive evaluation of their informational content based on a topic-specific benchmark (Multimedia Appendix 1) [9]. The benchmark was developed using published literature and evidence-based guidelines on the subject of interest as sources of information, in such a way that it covered the topic of PCI to an extent considered sufficient and comprehensible for nonprofessionals. It included information on the following aspects: definitions and introductory notions about PCI, types of coronary stents, indications for the procedure, preprocedural preparation of the patient, description of the procedure, the postprocedural period, what to know or do at home, risks, benefits, costs, other treatment options, general prevention and prophylaxis methods, as well as general warnings regarding alternative treatments. To ensure a practical grading manner, the benchmark was divided logically into 50 items. Their presence on the studied websites was evaluated regarding completeness (ie, the presence of the item on the studied website, evaluated in a binary fashion) and accuracy (ie, the extent to which the item was correctly presented on the studied website, graded on a 3-point scale). The benchmark was reviewed by medical professionals, specialists in the fields of cardiology and interventional radiology, from both Romania and the United States.

The data on the websites’ general characteristics and compliance with credibility criteria were collected by one operator, while the assessments regarding the websites’ informational content were performed independently by 2 evaluators for all websites. The websites’ compliance with the 12 selected eEurope 2002 credibility criteria was assessed in a binary fashion, with 1 point given to every criterion that was met. Based on the obtained sum, the relative credibility score of the given web page was calculated as previously described by Nădăşan et al [9]. Similarly, based on the points awarded for completeness and accuracy for each website, their relative completeness and accuracy scores were computed. All relative scores were reported on a 0-10 scale. The resulting scores were categorized as very poor (0-2), poor (2.1-4), average (4.1-6), good (6.1-8), or very good (8.1-10). The analyzed data are available in Multimedia Appendix 2.

For each included website, the degree of agreement between the 2 evaluators was assessed using the Cohen kappa statistic, a test that measures interrater reliability and is regarded as more robust than simply computing the percentage of agreement, as it adjusts for agreement occurring by chance. Kappa coefficients may range from −1 to 1. A kappa value of 1 indicates perfect agreement, while a value of 0 corresponds to the rate of agreement expected by chance alone. In our study, a coefficient of less than 0.8 prompted a reevaluation to reach a consensus. The Kolmogorov-Smirnov test was used to analyze the normality of the data, based on which the comparisons of data with normal and nonnormal distributions were performed using the 2-tailed Student t test and the 2-tailed Mann-Whitney test, respectively. The correlations between credibility scores on the one hand and completeness and accuracy scores on the other hand were analyzed using the Spearman rank correlation test.

The statistical analyses were conducted using IBM SPSS Statistics for Windows (version 22.0; IBM Corp). The threshold value for statistical significance was set at a value of α=.05. The obtained scores are presented as mean (SD).

As far as the websites’ credibility is concerned, the obtained scores were average at best, with English and German language websites acquiring the highest results (mean 6.0, SD 1.8 and mean 5.4, SD 1.3, respectively), significantly higher than those of Hungarian, Romanian, and Russian language websites, which were graded as poor. Although certain criteria (eg, displaying the owner’s name and contact information as well as providing a direct contact modality) were largely met by web pages in all 5 languages, other criteria (eg, providing authors’ credentials and accreditations, bibliographic references, articles’ dates of publication and last update, or offering a quality procedure statement) were scarcely included. This may raise a red flag since authorship and providing references are perceived as important indicators of medical information reliability. Apparently, Hungarian, Romanian, and Russian language websites’ owners may not pay enough attention to credibility or are unaware of this aspect. These findings are consistent with previously published literature investigating the credibility of web-based information about different medical topics [10-12]. Moreover, compliance with the credibility criteria for health-related websites as measured by the Health on the Net Code of Conduct has been shown to vary largely depending on the type of organization and health conditions [13].

Regarding the evaluation of the websites’ informational content, the completeness and accuracy of the data about PCI in the 5 studied languages were found to be rather unsatisfactory, with the obtained scores only getting average and poor labels. In terms of completeness, English language websites acquired the highest scores (mean 4.6, SD 1.6), significantly higher than those of the websites in the other 4 language subsamples. German and Hungarian language websites also performed significantly better than the Romanian and Russian websites. In terms of accuracy of data, English language websites had significantly higher scores than German, Romanian, and Russian but not Hungarian websites, which had significantly higher accuracy scores than Romanian and Russian websites. A relative superiority of English language health-related websites compared to Spanish ones has been observed as early as 2001 in a study covering multiple medical conditions (ie, breast cancer, depression, obesity, and childhood asthma), and it was more recently compared to Turkish websites focusing on an orthopaedic intervention [14,15]. Leaving aside methodological differences, the results of these studies call attention to possible language-mediated inequities and suggest that a multilingual approach to web-based documentation may provide more complete coverage of the topic. Apparently, in some countries such as Romania, the low quality of web-based information about PCI seems to be in line with the low number of PCIs per million individuals, as shown by the latest statistics published by the European Society of Cardiology [16]. Efforts to increase the quality of web-based information about PCI would be a reasonable step in countries where access to these interventions is wanting.

The correlation assessments did not find statistically significant relationships between the credibility of the PCI-related websites and the quality of their informational content, with one exception. In the case of German language websites, the compliance with credibility criteria exhibited statistically significant, moderate strength correlations with the websites’ coverage of the topic (ie, completeness and accuracy). The lack of consistent correlations between credibility and content quality has been previously reported in investigations focusing on various medical conditions, such as stroke and depression or procedures such as first aid instructions in case of choking [17-19]. The results may suggest that the selected credibility criteria are not reliable indicators of information quality on PCI-related websites in the studied languages, and therefore, cannot be recommended to nonprofessionals as marks of trustworthiness.

Despite the growing demand for web-based medical information, the recognition of the importance of patient participation in medical decision-making, and the impact of health-related web content on consumer health [20,21], the credibility and quality of websites about PCI—a procedure globally used to mitigate the consequences of the most common type of heart disease—has not yet been rigorously analyzed. To the best of our knowledge, this is the first study analyzing and comparing the quality of information about PCI on English, German, Hungarian, Romanian, and Russian websites aimed at the general population. The results of this study may be used to raise awareness among internet users about the limitations and potential hazards of using the web as a source of information about PCI. Engaging in safe internet browsing is crucial, as it may prevent poor decision-making and potential complications caused by delayed intervention as well as a deterioration of the physician-patient relationship [20,22]. Although the web-based environment is easily accessible and convenient, it is highly advisable that consumers engage in web-based medical documentation with precaution and always turn to medical professionals for advice. Furthermore, medical practitioners should fully acknowledge the reality of e-patients and handle it appropriately [23,24]. In this regard, the involvement of health professionals in the development of plain language and accurate web-based health resources and their involvement in providing guidance to patients with inadequate health literacy in accessing proper information on the internet could prove beneficial [25].

It is worth noting the strengths of this study. First, the inclusion of multiple languages, of which at least three are spoken by vast numbers of individuals worldwide, allows for the extrapolation of the results and recommendations to a large population. For instance, according to Ethnologue [26], English is the most widely used language around the globe, being spoken by approximately 1.5 billion people across more than 140 countries. Moreover, both Russian and German are among the top 15 most widely spoken languages, with nearly 260 and 135 million speakers, respectively.

Second, most of the previously published studies focus on assessing health-related web-based sources based on credibility (reliability), readability, or design criteria (eg, the Health on the Net Code, JAMA score, DISCERN score, Flesch-Kincaid readability test, and SMOG Readability Index) [27]. As acknowledged by the authors of the DISCERN instrument [28], not even this tool was designed to actually measure the scientific fidelity of the information. Our study addresses not only the credibility or reliability dimension but also the quality of the content by evaluating the completeness and accuracy of information based on an evidence-based, topic-specific quality benchmark.

Third, to minimize subjectivity and the human error factor, the content quality assessments were conducted by 2 independent evaluators.

The main limitations of the study are related to some inherent traits of web-based research. Internet users may turn to various search engines or use different keywords, consecutively obtaining different search results [29]. Moreover, the continuously changing dynamics of the web-based environment make it virtually impossible for the results of this study to be precisely replicated. Additionally, the sample size may be argued as small. However, most internet users limit their inquiries to the first Google Search results page (on average, the first 10 search results) [30]. Therefore, by simulating a popular search strategy among lay internet users, we are confident that our results are likely to reflect common experiences. The study brings valuable descriptive data on the quality of web-based PCI-related information in multiple languages and has the potential to raise awareness about the need for responsible use of health-related web resources.

The quality and reliability of the web-based information about PCI on English, German, Hungarian, Romanian, and Russian websites are rather unsatisfactory, and there are significant differences in the quality of information across the studied languages. It is safe to say that the internet does not provide the general public with good-quality medical information on the aforementioned topic. Moreover, the selected credibility criteria cannot be recommended as consistent indicators of information quality. Further efforts ought to be made by website developers to improve the trustworthiness of web-based health information. Since medical websites have become one of the most trusted sources of health-related documentation, it is crucial to both parties involved in the medical act (ie, lay users and medical practitioners) to develop awareness of the potential dangers internet documentation may pose. Our results could contribute to advances in the fields of preventive medicine or public health, supporting the importance of internet education among the general population.