The content of a page can change and is likely to change over time; this is one of the useful qualities of the Web, but also a dangerous one.
To monitor the evolution of Web page contents on sore throat over a 3 year period.
Two medical doctors independently evaluated 34 Web pages on sore throat. Pages were found using a metasearch engine. The evaluation factors were: the adherence of medical contents to a gold standard (American Academy of Pediatrics recommendations) composed of 5 subfactors (epidemiological, clinical, complications, diagnosis, and therapy); the completeness of the contents in terms of considered/missed factors of the gold standard; references to medical literature; and a specified last update of the page. During the observation period these sites were revisited twice, after 28 and 39 months, to examine any changes therein since the first visit.
The degree of adherence to the gold standard did not significantly change. Variations (both positive and negative) were recorded solely with regard to the update and references factors as well as with regard to the availability of the pages over time (18% disappeared during the observation period).
In 3 years medical contents have not changed significantly and despite the contemporary epochal Internet revolution (in terms of, eg, technology, graphics, and access) and the increase in the number of sites dealing with the issue of sore throat, there has been no corresponding qualitative increase in the contents of the pages monitored.
The Internet is a relatively-recent phenomenon and its evolution and use by nonprofessionals have developed especially in the last 5 years. The usefulness of medical cyberspace continues to grow and links, search engines, and the presentation of information as well as users' Web-surfing skills all seem to be steadily improving. The use of the Internet is also rapidly increasing in Italian households. Our previous study carried out on a population of parents in 1999 showed that approximately 19% of an unselected sample of parents used the Internet to acquire medical information [
The content of a page can change and is likely to change over time; this is one of the useful qualities of the Web, but also a dangerous one. Although many attempts have been made to assess, control, and assure the quality of Web-based medical information [
On August 12 1998 we searched the World Wide Web for the first time using Metacrawler, a metasearch system that in 1998 combined 6 different search engines (Alta Vista, Excite, Infoseek, Lycos, Webcrawler, and Yahoo). Because of the differences among the search engines (eg, Infoseek is a random-search tool and Alta Vista ranks the requested references in order of relevance) a metasearcher combines such different approaches in a unique "independent" collection that could be linked to the way in which parents — who have variable experience, culture, and computer ability — perform the search.
We used the English words:
To evaluate the quality of each Web site, we considered the adherence to some of the criteria suggested in literature available in 1998 on this topic: suitability of the medical contents, scientific citations, and date of creation and/or of the last update. We compared the medical contents of all these Web sites with the recommendations of the Committee on Infectious Diseases of the American Academy of Pediatrics [
Adherence
The median score of the present factors was used as the global evaluation of
Completeness
References
Update
According to these criteria, the quality of the Web pages was independently evaluated by 2 medical doctors (VC, a senior pediatrician, and PSB, a resident) to validate the medical information for parents. Each doctor evaluated every Web site individually without knowing the site's address or author. When the 2 authors disagreed, to ensure the reliability of the rating they talked about the disagreement until they reached a common value.
On December 12, 2000 and on November 12, 2001 all 34 sites included in the study were revisited to see if the pages still existed or if they had been substituted with new pages. All pages were reevaluated with the same methodology, comparing the content with copies of the original pages. The missing pages were manually searched in the Web (by means of Altavista, Google, and other search engines) starting from the address and/or the contents of the previously-evaluated pages. The results were analyzed through Wilcoxon's paired signed rank test and the McNemar test.
Sites evaluated and their respective homepage URL
|
|
Abersychan Surgery |
|
American Academy of Pediatrics |
|
American Medical Association |
|
Association of State and Territorial Directors of Health Promotion and Public Health Education (ASTDHPPHE) |
|
Baltimore County Public Schools |
|
Beijing Scene Publishing |
|
National Center for Emergency Medicine Informatics |
|
Children's Medical Center of Dallas |
|
ComWeb |
|
Darthmouth-Hitchcock |
|
Drugbase |
|
Eastern Wyoming College |
|
|
|
Group Health Cooperative |
|
Hawaii Department of Health |
|
HealthWorld Online |
|
HealthAnswers,Inc |
|
Housecall Medical Resources, Inc. |
|
linois Department of Public Health |
|
Kenyon College |
|
MayoClinic.com |
|
MedicineNet, Inc. |
|
Medscape Portals, Inc. |
|
National Institute of Allergy and Infectious Diseases |
|
Nightime Pediatrics Clinics, Inc. |
|
Seton Hall University |
|
St. James's Hospital |
|
The Nemours Foundation |
|
University of Iowa |
|
University of Colorado at Boulder |
|
University of Iowa Health Care |
|
University of Missouri-Rolla |
|
University of Rochester |
|
WellnessWeb |
|
During the observation period, 6 (18%) of the 34 pages monitored disappeared between August 1998 and November 2001; 3 pages were no longer available after 28 months with the remaining ones no longer active by 39 months.
Comparison of scores for 1998, 2000, and 2001
|
|
||
|
|||
|
|
|
|
Adherence | |||
1 | 3 | 3 | 2 |
2 | 1 | 1 | 1 |
3 | 5 | 4 | 4 |
4 | 5 | 5 | 3 |
5 | 20 | 18 | 18 |
Completeness | |||
1 | 0 | 0 | 0 |
2 | 4 | 4 | 4 |
3 | 6 | 4 | 3 |
4 | 8 | 6 | 6 |
5 | 16 | 17 | 15 |
References | |||
0 | 29 | 27 | 24 |
1 | 5 | 4 | 4 |
Update | |||
0 | 22 | 19 | 16 |
1 | 12 | 12 | 12 |
Number of sites evaluated | 34 | 31 | 28 |
Because only 1 Web site showed a large degree of variation in terms of adherence and completeness to the gold standard, no statistical analyses were performed. Most variations relate exclusively to update and references; to evaluate their evolution, we made the Wilcoxon's paired signed rank test and the McNemar test for the pairs of years 1998-2000 and 1998-2001. The differences were not significant (
If it is true that the Internet is continuously developing as regards the quantity of information available and the number of sites online, it seems that the same cannot be said for the quality of the information provided. This leads one to have doubts as to the effective greater dynamism of Web publications compared to publications printed on paper. Only 3 sites in our sample modified the pages relative to sore throat during the observation period and in 2 cases the changes did not have any significant impact on quality. With the exception of 1 site, the medical contents of the reevaluated sites remained similar to the initial version; apart from the foregoing, improvements were recorded only in aesthetics (2 sites) and a request for HON certification (1 site).
Of the 5 gold standard factors, the least common one is
In the observation period the relevant changes were restricted to
It was not always trivial to find a missing page through the initial address or via a simple path starting from the home page. The Web administrator often made a major change in the location of the page in the site. In some cases there were no direct links to reach the new location; instead, the new page was only accessible by using a search engine In 1 case, the Web site on which the page was published ceased to exist.
Our research was limited in terms of the observation period, the number of monitored sites and the unique medical issue considered. The idea of conducting a broader and more-systematic analysis of the evolution over time of the pediatric information available to parents via a suitable "Internet observatory" is an important challenge. Unfortunately, the huge number of sites and issues to be monitored require a correspondingly huge amount of resources; nevertheless there is the need to continuously improve the efforts to provide the final user such services.
None declared.