From November 2, 2011 to December 2, 2011, the YouTube site was searched using the following key words: “cardiovascular system examination”, “cardiovascular clinical examination”, “cardiovascular physical examination”, “respiratory system examination”, “respiratory clinical examination”, and “respiratory physical examination”. When searching YouTube, quotation marks were used with these terms to specify that these terms must be present. Only the first 10 pages of results for each search term were screened for related videos. The reason for not searching videos beyond page 10 of the search term results is that one is less likely to find videos covering the key words and users are less likely to go that far in their search [26]. We observed that videos not related to search words could be seen by the third and the fourth pages. By pages 8 to 10, videos not related to the search were found. Therefore, it was decided to go only as far as the first 10 pages for each search. Videos in the English language only were identified and the related URL was recorded.

The search was conducted by three authors (HAA, RAA, and SMA) independently using the search key words. The search results were evaluated and used to compile a common pool that was used in further analysis. The inclusion criteria were videos covering clinical examination of the cardiovascular or respiratory systems in adults. Videos were excluded if they were (1) not in the English language, (2) in the form of a lecture, (3) an advertisement or news, (4) discussing signs or symptoms of diseases affecting the cardiovascular or respiratory systems, (5) about simulated patients reflecting on their experiences or roles, (6) about patients with cardiovascular or respiratory disorders, (7) about drugs used in treating patients with cardiovascular or respiratory disorders, or (8) in the form of seminars or reviews of cardiovascular or respiratory systems. Duplicated videos were also excluded and repeats were treated as a single file for analysis. The repeat file with the greatest number of hits was used for analysis.

For each video, the following data were collected: title, duration of the video, number of days on YouTube, total number of viewers, and name of uploader/creator (organization, group of people, one person). The number of “likes” and “dislikes” (a crude scoring system that viewers can use to assign to each video they watched, though it is not necessarily used by all viewers) for each video was recorded. Because the number of days on YouTube varies widely among videos, we decided to calculate viewership/day as a more accurate parameter compared to total number of viewers. The viewership per day is the ratio of number of viewers to the number of days a video is on YouTube. The number of days is calculated from the day of uploading on YouTube up to December 1, 2011. This calculation of viewership/day was conducted for each video. The page number on which a video was placed was recorded. This is because the YouTube search algorithm is designed to show videos according to their relevance to key words used in the search and hence videos on the first three pages are more likely to be watched than videos on lower pages. Thus, the location of the videos may indirectly affect the number of “likes” and “dislikes” given by viewers.

The criteria used in evaluating the videos have been described in detail in an earlier work [25,35] with some modification to suit the study. In summary, the design of the criteria is based on four main domains: video content, technical aspects, authority/creator, and pedagogy used. The items in the criteria are grouped under two categories: major and minor. The major criteria comprise: (1) the video uses simulated patients or patients to demonstrate the examination, (2) contents about clinical examination are scientifically correct, (3) images are clear, (4) the creator and/or organization providing the video are mentioned, and (5) sounds are clear and background is free from noise. The minor criteria comprise: (1) the video covers the topic identified in the title, (2) designed at the level of undergraduate medical science students, (3) time to download is reasonable (about 10-15 minutes at the maximum, not interrupted or challenging to download as reported by the three evaluators), (4) information about the creator is up-to-date, (5) the educational objectives are stated, and (6) the topic is clearly presented. The criteria were used to categorize videos into educationally useful and non-educationally useful videos. We mean by “educationally useful” that a video provides scientifically correct and up-to-date knowledge and clinical instructions/skills accepted by educators in other teaching hospitals about cardiovascular and respiratory examinations. As per the basis of the evaluation criteria, educationally useful videos should fulfill the four domains (scientific content, technical aspects, authority/creator, and pedagogy used). Two scores were allocated for each item in the major criteria and one score was allocated to each item under the minor criteria. If an item was fulfilled, an allocated score was given; if not fulfilled, a zero was given. No half scores were used. As per our previous research work, educationally useful videos should fulfill all major criteria items as the minimum requirements plus at least three items under the minor criteria [35].

To standardize the evaluation of the content of each video and the process of clinical examination, the assessors used the textbook and video by Talley and O’Connor, “Clinical Examination”, as a reference to guide their assessment [36]. The content element in the criteria comprised the following: examiner introduces him/herself to the patient; patient is correctly positioned in bed; examination is conducted within time frame; examination covers the sequence of inspection, palpation, percussion, and auscultation; physical signs are correctly elicited; and examination is conducted in a professional manner. Prior to applying the criteria, we piloted its use. A total of 25 videos were randomly selected and used for this purpose. The criteria were applied independently by three assessors (SMA, HAA, and RAA). None of the assessors shared their findings or discussed the outcome of their evaluation. An Excel spreadsheet covering the three evaluations was examined by a fourth researcher (SAA). The agreement among the assessors was in the range of 96-98%. The findings were discussed among the researchers. The criteria were tested again independently by three assessors for another 25 videos. Videos were then rated independently by three assessors (SMA, HAA, and RAA). When videos were difficult to classify or when there was a disagreement among assessors, all researchers reviewed such videos in a meeting and reached a final agreement. This study was approved by the ethics committee at King Saud University College of Medicine.

The data were entered into Microsoft Excel 2010 and were checked before conducting any analysis. Analysis was conducted with SPSS software (version 18.0 for MS Windows) and was reported via mean, SD, percentage, and minimum and maximum; t tests and ANOVA (analysis of variance) were performed to determine significant differences. To assess the degree to which different judges or raters agreed in their assessment decisions, Cohen’s kappa for inter-rater reliability was used [37]. Pearson’s correlation coefficient (r) was calculated to determine if the viewership per day was correlated to the total scores given to each video. This relationship has also been examined to see if there is a correlation between the number of “likes” given by viewers to a video and the total scores given [38,39]. For all calculations, a P value <.05 was considered significant.

A total of 1920 YouTube videos were found on initial search and, on applying the inclusion criteria and visual examination of the videos, only 56 videos were relevant to clinical examination of the CVS (20 videos) and the RS (36 videos) (Table 1). Figure 1 summarizes how the YouTube searches and the number of videos in each category were refined on the basis of the inclusion and exclusion criteria. Examples of screenshots of these videos are shown in Figures 2-4.

The total duration of these video clips was 349 minutes, 11 seconds. The use of the criteria for categorizing the videos into useful and non-useful videos revealed that there were 9 CVS and 7 RS videos that provided useful information on clinical examination. These videos scored mean 14.9 (SD 0.33) and mean 15.0 (SD 0.00), respectively. The total duration of useful videos was 128 minutes, 6 seconds (71 minutes, 18 seconds for the CVS videos and 56 minutes, 48 seconds for the RS videos). The total viewers of all videos were 1,458,436. For both body systems, useful videos were viewed by 542,833 viewers (37.22% of total viewers, 542,833/1,458,436). Table 2 reveals detailed information about the 56 videos included in the study and the multimedia links show examples of useful videos about examination of the cardiovascular system (Multimedia Appendix 1) and the respiratory system (Multimedia Appendix 2).

Useful videos were created by doctors or professional bodies; a link to an organization such as ImedrxTV, geekymedics123, MedicalStudentApps, mosaicism105, or the name of the creator and his or her credentials, such as Professor S. Femando from the Open University of Sri Lanka, were shown. Some videos were linked to universities and known teaching institutes such as Manchester Medical School, St. George’s University of London, Central Manchester University Hospitals, the National Health Service, and the University of Wisconsin School of Medicine and Public Health.

Non-educationally useful videos scored mean 11.1 (SD 1.08) for CVS examination and mean 11.2 (SD 1.29) for the RS examination. Compared to educationally useful videos, the differences were significant (P<.001 for both CVS and RS videos) (Table 1). For both body systems, non-useful videos were viewed by 915,603 (62.78% of total viewers, 915,603/1,458,436). Non-educationally useful videos failed due to a number of reasons. The majority of the non-educationally useful videos failed to fulfill one of the major criterion items. Among those, more than 80% (33/40) were due to the image lacking clarity or no mention of the creator of the video. The concordance between the assessors on applying the criteria was 0.89, with a kappa score >.86.

The correlations between the total video scores and the number of viewers/day were not significant for the CVS videos (r=.06, P=.059) and the RS videos (r=.027, P=.827). Similarly, no correlation was found between likes or dislikes and the total number of scores given to CVS videos (r=.06, P=.059) or RS videos (r=.02, P=.832). No significant correlation was found between the video location on the first pages and the scores for like or dislike.

The aim of this study was to conduct an analysis of YouTube videos about CVS and RS physical examinations and to categorize these videos for educational purposes. Several methods have been described in evaluating videos [40-43]. However, none of these systems were useful for classifying videos about clinical skills. The system used in this study is simple, easy to apply, and covers four key elements, namely: scientific content, technicality, authority, and pedagogy parameters. It has also been tested in two previous studies and has shown a high level of inter-rater correlation while covering the key elements required for an educationally useful video [25,35].

The findings in this study indicated that there are 9 educationally useful videos on CVS examination and 7 on RS examination. These videos provided approximately 71 minutes, 18 seconds of CVS examination and 56 minutes, 48 seconds of RS examination that can be used in clinical learning and teaching purposes. Useful videos were linked mainly to universities or educational institutes. This indicates the involvement of universities and teaching institutes in promoting the use of educational videos as a resource to learners. This is particularly important with the shift of most universities toward self-directed learning and student-centered programs.

YouTube’s search engine, despite all precautions taken to target the search results to videos about examination of these two body systems, delivered more than 1800 videos that were not related to the search terms. Furthermore, the lack of correlation between the total score given to a video and the location of the video when searched (the page number) highlights the fact that the YouTube search algorithm is not well calibrated and unrelated videos usually appear despite the search filter provided by YouTube. Also, the lack of correlation between the total score given to a video to the “like” and “dislike” numbers as well as to viewership/day suggests the possibility that many viewers were watching substandard videos in their learning. Assuming that the majority of these viewers were medical/health students and trainees, these results highlight the need for YouTube to improve its search algorithm system to generate more accurate lists of videos that match with the search terms.

Much emphasis has been placed on physical examination in current medical curricula. To learn such skills, students usually rely on observing clinicians conducting examinations and then practicing on simulated patients and real patients. Recently, Duvivier et al (2012) in a study from Maastricht University reported that on average students devote 20% of their self-study time to skills training on physical examination [3]. They found that students use textbooks, examination guidelines, scientific articles, the Internet, videos/DVD, and scoring forms from previous OSCEs as their learning resources. Although the Internet and videos were among these resources, the study did not explore the sources of these videos and whether YouTube videos were used. Also, the study did not specify how much time they spend on average on resources such as videos and the Internet [3].

There is no substitute for witnessing a physical examination or a clinical procedure being performed live. Neither static images nor a description in a book outlining these techniques can offer the same impact as personal experience. Videos, however, are a medium that can transfer the experience and help in mastering such skills through repeated watching of techniques used and information provided. Online videos have become a routine and important tool in a student’s preparation for clinical skills. Attending clinical skills sessions, reading textbooks along with watching online videos, and practicing skills learned on patients have become important learning strategies in most clinical schools [44,45]. Based on cognitive psychological research, the use of videos will help expose students to the techniques of clinical examination, approaches for examining patients, and how to manage the sequence of technical steps in such examinations [45-50].

Considering the increasing number of learners using the Internet as their primary source of information, medical educators and clinical teachers should recognize the importance of YouTube in education and invest in using Web 2.0 in learning and teaching activities such as clinical teaching [8,13-14]. Although there are other links on the Web that provide free videos, we decided to examine the videos on YouTube for a number of reasons. First, YouTube is popular and usually preferred by users compared to other websites. Second, it is relatively easy to share videos on YouTube. Third, there is continuous improvement in the design of the YouTube site and each video is accompanied by useful data that reflect the evolution of online social networking and can be of use to researchers and viewers. Finally, YouTube has succeeded in providing social networking and enabling discussion among viewers, which could be useful to viewers as they share useful videos with each other.

This study shows that clinical teachers who are competent in clinical teaching and clinical educators, such as those who created the videos we identified in this study, can offer great service to medical and health students worldwide by placing their work on sharing websites such as YouTube. This can be part of knowledge transfer and scholarly work as it is created by academics, shared, and peer-reviewed [50]. Recently, YouTube launched YouTube EDU, an area of YouTube where video creators must possess high-quality credentials and provide evidence of significant mass of resources before they qualify to have their content included in YouTube EDU. This may have a major impact on improving successful search strategies particularly for YouTube videos designed for educational purposes.

This study represents a snapshot of available resources during November/December 2011, and, since then, there may have been more videos uploaded and made available. Given the continuous upload of videos on YouTube and the volume of new videos added to the system on a daily basis, further studies are needed to assess whether new videos of high quality and coverage have been added.

The small number of videos used in this study is a limitation to this study. The study was limited to videos in the English language and only those covering examination of adults, which may have contributed to the smaller number of videos found. However, the majority of videos uploaded on YouTube are in the English language. Also, there is the possibility that some videos were not labeled as such and thus were unidentifiable under our search terms, despite all our precautions including conducting the search of YouTube over 30 days independently by three researchers and using six search keywords to compile a common pool. The search in this study was limited to YouTube and there is the possibility that videos on other websites, such as those of medical and other health professional societies and medical journals, were not included.

This is possibly the first study assessing the educational value of YouTube videos about physical examination of the CVS and RS systems. Despite the small number of videos identified and found educationally useful, these videos can be used by medical students for independent learning and by clinical teachers as learning resources. The scoring system utilized by this study is simple, easy to apply, and could be used by other researchers on similar topics. The authors encourage other researchers to assess the tool and contribute to its improvement.