The Virtual Pathology Slide is an interactive microscope emulator that presents, via the Internet or CD-ROM, a complete 15.53 mm x 11.61 mm digitalized tissue section. The Virtual Pathology Slide mimics the use of a microscope in both the stepwise increase in magnification (from 16x up to 2000x) and in lateral motion in the X and Y Cartesian directions. This permits a pathologist to navigate to any area on a slide, at any magnification, similar to a conventional microscope.
The aim of this study was to assess the diagnostic accuracy and acceptability of the Virtual Pathology Slide.
Ten breast needle core biopsies were randomly selected and presented to 17 pathologists or trainee pathologists with at least 2 years experience in pathology practice. Participants were required to examine each case online and provide a diagnostic classification using online feedback forms. The recorded data permitted examination of interobserver variability and user satisfaction.
Agreement between original glass-slide diagnosis and consensus diagnosis using the Virtual Pathology Slide was reached in 9 out of 10 slides. Percentage concordance for slides lay in the range of 35.3% to 100% with an average percentage concordance between slides of 66.5%. The average Kappa statistics for interobserver agreement was 0.75 while average percentage concordance amongst participants was 66.5%. Participants looked at an average of 22 fields of view while examining each slide. Confidence: 81.25% of the participants indicated confidence using the Virtual Pathology Slide to make a diagnostic decision, with 56.25% describing themselves as "reasonably confident," 18.75% as "confident," and 6.25% as "very confident." Ease of use: 68.75% reported the system as "easy" or "very easy" to use. Satisfaction: 87.5% of participants expressed satisfaction with image quality, with 43.75% describing the image quality as "adequate," 25% describing it as "good," and 18.75% describing the image quality as "excellent." Pathologists with a working bandwidth greater than 20 kilobits per second found the download speed of the Virtual Pathology Slide "adequate" or better.
Results from this study show that the Virtual Pathology Slide can be used to make a correct diagnostic decision, and that the system is a realistic alternative to dynamic telepathology.
Telepathology is the practice of diagnostic pathology by a remote pathologist utilizing images of tissue specimens transmitted over a telecommunications network [
The diversity in telepathology systems reflects growing technological expertise in this area and the increasing importance of telepathology in education, training, quality assurance, and teleconsultation [
Image quality and the ability to make diagnostic decisions from electronically-compressed images is a contentious issue [
Recent improvements in Internet-browser technology have facilitated the development of interactive store-and-forward Web pages. These feature the ability to show the spatial relationship between individual images in low-power and high-power views. This technology is commonly visualized using a small image gallery constructed from one or two microscopic fields out of a possibility of thousands, displaying images of the same fields at higher magnifications [
To overcome problems attributable to sampling bias and interpretation resulting from limited field selection, telepathologists must be able to navigate to any field of view, at magnifications comparable to that of a conventional microscope, using images of sufficient resolution to render a correct diagnosis [
In an important new departure, the VPS can also record and quantify the diagnostic trace of a pathologist, as a discrete data set on a central server. This allows the decoupling of a pathologist's field selection from the technical functionality of the telepathology system. In this paper, we report on the development of this system, its acceptability among a group of evaluating pathologists, the level of diagnostic agreement among this group, and the potential future applications of the VPS in telepathology.
Screenshot of the Virtual Pathology Slide Web browser (for further screenshots, see
A comprehensive document detailing the scanning algorithm and system architecture of the VPS is in
To create VPS slides, an imaging workstation was developed in-house. An Olympus BX-40 microscope (Olympus, Melville, NY, USA) incorporating a 40x plan apochromat lens with a 0.95 numerical aperture was used. The microscope was fitted with a robotic stage (Prior Scientific Inc, Rockland, Mass, USA) and a JVC 3-CCD (3-chip charge-coupled device) video camera.
Using Optimas 6.5 imaging software (Media Cybernetics, Inc, Silver Spring, Md, USA), an algorithm was written in ALI (Analytical Language for Images) to perform a raster scan of 15.53 mm x 11.61 mm (180 mm 2) of tissue at 40x objective magnification. The VPS raster scan acquires 128 x 128 images in the X and Y Cartesian directions, one row at a time. Each acquired image represents 0.011 mm 2 at a resolution of 768 pixels by 574 pixels. Images were saved using a JPEG (Joint Photographic Experts Group) format at 10% compression, resulting in image-file sizes in the range of 100 to 150 KB (kilobytes). To build layers of lesser magnification, a second algorithm was developed, which tiles and resizes multiple images from the raster scan into composite images [
To view images via the Internet a graphical user interface was constructed [
A customized browser was developed to control the user's access to the VPS during dedicated studies, to optimize the integrity of recorded data, and to provide a uniform experience for users who would otherwise experience subtle differences due to variation in currently-existing versions of Web browsers.
The VPS browser is a Microsoft Foundation Class (MFC) application written in Visual C++, which utilizes Internet Explorer file libraries to behave as a customized browser. The VPS customized browser opens up prescribed Web pages on the VPS server. The VPS browser is optimized for PC users with Microsoft Internet Explorer 5 or greater.
When a user examines a VPS slide, data describing the user's interaction with the VPS is transmitted from the user's workstation to the VPS server and stored in an Oracle database. The VPS examination database is structured to contain the following data types:
This consists of data automatically recorded on the VPS server and includes parameters such as user's browser version, operating system, screen resolution, screen color depth, and IP (Internet Protocol) address.
This data records a user's "diagnostic pattern" as the user examines a slide. Information recorded includes image file name, image magnification, and the time spent viewing each image.
Diagnostic and descriptive data is submitted to the VPS server by participants, using HTML (Hypertext Markup Language) forms. Information recorded includes the report submitted by the user at the end of each slide examination and a final questionnaire. The observer also has the option to record or annotate every field of view examined.
The user has two choices on how he or she wishes to use the system. Users with high-speed Internet access can download the VPS browser from the VPS homepage and view images downloaded directly from the VPS server. To accommodate users with slow Internet connections, users may launch the VPS browser from a VPS CD and view images stored on the CD. However, an Internet connection is still required to record data on the VPS database, and to provide essential data for statistical analysis and playback facilities.
Ten needle core biopsies were obtained from the Department of Pathology, Mater Misericordiae Hospital, Dublin, Ireland. The slides were randomly selected by a pathologist (P.A.D.) with a special interest in breast pathology. The slides represented a range of diagnostic classifications. Two of the slides are presented in
Fifty-four pathologists with at least 2 years experience in pathology practice registered for the study. Of the 54 pathologists, 17 examined all 10 slides and 8 initiated the study but did not complete it. Of the 17 participants who completed the study, 8 were members of the European Working Group of Breast Screening Pathology. Of the 17 participants who examined all 10 slides, 13 subsequently completed a questionnaire on user perception of the VPS. Of the 8 participants that initiated the study but did not complete it, 3 completed the questionnaire.
Upon launching the VPS browser, participants were prompted to log in using the username and password they received at registration. This made them identifiable to the system. On successful log-in, the VPS needle core examination guidelines [
After stating they read the guidelines, users were permitted to browse the slides available for examination and select one from a slide gallery. The slide gallery displayed a thumbnail image of each slide and indicated the patient's age and sex, and a brief case description.
Upon selecting a slide for examination, participants were presented with the VPS user interface. While examining a slide, participants could if desired annotate the fields of view using the text area provided. Upon completing a slide examination, participants submitted an online report that provided a diagnostic classification for the case, using an adaptation of the Core Biopsy Reporting Guidelines for Non-operative Diagnostic Procedures and Reporting in Breast Cancer screening [
B1: Unsatisfactory/normal tissue only.
B2: Benign.
B3: Benign but of uncertain malignant potential.
B4: Suspicious of malignancy.
B5: Malignant.
For slides categorized as B5, participants were required to subclassify their decision as malignant, in-situ, or invasive. Upon making a classification, participants were returned to the slide gallery from which another slide could be selected for examination.
Utilization of this data allowed the following to be determined:
Percentage concordance for a user, calculated as the number of slides (expressed as a percentage) for which the user's diagnosis is in agreement with the consensus VPS diagnosis.
Percentage concordance of a slide, calculated as the percentage of users who concur as to the correct diagnosis of a slide.
Cohen's Kappa [
Participants who completed examination of the 10 slides were subsequently requested to complete an online questionnaire describing their experience using the VPS. Participants were asked to give a subjective evaluation of diagnostic confidence in using the VPS, reasons for uncertainty, an evaluation of image quality, and perceived download speed.
Two examples of the 10 breast needle core biopsies presented to 17 pathologists or trainee pathologists using the VPS (for all 10 images, see
Comparison of glass slide needle core surgical biopsy diagnosis and most-common Virtual Pathology Slide (VPS) diagnosis, in order of level of agreement (concordance) for each slide
|
||||||||||
|
|
|
|
|
|
|
|
|
|
|
Diagnosis Glass | B5 | B5 | B5 | B2 | B2 | B2 | B2 | B5 | B5 | B3 |
Diagnosis VPS | B5 | B5 | B5 | B2 | B2 | B2 | B2 | B5 | B5 | B4 |
Concordance, % | 100 | 94.1 | 82.4 | 76.5 | 64.7 | 58.8 | 52.9 | 52.9 | 47.1 | 35.3 |
Fields of view* | 243 | 208 | 464 | 410 | 463 | 309 | 591 | 431 | 487 | 299 |
A more-detailed analysis of the diagnostic classifications made by participants is described in
For each participant: years of experience in pathology practice, diagnostic classification of slides, level of agreement with each other (% concordance and Kappa index), and number of fields of view examined
|
|
|
|
|
|
|||||||||
|
|
|
|
|
|
|
|
|
|
|||||
5 | 5 | B5 | B5 | B5 | B2 | B2 | B2 | B2 | B4 | B5 | B4 | 90 | 0.97 | 321 |
62 | 5 | B5 | B5 | B5 | B2 | B2 | B3 | B2 | B5 | B4 | B4 | 80 | 0.94 | 326 |
35 | 5 | B5 | B5 | B5 | B2 | B2 | B1 | B1 | B4 | B5 | B4 | 70 | 0.94 | 122 |
10 | 5 | B5 | B5 | B5 | B2 | B3 | B3 | B2 | B5 | B4 | B4 | 70 | 0.91 | 157 |
39 | 5 | B5 | B5 | B5 | B1 | B3 | B3 | B2 | B5 | B5 | B5 | 60 | 0.90 | 343 |
55 | 5 | B5 | B5 | B5 | B2 | B2 | B2 | B2 | B5 | B3 | B3 | 80 | 0.87 | 289 |
87 | 5 | B5 | B5 | B5 | B1 | B2 | B2 | B2 | B3 | B5 | B3 | 70 | 0.86 | 130 |
18 | 3 | B5 | B5 | B4 | B1 | B3 | B2 | B1 | B5 | B4 | B3 | 40 | 0.86 | 252 |
68 | 5 | B5 | B5 | B5 | B2 | B3 | B2 | B2 | B5 | B4 | B2 | 70 | 0.85 | 228 |
65 | 5 | B5 | B5 | B5 | B2 | B2 | B3 | B1 | B3 | B4 | B4 | 60 | 0.80 | 234 |
22 | 5 | B5 | B5 | B5 | B2 | B2 | B2 | B5 | B5 | B5 | B5 | 80 | 0.75 | 204 |
41 | 5 | B5 | B5 | B5 | B1 | B2 | B2 | B2 | B2 | B4 | B4 | 70 | 0.75 | 216 |
7 | 5 | B5 | B5 | B5 | B2 | B2 | B2 | B1 | B4 | B2 | B3 | 60 | 0.73 | 121 |
1 | 5 | B5 | B5 | B4 | B2 | B4 | B2 | B2 | B5 | B5 | B1 | 70 | 0.67 | 223 |
75 | 5 | B5 | B5 | B5 | B2 | B2 | B2 | B5 | B4 | B5 | B2 | 70 | 0.65 | 120 |
36 | 3 | B5 | B5 | B2 | B2 | B3 | B3 | B4 | B5 | B2 | B2 | 40 | 0.26 | 201 |
6 | 5 | B5 | B2 | B5 | B2 | B2 | B4 | B5 | B3 | B5 | B2 | 50 | 0.23 | 418 |
|
66.5 | 0.76 | 23 |
* ID = identification number of participant.
† EXP = years of experience in pathology practice.
‡ Concordance = number of slides (expressed as a percentage) for which the user's diagnosis is in agreement with the consensus Virtual Pathology Slide diagnosis.
§ Kappa = Cohen's Kappa, a measure of agreement between observers, taking into account agreement that could occur by chance. Kappa greater than 0.7 indicates "substantial agreement."
The average Kappa value achieved by participants was 0.76. Participants 36 and 6 achieved a Kappa of 0.26 and 0.23 respectively indicating "fair agreement" [
The average percentage concordance for slides was 66.5% with a minimum concordance of 35.3% for slide 8 and a maximum concordance of 100% for slide 6. The percentage concordance for slide 5 was 47%. For all remaining slides there was greater than 50% agreement between participants.
The average number of fields of view examined by each participant was 23 per slide. Participant number 5, who achieved the highest Kappa, examined 321 views, while participant number 6, who had the lowest Kappa, examined 418 fields of view.
The highest number of number of fields of view examined for a particular slide was 118 by participant number 6 while examining slide 10. This slide had a percentage concordance between participants of 52.9%. The lowest number of views examined while examining a slide was 3; this was by participant 10 who achieved a Kappa score of 0.91 and agreed with the group consensus for slide 2. Diagnosis for slide 2 had a percentage concurrence amongst participants of 94%.
The average time taken for participants to examine a slide was recorded as 6 minutes 11 seconds. The maximum time taken to examine a slide was recorded as 12 minutes 49 seconds by participant number 36 with an average bandwidth of 20 kilobits per second while examining slide 7. The minimum examination time was recorded as 43 seconds by participant number 1 with an average bandwidth of 64 kilobits per second while examining slide 2.
Participants were asked to assess their own computer competency and the frequency with which they use a telepathology system. Participants described themselves as "advanced" (18.75%), "competent" (18.75%), or "adequately competent" (62.5%) with computers, while 44% of participants indicated they had never used a telepathology system prior to the study.
Ease of use of the Virtual Pathology Slide (VPS). Yrs = years of pathology experience. Percentages = percentage of 16 participants for that rating
Degree of confidence in using the Virtual Pathology Slide (VPS) to make a diagnostic decision. Yrs = years of pathology experience. Percentages = percentage of 16 participants for that rating
Perceived image quality of the Virtual Pathology Slide (VPS). Yrs = years of pathology experience. Percentages = percentage of 16 participants for that rating
The VPS system is a realistic alternative to dynamic telepathology, in terms of its ability to mimic a conventional microscope, its accessibility via the Internet, and its simplicity of operation. Of the 17 participants, 15 achieved a Kappa of between 0.97 and 0.65 and 14 attained a percentage concordance of between 90% and 60%. This demonstrates "substantial" agreement between users when using the VPS [
Participant 36 and participant 6 attained the lowest Kappa scores of 0.26 and 0.23 respectively. This reduced the overall average Kappa value considerably. Confidence in using the VPS was described as "reasonably confident" by participant 36, who had 3 years experience in pathology and examined 201 fields of view while examining the entire set of slides. Further analysis of the images viewed is necessary to elucidate reasons for the diagnostic decisions made by participant 36; however, inexperience with breast pathology coupled with insufficient examination of the slides may have contributed to poor performance. Use of telpathology was described as "infrequently" by participant 6 who was "confident" in making a diagnostic classification using the VPS and described the use of the VPS as "easy." However, participant 6 attributed some diagnostic uncertainty to "problems with assessing significance of small subtle lesions without having the whole slide to look at." Participant 6 examined 418 fields of view, the highest number examined by any participant.
The average percentage concordance for the entire set of slides was 66.5%. Full agreement between participants was achieved for slide 6, which demonstrates that full agreement can be achieved using the VPS.
The average number of views examined by participants while examining the entire set of slides was 230. The percentage concordance for a particular slide decreases as the average number of fields examined for that slide increases. For example, the average number of fields examined for slide 6 (100% concordance amongst participants) was 14.3, while the average number of fields examined for slide 10 (52.9% concordance amongst participants) was 34.8. Conversely, participants with a high Kappa score tend to view a greater number of fields of view than participants with a low Kappa score, suggesting that the greater the amount of tissue viewed by a pathologist, the more likely they are to make a correct diagnosis.
Slide 8 had the lowest level of concordance at 35.3%. This reduced the average percentage concordance for the set of slides by 3.46%.
Participants with 3 years or less experience did not have access to broadband Internet connection and recorded bandwidth speeds of less than 15 kilobits per second. These participants expressed least satisfaction with the VPS in terms of ease of use, image quality, and diagnostic confidence. All 3 participants, who indicated they were "not confident," attributed difficulty in using the VPS to poor download speed, with comments such as "Poor download speed was extremely slow and made the viewing experience disjointed and basically unworkable." Of these 3 participants, 2 had a working bandwidth of 12.6 kilobits per second and 31.5 kilobits per second respectively. A bandwidth could not be determined for the third, however the third did offer such comments as "too long to download images" and "problem was on my end, slow connection." High-speed broadband Internet connectivity is still unavailable to many pathologists. This is a major limiting factor for acceptability of Web-driven telepathology due to the time taken to download large image files over the Internet [
Participants were asked to comment on improvements to the VPS that they would like implemented. A number of participants suggested they would like additional magnification ranges. For example, "Navigation within the slide was disjointed and it was difficult to maintain perspective whilst moving from field to field. The range of magnifications was too limited, especially in the intermediate magnification range."
There are a growing number of interactive pathology sites available via the Internet [
The authors would like to thank the Irish Health Research Board (HRB), for their partial funding of this work. The authors would also like to thank the pathologists who took time to partake in this study and provided such useful and constructive feedback.
None declared.
Virtual Pathology Slide
Screenshots of the Virtual Pathology Web browser.
Technical Documentation.
Ten breast needle core biopsies presented to 17 pathologists or trainee pathologists using the VPS.