This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Mammography is considered the best imaging technique for breast cancer screening, and the radiographer plays an important role in its performance. Therefore, continuing education is critical to improving the performance of these professionals and thus providing better health care services.
Our goal was to develop an e-learning course on breast imaging for radiographers, assessing its efficacy, effectiveness, and user satisfaction.
A stratified randomized controlled trial was performed with radiographers and radiology students who already had mammography training, using pre- and post-knowledge tests, and satisfaction questionnaires. The primary outcome was the improvement in test results (percentage of correct answers), using intention-to-treat and per-protocol analysis.
A total of 54 participants were assigned to the intervention (20 students plus 34 radiographers) with 53 controls (19+34). The intervention was completed by 40 participants (11+29), with 4 (2+2) discontinued interventions, and 10 (7+3) lost to follow-up. Differences in the primary outcome were found between intervention and control: 21 versus 4 percentage points (pp),
This e-learning course is effective, especially for radiographers, which highlights the need for continuing education.
In Europe, breast cancer is responsible for one in every six deaths from cancer in women [
Mammography is currently considered the best imaging technique for breast cancer screening [
Radiographers play an important role in the performance of mammographic examinations; their knowledge requirements go beyond radiation exposure, positioning techniques, and other characteristics of the equipment used [
Radiographers are part of a multidisciplinary team, both during screening or diagnosis and intervention, allowing them to be aware of the clinical information as well as previous breast exams of the patient [
Beyond these technical issues, radiographers are usually the first professionals to have face-to-face contact with women in primary health care, during breast cancer screening. Therefore, they should be able to provide the patient with sound answers regarding the examination and the implications of the results [
This extended role could contribute to decreasing the mortality rate of breast cancer and thus reducing health care needs [
In this context, the knowledge of senology concepts is essential to the health professional, highly considered in some countries (such as in England, Denmark, Australia, and United States) [
Continuing education is known to improve the performance of professionals, providing better health care services [
The asynchronous ability, cost-savings, personalized learning, increased accessibility, ease of distribution, and updated content are some examples of e-learning advantages [
With the development of new information technology, there is a surplus of software that can be used to implement e-learning systems, ranging from websites and email to blogs, wikis, and discussion forums [
The potential of e-learning may not always translate into significant improvements in educational outcomes [
Most of the studies on the evaluation of e-learning processes rely on users’ satisfaction and knowledge [
One of the most cited questionnaires for user satisfaction with e-learning [
The work presented in the following sections aims to provide a new easy-to-use e-learning course, thus contributing to senology teaching, while emphasizing continuing education and professional development.
In this context, our research question is whether an e-learning system improves the senology knowledge of radiographers and radiology students. The objectives of this work include knowledge promotion and understanding of all aspects of breast illness and patient care required by radiographers. To achieve this, we proposed to (1) develop an easy-to-use course in e-learning environment, (2) assess its efficacy and effectiveness, and (3) assess the satisfaction of users.
An asynchronous e-learning course on breast imaging for radiographers was developed and evaluated for its efficacy, effectiveness, and satisfaction.
The target population in this study was all radiographers working at public health institutions in Porto’s metropolitan area, in Portugal, with ability to perform mammographic exams, and all radiography students attending the 3rd and 4th years of the radiology course at the School of Allied Health Science, Porto Polytechnic Institute who already had mammography training.
Hence the sample was stratified by professional status into two groups: students and radiographers, who were invited by email to participate in the study, after an individual request.
All radiographers and students included in the study were asked to perform the pre-knowledge test. Randomization was performed with the population elements in each stratum who accepted and effectively took the pretest. Therefore, 50% of the elements of each stratum were randomly allocated to the intervention group and contacted to participate in the course, after being properly informed of the randomization process. The remaining sample was allocated to the control group and afterward contacted to participate in a second test.
The communication with all participants was performed mainly by email (maximum three attempts) as described in
The intervention group was contacted in order to perform the e-learning course, and a similar process to the pretest enrollment occurred. Private asynchronous access was given during a 20-day period to those who agreed to do the course. Then, a final assessment test was delivered, to be answered within 5 days. Those who discontinued the intervention were requested nonetheless to perform the posttest, to allow an intention-to-treat analysis.
The control group was contacted 41 days after randomization to perform another test. During this period, neither radiographers nor students allocated to the control group had any formal educational activity concerning senology lectures given by our research team and were asked to answer tests in an honest manner, without consulting external sources, which could in the future evolve into a clear accreditation process [
The course was created and revised between October 2011 and January 2012. Contact with the participants occurred between February 1 and March 7, 2012, and the trial was conducted between March 7 and May 31, 2012.
Study design and post-implementation duration: numbers in boxes represent number of participants; numbers in arrows correspond to the elapsed time in days (maximum, average) at each phase.
The course was written in Portuguese and developed in Netbeans version 8.0, using simple and wide-spread technologies such as Hypertext Preprocessor (PHP), HyperText Markup Language (HTML), JavaScript, Cascading Style Sheets (CSS), and Extensible Markup Language (XML), and was hosted on the server at the Faculty of Medicine of the University of Porto. Website security was guaranteed through an authentication mechanism with username and password, and no collaborative activities [
The course instructions were available on the website, along with a glossary, and it was structured into four modules (
Description of the main contents included in the intervention e-learning course.
Module | Contents |
1. Breast anatomy and physiology of breast | Breast localization and superficial anatomy |
Breast tissues constitution | |
Radiological anatomy | |
Breast tissues patterns | |
Breast lesions localization | |
2. Breast cancer: multidisciplinary approach | Breast cancer statistics |
Breast cancer screening: Mammography | |
Breast cancer signs and symptoms | |
BI-RADS classification | |
Additional imaging techniques | |
Breast cancer management and treatment options | |
3. Breast pathology | Pathology of benign and malignant lesions |
Male breast cancer | |
4. Mammography: technical approach | Mammography history |
Technical aspects of the equipment and new technologies | |
Image quality control | |
Positioning techniques and indications for standard and additional mammographic views | |
Localization and biopsy techniques for non-palpable lesions | |
Radiographer role |
The content was presented using text, images, videos, and Prezi presentations. Diagnostic images were collected directly from the Breast Unit of S João, Hospital Center, with proper legal authorization. All content was reviewed by specialists from the same institution. Screenshots are presented in
Given the asynchronous characteristics of the course, learners could monitor the evolution of their learning through a status bar, giving feedback about the self-learning. At the end of each module, a summary of the key points and a self-assessment test of 6 multiple-choice questions were presented; correct answers were immediately available. The posttest was administered at the completion of the course. This posttest was a final assessment sent by email and available for 5 days after mail delivery. After successful completion, a certificate of attendance was sent to the participants.
Screenshot of a Prezi presentation in the e-learning course.
Screenshot of mammography views in the e-learning course.
Screenshot of mammography views and schemes in the e-learning course.
Screenshot of one of the four self-assessment tests included in the e-learning course.
Each test was structured with 8 multiple-choice questions related to the course modules (
Participants were asked to answer in an honest manner, without consulting external sources. This study consisted of two phases. In the first step, 4 individuals received one test, and the other 4 received the other test. The random delivery of the tests was accomplished, highlighting the independence of the questionnaires in relation to being the first or the second test answered. Then, in a second step, when an individual submitted the answers 24 hours later, the other test was sent. This pilot study was done between February 18 and March 1, 2012.
The focus of the data analysis was to determine whether there was a significant difference between the questions presented in the first and the second tests. For this observation, we used paired Wilcoxon tests to compare the number of incorrect answers of each individual between the first and the second test. As the questions were organized by themes, related to the modules of the course, the tests were also organized for each set of questions. The statistical analysis was performed using IBM SPSS Statistics software version 17.0, and the significance level was .05.
Through the data analysis, we can conclude that there are no differences among the modules in one test and the other: module 1 (
Demographic information such as age, gender, academic qualifications, years of professional experience, and routine mammography, as well as opinions regarding the need for continuing education and the receptivity of e-learning programs on any topic of professional interest, were also gathered.
The Web resource GoogleDocs was used to create and deliver online the tests, assigned to individuals randomly as pre- or post-knowledge tests.
Given the lack, to the best of our knowledge, of a questionnaire in Portuguese, we used the questionnaire proposed by Wang [
The primary outcome was the knowledge evolution observed between pre- and post-knowledge tests (each measured by the percentage of correct answers and failure defined using 50% cut-off), using a paired analysis the difference between pre- and post-knowledge test percentage of correct answers (referred to as “evolution” and measured in percentage points [pp]).
Effectiveness was assessed through the proportion of participants who completed the intervention. Satisfaction was mainly assessed through the last two items of the questionnaire.
Analysis was mostly performed according to the intention-to-treat strategy [
The sample was described by average (µ) and 95% confidence intervals for normally distributed variables, and median, 25 and 75 percentiles (P25; P75) for the remaining.
Homogeneity between the two groups was assessed using the Mann-Whitney
We considered a significance level of .05, and the analysis was carried out in IBM SPSS Statistics software, version 17.0.
Globally, 190 individuals were considered for inclusion (120 radiographers and 70 students) from which a total of 107 enrolled and answered the pre-knowledge test (68 radiographers and 39 students). The average time spent (in days) between study’s milestones can be observed in
According to the intention-to treat strategy, the intervention group included 2 radiographers and 2 students identified as lost to follow-up, as well as 1 radiographer and 1 student identified as “discontinued intervention” who answered the post-knowledge test. Globally, 46 individuals were included in the intervention group (14 students and 32 radiographers) and 42 in the control group (16 students and 26 radiographers).
Participant flow diagram showing enrolled sample and respective dropouts. Individuals who did not answer the participation request were considered "lost to follow-up"; participants who did not finish the course were considered “discontinued intervention”.
Of the 107 participants, 36.4% (39/107) were students and 63.6% (68/107) were radiographers; 79.4% (85/107) were female. The median age was 21 years old (P25=21; P75=22) for students, and 33 years old (28; 40) for radiographers. Overall, 10.3% (11/107) were third-year students, 26.2% (28/107) were fourth-year students, 1.0% (1/107) was credited due to professional experience, 7.5% (8/107) had bachelor’s degrees, 50.5% (54/107) had graduated, and 4.7% (5/107) had a master’s degree.
In the radiographer group, the median professional experience was 12 years (5; 17); 31% (21/68) individuals did not perform mammography at all, 53% (36/68) performed fewer than 30 per week, 6% (4/68) performed between 30 and 40 per week, and 10% (7/68) performed more than 40 per week.
In the pre-knowledge test, there was a failure rate of 14.0% (15/107). We observed that 51.4% (55/107) of the results were between 50-75% and 34.6% (37/107) had results better than 75%. In the post-knowledge test, the failure rate was 5.6% (6/107); 25.2% (27/107) of the results were between 50-75%, and 51.4% (55/107) had results better than 75%.
Intervention and control groups were globally comparable, with a difference only in female proportion in both groups for the radiographer stratum (88% vs 68%,
Both intervention and control groups had similar results in the pre-knowledge test (63% vs 63%,
The intervention group obtained better results in the post-knowledge test compared to the control group (88% vs 63%,
Participants had an overall positive evolution (μ=13 pp, 95% CI 8-18), which is higher in the intervention group (21 pp vs 4 pp,
Stratifying the results (left plot of
Considering a per-protocol analysis (right plot of
Significant differences in students could then be observed between intervention and control (25 pp vs 3 pp,
Improvement stratified by professional status. Left plot (intention-to-treat): Inconclusive result for students but clear effect for radiographers. Right plot (per-protocol): Clear effect for both students and radiographers.
Most participants (81%, 44/54) in the intervention group agreed to take the course (13 students and 31 radiographers), and only 9% (4/44) did not attend the full course: 2 students and 2 radiographers.
All participants who completed the e-learning course answered the satisfaction questionnaire (n=40). Considering global measures, 85% were satisfied with the e-learning system (students vs radiographers:
Of all e-learners, 10% (4/40) had previous e-learning experience, and 5% (2/40) performed it in the health area. However, the overall satisfaction did not differ between these participants and those who had no previous experience of e-learning (
Comparing students and radiographers, we did not find any significant differences besides age and academic qualifications, and therefore no additional confounding factors were considered for adjustment. Overall improvement was observed and attributed to the course.
Although the effect was not clear in students, we found differences in post-knowledge test results between the intervention and control groups, whereas such differences were absent in the pre-knowledge test. This result could be explained by the smaller sample size or higher proportion of individuals lost to follow-up, which could have resulted from being enrolled in other learning activities at the same time during the academic year. In addition, students were probably more prone to self-learning from other sources, or there was interest from students in the control group in learning more after performing the pre-knowledge test.
Regarding radiographers, those who were allocated to the intervention group significantly improved more than the controls did, which supports the importance of continuing education throughout their working lives.
The per-protocol analysis enhanced the influence of the course, exposing a significant effect on the students’ stratum. Considering that the real course effect is probably in between the two statistical analyses, we can conclude that this randomized controlled trial showed that the e-learning course improved the knowledge of those who attended (even if only part of) the intervention.
Overall, the developed course is efficacious, especially for radiographers, which highlights the need for continuing education, foreseeing also e-learning as an increasingly viable complement to the traditional method, especially since the technologies involved do not need heavy hosting requirements. This study also proved that the course is effective, since only 10% of the learners dropped out. Moreover, the course showed to have a high level of satisfaction, for both radiographers and students.
Since it was not possible to find in the literature any satisfaction questionnaire for e-learning systems in Portuguese, the validity of this satisfaction evaluation should be carefully considered, given the translation. A validation of the questionnaire used in this work could be a future project.
Another limitation of this work is the single learning institute with moderate sample size, affecting the generalizability of the results.
Another restraint is related to the learning and evaluation strategy, since we faced the risk of individuals resorting to external sources in order to provide correct answers to the tests. This situation creates a slight bias that is difficult to control considering the study design.
Nevertheless, randomization was performed after completing the pre-knowledge test, which yielded that the same willingness of participants to enter the study was demonstrated, regardless of the group where they were allocated.
Globally, this study underlines the importance of the radiographer as the health care professional who interacts first with women during the breast cancer screening process. We consider the high rate of participation an important aspect in our study (57% of radiographers and 56% of students), which reflects the great interest shown by these professionals to participate in scientific research, thereby promoting their professional category. They took advantage of learning opportunities, which shows that they are health care professionals committed to responding to the constant challenges of the profession.
This study contributes to the Portuguese radiographers’ continuing education, since we did not find any similar course related to breast imaging. It would be interesting to conduct additional assessments to demonstrate effective consolidation of knowledge gain. Future developments may include collaborative activities; for this first assessment, we believe such activities would confound the efficacy results.
Our main finding illustrates the knowledge improvement in senology that our e-learning course gave radiographers. We believe that this study highlights the importance of e-learning as a training platform, especially in light of budget constraints associated with the current economic climate. E-learning should be considered for continuing education, and directors should invest in it to improve the skills of their professionals and consequently enhance health care services [
European Society of Breast Cancer Statistics
Learning Management Systems
percentage points
randomized controlled trial
This work was done while the first author was a student in the Master’s program in Medical Informatics at the Faculties of Medicine and Sciences of the University of Porto which kindly supported this publication.
The authors would also like to thank S João Hospital Center, Radiology Department and Breast Unit, for the approval of images for the course, and to the Faculty of Medicine of the University of Porto for hosting the course website.
All the authors have made a significant contribution to this manuscript and have approved the final paper, thus meeting the criteria for authorship. All those entitled to authorship are listed as authors.
None declared.