JMIR Publications

Select Journals for Content Updates

When finished, please click submit.

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 02.04.12 in Vol 14, No 2 (2012): Mar-Apr

This paper is in the following e-collection/theme issue:

    Original Paper

    Breakeven, Cost Benefit, Cost Effectiveness, and Willingness to Pay for Web-Based Versus Face-to-Face Education Delivery for Health Professionals

    1Physiotherapy, Monash University, Melbourne, Australia

    2Allied Health Research Unit, Southern Health, Melbourne, Australia

    3Health Professions Education and Educational Research, Monash University, Melbourne, Australia

    4Health Workforce Education and Assessment Research, Monash University, Melbourne, Australia

    5Healthy Ageing Research Unit, Monash University, Melbourne, Australia

    Corresponding Author:

    Stephen Maloney, BApplSc (Physio), MPH

    Physiotherapy

    Monash University

    PO Box 527, Frankston

    Melbourne, 3199

    Australia

    Phone: 61 990 44240

    Fax:61 990 44812

    Email:


    ABSTRACT

    Background: The introduction of Web-based education and open universities has seen an increase in access to professional development within the health professional education marketplace. Economic efficiencies of Web-based education and traditional face-to-face educational approaches have not been compared under randomized controlled trial conditions.

    Objective: To compare costs and effects of Web-based and face-to-face short courses in falls prevention education for health professionals.

    Methods: We designed two short courses to improve the clinical performance of health professionals in exercise prescription for falls prevention. One was developed for delivery in face-to-face mode and the other for online learning. Data were collected on learning outcomes including participation, satisfaction, knowledge acquisition, and change in practice, and combined with costs, savings, and benefits, to enable a break-even analysis from the perspective of the provider, cost-effectiveness analysis from the perspective of the health service, and cost-benefit analysis from the perspective of the participant.

    Results: Face-to-face and Web-based delivery modalities produced comparable outcomes for participation, satisfaction, knowledge acquisition, and change in practice. Break-even analysis identified the Web-based educational approach to be robustly superior to face-to-face education, requiring a lower number of enrollments for the program to reach its break-even point. Cost-effectiveness analyses from the perspective of the health service and cost-benefit analysis from the perspective of the participant favored face-to-face education, although the outcomes were contingent on the sensitivity analysis applied (eg, the fee structure used).

    Conclusions: The Web-based educational approach was clearly more efficient from the perspective of the education provider. In the presence of relatively equivocal results for comparisons from other stakeholder perspectives, it is likely that providers would prefer to deliver education via a Web-based medium.

    Trial Registration: Australian New Zealand Clinical Trials Registry (ACTRN): 12610000135011; http://www.anzctr.org.au/trial_view.aspx?id=335135 (Archived by WebCite at http://www.webcitation.org/668POww4L)

    J Med Internet Res 2012;14(2):e47)

    doi:10.2196/jmir.2040

    KEYWORDS



    Introduction

    Continuing professional development for health professionals can change clinician behavior, affect patient outcomes, and influence the health of communities [1-3]. The availability of high-quality educational resources is not enough to ensure its uptake by clinicians or its delivery by educational institutions. Uptake is affected by the quality and sustainability of the product, and the accessibility and acceptability of its delivery [4].

    Traditional delivery of continuing professional development has used live modalities such as lectures, tutorials, seminars, and conferences [5]. Emerging technologies enabling interactive Web-based learning environments have introduced further choice for both the provider and recipient [6]. Web-based coursework holds great promise for overcoming the key barriers of time and cost associated with the professional isolation often experienced by practitioners in rural and remote regions [7]. Evaluation of costs relative to effects would provide an important metric in assessing the value of educational resources.

    In the context of the education of health professionals, there are costs associated with developing and delivering a course. Product success is determined by participant satisfaction and achievement of the defined learning targets [8]. No trials have compared the relative costs, benefits, and effectiveness of Web-based and face-to-face educational approaches for health professionals in a randomized controlled trial (RCT).

    Evidence may be lacking due to the rapid expansion in the availability and acceptability of information and communication technologies used in Web-based education [9]. Furthermore, contrasting findings from economic analyses of Web-based education are often confounded by change in the assumptions that underlie the analyses, caused by changes in social expectations of access to such technologies. This creates the need to update costing models in line with any changes in social expectations—for example, some of the costs once borne by providers in providing Internet access are now paid by users who rent access to the Internet [10].

    The cost effectiveness of Web-based versus face-to-face education for improving the practical skills of health professionals has not been previously reported [9-12]. This study examined the economic efficiency of Web-based versus face-to-face short-course delivery modalities in the context of falls prevention education for health professionals. The outcomes enable educational providers, health services, and health professional learners to make informed decisions about this type of investment in health professional education.


    Methods

    Design

    The economic evaluation was conducted as part of an RCT comparing the educational outcomes of two short courses constructed for the education of health professionals in the skill of exercise prescription for falls prevention [13]. We conducted three analyses: (1) break-even analysis from the perspective of the education provider, (2) cost-effectiveness analysis from the perspective of the health service (the employer of the health professional), and (3) cost-benefit analysis from the perspective of the participant. We included sensitivity analyses, displaying different permutations of the variables that construct the results, to tailor the findings to different educational or workplace settings.

    Participants and Setting

    Participants were eligible to enroll if they held a minimum qualification of a bachelor’s degree in any health science. We included physiotherapists, occupational therapists, nurses, and exercise physiologists working in Victoria, Australia. Participants were invited to take part in the study via the Victorian Department of Health email distribution channels.

    Measurements

    Measurements of Cost

    The Web-based arm of the RCT was offered as an ongoing educational product by the delivering university after the initial RCT was completed. The course was developed over four iterations, shown in Figure 1. The alpha version of the program was informed by research scoping activities and delivered to representative consumers who volunteered to participate, including content specialists, educational specialists, and community members. Course version beta was delivered to practitioners who held a bachelor’s degree in a health science, forming the RCT phase and the collection of data on learning outcomes and willingness to pay. Course version gamma was a fee-paying version of the course delivered to postgraduate clinicians, allowing validation of willingness to pay data. Course version delta was based on the modeled data, most closely simulating a realistic and ongoing short-course program.

    Data on the labor and capital required to provide the traditional and face-to-face education programs were collected either concurrently with the RCT or modeled afterward. Table 1 describes the approach to measuring these costs and the subsequent cost analyses [14,15]. We used market prices where known to reflect real-life costs of providing the program.

    Table 1. Method and analysis of cost items.
    View this table
    Figure 1. Strategy for development of the Web-based course and measurements used in the refinement and modeling of an acceptable cost framework. RCT = randomized controlled trial; WTP = willingness to pay.
    View this figure
    Measurements of Outcome

    We derived the measure of effectiveness used in the cost-effectiveness analysis from the mark awarded for the assignment and the examination score. An assessor, blind to group allocations, marked the course assignments. Examinations comprised multiple-choice questions that were automatically graded by the software that delivered the examinations.

    Mean maximum willingness to pay for the short course (beta version) was evaluated in the RCT. Participant responses were collected using an open response format and used as a proxy measure for benefit in the cost-benefit analysis [16,17]. Participants were given the option to answer a question in an anonymous Web-based questionnaire: “If this short course had not been a trial, what would be the maximum you would have been willing to pay for this course in Australian dollars (AUD)?” We asked this question in the context of four scenarios: (1) “If the course was not recognized as professional development (ongoing learning) points,” (2) “If the course had been subsidized 50% by your employer (you are indicating the presubsidized total cost),” (3) “If the course was recognized by your profession as professional development (ongoing learning) points,” (4) “If the course was recognized as prior learning (of approximately 5%–10%) toward a university postgraduate qualification, eg, master’s degree.”

    Procedure

    The RCT included concealed allocation and outcome assessment by a blinded assessor for comparing face-to-face versus Web-based educational interventions.

    Further details on the trial design, participants, setting, and interventions can be found in the accompanying paper of the original RCT [13].

    The trial was registered with the Australian New Zealand Clinical Trials Registry, ACTRN12610000135011. We obtained ethics approval through both the Southern Health Ethics Committee and the Monash University Human Research Ethics Committee.

    Analysis

    Break-even Analysis

    The break-even analysis, from the perspective of the education provider, estimated the minimum number of participants required for the course to operate without loss to the provider. The break-even point (Q: the number of participants required) is calculated by the point where the fixed costs (FC: costs that do not vary with participant numbers) and variable costs (VC: costs that vary with the number of participants) for each mode of program delivery are equaled by the savings (S) generated by participant fees, as represented by the equation Q = FC/(S – VC).

    Table 1 presents the costs considered and how they were valued. We conducted 1-way sensitivity analyses to account for possible variation in costs and savings depending on the resources and fee structure used by the education provider. Sensitivity analyses were (1) the number of hours paid for a course facilitator, (2) variation in the hourly remuneration of the facilitator, (3) class capacity (the number of students able to undertake the course at one time), (4) variation in the enrollment fee, and (5) variation in all associated costs combined.

    Cost-Effectiveness Analysis

    Health care providers often pay for their staff to attend professional development opportunities. A cost-effectiveness analysis, from the perspective of the health service, compared the relative impact of the two programs in clinical units—that is, the costs of increasing the number of trained clinicians and their measured level of clinical competence.

    We calculated the cost effectiveness for each course delivery method by first determining the quality of students’ education with each method, or quality-adjusted students educated (QASE), using the formula QASE = number of students educated × the group’s average grade. In this approach, average grade is used as a surrogate for measuring the improved ability of the staff member. To account for attrition, people who did not complete the course were given a zero in this weighting calculation. This is the measurement of effect in the incremental cost-effectiveness analysis. The incremental cost per QASE was calculated using the equation in Figure 2, resulting in an incremental cost-effectiveness ratio. Table 1 lists the costs considered in the cost-effectiveness analysis.

    We conducted a sensitivity analysis for the following scenarios: (1) participant attendance occurring during regular working hours, (2) participant attendance occurring during unpaid leave, (3) participant attendance occurring during participant leisure time, (4) an alternative fee structure based on profit relative to costs, and (5) whether student attrition was equivalent between the two educational approaches.

    Figure 2. Equation for calculating cost per quality-adjusted students educated (QASE) ratio. Cost F2F = cost to the health service of the face-to-face program; Cost Web-based = cost to the health service of the Web-based program; QASE F2F = number of quality-adjusted students educated with the face-to-face program; QASE Web-based = number of quality-adjusted students educated with the Web-based program.
    View this figure
    Cost-Benefit Analysis

    A cost-benefit analysis, from the perspective of the participant, considered costs and benefits associated with the education relevant to the participant in monetary terms to enable a direct comparison.

    Cost-benefit analysis requires the itemized costs experienced by each course participant along with the number of participants successfully completing the program. Net benefit, weighing the total expected costs against the total expected benefits, used the following equation: net benefit = (mean benefit face-to-face – mean cost face-to-face) – (mean benefit Web-based – mean cost Web-based), where mean benefit face-to-face is the benefit to the participant from the face-to-face method measured by the willingness-to-pay question; mean cost face-to-face is the cost to the participant to participate in the face-to-face method; mean benefit Web-based is the benefit to the participant from the Web-based method measured by the willingness-to-pay question; and mean cost Web-based is the cost to the participant in the Web-based method.

    Benefit was valued in monetary terms using the participant’s willingness to pay. Sensitivity analyses included the following variables: (1) whether the participant or health service paid for enrollment, and (2) whether the course occurred including during work hours or during leisure time. To test the construct validity of the willingness to pay values obtained from RCT participants, we compared the values obtained for the course contexts with Likert ratings of overall course satisfaction using Spearman rho. In this, we hypothesized that willingness to pay values should be associated with course satisfaction ratings if the willingness to pay values reflect the construct of the benefit from participation in the program [18].


    Results

    Demographics

    Participant demographics (Table 2) indicate a relatively even distribution of baseline characteristics between the face-to-face and Web-based mode participants.

    Table 2. Chi-square test outcomes for binary data and 2-sample t test outcomes for years since qualification.
    View this table

    Randomized Trial

    The RCT results indicated that there were no differences in outcomes between groups, except that Web-based education participants reported spending significantly more time (median, interquartile range of 1.0, 0.8–2.0 hours compared with 0.0, 0.0–1.0 hours) engaged with the additional learning materials than the face-to-face group (rank sum P = .002). The mean (SD) mark (used for calculating QASE in the cost-effectiveness analysis) for the combined examination and practical assignment grades was Web-based, 83.2% (9.9), and face-to-face, 81.6% (9.5).

    Break-Even Analysis

    Figure 3 presents the relationship between the costs and savings, for the primary scenario for face-to-face and Web-based delivery. Table 3 presents the fixed and variable costs considered in this analysis. Table 4 presents a sensitivity analysis, exploring the impact of variations in costs and savings. The break-even point for the primary Web-based scenario was obtained at 7 participants, whereas the primary scenario with the face-to-face delivery returned multiple break-even points (Table 5). Multiple break-even points occur in some of the sensitivity analyses when recurring fixed costs are incurred as a class reaches its enrollment capacity, causing the course to once again run at a loss until the new break-even point is reached as enrollments increase. This particular relationship, with the creation of multiple break-even points, is presented graphically for the face-to-face delivery approach in Figure 3.

    Table 3. Fixed and variable costs (AUD $) for Web-based and face-to-face course delivery, for a maximum class size of 20.
    View this table
    Table 4. Break-even and sensitivity analyses for Web-based course delivery mode.
    View this table
    Table 5. Break-even and sensitivity analyses for the face-to-face course delivery mode.
    View this table
    Figure 3. Savings versus costs for enrollment, with savings set at AUD $250 per participant and maximum class size of 20 participants.
    View this figure

    Cost-Effectiveness Analysis

    The face-to-face educational approach, in course iteration beta, began with 68 participants, with 49 students completing the summative assessments with an average grade of 81.6%, or 38.98 QASE. Of the 67 participants who began the Web-based delivery, 44 completed the program with an average grade of 83.2%, or 35.78 QASE.

    Through maintaining this attrition rate and mean grade for each delivery method, for a full class of 20 enrollments, the incremental cost-effectiveness ratio from the perspective of the health service for the primary analysis (scenario 1, Table 6) yields an incremental cost-effectiveness ratio of zero, therefore making face-to-face education more preferable from the health service perspective due to the higher rate of QASE.

    Table 6. Sensitivity analysis of incremental cost per quality-adjusted students educated (QASE) (D$/DQASE) for Web-based (Web) and face-to-face (F2F) course delivery.
    View this table

    Table 6 shows a sensitivity analysis. The scenarios include manipulations of the costs carried by the health service, including the impact of the timing of course delivery. The analyses include an alternative course fee structure to account for a provider business model that calculates the fee as a percentage of profit above costs, increasing the enrollment fee for face-to-face participants. The relationship (difference in gradients) between face-to-face and Web-based costs for the primary scenarios (Figure 3) yields an inflation factor of 2.1; therefore, we adjusted the alternative face-to-face fee to AUD $525.00. The sensitivity analysis also includes a scenario of equal attrition rates between the face-to-face and Web-based approaches.

    Cost-Benefit Analysis

    Table 7 presents estimated costs incurred by the participant, along with a sensitivity analysis. Net benefit to the individual participant was calculated as AUD $60.88 in favor of the face-to-face program delivery, meaning that the Web-based program would need to cost $60.88 less than the face-to-face program to create equivalent benefit for the consumer. Table 8 presents willingness to pay for each mode of program delivery. Table 9 presents sensitivity analyses of the net benefit.

    Table 9 presents the values of the cost-benefit analysis. The aforementioned alternative fee structure was applied in these analyses.

    Table 7. Primary analysis and sensitivity analyses of participant expenses (in AUD $) for Web-based versus face-to-face course delivery modes.
    View this table
    Table 8. Participant willingness to pay for Web-based versus face-to-face course delivery modes.
    View this table
    Table 9. Cost-benefit analysis results from the participant’s perspective considering varying scenarios for Web-based (Web) versus face-to-face (F2F) course delivery modes.
    View this table

    Discussion

    Economic evaluation of face-to-face versus Web-based delivery has shown that the outcome depends on the stakeholder’s perspective and the conditions applied to the analysis. Web-based education is the superior approach in the break-even analysis from the perspective of the education provider. The increased costs of face-to-face delivery create increased risk of financial loss if enrollment numbers are low. This is supported by what is seen in practice, where educational institutions are reluctant to run short courses in regional or remote areas where potential enrollments are lower. This barrier supports the notion that Web-based education has the potential to be more accessible and less discriminatory [4,19]. Cost-effectiveness analysis from the perspective of the health service favors face-to-face education, but the relatively small difference in quality of the clinicians’ knowledge obtained from face-to-face and Web-based education means that the preference for an educational approach largely depends on when the course is undertaken and its impact on covering service delivery. Cost-benefit analysis, from the perspective of the participant, produced the most participant benefit from face-to-face education. Aside from the strong cost-minimization advantages to the education provider, the other analyses are contingent on the unique conditions and sensitivity analysis applied. This further highlights the novel methods used in this study, demonstrating the strong relationship between the provider, the health service, and the participant.

    Economic analysis for health education has primarily focused on telemedicine technology or medical reviews by remote physicians, or has been concerned with the cost effectiveness of modalities for patient education [20,21] rather than upgrading the health professional’s skills. To our knowledge this is the first economic evaluation of competing approaches to providing continuing professional development with tested educational equivalence in an RCT including the outcomes of satisfaction, knowledge construction, and self-reported change in practice. It is possible that the findings may be somewhat context specific in terms of the content area and nature of the health disciplines involved, the specific subject of the educational materials, and the quality of the educational materials presented in the chosen medium. However, as the principles of Web-based and face-to-face education that we investigated could be transferred to other disciplines and settings, this report provides a model that can be applied to different scenarios, modified with associated unique assumptions.

    This study has highlighted the precarious balance that exists between the various stakeholders involved in education programs. Our break-even analysis highlighted that using the same fee structure for both Web-based and face-to-face modalities would lead to substantially greater profits for a Web-based course assuming equal enrollments; however, this would result in lower levels of benefits to the participants in the program. However, if the university were to choose to maintain the same relative profit ratio for Web-based and face-to-face courses, they would be able to offer the Web-based course at a lower cost to gain equivalent educational outcomes and participant benefit; society would therefore gain a greater volume of skilled professionals for a lower cost.

    Potential miscellaneous benefits from each delivery method were excluded from the analysis due to the difficulty in quantifying them. For example, it is hypothesized that face-to-face education may foster stronger feelings of an education community, socialization, and networking opportunities [22], whereas Web-based education may provide better maintenance of corporate knowledge that is often lost with key personnel in short-course training and lost to the profession in situations such as retirement or maternity leave [23].

    Other limitations that may have affected the findings of the analysis relate to the original RCT, the modeling of data, and underlying assumptions in the analysis formulas. The RCT measured the learning outcomes and change in practice behavior approximately 1 week following the conclusion of the training program. We do not know whether the effect of the training would have been different if the effects had been measured over a longer time frame. We also do not know whether the change in clinician practice behavior will result in real change in patient outcomes over time. We modeled data for the analysis on the fourth iteration of the training program, excluding costs involved in constructing the programs and additional expenses from “teething errors” in delivering the course with unfamiliar staff. These issues are particularly relevant to the introduction of the Web-based modality. Phelps et al [11] recognized that setting up a Web-based course, or converting an existing course to a Web-based modality, requires a large human capital investment. Our experience in creating the course resources for the RCT reflected this sentiment, but we also found that this argument was true of beginning a face-to-face program. Audiovisual resources for the Web-based course were ideally created for both course delivery methods, but rather than being used as direct student resources in the face-to-face delivery, they were used as a way of training face-to-face presenters and maintaining corporate knowledge. We anticipate this to be cost prohibitive for many organizations without significant information and communication technology (ICT) infrastructure or with staff unfamiliar with using basic computers and software to commence a Web-based course [9,10], particularly with the expectation of making short-term savings.

    Each formula used in the analysis involves calculations based on assumptions. These assumptions may be invalid depending on the context of the analysis or variation in reader perspective. Wells [24], Phelps et al [11], and Rumble [12] all reported that the shifting assumptions in information communication technologies affect the ability to contrast studies investigating the use of these technologies in education, such as who pays for what costs, the existing ICT infrastructure, and the degree of assumed user ICT literacy. For example, Rumble’s [12] investigation required the university to supply computers to all students involved in the study. It could be argued that students are now assumed to have access to their own personal computer or readily accessible university or workplace computer facilities. In our study we required students to have their own access to a computer and the Internet.

    The issue of who pays for what costs will naturally affect the economic evaluations. Our cost-benefit analysis assumed that students who did not complete the program did not obtain benefit from the small portion of the program that they may have completed. We averaged transport costs among the study cohort and limited them to transport by car. In reality, participants would have experienced a significant range of transport costs, which naturally depend on the location of the face-to-face delivery venue. The formulas used to calculate cost benefit and cost effectiveness require data on participant attrition in each delivery method. Participants were enrolled in the training program of the RCT at no cost. This lack of financial commitment to the program is likely to have negatively affected student attrition. This factor may have applied more to the Web-based education intervention, as it required a prolonged time commitment from students in its delivery over a 4-week period.

    Although we do not know the true generalizability of the findings, the transparency of this analysis will assist decision makers to tailor the findings to their area of interest and delivery setting, whether it is undergraduate or postgraduate education. Key factors in this decision making would include the ICT capabilities of the provider, similarity of their desired course to the constructivist pedagogy used in the practical skill-teaching methods of this analysis, and other particular circumstances or resources of the provider.

    We used willingness to pay as a proxy for determining participants’ perceived benefit of participating in the program. Willingness to pay may have been underreported if participants considered that their responses might potentially influence the cost of education in a market in which they are the consumer. We expect that this bias would have minimal impact on the analysis between the two interventions, as the effect is likely to be equal between the two groups. Another potential bias influencing the willingness to pay values, and therefore the benefit and cost effectiveness of the delivery methods, is a possible perception that Web-based education produces greater savings for the educational institution, and that these savings should be passed on to the participant via savings in course fees [25]. Drummond et al [18] noted that it is often impossible to validate willingness to pay values within economic analyses due to the modeling of data being commonly based on theoretical or hypothetical products or services. This study was able to provide values based on programs purposefully designed for equivalent content delivery and experienced by the participants who provided the willingness to pay values. Construct validity of the willingness to pay values obtained was demonstrated through the moderate degree of correlation between the willingness to pay values and the participants’ ratings of overall satisfaction (Table 9). Furthermore, the fee (AUD $250) used as the primary scenario in the economic evaluations was used in the fee-paying gamma version course iterations, a course that reached its full enrollment capacity (20 participants), indicating that this fee was a viable real-life fee structure and is comparable with fees for other Web-based professional development programs of equivalent length on the market.

    QASE as represented in the cost-effectiveness analysis assumes that the health care service costs are the same for each delivery group. This may be debated depending on whether the participant requires study leave to complete the course or completes the course out of work hours. QASE and cost per QASE (Δ$/ΔQASE) is also grounded in the assumption that a clinician with better skills in falls risk client management will be a greater asset to the health service, producing better health outcomes for the clients.

    Education providers vary in their opinion as to what practical class enrollment capacity is manageable in a face-to-face delivery environment. An increase in class capacity will often result in an overall decrease in fixed costs, affecting the result of a break-even analysis [26]. This is also true of Web-based education, although given the asynchronous nature of the student and teacher interaction, Web-based education is arguably less restricted by the course duration in providing supervision and feedback on participant practice, and in turn in producing greater flexibility in discerning the maximum class capacity.

    We have highlighted several areas for further research in exploring the economics of delivery approaches to continuing professional development. These include further investigation into the attrition rates when contrasting free versus full fee paying. Investigating the allocation of costs in continuing professional development for health care professionals, particularly the contribution to the course fee between the participant and the health service, would allow greater insights into the underlying assumptions in analyses. More broadly, we would encourage research into using this approach (ie, break-even analysis from the perspective of the provider, cost-effectiveness analysis from the perspective of the health service, and cost-benefit analysis from the perspective of the participant) to explore other educational approaches. Ultimately, an RCT would be conducted to investigate what may be the largest assumption in this study, that a more highly skilled clinician can affect health outcomes. For example, does the educational program to increase clinicians’ management of falls-risk clientele, studied in this evaluation, have the ability to reduce falls in the community?

    Conclusions

    Economic analysis of Web-based versus face-to-face training for improving the clinical performance of health professionals varies depending on the type of analysis and stakeholder perspective undertaken. The Web-based educational approach was clearly more efficient from the perspective of the education provider. In the presence of equivocal results for comparisons from other stakeholder perspectives, it is likely that providers will seek to deliver education via a Web-based medium in preference to an alternative face-to-face approach.

    With both Web-based and face-to-face delivery modalities offering their own unique strengths and weaknesses, it is important for decision makers to consider the application of these findings to their goals, their risk management capabilities, and their role in the delivery of high-quality health care to the community through the effective and efficient education of health service clinicians.

    Acknowledgments

    The authorship team would like to acknowledge the Department of Health, Victoria, Australia, for their sponsorship facilitating this research. Terry Haines was supported by a Career Development Award from the Australian National Health and Medical Research Council.

    Conflicts of Interest

    The authorship team has no conflicts of interest to declare in relation to the presented data or the original RCT on which it is based. Associate Professor Terry Haines is Director, Hospital Falls Prevention Solutions Pty Ltd. This company provides services for the prevention of falls in the hospital setting, which was not directly related to the subject of this research. Associate Professor Terry Haines has provided expert witness testimony in the area of prevention of falls in hospitals, which also was not directly related to the subject of this research.

    References

    1. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance. A systematic review of the effect of continuing medical education strategies. JAMA 1995 Sep 6;274(9):700-705. [Medline]
    2. Davis DA, Taylor-Vaisey A. Translating guidelines into practice. A systematic review of theoretic concepts, practical experience and research evidence in the adoption of clinical practice guidelines. CMAJ 1997 Aug 15;157(4):408-416 [FREE Full text] [Medline]
    3. Mazmanian PE, Davis DA. Continuing medical education and the physician as a learner: guide to the evidence. JAMA 2002 Sep 4;288(9):1057-1060. [Medline]
    4. Rovai AP, Downey JF. Why some distance education programs fail while others succeed in a global environment. Internet Higher Educ 2010;13(3):141-147. [CrossRef]
    5. Davis D, O'Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA 1999 Sep 1;282(9):867-874 [FREE Full text] [Medline]
    6. Bennett NL, Casebeer LL, Kristofco RE, Strasser SM. Physicians' Internet information-seeking behaviors. J Contin Educ Health Prof 2004;24(1):31-38. [CrossRef] [Medline]
    7. Curran VR, Fleet L, Kirby F. Factors influencing rural health care professionals' access to continuing professional education. Aust J Rural Health 2006 Apr;14(2):51-55. [CrossRef] [Medline]
    8. Jones D. A practical unit cost approach to budgeting and accountability in colleges. In: Levacic R, editor. Financial Management in Education. Milton Keynes: Open University Press; 1989.
    9. Ng KC. Costs and effectiveness of online courses in distance education. Open Learn 2000;15(3):301-308. [CrossRef]
    10. Bartley SJ, Golek JH. Evaluating the cost effectiveness of online and face-to-face instruction. Educ Technol Soc 2004;7(4):167-175.
    11. Phelps RH, Wells RA, Ashworth RL, Hahn HA. Effectiveness and costs of distance education usuing computer-mediated communication. Am J Distance Educ 1991;5(3):7-19. [CrossRef]
    12. Rumble B. Online costs: interactivity at a price. In: Mason R, Kaye A, editors. Communication, Computers and Distance Education. Oxford: Pergamon Press; 1989.
    13. Maloney S, Haas R, Keating JL, Molloy E, Jolly B, Sims J, et al. Effectiveness of Web-based versus face-to-face delivery of education in prescription of falls-prevention exercise to health professionals: randomized trial. J Med Internet Res 2011;13(4):e116 [FREE Full text] [CrossRef] [Medline]
    14. Becker G. A theory of the allocation of time. Econ J 1965;75(299):493-451.
    15. Feather P, Shaw WD. Estimating the cost of leisure time for recreation demand models. J Environ Econ Manage 1999;38(1):49-65. [CrossRef]
    16. Haines T, Kuys SS, Morrison G, Clarke J, Bew P. Cost-effectiveness analysis of screening for risk of in-hospital falls using physiotherapist clinical judgement. Med Care 2009 Apr;47(4):448-456. [CrossRef] [Medline]
    17. Bayoumi AM. The measurement of contingent valuation for health economics. Pharmacoeconomics 2004;22(11):691-700. [Medline]
    18. Drummond M, O'Brien B, Stoddart G, Torrance G. Methods for the Economic Evaluation of Health Care Programmes. 2nd edition. Oxford: Oxford University Press; 1997.
    19. Rabinowitz HK, Diamond JJ, Markham FW, Paynter NP. Critical factors for designing programs to increase the supply and retention of rural primary care physicians. JAMA 2001 Sep 5;286(9):1041-1048 [FREE Full text] [Medline]
    20. Bryce CL, Zickmund S, Hess R, McTigue KM, Olshansky E, Fitzgerald K, et al. Value versus user fees: perspectives of patients before and after using a web-based portal for management of diabetes. Telemed J E Health 2008 Dec;14(10):1035-1043. [CrossRef] [Medline]
    21. Currell R, Urquhart C, Wainwright P, Lewis R. Telemedicine versus face to face patient care: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2000(2):CD002098. [CrossRef] [Medline]
    22. Besser H, Bonn M. Impact of distance independent education. J Am Soc Inf Sci 1996;47(11):880-883. [CrossRef]
    23. Na Ubon A, Kimble C. Knowledge management in online distance education. In: Banks S, Goodyear P, Hodgson V, McConnell D, editors. Proceedings of the 3rd International Conference: Networked Learning 2002. Sheffield: Sheffield University Press; 2002.
    24. Wells RA. Computer-Mediated Communication for Distance Education: An International Review of Design, Teaching, and Institutional Issues. University Park, PA: American Center for the Study of Distance Education, College of Education, Pennsylvania State University; 1992.
    25. Haines TP, McPhail S. Patient preference for falls prevention in hospitals revealed through willingness-to-pay, contingent valuation survey. J Eval Clin Pract 2011 Apr;17(2):304-310. [CrossRef] [Medline]
    26. Carr S. Union publishes guide citing high cost of distance education. Chron High Educ 2001;47(35):39-41.


    Abbreviations

    ICT: information and communication technology
    QASE: quality-adjusted students educated
    RCT: randomized controlled trial


    Edited by G Eysenbach; submitted 03.01.12; peer-reviewed by L Warmerdam, S Fraser, B Das; comments to author 27.01.12; revised version received 13.02.12; accepted 22.02.12; published 02.04.12

    ©Stephen Maloney, Romi Haas, Jenny L Keating, Elizabeth Molloy, Brian Jolly, Jane Sims, Prue Morgan, Terry Haines. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 02.04.2012.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.