JMIR Publications

Select Journals for Content Updates

When finished, please click submit.

This paper is in the following e-collection/theme issue:

    Original Paper

    An Internet-Based Patient-Provider Communication System: Randomized Controlled Trial

    1Division of General Internal Medicine, University of Colorado at Denver and Health Sciences Center, Denver, CO, USA

    2Colorado Health Outcomes Program, University of Colorado at Denver and Health Sciences Center, Denver, CO, USA

    Corresponding Author:

    Chen-Tan Lin, MD

    UCDHSC Division of General Internal Medicine

    4200 E Ninth Avenue B180

    Denver, CO 80262

    USA

    Phone: +1 303 372 9084

    Fax:+1 303 372 9082

    Email:


    ABSTRACT

    Background: Internet-based interactive websites for patient communication (patient portals) may improve communication between patients and their clinics and physicians.

    Objective: The aim of the study was to assess the impact of a patient portal on patients’ satisfaction with access to their clinic and clinical care. Another aim was to analyze the content and volume of email messages and telephone calls from patients to their clinic.

    Methods: This was a randomized controlled trial with 606 patients from an academic internal medicine practice. The intervention “portal” group used a patient portal to send secure messages directly to their physicians and to request appointments, prescription refills, and referrals. The control group received usual care. We assessed patient satisfaction at the end of the 6-month trial period and compared the content of telephone and portal communications.

    Results: Portal group patients reported improved communication with the clinic (portal: 77/174 [44%] “a little better” or “a lot better;” control: 18/146 [12%]; χ2 = 38.8, df = 1, P < .001) and higher satisfaction with overall care (portal: 103/174 [59%] “very good” or “excellent;” control: 78/162 [48%]; χ2 = 4.1, df = 1, P = .04). Portal group patients also reported higher satisfaction with each of the portal’s services. Physicians received 1 portal message per day for every 250 portal patients. Total telephone call volume was not affected. Patients were more likely to send informational and psychosocial messages by portal than by phone. Of all surveyed patients, 162/341 (48%) were willing to pay for online correspondence with their physician. Of those willing to pay, the median amount cited was US $2 per message.

    Conclusions: Portal group patients demonstrated increased satisfaction with communication and overall care. Patients in the portal group particularly valued the portal’s convenience, reduced communication barriers, and direct physician responses. More online messages from patients contained informational and psychosocial content compared to telephone calls, which may enhance the patient-physician relationship.

    J Med Internet Res 2005;7(4):e47)

    doi:10.2196/jmir.7.4.e47

    KEYWORDS



    Introduction

    The Institute of Medicine report “Crossing the Quality Chasm” [1] cites “the free flow of information” and “the patient as the source of control” as key features of patient-centered care. Information technology can play a vital role in meeting patient needs. Internet applications may be increasingly useful now that 73% of US adults have access to the Internet [2]. Previous studies have reported that 90% of patients with Internet access would like to communicate via email with their physician and that 56% indicate that it would influence their choice of doctor [3,4].

    To meet this growing demand, some organizations have implemented Internet-based websites for communication between patients and their clinic and physician (patient portals) [4-10]. However, physicians have been hesitant about communicating online with patients, citing the potential for increased workload, increased medical liability, and risks to patient privacy [11-14].

    Previous studies of Internet-based patient-provider communication include a randomized controlled trial [5] and several observational studies [4,6,11,12]. These studies established that online messaging between patients and physicians was an important medium of communication that was well accepted by patients yet used rarely by physicians. Although patient satisfaction with online messaging has been described, direct comparisons between online messaging and telephone call volume and content are lacking.

    To better understand these issues, we conducted a randomized controlled trial while implementing a patient portal in an academic general internal medicine practice. This portal allowed patients to request appointments, prescription refills, and specialist referrals. It also allowed them to send secure electronic messages to their physician. We hypothesized that this portal would improve patient satisfaction with clinic operations. We were also interested in how the portal’s messaging system was used and how the volume and content of that use differed from telephone calls. Finally, we assessed the utility of the patient portal by asking patients to place a monetary value on its services.


    Methods

    Setting

    The study was conducted at an academic ambulatory internal medicine practice affiliated with the University of Colorado Hospital in Denver, CO. Fourteen physicians and staff in the practice were already using an electronic medical record (EMR) (Care Innovation, version 8.0, 3M Health Information Systems, Salt Lake City, UT), which included an electronic messaging system to document patients’ incoming telephone calls. All 14 physicians participated in the study, answering messages from portal patients as well as controls. Clinic staff typed the content of incoming calls into the EMR. Staff nurses retrieved these messages electronically, called the patient, and documented their conversation or consulted the physician electronically prior to calling the patient. At baseline, physicians also communicated occasionally with patients using standard electronic mail (Outlook 2000, Microsoft Corporation, Redmond, WA). The volume of these email messages was small.

    The study was approved by the Colorado Multiple Institutional Review Board.

    Recruitment and Randomization

    Patients were enrolled from August 2002 through February 2003. The study period began at the conclusion of enrollment and lasted 6 months from March 1, 2003 through August 31, 2003. Eligible patients were at least 18 years old, English speaking, and had experience using an Internet browser. Patients were recruited via descriptive brochures, a poster, and a research assistant in the practice waiting room and via additional brochures in the examination rooms. Two broadcast emails were sent to 6000 employees of the University of Colorado Health Science Center, many of whom were eligible patients. An article about the study was also distributed to 2000 employees in the hospital’s newsletter. Patients were enrolled into the study after completing an online informed consent and initial survey.

    Patients were consecutively assigned to intervention (access to the portal) or control (usual telephone care) groups by a research assistant according to a predetermined randomization scheme developed using SAS (SAS, version 8.2, SAS Institute Inc, Cary, NC), with equal numbers of portal and control participants in blocks of 10. Since patients in the portal group could send messages to physicians through the portal, physicians and clinic staff could not be blinded to the enrollment status of patients.

    Portal and Control Groups

    Portal group patients were instructed to register a username and password for the patient portal that the University of Colorado Hospital named “My Doctor’s Office” (Patient Online, version 6.0, IDX Systems Corporation, Burlington, VT). See Figure 1 and Multimedia Appendix 1 for screenshots of the portal website. They could then send requests for appointments, prescription refills, and referrals to the clinic staff and send clinical messages to their physician. Portal group patients were warned in advance not to send urgent messages using the portal. A clinic staff member copied incoming portal messages to the existing telephone messaging system three times daily (excluding weekends or holidays). Clinical messages were sent directly to the physician, who could send an electronic response to the patient or forward the message with instructions to clinic nurses. For patients in the control group, access to the portal was delayed until end of study. Instead, the control group received access to a website providing general health advice. Both portal and control patients could contact the clinic by telephone at their discretion or for urgent messages. The incoming telephone call triage system (for both portal and control patients) via the EMR was unchanged.

    Figure 1. Screenshot of My Doctor’s Office portal
    View this figure

    Broadcast emails were sent monthly during the study to patients in both groups. When these messages were returned due to an invalid email address, the research assistant attempted to retrieve correct contact information. Patients who could not be reached were disenrolled from the study. Also, portal group patients who had not registered their username and password were encouraged on five occasions via email to register.

    Outcome Measures

    Patient Satisfaction

    The primary outcome measures were patient satisfaction with the following: communication, overall care by the clinic, administrative requests (appointments, prescriptions, referrals), and clinical messaging (by portal and by telephone) with their physician. Patients completed surveys before and after the 6-month trial period (Multimedia Appendices 2-4). The initial survey assessed demographics and baseline satisfaction with clinic services. The final survey assessed satisfaction with clinic services for all participants and the perceived utility of the patient portal in the portal group. Satisfaction was assessed on a 5-point scale (1 = poor, 2 = fair, 3 = good, 4 = very good, 5 = excellent), with an option for patients to indicate that they “never did this.” Surveys were adapted by the investigators from prior instruments [4,6], pilot tested in a subset of nonstudy patients, and administered online. To maximize response rates, patients were contacted weekly by email for 4 weeks at the end of the 6-month trial period.

    Qualitative Content Analysis of Administrative Requests and Clinical Messages

    The patient portal tallied administrative requests and clinical messages. In all, 148/174 (85%) of portal patients and 142/166 (86%) of controls gave consent for investigators to review their medical record for the content of the clinical messages received via portal and telephone. The portal tracking system counted each incoming message separately, even if it was part of one clinical message exchange between patient and physician. During content review we only counted completed clinical message exchanges. We excluded clinical messages encompassing routine prescription refills, appointment requests, and referral requests from the content analysis.

    We compared and categorized the content of clinical messages sent by patients in the portal and control groups. We adopted categories from a previous study [15] (request test information, request test action, request medication information, request medication action, miscellaneous) and added the following categories: urgent message, biomedical concern, psychosocial concern (eg, depression, anxiety, insomnia), FYI (for your information), home monitoring, and prevention. We identified message responses in which the physician elaborated on the advice given by the triage nurse and those in which the physician responded directly to the patient by a telephone call or portal messaging. Two of the investigators (CTL, LW) independently categorized message content according to this schema and met to resolve any interrater disparities.

    Value to Patients

    We asked patients whether they would pay for electronic correspondence with their doctor and, if so, to indicate a dollar value for this correspondence, per completed transaction. We calculated the median and mean dollar amounts provided by the patients who responded that they would pay for this correspondence.

    Statistical Methods

    SAS was used for all statistical calculations. All tests of statistical significance were 2-tailed, with alpha = .05. With an anticipated sample size of approximately 300 participants per group and a 50% survey response rate, the study was designed to have 80% power to detect a difference of 15% in the proportions of the portal group and control group who rated overall communications with the clinic as “a lot better.”

    Data were examined to determine frequencies and normality of responses. Baseline characteristics and outcomes for both groups were assessed with Student t-tests or nonparametric Wilcoxon rank sum tests for count variables and chi-square tests for categorical variables or Fisher exact tests when appropriate. The nonparametric Kendall tau rank correlation was used to assess the correlation between categorical variables.


    Results

    The complete study data are included in Multimedia Appendix 5.

    Participants

    Demographic characteristics of the portal and control groups were comparable at the beginning of the study. Over 70% of the participants in both groups were college graduates and had a household income of at least US $45000 (Table 1). Proportionately, 96/305 (31%) of portal patients and 111/301 (37%) of controls registered using “uch.edu” or “uchsc.edu” email addresses and could be identified as employees of the University of Colorado Hospital or Health Sciences Center (χ2 = 2.0, df = 1, P = .16). This was reflective of the general patient population at the study clinic.

    Table 1. Baseline demographics
    View this table

    In all, 606 patients completed a baseline questionnaire. Of the 305 patients who were allocated to the portal group, 256 (84%) obtained a user account for the patient portal, and 95 (31%) used the portal during the trial period (Figure 2).

    A similar proportion of participants in the portal and control groups were lost to follow-up: 42 (14%) in the portal group and 46 (15%) in the control group. Of those who were sent the final survey, 175 (67%) of portal patients and 166 (65%) of controls responded. We compared overall satisfaction with care on the initial survey between participants who completed the study and those who did not (defined as those lost to follow-up plus those who never completed the final survey). Those who did not complete the study were less satisfied on the initial survey compared to those who did complete the study (completed study: 137/341 [40%] reported last interaction with clinic as “very good” or “excellent;” did not complete study: 84/265 [32%]; χ2 = 7.3, df = 1, P = .007). However, of those who completed the study, there was no significant difference in initial satisfaction with last clinic interaction between the portal group and controls (portal group: 106/305 [35%] reported last interaction with clinic as “very good” or “excellent;” control group: 115/301 [38%]; χ2 = 0.3, df = 1, P = .57).

    Figure 2. Flow of participants throughout study period
    View this figure

    Outcomes

    Patient Satisfaction

    At the end of the 6-month study, portal group patients were more likely to indicate that communication with the clinic had improved, and they were more likely to rate clinic services as “very good” or “excellent” compared to controls. Portal group patients were also more satisfied than controls with overall clinic services, and, for those portal group patients who used the administrative services, they were more satisfied with each of the services (appointments, refills, and referrals) and with clinical messaging (Table 2).

    In subgroup analysis, portal group patients who never used the portal (portal group nonusers) were similar to controls. The only “overall service” in which portal group nonusers differed from controls was satisfaction with “current communication with the clinic compared with the beginning of the study.” The only “specific service” in which portal group nonusers differed from controls was in the subgroup of patients who scheduled appointments.

    Table 2. Patient satisfaction at the end of 6-month study period
    View this table

    Patient satisfaction with the portal was high: of the 114/175 (65%) who reported using the portal, 80 (70%) indicated that they were “satisfied” or “very satisfied” with portal services, 92 (81%) indicated that it saved them a telephone call to the clinic, and 37 (33%) indicated that it saved them a visit to the clinic during the 6-month trial.

    Of the entire portal group, 132/175 (75%) indicated they were “likely” or “very likely” to use the portal in the future, and 149 (85%) said they would prefer to use the portal over the telephone for nonurgent messages.

    To determine whether frequent users of the portal were more satisfied with clinic services, we evaluated the association between the number of times patients logged on to the portal and their responses to questions about satisfaction. There were weak positive correlations between frequency of portal use and the following: satisfaction with portal services (r = 0.18, P = .02), improved communication with the clinic since baseline (r = 0.19, P = .01), and, for those who used it (n = 99), satisfaction with physician messaging (r = 0.17, P = .03).

    Qualitative Content Analysis of Administrative Requests and Clinical Messaging

    The 95 patients who actually used the portal sent a total of 175 administrative requests (88 appointments, 72 prescription refills, and 15 referrals) and 239 clinical messages. This translated to physicians receiving, on average, 1 clinical message per day for every 250 portal group patients enrolled. Monthly volumes were stable over the course of the study. Of these requests and messages, 27% were sent during clinic hours, and 73% were sent during nonclinic hours (Figure 3). Moreover, 52% were sent from 5 pm to midnight, 12% were sent from midnight to 8 am, and 9% were sent on weekends or holidays. Although not explicitly measured, clinic staff spent about 8 hours daily answering telephones and about 5 minutes daily triaging and responding to portal requests and messages.

    Figure 3. Incoming patient portal administrative requests and clinical messages by hour of the day (n = 414)
    View this figure

    We compared the clinical messages of patients in both study groups who consented to a review of their medical record. Portal patients called 110 times and sent 76 portal messages. Control patients called 126 times. The median number of telephone calls per patient and total messages (telephone plus portal) per patient was 0 for both the portal and control groups. There was no significant difference in the number of telephone calls (U = 1.1, P = .26) or total number of contacts (U = -1.1, P = .29) between the portal and control groups. The mean number of telephone messages per patient was 0.36 (SD = 1.25) in the portal group and 0.42 (SD = 1.06) in the control group, and the mean number of total (telephone plus portal) messages per patient was 0.61 (SD = 1.79) in the portal group and 0.42 (SD = 1.06) in the control group (this is the same as the number of telephone calls since this is the only way this group contacted the clinic). In aggregate, clinical messages from portal patients and controls were similar in content (data not shown).

    Within the portal group, we also compared the content of clinical messages sent by portal and by telephone (Table 3). Patients sent more FYI (for your information) and psychosocial messages via the portal. Only 2 portal messages were deemed urgent, and the receiving physicians did not consider these problematic.

    Table 3. Content of clinical messages from portal group (comparing phone and portal messages)
    View this table

    When physicians responded to clinical messages from the portal group, they were more likely to elaborate on the advice of the triage nurse when the message was received through the portal (physician input to portal message: 73/76 [96%]; physician input to phone message: 86/110 [78%]; χ2 = 11.6, df = 1, P < .001). They were also more likely to respond directly to the patient (physician direct response to portal message: 60/76 [79%]; physician direct response to phone message: 15/110 [14%]; χ2 = 79.7, df = 1, P < .001).

    Value to Patients

    In all, 162/341 (48%) of all survey respondents were willing to pay for electronic correspondence with their doctor. Of those willing to pay, the median amount reported per message was US $2, and the mean was $4.10.


    Discussion

    In this randomized controlled trial, patients with access to an Internet-based patient portal were more satisfied with their communication with the clinic and their overall care. These patients were also more satisfied with clinical messaging with their physicians and the process of requesting appointments, prescription refills, and referrals. Patients were more likely to send FYI and psychosocial messages via the portal than by telephone. The volume of incoming messages was minimal: 1 message daily for every 250 patients offered online access. Portal and control group clinical message volumes were not significantly different.

    Why were portal patients more satisfied than controls? First, the portal was convenient: 81% believed that the portal saved them a telephone call, and 33% believed it saved them a visit to the clinic. The portal allowed patients to send messages at all hours; indeed, 73% of incoming messages were sent during nonclinic hours. Second, the portal reduced barriers to communication—portal patients were more likely to send FYI and psychosocial messages. Patients may hesitate to “bother the doctor” with FYI messages by phone, but they feel more comfortable sending a portal message. Patients may prefer sending psychosocial messages by portal because it affords privacy and distance, avoiding the aggravation of being on hold and having to speak to an intermediary. One patient even suggested to the physician that the portal was a more comfortable medium for psychosocial discussion than in-person visits. Third, patients may have appreciated that portal messages were more likely to receive direct responses from the physician, whereas telephone calls tended to be mediated by a triage nurse. Finally, the portal was efficient, providing quick message responses that likely exceeded patient expectations. A substantial majority of messages were answered the same day, even though the portal states that responses may take up to 2 business days. This is consistent with other studies that demonstrated improved patient satisfaction with shorter response time [11] and with meeting or exceeding patient expectations [16,17].

    It is clear that patients increasingly desire and are satisfied with online messaging. Physicians are much less enamored with electronic communication, driven by fears of overwhelming volume, inappropriate messaging, and inadequate security [5,10,18,19]. The increasing publicity of patients demanding such service, the lack of demonstrated adverse effects, and the possibility of insurers reimbursing physicians for online communication may narrow this satisfaction gap [8,18,20].

    In our sample, the total number of incoming messages from portal patients (portal plus phone) was not significantly different from the total number of incoming messages from controls. This implies that patients replace phone calls with electronic messages. Although Katz et al [5] showed that total message volume increases with patient access to online messaging, others have shown a replacement of phone calls with online messages [4,18,19]. Although not specifically measured, both physicians and staff noted that responding to electronic messages took less time than responding to telephone messages, even after discounting the frustration of “telephone tag.” Others have corroborated this finding [18].

    Portal patients called more times than they sent online messages. Why? Urgent calls were one third of the phone call volume. Subtracting urgent calls, portal patients were equally likely to call as they were to send an online message. Adopting a new communications medium may occur gradually, with patients not trusting the new system, forgetting how to access it, or not thinking to use it.

    The clinical utility of incoming messages is beyond the scope of this study. It is not clear, for example, how FYI messages might have impacted care. At worst, one might imagine such messages “cluttering” the patient’s medical record. At best, it might “close the loop” when patients inform their physician of the success or failure of a treatment. Although we demonstrated improvement of patient satisfaction, we are unable to state whether quality of care was affected.

    Portal group patients who never used the system were similar to controls in their satisfaction with clinic services, except that they were more satisfied with “overall communication.” These patients may have felt that simply having the portal available if they needed it was advantageous.

    Some organizations are charging patients for portal clinical messaging. Since 53% of study participants would not pay for portal messages, this could shift portal messages to “free” telephone calls, reduce FYI and psychosocial messages, and affect satisfaction. Notably, some insurers are beginning to reimburse physicians for online communications, which may partially address this concern [8].

    Limitations

    This study has several limitations. Control group patients who continued emailing their physician may have diluted the difference between groups. Our patients had relatively high incomes, educational status, and familiarity with the Internet, and one third were University of Colorado or Health Science Center employees, so these results may not be generalizable to an Internet-naive, less affluent sample. Because of the nature of the intervention (online messaging vs telephone calls), the physicians and staff could not be blinded to the process and may have paid more attention to online messages, influencing the results. The study was conducted for only 6 months; patient satisfaction could have increased (due to increased familiarity) or decreased (due to fading of initial enthusiasm) if the study was carried out over a longer period of time. We note that our final sample size (N = 95 actual users) was smaller than our desired size of 150. We reported that the portal group achieved higher satisfaction than controls for “overall care” (P = .04). A larger sample would have provided a more precise estimate of effect. Lastly, because of our recruitment method, we were unable to collect information from patients who were eligible for the study but declined to participate. Despite initial randomization of patients and comparable demographic characteristics in dropouts, those who dropped out of the portal and control groups may be different, biasing the results of the final survey.

    Summary and Future Directions

    This randomized controlled study adds to the literature by describing possible underlying reasons for patient satisfaction with online communication: convenience, reduced communication barriers, and direct physician responses. Another novel finding was that more online messages from patients contain FYI and psychosocial content compared to telephone calls. These findings may explain why patient access to an Internet-based patient portal was associated with greater patient satisfaction with communication and overall care.

    A patient portal that supports online communication is a strong foundation on which to promote “care based on continuous healing relationships” [1] and encourage collaborative care. Further research is needed to evaluate more advanced portal services and their impact on patient satisfaction, empowerment, and medical outcomes. An advanced patient portal might include shared documentation by physicians and patients, patient access to test results and other aspects of their medical record, and shared decision support to patients and physicians for chronic care improvement.

    Acknowledgments

    The authors appreciate the assistance of John F Steiner, MD, MPH, in the design and critique of the study. Diane Fairclough, PhD, provided expert statistical analysis. Kelly Davis-McPeak provided tireless technical and webmaster support to researchers, clinic staff, and patients. Funding for the study was provided by a grant from the Division of General Internal Medicine. The authors also appreciate the participation of the patients, physicians, and staff of the University Medicine—Denver Practice.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Screenshots of My Doctor’s Office portal. PPT file, 436 KB


    Multimedia Appendix 2

    Baseline patient satisfaction questionnaire. PDF file, 92 KB


    Multimedia Appendix 3

    Final satisfaction questionnaire for subjects. PDF file, 128 KB


    Multimedia Appendix 4

    Final satisfaction questionnaire for controls. PDF file, 124 KB


    Multimedia Appendix 5

    Complete study data. PDF file, 52 KB


    References

    1. . Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press; 2001.
    2. . Email, research, news and weather, information about hobbies or special interests top the list of how people use the Internet as it continues to grow.   URL: http://www.harrisinteractive.com/harris_poll/index.asp?PID=527 [accessed 2005 Mar 12] [WebCite Cache]
    3. . Internet penetration at 66% of adults (137 million) nationwide.   URL: http://www.harrisinteractive.com/harris_poll/index.asp?PID=295 [accessed 2005 Mar 12] [WebCite Cache]
    4. Liederman EM, Morefield CS. Web messaging: a new tool for patient-physician communication. J Am Med Inform Assoc 2003;10(3):260-270. [Medline] [CrossRef]
    5. Katz SJ, Nissan N, Moyer CA. Crossing the digital divide: evaluating online communication between patients and their providers. Am J Manag Care 2004;10(9):593-598. [Medline]
    6. Sittig DF, King S, Hazelhurst BL. A survey of patient-provider e-mail communication: what do patients think? Int J Med Inf 2001;61(1):71-80. [Medline] [CrossRef]
    7. Fowles JB, Kind AC, Craft C, Kind EA, Mandel JL, Adlis S. Patients’ interest in reading their medical record: relation with clinical and sociodemographic characteristics and patients’; approach to health care. Arch Intern Med 2004;164(7):793-800. [Medline] [CrossRef]
    8. Delbanco T, Sands DZ. Electrons in flight — e-mail between doctors and patients. N Engl J Med 2004;350(17):1705-1707. [Medline] [CrossRef]
    9. Earnest MA, Ross SE, Wittevrongel L, Moore LA, Lin CT. Use of a patient-accessible electronic medical record in a practice for congestive heart failure: patient and physician experiences. J Am Med Inform Assoc 2004;11(5):410-417. [Medline] [CrossRef]
    10. Ross SE, Moore LA, Earnest MA, Wittevrongel L, Lin CT. Providing a Web-based online medical record with electronic communication capabilities to patients with congestive heart failure: randomized trial. J Med Internet Res 2004;6(2):e12. [Medline] [CrossRef]
    11. Kleiner KD, Akers R, Burke BL, Werner EJ. Parent and physician attitudes regarding electronic communication in pediatric practices. Pediatrics 2002;109(5):740-744. [Medline] [CrossRef]
    12. Moyer CA, Stern DT, Dobias KS, Cox DT, Katz SJ. Bridging the electronic divide: patient and provider perspectives on e-mail communication in primary care. Am J Manag Care 2002;8(5):427-433. [Medline]
    13. Mandl KD, Kohane IS, Brandt AM. Electronic patient-physician communication: problems and promise. Am Intern Med 1998;129(6):495-500. [Medline]
    14. . Standards for privacy of individually identifiable health information.   URL: http://www.hipaadvisory.com/regs/regs_in_PDF/finalprivmod.pdf [accessed 2005 Jul 12] [WebCite Cache]
    15. Kravitz RL, Bell RA, Franz CE. A taxonomy of requests by patients (TORP): a new system for understanding clinical negotiation in office practice. J Fam Pract 1999;48(11):872-878. [Medline]
    16. Lin CT, Albertson GA, Schilling LM, Cyran EM, Anderson SN, Ware L, et al. Is patients’ perception of time spent with the physician a determinant of ambulatory patient satisfaction? Arch Intern Med 2001;161(11):1437-1442. [Medline] [CrossRef]
    17. Thompson AG, Sunol R. Expectations as determinants of patient satisfaction: concepts, theory and evidence. Int J Qual Health Care 1995;7(2):127-144. [Medline] [CrossRef]
    18. Liederman EM, Lee JC, Baquero VH, Seites PG. Patient-physician web messaging. The impact on message volume and satisfaction. J Gen Intern Med 2005;20(1):52-57. [Medline] [CrossRef]
    19. Hassol A, Walker JM, Kidder D, Rokita K, Young D, Pierdon S, et al. Patient experiences and attitudes about access to a patient electronic health care record and linked web messaging. J Am Med Inform Assoc 2004 Nov;11(6):505-513. [Medline] [CrossRef]
    20. Freudenheim M. Digital Rx: take two aspirins and e-mail me in the morning. The New York Times. 2005 Mar 2.   URL: http://query.nytimes.com/gst/abstract.html?res=FB061FFC38590C718CDDAA0894DD404482 [accessed 2005 Mar 12] [WebCite Cache]

    Edited by G. Eysenbach; submitted 19.04.05; peer-reviewed by E Ammenwerth; comments to author 27.05.05; revised version received 01.07.05; accepted 09.07.05; published 05.08.05

    © Chen-Tan Lin, Loretta Wittevrongel, Laurie Moore, Brenda L Beaty, Stephen E Ross. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 5.8.2005. Except where otherwise noted, articles published in the Journal of Medical Internet Research are distributed under the terms of the Creative Commons Attribution License (http://www.creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited, including full bibliographic details and the URL (see "please cite as" above), and this statement is included.