JMIR Publications

Select Journals for Content Updates

When finished, please click submit.

This paper is in the following e-collection/theme issue:

    Original Paper

    Development and Evaluation of the Personal Patient Profile-Prostate (P3P), a Web-Based Decision Support System for Men Newly Diagnosed With Localized Prostate Cancer

    1Dana-Farber Cancer Institute, Boston, United States

    2University of Washington, Biobehavioral Nursing & Health Systems, Seattle, United States

    3University of Saskatchewan, Saskatoon, Canada

    4University of Washington, Department of Urology, Seattle, United States

    5University of Washington, Biobehavioral Nursing & Health Systems, Medical Education & Biomedical Informatics, Seattle, United States

    6Swedish Neuroscience Institute, Swedish Medical Center, Seattle, United States

    Corresponding Author:

    Donna L Berry, PhD RN FAAN

    Dana-Farber Cancer Institute

    44 Binney St

    CP 305

    Boston, 02115

    United States

    Phone: 1 617.632.1909

    Fax:1 617.582.8550

    Email:


    ABSTRACT

    Background: Given that no other disease with the high incidence of localized prostate cancer (LPC) has so many treatments with so few certainties related to outcomes, many men are faced with assuming some responsibility for the treatment decision along with guidance from clinicians. Men strongly consider their own personal characteristics and other personal factors as important and influential to the decision. Clinical researchers have not developed or comprehensively investigated interventions to facilitate the insight and prioritizing of personal factors along with medical factors that are required of a man in preparation for the treatment decision.

    Objectives: The purpose of this pilot study was to develop and evaluate the feasibility and usability of a Web-based decision support technology, the Personal Patient Profile-Prostate (P3P), in men newly diagnosed with LPC.

    Methods: Use cases were developed followed by infrastructure and content application. The program was provided on a personal desktop computer with a touch screen monitor. Participant responses to the query component of P3P determined the content of the multimedia educational and coaching intervention. The intervention was tailored to race, age, and personal factors reported as influencing the decision. Prepilot usability testing was conducted using a “think aloud” interview to identify navigation and content challenges. These issues were addressed prior to deployment in the clinic. A clinical pilot was conducted in an academic medical center where men sought consultation and treatment for LPC. Completion time, missing data, and acceptability were measured.

    Results: Prepilot testing included 4 men with a past diagnosis of LPC who had completed therapy. Technical navigation issues were documented along with confusing content language. A total of 30 additional men with a recent diagnosis of LPC completed the P3P program in clinic prior to consulting with a urologist regarding treatment options. In a mean time of 46 minutes (SD 13 minutes), participants completed the P3P query and intervention components. Of a possible 4560 items for 30 participants, 22 (0.5%) were missing. Acceptability was reported as high overall. The sections of the intervention reported as most useful were the statistics graphs, priority information topics, and annotated external website links.

    Conclusions: The P3P intervention is a feasible and usable program to facilitate treatment decision making by men with newly diagnosed LPC. Testing in a multisite randomized trial with a diverse sample is warranted.

    J Med Internet Res 2010;12(4):e67)

    doi:10.2196/jmir.1576

    KEYWORDS



    Introduction

    There is a growing body of evidence that men with a recent diagnosis of localized prostate cancer (LPC) conduct the treatment decision-making process by strongly considering their own personal characteristics and other personal factors [1-7]. Recently diagnosed men must often assume some responsibility for the treatment decision together with guidance from clinicians. The participation of patients with cancer in making decisions about treatment is promoted by virtually all interested parties including professional societies, researchers, and clinicians. Information widely available via the Internet, though of variable accuracy, has helped to hasten a new dynamic between patient and clinician. Because no other disease with the high incidence of prostate cancer has so many alternative treatments with so few certainties related to outcome, many men are faced with assuming some responsibility for the treatment decision along with guidance from clinicians.

    Yet clinical researchers have not comprehensively investigated interventions to facilitate the gaining of insight and the prioritizing personal factors as well as the decision making that are required of a man with a diagnosis of LPC. Decision support technologies provide much needed information to patients, but (1) focus solely on medical factors considered relevant by physicians (eg, histology, comorbidity, and age), (2) fail to customize the information to the personal characteristics of the patient [8-10], or (3) depend on interventions that have never been rigorously tested in randomized trials with diverse samples [2,11,12]. The goal of this ongoing program of research is to improve the decision-making experience for men with LPC by highlighting personal characteristics and factors that men bring to the treatment decision: their desired level of participation in decision making, the importance of potential outcomes and complications, current symptoms, priority information topics, the influence of others, race/ethnicity, and self-perception of age.

    This research was informed by O’Connor’s Decision Support Framework (DSF) [13]. The framework is most appropriate for health care situations in which careful deliberation is required because of many uncertainties and value-sensitive risk/benefits and for which the deliberation phase (deciding) requires substantially more effort than the implementation phase (undergoing a particular management strategy or therapy). The DSF is organized by (1) determinants of decisions, (2) decision support interventions, and (3) evaluation of both the process and outcomes of the decision support. Since our research team had documented the determinants, that is, the personal factors brought to the decision by men with LPC [1,7], we were poised to engage in the second step, designing the support intervention and evaluating the process.

    In this paper, we report the iterative development and initial evaluation of a tailored Internet patient decision support system, the Personal Patient Profile-Prostate (P3P), in which all of these factors are assessed and addressed. The research aims included the following system requirements: (1) specifications of use cases, (2) application architecture and content, (3) usability, and (4) feasibility.


    Methods

    Preclinical Design Overview

    An iterative development approach was employed, beginning with the development of use cases (see Multimedia Appendix 1) by members of the research team. Within these cases and through contextual inquiry with potential users and investigators (including content and informatics experts), we used a structured process for identifying end users, requirements, and application content. In addition to gathering end user requirements, we closely adhered to National Cancer Institute’s recommendations about appropriate user interface design [14]. The application was initially implemented in 2004. Detailed methods are presented below with respect to the preclinical design phase, with methods grouped by architecture, query, and intervention content.

    Application Architecture

    The application architecture for P3P utilizes an open source Web software platform and provides for a flexible survey environment [15]. The survey environment also enables the overlay of interventional content The design employs a modular, extensible approach built on the generalized storage and display of survey instruments. All survey content associated with a specific instrument is stored in a database, and “assessments” can be compiled from multiple survey instruments. Each survey instrument is represented as a reusable object containing questions, possible answers, and control logic. The software retrieves these objects and displays them to the user, recording answers as well as metadata such as time stamps and navigation information. A survey editor allows researchers to make changes to the content and the sequence of questions without software modifications. The survey framework also includes a patient manager to enter patient demographics and manage the patient data associated with a specific administration of an assessment in a clinical setting.

    The Web application was implemented using the PHP language and MySQL database on a Linux/Apache server, a development platform commonly known as "LAMP” [16]. All user interface components were implemented as dynamic server-side pages. Surveys were presented using templates, making it simple to adapt the system to conform to user interface guidelines and to a variety of device characteristics. Touch screen monitors were utilized to display and access the P3P program. This hardware, along with the navigational design of the program, eliminates the use of a mouse and scrolling. Keyboarding is not required but is available for optional open-ended items. User interface widgets (radio buttons, checkboxes, and navigation buttons) can be resized to work well with touch screens via a simple configuration option.

    Application Content

    Profile Query Component

    The purpose of the profile query was to gather the input necessary for the tailored intervention component. The P3P opening screen introduced the participant to the nature and purpose of the intervention, that is, to help make “the best choice for you,” providing information and guidance to understand the participant’s personal concerns. In addition to demographic characteristics, the following valid and reliable instruments were presented in the query component of P3P in order to generate the intervention (Table 1). All instruments were adapted for the touch screen by presenting 1 item per screen, with the exception of some items from the Expanded Prostate Cancer Index Composite Short Form 6.2002 (EPIC-SF) that were logically presented next to each other (see below). Additional questionnaires were presented to the patient during the query component which were outcomes of using the P3P tailored intervention in a future randomized trial, and we thought best to pilot test the inclusion of instruments to measure state anxiety [17] and decisional conflict [18]. Finally, we included an acceptability assessment [19]. Reading grade level of the application’s internal content was calculated using the Flesch-Kincaid measure for an average of 7.6 (SD 1.6) and ranging from 5.4 to 10.0. Skipping questions without answering was allowed.

    Table 1. Variables and questionnaire results used to generate the P3P tailored intervention
    View this table
    Personal Profile

    The Personal Profile was developed by the investigators based on the earlier qualitative work [1] and was designed for and used with 260 men during a descriptive quantitative study of the personal factors that influence men’s treatment decisions [7]. Face validity and test-retest reliability of the Personal Profile have been established [7]. The profile contains ranking of the following personal factors with regard to influence on the decision or importance to the decision: influential people (spouse/partner, family member, coworker, friend, and celebrity), influential outcomes (bladder, bowel and sexual function, and expected survival) and personal characteristics (confidence in the doctor, age, work, and recreational activities). Each of these item responses was listed as “no influence,” “a little influence,” “some influence,” or “a lot of influence.”

    Control Preferences Scale and Information Priorities

    The Control Preferences and Informational Priorities were the 2 components of the Patient Information Program (PIP) developed by Davison and colleagues [20]. The first component of the PIP uses the Control Preferences Scale modified by Davison [23] to elicit patients’ preferences for control over treatment decision making. The second component of the PIP focuses on identifying priority information topics and is based on a paper and pencil survey previously developed and validated by Davison in samples of men newly diagnosed with prostate cancer.

    Expanded Prostate Cancer Index Composite Short Form 6.2002

    Prostate cancer-targeted symptoms were assessed using the 4 prostate-targeted symptom domains developed by Wei and his colleagues [22]: Sexual, Hormonal, Urinary, and Bowel. Each scale of the EPIC-SF includes a function subscale and a “bother” item. In P3P, the final item of the EPIC-SF, assessing the patient’s perception of “how big a problem” for 5 hormonal symptoms, was displayed in a matrix on 1 screen.

    Outcome Measures (Did Not Create an Intervention Component)
    Anxiety

    Anxiety was measured with the 20-item State component of State-Trait Anxiety Inventory (STAI) [17], which is also known as the Self-Evaluation Questionnaire.

    Decisional Conflict

    The Decisional Conflict Scale (DCS) [18] measured the conflict inherent in the treatment decision encountered by the men in the sample. Of the subscales of the DCS, 2 (Uncertainty and Factors-Contributing-to-Uncertainty) are appropriate for use before or during decision-making and the third (Effectiveness of Decision Making) for use after the decision has been made.

    Intervention Component

    Immediately after a participant had completed the query component, the P3P intervention was delivered to him in 5 distinct sections. First, the participant was shown a screen that listed the levels of decisional control preference with the earlier selected level highlighted. The participant was then instructed to play the video clip matching the selected control level. Next, the 4 priority categories of “information needed today” that were ranked highest in the Patient Information Program component were displayed as brief narrative text on-screen summarizing the priority topic, and a full page teaching sheet was printed for each topic. Third, a statistics tutorial was displayed utilizing the highest ranked influential outcome: survival, bladder, bowel, or sexual function. The screen included explanatory text and an exemplar percentage chart such as 17 frowning faces and 83 smiling faces in a rectangular matrix of 100 faces.

    The fourth section began with a screen containing a menu of topics covering the influential factors that had been ranked as having some or a lot of influence on treatment decision making. When viewed, each topic included a brief narrative description of the issue and a corresponding video clip depicting a patient discussing the topic with a physician. Finally, an option to explore 4 reputable, informational prostate cancer websites [24] to which an annotated guide and links were provided was presented as the final intervention component. The clinicians on this investigative team reviewed the websites for current information.

    Table 2 summarizes how the intervention was tailored to the patient’s personal profile and provides links to screenshots in the Multimedia Appendices.

    Table 2. P3P intervention customization by the patient’s personal profile
    View this table
    Acceptability

    At the completion of the program, aspects of patient acceptability (easy, understandable, enjoy, helpful, time, value of information, and overall satisfaction) were measured for the entire P3P with the Acceptability E-scale, a 1 to 5 scale anchored by 1 = not easy at all (for example) to 5 = very easy (for example) [19]. We added 1 item, “value of information,” to our previous scale. An additional set of investigator-developed usefulness queries was presented using a similar 1 to 5 scale and focused on the unique sections of the intervention.

    Prepilot Usability Testing in Proxy Patients

    The University of Washington Human Subjects Division approved all study procedures and materials. In all, 4 English-speaking men who were at least 6 months post-prostate cancer treatment were recruited from a university-based prostate cancer clinic practice to test the usability of the P3P program. Each consenting participant was asked to complete the P3P prototype using a touch screen monitor and desktop computer in an informatics laboratory at the University of Washington, School of Nursing. Simultaneously, an audio-recorded cognitive clinical interview [25], also known as the “think-aloud” method [26], was conducted by a graduate nursing student (author JW). The purpose of this usability testing was to asses how men interacted with the technology and to identify problems with the interface and/or content. The subjects were asked to “think-aloud” as they went through the questions and the intervention on-screen in order to understand their experience of the system.

    Clinical Pilot Testing

    A total of 32 English-speaking men with newly diagnosed LPC who sought consultation at the University of Washington Medical Center’s Prostate Oncology Center were invited to participate by clinic nursing staff. Of these, 30 men provided informed consent and were enrolled by a research team member. Participants used the program on a touch screen monitor with a full-size keyboard connected to a desktop computer in the center’s patient education room. The team member was waiting in an adjacent room and available for assistance if needed and made summary notes of each participant’s session including any usability issues reported, feedback offered, and whether a spouse or partner viewed the program with the participant. The participant then proceeded to the consult visit with 1 or more prostate cancer specialty physicians within an hour of using the intervention. Figure 1 displays the clinical and application flow.

    Figure 1. Clinical and application flow
    View this figure

    Analysis

    Usability testing data from the prepilot were analyzed descriptively with quantitative content analysis [27]. In addition, the following quantitative measures were used for the clinical pilot results: completion time, data completeness, and acceptability scores. The questionnaire results will be reported elsewhere.


    Results

    Prepilot

    The program and cognitive interview took 1 to 1½ hours to complete. The interviews were audio-recorded and transcribed verbatim. Table 3 lists the tasks and observations and responses from the testing. Minor edits were made to the P3P program based on these formative findings.

    Table 3. Summary of participants’ responses and observations during prepilot usability testing (N=4)
    View this table

    Clinical Pilot

    Of the 30 participants, 26 (87%) completed query and intervention components of the P3P program in less than 1 hour (mean 46 minutes, SD 13 minutes, range 16 to 69 minutes). The sample had a mean age of 61, ranging from 45-74. Missing data were minimal: 15 participants answered every one of the 152 items, 8 men skipped 1 item (0.6%), and 7 men skipped 2 items (1.3%). Of a possible 4560 items for 30 participants, 22 (0.5%) were missing. All participants watched the tailored video clip describing the identified decision control preference, and 7 men watched additional clips representing other control preferences. In all, 10 men viewed a video clip from the menu of personal factor results. The majority, 28 of the 30 men, viewed at least 1 set of text and graphic statistics tutorials about survival, incontinence, impotence, or bowel disturbance.

    Overall acceptability of P3P was calculated as high (Table 5). At least 67% of the sample chose a 4 or 5 on each of the acceptability items

    Participants reported on the usefulness of the specific intervention components viewed (Table 6). Over half of all participants who viewed each component reported usefulness at 4 or 5 on the 5-point scale. In all, 5 participants did not have time to access or chose not to view the additional websites.

    There were no significant relationships between age, education, or work status and any usefulness outcome (data not shown). Many verbal questions/comments were made by participants regarding future home access to the program. Many participants also said that the information would have been more useful a few days prior to the consult with the urologist. Participants reported perceived time pressure in the clinic to finish viewing all components before being called in to see the physician. Additional comments were written by 6 men; of these, 5 offered critiques of wording or display in the program, and 1 man endorsed the content on family impact.

    Research team member notes were available for 23 of 30 sessions and indicated 2 usability problems: 1 participant was unsure how to navigate away from an informational prostate cancer website pop-up linked from the intervention, and another did not know how to print teaching sheets at the end of the program. An additional 3 men were noted to have trouble hearing the videos. Also, 13 men were reported to have viewed the intervention together with a spouse or partner, while 10 did not.

    Table 4. Demographic characteristics of participants (N = 30)
    View this table
    Table 5. Overall P3P Acceptability (N=30)
    View this table
    Table 6. P3P intervention component usefulness
    View this table

    Discussion

    The P3P was successfully developed, tested, and deployed in an academic medical center urology clinic by an interdisciplinary research team. The acceptability and usefulness scores plus verbal and written comments have given us areas with which to further revise and develop the program.

    There was a high level of interest in participating. Travel logistics were cited by the 2 men who did not participate as the primary reason for not enrolling. This problem together with comments requesting access to the program a few days before the options review visit indicated a strong need for home access to the intervention.

    Men may also have been more likely to view the menu items with which they were the least familiar, notably statistics. Only 10 men were able to, or chose to, view the video clips providing coaching on how to speak with one’s doctor regarding influential personal factors. We are uncertain as to why the other 20 men did not view the video clips although this could be explained by lack of time or misunderstanding of the screen instructions or menu display. The men may have been constrained by time at the completion of the program since they immediately went into their clinic visit with the urologist. There is some evidence from a subsequent cognitive interview study of African American men using P3P [28] that the navigation of the menu display for the video clips was not readily apparent.

    Methods of evaluation applied in this study have been used by other health technology evaluations of tailored intervention in 15 healthy women relevant to preventing osteoporosis [29] and 13 Hispanic family caregivers relevant to health promotion [30]. Findings of all these trials indicate that performance usability testing in settings that mirror the intended use setting can successfully identify areas and functions of the applications that require modification. Patient-centered decision support technologies posted on the Internet or provided in larger electronic health systems that have not been exposed to rigorous usability testing are inherently suspect for poor generalizability and potential end user dissatisfaction at best, or poor uptake and disuse, at worst.

    The reported acceptability of this tailored decision aid is comparable to that of other electronic, Web-based applications developed by this research team and colleagues. The electronic self-report assessment for cancer (ESRA-C) is a screening assessment for patients with all cancer diagnoses shown to be efficacious for improving clinician patient communication [31]. Wolpin et al [32] reported similarly high levels of acceptability for the ESRA-C application including 342 ambulatory patients with cancer who had completed the program at 2 time points. Intervention websites in health care have included quantitative acceptability measures typically developed by the researcher as study-specific scales or item sets and which have not been validated. For example, DiLorio and colleagues [33] reported the success of a Web-based self-management program for epilepsy from participants’ perspectives with regard to overall satisfaction and component-specific satisfaction. While we were unable to compare the scores because the instruments varied somewhat, the approach was feasible in both studies and both differentiated satisfaction with various program components. Furthermore, our scale for user self-report of satisfaction and acceptability has been tested for reliability and dimensionality. Tariman et al [19] analyzed a sample of 627 respondents and reported that the Acceptability E-scale was found to have a consistency coefficient of .76, good item-to-item and item-to-scale correlations, and was unidimensional.

    A large team of researchers recently developed the International Patient Decision Aid Standards Collaboration instrument (IPDASi) [34] as a means to evaluate decision support technologies. While the P3P contains each of the 9 applicable dimensions, a quantitative scoring of P3P has not been conducted using the IPDASi.

    Our pilot findings are limited by the evaluative scope of the study, namely feasibility, usability, and satisfaction with a new decision support system. Our participants were fairly well educated with most household incomes in the 4th or 5th quartile for the region and predominately white, precluding generalization beyond this group of men. However, the results have guided the redesign and deployment of P3P for testing in a multisite, randomized trial with a diverse sample [35]. The new application is accessible from remote (home) locations on varied hardware and software.

    In conclusion, our preliminary evidence suggests that the P3P is a useful and acceptable decision support system that feasibly can be deployed in a clinical practice setting. The program enables men with early stage prostate cancer to identify and understand the personal issues and factors that influence a treatment decision and coaches men to articulate those issues and factors to the consulting physician.

    Acknowledgments

    The authors wish to thank Martha Foster, RN, for her dedicated assistance in participant recruitment and study deployment. Andrew Ward, Cliff Solomon, Ashley Bond, Alan Gojdics, and John Balzarini were major contributors to video production. Funding was provided by the National Institutes of Health, grant number NIH R21 CA 100025; the sponsor had no role in manuscript preparation or approval.

    Conflicts of Interest

    None declared

    Multimedia Appendix 1

    Personal Patient Profile-Prostate (P3P): Use case in urology

    PDF file (Adobe PDF File), 49 KB

    Multimedia Appendix 2

    Screenshot of CPS video highlighted for patient’s selection

    PDF file (Adobe PDF File), 184 KB

    Multimedia Appendix 3

    Decisional control video: Active participation

    PDF file (Adobe PDF File), 651 KBWMV file (Windows Media Video) 6,144 KB

    Multimedia Appendix 4

    Influential people video: Coworker

    PDF file (Adobe PDF File), 635 KBWMV file (Windows Media Video) 9,216 KB

    Multimedia Appendix 5

    Screenshot of a statistics teaching text/graphic page

    PDF file (Adobe PDF File), 254 KB

    Multimedia Appendix 6

    Influential outcome video: Bladder

    PDF file (Adobe PDF File), 451 KBWMV file (Windows Media Video) 9,216 KB

    Multimedia Appendix 7

    Screenshot of an EPIC urinary domain item

    PDF file (Adobe PDF File), 49 KB

    Multimedia Appendix 8

    Screenshot of a current symptom coaching page

    PDF file (Adobe PDF File), 152 KB

    References

    1. Berry DL, Ellis WJ, Woods NF, Schwien C, Mullen KH, Yang C. Treatment decision-making by men with localized prostate cancer: the influence of personal factors. Urol Oncol 2003;21(2):93-100. [Medline]
    2. Feldman-Stewart D, Brundage MD, Van Manen L. A decision aid for men with early stage prostate cancer: theoretical basis and a test by surrogate patients. Health Expect 2001 Dec;4(4):221-234. [Medline]
    3. Davison BJ, Gleave ME, Goldenberg SL, Degner LF, Hoffart D, Berkowitz J. Assessing information and decision preferences of men with prostate cancer and their partners. Cancer Nurs 2002 Feb;25(1):42-49. [Medline]
    4. Patel HR, Mirsadraee S, Emberton M. The patient's dilemma: prostate cancer treatment choices. J Urol 2003 Mar;169(3):828-833. [CrossRef] [Medline]
    5. Steginga SK, Occhipinti S, Gardiner RA, Yaxley J, Heathcote P. Making decisions about treatment for localized prostate cancer. BJU Int 2002 Feb;89(3):255-260. [Medline]
    6. Diefenbach MA, Dorsey J, Uzzo RG, Hanks GE, Greenberg RE, Horwitz E, et al. Decision-making strategies for patients with localized prostate cancer. Semin Urol Oncol 2002 Feb;20(1):55-62. [Medline]
    7. Berry DL, Ellis WJ, Russell KJ, Blasko JC, Bush N, Blumenstein B, et al. Factors that predict treatment choice and satisfaction with the decision in men with localized prostate cancer. Clin Genitourin Cancer 2006 Dec;5(3):219-226. [Medline]
    8. Nexcura. Prostate Cancer Profiler Tool. 2010.   URL: https://www.cancerfacts.com/Secure/InterfaceSecure.asp?CB=14 [accessed 2010-06-06] [WebCite Cache]
    9. Moul JW, Esther TA, Bauer JJ. Implementation of a web-based prostate cancer decision site. Semin Urol Oncol 2000 Aug;18(3):241-244. [Medline]
    10. Onel E, Hamond C, Wasson JH, Berlin BB, Ely MG, Laudone VP, et al. Assessment of the feasibility and impact of shared decision making in prostate cancer. Urology 1998 Jan;51(1):63-66. [Medline]
    11. Diefenbach MA, Butz BP. A multimedia interactive education system for prostate cancer patients: development and preliminary evaluation. J Med Internet Res 2004 Jan 21;6(1):e3 [FREE Full text] [CrossRef] [Medline]
    12. Feldman-Stewart D, Brundage MD, Van Manen L, Skarsgard D, Siemens R. Evaluation of a question-and-answer booklet on early-stage prostate-cancer. Patient Educ Couns 2003 Feb;49(2):115-124. [Medline]
    13. O'Connor AM, Tugwell P, Wells GA, Elmslie T, Jolly E, Hollingworth G, et al. A decision aid for women considering hormone therapy after menopause: decision support framework and evaluation. Patient Educ Couns 1998 Mar;33(3):267-279. [Medline]
    14. Koyanl S, Bailey R, Nall J. Research-based Web Design & Usability Guidelines. Bethesda: National Cancer Institute: Office of Communications; 2003:1-128.
    15. Dockrey MR, Lober WB, Wolpin SE, Rae LJ, Berry DL. Distributed health assessment and intervention research software framework. AMIA Annu Symp Proc 2005:940. [Medline]
    16. Lee JM, Ware B. In: Ware B, editor. Open source Web development with LAMP: using Linux, Apache, MySQL, Perl, and PHP. Boston: Addison-Wesley; 2003.
    17. Spielberger C. In: Spielberger C, editor. State-Trait Anxiety inventory for Adults (Form Y). Palo Alto: Mind Garden, Inc.; 1983.
    18. O'Connor AM. Validation of a decisional conflict scale. Med Decis Making 1995;15(1):25-30. [Medline]
    19. Tariman JD, Berry DL, Halpenny B, Wolpin S, Schepp K. Validation and testing of the Acceptability E-scale for Web-based patient-reported outcomes in cancer care. Appl Nurs Res 2009 Sep 17. [CrossRef] [Medline]
    20. Davison BJ, Goldenberg SL, Gleave ME, Degner LF. Provision of individualized information to men and their partners to facilitate treatment decision making in prostate cancer. Oncol Nurs Forum 2003;30(1):107-114. [CrossRef] [Medline]
    21. Degner LF, Sloan JA, Venkatesh P. The Control Preferences Scale. Can J Nurs Res 1997;29(3):21-43. [Medline]
    22. Wei JT, Dunn RL, Sandler HM, McLaughlin PW, Montie JE, Litwin MS, et al. Comprehensive comparison of health-related quality of life after contemporary therapies for localized prostate cancer. J Clin Oncol 2002 Jan 15;20(2):557-566. [Medline]
    23. Davison BJ, Parker PA, Goldenberg SL. Patients' preferences for communicating a prostate cancer diagnosis and participating in medical decision-making. BJU Int 2004 Jan;93(1):47-51. [Medline]
    24. Lipp ER. Web resources for patients with prostate cancer: a starting point. Semin Urol Oncol 2002 Feb;20(1):32-38. [Medline]
    25. Drennan J. Cognitive interviewing: verbal data in the design and pretesting of questionnaires. J Adv Nurs 2003 Apr;42(1):57-63. [Medline]
    26. Kushniruk AW, Patel VL. Cognitive and usability engineering methods for the evaluation of clinical information systems. J Biomed Inform 2004 Feb;37(1):56-76. [CrossRef] [Medline]
    27. Neuendorf K. The Content Analysis Guidebook. Thousand Oaks: Sage Publications; 2002:-301.
    28. Jaja C, Pares-Avila J, Wolpin S, Berry D. Usability evaluation of the interactive Personal Patient Profile-Prostate decision support system with African American men. J Natl Med Assoc 2010 Apr;102(4):290-297. [Medline]
    29. Ryan P, Pumilia NJ, Henak B, Chang T. Development and performance usability testing of a theory-based, computerized, tailored intervention. Comput Inform Nurs 2009;27(5):288-98; quiz 299. [CrossRef] [Medline]
    30. Moore M, Bias RG, Prentice K, Fletcher R, Vaughn T. Web usability testing with a Hispanic medically underserved population. J Med Libr Assoc 2009 Apr;97(2):114-121. [CrossRef] [Medline]
    31. Berry D. Electronic Self-Report Assessment for Cancer (ESRA-C): Results of a Randomized Clinical Trial. 2008 Presented at: American Society of Clinical Oncology Annual Meeting; June 8, 2008; Chicago.
    32. Wolpin S, Berry D, Austin-Seymour M, Bush N, Fann JR, Halpenny B, et al. Acceptability of an Electronic Self-Report Assessment Program for patients with cancer. Comput Inform Nurs 2008;26(6):332-338. [CrossRef] [Medline]
    33. DiIorio C, Escoffery C, McCarty F, Yeager KA, Henry TR, Koganti A, et al. Evaluation of WebEase: an epilepsy self-management Web site. Health Educ Res 2009 Apr;24(2):185-197 [FREE Full text] [CrossRef] [Medline]
    34. Elwyn G, O'Connor AM, Bennett C, Newcombe RG, Politi M, Durand MA, et al. Assessing the quality of decision support technologies using the International Patient Decision Aid Standards instrument (IPDASi). PLoS One 2009;4(3):e4705 [FREE Full text] [CrossRef] [Medline]
    35. Berry DL. Using the Personal Patient Profile-Prostate (P4): evidence of the digital divide. 2008 Presented at: American Society of Clinical Oncology Genitourinary Cancers Symposium; February 14-16, 2008; San Francisco.


    Abbreviations

    DSF: Decision Support Framework
    ESRA-C: electronic self-report assessment for cancer
    EPIC-SF: Expanded Prostate Cancer Index Composite Short Form 6.2002
    IPDASi: International Patient Decision Aid Standards Collaboration instrument Personal
    PIP: Patient Information Program
    P3P: Personal Patient Profile-Prostate


    Edited by G Eysenbach; submitted 21.05.10; peer-reviewed by F Magrabi; comments to author 14.06.10; revised version received 22.06.10; accepted 03.11.10; published 17.12.10

    ©Donna L Berry, Barbara Halpenny, Seth Wolpin, B Joyce Davison, William J Ellis, William B Lober, Justin McReynolds, Jennifer Wulff. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 17.12.2010  

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.