Published on 18.10.10 in Vol 12, No 4 (2010): Oct-Dec
The Establishment of an Ethical Guideline for Genetic Testing Through Citizen Consensus via the Internet in Taiwan
Background: With the rapid advance of genetics, the application of genetic testing has become increasingly popular. Test results have had a tremendous impact on individuals who receive the test and his or her family. The ethical, legal, and social implications (ELSI) of genetic testing cannot be overlooked. The Internet is a potential tool for public engagement.
Objectives: This study aimed at establishing ethical guidelines for genetic testing in Taiwan through a participatory citizen consensus approach via the Internet.
Methods: The research method used was a citizen consensus conference modified by an Internet application and the Delphi technique. The citizen consensus conference is one of the public participation mechanisms. The draft ethical guidelines for genetic testing were written by an expert panel of 10. The Delphi technique was applied to a citizen panel recruited via the Internet until a consensus was reached. Our research population was restricted to people who had Internet access.
Results: Included in the citizen panel were 100 individuals. A total of 3 individuals dropped out of the process. The citizen panel was exposed to the issues through Internet learning and sharing. In all, 3 rounds of anonymous questionnaires were administered before a consensus was reached in terms of importance and feasibility. The result was ethical guidelines composed of 4 categories and 25 items. The 4 categories encompassed decision making (6 items), management of tissue samples (5 items), release of results (8 items), and information flow (6 items). On a scale of 1 to 10, the average (SD) importance score for the decision-making category was 9.41 (SD 0.58); for the management of tissue samples category, the average score was 9.62 (SD 0.49); for the release of results category, the average score was 9.34 (SD= 0.59); and for the information flow category, the average score was 9.6 (SD = 0.43). Exploratory analyses indicated that participants with higher education tended to attribute more importance to these guidelines.
Conclusions: The resulting recommended ethical guidelines had 4 categories and 25 items. We hope through the implementation of these guidelines that mutual trust can be established between health care profession and the general public with respect to genetic tests.
J Med Internet Res 2010;12(4):e47
A genetic test is defined as the analysis of human DNA, RNA, chromosomes, proteins, and certain metabolites in order to detect heritable disease-related genotypes, mutations, phenotypes, or karyotypes for clinical purposes . Every year, 4 million genetic tests are performed in the United States; many of these tests have been commercialized in England [ ]. Genetic testing has been conducted for various purposes such as the prediction of hereditary disease, diagnosis and treatment, disease prevention, health promotion, and newborn screening. It is foreseeable that the frequency of genetic testing will continue to grow rapidly in the future.
Nonetheless, the ethical, legal, and social implications (ELSI) of genetic testing cannot be overlooked. The test results often have a tremendous impact on the lives of the individuals who receive the test and his or her family. As such, the 1998 World Health Organization’s Proposed International Guidelines on Ethical Issues in Medical Genetics and Provisions of Genetic Services recommended that:
Every genetic test should be offered in such a way that individuals and families are free to refuse or accept according to their wishes and moral beliefs. All testing should be preceded by adequate information about the purpose and possible outcomes of the test and potential choices that might arise. Children should only be tested when it is for the purpose of better medical care .
More importantly, genetic testing is not only an issue of individual choice, but also of social choice. Health care professionals should conduct such testing in compliance with social norms so as to avoid the potential chaos that genetic testing is capable of creating.
Various ethical, legal, and social issues have been raised in the past regarding genetic testing. For instance, in 2008 the American College of Obstetricians and Gynecologists (ACOG) issued a committee opinion on ethical issues in genetic testing that included informed consent, prenatal genetic testing, genetic data and the family, genetic data and insurers and employers, genetics and assisted reproductive technology, in addition to other topics. The more contentious issues included confidentiality, privacy, the right of minors, the balance between the rights of individuals and their families, and potential discrimination by employers or insurers [- ].
Due to the clear need of guidance in translating genetic discoveries into clinical care, guidelines in this regard have been developed throughout the world . There have been fewer such efforts in Asia. The most notable are the guidelines for genetic testing published in Japan by 10 societies concerned with issues in genetic medicine [ ]. There are no such guidelines in Taiwan at present; therefore, the purpose of this research was to establish ethical guidelines for genetic testing in Taiwan through a citizen consensus approach via the Internet.
Most professional ethical guidelines have been written entirely within the profession. The use of expert focus groups is one of the commonly applied methods for developing ethical guidelines . However, it has been vigorously asserted that authors of ethical guidelines and the manner of their compilation will determine whether the guidelines themselves are ethical [ ]. Meeting the needs of the public is an important aspect of genetic testing and can be decided by the public. Although ethical guidelines for genetic testing are the conduct guidelines for health care professionals, allowing citizen participation in the formulation process gathers more public voices and facilitates meeting society’s expectations.
Engaging citizens in policy making has attracted much attention in recent years. Public participation, public engagement, or public involvement refer to interactions between the public and decision-making bodies, and the guiding principles for these activities are transparency and openness, which ensure that decisions are made based on the best available evidence .
For instance, member countries of the Organization for Economic Cooperation and Development (OECD) also strengthen their relations with citizens in order to improve the quality of policy . As stated in a policy brief produced by the OECD [ ], in addition to the necessity of having adequate information and consultation, active participation of citizens helps generate policy options. A citizen consensus conference is cited as one of the tools that can facilitate active participation. The OECD report also recognized information and communication technology (ICT) as a powerful tool to engage citizens.
There are 3 classes of public engagement based on the flow of information between participants and sponsors: public communication, public consultation, and public participation . In public communication, information is conveyed from sponsors of the initiatives to the public; in public consultation, information is conveyed from the public to the sponsors; while in public participation, information is exchanged between the public and the sponsors. Public engagement is enacted through a variety of structured mechanisms that are many in number but generally poorly defined [ ]. What works best when is a major concern.
The research method we applied was the citizen consensus conference modified by Internet application and Delphi technique.
Originating in Denmark, the citizen consensus conference is a method of public opinion extraction that gives ordinary citizens opportunities to make their voices heard in technology policy debates . This kind of conference provides lay citizens with sufficient information to deliberate public polices. Citizen consensus conferences are touted as being able to increase an ordinary citizen’s opportunities to participate in public affairs, and the policy dialogue process provides ordinary citizens with ample information to participate in public discussions and debates [ ]. The steps include (1) issue framing, (2) organizing the steering committee, (3) choosing the lay panel, (4) preparatory meetings, (5) formulating questions and choosing the expert panel, (6) conducting the public forum, and (7) writing the lay panel consensus statement [ ]. The process itself is lengthy and time consuming. With the widespread application of the Internet, and in light of the recommendation from the OECD, we sought to modify the processes in this research via the use of Internet communication.
According to a typology developed by Rowe , there are a total of 3 classes (public communication, public consultation and public participation) and 13 types of public engagement mechanisms. The citizen consensus conference, classified as participation type 1, focuses more on laypersons than experts [ ]. This type of mechanism is characterized by the controlled selection of participants, facilitated group discussion, unconstrained participant responses, and flexible information input from sponsors in the form of experts available for questioning [ ]. Since the aim was to have the guidelines decided by the public, and our research team has had past experiences with the same mechanism in a face-to-face fashion [ ], a citizen consensus conference modified with Internet application therefore became the mechanism of choice.
This research was divided into 3 phases, each of which is described in more detail below. In the first phase, we used the Internet to recruit citizens to join the research. We recruited and randomly sampled volunteers and obtained written consent from the final list of participants. The next phase was to invite the citizen panel to participate in Internet learning, sharing, and discussion. Written background material was distributed to the participants via email. Participants could engage in exchanges and discussions through the group email list where our expert panel also participated via group emailing. Meanwhile, we drafted ethical guidelines for genetic testing and invited an expert panel of 10 to review content validity. The third phase was to apply the Delphi technique to the citizen panel until a consensus was reached in terms of importance and feasibility of items contained in the guidelines.
By posting Internet announcements to recruit the lay panel, our research population sample was restricted to people who had Internet access. The announcement was posted for 1 month in popular local Internet portals such as Yam and Kijiji, as well as on websites of some universities and community colleges. The inclusion criteria were age greater than 20 years and an interest in the ethics of genetic testing. In total, 119 persons volunteered. Of the 119 in the preliminary sample, 47 had health care professional backgrounds, and 72 did not. After random sampling in a ratio of 4 to 6, the final panel included 40 people who had a health care background and 60 who did not.
The research questionnaire used in the Delphi processes was based on ethical guidelines drafted by the steering committee, which was composed of the principal investigator (PI) and the coinvestigators. The committee invited 10 experts (2 biomedical scientists, 2 clinical doctors, 2 ethics scholars, 2 lawyers, and 2 representatives from the biotechnology industry) to review the questionnaire. Each item’s content validity index (CVI) of .8 in terms of importance was preserved. Our first draft had 4 categories and 28 items; after the expert panel’s review, 4 categories and 21 items were retained in the questionnaire. Each item was scored on a scale of 1 to 10 in terms of importance and feasibility, with higher scores indicating higher levels of importance and feasibility.
The technique we applied is, in effect, a Delphi technique modified by the application of the Internet. The Delphi technique is one of the nominal group techniques that use questionnaires to build consensus. Conventionally, questionnaire surveys can only be completed by pen and paper, but these can now be conducted through the Internet. Administration of Web questionnaires has been reported in the literature to have the same reliability  as mailed pen and paper questionnaires but to have varied response rates [ ]. The advantages of Web-based questionnaire administration are time and cost savings, while the main disadvantage is that response rates depend on the level of Internet readiness of the target population [ ]. However, since our lay panel volunteered via the Internet before we started posting the questionnaires, a low response rate and lack of Internet readiness were not major concerns for our study.
In terms of data analyses, we applied CVI to determine expert content validity. Internal consistency calculations were used in the Delphi stage to determine whether a consensus had been reached. All other analyses were descriptive and exploratory in nature. The PI of this study was deemed responsible for the storage and confidentiality of the database.
In total, 3 persons dropped out of the lay panel during the process. From, we can see the composition of the initial citizen panel was 43% (42/97) male and 57% (55/97) female, with the largest age group consisting of 26 to 30 year olds (47/97 or 48%) and the second largest group consisting of 21 to 25 year olds (43/97 or 44%). In terms of education level, 57% (55/97) had or were currently engaged in a college education, while 43% (42/97) had or were engaged in a graduate education. Of note, most respondents (63/97 or 65%) declared having no religion. In terms of geographic distribution, although the respondents were scattered among 16 administrative areas, 60% (58/97) were residents from the capital area, that is, Taipei city and Taipei county.
In the Delphi stage, 3 rounds of anonymous questionnaires were conducted prior to consensus in terms of importance and feasibility, which achieved an internal consistency of Cronbach alpha .93. In the final stage, the ethical guidelines included 4 categories and 25 items. The 4 categories were: decision making, management of tissue samples, release of results, and information flow. The importance scores of each item ranged from 9.34 to 9.62 on average for these 4 categories ().
The decision-making category included 6 items that dealt primarily with autonomy and informed written consent. The management of tissue sample categories included 5 items that encompassed the scope of use, storage security, timing of destruction, and research problems. The release of the results category included 8 items and dealt with privacy issues, such as whether and how family members should be informed of test results. The information flow category included 6 items, primarily centered on confidentiality issues that emphasized how test results should be kept confidential from insurance companies, third persons, and employers. All categories and items within each category are presented in.
Whether personal characteristics affected the responses of the lay panel was further explored. In terms of importance scoring, only the variable of education had a significant influence on all 4 categories according to ANOVA. Post hoc Scheffe’s analyses indicated that the average scores of graduate students were significantly higher than those of college students in the categories of decision making (P = .047), release of results (P = .02), and information flow (P = .02), whereas in the management of tissue samples category, the average scores of educational groups in descending order were: doctoral students, people with a master degree, graduate students, and college students (P = .02). The results implied that participants with higher education tended to attribute more importance to these guidelines.
With the rapid advance of genetics, the application of genetic testing has become increasingly popular. The concept of genetic exceptionalism has been advanced by many ethicists, arguing that genetics should be subject to a more rigorous process of scrutiny due to the following reasons: (1) the results of genetic testing subject perfectly healthy individuals to discrimination due to potential future illnesses, (2) the uncertainties of genomic application including genetic testing and genetic treatment are still ominous, (3) there are consequences not only for the individuals who test, but also for family members .
Most scholars agree that, although it is imperative to respect autonomy and privacy in conducting genetic testing, health care professionals must also inform the examinee that it is his or her moral obligation to inform family members regarding hereditary risks [- ]. During our research process, we found that some ethicists contended that certain genetic tests required decision-making and sharing of information, that is, the undertaking of certain genetic tests need the approval of close family members of the same blood line, and the results of all relevant family members should be disclosed. Nonetheless, it appears from our tentative research results embodied in the draft guidelines that most Taiwanese are in favor of individual autonomy. This conflicting opinion is also reflected by the low degree of consensus in 2 of the items contained in our guidelines (items 3.5 and 3.6) compared with the other items. Item 3.5 says that the examinee or legal guardian of the examinee has the right to decide whether the family member who might be affected by the test results will be informed. And item 3.6 requires physicians and genetic counselors to encourage the examinee or the legal guardian of the examinee to disclose relevant information to the affected family members.
Because a breach of confidentiality in genetic information might affect family relations, employment, insurance, paternity law suits, and so on, and might further lead to stigmatization and discrimination, high standards of security must be established to ensure confidentiality [, ]. As a result, the guidelines resulting from our Delphi process also emphasized the importance of confidentiality and the restriction on information flow. Items 4.3 and 4.4 stress that test results should be kept confidential from insurance companies and irrelevant third persons. Item 4.5 states that when an insurer requires an individual to receive genetic testing as a condition of obtaining insurance, the individuals consent must be obtained in advance. Item 4.6 forbids employers from requesting that their employees receive genetic testing.
Another interesting phenomenon worth noting is that feasibility score averages tended to be lower than those of importance scores for the same item (). That is, although the statement was deemed important, the respondents were less confident regarding whether it could be executed in real life as written. As such, the establishment of mutual trust relating to genetic testing between health care professionals and the general public is vital for its conduct.
Furthermore, the lowest feasibility score was for item 4.6, which stated that employers be prohibited from subjecting their employees to genetic testing. This item clearly shows a lack of confidence in employers and reflects a worry that employers are unstoppable and will eventually control their employees through genetic testing in some way.
Another finding worth noting is that our lay panel inserted a contractual requirement for the health care industry. Item 4.2 states that the examiner should sign a contract with the examinee before testing to assure confidentiality. Under most health care circumstances, patients or recipients of care are requested to sign consent forms. Although the responsibilities of the physicians and caregivers are also specified in the consent forms, most people felt that their consent was sought merely to protect the health care professionals rather than themselves. In short, the requirement for informed consent was not deemed reciprocal by the general public. Therefore, it is natural for laypersons to think that they are entitled to written contracts from examiners to ensure that examiners fulfill their obligations, underlining the importance of the citizen consensus conference in giving voice to the general public.
Many countries have applied the techniques of the citizen consensus conference to explore public issues including Argentina, Australia, Austria, Japan, Netherlands, New Zealand, Canada, Denmark, France, German, Israel, Norway, Korea, Switzerland, the United Kingdom, and the United States as well as others . Citizen consensus conferences in these countries have covered a wide variety of issues. For instance, in Denmark, discussions topics have included gene technology in industry and agriculture, human genome mapping, transgenic animals, infertility, gene therapy, genetically modified foods, testing our genes, and others [ ]. Other countries have applied the same consensus mechanisms to similar topics, with gene-related topics frequently being among the topics of such discussions around the world.
Our research group adopted the same techniques to help with the revision of the code of ethics for nurses in Taiwan in 2005 and 2006 . Although we found this method useful, it was quite time consuming for lay panelists and therefore limiting in its widespread application. By modifying the guidelines with the help of the Internet and the Delphi technique, we were able to recruit a larger citizen panel to participate in the development of ethical guidelines for genetic testing. The downside was that the Internet and the larger panel size might have deterred effective communication among participants. Instances of face-to-face citizen consensus conferences, however, do not necessarily guarantee effective communication and decision making. In addition, the relatively small number of citizens who could physically participate in the consensus conference has been criticized as being unrepresentative of real-life consensus. In contrast, group emailing can increase the speed of group discussions and applications of the Delphi technique and can facilitate decision making if we wish to involve more citizens in the process.
Citizen consensus conferences need to have ample participation from the lay panel. As a result, for the modified Delphi technique to work for a citizen consensus conference, a good response rate to the questionnaires is very important. Internet technology has been reported to have the potential to decrease the time and cost in conducting a health care survey and increase response rates . Although conducting health surveys using the Internet has not always resulted in a good response rate [ ], the low dropout in our study was an example of how Internet-based studies could actually work better than traditional forms of data collection using surveys or questionnaires.
The composition of our citizen panel tended to be young urban students. This phenomenon was caused by our reliance on the Internet in this study because such students are more technology savvy and broadband services are more ubiquitous near city centers. Although this phenomenon limits the generalizability of our research results, the fact that the younger well-educated generation will ultimately provide the opinion leaders of the future suggests that our draft guidelines will likely be relevant for some time to come.
Another drawback is that we had a high percentage of health care professionals in the lay panel who were knowledgeable about genetic testing. Therefore, the composition of the lay panel was not representative of society. This difficulty arose due to the high percentage of respondents having medical backgrounds. It is reasonable to assume that those with medical backgrounds would be more interested in this issue than those without. Regardless, the common sense of health care professionals is important for forging societal consensus.
On the other hand, these limits to generalizability could have been caused by our advertising and recruiting strategy. Due to the limitation of funding, we relied solely on free Internet portals. If our recruitment announcements could have been placed in major commercial portals, such as Yahoo and Google, we might have been able to gather a more varied population of volunteers as potential participants.
Despite the growing interest in public participation, the real effectiveness of participation remains difficult to ascertain. The main difficulty comes from how to define effectiveness and how to make it operational . In the instance of consensus conference exercises, most evaluations from the past have only indicated that such consensus conferences were effective because of continuing application and wide audiences, which is not strong proof that they have been effective. It has been argued that rigorous evaluations using social science methodologies should be an important part of public-participation exercises. There are also attempts to establish a research agenda for evaluating public participation exercises [ ]. The fact that our study did not evaluate the effectiveness of the exercise itself remains a major limitation of its generalizability.
No doubt, the sample size of this study is inadequate to draw definitive conclusions about the sensitive issue at hand. This study was merely an attempt to forge some consensus in genetic testing through Internet public participation. Whether a larger number of participants would add strength to this policy making exercise remains to be seen. From the experience gained by conducting this study, we found that a sample size of 100 participants was fairly difficult to manage. For a citizen consensus conference to operate efficiently and effectively, researchers need to communicate with the participants constantly throughout the process. As the group gets larger, even with the help of information and communication technology, it becomes harder and harder to keep tract of the participants and make sure they keep their commitments. Other mechanisms need to be added if a broader participation is desired. There is no way in a democracy that citizen consensus conferences can replace all other policy-making mechanisms, such as referendums.
The ethical, legal, and social impact of genetic testing cannot be overlooked. Test results not only have a tremendous impact on the life of the individuals who receive the test, but also impact his or her family. This research helped to establish ethical guidelines for genetic testing using public participation via the Internet. The recommended ethical guidelines had 4 categories and 25 items; the 4 categories encompassed decision making, management of tissue samples, release of results, and information flow. We hope, through the implementation of these guidelines, mutual trust can be established between the health care professionals and the general public with respect to the application of genetic tests.
The authors thank the experts and citizens who participated in this study and would like to acknowledge the funding support provided by Taipei Medical University.
Conflicts of Interest
- Burke W. Genetic testing. N Engl J Med 2002 Dec 5;347(23):1867-1875. [CrossRef] [Medline]
- Hopkins MM, Nightingale P. Strategic risk management using complementary assets: Organizational capabilities and the commercialization of human genetic testing in the UK. Research Policy 2006;35:355-374. [CrossRef]
- Anderlik MR, Lisko EA. Medicolegal and ethical issues in genetic cancer syndromes. Semin Surg Oncol 2000 Jun;18(4):339-346. [Medline]
- Claes E, Evers-Kiebooms G, Boogaerts A, Decruyenaere M, Denayer L, Legius E. Communication with close and distant relatives in the context of genetic testing for hereditary breast and ovarian cancer in cancer patients. Am J Med Genet A 2003 Jan 1;116A(1):11-19. [CrossRef] [Medline]
- Hughes C, Lerman C, Schwartz M, Peshkin BN, Wenzel L, Narod S, et al. All in the family: evaluation of the process and content of sisters' communication about BRCA1 and BRCA2 genetic test results. Am J Med Genet 2002 Jan 15;107(2):143-150. [Medline]
- Pfeffer NL, Veach PM, LeRoy BS. An investigation of genetic counselors' discussion of genetic discrimination with cancer risk patients. J Genet Couns 2003 Oct;12(5):419-438. [Medline]
- Thompson HS, Valdimarsdottir HB, Duteau-Buck C, Guevarra J, Bovbjerg DH, Richmond-Avellaneda C, et al. Psychosocial predictors of BRCA counseling and testing decisions among urban African-American women. Cancer Epidemiol Biomarkers Prev 2002 Dec;11(12):1579-1585 [FREE Full text] [Medline]
- Venne VL, Botkin JR, Buys SS. Professional opportunities and responsibilities in the provision of genetic information to children relinquished for adoption. Am J Med Genet A 2003 May 15;119A(1):41-46. [CrossRef] [Medline]
- Genetics and Public Policy Center. Genetics and Public Policy Center. 2008. Genetic testing practice guidelines: translating genetic discoveries into clinical care URL: http://www.dnapolicy.org/images/issuebriefpdfs/Professional_Guidelines_Issue_Brief.pdf [accessed 2010-01-11] [WebCite Cache]
- ; Japan Society of Human Genetics, Japan Society of Obstetrics and Gynecology, Japan Society for Pediatric Genetics, Japanese Society for Familial Tumor, Japanese Society for Gene Diagnosis and Therapy, Japanese Society for Genetic Counseling, Japanese Society for Inherited Metabolic Diseases, Japanese Society of Laboratory Medicine, Japanese Society for Mass-screening, Japanese Teratology Society. Joint ELSI Committee of Genetic-Medicine-Related Societies. 2004. Gguidelines for Genetic Testing URL: http://jshg.jp/pdf/10academies_e.pdf [accessed 2010-01-11] [WebCite Cache]
- Verpeet E, Dierckx de Casterle B, Van der Arend A, Gastmans CA. Nurses' views on ethical codes: a focus group study. J Adv Nurs 2005 Jul;51(2):188-195. [CrossRef] [Medline]
- Pattison S. Are nursing codes of practice ethical? Nurs Ethics 2001 Jan;8(1):5-18. [Medline]
- Health Products and Food Branch. Health Canada Office of Consumer and Public Involvement. 2004. Public involvement framework URL: http://www.hc-sc.gc.ca/ahc-asc/alt_formats/hpfb-dgpsa/pdf/pubs/piframework-cadrepp-eng.pdf [accessed 2010-09-21] [WebCite Cache]
- Organization for Economic Cooperation and Development (OECD). Organization for Economic Cooperation and Development. 2001. Engaging citizens in policy-making: Information, consultation and public participation. OECD Public Management Policy Brief URL: http://www.oecd.org/dataoecd/24/34/2384040.pdf [accessed 2010-09-21] [WebCite Cache]
- Rowe G, Frewer LJ. A typology of public engagement mechanisms. Sci Technol Human Values 2005;30:251-290. [CrossRef]
- Sclove R. Town Meetings on Technology. Technol Review 1996;99:24-31.
- Guston D. Evaluating the impact of the first U.S. citizens? panel on Telecommunications and the Future of Democracy. Sci Technol Human Values 1999;24:451-482. [CrossRef]
- Lin CF, Lu MS, Chiang HH, Chung CC, Lin TL, Yin TJ, et al. Using a citizen consensus conference to revise the code of ethics for nurses in Taiwan. J Nurs Scholarsh 2007;39(1):95-101. [Medline]
- Wu RC, Thorpe K, Ross H, Micevski V, Marquez C, Straus SE. Comparing administration of questionnaires via the internet to pen-and-paper in patients with heart failure: randomized controlled trial. J Med Internet Res 2009;11(1):e3 [FREE Full text] [CrossRef] [Medline]
- Leece P, Bhandari M, Sprague S, Swiontkowski MF, Schemitsch EH, Tornetta P, et al. Internet versus mailed questionnaires: a randomized comparison (2). J Med Internet Res 2004 Sep 24;6(3):e30 [FREE Full text] [CrossRef] [Medline]
- Committee on Ethics, American College of Obstetricians and Gynecologists, Committee on Genetics, American College of Obstetricians and Gynecologists. ACOG Committee Opinion No. 410: Ethical issues in genetic testing. Obstet Gynecol 2008 Jun;111(6):1495-1502. [CrossRef] [Medline]
- The Loka Institute. The Loka Institute. 2007. Danish-style, citizen-based deliberative consensus conferences on science and technology policy worldwide URL: http://www.loka.org/TrackingConsensus.html [accessed 2010-01-11] [WebCite Cache]
- Rowe G, Frewer LJ. Evaluating public-participation exercises: a research agenda. Sci Technol Human Values 2004;29:512-556. [CrossRef]
|ACOG: American College of Obstetricians and Gynecologists|
|CVI: content validity index|
|ELSI: ethical, legal, and social implications|
|OECD: Organization for Economic Cooperation and Development|
|PI: principal investigator|
Edited by G Eysenbach; submitted 12.01.10; peer-reviewed by M Figueiredo, J Pramanik, S Singh; comments to author 14.02.10; revised version received 12.04.10; accepted 25.05.10; published 18.10.10
©Chiou-Fen Lin, Meei-Shiow Lu, Chun-Chih Chung, Che-Ming Yang. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 18.10.2010
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.