Published on in Vol 15, No 6 (2013): June

Crowdsourcing Participatory Evaluation of Medical Pictograms Using Amazon Mechanical Turk

Crowdsourcing Participatory Evaluation of Medical Pictograms Using Amazon Mechanical Turk

Crowdsourcing Participatory Evaluation of Medical Pictograms Using Amazon Mechanical Turk

Authors of this article:

Bei Yu1 ;   Matt Willis1 ;   Peiyuan Sun1 ;   Jun Wang1


  1. Lazard A, Schmidt A, Vu H, Byron M, Peters E, Boynton M, Brewer N. Icons for health effects of cigarette smoke: a test of semiotic type. Journal of Behavioral Medicine 2017;40(4):641 View
  2. Ng A, Chan A, Ho V. Comprehension by older people of medication information with or without supplementary pharmaceutical pictograms. Applied Ergonomics 2017;58:167 View
  3. Callaghan C. Open Source Process Insights From ‘Microbial Learning'. International Journal of Sociotechnology and Knowledge Development 2019;11(2):1 View
  4. Créquit P, Mansouri G, Benchoufi M, Vivot A, Ravaud P. Mapping of Crowdsourcing in Health: Systematic Review. Journal of Medical Internet Research 2018;20(5):e187 View
  5. Johnson P, Herrmann E, Johnson M. Opportunity costs of reward delays and the discounting of hypothetical money and cigarettes. Journal of the Experimental Analysis of Behavior 2015;103(1):87 View
  6. Wu B, Wu T, Ng Q. The Literature of Crowdsourcing in China: Evidence from International Journal of Knowledge Engineering 2015;1(3):172 View
  7. N. Fuertes J, R. Grindell S, Kestenbaum M, Gorman B. Sex, Parent Attachment, Emotional Adjustment, and Risk-Taking Behaviors. International Journal of High Risk Behaviors and Addiction 2016;Inpress(Inpress) View
  8. Lloyd J, Yen T, Pietrobon R, Wiener J, Ross S, Kokorowski P, Nelson C, Routh J. Estimating utility values for vesicoureteral reflux in the general public using an online tool. Journal of Pediatric Urology 2014;10(6):1026 View
  9. Kanji L, Xu S, Cavaco A. Assessing the Understanding of Pharmaceutical Pictograms among Cultural Minorities: The Example of Hindu Individuals Communicating in European Portuguese. Pharmacy 2018;6(1):22 View
  10. Lazard A, Atkinson L. Putting Environmental Infographics Center Stage. Science Communication 2015;37(1):6 View
  11. Short R, Middleton D, Befera N, Gondalia R, Tailor T. Patient-Centered Radiology Reporting: Using Online Crowdsourcing to Assess the Effectiveness of a Web-Based Interactive Radiology Report. Journal of the American College of Radiology 2017;14(11):1489 View
  12. Albarqouni S, Baur C, Achilles F, Belagiannis V, Demirci S, Navab N. AggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images. IEEE Transactions on Medical Imaging 2016;35(5):1313 View
  13. Mays D, Tercyak K. Framing Indoor Tanning Warning Messages to Reduce Skin Cancer Risks Among Young Women: Implications for Research and Policy. American Journal of Public Health 2015;105(8):e70 View
  14. Callaghan C. The probabilistic innovation theoretical framework. South African Journal of Economic and Management Sciences 2017;20(1) View
  15. Mortensen K, Hughes T. Comparing Amazon’s Mechanical Turk Platform to Conventional Data Collection Methods in the Health and Medical Research Literature. Journal of General Internal Medicine 2018;33(4):533 View
  16. Choi K, Cho J. Evaluation of Pictograms Related to Radiology on the Website of Hospitals. Journal of Radiological Science and Technology 2019;42(4):301 View
  17. Jiang R, Inouye B, Wang H, Tejwani R, Routh J. Crowdsourcing utility estimation for spina bifida in the general population. Journal of Pediatric Rehabilitation Medicine 2017;10(3-4):257 View
  18. van Beusekom M, Kerkhoven A, Bos M, Guchelaar H, van den Broek J. The extent and effects of patient involvement in pictogram design for written drug information: a short systematic review. Drug Discovery Today 2018;23(6):1312 View
  19. Crangle C, Kart J. A questions-based investigation of consumer mental-health information. PeerJ 2015;3:e867 View
  20. Kim S, Marsch L, Brunette M, Dallery J. Harnessing Facebook for Smoking Reduction and Cessation Interventions: Facebook User Engagement and Social Support Predict Smoking Reduction. Journal of Medical Internet Research 2017;19(5):e168 View
  21. Callaghan C. Scientific real-time research problem-solving and pharmaceutical innovation. African Journal of Science, Technology, Innovation and Development 2017;9(4):425 View
  22. Weissler J, Carney M, Enriquez F, Messa C, Broach R, Shapira M, Barg F, Fischer J. Using Crowdsourcing as a Platform to Evaluate Lay Perception of Prophylactic Mesh Placement. Journal of Surgical Research 2019;237:78 View
  23. Hosseini M, Shahri A, Phalp K, Taylor J, Ali R. Crowdsourcing: A taxonomy and systematic mapping study. Computer Science Review 2015;17:43 View
  24. Truitt A, Monsell S, Avins A, Nerenz D, Lawrence S, Bauer Z, Comstock B, Edwards T, Patrick D, Jarvik J, Lavallee D. Prioritizing research topics: a comparison of crowdsourcing and patient registry. Quality of Life Research 2018;27(1):41 View
  25. Cui L, Carter R, Zhang G. Evaluation of a Novel Conjunctive Exploratory Navigation Interface for Consumer Health Information: A Crowdsourced Comparative Study. Journal of Medical Internet Research 2014;16(2):e45 View
  26. Lin S, Thompson H, Hart L, Fu M, Demiris G. Evaluation of pharmaceutical pictograms by older “turkers”: A cross-sectional crowdsourced study. Research in Social and Administrative Pharmacy 2021;17(6):1079 View
  27. Dasgupta N, Freifeld C, Brownstein J, Menone C, Surratt H, Poppish L, Green J, Lavonas E, Dart R. Crowdsourcing Black Market Prices For Prescription Opioids. Journal of Medical Internet Research 2013;15(8):e178 View
  28. Kuang J, Argo L, Stoddard G, Bray B, Zeng-Treitler Q. Assessing Pictograph Recognition: A Comparison of Crowdsourcing and Traditional Survey Approaches. Journal of Medical Internet Research 2015;17(12):e281 View
  29. Lu W, Guttentag A, Elbel B, Kiszko K, Abrams C, Kirchner T. Crowdsourcing for Food Purchase Receipt Annotation via Amazon Mechanical Turk: A Feasibility Study. Journal of Medical Internet Research 2019;21(4):e12047 View
  30. Wazny K. Applications of crowdsourcing in health: an overview. Journal of Global Health 2018;8(1) View
  31. Zide M, Caswell K, Peterson E, Aberle D, Bui A, Arnold C. Consumers’ Patient Portal Preferences and Health Literacy: A Survey Using Crowdsourcing. JMIR Research Protocols 2016;5(2):e104 View
  32. Bateman D, Brady E, Wilkerson D, Yi E, Karanam Y, Callahan C. Comparing Crowdsourcing and Friendsourcing: A Social Media-Based Feasibility Study to Support Alzheimer Disease Caregivers. JMIR Research Protocols 2017;6(4):e56 View
  33. Mathew T, Kini J, Rajan J. Computational methods for automated mitosis detection in histopathology images: A review. Biocybernetics and Biomedical Engineering 2021;41(1):64 View
  34. Mittl G, Cook T, Hill P, Cho J, Shea J, Kahn C, Zafar H. Patient Understanding of Abnormal Imaging Findings Under Pennsylvania Act 112: A Call to Revise Mandated Notification Message Language. Journal of the American College of Radiology 2021;18(7):951 View
  35. He X, Zhang R, Alpert J, Zhou S, Adam T, Raisa A, Peng Y, Zhang H, Guo Y, Bian J. When text simplification is not enough: could a graph-based visualization facilitate consumers’ comprehension of dietary supplement information?. JAMIA Open 2021;4(1) View
  36. Velez R, Calderon D, Carey L, Aime C, Hultquist C, Yetman G, Kruczkiewicz A, Gorokhovich Y, Chen R. Advancing Data for Street-Level Flood Vulnerability: Evaluation of Variables Extracted from Google Street View in Quito, Ecuador. IEEE Open Journal of the Computer Society 2022;3:51 View
  37. Elor A, Whittaker S, Kurniawan S, Michael S. BioLumin: An Immersive Mixed Reality Experience for Interactive Microscopic Visualization and Biomedical Research Annotation. ACM Transactions on Computing for Healthcare 2022;3(4):1 View
  38. Amirkhanyan A, Roberts F, Meier K, Song M. Examining attitudes toward public participation across sectors: An experimental study of food assistance. Public Administration 2024 View

Books/Policy Documents

  1. Hosseini M, Phalp K, Taylor J, Ali R. Social Entrepreneurship. View
  2. Hosseini M, Phalp K, Taylor J, Ali R. Crowdsourcing. View
  3. Bouillon P, Gerlach J, Norré M, Spechbach H. Translation Technology in Accessible Health Communication. View