Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?


Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 03.12.20 in Vol 22, No 12 (2020): December

Preprints (earlier versions) of this paper are available at, first published Jun 18, 2020.

This paper is in the following e-collection/theme issue:


    COVID-19 Contact-Tracing Apps: Analysis of the Readability of Privacy Policies

    Family Medicine and Primary Care, Lee Kong Chian School of Medicine, Nanyang Technological University Singapore, Singapore, Singapore

    Corresponding Author:

    Melvyn Zhang, MBBS, MRCPsych

    Family Medicine and Primary Care

    Lee Kong Chian School of Medicine

    Nanyang Technological University Singapore

    11 Mandalay Road Level 18

    Clinical Sciences Building

    Singapore, 308322


    Phone: 65 63892504



    Apps that enable contact-tracing are instrumental in mitigating the transmission of COVID-19, but there have been concerns among users about the data collected by these apps and their management. Contact tracing is of paramount importance when dealing with a pandemic, as it allows for rapid identification of cases based on the information collected from infected individuals about other individuals they may have had recent contact with. Advances in digital technology have enabled devices such as mobile phones to be used in the contract-tracing process. However, there is a potential risk of users’ personal information and sensitive data being stolen should hackers be in the near vicinity of these devices. Thus, there is a need to develop privacy-preserving apps. Meanwhile, privacy policies that outline the risk associated with the use of contact-tracing apps are needed, in formats that are easily readable and comprehensible by the public. To our knowledge, no previous study has examined the readability of privacy policies of contact-tracings apps. Therefore, we performed a readability analysis to evaluate the comprehensibility of privacy policies of 7 contact-tracing apps currently in use. The contents of the privacy policies of these apps were assessed for readability using Readability Test Tool, a free web-based reliability calculator, which computes scores based on a number of statistics (ie, word count and the number of complex words) and indices (ie, Flesch Reading Ease, Flesch-Kincaid Reading Grade Level, Gunning Fog Index, and Simplified Measure of Gobbledygook index). Our analysis revealed that explanations used in the privacy policies of these apps require a reading grade between 7 and 14, which is considerably higher than the reading ability of the average individual. We believe that improving the readability of privacy policies of apps could be potentially reassuring for users and may help facilitate the increased use of such apps.

    J Med Internet Res 2020;22(12):e21572



    Contact tracing is of paramount importance when dealing with a pandemic such as COVID-19. It allows for the rapid identification of cases based on the information collected from infected individuals about their recent contact with other individuals [1]. Additionally, contact-tracing systems allow for the collection of further information about these contacts, in order to help minimize the spread of the disease [2]. Different contact tracing apps rely on different technologies, including GPS, Bluetooth, and millimeter-wave communication technologies. Conventionally, public health care workers can assist an infected patient to map out individuals with whom they might have been in close contact recently, and these individuals are then informed of their susceptibility to the infection. Thus, contact tracing enables the identification of potential cases and allows for the follow-up and rapid quarantining of susceptible individuals [1]. With advances in digital technology, devices such as a mobile phone can now be employed in the contact-tracing process. For instance, a recent article in Nature discusses 3 apps developed to rapidly identify contacts of patients with COVID-19, including an Australia-based app called COVIDSafe and similar apps being used in Germany and Egypt [3].

    Apps that enable contact tracing are instrumental in response to a public health emergency, but there have been concerns about the data they collect and how they are managed. Although there are potential benefits of using these apps, there are also ongoing concerns. For instance, in a recent commentary, Sharma et al [4] outlined the existing apps for COVID-19 contact tracing and concerns about data privacy. Another article in Nature cautioned against the accuracy of such contact-tracing apps and highlighted how these apps might render individuals susceptible to security breaches, given that most of these apps tap on Bluetooth functionality, potentially compromising the exchange of information [5]. This is an inherent risk that personal information and other sensitive personal data might be stolen if hackers happen to be in the vicinity of these devices [5]. A mixed-methods study in Norway analyzed the personal dataflows and the contents of privacy policies of 21 popular, free-to-use Android mobile apps [6]. Their results showed that 19 of the 21 apps studied transmitted personal data to about 600 different primary and third-party domains that were associated with tech companies in the United States. They also found that some apps tracked and shared data by default even when the app was not in use. The terms of use of some of these apps did not inform the users about the data sharing.

    This study highlights critical ethical issues of data protection, security, and privacy of data collated by smartphone apps [1] and the consequent need to develop privacy-preserving smartphone apps [7]. A scoping review of the privacy assessments of mobile health apps reported that the evaluation criteria used in studies have been heterogeneous and lacked objectivity [8]. This further emphasizes the need for a common evaluation tool to ensure that personal health data privacy is safeguarded. It has also been suggested that a “healthcare fiduciary” be developed to enhance international regulatory frameworks to increase data protection security [9].

    While we await the development of such privacy-preserving apps, privacy policies outlining the risks associated with the use of contact-tracing apps are needed, in a format that can be easily read and comprehended by the public. Readability of policy terms can be evaluated using validated tools that assess the complexity of the vocabulary and syntax, as well as the presentation of the content [10]. In other areas of health care, researchers have started to critique the readability of privacy policies. For instance, Robillard et al [10] focused on the availability and readability of privacy-related content of mental health apps and reported that most apps they studied did not include terms of agreement or a privacy policy. On the other hand, among the apps that had such policies in place, a reading ability more advanced than secondary education was required to comprehend the information. In relation to COVID-19, Basch et al [11] examined the information available on the internet and found that the readability levels required to comprehend the information exceeded that of the average American. The fact that higher-than-average readability levels are required to comprehend web-based information implies that the available information cannot be disambiguated, which might result in increased panic among the app users [11].

    Given this situation, we performed a readability analysis of the privacy policies of 7 contact-tracing apps, namely COVIDSafe (used in Australia) [12], BeAware (used in Bahrain) [13], CoronApp (used in Colombia) [14], GH COVID-19 Tracker (used in Ghana) [15], Rakning C-19 (used in Iceland) [16], NZ COVID Tracer (used in New Zealand) [17], and TraceTogether (used in Singapore) [18]. As previously highlighted by Basch et al [11], the provision of timely information, in a format that could be comprehended easily, would help individuals understand important information relevant to the pandemic and, in turn, allay any anxieties. A readability analysis of privacy policies is timely and pertinent, given the considerable number of contact-tracing apps now available and government agencies’ enforcement that individuals download and use these apps. As a result, individuals are now more likely to examine the privacy policies of the apps they use, to understand what data is being shared and how their personal information is being protected. Any difficulty in comprehending the information contained within these privacy policies could result in a reluctance to download and use such apps.

    Readability statistics of the privacy policies of the identified apps were computed using Readability Test Tool, a web-based reliability calculator [19]. This free resource computes the word count, Flesch Reading Ease, Flesch-Kincaid Reading Grade Level, Gunning Fog Index, Simplified Measure of Gobbledygook (SMOG) index, and the number of complex words [20]. For this evaluation, we used well-validated methods, based on previous studies that have examined readability [21,22]. The Flesch Reading Ease test evaluates the length of sentences and the number of polysyllabic words to determine the overall readability score; the score ranges from 0 to 100, with a higher score suggesting that the text is easy to read. The Flesh-Kincaid Reading Grade Level test evaluates the mean sentence and word length to compute reading complexity of the text; the score ranges from 1 to 12, corresponding to the US educational school grades, with scores higher than 12 indicative of college-level education and domain-specific experts. The Gunning Fog Index estimates the number of years of formal education required for an individual to understand the text on the first reading; the score ranges from 0 to 19+ and is representative of the readability level of the document. A Gunning Fog score of 0-6 is indicative of low literacy, a score of 7 or 8 is indicative of junior high school–level literacy, a score of 9-12 is indicative of high school–level literacy, a score of 13-16 is indicative of college-level literacy, a score of 17 or 18 is indicative of graduate-level literacy, and a score ≥19 suggests higher professional–level qualifications [23]. The SMOG index estimates the years of education needed to understand a piece of writing, by evaluating 10 sentences from the beginning, middle, and end of the document. The number of syllables in each section is then totaled and converted to a grade-level score [20]. Table 1 shows the readability scores for each of the 7 apps studied.

    Table 1. Readability scores for the privacy policies of different COVID-19 contact-tracing apps analyzed in this study.
    View this table

    Users of contact-tracing apps must be aware that the apps gather a lot of their personal data, some from self-reporting and some via sensors in their smartphone devices. Moreover, our findings suggest that the existing explanations in the privacy policies of these apps require a reading level between 7 and 14, which far exceeds many people’s reading ability. Apps like CoronApp [14] and NZ COVID Tracer [17] required the highest-grade level of comprehension (Figure 1), followed by BeAware [13], TraceTogether [18], GH COVID-19 Tracker [15], COVIDSafe [12], and Rakning C-19 [16], listed in order of decreasing readability ease. For example, in the United States, the average reading level is between grades 7 and 8 [24]. For the information to be accessible and achieve maximum impact among the general population, it should be written at a level no higher than grade 6 [22]. Hence, currently, the privacy policies of all the 7 apps analyzed in this study are considered “very difficult” to read and comprehend for the majority of individuals. In their analysis of the readability of online websites on COVID-19, Basch at al [11] highlighted how heightened levels of anxiety about the pandemic might further impair the understanding and interpretation of information, thus exacerbating fear.

    Figure 1. Overview of readability scores for each COVID-19 contact-tracing app evaluated.
    View this figure

    With more countries now exiting lockdowns, the use of contact-tracing apps will become more commonplace. While we await improvements to existing apps through the use of more secured technologies, the public must have access to readable terms of agreement or privacy policies to be aware of how their data are being collected, stored, and used. Improving the readability of privacy policies could be reassuring and could facilitate the adoption and eventual impact of these apps. Our study has highlighted COVID-19 contact-tracing apps with privacy policies that are readily understandable by the general public. Government agencies need to recognize this and to adapt their privacy policies accordingly, to ensure that every user can readily understand how their data are being stored and shared by the app. At a macro level, health care ministries and organizations could consider enhancing current regulatory frameworks to increase data protection security [9]. This may cause a trickle-down effect to app developers and companies and to the users, for safeguarding personal data.

    Several research implications arise from our study findings. We concur with the suggestions by Bahadori et al [23] that researchers could undertake a number of measures to improve app readability. Users are also occasionally involved in the conceptualization of the app and in user testing. With the increase in participatory research, potential users could perhaps be involved in the cocreation and drafting of the privacy policies for such apps. Academics and developers are encouraged to consider the average reading level of the population when they are drafting these policies. As highlighted by Bahadori et al [23], an effective way to do so is to reduce the length of the sentence and target towards a reading level of grade 6. For continued monitoring of user experiences, they also recommend determining whether readability needs to be improved on an ongoing basis. As these areas develop, an objective evaluation tool should also be developed to assess whether sufficient measures have been taken to safeguard the data of mobile app users. By increasing the level of trust that users have in how an app uses their data, more users will be confident of using these apps. This will bode well as health care research drives into the age of big data to improve health care services for everyone.


    MWZ is supported by a grant under the Singapore Ministry of Health’s National Medical Research Council (grant number NMRC/Fellowship/0048/2017) for PhD training. The funding source was not involved in any part of this project.

    Authors' Contributions

    MZ, AC, and HS jointly conceptualized the study. MZ and AC were involved in data extraction and verification of the extracted data. MZ worked on the first draft of the manuscript, and HS provided guidance and suggested revisions. MZ and AC amended the second draft of the manuscript. HS provided critical updates to the final manuscript. All authors read and approved the final manuscript.

    Conflicts of Interest

    None declared.


    1. Parker MJ, Fraser C, Abeler-Dörner L, Bonsall D. Ethics of instantaneous contact tracing using mobile phone apps in the control of the COVID-19 pandemic. J Med Ethics 2020 Jul;46(7):427-431 [FREE Full text] [CrossRef] [Medline]
    2. LI J, Guo X. Global deployment mappings and challenges of contact-tracing apps for COVID-19. SSRN Preprint posted online May 24, 2020. [CrossRef]
    3. Show evidence that apps for COVID-19 contact-tracing are secure and effective. Nature 2020 Apr;580(7805):563. [CrossRef] [Medline]
    4. Sharma T, Bashir M. Use of apps in the COVID-19 response and the loss of privacy protection. Nat Med 2020 Aug;26(8):1165-1167. [CrossRef] [Medline]
    5. Zastrow M. Coronavirus contact-tracing apps: can they slow the spread of COVID-19? Nature Epub ahead of print posted online on May 19, 2020. [CrossRef] [Medline]
    6. Brandtzaeg PB, Pultier A, Moen GM. Losing control to data-hungry apps: a mixed-methods approach to mobile app privacy. Social Science Computer Review 2018 May 31;37(4):466-488. [CrossRef]
    7. Yasaka T, Lehrich B, Sahyouni R. Peer-to-peer contact tracing: development of a privacy-preserving smartphone app. JMIR Mhealth Uhealth 2020 Apr 07;8(4):e18936 [FREE Full text] [CrossRef] [Medline]
    8. Benjumea J, Ropero J, Rivera-Romero O, Dorronzoro-Zubiete E, Carrasco A. Privacy assessment in mobile health apps: scoping review. JMIR Mhealth Uhealth 2020 Jul 02;8(7):e18868 [FREE Full text] [CrossRef] [Medline]
    9. Galvin HK, DeMuro PR. Developments in privacy and data ownership in mobile health technologies, 2016-2019. Yearb Med Inform 2020 Aug;29(1):32-43 [FREE Full text] [CrossRef] [Medline]
    10. Robillard JM, Feng TL, Sporn AB, Lai J, Lo C, Ta M, et al. Availability, readability, and content of privacy policies and terms of agreements of mental health apps. Internet Interv 2019 Sep;17:100243 [FREE Full text] [CrossRef] [Medline]
    11. Basch CH, Mohlman J, Hillyer GC, Garcia P. Public health communication in time of crisis: readability of on-line cOVID-19 information. Disaster Med Public Health Prep 2020 May 11:1-3 [FREE Full text] [CrossRef] [Medline]
    12. COVIDSafe App. Australian Government.   URL: [accessed 2020-06-04]
    13. BeAware Bahrain. Kingdom of Bahrain - eGovernment Apps Store.   URL: https:/​/apps.​​CMSWebApplication/​action/​ShowAppDetailsAction?selectedAppID=321&appLanguage=en [accessed 2020-06-04]
    14. CoronApp. Government of Colombia.   URL: [accessed 2020-06-04]
    15. GH COVID-19 Tracker App. Ministry of Communications, Ghana.   URL: [accessed 2020-06-04]
    16. Rakning C-19 App. Google Play Store.   URL: [accessed 2020-06-04]
    17. NZ COVID Tracer. Ministry of Health – Manatū Hauora.   URL: [accessed 2020-06-04]
    18. TraceTogether, safer together. A Singapore Government Agency Website.   URL: [accessed 2020-06-04]
    19. Readability test tool - Quick and easy way to test the readability of your work. WebFX.   URL: https:/​/www.​​tools/​read-able/​check.​php?tab=Test+By+Url&uri=https%3A%2F%2Ffacetagr.​com%2Fprivacy-policy%2F [accessed 2020-05-22]
    20. Grabeel K, Russomanno J, Oelschlegel S, Tester E, Heidel R. Computerized versus hand-scored health literacy tools: a comparison of Simple Measure of Gobbledygook (SMOG) and Flesch-Kincaid in printed patient education materials. J Med Libr Assoc 2018 Jan;106(1):38-45 [FREE Full text] [CrossRef] [Medline]
    21. Fowler L, Gillard C, Morain S. Readability and accessibility of terms of service and privacy policies for menstruation-tracking smartphone applications. Health Promot Pract 2020 Sep;21(5):679-683. [CrossRef] [Medline]
    22. Ayyaswami V, Padmanabhan D, Crihalmeanu T, Thelmo F, Prabhu A, Magnani J. Mobile health applications for atrial fibrillation: a readability and quality assessment. Int J Cardiol 2019 Oct 15;293:288-293. [CrossRef] [Medline]
    23. Bahadori S, Wainwright T, Ahmed O. Readability of information on smartphone apps for total hip replacement and total knee replacement surgery patients. J Patient Exp 2020 Jun;7(3):395-398 [FREE Full text] [CrossRef] [Medline]
    24. McKenzie J, Neiger B, Thackeray R. Planning, Implementing, and Evaluating Health Promotion Programs: A Primer. 7th Edition. New York, NY: Pearson; 2017.

    Edited by G Eysenbach, R Kukafka; submitted 18.06.20; peer-reviewed by C Shen, J Li, M Adly, A Dawood, A McLean, A Azzam; comments to author 09.09.20; revised version received 25.09.20; accepted 08.10.20; published 03.12.20

    ©Melvyn Zhang, Aloysius Chow, Helen Smith. Originally published in the Journal of Medical Internet Research (, 03.12.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.