This paper is in the following e-collection/theme issue:

JMIR Theme Issue: COVID-19 Special Issue (2458) Digital Contact Tracing, Digital Proximity Tracing, Precision Public Health (148) Outbreak and Pandemic Preparedness and Management (1740) Participatory Epidemiology and Surveillance (119) Surveillance Systems (465) Theme Issue: Novel Coronavirus (COVID-19) Outbreak Rapid Reports (1520) Security and Privacy of mHealth and uHealth (126)

Published on in Vol 23, No 2 (2021): February

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/23467, first published .
Technology, Privacy, and User Opinions of COVID-19 Mobile Apps for Contact Tracing: Systematic Search and Content Analysis

Technology, Privacy, and User Opinions of COVID-19 Mobile Apps for Contact Tracing: Systematic Search and Content Analysis

Technology, Privacy, and User Opinions of COVID-19 Mobile Apps for Contact Tracing: Systematic Search and Content Analysis

Authors of this article:

Mahmoud Elkhodr1 Author Orcid Image ;   Omar Mubin2 Author Orcid Image ;   Zainab Iftikhar3 Author Orcid Image ;   Maleeha Masood3 Author Orcid Image ;   Belal Alsinglawi2 Author Orcid Image ;   Suleman Shahid3 Author Orcid Image ;   Fady Alnajjar4 Author Orcid Image
Article Authors Cited by (59) Tweetations (5) Metrics

    Original Paper

    1School of Engineering and Technology, Central Queensland University, Sydney, Australia

    2School of Computer, Data and Mathematical Sciences, Western Sydney University, Rydalmere, Australia

    3Department of Computer Science, Syed Babar Ali School of Science and Engineering, Lahore University of Management Sciences, Lahore, Pakistan

    4Department of Computer Science and Software Engineering, College of Information Technology, United Arab Emirates University, Alain, United Arab Emirates

    Corresponding Author:

    Fady Alnajjar, PhD

    Department of Computer Science and Software Engineering

    College of Information Technology

    United Arab Emirates University

    Alain 15551

    Alain,

    United Arab Emirates

    Phone: 971 037135538

    Email: fady.alnajjar@uaeu.ac.ae


    Background: Many countries across the globe have released their own COVID-19 contact tracing apps. This has resulted in the proliferation of several apps that used a variety of technologies. With the absence of a standardized approach used by the authorities, policy makers, and developers, many of these apps were unique. Therefore, they varied by function and the underlying technology used for contact tracing and infection reporting.

    Objective: The goal of this study was to analyze most of the COVID-19 contact tracing apps in use today. Beyond investigating the privacy features, design, and implications of these apps, this research examined the underlying technologies used in contact tracing apps. It also attempted to provide some insights into their level of penetration and to gauge their public reception. This research also investigated the data collection, reporting, retention, and destruction procedures used by each of the apps under review.

    Methods: This research study evaluated 13 apps corresponding to 10 countries based on the underlying technology used. The inclusion criteria ensured that most COVID-19-declared epicenters (ie, countries) were included in the sample, such as Italy. The evaluated apps also included countries that did relatively well in controlling the outbreak of COVID-19, such as Singapore. Informational and unofficial contact tracing apps were excluded from this study. A total of 30,000 reviews corresponding to the 13 apps were scraped from app store webpages and analyzed.

    Results: This study identified seven distinct technologies used by COVID-19 tracing apps and 13 distinct apps. The United States was reported to have released the most contact tracing apps, followed by Italy. Bluetooth was the most frequently used underlying technology, employed by seven apps, whereas three apps used GPS. The Norwegian, Singaporean, Georgian, and New Zealand apps were among those that collected the most personal information from users, whereas some apps, such as the Swiss app and the Italian (Immuni) app, did not collect any user information. The observed minimum amount of time implemented for most of the apps with regard to data destruction was 14 days, while the Georgian app retained records for 3 years. No significant battery drainage issue was reported for most of the apps. Interestingly, only about 2% of the reviewers expressed concerns about their privacy across all apps. The number and frequency of technical issues reported on the Apple App Store were significantly more than those reported on Google Play; the highest was with the New Zealand app, with 27% of the reviewers reporting technical difficulties (ie, 10% out of 27% scraped reviews reported that the app did not work). The Norwegian, Swiss, and US (PathCheck) apps had the least reported technical issues, sitting at just below 10%. In terms of usability, many apps, such as those from Singapore, Australia, and Switzerland, did not provide the users with an option to sign out from their apps.

    Conclusions: This article highlighted the fact that COVID-19 contact tracing apps are still facing many obstacles toward their widespread and public acceptance. The main challenges are related to the technical, usability, and privacy issues or to the requirements reported by some users.

    J Med Internet Res 2021;23(2):e23467

    doi:10.2196/23467

    Keywords

    contact tracing; COVID-19; digital contact tracing apps 


    Overview

    The COVID-19 pandemic, the virus of which causes a highly contagious respiratory infection, has spread rapidly across the world and surpassed 20 million cases by early August 2020 [1]. The economic impact of the pandemic is felt globally with many countries slipping into recession. The COVID-19 pandemic is also turning into a job crisis, which is threatening to dismantle several industries, from aviation and manufacturing to services, tourism, and agriculture [2].

    The global public health and government responses to the pandemic have been fragmented due to the urgency of actions required as a result of the stochastic spread of the virus. Some countries are implementing policies to eradicate the virus, such as Vietnam and New Zealand [3]; some countries are trying to suppress and contain the spread of the virus, such as Australia [4]; and some countries are relying on building herd immunity, such as Sweden [5]. Nonetheless, the virus continues to spread arbitrarily between regions and countries, and the epicenter of the pandemic has been moving between continents. It started with China and moved to Italy, Spain, the United States, and Brazil, with India as the next in line. Several other countries are now experiencing a second wave after initially suppressing it, with clusters of new cases popping up in many countries [6].

    The speed of the authorities’ response has also proven to be a major key in containing the spread of the virus. For instance, many experts weighed in on the relatively slow response of Italy to contain the virus [7] and the fast response of South Korea in suppressing it [8]. Despite the variations in the worldwide governmental crisis responses to the pandemic and the lack of clear and uniform advice on matters as simple as the role of a mask in containing the spread of the virus [9], the measures and policies used worldwide to contain the virus remained mostly precautionary in the absence of a vaccine or a treatment. Consequently, the direct safety advice as a result of the COVID-19 pandemic continues to be about maintaining good hand hygiene, practicing social distancing between people, testing as soon as virus symptoms appear, quarantining, and, importantly, contact tracing.

    Contact tracing is the process of identifying, assessing, and managing people who have been exposed to a disease to prevent onward transmission [10]. Until a COVID-19 vaccine is commercially available to the public, contact tracing tools are vital in breaking the chains of transmission of the virus. This means identifying infected people and their close contacts, testing them, and isolating them for 14 days from day zero of the exposure. For countries that managed to control the exponential growth of the virus, known as flattening the curve, extensive contact tracing was essential in minimizing large-scale community transmissions. With countries recently coming out of lockdown and opening their economies and borders again, such as France and the United States, contact tracing is the key to rapidly identifying new cases; hence, maintaining low levels of community transmissions to remain successful in containing the outbreak of the virus. Thus, in addition to comprehensive testing capacity, contact tracing is increasingly becoming important in managing this pandemic until a vaccine or a reliable viral treatment is successful and made publicly available.

    For contact tracing to be beneficial in preventing onward transmission, and thereby reducing the impact of a second wave of a contagious disease such as COVID-19, it should be implemented systematically. This means having a system to securely collect, compile, and analyze data about individuals in real time, while not impinging on their privacy. As with the lack of a uniform and standardized global response to the pandemic, contact tracing technologies and approaches adopted by several countries were also diverse. For instance, on the same day in which Canada announced that they were working on a new contact tracing app [11], the United Kingdom was abandoning their contact tracing app, stating that the technology does not work [12].

    Background

    Contact tracing using a mobile app relies on the concept of proximity tracking. The concept behind contact tracing is to identify and keep a record of people who may have been in close proximity (eg, typically less than 1.5 meters) to other people. Therefore, once an individual is identified to be infected with COVID-19, the app will be used to retrieve and trace the other close contacts. There have been various implementations for contact tracing apps, and a range of technologies, security, and privacy approaches have been adopted across the globe. Notably, the effectiveness of these contact tracing technologies remains to be seen. More evidence is required to demonstrate whether these tools were successful in contact tracing and to determine their usefulness.

    Before the COVID-19 pandemic, a range of digital and mobile health tools had been utilized for the purposes of infectious disease control and public health interventions. Aba et al [13] illustrated the variety of functionalities provided by mobile apps to mitigate the spread of the Ebola virus in Africa, ranging from contract tracing to surveillance to case management. Mobile apps have also had a similar range of success in the use of public health interventions mitigating the spread of tuberculosis in Botswana [14].

    The current contact tracing apps for COVID-19, which have been widely used by several countries, mostly use Bluetooth as the underlying technology for proximity sensing. In an effort to contribute toward having a unified solution for contact tracing and to counter the limitations of using Bluetooth on the iOS platform [15], Apple and Google have also recently released a new framework to support contact tracing [16]. However, apps that implement this framework have not matured enough yet. Nonetheless, surveying the current apps in use; analyzing their privacy features, penetration, and intake; and measuring their reception by the public, including the ensuing issues faced, have not been fully explored. This is demonstrated in a survey of the prior literature, which is presented in Table S1 from Multimedia Appendix 1. We briefly summarize a review of the main studies from the literature below.

    The user acceptability of contact tracing apps in five countries hit by the pandemic using a survey were investigated in Altmann et al [17]; however, the study did not review specific contact tracing apps. Similarly, several studies [18-22] did not review current COVID-19 apps through direct access of app stores. For instance, the work reported in Anglemyer et al [18] did a meta-analysis on medical databases to review contact tracing apps. Others used various methodologies to conduct their reviews.

    Other works, such as the one reported in Collado-Borrell at al [23], attempted to identify smartphone apps that aimed to address the COVID-19 pandemic and analyzed their characteristics. However, the study did not investigate any specific app. It only classified the apps under specific categories, such as health, fitness, or medicine. The main security and data protection aspects relating to digital contact tracing frameworks and apps were also investigated in Martin at al [24]; the paper analyzed some of the privacy aspects, such as personal information access, data retention, and location tracking. The paper also highlighted some of the app’s public penetration; however, the study was only limited to apps from Google Play.

    An overview of mobile apps being currently used for COVID-19 and their assessment using the Mobile Application Rating Scale was reported in Davalbhakta et al [25]. This study was limited only to India, the United States, and the United Kingdom. Other works, such as the one reported in Vaudena [26], studied Bluetooth-based contact tracing solutions, including Decentralized Privacy-Preserving Proximity Tracing (DP-3T) and Temporary Contact Numbers (TCN) protocols, from a centralized versus decentralized point of view. As such, the vulnerabilities and the advantages of both solutions were systematically reviewed. The work focused more on the underlying architecture used and the level of privacy protection each one presented; however, it did not review specific implementations of COVID-19 apps. Only a few apps were used to represent the centralized and decentralized approaches.

    The work in Magklaras and Bojorquez [22] surveyed the data regulations and technology protocols relating to COVID-19 contact tracing apps. It also provided mapping for the global deployment of the COVID-19 contact tracing apps. The paper also discussed the challenges, including some privacy aspects, relating to Bluetooth-based contact tracking technologies. The work reported in Li and Guo [27] provided an in-depth review of COVID-19 tracing app technologies and processes, including app installation and registrations, encounter data processing and communication, and notifications. The paper also analyzed the security aspects of contact tracing app architectures (ie, centralized, decentralized, and hybrid) by assessing their risk against common security attacks, such as denial of service and carryover attacks. This paper as well as an additional review paper [28] discussed some users’ common concerns, but it did not qualitatively analyze any users’ reviews.

    Study Aims

    To this end, the work presented in this paper reviews and evaluates most categories of COVID-19 contact tracing mobile apps in use today. To our knowledge, this is the first research study that primarily investigated the public’s and users’ perceptions of COVID-19 contact tracing apps. This study also aimed at studying the privacy feature implementations and the level of penetration these apps achieved. In extension to the first aim, we aimed to determine the outreach of the collated apps in terms of number of downloads, as reported not only by the app stores but also by the authorities of each of the apps’ corresponding countries. This is in addition to providing a quantitative overview of the common complaints suggested by app users in connection to privacy, battery drainage, technical difficulties, bugs, crashes, and more. Additionally, in relation to the second aim, the underlying apps’ architecture and associated aspects, such as how the communication or handshake between two devices in proximity took place and then how close contacts were reported, were also analyzed. We also investigated the timeline of when these apps were introduced. Lastly, extending from the third aim, we attempted to understand the nature, type, and extent of data capture of the apps, such as granularity of data that was captured (ie, location, identification, and accomplices), duration of data retention, option to discard and delete records, and whether opt-out options were provided to the user without uninstalling the app.


    Selection of Apps, User Intake, and Penetration

    This study classified contact tracing apps based on the type of technology used for contact tracing of infected masses. This study identified six distinct technologies and an additional category commonly used or incorporated into COVID-19 tracing apps. These included Bluetooth, the DP-3T protocol, GPS, Pan-European Privacy-Preserving Proximity Tracing (PEPP-PT), the TCN protocol, Google and Apple, and other technologies, mainly the use of Quick Response (QR) codes paired with a digital diary. These technologies are outlined in Table S2 from Multimedia Appendix 1.

    The classification criteria considered the underlying technology used by the apps rather than classifying the apps based on geographical or other architectural features. This is because most of the apps in use today use Bluetooth. Therefore, classifying the apps based on the underlying technology ensures that the research is capturing most contact tracing solutions in use. For instance, contact tracing solutions used by Singapore, Australia, and Malaysia use the same technology (ie, BlueTrace). As such, there is little benefit to the research from surveying all three of these apps.

    Therefore, the research evaluated 13 apps corresponding to 10 countries and covered all the contact tracing technologies identified above. All apps were free to download. The inclusion criteria also ensured that most of the COVID-19-declared epicenters (ie, countries) were included in the sample, such as Italy and the United States. The evaluated apps also included countries that did relatively well in controlling the outbreak of COVID-19, such as Singapore; countries that had a low daily number of new infections (ie, Australia); and countries that had a medium-level daily number of new infections (ie, Pakistan). The Swiss app was included in this study, as Switzerland was among the few countries that did not implement a lockdown. Similarly, the Swedish app was also included, given Sweden’s unique approach to building herd immunity to combat COVID-19. Informational apps or unofficial contact tracing apps were excluded from this study, except for South Korea’s Corona 100m app, which uses GPS technology for contact tracing. This app was included because Corona 100m was among the first major contact tracing apps that launched across the globe and because South Korea is one of the few countries that managed to quickly suppress the transmission of the virus. Figure 1 shows the apps that were included in this study.

    ‎
    Figure 1. The 13 apps corresponding to 10 countries included in this study. DP-3T: Decentralized Privacy-Preserving Proximity Tracing; PEPP-PT: Pan-European Privacy-Preserving Proximity Tracing; QR: Quick Response.
    View this figure

    Table 1 details the architecture and approaches used by each of these technologies; these are as follows:

    1. Country: For each type of technology used, a sample of countries that use this technology and their contact tracing apps are provided. Where there is more than one app used in a country, the name of the corresponding app is provided. It is worth noting that this is not a comprehensive list. The aim is to evaluate some of the countries for the purpose of adding context to the data presented in the table rather than creating an inventory of apps. The next section provides more details on the selection and inclusion criteria of the apps evaluated in this study.
    2. Architecture: This criterion investigates whether or not the technology used by the contact tracing app incorporates the concept of uploading contact logs to a central reporting server. The criteria used are centralized, semicentralized, and decentralized. It has proven difficult to exclusively classify the architecture of each of these technologies, as implementations varied from one app to another. For instance, some apps uploaded contact logs to a central server, but the server did not have access to the uploaded contact logs, nor was it responsible for any further contact tracing processing, while others had access. As such, this criterion should be read in conjunction with the other criteria presented in Table 1, mainly the encounter handshake and infection reporting criteria.
    3. Encounter handshake: This refers to how two devices coming into close contact perform a handshake (ie, exchange identification data). Most of the technologies surveyed exchanged some form of a temporary ID, while others exchanged some form of a unique identifier that was either encrypted or in plain text, which also depended on the specific implementation of each of the apps.
    4. Infection reporting. This refers to how the contact log is reported to the central server and the role of this server in contact tracing. Most of the apps relied on the users to upload the contact logs. Implementations varied as well based on whether the health authorities had access to the contact logs or not.
    5. Privacy by design. As the name suggests, this criterion explored whether the technology embedded any privacy considerations into its design specifications.

    To analyze the users’ intake of each of the 13 apps under review and the penetration by these apps, this study extracted the following data for each of the apps: the name and country where the app was launched, the number of installs as per Google Play and as reported by the local news in the home country of the app, the penetration percentage as per Google Play installs and as reported by local news sources, and the launch date of each of the apps.

    The number of installs were not only sourced from Google Play but also from local news outlets from the home country of each of the corresponding apps. The penetration percentages sourced from Google Play and the ones extracted from local news sources were calculated by dividing the total number of installs by the total population of the home country.

    Table 1. The technologies of the contact tracing apps and their salient features.
    App informationApp technology

    		BluetoothDP-3TaGPSPEPP-PTbTCNcGoogle and AppleOthers
    Example countriesdAustralia;
    Singapore;
    Malaysia (MyTrace)    		Austria;
    Finland;
    The Netherlands    		Iceland (Ranking C-19);
    Italy (Diary);
    Jordan (Aman)    		France;
    Georgia;
    Italy (Immuni)    		Germany (ITO);
    Italy;
    The United States    		Canada;
    Switzerland (SwissCovid);
    Germany (Corona-Warn-App)    		New Zealand (digital diary);
    Australia, Canada, and New Zealand (GetHomeSafe); Malaysia
    (SELangkah)
    ArchitectureCentralizedDecentralizedCentralizedCentralizedSemicentralizedDecentralizedCentralized
    Encounter handshakeUsers exchange temporary IDs issued by the serverUnique 128-bit pseudorandom identifier by the serverVaries by implementation; some identify users by phone numbersUsers exchange temporary IDs issued by the serverTCNUnique identifiers that are encrypted with a secret daily key held by the sending deviceReal ID
    Infection reportingUser-triggered uploadUser-triggered upload, but
    the health
    authority never has access to contact log    		User-triggered uploadUser-triggered upload, but received massive privacy backlashesThe app notifies the user to potential infectionNot provisioned; delegated to app implementationVaries by implementation; mostly user triggered
    Privacy by designNoYesNoNoYesYesNo

    aDP-3T: Decentralized Privacy-Preserving Proximity Tracing.

    bPEPP-PT: Pan-European Privacy-Preserving Proximity Tracing.

    cTCN: Temporary Contact Numbers.

    dWhere there was more than one app used in a country, the name of the corresponding app is provided within parentheses.

    Investigating the Privacy-by-Design Features and Privacy Implementations of COVID-19 Contact Tracing Apps

    This study expands on previous work [29] that compared the privacy aspects of the COVIDSafe app (Australia, Bluetooth) and the COVID Tracer app (New Zealand, QR code). Each of the selected apps was downloaded and evaluated thoroughly. The study first identified the underlying technology used for contact tracing by an app and the amount of personal information each app collected (ie, personal information access). To do that, the following scale was used: if an app was only collecting the name, email, and phone number of the user, then the scale was designated as low; if, in addition to this personal information, the app collected the age of the user, then the scale was designated as medium; and if an app collected the name, email, phone number, age, and any additional information, such as the address, ethnicity, or location via GPS of the user, then this criterion was rated as high.

    Additionally, the study analyzed the location features and tracking capabilities for each of the apps. It investigated whether an app was tracking the movement of individuals or not (ie, location tracking). It also investigated whether the app under review knew the identity of the people in close proximity to the user or just their locations or IDs (ie, true identity vs temporary ID, such as with the TCN protocol). The criterion tracking and identifying proxies combined the encounter handshake and infection reporting features.

    Furthermore, this study investigated the record-keeping time frame of each app. This was achieved by researching the duration that the contact logs were kept on the device or the authority’s remote servers for each of the apps under review.

    In terms of user control, this study examined two criteria: the user’s right to forget and the geo-restrictions imposed on an app. The first criterion considered whether or not users were informed about the procedures to delete the records collected by an app. The opting-out criterion explored whether the users were able to sing in and out of the app under review. Lastly, for each app under review, the study investigated whether an app could be downloaded from anywhere or whether it was a home or region geo-restricted app (ie, geo-restriction). We referred to governments’ media releases, white papers, and developers’ announcements for the apps that were in testing phases or were not available on the Apple App Store or Google Play.

    Analyzing the Public Reception of COVID-19 Contact Tracing Apps

    We aimed to identify the audience uptake and users’ feedback of the COVID-19 contact tracing apps under review. Data were sourced by scraping the publicly available user reviews from the Apple App Store and Google Play webpages for each of the apps. Scraping is a process or tool used to extract data from a website; in this case, reviews from Google and Apple stores. Almost 30,000 reviews were scraped and analyzed in this study. The user reviews of each of the corresponding apps were then filtered and analyzed using a brute-force keyword search methodology; this means extracting the user reviews that contained a specific keyword used in the search. Table 2 lists the keywords used in scraping the reviews. The methodology used for analyzing these reviews also accounted for the variations of each of the keywords, referred to as subkeywords. For instance, the results of scraping and analyzing certain subkeywords—doesn’t work, didn’t work, not working, Doesn’t work, Didn’t work, and Not working—were all counted toward the results of the main keyword Malfunctioning. In other words, the results reported under the keyword Malfunctioning are a concatenation of each of the individual results returned by its list of subkeywords.

    Table 2. The keywords used in this study.
    KeywordsSubkeywords
    Drainagedrain; battery; Drain; Battery
    Spywarespy; spied; spyware; Spy; Spied; Spyware
    Malfunctioningdoesn’t work; didn’t work; not working; Doesn’t work; Didn’t work; Not working
    Crashescrash; freeze; Crash; Freeze
    Privacy concernsprivacy issue; privacy concern; location concern; tracking me; track me; tracking us; Privacy issue; Privacy concern; Location concern; Tracking me; Track me; Tracking us
    Ineffectiveuseless; rubbish; garbage; Useless; Rubbish; Garbage
    Bugsbug; buggy; Bug; Buggy
    Installation issuescan’t install; doesn’t install; couldn’t install; Can’t install; Doesn’t install; Couldn’t install
    Incompatiblecan’t download; couldn’t download; incompatible; Can’t download; Couldn’t download; Incompatible

    Selection of Apps, User Intake, and Penetration

    In this section, we initially describe results on app penetration. A challenging aspect of sourcing the data reported in Table 3 [30-40] was encountered when calculating the intake of the apps under study. For instance, the number of downloads for an app does not represent the true value of the actual intake. Downloading an app does not necessarily mean the app is being used. Users may simply download the app and never use it or uninstall it. In addition, there were little data available on the number of uninstalls for each of the surveyed apps. Regardless of this limitation, the number of installations for an app was not available on the Apple App Store. This has made the task of calculating the uptake of an app even more complex.

    Consequently, the research required access to a more precise estimate of the installation values as compared to what Google Play was showing. Therefore, apart from consulting Google Play’s number of installs, the study referred to reliable news sources to obtain the total number of registrations or downloads for each of the apps under review. The news sources were mainly from government or developer announcements, verifiable local news sources, and published research (ie, white papers). Some of the statistical information, such as the download intakes and any data sourced from local news, was available as of early July 2020. As such, there might be a slight variation in the values presented in Table 3 as compared those at the time of the archiving of this paper. Some apps were new, so this local value was not readily available for those either. Another challenge this research study ran into was the unavailability of some of the apps on the Google Play Store. This is because they were discontinued or because they were still in demo or beta stages.

    Table 3. Penetration and intake of the 13 selected contact tracing apps.
    CountryAppNo. of installs, nPenetration, %No. of days of the app\'s launch since patient zeroa, n


    		Local newsGoogle Play StorebLocal newsGoogle Play Storeb
    United StatesPathCheck SafePlacesN/Ac10,000N/A0.00193
    United StatesNOVIDN/A10,000N/A0.001110
    United StatesCare1933,000 [31]10,0000.010.00176
    ItalyImmuni2,700,000 [32]1,000,0004.471.65122
    ItalySM-COVID-1952,000 [33]50,0000.090.0873
    NorwaySmittestopp1,427,000 [34]100,00026.321.8450
    SingaporeTraceTogether2,100,000 [35]1,000,00035.8917.0957
    South KoreaCorona 100m1,000,000 [36]N/A1.95N/A20
    PakistanCoCareN/A500N/A0.001108
    AustraliaCOVIDSafe6,130,000 [37]1,000,00024.033.9291
    New ZealandNZ COVID Tracer573,000 [38]100,00011.882.0782
    SwitzerlandSwissCovid1,600,000 [39]500,00018.485.7890
    GeorgiaStop Covid100,000 [40]100,0002.51N/A50

    aThe first case in the country was reported on the Johns Hopkins Coronavirus Resource Center portal [30].

    bAll download values have been extracted from the app’s webpage on Google Play.

    cN/A: not applicable; data for apps were unavailable because the app was new, in the case of local news, or because it was discontinued or was still in demo or beta stages, in the case of the Google Play Store.

    The results of this study show that South Korea was the first country to use a mobile app for contact tracing during the COVID-19 pandemic. The South Korean app, Corona 100m, was only introduced after 20 days from the first detected case of COVID-19 in South Korea. This was followed by Singapore, Norway, and Georgia, which introduced their apps around 50 days since patient zero. The United States, Italy, and Pakistan were slower, as they introduced their contact tracing app around the 100-day mark. As reported by local news, the Singaporean (36%) and Norwegian (26%) apps had the highest penetration intake, followed by the Australian and Swiss apps, which had around 20% penetration, and the New Zealand app, which achieved around 11% penetration. Interestingly, the Italian and US apps had the lowest penetration values. The penetration intake of the apps on the Android platform, which was calculated based on the Google Play–reported number of installs for each of the apps, showed that all apps under review, except for the Singaporean app, had very poor intake (<5%).

    Furthermore, this research initially intended to calculate the success rate of each of the apps in contact tracing reporting. It also aimed to survey and compare the efficacy of the apps under review. However, this was challenged by the lack of any reliable relevant data available in relation to those aspects; thus, this part of the review had to be excluded. Therefore, it is unclear whether the early introduction of contact tracing apps has contributed toward their rapid public adoption or whether these apps have played a major role in the contact tracing efforts of COVID-19. Perhaps these apps have played some part in raising awareness among the public, as in the case of the Singaporean and Australian apps, which had higher penetration intake values and lower infection numbers comparatively. However, due to insufficient data, no conclusive results can be made on the correlation between the early introduction of a contact tracing app, its higher penetration intake, and the case where low number of COVID-19 transmissions were reported.

    Investigating the Privacy-by-Design Features and Privacy Implementations of COVID-19 Contact Tracing Apps

    In the subsequent sections in this paper, when referring to an app, the following notation shall be used: app name (country of origin, technology used for contact tracing).

    Table S3 from Multimedia Appendix 1 reviews the privacy features of the 13 apps evaluated in this study. Each of these apps was downloaded and evaluated thoroughly as per the criteria shown in Table S3 from Multimedia Appendix 1. The research also referred to white papers and developers’ announcements for the apps that were in their testing phases or were not available or accessible on the Apple App Store and/or Google Play. The same methodology was followed for the apps that were not available in English, such as Immuni (Italy, Google and Apple application programming interface [API]) [41], SM-COVID-19 (Italy, ReCoVer) [42], and Smittestopp (Norway, Bluetooth and GPS) [43].

    Nine of the apps were available for free on both the Apple App Store and Google Play. Two apps—SM-COVID-19 (Italy, Google and Apple) and CoCare (Pakistan, Bluetooth) [44]—were only available on Google Play, while Stop Covid (Georgia, PEPP-PT) [45] was only available on the Apple App Store. The Corona 100m app (South Korea, location) [46] was not available on both stores. Smittestopp (Norway, Bluetooth and GPS) was not available to download due to geo-restrictions. The Australian COVIDSafe app required an Australian phone number and a postcode to run.

    Bluetooth was the most frequently used underlying technology, employed by seven apps for digital contact tracing, whereas three apps performed contact tracing through location (eg, GPS). The apps using location as the underlying technology, namely Corona 100m (South Korea, location) and PathCheck SafePlaces (United States, location), tracked and recorded the locations visited by the users. Although Corona 100m (South Korea, location) was removed from Google Play, the app integrated GPS history, data from nationwide surveillance cameras, and credit card transactions. This has sparked privacy concerns, as users of the Corona 100m app could see the date when a COVID-19 patient was infected, along with his or her nationality, gender, age, and the locations they visited.

    The Norwegian, Singaporean, Georgian [45], and New Zealand [47] apps were among the apps that collected the most personal information from the users, while some other apps, such as the Swiss app [48] and the Italian Immuni app, did not collect any user information. Other apps ranged from simply collecting users’ phone numbers to additionally collecting their names or email addresses.

    Data destruction was incorporated into most of the apps, which automatically deleted the users’ records after 14 days, the observed minimum amount of time implemented in most of the apps. Some kept these records for 21 days (ie, Australia) and others for 30 days (ie, Switzerland and India); the New Zealand app kept them for 31 days, while the Georgian apps kept them for 3 years, the longest of any app.

    Three of the US apps—PathCheck (United States, location) [49], NOVID (United States, Bluetooth radio waves and ultrasound) [50], and Care19 (United States, GPS) [51]—did not require users to sign up before using their app. On the other hand, many apps, such as the Singaporean TraceTogether app [52], the Australian COVIDSafe app [53], and the Swiss and Indian apps, did not provide the users with an option to sign out from their app. It is noteworthy to mention that the data presented in Table S3 of Multimedia Appendix 1 are accurate as of June 30, 2020.

    Analyzing the Public Reception of COVID-19 Contact Tracing Apps

    Figure 2 shows the percent occurrence for each of the keywords for each app. Figure 3 shows the average ratings of the reviews for each keyword. For example, consider if a user left a review for one of the apps saying, “the app keeps on crashing,” and then gave it a rating of 2 stars. This review will then be counted toward the average mentions of the keyword crashes shown in Figure 2. The 2-star rating will also be counted toward the corresponding keyword average rating shown in Figure 3. All small values were rounded up to 0.001.

    ‎
    Figure 2. Percent occurrence of each keyword for each app. NA: not applicable; user reviews were unavailable, as the corresponding apps were not available on the corresponding platforms.
    View this figure
    ‎
    Figure 3. Average ratings out of 5 stars from user reviews in each category of each app on Google Play.
    View this figure

    Three of the applications—CoCare (Pakistan, Bluetooth), SM-COVID-19 (Italy, ReCoVer), and Corona 100m (South Korea, location)—were not available on the Apple App Store, whereas two apps—the Corona 100m (South Korea, location) and Stop Covid (Georgia, PEPP-PT)—were not available on Google Play. Based on the frequency of keyword occurrences, Drain, Malfunctioning, and Ineffective were the most frequent issues reported by the users in their reviews.

    On the Apple App Store, the keyword rubbish had a 13.33% occurrence for PathCheck SafePlaces (United States, location), 5.56% for NOVID (United States, Bluetooth), 5.40% for Immuni (Italy, Google and Apple API), and 9.09% for NZ COVID Tracer (New Zealand, digital diary). Similarly, many users did not find contact tracing apps functional. On the Apple App Store, many app users complained that their app did not work. This was represented by the keyword Malfunctioning, which had a 10.74% occurrence for NZ COVID Tracer (New Zealand, digital diary), 6.50% for COVIDSafe (Australia, Bluetooth), 6.67% for TraceTogether (Singapore, Bluetooth), 7.80% for Immuni (Italy, Google and Apple API), 11.11% for NOVID (United States, Bluetooth), and a sharp 12.77% occurrence for Care19 (United States, Apple and Google). Many users also had problems with the apps’ compatibility with their operating system and frequent crashes. For instance, CoCare (Pakistan, Bluetooth) had a 16.67% occurrence for the incompatibility issue.

    Interestingly, and as shown in Figure 4, no significant battery drainage issue had been reported for most of the reviewed apps. The privacy concerns reported by the users were also very minimal across all apps, as shown in Figure 5.

    ‎
    Figure 4. Percent occurrence of the keyword "drainage," pertaining to battery drainage.
    View this figure
    ‎
    Figure 5. Percent occurrence of the keywords "privacy concerns" and "spyware".
    View this figure

    Figures 6 and 7 provide overall insights into the technical issues reported by the users for each of the apps. These figures combine the results of the following keywords, along with their respective subkeywords: Malfunctioning, Crashes, Ineffective, Bugs, Installation issues, and Incompatible. It is obvious that most apps on the Apple App Store had the most reported technical issues when compared to their Google Play counterparts, except for the Swiss contact tracing app. The US PathCheck app had the least reported technical issues on Google Play, while the New Zealand app version on the Apple App Store had the most technical issues that were complained about across all apps and platforms.

    ‎
    Figure 6. Summary of technical issues reported for each of the apps. The plot shows results from the combination of the following keywords, along with their respective subkeywords: "Malfunctioning," "Crashes," "Ineffective," "Bugs," "Installation issues," and "Incompatible".
    View this figure
    ‎
    Figure 7. Comparison of user app reviews and their inclusion of various keywords.
    View this figure

    Principal Findings

    Our research study has highlighted the hindrances in the successful deployment of COVID-19 contact tracing apps. The use of mobile technologies for contact tracing has been met with a number of challenges [54], many of which also emerged in the contextual evaluation of user reviews on COVID-19 contact tracing apps as described in our study. Among these, the most popular were technical malfunctions and drainage of battery.

    Other challenges included privacy. Of course, this is anticipated, as you cannot expect to trace and track peoples’ movements by a government authority without addressing privacy issues [55]. Nonetheless, in addition to privacy, there were many other challenges and limitations hindering the anticipated efficacy from contact tracing apps.

    For instance, a mobile contact tracing app needs to be widely adopted by a population for it to be of benefit; this is challenging to achieve. Penetration of COVID-19 contact tracing apps remains low despite governments pushing for mass use [56]. The widespread adoption of contact tracing apps requires that people would have access to a smartphone and, in most cases, access to a reliable internet connection. Hence, in countries with large populations, such as Pakistan [57], the smartphone penetration percentage sits at only 15%, and in Indonesia this value sits at only 31%. Users may also feel uncomfortable if there is no clear opt-out strategy [58]. Nearly half of the apps reviewed in our study did not provide a transparent withdrawal avenue.

    Furthermore, the approaches used by contact tracing apps rely mostly on one single parameter (ie, proximity, such as via Bluetooth) [59]. However, proximity by itself is not enough to determine the risk of someone being exposed to the virus. There are a number of other parameters involved, such as being indoors or outdoors, being in a room with good air circulation or not, and the issue of surface infection exposure, irrespective of the proximity of an individual to an infected person. Furthermore, although as shown in our review that Bluetooth was one of the more popular technologies to implement contact tracing apps, every country’s regulations may differ [60]; hence, a one-size-fits-all approach may be problematic.

    Other challenges pertain to the limitations associated with the technology used for contact tracing. For instance, the use of GPS as a proximity technology is not reliable in indoor environments [61]. Determining the distance between two persons using Bluetooth technology also has its own set of challenges, such as signal strength attenuation caused by some environmental factors (eg, if the phone is placed inside a thick pocket or if the phone is at an angle facing a wall).

    Nevertheless, contact tracing technologies surveyed in this work have been found to use a locationless tracking approach; that is, the app does not trace or record people’s movements, obviously for privacy purposes. Therefore, most of these apps can only determine if two people were in proximity at a given time, but they do not keep a log of the users’ movements. Consider, for example, if an infected person, labelled as Pi, is in a supermarket and Pi touches an item at time t–1 at a location designated as Li. Another person who is not infected, designated as Pn, is at a location designated as Ln. There is no proximity between Pi and Pn. Now assume Pi leaves the store at time t, when at the same time (ie, at t) person Pn moves from Ln to Li. There is a high chance that Pn is going to be infected if they touch the same item Pi touched at t–1 (ie, surface infection exposure). To be able to capture this exposure, contact tracing apps require the use of a location-oriented tracking approach in which the locations and movements of people are compared against each other to determine the overlapped and colluded locations. Future work will explore the use of our already well-established location obfuscation technique [62] in a contact tracing solution. The work will aim at providing a location-oriented contact tracing app without impinging on users’ privacy.

    Limitations

    One of the challenges encountered in scraping the reviews was analyzing the apps that were not available in English. For example, most of the reviews for the Immuni app (Italy, Google and Apple), SM-COVID-19 (Italy, ReCoVer), and Smittestopp (Norway, Bluetooth and GPS) were available in the Italian and Norwegian languages, respectively. For these reviews, along with the rest of the app reviews that were in different languages, the keywords along with their subkeywords were translated into the language of their home app country. The results were incorporated when calculating the overall average values for all the apps. The translated keywords along with the subkeywords used can be found in Table S4 from Multimedia Appendix 1. Another limitation in our methodology for review scraping lies in the presence of false negatives in some of the reviews. This is one of the limitations of brute-force keyword search methodology. Take, for instance, one of the reviews for COVIDSafe (Australia, Bluetooth) on Google Play:

    Installed from its release. Worked. No problems at all. It doesn't drain the battery. It doesn't crash. It's totally fine. I haven't been dragged into the back of a van, taken to an underground bunker and questioned by spies.

    The review is classified as a false negative for the words drain and crash. It can be debated that the number of false negatives could have been reduced by simply taking the battery subkeyword out from the keyword search (ie, battery; drain). However, in doing so, the number of 1-star reviews were significantly reduced by more than 50%. For instance, with NZ COVID Tracer (New Zealand, digital diary), the 1-star reviews dropped from 23 to 10 after taking the word battery out of the search filter. The reason behind this is that the users’ reviews were not systematic. Most users represented their opinions in natural language. Some samples of 1-star reviews for COVIDSafe (Australia, Bluetooth) commenting on the app’s drainage issue are as follows:

    It is of no use whatsoever. A waste of money & a waste of my battery life.
    Battery went from 100% to zero in 5 hours with not much use. I usually get a full day out of it.
    Hard on the battery.

    Therefore, for the sake of including these comments, the subkeyword battery was not removed from the keyword search results.

    This study has a number of additional limitations. This paper is based solely on 13 evaluated apps. While the selection criteria ensured that apps were selected to represent each of the categories of technology used for contact tracing, it did not review all COVID-19 contact tracing apps. Also, the study relied on data that were extracted and accurate as of July 2020. Another major limitation of this work relates to the penetration intake calculations done for each of the apps. The study derived the percent penetration for each app by dividing the number of installs of an app by the population of the home country. This method suffers from several shortcomings. The number of downloads or installs cannot be precise; it ignores the fact that some users may install, uninstall, and reinstall the app several times or on more than one device owned by the user. A user may also download the app and never use it. Many users may not download an app yet may still post a review about it. For these reasons, the study attempted to gauge the penetration intake for each of the apps by analyzing the local government announcements and reports published by local news agencies. However, since the reported download values cannot be verified nor the methods used to derive them, the trustworthiness of these values also remains invalidated. Lastly, although the study evaluated about 30,000 user reviews, the reviews cannot not be verified.

    Conclusions

    While public health agencies attempt to understand the efficacy of nonpharmaceutical interventions [63], contact tracing has been a key part of the worldwide measure in response to the COVID-19 pandemic. For contact tracing to work effectively, solutions such as tracing apps should be implemented systematically. This requires the secure collection, processing, storage, and discarding of contact tracing information of people in real time, without impinging on their privacy and rights. The success of contact tracing apps greatly depends on their large uptake within a population, in addition to strong public health enforcement. This article highlighted the fact that COVID-19 contact tracing apps are still facing many obstacles toward their widespread and public acceptance.

    The main challenges are related to the technical, usability, and privacy issues or requirements reported by some users. This meant that most tracing apps were not publicly well-received and had low penetration levels, which hinders their effectiveness. For instance, only the Singaporean app had a penetration of slightly over 30%, the Australian and Swiss apps achieved penetration just below 20%, and the penetration values reported for most of the other apps were very poor, sitting at below 5%. The amount of personal data collected by the apps varied widely, with some apps not collecting data at all and others collecting a significant amount of sensitive data about the user, such as their ethnicity. The majority of the surveyed apps did not provide the user with options to opt out from the apps, such as logging out, without uninstalling them.

    The lack of a standardized contact tracing approach also meant that contact tracing apps used across the globe were fragmented and noninteroperable. As most countries are now coming out of lockdown and reopening their borders, there is an increased need for a cohesive, cross-border, and interoperable contact tracing app that can be used universally without impinging on users’ privacy. Additionally, there is a lack of available data on the effectiveness of COVID-19 contact tracing apps. As we progressively recuperate from this pandemic, there is a need to re-evaluate and re-examine the values and roles of contact tracing apps in controlling infectious diseases such as COVID-19.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Summary of papers related to contact tracing apps, the main technologies used, the privacy features, and translated keywords.

    DOCX File , 47 KB
    1. COVID-19 situation update worldwide. European Centre for Disease Prevention and Control. 2020.   URL: https://www.ecdc.europa.eu/en/geographical-distribution-2019-ncov-cases [accessed 2020-08-11]
    2. Akkermans J, Richardson J, Kraimer ML. The Covid-19 crisis as a career shock: Implications for careers and vocational behavior. J Vocat Behav 2020 Jun;119:103434 [FREE Full text] [CrossRef] [Medline]
    3. Baker MG, Kvalsvig A, Verrall AJ. New Zealand's COVID-19 elimination strategy. Med J Aust 2020 Sep;213(5):198-200.e1 [FREE Full text] [CrossRef] [Medline]
    4. Chang SL, Harding N, Zachreson C, Cliff OM, Prokopenko M. Modelling transmission and control of the COVID-19 pandemic in Australia. Nat Commun 2020 Nov 11;11(1):5710 [FREE Full text] [CrossRef] [Medline]
    5. Orlowski EJW, Goldsmith DJA. Four months into the COVID-19 pandemic, Sweden's prized is nowhere in sight. J R Soc Med 2020 Aug;113(8):292-298. [CrossRef] [Medline]
    6. Strzelecki A. The second worldwide wave of interest in coronavirus since the COVID-19 outbreaks in South Korea, Italy and Iran: A Google Trends study. Brain Behav Immun 2020 Aug;88:950-951 [FREE Full text] [CrossRef] [Medline]
    7. Ren X. Pandemic and lockdown: A territorial approach to COVID-19 in China, Italy and the United States. Eurasian Geogr Econ 2020 May 08;61(4-5):423-434 [FREE Full text] [CrossRef]
    8. Reis RF, de Melo Quintela B, de Oliveira Campos J, Gomes JM, Rocha BM, Lobosco M, et al. Characterization of the COVID-19 pandemic and the impact of uncertainties, mitigation strategies, and underreporting of cases in South Korea, Italy, and Brazil. Chaos Solitons Fractals 2020 Jul;136:109888 [FREE Full text] [CrossRef] [Medline]
    9. Chiu NC, Chi H, Tai YL, Peng CC, Tseng CY, Chen CC, et al. Impact of Wearing Masks, Hand Hygiene, and Social Distancing on Influenza, Enterovirus, and All-Cause Pneumonia During the Coronavirus Pandemic: Retrospective National Epidemiological Surveillance Study. J Med Internet Res 2020;22(8):e21257 [FREE Full text] [CrossRef] [Medline]
    10. Du L, Raposo VL, Wang M. COVID-19 Contact Tracing Apps: A Technologic Tower of Babel and the Gap for International Pandemic Control. JMIR Mhealth Uhealth 2020;8(11):e23194 [FREE Full text] [CrossRef] [Medline]
    11. Kleinman RA, Merkel C. Digital contact tracing for COVID-19. CMAJ 2020 Jun 15;192(24):E653-E656 [FREE Full text] [CrossRef] [Medline]
    12. Sabbagh D, Hern A. UK abandons contact-tracing app for Apple and Google model. The Guardian. 2020 Jun 18.   URL: https:/​/www.​theguardian.com/​world/​2020/​jun/​18/​uk-poised-to-abandon-coronavirus-app-in-favour-of-apple-and-google-models [accessed 2021-01-28]
    13. Tom-Aba D, Nguku PM, Arinze CC, Krause G. Assessing the concepts and designs of 58 mobile apps for the management of the 2014-2015 West Africa Ebola outbreak: Systematic review. JMIR Public Health Surveill 2018 Oct 29;4(4):e68 [FREE Full text] [CrossRef] [Medline]
    14. Littman-Quinn R, Chandra A, Schwartz A, Fadlelmola F, Ghose S, Luberti A, et al. mHealth applications for telemedicine and public health intervention in Botswana. In: Proceedings of the IST-Africa Conference. 2011 Presented at: IST-Africa Conference; May 11-13, 2011; Gaborone, Botswana p. 1-11   URL: https://ieeexplore.ieee.org/document/6107376
    15. Cunche M, Boutet A, Castelluccia C, Lauradoux C, Roca V. On Using Bluetooth-Low-Energy for Contact Tracing. Montbonnot-Saint-Martin, France: Inria Grenoble Rhône-Alpes; INSA de Lyon; 2020.   URL: https://hal.inria.fr/hal-02878346v5/document [accessed 2021-01-29]
    16. Michael K, Abbas R. Behind COVID-19 contact trace apps: The Google–Apple partnership. IEEE Consum Electron Mag 2020 Sep 1;9(5):71-76. [CrossRef]
    17. Altmann S, Milsom L, Zillessen H, Blasone R, Gerdon F, Bach R, et al. Acceptability of app-based contact tracing for COVID-19: Cross-country survey study. JMIR Mhealth Uhealth 2020 Aug 28;8(8):e19857 [FREE Full text] [CrossRef] [Medline]
    18. Anglemyer A, Moore T, Parker L, Chambers T, Grady A, Chiu K. Digital contact tracing technologies in epidemics: A rapid review. Cochrane Database Syst Rev 2020:1-46. [CrossRef]
    19. Budd J, Miller BS, Manning EM, Lampos V, Zhuang M, Edelstein M, et al. Digital technologies in the public-health response to COVID-19. Nat Med 2020 Aug;26(8):1183-1192. [CrossRef] [Medline]
    20. Kondylakis H, Katehakis DG, Kouroubali A, Logothetidis F, Triantafyllidis A, Kalamaras I, et al. COVID-19 Mobile Apps: A Systematic Review of the Literature. J Med Internet Res 2020;22(12):e23170 [FREE Full text] [CrossRef] [Medline]
    21. Baraniuk C. Covid-19 contact tracing: A briefing. BMJ 2020 May 13;369:m1859. [CrossRef] [Medline]
    22. Magklaras G, Bojorquez L. A review of information security aspects of the emerging COVID-19 contact tracing mobile phone applications. In: Proceedings of the International Symposium on Human Aspects of Information Security and Assurance. 2020 Presented at: International Symposium on Human Aspects of Information Security and Assurance; July 7-9, 2021; Mytilene, Greece p. 30-44. [CrossRef]
    23. Collado-Borrell R, Escudero-Vilaplana V, Villanueva-Bueno C, Herranz-Alonso A, Sanjurjo-Saez M. Features and functionalities of smartphone apps related to COVID-19: Systematic search in app stores and content analysis. J Med Internet Res 2020 Aug 25;22(8):e20334 [FREE Full text] [CrossRef] [Medline]
    24. Martin T, Karopoulos G, Hernández-Ramos JL, Kambourakis G, Nai Fovino I. Demystifying COVID-19 digital contact tracing: A survey on frameworks and mobile apps. Wirel Commun Mob Comput 2020 Oct 17;2020:1-29. [CrossRef]
    25. Davalbhakta S, Advani S, Kumar S, Agarwal V, Bhoyar S, Fedirko E, et al. A systematic review of smartphone applications available for coronavirus disease 2019 (COVID19) and the assessment of their quality using the Mobile Application Rating Scale (MARS). J Med Syst 2020 Aug 10;44(9):164 [FREE Full text] [CrossRef] [Medline]
    26. Vaudenay S. Centralized or decentralized? The contact tracing dilemma. OpenAIRE. 2020 Jun 01.   URL: https:/​/covid-19.​openaire.eu/​search/​publication?articleId=od_______185::b333a9cdf196cdc396fede6971196acc [accessed 2021-01-29]
    27. LI J, Guo X. Global deployment mappings and challenges of contact-tracing apps for COVID-19. SSRN 2020 May 24:1-7 [FREE Full text] [CrossRef]
    28. Ahmed N, Michelin RA, Xue W, Ruj S, Malaney R, Kanhere SS, et al. A survey of COVID-19 contact tracing apps. IEEE Access 2020;8:134577-134601. [CrossRef]
    29. Elkhodr M. New Zealand’s COVID-19 Tracer app won’t help open a ‘travel bubble’ with Australia anytime soon. The Conversation. 2020 May 20.   URL: https:/​/theconversation.​com/​new-zealands-covid-19-tracer-app-wont-help-open-a-travel-bubble-with-australia-anytime-soon-139026 [accessed 2021-01-28]
    30. COVID-10 dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JSU). Johns Hopkins Coronavirus Resource Center.   URL: https://coronavirus.jhu.edu/map.html [accessed 2021-01-28]
    31. Howell O'Neill P. Why one US state will have two coronavirus tracing apps. MIT Technology Review. 2020 May 20.   URL: https:/​/www.​technologyreview.com/​2020/​05/​20/​1002042/​why-one-us-state-will-have-two-coronavirus-tracing-apps/​ [accessed 2021-01-28]
    32. D'Alessandro J. La app anti pandemia Immuni vicina ai tre milioni di download. la Repubblica. 2020 Jun 16.   URL: https:/​/www.​repubblica.it/​tecnologia/​2020/​06/​16/​news/​immuni_vicina_ai_tre_milioni_di_download-259372693/​ [accessed 2021-01-28]
    33. SM-COVID-19.   URL: https://smcovid19.org/ [accessed 2021-01-28]
    34. Nikel D. Norway: 1.4 million people download coronavirus tracking app despite security concerns. Forbes. 2020 Apr 25.   URL: https:/​/www.​forbes.com/​sites/​davidnikel/​2020/​04/​25/​norway-14-million-people-download-coronavirus-tracking-app-despite-security-concerns/​?sh=fb00dc57832c [accessed 2021-01-28]
    35. TraceTogether. Government of Singapore.   URL: https://www.tracetogether.gov.sg/ [accessed 2021-01-28]
    36. Dujmovic J. Wildly popular coronavirus-tracker app helps South Koreans steer clear of outbreak areas. MarketWatch. 2020 Mar 18.   URL: https:/​/www.​marketwatch.com/​story/​wildly-popular-coronavirus-tracker-app-helps-south-koreans-steer-clear-of-outbreak-areas-2020-03-18 [accessed 2021-01-28]
    37. Meixner S. How many people have downloaded the COVIDSafe app and how central has it been to Australia's coronavirus response? ABC News. 2020 Jun 01.   URL: https:/​/www.​abc.net.au/​news/​2020-06-02/​coronavirus-covid19-covidsafe-app-how-many-downloads-greg-hunt/​12295130 [accessed 2021-01-28]
    38. Changelog: Covid Tracing Tracker updates as they happen. MIT Technology Review. 2020 May 15.   URL: https:/​/www.​technologyreview.com/​2020/​05/​15/​1001736/​changelog-covid-tracing-tracker-updates-as-they-happen/​ [accessed 2021-01-28]
    39. More than 1.6 million Swiss have downloaded Covid tracing app. The Local CH. 2020 Jun 29.   URL: https://www.thelocal.ch/20200629/more-than-800000-downloads-for-switzerlands-covid-app-in-three-days [accessed 2021-01-28]
    40. More than 100 thousand people downloaded covonavirus tracking app in Georgia. Georgian Journal. 2020 Apr 27.   URL: https:/​/www.​georgianjournal.ge/​society/​36537-more-than-100-thousand-people-downloaded-covonavirus-tracking-app-in-georgia.​html [accessed 2021-01-28]
    41. Immuni, Ministero della Salute. Google Play.   URL: https://play.google.com/store/apps/details?id=it.ministerodellasalute.immuni&hl=en_AU&gl=US [accessed 2021-01-28]
    42. SM_Covid19, Softmining Srl. Google Play.   URL: https:/​/play.​google.com/​store/​apps/​details?id=it.​softmining.​projects.​covid19.​savelifestyle&hl=en_AU&gl=US [accessed 2021-01-28]
    43. Smittestopp, Folkehelseinstituttet. Google Play.   URL: https:/​/play.​google.com/​store/​apps/​details?id=no.​fhi.​smittestopp_exposure_notification&hl=en_AU&gl=US [accessed 2021-01-28]
    44. CoCare, Cheetay. Google Play.   URL: https://play.google.com/store/apps/details?id=com.cocareapp&hl=en_AU&gl=US [accessed 2021-01-28]
    45. Stop Covid - let's fight this together, Ministry of IDPs, Labour, Health, Social Affairs. Google Play.   URL: https:/​/webcache.​googleusercontent.com/​search?q=cache:12nc9hpdEu8J:https:/​/play.​google.com/​store/​apps/​details%3Fid%3Dgov.​georgia.​novid20%26hl%3Den_CA%26gl%3DUS+&cd=1&hl=en&ct=clnk&gl=ca [accessed 2021-01-28]
    46. Review of mobile application technology to enhance contact tracing capacity for COVID-19. Johns Hopkins Center for Health Security. 2020 Apr 08.   URL: https:/​/www.​centerforhealthsecurity.org/​resources/​COVID-19/​COVID-19-fact-sheets/​200408-contact-tracing-factsheet.​pdf [accessed 2021-01-28]
    47. NZ COVID Tracer app. Ministry of Health NZ. 2020.   URL: https:/​/www.​health.govt.nz/​our-work/​diseases-and-conditions/​covid-19-novel-coronavirus/​covid-19-resources-and-tools/​nz-covid-tracer-app [accessed 2021-01-28]
    48. SwissCovid, Federal Office of Public Health FOPH. Google Play.   URL: https://play.google.com/store/apps/details?id=ch.admin.bag.dp3t&hl=en_AU&gl=US [accessed 2021-01-28]
    49. SafePlaces, Path Check, Inc. Google Play.   URL: https://play.google.com/store/apps/details?id=org.pathcheck.covidsafepaths&hl=en_AU&gl=US [accessed 2021-01-28]
    50. NOVID, Expii, Inc. Google Play.   URL: https://play.google.com/store/apps/details?id=com.expii.novid&hl=en_AU&gl=US [accessed 2021-01-28]
    51. Care19 Diary, ProudCrowd, LLC. Google Play.   URL: https://play.google.com/store/apps/details?id=com.proudcrowd.care&hl=en_AU&gl=US [accessed 2021-01-28]
    52. TraceTogether, Government Technology Agency. Google Play.   URL: https://play.google.com/store/apps/details?id=sg.gov.tech.bluetrace&hl=en_AU [accessed 2021-01-28]
    53. COVIDSafe app. Australian Government Department of Health. 2020.   URL: https://www.health.gov.au/resources/apps-and-tools/covidsafe-app [accessed 2021-01-28]
    54. Dahiya N, Kakkar AK. Mobile health: Applications in tackling the Ebola challenge. J Family Med Prim Care 2016;5(1):192-193 [FREE Full text] [CrossRef] [Medline]
    55. Rowe F, Ngwenyama O, Richet J. Contact-tracing apps and alienation in the age of COVID-19. Eur J Inf Syst 2020 Sep 13;29(5):545-562. [CrossRef]
    56. Morley J, Cowls J, Taddeo M, Floridi L. Ethical guidelines for COVID-19 tracing apps. Nature 2020 Jun;582(7810):29-31. [CrossRef] [Medline]
    57. Gu T. Newzoo. 2019 Sep 17.   URL: https:/​/newzoo.​com/​insights/​articles/​newzoos-global-mobile-market-report-insights-into-the-worlds-3-2-billion-smartphone-users-the-devices-they-use-the-mobile-games-they-play/​ [accessed 2021-01-28]
    58. Hutton L, Price BA, Kelly R, McCormick C, Bandara AK, Hatzakis T, et al. Assessing the Privacy of mHealth Apps for Self-Tracking: Heuristic Evaluation Approach. JMIR Mhealth Uhealth 2018;6(10) [FREE Full text] [CrossRef] [Medline]
    59. Zhao Q, Wen H, Lin Z, Xuan D, Shroff N. On the accuracy of measured proximity of Bluetooth-based contact tracing apps. In: Proceedings of the International Conference on Security and Privacy in Communication Systems. 2020 Presented at: International Conference on Security and Privacy in Communication Systems; October 21-23, 2020; Washington, WA p. 49-60. [CrossRef]
    60. Cho H, Ippolito D, Yu Y. Contact Tracing Mobile Apps for COVID-19: Privacy Considerations and Related Trade-offs. arXiv.org 2020:1-12 [FREE Full text]
    61. Poudyal A, van Heerden A, Hagaman A, Maharjan SM, Byanjankar P, Subba P, et al. Wearable Digital Sensors to Identify Risks of Postpartum Depression and Personalize Psychological Treatment for Adolescent Mothers: Protocol for a Mixed Methods Exploratory Study in Rural Nepal. JMIR Res Protoc 2019;8(8):e14734 [FREE Full text] [CrossRef] [Medline]
    62. Elkhodr M, Shahrestani S, Cheung H. A semantic obfuscation technique for the Internet of Things. In: Proceedings of the International Conference on Communications Workshops (ICC). 2014 Presented at: IEEE International Conference on Communications Workshops (ICC); June 10-14, 2014; Sydney, Australia p. 448-453. [CrossRef]
    63. Arsalan M, Mubin O, Alnajjar F, Alsinglawi B, Zaki N. Global and temporal COVID-19 risk evaluation. Front Public Health 2020;8:440 [FREE Full text] [CrossRef] [Medline]

    ‎
    API: application programming interface
    DP-3T: Decentralized Privacy-Preserving Proximity Tracing
    PEPP-PT: Pan-European Privacy-Preserving Proximity Tracing
    QR: Quick Response
    TCN: Temporary Contact Numbers

    Edited by R Kukafka; submitted 13.08.20; peer-reviewed by J Li, A Johnson; comments to author 25.09.20; revised version received 14.10.20; accepted 20.01.21; published 09.02.21

    Copyright

    ©Mahmoud Elkhodr, Omar Mubin, Zainab Iftikhar, Maleeha Masood, Belal Alsinglawi, Suleman Shahid, Fady Alnajjar. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 09.02.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.