Published on in Vol 23, No 5 (2021): May

Preprints (earlier versions) of this paper are available at, first published .
Electronic Tools to Bridge the Language Gap in Health Care for People Who Have Migrated: Systematic Review

Electronic Tools to Bridge the Language Gap in Health Care for People Who Have Migrated: Systematic Review

Electronic Tools to Bridge the Language Gap in Health Care for People Who Have Migrated: Systematic Review


1Patient-Reported Outcomes Unit (PROQOL), UMR 1123, Université de Paris, INSERM, F-75004, Paris, France

2Unité de Recherche Clinique en Economie de la Santé (URC-ECO), AP-HP, Hôpital Hôtel-Dieu, F-75004, Paris, France

3Faculty of Pharmacy, Ramaiah University of Applied Sciences, Karnataka, India

4Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States

5Service de Médecine Interne et d'Immunologie Clinique, Hôpital Bicêtre, F94270, Le Kremlin Bicêtre, France

Corresponding Author:

Frédérique Thonon, MPH, PhD

Patient-Reported Outcomes Unit (PROQOL), UMR 1123, Université de Paris, INSERM, F-75004

Hôpital Hôtel Dieu

1 place du Parvis Notre Dame

Paris, 75004


Phone: 33 140275543


Background: People who have migrated or with a language barrier may face significant hurdles in accessing health care. Some apps have been specifically developed to facilitate the dialogue between health care professionals and people who have migrated who have low-level language proficiency or to promote health among people who have migrated.

Objective: We conducted a systematic review to investigate development, acceptability, and effectiveness of these types of apps.

Methods: We conducted a search of PubMed, Scopus, and Embase databases. We included all study designs (qualitative, quantitative, mixed) reporting development, evaluation of efficacy, or acceptability of apps facilitating dialogue with a health professional or promoting health for people who have migrated, minorities, or tourists with a language barrier, using any outcome. Two researchers selected the studies independently. We collected general information about the app, information about health literacy and cultural adaptation, information about the development of the app, evidence on acceptability or efficacy, and information on app use. Data were collected by 2 researchers independently and results were reviewed to verify agreement and reported according to PRISMA (Preferred Reporting Items for Systematic Review and Meta-analysis).

Results: Positive results for translation apps included better communication, but with possible limitations, and reduced consultation time. Positive results for health promotion apps included improved quality of life and better management of chronic illnesses.

Conclusions: Overall, the apps had good levels of acceptability, though only half had their efficacy evaluated. In those evaluations, the endpoints were mostly related to reported behavior change and knowledge improvement, which is common for evaluations of health promotion programs. In the future, as more health apps are created, it is essential that apps that claim to have a public health objective undergo a rigorous evaluation of their acceptability, efficacy, and actual use. Indicators of outcomes beyond changes in behavior and knowledge should be reported; change in health status or access to care should also be reported. This systematic review has helped us note the characteristics associated with improved acceptability and efficacy, which can be helpful for the development of future apps.

J Med Internet Res 2021;23(5):e25131



People who have migrated may face significant delays and barriers in accessing health care, especially those who do not fluently speak the language of the host country. Research has been conducted to investigate language barriers in accessing care and prevention among people who have migrated and its health consequences. Pregnant women who have migrated and with low proficiency in the language of their new country of residence have found it more difficult to access care [1], had less knowledge about the benefits of folic acid and had lower folic acid intake [2], and had a higher risk of obstetric trauma [3]. A study [4] showed that a language barrier is significantly associated with a higher occurrence of serious medical events in pediatrics. Cancer studies show less screening for colorectal cancer [5,6], cervical cancer [6], and breast cancer [6,7] among people who have migrated who had low English proficiency. Others have found that adding an interpreter to a consultation where the patient has a language barrier (any type of interpretation) results in the reduction of obstetric interventions [8], better clinical outcomes in people with diabetes [7,8], a higher rate of breast cancer and colorectal cancer screening [7], and a higher rate of influenza vaccination [7]. Overall, it has also been shown that patients with a low proficiency in the language of the country of residence receive more preventive advice and more prescriptions and have fewer emergency visits if they use interpretation services [9].

Health care professionals commonly use several solutions to communicate with people who have migrated with low language proficiency, including printed guides and brochures, informal interpreters, professional interpreters in person, professional interpreters on phone or video conference, and general translation apps. Some apps or electronic tools have been specifically developed to be used in medical consultations to facilitate dialogue between health care professionals and people who have migrated who have low language proficiency or to promote health among people who have migrated. In the last decade, there has been a sharp increase in the number of medical apps developed [10], but their impact on patients or public health should be established.

We conducted a systematic review to examine the evidence related to the development, adaptation, acceptability, and effectiveness of electronic tools designed to help health care providers communicate with or promote health among people who have migrated and who have low levels of proficiency in the language of their country or low levels of health literacy. The aim of this review was to describe the existing tools and gather evidence about features that increase the acceptability and efficacy of such tools. Our work is part of a larger project designed to develop and evaluate an app to facilitate communication between people who have migrated who face a language barrier and health professionals regarding testing for HIV and different forms of viral hepatitis [11]. Lessons learned and evidence from similar apps could help design an app that has a significant impact on public health.

Search Strategy, Databases, and Keywords

We wrote a protocol prior to starting the systematic review and reported the results according to the PRISMA reporting guideline [12] (Multimedia Appendix 1). We conducted a search of 3 databases of scientific publications: PubMed, Scopus, and Embase. The keywords varied according to the database used. To determine keywords, we undertook a broad preliminary search and selected some articles (between 5 and 10) identified as meeting inclusion criteria determined by the authors. After selecting these articles, we searched for their related keywords, MeSH terms (Medical Subject Headings), terms in titles or abstracts, and EmTree terms. Then the selection of keywords was tested and different combinations were tested so that the total number of results was manageable and yielded relevant articles. The final selection of keywords was critically reviewed by a university librarian. The keywords for each database are detailed in Multimedia Appendix 2.

Inclusion and Exclusion Criteria

We used the inclusion and exclusion criteria detailed in Table 1. Although the primary focus of this review was people who have migrated, the preliminary search yielded articles reporting apps that helped bridge the language barrier for other populations, such as tourists needing emergency care and not speaking the language of the country visited or indigenous people whose primary language is different from the official language. Since these apps help health care providers communicate with patients with a language barrier, we decided to include them because we found they were relevant.

Table 1. Inclusion and exclusion criteria.
Criteria typeInclusion criteriaExclusion criteria

  • Written in English or French
  • Other languages

Date range
  • Published after 1998
  • Published before 1998

  • Original article, review, protocol, conference abstract, book chapter
  • Other types of publications: editorial, letter, notes, etc

  • Studies reporting the development of an electronic tool, including qualitative or quantitative studies of people who have migrated or health providers, mixed methods, literature reviews
  • Studies evaluating the acceptability of electronic tools, including qualitative or quantitative studies, usability studies, randomized or nonrandomized trials
  • Studies evaluating the efficacy of electronic tools, including randomized or nonrandomized trials, qualitative or quantitative studies, economic evaluations
  • Articles lacking information about the development or evaluation of an electronic tool
  • Studies exploring only the perceptions of users (people who have migrated or health professionals) related to e-health or a health issue


  • International people who have migrated not fluent with the language of the country they reside
  • People with no language barrier (internal people who have migrated, ethnic minorities, people who have migrated with no language barrier)

Communication barriers
  • Cultural minorities having a language barrier (eg, indigenous people whose first language is different from the official language)
  • Tourists
  • People with other type of communication barriers: deaf or hard-hearing people, people with a learning disability


  • Website, mobile (smartphone or tablet) apps, other electronic technology that allows interaction with user text message or email-based services
  • Tools using only print material, audio, or video


  • Technology designed to help communication between health care providers (eg, doctors, nurses, midwifes) and people who have migrated in any health care setting (eg, hospital, primary care)
  • Technology designed to promote healthy behavior among people who have migrated
  • Technology that aims to facilitate communication or translation in general settings but not designed specifically for the medical setting (eg, Google Translate, apps for tourists).


Development of an electronic tool
  • Themes emerging from interviews or focus groups, results from participants consultations
  • None

  • Comments from participants,, satisfaction surveys, data in app use or consultations
  • None

  • Changes in health outcomes (self-reported or measured with biomedical measures), changes in knowledge, attitudes, practices, and beliefs
  • None

Study Selection

A list of articles retrieved from all 3 journal databases was compiled. After excluding duplicates, 2 researchers (FT and SP) independently reviewed the title and abstracts of all documents for preselection. Articles were then reviewed in full for inclusion or exclusion; if an article was excluded, the reason was documented. We managed the selection of articles with Rayyan [13]. Differences of opinion regarding the inclusion of an article were managed by a third researcher (ORT). We also subsequently included relevant articles that were cited by articles that had been initially selected.

Data Collection

We used 2 data collection coding sheets (Google Forms): one for each article (type of publication, year of publication, and journal) and one for each app studied (general information about the app or electronic tool, information about health literacy and cultural adaptation, information about the development of the app or electronic tool, evidence about the acceptability or efficacy of the app or electronic tool, information about the use of the app or electronic tool). All articles were read by FT, and to improve the validity of results, we performed data triangulation by having a second author read articles independently.

When articles presented additional sources of information regarding an app, such as gray literature or a website, we retrieved data about the app from this source and noted the references of this additional source of information but did not use them in the article selection.

Quality and Risk of Bias Assessment

We evaluated the articles using ICROMS (Integrated Quality Criteria for Review of Multiple Study Designs) [14], which can be used for public health reviews that include several study designs, such as randomized controlled trials, controlled before-after, controlled interrupted times series, cluster randomized controlled trial, noncontrolled before-after, cohort studies and qualitative studies. This tool consists of 33 indicators grouped in 7 dimensions: (1) clear aims and justification; (2) managing bias in sampling or between groups, (3) managing bias in outcome measurement and blinding, (4) managing bias in follow-ups, (5) managing bias in other study aspects; (6) analytical rigor; and (7) managing bias in reporting or ethical considerations. Each indicator receives a score of 2 if the criteria for the indicator are met, 0 if this is not the case, or 1 if it is unknown whether the criteria were met. Where specified, we also found that it was necessary to use CHEERS (Consolidated Health Economic Evaluation Reporting Standards [15]), noting if the information required was available, incomplete, or not available using the same scoring system as ICROMS. It should be noted that this scoring system for protocols or medicoeconomic studies is not validated, nor is it part of the ICROMS tool.

Many articles related to eHealth are information technology usability studies. Usability studies are studies that aim to explore usability requirements, discover usability problems, and design solutions [16]. To our knowledge, there is no published article proposing a tool to evaluate the quality or bias of such studies. Usability studies mainly use either qualitative or quantitative methodologies [16], however, due to the usually small sample of participants [17] and approaches used, such as participants being asked to perform tasks and give oral feedback [18], they are closer to qualitative studies than they are to quantitative studies. Therefore, we assessed the quality and risk of bias of most of usability studies using the ICROMS tool adapted to qualitative studies, except usability studies using quantitative-only methodology, which were assessed as noncontrolled before-after. We also used the ICROMS tool to evaluate research protocols; we simply disregarded the questions that were not applicable.


The database search was carried out in October 2019. We retrieved a total of 15,752 articles from 3 databases after removing duplicates, of which 15,618 were excluded on the basis of title or abstract content. We assessed the full texts of 134 articles and excluded 87, most because the electronic tool described did not meet the inclusion criteria. We subsequently added 14 articles, because they were found either in the original background search or cited by articles that had been initially selected. In total, we included 61 articles (Figure 1). The difference between the very high number of articles originally retrieved and the number of articles selected can be explained by the fact that the original search yielded a number of articles related to translational research. The 61 selected articles were all read and analyzed by FT, and 51 (84%) were read and analyzed by a second reviewer (35 by SP and 16 by AVY).

Figure 1. Flowchart of article selection.
View this figure


The majority of articles were original research articles (55/61, 90%); 4 were published research protocols, and 2 were proceedings of scientific conferences. Of the 61 articles, most articles (27/61, 44%) were published in journals that specialized in medical information technology, 14 (23%) were published in journals related to a disease or medical specialty, 11 (18%) were published in journals that specialized in health promotion, 6 (10%) were published in general health journals, and 3 (5%) were published in nonmedical journals that specialized in information technology. All articles were published in English. The countries of affiliation of the first authors were USA (n=40), Germany (n=5), Australia (n=5), New Zealand (n=2), United Kingdom (n=2), Italy (n=1), Nigeria (n=1), China (n=1), Switzerland (n=1), Japan (n=1), Norway (n=1), and Spain (n=1).

Of the 61 articles, 21 articles reported qualitative studies (34%), 18 articles reported usability studies (30%), 9 articles reported nonrandomized interventional studies (15%), 7 articles reported randomized controlled trials (11%), 4 articles were protocols for either randomized controlled trials (n=3, 5%) or a mixed methods study (n=1, 2%), and 2 articles reported other types of studies (1 medicoeconomic study; 1 case study).

Quality and Risk of Bias

One article was a health economic study, for which CHEERS was used. We could not evaluate the quality of one article [19], as it is was a description of the tool and a single case study of its use on a patient. This article was included in our review because of its practical description of the tool.

The full results are available in Multimedia Appendix 3. Among the 7 randomized controlled trial articles, the mean score was 22 and the median score was 24 (the score ranges from 0 to 30, and the minimum score required for a study to be considered of robust quality is 22); 1 article did not meet the criterion to be considered of robust quality. Among the 21 qualitative studies, the average and median score were 18 and 18, respectively (the score ranges from 0 to 26, and the minimum required for a study to be considered of robust quality is 16); 7 articles did not meet the minimum to be considered of robust quality. Among the 9 noncontrolled before-after studies, the mean score was 22 and the median score was 23 (the score ranges from 0 to 30 and the minimum required for a study to be considered of robust quality is 22); 4 studies did not meet the minimum to be considered of robust quality. Among the 18 usability studies, 6 did not meet the minimum required for either qualitative or noncontrolled before-after studies to be considered of robust quality.

For the randomized controlled trials, the lowest scores were found on questions regarding allocation blinding and measurement blinding, and to a lower extent, reliability of primary outcome measures (lack of objectivity of outcome variables). For the qualitative studies and usability studies, the lowest scores were found for items of critically assessing researcher bias and lack of agreement. For noncontrolled before-after studies, the lowest scores were found regarding justification for and attempts to mitigate the lack of control group and, to a lower extent, lack of objectivity of outcome measures.

General Characteristics of Apps

In the 61 articles, a total of 48 apps were presented (Table 2). Approximately two-thirds of the electronic tools (n=30, 63%) were developed in the USA. Other countries represented were Germany (4 apps), Australia (4 apps); New Zealand (2 apps) and Italy, United Kingdom, Nigeria, China, Switzerland, Japan, Norway, and Spain (1 app each).

Of the 48 apps, 20 apps (42%) were designed for health promotion or prevention, rather than for one specific health care setting: 11 were designed for hospital care, 8 were designed for primary care, 5 were designed for therapeutic or patient education, 3 were designed for both primary care and hospital care, and one was designed for both health promotion and primary care; 14 apps (29%) were health promotion apps that were not related to one specific medical specialty or condition, and the most represented medical specialties were cancer (9/48, 19%), mental health (6/48, 13%). Health promotion is defined as the process of enabling people to increase control over and to improve their health [20].

One-third of apps (16/48, 33%) were designed solely to facilitate interactions between people who have migrated and health care providers during a consultation, while the remaining two-thirds (32/48, 67%) were designed to promote health among people who have migrated who face a language barrier. Of those 32 apps designed to promote health, 9 were adaptations for people who have migrated of existing apps, while 23 were new apps developed specifically for people who have migrated.

Most electronic tools (38/48, 79%) were in the form of a mobile app, while other types (text messaging, website) were less common. It should be noted that 2 electronic tools (2/48, 4%) included both a mobile app and a text messaging service. Almost two-thirds of apps (31/48, 65%) were interactive, meaning that they allowed feedback from the user, either to another user or to the app.

More than three-quarters (38/48, 79%) were specifically targeted to a group of people who have migrated, and mostly for people who have migrated from specific nationalities (24/38, 63%). Over two-thirds (33/48; 69%) had 1 language in addition to the source language, nine apps had 3 to 9 languages, five had 10 to 19 languages, and one app had over 20 languages. Information about the funding of the app was available for 33 (69%) out of the 48 apps: 30 apps (63%) had received a funding from either a public source, charitable source or crowd-funding, while 2 apps (4%) had received a funding from a mix of private and public or charitable source, and 1 app (2%) had received a private or industry funding.

Table 2. Characteristics of apps included in the analysis.
CharacteristicApps (n=48), n (%)
Country of development

USA30 (63)

Germany4 (8)

Australia4 (8)

New Zealand2 (4)

Italy1 (2)

United Kingdom1 (2)

Nigeria1 (2)

China1 (2)

Switzerland1 (2)

Japan1 (2)

Norway1 (2)

Spain1 (2)

Health promotion/prevention20 (42)

Hospital care11 (23)

Primary care8 (17)

Therapeutic or patient education5 (10)

Primary care and hospital care3 (6)

Health promotion/prevention and primary care1 (2)
Medical specialty

Health promotion without a focus on a medical specialty14 (29)

Cancer9 (19)

Mental health, psychiatry6 (13)

Infectious diseases5 (10)

Cardiovascular diseases, endocrinology (diabetes)3 (6)

Gynecology, pregnancy3 (6)

Emergency medicine, intensive care2 (4)

Addiction medicine2 (4)

Paramedical specialties (physiotherapy, occupational therapy, speech pathology, dietetic, podiatrists)2 (4)

Pediatrics1 (2)

Pulmonology1 (2)

Facilitating communication between migrant and health provider16 (33)

Promoting healthy behavior among people who have migrateda32 (67)

Including adaptation of existing apps9

Mobile app38 (79)

Text-messaging service6 (13)

Mobile app and text-messaging service2 (4)

Website for consultation2 (4)

Interactive31 (65)

Not interactive17 (35)
Target population

People who have migrated from specific nationalities24 (50)

Asylum seekers/refugees4 (8)

People who have migrated at risk from their occupation3 (6)

Indigenous people3 (6)

People who have migrated who are concerned by a specific health condition3 (6)

Other1 (2)

The app does not specifically target a group of people who have migrated10 (21)
Languages (in addition to the source language)

133 (69)

2-99 (19)

10-195 (10)

≥201 (2)
Fee required

Yes5 (10)

No9 (19)

Don\'t know34 (71)
Institution funding the development

Public/government11 (23)

Charitable or crowdfunding3 (6)

Mix of charitable and public/government2 (4)

Public/government, private and charitable1 (2)

Private/industry only15 (31)

No information about the funding of the app16 (33)

aOf the 32, 9 were adaptations of existing apps.

Health Literacy and Cultural Adaptation

There was information about how the translation was performed for only half of the apps (n=24); translation was performed by a professional translator (16/48, 33%); informally (4/48, 8%), usually by nonprofessional native speakers; or by a mix of professional and personal translation (4/48, 8%). Some form of quality control of the app was mentioned for 18 apps (38%), usually forward-backward translation or translation being checked by a native speaker. The translation included cultural adaptation on 23 apps (48%), half of the apps included pictures or pictograms (n=24). Two-thirds of apps had either an audio or video feature (32/48, 67%).

Information About the Development of the App or Electronic Tool

Of the 48 apps, 33 (69%) reported using scientific methods to develop the content of the app: qualitative studies (interviews or focus groups) for 15 (45%), use of a theoretical framework such as behavioral theories for 8 (24%), use of existing guidelines of curriculums (n=5, 15%), a mix of qualitative and quantitative methods for 4 (12%), and a mix of survey and use of theoretical framework for 1 (3%).

Of the 48 apps, less than half (22/48, 46%) reported involving users (people who have migrated) in the development, 18 apps (38%) reported involving health professionals in the development, and 5 apps (10%) reported involving other stakeholders such as charities.

Evidence About the Acceptability and Efficacy of the App or Electronic Tool

Many apps (32/48, 67%) had their acceptability evaluated (translation apps: 14/16, 88%; health promotion apps: 18/32, 56%). Of the 14 translation apps that had their acceptability evaluated, 3 were evaluated among people who had migrated only, 6 were evaluated among people who had migrated and health professionals, and 5 were evaluated among health professionals alone. Of the 18 apps designed to promote health among people who had migrated that have been evaluated for acceptability, 14 were evaluated among people who had migrated alone, 3 were evaluated among people who had migrated and health professionals, and 1 was evaluated among health professionals alone. Acceptability studies used mixed methods (20/32, 63%), used quantitative methods only (8/32, 25%), used qualitative methods only (7/32, 22%), were pilots (5/32, 16%), and randomized controlled trials (2/32, 6%). The endpoints used to measure acceptability were comments from participants (12/32, 38%), results from satisfaction survey (9/32, 28%), length of time of a consultation and satisfaction survey (4/32, 13%), data on the use of the app (3/32, 9%), a mix of survey and qualitative comments (2/32, 6%), and other outcomes (2/32, 6%). The Systems Usability Scale was mentioned in 5 evaluations. Other evaluation systems mentioned were the Technology Acceptance Model and the Stanford Communication with Physicians Scale. Among the 32 apps that had their acceptability evaluated, 25 (78%) reported an overall good or very good acceptability; 1 (3%) reported an adequate acceptability; for 3 apps (9%), the study was ongoing; and for 3 others, the results cannot be reported as they consisted of comments from participants or choice of a design. Of the 25 apps that reported a good or very acceptability, 8 had been developed involving users in the process.

Half of the apps (n=24) had their efficacy evaluated (translation apps: 2/16, 13%; health promotion apps: 22/32, 69%). Study designs were randomized controlled trials (9/24, 38%), nonrandomized trials (7/24, 29%), survey (3/24, 13%), qualitative (2/24, 8%), mixed methods (2/24, 8%), and economic analysis (1/24, 4%). The endpoints or outcomes used for those evaluations were reported behavior change (10/24, 42%), knowledge improvement (7/24, 29%), self-reported health markers, such as improved quality of life, better sleep, less anxiety (4/24, 17%), biometric health markers (2/24, 8%), cost-effectiveness (2/24, 8%), and accuracy of medical information (n=2, 8%). The total exceeds 100% as 11 apps used several different endpoints. Among the 24 apps that had their efficacy evaluated, 12 (50%) had significant positive results; 5 (12%) had partially positive results, meaning that the app showed significant efficacy in some measured outcomes but not all, or was effective in some population and not all; and 2 apps had nonsignificant results (8%); 5 studies were ongoing (21%). Details of the efficacy and acceptability studies are in Multimedia Appendix 4.

Positive outcomes (Table 3) reported for translation apps includes reducing the need to call an interpreter, especially in emergency situations [21], reduced consultation time [22,23], and reduced patient anxiety [24]. Negatives that were reported included limitations in the dialogue between health professionals and patients [25-27] and concerns about hindering the therapeutic relationship [28]. Health promotion apps had positive results in terms of acceptability and efficacy. Positive outcomes included improved quality of life and better management of chronic illnesses such as diabetes [29-31], cancer [32-34], HIV [35], depression [19,36], and addiction [37,38].

Table 3. Characteristics of health apps linked to better acceptability or efficacy.
App typeImportant points noted during the developmentCharacteristics linked to better acceptability or efficacy
TranslationMany experts recommend that culturally tailored materials be created de novo or in tandem, rather than as variations on existing materials
  • Speech is generally preferred to text
  • Including a button, equivalent to the patient’s “I do not understand the question”
  • Including a phrase for health care practitioners “I don’t understand your answer”
  • Integrating an option to directly call an interpreter in the app
  • Integrating a list of nearby hospitals for follow-up care
  • Including the option for patients to respond with pictures
  • Including the option for health care practitioners to save the conversation with a patient (with respect to data protection and confidentiality)
Health promotionAddressing both motivation as well as linguistic and sociocultural barriers and reassuring participants of confidentiality
  • Apps that personalize the experience for users are preferred
  • Including a help function and a tutorial
  • Include a tutorial provided by a virtual human rather than text
  • Culturally appropriate, with photos of a multigenerational family
  • Colorful and eye catching, but also professional, with easy-to-access information
  • Easy to navigate with simple and easy-to-understand information
  • Interactive with immediate feedback
  • Audio, videos and pictures
  • Provision of links for further information about a health issue
  • Use of humor considered very effective by target audiences
  • Including a list of frequently asked questions for users
  • Including the option for people who have migrated to learn medical terms in the language of the country they live in

Main Results

We synthesized evidence regarding the development, acceptability, and efficacy of health apps and electronic tools created to overcome the language barriers. Acceptability was evaluated in almost two-thirds of the apps and was generally high. Although health promotion or prevention programs specifically targeting people who have migrated might be complex to integrate into a health system, they are generally well accepted [39] and are a way to elevate health for all individuals.

Efficacy evaluations were only conducted for half of the apps. In those evaluations, the endpoints were mostly related to reported behavior change and knowledge improvement. Knowledge improvement, however, does not systematically lead to behavior change and the reported behavior change may not be long lasting. Changes in health outcomes are rarely measured in health promotion programs or health promotion research; this should be included as a goal of health communication tools. Indeed, a survey of researchers in health promotion highlighted that the majority of assessment measures changes in awareness, knowledge, skills, policy changes, changes in behavior, changes in community capacity, changes in organizational capacity, but changes in health outcomes are not cited [40]. In addition, several systematic reviews [41-43] examining the literature related to health promotion programs have shown that programs are rarely evaluated in terms of health outcomes. It seems, therefore, that assessments of health promotion apps are in line with current practices for assessments of health promotion programs, which focus on knowledge and reported practices but rarely on final health outcomes. Given the difficulty of measuring the health outcomes of health promotion programs or apps, it is necessary to develop new methods.

Although we could only retrieve information about the funding of 69% of apps (33/48), we found that the majority had either public or charitable funding, while only 2 apps received industry funding. Additionally, only 5 apps reported charging users a fee. That seems to suggest that both health promotion and medical translation apps mostly have a noncommercial purpose and were designed with a public health goal.

Only half of the apps had their efficacy evaluated, of which half had a significant positive result. Efficacy was evaluated more often for health promotion apps than for medical translation apps. As we have mentioned, communication difficulties between health providers and individuals with language barriers can have several negative consequences, such as less satisfaction with care [7], longer consultation time [44], lower adherence to treatment protocols [45], less health education messages delivered [9], and worse clinical outcomes [8]; therefore, it is of tremendous importance that medical translation apps are rigorously assessed, not only for their acceptability but also for their efficacy.

Comparison With Literature

Several literature reviews [46-48] have been conducted of health apps and multimedia-based health promotion programs; however, none has specifically examined apps that are focused specifically on language barriers. Two reviewed the use of mobile health technology use and implications in historically underserved and minority populations in the United States [46] and mobile health interventions to promote physical activity for Black and Hispanic women [47] but scarcely addressed language barriers. We found one systematic review evaluating consumer health information technology interventions toward US Spanish-speaking populations [48]. The study [48] focused on one specific population in a specific country (USA), for a very wide scope of electronic interventions (eg, radio, videos, text messages services) that is different from the scope of our article. In that systematic review, the most commonly used evaluation metrics were behavior, attitude change, usability, and knowledge retention. The results of the study [48] and of our own were similar.

Strengths and Limitations

We conducted an extensive search of the literature using 3 different databases of publications. The selection of articles was conducted by two researchers working independently; results were compared and differences were settled by a third party. Most articles were read and their data extracted by two researchers independently, and results were compared when analyzing data. This enabled us to have a higher quality of data and a reduced risk of bias. We extended the scope of our systematic review to both electronic tools designed for translation and those designed to promote health among people who have migrated with a language barrier. Although the electronic tools have seemingly different objectives, it is likely that, in the future, there will be hybrid apps that will be developed to integrate both objectives. Indeed, many primary care consultations include health education and advice from a health provider, and hospitals have been advised to include health promotion activities into their activities [49]. Development of new hybrid apps that includes both objectives will be able to learn from the evidence from the both types of apps we examined.

In this systematic review, we only included apps that were referred to in a scientific journal. Our analysis did not include the plethora of apps designed to promote heath or facilitate consultation for people who have migrated with a language barrier that were not the topic of published peer-reviewed articles. Since the objective of our review was to evaluate the evidence related to the development or evaluation of these types of apps, it was not relevant to include these other sources. In our review, we did not assess the technical characteristics of the apps studied. Indeed most articles included in this review did not give information about the technical characteristics of the apps. Many scales and evaluations systems have been created to that end [50-53]. Assessing the technical characteristics is time-consuming and not always possible as many of those apps are not available for public use. We suggest that when new apps are developed, they strive to achieve the technical qualities measured by such scales. Other systematic reviews [54] have examined an extensive range of apps designed to help communication between health care providers and people who have migrated beyond those published in peer-reviewed articles, which focus on technical characteristics, and which provide different types of information that are complementary to that provided herein.

Implication for Policy and Conclusion

As previously mentioned, people who do not speak the language of the country have poorer access to care, longer and less satisfactory consultations, and worse clinical outcomes. In this review, we found that translation apps showed good user satisfaction but had less data on changes in the process of care (consultation length, renouncing medical care) and no data on possible changes in clinical outcomes or medicoeconomic benefits. The evaluations of health promotion apps had positive results in terms of acceptability and efficacy; however the trials on efficacy mostly used self-reported outcomes, such as self-reported behavior changes or quality of life, rather than clinical outcomes. Most trials lacked randomization or control groups, blinding, or objective measures. This lower quality makes it difficult to draw clear conclusions on efficacy.

Future apps that are developed should include evaluation of clinical and possibly medicoeconomic benefits to draw clear recommendations on their use. The apps that were the most acceptable were those that integrated features beyond simple translation, such as making appointments with health professionals on the platform or entering basic information to prepare a visit. We recommend that future translation apps are created for medical visits integrate such. In both translation and health promotion apps, including audio and video features was most appreciated by users. We recommend integrating such features in the development of new apps.

In the future, more and more health apps will be created. Given the high cost of development [55], it is essential that apps that claim to have a public health objective undergo a rigorous evaluation of their acceptability, and efficacy. Future studies should also use strong epidemiological indicators as outcomes, such as changes in health status or access to care, rather than only using reported changes in behavior and knowledge.

The aim of this systematic review was to examine the evidence related to the development, adaptation, acceptability, and effectiveness of electronic tools designed to help health care providers communicate with or promote health among people who have migrated having a low proficiency in the language of the country of origin or a low level of health literacy. Our results, especially development and characteristics associated with a better acceptability and efficacy, should be of help to public health professionals who develop new apps.


We sincerely thank Corinne H Miller, Clinical Informationist at Galter Health Sciences Library & Learning Center of Northwestern University, Chicago, USA, for her help in the developing the search strategy and keywords.

The study was funded by the Agence Nationale de Recherches sur le SIDA et les Hépatites virales (grant ECTZ109287) and by Office Français de l’Immigration et de l’Intégration (20191104122918510), whom we thank for their support. The funders had no role in study design, data collection or analysis, manuscript writing, or decision to publish. JWG was supported in part by the National Institutes of Health (R01MD010440).

Authors' Contributions

FT, ORT, MD, SP, JWG, and OC designed the study. FT and SP developed the search and selection strategy, developed the data extraction form, and collected data. AVY collected data. FT, ORT, MD, and OC performed critical evaluation and discussion of results. FT drafted the manuscript. All authors reviewed the manuscript.

Conflicts of Interest

None declared.

Multimedia Appendix 1

PRISMA reporting checklist completed.

DOCX File , 20 KB

Multimedia Appendix 2

Keywords used for the database search.

DOCX File , 15 KB

Multimedia Appendix 3

Studies quality and risk assessment.

DOCX File , 64 KB

Multimedia Appendix 4

Summary of apps studied in this systematic review.

DOCX File , 71 KB

  1. Hoang H, Le Q, Kilpatrick S. Having a baby in the new land: a qualitative exploration of the experiences of Asian migrants in rural Tasmania, Australia. Rural Remote Health 2009 Mar 31;9(1):1-11. [CrossRef]
  2. van Eijsden M, van der Wal MF, Bonsel GJ. Folic acid knowledge and use in a multi-ethnic pregnancy cohort: the role of language proficiency. BJOG 2006 Dec;113(12):1446-1451. [CrossRef] [Medline]
  3. Sentell T, Chang A, Ahn HJ, Miyamura J. Maternal language and adverse birth outcomes in a statewide analysis. Women Health 2016 Sep 11;56(3):257-280 [FREE Full text] [CrossRef] [Medline]
  4. Cohen AL, Rivara F, Marcuse EK, McPhillips H, Davis R. Are language barriers associated with serious medical events in hospitalized pediatric patients? Pediatrics 2005 Sep;116(3):575-579. [CrossRef] [Medline]
  5. Sentell T, Braun KL, Davis J, Davis T. Colorectal cancer screening: low health literacy and limited English proficiency among Asians and Whites in California. J Health Commun 2013;18 Suppl 1:242-255 [FREE Full text] [CrossRef] [Medline]
  6. Thompson CA, Gomez SL, Chan A, Chan JK, McClellan SR, Chung S, et al. Patient and provider characteristics associated with colorectal, breast, and cervical cancer screening among Asian Americans. Cancer Epidemiol Biomarkers Prev 2014 Nov 02;23(11):2208-2217. [CrossRef]
  7. Flores G. The impact of medical interpreter services on the quality of health care: a systematic review. Med Care Res Rev 2005 Jun 18;62(3):255-299. [CrossRef] [Medline]
  8. Karliner LS, Jacobs EA, Chen AH, Mutha S. Do professional interpreters improve clinical care for patients with limited English proficiency? a systematic review of the literature. Health Serv Res 2007 Apr;42(2):727-754 [FREE Full text] [CrossRef] [Medline]
  9. Jacobs EA, Shepard DS, Suaya JA, Stone E. Overcoming language barriers in health care: costs and benefits of interpreter services. Am J Public Health 2004 May;94(5):866-869. [CrossRef] [Medline]
  10. Sweileh WM, Al-Jabi SW, AbuTaha AS, Zyoud SH, Anayah FMA, Sawalha AF. Bibliometric analysis of worldwide scientific literature in mobile - health: 2006-2016. BMC Med Inform Decis Mak 2017 May 30;17(1):72 [FREE Full text] [CrossRef] [Medline]
  11. Thonon F, Fahmi S, Rousset-Torrente O, Bessonneau P, Griffith J, Brown C, et al. Promoting HIV, Hepatitis B Virus, and Hepatitis C Virus Screening Among Migrants With a Language Barrier: Protocol for the Development and Evaluation of an Electronic App (Apidé). JMIR Res Protoc 2021 May 5;10(5):e22239 [FREE Full text] [CrossRef] [Medline]
  12. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015 Jan 01;4:1 [FREE Full text] [CrossRef] [Medline]
  13. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev 2016 Dec 05;5(1):210 [FREE Full text] [CrossRef] [Medline]
  14. Zingg W, Castro-Sanchez E, Secci F, Edwards R, Drumright L, Sevdalis N, et al. Innovative tools for quality assessment: integrated quality criteria for review of multiple study designs (ICROMS). Public Health 2016 Apr;133:19-37. [CrossRef]
  15. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, ISPOR Health Economic Evaluation Publication Guidelines-CHEERS Good Reporting Practices Task Force. Consolidated Health Economic Evaluation Reporting Standards (CHEERS)--explanation and elaboration: a report of the ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force. Value Health 2013 Mar;16(2):231-250 [FREE Full text] [CrossRef] [Medline]
  16. Yen P, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc 2012 May;19(3):413-422 [FREE Full text] [CrossRef] [Medline]
  17. Lewis JR. Sample sizes for usability studies: additional considerations. Hum Factors 1994 Jun;36(2):368-378. [CrossRef] [Medline]
  18. Nielsen J, Clemmensen T, Yssing C. Getting access to what goes on in people?s heads? reflections on the think-aloud technique. In: Proceedings of the second Nordic conference on Human-computer interaction. 2002 Presented at: Second Nordic conference on Human-Computer Interaction; October 19-23; Aarhus, Denmark p. 101. [CrossRef]
  19. Dahne J. ¿Aptívate!: a Spanish-language behavioral activation mobile app for delivery via primary care. Psychol Serv 2019;16(2):275.
  20. Ottawa Charter for Health Promotion. World Health Organization. 1986.   URL:,-1986 [accessed 1986-11-17]
  21. Day KJ, Song N. Attitudes and concerns of doctors and nurses about using a translation application for in-hospital brief interactions with Korean patients. J Innov Health Inform 2017 Oct 06;24(3):916 [FREE Full text] [CrossRef] [Medline]
  22. Silvera-Tawil D, Pocock C, Bradford D, Donnell A, Harrap K, Freyne J, et al. CALD Assist-Nursing: improving communication in the absence of interpreters. J Clin Nurs 2018 Nov 30;27(21-22):4168-4178. [CrossRef] [Medline]
  23. Freyne J, Bradford D, Pocock C, Silvera-Tawil D, Harrap K, Brinkmann S. Developing digital facilitation of assessments in the absence of an interpreter: participatory design and feasibility evaluation with allied health groups. JMIR Form Res 2018 Jan 09;2(1):e1 [FREE Full text] [CrossRef] [Medline]
  24. Taicher BM, Alam RI, Berman J, Epstein RH. Design, implementation, and evaluation of a computerized system to communicate with patients with limited native language proficiency in the perioperative period. Anesth Analg 2011;112(1):106-112. [CrossRef]
  25. Somers HL, Caress A, Evans DG, Johnson MJ, Lovel HJ, Mohamed Z. A computer-based aid for communication between patients with limited English and their clinicians, using symbols and digitised speech. Int J Med Inform 2008 Aug;77(8):507-517. [CrossRef] [Medline]
  26. Spechbach H, Gerlach J, Mazouri Karker S, Tsourakis N, Combescure C, Bouillon P. A speech-enabled fixed-phrase translator for emergency settings: crossover study. JMIR Med Inform 2019 May 07;7(2):e13167 [FREE Full text] [CrossRef] [Medline]
  27. Johnson M, Evans G, Mohamed Z, Caress A. The development and evaluation of alternative communication strategies to facilitate interactions with Somali refugees in primary care: a preliminary study. Inform Prim Care 2006 Sep 01;14(3):183-189 [FREE Full text] [CrossRef] [Medline]
  28. Villalobos O, Lynch S, DeBlieck C, Summers L. Utilization of a mobile app to assess psychiatric patients with limited English proficiency. Hisp J Behav Sci 2017 May 09;39(3):369-380. [CrossRef]
  29. Arora S, Burner E, Lam J, De Santos R, Menchine M. Assessing the satisfaction of mobile health (mHealth) amongst ED inner-city patients with diabetes who received the TExT-MED intervention. Acad Emerg Med 2013;20(5(Suppl. 1)):S181.
  30. Burner ER, Menchine MD, Kubicek K, Robles M, Arora S. Perceptions of successful cues to action and opportunities to augment behavioral triggers in diabetes self-management: qualitative analysis of a mobile intervention for low-income Latinos with diabetes. J Med Internet Res 2014 Jan 29;16(1):e25 [FREE Full text] [CrossRef] [Medline]
  31. Burner E, Menchine M, Taylor E, Arora S. Gender differences in diabetes self-management: a mixed-methods analysis of a mobile health intervention for inner-city Latino patients. J Diabetes Sci Technol 2013 Jan 01;7(1):111-118 [FREE Full text] [CrossRef] [Medline]
  32. Nápoles AM, Santoyo-Olsson J, Chacón L, Stewart AL, Dixit N, Ortiz C. Feasibility of a mobile phone app and telephone coaching survivorship care planning program among Spanish-speaking breast cancer survivors. JMIR Cancer 2019 Jul 09;5(2):e13543 [FREE Full text] [CrossRef] [Medline]
  33. Im EO, Ji X, Zhang J, Kim S, Lee Y, Chee E, et al. Issues in developing and evaluating a culturally tailored internet cancer support group. Comput Inform Nurs 2016;34(10):469. [CrossRef]
  34. Chee W, Lee Y, Im E, Chee E, Tsai H, Nishigaki M, et al. A culturally tailored Internet cancer support group for Asian American breast cancer survivors: a randomized controlled pilot intervention study. J Telemed Telecare 2017 Jul;23(6):618-626 [FREE Full text] [CrossRef] [Medline]
  35. Kurth AE, Chhun N, Cleland CM, Crespo-Fierro M, Parés-Avila JA, Lizcano JA, et al. Linguistic and cultural adaptation of a computer-based counseling program (CARE+ Spanish) to support HIV treatment adherence and risk reduction for people living with HIV/AIDS: a randomized controlled trial. J Med Internet Res 2016 Jul 13;18(7):e195 [FREE Full text] [CrossRef] [Medline]
  36. Dahne J, Collado A, Lejuez C, Risco CM, Diaz VA, Coles L, et al. Pilot randomized controlled trial of a Spanish-language behavioral activation mobile app (¡Aptívate!) for the treatment of depressive symptoms among united states Latinx adults with limited English proficiency. J Affect Disord 2019 May 01;250:210-217 [FREE Full text] [CrossRef] [Medline]
  37. Muroff J, Robinson W, Chassler D, López LM, Lundgren L, Guauque C, et al. An outcome study of the CASA-CHESS smartphone relapse prevention tool for Latinx Spanish-speakers with substance use disorders. Subst Use Misuse 2019 Mar 31;54(9):1438-1449. [CrossRef] [Medline]
  38. Muroff J, Robinson W, Chassler D, López LM, Gaitan E, Lundgren L, et al. Use of a smartphone recovery tool for Latinos with co-occurring alcohol and other drug disorders and mental disorders. J Dual Diagn 2017 Jul 10;13(4):280-290. [CrossRef]
  39. Duracinsky M, Thonon F, Bun S, Ben Nasr I, Dara AF, Lakhdari S, et al. Good acceptability of HIV, HBV, and HCV screening during immigration medical check-up amongst migrants in France in the STRADA study. PLoS ONE 2020 Jun 26;15(6):e0235260. [CrossRef]
  40. Jancey J, Burns S, Hendriks J, Pollard CM, Tohatoa J, Hallett J. Measuring health promotion research impact - what researchers think? Health Promot J Austr 2020 Jun 26:1-7. [CrossRef] [Medline]
  41. Torloni MR, Brizuela V, Betran AP. Mass media campaigns to reduce unnecessary caesarean sections: a systematic review. BMJ Glob Health 2020 Feb 26;5(2):e001935 [FREE Full text] [CrossRef] [Medline]
  42. García-Álvarez D, Faubel R. Strategies and measurement tools in physical activity promotion interventions in the university setting: a systematic review. Int J Environ Res Public Health 2020 Sep 08;17(18):6526 [FREE Full text] [CrossRef] [Medline]
  43. Charles Shapu R, Ismail S, Ahmad N, Lim PY, Abubakar Njodi I. Systematic review: effect of health education intervention on improving knowledge, attitudes and practices of adolescents on malnutrition. Nutrients 2020 Aug 13;12(8):2426 [FREE Full text] [CrossRef] [Medline]
  44. Fagan MJ, Diaz JA, Reinert SE, Sciamanna CN, Fagan DM. Impact of interpretation method on clinic visit length. J Gen Intern Med 2003 Aug;18(8):634-638 [FREE Full text] [CrossRef] [Medline]
  45. Tanser F, Bärnighausen T, Vandormael A, Dobra A. HIV treatment cascade in migrants and mobile populations. Curr Opin HIV AIDS 2015 Nov;10(6):430-438. [CrossRef] [Medline]
  46. Anderson-Lewis C, Darville G, Mercado RE, Howell S, Di Maggio S. mHealthtechnology use and implications in historically underserved and minority populations in the United States: systematic literature review. JMIR Mhealth Uhealth 2018 Jun 18;6(6):e128 [FREE Full text] [CrossRef] [Medline]
  47. Joseph RP, Royse KE, Benitez TJ. Systematic review of electronic and mobile health (e- and mHealth) physical activity interventions for African American and Hispanic women. J Phys Act Health 2019 Mar 01;16(3):230-239. [CrossRef] [Medline]
  48. Chaet AV, Morshedi B, Wells KJ, Barnes LE, Valdez R. Spanish-language consumer health information technology interventions: a systematic review. J Med Internet Res 2016 Aug 10;18(8):e214 [FREE Full text] [CrossRef] [Medline]
  49. Pelikan JM, Krajic K, Dietscher C. The health promoting hospital (HPH): concept and development. Patient Educ Couns 2001 Dec;45(4):239-243. [CrossRef]
  50. Brender J, Talmon J, de Keizer N, Nykänen P, Rigby M, Ammenwerth E. STARE-HI - Statement on Reporting of Evaluation Studies in Health Informatics: explanation and elaboration. Appl Clin Inform 2013;4(3):331-358 [FREE Full text] [CrossRef] [Medline]
  51. Robustillo Cortés MDLA, Cantudo Cuenca MR, Morillo Verdugo R, Calvo Cidoncha E. High quantity but limited quality in healthcare applications intended for HIV-infected patients. Telemed J E Health 2014 Aug;20(8):729-735. [CrossRef] [Medline]
  52. Nykänen P, Kaipio J. Quality of health IT evaluations. Stud Health Technol Inform 2016;222:303. [CrossRef]
  53. Yasini M, Beranger J, Desmarais P, Perez L, Marchand G. mHealth quality: a process to seal the qualified mobile health apps. Stud Health Technol Inform 2016;228:209.
  54. Khander A, Farag S, Chen KT. Identification and evaluation of medical translator mobile applications using an adapted applications scoring system. Telemed J E Health 2018 Aug;24(8):594-603. [CrossRef] [Medline]
  55. Mobile health app development costs $425,000 on average, likely continuing to rise. MobiHealthNews. 2018 Mar 26.   URL: https:/​/www.​​content/​mobile-health-app-development-costs-425000-average-likely-continuing-rise [accessed 2021-04-12]

CHEERS: Consolidated Health Economic Evaluation Reporting Standards
ICROMS: Integrated Quality Criteria for Review of Multiple Study Designs
HIV: human immunodeficiency virus
PRISMA: Preferred Reporting Items for Systematic Review and Meta-analysis

Edited by R Kukafka; submitted 03.11.20; peer-reviewed by E Neter, K Schroeder; comments to author 02.12.20; revised version received 05.03.21; accepted 16.03.21; published 06.05.21


©Frédérique Thonon, Swati Perrot, Abhijna Vithal Yergolkar, Olivia Rousset-Torrente, James W Griffith, Olivier Chassany, Martin Duracinsky. Originally published in the Journal of Medical Internet Research (, 06.05.2021.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.