This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
With their increasing availability in resource-limited settings, mobile phones may provide an important tool for participatory syndromic surveillance, in which users provide symptom data directly into a centralized database.
We studied the performance of a mobile phone app-based participatory syndromic surveillance system for collecting syndromic data (acute febrile illness and acute gastroenteritis) to detect dengue virus and norovirus on a cohort of children living in a low-resource and rural area of Guatemala.
Randomized households were provided with a mobile phone and asked to submit weekly reports using a symptom diary app (Vigilant-e). Participants reporting acute febrile illness or acute gastroenteritis answered additional questions using a decision-tree algorithm and were subsequently visited at home by a study nurse who performed a second interview and collected samples for dengue virus if confirmed acute febrile illness and norovirus if acute gastroenteritis. We analyzed risk factors associated with decreased self-reporting of syndromic data using the Vigilant-e app and evaluated strategies to improve self-reporting. We also assessed agreement between self-report and nurse-collected data obtained during home visits.
From April 2015 to June 2016, 469 children in 207 households provided 471 person-years of observation. Mean weekly symptom reporting rate was 78% (range 58%-89%). Households with a poor (<70%) weekly reporting rate using the Vigilant-e app during the first 25 weeks of observation (n=57) had a greater number of children (mean 2.8, SD 1.5 vs mean 2.5, SD 1.3; risk ratio [RR] 1.2, 95% CI 1.1-1.4), were less likely to have used mobile phones for text messaging at study enrollment (61%, 35/57 vs 76.7%, 115/150; RR 0.6, 95% CI 0.4-0.9), and were less likely to access care at the local public clinic (35%, 20/57 vs 67.3%, 101/150; RR 0.4, 95% CI 0.2-0.6). Parents of female enrolled participants were more likely to have low response rate (57.1%, 84/147 vs 43.8%, 141/322; RR 1.4, 95% CI 1.1-1.9). Several external factors (cellular tower collapse, contentious elections) were associated with periods of decreased reporting. Poor response rate (<70%) was associated with lower case reporting of acute gastroenteritis, norovirus-associated acute gastroenteritis, acute febrile illness, and dengue virus-associated acute febrile illness (
In a resource-limited area of rural Guatemala, a mobile phone app-based participatory syndromic surveillance system demonstrated a high reporting rate and good agreement between parental reported data and nurse-reported data during home visits. Several household-level and external factors were associated with decreased syndromic reporting. Poor reporting rate was associated with decreased syndromic and pathogen-specific case ascertainment.
Given numerous endemic and emerging infectious diseases, there is a need for improved surveillance capacity in low- and middle-income countries (LMICs). Mobile health (mHealth) systems, which take advantage of the increasing availability of mobile communication technology, offer a potential cost-effective tool to improve surveillance capacity in LMICs by obtaining real-time reporting directly from users.
Health care workers (HCWs) have successfully used mHealth platforms in LMICs for disease surveillance, transmission mapping of pathogens, and decision support [
An alternative approach to centralized surveillance is to “task shift” data collection from HCWs and health centers to the health care consumer. Participatory syndromic surveillance systems use mHealth tools for community members to directly self-report symptoms indicative of a particular disease [
To better understand the utility of participatory syndromic surveillance for emerging disease surveillance in LMICs, we studied the performance and acceptability of a mobile phone app-based participatory syndromic surveillance system in detecting acute febrile illness and acute gastroenteritis among a randomized cohort of children in a low-resource region of Guatemala with limited access to Internet and telecommunication.
The study was conducted in 25 communities within a 200 km2 catchment area along the coastal lowlands of southwest Guatemala. The populations living in these communities suffer from high levels of food insecurity, poverty, low access to health care, and high levels of diarrheal and respiratory disease [
An encrypted Android mobile phone app (Vigilant-e), developed by Integra IT (Bogota, Colombia), was used. The app allows study participants to directly enter and report symptoms or events using simplified question algorithms and decision-tree logic (see
The Vigilant-e app was configured with the participation of the study investigators, the study nurses, community members, and the Integra IT team. Given low education rates and mobile phone use in the region, the user interface was simplified to a minimum number of questions with simple language and visual aids when possible. Study personnel were trained and the app was field-tested to determine if mobile data coverage was acceptable. Although Internet coverage was variable, the Vigilant-e app was able to store data locally until data coverage was acquired, at which point the data were automatically uploaded into the study database. To avoid unnecessary consumption of available data from participants, all apps on their mobile phones were blocked except for Vigilant-e and WhatsApp as well as phone and text messaging apps, so participants could communicate with study nurses when needed.
Case definitions were created prior to study initiation. Acute gastroenteritis was defined as self-reported vomiting or diarrhea for at least 3 days or both for 1 day or more in the preceding week. Norovirus-associated acute gastroenteritis was defined as acute gastroenteritis with concurrent positive norovirus reverse transcription polymerase chain reaction (RT-PCR) testing at the time of sampling. Acute febrile illness was defined as self-reported fever for at least 2 days within the preceding week. Dengue fever was defined as acute febrile illness with positive dengue virus RT-PCR test or anti-DENV IgM serologic test at the time of sampling. If a child reported acute gastroenteritis or acute febrile illness for more than 2 weeks in a row, only the clinical data and sample from the first week were included.
All households were screened and enrolled using a two-stage cluster sampling strategy adapted from the World Health Organization lot quality assurance method, in which 30 clusters of 7 households were enrolled at random within the study catchment area, as previously described [
Beginning on October 2, 2015, if a participant did not submit the symptom diary in a given week, study nurses were allowed to phone the household and manually enter weekly symptom diary data into the study database instead of relying entirely on the Vigilant-e app. Beginning in early April 2016, all participating households were visited by a study nurse, any malfunctioning phones were replaced, and participants were reminded to submit weekly symptom data using the app. Prospective surveillance using the weekly symptom diary was continued through June 2016, at which point a final closeout visit was performed.
Serum samples from participants with acute febrile illness during prospective follow-up were tested for dengue virus by RT-PCR using the Centers for Disease Control and Prevention’s dengue virus assay (DENV-1-4) and IgM anti-DENV IgM enzyme-linked immunosorbent assay (ELISA, InBios Inc, Seattle, WA, USA). Stool specimens were collected using Copan FLOQSwabs (Brescia, Italy) either by rectal swab or on fresh (<2 hours old) stool sample and eluted in eNAT transport solution (Copan, Brescia, Italy) before testing, with both collection techniques previously demonstrating similar molecular viral yield [
Demographic variables were compared between the households with 70% or greater response rate and those with less than 70% response rate using a generalized linear model with at binomial response distribution and a log link function. The 70% cutoff was chosen based on response rates observed in previous participatory syndromic surveillance studies in non-LMICs [
The study was approved by the Colorado Multiple Institutional Review Board, the UVG Institutional Review Board, and the Guatemala Ministry of Health National Ethics Committee. The local Southwest Trifinio Community Advisory Board for Research agreed to the study.
The study enrolled 207 of 444 (46.6%) eligible households from April to September 2015, which included 469 children (
Study design and CONSORT diagram of study recruitment, enrollment, and completion. The participatory syndromic surveillance (PSS) cohort enrolled children from April to September 2015, followed by prospective observation with the weekly symptom diary for acute gastroenteritis and acute febrile illness episodes (dotted box) through June 2016.
Cellular phone ownership among households at enrollment varied with 12.1% (25/207) reporting no phone in the household at study initiation and 39.6% (82/207) reporting the use of a mobile phone. Of those households with cellular phones, 72.5% (150/207) used them for text messaging and 37.9% (69/182) for Internet access, of which 19.7% (13/66) accessed the Internet at least daily.
From April 2015 to June 2016, enrolled participants completed 471 person-years of prospective observation with a mean weekly symptom reporting rate of 78% (range 58%-89%) among enrolled children. During the first 25 weeks of observation, study participants with a low (<70%) weekly parental symptom reporting rate using the Vigilant-e app were more likely to be female than participants with a high (≥70%) weekly reporting rate (57% vs 44%, risk ratio [RR] 1.4, 95% CI 1.1-1.9). Households with a low (<70%) weekly symptom reporting rate (n=57) were more likely to have a greater number of children (mean 2.8, SD 1.5 vs mean 2.5, SD 1.3; RR 1.2, 95% CI 1.1-1.4), were less likely to have used mobile phones for text messaging at study enrollment (61% vs 77%, RR 0.6, 95% CI 0.4-0.9), and were less likely to access health care at the local public clinic (35% vs 67%, RR 0.4, 95% CI 0.2-0.6) than households with a ≥70% reporting rate (
Several external factors disrupted the weekly symptom reporting by study participants, including a period of staff turnover, the collapse of a local cellular tower, and contentious primary and secondary national elections (
When comparing symptom reporting using the Vigilant-e app to nurse-recorded reporting at the home visit (conducted within 48 hours), there was strong agreement between the Vigilant-e app and the home visit for all symptoms except bleeding, which was rarely reported (
Study participant and household characteristics and risk factors associated with low symptom diary app response rate (<70%), April to September 2015.
RR (CI)b |
||||||||||||||
Children enrolled, n | 469 | 322 | 147 | |||||||||||
Age (years), mean (SD) | 7.3 (4.7) | 7.1 (4.8) | 7.5 (4.4) | 1.0 (1.0-1.04) | .44 | |||||||||
Gender (female), n (%) | 225 (47.9) | 141 (43.8) | 84 (57.1) | 1.4 (1.1-1.9) | .008 | |||||||||
Child vaccinated (rotavirus), n (%) | 250 (53.3) | 165 (51.2) | 85 (57.8) | 1.2 (0.9-1.7) | .13 | |||||||||
Child attends school (if age ≥6 years), n (%) | 217 (85.1) | 143 (84.1) | 74 (87.1) | 0.8 (0.5-1.4) | .55 | |||||||||
Households enrolled, n | 207 | 150 | 57 | |||||||||||
Individuals in house, mean (SD) | 5.0 (1.8) | 4.9 (1.7) | 5.2 (2.0) | 1.1 (0.9-1.2) | .27 | |||||||||
Children enrolled per household, mean (SD) | 2.6 (1.4) | 2.5 (1.3) | 2.8 (1.5) | 1.2 (1.1-1.4) | .004 | |||||||||
Children aged ≤5 years enrolled per household, mean (SD) | 1.0 (0.8) | 1.0 (0.8) | 1.0 (0.8) | 1.0 (0.8-1.4) | .81 | |||||||||
Household cluster density, mean (SD) | 9.5 (8.6) | 9.9 (3.4) | 8.1 (8.0) | 0.8 (0.6-1.04) | .09 | |||||||||
Primary caregiver literacy, n (%) | 183 (88.4) | 131 (87.3) | 52 (91) | 1.3 (0.6-3.0) | .52 | |||||||||
Father’s education ≥secondary, n (%) | 51 (24.6) | 34 (22.7) | 17 (30) | 1.3 (0.8-2.1) | .28 | |||||||||
Mother’s education ≥secondary, n (%) | 38 (18.4) | 32 (21.3) | 6 (11) | 0.5 (0.2-1.1) | .10 | |||||||||
Health care at public clinic, n (%) | 121 (58.5) | 101 (67.3) | 20 (35) | 0.4 (0.2-0.6) | <.001 | |||||||||
Duration at current house (years), mean (SD) | 8.1 (3.4) | 8.04 (3.4) | 8.34 (8.8) | 1.0 (0.9-1.1) | .63 | |||||||||
Cellular phones per household, mean (SD) | 1.4 (1.1) | 1.4 (1.0) | 1.4 (1.1) | 1.0 (0.8-1.2) | .95 | |||||||||
No phone | 25 (12.1) | 16 (10.7) | 9 (16) | Ref | ||||||||||
No mobile phone | 99 (47.8) | 73 (48.7) | 26 (46) | 0.7 (0.4-1.4) | .32 | |||||||||
Mobile phone | 82 (39.6) | 60 (40.0) | 22 (39) | 0.7 (0.4-1.4) | .36 | |||||||||
Phones used for text messaging, n (%)c | 150 (72.5) | 115 (76.7) | 35 (61) | 0.6 (0.4-0.9) | .02 | |||||||||
Uses a phone with Internet, n (%)c | 69 (37.7) | 56 (42.1) | 13 (27) | 0.6 (0.4-1.1) | .10 | |||||||||
≤Weekly | 53 (80.3) | 45 (81.8) | 8 (73) | Ref | ||||||||||
≥Daily | 13 (19.7) | 10 (17.8) | 3 (27) | 1.5 (0.5-5.0) | .48 | |||||||||
100 | 92 (0.7) | 8 (0.3) | 0.2 (0.1-0.4) | <.001 | ||||||||||
Norovirus-associated acute gastroenteritis, n (%) | 12 (3.7) | 0 (0) | N/Cf | N/C | ||||||||||
122 | 112 (0.9) | 10 (0.3) | 0.2 (0.1-0.4) | <.001 | ||||||||||
Dengue-associated acute febrile illness, n (%) | 4 (1.2) | 0 (0) | N/C | N/C |
aWe were unable to model the random effects of multiple children per household due to relatively low numbers of children per household.
bRisk ratios (RR) and 95% confidence intervals were calculated using univariate generalized linear models, with dichotomous response rate in the first 25 weeks of surveillance (≥70% vs <70%) as the outcome of interest.
c12% of households are missing these variables.
d68% of all households are missing this variable because they said they did not use a phone with Internet access in the previous question.
eThe response rates reflect the first 25 weeks of surveillance, despite the longer syndromic reporting period (April 2015-June 2016).
fN/C: not calculated
Map showing clusters of participants with high (>70%; yellow circle) symptom diary app response rate versus moderate (40%-70%; orange circle) and low (<40%; red circle) response rates, in the Southwest Trifinio Region of Guatemala during the first 25 weeks of surveillance prior to allowing study nurses to manually enter syndromic data (April-October 2015).
Weekly syndromic reporting rate for acute febrile illness and acute gastroenteritis, April 2015-June 2016. Weekly syndromic reporting rate of participants using the Vigilant-e symptom diary mobile phone app (orange), manually entered data from nurse phone call (green), and combined mobile phone and manual data entry (blue). Several factors were associated with periods of decreased reporting, including a time period of high staff turnover (June-July 2015), a cellular tower collapse (August 2015), and primary and run-off presidential elections (October 2015). On October 2, 2015, study nurses were allowed to manually enter participant data if there was no response. In April 2016, study nurses performed an in-home visit to participating households to repair or replace malfunctioning phones and remind participants to use the Vigilant-e app for reporting when possible.
Agreement of symptom reporting among study participants between mobile phone symptom diary app and nurse home visit, April 2015 to June 2016.
Symptomsa | Kappa or Kendall taub | |||
Fever, n (%) | 62 (56.9) | 79 (69.9) | .57 | <.001 |
Fever duration (days), mean (SD) | 2.9 (1.3) | 3.0 (1.8) | .46 | <.001 |
Rash, n (%) | 15 (24.2) | 16 (20.3) | .59 | <.001 |
Pain, n (%) | 38 (61.3) | 45 (57.0) | .55 | <.001 |
Nausea, n (%) | 29 (46.8) | 32 (40.5) | .48 | <.001 |
Bleeding, n (%) | 3 (4.8) | 1 (1.3) | –.02 | .82 |
Vomiting, n (%) | 62 (57.4) | 29 (25.7) | .63 | <.001 |
Duration (days), mean (SD) | 2.5 (2.0) | 1.9 (0.9) | .69 | <.001 |
Maximum emesis/day, mean (SD) | 4.5 (2.8) | 3.6 (1.7) | .56 | .002 |
Diarrhea, n (%) | 33 (30.3) | 70 (62.0) | .61 | <.001 |
Diarrhea duration (days), mean (SD) | 3.2 (1.8) | 3.4 (1.8) | .78 | <.001 |
Maximum stools/day, mean (SD) | 4.7 (2.1) | 5.1 (2.2) | .29 | .006 |
aParticipants were asked additional symptom questions if they responded that “yes” their child had fever, diarrhea, or vomiting on the app or the nurse phone call. Nurses also asked the same questions using the same screening technique at the home visit (along with many more detailed questions). Symptoms included any reported symptom, regardless of duration.
bKappa statistic for categorical variables and Kendall tau for continuous variables.
Agreement between self-reported symptoms using the Vigilant-e app and study nurse-collected symptoms at home visit.
Days between app report and home visita | |||||||||
Kappa | Kappa | Kappa | |||||||
<1 | 79 | .70 | <.001 | .66 | <.001 | .65 | <.001 | ||
1 | 19 | .51 | .03 | .68 | .002 | .76 | .002 | ||
≥2 | 15 | .08 | .71 | .28 | .29 | .13 | .64 |
aAs the time interval increased between self-reported symptoms (Vigilant-e app) and nurse-collected symptoms (home visit), agreement between these reporting mechanisms decreased (kappa coefficient). If nurse-collected symptoms occurred within 1 day of self-report, kappa agreement was .65-.70.
In a resource-limited region of Guatemala with low literacy rates, we implemented a mobile phone-based participatory syndromic surveillance system with a high weekly response rate and a high rate of agreement between mobile phone parental reporting and nurse home visit reporting. We identified individual and community factors that led to decreased reporting, including female sex of the study participant, a greater number of children in the home, less prior experience with SMS text messaging, and lower utilization of local public health clinics. During our surveillance period, several external factors were associated with decreased reporting, including a cellular tower collapse and national elections, which highlight novel problems in conducting mobile phone-based surveillance. In addition, we demonstrated how contact with study participants, either with phone calls or home visits, may influence self-reporting. Finally, in this region of high norovirus and dengue virus burden [
Conducting prospective and accurate infectious disease surveillance in resource-limited settings is difficult for many reasons, including a lack of trained personnel, infrastructure, and diagnostic testing. Prospective surveillance studies have traditionally used weekly home visits or phone calls to collect syndromic data, followed by diagnostic testing in individuals meeting predefined case definitions, although these systems require significant resources and personnel [
We identified several practical lessons in performing mobile phone-based participatory syndromic surveillance in this resource-limited area. Although mobile phones are increasingly integrating into these communities, the telecommunication infrastructure is susceptible to interference, as demonstrated by decreased reporting following a cellular tower collapse. In addition, other external factors, including a period of high staff turnover that delayed reminder phone calls and widespread protests that impacted both participants and study personnel, were associated with periods of decreased self-reporting. Users were provided with limited data use per month and phones were locked to prevent use of non-study apps, but participants still found ways to circumvent this process. We found that regular communication between study personnel and participants led to improved reporting, but this required more personnel. Allowing study nurses to manually enter participant data into the database, instead of the participants, led to improved reporting overall but was associated with decreased self-reporting, somewhat undermining the participatory syndromic surveillance system. Prior to this intervention, the reporting rate (74%) was consistent with participatory syndromic surveillance studies in non-LMICs [
As mobile phones and data networks become increasingly integrated into resource-limited regions of the world, mobile phone-based participatory syndromic surveillance will likely become a more powerful tool to collect population-level syndromic data. Although we provided mobile phones to participants in our study, an important future step will be to allow users to download a symptom diary app onto their own mobile phones and encouraging syndromic self-reporting by providing some small incentive, such as prepaid airtime. This strategy, although still requiring personnel to maintain communication with participants and collect samples when needed, would allow a significant scale-up of the surveillance platform. And although syndromic data would be collected for all individuals, limiting diagnostic testing to a random or higher risk (meeting a more specific case definition) subset of participants would significantly reduce costs. This type of system could also be used for specific populations, such as in screening pregnant women for Zika virus, or during outbreaks of emerging pathogens such as Ebola. Because access to mobile phones is not evenly distributed within a population, it will be important for these types of surveillance programs to find strategies to ensure population-level representativeness.
The study had several strengths and limitations. We chose to perform the study in one of the most resource-limited regions of Guatemala where 70% of the population reports food insecurity and where 60% of households do not own a mobile phone [
In summary, we successfully implemented a mobile phone-based participatory syndromic surveillance system in a resource-limited region of Guatemala and identified several factors that positively or negatively impacted self-reporting. Self-reporting using a symptom diary mobile phone app (Vigilant-e) was accurate when compared to study nurse-collected data during a home visit. Future studies should evaluate mobile phone-based participatory syndromic surveillance for specific high-risk populations and pathogens at other sites, and should scale-up syndromic self-reporting with diagnostic testing performed only within a randomized or select subset of responders.
Sample screenshots from the Vigilant-e application (Integra IT, Bogota, Colombia) for weekly participatory syndromic surveillance in the Trifinio Region of Guatemala. Each week, subjects would select if any of the children enrolled from their household had fever or rash (Panel A). If the parent or guardian reports symptoms (red box), a decision-tree logic would then ask them to report symptoms for each child (Panel B). If they selected the child and then reported fever, vomiting, or diarrhea (not shown), another screen would appear asking additional syndromic questions (Panel C).
enzyme-linked immunosorbent assay
health care workers
Integrated Management of Childhood Illness
low- and middle-income countries
not calculated
risk ratio
reverse transcription polymerase chain reaction
We thank the families who participated in the study and the community leaders who facilitated its implementation. In addition, we are grateful to the following for their significant contributions to this research: CU Trifinio Research Team, including Neudy Carolina Rojop, Carmen Andrea Chacon, Jeniffer Yajaira Cardenas, Edwin Estuardo Hernandez, Edgar Eduardo Barrios, Macaria Genoveva Bail, Ruth Aide Ramirez Angel, Maria Eloin Dhaenes Vivar, Dulce Maria Camas, and Carlos Alvarez Guillen; Universidad del Valle de Guatemala: Mirsa Ariano and Erick Mollinedo; and Integra IT Colombia. This study was supported by an Investigator-Initiated Sponsored Research Grant from Takeda Pharmaceuticals (IISR-2014-100647). Dr Olson is supported by NIH/NCATS Colorado CTSI Grant Number UL1 TR001082 and the Children’s Hospital of Colorado Research Scholar Award. Contents are the authors’ sole responsibility and do not necessarily represent official NIH view.
Study design, execution, and interpretation of data was done by DO, ML, CCR, and EJA. Data acquisition was coordinated by APA, AZ, DO, ML, RZP, SRRC, and EJA. Laboratory analysis and interpretation was coordinated by MRL and CCR. Data analysis and report generation was performed by ML, KLC, DO, and EJA. Mapping was done by KLC. Preparation of the manuscript was led by DO, ML, and EJA. All authors reviewed and approved the final version.
Dr Asturias has served on an Advisory Board for Takeda Vaccines Inc and is partially supported by research grants from GlaxoSmithKline Biologicals and Takeda Vaccines Inc. Dr Lamb is partially supported by grants from GlaxoSmithKline Biologicals and Pantheryx Inc. Dr Olson is partially supported by a grant from Takeda Vaccines Inc. Ricardo Zambrano-Perilla and Sergio Ricardo Rodríguez-Castro are employed by Integra IT and own stake in the company.