This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Epidemiological studies on physical activity often lack inexpensive, objective, valid, and reproducible tools for measuring physical activity levels of participants. Novel sensing technologies built into smartphones offer the potential to fill this gap.
We sought to validate estimates of physical activity and determine the usability for large population-based studies of the smartphone-based CalFit software.
A sample of 36 participants from Barcelona, Spain, wore a smartphone with CalFit software and an Actigraph GT3X accelerometer for 5 days. The ease of use (usability) and physical activity measures from both devices were compared, including vertical axis counts (VT) and duration and energy expenditure predictions for light, moderate, and vigorous intensity from Freedson’s algorithm. Statistical analyses included (1) Kruskal-Wallis rank sum test for usability measures, (2) Spearman correlation and linear regression for VT counts, (3) concordance correlation coefficient (CCC), and (4) Bland-Altman plots for duration and energy expenditure measures.
Approximately 64% (23/36) of participants were women. Mean age was 31 years (SD 8) and mean body mass index was 22 kg/m2 (SD 2). In total, 25/36 (69%) participants recorded at least 3 days with at least 10 recorded hours of physical activity using CalFit. The linear association and correlations for VT counts were high (adjusted
The CalFit system had lower usability than the Actigraph GT3X because the application lacked a means to turn itself on each time the smartphone was powered on. The CalFit system may provide valid estimates to quantify and classify physical activity. CalFit may prove to be more cost-effective and easily deployed for large-scale population health studies than other specialized instruments because cell phones are already carried by many people.
Physical inactivity now ranks as the tenth leading cause of premature mortality worldwide [
Information on physical activity in epidemiological studies is generally obtained by questionnaires and more recently with accelerometers [
To address these problems and take advantage of the increased use and improved technology of smartphones, we developed CalFit [
The aim of this research is to study the usability of CalFit software and to assess the validity of its physical activity measures in real world situations by comparing its physical activity measures under free-living conditions with those obtained from a well-known and validated accelerometer, the Actigraph GT3X [
We enrolled volunteers to wear the CalFit phone and a conventional accelerometer for 5 days. Thirty-six participants were recruited by way of emails sent to colleagues from the Centre for Research in Environmental Epidemiology (CREAL) and to friends of colleagues as part of a larger study based on active travel behaviors. Inclusion criteria were to live and work or study in Barcelona, to live more than 10 minutes walking distance from the workplace or school, and be able to ride a bike for at least 20 minutes. Volunteers who met the eligibility requirements were enrolled in the study after an information session in which they were provided with details on study objectives and procedures. The field study took place from November 2011 to February 2012.
Our study protocol was approved by the Ethics Committee of Hospital del Mar Research Institute, and written informed consent was obtained from all the participants.
Each participant was given an Actigraph GT3X accelerometer [
Characteristics of CalFit and Actigraph GT3X.
Characteristics | Google G1 with CalFit | Actigraph GT3X |
Size | 11.7×5.6×1.7 cm | 3.8×3.7×1.8 cm |
Weight | 158 g | 27 g |
Placement | Frontal mean point between both anterior superior iliac spines | Anterior superior iliac spine of the right hip |
Sample rate | 10 Hz | 30 Hz |
Data storage | 16 GB | 16 MB |
Battery life | 18 hours | 31 days |
Accelerometer sensor | AK8976A triaxial accelerometer (Asahi Kasei Microsystems, Japan) | ADXL335 triaxial accelerometer (Analog Devices, Norwood, MA) |
Registered range of acceleration | ±2.8 |
±3 |
Outcomes (measured) | Acceleration of the 3 axes | Acceleration of the 3 axes |
Outcomes (estimated) | Not wearing; energy expenditure and duration of physical activity | Not wearing; standing, sitting, and lying; energy expenditure and duration of physical activity |
Data from both devices were summarized to 1-minute intervals. We merged data streams from both accelerometers identifying the time alignment that yielded the highest association (adjusted
The accelerometer nonwear intervals were defined as episodes of at least 40 consecutive minutes of 0 counts and below 0.3
Physical activity was defined as any minute with intensity equal or greater than 1.5 METs. Physical activity was partitioned into light, moderate, and vigorous levels of physical activity following the conventional cutoffs of 3 and 6 METs. The main summary measures of physical activity were vertical axis counts, and duration and intensity of physical activity.
Set of devices that were worn during 5 consecutive days.
Participants and physical activity characteristics are presented as number (percentage) for categorical variables, mean (SD) for continuous variables with normal distribution, or median (interquartile range, IQR) for continuous variables with non-normal distribution.
The comparison between CalFit and Actigraph GT3X was conducted using several approaches. First, to assess differences on usability as defined above, we performed a Kruskal-Wallis rank sum test (difference of medians components of usability). Second, the correlation and association between the vertical axis measures during coinciding time periods were assessed through a Spearman correlation and linear regression, respectively. Third, the agreement in the main summary measures of physical activity, as previously defined and during coinciding time periods, was studied using Lin’s concordance correlation coefficient (CCC) [
As a sensitivity analysis, previous comparisons were also performed during coinciding days with at least 10 hours of recorded activity, without control of the coinciding time periods, to test the influence of nonmeasured periods on physical activity agreement. All analyses were conducted using R-2.14.1 2011 (The R Foundation for Statistical Computing).
The sample consisted of 36 participants, most of which were women (23/36, 64%), with a mean age of 30.9 years (SD 7.9), and mean body mass index of 22.2 kg/m2 (SD 2.4) (
Of 180 possible days for recorded data, 19 were missing from CalFit and 8 from the Actigraph GT3X. During recorded days, there was a significant difference between the median time recorded: 22 hours for CalFit and 24 hours for Actigraph GT3X (
The main reasons for failed CalFit data collection among the 11 subjects who recorded less than 3 valid assessment of physical activity were: (1) 6 lost an average of 2 days of recording because CalFit was inadvertently turned off, (2) 2 had problems with phone battery life and their daily routine, and (3) 3 did not wear the phone.
Sociodemographic and physical characteristics of all participants (N=36).
Sample characteristics | Participants | |
Age (years), mean (SD) | 31 (8) | |
|
|
|
|
Male | 13 (36) |
BMI (kg/m2), mean (SD) | 22 (2) | |
|
|
|
|
More than high school | 33 (92) |
|
Less than high school | 3 (8) |
|
|
|
|
Spanish | 30 (83) |
|
Others | 6 (17) |
|
|
|
|
More than 2000 | 18 (50) |
|
Less than 2000 | 18 (50) |
|
|
|
|
Working | 32 (89) |
|
Studying | 4 (11) |
Comparison of usability characteristics between Actigraph and CalFit.
Characteristic | Actigraph GT3X | CalFit |
|
Days recorded (day), median (IQR) | 5 (5-5) | 5 (4.8-5.0) | .03 |
Recorded time (min), median (IQR) | 7200 (7200-7200) | 6474 (4635-7068) | <.001 |
Wearing time (min), median (IQR) | 4109 (3735-4373) | 2938 (2269-3652) | <.001 |
Time coincident (min), median (IQR) | 2825 (2110-3556) |
|
|
Recorded time within recorded days (hour/day), median (IQR) | 24 (24-24) | 22 (20-24) | <.001 |
Worn time within recorded days (hour/day), median (IQR) | 14 (12.5-15) | 11 (10-13) | <.001 |
Percent of worn time on recorded time within recorded days (%),median (IQR) | 58.5 (53-63) | 51.6 (46-58) | .03 |
Number of days with at least 10 wearing hours (day), median (IQR) | 5 (4-5) | 3 (2-4.2) | <.001 |
Participants with valid assessment of physical activity, n (%) | 34 (94) | 25 (69) | <.001 |
The linear regression and correlation analysis for average vertical (VT) axis measures from both devices during coinciding wear-time periods showed a high association (adjusted
The comparison of measures of light, moderate, and vigorous physical activity showed that less than 30% of the variability was attributable to the method of measurement (
Depending on the time inclusion criteria selected, the average difference between Actigraph GT3X and CalFit during light physical activity changed from a small but significant overestimation of 1.7% (95% CI 0.4-3.1) for coinciding time periods to a nonsignificant underestimation of –11.5 min (95% CI –27 to 4.3) for coinciding valid days (
Agreement between CalFit and Actigraph GT3X in vertical axis, duration, and energy expenditure in physical activity within the coinciding measurement time periods. (A) accelerometer vertical axis measures, (B) duration in physical activity, and (C) intensity of physical activity.
Agreement between CalFit and Actigraph GT3X for duration of light, moderate, and vigorous physical activity within the coinciding measurement time periods. (A) duration of light physical activity, (B) duration of moderate physical activity, and (C) duration of vigorous physical activity.
Comparison of average intensity recorded by CalFit and Actigraph GT3X within light, moderate, and vigorous physical activity identified by Actigraph.
Agreement between CalFit and Actigraph GT3X during light, moderate, and vigorous physical activity within the coinciding days with at least 10 hours of recorded activity. (A) duration of light physical activity per day, (B) duration of moderate physical activity per day, and (C) duration of vigorous physical activity per day.
This study assessed the usability and validity of CalFit software in a group of free-living volunteers. We compared CalFit to physical activity measures with those obtained from the Actigraph GT3X. The several approaches used to assess the properties of the CalFit showed that (1) there is a strong association between vertical axis measures from both devices; (2) the measures of duration and energy expenditure in overall, light, and moderate physical activity were highly concordant between devices, whereas vigorous physical activity was underestimated; (3) CalFit had lower usability compared to Actigraph GT3X resulting in a lower proportion of participants with valid assessment of physical activity; and (4) sensitivity analysis that compared the agreement within coinciding time periods to the agreement within coinciding days with at least 10 hours of recorded activity showed that the disparities in wearing-time periods between devices did not contribute to any significant bias into the measured validity.
To our knowledge, this is the first study to compare accelerometer use on smartphones to measure physical activity with a currently well-validated instrument [
This is also the first study to compare the validity of the vertical axis measures and to use the same algorithm for estimating physical activity in 2 different instruments. The association of the vertical axis measures between the 2 tools was high (adjusted
Concordance in physical activity measures across different definitions of time inclusion criteria showed that the results remained constant despite the shorter wearing time of CalFit. This suggests that the time charging the smartphone or the shorter battery life did not have a significant influence on the final measures. There was also a statistically significant bias toward underestimation in measures of vigorous physical activity estimated by CalFit compared to Actigraph GT3X. This may be partially explained by the fact that we used average VT measures instead of all measures per minute per participant and because we assumed a linear relationship between acceleration forces from smartphone and counts from Actigraph GT3X.
One of the main strengths of this study is the use and testing under free-living conditions. Participants maintained their daily routines, which is difficult to replicate in controlled environments. Another strength was the use of the concordance measures for quantifying physical activity in addition to the commonly used correlations. A third was using the Freedson algorithm of physical activity for both instruments, which is a valid algorithm for the different Actigraph models (CSA 1764, GT1M, and GT3X) [
The validation of smartphone accelerometry-based energy expenditure has implications for both epidemiologic research on physical activity as well as for the growth of the practice of medicine and public health by mobile applications (mHealth applications). Beyond the current CalFit application, which is focused on unobtrusive sensing of physical activity, may be novel mHealth smartphone applications that not only record physical activity, but attempt to intervene upon behavior [
The CalFit smartphone system has several advantages over conventional accelerometers because of geolocation information both from cell phone towers and Wi-Fi networks and from GPS satellites. This geolocation will allow us to improve the current physical activity algorithm by including information such as velocity of displacements, topographical challenges faced by participants (stairs, slopes), and the environments (home, work) where physical activity occurred. Furthermore, this tool also allows assessments of how the built and natural environment may affect behavior or lead to other exposures. Our research group has begun to demonstrate some of these advantages with the same participants by characterizing where the physical activity was done and quantifying the amount of pollution inhaled by participants in these environments [
One limitation of the present study was the use of a convenience sample of 36 participants with a high educational level to assess CalFit usability. However, this design has been efficient in detecting the problems in usability. Further work needs to be conducted in the population at large. Second, the use of the Actigraph GT3X accelerometer as a gold standard could be seen as a limitation, but it is the reference tool for assessing physical activity in real life for 5 days and has well-established validity [
Compared to the current gold standard instrument for population studies, the smartphones fitted with CalFit supply useful and valid estimates for quantifying and classifying physical activity under free-living conditions. Although user compliance for CalFit was lower than with the Actigraph GT3X, this difference would likely diminish if participants were allowed to load CalFit onto their existing smartphones, which will be feasible in the future. Such deployment would provide a cost-effective approach for large epidemiological studies and mHealth applications that rely upon measured physical activity.
concordance correlation coefficient
Global Positioning System
Metabolic Equivalent of Task
vertical
We thank Jaume Matamala and Meritxell Portella for the fieldwork done in this study. Funding for this project was provided by the Centre for Research in Environmental Epidemiology (CREAL) Internal Grant Program, the Coca-Cola Foundation through the Transportation Air pollution and Physical ActivitieS (TAPAS) Program, and NIH NIEHS grant R01-ES020409. Finally, we thank Dr Ruzena Bajcsy and seed funding from the Center for Information Technology in the Interest of Society (CITRIS), which were instrumental in the development of CalFit.
None declared.