Background: Demographic growth in conjunction with the rise of chronic diseases is increasing the pressure on health care systems in most OECD countries. Physical activity is known to be an essential factor in improving or maintaining good health. Walking is especially recommended, as it is an activity that can easily be performed by most people without constraints. Pedometers have been extensively used as an incentive to motivate people to become more active. However, a recognized problem with these devices is their diminishing accuracy associated with decreased walking speed. The arrival on the consumer market of new devices, worn indifferently either at the waist, wrist, or as a necklace, gives rise to new questions regarding their accuracy at these different positions.
Objective: Our objective was to assess the performance of 4 pedometers (iHealth activity monitor, Withings Pulse O2, Misfit Shine, and Garmin vívofit) and compare their accuracy according to their position worn, and at various walking speeds.
Methods: We conducted this study in a controlled environment with 21 healthy adults required to walk 100 m at 3 different paces (0.4 m/s, 0.6 m/s, and 0.8 m/s) regulated by means of a string attached between their legs at the level of their ankles and a metronome ticking the cadence. To obtain baseline values, we asked the participants to walk 200 m at their own pace.
Results: A decrease of accuracy was positively correlated with reduced speed for all pedometers (12% mean error at self-selected pace, 27% mean error at 0.8 m/s, 52% mean error at 0.6 m/s, and 76% mean error at 0.4 m/s). Although the position of the pedometer on the person did not significantly influence its accuracy, some interesting tendencies can be highlighted in 2 settings: (1) positioning the pedometer at the waist at a speed greater than 0.8 m/s or as a necklace at preferred speed tended to produce lower mean errors than at the wrist position; and (2) at a slow speed (0.4 m/s), pedometers worn at the wrist tended to produce a lower mean error than in the other positions.
Conclusions: At all positions, all tested pedometers generated significant errors at slow speeds and therefore cannot be used reliably to evaluate the amount of physical activity for people walking slower than 0.6 m/s (2.16 km/h, or 1.24 mph). At slow speeds, the better accuracy observed with pedometers worn at the wrist could constitute a valuable line of inquiry for the future development of devices adapted to elderly people.
Physical activity is universally recognized as playing an essential role in primary, secondary, and tertiary health prevention. This has been highlighted for patients with cardiovascular disease, diabetes, or osteoporosis, among other health hazards . With the widespread increase in life expectancy, these diseases become more frequent [ ], thereby exerting greater pressure and generating increasing costs on the health care system. One recognized way to reduce the cost associated with this progressively frail population is to improve their independence and health by keeping them physically active [ ]. Physical activity decreases the incidence of chronic diseases such as diabetes, hypertension, and obesity, among others [ ], and reduces hospitalization as well as the mortality rate [ , ]. It has been demonstrated that elderly people who remain physically active reduce their risk of falling, have decreased disability, succumb less to diseases, and maintain their independence longer [ , ]. Walking is a suitable physical activity for frail individuals, as well as being one of the preferred activities among older adults [ ]. Therefore, any intervention able to encourage walking activities should be promoted among this population. Goal-setting theory teaches us that measuring one’s activity, setting suitable goals, and receiving positive feedback on it is a motivating factor toward undertaking more physical activities [ ]. The low cost, small size, and simple ergonomics of pedometers make them particularly suited to motivate people to stay active by monitoring their activities [ ]. However, in order to successfully apply goal-setting theory, it is reasonable to expect a minimum level of accuracy from the selected pedometer. Indeed, irrelevant feedback can frustrate users and lead them to give up their objectives.
Frail individuals, such as diabetic, obese patients or those with heart failure, often walk at a slow pace (around 0.6 m/s  and as low as 25 m/min, or 0.4 m/s, for community ambulation [ ]). At such a pace, many pedometers show a lack of accuracy with relative errors going from 30% to 60% [ , - ]. A study on a group of patients with chronic heart failure testing the accuracy of the Omron HJ-720ITC pedometer reported an error close to 24% at 0.66 m/s, approximately 9% at 0.83 m/s, 5% at 1 m/s, approximately 3% at 1.16 m/s, and 1% at 1.33 m/s [ ]. The study of Marschollek et al [ ] compared 4 freely accessible pedometer algorithms on healthy people and on mobility-impaired geriatric inpatients in free walking. With healthy people, an error between 8.4% and 30.8% was observed, whereas with the geriatric population the error was between 28.1% and 62.1%. Another study [ ] comparing 5 pedometers (Omron HJ-105, Yamax Digiwalker 200, SportLine330, New-Lifestyles 2000, and ActiCal) on older adults reported a mean error of 9% for all devices at a self-selected speed. This error rose to 19% at 80 steps/min, 40% at 66 steps/min, and 56% at 50 steps/min. Fitbit Ultra worn on the wrist and on the hip was tested with the Samsung GT-19300 mobile phone in a study conducted by Lauritzen et al [ ]. This device was tested on 3 distinct populations: healthy adults, elderly people with normal mobility, and elderly people with reduced mobility using a rollator. The count produced for elderly people using a rollator had a greater than 60% error. Other studies have been conducted to investigate the influence of the position of the pedometer on its accuracy [ , ]. For instance, a study by Abel et al [ ] tested a pedometer at 3 positions at the waist (anterior, midaxillary, and posterior) for 3 different speeds (59, 72, and 86 m/s and at own pace) and took into account the influence of the waist circumference. Whereas the placement had no influence for a low waist circumference, the posterior position was best with a high waist circumference. Another study evaluated the Yamax SW-200 pedometer in 5 different positions at the waist (left midaxillary, left midthigh, umbilical, right midthigh, and right midaxillary). The tests of using the pedometer while walking on a treadmill on flat ground, as well as ascending and descending stairs, indicated a better performance when the pedometer was positioned in the left midaxillary position [ ].
Although, until recently, most pedometers were worn at the waist, a new generation entering the market offers more versatility and can be worn not only at the waist but also at the wrist or as a necklace. These new pedometer positions raise questions regarding their accuracy compared with the one worn at the waist. In order to investigate the influence of the position (wrist, waist, or necklace) in relation to the speed of movement, we conducted a comparative study of several pedometers by exploring the accuracy of their readings depending on the position of the device and the speed of movement.
We tested 4 commercially available pedometers at 4 different walking speeds: 3 at controlled speed (0.4 m/s, 0.6 m/s, and 0.8 m/s) and 1 at uncontrolled speed (natural speed of the participants) on a normalized 100 m long floor with equidistant marks. Each experiment was videotaped at normalized speed and synchronized to the participants. Pedometers were reset between each experiment and used with full charge power. The number of steps indicated by the pedometer was compared with the number of steps manually counted using the video.
From previous similar studies, we have identified that a minimum of 20 participants  is necessary to demonstrate significant differences between the experimental settings. Since slow walks simulated by adults do not produce acceleration patterns significantly different from those of frail individuals with reduced walking speed [ , ], we decided to recruit healthy people and ask them to walk at controlled paces. Participants were recruited on a voluntary basis with the only inclusion criterion being that they should be able to walk at least 500 m and not have any walking disabilities.
We used 4 different devices during this study: iHealth activity monitor (IH; iHealth Labs Inc, Mountain View, CA, USA), Withings Pulse O2 (WI; Withings, Issy-les-Moulineaux, France), Misfit Shine (MF; Misfit, Inc, Burlingame, CA, USA), and Garmin vívofit (GA; Garmin Ltd, Southampton, UK).lists their specifications. We selected these devices according to the following criteria: (1) 2 devices that can be worn at several positions, can count steps during an entire day, and can be integrated into a complete solution of health monitoring, (2) 1 device that is especially small, and (3) 1 device that integrates into a wide-ranging sport ecosystem.
iHealth Activity Monitor
iHealth is a brand specialized in health devices such as a glucometer and a blood pressure monitor. The IH can register the total number of steps during a day, distance travelled, and calories burned. It tracks sleep quality and can be placed either at the wrist or at the waist on a belt.
Withings Pulse O2
Withings commercializes devices such as a blood pressure monitor, a sleep monitor, a scale, and a pedometer. The WI can be placed at the wrist, on a belt, or on a shirt collar. This device tracks the number of steps, elevation, running time, calories burned, and distance travelled. It tracks users’ sleep quality, heart rate, and blood oxygen level.
Misfit doesn’t offer the same range of monitoring device as the 2 previous brands. We chose the MF for its very small size, an interesting feature that favors its acceptance by elderly people who are especially sensitive to stigmatization. It can be worn at the wrist, on a belt, or as a necklace. The device tracks the number of steps, distance travelled, calories burned, and the sleep pattern.
Garmin is a brand that covers a very large ecosystem of devices for sporting activities. The GA tracks the number of steps, calories burned, distance travelled, and sleep pattern. It can only be placed at the wrist.
The study took place in a flat area where ground markings indicated distances. We performed the study in 2 phases. In the first phase, we requested participants to walk 200 m at their preferred pace in order to assess the performance of the pedometers at natural speed. In the second phase, participants walked for a distance of 100 m at a controlled speed wearing all pedometers simultaneously. We selected 3 different walking speeds (0.4 m/s, 0.6 m/s, and 0.8 m/s) for our experiment. The slowest speed was set to 0.4 m/s, since several studies have recognized this speed as the minimum necessary for performing everyday activities [, , ]. The fastest speed was limited to 0.9 m/s, since this is the limit that defines normal speed [ ]. We relied on the methodology defined by Martin et al [ ] and used a metronome to constrain the cadence of the walker. In order to minimize intra- and interparticipant variation, their step length was also constrained using a string attached between their legs at the level of their ankles. As footstep length and cadence are related [ ], the string also enables footstep length to be limited in order to keep a natural ratio with cadence that should be adopted at a specific speed.
According to research, the relation between footstep length and cadence is 0.55 steps/min . Consequently, for each targeted speed, footstep length can be determined using the ratio in equation 1 expressing the relation between footstep length, speed, and cadence ( , equation 2). Once footstep length is calculated, the cadence can be simply derived by transforming the equation 1 ( , equation 3).
Based on equations 2 and 3, we calculated the various settings of the experiment, presented in.
|iHealth activity monitor||Withings Pulse O2||Misfit Shine||Garmin vívofit|
|Position||Wrist, belt||Wrist, belt, shirt collar||Wrist, belt, necklace||Wrist|
|Battery life||7 days||2 days||3 months||1 year|
|Footstep length (cm)||36||44||51|
Each participant was encouraged to practice walking under these conditions as long as required until they considered they could walk the 100 m comfortably at the desired speed. Each walk was videotaped in order to count the number of steps precisely during the analysis stage.
The actual footsteps were counted manually using the videotape by 2 independent (CW and FE) observers. If the number of steps counted did not match, the counting was restarted until they corresponded. This measure was then considered as the real step count to be compared with the count returned by the pedometers.
We calculated the relative error between real number of steps and steps registered by the pedometers according to equation 4.
For each speed and each position, we calculated the mean of the error. A 1-way analysis of variance (ANOVA) was conducted to evaluate whether there was a significant difference between the position for each speed and speed for each position of the group. The confidence interval was set at 95%.
Because we collected no personal data, we did not request institutional review board approval.
A total of 21 people participated in the study, 12 women and 9 men. The average age of the participants was 34.5 years (SD 15.7).
The results are presented in terms of the mean relative counting error at each speed and for each pedometer, as well as the average error for all participants (). The results are also presented graphically in and . contains a set of bar graphs highlighting the influence of speed on accuracy by position, and contains a set highlighting the influence of position on accuracy by speed.
|IHa||Wrist||10.21 (19.67)||14.79 (26.03)||26.97 (34.06)||62.64 (41.94)|
|Belt||0.55 (0.67)||5.12 (11.63)||16.29 (20.99)||56.45 (27.20)|
|WIb||Wrist||14.37 (23.78)||30.20 (31.43)||64.07 (41.76)||88.51 (25.35)|
|Belt||0.87 (0.80)||18.27 (29.07)||80.92 (27.00)||99.34 (3.79)|
|Necklace||1.52 (2.95)||29.07 (27.55)||88.46 (25.73)||99.84 (0.50)|
|MFc||Wrist||37.16 (47.81)||55.08 (8.10)||40.18 (32.67)||55.90 (33.19)|
|Belt||39.05 (48.54)||40.93 (41.62)||49.87 (39.41)||70.51 (45.08)|
|Necklace||10.93 (26.05)||34.93 (34.60)||53.09 (36.18)||63.32 (35.13)|
|GAd||Wrist||3.08 (5.65)||5.31 (9.38)||12.24 (16.92)||80.14 (25.41)|
aIH: iHealth activity monitor.
bWI: Withings Pulse O2.
cMF: Misfit Shine.
dGA: Garmin vívofit.
The MF pedometer generated an error higher than 30% in all cases, except at a natural pace when worn as a necklace (11%). When pedometers were worn at the wrist, the error was higher than 10% independently of the walking speed, except for GA. At a natural pace, every pedometer worn at the belt generated errors below 5%, except for MF. GA placed at the wrist and IH placed at the belt still had an error below 6% at 0.8 m/s and below 20% at 0.6 m/s. IH at the wrist and WI at the belt had an error below 20% at 0.8 m/s.
presents the mean error in terms of position and speed, with the results of a 1-way ANOVA.
Results According to Speed
The general tendency observed inhighlights the correlation between the decrease of speed and the increase of mean error. This tendency was verified for every pedometer at every position except for MF at the wrist. For this pedometer, there was similar relative error at 0.4 m/s and 0.8 m/s (approximately 55%), as well as at 0.6 m/s and at natural speed (approximately 40%).
Results According to Position
At a natural pace, when pedometers were placed at the wrist, the mean relative error was higher than when they are located at the 2 other positions (). At the belt, pedometers were less accurate than at the collar. At 0.8 m/s, the belt position generated results with the best accuracy, followed by the wrist and then the necklace. At 0.6 m/s, the wrist position generated the lowest error, followed by the belt and the necklace positions. The same tendency was observed at 0.4 m/s.
The 1-way ANOVA showed that at each position, the mean error differed at each selected speed, except at the belt, where it was at the limit of confidence. On the other hand, 1-way ANOVA didn’t reveal a significant difference in accuracy at the various positions for a given speed.
|Position||Speed (m/s)||Average||P value|
The influence of speed on accuracy can be clearly observed in, regardless of which pedometer or position was selected. The mean relative error significantly increased when speed decreased until it attained more than 50% at the slowest pace. The MF placed at the wrist did not show the same tendency, since the error at 0.6 m/s was lower than the error at 0.8 m/s. We attribute this phenomenon to the strong unreliability of this pedometer, which produced a high rate of error when worn at the wrist even at a user-preferred speed.
The reason for the important error at low speed may stem from the type of algorithm used in most pedometers. In order to count steps, many algorithms rely on vertical acceleration. As the vertical acceleration diminishes according to the walking speed, it is more difficult to detect every footstep at a slow pace. MF may not use this kind of algorithm, since it did not show the same result.
We did not observe a significant influence of the pedometer position on the accuracy of the readings. Generally speaking, at normal speed, wearing a pedometer at the wrist decreased the accuracy more noticeably than wearing it at the belt or as a necklace. When the speed decreased, however, pedometers worn at the wrist had the best accuracy, and those worn as a necklace had the worst. This can be explained by the fact that, during slow walking, the vertical acceleration of the body is low but the arms are usually still moving.
It should be noted that this study was conducted on healthy adults and not on individuals walking slowly due to some impairment. Controlling the walking speed by constraining cadence and pace length using a string and a metronome can potentially change the natural way of walking. In fact, it is difficult to normalize walking because everybody reacts differently to the string between their feet. Some participants easily adopted the required cadence, whereas others needed more concentration. Nevertheless, participants were allowed to practice walking with the string using the metronome cadence until they felt comfortable and were able to adopt a natural walk before the beginning of the experiment.
It remains questionable whether the tested pedometers are suitable for a slow-walking population. Responding to this question would require identifying which level of error remains acceptable while monitoring walking activity.
Other studies have shown a similar evolution of error in terms of speed [, , ]), that is, the error increases when speed decreases. The Omron HJ-720ITC pedometer was tested on patients with chronic heart failure [ ], producing an error close to 24% at 0.66 m/s, approximately 9% at 0.83 m/s, 5% at 1 m/s, approximately 3% at 1.16 m/s, and 1% at 1.33 m/s. Thus, when the speed increased, the error decreased. A study on older adults, comparing 5 pedometers [ ], reported mean errors from 9% for all devices at a self-selected speed to 56% at 50 steps/min. At 80 steps/min, the error was 19%, and at 66 steps/min, the error was approximately 40%. A study comparing 7 pedometers [ ], the DynaPort Movemonitor, Jawbone UP, Fitbit One, activPAL, Tractivity, Nike+ FuelBand, and Sensewear Armband, reported that the error increased during slow walking (around 1.6 km/h, or 0.4 m/s). But the error differences between speeds was pedometer dependent. Jawbone UP, Tractivity, Nike+ FuelBand, and Sensewear Armband showed a significant difference between slow speed and self-selected, fast speed. DynaPort Movemonitor, Fitbit One, and activPAL showed an error close to the other speeds with an error under 3.2% at every speed.
The main achievement of this study was to compare the influence of walking speed and pedometer position on the accuracy of pedometer readings. To our knowledge, ours is the first study that formally investigated this relation. This study showed that a reduction of walking speed negatively influenced the accuracy of the tested pedometers. Although this result would require a larger study to be confirmed, we observed that the position ensuring the best pedometer accuracy depended on the speed. At a normal pace, pedometers worn at the belt or as a necklace are more accurate, whereas for slow walkers, wearing pedometers at the wrist is the best choice. This study could open a valuable line of inquiry for the development of future devices for frail people, relying on the acceleration of arm movement to improve accuracy. Apart from this suggestion, this study underlines the conclusion that, before being used, a pedometer should first be assessed individually according to expected speed of movement before deciding on where to position of the device.
Conflicts of Interest
- Booth FW, Roberts CK, Laye MJ. Lack of exercise is a major cause of chronic diseases. Compr Physiol 2012 Apr;2(2):1143-1211 [FREE Full text] [CrossRef] [Medline]
- Christensen K, Doblhammer G, Rau R, Vaupel JW. Ageing populations: the challenges ahead. Lancet 2009 Oct 3;374(9696):1196-1208 [FREE Full text] [CrossRef] [Medline]
- Sugihara N, Watanabe M, Tomioka M, Braun KL, Pang L. Cost-benefit estimates of an elderly exercise program on Kaua'i. Hawaii Med J 2011 Jun;70(6):116-120 [FREE Full text] [Medline]
- Garber CE, Blissmer B, Deschenes MR, Franklin BA, Lamonte MJ, Lee I, American College of Sports Medicine. American College of Sports Medicine position stand. Quantity and quality of exercise for developing and maintaining cardiorespiratory, musculoskeletal, and neuromotor fitness in apparently healthy adults: guidance for prescribing exercise. Med Sci Sports Exerc 2011 Jul;43(7):1334-1359. [CrossRef] [Medline]
- Garcia-Aymerich J, Lange P, Benet M, Schnohr P, Antó JM. Regular physical activity reduces hospital admission and mortality in chronic obstructive pulmonary disease: a population based cohort study. Thorax 2006 Sep;61(9):772-778 [FREE Full text] [CrossRef] [Medline]
- Garcia-Aymerich J, Lange P, Serra I, Schnohr P, Antó JM. Time-dependent confounding in the study of the effects of regular physical activity in chronic obstructive pulmonary disease: an application of the marginal structural model. Ann Epidemiol 2008 Oct;18(10):775-783. [CrossRef] [Medline]
- Buchner DM, Wagner EH. Preventing frail health. Clin Geriatr Med 1992 Feb;8(1):1-17. [Medline]
- American College of Sports Medicine, Pescatello LS. ACSM's Guidelines for Exercise Testing and Prescription. Baltimore, MD: Wolters Kluwer/Lippincott Williams & Wilkins Health; 2014.
- Yusuf HR, Croft JB, Giles WH, Anda RF, Casper ML, Caspersen CJ, et al. Leisure-time physical activity among older adults. United States, 1990. Arch Intern Med 1996 Jun 24;156(12):1321-1326. [Medline]
- Tudor-Locke C. Taking Steps Toward Increased Physical Activity: Using Pedometers to Measure and Motivate. President?s Council on Physical Fitness and Sports Research Digest, Series 3, No. Washington, DC: President's Council on Physical Fitness and Sports; 2002:17.
- Cohen-Mansfield J, Werner P, Culpepper WJ, Wolfson M, Bickel E. Assessment of ambulatory behavior in nursing home residents who pace or wander: a comparison of four commercially available devices. Dement Geriatr Cogn Disord 1997;8(6):359-365. [Medline]
- Jehn M, Schmidt-Trucksäss A, Schuster T, Hanssen H, Halle M, Köhler F. Pedometer accuracy in patients with chronic heart failure. Int J Sports Med 2010 Mar;31(3):186-191. [CrossRef] [Medline]
- Perry J, Garrett M, Gronley JK, Mulroy SJ. Classification of walking handicap in the stroke population. Stroke 1995 Jun;26(6):982-989 [FREE Full text] [Medline]
- Marschollek M, Goevercin M, Wolf K, Song B, Gietzelt M, Haux R, et al. A performance comparison of accelerometry-based step detection algorithms on a large, non-laboratory sample of healthy and mobility-impaired persons. Conf Proc IEEE Eng Med Biol Soc 2008;2008:1319-1322. [CrossRef] [Medline]
- Martin JB, Krč KM, Mitchell EA, Eng JJ, Noble JW. Pedometer accuracy in slow walking older adults. Int J Ther Rehabil 2012 Jul 3;19(7):387-393 [FREE Full text] [Medline]
- Lauritzen J, Muñoz A, Luis Sevillano J, Civit A. The usefulness of activity trackers in elderly with reduced mobility: a case study. Stud Health Technol Inform 2013;192:759-762. [Medline]
- Campion MA, Lord RG. A control systems conceptualization of the goal-setting and changing process. Organ Behav Hum Perform 1982 Oct;30(2):265-287. [CrossRef]
- Abel MG, Hannon JC, Eisenman PA, Ransdell LB, Pett M, Williams DP. Waist circumference, pedometer placement, and step-counting accuracy in youth. Res Q Exerc Sport 2009 Sep;80(3):434-444. [CrossRef] [Medline]
- Horvath S, Taylor DG, Marsh JP, Kriellaars DJ. The effect of pedometer position and normal gait asymmetry on step count accuracy. Appl Physiol Nutr Metab 2007 Jun;32(3):409-415. [CrossRef] [Medline]
- Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res 1999 Jun;8(2):135-160. [Medline]
- Kim WS, Kim EY. Comparing self-selected speed walking of the elderly with self-selected slow, moderate, and fast speed walking of young adults. Ann Rehabil Med 2014 Feb;38(1):101-108 [FREE Full text] [CrossRef] [Medline]
- Arif M, Ohtaki Y, Nagatomi R, Inooka H. Estimation of the effect of cadence on gait stability in young and elderly people using approximate entropy technique. Meas Sci Rev 2004;4:29-40.
- Castell M, Sánchez M, Julián R, Queipo R, Martín S, Otero A. Frailty prevalence and slow walking speed in persons age 65 and older: implications for primary care. BMC Fam Pract 2013;14:86 [FREE Full text] [CrossRef] [Medline]
- Studenski S, Perera S, Patel K, Rosano C, Faulkner K, Inzitari M, et al. Gait speed and survival in older adults. JAMA 2011 Jan 5;305(1):50-58 [FREE Full text] [CrossRef] [Medline]
- Zijlstra A, de Bruin ED, Bruins N, Zijlstra W. The step length-frequency relationship in physically active community-dwelling older women. Eur J Appl Physiol 2008 Oct;104(3):427-434. [CrossRef] [Medline]
- Storm FA, Heller BW, Mazzà C. Step detection and activity recognition accuracy of seven physical activity monitors. PLoS One 2015;10(3):e0118723 [FREE Full text] [CrossRef] [Medline]
|ANOVA: analysis of variance|
|IH: iHealth activity monitor|
|GA: Garmin vívofit|
|MF: Misfit Shine|
|WI: Withings Pulse O2|
Edited by G Eysenbach; submitted 28.04.16; peer-reviewed by L Griebel, R Freak-Poli, P Elliott; comments to author 26.05.16; revised version received 20.08.16; accepted 14.09.16; published 06.10.16Copyright
©Frederic Ehrler, Chloé Weber, Christian Lovis. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 06.10.2016.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.