To examine the usability of currently available diabetes applications for the elderly, we performed an expert-based usability evaluation. With this method, usability experts put themselves in the role of potential or current users to examine products in terms of usability. We performed a summative evaluation as we exclusively included apps whose development was already finished [18].

Due to the high number of apps available for review, the usability evaluation was based on a representative 10% sample of existing diabetes apps as of April 2013. The sample was chosen on a random basis. The evaluation was performed by three independent experts, as suggested by Nielsen [19] and Barnum [20]. They were chosen due to their comprehensive experience in handling and testing mobile devices and applications with regard to usability for the elderly and operability of accessibility features. In addition, they had specific expertise in the field of diabetes and diabetes management. They were already involved in the accompanying systematic review and a survey of diabetes patients aged 50 or older and physicians investigating the acceptance factors of diabetes applications. The authors of this article were not involved in the usability evaluation for reasons of independence.

The basis for the usability evaluation was defined by a specially created set of usability criteria considering interaction processes, interface design, and comprehensibility of content (Table 2). Therefore, we reviewed usability guidelines (ISO, DIN) with explicit regard to the requirements of the elderly concerning mobile applications. Additionally, we considered usability requirements that have been proven as relevant in previous studies with this age group (Table 2). This guideline-based approach of usability testing is in accordance with the proposals of Nielsen [19], and Sarodnick and Brau [18]. We divided the selected criteria into main and subcriteria, added a clear description of their specific characteristics and defined respective assessment criteria. The experts rated each subcriterion and the expression of its characteristics by means of a 5-point Likert scale to grade the evaluation or by means of a dichotomous scale [16,19-21]. The main criteria were not evaluated themselves; their scores were calculated from the mean of the respective subcriteria.

To lower barriers for persons with reduced or limited cognitive and physical skills, iOS and Android offer different accessibility features. We tested the operability of three features for each tested app in a separate test run. We have chosen features that are relevant to the elderly and were offered by both operating systems:

According to the methodical approach of Barnum, the evaluators run through typical scenarios of use to conduct their evaluation [20]. They were asked to take the perspective of a diabetes patient aged 50 or older. Each expert tested the main functionalities of the app, listed in the app description (eg, record of blood glucose data and/or medication, plotting graphs, search for information on diabetes mellitus, etc). All Android apps were tested on a Samsung Galaxy Note 10.1. All iOS apps were tested on an iPad 4^thgeneration. If apps were offered for multiple platforms by the manufacturer, they were tested on an iPad 4^thgeneration.

The chosen method offers a high level of validity and comparability due to its guideline-based approach and closed response categories [22]. Additionally, a user-based test would not have been able to represent the pronounced heterogeneity among the age group 50 or older regarding health status, skills and preferences, experience in technology use, sociodemographics, etc, which is much more pronounced than for younger age groups [13,15,23-25]. At the same time, it would have been rather difficult to find test persons whose characteristics corresponded exactly to the distribution within the basic population.

In total, we evaluated 66 out of 656 diabetes apps within the usability evaluation (Multimedia Appendix 6): 29 apps available exclusively for the operating system iOS, 28 apps available exclusively for the operating system Android, and 9 apps available for both operating systems.

For all main and subcriteria, we averaged the evaluations of all three experts. The values of the main criteria represent the mean of the corresponding subcriteria (Table 6). The total usability score was calculated from all categories, which were determined by means of a 5-point Likert scale.

Analyzing the results, the majority of evaluations were in the range of 3.0 to 4.0, which corresponded to a moderate to good rating of the apps included in the 10% sample. All tested apps received the best rating for the subcriteria “use of understandable semantics” and “simple comprehensibility and interpretability of displayed images and depictions” with a total average value of 4.1 (Table 6). Independent of the operating system, all apps received the worst rating for the subcriterion “fault tolerance” (2.8) followed by “simple recognizability of click-sensitive areas” (3.0). It has to be mentioned that it was only possible to evaluate fault tolerance for apps offering data input (36.4%, 24/66) (Table 6). The values determined for all other subcriteria varied between 3.1 and 4.0. Comparing the total usability score between the different operating systems, we found no clear differences with values varying between 3.3 and 3.4 for both iOS/Android apps. The worst-rated app with a usability score of 2.5 was HealthFile LIVE! (developer: WakefieldSoft LLC) for Android (Multimedia Appendix 6). The best-rated app was Diabetic Recipes Volume I (developer: ECI) for Android with a usability score of 4.1. Password-protected services were offered by an average of 18.2% of the apps.

In the second run, we evaluated the three chosen accessibility features. The results show that their operability was rather limited. The highest values were observable for the screen reader features Voice over (iOS) and Talkback (Android); 25 (86.2%) of the 29 iOS apps offered the ability to read the screen content aloud compared to 19 (67.9%) of the 28 Android apps, and just 4 (44.4%) of the 9 apps designed for both operating systems. The feature “invert colors” showed no considerable improvement of color contrast compared to the results of our evaluation without testing this feature. The results for testing the feature “large type” differed widely. While none of the iOS apps offered this feature, 11/27 (40.7%) Android apps offered contents in large font (Table 6).

While conducting our systematic review, we hypothesized that usability decreases with an increasing number of functions. Hence, we additionally investigated the relationship between the main usability criteria and the number of functions by conducting several correlation analyses. The results shown in Table 7 indicate statistically significant negative relationships between the number of functions and all usability criteria, except “fault tolerance”. Thus, the number of functions and all usability criteria were significantly negative correlated with coefficients varying between −.29 and −.25 implicating that diabetes apps offering a wider range of functions performed worse in terms of usability.

Furthermore, we analyzed the relationship between the usability score and specific functions based on the differences in functionality we found in our systematic review. Therefore, we conducted multiple linear regression analysis to control for potential confounding effects of other functions offered by the same app (Table 8). It showed significant results only for two types of functions. The presence of a documentation function reduced the usability score on average by 0.36 points while the usability score of apps offering an analysis function was on average reduced by 0.21 points. According to R ², 25% of the variance of the usability score was explained by the model. All correlation and regression analyses were conducted with the statistical software Stata 11.1.

The systematic review showed that a large number of diabetes apps are available. Providers may be entering the market as a result of the rising number of patients suffering from diabetes. For users, especially patients, it becomes increasingly difficult to find an app in this plethora of options that is suitable for one’s own needs. This problem is caused by a lack of effective search criteria and filter functions in the app stores. More frequently, apps are chosen that appear first in the search results for diabetes apps. The sorting criteria in the app stores are not apparent. New apps from relatively unknown developers could have difficulties being listed among the first results.

At the same time, many apps offered similar functionalities, mostly a documentation function, which is consistent with earlier findings of Martínez-Pérez et al [3], Eng et al [5], Chomutare et al [6], and Demidowich et al [16]. Differences were found mostly in the design and the menu structure. Additionally, the majority of diabetes apps offered only one or two functions. An application that simultaneously informs and contributes to successful treatment by combining documentation, reminder, and advisory functions was not available as of April 2013. Such a multifunctional app would have a clear additional benefit, especially for newly diagnosed and elderly diabetes patients. At the same time, simple, understandable design, content, and menu navigation are needed. But several apps showed a lack of suitability and usability for its main target group diabetics, which is in accordance with the findings of Demidowich et al [16]. Some were apparently developed without intensive cooperation or prior (usability) tests with patients or professional health care personnel. The obligation for certification as a medical product does currently not exist, even though some diabetes apps are already certified, especially those linked to an external measurement device, eg, iBGStar Diabetes Manager (iOS), Bodytel Mobile (iOS and Android), or Diabetes Companion by mySugr (iOS and Android). Peer review processes of health apps are already offered by several platforms such as iMedical Apps [34], JMIR mHealth [35], or HealthOn [36]. This structure offers substantial and valuable support for users and their decision-making processes, but also in terms of quality assurance and improvement. In contrast, the Google Play Store, as one of the leading app stores, does not currently apply a peer review-based admission procedure for new apps. This lack of certification results in a lack of “[…] demonstrated safety and effectiveness, especially where information and trends are not just presented to patients, but used to make treatment recommendations” [37].

As an example, one app mainly offered labels like “after breakfast” or “after lunch” for the documentation of measured blood glucose values, which implies postprandial states. But, for most diabetics, the blood glucose values before eating are decisive to adjust the amount of insulin. Another app offered the feature to plot a graph labeled “HbA1c (glycated hemoglobin) according to day-time”. This neglects that the HbA1c is a value for long-term blood glucose. As a further example, some apps provided no option to modify, once entered, values at a later point in time. This misconception prevents a subsequent data amendment for example in the run-up of a doctors’ visit. At the same time, there is no opportunity to correct wrong values, for instance, as a result of erroneous data input.

Taking a look into the future, we expect that the data forwarding function, especially to the attending physician, will gain significantly more importance. A regular transmission of data to their physician linked with frequent feedback can be a valuable therapy support, particularly for people in rural regions that are or will become affected by a shortage of doctors [38,39]. Nevertheless, there still exist open questions concerning data security, network coverage, interoperability, documentation requirements, and coverage in health care plans, etc [38,40,41].

Additionally, the automated transmission of measured values in real time from the measuring device to the mobile device will probably spread and is an important driver for the perceived ease of use as El-Gayar et al point out [42]. At the moment, the input of the measured blood glucose values occurs manually in most of the cases, as earlier findings of Eng et al [5] and Chomutare et al [6] confirm. Some manufacturers already offer blood glucose meters that allow real-time data transmission of measured blood glucose data via Bluetooth to a mobile device (eg, GlucoTel [43], iBGStar [44]) (Figure 5). This function simplifies the process of documenting for the patient and, at the same time, it increases the reliability of the entered data and subsequent analysis.

Notwithstanding the functions offered by diabetes apps, their effects on patients’ self-management and, accordingly, on important indicators, as for example the HbA1c value, have to be evaluated. A comprehensive, representative, and long-term study investigating these health effects is lacking so far. But different studies focusing on the outcomes of mobile phone interventions, such as SMS, point out a slightly positive influence as shown in the reviews of Holtz et al [45] and Free et al [46].

As a supplement to our systematic review, we conducted an expert-based usability evaluation to examine the usability of currently available diabetes apps for patients aged 50 or older. Therefore, we focused on the age group with the highest diabetes prevalence. The results show moderate to good evaluations (range 3.0-4.0) for all reviewed usability criteria, which is in accordance with the results of Demidovich et al [16]. An exception was “fault tolerance” (Table 6). The main criteria, “comprehensibility”, rated best with a score of 4.0. In particular, the elderly benefit from easy, understandable semantics and easy, comprehensible, and interpretable images and depictions, due to their frequently limited experience in handling mobile devices and apps. Hence, it can lower inhibition thresholds, especially during the first time of use, and therefore increase acceptance among this age group. The same is true for the influence of “easily understandable feedback” (3.3) and an “intuitive usability” (3.5) (main criterion “usability”). However, these two subcriteria performed worse within our evaluation. The fact that most of the diabetes apps were in English or contained English/foreign language terms (Table 3) diminished the usability especially for non-English-speaking elderly in terms of comprehensibility. This can be seen as one optimization approach for future app development that is comparatively easy to implement.

The evaluation showed moderate results for the main criterion “presentation” (3.5). Our test of three accessibility features indicated a very good operability of the screen readers, especially for Voice over (86.2%) offered by iOS. However, the operability of the features “invert colors” (3.5) and “large type” (21.2%) was rather restricted. Additionally, the minority of diabetes apps (17.8% of the iOS apps) were developed specifically for tablet PCs. However, we assess them as more suitable and user-friendly for elderly diabetes patients due to their larger display and bigger illustrations. With increasing age, cognitive and physical skills are declining, such as eyesight, visual acuity, color vision, contrast detection, and hearing [11]. Especially elderly diabetes patients are often suffering from retinopathy. Shortcomings of diabetes apps concerning the presentation of information (color contrast, size of operating elements, option to flexibly adapt size of operating elements, etc) and the operability of accessibility features are deterring potentially interested patients from using diabetes apps from the outset. Therefore, a barrier-free access is a basic prerequisite for elderly patients to make them use diabetes apps. Against this background, all the other criteria we determined become of secondary importance in terms of usability.

The criterion “fault tolerance” rated worst with a score of 2.8 (Table 6). This means that the available diabetes apps were lacking an efficient fault management (criterion specified in Table 2). Especially inexperienced (elderly) users often have difficulties with inputting data. Some errors are unrecoverable or even cause the application to shut down, as Garcia et al already demonstrated in their analysis [17]. These results have to be viewed with great concern due to the fact that these apps are dealing with medical parameters. This becomes particularly serious if these values provide the basis for further calculations as, for example, the required dose of insulin. Our results demonstrate once again the meaningfulness of an automated transfer of measured values from the blood glucose meter to the mobile device [42]. Additionally, it could be helpful to limit choice to meaningful values, for example, by offering a numeric keyboard to enter blood glucose values.

Our correlation and regression analyses indicated a strong link between usability and the number and kind of functions. In particular, the number of functions and all main usability criteria were significantly negative correlated. These results cast a different light on the aforementioned outcomes of our usability test. Hence, the moderate to good usability scores applied mainly to apps offering a small range of functions. This relation inverted when we looked upon the considerably lower usability scores for multifunctional apps (Multimedia Appendix 6). Considering the special needs of elderly diabetes patients, they would benefit from a comprehensive and easily understandable support as already mentioned above. They are frequently affected by multimorbidity and polypharmacy, particularly after many years of suffering from diabetes [47].

Differed by functions, apps offering a documentation and an analysis function performed worse in terms of usability. This result is surprising as the documentation function is most commonly offered with a share of 53.0%. It can be a valuable support for all diabetes patients measuring and recording blood glucose level regularly. But as interviews with diabetes patients aged 50 or older (conducted in Germany in 2013) have shown, most of them prefer documentation by means of a conventional diary (results not shown). One reason they named was the aforementioned lack of usability and therefore a too complicated and time-consuming handling. Moreover, the use of these two functions is characterized by a higher level of human-technology interaction than, for example, the use of an information function. Of course, this can be accompanied by a wider scope of error sources and usability barriers.

Altogether, the potential of diabetes apps for assisting and supporting diabetes patients aged 50 or older is large. In particular, the target group aged between 50 and 60 years holds great potential as people of this age are already quite familiar with mobile devices and apps [48]. Now, app developers are facing the challenge of taking sufficient account of the usability criteria we examined and addressing those shortcomings. There is no need for a huge number of new app functions. It is more about improving what already exists.

The systematic review and the expert-based usability test were conducted within the project “InnoMedTec”. In that project, we investigate the question: “How should a mobile application be designed to support an effective self-management for diabetes patients aged 50 or older?” Our market analysis provided the basis for a survey among diabetes patients aged 50 or older and physicians, which we conducted in the second half of 2013. Within guided interviews, we investigated the current use, acceptance promoting/inhibiting factors, potentially needed support, and concrete design features for the development of a diabetes app. Merging the results of the systematic market review and the survey, a user- and needs-oriented prototype app for diabetics aged 50 or older will be developed this year. To guarantee usability and needs orientation, the prospective users and usability experts are involved in the product development process right from the beginning. User- and expert-based usability tests are performed regularly. The results are integrated continuously in the app optimization until its finalization.

Despite the huge amount of currently available diabetes apps, most of them offer a small number of similar functionalities. Patients and physicians should be directly involved during the app development to tackle the lack of usability and needs-orientation for its main target group diabetics. We think that data forwarding options and automated transmission of measured values to mobile devices will gain more importance in the future.

The usability of diabetes apps for patients aged 50 or older was moderate to good. But this result applied mainly to apps offering a small range of functions. Multifunctional apps performed considerably worse in terms of usability. Differed by functions, the documentation and analysis function indicated significantly lower usability scores. The operability of accessibility features for diabetes apps was quite limited, except for the feature “screen reader”.