Patients were recruited from 4 medical institutions across Japan with significant experience in conducting clinical trials (Textbox 1). Recruitment was conducted by attending physicians during the patients’ regular consultations at outpatient clinics. Before patients enrolled in the study, the study’s supervising physician presented the review board–approved consent form to each patient and explained the contents, which included information about diabetes, how the app functions, how to operate the app, data collection, risks and benefits of participating, and compensation. Once the study was explained and it was determined that patients understood the explanation, patients were requested to participate in the study, and consent given by the patient’s own free will was collected in writing via the consent form.

Patients who met the inclusion criteria (Textbox 2) and did not meet the exclusion criteria (Textbox 3) were eligible for the study. To focus the study on patients most likely to benefit and most suitable for participation, the criteria selected patients who have inadequate glycemic control but have not recently changed their blood glucose management. Minors and in-patient participants were not eligible due to concerns of ethics and safety. Participants were also limited to those who own a smartphone. Patients who might expect changes in their blood glucose levels during the study, such as women who are already pregnant or who are planning on becoming pregnant, were not eligible. The study focused on the population with T2DM, excluding patients with type 1 diabetes mellitus due to differences in the pathophysiology between the 2 diseases.

Following obtaining consent and a subsequent 2-week observational period, patients were screened against the criteria, and eligible patients were enrolled in the study. The 24-week study began with a 12-week treatment period that included consultations every 4 weeks, followed by a 12-week treatment sustainment period that included 1 consultation. Patients were randomized into either the intervention group or the control group in a 1:1 ratio by block randomization (block size of 4) with baseline hemoglobin A_1c (HbA_1c; less than 8% or 8% or greater) and hypoglycemic medication use (yes or no) as stratification factors. Patients were assigned an ID by the registration center for data tracking.

This study was a multi-institutional, prospective, and unblinded 2-armed randomized controlled trial carried out across 4 medical institutions in Tokyo, Japan. We chose to conduct an unblinded trial because we predicted that the use of a placebo app with minimal function would lead to the control group unblinding themselves.

We provided both groups with Bluetooth-enabled blood pressure cuffs (A&D, UC-352BLE) and scales (A&D, UA-651BLE). Patients in the intervention group were instructed to install the SMC-01 app onto their smartphones and use it to track diet, medication adherence, exercise, body weight, and blood pressure. Recordings of blood pressure and weight were collected automatically through the Bluetooth-enabled devices but could also be input manually. The rest of the data was input manually.

Patients in the control group recorded information about their diabetes self-management in a journal. They were asked to record which meals they ate (breakfast, lunch, dinner, or snack), whether they took their diabetes medication, whether they exercised and what type of exercise (aerobic or resistance training), their weight in kilograms, and their blood pressure along with when they measured it (morning, noon, or evening). Unlike the SMC-01 group, the control group did not record the content of meals, and only recorded whether they ate a meal or not. The format of a journal entry is shown in Figure 1.

SMC-01 (Figure 2) was designed to assist the treatment and management of T2DM, complementing, not replacing, physicians. SMC-01 breaks down goals set by physicians into smaller and specific actions and sends them to patients as daily feedback and challenges. Through this, it aims to increase the likelihood of long-term behavioral change by making goals more achievable and by addressing the issue of patients not knowing how to implement their physician’s advice.

SMC-01 consists of 4 main functions: facilitating data input, providing feedback in response to the data input, sending reminders, and setting daily challenges.

Patients receive feedback per Japanese diabetes treatment guidelines based on their daily behavior. This is implemented using Sketto, a chatbot within SMC-01 that sends messages to the patient based on patient-entered data. The content of these messages depends largely on whether the data the patient enters is within normal limits according to Japanese diabetes treatment guidelines. If the patient’s data are within normal limits, Sketto praises the patient for their hard work and encourages them to continue their diabetes self-management as well as their data logging. On the other hand, if the patient’s data are outside of normal limits, Sketto warns the patient about the possible implications of the measurement, such as poor health outcomes, and encourages the patient to seek the advice of a health professional. For example, when the patient records their weight, Sketto sends a message providing positive feedback on healthy lifestyle choices or caution on unhealthy lifestyle choices depending on the patient’s BMI, which SMC-01 calculates from the patient’s height registered at the initial set-up. When the patient records his or her meal data, Sketto sends a message depending on the patient’s reported satiety level and the time of day. For medication adherence, Sketto sends reminders if patients forget to record that they took their medication, and praises patients for recording medication adherence several days in a row. Examples of some of these messages are shown in Figure 3.

Sketto also sends patients health-related articles to increase patients’ knowledge about diabetes and health in general. Unlike the feedback described above, these articles are not personalized based on patient data. Patients can also review their records in the form of reports. These reports show the patient’s self-management over time, such as how many days the patient has exercised, their change in weight, and the days they took their medication. The supervising physicians of the study could review the patients’ self-management by viewing the app’s screen. Review of data input into the app by the study’s supervising physicians could be accomplished at any time, but the review was mainly carried out during visits 3 through 6.

Every morning, Sketto sends patients mini-challenges within the app that are designed to augment the patients’ assigned diet, medication adherence, and exercise-based behavior modifications. These challenges begin at the lowest level, and, depending on whether the patient succeeds, a similar challenge that is slightly harder or easier may be assigned the following day. Otherwise, a mini-challenge from another category will be assigned. The patient can also choose to repeat the same challenge on the following day. Examples of mini-challenges can be found in Table S1 in Multimedia Appendix 1.

Initial measurements were made at visit 1 at the beginning of the observation period. The study (Figure 4) included 6 in-person visits, a 2-week observation period, a 12-week intervention period, and a 12-week intervention sustainment period. If patients withdrew from the study at any point, observation and testing were carried out when the patient came to the health care facility to withdraw from the study. A detailed timeline of the study can be found in Table S2 in Multimedia Appendix 1.

From visit 1 until visit 6 or until measurement of efficacy outcomes had been completed, physicians in the study were prohibited from providing specific calorie intake and exercise amount goals to patients. Nonspecific advice, such as general directions to eat more slowly or eat less salt, was permitted. These limitations were set to ensure that changes in exercise amount and calorie intake could be attributed to the app and not instructions from physicians.

The primary outcome was the between-arm difference in change in HbA_1c values between the baseline and the end of the intervention period.

The study evaluated the following secondary measurements:

Self-maintenance adherence rates were calculated as follows:

The study was registered in the Japan Registry of Clinical Trials (jRCT2032200033) on May 17, 2020. It was approved by the research ethics committee of The Institute for Adult Diseases, Asahi Life Foundation (179-8). We conducted the study per the Ministerial Ordinance on Standards for Conducting Clinical Trials of Medical Devices (The Ministerial Ordinance on Good Clinical Practice). The study has been reported as per the eHealth CONSORT (Consolidated Standards of Reporting Trials) statement. We obtained written informed consent from all participants before the study, and study data have been anonymized. Participants were compensated JP ¥10,000 (roughly US $64) for their time.

We set the mean difference in the change in HbA_1c from the baseline that we were designing the study to detect to 0.5, as this is generally considered a clinically significant change in HbA_1c [26,27]. The SD common to both groups was set to 1 based on values from previous studies [23,28,29], and the 2-sided significance level was 5% with 90% power. The resulting required sample size is 86 for each group. Taking into account the likelihood of dropouts, the target sample size for each group was set at 100 cases (200 cases in total).

Per the intention-to-treat (ITT) analysis principle, all randomly assigned patients were included in the full analysis set (FAS). In the case that a patient was excluded from the analysis, justification for the exclusion was documented. Patients in the FAS who met all of the following criteria that did not violate the selection exclusion criteria and protocol were included in the per protocol set (PPS):

All patients that participated in the intervention period of the study were included in the safety analysis set. Demographic data and baseline values were compared between groups using the Fisher exact test for discrete values and a 2-tailed t test for continuous values.

As the primary analysis, an analysis of covariance was performed on the change in HbA_1c from baseline to the end of the intervention period (wk 12) in the FAS, with baseline values as the covariate, and the between-group differences and their 95% CIs were calculated. As a sensitivity analysis, the same analysis was performed for the PPS and for patients with HbA_1c measured at all time points (wk 0, 4, 8, 12, and 24). In addition, the FAS was analyzed with a linear mixed effects model that included all time points. The objective variable was the change in HbA_1c from baseline, the random effect was the patient, and the fixed effects were group, time point, group × time point interaction, and baseline HbA_1c. The amount of change for each group, the between-group difference in change, and their 95% CIs at each time point were calculated.

As a secondary analysis, an analysis of covariance was performed in the FAS on the change from baseline in HbA_1c, fasting blood glucose, fasting insulin, and fasting intact-proinsulin or insulin ratio at the end of the intervention period (wk 12) and at the end of the intervention sustainment period (wk 24) with baseline values as covariates. In addition, summary statistics of change and measurements were calculated for each time point (wk 0, 4, 8, 12, and 24) for each group. The percentage of HbA_1c less than 7% at each time point (wk 4, 8, 12, and 24) for each group was also calculated. However, the denominator was the FAS population excluding patients with HbA_1c less than 7% at baseline. Self-maintenance adherence rates for diet, exercise, and medication were assessed in the following time periods and compared between groups by a 2-tailed t test: week 0 to week 4, week 4 to week 8, week 8 to week 12, week 12 to week 24, week 0 to week 12, week 0 to week 24.

In addition, the same analyses on HbA_1c levels were performed for the following subgroups as in the primary and secondary analyses.

For analysis of safety for the safety analysis set, adverse events were classified for each group using system organ classes and preferred terms of the Japanese version of the Medical Dictionary for Regulatory Activities (version 23.0) [18], and the number of patients and occurrence rate for each system organ class or preferred term categories were tabulated. Additionally, the occurrence rates were compared between groups by Fisher exact test. For bugs in the SMC-01 group, the number of patients with bugs, occurrence rate, and the number of cases were tabulated. For each clinical test item and vital sign, levels and the difference from the beginning of the intervention period were calculated at each time point in each group as summary statistics.

All statistical tests were 2-tailed, with a significance level of 5%. CIs were calculated as 2-tailed 95% CIs. SAS (version 9.4; SAS Institute Inc) was used for statistical analysis.

This study was conducted from May 11, 2020, to April 5, 2021. In total, 210 participants were randomly assigned between groups, yielding 107 (intervention) and 103 (control) at baseline, 106 (intervention) and 102 (control) at the end of 12 weeks, and 105 (intervention) and 101 (control) at the end of 24 weeks (Figure 5). Patients were well matched in baseline characteristics (Table 1), although urine albumin or creatinine ratio was significantly higher in the intervention group than in the control group (42.50 mg/g of Cr in the intervention group versus 20.82 mg/g of Cr in the control group, P=.01).

Overall adherence rates were high during the intervention period (Table 2), at 82.28% in the SMC-01 group and 79.35% in the control group (P=.20). Average diet adherence rates were significantly higher in the control group than in the SMC-01 group at 4 weeks (96.39%, SD 10.67% and 99.01%, SD 2.42%, respectively; P=.02) and week 24 (91.57%, SD 22.44% and 97.63%, SD 10.37%, respectively; P=.01). Medication adherence rates were similar in both groups, with no statistically significant differences found at any time point (all P>.05). Average exercise self-management adherence rates were significantly higher in the SMC-01 group than in the control group, with 60.79% (SD 37.22%) and 49.23% (SD 37.92%; P=.03) at 4 weeks, 60.32% (37.24%) and 49.33% (38.52%; P=.04) at 8 weeks, and 59.29% (35.95%) and 47.34% (37.01%; P=.03) at the end of the intervention period.

Based on the analysis of covariance (Table 3), the change in HbA_1c from baseline to the end of the intervention period (12 wk) was –0.05% (95% CI –0.14% to 0.04%) in the SMC-01 group and 0.06% (95% CI –0.04% to 0.15%) in the control group, with a between-group difference (SMC-01 group – control group) of –0.11% (95% CI –0.24% to 0.03%, P=.11). The difference is not statistically significant.

Secondary analysis of the change in HbA_1c from baseline to the end of the intervention sustainment period (24 wk) revealed a between-group difference (SMC-01 group – control group) of –0.16% (95% CI –0.36% to 0.03%; P=.11). The difference is not statistically significant.

Secondary analysis of the change in fasting blood glucose from baseline to the end of the intervention period (12 wk) yielded a between-group difference (SMC-01 group – control group) of –4 (95% CI –10.6 to 2.6; P=.23). From baseline to the end of the intervention sustainment period (24 wk) the between-group difference (SMC-01 group – control group) was –4.8 (95% CI –12.6 to 3; P=.23). The differences are not statistically significant.

Secondary analysis of the change in fasting insulin from baseline to the end of the intervention period (12 wk) yielded a between-group difference (SMC-01 group – control group) of 0.24 (95% CI –0.64 to 1.13; P=.59). From baseline to the end of the intervention sustainment period (24 wk) the between-group difference (SMC-01 group – control group) was 0.19 (95% CI –0.47 to 0.85; P=.56). The differences are not statistically significant.

Secondary analysis of the change in fasting intact-proinsulin or insulin ratio from baseline to the end of the intervention period (12 wk) yielded a between-group difference (SMC-01 group – control group) of –0.01 (95% CI –0.04 to 0.02; P=.44). From baseline to the end of the intervention sustainment period (24 wk) the between-group difference (SMC-01 group – control group) was –0.01 (95% CI –0.03 to 0.01; P=.42). The differences are not statistically significant.

In the sensitivity analysis of the PPS and the complete set, ANCOVA of the between-group difference in the average change in HbA_1c from baseline to the end of the intervention period (wk 12) was not statistically significant (P=.14 and P=.11). Mixed model estimates of the change in HbA_1c from baseline in the FAS yielded a statistically significant difference at week 8 of –0.12 points (P=.04) and week 12 of –0.12 points (P=.04). The full sensitivity analysis results have been included in Multimedia Appendix 1.

In subgroup analysis of change in HbA_1c from baseline to the end of the intervention period (wk 12), statistically significant differences were found in the aged younger than 65 years subgroup (estimated values in the SMC-01 group and control group were –0.07% and 0.08%, respectively, P=.04, same order in subsequent listings) and the baseline HbA_1c less than 8% subgroup (–0.04% and 0.11%; P=.04). Subgroup analysis of change in HbA_1c from baseline to the end of the intervention sustainment period (wk 24) revealed no statistically significant difference between the 2 groups at any time point. The full subgroup analysis results have been included in Multimedia Appendix 1.

The results of the safety analysis are included in Multimedia Appendix 1. There were no adverse events or deaths caused by the device under study.

We did not find any statistically significant difference in outcomes between the SMC-01 and control groups. Further, HbA_1c values essentially did not change in either group. Patients who participate in clinical trials normally experience a change in health outcomes due to the Hawthorne effect and the placebo effect, regardless of the intervention. These effects in this study may have been small because this trial was conducted during the COVID-19 pandemic, when people were advised to stay home, possibly reducing changes in behavior such as exercise.

We can assess possible reasons for the failure of the intervention to change outcomes using behavioral change theory, specifically, the Theory of Planned Behavior [30]. When designing a lifestyle intervention, it is critical to view it as requiring a behavioral change, a perspective that was not used in the SMC-01 intervention. The first step is to clearly define the desired change in behavior. In this study, the desired change was a loose collection of behaviors associated with following physician guidance on a healthy lifestyle. This may have been too vague to be actionable compared to past successful interventions in diet and exercise that defined concrete actions [31-33]. The next consideration is the baseline lifestyle of the patients. It is possible that patients were already following physician lifestyle instructions, and that there was not much need for further lifestyle intervention via SMC-01.

Intention is key in the Theory of Planned Behavior framework. The intervention sought to have patients make a behavior change large enough to produce a significant change in glycemic control. Behavioral change is difficult for most people and requires strong motivation, and motivation was not evaluated as one of the inclusion criteria of this trial. If a patient does not intend to change or perform a behavior, they are very unlikely to do so [30]. Other studies have had success screening for patients who are in the contemplation, preparation, or action stages of the transtheoretical model of behavioral change, increasing the likelihood of strong motivation [34-36]. A 2015 study showed that 92% of Japanese patients with T2DM met this criterion [37]. A different intervention would be appropriate for patients who are not motivated to change [38]. Our study did not specify any screening test for high intention.

Another concern is the lack of clear information about patient behavior. The study examined elements of patient behavior such as app usage but was highly focused on the ultimate health changes rather than behavior. We can assume that the desired behavioral change would have led to improved glycemic control, as has been shown in many other studies [31,32,39], but we do not have enough data to confirm whether their behavior truly changed. Future studies of lifestyle intervention should carefully investigate how to measure behavioral changes. Additionally, SMC-01 may not have been used as intended due to problems with the protocol and its implementation. There may have been insufficient intervention design features. The study found high levels of usage, and yet the intervention did not change glycemic control. In this study, usage was measured as the percentage of days for which data was input at least once. User engagement with other features was not measured. These usage measurements may not have been the appropriate indicators of effective usage of SMC-01. Studies have shown that simply recording data does not improve health care outcomes [40,41]. A review of 32 studies found that adherence rates in mHealth interventions were inconsistent [42]. Screen-time measurements may be used to determine how often patients accessed certain features, providing insight into patient engagement with feedback and whether the patient truly received the intervention.

Another concern is possible insufficient data entry. Objective health data are crucial to mHealth interventions, as self-reported data are often unreliable [43,44]. Automatic data collection objectively measures behavior and was used for the measurement of weight and blood pressure in this study. These are intermediate health changes on the way to changed glycemic control, not behaviors. Future studies should apply automatic data collection of behavioral change. Finally, there may have been insufficient behavioral change information and directions suggested by SMC-01. Closed feedback loops form the backbone of mHealth interventions, but this feedback must be personalized and actionable [45-47].

We do not have sufficient data from this study to determine which, if any, of these possible issues contributed to the outcome. In future studies, addressing these possible issues during the study and app design phase is recommended. In future app design, a theoretical framework of behavioral change can be used to guide the implementation of features. It has been shown that basing interventions on behavior change theory improves health outcomes [47].

This study has some limitations. It was conducted in a Japanese population, and there are known differences between Japanese and other populations about lifestyle and the pathophysiology of T2DM [48,49]. This study was limited to patients who owned smartphones, which may have led to biases related to digital literacy and socioeconomic status. Social desirability bias also likely influenced results due to a lack of blinding.

To our knowledge, this study was the first trial in Japan that sought to obtain official approval for a smartphone app for T2DM as a medical device and will serve as a foundation for the design of future apps and clinical trials seeking medical device registration. mHealth devices for the management of chronic diseases will continue to improve, and future studies in Japan should continue to focus on gaining official approval. Information from this trial, along with the discussion of possible shortfalls, should help improve future efforts.

No statistically significant between-arm difference was found in the change in HbA_1c from baseline to the end of the 12-week intervention period. Future studies should improve clarity regarding the theory of the intervention, improve the methods used, and ensure the measurement of behavior and use throughout the study.