Tinnitus is the perception of a sound in the ears or head in the absence of a corresponding external sound source. It is very prevalent; it has been estimated that 10% to 15% of the adult population experiences tinnitus [1]. Various conditions are associated with developing tinnitus, such as ear disorders [2], exposure to loud noise, the presence of hearing loss, and increasing age [3]. Tinnitus experiences are highly heterogeneous in terms of how it manifests (eg, the types of sounds experienced, how individuals react to their sounds, the associated comorbidities, etc) and how individuals with tinnitus respond to treatment [4]. Although a majority of those with tinnitus are not bothered by their tinnitus, a significant number of people with tinnitus experience distressing tinnitus that affects their quality of life [5]. Nevertheless, tinnitus can affect people in different ways; the most common complaints include annoyance, irritability, fatigue, stress, sleep problems, and trouble concentrating [6]. Moreover, distressing tinnitus is often associated with an increased risk of anxiety and depression [7,8]. Various management strategies are used to help persons with tinnitus, including sound therapy (eg, hearing aids and masking); informational counseling for aiding people’s understanding of tinnitus; and psychological approaches that address unhelpful thought patterns and reactions to tinnitus, such as cognitive behavioral therapy (CBT). Of these, CBT has the highest level of research evidence on reducing tinnitus distress [9,10].

Although the use of CBT is recommended in many tinnitus practice guidelines [11], it is seldomly provided. This is partly due to a lack of trained professionals who can offer CBT for tinnitus in an in-person format. To overcome this barrier, internet-based CBT (ICBT) was developed in the late 1990s [12]. In ICBT, the treatment strategies are offered to individuals with tinnitus as self-help materials that are provided over the internet together with professional guidance [13]. The feasibility and efficacy of such an approach have been demonstrated among several populations in Sweden, Germany, Australia, the United Kingdom [14], and, more recently, the United States [15,16]. In general, studies have shown that nearly 50% to 60% of those who undergo ICBT will experience a clinically significant reduction in tinnitus distress [17,18]. To date, no strong predictors of ICBT outcomes have been identified to indicate who is likely to benefit from ICBT interventions. The predictors of outcomes that were identified when examining the long-term (1 year) outcomes of ICBT in the United Kingdom were higher baseline tinnitus severity, more engagement with the ICBT program (ie, more modules opened), and higher self-reported satisfaction with the intervention [18]. To further explore predictors of outcomes, various univariate and multivariate (ie, logistic and linear) regression models were applied to a combined data set of multiple ICBT studies [19]. These linear and logistic regression models identified education level (linear regression: P=.01; logistic regression: P<.001) and baseline tinnitus severity (linear regression: P<.001; logistic regression: P<.001) to be significant predictor variables that contribute to reductions in tinnitus severity following an ICBT intervention. As per the linear regression model, participants who had received disability allowance showed a 25.30-point less (95% CI −46.35 to −4.24) Tinnitus Functional Index (TFI) score reduction when compared to those who did not have to work less due to tinnitus after adjusting for baseline tinnitus severity and participants’ education levels.Although many other predictors, including age and tinnitus duration [19], were not identified to be significant under these linear models, these variables might have a nonlinear association with the response.

In the last 2 decades, various artificial intelligence and machine learning techniques have been developed and applied to hearing health data. Such approaches have mainly been used for disease profiling, although some studies have focused on the prediction of treatment outcomes [20-24]. It is noteworthy that the intervention trials in audiology and tinnitus research usually involve a few hundred participants and the collection of generally extensive data regarding demographic characteristics and clinical variables. Such a data set with many predictor variables may be best handled by exploratory data mining techniques, such as the use of tree-based models (eg, the random forest [RF] model). Such models tend to perform well even in the presence of multicollinearity among a large number of predictor variables, as these models decorrelate the variables [25]. For example, a recent study that examined various machine learning algorithms for predicting CBT treatment outcomes in the tinnitus population suggested that gradient boosted trees (area under the receiver operating characteristic curve [AUC]=0.89) have the best predictive power [21]. This study found that subjectively perceived tinnitus-related impairment, depression, sleep problems, physical health–related impairments in quality of life, the time spent on completing questionnaires, and educational level highly contributed to the model’s predictions. However, no previous studies have examined the application of artificial intelligence and machine learning techniques to ICBT outcomes in tinnitus research.

To further explore outcome predictors for ICBT, this study aimed to examine the applications of various exploratory data mining techniques based on decision tree models. In particular, we wanted to (1) investigate which types of decision tree models were the most applicable to ICBT outcome prediction (ie, models with the best accuracy and predictive power) and (2) identify the most relevant predictive factors of ICBT outcomes by using the most appropriate decision tree models.

We included 228 participants who previously underwent ICBT for tinnitus and whose data were collected as a part of 3 separate ICBT trials [17,18,26] that were conducted from 2016 to 2018. This study was a secondary analysis of these ICBT intervention studies. Ethical clearance was obtained from the Faculty of Science and Technology Research Ethics Panel of Anglia Ruskin University (reference numbers: FST/FREP/14/478 and FST/FREP/14/478) and the East of England–Cambridge South Research Ethics Committee (reference number: 16/EE/0148) and Health Research Authority (Integrated Research Application System project ID: 195565).

A heterogenous sample of individuals with tinnitus was obtained; thus, the sample represented typical tinnitus populations, as seen in Multimedia Appendix 1. The average age was 55.14 years (SD 12.92 years), and 98 out of the 228 (43%) participants were females. The majority of participants (154/228, 67.5%) had long-standing tinnitus with a mean duration of 17.68 years (SD 19.42 years). Of the 228 participants, 59 (25.9%) had completed high school education, 61 (26.8%) had an undergraduate degree, and only 30 (13.2%) had a postgraduate degree. Approximately 48% (109/228, 47.8%) of the participants experienced tinnitus in both ears, 26.8% (61/228) of them experienced tinnitus in 1 ear, and the others reported experiencing tinnitus in their head or in other locations. The majority (159/228, 69.7%) of participants did not wear hearing aids, and 25.4% (58/228) of them had sought tinnitus treatment previously.

The study participants completed an 8-week ICBT intervention that was presented in a self-help format [13,27]. The intervention was administered by using a secure e-platform [28,29]. During this 8-week period, participants were presented with 2 to 3 learning modules that contained various elements of CBT that were specifically adapted for tinnitus, including applied relaxation, cognitive restructuring, and imagery. The digital materials were presented by using text, images, and videos. In addition, various exercises were presented in these learning modules to improve engagement.

The baseline data collection included an extensive questionnaire that focused on demographics and tinnitus-related and treatment-related information. Outcome data were gathered by using standardized primary and secondary self-reported questionnaires, which were administered before (baseline), during (weekly), and after the intervention. The primary outcome was a change in tinnitus severity, as measured by the TFI [30]. The secondary outcome measures included the Insomnia Severity Index [31] (a measure of insomnia), the Generalized Anxiety Disorder-7 [32] (a measure of anxiety); the Patient Health Questionnaire-9 [33] (a measure of depressive symptoms); the Hearing Handicap Inventory for Adults Screening version [34] (a measure of self-reported hearing disability); the Hyperacusis Questionnaire [35], which was used to assess the presence hyperacusis (ie, reduced tolerance to everyday sounds); the Cognitive Failures Questionnaire [36], which was used to assess cognitive functions; and the Satisfaction with Life Scales [37], which were used to assess global life satisfaction.

Undertaking ICBT significantly reduced tinnitus severity scores (t₂₂₇=16.37; P<.001) from a mean baseline severity score of 57.93 (SD 19.17) to a mean post-ICBT severity score of 34.22 (SD 22.78), as measured by the TFI. A clinically significant 13-point change in TFI scores was achieved by 150 of the 228 participants (65.8%) after the intervention.

Table 1 contains the model evaluation information of all 6 decision tree classifiers that were based on the test data. Following training via the 3-fold cross-validation method, the mean accuracies of the six decision tree classifiers ranged between a minimum of 56.3% (the C5.0 model) to a maximum of 71.8% (the GB model). Model predictions showed variations in their sensitivity (range 68.6%-78.3%), specificity (range 31.1%-64%), and AUC values (range 0.52-0.69).

None of the six models were considered robust, as their AUC values were below 0.80. As the CART and GB classifiers were found to be superior compared to the other four models when considering all evaluation measurements as a whole (accuracy, sensitivity, specificity, and the AUC), these two models were further examined.

The SHAP framework was applied to the CART and GB classifiers to estimate each predictor variable's importance in predicting ICBT outcomes (Figure 1). Variables with larger SHAP values are relatively more important in terms of their contributions (ie, feature contribution) to a model prediction. Education level (average SHAP values: GB model=0.053; CART model=0.079) was identified as the most important influencing factor in both models. Although they were not ranked in the same order, the other features that ranked within the top 10 features for both models were employment type (average SHAP values: GB model=0.019; CART model=0.051), hearing aid usage (average SHAP values: GB model=0.019; CART model=0.040), and tinnitus maskability (average SHAP value: GB model=0.015; CART model=0.040). The differences between these models were that the GB model ranked baseline tinnitus severity (average SHAP value=0.041), how often tinnitus is heard (average SHAP value=0.022), insomnia (average SHAP value=0.021), the use of medication for tinnitus (average SHAP value=0.024), and the presence of a psychological condition (average SHAP value=0.015) among the top 10 features, whereas the CART model ranked the presence of multiple sounds (average SHAP value=0.049), loud noise exposure (average SHAP value=0.042), and tinnitus location (average SHAP value=0.023) as key features.

Figures 2 and 3 present the effect that each feature category had on the outcome variable, as decided by the best GB and CART decision tree models. Features’ impacts on the two classes are presented in 2 separate plots for each feature (“1” indicates the effect on the successful treatment class and “0” indicates the effect on the unsuccessful treatment class). Positive SHAP values in each successful treatment group indicate a higher log odd of achieving a 13-point or more tinnitus severity score reduction (ie, on the TFI for a given category) for a feature and vice versa. This log odd is relative to the training set average. Figures 2 and 3 both depict positive SHAP values for the participants who had a vocational training degree or a master’s degree (or above), those with higher levels of education (postgraduate degrees), and those who were using a hearing aid in only 1 ear. As per the GB model, this reduction was more likely to occur for participants with insomnia (scores of 14 or less on the Insomnia Severity Index), a psychological condition, and tinnitus that can be described as a buzzing sound and for participants who had median baseline tinnitus severity scores (ie, pre-TFI assessment) of >55.2.

Figure 4 presents the final decision tree model, which has 10 nodes. A detailed explanation is provided in Multimedia Appendix 3. In this model, homogenous groups were formed by creating binary splits at each node. The decision nodes represent the treatment groups (either 0 or 1) that were the most likely to achieve tinnitus severity reduction, and the decisions in the tree were based on the characteristics of each group. These characteristics were represented by each branch (based on their corresponding feature values) of the tree. This model showed that higher education level, tinnitus maskability, hearing aid usage, the presence of multiple tinnitus sounds, loud noise exposure, employment type, the presence of hearing loss, and tinnitus location were important factors for determining treatment outcomes. The following participant groups had at least an 85% chance of achieving a TFI score reduction of 13 points or more:

In this study, we applied 6 different decision tree models to ICBT data. Although none of the six models attained an excellent or outstanding status, the CART and GB models’ discriminative power can be considered to be satisfactory, given the moderate sample size with just 33 predictive factors (features). This is consistent with a recent study that used 10 decision tree models to predict the outcomes of CBT for patients with tinnitus (N=1416) and found that the GB model with 26 predictive factors had the best predictive power; it had an AUC value of 0.89 [21].

Further work however is needed in this area to determine which models and how many vital factors may result in an optimal predictive model. A larger sample size would likely improve the results. However, we are not sure if just adding factors would be helpful. This is because Niemann et al [21] included 205 factors in their analysis, and of these, only 26 were helpful in achieving optimal results. Moreover, of the 26 factors, only a handful had the most considerable effect. For instance, only 1 factor (ie, tinnitus impairment in terms of loudness, frequency, and distress) resulted in an AUC of 0.79, only 3 features resulted in an AUC of 0.85, and only 8 factors resulted in an AUC of 0.85. These results indicate that including key factors with high predictive power may be a better approach than just adding all of the possible factors.

Among the best decision tree models (ie, the CART and GB models), various factors were found to be critical predictors of ICBT outcomes. These included demographics (ie, education level, employment type, and the presence of a psychological condition), tinnitus and hearing-related factors (ie, baseline tinnitus severity, tinnitus location, how often tinnitus is heard, a buzzing type of tinnitus, tinnitus maskability, and hearing loss type), treatment-related factors (ie, hearing aid usage), and clinical factors (ie, insomnia). However, education level was the most notable predictor among these.

Participants who had a master’s degree or above had an 88% chance of achieving a successful outcome. This is understandable, as the ability to read, understand, and follow instructions is key for undergoing self-help interventions. However, it is likely that the way in which the materials were written may also have played a role. For instance, UK ICBT materials are written at a ninth-grade reading level [49], which may require higher literacy skills. However, these materials have been rewritten at a sixth-grade reading level and below [49] to ensure accessibility for those with lower education levels. More and more people, including those with lower education, are using the internet and participating in internet-based treatments, particularly due to the constraints that have been placed on health care during the COVID-19 pandemic [50-52]. In our sample, over 85% (198/228, 86.8%) of the participants had a below–master-level education. This highlights the need for making ICBT more accessible to increase the chances of achieving improved outcomes for those with lower education levels.

Baseline tinnitus severity was found to be another critical factor for predicting ICBT outcomes in the GB model. Our previous studies on 1-year outcomes [53] and our previous application of univariate and multivariate analyses to this study’s sample [19] identified baseline tinnitus severity as a critical predictive variable. In the Niemann et al [21] study, tinnitus loudness, frequency, and distress, which were measured by using a visual analog scale, were found to be key predictive factors. Further, tinnitus distress, which was measured by using the German version of the Tinnitus Questionnaire (a tool that is comparable to the TFI in this study), was not found to be the key predictive factor. However, based on both clinical experiences and findings from many previous studies, baseline tinnitus severity is an important factor for determining treatment outcomes. The clinical factors depression and anxiety were among the key predictive factors in the GB model. A recent clinical trial conducted by Beukes et al [54], as well as the Niemann et al [21] study, determined that those with high levels of depression had a better chance of achieving success.

Although various other tinnitus and hearing-related variables could have played a role in determining the outcomes of ICBT, the predictive power of our models was relatively low based on the AUC values. Nevertheless, it would be useful for hearing health care professionals to examine these factors when deciding on the candidacy of self-help psychological interventions such as ICBT. Moreover, it would be useful for future studies to examine any additional factors (eg, health literacy) that may have a bearing on ICBT outcomes.

Although this study is among the first to apply data mining models to ICBT data, it has several limitations. The sample size was limited, and this may have contributed to the low predictive accuracies of the models. The exploratory decision tree models worked better when including a large number of predictive factors. In this study, we only included 33 predictive factors in our models, and this may have limited the performance of our models. Further, we may have missed some important factors (eg, health literacy) that have a bearing on ICBT outcomes.

The inclusion and exclusion criteria that were used in the three trials from which our data were generated may have resulted in a sample with high tinnitus severity levels that may not be representative of the general tinnitus population. This may have also contributed to our limited key findings. Future studies could include more extensive samples of heterogeneous patients with tinnitus as well as all of the possible predictive factors that could help with improving our models’ predictive power. Moreover, developing nonlinear classifiers with artificial neural networks and support vector machines could help with achieving higher prediction accuracies and should be examined in future studies.

In conclusion, tree models, especially the CART and GB models, appear to be promising in predicting ICBT outcomes. Future studies should be undertaken with larger sample sizes and include a more comprehensive range of predictive factors to improve their models’ predictive power.