We conducted both a CEA with three measures of improved social participation as outcome and a cost-utility analysis (CUA) with quality-adjusted life years (QALYs) gained as outcome. The outcome measures are described in more detail below.

The outcome of interest in the CEA was social participation. Social participation was operationalized in three ways: (1) objective social participation as the amount of time spent with others, (2) subjective social participation as momentary anxiety, and (3) subjective social participation as momentary paranoia. Momentary in this context meant that it was measured in real life during social company. All three outcomes were assessed in real time using the ecological sampling method (ESM). ESM is a structured diary method in which individuals are asked in daily life to report their thoughts, feelings, and symptoms, as well as the appraisal of the present social context. To that end, all participants carried an electronic device (PsyMate) for the ESM assessments. The device beeped at semirandom moments 10 times a day over 6 days. At each beep, the device collected self-assessments on a 7-point Likert scale ranging from 1 (not at all) to 7 (very). A positive treatment response on each of the three outcome measures was defined as an improvement of at least 20% at 6 months follow-up relative to the patient’s baseline score.

The outcome in the CUA was the QALY derived from the GPTS [13]. The GPTS is an established broad measure of paranoid-delusional functioning that has long been used as an outcome measure. This instrument was chosen to be able to compare results with earlier CUA research on the subject. Mean GPTS scores at each measurement were first converted into the standard mean difference (SMD) by dividing the raw mean change scores by the SD of the GPTS at baseline in the control condition. In a next step, we used Sanderson et al’s conversion factor [14] of 0.1835 (ie, the average of 0.209 using a rating scale and 0.158 using time trade-off), such that a change of 1 standard unit (ie, SMD) on the GPTS is equal to a corresponding change of 0.18 utility. The utility is a quality of life valuation and is needed to compute QALY gains in the VR-CBT condition relative to the TAU condition over the full 6 months between baseline and follow-up.

Societal costs were computed by adding (1) the direct medical costs of health care services use including the costs of antipsychotic medication and, in the experimental condition, the additional costs of adjunctive VR-CBT treatment; (2) direct nonmedical costs of travel; and (3) indirect costs stemming from lower productivity. For each participant, cost data over the last 3 months were collected at each of three measurement points. Resource use data, for costing, were collected using the Trimbos Institute and Institute of Medical Technology Assessment Questionnaire for Costs Associated with Psychiatric Illness (TiC-P) [15]. The TiC-P is the most widely used health service interview in the Netherlands. It consists of questions on the number of contacts by type of health care provider and questions on productivity losses. A health service questionnaire is a valid and reliable method for quantifying costs in trial-based economic evaluations in health care [16]. A cross-validation sample comparing TiC-P self-report to electronic patient files showed all data to be reliable, except for the number of reported sessions with a psychologist (data available upon request from first author). Not all patients had understood that they needed to incorporate the 16 VR-CBT sessions into their TiC-P self-report. This information was, therefore, 100% cross-checked using electronic patient files. The main cost driver was admission to psychiatric hospitals, so the number of days admitted to a psychiatric hospital was also 100% cross-checked against electronic patient files and corrected where needed.

Direct medical costs were calculated by multiplying health service units (eg, sessions, visits, and hospital days) with their standard economic cost price (see Multimedia Appendix 1). We also added the medication costs, consisting of antipsychotic and antidepressant medication. Corresponding costs were calculated as the cost price per standard daily dose, as reported in the Dutch Pharmaceutical Compass [17], multiplied by the number of prescription days, plus the pharmacist’s dispensing costs of €6 per monthly prescription or €12 for a first-time prescription [18].

For VR therapy hardware, software and training costs were calculated. Total yearly costs for one VR system was €23,995, according to CleVR BV, a company who builds VR sets. Yearly costs for training and supervision of the psychologists was €13,400. Per-patient costs per 16 VR-CBT treatment sessions was €373.95.

Travel costs arose when participants had to make return trips for receiving health care at health services. Travel costs were computed as the average distance to a health service (7 km) multiplied by the costs per km (€0.21) [18].

Research assistants monitored changes in the participants’ work status at baseline and at 3 and 6 months postbaseline using the TiC-P. Productivity losses in paid work were calculated according to the human capital approach [19], reflecting changes in the contractual number of hours worked per week and adjusting these for work-loss days arising from sick leave over the full period of 6 months using gender-specific hourly productivity costs [18]. Costs were originally expressed in Euros for the reference year 2014, but indexed to 2015 using the consumer price index as reported by Statistics Netherlands. In the reference year 2015, 1 Euro in the Netherlands equaled 1.235 US$.

Following the CONSORT (Consolidated Standards of Reporting Trials) and CHEERS (Consolidated Health Economic Evaluation Reporting Standards) guidelines, all our analyses adhered to the intention-to-treat principle. To that end, missing values were imputed using multiply imputed chained equations (MICE) for nonparametric data with M of 100 bootstraps for each incomplete variable. Baseline variables predictive of effects (ie, QALYs and treatment response) were used for imputation, such as baseline data of the variable with missing values, treatment condition, ethnicity, education, sex, age, and safety behaviors at baseline. Safety behaviors, such as avoiding eye contact or escaping from social situations, were measured using the Safety Behaviour Questionnaire-Persecutory Delusions (SBQ-PD) [20]. Time spent with others showed a large difference at baseline despite randomization and was added as covariate in the CEA where time spent with others was used as the treatment response outcome of interest.

Both the CUA and CEA were conducted from the societal perspective. In each of these analyses, the incremental cost-effectiveness ratios (ICERs) were calculated as the between-group cost difference divided by the between-group effect difference. The ICER is interpreted as the additional costs per additional unit effect (ie, per additional treatment responder; per QALY gained). Cost and effect differences were obtained from seemingly unrelated regression equations of costs and effects, thus allowing for correlated residuals in the equations. The seemingly unrelated regression equations (SURE) models were bootstrapped 5000 times. In each bootstrap step, the mean cost differences and the mean outcome differences were computed and these were plotted on the cost-effectiveness plane. Finally, cost-effectiveness acceptability curves (CEACs) were graphed. CEACs inform decision makers about the likelihood that an intervention is deemed cost-effective, given a range of willingness-to-pay ceilings for gaining 1 QALY and gaining 1 treatment responder. All analyses were conducted in Stata, version 13.1 (StataCorp).

The following sensitivity analyses were carried out. First, a sensitivity analysis was done including safety behavior at baseline as a covariate because despite randomization there was a significant difference at baseline, and it was found to be the main mediator in reducing paranoid ideation [10]. Second, a sensitivity analysis was done including psychiatric admission costs at baseline as a covariate because there was a large difference between groups at baseline. Third, a sensitivity analysis was done including both safety behavior at baseline and psychiatric admission costs at baseline as covariates in the model.

The treatment response rate regarding the time spent with others was 13 patients out of 58 (22%) in the control group and 24 patients out of 58 (41%) in the experimental group. The baseline-adjusted between-group difference between the response rates (ie, the incremental effect) was 0.23, which was statistically significant (SE=0.076, t₁₁₃=3.07, 95% CI 0.08-0.38, P=.003).

The treatment response rate with regard to momentary anxiety was 17 patients out of 58 (29%) in the control group and 24 patients out of 58 (41%) in the experimental group. The between-group difference between the treatment response rates (ie, incremental effect) was 0.12, but this difference was not statistically significant (SE=0.089, t₁₁₄=1.36, 95% CI –0.055 to 0.290, P=.18).

The treatment response rate in momentary GPTS paranoia was 11 patients out of 58 (19%) in the control group and 28 patients out of 58 (48%) in the experimental group. The between-group difference in the response rates was 0.29 and was statistically significant (SE=0.0841, t₁₁₄=3.48, 95% CI 0.126-0.460, P=.001).

The SMD of GPTS paranoia was 0.523, which was statistically significant (SE=0.120, t₁₁₄=4.37, 95% CI 0.285-0.760, P<.001). Using Sanderson et al’s conversion factor [14] of 0.1835 and taking into account a follow-up period of half a year, this became a QALY gain of 0.048 (0.523 × 0.1835 × 0.5) favoring the VR-CBT condition and this was statistically significant (SE=0.011, t₁₁₄=4.37, 95% CI 0.026-0.069, P<.001).

As can be seen in Table 2, the average per-patient health care costs in the TAU group was €1396 at baseline, €648 at posttreatment, and €1039 at follow-up. The average per-patient health care costs in the VR-CBT group was €1918 at baseline, €3031 at posttreatment, and €887 at follow-up. This includes €373.95 per patient for VR-related costs included in the posttreatment costs.

The cumulative costs per patient between baseline and follow-up, including the costs of VR-CBT, were €1686 and €3917 in the TAU and VR-CBT conditions, respectively. The between-group difference was €2231 (€3917–€1686) and was statistically significant (SE=663, t₁₁₄=3.36, P=.001) when not adjusted for the initial cost difference between the conditions at baseline. After adjustment for baseline costs, the incremental health care costs became slightly less at €2170 and retained statistical significance (SE=661, t₁₁₃=3.28, P=.001).

A total of 11.2% (13/116) of the participants had paid work. The average costs stemming from productivity losses per person for the TAU group was €224 at baseline, €214 at posttreatment, and €104 at follow-up. The average costs stemming from productivity losses per person for the VR-CBT group was €553 at baseline, €359 posttreatment and €28 at follow-up. The cumulative costs per patient between baseline and follow-up were €317 and €387 in the TAU and VR-CBT conditions, respectively. The between-group difference was €70 (€387–€317) and was not statistically significant (SE=274, t₁₁₄=–0.26, P=.80).

The average costs stemming from travel per person for the TAU group was €29 at baseline, €23 at posttreatment, and €24 at follow-up. The average travel costs per person for the VR-CBT group was €31 at baseline, €60 at posttreatment, and €28 at follow-up. The cumulative travel costs per patient between baseline and follow-up were €47 and €88 in the TAU and VR-CBT conditions, respectively. The between-group difference was €41 (€88–€47) and was statistically significant (SE=6, t₁₁₄=–6.73, P<.001).

The cumulative societal costs per patient between baseline and follow-up were €2050 and €4393 in the TAU and VR-CBT conditions, respectively. The between-group difference was €2343 (€4293–€2050) and was statistically significant (SE=747, t₁₁₄=–3.14, P=.002).

The mean incremental costs for a positive treatment responder was as follows:

Figures 2 to 4 depict the distribution of the 5000 bootstrapped ICERs over the cost-effectiveness plane for each of the social participation measures. Figure 2 depicts time spent with others, the plane illustrates 99.70% of the ICERs fall in the northeast quadrant, indicating that more QALYs are gained for higher costs. Figure 3 depicts momentary anxiety, the plane illustrates 90.74% of the ICERs fall in the northeast quadrant, indicating that more QALYs are gained for higher costs. Figure 4 depicts momentary paranoia, the plane illustrates 99.74% of the ICERs fall in the northeast quadrant, indicating that more QALYs are gained for higher costs.

Figure 5 depicts the distribution of the bootstrapped ICERs over the cost-effectiveness plane, with the vast majority of the ICERs in the northeast quadrant, indicating that more QALYs are gained but for higher costs, while 0.02% of the simulated ICERs fall in the southeast quadrant (ie, QALY gains for lower costs) for the VR-CBT group compared with the TAU group.

The mean incremental cost per QALY was €48,868. When looking at the acceptability curve in Figure 5, a higher probability that the VR-CBT intervention is deemed cost-effective can also be calculated. For an 80% certainty of cost-effectiveness, the incremental cost for gaining 1 QALY is €66,161, which falls well below the willingness-to-pay ceiling of €80,000 in the Netherlands for a severely disabling condition, such as schizophrenia characterized by paranoid delusions [21].

Looking at the three treatment responses, at 50% probability of being cost-effective, the costs are as mentioned: time spent with others, €10,069; momentary anxiety, €19,525; and momentary paranoia, €8079. Supposing that a decision maker needs an 80% certainty, time spent with others will have to be valued at €14,293 per treatment responder; momentary anxiety at €50,000; and momentary paranoia at €11,342.

When including safety behavior at baseline as a covariate, the incremental costs per treatment responder on time spent with others became €9136; momentary anxiety became €17,535; and momentary paranoia became €7219. When including safety behavior at baseline as a covariate, the incremental costs per QALY gained became €44,597. Overall, incremental costs were somewhat lower when the baseline difference of safety behaviors was included in the analysis.

When including psychiatric admission costs at baseline as a covariate, the incremental costs per treatment responder on time spent with others became €9729; momentary anxiety became €18,879; and momentary paranoia became €7750. When including psychiatric admission at baseline as a covariate, the incremental costs per QALY gained became €47,308. Overall, incremental costs were somewhat lower when the baseline difference of psychiatric admission costs was included in the analysis.

When including both psychiatric admission costs at baseline and safety behavior at baseline as covariates, the incremental costs per treatment responder on time spent with others became €8592; momentary anxiety became €16,597; and momentary paranoia became €6800. When including both psychiatric admission costs and safety behavior at baseline as covariates, the incremental costs per QALY gained became €42,030. Overall, incremental costs were lower when the baseline differences of both safety behaviors and psychiatric admission costs were included in the analysis.