This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
Evidence was found for the effectiveness of virtual reality-based cognitive behavioral therapy (VR-CBT) for treating paranoia in psychosis, but health-economic evaluations are lacking.
This study aimed to determine the short-term cost-effectiveness of VR-CBT.
The health-economic evaluation was embedded in a randomized controlled trial evaluating VR-CBT in 116 patients with a psychotic disorder suffering from paranoid ideation. The control group (n=58) received treatment as usual (TAU) for psychotic disorders in accordance with the clinical guidelines. The experimental group (n=58) received TAU complemented with add-on VR-CBT to reduce paranoid ideation and social avoidance. Data were collected at baseline and at 3 and 6 months postbaseline. Treatment response was defined as a pre-post improvement of symptoms of at least 20% in social participation measures. Change in quality-adjusted life years (QALYs) was estimated by using Sanderson et al’s conversion factor to map a change in the standardized mean difference of Green’s Paranoid Thoughts Scale score on a corresponding change in utility. The incremental cost-effectiveness ratios were calculated using 5000 bootstraps of seemingly unrelated regression equations of costs and effects. The cost-effectiveness acceptability curves were graphed for the costs per treatment responder gained and per QALY gained.
The average mean incremental costs for a treatment responder on social participation ranged between €8079 and €19,525, with 90.74%-99.74% showing improvement. The average incremental cost per QALY was €48,868 over the 6 months of follow-up, with 99.98% showing improved QALYs. Sensitivity analyses show costs to be lower when relevant baseline differences were included in the analysis. Average costs per treatment responder now ranged between €6800 and €16,597, while the average cost per QALY gained was €42,030.
This study demonstrates that offering VR-CBT to patients with paranoid delusions is an economically viable approach toward improving patients’ health in a cost-effective manner. Long-term effects need further research.
International Standard Randomised Controlled Trial Number (ISRCTN) 12929657; http://www.isrctn.com/ISRCTN12929657
Psychotic disorders impose a large disease burden—morbidity plus mortality—on the population, and in its wake, substantial economic costs occur for society and health care systems. The main drivers of societal costs of schizophrenia are health care costs and productivity losses, but patients and their families also incur substantial costs [
Paranoid ideation is a common delusion in individuals with a psychotic disorder. Even when medicinal treatment is successful, paranoid thoughts and anxiety often remain because of conditioned avoidance and other acquired safety behaviors in social situations [
The health-economic evaluation was embedded in a randomized controlled trial evaluating VR-CBT in 116 patients with a psychotic disorder suffering from paranoid ideation [
Participants were recruited at seven treatment centers in the Netherlands between April 1, 2014, and December 31, 2015. To be included, participants had to meet the following criteria: (1) 18-65 years of age; (2) DSM-IV (Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition) diagnosis of schizophrenia, schizophreniform disorder, schizoaffective disorder, delusional disorder, or psychotic disorder not otherwise specified; (3) suffering from at least mild paranoia, as assessed by Green’s Paranoid Thoughts Scale (GPTS) (score of >40); and (4) self-report of avoiding at least one social situation. Exclusion criteria were as follows: (1) insufficient competency of Dutch language; (2) IQ below 70; and (3) a concurrent diagnosis of epilepsy. Assessments were performed at baseline and at 3 and 6 months postbaseline.
All participants continued to receive TAU (ie, antipsychotic medication, regular contact with a psychiatrist to manage symptoms, and regular contact with a psychiatric nurse). Participants in the experimental condition also received therapist-led VR-CBT. VR-CBT treatment consisted of 16 biweekly sessions of 60 minutes each, using 40 minutes for exposure and behavioral exercises in virtual social environments. The therapist used an individual case formulation to help patients falsify their harm expectancies. No homework exercises were given between VR-CBT sessions. The treatment protocol, in Dutch, is available from the corresponding author.
We conducted both a CEA with three measures of improved social participation as outcome and a cost-utility analysis (CUA) with quality-adjusted life years (QALYs) gained as outcome. The outcome measures are described in more detail below.
The outcome of interest in the CEA was social participation. Social participation was operationalized in three ways: (1) objective social participation as the amount of time spent with others, (2) subjective social participation as momentary anxiety, and (3) subjective social participation as momentary paranoia.
The outcome in the CUA was the QALY derived from the GPTS [
Societal costs were computed by adding (1) the direct medical costs of health care services use including the costs of antipsychotic medication and, in the experimental condition, the additional costs of adjunctive VR-CBT treatment; (2) direct nonmedical costs of travel; and (3) indirect costs stemming from lower productivity. For each participant, cost data over the last 3 months were collected at each of three measurement points. Resource use data, for costing, were collected using the Trimbos Institute and Institute of Medical Technology Assessment Questionnaire for Costs Associated with Psychiatric Illness (TiC-P) [
Direct medical costs were calculated by multiplying health service units (eg, sessions, visits, and hospital days) with their standard economic cost price (see
For VR therapy hardware, software and training costs were calculated. Total yearly costs for one VR system was €23,995, according to CleVR BV, a company who builds VR sets. Yearly costs for training and supervision of the psychologists was €13,400. Per-patient costs per 16 VR-CBT treatment sessions was €373.95.
Travel costs arose when participants had to make return trips for receiving health care at health services. Travel costs were computed as the average distance to a health service (7 km) multiplied by the costs per km (€0.21) [
Research assistants monitored changes in the participants’ work status at baseline and at 3 and 6 months postbaseline using the TiC-P. Productivity losses in paid work were calculated according to the human capital approach [
Following the CONSORT (Consolidated Standards of Reporting Trials) and CHEERS (Consolidated Health Economic Evaluation Reporting Standards) guidelines, all our analyses adhered to the intention-to-treat principle. To that end, missing values were imputed using multiply imputed chained equations (MICE) for nonparametric data with M of 100 bootstraps for each incomplete variable. Baseline variables predictive of effects (ie, QALYs and treatment response) were used for imputation, such as baseline data of the variable with missing values, treatment condition, ethnicity, education, sex, age, and safety behaviors at baseline. Safety behaviors, such as avoiding eye contact or escaping from social situations, were measured using the Safety Behaviour Questionnaire-Persecutory Delusions (SBQ-PD) [
Both the CUA and CEA were conducted from the societal perspective. In each of these analyses, the incremental cost-effectiveness ratios (ICERs) were calculated as the between-group cost difference divided by the between-group effect difference. The ICER is interpreted as the additional costs per additional unit effect (ie, per additional treatment responder; per QALY gained). Cost and effect differences were obtained from seemingly unrelated regression equations of costs and effects, thus allowing for correlated residuals in the equations. The seemingly unrelated regression equations (SURE) models were bootstrapped 5000 times. In each bootstrap step, the mean cost differences and the mean outcome differences were computed and these were plotted on the cost-effectiveness plane. Finally, cost-effectiveness acceptability curves (CEACs) were graphed. CEACs inform decision makers about the likelihood that an intervention is deemed cost-effective, given a range of willingness-to-pay ceilings for gaining 1 QALY and gaining 1 treatment responder. All analyses were conducted in Stata, version 13.1 (StataCorp).
The following sensitivity analyses were carried out. First, a sensitivity analysis was done including safety behavior at baseline as a covariate because despite randomization there was a significant difference at baseline, and it was found to be the main mediator in reducing paranoid ideation [
After providing informed consent, 116 participants agreed to participate: 58 (50.0%) in the control condition and 58 (50.0%) in the experimental condition (see
Baseline characteristics of the sample can be found in
Trial flow diagram. *Specification of participants lost to posttreatment: 6 declined further participation and 2 were lost due to clerical errors by therapist. ‡Specification of participants lost to follow-up: 9 declined further participation, 1 died of unrelated causes, and 2 were lost due to clerical errors by therapist. First published in Lancet Psychiatry (Pot-Kolder et al, 2018). VR-CBT: virtual reality-based cognitive behavioral therapy.
Characteristics of the study sample at baseline.
Characteristic | VR-CBTa (n=58) | Treatment as usual (TAU) (n=58) | |
Gender (male), n (%) | 40 (69) | 42 (72) | |
Age in years, mean (SD) | 36.5 (9.8) | 39.5 (10.0) | |
Non-Dutch origin, n (%) | 15 (26) | 25 (43) | |
|
|
|
|
|
No education or primary | 16 (28) | 16 (28) |
|
Vocational | 18 (31) | 24 (41) |
|
Secondary | 9 (16) | 9 (16) |
|
Higher | 15 (26) | 9 (16) |
|
|
|
|
|
Schizophrenia, n (%) | 46 (79) | 49 (85) |
|
Schizoaffective disorder, n (%) | 1 (2) | 5 (9) |
|
Delusional disorder, n (%) | 1 (2) | 0 (0) |
|
Psychotic disorder (not otherwise specified), n (%) | 10 (17) | 4 (7) |
|
Duration of illness in years, mean (SD) | 13.3 (10.6) | 14.9 (9.5) |
|
|
|
|
|
Antipsychotics, n (%) | 54 (93) | 57 (98) |
|
Olanzapine equivalent of prescribed antipsychotic medication (mg/day), mean (SD) | 10.5 (6.8) | 11.0 (8.3) |
|
Antidepressants, n (%) | 15 (26) | 17 (29) |
Paid work, n (%) | 8 (14) | 5 (9) | |
Safety behaviors, mean (SD) | 28.8 (14.2) | 21.1 (16.0) |
aVR-CBT: virtual reality-based cognitive behavioral therapy.
bDSM-IV: Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition.
Average per-participant costs per 3-month period in Euros for the year 2015 and average outcomes by measurement and condition.
Costs and outcomes | Baseline | Posttreatment (3 months) | Follow-up (6 months) | ||||
|
VR-CBTa | TAUb | VR-CBT | TAU | VR-CBT | TAU | |
|
|
|
|
|
|
|
|
|
Health care costs | 1918 (5178) | 1396 (3146) | 3031 (3189) | 648 (960) | 887 (1160) | 1039 (2640) |
|
Travel costs | 31 (23) | 29 (26) | 60 (34) | 23 (15) | 28 (22) | 24 (16) |
|
Productivity loss | 553 (2730) | 224 (1214) | 359 (1205) | 214 (1127) | 28 (161) | 102 (588) |
|
Total (societal) costs | 2502 (6246) | 1649 (3570) | 3076 (3469) | 885 (1589) | 943 (1185) | 1165 (2766) |
|
|
|
|
|
|
|
|
|
GPTSc paranoia (score) | 85 (34) | 77 (31) | 70 (31) | 75 (31) | 67 (33) | 75 (33) |
|
Time spent with others (proportion) | 0.416 (0.256) | 0.364 (0.266) | 0.404 (0.226) | 0.323 (0.266) | 0.419 (0.209) | 0.340 (0.273) |
|
Momentary anxiety (scored) | 2.986 (1.120) | 3.259 (1.484) | 2.586 (1.089) | 3.221 (1.495) | 2.645 (1.095) | 3.218 (1.388) |
|
Momentary paranoia (scored) | 3.064 (1.393) | 3.259 (1.418) | 2.714 (1.291) | 3.221 (1.518) | 2.719 (1.293) | 3.218 (1.467) |
aVR-CBT: virtual reality-based cognitive behavioral therapy.
bTAU: treatment as usual.
cGPTS: Green’s Paranoid Thoughts Scale.
dScores are on a 7-point Likert scale ranging from 1 (not at all) to 7 (very).
The treatment response rate regarding the time spent with others was 13 patients out of 58 (22%) in the control group and 24 patients out of 58 (41%) in the experimental group. The baseline-adjusted between-group difference between the response rates (ie, the incremental effect) was 0.23, which was statistically significant (SE=0.076, t113=3.07, 95% CI 0.08-0.38,
The treatment response rate with regard to momentary anxiety was 17 patients out of 58 (29%) in the control group and 24 patients out of 58 (41%) in the experimental group. The between-group difference between the treatment response rates (ie, incremental effect) was 0.12, but this difference was not statistically significant (SE=0.089, t114=1.36, 95% CI –0.055 to 0.290,
The treatment response rate in momentary GPTS paranoia was 11 patients out of 58 (19%) in the control group and 28 patients out of 58 (48%) in the experimental group. The between-group difference in the response rates was 0.29 and was statistically significant (SE=0.0841, t114=3.48, 95% CI 0.126-0.460,
The SMD of GPTS paranoia was 0.523, which was statistically significant (SE=0.120, t114=4.37, 95% CI 0.285-0.760,
As can be seen in
The cumulative costs per patient between baseline and follow-up, including the costs of VR-CBT, were €1686 and €3917 in the TAU and VR-CBT conditions, respectively. The between-group difference was €2231 (€3917–€1686) and was statistically significant (SE=663, t114=3.36,
A total of 11.2% (13/116) of the participants had paid work. The average costs stemming from productivity losses per person for the TAU group was €224 at baseline, €214 at posttreatment, and €104 at follow-up. The average costs stemming from productivity losses per person for the VR-CBT group was €553 at baseline, €359 posttreatment and €28 at follow-up. The cumulative costs per patient between baseline and follow-up were €317 and €387 in the TAU and VR-CBT conditions, respectively. The between-group difference was €70 (€387–€317) and was not statistically significant (SE=274, t114=–0.26,
The average costs stemming from travel per person for the TAU group was €29 at baseline, €23 at posttreatment, and €24 at follow-up. The average travel costs per person for the VR-CBT group was €31 at baseline, €60 at posttreatment, and €28 at follow-up. The cumulative travel costs per patient between baseline and follow-up were €47 and €88 in the TAU and VR-CBT conditions, respectively. The between-group difference was €41 (€88–€47) and was statistically significant (SE=6, t114=–6.73,
The cumulative societal costs per patient between baseline and follow-up were €2050 and €4393 in the TAU and VR-CBT conditions, respectively. The between-group difference was €2343 (€4293–€2050) and was statistically significant (SE=747, t114=–3.14,
The mean incremental costs for a positive treatment responder was as follows:
Time spent with others: €2343/0.23=€10,069.
Momentary anxiety: €2343/0.12=€19,525.
Momentary paranoia: €2343/0.29=€8079.
The mean incremental cost per QALY: €2343/0.048=€48,868.
Cost-effectiveness plane and willingness to pay (WTP) acceptability curve for time spent with others.
Cost-effectiveness plane and willingness to pay (WTP) acceptability curve for momentary anxiety.
Cost-effectiveness plane and willingness to pay (WTP) acceptability curve for momentary paranoia.
Cost-effectiveness plane and willingness to pay (WTP) acceptability curve for quality-adjusted life year (QALY) gain (costs per QALY gained) after 6 months.
The mean incremental cost per QALY was €48,868. When looking at the acceptability curve in
Looking at the three treatment responses, at 50% probability of being cost-effective, the costs are as mentioned:
When including safety behavior at baseline as a covariate, the incremental costs per treatment responder on
When including psychiatric admission costs at baseline as a covariate, the incremental costs per treatment responder on
When including both psychiatric admission costs at baseline and safety behavior at baseline as covariates, the incremental costs per treatment responder on
This study aimed to get an impression of short-term cost-effectiveness of VR-CBT for patients with paranoid delusions in comparison to TAU from a societal perspective. Data were collected 6 months after baseline at follow-up. Costs per treatment responder gained were estimated to be between €8079 and €19,525 for different aspects of social participation, with between 90.74% and 99.74% showing improvement. Cost per QALY gained at follow-up was estimated to be €48,868 with 99.98% showing improved QALYs. Sensitivity analyses showed costs to be lower when baseline differences in both safety behavior and psychiatric admission costs were included in the analysis. Costs per treatment responder gained were then estimated to be between €6800 and €16,597, with cost per QALY gained at €42,030.
How much a society values solidarity with people burdened by disease will determine if guidelines are translated to actual treatment of patients. While the VR-CBT treatment condition is more expensive than TAU only, that was to be expected, as the aim was to add to existing treatment. Results show that this addition improves social participation for people with a psychotic disorder suffering from paranoid ideation. We see this improvement for time spent with others, momentary paranoia, momentary anxiety, and paranoid ideation, via the GPTS.
Engaging in psychological therapy is challenging for many patients suffering from paranoid ideation and treatment results vary. There are several aspects that favor VR treatment. Person-specific behavioral exposure is an important part of increasing treatment effect [
Interestingly, during the follow-up we see that the VR-CBT group resulted in decreased health care costs and decreased costs due to productivity loss compared to the TAU-only group. There were no psychiatric admission days at follow-up for the VR-CBT group. To determine whether this was a coincidence or a trend, a much longer follow-up period is needed. Short-term societal costs were between €8079 and €19,525 for a positive treatment response. A disability weight of zero represents no loss of health and a weight of 1 represents health loss equivalent to death [
The study has several limitations. First, data were collected only 6 months postbaseline. Any longer-term effects and costs are unknown. There are indications that cost-effectiveness for treatment of psychotic symptoms improves with time [
This study found VR-CBT to be cost-effective in the short term from a societal perspective. However, the effect of additional VR-CBT sessions and long-term effects need to be determined while using direct measurement of QALYs.
Direct medical costs. Prices are from 2015 and are in Euros. *8 (sessions) × 2.5 (hours) × 2 (therapists) × 112 (Euros per contact-hour) / 8 (participants) = Є560. WRAP: Wellness Recovery Action Plan.
3 degrees of freedom
cognitive behavioral therapy
cost-effectiveness analysis
cost-effectiveness acceptability curve
Consolidated Health Economic Evaluation Reporting Standards
Consolidated Standards of Reporting Trials
cost-utility analysis
Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition
European Quality of Life Five Dimension Scale
ecological sampling method
Green’s Paranoid Thoughts Scale
incremental cost-effectiveness ratio
International Standard Randomised Controlled Trial Number
multiply imputed chained equations
quality-adjusted life year
Safety Behaviour Questionnaire-Persecutory Delusions
standard mean difference
seemingly unrelated regression equations
treatment as usual
Trimbos Institute and Institute of Medical Technology Assessment Questionnaire for Costs Associated with Psychiatric Illness
virtual reality
virtual reality-based cognitive behavioral therapy
Vrije Universiteit
Funding was received from Fonds NutsOhra, Stichting tot Steun VCVGZ.
None declared.