We conducted a pragmatic, mixed methods, cluster-randomized controlled trial with an embedded qualitative study in 19 general practices across the West Midlands, United Kingdom (10 intervention, 9 control) in early 2024. A cluster design was necessary because, once installed, the CDSS became accessible to all clinicians in the practice. Individual-level randomization was therefore not feasible due to the risk of contamination [25]. The trial protocol is publicly available [26]. Practices were recruited between February and April 2023, and randomized in a single batch in January 2024. The intervention period ran from February to June 2024, with each practice contributing a 3-month recruitment window plus 3 months of follow-up EHR data. Qualitative interviews were conducted following the intervention period, between June and August. Data extraction and analysis were completed in May 2025.

Participating clinicians included GPs and FCPs. GP practice involvement, consent, licensing, and training were supported by the National Institute for Health Research Clinical Research Network and a local health care team from the Birmingham and Solihull Integrated Care Board.

The trial is reported in accordance with the CONSORT-EHEALTH (Consolidated Standards of Reporting Trials of Electronic and Mobile Health Applications and Online Telehealth) guidelines [27], and the completed CONSORT-EHEALTH checklist (version 1.6.1) is provided in Checklist 1.

The original trial protocol included patient-reported outcome measures data collection via a trial-specific computer prompt during consultations. However, it failed to operate as intended, resulting in low recruitment rates. Following this, an ethics amendment was approved to abandon patient-reported outcome measure collection and instead rely solely on anonymized EHR data for all outcomes and to change the primary outcome to care quality.

The unit of allocation and intervention within this cluster-randomized trial was the GP practice. GP practices were randomized 1:1 by an independent statistician using stratified randomization based on registered patient list size. Due to the nature of the intervention, clinician blinding of clinicians was not feasible. Patients were not informed of group allocation. The study used routinely collected, anonymized EHR data, and the Health Research Authority approved processing of these data without individual patient consent (IRAS 326002; REC 23/YH/0105). Patients who had opted out of data sharing for research were excluded. Outcome analysis was performed by a statistician blinded to group allocation.

The trial was powered to detect a 20% absolute reduction in nonevidence-based management options in the intervention arm, assuming a baseline proportion of 40%. With an intracluster correlation coefficient (ICC) of 0.01 and a 2-tailed α of .05, a total of 400 patients (200 in each arm; 20 per cluster) provided more than 90% power for superiority testing. For the primary outcome, a random subsample of 120 patient records (60 per arm) was selected. Using PASS 2022 (NCSS, LLC) and the procedure for testing 2 proportions in a cluster-randomized design (2-sided unpooled Z-test), we estimated power for a design of 10 clusters per arm with 6 records per cluster (60 per arm). Assuming proportions of 0.625 and 0.375 in the 2 arms (a 25 percentage-point absolute difference), an ICC of 0.01, and a 2-sided α of .05, this provided approximately 79% power. The assumed proportions were set close to 0.5, where the binomial variance and hence the required sample size are greatest, giving a conservative estimate [25]. In the event, the sampled records came from 7 practices per arm with a mean of 8.6 records per practice. At an ICC of 0.01, the larger cluster size has a negligible effect on the design effect, but estimating between-practice variance from 7 rather than 10 clusters per arm reduces precision, so realized power was somewhat below the 79% estimated at the design stage. The observed proportion of control records containing suboptimal management (15/60, 25%) was lower than the 40% assumed for the trial-level calculation at the design stage, although the absolute between-arm difference observed (22 percentage points) was close to the 25 percentage points the subsample was powered to detect.

Inclusion criteria for the anonymized EHR analysis were consultations in adults aged ≥18 years with an eligible MSK code [28] during a 3-month trial window (February to July 2024, depending on practice onboarding). Patients who had opted out of data sharing for research were excluded. Clinicians were prompted to record pain intensity scores during MSK consultations. Qualitative participants included purposively sampled patients, clinicians from intervention practices, and secondary care specialist triage clinicians receiving CDSS-facilitated referrals.

The intervention was CrossCover Orthopathway CDSS [23], a UK Conformity Assessed (UKCA)-marked class 1 medical device featuring MSK pathways for the elbow, foot and ankle, hand and wrist, hip, knee, paediatrics, shoulder, and spinal pain. The pathways were co-designed by a local multidisciplinary team and built on national MSK guidance; clinical content for each body-region pathway was validated by consultant physiotherapists, orthopedic and rheumatology specialists, and GPs, while CrossCover delivered digital integration, referral automation, and the safety and feedback logging. A CDSS link appeared during a consultation after clinicians entered a relevant MSK diagnosis code into the EHR. Training was offered to all intervention clinicians, although typically a single clinician per practice cascaded training internally, as the CDSS was new and in the process of being adopted locally at the time of the trial. No changes were made to the orthopathway intervention during the study period. There were no software updates, bug fixes, content or functionality changes, or system downtime that affected the delivery of the intervention.

Control practices delivered usual care with no restrictions on the information sources clinicians might use; the only trial-specific addition was a prompt to record pain scores.

Clinician engagement with the CDSS was assessed using in-platform analytics. Metrics included the proportion of eligible patients for whom the tool was used, the average time spent using it, the number and proportion of uses by GPs and FCPs, and the rate at which users opted out of the CDSS’s treatment recommendations.

Clinicians could report safety concerns using an in-platform issue log. Reported issues were reviewed and categorized independently by members of the academic trial team, without input from the CDSS industry partner. Using a structured classification framework [31], each issue was grouped by its likely cause (such as data entry errors, algorithm inconsistencies, user interface design, or system integration problems), type of impact (eg, accuracy or timeliness), and severity of risk of patient harm (low, moderate, or high).

A validated carbon footprint and cost calculator was used to estimate resource use across 63 common MSK management options [5]. Health care utilization data were extracted using SNOMED CT-coded EHRs data and standardized methods [28], across primary, community, and secondary care settings [5]. While the full methodology and results of this substudy are reported elsewhere, key overall findings are included here to support broader impact interpretation. Costs are reported in pounds sterling (£) with conversions to US dollars at the 2024 average exchange rate of £1 = US $1.28.

All quantitative analyses were performed using Stata v17 (StataCorp LLC). The PP primary analysis used hierarchical (multilevel) generalized linear regression models to examine treatment differences across groups, adjusting for practice list size (fixed effect) and clustering by GP practice (random effect). Binary outcomes used logit links, while count outcomes used negative binomial models to account for overdispersion. The primary analysis tested for superiority using a 2-sided 5% α, with 95% CIs.

Two secondary comparisons were performed in relation to the scenarios of CDSS use within the trial. First, the ITT analysis compared participants in intervention vs control practices to explore the effectiveness of the CDSS across all consultations, whether the CDSS was used or not. Second, a PP analysis compared consultations in which the CDSS was used with control consultations to explore the efficacy of the CDSS when used. A post hoc sensitivity analysis used cluster-level modeling to test the robustness of the findings, given the relatively small number of clusters [32].

A qualitative study was embedded within the trial to explore clinician experiences of the CDSS. The full methodology and results are reported in a separate manuscript (Lee K, Wright T, Birkinshaw H, Hill JC, unpublished data, 2026). However, to support interpretation of clinician intervention engagement, selected findings are summarized here. Semistructured interviews were conducted online via Microsoft Teams and audio-recorded. A topic guide was developed by the study team and refined during data collection. Transcripts were anonymized and analyzed using reflexive thematic analysis [33], with coding supported by NVivo software (Lumivero). We reported the qualitative methods and findings in accordance with SRQR (Standards for Reporting Qualitative Research) guidelines [34].

Ethical approval was granted by the Health Research Authority on June 8, 2023 (IRAS project ID: 326002; REC reference: 23/YH/0105). Written, informed consent was obtained from all qualitative participants.

Of the 30 GP practices approached, 20 provided final consent and were randomized; 19 actively participated in the trial (10 intervention, 9 control) (Figure 1). Across these 19 practices, 10,182 unique adult patients consulted for an MSK pain condition during the study period (4268 in intervention practices, 5914 in control practices).

The CDSS was used in 365 (8.6%) intervention patients, involving 56 clinicians across 8 of the 10 intervention patient practices, and there were 397 uses across those 365 patients. Patients had a mean age of more than 50 years; 59.0% (6005/10,182) were female. The most common presenting complaints were spinal, knee/leg, and multisite pain. Baseline characteristics are summarized in Table 1.

Table 2 presents the results of the blinded clinical record review; interrater agreement on the 110 independently reviewed records was complete (Fleiss κ 1.0). The sampled records were drawn from 7 practices in each arm. The 60 control records comprised 101 consultations, all of which were assessed. The 60 intervention records comprised 105 consultations, of which 68 (64.8%) involved CDSS use and were therefore assessable under the PP definition.

Suboptimal management was identified in 2 of 60 (3%) intervention records and 15 of 60 (25%) control records. Adjusted analysis indicated a significantly lower likelihood of suboptimal care in the intervention group (adjusted odds ratio (OR) 12.30, 95% CI 2.52‐59.90; P=.002). When expressed per assessed consultation, suboptimal management was identified in 2 of 68 (3%) CDSS-guided consultations, compared with 15 of 101 (14.9%) control consultations.

In a sensitivity analysis assessing all consultations in both arms (ie, intervention consultations not restricted to CDSS use), suboptimal management was identified in 8 of 60 (13%) intervention records, giving an adjusted OR of 2.74 (95% CI 1.00‐7.48; P=.05). The direction of effect was consistent across both definitions, although the difference was borderline significant when management at intervention consultations where the CDSS had not been used was included.

Among the 37 intervention consultations in which the CDSS was not used, suboptimal management was identified in 6 (16%). This was closely comparable to the rate in control consultations (15/101, 14.9%) and several times higher than in consultations where the CDSS was used (2/68, 3%). Care delivered within intervention practices when the tool was not opened was therefore indistinguishable from usual care, which suggests the difference observed is consistent with an effect of CDSS use rather than underlying differences between practices. This comparison was conducted post hoc at the request of peer reviewers.

Findings in the control arm were concentrated in 2 indicators: multiple attendances with only a prescription issued and no documentation of advice, investigation or referral (5 vs 0), and spinal imaging for low back pain without red flag indicators (4 vs 0). Together these accounted for 9 of the 15 instances identified in control records. Across the prescribing and injection indicators, counts were similar or slightly higher in the intervention arm. A full breakdown by indicator under both definitions is provided in Multimedia Appendix 1.

Table 3 presents secondary outcomes from the ITT analysis that explored effectiveness in all consultations. No statistically significant differences were found between groups, except for a higher rate of recorded self-management and exercise advice in the intervention group. A before-and-after sensitivity analysis of orthopedic referral rates, using secondary care data, showed no material differences between groups in either the preimplementation or postimplementation period, supporting the robustness of the main findings.

Table 4 presents the results from the PP analysis to explore the efficacy of the CDSS, comparing consultations where the CDSS was used (n=365) with controls. CDSS-guided care was associated with fewer prescriptions and fit notes and with more frequent urgent specialist referrals (a proxy for better red flag identification), community referrals, and provision of self-management advice. Post hoc sensitivity analyses at the cluster level supported these findings.

Clinician engagement with the CDSS was limited. Over the 3-month intervention period, there were 397 uses of the CDSS among 8.6% (365/4268) eligible patients. Use was uncommon in follow-up appointments. A total of 56 clinicians used the CDSS: 41 GPs and 15 FCPs. GPs accounted for 56% (221/397) of uses and FCPs for 44% (176/397). The mean use time per consultation was 4 minutes 39 seconds (SD 4.36), and a 5% opt-out rate was observed for CDSS-generated treatment recommendations.

There were 24 safety log submissions in 365 (6.6%) uses of the CDSS, categorized as follows: no high-risk issues, 3 moderate-risk (1 algorithm labeling error and 2 integration issues), and 21 low-risk or system improvement suggestions. Overall, the CDSS was judged to be safe, with no reports of patient harm, and was seen as responsive to clinician feedback.

Semistructured interviews with clinicians (n=18) identified key barriers to CDSS use. GPs noted that its MSK-specific focus conflicted with the multimorbidity and time constraints typical of general practice consultations. The binary nature of outputs (eg, referral/no referral) was considered limiting for conditions requiring longer-term monitoring. Other barriers included the perceived learning curve for new software, concerns over growing reliance on algorithms, and the sense that the initiative prioritized secondary care goals over primary care needs. Participants emphasized the importance of improved integration, communication, and incentives to support uptake. Further qualitative results are reported in a separate manuscript.

Modeling suggested that the intervention could reduce annual carbon dioxide equivalent (CO₂e) emissions across Birmingham and Solihull Integrated Care Board by 940 tonnes (equivalent to 3.36 million petrol car miles) and generate annual cost savings of £1.6 million (US $2.05 million). The largest carbon contributor was prescribing, while the main cost driver was GP appointments. On a per-person basis, CDSS-guided care reduced mean emissions by 4.14 kg CO₂e and costs by £7.10 (US $9.09), representing an 18% reduction in emissions and a 10% reduction in costs. Full methodology and detailed results are reported in a separate publication (Naylor R, Malcolm R, Pegg M, et al, unpublished data, 2026).

This mixed methods, cluster-randomized trial evaluated the CrossCover Orthopathway CDSS in over 10,000 MSK consultations in UK general practice. In a PP analysis, the intervention was associated with lower guideline nonadherent care: suboptimal management was recorded in 3% (2/60) of CDSS-guided consultations compared with 15/60 (25%) of control records. This estimate is imprecise: it rests on 2 events in the intervention arm and 15 in the control arm, and the CI around the OR is correspondingly wide. The difference was concentrated in 2 of the 12 indicators: repeated attendance without documented advice, investigation, or referral, and imaging for nonspecific low back pain without red flags. Across the prescribing and injection indicators, counts were similar or slightly higher in the intervention arm, which is consistent with the CDSS’s focus on referral and pathway decisions rather than on what to prescribe.

A post hoc comparison within the intervention arm provides some reassurance that this reflects the tool rather than differences between practices. Among intervention consultations in which the CDSS was not used, suboptimal management was identified at a rate closely comparable to that in control consultations and several times higher than in consultations in which the tool was used. Care delivered within intervention practices when the CDSS was not opened, therefore, resembled usual care.

In the ITT secondary outcome analysis, which reflects the real-world effectiveness of the CDSS, the intervention group also received more documented self-management and exercise advice. In the PP secondary outcome analysis, additional benefits included reduced analgesic prescribing and fit-note certification, alongside increased referrals to physiotherapy and urgent secondary care, consistent with the CDSS’s escalation logic for red-flag presentations, such as suspected cauda equina syndrome or fracture.

However, CDSS usage occurred in only 8.6% of eligible patients, reflecting persistent barriers to clinician engagement. This corroborates findings from other trials—including STarT MSK [35], where tools were perceived as disruptive to consultation flow. Low uptake is not unique to this intervention: a systematic review of CDSS uptake found that use is seldom reported and, where reported, is often low and identified this as a major and potentially modifiable barrier to overall CDSS effectiveness [36]. While user feedback in this study was largely positive, with an average usage time under 5 minutes and low opt-out rates, broader adoption remains a key challenge. Notably, usage appeared influenced by clinical role: GPs, whose caseloads span many conditions, encountered the CDSS less frequently, limiting habituation. In contrast, FCPs, whose caseloads are MSK-specific, engaged more frequently with the tool. Direct evidence linking caseload concentration to sustained CDSS engagement is limited, but this suggests that FCPs may represent a priority group for CDSS implementation in primary care.

Interventions aiming to change clinician behavior in MSK care have a mixed record. Single-component implementation efforts have consistently failed to shift low back pain practice, with multifaceted approaches appearing necessary [37], and behavioral nudging trials such as those led by Maher et al [17] have shown limited success. Clinical decision support systems (CDSSs) perform better, but modestly and selectively: across 122 trials involving over 1.2 million patients, CDSSs increased the proportion of patients receiving the specific care targeted by a tool by 5.8% (95% CI 4.0%‐7.6%), with marked heterogeneity between targets and a median improvement of only 0.3% in clinical end points [38]. Effects concentrate on the decisions a tool actually prompts rather than generalizing across care [39].

That pattern helps explain the shape of our findings. Orthopathway addressed referral and pathway decisions, and it was in referral and imaging that differences emerged; it offered little guidance on analgesic choice or the timing of joint injections, and in those domains we found no benefit. Guideline-discordant imaging for low back pain remains common in primary care [40], and our imaging finding is consistent with evidence that decision support and targeted reminders can reduce it [41]. A more recent meta-analysis found no overall effect, based on low-certainty evidence, but concluded that educational interventions are unlikely to succeed while organizational and policy-level interventions are more promising [42], a distinction that fits with a tool embedded in the consultation workflow rather than delivered as training. Prescribing, by contrast, has proved responsive to audit and feedback rather than to point-of-care prompts, as in the UK CROP study [43], suggesting that different levers may be required for prescribing than for pathway decisions.

Previous MSK CDSS trials, including SupportPrim [14] and STarT MSK [15], did not assess care quality directly and showed limited influence on clinician decision-making. Our blinded review of records against prespecified indicators offers a feasible method for assessing digital interventions intended to improve care quality and adds to evidence that deimplementation of low-value practice is achievable when workflow and infrastructure are changed, rather than when information alone is supplied [44]. Our findings suggest that a clinician-facing CDSS may improve some aspects of MSK care quality when it is used, although the magnitude of that effect remains uncertain.

These findings also align with ongoing safety concerns in primary care, particularly those addressed by National Health Service (NHS) England Jess’s Rule, which advises clinicians to “reflect, review, and rethink” when patients reattend three times without diagnostic clarity or documented progression [45]. Repeated prereferral consultations are an established marker of missed diagnostic opportunity in other clinical areas, notably cancer diagnosis [46], although the evidence base for reattendance as a quality marker in MSK care specifically is less developed. In our sample, reattendance without documented change in management was among the most frequent forms of suboptimal care in the control arm and was not observed at CDSS-guided consultations, suggesting that embedded structured support may help operationalize prompts of this kind in routine practice.

Strengths of this trial include its real-world design, mixed methods approach, large-scale anonymized EHR data, and use of independent, blinded outcome review. It also incorporated safety, environmental, and economic analyses, enabling a broader assessment of impact. Limitations span multiple domains. In relation to data collection, the follow-up period was short (3 mo); there was an absence of data collection on control clinicians’ information-seeking behavior; and the failure of the patient-reported outcome measure data collection component prevented evaluation of patient-reported outcomes.

The primary outcome should be interpreted cautiously. It rests on a small number of events, and effect estimates derived from sparse data are prone to inflation and imprecision [47], which is reflected in the wide CI around our OR. The composite was also driven by 2 of the 12 indicators; composite outcomes are frequently dominated by a subset of their components, and effects should not be assumed to apply uniformly across the whole composite [48,49]. One of these 2 indicators, repeated attendance without adequate management, involves a degree of subjective clinical judgement, and the appropriateness of imaging for nonspecific low back pain can be difficult to establish from retrospective record review, even with the full consultation notes available. Reviewers were blinded and applied pre-specified indicators, but these constraints limit the precision with which the size of the effect can be estimated.

The attribution rule used in the PP analysis is a further limitation. Suboptimal management in the intervention arm was counted only where it occurred at a consultation in which the CDSS was used, whereas all control consultations contributed. Intervention records therefore had fewer assessable consultations on average (1.13, compared with 1.68 in the control arm). The sensitivity analysis assessing all consultations in both arms showed an effect in the same direction, but smaller and not robust. More broadly, ITT analysis with low uptake estimates the effect of being offered the intervention rather than of receiving it and dilutes the effect of the tool itself [50]; our ITT analysis may additionally have been underpowered, with higher-than-expected ICCs contributing to wide CIs. In relation to the CDSS and its integration, pathway coverage was limited at the time of the study, and the CDSS sat outside the EHR system, so users may have reflected an early-adopter population rather than all primary care clinicians.

Future research should evaluate implementation strategies to increase uptake, particularly those enhancing workflow integration and aligning with incentive structures (such as the advice and guidance scheme). Longer-term evaluations are also needed to assess sustainability, patient outcomes, and wider NHS system impacts. Notably, this trial directly led to the full EMIS and SystmOne integration of the CDSS in January 2025, demonstrating how early digital evaluations can identify barriers that are subsequently resolved through iterative design improvements.

When used, the CrossCover CDSS was associated with lower rates of nonevidence-based MSK care in UK general practice, although the size of this effect remains uncertain. Benefits were concentrated in referral and imaging decisions, the areas the tool addressed, with little evidence of effect on prescribing or injections. The tool was safe, adaptable, and associated with reductions in both carbon emissions and health care costs, but overall impact was limited by low clinician uptake. Broader adoption will require improved system integration, better alignment with clinical workflows, and the use of incentive frameworks. Further research is needed to evaluate long-term clinical, environmental, economic, and patient-reported outcomes.