In the United Kingdom, each patient registers with a single general practitioner practice. Information about their primary care consultations, prescriptions, investigation results, and certified sickness and mortality data are recorded in computerized medical records systems. Each patient has a unique identifier, the NHS number, which allows data linkage with other data sets, including the hospital data, Hospital Episode Statistics, death certificate data provided by the Office for National Statistics and the NHS prescribing data set [30].

We used pseudonymized data from the RSC to construct and validate a revised version of CMMS based on the limited set of SNOMED CT codes (we refer to this as the GDPPR-modified CMMS). RSC data are stored in the Oxford RCGP Digital Informatics Hub (ORCHID) trusted research environment. The RCGP RSC extracts data from just under 2000 general practices in England [31] and provides a data set that is representative of individuals in England.

We applied the same analytical approach and the same inclusion criteria as in our previous study, where we developed and validated a new CMMS for RSC using the more extensive list of SNOMED CT codes (we refer to this as the RSC-modified CMMS) [21]. We included people aged 16 years or older on the index date registered with a practice for 12 months or longer. Three separate data sets were sampled from the RSC as described previously (Figures S1 and S2 in Multimedia Appendix 1 [21]) and they are (1) derivation data set (n=500,000); (2) validation data set 1 (n=250,000) with the same study start and study end date as the derivation set (synchronous outcome); and (3) validation data set 2 (n=250,000) with 12-month outcome at a different time point to the derivation data set (asynchronous outcome), and 60-month outcome occurring at the same time point as the 12-month outcome of the derivation data set (synchronous outcome), as illustrated in Figure 1 [21]. These 3 data sets were generally comparable in terms of age, sex, number of conditions, and follow-up time.

In selecting conditions for a new modified CMMS model, we first considered all 37 conditions that were included in the original CMMS development and validation and used the same definitions and prescriptions [17]. Following the approach we previously developed for the RSC-modified CMMS with SNOMED CT [21], we carefully curated the conditions within the limited set of SNOMED CT codes in the GDPPR. We performed a confirmatory study concerning SNOMED CT coverage in the GDPPR by carrying out a statistical and clinical matching of the conditions between the GDPPR and RSC (Figure 2). A matching percentage for each condition was defined as the prevalence ratio, that is, the prevalence in the RSC data set but included in the GDPPR set of SNOMED CT codes divided by the prevalence included in the RSC set of SNOMED CT codes. Therefore, a ratio less than 100% indicated a higher prevalence of that condition within the RSC data set using its set of RSC SNOMED CT codes, compared to the prevalence using the GDPPR SNOMED CT list on the same data set. We set an 85% threshold for inclusion in the development of the GDPPR-modified CMMS model unless there was a clinical reason to accept a lower threshold.

As the RSC provides a data set that is representative of England, we assumed that the actual prevalence of each condition in the RSC data set is similar to that in the GDPPR data set, and therefore, the RCS data set provided a suitable environment to develop the GDPPR-modified CMMS. We developed this GDPPR-modified CMMS in the RSC data set because it offered a complete set of SNOMED CT codes for each clinical concept, and we could replicate the reduced data set within GDPPR and then compare the case finding with each approach.

Using the previously described method [21], we used time-to-mortality Cox proportional hazards models based on the development data set. We first included the conditions as binary indicators with sex and age (in decades) and a quadratic age term as covariates. We then carried out a variable reduction process via backward elimination and using the Akaike information criteria as the stopping criterion [32]. The goal was to be parsimonious with the number of variables needed for implementation. This was carried out using the “fastbw” function from the rms R package. Model performance evaluation was based on discrimination and calibration. Discrimination was assessed by the pseudo R², Somers D, and Harrell C [33]. Model calibration was evaluated by plotting a calibration curve and recalibration was carried out by resampling using cross-validation to correct for optimism or overfitting. This was implemented using the “calibrate” function in the rms R package. The model was developed, performance was evaluated on the derivation data set, and we then evaluated the performance of the models on the 2 validation data sets. All data preparation and analyses were conducted in R (version 4.1.0; R Core Team) [34], using the following R packages lme4 (version 1.1-27) [35], lubridate (version 1.7.10) [36,37], randomizr (version 0.20.0) [37], rms (version 6.2-0) [38], survival (version 3.2-11) [39,40], tableone (version 0.12.0) [41], and tidyverse (version 1.3.1) [42].

This development of the CMMS for use in GDPPR was performed as part of the RAVEN study which received ethical approval (Integrate Research Application Service number 300259) and was approved by the Health Research Authority’s Bromley Research Ethics Committee reference 21/HRA/1971, on October 8, 2021. NHS England hosts the national safe haven for patient data. The legal basis for this is Regulation 3 of the Health Service (Control of Patient Information Regulations) 2002 [43]. Pseudonymized data extracted from the practices are kept in a secured server at ORCHID, which is an NHS England policy-compliant trusted research environment (organization code EE133863-MSD-NDPCHS).

In this study, we developed and validated a modified version of a single measure of multimorbidity, CMMS, for use within a national data set created during the pandemic from all existing primary care data collections, GDPPR, and using its limited set of SNOMED CT codes. The initial model included 22 conditions from the set of 37 in the original CMMS model and the reduced 17-condition model showed an identical performance in predicting mortality to the 22-condition model and a similar performance compared to the original 37-condition CMMS and the previous modification which was based on an extensive SNOMED CT data set.

The GDPPR database remains available and is listed in the NHS Data Model and Dictionary [44]. It is now one of the data collections available through the NHS England Secure Data Environment [45], which was created as part of NHS England’s Data Saves Lives policy, following the Goldacre report [46].

This new single measure of multimorbidity will help us measure vaccine effectiveness using GDPPR, NHS England’s nationwide database. We will use this GDPPR-modified CMMS score in the RAVEN study to build on the existing evidence base [47,48]. Several observational studies have shown that the effectiveness of vaccination could be suboptimal in people with multimorbidity [49], and thus it is important to be able to explore and adjust for multimorbidity.

This tool may also be useful in a wider range of studies of people with multimorbidity, including vaccine and post-authorization safety studies. The risk of hospitalization, admission to intensive care unit beds, and mortality in people with multimorbidity are significantly higher than in the general population [50,51]. In an observational study of hospitalized patients in the United Kingdom with COVID-19, the crude mortality in people with multimorbidity, compared to single comorbidity, after adjusting for the demographic factors, was more than double (1492/3961, 37.7% vs 341/1971, 17.3%) [52]. It was estimated to reduce 63.5% (1905/3000) of deaths by prioritizing people with multimorbidity [53]. Therefore, people with multimorbidity were prioritized for vaccine rollout [54].

Our previous study showed that reducing the original CMMS variables from 37 to 21 did not compromise mortality predictability in people with multimorbidity [21]. In this study, we have demonstrated that the number of conditions can be reduced to 17 to match the data available in GDPPR and still can be a very good predictor of mortality.

We focused on mortality and this is the outcome most often reported in development studies of comorbidity indices [55]. There are many published comorbidity indices and they vary according to their time of development (and thus the number of modifications), derivation population, conditions (predictors), prediction horizon, outcome predicted, and data source [52]. Historically, the mortality indices have been designed for people in hospitals, using secondary care coding systems and they have provided predictions of in-hospital mortality and mortality between 6 months and 5 years [55]. The GDPPR-modified CMMS is the latest adaptation to the original CMMS [17] for predicting mortality in primary care. The predictions for the CMMS and its modified versions are for the same prediction horizons of 1 year and 5 years. The previous modification adapted the CMMS to conditions defined by the internationally recognized SNOMED CT coding system [21], as the original CMMS was based on a population in the United States and conditions defined by the Read clinical terminology, which is no longer used in England. Both modifications of the CMMS have been developed on English populations with a lower minimum age compared to that for the original version (16 years as opposed to 21 years).

This GDPPR-modified version produces a multimorbidity index for mortality based on conditions defined by the limited SNOMED CT list of the GDPPR. The RSC provided the development data set for both modifications of the CMMS. The RCS has a complete set of SNOMED CT codes. Its data set includes people registered in a fraction of English general practices, and the assumption of this study is that the underlying prevalence of each CMMS condition in people in the RSC data set is the same as those in the larger GDPPR data set. While the RSC is recruited to be nationally representative, there may inevitably be differences [24].

The GDPPR includes the English primary care data used by the British Heart Foundation’s Data Science Centre for their COVID-19 and cardiovascular diseases Consortium’s work. Our version of CMMS could be deployed by this and other groups using GDPPR or underlying primary care (General Practice Extraction Service) data [56].

The main strength of this study is that it built on our expertise in developing a version of CMMS that could be applied to routine clinical data recorded using SNOMED CT. This terminology is used internationally [21]. We overcame the limitations of the relatively limited number of SNOMED clinical terms in GDPPR and demonstrated that a 17-condition CMMS could run in the GDDPR data set of 54 million individuals.

There are several limitations of this study. We only predicted the mortality risk and did not predict the hospitalization or intensive care unit admission risk. The conditions were a subset of those included in the original CMMS study, which arose from a review of multimorbidity literature at the time of its development. The nature of disease and treatments, as to population characteristics, will change over time. Hence, the new CMMS versions will need to be updated regularly, and this may involve adding conditions that were not included in the original CMMS. Although we split our initial data set randomly into development and validation data sets, we have performed simple temporal external validation [57]. A more robust form of external validation would involve investigating the generalizability to other countries (geographical validation) or other settings (domain validation), but neither is relevant in this case as we are considering a national data set, and we have developed the new CMMS using what we assume to be a representative sample of the adult English population. The generalizability could be tested further on other primary care data such as Clinical Practice Research Datalink [18], which provides a database of anonymized health records for another sample of English general practitioner practices.

This latest modification of the CMMS provides a new validated single multimorbidity measure, which was generated through a combination of unique access to data and expertise in validation. The RSC provided nationally representative and comprehensive primary care data. The study team had experience in developing a validated CMMS version to use within SNOMED CT. This combination meant that it was possible to develop and validate a new version of CMMS for use in the national English data set, the GDPPR. Our previous study showed that reducing the original CMMS variables from 37 to 21 did not compromise mortality predictability in people with multimorbidity [21]. In this study, we have demonstrated that the number of conditions can be reduced to 17 to match the data available in GDPPR and still can be a very good predictor of mortality. Therefore, researchers using this national database, or looking for a further reduced CMMS measure, can use this 17-component single measure of comorbidity.

The approach used in this study could also be applied in other contexts. Our approach has been to replicate a validated multimorbidity measure in a smaller, but complete and high data quality sentinel network database, the RSC. Within the RSC we could ensure the model performs as well as the one run on complete data [21]. Additionally, developing and validating this reduced CMMS model in the RSC required less processing time. This may make this reduced version more attractive to other users, should processing time be at a premium.