All PRO data have been collected using the AmbuFlex/WestChronic PRO system, which was developed by the author in 2004 and has been continuously updated since. The system has been described in detail elsewhere [4,5]. The first version of the generic PRO system, WestChronic, was developed for mixed-mode (Web and paper) collection of PRO data for research purposes in clinical epidemiological studies with repetitive measurements. Based on experienced feasibility and high response rates, it was decided in 2007 to develop this system into a flexible, multipurpose PRO system. The goal was to use clinical PRO data as the basis for outpatient follow-up in selected patient groups.

In AmbuFlex, outpatients report their symptoms from home at regular intervals instead of attending fixed visits at the outpatient clinic. The PRO measures are used to decide whether a patient needs or wishes an outpatient visit, and were developed to promote better symptom monitoring, more patient-centered care, and more efficient use of resources [5]. Specific questionnaires have been developed for each diagnostic group. The AmbuFlex concept consists of three generic elements: PRO data collection, PRO-based automated decision algorithm, and PRO-based graphical overview for clinical decision support.

The AmbuFlex/WestChronic system supports dynamic mixed-mode data collection with the use of the internet or paper forms, as well as automated communication to the patient and the clinician via personalized letters, emails, text messages, and secure electronic communication. All information regarding implemented projects, items and questionnaires, communication, clinical users, and patients resides in tables in a Structured Query Language database residing in the server park of Region Central Denmark. All administration of projects, questionnaires, users, and patients is supported by the system’s software and managed in browser windows.

The system has several integrations (Table 1 and Figure 1). All Danish citizens are assigned a unique 10-digit number (Civil Personal Registration [CPR] number), and continuously updated information on their current postal address and vital status is available from the Danish Civil Registration System [6]. This information is automatically collected online prior to any approach to patients. On-demand printing of questionnaires and letters, as well as scanning of incoming questionnaires with subsequent optical character recognition, is controlled by the system software, and results about all variables end up in result tables for the individual implemented projects in the same database. This occurs irrespective of whether Web or paper forms are used, and all results are instantaneously accessible. WestChronic may implement an arbitrary number of PRO projects with individual protocols, questionnaires, patients, and clinical users. For the patient and clinician, each implemented project appears as a unique PRO project with its own logo, domain, website, accompanying letters, contact information, etc. A new and rewritten version is underway.

The data for the present paper were generated by an ad hoc application in the system and transferred for analysis in the R package (R Foundation for Statistical Computing, Vienna, Austria). Patients who had been referred but not yet answered at least one questionnaire were not included. The data collection in the first project started in September 2004, and during 2004 a total of 301 questionnaires were collected from 241 patients; this period was considered a pilot phase and not included in the analysis.

The analysis of trends was based on a generic model applicable for any clinical health data [7]. Each application of PRO data is divided into four processes: patient identification (eg, registers, consecutive or individual referral), data collection (logistics and organization), data aggregation (at patient or at some specified group level) and the actual data use (Figure 2). In the present paper, the last two processes will be described together. The patient identification process involves the identification of patient(s) from whom data are to be collected, the data collection process involves the actual collection of health data, including logistic procedures, the data aggregation process involves the management and organization of collected data for the data use process, and the data use process involves the use of the health data for the purpose of the specified activity. Each process may be repeated or may take place simultaneously with the previous process (from [7]).

An overview of selected projects is given in Table 2. In the first cohort studies, emphasis was on response rate, which was promoted by up to three reminders using email and letters and by giving the patient a free choice between Web and paper questionnaires. The maximum response rate was reached in patients with prostate cancer (Table 2), where the first questionnaire was answered by 95.9% of the patients. The PRO-based algorithm was first introduced in a screening program for patients with coronary heart diseases, and the AmbuFlex principle was fully unfolded in the implantation in epilepsy patients (Table 2, Figures 1 and 3).

Figure 4 and Table 3 show the development of the number of projects, patients, and received questionnaires. When considering that 2019 is only partly included, an exponential model of number of received questionnaires per calendar year explains 94% of the variation (Figure 4). At item level, a total of 13,538,760 responses have been received in the period. Until 5 years ago, web- and paper-based questionnaires were used with similar frequencies, but during the last 4 years web-based questionnaires have almost completely taken over. Projects where data was used at group level dominated until 2014, but PRO data used at the patient level is now by far the most frequent (Figure 5). Most projects and about one-third of the questionnaires now have an attached algorithm (AmbuFlex) by which the incoming answers are classified into two or three groups (Table 3 and Figure 3).

In the first part of the 15-year period, all projects were register-based, epidemiologic cohort studies with defined inclusion criteria, while from 2015 onwards PRO data for clinical use at the patient level has dominated the scene (Table 3 and Figure 5). In the latter case, patients are individually referred by a clinician. Patient cohorts are still created at group level and are used in research for topics like outcome measures, validation, and studies of determinants of referral. In 2019, 16,062 new patients were included, of which 73.8% (11,854/16,062) were referred individually by a clinician while the rest were included as part of a group by means of batch processing.

Fundamental changes have taken place with respect to PRO data collection over the course of very few years. Until 2014, paper-based communication was used in more than 90% of the approaches (Figure 6), while electronic communication was predominantly used in epidemiological studies with multiple measurements where about half of the communication was electronic (emailing and web-based answering) (Figures 4 and 6). A minor revolution took place November 1, 2014. From this date, all communication between the Danish authorities (including all public hospitals), and Danish citizens had to use a national, secure electronic mail system, called e-boks. All citizens are provided an e-boks, but while its use was previously voluntary, only people with disabilities can now be granted a dispensation and continue to receive paper letters from places like hospitals. After the AmbuFlex/WestChronic system gained access to e-boks (Table 1), this channel rapidly became the most important communication method to reach patients (Figure 6). These changes in communication methods to and from patients had consequences for turnaround time (ie, the time from asking the patient to answer until the questionnaire is received) (Figure 7). A total of 75% of the answers were received within 17 days in 2005 due to the homogeneous cohorts, but turnaround time increased slowly until 2014. At this time, the Danish Mail decreased their services due to economic problems caused by the steep fall in paper-based mail. The historic low turnaround time in 2019 is caused by a combination of e-boks communication and the web-answering method. Today, the first answers are received within minutes. Use of ordinary, unsecure email ended in 2018. Text messaging is still used for reminders in some projects but is mainly used as a 2-factor authentication tool, where the patient, after entering their credentials (CPR number), receives a 6-digit cipher code as a text message.

While data from group-level projects are aggregated and used in the analysis phases of each epidemiological study, data in patient-level projects are, or should be, aggregated and used the date the questionnaire is received; however, this is not always the case. If PRO data are used as an add-on in connection with ordinary organized outpatient follow-up, it is up to each clinician to decide to open the PRO overview or not. In a multicenter project covering eight Danish hospitals, the patients treated for lung cancer answered a PRO questionnaire each time they went to the hospital for a follow-up visit, with the idea being that the PRO data should be used to enforce patient-clinician communication during the consultation (Table 2). For legal reasons, the AmbuFlex/WestChronic system keep automated log files of when and to whom the PRO data are shown. In total, a minority of responses (47%) were seen by a clinician (Figure 8). There were major differences between departments, ranging from 14-93% [34]. Similar figures have been found in other projects where the PRO data are an add-on to the existing outpatient set-up. In AmbuFlex, however, the PRO data are not an add-on but the basis for the follow-up. Each time a questionnaire is received, it is either handled automatically (green response, Figure 3) or included on a list where it remains until a clinician has accessed it and made a clinical decision regarding whether the patient should be contacted or not. Therefore, virtually no AmbuFlex questionnaires remain unnoticed.