We designed multiple choices in different scenarios consisting of 6 different randomly selected attributes using conjoint-related techniques. Questionnaires were created by Sawtooth software’s Lighthouse Studio modules (version 9.8.1) for general interviews and choice-based conjoint (CBC) scenario design. Respondents were aged 18 to 85 years. Meanwhile, in this data analysis section, we aimed to visualize and measure the percentage weight and importance of different attributes with selected models. From the perspective of public health, McFadden’s conditional logit [15], which is also known as multinomial logit (MNL) [16], is widely applied to organize, analyze, and predict the data we had and help with further analysis of statistical significance. However, the foundation of using this model is that we acquiesce that unobserved heterogeneity of preference does not exist across respondents. Therefore, we then need to introduce a MXL model [17] and a GMNL model [18], both of which take unobserved heterogeneity of preference into account. The LCM [19] might also be an appropriate model to apply here, since it divides the data into various groups with fixed segmented size and the evident probability of latent membership [20]. With different latent classes, we can clearly distinguish the most important attributes or attributes’ levels of each class and summarize these attributes with a significant percentage weight.

Random utility theory [21] is the basic principle of DCE. The principle assumes that all the choice selectors have M different choice alternatives, and each choice alternative corresponds to a utility W. The utility W is consistent with a combination of fixed and random utility. The stationary utility U can be explained by some observable elements x, while the random factor ε represents the influence and interference unobserved utility and possible error. The goal of choice selectors is to choose the best combination with the supreme utility; then the probability of each combination alternative being selected can be expressed as a function of its fixed utility: P = F(U). The specific form of the function depends on the distribution of random effects. In most model settings, the utility that is optical, U_v, will be expressed as a linear combination of elements x, that is, U_v = βx. β is a coefficient, and its value and significance level can be estimated from the observation data.

Based on the relevant literature [22-24], we have assumed that the patients’ preference or satisfaction with the medical choices mainly depends on some specific features that make up the essential attributes of our experiment. Moreover, we performed a pilot test to get the attributes and levels that we need in our research. Patients in the outpatient queue of the First Affiliated Hospital of Jinan University (Guangzhou Overseas Chinese Hospital) and the First Affiliated Hospital of Sun Yat-sen University were interviewed and invited to have a discussion of what attributes patients are most likely to attach importance to. In addition, the possible attributes’ levels are hypothesized and set in a certain sequence in our questionnaire; for instance, accuracy ranks from 0% to 100%.

Therefore, in our questionnaire, six attributes have been contained for our experiment: (1) diagnostic method; (2) outpatient waiting time before being asked; (3) diagnosis time; (4) accuracy (ratio of correct diagnosis); (5) follow-up after diagnosis (whether the outpatient doctor/AI doctor can follow up and follow up at any time); and (6) diagnostic expenses. Every attribute and its levels are presented in Table 1.

The questionnaire contains two sections. In the first section, which is also known as demographic questions, we aim to allocate the respondents’ basic information: age, gender, and highest education level. In the second section, we use the CBC function in Sawtooth software to create various combinations of scenarios for respondents to choose from.

When we use the factorials method [25] to analyze the attributes to give the combinations of scenarios, we encounter several obstacles. Since we have 6 attributes that give 3240 (3×6×3×5×2×6) possibilities, we assume that we have 6 random questions and 100 sets, which also gives 600 different combinations. The difficulty is how to extract 600 representative combinations from 3240 combinations and obey two basic principles [26] at the same time: (1) balance and (2) orthogonality. Balance means each attribute level appears equally often within an attribute, and level 1 in attribute 1 equals level 2 in attribute 2. Orthogonality means each pair of levels appears equally often across all pairs of attributes. However, it is almost unrealistic to handle such a huge task. Therefore, we use Sawtooth software to help us select suitable combinations. We set six random tasks, one fixed task, and two concepts per task (excluding “None Option”). Meanwhile, we set the sample size 500 and assume that 5% of respondents would choose “None Option” and finally design the test.

The standard error of almost all the attributes’ levels is <0.05. Since the expense is a continuous variable of which the standard error could be slightly higher than 0.05, a sample size of 500 should be sufficient for our experiment. One of the CBC tasks of the DCE questionnaire has been presented in Table 2.

We sent our website link containing our DCE questionnaire through social media apps such as Facebook and WeChat. In addition, we gave respondents some rewards such as Mi bands or a small cash payment if the respondents could fully complete the questionnaire.

428 participants (aged 18-85 years) who provided complete data were included in the analysis. Among those with complete data, 206 (48.1%) were male and 222 (51.9%) were female, while 2 of them were pregnant women.

The average utility values of all the attributes’ levels were measured using the Utility Scaling Method with zero-centered differences. The highest utility levels of the six hypothesized attributes are “AI diagnosis + clinicians’ confirmation” (of “diagnosis method”), “20 min” (of “outpatient waiting time”), “15 min” (of “diagnosis time”), “100%” (of “accuracy”), “Yes” (of “follow-up”), and “¥0” (of “diagnostic expenses”), respectively. The most important attribute is “accuracy,” meaning that the majority of respondents ranked that attribute (Table 3).

In general, it is obvious that attribute “accuracy” was the most important factor and most preferred when facing the diagnosis. As is presented in Figure 1, the percentage importance of attribute “accuracy” was 39.29%, which undoubtedly shows the position of accuracy of diagnosis in people’s minds. Attributes “diagnostic expenses” and “diagnostic method” ranked second and third, respectively.

The results of the logit analysis of all attributes’ levels are presented in Table 4. For the diagnosis method, the coefficient of level “AI + clinician” is positive, which means that the level “AI + clinician” is positively correlated with people’s preference and utility. In addition, the coefficients of levels “0 min,” “20 min,” and “40 min” (of “outpatient waiting time”) are positive, and others are negative. It is clear that people have a preference for shorter outpatient waiting time. For other attributes’ levels, people prefer to choose the higher accuracy level, greater possibility to follow up after diagnosis, and lower diagnosis expense. However, for the attribute “diagnosis time,” people unexpectedly prefer a longer diagnosis time rather than the “0 min” option.

For the P value of these attributes’ levels, we assume that if the P value of a level is less than .05, then this level is statistically significant; when the P value of a level is less than .001, then this level is extremely statistically significant. We found that “AI + clinician” and “AI diagnosis” for “diagnosis methods” are extremely statistically significant; “20 min” of outpatient waiting time is extremely statistically significant; “60 min” and “80 min” are statistically significant; “60%,” “70%,” “90%,” “100%” of “accuracy” are extremely statistically significant; both levels of attribute “follow up after diagnosis” are extremely statistically significant. “¥0’” of attribute “diagnostic expenses” is extremely statistically significant, and all the other levels of diagnostic expenses are statistically significant.

The odds ratio is a commonly used indicator in case-control epidemiological studies. In our analysis and calculation results (Table 4), we find that some odds ratios of the attributes’ levels compared to the reference level are greater than one, which means that the probability of people’s choosing of these levels is higher than the previous one or the reference. Taking the level “Clinician” of attribute “diagnostic methods” as the reference, level “AI + clinician” has an odds ratio of 1.559 (95% CI 1.446-1.682). The odds ratio of level “20 min” (of “outpatient waiting time”) is 1.136 (95% CI 1.014-1.274) with the reference of level “0 min.” Levels “15 min” and “30 min” (of “diagnosis time”) have odds ratios 1.077 and 1.067, respectively, with the reference of level “0 min” (95% CI 0.999-1.162 and 0.984-1.143, respectively). All the levels of the attribute “accuracy” were greater than one, which means the preference weights increase with the accuracy. Meanwhile, all the odds ratios of expense compared to the reference are smaller than zero, which refers to a preference of “free diagnosis” for the majority of people.

We compared these potential models and selected the model that maximized the area under the receiver operating characteristic curve and minimized the Akaike information criterion (AIC) [29,30] and Bayesian information criterion (BIC) [31] to penalize for model complexity. According to AIC, 5 classes should be the best choice for our model. However, if we must choose with BIC, then a 2-class option should be the most appropriate one, since 2-class has the lowest BIC. Under such circumstances, we compare ABIC, which means sample size–adjusted BIC [32] and involves sample size value. After comparing, the 3-class option has the lowest value of ABIC. Therefore, the most suitable number of latent classes in our model was 3 (Table 5 and 6). First of all, we divided all 428 respondents into 3 classes with segment sizes of 174 (40.7%), 30 (7.0%), and 224 (52.3%). The average maximum membership probability is around 0.87 and the percent certainty was 35.30, which is relatively low, meaning that there was not much uncertainty according to the respondents divided into classes.

For class 1, t ratios for attributes “diagnostic expenses,” “‘accuracy,” and “follow-up after diagnosis” were significant across all treatment modalities. Attribute “accuracy” was the most important factor for patients with 52.8%, followed by “diagnostic expenses” and “follow-up after diagnosis” with percentage importance 13.51% and 13.10%, respectively (Figure 1). Meanwhile, the span of percentage weights of “accuracy” is obvious (Figure 2), from −2.33 to 2.52. Meanwhile, the span of attribute “diagnostic expenses” is from −0.819 to 0.423. The preference weights of “diagnostic expenses” decrease with the increasing of the expenses. The preference weight of attribute “follow-up after diagnosis” is from −0.602 to 0.602. This is −0.602 for “No follow-up” and 0.602 for “follow-up”, which is symmetrical. In addition, the odds ratio of level “AI + clinicians” was 1.247 (95% CI 1.036-1.463), meaning that the majority of patients prefer durable treatments over a single treatment. As well, all the levels of attribute “diagnosis time” with the reference level “0 min” are larger than one. At the same time, the odds ratio (Table 7) of the levels of attribute “accuracy” increases with the accuracy rate, meaning that people’s preference weight increases with the accuracy.

For class 2, the attribute “accuracy,” as well as “diagnostic expenses,” was relatively important. Meanwhile, “diagnostic expenses” was (surprisingly) the most important for respondents among the attributes with a percentage weight of 39.09%, followed by “accuracy” with a percentage of 31.75% (Figure 1). The span of the percentage weights of these two attributes has been presented in Figure 3. The percentage weights for attribute “accuracy” were from −1.717 to 1.480, and −2.10 to 1.830 for “diagnostic expenses” (Figure 3). From P values, we find that almost all the levels are not statistically significant except the “100%” level of “accuracy” and level “¥50” of “diagnostic expenses.” The odds ratio (Table 7) displays that the AI method is the best of three methods. The odds ratio of levels of “outpatient waiting time” are all greater than 1, and patients’ preference weight still increases with the increasing of “accuracy” as in class 1. For attribute “diagnostic expenses,” only the level “¥50” has an odds ratio greater than 1.

For class 3, attributes “diagnosis method” and “outpatient waiting time” were more statistically significant across all the attributes. For respondents in this class, the percentage importance of “diagnosis method” and “outpatient waiting time” were 29.09% and 30.83%, respectively (Figure 1). From Figure 4, we find that the span for diagnostic methods is from −0.364 to 0.518. Meanwhile, the span for diagnostic expenses is from −0.538 to 0.397. In addition, the odds ratio of level “AI + clinicians” of attribute “diagnostic method” was 1.985 (95% CI 1.769-2.167), which is greater than 1, meaning that respondents in class 3 also prefer durable diagnosis mode rather than single mode, similar to the condition of class 1. All of the odds ratios of levels of attributes “outpatient waiting time” and “accuracy” are greater than 1. Meanwhile, the odds ratio continues increasing with the increase of accuracy, the same condition of the previous two classes.

For these 3 latent classes, attribute “accuracy” is the most preferred factor in two classes (class 1 and class 2), while “diagnostic expenses” is the most preferred factor in class 3. In addition, the odds ratio of levels of attribute “accuracy” always increases with the increasing of accuracy rate, meaning that people’s preference will always grow with the increase, and higher accuracy is always preferred.

Several different models were used in our study to research people’s preference for various attributes, including a GMNL model and LCM; both models presented with their own merits and shortcomings.

For the GMNL analysis and the utility report, we found that “accuracy” was the most important thing for respondents and most of the levels of attributes “diagnostic methods” and “accuracy” were statistically significant. From the odds ratio, most of the respondents have a preference for the AI diagnosis plus clinicians’ confirmation with outpatient waiting time of 20 minutes, diagnosis time of 15 minutes, and 100% accuracy. We would expect that the outpatient waiting time of 0 minutes and diagnosis time of 0 minutes should be preferred. However, the preferred choice was not 0 minutes. We assume that some people believe that a longer diagnosis time can give them more credibility and a feeling of safety. Furthermore, the odds ratio of 1.559 for “AI diagnosis + clinicians’ confirmation” was evidently higher than only “AI diagnosis,” which reflects that, for now, the vast majority of people still cannot totally trust the AI diagnosis due to the factors of AI uncertainty. AI diagnosis acting in concert with clinicians will significantly guarantee the accuracy. Another defect of AI diagnosis is that patients follow up after diagnosis, which means that patients who cannot receive follow-up have no alternatives but to head to clinicians as outpatients, which indirectly results in missing the best patient treatment time. That also leads to huge inconvenience to both patients and clinicians. Those who have an evident preference for only AI diagnosis may as a result of freshness and relatively higher diagnosis time of AI method, with sacrifice of accuracy in their mind. Thus, we can still consider the clinicians’ diagnosis to remain irreplaceable, at least in the short term. The limitation of the GMNL model was also obvious; some respondents with preferences for other diagnostic methods and different diagnosis expenses cannot be reflected.

From the LCM, our study finds that respondents who are divided into 3 classes show different preferences and different patient profiles. There was slight heterogeneity compared to the GMNL model. Specifically, respondents in class 1 and class 2 still attach importance to “‘accuracy”; however, respondents in class 3 mostly pay attention to “diagnostic expenses.” Although medical insurance and accessibility are quite advanced today, some patients hardly get access to basic medical diagnosis or treatment, particularly in some underdeveloped or remote areas in China. Some old people would rely on their own self-healing function or immune system [33] rather than go to the hospital due to their outdated concepts of unaffordable diagnosis and treatment expenses. Therefore, the acceptability of AI diagnosis or even modern medical techniques among old people is palpably lower than among new generations in China due to the concepts of cost. In addition, we hypothesized that the numbers of hospitals in remote areas is undoubtedly lower than those in urban or advanced areas, which results in the people living in remote areas having to undertake the transportation time and cost. The long transit time and cost could also sometimes be fatal to these people since that would also force them to stay at home and miss the most appropriate diagnosis and treatment time. To sum up, the promotion and spread of AI diagnosis cannot ignore the need to set a suitable diagnosis price or give some discount and bonus according to the wealth status of patients.

Several previous studies [13] have found that most patients believe the outpatient waiting time plays a vital role in their decision-making behavior when faced with various choices of hospitals and clinicians. In addition, few studies [14] have attached importance to the quality and quantity of follow-up after clinical diagnosis. However, all of these studies ignored the effect of diagnosis accuracy and diagnosis expense in patients’ trade-offs. Especially in the era of artificial intelligence, there are few studies addressing the accuracy of AI versus human clinicians rather than the outpatient waiting time or other factors. With diagnostic accuracy, clinically, the AI system can be programmed to probe and mark out some cancer indications such as prostate cancer and is more accurate than experts [34]. The clinical application could reduce pathology workload in this epidemic and in future clinical work. An AI system with expert-level grading performance might contribute a second opinion, aid in standardizing grading, and provide expert advice in areas with poor sanitation. As cloud computing capabilities come closer to life, overcomes limited memory, and central processing unit power [35]. We believe ever more and more people will trust the rapid diagnostic capabilities of AI. There is a lot of SARS-CoV-2–related scientific research undertaken to use AI to combat this pandemic by deploying new methods in the development of vaccines and drugs, as well as for public awareness [36]. Relying on the advantages of AI for medical auxiliary diagnosis, image analysis, remote consultation, etc during the outbreak of COVID-19, many AI devices have been used in first-line medicine. Moreover, AI is reducing cross infection. It has played an important role in therapeutic innovation. Health QR Code, which is a fusion of AI and big data, is a mobile phone app for everyone and uses red, yellow, and green colors to provide simple and effective intelligent services for personnel communication and economic and trade exchanges in the China in the “postepidemic era.”

AI will continue to play an increasingly important role in controlling the public health crisis to save lives and economic recovery. AI contact tracing based on mobile communication technology becomes more mature [37]. The AI systems combining computed tomography and clinical symptoms can help to quickly diagnose SARS-CoV-2 patients [10]. A surrogate rapid diagnosis technique that is based on a deep learning neural network can be applied for discovering SARS-CoV-2 by analyzing the visual chest radiography imaging of patients [38].

Segments of patients’ preferences for these diagnosis options seem to be homogenous and convergent. All the attributes hypothesized and attributes’ levels are evidently not ignorable during the implementation and widespread use of AI diagnosis techniques. People’s preference for “accuracy” was obvious across different classes. Although “online treatment” has become more common today, accuracy has been sacrificed in exchange for so-called convenience, which is totally unwise. In addition, AI diagnosis technique developers as well as technology sellers, including hospitals, should take diagnosis expense into consideration and make pricing rules more flexible in the light of areas’ economic development and individual patients’ wealth status.

AI will definitely have a potential market and bright future, especially in the ongoing COVID-19 pandemic, since the AI diagnostic technology can ease the requirements of professional clinicians worldwide, particularly in rural areas.