This study was approved by the institutional Clinical Research Ethics Committee of Xiangya Hospital (No.2021101068).

This study collected information about consecutive multicenter patients with psoriasis aged 18 years and older from January 2015 to July 2021.

A data set of 14,096 images from 2367 patients with psoriasis taken by digital cameras or smartphones was established for this study. We used 1962 patients from January 2015 to April 2021 to train the model and the other 405 patients from May 2021 to July 2021 to validate it. All patients gave their informed consent before collecting images. Each image was captured with either a digital camera (SONY DSC-HX50) or a smartphone (Apple iPhone 6s to 12). Three clinicians who have more than 10 years of experience in assessing psoriasis severity were invited to label the images independently. The final PASI scores of each image were obtained from the average of their scores, which can be viewed as ground truth labeling. To further ensure the reliability of the data, each data point was also checked by at least one professor-level doctor. Considering the images were taken under different lighting conditions, we normalized the input images to ensure the generalizability of the model. An offline automatic color enhancement algorithm [19] was implemented to reduce the influence of varied color and luminance conditions.

A severity assessment model was proposed to simulate how dermatologists calculate PASI scores in the clinical practice (Figure 1). The model receives N lesion images from each body part as inputs and predicts the redness of erythema (0-4), the thickness of induration (0-4), the scaling of desquamation (0-4), and the area ratio (0-6).

This model consists of an image-processing block, a multiview feature enhancement block, and a cross-revise output block. During the training stage, the image-processing block first conducts random cropping of the input image and then resizes the cropped image to 800 × 1024 pixels. Then, a convolutional neural network (CNN) encoder is used to extract visual features from the processed image, in which another attention branch is specially designed to generate attention features for the regions of interest (ROIs). In this task, ROIs denote the regions with skin lesions in the input images. Afterward, the visual feature and the attention feature are combined as the final feature representation, in which the features unrelated to ROIs are suppressed. In this study, a basic EfficientNet-B0 model was applied as the CNN encoder, and 2143 images containing manually skin lesions marked by bounding boxes served as the supervisory signals to the attention branch in the heat map mask format.

Furthermore, a multiview feature enhancement block that merges features from multiple input images was designed to enable the model to combine vision features from different perspectives. Specifically, this block handles the visual and attention features by calculating their maximum and average values to convert these image-level features into patient-level features. Then, the multihead output block uses the obtained patient-level features as input and sends them into 2 output headers with different optimization objectives, that is, a regression header with a smooth L1 loss and a classification header with a softmax cross-entropy loss. Moreover, we designed an extra cross-teacher loss [20] that revises the output of either output header by the other headers to learn more prior knowledge from the severity score annotated by dermatologists. Finally, the regression output is used as the final output. This may simulate the assessment method used by dermatologists in real clinical practice.

Patients who visited hospitals from May 2021 to July 2021 were included in a separate validation cohort, which was conducted as a prospective analysis.

The accuracy of the predicted PASI score was first evaluated. The mean average error (MAE) was used as the evaluation metric, which measures the average squared difference between the estimated values and the target values. In this study, we measured the absolute distance from the predicted score obtained from our model to the actual PASI score diagnosed by dermatologists. Besides the overall score, the accuracy of the 4 subscores of PASI was also evaluated. Erythema, induration, and desquamation are represented by integer values ranging from 0 to 4, and the area ratio is represented by an integer value ranging from 0 to 6. Therefore, we formulated the subscore predictions as 5-label and 7-label classifications, respectively, and used the classification accuracy as the evaluation metric. The trend predicted by the proposed model was further evaluated. If the PASI score increases, it potentially indicates a deterioration of psoriasis. If the PASI score decreases, it potentially indicates an improvement of psoriasis.

The proposed neural model was compared with 43 experienced dermatologists from 18 hospitals. Similar to the model validation, 2 perspectives revealed the comparison: (1) the accuracy of PASI scoring for each visit and (2) the severity ranking between 2 visits. To conduct a fair comparison, we provided the same skin lesion images that were not included in the training and testing data set to both the dermatologists and the proposed model. A labeling website was built for dermatologists to annotate the images on the web.

In all, 429 patients’ visits were followed up. Among them, 214 visits were of low severity (PASI≤5), 119 visits were of medium severity (5<PASI≤10), and 96 visits were of high severity (PASI>10). All pairs of these visits were enumerated, that is, C²₄₂₉=91,806 pairs in total. For each pair, we predicted the PASI score of these 2 visits. If the order of the predicted PASI scores was the same as the order of manual annotated PASI scores, we regarded it as a successful trend prediction; otherwise, we regarded it as an incorrect prediction.

We deployed the proposed severity assessment model into an app named SkinTeller on the WeChat platform. WeChat is a social network platform that is similar to WhatsApp and Line. WeChat processes 1.2 billion monthly active users [21]. Since WeChat allows users to launch mini-apps without installation or configuration, the app provides convenient access to the proposed model for both dermatologists and patients with psoriasis.

In this study, we integrated the proposed image-AI–based PASI-estimating model into the SkinTeller app and optimized the process of psoriasis severity assessment (Figure 2). Patients were allowed to take images of skin lesions and complete their medical history. The proposed model should output the PASI score with higher accuracy, and its consistency should outperform the actual clinical applications. We are also exploring making decisions based on psoriasis diagnosis and treatment guidelines and the experience of doctors and designing the product to automatically classify psoriasis severity into mild, moderate, and severe conditions based on the score and provide treatment recommendations. For example, patients with mild conditions should be treated with phototherapy and topical drugs, whereas patients with severe conditions should be recommended to biologic treatment. This work is important for both doctors and patients.

Generally, with more photos, the model has more information about the skin condition and therefore is more likely to predict an accurate PASI score. As a result, the first ablation study was conducted on the number of input lesion images per body part. Table 1 illustrates the performance of the model with different numbers of input images. The MAE decreased from 3.47 to 2.25 when the number of input image increased from 1 to 2. When entering 3 images, the performance of our model reached the peak and achieved the smallest MAE of 2.05. By the first ablation study, the error of the model is at minimum when given 3 images. In other words, this situation is closer to the real PASI score. The next ablation study compared different fusion strategies for multiple images to reveal the performance of the multiview feature enhancement block. As shown in Table 1, the combination of multiple aggregation methods outperformed the methods individually, whether in the total score or the 4 subindicators of PASI. The third study evaluated the performance of different output headers: the classification header, the regression header, and the cross-teacher header. According to the definition of the PASI score, the values of erythema, induration, desquamation, and area ratio are all discrete integers. In this manner, we formulated severity rating as a classification task. However, considering the ordering of different values, the task can also be formulated as a regression problem. According to Table 1, the aggregation of both the regression header and the classification header with the cross-teacher header outperformed other approaches.

For different groups of patients with varied degrees of severity, the performance of the proposed model decreased when the degree of severity increased (Table 1). For patients with low and medium severity, the MAE of the overall PASI score was less than 2; for patients with high severity, the MAE enlarged to 3.29. This is because the PASI score includes multiplication operations between subscores. Since the values of these subscores are already very large, a small variation of one subscore could result in a big variation of the total PASI score.

In all, 43 dermatologists from 18 hospitals participated and finished the labeling. They consisted of 13 professors (at least 15 years of dermatological experience) and 30 attending or resident physicians (at least 5 years of dermatological experience). As shown in Table 2, the proposed model outperformed dermatologists with a 33.2% improvement in the total PASI score and 23%, 7%, 11%, and 12% improvements in accuracy in the 4 subscores of erythema, induration, desquamation, and area ratio, respectively. The greatest improvement is erythema, which is also because it has the strongest subjectivity. The proposed model ranked eighth among all 43 dermatologists using MAE as the main measurement (dermatologist mean MAE=4.67; range 2.51-7.49; proposed model MAE=3.12). The comparison proved the effectiveness of the proposed severity rating model, considering that the majority of participant dermatologists are experienced in PASI scoring.

The accuracy in predicting the direction of severity progress with different ranges of PASI scores were further evaluated (Table 3). The average trend accuracy was 84.81%, indicating that the proposed model could accurately predict the trend of the severity degree for over 80% of cases. We also found that if the score gap between 2 visits is larger than 5 PASI score points, that is, if the severity degree changes from severe to moderate or from moderate to mild, we could predict the event at an accuracy level as high as 96.09%. However, the interagreement between dermatologists was poor. For top-ranked dermatologists, they tended to better agree with each other, whereas low-ranked dermatologists were more likely to disagree with each other (Multimedia Appendix 3).

A complete treatment course of a patient with psoriasis was tracked. Figure 3 shows the PASI scores of the proposed system and doctors at different treatment phases. The trend of the severity was generally consistent, whereas there were some small discrepancies in the scores from the AI and 3 doctors. The AI and all 3 doctors assigned psoriasis scores over 15 as the original status. Large areas of erythema and desquamation were observed all over the skin surface of the patient, and most of the subscores for the erythema metric were 3 or 4 (Figure 3 and Table 4). After the first treatment phase, the patient exhibited a substantial treatment response with the disappearance of lesions on the head with all scores being 0. The PASI score suggested that the treatment achieved substantial improvements in the erythema, induration, and desquamation of the patient’s lesions. However, due to the excessive size of the patient’s lesions, the PASI scores of the lower limbs assigned by the AI and all 3 doctors remained high after the first treatment phase, especially in the area ratio (PASI scores of 2, 3, 3, and 2, respectively) and erythema (PASI scores of 3, 3, 2, and 2, respectively). After 2 phases of treatment, the AI and all 3 doctors considered the lesions on the head and upper limbs to be completely relieved, which resulted in a PASI score of 0.

Similarly, we reported on another patient with psoriasis to present the PASI scores between AI and dermatologists during different treatment phases (Multimedia Appendix 4).

The proposed severity assessment model was deployed as an app on the WeChat platform. Functions and the user interface are shown in Figure 4. Relying on physician resources and high-quality algorithms, the SkinTeller app provides the function of AI-driven diagnosis, generic clinical science knowledge, health consultation, and recommendations to suitable hospitals and doctors. Doctors can also conduct follow-ups on the web. Patients can easily access the specialist-level services, and doctors in primary hospitals might also reach the clinical level of domain experts (Figure 4).

Furthermore, 43 dermatologists from different levels of hospitals were interviewed about SkinTeller (Multimedia Appendix 5). In all, 91% (39/43) of dermatologists perceived SkinTeller as helpful, relieving them from tedious and error-prone PASI scoring, especially dermatologists from country or community hospitals. Additionally, 86% (37/43) considered the SkinTeller app to have efficiently and accurately rated the degree of psoriasis severity. Regarding the management tool for chronic disease, 72% (31/43) confirmed that it might assist them in arranging the re-examination schedule and reminding their patients automatically. Almost all dermatologists (42/43, 98%) were willing to recommend the SkinTeller to colleagues (Table 5).

Regarding potential disadvantages of SkinTeller, 88% (38/43) of dermatologists recommended further improvements by providing alternative treatment options and predicting the possible patient outcomes for the doctor.

To solve the problem of PASI scoring in the real clinical practice, an image-AI–based system was developed based on a large multicenter database of patients with psoriasis. In this system, a multiview deep feature enhancement block to combine features from multiple perspectives was designed to simulate the clinical process of PASI calculation as dermatologists. The proposed model outperforms the average performance of 43 experienced dermatologists. Perhaps most importantly, we deployed a mobile app, SkinTeller, integrated with the model, which has been used 3369 times for PASI scoring in 1497 patients from 18 hospitals, and its excellent performance was confirmed by a feedback survey of 43 dermatologist users. This study may provide a promising alternative for accurate assessment and self-management of psoriasis.

About dilemmas in PASI, we conducted a multicenter questionnaire-based investigation among 346 dermatologists (Multimedia Appendix 2). Only 30.9% dermatologists preferred to evaluate the severity of psoriasis by scales. Two major disadvantages of PASI were recognized—having difficulties in calculating the score and the subjectivity of results—which made the evaluating procedure time-consuming and results unreliable. The shortcomings of each part of PASI were probably the sources of prejudice by the doctors. Our solution potentially overcomes the shortcomings of PASI and augments the clinical workflow of psoriasis management. First, SkinTeller integrated with the proposed image-AI–based severity assessment model provides the function of intelligent psoriasis severity rating and recommends personalized treatment. Second, an effective chronic disease management tool in the SkinTeller assists in tracking the condition. Third, under the support of physical hospitals and the construction of internet hospitals, multiple rounds of interactive self-inspection systems and intelligent referral and triage systems are formed.

Prior to our study, Schaap et al [15] constructed a system for automating PASI scores. Fink et al [16] proposed a filter-based image processing pipeline to determine the PASI score, but a limitation is the nonstandardized body posture of patients with immobility or noncooperation when collecting images. In our study, we have established a multicenter collaboration system for psoriasis data collection and formed a psoriasis data collection standard. The real data from patients with psoriasis that had the blur and occlusion removed were used to establish the model, which is fundamental to our study. First, we collected data from 21,524 patients with psoriasis with 5-20 clinical images per patient on average, resulting in the large sample size of the database. Recent studies reported the performance of an image-based automated deep learning system trained using 1731 images [15], and the observer-independent assessment system of psoriasis-affected areas included a total sample of only 259 photographs [20]. Second, the reference value of the PASI was calculated by 3 doctors independently with the supervision of 3 senior doctors, which ensures that the value is reliable. Of course, there are the other score systems such as Physician Global Assessment (PGA) and Body Surface Area (BSA) that are also commonly used in clinic. These scoring methods have different emphases and evaluation indexes. Our system is trained and developed based on PASI score data and outperforms those of dermatologists. However, after changing the task to PGA or BSA scores, the performance can be reduced because the training is inconsistent with the task. In the future, we will try other score methods, including PGA score, BSA score, etc, for psoriasis and strive to upgrade the system based on existing technologies [22] for most scenarios.

Our team has long been researching the intelligent diagnosis and treatment of psoriasis [23]. Besides the large-scale psoriasis database mentioned above, we also proposed a deep learning system to estimate the overall severity score and all intermediate metrics. In a novel approach to model construction [24], skin lesions were first mapped into a special-designed color space and then classified into 3 different types [25]; erythema scores were scored with the K-nearest neighbor algorithm [26], and a Gaussian mixture model was applied to segment skin lesions into different color channels and then scored based on the trichromatic bands. For CNN-based approaches, one model [27] estimates the severity levels of 3 indicators (erythema, induration, and desquamation) by adding 3 output heads in a single deep neural network [28]. These existing studies mainly focus on a partial list of aspects related to the severity of psoriasis and only generate severity levels with 5 discrete severity categories. They do not meet the clinical needs of psoriasis treatment, which requires an overall severity level for convenience and a larger and finer score range to monitor the severity in a timely and precise manner. If the patient’s PASI score decreases after treatment, then the management plan is effective; otherwise, the management plan must be adjusted. In this paper, we proposed to quantitatively estimate the overall severity score and all intermediate metrics. This method makes the psoriasis severity rating more convenient and consistent, especially when the difference in scores between 2 visits is larger than 5, indicating that the model showed clinically satisfying consistency in patients with psoriasis with substantial disease changes. Second, in the clinical diagnosis of psoriasis, multiple perspectives can bring more information to dermatologists. Therefore, to approximate the diagnosis scenario, we designed a multiview feature enhancement block, which integrates features from multiple input images to combine visual features from different perspectives. Third, our system has been comprehensively verified by comparison with doctors, ablation experiments, questionnaire feedback, and so on. In the ablation experiment, 3 input images were considered to have the smallest MAE. Lastly, our system can also assist doctors in making treatment decisions. On this basis, we also developed an app for the whole course management. This system will become a reliable reference for dermatologists to manage patients, predict progression in the long term, and avoid limitations in the evaluation results caused by the subjective judgment of clinicians.

Our study also has limitations. First, our clinical data set was limited to patients of Chinese origin, which would require retraining with local pictures if our approach were to be established elsewhere. Skin color would be expected to have some influence on the accuracy of the PASI score, especially on the subscore of erythema. Second, the data we used were collected from many institutions, which is conducive to the more general applicability of the development results but may also include bias by the different devices used for photography, environmental factors, and so on. Third, future work will be required to develop more user-friendly functions on SkinTeller. We also call for more efforts to assess the impact of this model and resulting software on clinical practice.

The proposed image-AI–based system can provide more objective and accurate assessment for severity of patients with psoriasis and outperforms experienced dermatologists. It can also predict the direction of severity progress with different ranges of PASI scores precisely. The SkinTeller app integrated with the proposed system show enormous potentialities for precise personalized treatment and chronic disease self-management in patients with psoriasis.