This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Psoriasis is one of the most frequent inflammatory skin conditions and could be treated via tele-dermatology, provided that the current lack of reliable tools for objective severity assessments is overcome. Psoriasis Area and Severity Index (PASI) has a prominent level of subjectivity and is rarely used in real practice, although it is the most widely accepted metric for measuring psoriasis severity currently.
This study aimed to develop an image–artificial intelligence (AI)–based validated system for severity assessment with the explicit intention of facilitating long-term management of patients with psoriasis.
A deep learning system was trained to estimate the PASI score by using 14,096 images from 2367 patients with psoriasis. We used 1962 patients from January 2015 to April 2021 to train the model and the other 405 patients from May 2021 to July 2021 to validate it. A multiview feature enhancement block was designed to combine vision features from different perspectives to better simulate the visual diagnostic method in clinical practice. A classification header along with a regression header was simultaneously applied to generate PASI scores, and an extra cross-teacher header after these 2 headers was designed to revise their output. The mean average error (MAE) was used as the metric to evaluate the accuracy of the predicted PASI score. By making the model minimize the MAE value, the model becomes closer to the target value. Then, the proposed model was compared with 43 experienced dermatologists. Finally, the proposed model was deployed into an app named SkinTeller on the WeChat platform.
The proposed image-AI–based PASI-estimating model outperformed the average performance of 43 experienced dermatologists with a 33.2% performance gain in the overall PASI score. The model achieved the smallest MAE of 2.05 at 3 input images by the ablation experiment. In other words, for the task of psoriasis severity assessment, the severity score predicted by our model was close to the PASI score diagnosed by experienced dermatologists. The SkinTeller app has been used 3369 times for PASI scoring in 1497 patients from 18 hospitals, and its excellent performance was confirmed by a feedback survey of 43 dermatologist users.
An image-AI–based psoriasis severity assessment model has been proposed to automatically calculate PASI scores in an efficient, objective, and accurate manner. The SkinTeller app may be a promising alternative for dermatologists’ accurate assessment in the real world and chronic disease self-management in patients with psoriasis.
Psoriasis is a chronic, immune-mediated disease that causes pain, disfigurement, and disability, for which there is no cure, as recognized by the World Health Organization [
Psoriasis Area and Severity Index (PASI) is the most widely accepted metric for measuring psoriasis severity currently [
Deep neural models have been proposed for diagnosing and evaluating multiple skin diseases [
In this study, we proposed a deep learning system based on a psoriasis-specific image database and developed a mobile app (named SkinTeller) integrated with this model to assess psoriasis severity in clinical practice. Results from multiple centers highlight the potential to achieve precise treatment and management for psoriasis.
This study was approved by the institutional Clinical Research Ethics Committee of Xiangya Hospital (No.2021101068).
This study collected information about consecutive multicenter patients with psoriasis aged 18 years and older from January 2015 to July 2021.
A data set of 14,096 images from 2367 patients with psoriasis taken by digital cameras or smartphones was established for this study. We used 1962 patients from January 2015 to April 2021 to train the model and the other 405 patients from May 2021 to July 2021 to validate it. All patients gave their informed consent before collecting images. Each image was captured with either a digital camera (SONY DSC-HX50) or a smartphone (Apple iPhone 6s to 12). Three clinicians who have more than 10 years of experience in assessing psoriasis severity were invited to label the images independently. The final PASI scores of each image were obtained from the average of their scores, which can be viewed as ground truth labeling. To further ensure the reliability of the data, each data point was also checked by at least one professor-level doctor. Considering the images were taken under different lighting conditions, we normalized the input images to ensure the generalizability of the model. An offline automatic color enhancement algorithm [
A severity assessment model was proposed to simulate how dermatologists calculate PASI scores in the clinical practice (
This model consists of an image-processing block, a multiview feature enhancement block, and a cross-revise output block. During the training stage, the image-processing block first conducts random cropping of the input image and then resizes the cropped image to 800 × 1024 pixels. Then, a convolutional neural network (CNN) encoder is used to extract visual features from the processed image, in which another attention branch is specially designed to generate attention features for the regions of interest (ROIs). In this task, ROIs denote the regions with skin lesions in the input images. Afterward, the visual feature and the attention feature are combined as the final feature representation, in which the features unrelated to ROIs are suppressed. In this study, a basic EfficientNet-B0 model was applied as the CNN encoder, and 2143 images containing manually skin lesions marked by bounding boxes served as the supervisory signals to the attention branch in the heat map mask format.
Furthermore, a multiview feature enhancement block that merges features from multiple input images was designed to enable the model to combine vision features from different perspectives. Specifically, this block handles the visual and attention features by calculating their maximum and average values to convert these image-level features into patient-level features. Then, the multihead output block uses the obtained patient-level features as input and sends them into 2 output headers with different optimization objectives, that is, a regression header with a smooth L1 loss and a classification header with a softmax cross-entropy loss. Moreover, we designed an extra cross-teacher loss [
The structure of the proposed image–artificial intelligence (AI)–based Psoriasis Area and Severity Index (PASI) assessment model, which refers to the PASI score rating module. Avg: average; C: channel; conv: convolution; FC: fully connected; H: height; ROI: region of interest; SE: squeeze and excitation; W: width.
Patients who visited hospitals from May 2021 to July 2021 were included in a separate validation cohort, which was conducted as a prospective analysis.
The accuracy of the predicted PASI score was first evaluated. The mean average error (MAE) was used as the evaluation metric, which measures the average squared difference between the estimated values and the target values. In this study, we measured the absolute distance from the predicted score obtained from our model to the actual PASI score diagnosed by dermatologists. Besides the overall score, the accuracy of the 4 subscores of PASI was also evaluated. Erythema, induration, and desquamation are represented by integer values ranging from 0 to 4, and the area ratio is represented by an integer value ranging from 0 to 6. Therefore, we formulated the subscore predictions as 5-label and 7-label classifications, respectively, and used the classification accuracy as the evaluation metric. The trend predicted by the proposed model was further evaluated. If the PASI score increases, it potentially indicates a deterioration of psoriasis. If the PASI score decreases, it potentially indicates an improvement of psoriasis.
The proposed neural model was compared with 43 experienced dermatologists from 18 hospitals. Similar to the model validation, 2 perspectives revealed the comparison: (1) the accuracy of PASI scoring for each visit and (2) the severity ranking between 2 visits. To conduct a fair comparison, we provided the same skin lesion images that were not included in the training and testing data set to both the dermatologists and the proposed model. A labeling website was built for dermatologists to annotate the images on the web.
In all, 429 patients’ visits were followed up. Among them, 214 visits were of low severity (PASI≤5), 119 visits were of medium severity (5<PASI≤10), and 96 visits were of high severity (PASI>10). All pairs of these visits were enumerated, that is, C2429=91,806 pairs in total. For each pair, we predicted the PASI score of these 2 visits. If the order of the predicted PASI scores was the same as the order of manual annotated PASI scores, we regarded it as a successful trend prediction; otherwise, we regarded it as an incorrect prediction.
We deployed the proposed severity assessment model into an app named SkinTeller on the WeChat platform. WeChat is a social network platform that is similar to WhatsApp and Line. WeChat processes 1.2 billion monthly active users [
In this study, we integrated the proposed image-AI–based PASI-estimating model into the SkinTeller app and optimized the process of psoriasis severity assessment (
Overview of the workflow of the proposed model and the SkinTeller mobile app. (a) The image–artificial intelligence (AI)–based Psoriasis Area and Severity Index (PASI)–estimating model. (b) The workflow of the SkinTeller app that is integrated with the proposed model. (c) The clinical significance for both doctors and patients. CNN: convolutional neural network; F: female; M: male; N: no; Y: yes.
Generally, with more photos, the model has more information about the skin condition and therefore is more likely to predict an accurate PASI score. As a result, the first ablation study was conducted on the number of input lesion images per body part.
For different groups of patients with varied degrees of severity, the performance of the proposed model decreased when the degree of severity increased (
Ablation studies.
Item | Area ratio (%) | Erythema (%) | Desquamation (%) | Induration (%) | MAEa of PASIb | ||||||
|
|||||||||||
|
1 | 63.7 | 59.29 | 53.64 | 55.65 | 3.47 | |||||
|
2 | 70.69 | 63.41 | 56.23 | 62.55 | 2.25 | |||||
|
3 |
|
|
57.76 |
|
|
|||||
|
4 | 69.16 | 56.23 |
|
62.55 | 2.08 | |||||
|
|||||||||||
|
Only max | 64.94 | 59.39 | 55.65 | 61.4 | 2.73 | |||||
|
Only mean | 69.83 | 62.16 | 53.26 | 61.49 | 2.31 | |||||
|
Max + mean |
|
|
|
|
|
|||||
|
|||||||||||
|
Only regression | 68.87 | 61.49 | 57.95 | 60.82 | 2.14 | |||||
|
Only classification | 62.55 |
|
52.97 | 59.2 | 2.61 | |||||
|
Regression + classification | 69.73 | 63.89 |
|
63.31 | 2.21 | |||||
|
Regression + classification with cross-teacher |
|
64.56 | 57.76 |
|
|
|||||
|
|||||||||||
|
All (N=429) | 67.54 | 63.46 | 58.39 | 61.6 | 1.97 | |||||
|
Mild (0-5; n=214) | 70.27 | 59.1 | 53.64 | 59.1 | 1.70 | |||||
|
Moderate (6-10; n=119) | 66.10 | 67.61 | 66.67 | 58.33 | 1.48 | |||||
|
Severe (>10; n=96) | 63.46 | 67.31 | 57.14 | 71.98 | 3.29 |
aMAE: mean absolute error.
bPASI: Psoriasis Area and Severity Ratio.
cItalicized values indicate the best performance.
In all, 43 dermatologists from 18 hospitals participated and finished the labeling. They consisted of 13 professors (at least 15 years of dermatological experience) and 30 attending or resident physicians (at least 5 years of dermatological experience). As shown in
Performance comparison of the deep learning system versus the dermatologists (professors and attending or resident physicians). The proposed model achieved better results than dermatologists in the 4 subscores of Psoriasis Area and Severity Index (PASI), including erythema, induration, desquamation, and area ratio.
|
Erythema (%) | Induration (%) | Desquamation (%) | Area ratio (%) |
Dermatologists (professors) | 39 | 36 | 37 | 60 |
Dermatologists (attending or resident physicians) | 43 | 44 | 43 | 50 |
Dermatologists (combined) | 42 | 44 | 43 | 54 |
Proposed model | 45 | 42 | 43 | 55 |
The accuracy in predicting the direction of severity progress with different ranges of PASI scores were further evaluated (
The consistent trend between artificial intelligence and doctors among different Psoriasis Area and Severity Index (PASI) score gaps.
PASI score gap | Trend consistency (%) |
Average | 84.81 |
>10 | 98.79 |
6-10 | 96.09 |
0-5 | 73.26 |
A complete treatment course of a patient with psoriasis was tracked.
Similarly, we reported on another patient with psoriasis to present the PASI scores between AI and dermatologists during different treatment phases (
The practical application of the proposed image-AI–based PASI-estimating model. (a) The PASI scores between the AI and 3 doctors for a patient with psoriasis at different treatment phases. (b) The clinical images of the patient at different treatment phases. AI: artificial intelligence; PASI: Psoriasis Area and Severity Index.
The Psoriasis Area and Severity Index (PASI) scores between artificial intelligence (AI) and 3 doctors for a patient with psoriasis at different treatment phases.
|
Area ratio | Erythema | Induration | Desquamation | PASI score | |||
|
||||||||
|
|
18.4 | ||||||
|
|
Head | 1 | 2 | 1 | 1 |
|
|
|
|
Trunk | 2 | 3 | 2 | 3 |
|
|
|
|
Upper limb | 1 | 3 | 1 | 2 |
|
|
|
|
Lower limb | 3 | 4 | 3 | 3 |
|
|
|
|
17.2 | ||||||
|
|
Head | 1 | 2 | 1 | 1 |
|
|
|
|
Trunk | 2 | 3 | 2 | 3 |
|
|
|
|
Upper limb | 1 | 3 | 1 | 2 |
|
|
|
|
Lower limb | 3 | 3 | 3 | 3 |
|
|
|
|
19.3 | ||||||
|
|
Head | 1 | 1 | 1 | 1 |
|
|
|
|
Trunk | 2 | 3 | 1 | 2 |
|
|
|
|
Upper limb | 1 | 2 | 1 | 2 |
|
|
|
|
Lower limb | 4 | 3 | 3 | 3 |
|
|
|
|
17.3 | ||||||
|
|
Head | 1 | 2 | 1 | 1 |
|
|
|
|
Trunk | 2 | 3 | 1 | 3 |
|
|
|
|
Upper limb | 1 | 2 | 1 | 1 |
|
|
|
|
Lower limb | 3 | 4 | 3 | 3 |
|
|
|
||||||||
|
|
6.8 | ||||||
|
|
Head | 0 | 0 | 0 | 0 |
|
|
|
|
Trunk | 2 | 2 | 1 | 1 |
|
|
|
|
Upper limb | 1 | 1 | 0 | 1 |
|
|
|
|
Lower limb | 2 | 3 | 1 | 1 |
|
|
|
|
9.6 | ||||||
|
|
Head | 0 | 0 | 0 | 0 |
|
|
|
|
Trunk | 2 | 2 | 1 | 1 |
|
|
|
|
Upper limb | 0 | 0 | 0 | 0 |
|
|
|
|
Lower limb | 3 | 3 | 1 | 2 |
|
|
|
|
9.7 | ||||||
|
|
Head | 1 | 1 | 0 | 0 |
|
|
|
|
Trunk | 2 | 1 | 1 | 1 |
|
|
|
|
Upper limb | 1 | 1 | 1 | 1 |
|
|
|
|
Lower limb | 3 | 2 | 2 | 2 |
|
|
|
|
6.6 | ||||||
|
|
Head | 0 | 0 | 0 | 0 |
|
|
|
|
Trunk | 2 | 1 | 1 | 1 |
|
|
|
|
Upper limb | 1 | 2 | 1 | 1 |
|
|
|
|
Lower limb | 2 | 2 | 1 | 2 |
|
|
|
||||||||
|
|
3.5 | ||||||
|
|
Head | 0 | 0 | 0 | 0 |
|
|
|
|
Trunk | 3 | 3 | 2 | 3 |
|
|
|
|
Upper limb | 0 | 0 | 0 | 0 |
|
|
|
|
Lower limb | 2 | 2 | 1 | 1 |
|
|
|
|
6.6 | ||||||
|
|
Head | 0 | 0 | 0 | 0 |
|
|
|
|
Trunk | 2 | 2 | 1 | 2 |
|
|
|
|
Upper limb | 0 | 0 | 0 | 0 |
|
|
|
|
Lower limb | 3 | 1 | 1 | 1 |
|
|
|
|
5.6 | ||||||
|
|
Head | 0 | 0 | 0 | 0 |
|
|
|
|
Trunk | 2 | 1 | 1 | 2 |
|
|
|
|
Upper limb | 0 | 0 | 0 | 0 |
|
|
|
|
Lower limb | 2 | 2 | 1 | 1 |
|
|
|
|
4.2 | ||||||
|
|
Head | 0 | 0 | 0 | 0 |
|
|
|
|
Trunk | 2 | 2 | 1 | 1 |
|
|
|
|
Upper limb | 0 | 0 | 0 | 0 |
|
|
|
|
Lower limb | 2 | 1 | 1 | 1 |
|
The proposed severity assessment model was deployed as an app on the WeChat platform. Functions and the user interface are shown in
Furthermore, 43 dermatologists from different levels of hospitals were interviewed about SkinTeller (
Regarding potential disadvantages of SkinTeller, 88% (38/43) of dermatologists recommended further improvements by providing alternative treatment options and predicting the possible patient outcomes for the doctor.
Page introduction and service module of the mobile app SkinTeller. (A) On the first page, mobile app SkinTeller includes the function of severity score (severity rating and psoriasis diagnosis), screening (intelligent diagnosis), patient list (managing the patients who the current dermatologist is response of), statistics (providing data analysis of patients who the current dermatologist is response of), psoriasis knowledge (recommending the articles of psoriasis self-management), calendar reminder (reminding the patients of hospital revisits and medicine dosage), daily recording (recording the vital signs, such as blood pressure, pulse and the medical dosage history), questionnaire scale (Self-rating Anxiety Scale, Self-rating Depression Scale, Health-Related Quality of Life, etc) and nearly hospitals (providing information about nearby hospitals to facilitate patient treatment). (B) On the second page, multimodal data input page (including metadata and images). (C) On the third page, the example of photo guide which instruct the patient how to take pictures of each part of the body. (D) on the last page, the result page includes the overall PASI score as well as all 16 subscores. PASI: Psoriasis Area and Severity Index.
Feedback from 43 dermatologists on the use of the SkinTeller mobile app.
Question, response | Dermatologist (N=43), n (%) | ||
|
|||
|
Very helpful | 12 (28) | |
|
A lot helpful | 27 (63) | |
|
Generally helpful | 4 (9) | |
|
|||
|
Actively recommend | 21 (49) | |
|
Recommend | 21 (49) | |
|
It depends | 1 (2) | |
|
|||
|
Quickly and accurately assesses the condition of psoriasis | 37 (86) | |
|
Provides follow-up plans of patients for doctors and has a function to remind patients | 31 (72) | |
|
Can better guide treatment | 30 (70) | |
|
Can better judge the prognosis | 25 (58) | |
|
|||
|
The patient cannot cooperate | 19 (44) | |
|
Inconvenient operation | 17 (40) | |
|
The accuracy is not high | 11 (26) | |
|
Unreasonable application scenarios | 6 (14) | |
|
Others | 2 (5) |
To solve the problem of PASI scoring in the real clinical practice, an image-AI–based system was developed based on a large multicenter database of patients with psoriasis. In this system, a multiview deep feature enhancement block to combine features from multiple perspectives was designed to simulate the clinical process of PASI calculation as dermatologists. The proposed model outperforms the average performance of 43 experienced dermatologists. Perhaps most importantly, we deployed a mobile app, SkinTeller, integrated with the model, which has been used 3369 times for PASI scoring in 1497 patients from 18 hospitals, and its excellent performance was confirmed by a feedback survey of 43 dermatologist users. This study may provide a promising alternative for accurate assessment and self-management of psoriasis.
About dilemmas in PASI, we conducted a multicenter questionnaire-based investigation among 346 dermatologists (
Prior to our study, Schaap et al [
Our team has long been researching the intelligent diagnosis and treatment of psoriasis [
Our study also has limitations. First, our clinical data set was limited to patients of Chinese origin, which would require retraining with local pictures if our approach were to be established elsewhere. Skin color would be expected to have some influence on the accuracy of the PASI score, especially on the subscore of erythema. Second, the data we used were collected from many institutions, which is conducive to the more general applicability of the development results but may also include bias by the different devices used for photography, environmental factors, and so on. Third, future work will be required to develop more user-friendly functions on SkinTeller. We also call for more efforts to assess the impact of this model and resulting software on clinical practice.
The proposed image-AI–based system can provide more objective and accurate assessment for severity of patients with psoriasis and outperforms experienced dermatologists. It can also predict the direction of severity progress with different ranges of PASI scores precisely. The SkinTeller app integrated with the proposed system show enormous potentialities for precise personalized treatment and chronic disease self-management in patients with psoriasis.
Definition of the Psoriasis Area and Severity Index (PASI) scoring.
Questionnaire on the severity assessment of patients with psoriasis.
Interagreement of dermatologists.
The other case study.
Questionnaire on the use of the app (named SkinTeller).
artificial intelligence
Body Surface Area
convolutional neural network
mean absolute error
Psoriasis Area and Severity Index
Physician Global Assessment
region of interest
This work was supported by the Project of Intelligent Management Software for Multimodal Medical Big Data for New Generation Information Technology, Ministry of Industry and Information Technology of People’s Republic of China (TC210804V), the Hunan Provincial Innovation Foundation for Postgraduate, and the Fundamental Research Funds for the Central Universities of Central South University.
None declared.