This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.
Skin and subcutaneous disease is the fourth-leading cause of the nonfatal disease burden worldwide and constitutes one of the most common burdens in primary care. However, there is a severe lack of dermatologists, particularly in rural Chinese areas. Furthermore, although artificial intelligence (AI) tools can assist in diagnosing skin disorders from images, the database for the Chinese population is limited.
This study aims to establish a database for AI based on the Chinese population and presents an initial study on six common skin diseases.
Each image was captured with either a digital camera or a smartphone, verified by at least three experienced dermatologists and corresponding pathology information, and finally added to the Xiangya-Derm database. Based on this database, we conducted AI-assisted classification research on six common skin diseases and then proposed a network called Xy-SkinNet. Xy-SkinNet applies a two-step strategy to identify skin diseases. First, given an input image, we segmented the regions of the skin lesion. Second, we introduced an information fusion block to combine the output of all segmented regions. We compared the performance with 31 dermatologists of varied experiences.
Xiangya-Derm, as a new database that consists of over 150,000 clinical images of 571 different skin diseases in the Chinese population, is the largest and most diverse dermatological data set of the Chinese population. The AI-based six-category classification achieved a top 3 accuracy of 84.77%, which exceeded the average accuracy of dermatologists (78.15%).
Xiangya-Derm, the largest database for the Chinese population, was created. The classification of six common skin conditions was conducted based on Xiangya-Derm to lay a foundation for product research.
Skin conditions affect 1.9 billion people [
The total number of dermatologists in China is less than 30,000, which results in a large gap between the supply and demand of dermatological resources, especially in primary care hospitals. In addition, some long-standing problems in the Chinese medical industry still exist, such as the uneven distribution of high-quality doctor resources, primary care hospital doctors prone to misdiagnosing patients, missed diagnoses, and insufficient knowledge of rare diseases [
Using deep learning methods for skin disease identification requires a large amount of data. Skin images from the Chinese population are limited. Most public databases on skin diseases are from the US or European Union (EU) population, with a relatively small size and without corresponding pathology information to support their validity [
In this paper, we aim to build a database consisting of skin images based on the Chinese population and present an AI-based method for classifying six common skin diseases to lay a foundation for further product research.
As approved by the ethics committee of Central South University, we captured each image with either a digital camera or a smartphone in the Dermatology Outpatient Department, which relies on the Xiangya Big Data Collection Platform. The images were collected according to uniform standards from many cooperating hospitals in Nanjing, Guangzhou, Wuhan, Changsha (cities in China), etc. Most of these images have corresponding pathology information to ensure the accuracy of diagnosis information. For the other common and easy-to-diagnose skin diseases, images were verified by at least three experienced dermatologists. Meanwhile, the dermatologists were asked to annotate the specific location of the skin lesions in each image using an open source software called LabelImg [
After the construction of Xiangya-Derm, we conducted research on the classification of some common skin diseases from the Xiangya-Derm database using a convolutional neural network (CNN). First, six different subtypes of common dermatoses with additional data magnitudes and common outpatients (psoriasis, basal cell carcinoma, seborrheic keratosis, pemphigus, eczema, and lupus erythematosus) were selected, and a data set consisting of 5660 clinical images was constructed. These six subtypes of skin diseases were selected because they are the most common and representative skin diseases in primary care hospitals in China and because more data were available for these diseases than for any other. This data set contained common skin disease types, such as skin tumors, erythema, and scaling disease. In this data set, the image numbers of psoriasis and other diseases were unbalanced. If the data set had been used directly to classify each type of disease separately, it would have created a severe bias in the network against psoriasis. Therefore, we downsampled psoriasis samples to balance the number of samples among different categories. Moreover, we re-established a database containing six types of diseases, totaling approximately 3000 images, which ensured that the number of samples in different categories was within a reasonable range. In this range, the disease with the largest number of samples had over 1000 images, while the disease with the fewest samples had fewer than 200 images.
Based on a balanced number of samples in different categories, we proposed a network, called Xy-SkinNet, that was trained to be a six-category model for these six common types of diseases. Before we constructed the model, images in the database for modeling were checked. We found that the background of the clinical images in our database was complex and the size of lesions was relatively small. To overcome these problems and improve the classification performance, the model was designed to first detect the regions of interest that contained lesions. A specially designed information fusion block was then used to output the result by considering all the regions. The network used modified ResNet (M-ResNET) as the backbone network and Faster R-CNN [
Faster R-CNN is an improved version of R-CNN [
Structure of Xy-SkinNet. CNN: convolutional neural network.
To evaluate the performance of Xy-SkinNet and validate its effectiveness in a clinical situation, we conducted a contrast experiment with 31 dermatologists of different seniority levels, including professors, senior attending doctors, young attending doctors, and medical students of dermatology. In our contrast experiment, Xy-SkinNet and all dermatologists had to identify the type of disease from a set of 100 clinical images without a time limit. Their performances were evaluated using accuracy as the metric. To ensure the fairness and validity of the comparison, all the images used contained only one type of skin disease and had been previously verified using corresponding histopathological information. Histopathological examination was the golden standard for diagnoses of these six types of skin diseases. Compared with other diagnostic technologies, histopathological examination was more accurate and objective. If there were some cases whose histopathological features were atypical, we let another three specialists (who did not take part in human vs. computer competition) determine a final diagnosis. Images for this contrast experiment were selected by dermatologists who did not participate in the experiment, and the images were not included in the training or testing data sets used in the process of building of our model.
Xiangya-Derm, as a new Chinese database, contains nearly 20 years of data from 15 hospitals, including Xiangya Hospital Central South University, the Third Hospital of Central South University, and Henan Provincial People’s Hospital. Most images were matched with pathology information to ensure the accuracy of the image diagnostic information. Other common skin diseases that are easily recognized by the naked eye were independently confirmed by three dermatologists. Each patient’s clinical history was recorded on the Xiangya Big Data Collection Platform. It is worth mentioning that most of the data in the Xiangya-Derm database have been labeled with bounding boxes by skin experts (
There are over 150,000 images in the Xiangya-Derm database, including 571 different skin diseases. Approximately, 60,000 images have been assigned annotations of skin type and lesion location. Common skin diseases, such as benign and malignant skin tumors, erythema papule scales, allergic skin diseases, and bullous skin diseases, are included (
For common skin diseases, the diagnosis was independently made by three dermatologists based on history, clinical images, and dermoscopic features (
The database mainly consists of six types of skin diseases: malignant tumor, benign tumor, erythema papule scale, connective tissues, allergic, and bullous (
Pattern of the Xiangya Big Data Collection Platform.
Branching tree diagram of the Xiangya-Derm database.
The performance when using different CNN structures as the backbone model on the testing data set is shown in
The comparison between the experimental results of Xy-SkinNet and the average performance of dermatologists is shown in
Moreover, we recorded the time costs for Xy-SkinNet and each dermatologist in our experiments. The results showed that Xy-SkinNet takes only 10 seconds to identify all 100 clinical images, while some dermatologists took over 2500 seconds or over 25 seconds for a single image. Even dermatologists with minimal time costs took up to 1216 seconds to classify the images. These results showed that the method adopted in this paper can achieve comparable performance in terms of accuracy to that of dermatologists and outperform dermatologists in terms of time costs.
The comparison of accuracies (%) on six different skin diseases between Xy-SkinNet and the average of dermatologists with different levels in the external data set provided by dermatologists are shown in
Classification accuracies (%) of Xy-SkinNet.a
Backbone | PSOb (%) | BCCc (%) | SEKd (%) | PEMe (%) | ECZf (%) | LUEg (%) | Mean (%) | |
|
||||||||
|
ResNet-50 | 61.9 | 20.0 | 60.0 | 70.0 | 45.5 | 40.0 | 49.57 |
|
ResNet-101 | 54.8 | 20.0 | 70.0 | 80.0 | 59.1 | 50.0 | 55.65 |
|
ResNet-152 | 69.4 | 40.0 | 75.0 | 80.0 | 54.5 | 50.0 | 61.48 |
|
Xy-SkinNet | 64.1 | 50.0 | 70.0 | 75.0 | 83.9 | 45.5 | 64.75 |
|
||||||||
|
ResNet-50 | 66.7 | 70.0 | 75.0 | 90.0 | 68.2 | 60.0 | 71.65 |
|
ResNet-101 | 61.9 | 40.0 | 80.0 | 90.0 | 72.7 | 70.0 | 69.10 |
|
ResNet-152 | 71.6 | 80.0 | 90.0 | 90.0 | 72.7 | 60.0 | 77.38 |
|
Xy-SkinNet | 67.7 | 70.0 | 90.0 | 90.0 | 100.0 | 90.9 | 84.77 |
aThe classification accuracies (%) of Xy-SkinNet on six different skin diseases using different backbone networks as the feature extractor are shown. The ResNet-152 model showed better performance than the ResNet-50 and ResNet-101 models in terms of top 1 and top 3 mean accuracies. However, our original algorithm performed better than the others, especially in terms of the top 3 accuracies.
bPSO: psoriasis.
cBCC: basal cell carcinoma.
dSEK: seborrheic keratosis.
ePEM: pemphigus.
fECZ: eczema.
gLUE: lupus erythematosus.
Comparison of accuracy (%) between Xy-SkinNet and the average of dermatologists.
Class | Sample size | Top 1 (computer; %) | Top 1 (dermatologists; %) | Top 3 (computer; %) | Top 3 (dermatologists; %) |
LUEa | 11 | 45.5 | 60.6 | 90.9 | 70.0 |
ECZb | 31 | 83.9 | 47.5 | 100.0 | 76.9 |
SEKc | 10 | 70.0 | 60.8 | 90.0 | 77.9 |
PSOd | 93 | 64.1 | 75.7 | 67.7 | 87.2 |
BCCe | 10 | 50.0 | 67.4 | 70.0 | 84.9 |
PEMf | 20 | 75.0 | 60.8 | 90.0 | 72.0 |
All | 175 | 64.75 | 62.13 | 84.77 | 78.15 |
aLUE: lupus erythematosus.
bECZ: eczema.
cSEK: seborrheic keratosis.
dPSO: psoriasis.
eBCC: basal cell carcinoma.
fPEM: pemphigus.
ROC curve of the six-category classification model. The three blue lines represent the mean ROC curves over the ResNet50, ResNet101, and ResNet152 runs. Each triangular point represents the results of one dermatologist. Results suggested that the mean performance of the dermatologists was lower than that of the models for most diseases. ROC: receiver operating characteristic.
In this study, we established a new database, Xiangya-Derm, which consists of over 150,000 clinical images of 571 different skin diseases in the Chinese population. Xiang-Derm is the first integrated, normative database based on skin conditions in the Chinese population. Based on this database, we selected six common skin diseases and proposed an AI network, Xy-SkinNet. The top 1 and top 3 diagnostic accuracies of Xy-SkinNet were higher than those of dermatologists from the Department of Dermatology. This study was an attempt at exploring AI products and services and has successfully set the stage for future development. An increasing number of studies are incorporating clinical images [
There are already some open databases, such as AtlasDerm, Derm101, and Dermnet. Considering differences in skin color, Xiangya-Derm can provide data for realizing AI diagnosis of skin diseases among the Chinese population. Many existing databases lack medical history information, especially information about pathological diagnosis, and, potentially, contain some misdiagnosed photos. Notably, one of the greatest advantages of Xiangya-Derm is that most images contain corresponding skin pathology results, providing the category annotation of a gold standard, which can be most effectively applied to various research studies and in the development of AI. This feature ensures that the diagnostic information about pictures used for deep learning is accurate and reduces the diagnostic errors caused by misdiagnosis. Of course, there are also a small number of unmatched pictures in our database, which is correlated with the lack of corresponding dermoscopic images.
In addition, XiangyaDerm provides image data with the location for all skin lesions, thus enabling researchers to apply object detection algorithms in computer vision for the automatic diagnosis of skin diseases. Moreover, each image has a full set of clinical information about the patient, including demographic information, complaints, current medical history, past medical history, and family history. Given the complete set of big data, conducting further research on AI diagnosis using multimodal data, which is more coincident with the real-world diagnosis process and more intuitive for both doctors and patients, is achievable.
Many attempts have been made in the field of automatic diagnosis of dermatosis. For example, Alćonet et al [
CNNs have presented varying results in medical fields when using an improved algorithm [
Relevant research reports also increasingly support that deep learning systems can help improve the accuracy of nondermatologists or clinicians with a lack of experience, especially in primary care [
Because there are few dermatologists in China’s rural areas, we plan to build a multidimensional AI platform that assists Chinese primary dermatologists in diagnosing skin diseases through deep learning and the construction of Xiangya-Derm. In light of the lack of criteria for skin disease data sets, we would like to encourage the establishment of standards in the future. It would be beneficial to create a new data set or improve the existing data sets with multimodal clinical information.
Although we included six types of skin diseases in our data set, a large number of skin diseases were not considered. One of the reasons for this is that these six diseases cover most of the skin conditions in clinical practice. With this database and the development of AI products, new data for rare skin diseases can be collected at the same time as the diagnosis of common skin diseases. In addition, the quality of health services has improved in China, and some skin diseases, such as bacterial and parasitic skin diseases, are rarely seen. Thus, it is difficult to collect data on these diseases. Moreover, the clinical images in our database were collected from the Chinese population, making the database ineffective for training an intelligent global skin disease diagnosis system.
Xiangya-Derm, the largest database for skin diseases in the Chinese population, was created, and six common skin conditions were classified based on Xiangya-Derm to lay a foundation for product research.
artificial intelligence
area under the curve
convolutional neural network
European Union
modified ResNet
positron emission tomography–computed tomography
receiver operating characteristic
region proposal network
This work is supported by the National Key R&D Program of China (2018YFC0117004); the Outstanding Youth Project of Hunan, China (2020RC3059); the Hunan Provincial Innovation Foundation for Postgraduate; and the Fundamental Research Funds for the Central University of Central South University.
None declared.