To develop and evaluate a DNN model, we used data from Side Effect Resource (SIDER) [23], a database of drugs with side effects, which contains information on medicines in the market since 2009 and their recorded ADRs [24]. We collected the ADR information from 2009 and 2012 from SIDER to represent the simulated prospective approach. In total, 746 drugs and 1325 side effect terms related to these drugs were recorded in both years. Additional 232 drugs appeared only in the 2012 dataset as new drugs. It is important to monitor the ADRs of drugs throughout their life cycle, from the preclinical research phase to postmarket surveillance. The fundamental properties of drugs rely on preclinical in vitro safety profiling that involves the testing of compounds with chemical and biological properties. Therefore, we extracted these properties as a part of the features in the model. We extracted the 17 molecular descriptors of drugs from PubChem [25] (Textbox 1). We utilized the biological features from DrugBank [26] to represent the biomolecular interactions and pathways. These features contain the targets, enzymes, transporters, and carriers of each drug and their actions.

For enriching the scientific evidence and enhancing the detection of ADRs, we collected millions of papers from the Medical Literature Analysis and Retrieval System Online (MEDLINE) [27] to be used as auxiliary data to enrich the information about each drug. We used the name of each drug as the query term and selected all the papers related to the drug, published before 2009, such as case reports, clinical trials, and observational studies. The reason for collecting the papers published before 2009 is that we wanted to simulate the progress of drug surveillance from 2009 to 2012.

We treated ADR identification as an information retrieval problem, such that our model could discover the potential relationships between each drug and the 1325 side effects recorded. We represented the prediction target of 1325 dimensions with a binary profile of elements corresponding to the presence or absence of side effects with 1 or 0, Y ∈ ℕ^{n ×1325} with n being the number of drugs. Each drug was associated with 3 types of features: the chemical properties, biological properties, and information from the literature. In addition, the known ADR records of drugs were included. After preprocessing, we filtered out 2 empty properties: the Isotope Atom Count and the Undefined Bond Stereocenter Count. Subsequently, the feature of chemical properties was represented using a 15-dimensional vector, with X_Chem ∈ ℝ^{n ×15} for each element. The biological properties, extracted from DrugBank, contained 4 phases of information including the carriers, enzymes (for drug metabolism), protein targets, and transporters (for drug transportation). After preprocessing, we utilized the biological information to represent each drug with the 1048-dimensional vector, which included 788 protein targets, 162 enzymes, 85 transporters, and 13 carriers, with X_Bio ∈ ℕ^{n ×1048} for different action types in each element. The known ADR records of a drug played an important role in identifying potential ADRs. Thus, we leveraged this information to predict potential ADRs that appeared in 2012.

The biomedical literature played an important role in this study because it contains a large amount of information related to drugs and ADRs such as clinical notes and case reports. However, one of the issues in extracting the drug information from biomedical literature is the uncertainty regarding which words or documents represent the drug. Therefore, we trained the model to understand the semantic features of drugs from 2.3 million biomedical papers on 764 drugs introduced before 2009 by utilizing one of the most popular embedding methods Word2Vec [28-30] to model it using the skip-gram model. Subsequently, we used the vector of the drug name as the drug vector (drug2vec, D2V), X_D2V ∈ ℝ^{n ×400}, with n being the number of drugs. We observed that the features represented by D2V were more comprehensive than the intrinsic features in the experiments. However, as we used drug names as the query term, we could not identify papers related to 232 new drugs. To address drugs that were not observed during the embedding training step, we expanded the D2V by introducing a drug description mapping function. Using V_D2V to denote the word-embedding space of the drug vector existing in the training and V_DDV to denote the new drug description vector the summation of each word vector related to the new drug in the papers, the mapping function f (v): V_DDV→ V_D2V was developed and parameterized by a W, such that v′=Wv for v ∈ V_DDV and v′ ∈ V_D2V [31] using the least absolute shrinkage and selection operator regression for training the W [32]. This expansion method enabled the model to process new drugs, making it more flexible.

We designed a DNN model that can identify and predict the ADRs of a drug with different requirements. This model (Figure 1) was based on one of the common DNN architectures, multilayer perceptron [33], which has been successfully applied in several prior studies. We added the dropout layer between each dense layer to avoid model overfitting in the training step [34,35]. The process of the nonlinear transformation in each layer ensured that the model could learn more information from the input data. The model aimed at identifying and predicting potential ADRs of a drug. The identification function is fulfilled by seeking the potential ADRs of drugs by using known records of ADRs and various features of drugs. The prediction function pertains to the detection of potential ADRs of new drugs. We designed 2 kinds of feature representations (Figure 2) to distinguish the tasks of identification and prediction. In prediction, we assumed that the ADR record for a new drug was empty. Therefore, the feature of known ADR records with zero indicated that the possible ADRs of new drugs relied only on the semantic feature (drug2vec), chemical properties, and biological properties.

We treated the identification task as an information retrieval problem because drugs may have more than one ADR. Therefore, we designed the last layer with 1325 hidden nodes, which was equal to the number of ADRs in the dataset. Evaluating the probability of ADR y of a given drug x, we defined the p (y ∣ x) = σ(Wh + b), σ as the sigmoid function, to transfer the hidden vector h to the value between 0 and 1. We learned the p (y ∣ x) by minimizing the cross-entropy with D. When θ denoted all parameters of the model, the objective function ℒ(D, θ) was formulated as follows:

In this study, we aimed to increase the diversity of information on drugs to improve our ability to detect ADRs. Accordingly, we extracted information from the chemical and biological properties of drugs and from the existing biomedical literature. The MEDLINE was selected as the source for biomedical literature to identify important auxiliary data because it contains several types of biomedical papers, such as clinical trials, case reports, and observational studies, related to drugs. However, it was difficult to use keywords to identify specific drugs from millions of papers and words. Therefore, we utilized 2.3 million biomedical papers to identify the semantic features of drug using the skip-gram model in Word2Vec. In particular, for a central word w_t under consideration, the probability p (w_{t ±i} ∣ w_t ) of predicting the surrounding word w_{t ±i} depended on the w_t . The subscript t indicates a target word, such as the drug name “Dantrolene,” and the i represented the windows size of the target word. This feature helped us extract the latent information of each word, including the words comprising the drug’s name. After this word-embedding training, we investigated whether the semantic feature (D2V) could represent the properties of drugs, such as biomedical and chemical properties. We visualized the relationships between the learned drug-embedding vectors based on their similarity (Figure 5). This graph included 746 drugs and presented the relationship of these drugs with those of the latent information learned from the semantic feature. Each node presents a drug and the edge represents the similarity between other drugs. The more similar the drugs in each pair were, the closer they were in the graph. The larger nodes represented the drugs that were more similar to other drugs. We found that the model seems to cluster the drugs used in specific treatments.

For instance, the drugs in the cluster with the blue circle shown in Figure 5 (image on the top) comprised antidepressants such as trazodone, citalopram, clomipramine, and paroxetine. The cluster to the right contained antibiotics such as ampicillin, ceftazidime, cefpodoxime, and cefotaxime shown is Figure 5 (image in the middle). Moreover, the cluster at the bottom included ophthalmic medications such as fluorometholone, levocabastine, brinzolamide, and dipiverfrin (Figure 5, image at the bottom). There were other small clusters with their own specific treatment. Thus, we learned the relationships among drugs and the latent information from the papers included in the embedding function at the text-level. Accordingly, the semantic feature had a great effect on the performance of our model.

Subsequently, we examined the ability of this model to perform its identification and prediction functions with reference to serious ADRs defined by the Micromedex. Using the identification function of the model, we ranked the potential ADRs in a list by the probability of their occurrence (Table 3). One of the reasons why the probability of ADR presented in Table 3 was not highly prominent was that the positive samples of these ADRs were rarely reported in this dataset. However, our model could identify that hydroxychloroquine led to muscle cramps, which is a serious ADR that occurs in severe neuromuscular disease. In addition, we extracted 5 drugs that were recorded only in 2012 to examine the prediction function of this model.

Findings revealed that our model has the capacity to predict the serious ADRs of new drugs. For instance, the model predicted that Zoladex could lead to a serious ADR, myocardial infarction, which is one of the commonest causes of death in developing countries.

This study has several limitations that need to be addressed in future studies. First, data diversity plays an important role in the model. We only used the data published in SIDER. Our model will be more persuasive and reliable if we can include more data from different datasets. Because the chemical and biological properties of drugs contribute most to their effects on human, the more the databases of drug properties included, the better the performance of our model. On the other hand, if we have access to more open-source data, including clinical trials, spontaneous reporting systems, and EMRs with support from government and pharmaceutical industry, our model will have better prediction. Furthermore, our model focused on the ADR prediction and identification. To identify the probability of occurrence of each ADR, we set 1325 hidden nodes and the total number of ADRs in the dataset in the output layer. In other words, although we had a mapping function to address new drugs, this model could only predict existing ADRs. Therefore, in the future work, we plan to utilize more detailed features such as drug-ADR interaction [13], drug-drug interaction, and ADR-ADR interaction networks for the prediction of ADRs. Furthermore, we also plan to investigate other embedding approaches to represent the ADRs to help predict the relationships between drugs and new ADRs.

We developed a novel ADR detection model based on the biological and chemical properties of drugs and the D2V (the semantic feature). After discussing the drug similarities with domain experts from the National Cheng Kung University Hospital and the Institute of Clinical Pharmacy and Pharmaceutical Sciences, we found out that the D2V can represent a characteristic of the drug. Our model could not only discover the potential ADRs of drugs but also predict the possible ADRs of new drugs. To discover potential ADRs based on the previous records, our model could identify the hidden relationship between ADR-ADR interactions. Furthermore, to predict the possible ADRs of a new drug without any previous ADR records, using the D2V feature, our mapping function exhibited good profiling for transferring the drug description into the D2V. The model exhibited good performance on both tasks and generated the most suitable results. It will help pharmacists and health care providers to understand the potential risk of side effect of drugs and address the issue of underreporting of spontaneous reports. Above all, our model will aid pharmacovigilance by identifying and predicting potential ADRs.