Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 03.01.20 in Vol 22, No 1 (2020): January

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/15684, first published Jul 29, 2019.

This paper is in the following e-collection/theme issue:

    Original Paper

    Health Effects Associated With Electronic Cigarette Use: Automated Mining of Online Forums

    University of California, Riverside, Riverside, CA, United States

    *these authors contributed equally

    Corresponding Author:

    Prue Talbot, BSc, MSc, PhD

    University of California, Riverside

    2320 Spieth Hall

    900 University Ave

    Riverside, CA, 92521

    United States

    Phone: 1 9518273768

    Email: talbot@ucr.edu


    ABSTRACT

    Background: Our previous infodemiological study was performed by manually mining health-effect data associated with electronic cigarettes (ECs) from online forums. Manual mining is time consuming and limits the number of posts that can be retrieved.

    Objective: Our goal in this study was to automatically extract and analyze a large number (>41,000) of online forum posts related to the health effects associated with EC use between 2008 and 2015.

    Methods: Data were annotated with medical concepts from the Unified Medical Language System using a modified version of the MetaMap tool. Of over 1.4 million posts, 41,216 were used to analyze symptoms (undiagnosed conditions) and disorders (physician-diagnosed terminology) associated with EC use. For each post, sentiment (positive, negative, and neutral) was also assigned.

    Results: Symptom and disorder data were categorized into 12 organ systems or anatomical regions. Most posts on symptoms and disorders contained negative sentiment, and affected systems were similar across all years. Health effects were reported most often in the neurological, mouth and throat, and respiratory systems. The most frequently reported symptoms and disorders were headache (n=939), coughing (n=852), malaise (n=468), asthma (n=916), dehydration (n=803), and pharyngitis (n=565). In addition, users often reported linked symptoms (eg, coughing and headache).

    Conclusions: Online forums are a valuable repository of data that can be used to identify positive and negative health effects associated with EC use. By automating extraction of online information, we obtained more data than in our prior study, identified new symptoms and disorders associated with EC use, determined which systems are most frequently adversely affected, identified specific symptoms and disorders most commonly reported, and tracked health effects over 7 years.

    J Med Internet Res 2020;22(1):e15684

    doi:10.2196/15684

    KEYWORDS



    Introduction

    Background

    At the time of their introduction 10 years ago, there was little information on the health effects associated with electronic cigarettes (ECs); nevertheless, they were often considered safer than conventional cigarettes because they do not burn tobacco and therefore produce aerosols with fewer chemicals. Since their introduction, a wide range of studies concerning the health effects associated with ECs have been conducted using various approaches that include online informatics and survey studies [1-6], short-term physiological assessments of EC use on human health [7,8], and in vitro and in vivo cytotoxicity studies [9-15]. Although these studies are limited mainly to acute exposures, they often suggest that EC use is not harm free [16]. Summaries of the health effect data and case report information on ECs can be found in 2 recent reviews [17,18].

    Infodemiological approaches, which mine data from the internet and social media, have yielded new information such as EC topography and the effects of EC use on human health [1,19-22]. For example, in a previous study, we mined internet data on EC puffing topography and showed that puff duration is about twice as long for EC users than conventional smokers [22]. In addition, topography is highly variable among EC users, who generally intake much larger volumes of aerosol than cigarette smokers [23]. In our prior infodemiological study, we mined information manually from major EC online health forums and identified numerous negative and some positive health effects that users attributed to ECs [1]. This was a useful approach; however, manual mining methods are labor intensive, limit the number of posts that can be reasonably extracted and analyzed, and are not practical for examining large amounts of data over time.

    Objectives

    The objective of this study was to use automated computer methods to mine an online forum and extract a large set of posts dealing with the effects of EC use on human health. These data were analyzed to identify the symptoms (undiagnosed conditions) and disorders (physician-diagnosed terminology) associated with EC use. Data were analyzed over a 7-year period, and the sentiment in each post (positive, negative, and neutral) was determined.


    Methods

    Datasets

    We collected data posted between January 2008 and July 2015 on a large EC discussion forum. Data from 2008 and 2015 were each collected for approximately 6 months. We analyzed the layout of the website and built a crawler in Java using the Java library jsoup [24], which is designed to extract and parse information from HTML pages. The posts were collected from 7 subforums. The total number of discussion threads was 2330 and the total number of posts was 1,450,896. As the primary goal of this study was to evaluate the health effects produced by ECs, we focused on those posts that belonged to the 7 health subforums, which contained 41,216 posts. We emphasize that all collected data are publicly available, including discussion threads and users’ information. Figure 1 shows the overall pipeline used for our analysis.

    Figure 1. Online forum data pipeline showing processing, post sorting, and classification workflow.
    View this figure

    Medical Concepts

    We used a modified version of the MetaMap tool [25] to annotate each post with a set of medical concepts from the Unified Medical Language System (UMLS). UMLS is a repository of a large number of biomedical controlled vocabularies [26]. In UMLS, there are 15 high-level semantic groups, which were created to help reduce complexity by grouping the semantic types [27]. In this work, we analyzed 2 semantic types, sign or symptoms and disorder or syndrome, which belong to the Symptoms and Disorders semantic groups. Each concept in UMLS can be assigned to multiple semantic types, but only to 1 semantic group [27]. As MetaMap was built to annotate the natural language text in biomedical academic publications, it is not very effective out-of-the-box on social media posts, as it successfully maps the medical terms most of the time, and not the descriptive or nonmedical terms [28]. To improve the tool’s mapping efficiency, we manually examined and removed misclassified UMLS concepts generated by MetaMap by performing the following steps:

    1. For the 2 semantic types (symptoms and disorders), we ordered the concepts by their frequencies.
    2. We analyzed the different terms mapped to each concept.
    3. We removed the misclassified concepts from our results. Examples of misclassified concepts include:
      1. mod, which refers to vape mods, was mapped to Type 2 diabetes mellitus (C0011860)
      2. ect, which is a type of vape mod, was mapped to Benign Rolandic epilepsy (C2363129)
      3. pic was mapped to Punctate inner choroidopathy (C0730321)

    For each semantic type, we reported the most frequent disorders and symptoms overall and by year.

    Sentiment

    To measure the positive and negative health effects produced by EC use, we used a supervised learning classifier (Random Forest) on a set of manually labeled posts to predict the sentiment for unseen posts. We randomly selected 1080 posts, which were labeled independently by 3 of the authors as the following:

    • Negative: if a post clearly contained a health effect or unpleasant experience or complaint that co-occurred with the use of EC.
    • Positive: if a post clearly mentioned a health improvement or a recovery from previous health effects when switching from smoking analogs to EC.
    • Neutral: if a post did not express any sentiment.

    Our interpretation of positive and negative is different from typical sentiment classifications, and mainly focuses on health-related effects. We first asked the labelers to categorize 400 posts, and then we measured the intercoder reliability between the labelers. Using ReCal [29], an online tool to calculate the reliability for the masses, the agreement was 80.53% using the Average Pairwise Percent Agreement measure. Owing to the high agreement, the rest of the posts were split evenly among the labelers to categorize. Table 1 shows the class distribution of our sample data with examples for each class; 44.7% (179/400) of posts were labeled as negative, 38.5% (154/400) as neutral, and 16.7% (67/400) as positive.

    Table 1. Sample data summary.
    View this table

    Using Weka machine-learning toolkit v. 3.8.1 [30], we first filtered our sample data after many experiments using StringToWordVector class filter, which filters strings into N-grams using WordTokenizer class, with the following settings: (1) convert all words to lower case, (2) remove stop words, (3) stem words using Weka built-in stemmer, (4) keep only terms that appear at least twice, and (5) retain unigram, bigram, and trigram. We then split the sample data as follows: (1) 962 posts for the training test and (2) 118 posts for the test set. We then trained our data using the Random Forest classifier; however, the classifier’s initial accuracy was not satisfactory.

    To improve the classifier’s accuracy, we needed to address a well-known issue in our sample data, which is the imbalanced class distribution [31]. The Positive class, as seen in Table 1, only covers 16.7% (67/400) of the data, whereas the Neutral class covers 38.50% (154/400) and the Negative class covers 44.7% (179/400). Thus, we oversampled the Positive class by duplicating the posts which were labeled Positive in the training set only. Table 2 shows the new class distribution for the training set, namely Training (extended). Another approach we used to improve the accuracy is annotating all the posts in the sample data with the ancestors of the medical concepts mentioned in the posts. For example, if pneumonia is mentioned in a post, then we append with Disorder of lung.

    After using the new training set, the classifier’s accuracy increased from 66.95% to 75.42%. Table 3 reports for each class 3 different measures, including precision, recall, and F-measure. As seen in the table, the classifier is most accurate on the Negative class (F-measure=0.79), followed by Positive and Neutral classes.

    Table 2. Training data summary (N=400).
    View this table
    Table 3. Test data classification accuracy (N=118).
    View this table

    Data Categorization and Analysis

    All health-related effects (symptoms and disorders) data reported by EC users in posts were collected iteratively and sorted into Microsoft Excel spreadsheets. The symptoms and disorders were further grouped according to the organ system and anatomical region, which we defined as systems previously [1]. When a symptom could have been associated with more than 1 system, the health effect was assigned to the system for which it had the strongest fit (eg, improved sense of taste was assigned to sensory but could have been mouth/throat). Frequency distributions for the overall grouped data in each system for symptoms and disorders were plotted using GraphPad Prism (GraphPad, San Diego). In addition, the sentiment for each post was grouped according to their positive, neutral, and negative sentiment as described in the Methods section.


    Results

    Overall Frequency of Reported Symptoms and Disorders Classified by System or Anatomical Region

    The 41,216 posts we collected spanned the years from 2008 to 2015 (2008 and 2015 were half years). We analyzed the frequency of reports for various symptoms and disorders by consolidating the reported health effects into structural or physiological systems (eg, sore throat was classified into mouth and throat; Figure 2). The 5 systems that had the most reports for symptoms were neurological (n=3623), respiratory (n=1995), digestive (n=1637), mouth and throat (n=1390), and integumentary (n=853; Figure 2). The top 5 systems for disorders were respiratory (n=2972), mouth and throat (n=1986), neurological (n=1143), integumentary (n=1123), and immune (n=739; Figure 2). For both symptoms and disorders, a majority of the posts were associated with negative sentiment across all systems (Figure 2).

    Figure 2. Frequency distribution of reported symptom (A) and disorder (B) posts grouped into their systems or anatomical regions. The frequency of positive, neutral, and negative posts is shown for symptoms (A) and for disorders (B).
    View this figure

    Symptom and Disorder Frequency and Sentiment Distribution Over Time

    After examining overall frequency distribution for all posts, we grouped the posts according to their years for analysis in their symptom or disorder categories. Across all years for both symptoms and disorders, we found the frequency distribution of reports per year. In addition, the posts for symptoms and disorders were categorized according to sentiment (positive, negative, and neutral), and their frequency per year was summarized in stacked bar graphs for each year (Figure 3).

    Figure 3. The frequency distribution of positive, neutral, and negative sentiment was assigned for reported symptoms (A-H) and disorders (I-P).
    View this figure

    For the symptoms, the posts with the most reports were consistently found in the neurological, respiratory, digestive, integumentary, and mouth and throat systems. For all years except 2008, the neurological and respiratory systems were the top 2 systems. The digestive, integumentary, and mouth and throat alternated in some years, but were generally in the top 5 systems with the most posts in each of the years.

    Similarly, the posts containing disorders associated with EC use had similar results for their top 5 system categories across the 7 years of reporting. The 2 top systems reported between 2008 and 2012 were the respiratory and mouth/throat. Alternating in the top 5 disorders were the integumentary, neurological, and immune systems.

    Negative sentiment was associated with most symptoms and disorders in each system or anatomical region (Figure 3), with some increases reported in positive health effects in 2015 for the disorders (Figure 3). It should also be noted that we only have partial reporting for 2015 because data collection was terminated by the EC forum.

    Specific Symptoms and Disorders in Systems With the Most Reports

    Heat maps were made by plotting the frequency with which individual symptoms/disorders occurred for all 41,216 posts (Figures 4 and 5; Multimedia Appendices 1 and 2). Symptoms with fewer than 2 posts are listed in Multimedia Appendix 3. The total number of negative and positive posts for each symptom or disorder is shown on a log scale ranging from high (red) to low (blue). White represents a zero-post frequency. Numerous negative symptoms were reported for each system. Typically, about 16.52% (6807/41,216) of the symptoms were reported frequently (red), while the majority often occurred in fewer than 100 posts (blue to purple). In the neurological system, the most common symptoms included: headaches (n=939), fatigue/tired/malaise (n=468), nausea (n=290), dizziness (n=183), and lightheadedness (n=113; Figure 4). In the respiratory system, the negative effects included: coughing (n=852), wheezing (n=298), dyspnea (n=235), and excessively deep breathing (n=112). The most reported symptoms in the digestive system were: heartburn (n=327), cramping (n=303), flatus (n=176), and constipation (n=113). In the mouth/throat and integumentary systems, common symptoms were: pain in throat (n=643), harsh voice quality (n=175), pharyngeal dryness (n=147), itching skin (n=565), and dry skin (n=121; Figure 4). Other commonly reported symptoms involved aching and chest pains as well as immune symptoms related to the cold and flu.

    Although positive symptoms were not frequently reported in this online forum, those reports that were posted most often dealt with improvements in the neurological (n=77), respiratory (n=60), digestive (n=19), and mouth and throat (n=18) systems (Figure 4). In the neurological system, these include improvement in tiredness (n=12) and insomnia (n=8). For respiratory system, these symptoms included improvements in wheezing (n=17), dyspnea (n=14), and coughing (n=8). In the digestive and mouth and throat systems, improvements were found in cramp (n=5) and halitosis (n=5). Other systems and anatomical regions had fewer than 10 total positive reports.

    For each system/anatomical region, there were 1 to 3 top disorders. In the respiratory system, the most common disorders were asthma (n=916), chronic obstructive pulmonary disorder (COPD; n=471), pneumonia (n=367), and bronchitis (n=232; Figure 5). In mouth and throat, the most common disorders were pharyngitis (n=565), aptyalism (n=377), and ulcer of mouth (n=207). The most reported disorders in the neurological system were dehydration (n=403) and migraine (n=103). Most disorders were reported in the respiratory, mouth and throat, neurological, integumentary, and immune systems (Figure 5), whereas the remaining systems had fewer reported disorders (Multimedia Appendix 2).

    Figure 4. Heat map of specific symptoms reported in the neurological, respiratory, digestive, mouth and throat, and integumentary systems. The total number of posts for each symptom is shown on a log scale ranging from high (red) to low (blue).
    View this figure
    Figure 5. Heat map of specific disorders reported in the respiratory, mouth and throat, neurological, integumentary, and immune systems. The total number of posts for each disorder is shown on a log scale ranging from high (red) to low (blue).
    View this figure

    To compare the frequency with which different symptoms/disorders appeared across different systems, frequency distribution graphs were created (Figures 6 and 7). Graphs show only those symptoms/disorders with over 100 posts (Figures 6 and 7). These data were sorted by negative sentiment as negative effects were most commonly reported and were of most interest. In total, 25 symptoms and 22 disorders had over 100 posts. The 5 top symptoms in the 41,216 posts were: headache, coughing, pain in throat, itching, and malaise (Figure 6). The top 5 disorders in the dataset were dehydration, asthma, pharyngitis, common cold, and aptyalism (Figure 7). These symptoms and disorders are the most commonly reported conditions in our dataset.

    Figure 6. Frequency distribution of specific symptoms with over 100 posts and frequency distribution of their systems or anatomical regions (inset). Digest.: digestive; Integ.: integumentary; Mo./Th.: mouth and throat; Musc./Skel.: muscular/skeletal; Neuro.: neurological; Resp.: respiratory.
    View this figure
    Figure 7. Frequency distribution of specific disorders with over 100 posts and frequency distribution of their systems or anatomical regions (inset). Neuro.: neurological; Resp.: respiratory; Mo./Th.: mouth and throat; Integ.: integumentary; Digest.: digestive; Circ.: circulatory; Endoc.: endocrine.
    View this figure

    Identification of Frequently Reported Paired Symptoms

    A total of 46 paired symptoms were frequently reported (Figure 8; Multimedia Appendix 4). Those with over 30 reports included a combination of neurological-neurological symptoms (eg, nausea and headache), respiratory-respiratory symptoms (eg, wheezing and coughing), and/or neurological-respiratory-mouth and throat symptoms (eg, pain in throat and headache; coughing and headache). The results in the top symptom pairings reflect the abundance of symptoms reported in their respective categories. As for those pairings occurring in less than 30 posts, various combinations of neurological, respiratory, integumentary, and digestive related symptoms.

    Figure 8. Graph showing frequency with which symptoms in various systems were linked. Digest.: digestive; Integ.: integumentary; Mo./Th.: mouth and throat; Musc./Skel.: muscular/skeletal; Neuro.: neurological; Resp.: respiratory.
    View this figure

    Discussion

    Principal Findings

    The internet is a dynamic resource containing information that can be mined to learn about the health effects associated with EC use. In our previous study, we manually mined 632 posts from 3 online EC forums to identify both positive and negative health effects reported by EC users [1]. Manual mining of such information is time consuming, labor intense, and limited by the number of posts that can be realistically evaluated. To take additional advantage of the internet as a repository of EC-related health information, we developed an automated method that was used to extract over 1 million posts from an EC forum. These posts were then filtered to yield over 41,000 health-related posts for detailed analysis. By automating the extraction process, we collected 100 times more posts for analysis and tracked responses over 7 years, an option that would be too time consuming to perform manually. The data showed a variety of positive and negative symptoms/disorders, demonstrating that the internet is a valuable resource for acquiring new data related to EC usage and their associated health effects.

    The results from this study are in overall good agreement with our prior publication [1]. In both studies, the neurological and respiratory systems were most often reported to have adverse effects associated with EC use. Both studies reported similar positive and negative health effects in various systems, and within all systems, a small number of self-reported symptoms and disorders occurred at high frequency (Figures 6 and 7). However, with the power of automated mining, we were also able to (1) identify numerous symptoms and disorders, many of which were not previously reported, along with their frequencies; (2) report new data in the disorders category for each system; (3) identify those symptoms/disorders that users have reported most frequently which will be of interest to health care providers treating patients using EC products; (4) show that health effects were similar over a 7-year period; (5) evaluate positive and negative sentiments for symptoms and disorders; (6) evaluate symptoms that are linked to each other; and (7) identify top symptoms (eg, wheezing) and disorders (eg, asthma, COPD, and pharyngitis) that are associated with inflammation. In addition, some symptoms (eg, headache and nausea) and disorders (eg, pneumonia) that occurred with the highest frequency in the neurological and respiratory system have also appeared in a number of case reports in the EC literature [17,32-35]. Also, in agreement with our prior study, some reported health outcomes attributed to EC use were positive [1]. These included reduction in symptoms such as coughing and wheezing and disorders such as asthma, COPD, and common cold. It is very likely that there were real health benefits for some individuals, especially for those switching from conventional to EC use, and this is supported by other publications [36-38].

    There are numerous reports on the health effects of EC, many of which are in agreement with our data. In the neurological system, the most commonly reported adverse symptoms we observed included headache, fatigue, nausea, dizziness, and seizures, which have also been reported in human studies [39-42]. Headaches have been reported to the Food and Drug Administration (FDA) by EC users [43], they were a common side effect in various surveys and online studies [4,44-46] and were reported in human studies in which participants used different EC devices and refill fluids with varying nicotine concentrations [47-49]. In 1 case report, an adult male experienced severe headaches/migraines and seizures for 1 week before being diagnosed with reversible cerebral vasoconstriction syndrome related to EC use [32], and in a second case, an adolescent female developed persistent daily headaches after a single EC use [50]. Nausea and dehydration were commonly reported symptoms and disorders of the neurological system. These symptoms have been associated with headache and fatigue/tiredness [51-54]. In our study, headache and nausea were frequently reported together, demonstrating that symptoms associated with EC use may be linked. The frequency distribution showing symptoms and disorders with over 100 posts also revealed that digestive symptoms such as heartburn and circulatory disorders such as myocardial infarction were highly reported. Digestive symptoms related to EC use have not been previously focused on and may be important to pay attention to, and disorders such as myocardial infarction associated with EC have recently received more attention from epidemiology studies [55].

    For the respiratory system, the most frequently reported symptoms included coughing, wheezing, and dyspnea, and the top paired respiratory symptoms were coughing-wheezing. In the national Population Assessment of Tobacco Health (PATH) and in some human surveys, EC use was associated with increased wheezing (an important potential risk factor for respiratory disease) [56-59]. In our study, the top disorders were asthma, COPD, pneumonia, bronchitis, and sinusitis, which have a common theme of inflammation. Human studies and surveys have shown that adolescents and adults associated chronic bronchitis symptoms (eg, cough, phlegm, or dyspnea) with EC use [57,58]. Epidemiological studies have linked EC use to both COPD and asthma [56], and the PATH study showed that dual use of EC and conventional cigarettes aggravated this risk [56]. Frequently reported respiratory disorders in our study such as pneumonia and bronchitis have appeared in several EC case reports, most of which deal with lung inflammation and pneumonia-linked incidents [34,35,60,61]. Some of the patients in these case reports had no preexisting health conditions but presented with coughing, wheezing, and dyspnea. They typically recovered from their respiratory disorders after discontinuing EC use.

    The circulatory, mouth/throat, chest, integumentary, and immunological systems were also affected by EC use in our study. Symptoms such as pain in throat, dry skin, pounding heart, and chest pain have been reported in survey/human EC studies [62]. Myocardial infarction, which was a top disorder for the circulatory system in our data, has been described in a case report after the patient used an EC with high nicotine [63]. A link between EC use and myocardial infarction was also found in a recent national survey adjusted for conventional smoking and other risk factors [55]. EC users are potentially susceptible to periodontal disease and increased plaque formation, which could lead to dental caries (also reported in our paper) [64]. Other reported disorders in our study, such as common cold and diabetes mellitus, are immunologically based, and multiple studies have shown EC aerosol exposure can induce inflammatory response [65]. Mice exposed to EC aerosol have impaired pulmonary viral and bacterial clearance ability that can lead to increased bacterial resistance, implying that EC users are more susceptible to cold and flu, a common complaint in our online studies. Experimental studies have further demonstrated that EC aerosol exposure can result in oxidative stress [13,66,67], suggesting chronic use could trigger inflammation, leading to progressive inflammatory disorders in the respiratory system.

    Electronic Cigarette Aerosol Chemicals That May Produce Adverse Health Effects

    EC refill fluids and aerosols are complex mixtures that contain flavor chemicals, solvents, nicotine, and metals that could contribute to adverse health effects (Table 4). Although most flavor chemicals in EC are generally regarded as safe (GRAS) for ingestion, their inhalation safety has usually not been established [68]. Some EC products contain high concentrations of flavor chemicals that exceed the National Institute of Occupational Safety and Health limits [69-75] and the concentrations normally used in consumer products [71,72]. Many EC flavor chemicals are classified as irritants [72] and are cytotoxic when tested in vitro at concentrations below those in EC products [71,72]. Cinnamaldehyde, which is used in refill fluids including those that do not have cinnamon in their name [10], is highly cytotoxic in the 3-(4,5-dimethylthiazol-2-yl) -2,5-diphenyltetrazolium bromide [10,69] and impaired respiratory response in immunological assays [76-78] and in vivo assays. In cell, animal, and human studies, EC flavor chemicals (eg, citrus/fruit and chocolate) caused an increase in reactive oxygen species leading to tissue and DNA damage and inflammation [67], which could in turn lead to mutations and disease progression. Some flavor chemicals, such as alcohols and phenols, can dilate blood vessels and cause headache, nausea, and fatigue. Prolonged inhalation of flavor chemicals, such as benzaldehyde, ethyl butanoate, diacetyl and its derivatives (2,3-pentadione, acetoin), triacetin, and limonene can elicit headaches, dizziness, and/or respiratory symptoms. Diacetyl (2,3-butanedione), a diketone associated with respiratory symptoms (wheezing and shortness of breath) and bronchiolitis obliterans, is in some EC refill fluids and can form as a reaction product during aerosolization [70,74]. Additional reaction products that form in EC aerosols (eg, aldehydes, acetals, and oxides) can be harmful to humans and elicit various symptoms, including pain.

    Table 4. Examples of chemical components in electronic cigarettes that may cause major symptoms/disorders with reference citations of studies.
    View this table

    Elements/metals (eg, aluminum, copper, cadmium, chromium, iron, nickel, silicon, lead, cobalt, and zinc) have been identified in EC aerosols [130,131]. In studies not involving ECs, these elements/metals have been linked to neurological (headache, nausea, and dizziness) and respiratory (eg, coughing, wheezing, shortness of breath, and bronchial/pulmonary irritations impairment) symptoms [132]. A positive correlation has been reported between human EC use and internal concentrations of nickel and chromium [133]. We found pneumoconiosis due to silica frequently reported by users, which could be caused by silica particles in EC aerosols [130]. Inhalation of silicon particles can elicit cough, inflammation, and lung fibrosis [134]. Although there is not a consensus on whether element/metal concentrations in EC aerosols are high enough to produce these effects [135], some evidence suggests that they could be a factor [130,136]. A female patient with no history of allergic disease tested positive for nickel allergy after being diagnosed with dermatitis caused by corrosion of the EC device [119]. Some of the top symptoms and disorders in our study also relate to inflammation of the skin (eg, itching and eczema), and this may be attributed to direct exposure and allergic reactions to EC products [119,120].

    Propylene glycol and glycerin, 2 solvents in EC aerosols, are generally considered safe for ingestion; however, they are known respiratory tract and integumentary irritants [137]. When heated, propylene glycol can produce toxic aldehyde reaction products, such as acetaldehyde and formaldehyde [138], which can cause cellular and tissue damage in the body [67]. Inhalation of aldehyde fumes can cause dizziness, nausea, and headaches in humans [113], and formaldehyde can cause coughing, wheezing, pneumonia, bronchitis, and neurological and cardiovascular symptoms (eg, headaches, nausea, heart palpitations) [139]. Inhalation of propylene glycol mists can elicit both neurological and respiratory symptoms, such as nausea, wheezing, shortness of breath, and cough [101,140] and can exacerbate asthma [101,140]. Propylene glycol and glycerin produce 15 different aerobic thermal degradation products through hydrogen abstraction, oxidation, and cleavage reactions [138]. Several of these (eg, formaldehyde, formic acid, acetaldehyde, and acrolein) are carcinogens or have genotoxic potential [141,142]. Some of these byproducts are hemiacetals (such as formic acid and formaldehyde), which equilibrate and persist in the aerosols inhaled by the users.

    Nicotine, a major component in most EC fluids, has various neurological, respiratory, digestive, mouth/throat, and circulatory system effects that overlap the symptoms/disorders observed in our study. Most cases of EC nicotine poisonings result from oral ingestion or intravenous injection [17] and are characterized by symptoms such as vomiting, nausea, dizziness, headaches, and more severe effects that can lead to death. The side effects of nicotine inhalation include headache, nausea, mouth/throat pain, cough, and heartburn [143]. Some users in our study may have been weaning themselves off nicotine or using devices with poor nicotine delivery leading to nicotine withdrawal, which could produce symptoms such as dizziness and anxiety [144]. Nicotine can trigger a dose-dependent loss of the endothelial barrier which has been shown to rapidly increase lung inflammation and oxidative stress in mice [15]. Nicotine in EC aerosols can induce glucose deprivation in the brain, which could lead to enhanced ischemic brain injury and or stroke risk. In addition, EC refill fluids may contain free-base nicotine (a form more addictive) [145], which can lead to greater deposition in the mouth and throat and upper respiratory tract [146].

    Recently, an e-cigarette, or vaping, product use associated lung injury (EVALI) epidemic has been identified by the Centers for Disease Control and Prevention (CDC) [147]. As of December 2019, at least 2409 cases of lung injury have been reported to the CDC from 50 states, the District of Columbia, and 2 US territories [147]. In addition, 52 deaths associated with vaping were confirmed by 26 states and the District of Columbia [147]. Some of the commonly reported symptoms in presenting patients included chest pain, shortness of breath, cough, nausea, vomiting, diarrhea, fever, chills, fatigue/malaise, and headache [148-150], all of which are reported in this study. In addition to lung-related disease, some case reports included neurological and gastrointestinal symptoms [148,150], which overlap those found in our study.

    The sudden uptick in health-related symptoms and conditions related to vaping comes at least 10 years after the products have gained widespread popularity in the United States, including the rise in popularity of JUUL and marijuana (THC) vape products. Our data show that many of the symptoms characterizing the current patients have been reported online for at least 7 years, suggesting that cases similar to those in the current epidemic have existed previously and been unreported or not linked to vaping. Our data further suggest that this epidemic will continue to grow given the many reports of symptoms characteristic of EVALI on the internet. The specific causes of the reported health effects are not yet known, but it is important to continue vigilant reporting of cases, tracking symptoms, and ongoing research on the health effects related to EC use to understand and contain the vaping epidemic.

    Limitations

    Our data may underestimate positive health effects, which EC users are less likely to post on online forums. The factors causing the symptoms and disorders reported by EC users could be complex and will require further investigations. Demographic data on the study population were not extractable. It is not known if any individuals were dual users or if they had preexisting health conditions that may have affected their response to EC.

    Conclusions

    This study is the first to use automated methods to analyze posts on an EC website over a span of 7 years and to identify the symptoms and disorders most frequently reported online by EC users. We demonstrate the value of using automated methods to acquire and analyze large datasets thereby increasing the power of infodemiological analyses. In addition, from our dataset, we identified a condensed list of symptoms and disorders and ranked them according to post frequency. These symptoms and disorders reported in our study may be of interest to physicians and health care providers who are treating patients using EC and could potentially be reported more frequently by EC users. Moreover, informative data were collected from a large population of EC vapers irrespective of their EC products and individual topographies and was not limited to a small selection of EC products or human subjects, as is often the case with experimental studies and case reports. Data collected using our automated method contribute to the growing body of knowledge linking EC use to adverse health effects, mainly in the mouth and throat and the neurological, respiratory, digestive, and integumentary systems. Our study identified hundreds of negative effects that were not previously described in case reports and peer-reviewed literature. The results from our study are in good agreement with previous surveys, human studies, and case reports. Although many of the symptoms that were reported with high frequency are not life-threatening (eg, headache, coughing, heartburn, sore throat), they can be disabling and reduce the quality of life. Of particular concern are the respiratory disorders that appeared with high frequency, such as asthma, COPD, pneumonia, and bronchitis, which not only severely impact the quality of life but may also be life threatening. Our data support the idea that EC use is not free of adverse health effects and that it is important to continue tracking the health of EC users. Advances in internet data mining provide a novel method for monitoring the health of EC users over time. Infodemiological data gathered on EC users will be valuable to physicians, regulatory agencies, and the users themselves.

    Acknowledgments

    The research reported in this publication was partially supported by the National Institute of Drug Addiction, National Institute of Environmental Health Sciences, and the FDA Center for Tobacco Products (CTP) Grant #s R01DA036493 and R01ES029741 (CTP) to PT. MH was supported in part by a Cornelius Hopper Fellowship and Predoctoral Dissertation Fellowship from the Tobacco-Related Research Program of California (Grant # 28DT-0009). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The authors are grateful to Malcolm Tran, Jacqueline Wong, and Avni Parekh for their help in organizing and sorting the data. They would like to thank Kevin Huang for his help with the manuscript and references and Dr Duc Nguyen for reading the manuscript.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Supplementary Figure 1: Heat map of symptoms less frequently reported in forums along with their associated systems. The total number of posts for each symptom is shown on a log scale.

    PNG File , 109 KB

    Multimedia Appendix 2

    Supplementary Figure 2: Heat map of disorders less frequently reported in forums along with their associated systems. The total number of posts for each disorder is shown on a log scale.

    PNG File , 86 KB

    Multimedia Appendix 3

    Supplementary Table 1: Listing of all symptoms not included in heat map.

    DOCX File , 15 KB

    Multimedia Appendix 4

    Supplementary Table 2: Listing of all linked symptoms by frequency.

    DOCX File , 14 KB

    References

    1. Hua M, Alfi M, Talbot P. Health-related effects reported by electronic cigarette users in online forums. J Med Internet Res 2013 Apr 8;15(4):e59 [FREE Full text] [CrossRef] [Medline]
    2. Polosa R, Morjaria JB, Caponnetto P, Campagna D, Russo C, Alamo A, et al. Effectiveness and tolerability of electronic cigarette in real-life: a 24-month prospective observational study. Intern Emerg Med 2014 Aug;9(5):537-546. [CrossRef] [Medline]
    3. Ayers JW, Leas EC, Allem J, Benton A, Dredze M, Althouse BM, et al. Why do people use electronic nicotine delivery systems (electronic cigarettes)? A content analysis of Twitter, 2012-2015. PLoS One 2017;12(3):e0170702 [FREE Full text] [CrossRef] [Medline]
    4. Li Q, Zhan Y, Wang L, Leischow SJ, Zeng DD. Analysis of symptoms and their potential associations with e-liquids' components: a social media study. BMC Public Health 2016 Jul 30;16:674 [FREE Full text] [CrossRef] [Medline]
    5. Soule EK, Nasim A, Rosas S. Adverse effects of electronic cigarette use: a concept mapping approach. Nicotine Tob Res 2016 May;18(5):678-685 [FREE Full text] [CrossRef] [Medline]
    6. Chen Z, Zeng DD. Mining online e-liquid reviews for opinion polarities about e-liquid features. BMC Public Health 2017 Jul 7;17(1):633 [FREE Full text] [CrossRef] [Medline]
    7. Vardavas CI, Anagnostopoulos N, Kougias M, Evangelopoulou V, Connolly GN, Behrakis PK. Short-term pulmonary effects of using an electronic cigarette: impact on respiratory flow resistance, impedance, and exhaled nitric oxide. Chest 2012 Jun;141(6):1400-1406. [CrossRef] [Medline]
    8. Moheimani RS, Bhetraratana M, Yin F, Peters KM, Gornbein J, Araujo JA, et al. Increased cardiac sympathetic activity and oxidative stress in habitual electronic cigarette users: implications for cardiovascular risk. JAMA Cardiol 2017 Mar 1;2(3):278-284 [FREE Full text] [CrossRef] [Medline]
    9. Bahl V, Lin S, Xu N, Davis B, Wang Y, Talbot P. Comparison of electronic cigarette refill fluid cytotoxicity using embryonic and adult models. Reprod Toxicol 2012 Dec;34(4):529-537. [CrossRef] [Medline]
    10. Behar RZ, Davis B, Wang Y, Bahl V, Lin S, Talbot P. Identification of toxicants in cinnamon-flavored electronic cigarette refill fluids. Toxicol In Vitro 2014 Mar;28(2):198-208. [CrossRef] [Medline]
    11. Behar RZ, Davis B, Bahl V, Lin S, Talbot P. Commentary in response to the letter from Farsalinos et al. regarding our publication entitled: 'identification of toxicants in cinnamon-flavored electronic cigarette refill fluids'. Toxicol In Vitro 2014 Dec;28(8):1521-1522. [CrossRef] [Medline]
    12. Yu V, Rahimy M, Korrapati A, Xuan Y, Zou AE, Krishnan AR, et al. Electronic cigarettes induce DNA strand breaks and cell death independently of nicotine in cell lines. Oral Oncol 2016 Jan;52:58-65 [FREE Full text] [CrossRef] [Medline]
    13. Lerner CA, Sundar IK, Yao H, Gerloff J, Ossip DJ, McIntosh S, et al. Vapors produced by electronic cigarettes and e-juices with flavorings induce toxicity, oxidative stress, and inflammatory response in lung epithelial cells and in mouse lung. PLoS One 2015;10(2):e0116732 [FREE Full text] [CrossRef] [Medline]
    14. Sussan TE, Gajghate S, Thimmulappa RK, Ma J, Kim J, Sudini K, et al. Exposure to electronic cigarettes impairs pulmonary anti-bacterial and anti-viral defenses in a mouse model. PLoS One 2015;10(2):e0116861 [FREE Full text] [CrossRef] [Medline]
    15. Schweitzer KS, Chen SX, Law S, Van Demark M, Poirier C, Justice MJ, et al. Endothelial disruptive proinflammatory effects of nicotine and e-cigarette vapor exposures. Am J Physiol Lung Cell Mol Physiol 2015 Jul 15;309(2):L175-L187 [FREE Full text] [CrossRef] [Medline]
    16. Drummond MB, Upson D. Electronic cigarettes. Potential harms and benefits. Ann Am Thorac Soc 2014 Feb;11(2):236-242 [FREE Full text] [CrossRef] [Medline]
    17. Hua M, Talbot P. Potential health effects of electronic cigarettes: a systematic review of case reports. Prev Med Rep 2016 Dec;4:169-178 [FREE Full text] [CrossRef] [Medline]
    18. Pisinger C, Døssing M. A systematic review of health effects of electronic cigarettes. Prev Med 2014 Dec;69:248-260 [FREE Full text] [CrossRef] [Medline]
    19. Eysenbach G. Infodemiology: The epidemiology of (mis)information. Am J Med 2002 Dec 15;113(9):763-765. [CrossRef] [Medline]
    20. Eysenbach G. Infodemiology and infoveillance: framework for an emerging set of public health informatics methods to analyze search, communication and publication behavior on the Internet. J Med Internet Res 2009 Mar 27;11(1):e11 [FREE Full text] [CrossRef] [Medline]
    21. Eysenbach G. Infodemiology and infoveillance tracking online health information and cyberbehavior for public health. Am J Prev Med 2011 May;40(5 Suppl 2):S154-S158. [CrossRef] [Medline]
    22. Hua M, Yip H, Talbot P. Mining data on usage of electronic nicotine delivery systems (ENDS) from YouTube videos. Tob Control 2013 Mar;22(2):103-106. [CrossRef] [Medline]
    23. Behar RZ, Hua M, Talbot P. Puffing topography and nicotine intake of electronic cigarette users. PLoS One 2015;10(2):e0117222 [FREE Full text] [CrossRef] [Medline]
    24. Hedley J. jsoup Java HTML Parser, with best of DOM, CSS, and jquery. 2009.   URL: https://jsoup.org/ [accessed 2019-02-07]
    25. Aronson AR. Effective mapping of biomedical text to the UMLS Metathesaurus: the MetaMap program. Proc AMIA Symp 2001:17-21 [FREE Full text] [Medline]
    26. Bodenreider O. The Unified Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Res 2004 Jan 1;32(Database issue):D267-D270 [FREE Full text] [CrossRef] [Medline]
    27. Bodenreider O, McCray AT. Exploring semantic groups through visual approaches. J Biomed Inform 2003 Dec;36(6):414-432 [FREE Full text] [CrossRef] [Medline]
    28. Denecke K, Soltani N. Where Humans Meet Machines: Innovative Solutions for Knotty Natural-Language Problems. New York: Springer; 2013.
    29. Freelon D. dfreelon. 2017. ReCal: reliability calculation for the masses.   URL: http://dfreelon.org/utils/recalfront/ [accessed 2017-11-02]
    30. Frank E, Hall MA, Witten IH. The WEKA Workbench in Data Mining: Practical Machine Learning Tools and Techniques. Fourth Edition. San Francisco: Morgan Kaufmann Publishers; 2016.
    31. Sun Y, Wong AK, Kamel MS. Classification of imbalanced data: a review. Int J Patt Recogn Artif Intell 2011;23(4):687-719. [CrossRef]
    32. Vannier S, Ronziere T, Ferre JC, Lassalle V, Verin M. Reversible cerebral vasoconstriction syndrome triggered by an electronic cigarette: case report. Eur J Neurol 2015 May;22(5):e64-e65. [CrossRef] [Medline]
    33. McCauley L, Markin C, Hosmer D. An unexpected consequence of electronic cigarette use. Chest 2012 Apr;141(4):1110-1113. [CrossRef] [Medline]
    34. Khan MS, Khateeb F, Akhtar J, Khan Z, Lal A, Kholodovych V, et al. Organizing pneumonia related to electronic cigarette use: a case report and review of literature. Clin Respir J 2018 Mar;12(3):1295-1299. [CrossRef] [Medline]
    35. Viswam D, Trotter S, Burge PS, Walters GI. Respiratory failure caused by lipoid pneumonia from vaping e-cigarettes. BMJ Case Rep 2018 Jul 6;2018:pii: bcr-2018-224350. [CrossRef] [Medline]
    36. Dawkins L, Turner J, Crowe E. Nicotine derived from the electronic cigarette improves time-based prospective memory in abstinent smokers. Psychopharmacology (Berl) 2013 Jun;227(3):377-384. [CrossRef] [Medline]
    37. Dawkins L, Turner J, Hasna S, Soar K. The electronic-cigarette: effects on desire to smoke, withdrawal symptoms and cognition. Addict Behav 2012 Aug;37(8):970-973. [CrossRef] [Medline]
    38. Berg CJ, Barr DB, Stratton E, Escoffery C, Kegler M. Attitudes toward e-cigarettes, reasons for initiating e-cigarette use, and changes in smoking behavior after initiation: a pilot longitudinal study of regular cigarette smokers. Open J Prev Med 2014 Oct;4(10):789-800 [FREE Full text] [CrossRef] [Medline]
    39. Motooka Y, Matsui T, Slaton RM, Umetsu R, Fukuda A, Naganuma M, et al. Adverse events of smoking cessation treatments (nicotine replacement therapy and non-nicotine prescription medication) and electronic cigarettes in the Food and Drug Administration Adverse Event Reporting System, 2004-2016. SAGE Open Med 2018;6:2050312118777953 [FREE Full text] [CrossRef] [Medline]
    40. Food and Drug Administration. 2019. Some E-cigarette Users Are Having Seizures, Most Reports Involving Youth and Young Adults.   URL: https:/​/www.​fda.gov/​tobacco-products/​ctp-newsroom/​some-e-cigarette-users-are-having-seizures-most-reports-involving-youth-and-young-adults [accessed 2019-11-20]
    41. Meo SA, Al Asiri SA. Effects of electronic cigarette smoking on human health. Eur Rev Med Pharmacol Sci 2014;18(21):3315-3319 [FREE Full text] [Medline]
    42. Zvolensky MJ, Garey L, Mayorga NA, Peraza N, Rogers AH, Ditre JW, et al. Pain severity and anxiety sensitivity interplay among exclusive and dual electronic cigarette users. J Stud Alcohol Drugs 2019 Mar;80(2):211-219. [CrossRef] [Medline]
    43. Chen IL. FDA summary of adverse events on electronic cigarettes. Nicotine Tob Res 2013 Feb;15(2):615-616. [CrossRef] [Medline]
    44. Baweja R, Curci KM, Yingst J, Veldheer S, Hrabovsky S, Wilson SJ, et al. Views of experienced electronic cigarette users. Addict Res Theory 2016;24(1):80-88 [FREE Full text] [CrossRef] [Medline]
    45. Volesky KD, Maki A, Scherf C, Watson LM, Cassol E, Villeneuve PJ. Characteristics of e-cigarette users and their perceptions of the benefits, harms and risks of e-cigarette use: survey results from a convenience sample in Ottawa, Canada. Health Promot Chronic Dis Prev Can 2016 Jul;36(7):130-138 [FREE Full text] [CrossRef] [Medline]
    46. Mohamed MH, Rahman A, Jamshed S, Mahmood S. Effectiveness and safety of electronic cigarettes among sole and dual user vapers in Kuantan and Pekan, Malaysia: a six-month observational study. BMC Public Health 2018 Aug 20;18(1):1028 [FREE Full text] [CrossRef] [Medline]
    47. Caponnetto P, Campagna D, Cibella F, Morjaria JB, Caruso M, Russo C, et al. EffiCiency and Safety of an eLectronic cigAreTte (ECLAT) as tobacco cigarettes substitute: a prospective 12-month randomized control design study. PLoS One 2013;8(6):e66317 [FREE Full text] [CrossRef] [Medline]
    48. Cravo AS, Bush J, Sharma G, Savioz R, Martin C, Craige S, et al. A randomised, parallel group study to evaluate the safety profile of an electronic vapour product over 12 weeks. Regul Toxicol Pharmacol 2016 Nov 15;81(Suppl 1):S1-14 [FREE Full text] [CrossRef] [Medline]
    49. Cooper M, Harrell MB, Perry CL. A qualitative approach to understanding real-world electronic cigarette use: implications for measurement and regulation. Prev Chronic Dis 2016 Jan 14;13:E07 [FREE Full text] [CrossRef] [Medline]
    50. Csere A, Sahai-Srivastava S. The Development of New Daily Persistent Headache after Electronic Cigarette Use in an Adolescent. In: Proceedings of the American Headache Society 57th Annual Scientific Meeting. 2015 Presented at: AHS'15; June 18-21, 2015; Washington, DC.
    51. OECD SIDS. OECD's Work on Co-operating in the Investigation. 2002. Screening information data set (SIDS): SIDS initial assessment report for SIAM 14; glycerol: CAS no: 56-81-5.   URL: https://hpvchemicals.oecd.org/ui/handler.axd?id=4b0a2d87-3183-40d4-84f5-0e118c647b19 [accessed 2018-06-02]
    52. Cosmetic Ingredient Review. 2014. Safety Assessment of Glycerin as Used in Cosmetics: Draft Report for Panel Review.   URL: https://www.cir-safety.org/sites/default/files/glycerin.pdf [accessed 2018-02-01]
    53. Venable JR, McClimans CD, Flake RE, Dimick DB. A fertility study of male employees engaged in the manufacture of glycerine. J Occup Med 1980 Feb;22(2):87-91. [CrossRef] [Medline]
    54. National Academies of Sciences, Engineering, and Medicine. Public Health Consequences of E-Cigarettes. Washington, DC: The National Academies Press; 2018.
    55. Alzahrani T, Pena I, Temesgen N, Glantz SA. Association between electronic cigarette use and myocardial infarction. Am J Prev Med 2018 Oct;55(4):455-461 [FREE Full text] [CrossRef] [Medline]
    56. Li D, Sundar IK, McIntosh S, Ossip DJ, Goniewicz ML, O'Connor RJ, et al. Association of smoking and electronic cigarette use with wheezing and related respiratory symptoms in adults: cross-sectional results from the Population Assessment of Tobacco and Health (PATH) study, wave 2. Tob Control 2019 Feb 13. [CrossRef] [Medline]
    57. McConnell R, Barrington-Trimis JL, Wang K, Urman R, Hong H, Unger J, et al. Electronic cigarette use and respiratory symptoms in adolescents. Am J Respir Crit Care Med 2017 Apr 15;195(8):1043-1049 [FREE Full text] [CrossRef] [Medline]
    58. Hedman L, Backman H, Stridsman C, Bosson JA, Lundbäck M, Lindberg A, et al. Association of electronic cigarette use with smoking habits, demographic factors, and respiratory symptoms. JAMA Netw Open 2018 Jul 6;1(3):e180789 [FREE Full text] [CrossRef] [Medline]
    59. Moazed F, Calfee CS. The canary in the coal mine is coughing: electronic cigarettes and respiratory symptoms in adolescents. Am J Respir Crit Care Med 2017 Apr 15;195(8):974-976 [FREE Full text] [CrossRef] [Medline]
    60. Agustin M, Yamamoto M, Cabrera F, Eusebio R. Diffuse alveolar hemorrhage induced by vaping. Case Rep Pulmonol 2018;2018:9724530 [FREE Full text] [CrossRef] [Medline]
    61. Shields PG, Berman M, Brasky TM, Freudenheim JL, Mathe E, McElroy JP, et al. A review of pulmonary toxicity of electronic cigarettes in the context of smoking: a focus on inflammation. Cancer Epidemiol Biomarkers Prev 2017 Aug;26(8):1175-1191 [FREE Full text] [CrossRef] [Medline]
    62. Tang L. Pain Associated with the Use of Electronic Cigarettes. IntechOpen 2019 [FREE Full text] [CrossRef]
    63. Kivrak T, Sunbul M, Durmus E, Dervisova R, Sari I, Yesildag O. Acute myocardial infarction due to liquid nicotine in a young man. Ther Adv Cardiovasc Dis 2014 Feb;8(1):32-34. [CrossRef] [Medline]
    64. Javed F, Kellesarian SV, Sundar IK, Romanos GE, Rahman I. Recent updates on electronic cigarette aerosol and inhaled nicotine effects on periodontal and pulmonary tissues. Oral Dis 2017 Nov;23(8):1052-1057 [FREE Full text] [CrossRef] [Medline]
    65. Kaur G, Pinkston R, Mclemore B, Dorsey WC, Batra S. Immunological and toxicological risk assessment of e-cigarettes. Eur Respir Rev 2018 Mar 31;27(147):pii: 170119 [FREE Full text] [CrossRef] [Medline]
    66. Anderson C, Majeste A, Hanus J, Wang S. E-cigarette aerosol exposure induces reactive oxygen species, DNA damage, and cell death in vascular endothelial cells. Toxicol Sci 2016 Dec;154(2):332-340. [CrossRef] [Medline]
    67. Muthumalage T, Prinz M, Ansah KO, Gerloff J, Sundar IK, Rahman I. Inflammatory and oxidative responses induced by exposure to commonly used e-cigarette flavoring chemicals and flavored e-liquids without nicotine. Front Physiol 2017;8:1130 [FREE Full text] [CrossRef] [Medline]
    68. Hallagan JB, Hall RL. Under the conditions of intended use - New developments in the FEMA GRAS program and the safety assessment of flavor ingredients. Food Chem Toxicol 2009 Feb;47(2):267-278. [CrossRef] [Medline]
    69. Behar RZ, Wang Y, Talbot P. Comparing the cytotoxicity of electronic cigarette fluids, aerosols and solvents. Tob Control 2018 May;27(3):325-333 [FREE Full text] [CrossRef] [Medline]
    70. Behar RZ, Luo W, McWhirter KJ, Pankow JF, Talbot P. Analytical and toxicological evaluation of flavor chemicals in electronic cigarette refill fluids. Sci Rep 2018 May 29;8(1):8288 [FREE Full text] [CrossRef] [Medline]
    71. Omaiye EE, McWhirter KJ, Luo W, Tierney PA, Pankow JF, Talbot P. High concentrations of flavor chemicals are present in electronic cigarette refill fluids. Sci Rep 2019 Feb 21;9(1):2468 [FREE Full text] [CrossRef] [Medline]
    72. Hua M, Omaiye EE, Luo W, McWhirter KJ, Pankow JF, Talbot P. Identification of cytotoxic flavor chemicals in top-selling electronic cigarette refill fluids. Sci Rep 2019 Feb 26;9(1):2782 [FREE Full text] [CrossRef] [Medline]
    73. Farsalinos KE, Kistler KA, Gillman G, Voudris V. Evaluation of electronic cigarette liquids and aerosol for the presence of selected inhalation toxins. Nicotine Tob Res 2015 Feb;17(2):168-174 [FREE Full text] [CrossRef] [Medline]
    74. Varlet V, Farsalinos K, Augsburger M, Thomas A, Etter J. Toxicity assessment of refill liquids for electronic cigarettes. Int J Environ Res Public Health 2015 Apr 30;12(5):4796-4815 [FREE Full text] [CrossRef] [Medline]
    75. Allen JG, Flanigan SS, LeBlanc M, Vallarino J, MacNaughton P, Stewart JH, et al. Flavoring chemicals in e-cigarettes: diacetyl, 2, 3-pentanedione, and acetoin in a sample of 51 products, including fruit-, candy-, and cocktail-flavored e-cigarettes. Environ Health Perspect 2016 Jun;124(6):733-739 [FREE Full text] [CrossRef] [Medline]
    76. Clapp PW, Pawlak EA, Lackey JT, Keating JE, Reeber SL, Glish GL, et al. Flavored e-cigarette liquids and cinnamaldehyde impair respiratory innate immune cell function. Am J Physiol Lung Cell Mol Physiol 2017 Aug 1;313(2):L278-L292 [FREE Full text] [CrossRef] [Medline]
    77. Clapp PW, Jaspers I. Electronic cigarettes: their constituents and potential links to asthma. Curr Allergy Asthma Rep 2017 Oct 5;17(11):79 [FREE Full text] [CrossRef] [Medline]
    78. Clapp PW, Lavrich KS, van Heusden CA, Lazarowski ER, Carson JL, Jaspers I. Cinnamaldehyde in flavored e-cigarette liquids temporarily suppresses bronchial epithelial cell ciliary motility by dysregulation of mitochondrial function. Am J Physiol Lung Cell Mol Physiol 2019 Mar 1;316(3):L470-L486. [CrossRef] [Medline]
    79. Ni Y, Zhang G, Kokot S. Simultaneous spectrophotometric determination of maltol, ethyl maltol, vanillin and ethyl vanillin in foods by multivariate calibration and artificial neural networks. Food Chem 2005;89(3):465-473. [CrossRef]
    80. Yue Y, Liu J, Yao M, Yao X, Fan J, Ji H. The investigation of the binding behavior between ethyl maltol and human serum albumin by multi-spectroscopic methods and molecular docking. Spectrochim Acta A Mol Biomol Spectrosc 2012 Oct;96:316-323. [CrossRef] [Medline]
    81. Gonullu H, Gonullu E, Karadas S, Arslan M, Kalemci O, Aycan A, et al. The levels of trace elements and heavy metals in patients with acute migraine headache. J Pak Med Assoc 2015 Jul;65(7):694-697 [FREE Full text] [Medline]
    82. Agency for Toxic Substances and Disease Registry (ATSDR). Atlanta, GA: US Department of Health and Human Services, Public Health Service; 2004 Sep. Toxicological profile for Copper.   URL: https://www.atsdr.cdc.gov/toxprofiles/tp132.pdf [accessed 2019-07-25]
    83. Brodkin E, Copes R, Mattman A, Kennedy J, Kling R, Yassi A. Lead and mercury exposures: interpretation and action. Can Med Assoc J 2007 Jan 2;176(1):59-63 [FREE Full text] [CrossRef] [Medline]
    84. OECD's Work on Co-operating in the Investigation. 2001. Screening information data set (SIDS): SIDS initial assessment profile for SIAM 11; Dipropylene glycol (mixed isomers and dominant isomer).   URL: https://hpvchemicals.oecd.org/ui/handler.axd?id=40da06b1-a855-4c0c-bc21-bbc856dca725 [accessed 2019-11-20]
    85. NJ Health. 2009. Hazardous Substances Fact Sheet: Propylene Glycol.   URL: http://nj.gov/health/eoh/rtkweb/documents/fs/3595.pdf [accessed 2019-07-25]
    86. Centers for Disease Control and Prevention. 2018. NIOSH: Nicotine.   URL: https://www.cdc.gov/niosh/npg/npgd0446.html
    87. Mishra A, Chaturvedi P, Datta S, Sinukumar S, Joshi P, Garg A. Harmful effects of nicotine. Indian J Med Paediatr Oncol 2015;36(1):24-31 [FREE Full text] [CrossRef] [Medline]
    88. Taylor FR. Tobacco, nicotine, and headache. Headache 2015;55(7):1028-1044. [CrossRef] [Medline]
    89. Ash M, Ash I. Handbook of Green Chemicals. Second Edition. New York, NY: Synapse Information Resources Inc; 2006.
    90. Assi MA, Hezmee MN, Haron AW, Sabri MY, Rajion MA. The detrimental effects of lead on human and animal health. Vet World 2016 Jun;9(6):660-671 [FREE Full text] [CrossRef] [Medline]
    91. NJ Health. 2009. Hazardous Substances Fact Sheet: Benzaldehyde.   URL: https://nj.gov/health/eoh/rtkweb/documents/fs/0196.pdf [accessed 2019-07-25]
    92. Environmental XPRT. 2010. Overview of Airborne metals regulations, exposure limits, health effects, and contemporary research case study.   URL: https:/​/www.​environmental-expert.com/​articles/​overview-of-airborne-metals-regulations-exposure-limits-health-effects-and-contemporary-research-cas-450906 [accessed 2019-07-25]
    93. Jaishankar M, Tseten T, Anbalagan N, Mathew BB, Beeregowda KN. Toxicity, mechanism and health effects of some heavy metals. Interdiscip Toxicol 2014 Jun;7(2):60-72 [FREE Full text] [CrossRef] [Medline]
    94. Centers for Disease Control and Prevention. 2018. NIOSH: Nicotine: Systemic Agent.   URL: https://www.cdc.gov/niosh/ershdb/emergencyresponsecard_29750028.html [accessed 2019-07-25]
    95. Agency for Toxic Substances and Disease Registry. 2014. Medical Management Guidelines for Phenol.   URL: https://www.atsdr.cdc.gov/mmg/mmg.asp?id=144&tid=27#bookmark05 [accessed 2019-07-25]
    96. Callahan-Lyon P. Electronic cigarettes: human health effects. Tob Control 2014 May;23(Suppl 2):ii36-ii40 [FREE Full text] [CrossRef] [Medline]
    97. Pesta DH, Angadi SS, Burtscher M, Roberts CK. The effects of caffeine, nicotine, ethanol, and tetrahydrocannabinol on exercise performance. Nutr Metab (Lond) 2013 Dec 13;10(1):71 [FREE Full text] [CrossRef] [Medline]
    98. NJ Health. 1998. Hazardous Substances Fact Sheet: ethyl butyrate.   URL: http://nj.gov/health/eoh/rtkweb/documents/fs/0862.pdf [accessed 2019-11-20]
    99. Park RM, Gilbert SJ. Pulmonary impairment and risk assessment in a diacetyl-exposed population: microwave popcorn workers. J Occup Environ Med 2018 Jun;60(6):496-506 [FREE Full text] [CrossRef] [Medline]
    100. Toxnet. 2015. Triacetin.   URL: https://toxnet.nlm.nih.gov/cgi-bin/sis/search/a?dbs+hsdb:@term+@DOCNO+585 [accessed 2019-07-25]
    101. Wieslander G, Norbäck D, Lindgren T. Experimental exposure to propylene glycol mist in aviation emergency training: acute ocular and respiratory effects. Occup Environ Med 2001 Oct;58(10):649-655 [FREE Full text] [CrossRef] [Medline]
    102. Dicpinigaitis PV. Effect of tobacco and electronic cigarette use on cough reflex sensitivity. Pulm Pharmacol Ther 2017 Dec;47:45-48. [CrossRef] [Medline]
    103. Nemery B. Metal toxicity and the respiratory tract. Eur Respir J 1990 Feb;3(2):202-219. [Medline]
    104. Boulay ME, Henry C, Bossé Y, Boulet LP, Morissette MC. Acute effects of nicotine-free and flavour-free electronic cigarette use on lung functions in healthy and asthmatic individuals. Respir Res 2017 Feb 10;18(1):33 [FREE Full text] [CrossRef] [Medline]
    105. Cooper RG. Zinc toxicology following particulate inhalation. Indian J Occup Environ Med 2008 Apr;12(1):10-13 [FREE Full text] [CrossRef] [Medline]
    106. United States Environmental Protection Agency. 2000. Selenium Compounds.   URL: https://www.epa.gov/sites/production/files/2016-09/documents/selenium-compounds.pdf [accessed 2019-07-25]
    107. Norbäck D, Wieslander G, Edling C. Occupational exposure to volatile organic compounds (VOCs), and other air pollutants from the indoor application of water-based paints. Ann Occ Hyg 1995;39(6):783-794. [CrossRef]
    108. Paik JH, Kang S, Durey A, Kim JH, Kim AJ. Symptomatic bradycardia due to nicotine intoxication. Rev Bras Ter Intensiva 2018 Mar;30(1):121-126 [FREE Full text] [CrossRef] [Medline]
    109. Quirce S, Fernández-Nieto M, del Pozo V, Sastre B, Sastre J. Occupational asthma and rhinitis caused by eugenol in a hairdresser. Allergy 2008 Jan;63(1):137-138. [CrossRef] [Medline]
    110. Bein K, Leikauf GD. Acrolein - a pulmonary hazard. Mol Nutr Food Res 2011 Sep;55(9):1342-1360. [CrossRef] [Medline]
    111. Centers for Disease Control and Prevention. 2017. Flavorings-related lung disease.   URL: https://www.cdc.gov/niosh/topics/flavorings/exposure.html [accessed 2019-07-25]
    112. Sisson JH. Alcohol and airways function in health and disease. Alcohol 2007 Aug;41(5):293-307 [FREE Full text] [CrossRef] [Medline]
    113. Gorguner M, Akgun M. Acute inhalation injury. Eurasian J Med 2010 Apr;42(1):28-35 [FREE Full text] [CrossRef] [Medline]
    114. Chang CC, Incaudo GA, Gershwin ME. Diseases of the Sinuses: A Comprehensive Textbook of Diagnosis and Treatment. New York, NY: Springer-Verlag; 2014.
    115. Sunderman FW. Nasal toxicity, carcinogenicity, and olfactory uptake of metals. Ann Clin Lab Sci 2001 Jan;31(1):3-24. [Medline]
    116. Kim SA, Smith S, Beauchamp C, Song Y, Chiang M, Giuseppetti A, et al. Cariogenic potential of sweet flavors in electronic-cigarette liquids. PLoS One 2018;13(9):e0203717 [FREE Full text] [CrossRef] [Medline]
    117. Wychowanski P, Malkiewicz K. Evaluation of metal ion concentration in hard tissues of teeth in residents of Central Poland. Biomed Res Int 2017;2017:6419709 [FREE Full text] [CrossRef] [Medline]
    118. Liu S, Wu T, Zhou X, Zhang B, Huo S, Yang Y, et al. Nicotine is a risk factor for dental caries: an study. J Dent Sci 2018 Mar;13(1):30-36 [FREE Full text] [CrossRef] [Medline]
    119. Maridet C, Atge B, Amici J, Taïeb A, Milpied B. The electronic cigarette: the new source of nickel contact allergy of the 21st century? Contact Dermatitis 2015 Jul;73(1):49-50. [CrossRef] [Medline]
    120. Shim TN, Kosztyuova T. Allergic contact dermatitis to electronic cigarette. Dermatitis 2018;29(2):94-95. [CrossRef] [Medline]
    121. Nguyen TT, Higashi T, Kambayashi Y, Anyenda EO, Michigami Y, Hara J, et al. A longitudinal study of association between heavy metals and itchy eyes, coughing in chronic cough patients: related with non-immunoglobulin E mediated mechanism. Int J Environ Res Public Health 2016 Jan 7;13(1):pii: E110 [FREE Full text] [CrossRef] [Medline]
    122. Andersen KE, Storrs FJ. [Skin irritation caused by propylene glycols]. Hautarzt 1982 Jan;33(1):12-14. [Medline]
    123. Lee IW, Ahn SK, Choi EH, Lee SH. Urticarial reaction following the inhalation of nicotine in tobacco smoke. Br J Dermatol 1998 Mar;138(3):486-488. [CrossRef] [Medline]
    124. Agency for Toxic Substances and Disease Registry. Atlanta, GA: US Department of Health and Human Services, Public Health Service; 2015. Public Health Statement for Propylene Glycol.   URL: https://www.atsdr.cdc.gov/toxprofiles/tp189-c1-b.pdf [accessed 2019-11-20]
    125. Sarici G, Cinar S, Armutcu F, Altinyazar C, Koca R, Tekin NS. Oxidative stress in acne vulgaris. J Eur Acad Dermatol Venereol 2010 Jul;24(7):763-767. [CrossRef] [Medline]
    126. Mills CM, Peters TJ, Finlay AY. Does smoking influence acne? Clin Exp Dermatol 1993 Mar;18(2):100-101. [CrossRef] [Medline]
    127. Misery L. Nicotine effects on skin: are they positive or negative? Exp Dermatol 2004 Nov;13(11):665-670. [CrossRef] [Medline]
    128. Wait M. WebMD. 2014. Why Tobacco Use May Make Heartburn Worse.   URL: https://www.webmd.com/heartburn-gerd/features/heartburn-tobacco-connection [accessed 2019-07-25]
    129. Mueller PD, Benowitz NL. Toxicologic causes of acute abdominal disorders. Emerg Med Clin North Am 1989 Aug;7(3):667-682. [Medline]
    130. Williams M, Villarreal A, Bozhilov K, Lin S, Talbot P. Metal and silicate particles including nanoparticles are present in electronic cigarette cartomizer fluid and aerosol. PLoS One 2013;8(3):e57987 [FREE Full text] [CrossRef] [Medline]
    131. Olmedo P, Goessler W, Tanda S, Grau-Perez M, Jarmul S, Aherrera A, et al. Metal concentrations in e-cigarette liquid and aerosol samples: the contribution of metallic coils. Environ Health Perspect 2018 Feb 21;126(2):027010 [FREE Full text] [CrossRef] [Medline]
    132. Geiger A, Cooper J. Overview of Airborne Metals: Regulations, Exposure Limits, Health Effects, and Contemporary Research. Portland, OR: Cooper Environmental Services; 2010.
    133. Aherrera A, Olmedo P, Grau-Perez M, Tanda S, Goessler W, Jarmul S, et al. The association of e-cigarette use with exposure to nickel and chromium: a preliminary study of non-invasive biomarkers. Environ Res 2017 Nov;159:313-320. [CrossRef] [Medline]
    134. Sato T, Shimosato T, Klinman DM. Silicosis and lung cancer: current perspectives. Lung Cancer (Auckl) 2018;9:91-101 [FREE Full text] [CrossRef] [Medline]
    135. Palazzolo DL. Electronic cigarettes and vaping: a new challenge in clinical medicine and public health. A literature review. Front Public Health 2013 Nov 18;1:56 [FREE Full text] [CrossRef] [Medline]
    136. Hess CA, Olmedo P, Navas-Acien A, Goessler W, Cohen JE, Rule AM. E-cigarettes as a source of toxic and potentially carcinogenic metals. Environ Res 2017 Jan;152:221-225 [FREE Full text] [CrossRef] [Medline]
    137. Dalton P, Soreth B, Maute C, Novaleski C, Banton M. Lack of respiratory and ocular effects following acute propylene glycol exposure in healthy humans. Inhal Toxicol 2018 Feb;30(3):124-132. [CrossRef] [Medline]
    138. Jensen RP, Strongin RM, Peyton DH. Solvent chemistry in the electronic cigarette reaction vessel. Sci Rep 2017 Feb 14;7:42549 [FREE Full text] [CrossRef] [Medline]
    139. Occupational Safety and Health Administration. 1910.1048 App C - Medical surveillance - Formaldehyde.   URL: https://www.osha.gov/laws-regs/regulations/standardnumber/1910/1910.1048AppC [accessed 2019-11-20]
    140. Cotta KI, Stephen CD, Mohammad NU. A review on the safety of inhalation of propylene glycol in e-cigarettes. Glob J Pharm Pharm Sci 2017;2(2):1-2. [CrossRef]
    141. Watson AY, Bates RR, Kennedy D. Health effects of aldehydes and alcohols in mobile source emissions. In: Air Pollution, the Automobile, and Public Health. Washington, DC: National Academies Press; 1988.
    142. Kawanishi M, Matsuda T, Yagi T. Genotoxicity of formaldehyde: molecular basis of DNA damage and mutation. Front Environ Sci 2014;2:36. [CrossRef]
    143. Drugs. 2019. Nicotine Side Effects.   URL: https://www.drugs.com/sfx/nicotine-side-effects.html#moreResources [accessed 2019-07-17]
    144. McLaughlin I, Dani JA, de Biasi M. Nicotine withdrawal. Curr Top Behav Neurosci 2015;24:99-123 [FREE Full text] [CrossRef] [Medline]
    145. Duell AK, Pankow JF, Peyton DH. Free-Base Nicotine Determination in Electronic Cigarette Liquids by 1H NMR Spectroscopy. Chem Res Toxicol 2018 Jun 18;31(6):431-434 [FREE Full text] [CrossRef] [Medline]
    146. Henningfield J, Pankow J, Garrett B. Ammonia and other chemical base tobacco additives and cigarette nicotine delivery: issues and research needs. Nicotine Tob Res 2004 Apr;6(2):199-205. [CrossRef] [Medline]
    147. Center for Disease Control and Prevention. 2019 Nov 10. Outbreak of Lung Injury Associated with the Use of E-Cigarette, or Vaping, Products.   URL: https://www.cdc.gov/tobacco/basic_information/e-cigarettes/severe-lung-disease.html [accessed 2019-11-10]
    148. Layden JE, Ghinai I, Pray I, Kimball A, Layer M, Tenforde M, et al. Pulmonary Illness Related to E-Cigarette Use in Illinois and Wisconsin - Preliminary Report. N Engl J Med 2019 Sep 6. [CrossRef] [Medline]
    149. Davidson K, Brancato A, Heetderks P, Mansour W, Matheis E, Nario M, et al. Outbreak of Electronic-Cigarette-Associated Acute Lipoid Pneumonia - North Carolina, July-August 2019. MMWR Morb Mortal Wkly Rep 2019 Sep 13;68(36):784-786 [FREE Full text] [CrossRef] [Medline]
    150. Henry TS, Kanne JP, Kligerman SJ. Imaging of vaping-associated lung disease. N Engl J Med 2019 Oct 10;381(15):1486-1487. [CrossRef] [Medline]


    Abbreviations

    CDC: Centers for Disease Control and Prevention
    COPD: chronic obstructive pulmonary disorder
    CTP: Center for Tobacco Products
    EC: electronic cigarette
    FDA: Food and Drug Administration
    PATH: Population Assessment of Tobacco Health
    UMLS: Unified Medical Language System
    VAPI: vaping-associated pulmonary illness


    Edited by G Eysenbach; submitted 29.07.19; peer-reviewed by L Wang, K McCausland; comments to author 03.10.19; revised version received 11.10.19; accepted 20.10.19; published 03.01.20

    ©My Hua, Shouq Sadah, Vagelis Hristidis, Prue Talbot. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 03.01.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.