The user profile selection was conducted October 15-29, 2014. During that time period, we retrieved YouTube videos using 2 search terms, namely “pro-ana” and “anti pro-ana.” Based on Google Trends queries, these were the most popular words used to describe the positive and negative stance toward anorexia. Using additional synonyms would not have changed the search results because of the YouTube search engine’s use of query expansion, which involves applying synonyms of search terms to increase recall. We received similar results using “pro-ana,” “proana,” and “pro-anorexia” and their “anti” counterparts with more than 90% coverage. These search terms were able to cover part of “thinspiration,” “thinspo,” “anti-thinspiration,” and “anti-thinspo” hits as well.

We selected the most popular profiles that included uploaded video material from the past 24 months concerning either the pro-anorexia or anti-pro-anorexia stance on anorexia, limiting both profile lists to 25 on the basis of their popularity. The popularity of profiles selected was determined by both video views and channel subscriptions. Notably, the YouTube search engine is also likely to list these channels first due to these criteria. All of the selected profiles were in English as were the included video comments posted during the previous 24 months. In addition, we checked that the profiles were either pro-anorexia or anti-pro-anorexia. The most popular material was created by individuals who often included video blogs concerning their opinion on anorexia. Our data did not include profiles that might be considered official profiles of governmental or nongovernmental organizations.

The second phase of data collection involved the retrieval of information about videos and comments through an automatic crawler using the YouTube Data Application Programming Interface [24]. On November 3-14, 2014, we extracted all of the video comments provided by YouTube as well as any available information of the videos, including title, date, description, and number of views, likes, and dislikes. This approach was similar to that of previous research on YouTube analyses of political campaigns [17] and video popularity [25]. The total video comment material included 1163 videos uploaded by 50 YouTube users. Although all of the users hosted video content concerning anorexia, some included a variety of other types of content. We first excluded 671 videos that did not include anorexia content. In addition, 97 videos were omitted because they did not have any comments. Hence, 395 videos with a total of 12,161 comments were included in the final sample.

User information included the date when the user joined YouTube, their physical location (country), and the number of subscribers to their channel. The gender of profile users was checked separately by viewing profile pages and videos (1=female; 2=male; 3=other). Video information included upload time, duration, number of video views, numbers of video likes and dislikes, and the total number of video comments and commenter usernames.

Video comments were assessed with the SentiStrength automatic sentiment analysis tool, which uses an algorithm to estimate the sentiment content of texts based on lexical information consisting of a list of sentiment words (approximately 3000) and grammatical categories (eg, negation) [26-28]. SentiStrength is particularly useful when working with big data and short comments, such as those of Twitter or YouTube, because it extracts both negative and positive sentiment strength simultaneously from a given short text, enabling the analysis of texts expressing both positivity and negativity at the same time. Notably, research on linguistics and psychology both show that textual statements can have loadings in both negative and positive scales simultaneously [28,29]. Positive ratings vary from 1 (not positive) to 5 (extremely positive) and negative ratings vary from -1 (not negative) to -5 (extremely negative).

We ran interrater reliability checks throughout the data collection period. Interrater reliability of the YouTube profile users and video contents were assessed by 2 independent raters. Average interrater agreement was 90.68% (Cohen’s κ=.80).

The reliability of the SentiStrength tool was tested in the study. We ran the test with a random sample of 1000 comments with 3 blind reviewers who were given instruction to code the comments on both the positive and negative axis. The raters had, on average, 47.03% (470/1000) full agreement with SentiStrength on the positive scale and 63.70% (637/1000) on the negative scale. In all, 87.20% (872/1000) of their answers showed high similarity (ie, within 1 point of SentiStrength rating) on the positive scale and 88.30% (883/1000) on the negative scale. Mean correlation between raters and SentiStrength was ρ=.63 on the positive scale and ρ=.69 on the negative scale.

Table 1 shows means and 95% confidence intervals for SentiStrength and raters. It also shows the full agreement, close agreement, Cohen’s kappa, and Spearman rho figures for each rater. Based on a 2-sample Mann-Whitney U test, the difference between SentiStrength and raters 1 and 2 was not statistically significant on the positive scale (rater 1: Z=1.43, P=.15; rater 2: Z=1.90, P=.06). However, this difference was significant between SentiStrength and rater 3 (Z=–10.27, P<.001). On the negative scale, there was no statistically significant difference between SentiStrength and rater 3 (Z=–0.63, P=.53). However, raters 1 and 2 were statistically different on the negative scale (rater 1: Z=–2.80, P=.005; rater 2: Z=–3.63, P<.001). Overall, our results indicate that SentiStrength’s precision is within human-level accuracy in estimating sentiments from text.

Our key interest was to analyze responses to both pro-anorexia videos and anti-pro-anorexia videos. First, we provided descriptive statistics on both video types. These included a Mann-Whitney 2-sample U test for the comparison of pro-anorexia and anti-pro-anorexia communities. Our main analyses were based on ordinary least squares (OLS) regression models. We analyzed both positive and negative feedback to the videos by investigating the content of comments and video likes and dislikes. Both negative and positive SentiStrength scales as well as video likes and dislikes were used as dependent variables.

Our regression models controlled for the number of video views, number of comments, number of months the video had been available on YouTube, and video duration. In light of the skewed distribution of these variables, we used logarithmic transformation (natural logarithm) in the analysis. We also controlled for the country of the video uploader (dummy coded, 0=non-English speaking, 1=English speaking) and activity of the uploader as a commentator (0=no; 1=yes). The models predicting positive and negative sentiments were adjusted to account for the clustering of observations on the level of videos. This procedure had an impact on standard errors. Therefore, we report the adjusted predictions on the regression models (with the other variables in the model set at their means) for both positive and negative sentiments as well as for video likes and dislikes.

Of 50 uploaders, 46 were female (92%) and the rest represented the category of unknown or “other” gender (eg, transgender). The uploaders came from 13 different countries, almost half from the United States (44%, 22/50); 74% (37/50) were from English-speaking countries. Their user profiles were, on average, 4 years old (mean 52.87 months, SD 28.23, range 9-102) and they had a mean of 2208 subscribers (SD 10,298.54, range 0-70,285). Uploaders were not active in writing comments to other videos. In addition, pro-anorexia profile users and anti-pro-anorexia profile users did not interact in our dataset. There were only 2 comments written by a pro-anorexia uploader to anti-pro-anorexia videos, whereas there were only 4 comments vice versa.

Our data included 133 pro-anorexia videos and 262 anti-pro-anorexia videos. As Table 2 shows, these 395 videos had a total of 12,161 comments from 7903 commenters. Only 1.00% (79/7903) of the commenters commented on both pro-anorexia and anti-pro-anorexia videos. Uploaders wrote only 5.80% (705/12,161) of the comments. The videos had more than 6 million total views indicating that this kind of material is widely accessed by YouTube users. Videos had a mean of 15,496 views (SD 37,865.37, range 57-280,253). Table 2 also shows that anti-pro-anorexia videos were more popular based on the number of video views, comments, and commenters. Anti-pro-anorexia videos received more likes than pro-anorexia videos (Z=–3.21, P=.001) and they also received more positive comments based on SentiStrength analysis (Z=–3.78, P<.001).

Our study has its limitations. First, we examined only communities and videos within YouTube; therefore, the results cannot be generalized to other online communities that may function differently depending on varying social media user interfaces. Second, our work is limited to English-language content because it was used as a selection criterion. Thus, results might not apply to videos and comments written in other languages. Third, we limited our analysis to 50 uploaders who were selected based on their popularity. However, despite limiting ourselves to 50 users, the analysis involved a total of 1163 videos, representing a large dataset through limited sources. Fourth, our study focused on 2 contrasting search words that were most commonly used to express either a “pro” or “anti” statement toward anorexia. Despite this limited search criteria, the YouTube search engine optimization guaranteed coverage of synonyms. Thus, our search results are not compromised by using only 2 search words.

Social media has been recognized as a valuable medium for health behavior identification and communication with adolescents and young adults [34]. One specific upside of this is its allowance for engaging so-called hard-to-reach populations [35]. Health and mental professionals working with young people would benefit from understanding social media communities as a whole. Although pro-anorexia content is now available in multiple social media sites [21], some are massively popular, such as YouTube. It is important to increase awareness of social media’s relevance among clinicians and educators [21,36]. Because YouTube does not restrict the material shown unless copyrights are broken, different censorship measures cannot be taken. Official routes may not be the most efficient in social media. Our results point out that YouTube users were able to respond to communities such as pro-anorexia and, in fact, those videos opposing pro-anorexia communities were more positively commented on and rated. A recognition of the dynamics of social media is important in understanding what groups or communities at-risk users associate with. This combination of dynamics and group identification would be important knowledge for any clinician treating eating disorders. Furthermore, the existing online material can be used for educational purposes. In addition, it would be possible to develop automatic tools based on thematic search and sentiment analysis to detect the most relevant online discussions on eating disorders in real time. Future studies should continue working with sentiments expressed on online social media platforms preferred by children and adolescents toward improved assessment of the effects of opposing points of view regarding health issues.

Earlier studies have shown that pro-anorexia communities are active online and encourage unhealthy behavior [1-6]. Our study provides a more dynamic view of social media as we compared the YouTube user responses to both pro-anorexia and anti-pro-anorexia videos. The study showed that the pro-anorexia community has online opponents within the YouTube user community. Many anti-pro-anorexia videos were in fact pro-recovery videos. These videos promote help for anorexics and oppose the pro-anorexia community. These videos were also more popular, gaining more positive feedback and comments than pro-anorexia videos. Therefore, anti-pro-anorexia content appears to be a counterforce on YouTube. This study serves to benefit professionals working with young people by providing them with a deeper understanding of social media activities and the sentiments expressed therein.