This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.
The #MeToo movement sparked an international debate on the sexual harassment, abuse, and assault and has taken many directions since its inception in October of 2017. Much of the early conversation took place on public social media sites such as Twitter, where the hashtag movement began.
The aim of this study is to document, characterize, and quantify early public discourse and conversation of the #MeToo movement from Twitter data in the United States. We focus on posts with public first-person revelations of sexual assault/abuse and early life experiences of such events.
We purchased full tweets and associated metadata from the Twitter Premium application programming interface between October 14 and 21, 2017 (ie, the first week of the movement). We examined the content of novel English language tweets with the phrase “MeToo” from within the United States (N=11,935). We used machine learning methods, least absolute shrinkage and selection operator regression, and support vector machine models to summarize and classify the content of individual tweets with revelations of sexual assault and abuse and early life experiences of sexual assault and abuse.
We found that the most predictive words created a vivid archetype of the revelations of sexual assault and abuse. We then estimated that in the first week of the movement, 11% of novel English language tweets with the words “MeToo” revealed details about the poster’s experience of sexual assault or abuse and 5.8% revealed early life experiences of such events. We examined the demographic composition of posters of sexual assault and abuse and found that white women aged 25-50 years were overrepresented in terms of their representation on Twitter. Furthermore, we found that the mass sharing of personal experiences of sexual assault and abuse had a large reach, where 6 to 34 million Twitter users may have seen such first-person revelations from someone they followed in the first week of the movement.
These data illustrate that revelations shared went beyond acknowledgement of having experienced sexual harassment and often included vivid and traumatic descriptions of early life experiences of assault and abuse. These findings and methods underscore the value of content analysis, supported by novel machine learning methods, to improve our understanding of how widespread the revelations were, which likely amplified the spread and saliency of the #MeToo movement.
Public discourse on sensitive topics, ranging from sexual violence to health (mis)behaviors, increasingly occurs on social networking platforms such as Twitter [
The recent explosion in the public discourse on sexual violence is an interesting case to consider. Sexual violence, including sexual harassment, abuse, and assault, is highly pervasive and has long-term behavioral and mental health sequelae [
Ms Milano’s tweet immediately went viral, with 1,595,453 tweets posted in the first week, and ignited a movement where victims of sexual assault, abuse, and harassment felt empowered to divulge as much, or as little, information as they wanted about their personal experiences. The phrase “MeToo” was coined by Tarana Burke, a civil rights activist, as a way to raise awareness and provide support for survivors of sexual violence. Ms Milano’s use of this phrase in her tweet, as opposed to other more explicit hashtags such as #BeenRapedNeverReported, allowed posters to retain some privacy about the details of the event and still participate. This led to millions of users joining the conversation and subsequent normalizing of the revelations. Meanwhile, the wide use of the generic #MeToo encouraged an outpour of detailed revelations. Because of the massive size of the #MeToo movement, many who may not have experienced sexual violence were confronted with the knowledge that members of their network had.
In this study, we aim to describe the public disclosures of sexual violence within the first week of the #MeToo movement. Here, we document the content, quantify the scale, and present the demographic characteristics of Twitter users who disclose incidents of sexual assault/abuse in the early conversation on Twitter for the #MeToo movement. We use simple machine learning tools to create the archetype of tweets, which often included detailed accounts of sexual assault and abuse and early life experience of such events among women from all walks of life. Next, we categorize the content of individual-level tweets to estimate the proportion of all #MeToo tweets with such revelations in the first week of the movement. Furthermore, we use our categorization to detail the demographic characteristics of posters with revealed events and the reach of the revealed events on the Twitter platform generally.
The data for this project are tweets, short messages of 140 characters or less, sent from a Twitter user (Twitter handle) to their network of followers. Twitter data are considered existing data in the public domain and therefore exempt from human subject review.
We applied for and were granted access to the Twitter Premium application programming interface (API) platform, which allows users to purchase and query all nondeleted public tweets since the first tweet posted in 2006. We purchased both the counts and full tweets from historical Twitter data between October 14 and 21, 2017 (ie, the first week of the movement). Unlike other social media movements that took several months to take hold, the #MeToo movement had the greatest activity in the first week (
The analytic sample for our content analyses comprised the full text and associated metadata for 97% of novel English language US-based tweets in the study period (N=11,935).
Comparison of relative time patterns of novel English tweets including MeToo and BlackLivesMatter. MeToo counts are on the left axis, and BlackLivesMatter counts are on the right axis. BLM: BlackLivesMatter.
Data flow chart. LASSO: least absolute shrinkage and selection operator; SVM: support vector machine.
We first present evidence that the sample of novel English #MeToo tweets in the United States, which we selected for analysis, was comparable in terms of time trends to all novel English #MeToo tweets. We graphed the number of #MeToo tweets per hour starting on October 14, to show the time trends comparing all #MeToo tweets, novel English #MeToo tweets, and US-based geotagged novel English language #MeToo tweets. We also report quantitative measures of correlation on the number of tweets in each of these categories by hour (the Pearson correlation and coefficient of determination).
Before analyzing the data, both authors reviewed approximately 2000 tweets from the first week of the movement. Based on this initial review, we chose to focus the analysis on first-person revelations of sexual assault and abuse and childhood experience of sexual assault and abuse. The tweets reviewed revealed several categories of comments with #MeToo hashtags. They included (1) tweets with #MeToo that were support statements (ie, “How many women not in the spotlight have #MeToo #Notaceleb #StillAStar. Proud of them! As Bosch says- ‘everybody counts or nobody counts’”), (2) statements with ambiguous revelations of events (ie, “MeToo” alone and “Kept it buried down for many years and didn't even really realize how it impacted me.”), (3) statements with detailed revelations of events (ie, “I was sexually assaulted by a family member when I was 8 the trauma
Based on this initial review, we chose to focus on the second and third categories and created an annotation rubric to categorize first-person revelations. For ambiguous statements (ie, category 2) where it was clear an event occurred, such as a revelation with an alleged assailant’s name or a situation but little details on the actual actions, we decided that this recollection likely reflects enduring trauma and should be categorized as a revealed abuse or assault. If the ambiguous events were experienced in childhood, it would be classified as an early life experience of abuse or assault (“I was a child too scared to speak. Till this day, it still haunts me”). If the statement was too ambiguous, did not make references to childhood, or provided little detail, we did not classify it as a case of abuse (ie, “Sometimes the stories we don't share are the ones that affect us and continue to scare us the most” and “Tried to tweet about my #metoo moment and deleted it bc it still feels like my fault. The voices you aren’t hearing in this are deafening”). For the third category—the clearest case—based on the purported event, we categorized the revelations as abuse or assault and early life experience based on the details provided.
We randomly selected a subset of 650 tweets from the novel English US-based #MeToo tweets as the training set for the analysis. Based on our rubric, both authors classified the 650 randomly selected tweets. Each tweet was categorized along two main dimensions: if the tweet revealed details of an experience of sexual assault and abuse and if the details suggest that the event happened in early life. All other types of tweets (support, too ambiguous, or other) were categorized as not revealing an incident of abuse or assault. We used an approximate age cut off of 22 years (or references to college or earlier schooling) to distinguish and delineate early life experience. This was chosen to capture college experience under the category of early life. All other tweets were categorized as “other.” The concordance between the two authors’ categorizations was 94% for sexual abuse and assault and 98% for childhood experience. Most disagreements on the abuse/assault categorization were on cases with ambiguous revelations, which could be argued either way. Most disagreements on childhood experience were in cases where the revealed event was placed in the past. Many cases point to an event in the distant past, but it was hard to ascertain the age of the poster at the time of the revealed event, because we did not have the current age of the poster. Given these minor differences in interpretation, we decided to use one author’s (BC) categorization in the training set. To assess our model performance, we used the other author’s (SM) classification in the test set for calculating positive and negative predictive values.
Tweets were categorized along two dimensions: if they disclose an experience of sexual assault and abuse (red in
Classification flow chart.
To provide exemplars of the disclosed tweets, we used simple supervised machine learning methods— least absolute shrinkage and selection operator (LASSO) regression models—on the training sample to find most predictive words for classification along both dimensions [
LASSO regressions are a common tool for economists who are using text as data and performing simple computational linguistic analyses [
After removing stop words, we had 11,931 unique words in our training set from 650 tweets. We stemmed each word, examined the list for misspelling, and considered words (and stubs) that appear in at least five tweets to limit the sparsity of our sample, leading to 1186 unique words. From these words, the authors selected 109 words related to sexual abuse and assault. This filtering limited the sparsity of the data further. We then used LASSO regressions on a matrix of 650 tweets coded for sexual assault/abuse (or coded for early life experience) and 109 words, each of which was treated as a binary flag if it appeared in each tweet. For example, if the tweet included the word “Rape” or “Raped” in it, then the variable “Rape” was coded as 1.
The LASSO model minimizes an objective function
Equation 1:
The LASSO models were implemented in the statistical software R ([computer program] Version 3.5.0. Vienna, Austria: R Foundation for Statistical Computing; 2018) using the “glmnet” package. The results of the LASSO model are validated with ten-fold cross-validation. The cross-validation allows us to pick the model with the λ value having the lowest mean squared error or model variance. From this selected model, we obtained a list of the 35 most predictive words for tweets with revelations of sexual assault/abuse words and 34 most predictive words for tweets with revelations of early life experience of sexual assault/abuse words. These word lists were then categorized by the authors.
We then used the same training sample of tweets to train the support vector machine (SVM) models to classify the remaining sample of tweets (N=11,285) along the same two dimensions—experience of sexual assault/abuse and early life experience of sexual assault/abuse [
An SVM is a discriminative classifier that is used on training data to define a separating hyperplane in multidimensional space and then uses this hyperplane to categorize new data. In the training set, data points that are closer to a potential defining hyperplane (ie, support vectors) are given more weight. The goal of the underlying algorithm is to maximize the margin between the support vector data points and the separating hyperplane using a cost function [
In our case, we have a matrix of 11,931 unique words (dimensions) and 650 labeled outcome data. From this information, the algorithm defines a separating hyperplane. Based on this hyperplane, the remaining 11,285 tweets are categorized. To perform the SVM model operations, the package “RTextTools” was used in R software.
We assessed the quality of the SVM model’s classification with four test sets, two for each classification category, to estimate positive and negative predictive values. To calculate the positive predictive value (PPV) and negative predictive value (NPV), we sampled 50 tweets four times as test sets. The first two samples were used to assess PPV and NPV for the model predictions of sexual assault and abuse. The second two samples were used to assess PPV and NPV for the model predictions of early life experience of sexual assault and abuse. Given that BC’s classification was used in the training set, we used SM’s classification on the test set to assess the model.
For each sample of the first two samples, one author (SM) assessed the content of the tweet for a revelation of an experience of sexual assault and abuse. We treated the human-assessed content as the gold standard and calculated the proportion of the time that the SVM algorithm’s classification was the same as the human classification. PPV was calculated as the number of true positives (agreement between human and algorithm) divided by the number of positive cases found by the SVM algorithm. NPV was calculated as the number of true negatives divided by the number of negative cases found by the SVM algorithm. We repeat the process and manually categorize the content of the last two samples for revelations of early life experience of sexual assault or abuse to calculate PPV and NPV
Based on these classifications, we used a previously vetted commercial service, Demographics Pro [
Author BC’s classification of the 650 randomly selected tweets in the training set revealed that approximately 19% of these tweets included a first-person revelation of sexual assault/abuse. BC’s classification was used in supervised machine learning methods with LASSO regressions to identify the most predictive words.
Hourly counts of "MeToo" tweets. Top: Hourly counts of "MeToo" tweets by category (overall, novel English, and geotagged novel English). Bottom: Hourly counts of all novel English language tweets with the phrase "MeToo" and hourly counts of all novel English language geotagged tweets in the United States.
Age
College
First time
Fifteen
Grade
Hasnumber (Hasnumber is an overall indicator of whether there is a number in the text.)
Kindergarten
Old
Year
Years ago
Years old
Boyfriend
Coworker
Date
Father
Man
Rapist
Police
Stranger
Teacher
Uncle
Advantage
Chase
Grab
Grope
Rape
Shout
Arm
Boob
Butt
Drug
Drunk
Daylight
Door
Interview
Frat (This term can indicate a location, time/age, or an implied level of intoxication. The term frat can indicate a time (college is when most people go to frats), a place such as a frat house, and a level of intoxication because frats are often where alcohol is served.)
Of the 650 randomly selected tweets in the training set, the authors categorized 5% of the tweets as having indicated an early experience of sexual assault/abuse.
Next, we use the SVM algorithm on the same 650 classified tweets in the training set. Of the remaining 11,285 tweets, the SVM algorithm categorized 1287 tweets (11.4%) as having a revelation of sexual assault/abuse and 657 tweets (5.8%) as having a revelation of early sexual assault/abuse (
In
Based on the SVM classification, we then used the Demographics Pro prediction service to understand and compare the demographics of posters of sexual assault/abuse during the early #MeToo movement. We present these demographics in
Finally, from the SVM classification, we used the provided metadata on the number of followers of posters who revealed sexual assault/abuse in the first week of the #MeToo movement to better understand the reach of such revelations. Based on posters’ follower count, we took the 25%-75% range of followers from users with such experiences and calculated a lower bound range on the number of Twitter user who may have seen a first-person revelation. This simplistic calculation provided a range of 5,955,342 to 34,251,628 Twitter users for the reach, which we believe is a substantial lower bound (
We believe the reported reach is an underestimate, because we did not include replies or retweets and our algorithm has a higher false negative rate. We did not examine the network of people who reveal events, which could be highly overlapping and would mean that many users would have seen multiple tweets with revelations. Furthermore, the distribution of followers who posted about sexual assault does not seem to vary substantially from the distribution of followers for Twitter users overall (
Age
College
First time
Freshman
Grade
Hasnumber (Hasnumber is an overall indicator of whether there is a number in the text.)
High school
Kindergarten
Old
School
Years ago
Years old
Cop
Coworker
Date
Doctor
Father
Male
Neighbor
Rapist
Step father
Teacher
Uncle
Rape
Rip
Screw
Arm
Butt
Pussy
Afraid
Asleep
Drunk
Concert
Counts and percent of #MeToo tweets with disclosures of sexual abuse/assault and early experience tweets by date.
Date | Total, na | Abuse/assault, n (%) | Early experience, n (%) |
10/15/17 | 371 | 43 (11.59) | 25 (6.74) |
10/16/17 | 5987 | 817 (13.65) | 420 (7.02) |
10/17/17 | 3174 | 336 (10.59) | 142 (4.47) |
10/18/17 | 1155 | 113 (9.78) | 54 (4.68) |
10/19/17 | 676 | 57 (8.43) | 21 (3.11) |
10/20/17 | 356 | 31 (8.71) | 14 (3.93) |
10/21/17 | 215 | 14 (6.51) | 12 (5.58) |
aNumber of geotagged novel English language tweets in United States.
Demographic characteristics of abuse/assault and early life experience samples among unique Twitter users.
Characteristic | US census, %a | Twitter overall, %b | Abuse/assault sample (N=1168), %b,c | Early experience sample (N=612), %b,c | |
|
|
||||
|
Male | 49.2 | 45.8 | 10.6 | 9.2 |
|
Female | 50.8 | 54.2 | 89.4 | 90.8 |
|
|
||||
|
≤19 | 25.4 | 25.02 | 15.2 | 13.1 |
|
20-24 | 6.70 | 45.33 | 25.5 | 24.1 |
|
25-29 | 7.10 | 16.10 | 20 | 20 |
|
30-34 | 6.70 | 7.16 | 17.8 | 22 |
|
35-39 | 6.60 | 2.40 | 8.1 | 7.3 |
|
40-49 | 12.5 | 3.25 | 9.7 | 9 |
|
50-59 | 13.3 | 0.49 | 2.6 | 3.7 |
|
≥60 | 21.7 | 0.25 | 1 | 0.8 |
|
|
||||
|
White/Caucasian | 60.7 | 78.7 | 90.7 | 89.8 |
|
Hispanic | 18.1 | 7.6 | 6.2 | 6.1 |
|
African American | 13.4 | 13.1 | 2.6 | 3.3 |
|
Asian | 5.8 | 0.6 | 0.4 | 0.8 |
|
Native American/Pacific Islander | 1.5 | —d | — | — |
aAge distribution based on 2017 American Community Survey 1-Year Estimates (July 1, 2017).
bProportions provided by Demographics Pro on October 18, 2018.
cBased on our classification in the analytical sample of geotagged novel English language tweets in the United States.
dNot available.
We conduct the first quantitative text analysis of the content of the early conversation in the #MeToo movement on Twitter, which was the largest hashtag movement on Twitter in 2017 [
There are key limitations to this study that should be noted. First, based on financial considerations, we could not extract all novel English language posts during the first week of the #MeToo movement. Instead, we chose to focus on those geotagged within the United States. This restriction made the purchase of almost all the tweets that fit this category possible. To examine the representativeness of these tweets, we examined time patterns relative to all #MeToo posts and novel English language #MeToo posts (
Despite the noted limitations, our results highlight the magnitude of the mass sharing of personal experiences of sexual assault/abuse, filled with narratives of early life experience, which enabled the spread of the #MeToo movement and had a broad reach across Twitter. Further, these tweets and the archetypes presented here provide rich details to augment existing statistics captured traditionally from survey data and small in-depth studies of sexual assault and abuse survivors. The archetype provides a picture of what the public saw. The candid and revealing statements reminded followers and the public alike of the magnitude sexual violence and how it is often first experienced in early life and hidden. Therefore, the descriptive narratives could be used in public health survey development to assess whether there has been a change or a deeper public understanding of the prevalence, early life experience, and enduring trauma of sexual assault and abuse.
The summary of the content of tweets presented in this study highlights the initial conversation and demographics of participants in the conversation in the early stages of the #MeToo movement. Although our results present a snapshot of the public discourse on sexual violence and the initial participants, future work could examine the content and directions of the national conversation, which has since taken many directions and varies by populations.
Details of the selection process.
All versus novel tweets: example table.
Estimates of number, proportion, and reach of MeToo tweets from Oct 14 to 21, 2017.
Comparison of Twitter followers for abuse/assault and early experience sample against all twitter users.
application programming interface
least absolute shrinkage and selection operator
negative predictive value
ordinary least squares model
positive predictive value
support vector machine
None declared.