We collected a corpus of tweets containing the hashtag #WhyIStayed or #WhyILeft using the Twitter and Topsy application programming interfaces, of which the latter is currently defunct. This corpus spans the beginning of September (the start of the trend) to the beginning of October 2014, when volume dropped to background levels. The majority of tweets are from the first week of the trend’s creation (Figure 1).

One way to address our main research question is to ask whether #WhyIStayed and #WhyILeft hashtags indicated distinct classes of micronarratives or were merely framing devices. To address this question, we trained naïve Bayes, linear support vector machine (SVM), and radial basis function (RBF) SVM classifiers from the scikit-learn python library [15], using the hashtags as ground truth, and various language features as input. We then use the SVMs to report the features that have the most predictive power. We also considered neural models [16,17], but it was harder to make sense of these results.

Four of the authors of this paper performed the annotation task. The overall agreement overlap was 0.77. Randolph’s free-marginal multirater kappa [18] score was 0.72. We chose this multirater kappa because it allows any distribution of class labels that annotators assign (ie, it is free-marginal), unlike Fleiss’ multirater kappa, which assumes a predetermined distribution [18]. According to the resulting annotations (Table 1), on average (over all annotators), 35.28% (1176/3333) of the instances were reasons for staying (S), 40.98% (1366/3333) were reasons for leaving (L), 13.83% (461/3333) were meta comments, 1.77% (59/3333) were jokes, 1.41% (47/3333) were ads, and 6.72% (224/3333) did not match prior categories (other).

The limited contextual information that such tweets provided sometimes made it difficult to interpret unambiguously. For instance because i was slowly dying anyway was marked as S by two annotators and L by the other two. The annotators disagreed on whether the victim decided to stay out of a sense of resignation or left because they felt there was nothing left to lose. (The ground truth label is #WhyILeft.) Another example of disagreement was two years of bliss, followed by uncertainty and fear. (This tweet’s label is #WhyIStayed.) However, our results show that most tweets contained enough information for humans to infer their original hashtag labels, with annotators correctly identifying L more frequently than S.

Basic lexical statistics in the balanced devset before lowercasing, stoplisting, and lemmatizing are shown in Table 2. The top nine most frequent unigrams, bigrams, and trigrams of words in the balanced dataset after lowercasing, stoplisting, and lemmatizing are shown in Tables 3 and 4. In order for word and ngram counts to reflect the words representing content rather than the words serving grammatical functions or tweet markup, before making these counts, we stoplisted, lemmatized, and excluded start-of-sentence and end-of-sentence tokens from each tweet.

Each table reveals new insights into the #WhyIStayed/#WhyILeft movement, so we examine each in turn.

The six words appearing in both columns of the unigram table (love, leave, abuse, want, and know) reveal common themes shared by both hashtags (Table 3). Our bi- and tri- gram analyses (presented below) reveal differences in how these common words are used. However, from the unigram perspective, the commonality of these words suggests, in the case of leave and abuse, that both #WhyIStayed and #WhyILeft tweets are often framed in terms of a leaving event, and that both often acknowledge the abuse that happened. This is perhaps not surprising, as the tags themselves suggest looking back on past abusive relationships. Love and want reveal that strong emotional forces are associated with both hashtags. Know indicates that knowledge (or lack thereof) is associated with both hashtags.

Of the words found in one list only, think and believe in the #Stayed list and realize in the #Left list suggest a transition between staying and leaving that involves learning the truth about false beliefs. The presence of tell and say in the #Stayed list suggests that an abuser coerced, deceived, or in some other way, emotionally manipulated the narrator. Deserve and life in the #Left list express positive sentiments.

Deserve better—the most common bigram by far—indicates that a change in one’s sense of self dignity or fairness plays a major role in leaving. Both realize and deserve occur three times in the #Left list, suggesting that increased awareness, particularly from the perspective of justice or fairness, were important forces in helping narrators leave abusive relationships (Table 3).

Two words (love and deserve) appear in both lists. Each is paired with think in #Stayed and realized in #Left, suggesting a change in awareness of what love and fair treatment really are, respectively. That deserve also appears with know for #Left suggests determination.

The presence of emotional abuse in the #Stayed list confirms that, to the narrators of these tweets, emotional manipulation played a role in their staying in abusive relationships.

One telling bigram pairing is kill leave from the #Stayed list and want live from the #Left list. They indicate that people both remained and left abusive relationships out of fear for the safety of self and loved ones (the latter being supported by considerations about children, such as year old, want daughter, in the #Left list).

The most frequent trigrams in each list make feel like and realize deserve better, respectively, make an interesting pair. They seem to indicate the important role emotional manipulation plays in staying, and that leaving is precipitated by a realization or epiphany, perhaps sometimes ones that break the spell of emotional abuse (Table 4).

Hit, yell, and kill all appear in the #WhyIStayed list, suggesting that violence is an important force in keeping people in abusive relationships. Notably, man appears twice in this list. The trigram church support spousal shows the important role institutions and moral or religious values play.

Better, deserve, and daughter each appear four times in the #WhyILeft list. This shows that concerns about the welfare of dependents (note that son also appears on the list) or a desire for a better life (and that such a life is deserved) drive decisions to leave. The most common word in the #Stayed list, url (which appears four times), is harder to interpret; recall that, to protect privacy, we removed all URLs and replaced them with this token.

Fourteen percent of the dev- and test-set tweets have a subject-verb-object (SVO) structure, in which (a) the abuser is doing something to the victim, or (b) the victim is explaining something about the abuser or self. Such SVO structures represent the largest proportion of the total number of dependency structures in this data. Thus, we focus on its exploration in both corpus analytics and automated classification. These SVO structures provide insight into the abuser-victim relationship while maintaining sentence-level structures large enough to convey or indicate syntactic relationships, which tend to be more interpretable than isolated words.

We used the following conditional model to identify the most indicative verbs in terms of predicting hashtag class (#WhyIStayed vs #WhyILeft), among SVO tweets. Starting at the lemmatized predicate verb in each dependency parse, whenever the predicate verb followed an abuser subject word and preceded a victim object word, we added it to a distribution conditioned on class. The abuser subject words were he, bf, boyfriend, father, dad, husband, brother, and man for a male abuser, she, gf, girlfriend, mother, mom, wife, sister, and woman for a female abuser, and finally, pastor, abuser, offender, ex, x, lover, church, and they were used as neutral references. The victim object words were me, sister, brother, child, kid, baby, friend, her, him, man, and woman. These are denoted here as abusers onto victim structures. Analogous methods were used to extract structures in which the victim was the subject. We then determined the most indicative verb predicates from these conditional frequency distributions using the following formula for each such predicate:

where count_left and count_stayed are the number of times the verb appears in #WhyILeft or #WhyIStayed tweets, respectively. Table 5 shows those where the ratio is greater than 0.70 and, to avoid a bias toward lower frequency verbs, the total count exceeds a threshold of 0.5% of the total number of instances. In the case of abuser onto victim, the resulting frequency threshold was 11, and in the victim as subject, it was 68.

Table 5 shows that physical abusers on victim verbs like choke and kill are associated with #WhyILeft, whereas for the victim as subject verbs realize appears as the most indicative verb in the data, along with find and learn. Additionally, a predominance of verbs about cognitive manipulation appear in #WhyIStayed tweets, such as convince, promise, believe, think, !think (where “!” denotes negation, for example, I did not think he would...) and tell. Heise et al [19] suggested that emotional dependence and an optimistic hope for change are reasons for staying, which these manipulative verbs seem to corroborate. Other interesting findings are the equal and opposite effects of love and !love, and the verb !hit, which suggests that perhaps the narrators believed they were not experiencing abuse because it was not physical, or that they feared physical retribution if they tried to leave.

We used naïve Bayes, logistic regression, linear SVM, and RBF SVM classifier methods to automatically predict whether a tweet was tagged with #WhyIStayed or #WhyILeft. The RBF SVM method performed slightly better than the others, achieving a maximum accuracy of 78% (SD 1%) on the devset and 78% on the test set using a subset of features and hyperparameters: max df=12%, C=10, gamma=1. To better understand which linguistic features and preprocessing steps were most important to these classifiers, we performed feature ablation, following the procedure in Fraser et al [20], to determine the most important features the classifier used for prediction. Interestingly, the SVO features combined with ngrams worsened performance slightly, perhaps because of trigrams capturing the majority of SVO cases, but likely also because they just covered a small fraction of the dataset. The highest accuracy, nearly 78% on the test set, used a combination of ngrams and retweet counts for features and informal register (tone) replacement in the preprocessing step.

We then used the confidence score of the linear SVM (defined as the distance from the classifier’s separating hyperplane in the feature space of the model) on each feature, taken as a single input to the SVM, as an estimate of that feature’s discriminativeness, or ability to distinguish between the hashtag class labels (Tables 5 and 6). This method can be seen as an alternative to the ngram count, which measures the predictive power of each ngram, rather than its frequency [21].

The SVM picked up on many of the same reasons for leaving and staying as those shown in Tables 3 and 4, but also revealed new ones, including tired, finally, and strong, which appear on the #WhyILeft list (Table 6). These seem to suggest less an epiphany or triggering crisis and more a sense that the narrator was aware of and tolerated abuse for a long time until it became too much to bear.

For staying, language about cognitive and verbal manipulation was prominent (think, believe, convince, tell, say, and feel). Several new reasons also appeared: try leave, money, and young. The phrase try leave backs up claims in clinical literature that it is often difficult to gain external support to leave, and that victims of abuse frequently go through cycles of abuse that involve leaving and returning multiple times [19]. Financial distress is often a key factor for staying [19,22], so it is no surprise that money appears as a top feature for the SVM. The word young suggests that many were too young to leave or too naïve (due to their youth) to recognize that their relationship was abusive.

Restricting the dev- and test-sets to just those instances having an SVO structure, we trained the naïve Bayes, linear SVM, and RBF SVM. The linear SVM performed best, yielding 72% accuracy.

Table 7 shows the top SVO structures using the confidence score of the linear SVM on each data item. Some interesting structures not found in Table 5 appear here. For example, the #WhyILeft list reveals interventions from nonabusers (sister tell me). Taking a closer look at the supporting tweets, for example because my sorority sisters and roommates told me nothing about how he treated me was okay, suggests that these SVO structures refer to social support to which the victim has access. In the #WhyIStayed class, church tell me once again shows that religious institutions can play a role in keeping victims in abusive relationships. Several tweets indicated that their church condoned abuse as a means of avoiding embarrassment and divorce, for example because the church told me that it was my responsibility as a godly wife to not embarrass him and just pray.

Our analysis shows that the process of leaving often involves a better understanding of the reality of an abusive relationship, or coming to terms with a long anticipated but hazardous decision to leave. The words and phrases found in our results tend to be about the pressures of staying versus leaving or the dynamics involved in leaving (Figure 2). Buel [22] explains why women may choose to stay in abusive relationships, including fear of retaliation, lack of financial independence, concern for their children, emotional dependence, lack of support from friends and family, fear of divorce and the potential to lose custody of their children, and/or an optimistic hope through love that their abuser will change. Children are often a factor in keeping victims in abusive relationships, and many victims will finally leave an abusive situation once their children have grown. The words and structures found in our results support many of these observations (eg, church tell me, emotional abuse, and daughter deserve better).

Heise et al [19] explain why victims of abuse leave and describe the dynamics of leaving. For instance, an increase in violence sometimes triggers a realization that their abuser will not change, that things are going to get worse, that the violence is going to affect their children, that they may be killed, etc. Support from friends, family, or society often allows those abused to leave. In any case, leaving is frequently a difficult process, involving cycles of denial, self-blame, and doubt. We found many of these same pressures and dynamics in our results (eg, realize love, want live, sister tell me).

Many of the findings from this study support a four-level ecological model [25,26] proposed by Heise et al [19] and expanded on by the World Health Organization [27]. All four levels that increase the likelihood that a man will abuse his partner are found in these data to varying degrees.

We note several limitations in this research.

Computational methods are applied to better understand the #WhyIStayed/#WhyILeft movement. Recent studies of this movement are concurrent with ours [29,30]. However, they were about their historical and qualitative aspects, or were based on a tiny sample of the data available. Our work is complementary to these studies, and our goal was to provide quantitative results that lend insight and credibility to these prior qualitative and clinical observations.

An extensive body of work explores how to extract affective information and other subjective signals from social media [31-35]. Adding part of speech tags to ngrams is often attempted as well as creating word classes via data inspection, using morphosyntactic features, and exploiting the sentiment of text instances. For instance, in Xu et al [36], linear models with ngrams are recommended for their simplicity and high accuracy, although in Lamb et al [37], word classes, Twitter-specific stylometry (retweet counts, hashtags, user mentions, and emoticons), and an indicator for phrases beginning with a verb were found to be helpful over ngrams on two different tasks.

Many of these works are motivated by commercial applications, for example, mining to extract individuals’ sentiments about products or services [38,39]. Another stream of research focuses specifically on modeling, extracting, and/or tracking emotions on social media. Some of these works deal with emotions independent of context [40-42]. Other studies have studied their correlations with time [43,44] or other socioeconomic phenomena [45]. Still others model emotion as a social contagion [46-48] or focus on specific contexts, such as employment [49]. More recently, researchers have focused on specific emotional conditions or behavioral phenomena.

With respect to behavioral health, Coppersmith et al [50-52] built classifiers for detecting a number of mental health conditions, including major depression, posttraumatic stress disorder, seasonal affective disorder, and attention deficit hyperactivity disorder, by training tweets that match regular expressions related to each condition. De Choudhury et al [53,54] collected labels to consider as ground truth regarding the presence of major depression using crowd sourcing. Other researchers have focused on specific health issues, including posttraumatic stress disorder [51], early detection of epidemics [37,55], and bullying tweets [36,56]. A number of recent papers have studied suicidality on social media [57,58], or risk factors for suicide such as distress [59]. Closer to this paper and concurrent with our research, sexual abuse disclosures via anonymous Reddit were studied qualitatively by O’Neill [33] and quantitatively by Andalibi et al [60]. Subramani et al [61] performed a similar quantitative analysis of Facebook pages related to domestic violence. Karlekar and Bansal [10] studied the use of machine learning to extract narratives of sexual harassment from the SafeCity web-based forum.

The research presented here demonstrates the power of social media to uncover meaningful structural, semantic, linguistic, and textual characteristics, including actions, stakeholders, and situations related to abusive relationships. It revealed micronarratives in tweeted reasons for staying versus leaving abusive relationships. A classifier for distinguishing between tweeted reasons for staying versus leaving abusive relationships achieved an accuracy of 78%. Our textual analysis, in showing that partners leave violent relationships after an epiphany of self-realization, is validated in the clinical literature. Moreover, the sheer volume of data present in social media suggests the potential to learn more details about the nature and dynamics of interpersonal violence than—due to the stigma and shame related to disclosing stories of victimization that the #WhyIStayed/#WhyILeft movement has helped erode—are currently known and may potentially help clinicians to reduce the harm caused by abusive relationships.

There are a number of interesting directions for future work. For instance, social media data with course-grained geotags could be used to study whether reasons for staying and leaving differ across geographical locations, or how varying community-level characteristics of those locations (eg, poverty level, population density, education levels, etc.) affect IPV victims. Analysis of web abuse discourse across varied media would strengthen the present findings if they overlapped, and perhaps lead to a better understanding of how victims make sense of and manage IPV and abuse.