In this study, a self-developed, online, experimental platform was used to recruit participants. We first randomly sent invitations to approximately 20,000 Weibo users whose profiles indicated they were adults. Then, users who were willing to participate in our experiment were guided to the online experimental platform. Between September 2012 and February 2013, a total of 2428 Weibo users participated in the experiment. To ensure that the participants were active users of Weibo and that the psychological questionnaire was valid, we excluded (1) participants who had posted fewer than 500 Weibo texts since the initial registration of personal accounts and (2) participants who completed the questionnaire over too long or too short a period of time. Specifically, considering that PWB was part of the entire collected questionnaire (18 of 72 questions), we excluded participants who completed the entire questionnaire in less than 300 seconds or more than 1500 seconds (during the pre-experiment, the total time to complete the questionnaire was 6 minutes to 15 minutes). After normalizing the PWB score distribution (the changes in the label set’s distribution are shown in Figures 1 and 2), 1427 samples were ultimately included in this study, including 540 men and 887 women. In the end, a total of 471,173 posts were included in the analysis of this study.

The ethical conduct of the research was approved by the Institutional Review Board at the Institute of Psychology, Chinese Academy of Sciences, under code H15009.

In this study, we used multiobjective regression to consider all PWB dimensions as a whole to develop a novel model. However, a basic premise of this approach is that these dimensions are correlated (even weakly) [12]. Therefore, we first analyzed the Pearson correlation coefficients between PWB dimensions. Next, considering that we introduced domain knowledge to build the prediction model, we analyzed the feature importance of each psychological lexicon after using the attribute of the extra trees algorithm to output the feature importance of the language features that were used for modeling. In detail, we calculated the sum of the feature importance and the average feature importance (the ratio of the sum of the feature importance of each lexicon compared with the number of features included in each lexicon) of each lexicon. Last, the method by Wang et al [11] was used as a reference to verify the reliability and validity of the model. The split-half reliability for the prediction model was assessed between the “split-half” scores using Pearson correlation coefficients. Multitrait-multimethod matrix analysis and criterion validity analysis were conducted on the “whole” data to test the model’s validity [11]. All the Pearson correlation coefficients were calculated using SPSS 22.0 (IBM Corp, Armonk, NY) [50].

First, we collected demographic information from the participants. Among all the participants, 62.2% (887/1427) were women. Participants' ages ranged from 18 years to 80 years, with a median age of 23 (SD 5.8) years. Most of the participants were located in cities (1318/1427, 92.4%). Among the study population, 52.1% (743/1427) of the participants had a bachelor’s degree, and 41.4% (591/1427) of them had a high school degree or less. The vast majority (1305/1427, 91.4%) of the number of posts by participants were concentrated between 1000 and 5000 posts. Table 2 displays the demographic information of these individuals.

We used the attribute of the extra trees algorithm to output the feature importance of the language features that were used for modeling. The importance of a feature indicates how important the feature was to the model building process. The sum of the values of the importance of the features used was 1. All 120 linguistic features were used to train the prediction model. As shown in Table 3, the sum of the importance of each lexicon was calculated. Since total feature importance increases with the number of features, we also calculated the average feature importance for each lexicon, which is the ratio of the sum of the importance of each lexicon compared with the number of features included in each lexicon (see Table 3).

The results of this study showed that the SC-LIWC made the greatest contribution to predicting PWB, reaching a total importance of more than 70%. However, with respect to average feature importance, the word-average importance of the SC-LIWC lexicon was the lowest. The linguistic features of CVD performed the best during the model building process, followed by CSD and MMD and then by Weibo-5BML and MFD. The levels of importance of the features of these 5 lexicons were higher than that of SC-LIWC.

The Pearson correlation coefficient r can describe the degree of linear correlation between 2 variables; r’s absolute value stands for the strength of the correlation. Table 4 shows that all the dimensions of the actual scale scores exhibited a significant positive correlation.

Table 4 presents the matrix of the Pearson correlation coefficients between variables. The subdimensions of PWB were included in the multitrait-multimethod matrix, including environmental mastery, positive relations with others, autonomy, personal growth, self-acceptance, and purpose in life, and 2 methods were involved in this process, including the PWB subscales and the linguistic prediction model.

Structural validity includes 2 different aspects, specifically discriminant validity and convergent validity. In Table 4, the correlation coefficients using different methods to measure the same traits are all greater than the other numbers, which indicates that the convergent validity of our multiobjective prediction model is good. However, the discriminant validity of this model is fair. Except for the dimension of autonomy, not all the correlation coefficients using different methods to measure the same traits are greater than the corresponding coefficients using the same method to measure different traits.

The Pearson correlation coefficients between the effective standard and the predicted scores of the “whole” data are shown as R₁ in Table 5. When measuring a psychological variable using different assessment instruments or methods, the correlation coefficient between different instruments or methods is typically approximately 0.39 to 0.68 [51]. Our results are all significantly greater than or equal to 0.49, which means that the developed model exhibits good criterion validity.

The Pearson correlation coefficient of the predicted scores for the 2 sets of “split-half” data are displayed in Table 5 (see R₂), with all dimensions reaching the level of significance, which suggests that the model has good split-half reliability.

This study tested the ability of the prediction model to predict well-being levels based on linguistic data drawn from social media. We collected PWB scale scores as the output, and based on 6 psychological lexicons, we extracted linguistic features of posts on Weibo as the input. We then developed a multiobjective prediction model featuring 6 dimensions. Subsequently, the importance of the 6 lexicons used for model building was analyzed. At last, we explored the linguistic prediction model's criterion validity, structural validity (both to verify the model’s validity), and split-half reliability (to verify the model’s reliability). The results suggest that the prediction model had good criterion validity and split-half reliability. However, when it comes to structural validity, the convergent validity of the model was really good, but the discriminant validity was not satisfactory.

In this study, we used domain knowledge to extract features, which increases the interpretability of this prediction model. The results of this study showed that the SC-LIWC had the highest contribution to predicting PWB, with a total importance of more than 70%. This finding is consistent with those in several previous studies that used LIWC as a closed-vocabulary method of extracting social media features [15,47,52] and provides evidence of the validity of SC-LIWC [31]. However, regarding the average importance of these 6 lexicons, the word-average importance of the SC-LIWC lexicon was the lowest. We speculate that this deficiency is due to the fact that SC-LIWC is a general psycholinguistic lexicon. In fact, we were able to use SC-LIWC to distinguish people’s emotional states, intentions, thinking styles, and individual differences, but SC-LIWC is not a targeted measurement of PWB nor other psychological characteristics associated with PWB. Therefore, like our hypothesis, the linguistic features extracted using the other 5 lexicons based on domain knowledge performed better [52,53]. We also found that the linguistic features of CVD performed the best. One possible explanation for this finding is that people cannot define mental health in isolation from culture [5]. PWB, as an important component of well-being, cannot be explained by theories that lack reference to culture [54]. Furthermore, all understanding of PWB is based on moral visions, which also explains the good performance of MMD and MFD [54]. In addition, our finding concerning the good performance of CSD is also in line with those of previous research that reported a significant negative correlation between people's PWB level and their suicidal attitudes [38]. As suggested by Gubler et al [55], this study investigated the correlation between emotion and PWB using Weibo-5BML to extract features and reported primary evidence.

Consistent with previous research pertaining to the prediction of well-being based on social media [15-17], the high criterion validity suggests that users’ PWB could also be identified solely based on social media posts. In addition, even if some state-of-the-art methods, such as deep learning [56] and open-vocabulary methods [19], were not used in this investigation, the prediction model could achieve better accuracy by relying solely on domain knowledge. Another finding was that all 6 dimensions of PWB had similar criterion validity. One possible explanation for this fact is that these 6 dimensions are all distinct components of positive psychological functioning and serve as a theoretical foundation for generating a multidimensional model of well-being [23,57]. Another such reason is the fact that we chose a multiobjective regression algorithm to preserve the correlations among PWB dimensions.

All dimensions also exhibited fairly high convergent validity. Specifically, the correlations between different approaches measuring the same trait were all stronger than the correlations between different approaches measuring different traits. As we would expect, this suggested that the 2 methods measured the same trait. Numerous earlier studies have confirmed that diverse types of mental health status are expressed in various ways in language and even the same type of mental health status also has different expressions in language at different levels [15,35,58]. Our finding indicates that different components of PWB may have specific expressions in language that are more detailed and refined. These unique expressions of each component should be explored in the future. Furthermore, given the strong correlations (0.24-0.85) among the scores of the subscales in the scale by Ryff and Keyes [23] and the strong correlations (0.19-0.61) between the predicted scores, it is reasonable the model failed to exhibit good discriminant validity.

Referring to the reliability, our results suggest a good level of split-half reliability across all dimensions (ranging from 0.65 to 0.85), which is similar to results (ranging from 0.66 to 0.88) reported in earlier studies that have examined the PWB scale by Ryff and Keyes [57,59,60]. The believable reliability result indicates that the linguistic features we extracted based on domain knowledge, rather than other random factors, can illustrate people’s psychological status to a certain extent. According to studies that have argued for the robustness of using social media to reflect people’s implicit traits [22,61], such variations may stem from other personal characteristics.

This study validated the predictive ability and reliability of machine learning models in the context of ground truth mental health data, which is an important issue that must be addressed. Another unexpected finding of this study is that our results confirmed that the application of specific domain knowledge could help improve the predictive performance of machine learning models and contribute to the interpretability of such models. Although closed vocabulary analysis was proved not as good as open vocabulary analysis in a previous study [19], our model still achieved relatively advanced prediction accuracy [62]. This also proves the importance of introducing domain knowledge training prediction models from the side. The prediction model trained in this study incorporated knowledge from experts in the form of psychological lexicons that depict the linguistic patterns associated with different levels of PWB, which improved the model performance. It should be noted that the current prediction model still cannot replace scales nor dictate the judgment of a counselor but instead provides alerts and decision-making support. However, our study still has positive implications for practical applications in nonprofessional settings. Specifically, because the method of measuring mental health using social media data has been endorsed by our findings, it could be used for self-examination, helping to gain a clearer picture of oneself. In addition, the method could also be used for large-scale user studies to quickly provide feedback on the psychological state of the public, which is of great help to social governance.

A limitation of this study is that most of our participants were young women. The participants in this study were randomly sampled from social media, which might have caused some degree of sampling bias. In addition, this study was only carried out in the Chinese Weibo online environment, so the findings cannot be generalized to other languages or online platforms. Last, the performance of the prediction model tested in a new data set is unknown.

In this study, we used the closed vocabulary method and a machine learning algorithm to develop our prediction model. Utilizing deep learning algorithms for natural language processing may bring some better outcomes, but this aspect was not the focus of this study. We could further combine domain knowledge and a deep learning method in the future. In addition, considering that the quality of the online questionnaire depends on the participants' conscious compliance with the rules for completing the questionnaire, we chose the 18-item version of the PWB scale by Ryff and Keyes to reduce the time required to complete the scale and ensure participants’ attention. Further studies could choose other scales to measure individuals’ well-being more precisely. For example, considering the discriminant validity was not satisfactory in this work, using scales (eg, Oxford Happiness Inventory) that combine different dimensions to a single target is also worth trying. In addition, future work could compare the prediction performance between a single additive target and multiple targets. Last but not the least, many studies have used prediction models to compare changes before and after in the same individuals [58,63]. These studies, including this study, all have a common assumption; that is, individuals before and after a certain time node are regarded as different individuals. This may not be sufficiently accurate to measure people’s longitudinal changes. Further work needs to be done to update the data set by merging longitudinal data into cross-sectional data; then, this more comprehensive data set could be used to build prediction models so as to improve the prediction accuracy within participants.

This study proposed a linguistic prediction model based on social media. By confirming the validity and reliability of the model, this study verified the predictive power of social media corresponding to ground truth mental health data from the perspective of PWB. Our study has positive implications for the use of social media to predict mental health in nonprofessional settings such as self-monitoring or large-scale user studies. Future research can explore natural language processing methods combined with domain knowledge to enhance the interpretability of deep learning models.