Early Detection of Mental Disorder via Social Media Posts Using Deep Learning Models
Abstract:
Mental health, which has as equally important effects on people’s life as physical health, is receiving more and more attention nowadays, especially with a significant increase of pressure brought by the fast-paced evolution of technology and society. The diagnosis of mental health symptoms, however, mostly relies on the interpretation of languages and behaviors by experienced psychologists, who are not accessible for the great population. Depression causes cognitive and motor changes that affect speech production: reduction in verbal activity productivity, prosodic speech irregularities, and monotonous speech have all been shown to be symptomatic of depression. In this study, we aim to provide a machine learning-based model that could give an initial diagnosis of mental health problems for individuals and screen the risk of developing mental health issues for the general population. This AI-driven model focuses on the understanding and analysis of people’s daily public comments/posts and captures the peoples’ mental health status embedded in the semantic and syntactic structure in those online posts. Powered by an AI-driven data crawler, we are able to provide real-time statistics of public mental health status by analyzing public comments and producing suggestive data analysis for the public health department to take precautions/corresponding measures to protect not only the physical but the mental health of the public.Bibliography/Citations:
1] M. Reinert, D. Fritze, T. Nguyen, The state of mental health in america
2022.
[2] G. Park, H. A. Schwartz, J. C. Eichstaedt, M. L. Kern, M. Kosinski, D. J. Stillwell, L. H. Ungar, M. E. Seligman, Automatic personality assessment through social media language., Journal of personality and social psychology 108 (6) (2015) 934.
[3] M. De Choudhury, M. Gamon, S. Counts, E. Horvitz, Predicting depression via social media, in: Seventh international AAAI conference on weblogs and social media, 2013.
[4] A. G. Reece, A. J. Reagan, K. L. Lix, P. S. Dodds, C. M. Danforth, E. J. Langer, Forecasting the onset and course of mental illness with twitter data, Scientific reports 7 (1) (2017) 1–11.
[5] S. Tsugawa, Y. Kikuchi, F. Kishino, K. Nakajima, Y. Itoh, H. Ohsaki, Recognizing depression from twitter activity, in: Proceedings of the 33rd annual ACM conference on human factors in computing systems, 2015, pp. 3187–3196.
[6] G. Gkotsis, A. Oellrich, S. Velupillai, M. Liakata, T. J. Hubbard, R. J Dobson, R. Dutta, Characterisation of mental health conditions in social media using informed deep learning, Scientific reports 7 (1) (2017) 1–11.
[7] J. Du, Y. Zhang, J. Luo, Y. Jia, Q. Wei, C. Tao, H. Xu, Extracting psychiatric stressors for suicide from social media using deep learning, BMC medical informatics and decision making 18 (2) (2018) 77–87.
[8] J. Kim, J. Lee, E. Park, J. Han, A deep learning model for detecting mental illness from user content on social media, Scientific reports 10 (1) (2020) 1–6.
[9] M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, L. Zettlemoyer, Deep contextualized word representations, in: Proc. of NAACL, 2018.
[10] A. Radford, K. Narasimhan, T. Salimans, I. Sutskever, Improving language understanding by generative pre-training.
[11] A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al., Language models are unsupervised multitask learners, OpenAI blog 1 (8) (2019) 9.
[12] T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., Language models are few-shot learners, arXiv preprint arXiv:2005.14165.
[13] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint arXiv:1810.04805.
[14] K. Saha, A. Sharma, Causal factors of effective psychosocial outcomes in online mental health communities, in: Proceedings of the International AAAI Conference on Web and Social Media, Vol. 14, 2020, pp. 590–601.
[15] A. Sharma, A. S. Miner, D. C. Atkins, T. Althoff, A computational approach to understanding empathy expressed in text-based mental health support, in: EMNLP, 2020.
[16] E. Sharma, M. De Choudhury, Mental health support and its relationship to linguistic accommodation in online communities, in: Proceedings of the 2018 CHI conference on human factors in computing systems, 2018, pp. 1–13.
[17] N. Friedrich, T. D. Bowman, W. G. Stock, S. Haustein, Adapting sentiment analysis for tweets linking to scientific papers, arXiv preprint arXiv:1507.01967.
[18] R. Rehurek, P. Sojka, Gensim–python framework for vector space modelling, NLP Centre, Faculty of Informatics, Masaryk University, Brno, Czech Republic 3 (2).
[19] V. Sanh, L. Debut, J. Chaumond, T. Wolf, Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter, arXiv preprint arXiv:1910.01108
Additional Project Information
Project files
Research Plan:
Rationale:
Mental health, which has as equally important effects on people’s life as physical health, is receiving more and more attention nowadays, especially with a significant increase of pressure brought by the fast-paced evolution of technology and society. The diagnosis of mental health symptoms, however, mostly relies on the interpretation of languages and behaviors by experienced psychologists, who are not accessible for the great population. Depression causes cognitive and motor changes that affect speech production: reduction in verbal activity productivity, prosodic speech irregularities and monotonous speech have all been shown to be symptomatic of depression. In this study, we aim to provide a deep learning based model that could give initial diagnosis of mental health problems for individuals and screen the risk of developing mental health issues. This AI-driven model focuses on the understanding and analysis of people’s daily public comments/posts and captures the peoples’ mental health status embedded in the semantic and syntactic structure in those online posts through the state-of-the-art language model Bidirectional Encoder Representations from Transformers (BERT) for natural language processing (NLP).
A. Question or Problem being addressed:
- Can natural language processing models find subtle language patterns using raw text of online posts to detect the presence of mental health disorder(s)?
- How can the performance of the models investigated be measured and improved?
- Among all the models investigated, which model is the most effective model for detecting mental disorder signs from social media text?
B. Goals/Expected Outcomes/Hypotheses: Natural language processing models can find subtle language patterns using raw text of online posts to detect the presence of mental health disorder(s). State of the art pre-trained language models can improve the accuracy of such predictive power.
C. Description in detail of method or procedures
Procedures:
- Collect data that has social media text labeled as having or not having presence of mental health disorders from pre-existing labeled data sets of the mental health status of social media posts, such as Dreaddit and TalkLife.
- Develop a script to collect social media data from Twitter, Reddit, and other social media platforms.
- Design scripts to do pre-preprocessing on the collected data, such as converting all cases to lowercase, removing URLs and symbols, expanding contractions, and removing punctuation and stop words.
- Construct baseline models that can be used to evaluate other deep learning models, such as a random predictor model, Sentence2vec model, and Long short-term memory (LSTM) model.
- Construct the deep learning models, which are the pre-trained BERT model and fine-tuned BERT model. Train over training portion of data set.
- Design a quantitative test solution to evaluate deep learning models.
- Complete experiments running the models over the dataset and recording the accuracy of what percentage of the validation data that the model correctly predicts the label of.
- Analyze qualitative data to compare each deep learning model, including specific samples that do not contain key words.
- Look for the most effective model for detecting mental disorder signs from social media text based on qualitative and quantitative results.
Questions and Answers
1. What was the major objective of your project and what was your plan to achieve it?
- Was that goal the result of any specific situation, experience, or problem you encountered?
- Were you trying to solve a problem, answer a question, or test a hypothesis?
The major objective of my project was to determine if machine learning models could be applied to effectively predict the presence of mental health disorders in social media posts. This was the result of my personal encounters with the diminishing mental health of my peers and the wider public especially during the pandemic, and supported by further research about declining mental health. Thus, I attempted to test the hypothesis that machine learning models would be able to detect language patterns in social media text by constructing different models, with a focus on pretrained and finetuned Bidirectional Encoder Representations from Transformers (BERT) models, running experiments, and evaluating the performances.
2. What were the major tasks you had to perform in order to complete your project?
- Collect data that has social media text labeled as having or not having presence of mental health disorders from pre-existing labeled data sets of the mental health status of social media posts, such as Dreaddit and TalkLife.
- Develop a script to collect social media data from Twitter, Reddit, and other social media platforms.
- Design scripts to do pre-preprocessing on the collected data, such as converting all cases to lowercase, removing URLs and symbols, expanding contractions, and removing punctuation and stop words.
- Construct baseline models that can be used to evaluate other deep learning models, such as a random predictor model, Sentence2vec model, and Long short-term memory (LSTM) model.
- Construct the deep learning models, which are the pre-trained BERT model and fine-tuned BERT model. Train over training portion of data set.
- Design a quantitative test solution to evaluate deep learning models.
- Complete experiments running the models over the dataset and recording the accuracy of what percentage of the validation data that the model correctly predicts the label of.
- Analyze qualitative data to compare each deep learning model, including specific samples that do not contain key words.
- Look for the most effective model for detecting mental disorder signs from social media text based on qualitative and quantitative results.
3. What is new or novel about your project?
- Is there some aspect of your project's objective, or how you achieved it that you haven't done before?
- Is your project's objective, or the way you implemented it, different from anything you have seen?
- If you believe your work to be unique in some way, what research have you done to confirm that it is?
Previously existing projects have investigated detecting mental health issues in social media posts with alternative methods, such as with traditional statistical methods. Other projects have applied deep learning methods of convolutional neural networks (CNN) and long short-term memory networks (LSTM) on user content from social media, such as in Gkotsis et all 17, Du et al 18, and Kim et all 20. BERT pretraining of bidirectional transformers for natural language processing (NLP) has also been detailed, such as in Devlin et all 18. However, I uniquely applied and tested the pretrained and finetuned BERT models for the specific problem of social media text.
4. What was the most challenging part of completing your project?
- What problems did you encounter, and how did you overcome them? Too little data
- What did you learn from overcoming these problems?
The most challenging part of completing the project was gathering and generating the high-quality labeled dataset. Because data forms the basis for training and validating the best possible models, I wanted the most high quality data to begin with. However, existing data sets were limited. Thus, I decided to supplement the data set with data I scraped from many alternative sources, such as from social media platforms on servers and posts tagged or hashtagged with particular keywords. This taught me to be creative in my problem solving, and to look in a variety of different places for answers when they did not present themselves to me all in the first place I looked.
5. If you were going to do this project again, are there any things you would do differently the next time?
I would explore more standard text processing techniques and use existing robust text processing packages for initial text processing such as spelling checks. The text processing steps I have done are still relatively simple and don’t handle the network languages used among the young people, so it could affect the language model, especially the pretrained BERT model accuracy, if those new network language works are not considered in the pretraining. I would also go for more powerful language models such as BERT and GPT-3 (light version) instead of spending too much time trying simple word2vec and RNN models. They provide a good learning experience, yet from a project perspective, those relatively simple models didn’t provide satisfying results and based on this learning experience, I would focus more time on state of the art complex language models. Finally, I could extend the project and build and automatic pipeline to have the developed model serving online and collecting real time feedback, which could be used to further improve the model accuracy and capability.
6. Did working on this project give you any ideas for other projects?
Working on this project made me realize the power of NLP at solving a lot of traditional heavy labor text-related work. I could also extend the skills and knowledge I gained from this project to other NLP related projects, such as auto-comedy script generation, auto linguistic tutoring for children, and audio to script. My experience also inspires me to further develop my solutions and make them more applicable, such as by creating an interactable platform for users to input their social media information and receive rapid results. This might enhance the accessibility of the tool to a larger audience, which was one of my ultimate goals.
7. How did COVID-19 affect the completion of your project?
COVID-19 did not have major effects on the completion of my project because I was able to work by myself, while utilizing tools accessed online.