A Selective Masked Language Model for Implicit Emotion Recognition

Abstract

This thesis leverages the Masked Language Modeling (MLM) task, which has significantly advanced natural language processing (NLP) by enabling models to learn rich contextual word representations. It utilizes the transformer-based BERT model, renowned for its powerful contextual understanding capabilities. To address the challenging task of implicit emotion recognition, various masking strategies were investigated, including random masking, PMI-based masking, TF-IDF based masking, lexicon-based masking, and gradient masking. Additionally, a novel hybrid masking strategy was introduced, combining the strengths of PMI and random masking to enhance model performance. This hybrid approach achieved the best results, with an F1-score of 66.34%, highlighting its effectiveness in improving emotion recognition accuracy.