Using neural networks for binary classification problem of social media messages in the field of urban management

This study explores the application of various BERT architecture models, part of the transformer family, in solving the task of classifying messages from social networks. The relevance of the work is determined by the need to develop a solution to this problem in order to improve the efficiency of information processing in urban management. To achieve this, a fine-tuning approach was adopted, leveraging pre-trained models from open sources based on available annotated data. Various models based on this architecture were utilized, such as BERT, RuBERT, RoBERTa, among others. The results demonstrated the higher effectiveness of these models across multiple metrics including accuracy, recall, processing speed, and false positive rate. To maximize recall metric, the threshold for assigning an object to the positive class was lowered, resulting in a slight decrease in model precision. Models pre-trained on Russian texts and social media texts performed particularly well. The best-performing model was ai-forever/ruBert-base, which significantly reduced the number of false positive responses.