Arabic text summarization using three-layer bidirectional long short-term memory (BILSTM) architecture

This work presents an improved approach to the challenging problem of Arabic Text Summarization (ATS) by introducing a novel model that seamlessly integrates state-of-the-art neural network architectures with advanced Natural Language Processing (NLP) techniques. Inspired by classical ATS approaches, our model leverages a three-layer Bidirectional Long Short-Term Memory (BiLSTM) architecture which is augmented with Transformer-based attention mechanisms and AraBERT for preprocessing, to successfully tackle the notoriously challenging peculiarities of the Arabic language. To boost performance, our model further draws upon the power of contextual embeddings from models such as GPT-3, and through the use of advanced data augmentation techniques including back-translation and paraphrasing. To further improve performance, our approach integrates novel techniques for training and uses Bayesian Optimization to perform hyperparameter optimization. The model evaluated against state-of-the-art datasets such as the Arabic Headline Summary (AHS) and Arabic Mogalad_Ndeef (AMN) and reported on traditional evaluation metrics including: ROUGE-1, ROUGE-2, ROUGE-L, BLEU, METEOR and BERTScore. This work is significant because it presents an important step forward in the task of Arabic Text Summarization (ATS) towards summarizing text to be not only coherent and concise, but also authentic and culturally relevant in an effort to push forward NLP research and applications for Arabic.