Biomedical Text Classification Using LSTM, GRU and Bahdanau Attention

Abstract

This research evaluates the performance of deep learning models—Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM), with Bahdanau attention added—for biomedical text classification using two datasets: the Memorial Sloan Kettering Cancer Center (MSKCC ) dataset for binary classification of genetic mutations and the Medical Abstracts dataset for multi-class disease categorization. The experiments showed that GRU models generally offer better training efficiency and balanced accuracy than LSTM models, and that class weighting proved more effective than Synthetic Minority Over-sampling Technique (technique for oversampling minority classes) (SMOTE) in handling class imbalance. While few-shot learning remains challenging, models using Biomedical Natural Language Processing (BioMedNLP) contextual embeddings combined with attention mechanisms demonstrated promising generalization, particularly in low-resource scenarios. These findings support the development of more robust and equitable Natural Language Processing (NLP) systems for biomedical applications.