METHOD FOR IDENTIFYING PROMISING RESEARCH TOPICS BASED ON HYBRID TRANSFORMER ARCHITECTURES USING SCIENTOMETRIC DATABASES

Abstract

The article addresses the critical challenge of automating the identification of promising research topics within largescale scientometric databases. The exponential growth of scientific publications creates significant information overload, complicating the discovery of latent trends. To solve this, the study proposes a universal method based on a hybrid deep learning architecture that integrates the strengths of Convolutional Neural Networks (CNN) and Transformer architectures. The core of the proposed architecture utilizes a CNN layer as a local feature extractor to identify stable terminological patterns (n-grams) within article abstracts. This is followed by a cascade of three Transformer layers with a MultiHeadAttention mechanism, which models global semantic dependencies, ensuring high relevance in specialized domains. To handle the significant class imbalance typical of scientometric data (where «High Impact» topics account for approximately 28 %), the Focal Loss function was implemented. This allows the model to concentrate on «hard» examples, significantly improving the detection of emerging breakthrough directions. The method was validated using a Scopus metadata dataset from the IoT Security domain, comprising 4833 publications from 2020–2025. Experimental results show that the model achieves a Recall of 0.80 and an F1-score of 0.61, outperforming previous LSTM-based approaches. Such a balance of metrics is methodologically justified for research discovery tasks, where minimizing the risk of missing a «breakthrough» topic (Type II error) is the primary priority. Furthermore, the approach includes a trend forecasting module based on linear regression of citation time series, enabling the prediction of a topic’s future popularity. The developed solution is universal and can be adapted to any scientific or applied field to accelerate research discovery and support decision-making in grant allocation.