Enhancing Voice Phishing Detection Using Multilingual Back-Translation and SMOTE: An Empirical Study

Abstract

With the widespread global trend of voice phishing or vishing attacks, the development of effective detection models using artificial intelligence (AI) has been hindered by the lack of high-quality and large volumes of data. This lack of data reflecting a real vishing scenario often leads to imbalanced datasets and biased detection models. Therefore, we present in this paper a data augmentation (DA) method for expanding the imbalanced Korean call content vishing (KorCCVi) dataset to address the existing data asymmetry problem and enhance the performance of Korean vishing detection. The proposed approach for DA involves using the back-translation (BT) method with three different intermediate languages: English, Chinese, and Japanese. The proposed method offers several advantages over the traditional synthetic minority oversampling technique (SMOTE), which is the main technique used to compare with our multilingual BT approach. Using these two DA techniques, several machine learning (ML) and deep learning (DL) models were trained on the original imbalanced dataset, the dataset balanced with SMOTE and its variants, and the dataset augmented with our method. We analyzed the impact of these DA methods on the performance of the models, demonstrated the benefits of each approach, and suggested the most suitable approach. The performance of the trained models was evaluated using the accuracy, precision, recall, and F1-score metrics. The experimental results demonstrated that the proposed multilingual BT method effectively expands the dataset while preserving its contextual and linguistic characteristics. The average performance of the models revealed that those trained on the augmented dataset outperformed the other models. They achieved F1-scores of 98.91% for the back-translated data, 98.14% for the original data, and 97.23% for SMOTE.