Cross-Attention Model for Multi-modal Bio-Signal Processing - Heesoo, Son; Sangseok, Lee; Sael, Lee (Proceedings - 2022 IEEE International Conference on Big Data and Smart Computing, BigComp 2022, 2022)
In order to comprehensively predict human condition, it is beneficial to analyze various bio-signals obtained from human body. Existing multi-modal deep sequence models are often very complex models that involve significantly more parameters than single-modal models. However, since the number of multi-modal signal data is small compared to single-modal, more efforts is needed to reduce the model complexity of multi-modal models. We introduce a multi-modal sequence classification model based on our cross-attention blocks, which aims to reduce the number of parameters involved in cross-referencing different modes. We compare our method with baseline sequential deep learning models, LSTM and Transformer, and a competitor. We test our methods on two public datasets and a dataset obtained from construction works. We show that our model outperforms compared methods in accuracy and number of parameters when the number of modes increases.
Keywords: Attention model; Biosignal processing; Biosignals; Cross-attention; Human bodies; Human conditions; Modal models; Multi-modal; Sequence models; Single-modal; bio-signals; cross-attention; multi-modal analysis