Ajou University repository

Multi-Channel Spatio-Temporal Transformer for Sign Language Production
Citations

SCOPUS

0

Citation Export

DC Field Value Language
dc.contributor.authorMa, Xiaohan-
dc.contributor.authorJin, Rize-
dc.contributor.authorChung, Tae Sun-
dc.date.issued2024-01-01-
dc.identifier.urihttps://aurora.ajou.ac.kr/handle/2018.oak/37104-
dc.identifier.urihttps://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85195911057&origin=inward-
dc.description.abstractThe task of Sign Language Production (SLP) in machine learning involves converting text-based spoken language into corresponding sign language expressions. Sign language conveys meaning through the continuous movement of multiple articulators, including manual and non-manual channels. However, most current Transformer-based SLP models convert these multi-channel sign poses into a unified feature representation, ignoring the inherent structural correlations between channels. This paper introduces a novel approach called MCST-Transformer for skeletal sign language production. It employs multi-channel spatial attention to capture correlations across various channels within each frame, and temporal attention to learn sequential dependencies for each channel over time. Additionally, the paper explores and experiments with multiple fusion techniques to combine the spatial and temporal representations into naturalistic sign sequences. To validate the effectiveness of the proposed MCST-Transformer model and its constituent components, extensive experiments were conducted on two benchmark sign language datasets from diverse cultures. The results demonstrate that this new approach outperforms state-of-the-art models on both datasets.-
dc.description.sponsorshipThis work was supported by the Institute of Information & communications Technology Planning & Evaluation (IITP) under the Artificial Intelligence Convergence Innovation Human Resources Development (IITP-2024-RS-2023-00255968) grant, the ITRC (Information Technology Research Center) support program (IITP-2021-0-02051) funded by the Korea government (MSIT), and the Foreign Intelligence support program funded by Shijiazhuang Science and Technology Bureau (Project No. 20240024).-
dc.language.isoeng-
dc.publisherEuropean Language Resources Association (ELRA)-
dc.subject.meshLanguage production-
dc.subject.meshMachine-learning-
dc.subject.meshMulti channel-
dc.subject.meshProduction models-
dc.subject.meshSign language-
dc.subject.meshSign language production-
dc.subject.meshSpatio-temporal-
dc.subject.meshSpatio-temporal fusions-
dc.subject.meshSpoken languages-
dc.subject.meshTransformer-
dc.titleMulti-Channel Spatio-Temporal Transformer for Sign Language Production-
dc.typeConference-
dc.citation.conferenceDate2024.5.20. ~ 2024.5.25.-
dc.citation.conferenceNameJoint 30th International Conference on Computational Linguistics and 14th International Conference on Language Resources and Evaluation, LREC-COLING 2024-
dc.citation.edition2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings-
dc.citation.endPage11712-
dc.citation.startPage11699-
dc.citation.title2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings-
dc.identifier.bibliographicCitation2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC-COLING 2024 - Main Conference Proceedings, pp.11699-11712-
dc.identifier.scopusid2-s2.0-85195911057-
dc.subject.keywordSign Language Production-
dc.subject.keywordSpatio-Temporal Fusion-
dc.subject.keywordTransformer-
dc.type.otherConference Paper-
dc.subject.subareaTheoretical Computer Science-
dc.subject.subareaComputational Theory and Mathematics-
dc.subject.subareaComputer Science Applications-
Show simple item record

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Related Researcher

Chung, Tae-Sun Image
Chung, Tae-Sun정태선
Department of Software and Computer Engineering
Read More

Total Views & Downloads

File Download

  • There are no files associated with this item.