Ajou University repository

GTA: Gated Toxicity Avoidance for LM Performance Preservationoa mark
Citations

SCOPUS

0

Citation Export

Publication Year
2023-01-01
Journal
Findings of the Association for Computational Linguistics: EMNLP 2023
Publisher
Association for Computational Linguistics (ACL)
Citation
Findings of the Association for Computational Linguistics: EMNLP 2023, pp.14747-14763
Mesh Keyword
Generation methodLanguage modelModel generationNovel solutionsPerformanceSimple++State of the artText generationsToxicity reduction
All Science Classification Codes (ASJC)
Computational Theory and MathematicsComputer Science ApplicationsInformation SystemsLanguage and LinguisticsLinguistics and Language
Abstract
Caution: This paper includes offensive words that could potentially cause unpleasantness. The fast-paced evolution of generative language models such as GPT-4 has demonstrated outstanding results in various NLP generation tasks. However, due to the potential generation of offensive words related to race or gender, various Controllable Text Generation (CTG) methods have been proposed to mitigate the occurrence of harmful words. However, existing CTG methods not only reduce toxicity but also negatively impact several aspects of the language model's generation performance, including topic consistency, grammar, and perplexity. This paper explores the limitations of previous methods and introduces a novel solution in the form of a simple Gated Toxicity Avoidance (GTA) that can be applied to any CTG method. We also evaluate the effectiveness of the proposed GTA by comparing it with state-of-the-art CTG methods across various datasets. Our findings reveal that gated toxicity avoidance efficiently achieves comparable levels of toxicity reduction to the original CTG methods while preserving the generation performance of the language model.
Language
eng
URI
https://aurora.ajou.ac.kr/handle/2018.oak/37011
https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85183309567&origin=inward
DOI
https://doi.org/2-s2.0-85183309567
Type
Conference
Funding
This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No.2022-0-00680, Abductive inference framework using omni-data for understanding complex causal relations) and (IITP-2023-No.RS-2023-00255968, Artificial Intelligence Convergence Innovation Human Resources Development) We received support from the Google TPU Research Cloud to train the models required for this study. To make Fig 1, 2, and 3, we used commercially available public icons. Robot icons910 are created by itim2101 - Flaticon. Gate icon11 and Bearded male icon12 is create by Freepik - Flati-con.This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No.2022-0-00680, Abductive inference framework using omni-data for understanding complex causal relations) and (IITP-2023No.RS-2023-00255968, Artificial Intelligence Convergence Innovation Human Resources Development)
Show full item record

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

Related Researcher

Cho, Hyunsouk Image
Cho, Hyunsouk조현석
Department of Software and Computer Engineering
Read More

Total Views & Downloads

File Download