Is your chatbot perplexing?: Confident personalized conversational agent for consistent chit-chat dialogue - Na, Young Yun; Park, Junekyu; Sohn, Kyung Ah (ICAART 2021 - Proceedings of the 13th International Conference on Agents and Artificial Intelligence, 2021)
Chatbots are being researched and employed not only in academic settings but also in many fields as an application. Ultimately, conversational agents attempt to produce human-like responses along with dialogues. To achieve this goal, we built a novel framework that processes complex data consisting of personalities and utterances and fine-tuned a large-scale self-attention-based language model. We propose a consistent personalized conversational agent(CPC-Agent) for the framework. Our model was designed to utilize the complex knowledge of a dataset to achieve accuracy and consistency. Together with a distractor mechanism, we could generate confident responses. We compared our model to state-of-the-art models using automated metrics. Our model scored 3.29 in perplexity, 17.59 in F1 score, and 79.5 in Hits@1. In particular, the perplexity result was almost four times smaller than that of the current state-of-the-art model that scored 16.42. In addition, we conducted a human evaluation of each model to determine its response quality because the automatic evaluation metrics in dialogue tasks are still considered insufficient. Our model achieved the best rates from the voters, which indicated that our model is adequate for practical use.
Keywords: Automatic evaluation; Complex data; Conversational agents; Human evaluation; Human like; Language model; Practical use; State of the art; Chatbot; Confidence; Dialogue; Natural language processing; Personalized