Is your chatbot perplexing?: Confident personalized conversational agent for consistent chit-chat dialogue

Abstract

Chatbots are being researched and employed not only in academic settings but also in many fields as an application. Ultimately, conversational agents attempt to produce human-like responses along with dialogues. To achieve this goal, we built a novel framework that processes complex data consisting of personalities and utterances and fine-tuned a large-scale self-attention-based language model. We propose a consistent personalized conversational agent(CPC-Agent) for the framework. Our model was designed to utilize the complex knowledge of a dataset to achieve accuracy and consistency. Together with a distractor mechanism, we could generate confident responses. We compared our model to state-of-the-art models using automated metrics. Our model scored 3.29 in perplexity, 17.59 in F1 score, and 79.5 in Hits@1. In particular, the perplexity result was almost four times smaller than that of the current state-of-the-art model that scored 16.42. In addition, we conducted a human evaluation of each model to determine its response quality because the automatic evaluation metrics in dialogue tasks are still considered insufficient. Our model achieved the best rates from the voters, which indicated that our model is adequate for practical use.