Continual Learning with Flexible Preservation

Abstract

To conquer several different tasks using only a single model, various attempts are being made in the name of continual learning. However, in the real world, when a new type of task or data emerges, it takes a considerable amount of time to collect enough data for the model to learn. Which means, when learning a new task, a problem occurs in which sufficient data required for learning is not obtained. In this study, we propose rigid Elastic Weight Consolidation(rEWC) to overcome these problems by improving Elastic Weight Consolidation(EWC), which prevents catastrophic forgetting based on weight regularization method. The proposed method showed better prediction performance for newly added tasks while showing less catastrophic forgetting than the existing EWC. Furthermore, in a situation where the number of data that can be learned decreases as the task number increases, it showed performance that overwhelms other widely known continual learning models, including EWC.