Deep learning models with encoder-decoder architecture become popular in automatic speech recognition systems, due to their success in sequential prediction tasks. Recently, the conformer model has greatly improved the accuracy of speech recognition. However, similar to transformer models, its training relies on a large amount of data. This paper explores an efficient few-shot learning strategy. Specifically, a spec-augment approach is proposed to augment the speech dataset, then a novel loss function, anti-focal loss, is introduced to encourage fast convergence in a small-scale, unbalanced data setting. Extensive experiments on aishell-l dataset show that our model outperforms state-of-the-art approaches under limited support data, in terms of convergence speed and generalization ability.
