Few-Shot Text Classification with an Efficient Prompt Tuning Method in Meta-Learning Framework

Abstract

Meta-learning stands as a prevalent framework utilized in few-shot learning methods. Nonetheless, its efficacy hinges on substantial data availability during meta-training. Recent work adeptly tackled this hurdle by synergizing prompt tuning with the meta-learning paradigm, consequently attaining unparalleled performance on four benchmarks (FewRel, HuffPost, Reuters and Amazon). Nonetheless, the implementation efficacy of the previous method leaves room for enhancement, which is especially crucial when tuning larger language models. To this end, we introduce another expedited prompt tuning approach nested within the meta-learning framework. The novel approach normalizes the label information and sample information and uses the regression method to obtain the closed-form solution of each few-shot task, which significantly enhances inference speed, achieving a twofold improvement, while concurrently elevating average accuracy by [Formula: see text]% on the same benchmarks. Moreover, it demonstrates enhanced stability when faced with limited meta-training data, which is more applicable in many real scenarios where parallel data is rare. The source code is available to reproduce the results (http://github.com/Dr-Lv/EMPT).