Active Learning: Encoder-Decoder-Outlayer and Vector Space Diversification Sampling

Abstract

This study introduces a training pipeline comprising two components: the Encoder-Decoder-Outlayer framework and the Vector Space Diversification Sampling method. This framework efficiently separates the pre-training and fine-tuning stages, while the sampling method employs pivot nodes to divide the subvector space and selectively choose unlabeled data, thereby reducing the reliance on human labeling. The pipeline offers numerous advantages, including rapid training, parallelization, buffer capability, flexibility, low GPU memory usage, and a sample method with nearly linear time complexity. Experimental results demonstrate that models trained with the proposed sampling algorithm generally outperform those trained with random sampling on small datasets. These characteristics make it a highly efficient and effective training approach for machine learning models. Further details can be found in the project repository on GitHub.