Affiliation:
1. School of Cyber Security and Computer, Hebei University, Baoding 071002, China
Abstract
Artificially generated datasets often exhibit biases, leading conventional deep neural networks to overfit. Typically, a weighted function adjusts sample impact during model updates using weighted loss. Meta-neural networks, trained with meta-learning principles, generalize well across tasks, acquiring generalized weights. This enables the self-generation of tailored weighted functions for data biases. However, datasets may simultaneously exhibit imbalanced classes and corrupted labels, posing a challenge for current meta-models. To address this, this paper presents Meta-Loss Reweighting Network (MLRNet) with fusion attention features. MLRNet continually evolves sample loss values, integrating them with sample features from self-attention layers in a semantic space. This enhances discriminative power for biased samples. By employing minimal unbiased meta-data for guidance, mutual optimization between the classifier and the meta-model is conducted, endowing biased samples with more reasonable weights. Experiments on English and Chinese benchmark datasets including artificial and real-world biased data show MLRNet’s superior performance under biased data conditions.
Subject
Fluid Flow and Transfer Processes,Computer Science Applications,Process Chemistry and Technology,General Engineering,Instrumentation,General Materials Science
Reference46 articles.
1. Ren, M., Zeng, W., Yang, B., and Urtasun, R. (2018, January 10–15). Learning to reweight examples for robust deep learning. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden.
2. Gong, R., Qin, X., and Ran, W. (2023). Prompt-Based Graph Convolution Adversarial Meta-Learning for Few-Shot Text Classification. Appl. Sci., 13.
3. Learning from imbalanced data;He;IEEE Trans. Knowl. Data Eng.,2009
4. Understanding deep learning (still) requires rethinking generalization;Zhang;Commun. ACM,2021
5. Hendrycks, D., Mazeika, M., Wilson, D., and Gimpel, K. (2018). Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise. arXiv.