From Scores to Predictions in Multi-Label Classification: Neural Thresholding Strategies
-
Published:2023-06-27
Issue:13
Volume:13
Page:7591
-
ISSN:2076-3417
-
Container-title:Applied Sciences
-
language:en
-
Short-container-title:Applied Sciences
Author:
Draszawka Karol1ORCID, Szymański Julian1ORCID
Affiliation:
1. Department of Computer Systems Architecture, Faculty of Electronics, Telecommunications and Informatics, Gdańsk University of Technology, 80-233 Gdańsk, Poland
Abstract
In this paper, we propose a novel approach for obtaining predictions from per-class scores to improve the accuracy of multi-label classification systems. In a multi-label classification task, the expected output is a set of predicted labels per each testing sample. Typically, these predictions are calculated by implicit or explicit thresholding of per-class real-valued scores: classes with scores exceeding a given threshold value are added to a prediction set. In our work, we propose a neural network-based thresholding phase for multi-label classification systems and examine its influence on the overall classification performance measured by micro- and macro-averaged F1 scores on synthetic and real datasets. In contrast to classic thresholding methods, our approach has the unique property of being able to recover from scoring errors, because each decision about a given label prediction depends on the corresponding class score, as well as on all the other class scores for a given sample at once. The method can be used in combination with any classification system that outputs real-valued class scores. The proposed thresholding methods are trained offline, after the completion of the scoring phase. As such, it can be considered a universal fine-tuning step that can be employed in any multi-label classification system that seeks to find the best multi-label predictions based on class scores. In our experiments on real datasets, the input class scores were obtained from two third-party baseline classification systems. We show that our approach outperforms the traditional thresholding methods, which results in the improved performance of all tested multi-label classification tasks. In terms of relative improvement, on real datasets, the micro-F1 score is higher by up to 40.6%, the macro-F1 score is higher by up to 3.6%, and the averaged micro–macro-F1 score is higher by up to 30.1%, considering single models only. We show that ensembles and hybrid models give even better results. We show examples of successful extreme recoveries, where the system, equipped with our method, was able to correctly predict labels, which were highly underscored after the scoring phase.
Subject
Fluid Flow and Transfer Processes,Computer Science Applications,Process Chemistry and Technology,General Engineering,Instrumentation,General Materials Science
Reference55 articles.
1. Partalas, I., Kosmopoulos, A., Baskiotis, N., Artières, T., Paliouras, G., Gaussier, E., Androutsopoulos, I., Amini, M.R., and Gallinari, P. (2023, May 26). LSHTC: A Benchmark for Large-Scale Text Classification. Available online: https://hal.science/hal-01691460. 2. LightXML: Transformer with Dynamic Negative Sampling for High-Performance Extreme Multi-label Text Classification;Jiang;Proc. Aaai Conf. Artif. Intell.,2021 3. Label-representative graph convolutional network for multi-label text classification;Vu;Appl. Intell.,2022 4. Hybrid embedding-based text representation for hierarchical multi-label text classification;Ma;Expert Syst. Appl.,2022 5. Khataei Maragheh, H., Gharehchopogh, F.S., Majidzadeh, K., and Sangar, A.B. (2022). A New Hybrid Based on Long Short-Term Memory Network with Spotted Hyena Optimization Algorithm for Multi-Label Text Classification. Mathematics, 10.
|
|