Affiliation:
1. Institute for Infocomm Research, Singapore
2. Carnegie Mellon University, Pittsburgh, PA
3. Georgia Institute of Technology, Atlanta, GA
4. National University of Singapore, Singapore
Abstract
We propose a maximal figure-of-merit (MFoM)-learning approach for robust classifier design, which directly optimizes performance metrics of interest for different target classifiers. The proposed approach, embedding the decision functions of classifiers and performance metrics into an overall training objective, learns the parameters of classifiers in a decision-feedback manner to effectively take into account both positive and negative training samples, thereby reducing the required size of positive training data. It has three desirable properties: (a) it is a performance metric, oriented learning; (b) the optimized metric is consistent in both training and evaluation sets; and (c) it is more robust and less sensitive to data variation, and can handle insufficient training data scenarios. We evaluate it on a text categorization task using the Reuters-21578 dataset. Training an F
1
-based binary tree classifier using MFoM, we observed significantly improved performance and enhanced robustness compared to the baseline and SVM, especially for categories with insufficient training samples. The generality for designing other metrics-based classifiers is also demonstrated by comparing precision, recall, and F
1
-based classifiers. The results clearly show consistency of performance between the training and evaluation stages for each classifier, and MFoM optimizes the chosen metric.
Publisher
Association for Computing Machinery (ACM)
Subject
Computer Science Applications,General Business, Management and Accounting,Information Systems
Reference42 articles.
1. Exploiting latent semantic information in statistical language modeling;Bellegarda J. R.;Proc. IEEE.,2000
2. Breiman L. Friedman J. H. Olshen R. A. and Stone C. J. 1984. Classification and Regression Trees. Wadsworth Belmont CA. Breiman L. Friedman J. H. Olshen R. A. and Stone C. J. 1984. Classification and Regression Trees. Wadsworth Belmont CA.
Cited by
16 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献