Generalization Analysis of Pairwise Learning for Ranking With Deep Neural Networks

Abstract

Abstract Pairwise learning is widely employed in ranking, similarity and metric learning, area under the ROC curve (AUC) maximization, and many other learning tasks involving sample pairs. Pairwise learning with deep neural networks was considered for ranking, but enough theoretical understanding about this topic is lacking. In this letter, we apply symmetric deep neural networks to pairwise learning for ranking with a hinge loss ϕh and carry out generalization analysis for this algorithm. A key step in our analysis is to characterize a function that minimizes the risk. This motivates us to first find the minimizer of ϕh-risk and then design our two-part deep neural networks with shared weights, which induces the antisymmetric property of the networks. We present convergence rates of the approximation error in terms of function smoothness and a noise condition and give an excess generalization error bound by means of properties of the hypothesis space generated by deep neural networks. Our analysis is based on tools from U-statistics and approximation theory.