Information Based Universal Feature Extraction in Shallow Networks

Abstract

In many real-world image based pattern recognition tasks, the extraction and usage of task-relevant features are the most crucial part of the diagnosis. In the standard approach, either the features are given by common sense like edges or corners in image analysis, or they are directly determined by expertise. They mostly remain task-specific, although human may learn the life time, and use different features too, although same features do help in recognition. It seems that a universal feature set exists, but it is not yet systematically found. In our contribution, we try to find such a universal image feature set that is valuable for most image related tasks. We trained a shallow neural network for recognition of natural and non-natural object images before different backgrounds, including pure texture and handwritten digits, using a Shannon information-based algorithm and learning constraints. In this context, the goal was to extract those features that give the most valuable information for classification of the visual objects, hand-written digits and texture datasets by a one layer network and then classify them by a second layer. This will give a good start and performance for all other image learning tasks, implementing a transfer learning approach. As result, in our case we found that we could indeed extract unique features which are valid in all three different kinds of tasks. They give classification results that are about as good as the results reported by the corresponding literature for the specialized systems, or even better ones.