Learning and Technical Market -Effects of In-Sample Data Selection-

Abstract

For a learning system, nothing is more influential than data to be learned. Any data, whether in-sample or out-of-sample, is intrinsically particular. This is typically true for complex market data. Data for the last year on a single stock name, for instance, is only a small segment of data patterns if compared to unseen out-of-sample data having potentially numerous patterns. Any technical system trained with limited in-sample data is more or less particular, thus likely becomes to be overfitted. If in-sample data is particular, its performance for out-of-sample data is expected to be poor, no matter how well the technical system is trained. In other words, a better trained technical system frequently has lower performance for out-of sample data. If not well controlled, training is generally a particularity-seeking process of in-sample data, requiring some mechanism outside of the learning process. Possible solutions may exist in in-sample data selection, learning sophistication, or technical indicator effectiveness. This study is intended to provide a better understanding of technical markets and learning, and suggestions on in-sample data selection. Experiments examine how the selection of in-sample data affects the performance of a target system for out-of-sample data and show technical market and learning phenomena using genetic algorithms (GA).