Greedy Knot Selection Algorithm for Restricted Cubic Spline Regression

Abstract

Abstract Non-linear regression modeling is common in many fields for prediction purposes or estimating relationships between predictor and response variables. For example, restricted cubic spline regression can model non-linear relationships as third-order polynomials joined at knot points. The standard approach is to place knots by a regular sequence of quantiles between the outer boundaries. A regression curve can easily be fitted to the sample using a relatively high number of knots. The problem is then overfitting, where a regression model has a good fit to the given sample but does not generalize well to other samples. A low knot count is therefore preferred. However, the standard knot selection process can lead to underperformance in the sparser regions of the predictor variable, especially when using a low number of knots. It can also lead to overfitting in the denser regions. We present a simple greedy search algorithm using a backward method for knot selection that shows reduced prediction error and Bayesian information criterion (BIC) scores compared to the standard knot selection process in simulation experiments. We have implemented the algorithm as part of an open-source R-package, knutar.