Stacking method for determining weights in partial least squares model averaging

Abstract

Model averaging has been developed to improve prediction accuracy in high dimensional regression. This approach is a weighted linear combination of some regression models. Two important procedures in model averaging are construction the candidate models and determination the weights. This paper evaluated performance of partial least squares model averaging (PLSMA) with some weights and proposed stacking as a method for determining the weights. Stacking determines the weights by minimizing least squares error over candidate models. Our proposed method (stacked-PLSMA) was evaluated in a simulation experiment and compared to equal weight, Akaike information criterion (AIC) weight, and Bayesian information criterion (BIC) weight. The result showed that stacked-PLSMA yielded smaller prediction error with high consistency than the other weights.