Solving Multicolinearity Problem in a Linear Regression: A Comparative Study of Ordinary Least Squares and Partial Least Squares Regression

Abstract

Ordinary Least Squares (OLS) estimator usually yields inefficient estimates when multicollinearity is present in a Linear Regression Model. The inefficiency of OLS can be mitigated by Partial Least Squares Regression (PLSR). However, this method requires selecting latent variables in order to yield efficient estimates of regression parameters. This paper proposes using weighted standard errors and ranking standard errors of regression coefficients for latent variable extraction, alongside model selection methods such as Akaike Information Criterion (AIC), Bayesian Information Criterion (BIC), Adjusted R Squared (ARS), and Standard Error of Regression Coefficient (SER). Monte-Carlo experiments on a Linear Regression Model with six explanatory variables were conducted one thousand (1000) times across eleven levels of multicollinearity [low (0.0, 0.2 and 0.4), moderate (0.6 and 0.8), high (0.95, 0.96, 0.97, 0.98,) and very high (0.99 and 0.999)] and five levels of sample size [small (20), medium (30 and 50) and large (100 and 250)]. Results indicated that while OLS estimates are preferred overall, Partial Least Squares with BIC becomes preferable at high multicollinearity levels. Additionally, the efficiency of OLS improves with larger sample sizes despite multicollinearity.