A note on ridge regression modeling techniques


Abstract


In this study, the techniques of ridge regression model as alternative to the classical ordinary least square (OLS) method in the presence of correlated predictors were investigated. One of the basic steps for fitting efficient ridge regression models require that the predictor variables be scaled to unit lengths or to have zero means and unit standard deviations prior to parameters’ estimations. This was meant to achieve stable and efficient estimates of the parameters in the presence of multicollinearity in the data. However, despite the benefits of this variable transformation on ridge estimators, many published works on ridge regression practically ignored it in their parameters’ estimations. This work therefore examined the impacts of scaled collinear predictor variables on ridge regression estimators. Various results from simulation studies underscored the practical importance of scaling the predictor variables while fitting ridge regression models. A real life data set on import activities in the French economy was employed to validate the results from the simulation studies.

DOI Code: 10.1285/i20705948v7n2p343

Keywords: Ridge regression; orthogonality; shrinkage parameter; scaling; ordinary least squares; mean square error

References


Bradly, R. A., Srivastava, S. S. (1997). Correlation in polynomial regression. URL: http://stat.fsu.edu/techreports/M409.pdf

Cannon, A. J. (2009). Negative ridge regression parameters for improving the covariance structure of multivariate linear downscaling models. Int. J. Climatol., 29, 761-769.

Chatterjee, S., Hadi, A. S. (2006). Regression Analysis by Example. John Wiley & Sons, Inc., Hoboken, New Jersey.

Dorugade, A. V., Kashid, D. N. (2010). Alternative method for choosing ridge parameter for regression. Applied Mathematical Science, 4(9), 447-456.

El-Dereny, M. and Rashwan, N. I. (2011). Solving Multicollinearity Problem Using Ridge Regression Models. Int. J. Contemp. Math. Sciences, 6(12), 585-600.

Faraway, J. J. (2002). Practical regression and ANOVA using R. http://cran.r-project.org/doc/contrib/Faraway-PRA.pdf

Fearn, T. (1993). A misuse of ridge regression in the calibration of a near infrared reflectance instrument. Applied Statistics, 32, 73-79.

Hoerl, A. E., Kennard, R. W., Hoerl, R. W.(1985). Practical use of ridge regression: A challenge met. Applied Statistics, 34(2), 114-120.

Hoerl, A.E., Kennard, R.W., Baldwin, K.F. (1975). Ridge regression: Some simulations.Communications in Statistics, 4, 105-123.

Hoerl, A. E., Kennard, R.W.(1970a). Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12, 55-67.

Hoerl, A. E., Kennard, R. W. (1970b). Ridge regression: Applications to nonorthogonal problems. Technometrics, 12, 69-82, 1970b.

Khalaf, G., Shukur, G. (2005). Choosing ridge parameter for regression problem. Communications in Statistics–Theory and Methods, 34, 1177-1182.

Kibria, B. M. (2003). Performance of some ridge regression estimators. Communication in Statistics – Simulation and Computation, 32, 419-435.

Lawless, J. F., Wang, P. A. (1976). Simulation study of ridge and other regression estimators. Communications in Statistics –Theory and Methods, 14, 1589-1604.

Longley, J. W.(1976). An appraisal of least-squares programs from the point of view of the user. Journal of the American Statistical Association, 62, 819–841.

Lin, L., Kmenta, J. (1982). Ridge Regression under Alternative Loss Criteria. The Review of Economics and Statistics, 64(3), 488-494.

Malinvaud, E. (1968). Statistical Methods of Econometrics, Rand-McNally, Chicago.

Mardikyan, S., Cetin, E.(2008). Efficient Choice of Biasing Constant for Ridge Regression. Int. J. Contemp. Math. Sciences, 3, 527-547.

Marquardt, D. W., Snee, R. D. (1975). Ridge regression in practice. The American Statistician, 29(1), 3-20.

Muniz, G., Kibria, B. M.(2009). On Some Ridge Regression estimator: An empirical comparison. Communication in Statistics–Simulation and Computation, 38, 62-630.

Myers, R. H.(1986). Classical and Modern Regression with Applications. PWS-KENT Publishing Company, Massachusetts.

Sparks, R. (2004). SUR Models Applied To an Environmental Situation with Missing Data and Censored Values. Journal of Applied Mathematics and Decision Sciences, 8(1), 15-32, 2004.

Wethril, H. (1986). Evaluation of ordinary Ridge Regression. Bulletin of Mathematical Statistics, 18, 1-35, 1986.

Yahya, W.B., Adebayo, S.B., Jolayemi, E.T., Oyejola, B.A., Sanni, O.O.M. (2008). Effects of non-orthogonality on the efficiency of seemingly unrelated regression (SUR) models. InterStat Journals, 1-29.


Full Text: pdf
Bookmark and Share


Creative Commons License
This work is licensed under a Creative Commons Attribuzione - Non commerciale - Non opere derivate 3.0 Italia License.