Email this article An adaptive sampling and weighted ensemble of surrogate models for high dimensional global optimization problems
Abstract
The modern engineering design optimization relies heavily on high- fidelity computer. Even though, the computing ability of computers have increased drastically, design optimization based on high-fidelity simulations is still time consuming and impractical. Surrogate modeling is a technique to replace the high-fidelity simulations. This paper presents a novel approach, named weighted ensemble of surrogates (WESO) for computationally intensive optimization problems, The focus is on multi-modal functions to identify its global optima with relatively few function evaluations. WESO search mechanism falls in two steps, explore and fit. The “explore” step is based on exploring the whole design region by generating sample points (agents) using Latin hypercube sampling (LHS) technique to gain prior knowledge about the function of interest (learning phase). The “fit” step is to train and fit a weighted ensemble of surrogate models over the promising region (training phase) to mimic the computationally intensive true function and replace it with a surrogate model (cheap function). The surrogates are then utilized to select candidates’ decision variable points at which the true objective function and constraints’ functions to be evaluated. Weights are then determined, assigned and an ensemble of surrogate gets constructed using the candidate sample points where optimization can be carried out. WESO has been evaluated on classical benchmark functions embedded in larger dimensional spaces. WESO was also tested on the aerodynamic shape optimization of turbo-machinery airfoils to demonstrate its ability in handling computationally intensive optimization problems. The results showed to what extent combinations of models can perform better than single surrogate models and provide insights into the scalability and robustness of the approach. WESO can successfully identify near global solutions, faster than other classical global optimization algorithms.
References
Myers RH, Montgomery DC, Anderson-Cook CM (2016) Response surface methodology: process and product optimization using designed experiments. J. Wiley & Sons.
Matheron G (1963) Principles of geo statistics. Economic Geology 58(8):1246-1266.
Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Statistical Science 4(4):409-423.
Kerh TF, Lai JS, Gunaratnam D, Saunders R (2008) Evaluation of Seismic Design Values in the Taiwan Building Code by Using Artificial Neural Network. Computer Modeling in Engineering & Sciences 26(1):1-12.
Gutmann HM (2001) A radial basis function method for global optimization. Journal of Global Optimization 19(3):201-227.
Sun G, Li G, Gong Z, He G, Li Q (2011) Radial basis functional model for multi-objective sheet metal forming optimization. Engineer Optimization 43(12):1351-1366.
Smola, AJ, Schölkopf, B (2004) A tutorial on support vector regression. Statistics and computing, 14(3):199-222.
Friedman, J. H. (1991), “Multivariate Adaptive Regression Splines.” The Annals of Statistics, 19 (1): 1-67.
M.P. Perrone, L.N. Cooper, When Networks Disagree: Ensemble Methods for Hybrid Neural Networks, In: R.J. Mammone, ed., Artificial Neural Networks for Speech and Vision, Chapman and Hall, London, pp 126–142,1993.
L.E. Zerpa, N.V. Queipo, S. Pintos, J. Salager, An Optimization Methodology of Alkaline–Surfactant–Polymer Flooding Processes Using Field Scale Numerical Simulation and Multiple Surrogates, Journal of Petroleum Science and Engineering, vol. 47, pp. 197–208, 2005.
. T. Goel, R.T. Haftka,W. Shyy, N.V.Queipo, Ensemble of Surrogates, Struct. Multidisc. Optim. vol. 33, pp. 199–216, 2007.
. G. Evensen, The Ensemble Kalman Flter: Theoretical Formulation and Practical Implementation, Ocean Dyn., vol 53, pp. 343–367, 2003.
. D. Madigan, A.E. Raftery, Model Selection and Accounting for Model Uncertainty in Graphical Models using Occam’sWindow, J AmStat Assoc, vol. 89(428), pp. 1535–1546, 1994.
. S.T. Buckland, K.P. Burnham, N.H. Augustin,Model Selection: An Integral Part of Inference, Biometrics, vol. 53, pp. 603–618, 1997.
I. Kaymaz, C.A. McMathon, A Response Surface Method Based on Weighted Regression for Structural Reliability Analysis, Probabilistic Engineering Mechanics, vol. 20, pp. 11–17, 2005.
Rommel G. Regis & Christine A. Shoemaker (2013) Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization,” Engineering Optimization, 45:5, 529-555
Martina Friese, Thomas Bartz-Beielstein, Michael Emmerich, “Building Ensembles of Surrogates by Optimal Convex Combination,” Bioinspired Optimization Methods and their Application, Bled, Slovenia, May 2016, Volume: Jožef Stefan Institute, Lubljana, Slowenien, 131-143
Acar, E.; Rais-Rohani, M. Ensemble of metamodels with optimized weight factors. Struct. Multidiscip. Optim. 2009, 37, 279–294.
]Yi Zhang, Xiaoqian Chen; Ning Wang; Wen Yao; Bingxiao Du, “Ensemble of surrogates based on error classification by unsupervised learning,” 2016 IEEE Congress on Evolutionary Computation (CEC), 24-29 July 2016, Vancouver, BC, Canada
E. Sanchez, S. Pintos, N.V.Queipo, Toward an Optimal Ensemble of Kernelbased Approximations with Engineering Applications, Struct. Multi. disc. Optim., vol. 36(3), pp. 247–261, 2008.
F.A.C. Viana, R.T. Haftka, V. Steffen, Jr., Multiple Surrogates: How Cross-Validation Errors Can Help Us to Obtain the Best Predictor, Struct. Multidisc., Optim., vol. 39, pp. 439–457, 2009.
M.P. Perrone, L.N. Cooper, When Networks Disagree: Ensemble Methods for Hybrid Neural Networks, In: R.J. Mammone, ed., Artificial Neural Networks for Speech and Vision, Chapman and Hall, London, pp 126–142, 1993.
Adel Younis and Zuomin Dong (2012) Global Optimization Using Mixed Surrogates and Space Elimination in Computationally Intensive Engineering Designs, International Journal for Computational Methods in Engineering Science and Mechanics, 13:4, 272-289
E. Acar and M. Rais-Rohani, Ensemble of Metamodels with Optimized Weight Factors, Struct and Multidisc Optim., vol. 37, pp. 279–294, 2009.
H.C.W. Lau, G.T.S. Ho, Yi Zhao, “A demand forecast model using a combination of surrogate data analysis and optimal neural network approach,” Decision Support Systems, Volume 54, Issue 3, 2013, Pages 1404-1416
Taimoor Akhtar and Christine A. Shoemaker, “Combining local surrogates and adaptive restarts for global optimization of moderately expensive functions,” AIP Conference Proceedings 2070, 020048 (2019)
Alizadeh, R., Jia, L., Nellippallil, A., Wang, G., Hao, J., Allen, J., & Mistree, F. (2019). Ensemble of surrogates and cross-validation for rapid and accurate predictions using small data sets. Artificial Intelligence for Engineering Design, Analysis and Manufacturing, 33(4), 484-501.
Ye, P.; Pan, G. Selecting the Best Quantity and Variety of Surrogates for an Ensemble Model. Mathematics 2020, 8, 1721.
Dhamotharan, V.; Jadhav, P.D.; Ramu, P. Optimal design of savonius wind turbines using ensemble of surrogates and CFD analysis. Struct. Multidiscip. Optim. 2018, 58, 2711–2726.
Gu, X.; Lu, J.; Wang, H. Reliability-based design optimization for vehicle occupant protection system based on ensemble of metamodels. Struct. Multidiscip. Optim. 2015, 51, 533–546.
Cheng, K.; Lu, Z. Structural reliability analysis based on ensemble learning of surrogate models. Struct. Saf. 2020, 83, 101905.
Acar, E. Various approaches for constructing an ensemble of metamodels using local measures. Struct. Multidiscip. Optim. 2010, 42, 879–896.
Lee, Y.; Choi, D.H. Pointwise ensemble of meta-models using v nearest points cross- validation. Struct. Multidiscip. Optim. 2014, 50, 383–394.
Amir Safari1, Adel Younis, Gary Wang, Hirpa Lemu and Zuomin Dong, “Development of a metamodel assisted sampling approach to aerodynamic shape optimization problems,” Journal of Mechanical Science and Technology 29 (5) (2015) 2013~2024
Full Text: pdf
