Two-phase Sequential Approximate Optimization using Radial Basis Function Surrogate Model with Cumulative Distribution Function Transformation

Abstract

Surrogate model-based sequential approximate optimization (SAO) method has been developed to achieve efficiently a global optimum solution of optimization problems by using the sequentially constructed surrogate model. The key feature here is the balance between the exploitation that searches for the region of interest and the exploration that finds the unexplored region. This method can be categorized into two types. The first type employs one phase to choose sample points with probability metrics. The second type separates the exploitation phases and the exploration phase, and then each phase is carried out alternately. In the first type, however, the probabilities become zero in most regions, which lead to the flat and locally sharp shape in the sampling criterion. It occurs the problem that fails to detect the region of interest in optimization. The probability-based criterion has a high value that is likely to be improved and/or be explored, which could lead to inefficient optimization. Even with separated phases, inefficient optimization could be performed, because each phase has no relation towards improving the current minimum. Moreover, the condition for carrying out the exploitation is not clearly defined. Lastly, regardless of the type, the accuracy of the surrogate model is an important parameter, because information obtained from the constructed surrogate model is used to exploit the region of interest. In this dissertation, a surrogate model-based SAO method using two phases is proposed. In the exploitation phase, the region of interest is restricted to the local domain to effectively find the minimum. The exploration phase is performed while minimizing the objective function. The condition for carrying out the exploitation phase is defined according to the possibility of improvement of objective function. In both phases, the proposed probability density model (PDM) is used. In the proposed model, surrogate modeling is performed after the transformation of outputs by using the cumulative distribution function (CDF). The CDF transformation magnifies the region of interest, in which sample points become dense during the surrogate model-based SAO. It makes outputs more distinctive around the interesting zone and strictly normalizes the outputs between 0 and 1. Then, the transformed outputs are interpolated by the radial basis function surrogate model using the Gaussian basis function. The second derivative of the PDM is calculated and the convexity of the PDM is satisfied for the certain range and the condition. Thus, the condition and the range of exploitation phase are derivable. In result, the PDM provides a local minimum by representing the convex shape. In order to compare the performances of the proposed method with the performances of the existing methods, mathematical examples with various features are employed. In two-dimensional optimization problems, 8 mathematical examples are classified according to the direction to minimization of the objective function and the shapes of the feasible region. Four multi-dimensional examples are also solved to analyze behaviors of the proposed method in high dimensional problems. As design applications, a piezoelectric energy harvester by ABAQUS and a combat vehicle by MapleSim are formulated. As a result, for problems with the unidirectional objective functions regardless of the shape of the feasible region, the proposed method outperforms the existing methods in the efficiency and the robustness of optimization because of the exploitation phase with the restricted local domain and the minimization of objective function in the exploration phase.|순차근사최적화 기법은 순차적으로 구축된 대체모델의 정보를 활용하여 전역최적해를 효율적으로 도출하는 것을 목적으로 한다. 순차근사최적화는 관심영역을 탐색하는 개척 (exploitation)과 설계영역 중 빈 공간을 탐색하는 탐험 (exploration)을 조율하여 전역최적해를 도출한다. 순차근사최적화 기법은 개척과 탐험을 하나의 단계로 조합하여 사용하는 일단계 기법과 개척과 탐험을 분리하여 각각의 단계로 사용하는 이단계 기법으로 나뉜다. 기존의 일단계 기법은 개척과 탐험을 조합하기 위해 확률기반 척도 (criterion)를 제안하였다. 하지만, 확률 값은 대부분의 설계영역에서 0의 확률을 나타내고, 높은 확률 값은 국부적으로 존재한다. 따라서, 확률기반 척도 (criterion)는 평평하거나 뾰족한 형태가 되고, 이 형태는 최적화 알고리즘이 탐지하기 어렵다. 또한, 개선영역과 탐험영역의 확률 값이 동시에 높은 값을 갖기 때문에, 효율성이 저하될 수 있다. 이단계 기법의 경우에도, 각 단계가 목적함수의 개선가능성으로 연계되어있지 않기 때문에 비효율적이며, 각 단계를 구분하는 조건 또한 명확하지 않다. 한편, 기존의 두 가지 기법 모두 대체모델의 정보를 이용하여 최적화를 수행하기 때문에, 기법들의 성능이 대체모델의 정확성에 의존적이다. 본 논문에서는, 새로운 이단계 순차근사최적화 기법을 제안한다. 제안하는 기법에서는 효과적으로 최적해를 찾기 위하여 설계영역을 국부영역으로 한정하여 개척단계를 수행한다. 탐험단계는 탐험과 목적함수의 최소화를 동시에 고려하여 효율성을 증가하도록 한다. 제안하는 기법은 탐험단계 수행 중, 목적함수의 개선이 가능한 영역을 찾았을 경우 개척단계를 수행하도록 한다. 두 단계에서는 확률밀도모델 (probability density model)을 제안하여 사용한다. 확률밀도모델은 먼저 누적분포함수를 이용하여 출력 값을 변환한 후 대체모델을 구성한다. 누적분포함수 변환은 순차근사최적화 도중 표본의 밀도가 증가하는 관심영역을 강조한다. 본 변환은 관심영역의 값들을 명확히 하며, 0과 1사이로 엄밀하게 정규화한다. 변환된 출력 값은 가우시안기저함수 기반의 방사형기저함수 대체모델을 이용하여 근사 한다. 확률밀도모델의 이계도함수로부터 볼록성 (convexity)을 만족할 조건을 정리하면, 특정 범위와 조건이 유도된다. 따라서, 볼록성으로부터 개척단계를 수행하기 위한 조건과 범위를 정의하고, 확률밀도모델을 이용하여 범위 내에 유일하게 존재하는 전역최적해를 도출한다. 기존의 순차근사최적화 기법과 제안하는 기법의 성능을 비교하기 위해, 함수의 형태가 다른 수학예제들을 선별하였다. 비제약최적화 문제들은 목적함수의 최소화 방향과 유용영역의 형태로 구분하였다. 또한, 유용영역이 매우 작거나 큰 형태에서 다변수 문제에서의 제안하는 알고리즘의 거동을 확인하기 위해 다변수 수학예제의 최적화를 수행하였다. 공학 적용 예제로써, Abaqus 기반 압전 하베스터와 MapleSim 기반 전투차량을 사용하였다. 그 결과, 유용영역 형태와 상관없이 목적함수의 최소화가 한 방향일 때, 타 기법에 비해 뛰어난 효율성과 강건성을 보였다. 이러한 특징은 제한된 영역으로 개척단계를 수행한 것과 탐험단계에서 목적함수의 최소화를 고려한 것에서 기인하였다.; Surrogate model-based sequential approximate optimization (SAO) method has been developed to achieve efficiently a global optimum solution of optimization problems by using the sequentially constructed surrogate model. The key feature here is the balance between the exploitation that searches for the region of interest and the exploration that finds the unexplored region. This method can be categorized into two types. The first type employs one phase to choose sample points with probability metrics. The second type separates the exploitation phases and the exploration phase, and then each phase is carried out alternately. In the first type, however, the probabilities become zero in most regions, which lead to the flat and locally sharp shape in the sampling criterion. It occurs the problem that fails to detect the region of interest in optimization. The probability-based criterion has a high value that is likely to be improved and/or be explored, which could lead to inefficient optimization. Even with separated phases, inefficient optimization could be performed, because each phase has no relation towards improving the current minimum. Moreover, the condition for carrying out the exploitation is not clearly defined. Lastly, regardless of the type, the accuracy of the surrogate model is an important parameter, because information obtained from the constructed surrogate model is used to exploit the region of interest. In this dissertation, a surrogate model-based SAO method using two phases is proposed. In the exploitation phase, the region of interest is restricted to the local domain to effectively find the minimum. The exploration phase is performed while minimizing the objective function. The condition for carrying out the exploitation phase is defined according to the possibility of improvement of objective function. In both phases, the proposed probability density model (PDM) is used. In the proposed model, surrogate modeling is performed after the transformation of outputs by using the cumulative distribution function (CDF). The CDF transformation magnifies the region of interest, in which sample points become dense during the surrogate model-based SAO. It makes outputs more distinctive around the interesting zone and strictly normalizes the outputs between 0 and 1. Then, the transformed outputs are interpolated by the radial basis function surrogate model using the Gaussian basis function. The second derivative of the PDM is calculated and the convexity of the PDM is satisfied for the certain range and the condition. Thus, the condition and the range of exploitation phase are derivable. In result, the PDM provides a local minimum by representing the convex shape. In order to compare the performances of the proposed method with the performances of the existing methods, mathematical examples with various features are employed. In two-dimensional optimization problems, 8 mathematical examples are classified according to the direction to minimization of the objective function and the shapes of the feasible region. Four multi-dimensional examples are also solved to analyze behaviors of the proposed method in high dimensional problems. As design applications, a piezoelectric energy harvester by ABAQUS and a combat vehicle by MapleSim are formulated. As a result, for problems with the unidirectional objective functions regardless of the shape of the feasible region, the proposed method outperforms the existing methods in the efficiency and the robustness of optimization because of the exploitation phase with the restricted local domain and the minimization of objective function in the exploration phase.