Deep learning is definitely notable in various classification field. DNN(Deep Neural Network) is a kind of algorithm for deep learning. It becomes important to choose a suitable activation function to make learn the various DNN structure. In this paper, we summarize the history of activation function playing a key role in DNN and propose an improvement direction for state of the art activation function.