Effective Multi-Scale Recurrent Neural Networks Model Based on Multiresolution Analysis Method

Abstract

Recurrent neural networks (RNNs) are a deep learning model widely used for various time series analysis tasks such as prediction, classification, and anomaly detection by reflecting past information in time series data. However, RNN models have a limited ability to capture both short-term and long-term patterns that are evident at multiple scales in the input data. For this purpose, studies such as multi-layer RNNs that combine multiple layers with skip-connection from several layers and nodes have been proposed in the literature. However, such RNN models suffer from a poor ability to capture patterns at various scales mainly due to their mere increased complexity and inability to reflect multi resolution. In this paper, we propose a novel multi-scale RNN model that effectively bring in multi-scale data patterns in network layers based on multi-scale analysis. Reminiscent of wavelet analysis, the model doubles the scale of the input data in a layer, recursively combining the scaled data with network outputs. The configuration is to simultaneously capture both the short-term and the long-term patterns of the input data. To evaluate the superiority of our model in comparison with other RNN-based models, we conduct extensive experiments on simulation data and various real-life data.