Artificial Intelligence-based Demand Response Management for Industrial Facilities: A Deep Reinforcement Learning Approach

Abstract

With developments in information and communication technology, as well as advanced metering infrastructure (AMI), smart grids (SGs) using digital bidirectional energy transmission provides considerably more flexible and stable services than conventional grid. This situation has led to significant interest in improving energy efficiency, with one of the most popular approaches being demand response (DR). By offering users a certain amount of financial incentive or dynamic electricity price, the function of DR is to adjust the user's electricity consumption pattern, shift the high price demand to the low price period, reduce the user's electricity cost, and ensure the stability and safety of the power grid. As a major consumer of energy, the industrial sector must assume the responsibility for improving energy efficiency and reducing carbon emissions. However, compared to the large degree of DR participation in the residential and commercial sectors, the industrial potential of DR is not well understood. The main barriers can be attributed to two aspects. The first problem is industrial diversity. Unlike the residential and commercial sectors, the energy consumption of production lines using diverse items of equipment varies considerably. Successful DR implementation requires a high-resolution model capturing the physical characteristics of all equipment in the system. However, it is not easy to model an entire industrial facility. Specifically, apart from electricity management, raw, intermediate, and auxiliary resources for production must all be considered. The second problem is the need to guarantee daily production; potentially, DR could result in production losses or cost increases because production must be shifted. Thus, industrial customers are wary of DR programs. To address the above issues, an artificial intelligence-based model-free DR scheme for industrial facilities was developed in this study. Rooted in behavioral psychology, reinforcement learning (RL) is a distinctive member of the machine learning (ML) family. No need the detailed mathematical model of the target system, RL solves the optimal decision-making issues by learning new experiences through trial-and-error. This kind of model-free characteristic brings great benefits in applying DR to industrial facilities. In practical terms, we first formulated the industrial DR problem with the goal of reducing energy costs while ensuring the completion of production tasks. We present the composition of the state, action, and reward for RL in detail. Then, we designed an actor-critic-based deep reinforcement learning (DRL) algorithm to determine the optimal energy management policy. We then confirmed the validity of our scheme by applying it to a real-world industry. Our algorithm identified an optimal energy consumption schedule, reducing energy costs without compromising production.