47 0

Learning to Schedule Network Resources Throughput and Delay Optimally Using Q+-Learning

Title
Learning to Schedule Network Resources Throughput and Delay Optimally Using Q+-Learning
Author
이주현
Issue Date
2021-01
Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Citation
IEEE-ACM TRANSACTIONS ON NETWORKING, v. 29, Issue. 2, Page. 750-763
Abstract
As network architecture becomes complex and the user requirement gets diverse, the role of efficient network resource management becomes more important. However, existing throughput-optimal scheduling algorithms such as the max-weight algorithm suffer from poor delay performance. In this paper, we present reinforcement learning-based network scheduling algorithms for a single-hop downlink scenario which achieve throughput-optimality and converge to minimal delay. To this end, we first formulate the network optimization problem as a Markov decision process (MDP) problem. Then, we introduce a new state-action value function called Q + -function and develop a reinforcement learning algorithm called Q + -learning with UCB (Upper Confidence Bound) exploration which guarantees small performance loss during a learning process. We also derive an upper bound of the sample complexity in our algorithm, which is more efficient than the best known bound from Q-learning with UCB exploration by a factor of γ 2 where γ is the discount factor of the MDP problem. Finally, via simulation, we verify that our algorithm shows a delay reduction of up to 40.8% compared to the max-weight algorithm over various scenarios. We also show that the Q + -learning with UCB exploration converges to an ε-optimal policy 10 times faster than Q-learning with UCB.
URI
https://ieeexplore.ieee.org/abstract/document/9336288https://repository.hanyang.ac.kr/handle/20.500.11754/167026
ISSN
1063-6692; 1558-2566
DOI
10.1109/TNET.2021.3051663
Appears in Collections:
COLLEGE OF ENGINEERING SCIENCES[E](공학대학) > ELECTRICAL ENGINEERING(전자공학부) > Articles
Files in This Item:
There are no files associated with this item.
Export
RIS (EndNote)
XLS (Excel)
XML


qrcode

Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.

BROWSE