Online Learning for Joint Beam Tracking and Pattern Optimization in Massive MIMO Systems

Abstract

In this paper, we consider a joint beam tracking and pattern optimization problem for massive multiple input multiple output (MIMO) systems in which the base station (BS) selects a beamforming codebook and performs adaptive beam tracking taking into account the user mobility. A joint adaptation scheme is developed in a two-phase reinforcement learning framework which utilizes practical signaling and feedback information. In particular, an inner agent adjusts the transmission beam index for a given beamforming codebook based on short-term instantaneous signal-to-noise ratio (SNR) rewards. In addition, an outer agent selects the beamforming codebook based on long-term SNR rewards. Simulation results demonstrate that the proposed online learning outperforms conventional codebook-based beamforming schemes using the same number of feedback information. It is further shown that joint beam tracking and beam pattern adaptation provides a significant SNR gain compared to the beam tracking only schemes, especially as the user mobility increases.