Continual Learning to Generalize Forwarding Strategies for Diverse Mobile Wireless Networks

Abstract

Deep reinforcement learning (DRL) has been successfully used to design forwarding strategies for multi-hop mobile wireless networks. While such strategies can be used directly for networks with varied connectivity and dynamic conditions, developing generalizable approaches that are effective on scenarios significantly different from the training environment remains largely unexplored. In this paper, we propose a framework to address the challenge of generalizability by (i) developing a generalizable base model considering diverse mobile network scenarios, and (ii) using the generalizable base model for new scenarios, and when needed, fine-tuning the base model using a small amount of data from the new scenarios. To support this framework, we first design new features to characterize network variation and feature quality, thereby improving the information used in DRL-based forwarding decisions. We then develop a continual learning (CL) approach able to train DRL models across diverse network scenarios without ``catastrophic forgetting.'' Using extensive evaluation, including real-world scenarios in two cities, we show that our approach is generalizable to unseen mobility scenarios. Compared to a state-of-the-art heuristic forwarding strategy, it leads to up to 78% reduction in delay, 24% improvement in delivery rate, and comparable or slightly higher number of forwards.