Allocating Network Services in Space Air Ground Integrated Networks with Federated Deep Reinforcement Learning

2024

Space-Air-Ground Integrated Networks (SAGINs) are proving to be an interesting, and

increasingly feasible with the rise of Internet providers like Starlink. This, in addition to

increasing awareness of virtualization technologies like Network Function Virtualization

(NFV), serve as promising avenues that may be explored for new types of communication

networks like 6G. Though these approaches are not without problems, as the SAGIN

suffers from an inherent heterogeneous design that is difficult to work with, and as NFV

must tackle both the chaining- and placement problem.

There are, currently, no widely accepted practical solutions to the combined chaining-

and placement problem. That is why it is of great interest to try and engineer solutions

to it. A prominent approach in this field has been to train Deep Reinforcement Learning

(DRL) agents to allocate the Network Service (NS) in the communication network, in

this case a SAGIN. Another approach of interest here is the Federated Learning (FL)

approach, which may serve to counteract some of the heterogeneity of the SAGIN. Thus:

The main aim of this thesis has been to train a Federated DRL (FDRL) agent to be able

to allocate NSs in SAGIN.

The thesis shows that the FDRL agent is able to compete with both a regular DRL agent,

and the known heuristic algorithm Global Resource Capacity (GRC). It outperforms the

GRC agent in all of the widely recognized metrics: Acceptance Rate (ACR), Average

Revenue to Cost ratio (ARC), and Average Revenue (AR). Though the FDRL agent

must capitulate to the regular DRL agent.

This thesis functions as an exploration into the feasibility and applicability of the FDRL

approach in this SAGIN scenario. The results indicate that it is certainly possible to

achieve FDRL without too many complexities, though there are limitations involved.

UIS