Multi-Agent Deep Reinforcement Learning for Spectrum Management in V2X with Social Roles

In a vehicle-to-everything (V2X) communication system involving multiple vehicle types, there is a more challenging and practical problem compared to a single-type scenario. Each vehicle type acts autonomously with distinct communication policies. While prior knowledge can establish behavior for each agent type, it may reduce the adaptability and versatility of the system. This paper proposes a role-oriented actor-critic (ROAC) approach, where vehicles of similar types share similar policies in a satellite-assisted V2X network for more precise and effective spectrum management. The vehicles are trained to optimize system utility by selecting transmission modes, power levels, and sub-channels. The social role properties enable each agent to make better decisions based on the environment and its type. The ROAC model provides 8-10% higher normalized system utility over other advanced methods, even with vehicle-role extension, in situations with heavier traffic.