On Federated Multi-Armed Bandits for Mobile Social Networks

Multi-armed bandits (MABs) are widely used for decision making processes, in which an agent tries to maximize a long-term reward by balancing exploiting and exploring actions. In this paper, we are particularly interested in federated MABs for mobile social networks (MSNs), where a collection of agents learn action values by locally observing rewards and exchanging estimated action values with other agents. Our research differs from the existing federated MABs in the underlying network setting, such that agents can communicate with each other only at opportunistic contact events. To this end, we first design the weighted-connectivity (WC) centrality to quantify the importance of agents in an MSN, and then, we propose the weighted-connectivity upper confidence bound (WC-UCB) algorithm for the MSN contexts. The key idea to reduce biases at each agent and to utilize limited opportunities of federated updates is to prioritize the estimates of action-value functions computed by the agents with a high WC centrality. In addition, the performance bound in terms of the cumulative regret is analyzed. The performance of the proposed algorithm is evaluated by simulations using real mobility traces and the results demonstrate that our WC-UCB outperforms the state-of-the-art algorithms in terms of the average reward and the cumulative regret.