Boosting Fairness and Robustness in Over-the-Air Federated Learning: Abstract and Introduction

Table of Links

Abstract and Introduction

Problem Setup

Federated fair over-the-air learning (FedAir) Algorithm

Convergence Properties

Numerical Example

Conclusion and References

Abstract

Over-the-Air Computation is a beyond-5G communication strategy that has recently been shown to be useful for the decentralized training of machine learning models due to its efficiency. In this paper, we propose an Over-the-Air federated learning algorithm that aims to provide fairness and robustness through minmax optimization. By using the epigraph form of the problem at hand, we show that the proposed algorithm converges to the optimal solution of the minmax problem. Moreover, the proposed approach does not require reconstructing channel coefficients by complex encoding decoding schemes as opposed to state-of-the-art approaches. This improves both efficiency and privacy.

I. INTRODUCTION

For large-scale systems, where information exchange and cooperation are vital, one critical challenge for the averaging based federated learning algorithms is heterogeneity [8]– [10]. When the data is heterogeneous, i.e., the fact that agents observe data from different distributions, the parameter vectors minimizing the local cost functions will in general vary significantly between different agents, and minimizing the cost function (2) may not be desirable. Instead, using a worst-case optimization problem may reflect practical requirements more accurately. Another problem that we encounter in large-scale networks is that the communication load on the overall system increases with the number of agents [11]–[14]. When multiple agents transmit information at the same time and in the same frequency band, signals are affected by the physical phenomenon of interference. Standard communication protocols[1] prevent interference by transmitting signals orthogonally. However, these techniques are not resource-efficient in the sense that they increase the need for bandwidth or the number of communication rounds, which in general leads to a decrease in total throughput and efficiency [15], [16].

In this paper, we present a federated learning algorithm that aims to improve the performance of the worst performing agent in the system, thus providing fairness and robustness against heterogeneity. We leverage a beyond-5G communication strategy, called Over-the-Air Computation, which is more efficient as the number of agents grows [16]. Unlike the existing literature on Over-the-Air computation, e.g, [8], [17], the proposed algorithm can operate despite the inherently unknown nature of channel coefficients. We do not assume any knowledge of (nor the capability to reconstruct) channel coefficients, and therefore we will not need extra preprocessing efforts to reconstruct the channel, which makes the proposed scheme more time and resource efficient. Moreover, since the channel coefficients are completely unknown, privacy is inherently guaranteed, as discussed in [18].

The remainder of this paper is organized as follows: we present the problem setup in Section II. In Section III, we introduce the proposed algorithm, whose convergence analysis is presented in Section IV. A numerical example is presented in Section V. Concluding remarks are given in Section VI.

[1] In TDMA (Time Division Multiple Access), agents are assigned different time slots when they can transmit, whereas in FDMA (Frequency Division Multiple Access), different frequency bands are allocated to different users.