Boosting Fairness and Robustness in Over-the-Air Federated Learning: Conclusion and References

by Computational Technology for AllOctober 27th, 2024

Too Long; Didn't Read

This paper presents a federated learning algorithm using Over-the-Air computation for fairness and robustness, optimizing performance in decentralized networks.

featured image - Boosting Fairness and Robustness in Over-the-Air Federated Learning: Conclusion and References

Authors:

(1) Halil Yigit Oksuz, Control Systems Group at Technische Universitat Berlin, Germany and Exzellenzcluster Science of Intelligence, Technische Universitat Berlin, Marchstr. 23, 10587, Berlin, Germany;

(2) Fabio Molinari, Control Systems Group at Technische Universitat Berlin, Germany;

(3) Henning Sprekeler, Exzellenzcluster Science of Intelligence, Technische Universit¨at Berlin, Marchstr. 23, 10587, Berlin, Germany and Modelling Cognitive Processes Group at Technische Universit¨at Berlin, Germany;

(4) Jorg Raisch, Control Systems Group at Technische Universitat Berlin, Germany and Exzellenzcluster Science of Intelligence, Technische Universitat Berlin, Marchstr. 23, 10587, Berlin, Germany.

Table of Links

Abstract and Introduction

Problem Setup

Federated fair over-the-air learning (FedAir) Algorithm

Convergence Properties

Numerical Example

Conclusion and References

VI. CONCLUSION

In this paper, we have introduced the FedFAir algorithm which uses Over-the-Air Computation to carry out efficient decentralized learning while providing fairness and improved performance. We have shown that the FedFAir algorithm converges to an optimal solution of the minimax problem. Furthermore, we have also illustrated our theoretical findings with a numerical example.

Future research will include the development of resilient federated learning algorithms when there are malicious agents in the system.

REFERENCES

[1] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics. PMLR, 2017, pp. 1273–1282.

[2] V. Smith, C.-K. Chiang, M. Sanjabi, and A. S. Talwalkar, “Federated multi-task learning,” Advances in neural information processing systems, vol. 30, 2017.

[3] K. Bonawitz, H. Eichner, W. Grieskamp, D. Huba, A. Ingerman, V. Ivanov, C. Kiddon, J. Konecny, S. Mazzocchi, B. McMahan et al., “Towards federated learning at scale: System design,” Proceedings of Machine Learning and Systems, vol. 1, pp. 374–388, 2019.

[4] Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated machine learning: Concept and applications,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 10, no. 2, pp. 1–19, 2019.

[5] A. Nedic, “Distributed gradient methods for convex machine learning problems in networks: Distributed optimization,” IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 92–101, 2020.

[6] V. P. Chellapandi, A. Upadhyay, A. Hashemi, and S. H. Zak, “On ˙ the convergence of decentralized federated learning under imperfect information sharing,” IEEE Control Systems Letters, 2023.

[7] T. Omori and K. Kashima, “Combinatorial optimization approach to client scheduling for federated learning,” IEEE Control Systems Letters, 2023.

[8] T. Sery, N. Shlezinger, K. Cohen, and Y. C. Eldar, “Over-the-air federated learning from heterogeneous data,” IEEE Transactions on Signal Processing, vol. 69, pp. 3796–3811, 2021.

[9] T. Gafni, N. Shlezinger, K. Cohen, Y. C. Eldar, and H. V. Poor, “Federated learning: A signal processing perspective,” IEEE Signal Processing Magazine, vol. 39, no. 3, pp. 14–41, 2022.

[10] M. Ye, X. Fang, B. Du, P. C. Yuen, and D. Tao, “Heterogeneous federated learning: State-of-the-art and research challenges,” ACM Computing Surveys, vol. 56, no. 3, pp. 1–44, 2023.

[11] J. Koneˇcn `y, H. B. McMahan, F. X. Yu, P. Richt´arik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency,” arXiv preprint arXiv:1610.05492, 2016.

[12] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: Challenges, methods, and future directions,” IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 50–60, 2020.

[13] P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings et al., “Advances and open problems in federated learning,” Foundations and Trends® in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021.

[14] H. Y. Oksuz, F. Molinari, H. Sprekeler, and J. Raisch, “Federated learning in wireless networks via over-the-air computations,” in 2023 62nd IEEE Conference on Decision and Control (CDC), 2023, pp. 4379–4386.

[15] F. Molinari, N. Agrawal, S. Sta´nczak, and J. Raisch, “Max-consensus over fading wireless channels,” IEEE Transactions on Control of Network Systems, vol. 8, no. 2, pp. 791–802, 2021.

[16] M. Frey, I. Bjelakovi´c, and S. Sta´nczak, “Over-the-air computation in correlated channels,” IEEE Transactions on Signal Processing, vol. 69, pp. 5739–5755, 2021.

[17] K. Yang, T. Jiang, Y. Shi, and Z. Ding, “Federated learning via overthe-air computation,” IEEE Transactions on Wireless Communications, vol. 19, no. 3, pp. 2022–2035, 2020.

[18] F. Molinari and J. Raisch, “Exploiting wireless interference for distributively solving linear equations,” IFAC-PapersOnLine, vol. 53, no. 2, pp. 2999–3006, 2020.

[19] D. Bertsekas, A. Nedic, and A. Ozdaglar, Convex analysis and optimization. Athena Scientific, 2003, vol. 1.

[20] M. Mohri, G. Sivek, and A. T. Suresh, “Agnostic federated learning,” in International Conference on Machine Learning. PMLR, 2019, pp. 4615–4625.

[21] Z. Hu, K. Shaloudegi, G. Zhang, and Y. Yu, “Federated learning meets multi-objective optimization,” IEEE Transactions on Network Science and Engineering, vol. 9, no. 4, pp. 2039–2051, 2022.

[22] D. P. Bertsekas, “Necessary and sufficient conditions for a penalty method to be exact,” Mathematical programming, vol. 9, no. 1, pp. 87–99, 1975.

[23] K. Srivastava, A. Nedi´c, and D. Stipanovi´c, “Distributed min-max optimization in networks,” in 2011 17th International Conference on Digital Signal Processing (DSP). IEEE, 2011, pp. 1–8.

[24] R. Ahlswede, “Multi-way communication channels,” in Second International Symposium on Information Theory: Tsahkadsor, Armenia, USSR, Sept. 2-8, 1971, 1973.

[25] A. Giridhar and P. Kumar, “Toward a theory of in-network computation in wireless sensor networks,” IEEE Communications Magazine, vol. 44, no. 4, pp. 98–107, apr 2006.

[26] F. Molinari, N. Agrawal, S. Sta´nczak, and J. Raisch, “Over-the-air max-consensus in clustered networks adopting half-duplex communication technology,” IEEE Transactions on Control of Network Systems, 2022.

[27] W. Rudin et al., Principles of mathematical analysis. McGraw-hill New York, 1976, vol. 3.

[28] D. Tse and P. Viswanath, Fundamentals of Wireless Communication. Cambridge University Press, 2012.

[29] A. F. Molisch, Wireless communications. John Wiley & Sons, 2012.

[30] B. Sklar, “Rayleigh fading channels in mobile digital communication systems. i. characterization,” IEEE Communications magazine, vol. 35, no. 7, pp. 90–100, 1997.

[31] R. G. Bartle and D. R. Sherbert, Introduction to real analysis. Wiley New York, 2000.

[32] B. T. Polyak, Introduction to optimization. optimization software. Inc., Publications Division, New York, 1987.

[33] Y. I. Alber, A. N. Iusem, and M. V. Solodov, “On the projected subgradient method for nonsmooth convex optimization in a hilbert space,” Mathematical Programming, vol. 81, pp. 23–35, 1998.