An Overview of the Current State-of-the-Art Federated Learning Methodologies

Table of Links

Highlights

Abstract & Introduction

Federated Learning: Collaborative and Privacy-Preserving Model Training

State of the Art Federated Learning Software Platforms and Testbeds

Sensors and Edge-Sensing Devices

Federated Learning Applications on Sensing Devices

Discussion

Conclusions, Acknowledgment, and References

7. Conclusion

This paper has provided an overview of the current state-of-the-art FL methodologies applied to sensing devices, focusing on IoT sensors, mobile devices, and wearables. By examining the challenges and opportunities in applying FL to sensing tasks, the paper aims to inspire further research in this domain.

The discussion section has raised several important points for consideration. It highlighted the need to address the asynchrony in FL, leverage unlabelled on-device data, explore FL applications in Human Activity Recognition (HAR) and affective computing, and adapt FL methodologies from simulation environments to real-world applications on sensing devices.

Efficient wireless communication, resource efficiency, and personalization are crucial when applying FL methodologies to sensing devices. Future research should focus on developing wireless communication techniques that minimize data exchange, designing algorithms with resource constraints in mind, and enabling privacy-preserving personalization. By addressing these challenges, FL methodologies can be effectively applied in real-world sensing applications, considering sensing devices’ unique constraints and requirements.

Moreover, the discussion emphasizes the need for studies that bridge the gap between simulation and real-world deployment, ensuring the practicality and effectiveness of FL methodologies in real-world sensing applications. By adapting FL methodologies to real-world settings, researchers can address additional challenges, such as communication constraints, privacy concerns, and device heterogeneity, and validate the performance of FL in diverse sensing domains.

In conclusion, this paper aims to provide a comprehensive review for researchers and practitioners interested in applying FL methodologies to edge sensing devices to contribute to developing intelligent and efficient systems in diverse application domains by highlighting the specific challenges and opportunities associated with these devices.

Acknowledgment

T¨ubitak Bideb 2211-A academic reward is gratefully acknowledged. This research has been supported by the Bo˘gazi¸ci University Research Fund, project number: 19301P.

References

S. Dey, A. Mukherjee, H. S. Paul, A. Pal, Challenges of using edge devices in iot computation grids, in: 2013 International Conference on Parallel and Distributed Systems, IEEE, 2013, pp. 564–569.

L. Atzori, A. Iera, G. Morabito, The internet of things: A survey, Computer networks 54 (2010) 2787–2805.

P. Voigt, A. Von dem Bussche, The eu general data protection regulation (gdpr), A Practical Guide, 1st Ed., Cham: Springer International Publishing 10 (2017) 10–5555.

M. Capra, R. Peloso, G. Masera, M. Ruo Roch, M. Martina, Edge computing: A survey on the hardware requirements in the internet of things world, Future Internet 11 (2019) 100.

Q. Yang, Y. Liu, T. Chen, Y. Tong, Federated machine learning: Concept and applications, ACM Transactions on Intelligent Systems and Technology (TIST) 10 (2019) 1–19.

W. Yu, F. Liang, X. He, W. G. Hatcher, C. Lu, J. Lin, X. Yang, A survey on the edge computing for the internet of things, IEEE access 6 (2017) 6900–6919.

D. C. Nguyen, M. Ding, P. N. Pathirana, A. Seneviratne, J. Li, H. V. Poor, Federated learning for internet of things: A comprehensive survey, IEEE Communications Surveys & Tutorials 23 (2021) 1622–1658.

M. Aledhari, R. Razzak, R. M. Parizi, F. Saeed, Federated learning: A survey on enabling technologies, protocols, and applications, IEEE Access 8 (2020) 140699–140725.

J. Xu, B. S. Glicksberg, C. Su, P. Walker, J. Bian, F. Wang, Federated learning for healthcare informatics, Journal of Healthcare Informatics Research 5 (2021) 1–19.

Q. Wu, K. He, X. Chen, Personalized federated learning for intelligent iot applications: A cloud-edge based framework, IEEE Open Journal of the Computer Society 1 (2020) 35–44.

E. Sannara, F. Portet, P. Lalanda, V. German, A federated learning aggregation algorithm for pervasive computing: Evaluation and comparison, in: 2021 IEEE International Conference on Pervasive Computing and Communications (PerCom), IEEE, 2021, pp. 1–10.

F. Wang, A. Preininger, Ai in health: state of the art, challenges, and future directions, Yearbook of medical informatics 28 (2019) 016–026.

N. Rieke, J. Hancox, W. Li, F. Milletari, H. R. Roth, S. Albarqouni, S. Bakas, M. N. Galtier, B. A. Landman, K. Maier-Hein, et al., The future of digital health with federated learning, NPJ digital medicine 3 (2020) 119.

M. Aledhari, R. Razzak, R. M. Parizi, F. Saeed, Federated learning: A survey on enabling technologies, protocols, and applications, IEEE Access 8 (2020) 140699–140725.

Q. Jing, W. Wang, J. Zhang, H. Tian, K. Chen, Quantifying the performance of federated transfer learning, arXiv preprint arXiv:1912.12795 (2019).

J. Yao, M. Mouhoub, A. Manashty, Y. Morgan, et al., Analysis of Model Aggregation Techniques in Federated Learning, Ph.D. thesis, Faculty of Graduate Studies and Research, University of Regina, 2021.

P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings, et al., Advances and open problems in federated learning, Foundations and Trends® in Machine Learning 14 (2021) 1–210.

A. Saeed, F. D. Salim, T. Ozcelebi, J. Lukkien, Federated self-supervised learning of multisensor representations for embedded intelligence, IEEE Internet of Things Journal 8 (2020) 1030–1040.

T. Li, A. K. Sahu, A. Talwalkar, V. Smith, Federated learning: Challenges, methods, and future directions, IEEE signal processing magazine 37 (2020) 50–60.

J. Koneˇcn`y, H. B. McMahan, F. X. Yu, P. Richt´arik, A. T. Suresh, D. Bacon, Federated learning: Strategies for improving communication efficiency, arXiv preprint arXiv:1610.05492 (2016).

K. Wang, R. Mathews, C. Kiddon, H. Eichner, F. Beaufays, D. Ramage, Federated evaluation of on-device personalization, arXiv preprint arXiv:1910.10252 (2019).

K. Bonawitz, H. Eichner, W. Grieskamp, D. Huba, A. Ingerman, V. Ivanov, C. Kiddon, J. Koneˇcn`y, S. Mazzocchi, B. McMahan, et al., Towards federated learning at scale: System design, Proceedings of machine learning and systems 1 (2019) 374–388.

K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, K. Seth, Practical secure aggregation for privacypreserving machine learning, in: proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017, pp. 1175– 1191.

Z. Zhou, S. Yang, L. Pu, S. Yu, Cefl: Online admission control, data scheduling, and accuracy tuning for cost-efficient federated learning across edge nodes, IEEE Internet of Things Journal 7 (2020) 9341–9356.

Y. Jiang, R. Cong, C. Shu, A. Yang, Z. Zhao, G. Min, Federated learning based mobile crowd sensing with unreliable user data, in: 2020 IEEE 22nd International Conference on High Performance Computing and Communications; IEEE 18th International Conference on Smart City; IEEE 6th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), IEEE, 2020, pp. 320–327.

B. McMahan, E. Moore, D. Ramage, S. Hampson, B. A. y Arcas, Communication-efficient learning of deep networks from decentralized data, in: Artificial intelligence and statistics, PMLR, 2017, pp. 1273–1282.

N. Shlezinger, M. Chen, Y. C. Eldar, H. V. Poor, S. Cui, Uveqfed: Universal vector quantization for federated learning, IEEE Transactions on Signal Processing 69 (2020) 500–514.

S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, A. T. Suresh, Scaffold: Stochastic controlled averaging for federated learning, in: International conference on machine learning, PMLR, 2020, pp. 5132–5143.

A. Reisizadeh, A. Mokhtari, H. Hassani, A. Jadbabaie, R. Pedarsani, Fedpaq: A communication-efficient federated learning method with periodic averaging and quantization, in: International Conference on Artificial Intelligence and Statistics, PMLR, 2020, pp. 2021–2031.

H. Wang, M. Yurochkin, Y. Sun, D. Papailiopoulos, Y. Khazaeni, Federated learning with matched averaging, arXiv preprint arXiv:2002.06440 (2020).

M. Zaharia, A. Konwinski, A. D. Joseph, R. H. Katz, I. Stoica, Improving mapreduce performance in heterogeneous environments., in: Osdi, volume 8, 2008, p. 7.

C. Xie, S. Koyejo, I. Gupta, Asynchronous federated optimization, arXiv preprint arXiv:1903.03934 (2019).

W. Wu, L. He, W. Lin, R. Mao, C. Maple, S. Jarvis, Safa: A semiasynchronous protocol for fast federated learning with low overhead, IEEE Transactions on Computers 70 (2020) 655–668.

M. Duan, D. Liu, X. Chen, Y. Tan, J. Ren, L. Qiao, L. Liang, Astraea: Selfbalancing federated learning for improving classification accuracy of mobile deep learning applications, in: 2019 IEEE 37th international conference on computer design (ICCD), IEEE, 2019, pp. 246–254.

J. Liu, J. H. Wang, C. Rong, Y. Xu, T. Yu, J. Wang, Fedpa: An adaptively partial model aggregation strategy in federated learning, Computer Networks 199 (2021) 108468.

Y. Gao, L. Liu, X. Zheng, C. Zhang, H. Ma, Federated sensing: Edgecloud elastic collaborative learning for intelligent sensing, IEEE Internet of Things Journal 8 (2021) 11100–11111.

I. Erg¨un, H. U. Sami, B. G¨uler, Communication-efficient secure aggregation for federated learning, in: GLOBECOM 2022-2022 IEEE Global Communications Conference, IEEE, 2022, pp. 3881–3886.

C. He, S. Li, J. So, X. Zeng, M. Zhang, H. Wang, X. Wang, P. Vepakomma, A. Singh, H. Qiu, et al., Fedml: A research library and benchmark for federated machine learning, arXiv preprint arXiv:2007.13518 (2020).

J. Xu, S. Wang, L. Wang, A. C.-C. Yao, Fedcm: Federated learning with client-level momentum, arXiv preprint arXiv:2106.10874 (2021).

T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, V. Smith, Federated optimization in heterogeneous networks, Proceedings of Machine learning and systems 2 (2020) 429–450.

T. Semwal, A. Mulay, A. M. Agrawal, Fedperf: A practitioners’ guide to performance of federated learning algorithms (2020).

I. Kholod, E. Yanaki, D. Fomichev, E. Shalugin, E. Novikova, E. Filippov, M. Nordlund, Open-source federated learning frameworks for iot: A comparative review and analysis, Sensors 21 (2020) 167.

TFF, Tensorflow federated: Machine learning on decentralized data, Accessed: 21 September 2023. https://www.tensorflow.org/federated.

S. Caldas, S. M. K. Duddu, P. Wu, T. Li, J. Koneˇcn`y, H. B. McMahan, V. Smith, A. Talwalkar, Leaf: A benchmark for federated settings, arXiv preprint arXiv:1812.01097 (2018).

O. Community, Open-source python 3 based library developed by the openmined community, Accessed: 21 September 2023. https://blog. openmined.org/tag/pysyft/.

D. J. Beutel, T. Topal, A. Mathur, X. Qiu, J. Fernandez-Marques, Y. Gao, L. Sani, K. H. Li, T. Parcollet, P. P. B. de Gusm˜ao, et al., Flower: A friendly federated learning research framework, arXiv preprint arXiv:2007.14390 (2020).

Fedai, An industrial grade federated learning framework, Accessed: 21 September 2023. https://fate.fedai.org/.

Intel, Open-source python 3 based library for federated learning developed by the intel, Accessed: 21 September 2023. https://github.com/intel/ openfl.

M. Popovic, M. Popovic, I. Kastelan, M. Djukic, S. Ghilezan, A simple python testbed for federated learning algorithms, arXiv preprint arXiv:2305.20027 (2023).

H. Cho, A. Mathur, F. Kawsar, Flame: Federated learning across multidevice environments, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6 (2022) 1–29.

R. Jaren, E. Scott, L. Chapin, The internet of things: an overviewunderstanding the issues and challenges of a more connected world, Internet Soc (2015).

R. Want, B. N. Schilit, S. Jenson, Enabling the internet of things, Computer 48 (2015) 28–35.

H. U. Rehman, M. Asif, M. Ahmad, Future applications and research challenges of iot, in: 2017 International conference on information and communication technologies (ICICT), IEEE, 2017, pp. 68–74.

J. McCann, L. Quinn, S. McGrath, E. O’Connell, Towards the distributed edge–an iot review, in: 2018 12th International Conference on Sensing Technology (ICST), IEEE, 2018, pp. 263–268.

J. Feng, C. Rong, F. Sun, D. Guo, Y. Li, Pmf: A privacy-preserving human mobility prediction framework via federated learning, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4 (2020) 1–21.

K. Sozinov, V. Vlassov, S. Girdzijauskas, Human activity recognition using federated learning, in: 2018 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Ubiquitous Computing & Communications, Big Data & Cloud Computing, Social Computing & Networking, Sustainable Computing & Communications (ISPA/IUCC/BDCloud/SocialCom/SustainCom), IEEE, 2018, pp. 1103– 1111.

S. Ek, F. Portet, P. Lalanda, G. Vega, Evaluation of federated learning aggregation algorithms: application to human activity recognition, in: Adjunct proceedings of the 2020 ACM international joint conference on pervasive and ubiquitous computing and proceedings of the 2020 ACM international symposium on wearable computers, 2020, pp. 638–643.

C. Srinivasan, B. Rajesh, P. Saikalyan, K. Premsagar, E. S. Yadav, A review on the different types of internet of things (iot), Journal of Advanced Research in Dynamical and Control Systems 11 (2019) 154–158.

D. C. Nguyen, M. Ding, P. N. Pathirana, A. Seneviratne, J. Li, H. V. Poor, Federated learning for internet of things: A comprehensive survey, IEEE Communications Surveys & Tutorials 23 (2021) 1622–1658.

G. Szegedi, P. Kiss, T. Horv´ath, Evolutionary federated learning on eegdata., in: ITAT, 2019, pp. 71–78.

T. Wang, Y. Du, Y. Gong, K.-K. R. Choo, Y. Guo, Applications of federated learning in mobile health: Scoping review, Journal of Medical Internet Research 25 (2023) e43006.

G. O. Ogbuabor, J. C. Augusto, R. Moseley, A. van Wyk, Context-aware support for cardiac health monitoring using federated machine learning, in: Artificial Intelligence XXXVIII: 41st SGAI International Conference on Artificial Intelligence, AI 2021, Cambridge, UK, December 14–16, 2021, Proceedings 41, Springer, 2021, pp. 267–281.

R. W. Picard, Affective computing, MIT press, 2000

Y. S. Can, C. Ersoy, Privacy-preserving federated deep learning for wearable iot-based biomedical monitoring, ACM Transactions on Internet Technology (TOIT) 21 (2021) 1–17.

D.-V. Nguyen, K. Zettsu, Spatially-distributed federated learning of convolutional recurrent neural networks for air pollution prediction, in: 2021 IEEE International Conference on Big Data (Big Data), IEEE, 2021, pp. 3601–3608.

S. Chen, G. Long, T. Shen, J. Jiang, Prompt federated learning for weather forecasting: Toward foundation models on meteorological data, arXiv preprint arXiv:2301.09152 (2023).

J. So, B. G¨uler, A. S. Avestimehr, Turbo-aggregate: Breaking the quadratic aggregation barrier in secure federated learning, IEEE Journal on Selected Areas in Information Theory 2 (2021) 479–489.

J. C. Jiang, B. Kantarci, S. Oktug, T. Soyata, Federated learning in smart city sensing: Challenges and opportunities, Sensors 20 (2020) 6230.

Y. Liu, J. James, J. Kang, D. Niyato, S. Zhang, Privacy-preserving traffic flow prediction: A federated learning approach, IEEE Internet of Things Journal 7 (2020) 7751–7763.

D. C. Nguyen, M. Ding, P. N. Pathirana, A. Seneviratne, J. Li, H. V. Poor, Federated learning for internet of things: A comprehensive survey, IEEE Communications Surveys & Tutorials 23 (2021) 1622–1658.

A. M. Elbir, B. Soner, S. C¸ ¨oleri, D. G¨und¨uz, M. Bennis, Federated learning in vehicular networks, in: 2022 IEEE International Mediterranean Conference on Communications and Networking (MeditCom), IEEE, 2022, pp. 72–77.

X. Liang, Y. Liu, T. Chen, M. Liu, Q. Yang, Federated transfer reinforcement learning for autonomous driving, in: Federated and Transfer Learning, Springer, 2022, pp. 357–371.

Z. Ge, Z. Song, S. X. Ding, B. Huang, Data mining and analytics in the process industry: The role of machine learning, Ieee Access 5 (2017) 20590– 20616.

V. Pruckovskaja, A. Weissenfeld, C. Heistracher, A. Graser, J. Kafka, P. Leputsch, D. Schall, J. Kemnitz, Federated learning for predictive maintenance and quality inspection in industrial applications, arXiv preprint arXiv:2304.11101 (2023).

X. Han, H. Yu, H. Gu, Visual inspection with federated learning, in: Image Analysis and Recognition: 16th International Conference, ICIAR 2019, Waterloo, ON, Canada, August 27–29, 2019, Proceedings, Part II 16, Springer, 2019, pp. 52–64.

F. Liu, X. Wu, S. Ge, W. Fan, Y. Zou, Federated learning for vision-andlanguage grounding problems, in: Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, 2020, pp. 11572–11579.

S. Zhai, X. Jin, L. Wei, H. Luo, M. Cao, Dynamic federated learning for gmec with time-varying wireless link, IEEE Access 9 (2021) 10400–10412.

N. I. Mowla, N. H. Tran, I. Doh, K. Chae, Federated learning-based cognitive detection of jamming attack in flying ad-hoc network, IEEE Access 8 (2019) 4338–4350.

W. Zhou, Y. Li, S. Chen, B. Ding, Real-time data processing architecture for multi-robots based on differential federated learning, in: 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), IEEE, 2018, pp. 462–471.

B. Liu, L. Wang, M. Liu, C.-Z. Xu, Federated imitation learning: A novel framework for cloud robotic systems with heterogeneous sensor data, IEEE Robotics and Automation Letters 5 (2020) 3509–3516.

B. Liu, L. Wang, M. Liu, Lifelong federated reinforcement learning: a learning architecture for navigation in cloud robotic systems, IEEE Robotics and Automation Letters 4 (2019) 4555–4562.

M. Fu, Y. Shi, Y. Zhou, Federated learning via unmanned aerial vehicle, IEEE Transactions on Wireless Communications (2023).

W. Li, W. Yang, G. Jin, J. Chen, J. Li, R. Huang, Z. Chen, Clustering federated learning for bearing fault diagnosis in aerospace applications with a self-attention mechanism, Aerospace 9 (2022) 516.

T. Zeng, O. Semiari, M. Mozaffari, M. Chen, W. Saad, M. Bennis, Federated learning in the sky: Joint power allocation and scheduling with uav swarms, in: ICC 2020-2020 IEEE International Conference on Communications (ICC), IEEE, 2020, pp. 1–6.

J. Ebenezer, P. G. Krishna, M. Poojitha, A. V. Krishna, Plant leaf disease detection and classification with cnn and federated learning approach, in: Conference of Innovative Product Design and Intelligent Manufacturing System, Springer, 2022, pp. 513–523.

A. Abu-Khadrah, A. M. Ali, M. Jarrah, An amendable multi-function control method using federated learning for smart sensors in agricultural production improvements, ACM Transactions on Sensor Networks (2023).

This paper is available on Arxiv under a CC 4.0 license.