Comparative Federated Algorithms for Solving Non- IID Data Challenges
Abstract
This study evaluates three Federated Learning (FL) algorithms—FedAvg, Federated Proximal (FedProx), and MOON—by assessing their performance in Independent and Identically Distributed (IID) and non-IID settings. We found that FedAvg performs best in IID scenarios, offering quick convergence and high accuracy. However, MOON stood out as the top performer in non-IID settings, thanks to its contrastive learning method, providing better stability and accuracy across heterogeneous data. FedProx improved over FedAvg in handling non-IID data but was less effective than MOON. Our findings suggest that for environments with IID data, FedAvg is ideal, while MOON is more suitable for non-IID cases. We also highlight the need for further research into personalized FL, regularization techniques, and multimodal data integration.
Keywords:
Federated learning, FedAvg, FedProx, MOON, Data heterogeneityReferences
- [1] Abdulrahman, S., Tout, H., Ould-Slimane, H., Mourad, A., Talhi, C., & Guizani, M. (2021). A survey on federated learning: The journey from centralized to distributed on-site learning and beyond. IEEE internet of things journal, 8(7), 5476–5497. https://doi.org/10.1109/JIOT.2020.3030072
- [2] Huang, C., Huang, J., & Liu, X. (2022). Cross-silo federated learning: Challenges and opportunities. https://doi.org/10.48550/arXiv.2206.12949
- [3] Li, Q., Diao, Y., Chen, Q., & He, B. (2022). Federated learning on non-iid data silos: An experimental study. 2022 IEEE 38th international conference on data engineering (Icde) (pp. 965–978). IEEE. https://doi.org/10.1109/ICDE53745.2022.00077
- [4] Dashan Gao, Xin Yao, Q. Y. (2022). A survey on heterogeneous federated learning. https://doi.org/10.48550/arXiv.2210.04505
- [5] Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2020). Federated optimization in heterogeneous networks. Proceedings of machine learning and systems (Vol. 2, pp. 429–450). MLSys. https://B2n.ir/qp5832
- [6] Che, L., Wang, J., Zhou, Y., & Ma, F. (2023). Multimodal federated learning: A survey. Sensors, 23(15), 6986. https://doi.org/10.3390/s23156986
- [7] McMahan, B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. y. (2017). Communication-efficient learning of deep networks from decentralized data. Proceedings of the 20th international conference on artificial intelligence and statistics (Vol. 54, pp. 1273–1282). PMLR. https://proceedings.mlr.press/v54/mcmahan17a.html
- [8] Hartmann, M., Danoy, G., & Bouvry, P. (2024). FedPref: Federated learning across heterogeneous multi-objective preferences. ACM transactions on modeling and performance evaluation of computing systems. Association for Computing Machinery. https://doi.org/10.1145/3708984
- [9] McMahan, H., Moore, E., & Ramage, D. (2016). Federated learning of deep networks using model averaging. https://b2n.ir/md1975
- [10] Sun, L., & Wu, J. (2023). A scalable and transferable federated learning system for classifying healthcare sensor data. IEEE journal of biomedical and health informatics, 27(2), 866–877. https://doi.org/10.1109/JBHI.2022.3171402
- [11] Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., & Chandra, V. (2018). Federated learning with non-IID data. http://arxiv.org/abs/1806.00582
Issue 1