To see the other types of publications on this topic, follow the link: Non-identically distributed data.
Journal articles on the topic 'Non-identically distributed data'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Non-identically distributed data.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
A AlSaiary, Zakeia. "Analyzing Order Statistics of Non-Identically Distributed Shifted Exponential Variables in Numerical Data." International Journal of Science and Research (IJSR) 13, no. 11 (November 5, 2024): 1264–70. http://dx.doi.org/10.21275/sr241116231011.
Full text
2
Tiurev, Konstantin, Peter-Jan H. S. Derks, Joschka Roffe, Jens Eisert, and Jan-Michael Reiner. "Correcting non-independent and non-identically distributed errors with surface codes." Quantum 7 (September 26, 2023): 1123. http://dx.doi.org/10.22331/q-2023-09-26-1123.
Full textAbstract:
A common approach to studying the performance of quantum error correcting codes is to assume independent and identically distributed single-qubit errors. However, the available experimental data shows that realistic errors in modern multi-qubit devices are typically neither independent nor identical across qubits. In this work, we develop and investigate the properties of topological surface codes adapted to a known noise structure by Clifford conjugations. We show that the surface code locally tailored to non-uniform single-qubit noise in conjunction with a scalable matching decoder yields an increase in error thresholds and exponential suppression of sub-threshold failure rates when compared to the standard surface code. Furthermore, we study the behaviour of the tailored surface code under local two-qubit noise and show the role that code degeneracy plays in correcting such noise. The proposed methods do not require additional overhead in terms of the number of qubits or gates and use a standard matching decoder, hence come at no extra cost compared to the standard surface-code error correction.
3
Zhu, Feng, Jiangshan Hao, Zhong Chen, Yanchao Zhao, Bing Chen, and Xiaoyang Tan. "STAFL: Staleness-Tolerant Asynchronous Federated Learning on Non-iid Dataset." Electronics 11, no. 3 (January 20, 2022): 314. http://dx.doi.org/10.3390/electronics11030314.
Full textAbstract:
With the development of the Internet of Things, edge computing applications are paying more and more attention to privacy and real-time. Federated learning, a promising machine learning method that can protect user privacy, has begun to be widely studied. However, traditional synchronous federated learning methods are easily affected by stragglers, and non-independent and identically distributed data sets will also reduce the convergence speed. In this paper, we propose an asynchronous federated learning method, STAFL, where users can upload their updates at any time and the server will immediately aggregate the updates and return the latest global model. Secondly, STAFL will judge the user’s data distribution according to the user’s update and dynamically change the aggregation parameters according to the user’s network weight and staleness to minimize the impact of non-independent and identically distributed data sets on asynchronous updates. The experimental results show that our method performs better on non-independent and identically distributed data sets than existing methods.
4
Wu, Jikun, JiaHao Yu, and YuJun Zheng. "Research on Federated Learning Algorithms in Non-Independent Identically Distributed Scenarios." Highlights in Science, Engineering and Technology 85 (March 13, 2024): 104–12. http://dx.doi.org/10.54097/7newsv97.
Full textAbstract:
Federal learning is distributed learning and is mainly training locally by using multiple distributed devices. After receiving a local parameter, a server performs aggregation and performs multiple iterations until convergence to a final stable model. However, in actual application, due to different preferences of clients and differences in local data of different clients, data in federal learning may be not independently and identically distributed. (Non-Independent Identically Distribution). The main research work of this article is as follows: 1)Analyze and summarize the methods and techniques for solving the non-IID data problem in past experiments.2) Perform in-depth research on the basic methods of federal learning on non-IID data, such as FedAvg and FedProx.3) By using the FedAvg algorithm, using the CIFAR-10 data set, the simulation method is used to simulate the number of types contained in each client, and the distribution of the data set divided according to the distribution of Dirichlet to simulate the non-independent identical distribution of data. The detailed data analysis is made on the influence of the data on the accuracy and loss of model training.
5
Jiang, Yingrui, Xuejian Zhao, Hao Li, and Yu Xue. "A Personalized Federated Learning Method Based on Knowledge Distillation and Differential Privacy." Electronics 13, no. 17 (September 6, 2024): 3538. http://dx.doi.org/10.3390/electronics13173538.
Full textAbstract:
Federated learning allows data to remain decentralized, and various devices work together to train a common machine learning model. This method keeps sensitive data local on devices, protecting privacy. However, privacy protection and non-independent and identically distributed data are significant challenges for many FL techniques currently in use. This paper proposes a personalized federated learning method (FedKADP) that integrates knowledge distillation and differential privacy to address the issues of privacy protection and non-independent and identically distributed data in federated learning. The introduction of a bidirectional feedback mechanism enables the establishment of an interactive tuning loop between knowledge distillation and differential privacy, allowing dynamic tuning and continuous performance optimization while protecting user privacy. By closely monitoring privacy overhead through Rényi differential privacy theory, this approach effectively balances model performance and privacy protection. Experimental results using the MNIST and CIFAR-10 datasets demonstrate that FedKADP performs better than conventional federated learning techniques, particularly when handling non-independent and identically distributed data. It successfully lowers the heterogeneity of the model, accelerates global model convergence, and improves validation accuracy, making it a new approach to federated learning.
6
Babar, Muhammad, Basit Qureshi, and Anis Koubaa. "Investigating the impact of data heterogeneity on the performance of federated learning algorithm using medical imaging." PLOS ONE 19, no. 5 (May 15, 2024): e0302539. http://dx.doi.org/10.1371/journal.pone.0302539.
Full textAbstract:
In recent years, Federated Learning (FL) has gained traction as a privacy-centric approach in medical imaging. This study explores the challenges posed by data heterogeneity on FL algorithms, using the COVIDx CXR-3 dataset as a case study. We contrast the performance of the Federated Averaging (FedAvg) algorithm on non-identically and independently distributed (non-IID) data against identically and independently distributed (IID) data. Our findings reveal a notable performance decline with increased data heterogeneity, emphasizing the need for innovative strategies to enhance FL in diverse environments. This research contributes to the practical implementation of FL, extending beyond theoretical concepts and addressing the nuances in medical imaging applications. This research uncovers the inherent challenges in FL due to data diversity. It sets the stage for future advancements in FL strategies to effectively manage data heterogeneity, especially in sensitive fields like healthcare.
7
Layne, Elliot, Erika N. Dort, Richard Hamelin, Yue Li, and Mathieu Blanchette. "Supervised learning on phylogenetically distributed data." Bioinformatics 36, Supplement_2 (December 2020): i895—i902. http://dx.doi.org/10.1093/bioinformatics/btaa842.
Full textAbstract:
Abstract Motivation The ability to develop robust machine-learning (ML) models is considered imperative to the adoption of ML techniques in biology and medicine fields. This challenge is particularly acute when data available for training is not independent and identically distributed (iid), in which case trained models are vulnerable to out-of-distribution generalization problems. Of particular interest are problems where data correspond to observations made on phylogenetically related samples (e.g. antibiotic resistance data). Results We introduce DendroNet, a new approach to train neural networks in the context of evolutionary data. DendroNet explicitly accounts for the relatedness of the training/testing data, while allowing the model to evolve along the branches of the phylogenetic tree, hence accommodating potential changes in the rules that relate genotypes to phenotypes. Using simulated data, we demonstrate that DendroNet produces models that can be significantly better than non-phylogenetically aware approaches. DendroNet also outperforms other approaches at two biological tasks of significant practical importance: antiobiotic resistance prediction in bacteria and trophic level prediction in fungi. Availability and implementation https://github.com/BlanchetteLab/DendroNet.
8
Shahrivari, Farzad, and Nikola Zlatanov. "On Supervised Classification of Feature Vectors with Independent and Non-Identically Distributed Elements." Entropy 23, no. 8 (August 13, 2021): 1045. http://dx.doi.org/10.3390/e23081045.
Full textAbstract:
In this paper, we investigate the problem of classifying feature vectors with mutually independent but non-identically distributed elements that take values from a finite alphabet set. First, we show the importance of this problem. Next, we propose a classifier and derive an analytical upper bound on its error probability. We show that the error probability moves to zero as the length of the feature vectors grows, even when there is only one training feature vector per label available. Thereby, we show that for this important problem at least one asymptotically optimal classifier exists. Finally, we provide numerical examples where we show that the performance of the proposed classifier outperforms conventional classification algorithms when the number of training data is small and the length of the feature vectors is sufficiently high.
9
Lv, Yankai, Haiyan Ding, Hao Wu, Yiji Zhao, and Lei Zhang. "FedRDS: Federated Learning on Non-IID Data via Regularization and Data Sharing." Applied Sciences 13, no. 23 (December 4, 2023): 12962. http://dx.doi.org/10.3390/app132312962.
Full textAbstract:
Federated learning (FL) is an emerging decentralized machine learning framework enabling private global model training by collaboratively leveraging local client data without transferring it centrally. Unlike traditional distributed optimization, FL trains the model at the local client and then aggregates it at the server. While this approach reduces communication costs, the local datasets of different clients are non-Independent and Identically Distributed (non-IID), which may make the local model inconsistent. The present study suggests a FL algorithm that leverages regularization and data sharing (FedRDS). The local loss function is adapted by introducing a regularization term in each round of training so that the local model will gradually move closer to the global model. However, when the client data distribution gap becomes large, adding regularization items will increase the degree of client drift. Based on this, we used a data-sharing method in which a portion of server data is taken out as a shared dataset during the initialization. We then evenly distributed these data to each client to mitigate the problem of client drift by reducing the difference in client data distribution. Analysis of experimental outcomes indicates that FedRDS surpasses some known FL methods in various image classification tasks, enhancing both communication efficacy and accuracy.
10
Zhang, Xufei, and Yiqing Shen. "Non-IID federated learning with Mixed-Data Calibration." Applied and Computational Engineering 45, no. 1 (March 15, 2024): 168–78. http://dx.doi.org/10.54254/2755-2721/45/20241048.
Full textAbstract:
Federated learning (FL) is a privacy-preserving and collaborative machine learning approach for decentralized data across multiple clients. However, the presence of non-independent and non-identically distributed (non-IID) data among clients poses challenges to the performance of the global model. To address this, we propose Mixed Data Calibration (MIDAC). MIDAC mixes M data points to neutralize sensitive information in each individual data point and uses the mixed data to calibrate the global model on the server in a privacy-preserving way. MIDAC improves global model accuracy with low computational overhead while preserving data privacy. Our experiments on CIFAR-10 and BloodMNIST datasets validate the effectiveness of MIDAC in improving the accuracy of federated learning models under non-IID data distributions.
11
Alotaibi, Basmah, Fakhri Alam Khan, and Sajjad Mahmood. "Communication Efficiency and Non-Independent and Identically Distributed Data Challenge in Federated Learning: A Systematic Mapping Study." Applied Sciences 14, no. 7 (March 24, 2024): 2720. http://dx.doi.org/10.3390/app14072720.
Full textAbstract:
Federated learning has emerged as a promising approach for collaborative model training across distributed devices. Federated learning faces challenges such as Non-Independent and Identically Distributed (non-IID) data and communication challenges. This study aims to provide in-depth knowledge in the federated learning environment by identifying the most used techniques for overcoming non-IID data challenges and techniques that provide communication-efficient solutions in federated learning. The study highlights the most used non-IID data types, learning models, and datasets in federated learning. A systematic mapping study was performed using six digital libraries, and 193 studies were identified and analyzed after the inclusion and exclusion criteria were applied. We identified that enhancing the aggregation method and clustering are the most widely used techniques for non-IID data problems (used in 18% and 16% of the selected studies), and a quantization technique was the most common technique in studies that provide communication-efficient solutions in federated learning (used in 27% and 15% of the selected studies). Additionally, our work shows that label distribution skew is the most used case to simulate a non-IID environment, specifically, the quantity label imbalance. The supervised learning model CNN model is the most commonly used learning model, and the image datasets MNIST and Cifar-10 are the most widely used datasets when evaluating the proposed approaches. Furthermore, we believe the research community needs to consider the client’s limited resources and the importance of their updates when addressing non-IID and communication challenges to prevent the loss of valuable and unique information. The outcome of this systematic study will benefit federated learning users, researchers, and providers.
12
Wang, Zhao, Yifan Hu, Shiyang Yan, Zhihao Wang, Ruijie Hou, and Chao Wu. "Efficient Ring-Topology Decentralized Federated Learning with Deep Generative Models for Medical Data in eHealthcare Systems." Electronics 11, no. 10 (May 12, 2022): 1548. http://dx.doi.org/10.3390/electronics11101548.
Full textAbstract:
By leveraging deep learning technologies, data-driven-based approaches have reached great success with the rapid increase of data generated for medical applications. However, security and privacy concerns are obstacles for data providers in many sensitive data-driven scenarios, such as rehabilitation and 24 h on-the-go healthcare services. Although many federated learning (FL) approaches have been proposed with DNNs for medical applications, these works still suffer from low usability of data due to data incompleteness, low quality, insufficient quantity, sensitivity, etc. Therefore, we propose a ring-topology-based decentralized federated learning (RDFL) scheme for deep generative models (DGM), where DGM is a promising solution for solving the aforementioned data usability issues. Our RDFL schemes provide communication efficiency and maintain training performance to boost DGMs in target tasks compared with existing FL works. A novel ring FL topology and a map-reduce-based synchronizing method are designed in the proposed RDFL to improve the decentralized FL performance and bandwidth utilization. In addition, an inter-planetary file system (IPFS) is introduced to further improve communication efficiency and FL security. Extensive experiments have been taken to demonstrate the superiority of RDFL with either independent and identically distributed (IID) datasets or non-independent and identically distributed (Non-IID) datasets.
13
Aggarwal, Meenakshi, Vikas Khullar, Nitin Goyal, Abdullah Alammari, Marwan Ali Albahar, and Aman Singh. "Lightweight Federated Learning for Rice Leaf Disease Classification Using Non Independent and Identically Distributed Images." Sustainability 15, no. 16 (August 9, 2023): 12149. http://dx.doi.org/10.3390/su151612149.
Full textAbstract:
Rice (Oryza sativa L.) is a vital food source all over the world, contributing 15% of the protein and 21% of the energy intake per person in Asia, where most rice is produced and consumed. However, bacterial, fungal, and other microbial diseases that have a negative effect on the health of plants and crop yield are a major problem for rice farmers. It is challenging to diagnose these diseases manually, especially in areas with a shortage of crop protection experts. Automating disease identification and providing readily available decision-support tools are essential for enabling effective rice leaf protection measures and minimising rice crop losses. Although there are numerous classification systems for the diagnosis of rice leaf disease, no reliable, secure method has been identified that meets these needs. This paper proposes a lightweight federated deep learning architecture while maintaining data privacy constraints for rice leaf disease classification. The distributed client–server design of this framework protects the data privacy of all clients, and by using independent and identically distributed (IID) and non-IID data, the validity of the federated deep learning models was examined. To validate the framework’s efficacy, the researchers conducted experiments in a variety of settings, including conventional learning, federated learning via a single client, as well as federated learning via multiple clients. The study began by extracting features from various pre-trained models, ultimately selecting EfficientNetB3 with an impressive 99% accuracy as the baseline model. Subsequently, experimental results were conducted using the federated learning (FL) approach with both IID and non-IID datasets. The FL approach, along with a dense neural network trained and evaluated on an IID dataset, achieved outstanding training and evaluated accuracies of 99% with minimal losses of 0.006 and 0.03, respectively. Similarly, on a non-IID dataset, the FL approach maintained a high training accuracy of 99% with a loss of 0.04 and an evaluation accuracy of 95% with a loss of 0.08. These results indicate that the FL approach performs nearly as well as the base model, EfficientNetB3, highlighting its effectiveness in handling both IID and non-IID data. It was found that federated deep learning models with multiple clients outperformed conventional pre-trained models. The unique characteristics of the proposed framework, such as its data privacy for edge devices with limited resources, set it apart from the existing classification schemes for rice leaf diseases. The framework is the best alternative solution for the early classification of rice leaf disease because of these additional features.
14
Niang, Aladji Babacar, Gane Samb Lo, Cherif Mamadou Traoré, and Amadou Ball. "\(\ell^{\infty}\) Poisson invariance principles from two classical Poisson limit theorems and extension to non-stationary independent sequences." Afrika Statistika 17, no. 1 (January 1, 2022): 3125–43. http://dx.doi.org/10.16929/as/2022.3125.198.
Full textAbstract:
The simple Lévy Poisson process and scaled forms are explicitly constructed from partial sums of independent and identically distributed random variables and from sums of non-stationary independent random variables. For the latter, the weak limits are scaled Poisson processes. The method proposed here prepares generalizations to dependent data, to associated data in the first place.
15
Niang, Aladji Babacar, Gane Samb Lo, Cherif Mamadou Moctar Traoré, and Amadou Ball. "\(\ell^{\infty}\) Poisson invariance principles from two classical Poisson limit theorems and extension to non-stationary independent sequences." Afrika Statistika 17, no. 1 (January 1, 2022): 3125–43. http://dx.doi.org/10.16929/as/3125.3115.198.
Full textAbstract:
The simple Lévy Poisson process and scaled forms are explicitly constructed from partial sums of independent and identically distributed random variables and from sums of non-stationary independent random variables. For the latter, the weak limits are scaled Poisson processes. The method proposed here prepares generalizations to dependent data, to associated data in the first place.
16
Wu, Xia, Lei Xu, and Liehuang Zhu. "Local Differential Privacy-Based Federated Learning under Personalized Settings." Applied Sciences 13, no. 7 (March 24, 2023): 4168. http://dx.doi.org/10.3390/app13074168.
Full textAbstract:
Federated learning is a distributed machine learning paradigm, which utilizes multiple clients’ data to train a model. Although federated learning does not require clients to disclose their original data, studies have shown that attackers can infer clients’ privacy by analyzing the local models shared by clients. Local differential privacy (LDP) can help to solve the above privacy issue. However, most of the existing federated learning studies based on LDP, rarely consider the diverse privacy requirements of clients. In this paper, we propose an LDP-based federated learning framework, that can meet the personalized privacy requirements of clients. We consider both independent identically distributed (IID) datasets and non-independent identically distributed (non-IID) datasets, and design model perturbation methods, respectively. Moreover, we propose two model aggregation methods, namely weighted average method and probability-based selection method. The main idea, is to weaken the impact of those privacy-conscious clients, who choose relatively small privacy budgets, on the federated model. Experiments on three commonly used datasets, namely MNIST, Fashion-MNIST, and forest cover-types, show that the proposed aggregation methods perform better than the classic arithmetic average method, in the personalized privacy preserving scenario.
17
Bejenar, Iuliana, Lavinia Ferariu, Carlos Pascal, and Constantin-Florin Caruntu. "Aggregation Methods Based on Quality Model Assessment for Federated Learning Applications: Overview and Comparative Analysis." Mathematics 11, no. 22 (November 10, 2023): 4610. http://dx.doi.org/10.3390/math11224610.
Full textAbstract:
Federated learning (FL) offers the possibility of collaboration between multiple devices while maintaining data confidentiality, as required by the General Data Protection Regulation (GDPR). Though FL can keep local data private, it may encounter problems when dealing with non-independent and identically distributed data (non-IID), insufficient local training samples or cyber-attacks. This paper introduces algorithms that can provide a reliable aggregation of the global model by investigating the accuracy of models received from clients. This allows reducing the influence of less confident nodes, who were potentially attacked or unable to perform successful training. The analysis includes the proposed FedAcc and FedAccSize algorithms, together with their new extension based on the Lasso regression, FedLasso. FedAcc and FedAccSize set the confidence in each client based only on local models’ accuracy, while FedLasso exploits additional details related to predictions, like predicted class probabilities, to support a refined aggregation. The ability of the proposed algorithms to protect against intruders or underperforming clients is demonstrated experimentally using testing scenarios involving independent and identically distributed (IID) data as well as non-IID data. The comparison with the established FedAvg and FedAvgM algorithms shows that exploiting the quality of the client models is essential for reliable aggregation, which enables rapid and robust improvement in the global model.
18
Tayyeh, Huda Kadhim, and Ahmed Sabah Ahmed AL-Jumaili. "Balancing Privacy and Performance: A Differential Privacy Approach in Federated Learning." Computers 13, no. 11 (October 24, 2024): 277. http://dx.doi.org/10.3390/computers13110277.
Full textAbstract:
Federated learning (FL), a decentralized approach to machine learning, facilitates model training across multiple devices, ensuring data privacy. However, achieving a delicate privacy preservation–model convergence balance remains a major problem. Understanding how different hyperparameters affect this balance is crucial for optimizing FL systems. This article examines the impact of various hyperparameters, like the privacy budget (ϵ), clipping norm (C), and the number of randomly chosen clients (K) per communication round. Through a comprehensive set of experiments, we compare training scenarios under both independent and identically distributed (IID) and non-independent and identically distributed (Non-IID) data settings. Our findings reveal that the combination of ϵ and C significantly influences the global noise variance, affecting the model’s performance in both IID and Non-IID scenarios. Stricter privacy conditions lead to fluctuating non-converging loss behavior, particularly in Non-IID settings. We consider the number of clients (K) and its impact on the loss fluctuations and the convergence improvement, particularly under strict privacy measures. Thus, Non-IID settings are more responsive to stricter privacy regulations; yet, with a higher client interaction volume, they also can offer better convergence. Collectively, knowledge of the privacy-preserving approach in FL has been extended and useful suggestions towards an ideal privacy–convergence balance were achieved.
19
Liu, Ying, Zhiqiang Wang, Shufang Pang, and Lei Ju. "Distributed Malicious Traffic Detection." Electronics 13, no. 23 (November 28, 2024): 4720. http://dx.doi.org/10.3390/electronics13234720.
Full textAbstract:
With the wide deployment of edge devices, distributed network traffic data are rapidly increasing. Traditional detection methods for malicious traffic rely on centralized training, in which a single server is often used to aggregate private traffic data from edge devices, so as to extract and identify features. However, these methods face difficult data collection, heavy computational complexity, and high privacy risks. To address these issues, this paper proposes a federated learning-based distributed malicious traffic detection framework, FL-CNN-Traffic. In this framework, edge devices utilize a convolutional neural network (CNN) to process local detection, data collection, feature extraction, and training. A server aggregates model updates from edge devices using four federated learning algorithms (FedAvg, FedProx, Scaffold, and FedNova) to build a global model. This framework allows multiple devices to collaboratively train a model without sharing private traffic data, addressing the “Data Silo” problem while ensuring privacy. Evaluations on the USTC-TFC2016 dataset show that for independent and identically distributed (IID) data, this framework can reach or exceed the performance of centralized deep learning methods. For Non-IID data, this framework outperforms other neural networks based on federated learning, with accuracy improvements ranging from 2.59% to 4.73%.
20
Leroy, Fanny, Jean-Yves Dauxois, and Pascale Tubert-Bitter. "On the Parametric Maximum Likelihood Estimator for Independent but Non-identically Distributed Observations with Application to Truncated Data." Journal of Statistical Theory and Applications 15, no. 1 (2016): 96. http://dx.doi.org/10.2991/jsta.2016.15.1.8.
Full text
21
DIB, ABDESSAMAD, MOHAMED MEHDI HAMRI, and ABBES RABHI. "ASYMPTOTIC NORMALITY SINGLE FUNCTIONAL INDEX QUANTILE REGRESSION UNDER RANDOMLY CENSORED DATA." Journal of Science and Arts 22, no. 4 (December 30, 2022): 845–64. http://dx.doi.org/10.46939/j.sci.arts-22.4-a07.
Full textAbstract:
The main objective of this paper is to estimate non-parametrically the quantiles of a conditional distribution based on the single-index model in the censorship model when the sample is considered as an independent and identically distributed (i.i.d.) random variables. First of all, a kernel type estimator for the conditional cumulative distribution function (cond-cdf) is introduced. Afterwards, we give an estimation of the quantiles by inverting this estimated cond-cdf, the asymptotic properties are stated when the observations are linked with a single-index structure. Finally, a simulation study is carried out to evaluate the performance of this estimate.
22
Jahani, Khalil, Behzad Moshiri, and Babak Hossein Khalaj. "A Survey on Data Distribution Challenges and Solutions in Vertical and Horizontal Federated Learning." Journal of Artificial Intelligence, Applications, and Innovations 1, no. 2 (2024): 55–71. https://doi.org/10.61838/jaiai.1.2.5.
Full textAbstract:
Federated learning is a novel way of training machine learning models on data that is distributed across multiple devices, such as smartphones and IoT sensors, without compromising privacy, efficiency, or security. However, federated learning faces a significant challenge when the data on each device is not independent and identically distributed (non-IID), which means that the data may have different distributions, sizes, or qualities. non-IID data is a major challenge for federated learning, as it affects the accuracy and participation of the local devices. Most existing methods focus on improving the model, algorithm, or framework of federated learning to deal with non-IID data. However, there is a lack of systematic and up-to-date reviews on this topic. In this paper, we survey different approaches to address the challenge of non-IID data in Vertical Federated Learning (VFL) and Horizontal Federated Learning (HFL). We organize the existing literature based on the perspective of the researcher and the sub-tasks involved in each approach. Our goal is to provide a comprehensive and systematic overview of the problem and its solutions.
23
Zhang, Jianfei, and Zhongxin Li. "A Clustered Federated Learning Method of User Behavior Analysis Based on Non-IID Data." Electronics 12, no. 7 (March 31, 2023): 1660. http://dx.doi.org/10.3390/electronics12071660.
Full textAbstract:
Federated learning (FL) is a novel distributed machine learning paradigm. It can protect data privacy in distributed machine learning. Hence, FL provides new ideas for user behavior analysis. User behavior analysis can be modeled using multiple data sources. However, differences between different data sources can lead to different data distributions, i.e., non-identically and non-independently distributed (Non-IID). Non-IID data usually introduce bias in the training process of FL models, which will affect the model accuracy and convergence speed. In this paper, a new federated learning algorithm is proposed to mitigate the impact of Non-IID data on the model, named federated learning with a two-tier caching mechanism (FedTCM). First, FedTCM clustered similar clients based on their data distribution. Clustering reduces the extent of Non-IID between clients in a cluster. Second, FedTCM uses asynchronous communication methods to alleviate the problem of inconsistent computation speed across different clients. Finally, FedTCM sets up a two-tier caching mechanism on the server for mitigating the Non-IID data between different clusters. In multiple simulated datasets, compared to the method without the federated framework, the FedTCM is maximum 15.8% higher than it and average 12.6% higher than it. Compared to the typical federated method FedAvg, the accuracy of FedTCM is maximum 2.3% higher than it and average 1.6% higher than it. Additionally, FedTCM achieves more excellent communication performance than FedAvg.
24
Chen, Runzi, Shuliang Zhao, and Zhenzhen Tian. "A Multiscale Clustering Approach for Non-IID Nominal Data." Computational Intelligence and Neuroscience 2021 (October 11, 2021): 1–10. http://dx.doi.org/10.1155/2021/8993543.
Full textAbstract:
Multiscale brings great benefits for people to observe objects or problems from different perspectives. Multiscale clustering has been widely studied in various disciplines. However, most of the research studies are only for the numerical dataset, which is a lack of research on the clustering of nominal dataset, especially the data are nonindependent and identically distributed (Non-IID). Aiming at the current research situation, this paper proposes a multiscale clustering framework based on Non-IID nominal data. Firstly, the benchmark-scale dataset is clustered based on coupled metric similarity measure. Secondly, it is proposed to transform the clustering results from benchmark scale to target scale that the two algorithms are named upscaling based on single chain and downscaling based on Lanczos kernel, respectively. Finally, experiments are performed using five public datasets and one real dataset of the Hebei province of China. The results showed that the method can provide us not only competitive performance but also reduce computational cost.
25
Yan, Jiaxing, Yan Li, Sifan Yin, Xin Kang, Jiachen Wang, Hao Zhang, and Bin Hu. "An Efficient Greedy Hierarchical Federated Learning Training Method Based on Trusted Execution Environments." Electronics 13, no. 17 (September 6, 2024): 3548. http://dx.doi.org/10.3390/electronics13173548.
Full textAbstract:
With the continuous development of artificial intelligence, effectively solving the problem of data islands under the premise of protecting user data privacy has become a top priority. Federal learning is an effective solution to the two significant dilemmas of data islands and data privacy protection. However, there are still some security problems in federal learning. Therefore, this study simulates the data distribution in a hardware-based trusted execution environment in the real world through two processing methods: independent identically distributed and non-independent identically distributed methods. The basic model uses ResNet164 and innovatively introduces a greedy hierarchical training strategy to gradually train and aggregate complex models to ensure that the training of each layer is optimized under the premise of protecting privacy. The experimental results show that under the condition of an IID data distribution, the final accuracy of the greedy hierarchical model reaches 86.72%, which is close to the accuracy of the unpruned model at 89.60%. In contrast, under the non-IID condition, the model’s performance decreases. Overall, the TEE-based hierarchical federated learning method shows reasonable practicability and effectiveness in a resource-constrained environment. Through this study, the advantages of the greedy hierarchical federated learning model with regard to enhancing data privacy protection, optimizing resource utilization, and improving model training efficiency are further verified, providing new ideas and methods for solving the data island and data privacy protection problems.
26
Gao, Huiguo, Mengyuan Lee, Guanding Yu, and Zhaolin Zhou. "A Graph Neural Network Based Decentralized Learning Scheme." Sensors 22, no. 3 (January 28, 2022): 1030. http://dx.doi.org/10.3390/s22031030.
Full textAbstract:
As an emerging paradigm considering data privacy and transmission efficiency, decentralized learning aims to acquire a global model using the training data distributed over many user devices. It is a challenging problem since link loss, partial device participation, and non-independent and identically distributed (non-iid) data distribution would all deteriorate the performance of decentralized learning algorithms. Existing work may restrict to linear models or show poor performance over non-iid data. Therefore, in this paper, we propose a decentralized learning scheme based on distributed parallel stochastic gradient descent (DPSGD) and graph neural network (GNN) to deal with the above challenges. Specifically, each user device participating in the learning task utilizes local training data to compute local stochastic gradients and updates its own local model. Then, each device utilizes the GNN model and exchanges the model parameters with its neighbors to reach the average of resultant global models. The iteration repeats until the algorithm converges. Extensive simulation results over both iid and non-iid data validate the algorithm’s convergence to near optimal results and robustness to both link loss and partial device participation.
27
Zhou, Yuwen, Yuhan Hu, Jing Sun, Rui He, and Wenjie Kang. "A Semi-Federated Active Learning Framework for Unlabeled Online Network Data." Mathematics 11, no. 8 (April 21, 2023): 1972. http://dx.doi.org/10.3390/math11081972.
Full textAbstract:
Federated Learning (FL) is a newly emerged federated optimization technique for distributed data in a federated network. The participants in FL that train the model locally are classified into client nodes. The server node assumes the responsibility to aggregate local models from client nodes without data moving. In this regard, FL is an ideal solution to protect data privacy at each node of the network. However, the raw data generated on each node are unlabeled, making it impossible for FL to apply these data directly to train a model. The large volume of data annotating work prevents FL from being widely applied in the real world, especially for online scenarios, where the data are generated continuously. Meanwhile, the data generated on different nodes tend to be differently distributed. It has been proved theoretically and experimentally that non-independent and identically distributed (non-IID) data harm the performance of FL. In this article, we design a semi-federated active learning (semi-FAL) framework to tackle the annotation and non-IID problems jointly. More specifically, the server node can provide (i) a pre-trained model to help each client node annotate the local data uniformly and (ii) an estimation of the global gradient to help correct the local gradient. The evaluation results demonstrate our semi-FAL framework can efficiently handle unlabeled online network data and achieves high accuracy and fast convergence.
28
Wang, Jinru, Zijuan Geng, and Fengfeng Jin. "Optimal Wavelet Estimation of Density Derivatives for Size-Biased Data." Abstract and Applied Analysis 2014 (2014): 1–13. http://dx.doi.org/10.1155/2014/512634.
Full textAbstract:
A perfect achievement has been made for wavelet density estimation by Dohono et al. in 1996, when the samples without any noise are independent and identically distributed (i.i.d.). But in many practical applications, the random samples always have noises, and estimation of the density derivatives is very important for detecting possible bumps in the associated density. Motivated by Dohono's work, we propose new linear and nonlinear wavelet estimatorsf^lin(m),f^non(m)for density derivativesf(m)when the random samples have size-bias. It turns out that the linear estimationE(∥f^lin(m)-f(m)∥p)forf(m)∈Br,qs(A,L)attains the optimal covergence rate whenr≥p, and the nonlinear oneE(∥f^lin(m)-f(m)∥p)does the same ifr<p.
29
Efthymiadis, Filippos, Aristeidis Karras, Christos Karras, and Spyros Sioutas. "Advanced Optimization Techniques for Federated Learning on Non-IID Data." Future Internet 16, no. 10 (October 13, 2024): 370. http://dx.doi.org/10.3390/fi16100370.
Full textAbstract:
Federated learning enables model training on multiple clients locally, without the need to transfer their data to a central server, thus ensuring data privacy. In this paper, we investigate the impact of Non-Independent and Identically Distributed (non-IID) data on the performance of federated training, where we find a reduction in accuracy of up to 29% for neural networks trained in environments with skewed non-IID data. Two optimization strategies are presented to address this issue. The first strategy focuses on applying a cyclical learning rate to determine the learning rate during federated training, while the second strategy develops a sharing and pre-training method on augmented data in order to improve the efficiency of the algorithm in the case of non-IID data. By combining these two methods, experiments show that the accuracy on the CIFAR-10 dataset increased by about 36% while achieving faster convergence by reducing the number of required communication rounds by 5.33 times. The proposed techniques lead to improved accuracy and faster model convergence, thus representing a significant advance in the field of federated learning and facilitating its application to real-world scenarios.
30
Seol, Mihye, and Taejoon Kim. "Performance Enhancement in Federated Learning by Reducing Class Imbalance of Non-IID Data." Sensors 23, no. 3 (January 19, 2023): 1152. http://dx.doi.org/10.3390/s23031152.
Full textAbstract:
Due to the distributed data collection and learning in federated learnings, many clients conduct local training with non-independent and identically distributed (non-IID) datasets. Accordingly, the training from these datasets results in severe performance degradation. We propose an efficient algorithm for enhancing the performance of federated learning by overcoming the negative effects of non-IID datasets. First, the intra-client class imbalance is reduced by rendering the class distribution of clients close to Uniform distribution. Second, the clients to participate in federated learning are selected to make their integrated class distribution close to Uniform distribution for the purpose of mitigating the inter-client class imbalance, which represents the class distribution difference among clients. In addition, the amount of local training data for the selected clients is finely adjusted. Finally, in order to increase the efficiency of federated learning, the batch size and the learning rate of local training for the selected clients are dynamically controlled reflecting the effective size of the local dataset for each client. In the performance evaluation on CIFAR-10 and MNIST datasets, the proposed algorithm achieves 20% higher accuracy than existing federated learning algorithms. Moreover, in achieving this huge accuracy improvement, the proposed algorithm uses less computation and communication resources compared to existing algorithms in terms of the amount of data used and the number of clients joined in the training.
31
Lee, Suchul. "Distributed Detection of Malicious Android Apps While Preserving Privacy Using Federated Learning." Sensors 23, no. 4 (February 15, 2023): 2198. http://dx.doi.org/10.3390/s23042198.
Full textAbstract:
Recently, deep learning has been widely used to solve existing computing problems through large-scale data mining. Conventional training of the deep learning model is performed on a central (cloud) server that is equipped with high computing power, by integrating data via high computational intensity. However, integrating raw data from multiple clients raises privacy concerns that are increasingly being focused on. In federated learning (FL), clients train deep learning models in a distributed fashion using their local data; instead of sending raw data to a central server, they send parameter values of the trained local model to a central server for integration. Because FL does not transmit raw data to the outside, it is free from privacy issues. In this paper, we perform an experimental study that explores the dynamics of the FL-based Android malicious app detection method under three data distributions across clients, i.e., (i) independent and identically distributed (IID), (ii) non-IID, (iii) non-IID and unbalanced. Our experiments demonstrate that the application of FL is feasible and efficient in detecting malicious Android apps in a distributed manner on cellular networks.
32
Zhao, Puning, Fei Yu, and Zhiguo Wan. "A Huber Loss Minimization Approach to Byzantine Robust Federated Learning." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 19 (March 24, 2024): 21806–14. http://dx.doi.org/10.1609/aaai.v38i19.30181.
Full textAbstract:
Federated learning systems are susceptible to adversarial attacks. To combat this, we introduce a novel aggregator based on Huber loss minimization, and provide a comprehensive theoretical analysis. Under independent and identically distributed (i.i.d) assumption, our approach has several advantages compared to existing methods. Firstly, it has optimal dependence on epsilon, which stands for the ratio of attacked clients. Secondly, our approach does not need precise knowledge of epsilon. Thirdly, it allows different clients to have unequal data sizes. We then broaden our analysis to include non-i.i.d data, such that clients have slightly different distributions.
33
Valente Neto, Ernesto, Solon Peixoto, and Júlio César Anjos. "EnBaSe: Enhancing Image Classification in IoT Scenarios through Entropy-Based Selection of Non-IID Data." Learning and Nonlinear Models 23, no. 1 (February 28, 2025): 49–66. https://doi.org/10.21528/lnlm-vol23-no1-art4.
Full textAbstract:
This study presents an analysis of the scalability and dispersion of results in Federated Learning (FL) using two algorithms: EnBaSe, based on entropy, and Random, a random selection approach. The Random algorithm ensures that each member of the population has an equal probability of inclusion. At the same time, EnBaSe calculates the information gain and selects the most informative samples for the neural network. Both algorithms were applied in federated learning scenarios with data distributed non-independently and non-identically (Non-IID). The MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 datasets were used for the evaluation, representing different levels of computer vision classification. The results show that the EnBaSe algorithm achieves high accuracy while halving computational and energy costs compared to training with all samples from the datasets. In addition, EnBaSe demonstrated greater resilience to variability, showing low variance and a more stable distribution, especially in Internet of things (IoT) environments with limited computational resources.
34
Firdaus, Muhammad, Siwan Noh, Zhuohao Qian, Harashta Tatimma Larasati, and Kyung-Hyune Rhee. "Personalized federated learning for heterogeneous data: A distributed edge clustering approach." Mathematical Biosciences and Engineering 20, no. 6 (2023): 10725–40. http://dx.doi.org/10.3934/mbe.2023475.
Full textAbstract:
<abstract><p>Federated learning (FL) is a distributed machine learning technique that allows multiple devices (e.g., smartphones and IoT devices) to collaborate in the training of a shared model with each device preserving the privacy of its local data. However, the highly heterogeneous distribution of data among clients in FL can result in poor convergence. In addressing this issue, the concept of personalized federated learning (PFL) has emerged. PFL aims to tackle the effects of non-independent and identically distributed data and statistical heterogeneity and to achieve personalized models with rapid model convergence. One approach is clustering-based PFL, which utilizes group-level client relationships to achieve personalization. However, this method still relies on a centralized approach, whereby the server coordinates all processes. To address these shortcomings, this study introduces a blockchain-enabled distributed edge cluster for PFL (BPFL) that combines the benefits of blockchain and edge computing. Blockchain technology can be used to enhance client privacy and security by recording transactions on immutable distributed ledger networks, thereby improving client selection and clustering. The edge computing system offers reliable storage and computation such that computational processing is locally performed in the edge infrastructure to be closer to clients. Thus, the real-time services and low-latency communication of PFL are improved. However, further work is required to develop a representative dataset for the examination of related types of attacks and defenses for a robust BPFL protocol.</p></abstract>
35
Chu, Patrick K. K. "Study on the Non-Random and Chaotic Behavior of Chinese Equities Market." Review of Pacific Basin Financial Markets and Policies 06, no. 02 (June 2003): 199–222. http://dx.doi.org/10.1142/s0219091503001055.
Full textAbstract:
After the stock market crash of October 19, 1987, interest in nonlinear dynamics and chaotic dynamics have increased in the field of financial analysis. The extent that the daily return data from the Shanghai Stock Exchange Index and the Shenzhen Stock Exchange Index exhibit non-random, nonlinear and chaotic characteristics are investigated by employing various tests from chaos theory. The Hurst exponent in R/S analysis rejects the hypothesis that the index return series are random, independent and identically distributed. The BDS test provides evidence for nonlinearity. The estimated correlation dimensions provide evidence for deterministic chaotic behaviors.
36
Knight, John L., and Stephen E. Satchell. "The Cumulant Generating Function Estimation Method." Econometric Theory 13, no. 2 (April 1997): 170–84. http://dx.doi.org/10.1017/s0266466600005715.
Full textAbstract:
This paper deals with the use of the empirical cumulant generating function to consistently estimate the parameters of a distribution from data that are independent and identically distributed (i.i.d.). The technique is particularly suited to situations where the density function is unknown or unbounded in parameter space. We prove asymptotic equivalence of our technique to that of the empirical characteristic function and outline a six-step procedure for its implementation. Extensions of the approach to non-i.i.d. situations are considered along with a discussion of suitable applications and a worked example.
37
Gao, Yuan. "Federated learning: Impact of different algorithms and models on prediction results based on fashion-MNIST data set." Applied and Computational Engineering 86, no. 1 (July 31, 2024): 210–18. http://dx.doi.org/10.54254/2755-2721/86/20241594.
Full textAbstract:
The realm of federated learning is rapidly advancing amid the era of big data. Therefore, how to select a suitable federated learning algorithm to achieve realistic tasks has become particularly critical. In this study, we explore the impact of different algorithms and models on the prediction results of Federated Learning (FL) using the Fashion-MNIST data set. Federated Learning enhances data privacy and reduces latency by training models directly on local devices since it is a decentralized machine learning approach. We analyze the performance of several FL algorithms including Federated Averaging (FedAvg), Federated Stochastic Gradient Descent (FedSGD), Federated Proximal (FedProx), and SCAFFOLD. Our experiments reveal significant differences in accuracy and stability among these algorithms, highlighting their strengths and weaknesses in handling non-IID (Non-Independent and Identically Distributed) data. FedProx demonstrate superior performance in terms of accuracy and robustness, making them suitable for complex federated learning environments. These discoveries offer crucial insights for choosing suitable FL algorithms and models in practical applications.
38
Choi, Jai Won, Balgobin Nandram, and Boseung Choi. "Combining Correlated P-values From Primary Data Analyses." International Journal of Statistics and Probability 11, no. 6 (October 20, 2022): 12. http://dx.doi.org/10.5539/ijsp.v11n6p12.
Full textAbstract:
Research results on the same subject, extracted from scientific papers or clinical trials, are combined to determine a consensus. We are primarily concerned with combining p-values from experiments that may be correlated. We have two methods, a non-Bayesian method and a Bayesian method. We use a model to combine these results and assume the combined results follow a certain distribution, for example, chi-square or normal. The distribution requires independent and identically distributed (iid) random variables. When the data are correlated or non-iid, we cannot assume such distribution. In order to do so, the combined results from the model need to be adjusted, and the adjustment is done “indirectly” through two test statistics. Specifically, one test statistic (TS** ) is obtained for the non-iid data and the other is the test statistic (TS) is obtained for iid data. We use the ratio between the two test statistics to adjust the model test statistic (TS**) for its non-iid violation. The adjusted TS** is named as “effective test statistics” (ETS), which is then used for statistical inferences with the assumed distribution. As it is difficult to estimate the correlation, to provide a more coherent method for combining p-values, we also introduce a novel Bayesian method for both iid data and non-iid data. The examples are used to illustrate the non-Bayesian method and additional examples are given to illustrate the Bayesian method.
39
Tan, Qingjie, Bin Wang, Hongfeng Yu, Shuhui Wu, Yaguan Qian, and Yuanhong Tao. "DP-FEDAW: FEDERATED LEARNING WITH DIFFERENTIAL PRIVACY IN NON-IID DATA." International Journal of Engineering Technologies and Management Research 10, no. 5 (May 20, 2023): 34–49. http://dx.doi.org/10.29121/ijetmr.v10.i5.2023.1328.
Full textAbstract:
Federated learning can effectively utilize data from various users to coordinately train machine learning models while ensuring that data does not leave the user's device. However, it also faces the challenge of slow global model convergence and even the leakage of model parameters under heterogeneous data. To address this issue, this paper proposes a federated weighted average with differential privacy (DP-FedAW) algorithm, which studies the security and convergence issues of federated learning for Non-independent identically distributed (Non-IID) data. Firstly, the DP-FedAW algorithm quantifies the degree of Non-IID for different user datasets and further adjusts the aggregation weights of each user, effectively alleviating the model convergence problem caused by differences in Non-IID data during the training process. Secondly, a federated weighted average algorithm for privacy protection is designed to ensure that the model parameters meet differential privacy requirements. In theory, this algorithm effectively provides privacy and security during the training process while accelerating the convergence of the model. Experiments have shown that compared to the federated average algorithm, this algorithm can converge faster. In addition, with the increase of the privacy budget, the model's accuracy gradually tends to be without noise while ensuring model security. This study provides an important reference for ensuring model parameter security and improving the algorithm convergence rate of federated learning towards the Non-IID data.
40
Shan, Ang, and Fengkai Yang. "Bayesian Inference for Finite Mixture Regression Model Based on Non-Iterative Algorithm." Mathematics 9, no. 6 (March 10, 2021): 590. http://dx.doi.org/10.3390/math9060590.
Full textAbstract:
Finite mixtures normal regression (FMNR) models are widely used to investigate the relationship between a response variable and a set of explanatory variables from several unknown latent homogeneous groups. However, the classical EM algorithm and Gibbs sampling to deal with this model have several weak points. In this paper, a non-iterative sampling algorithm for fitting FMNR model is proposed from a Bayesian perspective. The procedure can generate independently and identically distributed samples from the posterior distributions of the parameters and produce more reliable estimations than the EM algorithm and Gibbs sampling. Simulation studies are conducted to illustrate the performance of the algorithm with supporting results. Finally, a real data is analyzed to show the usefulness of the methodology.
41
Agrawal, Shaashwat, Sagnik Sarkar, Mamoun Alazab, Praveen Kumar Reddy Maddikunta, Thippa Reddy Gadekallu, and Quoc-Viet Pham. "Genetic CFL: Hyperparameter Optimization in Clustered Federated Learning." Computational Intelligence and Neuroscience 2021 (November 18, 2021): 1–10. http://dx.doi.org/10.1155/2021/7156420.
Full textAbstract:
Federated learning (FL) is a distributed model for deep learning that integrates client-server architecture, edge computing, and real-time intelligence. FL has the capability of revolutionizing machine learning (ML) but lacks in the practicality of implementation due to technological limitations, communication overhead, non-IID (independent and identically distributed) data, and privacy concerns. Training a ML model over heterogeneous non-IID data highly degrades the convergence rate and performance. The existing traditional and clustered FL algorithms exhibit two main limitations, including inefficient client training and static hyperparameter utilization. To overcome these limitations, we propose a novel hybrid algorithm, namely, genetic clustered FL (Genetic CFL), that clusters edge devices based on the training hyperparameters and genetically modifies the parameters clusterwise. Then, we introduce an algorithm that drastically increases the individual cluster accuracy by integrating the density-based clustering and genetic hyperparameter optimization. The results are bench-marked using MNIST handwritten digit dataset and the CIFAR-10 dataset. The proposed genetic CFL shows significant improvements and works well with realistic cases of non-IID and ambiguous data. An accuracy of 99.79% is observed in the MNIST dataset and 76.88% in CIFAR-10 dataset with only 10 training rounds.
42
Zhang, You, Jin Wang, Liang-Chih Yu, Dan Xu, and Xuejie Zhang. "Multi-Attribute Multi-Grained Adaptation of Pre-Trained Language Models for Text Understanding from Bayesian Perspective." Proceedings of the AAAI Conference on Artificial Intelligence 39, no. 24 (April 11, 2025): 25967–75. https://doi.org/10.1609/aaai.v39i24.34791.
Full textAbstract:
Current neural networks often employ multi-domain-learning or attribute-injecting mechanisms to incorporate non-independent and identically distributed (non-IID) information for text understanding tasks by capturing individual characteristics and the relationships among samples. However, the extent of the impact of non-IID information and how these methods affect pre-trained language models (PLMs) remains unclear. This study revisits the assumption that non-IID information enhances PLMs to achieve performance improvements from a Bayesian perspective, which unearths and integrates non-IID and IID features. Furthermore, we proposed a multi-attribute multi-grained framework for PLM adaptations (M2A), which combines multi-attribute and multi-grained views to mitigate uncertainty in a lightweight manner. We evaluate M2A through prevalent text-understanding datasets and demonstrate its superior performance, mainly when data are implicitly non-IID, and PLMs scale larger.
43
Zhang, Kainan, Zhipeng Cai, and Daehee Seo. "Privacy-Preserving Federated Graph Neural Network Learning on Non-IID Graph Data." Wireless Communications and Mobile Computing 2023 (February 3, 2023): 1–13. http://dx.doi.org/10.1155/2023/8545101.
Full textAbstract:
Since the concept of federated learning (FL) was proposed by Google in 2017, many applications have been combined with FL technology due to its outstanding performance in data integration, computing performance, privacy protection, etc. However, most traditional federated learning-based applications focus on image processing and natural language processing with few achievements in graph neural networks due to the graph’s nonindependent identically distributed (IID) nature. Representation learning on graph-structured data generates graph embedding, which helps machines understand graphs effectively. Meanwhile, privacy protection plays a more meaningful role in analyzing graph-structured data such as social networks. Hence, this paper proposes PPFL-GNN, a novel privacy-preserving federated graph neural network framework for node representation learning, which is a pioneer work for graph neural network-based federated learning. In PPFL-GNN, clients utilize a local graph dataset to generate graph embeddings and integrate information from other collaborative clients to utilize federated learning to produce more accurate representation results. More importantly, by integrating embedding alignment techniques in PPFL-GNN, we overcome the obstacles of federated learning on non-IID graph data and can further reduce privacy exposure by sharing preferred information.
44
Hu, Cheng, Scarlett Chen, and Zhe Wu. "Economic Model Predictive Control of Nonlinear Systems Using Online Learning of Neural Networks." Processes 11, no. 2 (January 20, 2023): 342. http://dx.doi.org/10.3390/pr11020342.
Full textAbstract:
This work focuses on the development of a Lyapunov-based economic model predictive control (LEMPC) scheme that utilizes recurrent neural networks (RNNs) with an online update to optimize the economic benefits of switched non-linear systems subject to a prescribed switching schedule. We first develop an initial offline-learning RNN using historical operational data, and then update RNNs with real-time data to improve model prediction accuracy. The generalized error bounds for RNNs updated online with independent and identically distributed (i.i.d.) and non-i.i.d. data samples are derived, respectively. Subsequently, by incorporating online updating RNNs within LEMPC, probabilistic closed-loop stability, and economic optimality are achieved simultaneously for switched non-linear systems accounting for the RNN generalized error bound. A chemical process example with scheduled mode transitions is used to demonstrate that the closed-loop economic performance under LEMPC can be improved using an online update of RNNs.
45
Zhou, Yueying, Gaoxiang Duan, Tianchen Qiu, Lin Zhang, Li Tian, Xiaoying Zheng, and Yongxin Zhu. "Personalized Federated Learning Incorporating Adaptive Model Pruning at the Edge." Electronics 13, no. 9 (May 1, 2024): 1738. http://dx.doi.org/10.3390/electronics13091738.
Full textAbstract:
Edge devices employing federated learning encounter several obstacles, including (1) the non-independent and identically distributed (Non-IID) nature of client data, (2) limitations due to communication bottlenecks, and (3) constraints on computational resources. To surmount the Non-IID data challenge, personalized federated learning has been introduced, which involves training tailored networks at the edge; nevertheless, these methods often exhibit inconsistency in performance. In response to these concerns, a novel framework for personalized federated learning that incorporates adaptive pruning of edge-side data is proposed in this paper. This approach, through a two-staged pruning process, creates customized models while ensuring strong generalization capabilities. Concurrently, by utilizing sparse models, it significantly condenses the model parameters, markedly diminishing both the computational burden and communication overhead on edge nodes. This method achieves a remarkable compression ratio of 3.7% on the Non-IID dataset FEMNIST, with the training accuracy remaining nearly unaffected. Furthermore, the total training duration is reduced by 46.4% when compared with the standard baseline method.
46
Zhao, Bo, Peng Sun, Tao Wang, and Keyu Jiang. "FedInv: Byzantine-Robust Federated Learning by Inversing Local Model Updates." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 8 (June 28, 2022): 9171–79. http://dx.doi.org/10.1609/aaai.v36i8.20903.
Full textAbstract:
Federated learning (FL) is a privacy-preserving distributed machine learning paradigm that enables multiple clients to collaboratively train statistical models without disclosing raw training data. However, the inaccessible local training data and uninspectable local training process make FL susceptible to various Byzantine attacks (e.g., data poisoning and model poisoning attacks), aiming to manipulate the FL model training process and degrade the model performance. Most of the existing Byzantine-robust FL schemes cannot effectively defend against stealthy poisoning attacks that craft poisoned models statistically similar to benign models. Things worsen when many clients are compromised or data among clients are highly non-independent and identically distributed (non-IID). In this work, to address these issues, we propose FedInv, a novel Byzantine-robust FL framework by inversing local model updates. Specifically, in each round of local model aggregation in FedInv, the parameter server first inverses the local model updates submitted by each client to generate a corresponding dummy dataset. Then, the server identifies those dummy datasets with exceptional Wasserstein distances from others and excludes the related local model updates from model aggregation. We conduct an exhaustive experimental evaluation of FedInv. The results demonstrate that FedInv significantly outperforms the existing robust FL schemes in defending against stealthy poisoning attacks under highly non-IID data partitions.
47
Yang, Dezhi, Xintong He, Jun Wang, Guoxian Yu, Carlotta Domeniconi, and Jinglin Zhang. "Federated Causality Learning with Explainable Adaptive Optimization." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 15 (March 24, 2024): 16308–15. http://dx.doi.org/10.1609/aaai.v38i15.29566.
Full textAbstract:
Discovering the causality from observational data is a crucial task in various scientific domains. With increasing awareness of privacy, data are not allowed to be exposed, and it is very hard to learn causal graphs from dispersed data, since these data may have different distributions. In this paper, we propose a federated causal discovery strategy (FedCausal) to learn the unified global causal graph from decentralized heterogeneous data. We design a global optimization formula to naturally aggregate the causal graphs from client data and constrain the acyclicity of the global graph without exposing local data. Unlike other federated causal learning algorithms, FedCausal unifies the local and global optimizations into a complete directed acyclic graph (DAG) learning process with a flexible optimization objective. We prove that this optimization objective has a high interpretability and can adaptively handle homogeneous and heterogeneous data. Experimental results on synthetic and real datasets show that FedCausal can effectively deal with non-independently and identically distributed (non-iid) data and has a superior performance.
48
Tursunboev, Jamshid, Yong-Sung Kang, Sung-Bum Huh, Dong-Woo Lim, Jae-Mo Kang, and Heechul Jung. "Hierarchical Federated Learning for Edge-Aided Unmanned Aerial Vehicle Networks." Applied Sciences 12, no. 2 (January 11, 2022): 670. http://dx.doi.org/10.3390/app12020670.
Full textAbstract:
Federated learning (FL) allows UAVs to collaboratively train a globally shared machine learning model while locally preserving their private data. Recently, the FL in edge-aided unmanned aerial vehicle (UAV) networks has drawn an upsurge of research interest due to a bursting increase in heterogeneous data acquired by UAVs and the need to build the global model with privacy; however, a critical issue is how to deal with the non-independent and identically distributed (non-i.i.d.) nature of heterogeneous data while ensuring the convergence of learning. To effectively address this challenging issue, this paper proposes a novel and high-performing FL scheme, namely, the hierarchical FL algorithm, for the edge-aided UAV network, which exploits the edge servers located in base stations as intermediate aggregators with employing commonly shared data. Experiment results demonstrate that the proposed hierarchical FL algorithm outperforms several baseline FL algorithms and exhibits better convergence behavior.
49
Lee, Yi-Chen, Wei-Che Chien, and Yao-Chung Chang. "FedDB: A Federated Learning Approach Using DBSCAN for DDoS Attack Detection." Applied Sciences 14, no. 22 (November 7, 2024): 10236. http://dx.doi.org/10.3390/app142210236.
Full textAbstract:
The rise of Distributed Denial of Service (DDoS) attacks on the internet has necessitated the development of robust and efficient detection mechanisms. DDoS attacks continue to present a significant threat, making it imperative to find efficient ways to detect and prevent these attacks promptly. Traditional machine learning approaches raise privacy concerns when handling sensitive data. In response, federated learning has emerged as a promising paradigm, allowing model training across decentralized devices without centralizing data. However, challenges such as the non-IID (Non-Independent and Identically Distributed) problem persist due to data distribution imbalances among devices. In this research, we propose personalized federated learning (PFL) as a solution for detecting DDoS attacks. PFL preserves data privacy by keeping sensitive information localized on individual devices during model training, thus addressing privacy concerns that are inherent in traditional approaches. In this paper, we propose federated learning with DBSCAN clustering (FedDB). By combining personalized training with model aggregation, our approach effectively mitigates the common challenge of non-IID data in federated learning setups. The integration of DBSCAN clustering further enhances our method by effectively handling data distribution imbalances and improving the overall detection accuracy. Results indicate that our proposed model improves performance, achieving relatively consistent accuracy across all clients, demonstrating that our method effectively overcomes the non-IID problem. Evaluation of our approach utilizes the CICDDOS2019 dataset. Through comprehensive experimentation, we demonstrate the efficacy of personalized federated learning in enhancing detection accuracy while safeguarding data privacy and mitigating non-IID concerns.
50
Sharma, Shagun, Kalpna Guleria, Ayush Dogra, Deepali Gupta, Sapna Juneja, Swati Kumari, and Ali Nauman. "A privacy-preserved horizontal federated learning for malignant glioma tumour detection using distributed data-silos." PLOS ONE 20, no. 2 (February 11, 2025): e0316543. https://doi.org/10.1371/journal.pone.0316543.
Full textAbstract:
Malignant glioma is the uncontrollable growth of cells in the spinal cord and brain that look similar to the normal glial cells. The most essential part of the nervous system is glial cells, which support the brain’s functioning prominently. However, with the evolution of glioma, tumours form that invade healthy tissues in the brain, leading to neurological impairment, seizures, hormonal dysregulation, and venous thromboembolism. Medical tests, including medical resonance imaging (MRI), computed tomography (CT) scans, biopsy, and electroencephalograms are used for early detection of glioma. However, these tests are expensive and may cause irritation and allergic reactions due to ionizing radiation. The deep learning models are highly optimal for disease prediction, however, the challenge associated with it is the requirement for substantial memory and storage to amalgamate the patient’s information at a centralized location. Additionally, it also has patient data-privacy concerns leading to anonymous information generalization, regulatory compliance issues, and data leakage challenges. Therefore, in the proposed work, a distributed and privacy-preserved horizontal federated learning-based malignant glioma disease detection model has been developed by employing 5 and 10 different clients’ architectures in independent and identically distributed (IID) and non-IID distributions. Initially, for developing this model, the collection of the MRI scans of non-tumour and glioma tumours has been done, which are further pre-processed by performing data balancing and image resizing. The configuration and development of the pre-trained MobileNetV2 base model have been performed, which is then applied to the federated learning(FL) framework. The configurations of this model have been kept as 0.001, Adam, 32, 10, 10, FedAVG, and 10 for learning rate, optimizer, batch size, local epochs, global epochs, aggregation, and rounds, respectively. The proposed model has provided the most prominent accuracy with 5 clients’ architecture as 99.76% and 99.71% for IID and non-IID distributions, respectively. These outcomes demonstrate that the model is highly optimized and generalizes the improved outcomes when compared to the state-of-the-art models.