Academic literature on the topic 'Adversarial robustness'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the lists of relevant articles, books, theses, conference reports, and other scholarly sources on the topic 'Adversarial robustness.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Journal articles on the topic "Adversarial robustness"
1
Doan, Bao Gia, Shuiqiao Yang, Paul Montague, Olivier De Vel, Tamas Abraham, Seyit Camtepe, Salil S. Kanhere, Ehsan Abbasnejad, and Damith C. Ranashinghe. "Feature-Space Bayesian Adversarial Learning Improved Malware Detector Robustness." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 12 (June 26, 2023): 14783–91. http://dx.doi.org/10.1609/aaai.v37i12.26727.
Full textAbstract:
We present a new algorithm to train a robust malware detector. Malware is a prolific problem and malware detectors are a front-line defense. Modern detectors rely on machine learning algorithms. Now, the adversarial objective is to devise alterations to the malware code to decrease the chance of being detected whilst preserving the functionality and realism of the malware. Adversarial learning is effective in improving robustness but generating functional and realistic adversarial malware samples is non-trivial. Because: i) in contrast to tasks capable of using gradient-based feedback, adversarial learning in a domain without a differentiable mapping function from the problem space (malware code inputs) to the feature space is hard; and ii) it is difficult to ensure the adversarial malware is realistic and functional. This presents a challenge for developing scalable adversarial machine learning algorithms for large datasets at a production or commercial scale to realize robust malware detectors. We propose an alternative; perform adversarial learning in the feature space in contrast to the problem space. We prove the projection of perturbed, yet valid malware, in the problem space into feature space will always be a subset of adversarials generated in the feature space. Hence, by generating a robust network against feature-space adversarial examples, we inherently achieve robustness against problem-space adversarial examples. We formulate a Bayesian adversarial learning objective that captures the distribution of models for improved robustness. To explain the robustness of the Bayesian adversarial learning algorithm, we prove that our learning method bounds the difference between the adversarial risk and empirical risk and improves robustness. We show that Bayesian neural networks (BNNs) achieve state-of-the-art results; especially in the False Positive Rate (FPR) regime. Adversarially trained BNNs achieve state-of-the-art robustness. Notably, adversarially trained BNNs are robust against stronger attacks with larger attack budgets by a margin of up to 15% on a recent production-scale malware dataset of more than 20 million samples. Importantly, our efforts create a benchmark for future defenses in the malware domain.
2
Zhou, Xiaoling, Nan Yang, and Ou Wu. "Combining Adversaries with Anti-adversaries in Training." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 9 (June 26, 2023): 11435–42. http://dx.doi.org/10.1609/aaai.v37i9.26352.
Full textAbstract:
Adversarial training is an effective learning technique to improve the robustness of deep neural networks. In this study, the influence of adversarial training on deep learning models in terms of fairness, robustness, and generalization is theoretically investigated under more general perturbation scope that different samples can have different perturbation directions (the adversarial and anti-adversarial directions) and varied perturbation bounds. Our theoretical explorations suggest that the combination of adversaries and anti-adversaries (samples with anti-adversarial perturbations) in training can be more effective in achieving better fairness between classes and a better tradeoff between robustness and generalization in some typical learning scenarios (e.g., noisy label learning and imbalance learning) compared with standard adversarial training. On the basis of our theoretical findings, a more general learning objective that combines adversaries and anti-adversaries with varied bounds on each training sample is presented. Meta learning is utilized to optimize the combination weights. Experiments on benchmark datasets under different learning scenarios verify our theoretical findings and the effectiveness of the proposed methodology.
3
Goldblum, Micah, Liam Fowl, Soheil Feizi, and Tom Goldstein. "Adversarially Robust Distillation." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 3996–4003. http://dx.doi.org/10.1609/aaai.v34i04.5816.
Full textAbstract:
Knowledge distillation is effective for producing small, high-performance neural networks for classification, but these small networks are vulnerable to adversarial attacks. This paper studies how adversarial robustness transfers from teacher to student during knowledge distillation. We find that a large amount of robustness may be inherited by the student even when distilled on only clean images. Second, we introduce Adversarially Robust Distillation (ARD) for distilling robustness onto student networks. In addition to producing small models with high test accuracy like conventional distillation, ARD also passes the superior robustness of large networks onto the student. In our experiments, we find that ARD student models decisively outperform adversarially trained networks of identical architecture in terms of robust accuracy, surpassing state-of-the-art methods on standard robustness benchmarks. Finally, we adapt recent fast adversarial training methods to ARD for accelerated robust distillation.
4
Tack, Jihoon, Sihyun Yu, Jongheon Jeong, Minseon Kim, Sung Ju Hwang, and Jinwoo Shin. "Consistency Regularization for Adversarial Robustness." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 8 (June 28, 2022): 8414–22. http://dx.doi.org/10.1609/aaai.v36i8.20817.
Full textAbstract:
Adversarial training (AT) is currently one of the most successful methods to obtain the adversarial robustness of deep neural networks. However, the phenomenon of robust overfitting, i.e., the robustness starts to decrease significantly during AT, has been problematic, not only making practitioners consider a bag of tricks for a successful training, e.g., early stopping, but also incurring a significant generalization gap in the robustness. In this paper, we propose an effective regularization technique that prevents robust overfitting by optimizing an auxiliary `consistency' regularization loss during AT. Specifically, we discover that data augmentation is a quite effective tool to mitigate the overfitting in AT, and develop a regularization that forces the predictive distributions after attacking from two different augmentations of the same instance to be similar with each other. Our experimental results demonstrate that such a simple regularization technique brings significant improvements in the test robust accuracy of a wide range of AT methods. More remarkably, we also show that our method could significantly help the model to generalize its robustness against unseen adversaries, e.g., other types or larger perturbations compared to those used during training. Code is available at https://github.com/alinlab/consistency-adversarial.
5
Liang, Youwei, and Dong Huang. "Large Norms of CNN Layers Do Not Hurt Adversarial Robustness." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 10 (May 18, 2021): 8565–73. http://dx.doi.org/10.1609/aaai.v35i10.17039.
Full textAbstract:
Since the Lipschitz properties of convolutional neural networks (CNNs) are widely considered to be related to adversarial robustness, we theoretically characterize the L-1 norm and L-infinity norm of 2D multi-channel convolutional layers and provide efficient methods to compute the exact L-1 norm and L-infinity norm. Based on our theorem, we propose a novel regularization method termed norm decay, which can effectively reduce the norms of convolutional layers and fully-connected layers. Experiments show that norm-regularization methods, including norm decay, weight decay, and singular value clipping, can improve generalization of CNNs. However, they can slightly hurt adversarial robustness. Observing this unexpected phenomenon, we compute the norms of layers in the CNNs trained with three different adversarial training frameworks and surprisingly find that adversarially robust CNNs have comparable or even larger layer norms than their non-adversarially robust counterparts. Furthermore, we prove that under a mild assumption, adversarially robust classifiers can be achieved using neural networks, and an adversarially robust neural network can have an arbitrarily large Lipschitz constant. For this reason, enforcing small norms on CNN layers may be neither necessary nor effective in achieving adversarial robustness. The code is available at https://github.com/youweiliang/norm_robustness.
6
Wang, Desheng, Weidong Jin, and Yunpu Wu. "Between-Class Adversarial Training for Improving Adversarial Robustness of Image Classification." Sensors 23, no. 6 (March 20, 2023): 3252. http://dx.doi.org/10.3390/s23063252.
Full textAbstract:
Deep neural networks (DNNs) have been known to be vulnerable to adversarial attacks. Adversarial training (AT) is, so far, the only method that can guarantee the robustness of DNNs to adversarial attacks. However, the robustness generalization accuracy gain of AT is still far lower than the standard generalization accuracy of an undefended model, and there is known to be a trade-off between the standard generalization accuracy and the robustness generalization accuracy of an adversarially trained model. In order to improve the robustness generalization and the standard generalization performance trade-off of AT, we propose a novel defense algorithm called Between-Class Adversarial Training (BCAT) that combines Between-Class learning (BC-learning) with standard AT. Specifically, BCAT mixes two adversarial examples from different classes and uses the mixed between-class adversarial examples to train a model instead of original adversarial examples during AT. We further propose BCAT+ which adopts a more powerful mixing method. BCAT and BCAT+ impose effective regularization on the feature distribution of adversarial examples to enlarge between-class distance, thus improving the robustness generalization and the standard generalization performance of AT. The proposed algorithms do not introduce any hyperparameters into standard AT; therefore, the process of hyperparameters searching can be avoided. We evaluate the proposed algorithms under both white-box attacks and black-box attacks using a spectrum of perturbation values on CIFAR-10, CIFAR-100, and SVHN datasets. The research findings indicate that our algorithms achieve better global robustness generalization performance than the state-of-the-art adversarial defense methods.
7
Bui, Anh Tuan, Trung Le, He Zhao, Paul Montague, Olivier DeVel, Tamas Abraham, and Dinh Phung. "Improving Ensemble Robustness by Collaboratively Promoting and Demoting Adversarial Robustness." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 8 (May 18, 2021): 6831–39. http://dx.doi.org/10.1609/aaai.v35i8.16843.
Full textAbstract:
Ensemble-based Adversarial Training is a principled approach to achieve robustness against adversarial attacks. An important technicality of this approach is to control the transferability of adversarial examples between ensemble members. We propose in this work a simple, but effective strategy to collaborate among committee models of an ensemble model. This is achieved via the secure and insecure sets defined for each model member on a given sample, hence help us to quantify and regularize the transferability. Consequently, our proposed framework provides the flexibility to reduce the adversarial transferability as well as promote the diversity of ensemble members, which are two crucial factors for better robustness in our ensemble approach. We conduct extensive and comprehensive experiments to demonstrate that our proposed method outperforms the state-of-the-art ensemble baselines, at the same time can detect a wide range of adversarial examples with a near perfect accuracy.
8
Li, Xin, Xiangrui Li, Deng Pan, and Dongxiao Zhu. "Improving Adversarial Robustness via Probabilistically Compact Loss with Logit Constraints." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 10 (May 18, 2021): 8482–90. http://dx.doi.org/10.1609/aaai.v35i10.17030.
Full textAbstract:
Convolutional neural networks (CNNs) have achieved state-of-the-art performance on various tasks in computer vision. However, recent studies demonstrate that these models are vulnerable to carefully crafted adversarial samples and suffer from a significant performance drop when predicting them. Many methods have been proposed to improve adversarial robustness (e.g., adversarial training and new loss functions to learn adversarially robust feature representations). Here we offer a unique insight into the predictive behavior of CNNs that they tend to misclassify adversarial samples into the most probable false classes. This inspires us to propose a new Probabilistically Compact (PC) loss with logit constraints which can be used as a drop-in replacement for cross-entropy (CE) loss to improve CNN's adversarial robustness. Specifically, PC loss enlarges the probability gaps between true class and false classes meanwhile the logit constraints prevent the gaps from being melted by a small perturbation. We extensively compare our method with the state-of-the-art using large scale datasets under both white-box and black-box attacks to demonstrate its effectiveness. The source codes are available at https://github.com/xinli0928/PC-LC.
9
Yang, Shuo, Tianyu Guo, Yunhe Wang, and Chang Xu. "Adversarial Robustness through Disentangled Representations." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 4 (May 18, 2021): 3145–53. http://dx.doi.org/10.1609/aaai.v35i4.16424.
Full textAbstract:
Despite the remarkable empirical performance of deep learning models, their vulnerability to adversarial examples has been revealed in many studies. They are prone to make a susceptible prediction to the input with imperceptible adversarial perturbation. Although recent works have remarkably improved the model's robustness under the adversarial training strategy, an evident gap between the natural accuracy and adversarial robustness inevitably exists. In order to mitigate this problem, in this paper, we assume that the robust and non-robust representations are two basic ingredients entangled in the integral representation. For achieving adversarial robustness, the robust representations of natural and adversarial examples should be disentangled from the non-robust part and the alignment of the robust representations can bridge the gap between accuracy and robustness. Inspired by this motivation, we propose a novel defense method called Deep Robust Representation Disentanglement Network (DRRDN). Specifically, DRRDN employs a disentangler to extract and align the robust representations from both adversarial and natural examples. Theoretical analysis guarantees the mitigation of the trade-off between robustness and accuracy with good disentanglement and alignment performance. Experimental results on benchmark datasets finally demonstrate the empirical superiority of our method.
10
Li, Zhuorong, Chao Feng, Minghui Wu, Hongchuan Yu, Jianwei Zheng, and Fanwei Zhu. "Adversarial robustness via attention transfer." Pattern Recognition Letters 146 (June 2021): 172–78. http://dx.doi.org/10.1016/j.patrec.2021.03.011.
Full textDissertations / Theses on the topic "Adversarial robustness"
1
Engstrom, Logan(Logan G. ). "Understanding the landscape of adversarial robustness." Thesis, Massachusetts Institute of Technology, 2019. https://hdl.handle.net/1721.1/123021.
Full textAbstract:
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019
Cataloged from student-submitted PDF version of thesis.
Includes bibliographical references (pages 108-115).
Despite their performance on standard tasks in computer vision, natural language processing and voice recognition, state-of-the-art models are pervasively vulnerable to adversarial examples. Adversarial examples are inputs that have been slightly perturbed--such that the semantic content is the same--as to cause malicious behavior in a classifier. The study of adversarial robustness has so far largely focused on perturbations bound in l[subscript p]-norms, in the case where the attacker knows the full model and controls exactly what input is sent to the classifier. However, this threat model is unrealistic in many respects. Models are vulnerable to classes of slight perturbations that are not captured by l[subscript p] bounds, adversaries realistically often will not have full model access, and in the physical world it is not possible to exactly control what image is sent to the classifier. In our exploration we successfully develop new algorithms and frameworks for exploiting vulnerabilities even in restricted threat models. We find that models are highly vulnerable to adversarial examples in these more realistic threat models, highlighting the necessity of further research to attain models that are truly robust and reliable.
by Logan Engstrom.
M. Eng.
M.Eng. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science
2
Zhang, Jeffrey M. Eng Massachusetts Institute of Technology. "Enhancing adversarial robustness of deep neural networks." Thesis, Massachusetts Institute of Technology, 2019. https://hdl.handle.net/1721.1/122994.
Full textAbstract:
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019
Cataloged from student-submitted PDF version of thesis.
Includes bibliographical references (pages 57-58).
Logit-based regularization and pretrain-then-tune are two approaches that have recently been shown to enhance adversarial robustness of machine learning models. In the realm of regularization, Zhang et al. (2019) proposed TRADES, a logit-based regularization optimization function that has been shown to improve upon the robust optimization framework developed by Madry et al. (2018) [14, 9]. They were able to achieve state-of-the-art adversarial accuracy on CIFAR10. In the realm of pretrain- then-tune models, Hendrycks el al. (2019) demonstrated that adversarially pretraining a model on ImageNet then adversarially tuning on CIFAR10 greatly improves the adversarial robustness of machine learning models. In this work, we propose Adversarial Regularization, another logit-based regularization optimization framework that surpasses TRADES in adversarial generalization. Furthermore, we explore the impact of trying different types of adversarial training on the pretrain-then-tune paradigm.
by Jeffry Zhang.
M. Eng.
M.Eng. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science
3
PINTOR, MAURA. "Towards Debugging and Improving Adversarial Robustness Evaluations ." Doctoral thesis, Università degli Studi di Cagliari, 2022. http://hdl.handle.net/11584/328882.
Full textAbstract:
Despite exhibiting unprecedented success in many application domains, machine‐learning models have been shown to be vulnerable to adversarial examples, i.e., maliciously perturbed inputs that are able to subvert their predictions at test time. Rigorous testing against such perturbations requires enumerating all possible outputs for all possible inputs, and despite impressive results in this field, these methods remain still difficult to scale to modern deep learning systems. For these reasons, empirical methods are often used. These adversarial perturbations are optimized via gradient descent, minimizing a loss function that aims to increase the probability of misleading the model’s predictions. To understand the sensitivity of the model to such attacks, and to counter the effects, machine-learning model designers craft worst-case adversarial perturbations and test them against the model they are evaluating. However, many of the proposed defenses have been shown to provide a false sense of security due to failures of the attacks, rather than actual improvements in the machine‐learning models’ robustness. They have been broken indeed under more rigorous evaluations. Although guidelines and best practices have been suggested to improve current adversarial robustness evaluations, the lack of automatic testing and debugging tools makes it difficult to apply these recommendations in a systematic and automated manner. To this end, we tackle three different challenges: (1) we investigate how adversarial robustness evaluations can be performed efficiently, by proposing a novel attack that can be used to find minimum-norm adversarial perturbations; (2) we propose a framework for debugging adversarial robustness evaluations, by defining metrics that reveal faulty evaluations as well as mitigations to patch the detected problems; and (3) we show how to employ a surrogate model for improving the success of transfer-based attacks, that are useful when gradient-based attacks are failing due to problems in the gradient information. To improve the quality of robustness evaluations, we propose a novel attack, referred to as Fast Minimum‐Norm (FMN) attack, which competes with state‐of‐the‐art attacks in terms of quality of the solution while outperforming them in terms of computational complexity and robustness to sub‐optimal configurations of the attack hyperparameters. These are all desirable characteristics of attacks used in robustness evaluations, as the aforementioned problems often arise from the use of sub‐optimal attack hyperparameters, including, e.g., the number of attack iterations, the step size, and the use of an inappropriate loss function. The correct refinement of these variables is often neglected, hence we designed a novel framework that helps debug the optimization process of adversarial examples, by means of quantitative indicators that unveil common problems and failures during the attack optimization process, e.g., in the configuration of the hyperparameters. Commonly accepted best practices suggest further validating the target model with alternative strategies, among which is the usage of a surrogate model to craft the adversarial examples to transfer to the model being evaluated is useful to check for gradient obfuscation. However, how to effectively create transferable adversarial examples is not an easy process, as many factors influence the success of this strategy. In the context of this research, we utilize a first-order model to show what are the main underlying phenomena that affect transferability and suggest best practices to create adversarial examples that transfer well to the target models.
4
Cina', Antonio Emanuele <1995>. "On the Robustness of Clustering Algorithms to Adversarial Attacks." Master's Degree Thesis, Università Ca' Foscari Venezia, 2019. http://hdl.handle.net/10579/15430.
Full textAbstract:
Machine learning is becoming more and more used by businesses and private users as an additional tool for aiding in decision making and automation processes. However, over the past few years, there has been an increased interest in research related to the security or robustness of learning models in presence of adversarial examples. It has been discovered that wisely crafted adversarial perturbations, unaffecting human judgment, can significantly affect the performance of the learning models. Adversarial machine learning studies how learning algorithms can be fooled by crafted adversarial examples. In many ways it is a recent research area, mainly focused on the analysis of supervised models, and only few works have been done in unsupervised settings. The adversarial analysis of this learning paradigm has become imperative as in recent years unsupervised learning has been increasingly adopted in multiple security and data analysis applications. In this thesis, we are going to show how an attacker can craft poisoning perturbations on the input data for reaching target goals. In particular, we are going to analyze the robustness of two fundamental applications of unsupervised learning, feature-based data clustering and image segmentation. We are going to show how an attacker can craft poisoning perturbations against the two applications. We choose 3 very well known clustering algorithms (K-Means, Spectral and Dominant Sets clustering) and multiple datasets for analyzing the robustness provided by them against adversarial examples, crafted with our designed algorithms.
5
Allenet, Thibault. "Quantization and adversarial robustness of embedded deep neural networks." Electronic Thesis or Diss., Université de Rennes (2023-....), 2023. https://ged.univ-rennes1.fr/nuxeo/site/esupversions/5f524c49-7a4a-4724-ae77-9afe383b7c3c.
Full textAbstract:
Les réseaux de neurones convolutifs et les réseaux neurones récurrents (RNN) ont été largement utilisés dans de nombreux domaines tels que la vision par ordinateur, le traitement naturel du langage et le traitement du signal. Néanmoins, la charge de calcul et le besoin en bande passante mémoire impliqués dans l'inférence des réseaux de neurones profonds empêchent souvent leur déploiement sur des cibles embarquées à faible ressources. De plus, la vulnérabilité des réseaux de neurones profonds à de petites perturbations sur les entrées remet en question leur déploiement pour des applications impliquant des décisions de haute criticité. Pour relever ces défis, cette thèse propose deux principales contributions. D'une part, nous proposons des méthodes de compression pour rendre les réseaux de neurones profonds plus adaptés aux systèmes embarqués ayant de faibles ressources. D'autre part, nous proposons une nouvelle stratégie pour rendre les réseaux de neurones profonds plus robustes aux attaques adverses en tenant compte des ressources limitées des systèmes embarqués. Dans un premier temps, nous présentons une revue de la littérature sur des principes et des outils de bases de l'apprentissage profond, des types de réseaux de neurones reconnus et un état de l'art sur des méthodes de compression de réseaux de neurones. Ensuite, nous présentons deux contributions autour de la compression des réseaux de neurones profonds : une étude de transférabilité du Lottery Ticket sur les RNN et une méthode de quantification à l’apprentissage. L’étude de transférabilité du Lottery Ticket sur les RNN analyse la convergence des RNN et étudie son impact sur l'élagage des paramètres pour des taches de classification d'images et de modélisation du langage. Nous proposons aussi une méthode de prétraitement basée sur le sous-échantillonnage des données qui permet une convergence plus rapide des LSTM tout en préservant les performances applicatives. Avec la méthode Disentangled Loss Quantization Aware Training (DL-QAT), nous proposons d'améliorer une méthode de quantification avancée avec des fonctions de coût favorables à la quantification afin d'atteindre des paramètres binaires. Les expériences sur ImageNet-1k avec DL-QAT montrent une amélioration de près de 1 % sur la précision du score de ResNet-18 avec des poids binaires et des activations de 2 bits. Il apparaît clairement que DL-QAT fournit le meilleur profil du compromis entre l'empreinte mémoire et la performance applicative. Ce travail étudie ensuite la robustesse des réseaux de neurones face aux attaques adverses. Après avoir présenté l'état de l'art sur les attaques adverses et les mécanismes de défense, nous proposons le mécanisme de défense Ensemble Hash Defense (EHD). EHD permet une meilleure résistance aux attaques adverses basées sur l'approximation du gradient tout en préservant les performances de l'application et en ne nécessitant qu'une surcharge de mémoire au moment de l'inférence. Dans la meilleure configuration, notre système réalise des gains de robustesse significatifs par rapport aux modèles de base et à une approche de robustesse basée sur la fonction de coût. De plus, le principe de l'EHD la rend complémentaire à d'autres méthodes d'optimisation robuste qui permettraient d'améliorer encore la robustesse du système final. Dans la perspective de l'inférence sur cible embarquée, la surcharge mémoire introduite par l'EHD peut être réduite par la quantification ou le partage de poids. En conclusion, les travaux de cette thèse ont proposé des méthodes de compression de réseaux de neurones et un système de défense pour résoudre des défis importants, à savoir comment rendre les réseaux de neurones profonds plus robustes face aux attaques adverses et plus faciles à déployer sur les plateformes à ressources limitées. Ces travaux réduisent davantage l'écart entre l'état de l'art des réseaux neurones profonds et leur exécution sur des cibles embarquées à faible ressourcesConvolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have been broadly used in many fields such as computer vision, natural language processing and signal processing. Nevertheless, the computational workload and the heavy memory bandwidth involved in deep neural networks inference often prevents their deployment on low-power embedded devices. Moreover, deep neural networks vulnerability towards small input perturbations questions their deployment for applications involving high criticality decisions. This PhD research project objective is twofold. On the one hand, it proposes compression methods to make deep neural networks more suitable for embedded systems with low computing resources and memory requirements. On the other hand, it proposes a new strategy to make deep neural networks more robust towards attacks based on crafted inputs with the perspective to infer on edge. We begin by introducing common concepts for training neural networks, convolutional neural networks, recurrent neural networks and review the state of the art neural on deep neural networks compression methods. After this literature review we present two main contributions on compressing deep neural networks: an investigation of lottery tickets on RNNs and Disentangled Loss Quantization Aware Training (DL-QAT) on CNNs. The investigation of lottery tickets on RNNs analyze the convergence of RNNs and study its impact when subject to pruning on image classification and language modelling. Then we present a pre-processing method based on data sub-sampling that enables faster convergence of LSTM while preserving application performance. With the Disentangled Loss Quantization Aware Training (DL-QAT) method, we propose to further improve an advanced quantization method with quantization friendly loss functions to reach low bit settings like binary parameters where the application performance is the most impacted. Experiments on ImageNet-1k with DL-QAT show improvements by nearly 1\% on the top-1 accuracy of ResNet-18 with binary weights and 2-bit activations, and also show the best profile of memory footprint over accuracy when compared with other state-of-the art methods. This work then studies neural networks robustness toward adversarial attacks. After introducing the state of the art on adversarial attacks and defense mechanisms, we propose the Ensemble Hash Defense (EHD) defense mechanism. EHD enables better resilience to adversarial attacks based on gradient approximation while preserving application performance and only requiring a memory overhead at inference time. In the best configuration, our system achieves significant robustness gains compared to baseline models and a loss function-driven approach. Moreover, the principle of EHD makes it complementary to other robust optimization methods that would further enhance the robustness of the final system and compression methods. With the perspective of edge inference, the memory overhead introduced by EHD can be reduced with quantization or weight sharing. The contributions in this thesis have concerned optimization methods and a defense system to solve an important challenge, that is, how to make deep neural networks more robust towards adversarial attacks and easier to deployed on the resource limited platforms. This work further reduces the gap between state of the art deep neural networks and their execution on edge devices
6
Ebrahimi, Javid. "Robustness of Neural Networks for Discrete Input: An Adversarial Perspective." Thesis, University of Oregon, 2019. http://hdl.handle.net/1794/24535.
Full textAbstract:
In the past few years, evaluating on adversarial examples has become a standard
procedure to measure robustness of deep learning models. Literature on adversarial
examples for neural nets has largely focused on image data, which are represented as
points in continuous space. However, a vast proportion of machine learning models
operate on discrete input, and thus demand a similar rigor in understanding their
vulnerabilities and robustness. We study robustness of neural network architectures
for textual and graph inputs, through the lens of adversarial input perturbations.
We will cover methods for both attacks and defense; we will focus on 1) addressing
challenges in optimization for creating adversarial perturbations for discrete data;
2) evaluating and contrasting white-box and black-box adversarial examples; and 3)
proposing efficient methods to make the models robust against adversarial attacks.
7
CARBONE, GINEVRA. "Robustness and Interpretability of Neural Networks’ Predictions under Adversarial Attacks." Doctoral thesis, Università degli Studi di Trieste, 2023. https://hdl.handle.net/11368/3042163.
Full textAbstract:
Le reti neurali profonde (DNNs) sono potenti modelli predittivi, che superano le capacità umane in una varietà di task. Imparano sistemi decisionali complessi e flessibili dai dati a disposizione e raggiungono prestazioni eccezionali in molteplici campi di apprendimento automatico, dalle applicazioni dell'intelligenza artificiale, come il riconoscimento di immagini, parole e testi, alle scienze più tradizionali, tra cui medicina, fisica e biologia. Nonostante i risultati eccezionali, le prestazioni elevate e l’alta precisione predittiva non sono sufficienti per le applicazioni nel mondo reale, specialmente in ambienti critici per la sicurezza, dove l'utilizzo dei DNNs è fortemente limitato dalla loro natura black-box. Vi è una crescente necessità di comprendere come vengono eseguite le predizioni, fornire stime di incertezza, garantire robustezza agli attacchi avversari e prevenire comportamenti indesiderati.
Anche le migliori architetture sono vulnerabili a piccole perturbazioni nei dati di input, note come attacchi avversari: manipolazioni malevole degli input che sono percettivamente indistinguibili dai campioni originali ma sono in grado di ingannare il modello in predizioni errate. In questo lavoro, dimostriamo che tale fragilità è correlata alla geometria del manifold dei dati ed è quindi probabile che sia una caratteristica intrinseca delle predizioni dei DNNs. Questa
condizione suggerisce una possibile direzione al fine di ottenere robustezza agli attacchi: studiamo la geometria degli attacchi avversari nel limite di un numero infinito di dati e di pesi per le reti neurali Bayesiane, dimostrando che, in questo limite, sono immuni agli attacchi avversari gradient-based. Inoltre, proponiamo alcune tecniche di training per migliorare la robustezza delle architetture deterministiche. In particolare, osserviamo sperimentalmente che ensembles di reti neurali addestrati su proiezioni casuali degli input originali in spazi basso-dimensionali sono più resistenti agli attacchi.
Successivamente, ci concentriamo sul problema dell'interpretabilità delle predizioni delle reti nel contesto delle saliency-based explanations. Analizziamo la stabilità delle explanations soggette ad attacchi avversari e dimostriamo che, nel limite di un numero infinito di dati e di pesi, le interpretazioni Bayesiane sono più stabili di quelle fornite dalle reti deterministiche. Confermiamo questo comportamento in modo sperimentale nel regime di un numero finito di dati.
Infine, introduciamo il concetto di attacco avversario alle sequenze di amminoacidi per protein Language Models (LM). I modelli di Deep Learning per la predizione della struttura delle proteine, come AlphaFold2, sfruttano le architetture Transformer e il loro meccanismo di attention per catturare le proprietà strutturali e funzionali delle sequenze di amminoacidi. Nonostante l'elevata precisione delle predizioni, perturbazioni biologicamente piccole delle sequenze di input, o anche mutazioni di un singolo amminoacido, possono portare a strutture 3D sostanzialmente diverse. Al contempo, i protein LMs sono insensibili alle mutazioni che inducono misfolding o disfunzione (ad esempio le missense mutations). In particolare, le predizioni delle coordinate 3D non rivelano l'effetto di unfolding indotto da queste mutazioni. Pertanto, esiste un'evidente incoerenza tra l'importanza biologica delle mutazioni e il conseguente cambiamento nella predizione strutturale. Ispirati da questo problema, introduciamo il concetto di perturbazione avversaria delle sequenze proteiche negli embedding continui dei protein LMs. Il nostro metodo utilizza i valori di attention per rilevare le posizioni degli amminoacidi più vulnerabili nelle sequenze di input. Le mutazioni avversarie sono biologicamente diverse dalle sequenze di riferimento e sono in grado di alterare in modo significativo le strutture 3D.Deep Neural Networks (DNNs) are powerful predictive models, exceeding human capabilities in a variety of tasks. They learn complex and flexible decision systems from the available data and achieve exceptional performances in multiple machine learning fields, spanning from applications in artificial intelligence, such as image, speech and text recognition, to the more traditional sciences, including medicine, physics and biology. Despite the outstanding achievements, high performance and high predictive accuracy are not sufficient for real-world applications, especially in safety-critical settings, where the usage of DNNs is severely limited by their black-box nature. There is an increasing need to understand how predictions are performed, to provide uncertainty estimates, to guarantee robustness to malicious attacks and to prevent unwanted behaviours. State-of-the-art DNNs are vulnerable to small perturbations in the input data, known as adversarial attacks: maliciously crafted manipulations of the inputs that are perceptually indistinguishable from the original samples but are capable of fooling the model into incorrect predictions. In this work, we prove that such brittleness is related to the geometry of the data manifold and is therefore likely to be an intrinsic feature of DNNs’ predictions. This negative condition suggests a possible direction to overcome such limitation: we study the geometry of adversarial attacks in the large-data, overparameterized limit for Bayesian Neural Networks and prove that, in this limit, they are immune to gradient-based adversarial attacks. Furthermore, we propose some training techniques to improve the adversarial robustness of deterministic architectures. In particular, we experimentally observe that ensembles of NNs trained on random projections of the original inputs into lower dimensional spaces are more resilient to the attacks. Next, we focus on the problem of interpretability of NNs’ predictions in the setting of saliency-based explanations. We analyze the stability of the explanations under adversarial attacks on the inputs and we prove that, in the large-data and overparameterized limit, Bayesian interpretations are more stable than those provided by deterministic networks. We validate this behaviour in multiple experimental settings in the finite data regime. Finally, we introduce the concept of adversarial perturbations of amino acid sequences for protein Language Models (LMs). Deep Learning models for protein structure prediction, such as AlphaFold2, leverage Transformer architectures and their attention mechanism to capture structural and functional properties of amino acid sequences. Despite the high accuracy of predictions, biologically small perturbations of the input sequences, or even single point mutations, can lead to substantially different 3d structures. On the other hand, protein language models are insensitive to mutations that induce misfolding or dysfunction (e.g. missense mutations). Precisely, predictions of the 3d coordinates do not reveal the structure-disruptive effect of these mutations. Therefore, there is an evident inconsistency between the biological importance of mutations and the resulting change in structural prediction. Inspired by this problem, we introduce the concept of adversarial perturbation of protein sequences in continuous embedding spaces of protein language models. Our method relies on attention scores to detect the most vulnerable amino acid positions in the input sequences. Adversarial mutations are biologically diverse from their references and are able to significantly alter the resulting 3D structures.
8
Itani, Aashish. "COMPARISON OF ADVERSARIAL ROBUSTNESS OF ANN AND SNN TOWARDS BLACKBOX ATTACKS." OpenSIUC, 2021. https://opensiuc.lib.siu.edu/theses/2864.
Full textAbstract:
n recent years, the vulnerability of neural networks to adversarial samples has gained wide attention from machine learning and deep learning communities. Addition of small and imperceptible perturbations to the input samples can cause neural network models to make incorrect prediction with high confidence. As the employment of neural networks on safety critical application is rising, this vulnerability of traditional neural networks to the adversarial samples demand for more robust alternative neural network models. Spiking Neural Network (SNN), is a special class of ANN, which mimics the brain functionality by using spikes for information processing. The known advantages of SNN include fast inference, low power consumption and biologically plausible information processing. In this work, we experiment on the adversarial robustness of the SNN as compared to traditional ANN, and figure out if SNN can be a candidate to solve the security problems faced by ANN.
9
Yang, Shuo. "Adversarial Data Generation for Robust Deep Learning." Thesis, The University of Sydney, 2021. https://hdl.handle.net/2123/27291.
Full textAbstract:
The success of deep learning is inseparable from the support of massive data. The vast amount of high-quality data is one of the most crucial prerequisites to train a robust deep learning model. However, the raw data collected in the real world usually has some defects which prevent their direct usage for training. In this thesis, we propose to employ the generative adversarial learning framework to generate high-quality data for training robust deep learning models.
We focus on the generation of two of the most common data types: time series and images. For time series, we propose an adversarial recurrent time series imputation model to reconstruct the incomplete time series. Specifically, our model modifies the traditional Recurrent Neural Network (RNN) architecture to better capture the temporal dependencies and feature correlations and legitimately combines them to impute the missing part. Besides, we employ an element-wise generative adversarial learning framework to train the modified recurrent structure to generate more realistic data. Experiments on several real-world time series datasets demonstrate encouraging improvement of our model on the imputation performance as well as the classification accuracy.
For image classification, recent studies find that deep learning models trained with natural images can be vulnerable to a kind of adversarially generated perturbations, namely adversarial perturbations. In this work, we improve the adversarial example generation and traditional AT framework from four aspects: adaptive perturbation size, diversified adversarial examples, stabilizing AT with massive contrastive adversaries, and adversarial robustness through representation disentanglement. The models presented above all demonstrate empirical remarkable performance improvement in terms of the quality of adversarial examples and model robustness.
10
Uličný, Matej. "Methods for Increasing Robustness of Deep Convolutional Neural Networks." Thesis, Högskolan i Halmstad, Akademin för informationsteknologi, 2015. http://urn.kb.se/resolve?urn=urn:nbn:se:hh:diva-29734.
Full textAbstract:
Recent discoveries uncovered flaws in machine learning algorithms such as deep neural networks. Deep neural networks seem vulnerable to small amounts of non-random noise, created by exploiting the input to output mapping of the network. Applying this noise to an input image drastically decreases classication performance. Such image is referred to as an adversarial example. The purpose of this thesis is to examine how known regularization/robustness methods perform on adversarial examples. The robustness methods: dropout, low-pass filtering, denoising autoencoder, adversarial training and committees have been implemented, combined and tested. For the well-known benchmark, the MNIST (Mixed National Institute of Standards and Technology) dataset, the best combination of robustness methods has been found. Emerged from the results of the experiments, ensemble of models trained on adversarial examples is considered to be the best approach for MNIST. Harmfulness of the adversarial noise and some robustness experiments are demonstrated on CIFAR10 (The Canadian Institute for Advanced Research) dataset as well. Apart from robustness tests, the thesis describes experiments with human classification performance on noisy images and the comparison with performance of deep neural network.
Books on the topic "Adversarial robustness"
1
Adversarial Robustness for Machine Learning. Elsevier, 2023. http://dx.doi.org/10.1016/c2020-0-01078-9.
Full text
2
Adversarial Robustness for Machine Learning Models. Elsevier Science & Technology Books, 2022.
Find full text
3
Adversarial Robustness for Machine Learning Models. Elsevier Science & Technology, 2022.
Find full text
4
Machine Learning Algorithms: Adversarial Robustness in Signal Processing. Springer International Publishing AG, 2022.
Find full textBook chapters on the topic "Adversarial robustness"
1
Göpfert, Christina, Jan Philip Göpfert, and Barbara Hammer. "Adversarial Robustness Curves." In Machine Learning and Knowledge Discovery in Databases, 172–79. Cham: Springer International Publishing, 2020. http://dx.doi.org/10.1007/978-3-030-43823-4_15.
Full text
2
Mao, Chengzhi, Amogh Gupta, Vikram Nitin, Baishakhi Ray, Shuran Song, Junfeng Yang, and Carl Vondrick. "Multitask Learning Strengthens Adversarial Robustness." In Computer Vision – ECCV 2020, 158–74. Cham: Springer International Publishing, 2020. http://dx.doi.org/10.1007/978-3-030-58536-5_10.
Full text
3
Ríos Insua, David, Fabrizio Ruggeri, Cesar Alfaro, and Javier Gomez. "Robustness for Adversarial Risk Analysis." In Robustness Analysis in Decision Aiding, Optimization, and Analytics, 39–58. Cham: Springer International Publishing, 2016. http://dx.doi.org/10.1007/978-3-319-33121-8_3.
Full text
4
Günnemann, Stephan. "Graph Neural Networks: Adversarial Robustness." In Graph Neural Networks: Foundations, Frontiers, and Applications, 149–76. Singapore: Springer Singapore, 2022. http://dx.doi.org/10.1007/978-981-16-6054-2_8.
Full text
5
Cao, Houze, and Meng Xue. "Adversarial Training for Better Robustness." In Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, 75–84. Cham: Springer Nature Switzerland, 2023. http://dx.doi.org/10.1007/978-3-031-35982-8_6.
Full text
6
Reyes-Amezcua, Ivan, Gilberto Ochoa-Ruiz, and Andres Mendez-Vazquez. "Adversarial Robustness on Artificial Intelligence." In What AI Can Do, 419–31. Boca Raton: Chapman and Hall/CRC, 2023. http://dx.doi.org/10.1201/b23345-24.
Full text
7
Komiyama, Ryota, and Motonobu Hattori. "Adversarial Minimax Training for Robustness Against Adversarial Examples." In Neural Information Processing, 690–99. Cham: Springer International Publishing, 2018. http://dx.doi.org/10.1007/978-3-030-04179-3_61.
Full text
8
Tang, Keke, Tianrui Lou, Xu He, Yawen Shi, Peican Zhu, and Zhaoquan Gu. "Enhancing Adversarial Robustness via Anomaly-aware Adversarial Training." In Knowledge Science, Engineering and Management, 328–42. Cham: Springer Nature Switzerland, 2023. http://dx.doi.org/10.1007/978-3-031-40283-8_28.
Full text
9
Zhang, Chaoning, Kang Zhang, Chenshuang Zhang, Axi Niu, Jiu Feng, Chang D. Yoo, and In So Kweon. "Decoupled Adversarial Contrastive Learning for Self-supervised Adversarial Robustness." In Lecture Notes in Computer Science, 725–42. Cham: Springer Nature Switzerland, 2022. http://dx.doi.org/10.1007/978-3-031-20056-4_42.
Full text
10
Vaishnavi, Pratik, Tianji Cong, Kevin Eykholt, Atul Prakash, and Amir Rahmati. "Can Attention Masks Improve Adversarial Robustness?" In Communications in Computer and Information Science, 14–22. Cham: Springer International Publishing, 2020. http://dx.doi.org/10.1007/978-3-030-62144-5_2.
Full textConference papers on the topic "Adversarial robustness"
1
Bai, Tao, Jinqi Luo, Jun Zhao, Bihan Wen, and Qian Wang. "Recent Advances in Adversarial Training for Adversarial Robustness." In Thirtieth International Joint Conference on Artificial Intelligence {IJCAI-21}. California: International Joint Conferences on Artificial Intelligence Organization, 2021. http://dx.doi.org/10.24963/ijcai.2021/591.
Full textAbstract:
Adversarial training is one of the most effective approaches for deep learning models to defend against adversarial examples. Unlike other defense strategies, adversarial training aims to enhance the robustness of models intrinsically. During the past few years, adversarial training has been studied and discussed from various aspects, which deserves a comprehensive review. For the first time in this survey, we systematically review the recent progress on adversarial training for adversarial robustness with a novel taxonomy. Then we discuss the generalization problems in adversarial training from three perspectives and highlight the challenges which are not fully tackled. Finally, we present potential future directions.
2
Hsiung, Lei, Yun-Yun Tsai, Pin-Yu Chen, and Tsung-Yi Ho. "CARBEN: Composite Adversarial Robustness Benchmark." In Thirty-First International Joint Conference on Artificial Intelligence {IJCAI-22}. California: International Joint Conferences on Artificial Intelligence Organization, 2022. http://dx.doi.org/10.24963/ijcai.2022/851.
Full textAbstract:
Prior literature on adversarial attack methods has mainly focused on attacking with and defending against a single threat model, e.g., perturbations bounded in Lp ball. However, multiple threat models can be combined into composite perturbations. One such approach, composite adversarial attack (CAA), not only expands the perturbable space of the image, but also may be overlooked by current modes of robustness evaluation. This paper demonstrates how CAA's attack order affects the resulting image, and provides real-time inferences of different models, which will facilitate users' configuration of the parameters of the attack level and their rapid evaluation of model prediction. A leaderboard to benchmark adversarial robustness against CAA is also introduced.
3
Guo, Xiaohui, Richong Zhang, Yaowei Zheng, and Yongyi Mao. "Robust Regularization with Adversarial Labelling of Perturbed Samples." In Thirtieth International Joint Conference on Artificial Intelligence {IJCAI-21}. California: International Joint Conferences on Artificial Intelligence Organization, 2021. http://dx.doi.org/10.24963/ijcai.2021/343.
Full textAbstract:
Recent researches have suggested that the predictive accuracy of neural network may contend with its adversarial robustness. This presents challenges in designing effective regularization schemes that also provide strong adversarial robustness. Revisiting Vicinal Risk Minimization (VRM) as a unifying regularization principle, we propose Adversarial Labelling of Perturbed Samples (ALPS) as a regularization scheme that aims at improving the generalization ability and adversarial robustness of the trained model. ALPS trains neural networks with synthetic samples formed by perturbing each authentic input sample towards another one along with an adversarially assigned label. The ALPS regularization objective is formulated as a min-max problem, in which the outer problem is minimizing an upper-bound of the VRM loss, and the inner problem is L1-ball constrained adversarial labelling on perturbed sample. The analytic solution to the induced inner maximization problem is elegantly derived, which enables computational efficiency. Experiments on the SVHN, CIFAR-10, CIFAR-100 and Tiny-ImageNet datasets show that the ALPS has a state-of-the-art regularization performance while also serving as an effective adversarial training scheme.
4
Byun, Junyoung, Hyojun Go, Seungju Cho, and Changick Kim. "Exploiting Doubly Adversarial Examples for Improving Adversarial Robustness." In 2022 IEEE International Conference on Image Processing (ICIP). IEEE, 2022. http://dx.doi.org/10.1109/icip46576.2022.9897374.
Full text
5
Cheng, Minhao, Qi Lei, Pin-Yu Chen, Inderjit Dhillon, and Cho-Jui Hsieh. "CAT: Customized Adversarial Training for Improved Robustness." In Thirty-First International Joint Conference on Artificial Intelligence {IJCAI-22}. California: International Joint Conferences on Artificial Intelligence Organization, 2022. http://dx.doi.org/10.24963/ijcai.2022/95.
Full textAbstract:
Adversarial training has become one of the most effective methods for improving robustness of neural networks. However, it often suffers from poor generalization on both clean and perturbed data. Current robust training method always use a uniformed perturbation strength for every samples to generate adversarial examples during model training for improving adversarial robustness. However, we show it would lead worse training and generalizaiton error and forcing the prediction to match one-hot label. In this paper, therefore, we propose a new algorithm, named Customized Adversarial Training (CAT), which adaptively customizes the perturbation level and the corresponding label for each training sample in adversarial training. We first show theoretically the CAT scheme improves the generalization. Also, through extensive experiments, we show that the proposed algorithm achieves better clean and robust accuracy than previous adversarial training methods. The full version of this paper is available at https://arxiv.org/abs/2002.06789.
6
Megyeri, Istvan, Istvan Hegedus, and Mark Jelasity. "Adversarial Robustness of Model Sets." In 2020 International Joint Conference on Neural Networks (IJCNN). IEEE, 2020. http://dx.doi.org/10.1109/ijcnn48605.2020.9206656.
Full text
7
Stutz, David, Matthias Hein, and Bernt Schiele. "Disentangling Adversarial Robustness and Generalization." In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2019. http://dx.doi.org/10.1109/cvpr.2019.00714.
Full text
8
Rozsa, Andras, Manuel Gunther, and Terrance Boult. "Adversarial Robustness: Softmax versus Openmax." In British Machine Vision Conference 2017. British Machine Vision Association, 2017. http://dx.doi.org/10.5244/c.31.156.
Full text
9
Hosseini, Hossein, Sreeram Kannan, and Radha Poovendran. "Dropping Pixels for Adversarial Robustness." In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 2019. http://dx.doi.org/10.1109/cvprw.2019.00017.
Full text
10
Ben-Eliezer, Omri, and Eylon Yogev. "The Adversarial Robustness of Sampling." In SIGMOD/PODS '20: International Conference on Management of Data. New York, NY, USA: ACM, 2020. http://dx.doi.org/10.1145/3375395.3387643.
Full textReports on the topic "Adversarial robustness"
1
Rudner, Tim, and Helen Toner. Key Concepts in AI Safety: Robustness and Adversarial Examples. Center for Security and Emerging Technology, March 2021. http://dx.doi.org/10.51593/20190041.
Full textAbstract:
This paper is the second installment in a series on “AI safety,” an area of machine learning research that aims to identify causes of unintended behavior in machine learning systems and develop tools to ensure these systems work safely and reliably. The first paper in the series, “Key Concepts in AI Safety: An Overview,” described three categories of AI safety issues: problems of robustness, assurance, and specification. This paper introduces adversarial examples, a major challenge to robustness in modern machine learning systems.