Journal articles on the topic 'Model-agnostic methods'
To see the other types of publications on this topic, follow the link: Model-agnostic methods.
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Model-agnostic methods.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Su, Houcheng, Weihao Luo, Daixian Liu, Mengzhu Wang, Jing Tang, Junyang Chen, Cong Wang, and Zhenghan Chen. "Sharpness-Aware Model-Agnostic Long-Tailed Domain Generalization." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 13 (March 24, 2024): 15091–99. http://dx.doi.org/10.1609/aaai.v38i13.29431.
Abstract:
Domain Generalization (DG) aims to improve the generalization ability of models trained on a specific group of source domains, enabling them to perform well on new, unseen target domains. Recent studies have shown that methods that converge to smooth optima can enhance the generalization performance of supervised learning tasks such as classification. In this study, we examine the impact of smoothness-enhancing formulations on domain adversarial training, which combines task loss and adversarial loss objectives. Our approach leverages the fact that converging to a smooth minimum with respect to task loss can stabilize the task loss and lead to better performance on unseen domains. Furthermore, we recognize that the distribution of objects in the real world often follows a long-tailed class distribution, resulting in a mismatch between machine learning models and our expectations of their performance on all classes of datasets with long-tailed class distributions. To address this issue, we consider the domain generalization problem from the perspective of the long-tail distribution and propose using the maximum square loss to balance different classes which can improve model generalizability. Our method's effectiveness is demonstrated through comparisons with state-of-the-art methods on various domain generalization datasets. Code: https://github.com/bamboosir920/SAMALTDG.
2
Pugnana, Andrea, and Salvatore Ruggieri. "A Model-Agnostic Heuristics for Selective Classification." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 8 (June 26, 2023): 9461–69. http://dx.doi.org/10.1609/aaai.v37i8.26133.
Abstract:
Selective classification (also known as classification with reject option) conservatively extends a classifier with a selection function to determine whether or not a prediction should be accepted (i.e., trusted, used, deployed). This is a highly relevant issue in socially sensitive tasks, such as credit scoring. State-of-the-art approaches rely on Deep Neural Networks (DNNs) that train at the same time both the classifier and the selection function. These approaches are model-specific and computationally expensive. We propose a model-agnostic approach, as it can work with any base probabilistic binary classification algorithm, and it can be scalable to large tabular datasets if the base classifier is so. The proposed algorithm, called SCROSS, exploits a cross-fitting strategy and theoretical results for quantile estimation to build the selection function. Experiments on real-world data show that SCROSS improves over existing methods.
3
Satrya, Wahyu Fadli, and Ji-Hoon Yun. "Combining Model-Agnostic Meta-Learning and Transfer Learning for Regression." Sensors 23, no. 2 (January 4, 2023): 583. http://dx.doi.org/10.3390/s23020583.
Abstract:
For cases in which a machine learning model needs to be adapted to a new task, various approaches have been developed, including model-agnostic meta-learning (MAML) and transfer learning. In this paper, we investigate how the differences in the data distributions between the old tasks and the new target task impact performance in regression problems. By performing experiments, we discover that these differences greatly affect the relative performance of different adaptation methods. Based on this observation, we develop ensemble schemes combining multiple adaptation methods that can handle a wide range of data distribution differences between the old and new tasks, thus offering more stable performance for a wide range of tasks. For evaluation, we consider three regression problems of sinusoidal fitting, virtual reality motion prediction, and temperature forecasting. The evaluation results demonstrate that the proposed ensemble schemes achieve the best performance among the considered methods in most cases.
4
Atallah, Rasha Ragheb, Amirrudin Kamsin, Maizatul Akmar Ismail, and Ahmad Sami Al-Shamayleh. "NEURAL NETWORK WITH AGNOSTIC META-LEARNING MODEL FOR FACE-AGING RECOGNITION." Malaysian Journal of Computer Science 35, no. 1 (January 31, 2022): 56–69. http://dx.doi.org/10.22452/mjcs.vol35no1.4.
Abstract:
Face recognition is one of the most approachable and accessible authentication methods. It is also accepted by users, as it is non-invasive. However, aging results in changes in the texture and shape of a face. Hence, age is one of the factors that decreases the accuracy of face recognition. Face aging, or age progression, is thus a significant challenge in face recognition methods. This paper presents the use of artificial neural network with model-agnostic meta-learning (ANN-MAML) for face-aging recognition. Model-agnostic meta-learning (MAML) is a meta-learning method used to train a model using parameters obtained from identical tasks with certain updates. This study aims to design and model a framework to recognize face aging based on artificial neural network. In addition, the face-aging recognition framework is evaluated against previous frameworks. Furthermore, the performance and the accuracy of ANN-MAML was evaluated using the CALFW (Cross-Age LFW) dataset. A comparison with other methods showed superior performance by ANN-MAML.
5
Zafar, Muhammad Rehman, and Naimul Khan. "Deterministic Local Interpretable Model-Agnostic Explanations for Stable Explainability." Machine Learning and Knowledge Extraction 3, no. 3 (June 30, 2021): 525–41. http://dx.doi.org/10.3390/make3030027.
Abstract:
Local Interpretable Model-Agnostic Explanations (LIME) is a popular technique used to increase the interpretability and explainability of black box Machine Learning (ML) algorithms. LIME typically creates an explanation for a single prediction by any ML model by learning a simpler interpretable model (e.g., linear classifier) around the prediction through generating simulated data around the instance by random perturbation, and obtaining feature importance through applying some form of feature selection. While LIME and similar local algorithms have gained popularity due to their simplicity, the random perturbation methods result in shifts in data and instability in the generated explanations, where for the same prediction, different explanations can be generated. These are critical issues that can prevent deployment of LIME in sensitive domains. We propose a deterministic version of LIME. Instead of random perturbation, we utilize Agglomerative Hierarchical Clustering (AHC) to group the training data together and K-Nearest Neighbour (KNN) to select the relevant cluster of the new instance that is being explained. After finding the relevant cluster, a simple model (i.e., linear model or decision tree) is trained over the selected cluster to generate the explanations. Experimental results on six public (three binary and three multi-class) and six synthetic datasets show the superiority for Deterministic Local Interpretable Model-Agnostic Explanations (DLIME), where we quantitatively determine the stability and faithfulness of DLIME compared to LIME.
6
Tak, Jae-Ho, and Byung-Woo Hong. "Enhancing Model Agnostic Meta-Learning via Gradient Similarity Loss." Electronics 13, no. 3 (January 29, 2024): 535. http://dx.doi.org/10.3390/electronics13030535.
Abstract:
Artificial intelligence (AI) technology has advanced significantly, now capable of performing tasks previously believed to be exclusive to skilled humans. However, AI models, in contrast to humans who can develop skills with relatively less data, often require substantial amounts of data to emulate human cognitive abilities in specific areas. In situations where adequate pre-training data is not available, meta-learning becomes a crucial method for enhancing generalization. The Model Agnostic Meta-Learning (MAML) algorithm, which employs second-order derivative calculations to fine-tune initial parameters for better starting points, plays a pivotal role in this area. However, the computational demand of this method can be challenging for modern models with a large number of parameters. The concept of the Approximate Hessian Effect is introduced in this context, examining the effectiveness of second-order derivatives in identifying initial parameters conducive to high generalization performance. The study suggests the use of cosine similarity and squared error (L2 loss) as a loss function within the Approximate Hessian Effect framework to modify gradient weights, aiming for more generalizable model parameters. Additionally, an algorithm that relies on first-order calculations is presented, designed to achieve performance levels comparable to MAML. This approach was tested and compared with traditional MAML methods using both the MiniImagenet dataset and a modified MNIST dataset. The results were analyzed to evaluate its efficiency. Compared to previous studies that achieved good performance using only the first derivative, this approach is more efficient because it does not require iterative loops to converge on additional loss functions. Additionally, there is potential for further performance enhancement through hyperparameter tuning.
7
Hou, Xiaoyu, Jihui Xu, Jinming Wu, and Huaiyu Xu. "Cross Domain Adaptation of Crowd Counting with Model-Agnostic Meta-Learning." Applied Sciences 11, no. 24 (December 17, 2021): 12037. http://dx.doi.org/10.3390/app112412037.
Abstract:
Counting people in crowd scenarios is extensively conducted in drone inspections, video surveillance, and public safety applications. Today, crowd count algorithms with supervised learning have improved significantly, but with a reliance on a large amount of manual annotation. However, in real world scenarios, different photo angles, exposures, location heights, complex backgrounds, and limited annotation data lead to supervised learning methods not working satisfactorily, plus many of them suffer from overfitting problems. To address the above issues, we focus on training synthetic crowd data and investigate how to transfer information to real-world datasets while reducing the need for manual annotation. CNN-based crowd-counting algorithms usually consist of feature extraction, density estimation, and count regression. To improve the domain adaptation in feature extraction, we propose an adaptive domain-invariant feature extracting module. Meanwhile, after taking inspiration from recent innovative meta-learning, we present a dynamic-β MAML algorithm to generate a density map in unseen novel scenes and render the density estimation model more universal. Finally, we use a counting map refiner to optimize the coarse density map transformation into a fine density map and then regress the crowd number. Extensive experiments show that our proposed domain adaptation- and model-generalization methods can effectively suppress domain gaps and produce elaborate density maps in cross-domain crowd-counting scenarios. We demonstrate that the proposals in our paper outperform current state-of-the-art techniques.
8
Chen, Zhouyuan, Zhichao Lian, and Zhe Xu. "Interpretable Model-Agnostic Explanations Based on Feature Relationships for High-Performance Computing." Axioms 12, no. 10 (October 23, 2023): 997. http://dx.doi.org/10.3390/axioms12100997.
Abstract:
In the explainable artificial intelligence (XAI) field, an algorithm or a tool can help people understand how a model makes a decision. And this can help to select important features to reduce computational costs to realize high-performance computing. But existing methods are usually used to visualize important features or highlight active neurons, and few of them show the importance of relationships between features. In recent years, some methods based on a white-box approach have taken relationships between features into account, but most of them can only work on some specific models. Although methods based on a black-box approach can solve the above problems, most of them can only be applied to tabular data or text data instead of image data. To solve these problems, we propose a local interpretable model-agnostic explanation approach based on feature relationships. This approach combines the relationships between features into the interpretation process and then visualizes the interpretation results. Finally, this paper conducts a lot of experiments to evaluate the correctness of relationships between features and evaluates this XAI method in terms of accuracy, fidelity, and consistency.
9
Hu, Cong, Kai Xu, Zhengqiu Zhu, Long Qin, and Quanjun Yin. "Multi-Agent Chronological Planning with Model-Agnostic Meta Reinforcement Learning." Applied Sciences 13, no. 16 (August 11, 2023): 9174. http://dx.doi.org/10.3390/app13169174.
Abstract:
In this study, we propose an innovative approach to address a chronological planning problem involving the multiple agents required to complete tasks under precedence constraints. We model this problem as a stochastic game and solve it with multi-agent reinforcement learning algorithms. However, these algorithms necessitate relearning from scratch when confronted with changes in the chronological order of tasks, resulting in distinct stochastic games and consuming a substantial amount of time. To overcome this challenge, we present a novel framework that incorporates meta-learning into a multi-agent reinforcement learning algorithm. This approach enables the extraction of meta-parameters from past experiences, facilitating rapid adaptation to new tasks with altered chronological orders and circumventing the time-intensive nature of reinforcement learning. Then, the proposed framework is demonstrated through the implementation of a method named Reptile-MADDPG. The performance of the pre-trained model is evaluated using average rewards before and after fine-tuning. Our method, in two testing tasks, improves the average rewards from −44 to −37 through 10,000 steps of fine-tuning in two testing tasks, significantly surpassing the two baseline methods that only attained −51 and −44, respectively. The experimental results demonstrate the superior generalization capabilities of our method across various tasks, thus constituting a significant contribution towards the design of intelligent unmanned systems.
10
Xue, Tianfang, and Haibin Yu. "Unbiased Model-Agnostic Metalearning Algorithm for Learning Target-Driven Visual Navigation Policy." Computational Intelligence and Neuroscience 2021 (December 8, 2021): 1–12. http://dx.doi.org/10.1155/2021/5620751.
Abstract:
As deep reinforcement learning methods have made great progress in the visual navigation field, metalearning-based algorithms are gaining more attention since they greatly improve the expansibility of moving agents. According to metatraining mechanism, typically an initial model is trained as a metalearner by existing navigation tasks and becomes well performed in new scenes through relatively few recursive trials. However, if a metalearner is overtrained on the former tasks, it may hardly achieve generalization on navigating in unfamiliar environments as the initial model turns out to be quite biased towards former ambient configuration. In order to train an impartial navigation model and enhance its generalization capability, we propose an Unbiased Model-Agnostic Metalearning (UMAML) algorithm towards target-driven visual navigation. Inspired by entropy-based methods, maximizing the uncertainty over output labels in classification tasks, we adopt inequality measures used in Economics as a concise metric to calculate the loss deviation across unfamiliar tasks. With succinctly minimizing the inequality of task losses, an unbiased navigation model without overperforming in particular scene types can be learnt based on Model-Agnostic Metalearning mechanism. The exploring agent complies with a more balanced update rule, able to gather navigation experience from training environments. Several experiments have been conducted, and results demonstrate that our approach outperforms other state-of-the-art metalearning navigation methods in generalization ability.
11
Moskalenko, V. V. "MODEL-AGNOSTIC META-LEARNING FOR RESILIENCE OPTIMIZATION OF ARTIFICIAL INTELLIGENCE SYSTEM." Radio Electronics, Computer Science, Control, no. 2 (June 30, 2023): 79. http://dx.doi.org/10.15588/1607-3274-2023-2-9.
Abstract:
Context. The problem of optimizing the resilience of artificial intelligence systems to destructive disturbances has not yet been fully solved and is quite relevant for safety-critical applications. The task of optimizing the resilience of an artificial intelligence system to disturbing influences is a high-level task in relation to efficiency optimization, which determines the prospects of using the ideas and methods of meta-learning to solve it. The object of current research is the process of meta-learning aimed at optimizing the resilience of an artificial intelligence system to destructive disturbances. The subjects of the study are architectural add-ons and the meta-learning method which optimize resilience to adversarial attacks, fault injection, and task changes.
Objective. Stated research goal is to develop an effective meta-learning method for optimizing the resilience of an artificial intelligence system to destructive disturbances.
Method. The resilience optimization is implemented by combining the ideas and methods of adversarial learning, fault-tolerant learning, model-agnostic meta-learning, few-shot learning, gradient optimization methods, and probabilistic gradient approximation strategies. The choice of architectural add-ons is based on parameter-efficient knowledge transfer designed to save resources and avoid the problem of catastrophic forgetting.
Results. A model-agnostic meta-learning method for optimizing the resilience of artificial intelligence systems based on gradient meta-updates or meta-updates using an evolutionary strategy has been developed. This method involves the use of tuner and metatuner blocks that perform parallel correction of the building blocks of a original deep neural network. The ability of the proposed approach to increase the efficiency of perturbation absorption and increase the integral resilience indicator of the artificial intelligence system is experimentally tested on the example of the image classification task. The experiments were conducted on a model with the ResNet-18 architecture, with an add-on in the form of tuners and meta-tuners with the Conv-Adapter architecture. In this case, CIFAR-10 is used as a base set on which the model was trained, and CIFAR-100 is used as a set for generating samples on which adaptation is performed using a few-shot learning scenarios. We compare the resilience of the artificial intelligence system after pre-training tuners and meta-tuners using the adversarial learning algorithm, the fault-tolerant learning algorithm, the conventional model-agnostic meta-learning algorithm, and the proposed meta-learning method for optimizing resilience. Also, the meta-learning algorithms with meta-gradient updating and meta-updating based on the evolutionary strategy are compared on the basis of the integral resilience indicator.
Conclusions. It has been experimentally confirmed that the proposed method provides a better resilience to random bit-flip injection compared to fault injection training by an average of 5%. Also, the proposed method provides a better resilience to Ladversarial evasion attacks compared to adversarial training by an average of 4.8%. In addition, an average 4.8% increase in the resilience to task changes is demonstrated compared to conventional fine-tuning of tuners. Moreover, meta-learning with an evolutionary strategy provides, on average, higher values of the resilience indicator. On the downside, this meta-learning method requires more iterations.
12
Schmidt, Henri, Palash Sashittal, and Benjamin J. Raphael. "A zero-agnostic model for copy number evolution in cancer." PLOS Computational Biology 19, no. 11 (November 9, 2023): e1011590. http://dx.doi.org/10.1371/journal.pcbi.1011590.
Abstract:
Motivation New low-coverage single-cell DNA sequencing technologies enable the measurement of copy number profiles from thousands of individual cells within tumors. From this data, one can infer the evolutionary history of the tumor by modeling transformations of the genome via copy number aberrations. Copy number aberrations alter multiple adjacent genomic loci, violating the standard phylogenetic assumption that loci evolve independently. Thus, specialized models to infer copy number phylogenies have been introduced. A widely used model is the copy number transformation (CNT) model in which a genome is represented by an integer vector and a copy number aberration is an event that either increases or decreases the number of copies of a contiguous segment of the genome. The CNT distance between a pair of copy number profiles is the minimum number of events required to transform one profile to another. While this distance can be computed efficiently, no efficient algorithm has been developed to find the most parsimonious phylogeny under the CNT model. Results We introduce the zero-agnostic copy number transformation (ZCNT) model, a simplification of the CNT model that allows the amplification or deletion of regions with zero copies. We derive a closed form expression for the ZCNT distance between two copy number profiles and show that, unlike the CNT distance, the ZCNT distance forms a metric. We leverage the closed-form expression for the ZCNT distance and an alternative characterization of copy number profiles to derive polynomial time algorithms for two natural relaxations of the small parsimony problem on copy number profiles. While the alteration of zero copy number regions allowed under the ZCNT model is not biologically realistic, we show on both simulated and real datasets that the ZCNT distance is a close approximation to the CNT distance. Extending our polynomial time algorithm for the ZCNT small parsimony problem, we develop an algorithm, Lazac, for solving the large parsimony problem on copy number profiles. We demonstrate that Lazac outperforms existing methods for inferring copy number phylogenies on both simulated and real data.
13
Hasan, Md Mahmudul. "Understanding Model Predictions: A Comparative Analysis of SHAP and LIME on Various ML Algorithms." Journal of Scientific and Technological Research 5, no. 1 (2024): 17–26. http://dx.doi.org/10.59738/jstr.v5i1.23(17-26).eaqr5800.
Abstract:
To guarantee the openness and dependability of prediction systems across multiple domains, machine learning model interpretation is essential. In this study, a variety of machine learning algorithms are subjected to a thorough comparative examination of two model-agnostic explainability methodologies, SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-Agnostic Explanations). The study focuses on the performance of the algorithms on a dataset in order to offer subtle insights on the interpretability of models when faced with various algorithms. Intriguing new information on the relative performance of SHAP and LIME is provided by the findings. While both methods adequately explain model predictions, they behave differently when applied to other algorithms and datasets. The findings made in this paper add to the continuing discussion on model interpretability and provide useful advice for utilizing SHAP and LIME to increase transparency in machine learning applications.
14
Labaien Soto, Jokin, Ekhi Zugasti Uriguen, and Xabier De Carlos Garcia. "Real-Time, Model-Agnostic and User-Driven Counterfactual Explanations Using Autoencoders." Applied Sciences 13, no. 5 (February 24, 2023): 2912. http://dx.doi.org/10.3390/app13052912.
Abstract:
Explainable Artificial Intelligence (XAI) has gained significant attention in recent years due to concerns over the lack of interpretability of Deep Learning models, which hinders their decision-making processes. To address this issue, counterfactual explanations have been proposed to elucidate the reasoning behind a model’s decisions by providing what-if statements as explanations. However, generating counterfactuals traditionally involves solving an optimization problem for each input, making it impractical for real-time feedback. Moreover, counterfactuals must meet specific criteria, including being user-driven, causing minimal changes, and staying within the data distribution. To overcome these challenges, a novel model-agnostic approach called Real-Time Guided Counterfactual Explanations (RTGCEx) is proposed. This approach utilizes autoencoders to generate real-time counterfactual explanations that adhere to these criteria by optimizing a multiobjective loss function. The performance of RTGCEx has been evaluated on two datasets: MNIST and Gearbox, a synthetic time series dataset. The results demonstrate that RTGCEx outperforms traditional methods in terms of speed and efficacy on MNIST, while also effectively identifying and rectifying anomalies in the Gearbox dataset, highlighting its versatility across different scenarios.
15
Sun, Yifei, Cheng Song, Feng Lu, Wei Li, Hai Jin, and Albert Y. Zomaya. "ES-Mask: Evolutionary Strip Mask for Explaining Time Series Prediction (Student Abstract)." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 13 (June 26, 2023): 16342–43. http://dx.doi.org/10.1609/aaai.v37i13.27031.
Abstract:
Machine learning models are increasingly used in time series prediction with promising results. The model explanation of time series prediction falls behind the model development and makes less sense to users in understanding model decisions. This paper proposes ES-Mask, a post-hoc and model-agnostic evolutionary strip mask-based saliency approach for time series applications. ES-Mask designs the mask consisting of strips with the same salient value in consecutive time steps to produce binary and sustained feature importance scores over time for easy understanding and interpretation of time series. ES-Mask uses an evolutionary algorithm to search for the optimal mask by manipulating strips in rounds, thus is agnostic to models by involving no internal model states in the search. The initial experiments on MIMIC-III data set show that ES-Mask outperforms state-of-the-art methods.
16
Wu, Gang, Junjun Jiang, Kui Jiang, and Xianming Liu. "Learning from History: Task-agnostic Model Contrastive Learning for Image Restoration." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 6 (March 24, 2024): 5976–84. http://dx.doi.org/10.1609/aaai.v38i6.28412.
Abstract:
Contrastive learning has emerged as a prevailing paradigm for high-level vision tasks, which, by introducing properly negative samples, has also been exploited for low-level vision tasks to achieve a compact optimization space to account for their ill-posed nature. However, existing methods rely on manually predefined and task-oriented negatives, which often exhibit pronounced task-specific biases. To address this challenge, our paper introduces an innovative method termed 'learning from history', which dynamically generates negative samples from the target model itself. Our approach, named Model Contrastive Learning for Image Restoration (MCLIR), rejuvenates latency models as negative models, making it compatible with diverse image restoration tasks. We propose the Self-Prior guided Negative loss (SPN) to enable it. This approach significantly enhances existing models when retrained with the proposed model contrastive paradigm. The results show significant improvements in image restoration across various tasks and architectures. For example, models retrained with SPN outperform the original FFANet and DehazeFormer by 3.41 and 0.57 dB on the RESIDE indoor dataset for image dehazing. Similarly, they achieve notable improvements of 0.47 dB on SPA-Data over IDT for image deraining and 0.12 dB on Manga109 for a 4x scale super-resolution over lightweight SwinIR, respectively. Code and retrained models are available at https://github.com/Aitical/MCLIR.
17
Li, Ding, Yan Liu, and Jun Huang. "Assessment of Software Vulnerability Contributing Factors by Model-Agnostic Explainable AI." Machine Learning and Knowledge Extraction 6, no. 2 (May 16, 2024): 1087–113. http://dx.doi.org/10.3390/make6020050.
Abstract:
Software vulnerability detection aims to proactively reduce the risk to software security and reliability. Despite advancements in deep-learning-based detection, a semantic gap still remains between learned features and human-understandable vulnerability semantics. In this paper, we present an XAI-based framework to assess program code in a graph context as feature representations and their effect on code vulnerability classification into multiple Common Weakness Enumeration (CWE) types. Our XAI framework is deep-learning-model-agnostic and programming-language-neutral. We rank the feature importance of 40 syntactic constructs for each of the top 20 distributed CWE types from three datasets in Java and C++. By means of four metrics of information retrieval, we measure the similarity of human-understandable CWE types using each CWE type’s feature contribution ranking learned from XAI methods. We observe that the subtle semantic difference between CWE types occurs after the variation in neighboring features’ contribution rankings. Our study shows that the XAI explanation results have approximately 78% Top-1 to 89% Top-5 similarity hit rates and a mean average precision of 0.70 compared with the baseline of CWE similarity identified by the open community experts. Our framework allows for code vulnerability patterns to be learned and contributing factors to be assessed at the same stage.
18
Chen, Mingyang, Wen Zhang, Zhen Yao, Yushan Zhu, Yang Gao, Jeff Z. Pan, and Huajun Chen. "Entity-Agnostic Representation Learning for Parameter-Efficient Knowledge Graph Embedding." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 4 (June 26, 2023): 4182–90. http://dx.doi.org/10.1609/aaai.v37i4.25535.
Abstract:
We propose an entity-agnostic representation learning method for handling the problem of inefficient parameter storage costs brought by embedding knowledge graphs. Conventional knowledge graph embedding methods map elements in a knowledge graph, including entities and relations, into continuous vector spaces by assigning them one or multiple specific embeddings (i.e., vector representations). Thus the number of embedding parameters increases linearly as the growth of knowledge graphs. In our proposed model, Entity-Agnostic Representation Learning (EARL), we only learn the embeddings for a small set of entities and refer to them as reserved entities. To obtain the embeddings for the full set of entities, we encode their distinguishable information from their connected relations, k-nearest reserved entities, and multi-hop neighbors. We learn universal and entity-agnostic encoders for transforming distinguishable information into entity embeddings. This approach allows our proposed EARL to have a static, efficient, and lower parameter count than conventional knowledge graph embedding methods. Experimental results show that EARL uses fewer parameters and performs better on link prediction tasks than baselines, reflecting its parameter efficiency.
19
Shozu, Kanto, Masaaki Komatsu, Akira Sakai, Reina Komatsu, Ai Dozen, Hidenori Machino, Suguru Yasutomi, et al. "Model-Agnostic Method for Thoracic Wall Segmentation in Fetal Ultrasound Videos." Biomolecules 10, no. 12 (December 17, 2020): 1691. http://dx.doi.org/10.3390/biom10121691.
Abstract:
The application of segmentation methods to medical imaging has the potential to create novel diagnostic support models. With respect to fetal ultrasound, the thoracic wall is a key structure on the assessment of the chest region for examiners to recognize the relative orientation and size of structures inside the thorax, which are critical components in neonatal prognosis. In this study, to improve the segmentation performance of the thoracic wall in fetal ultrasound videos, we proposed a novel model-agnostic method using deep learning techniques: the Multi-Frame + Cylinder method (MFCY). The Multi-frame method (MF) uses time-series information of ultrasound videos, and the Cylinder method (CY) utilizes the shape of the thoracic wall. To evaluate the achieved improvement, we performed segmentation using five-fold cross-validation on 538 ultrasound frames in the four-chamber view (4CV) of 256 normal cases using U-net and DeepLabv3+. MFCY increased the mean values of the intersection over union (IoU) of thoracic wall segmentation from 0.448 to 0.493 for U-net and from 0.417 to 0.470 for DeepLabv3+. These results demonstrated that MFCY improved the segmentation performance of the thoracic wall in fetal ultrasound videos without altering the network structure. MFCY is expected to facilitate the development of diagnostic support models in fetal ultrasound by providing further accurate segmentation of the thoracic wall.
20
Alinia, Parastoo, Asiful Arefeen, Zhila Esna Ashari, Seyed Iman Mirzadeh, and Hassan Ghasemzadeh. "Model-Agnostic Structural Transfer Learning for Cross-Domain Autonomous Activity Recognition." Sensors 23, no. 14 (July 12, 2023): 6337. http://dx.doi.org/10.3390/s23146337.
Abstract:
Activity recognition using data collected with smart devices such as mobile and wearable sensors has become a critical component of many emerging applications ranging from behavioral medicine to gaming. However, an unprecedented increase in the diversity of smart devices in the internet-of-things era has limited the adoption of activity recognition models for use across different devices. This lack of cross-domain adaptation is particularly notable across sensors of different modalities where the mapping of the sensor data in the traditional feature level is highly challenging. To address this challenge, we propose ActiLabel, a combinatorial framework that learns structural similarities among the events that occur in a target domain and those of a source domain and identifies an optimal mapping between the two domains at their structural level. The structural similarities are captured through a graph model, referred to as the dependency graph, which abstracts details of activity patterns in low-level signal and feature space. The activity labels are then autonomously learned in the target domain by finding an optimal tiered mapping between the dependency graphs. We carry out an extensive set of experiments on three large datasets collected with wearable sensors involving human subjects. The results demonstrate the superiority of ActiLabel over state-of-the-art transfer learning and deep learning methods. In particular, ActiLabel outperforms such algorithms by average F1-scores of 36.3%, 32.7%, and 9.1% for cross-modality, cross-location, and cross-subject activity recognition, respectively.
21
Apicella, A., F. Isgrò, R. Prevete, and G. Tamburrini. "Middle-Level Features for the Explanation of Classification Systems by Sparse Dictionary Methods." International Journal of Neural Systems 30, no. 08 (July 14, 2020): 2050040. http://dx.doi.org/10.1142/s0129065720500409.
Abstract:
Machine learning (ML) systems are affected by a pervasive lack of transparency. The eXplainable Artificial Intelligence (XAI) research area addresses this problem and the related issue of explaining the behavior of ML systems in terms that are understandable to human beings. In many explanation of XAI approaches, the output of ML systems are explained in terms of low-level features of their inputs. However, these approaches leave a substantive explanatory burden with human users, insofar as the latter are required to map low-level properties into more salient and readily understandable parts of the input. To alleviate this cognitive burden, an alternative model-agnostic framework is proposed here. This framework is instantiated to address explanation problems in the context of ML image classification systems, without relying on pixel relevance maps and other low-level features of the input. More specifically, one obtains sets of middle-level properties of classification inputs that are perceptually salient by applying sparse dictionary learning techniques. These middle-level properties are used as building blocks for explanations of image classifications. The achieved explanations are parsimonious, for their reliance on a limited set of middle-level image properties. And they can be contrastive, because the set of middle-level image properties can be used to explain why the system advanced the proposed classification over other antagonist classifications. In view of its model-agnostic character, the proposed framework is adaptable to a variety of other ML systems and explanation problems.
22
Diprose, William K., Nicholas Buist, Ning Hua, Quentin Thurier, George Shand, and Reece Robinson. "Physician understanding, explainability, and trust in a hypothetical machine learning risk calculator." Journal of the American Medical Informatics Association 27, no. 4 (February 27, 2020): 592–600. http://dx.doi.org/10.1093/jamia/ocz229.
Abstract:
Abstract Objective Implementation of machine learning (ML) may be limited by patients’ right to “meaningful information about the logic involved” when ML influences healthcare decisions. Given the complexity of healthcare decisions, it is likely that ML outputs will need to be understood and trusted by physicians, and then explained to patients. We therefore investigated the association between physician understanding of ML outputs, their ability to explain these to patients, and their willingness to trust the ML outputs, using various ML explainability methods. Materials and Methods We designed a survey for physicians with a diagnostic dilemma that could be resolved by an ML risk calculator. Physicians were asked to rate their understanding, explainability, and trust in response to 3 different ML outputs. One ML output had no explanation of its logic (the control) and 2 ML outputs used different model-agnostic explainability methods. The relationships among understanding, explainability, and trust were assessed using Cochran-Mantel-Haenszel tests of association. Results The survey was sent to 1315 physicians, and 170 (13%) provided completed surveys. There were significant associations between physician understanding and explainability (P < .001), between physician understanding and trust (P < .001), and between explainability and trust (P < .001). ML outputs that used model-agnostic explainability methods were preferred by 88% of physicians when compared with the control condition; however, no particular ML explainability method had a greater influence on intended physician behavior. Conclusions Physician understanding, explainability, and trust in ML risk calculators are related. Physicians preferred ML outputs accompanied by model-agnostic explanations but the explainability method did not alter intended physician behavior.
23
Liu, Jiakang, and Hua Huo. "DFENet: Double Feature Enhanced Class Agnostic Counting Methods." Frontiers in Computing and Intelligent Systems 6, no. 1 (December 1, 2023): 70–76. http://dx.doi.org/10.54097/fcis.v6i1.14.
Abstract:
Object counting is a basic computer vision task, which can estimate the number of each object in an image, thus providing valuable information. In dense scenes, there are huge differences in target individual scale, and the different target individual scale leads to low accuracy of target count. In addition, most of the existing target count datasets in the field require a lot of manual creation and annotation, which increases the cost and difficulty of the dataset, lack of ease of use and portability. To solve these problems, this paper proposes a class agnostic counting method Double Feature Enhancement Net based on improved Bilinear Matching Network+ (BMNet+). By introducing the feature enhancement module based on the principle of conditional random field and the adaptively spatial feature fusion module, combined with the feature similarity measurement strategy of bilinear matching network, the method can effectively extract the target features of different scales, enhance the adaptability to the targets with large scale changes, and improve the counting performance of the network. Experiments were carried out on FSC-147 data set, and the experimental results show that the proposed model has been further improved in counting accuracy. The MAE and MSE of the verification set are 15.03 and 54.53 respectively. In the test set, MAE reaches 13.65, MSE reaches 89.54, and the counting performance is at the advanced level in the field.
24
Moon, Jae-pil, Jin-Guk Kim, Choong-Heon Yang, and Su-Bin Park. "A Case Study on the Application of Model-agnostic Methods for the Post hoc Interpretation of A Machine Learning Model :." International Journal of Highway Engineering 24, no. 3 (June 30, 2022): 83–95. http://dx.doi.org/10.7855/ijhe.2022.24.3.083.
25
Li, Jolen, Christoforos Galazis, Larion Popov, Lev Ovchinnikov, Tatyana Kharybina, Sergey Vesnin, Alexander Losev, and Igor Goryanin. "Dynamic Weight Agnostic Neural Networks and Medical Microwave Radiometry (MWR) for Breast Cancer Diagnostics." Diagnostics 12, no. 9 (August 23, 2022): 2037. http://dx.doi.org/10.3390/diagnostics12092037.
Abstract:
Background and Objective: Medical microwave radiometry (MWR) is used to capture the thermal properties of internal tissues and has usages in breast cancer detection. Our goal in this paper is to improve classification performance and investigate automated neural architecture search methods. Methods: We investigated extending the weight agnostic neural network by optimizing the weights using the bi-population covariance matrix adaptation evolution strategy (BIPOP-CMA-ES) once the topology was found. We evaluated and compared the model based on the F1 score, accuracy, precision, recall, and the number of connections. Results: The experiments were conducted on a dataset of 4912 patients, classified as low or high risk for breast cancer. The weight agnostic BIPOP-CMA-ES model achieved the best average performance. It obtained an F1-score of 0.933, accuracy of 0.932, precision of 0.929, recall of 0.942, and 163 connections. Conclusions: The results of the model are an indication of the promising potential of MWR utilizing a neural network-based diagnostic tool for cancer detection. By separating the tasks of topology search and weight training, we can improve the overall performance.
26
R, Jain. "Transparency in AI Decision Making: A Survey of Explainable AI Methods and Applications." Advances in Robotic Technology 2, no. 1 (January 19, 2024): 1–10. http://dx.doi.org/10.23880/art-16000110.
Abstract:
Artificial Intelligence (AI) systems have become pervasive in numerous facets of modern life, wielding considerable influence in critical decision-making realms such as healthcare, finance, criminal justice, and beyond. Yet, the inherent opacity of many AI models presents significant hurdles concerning trust, accountability, and fairness. To address these challenges, Explainable AI (XAI) has emerged as a pivotal area of research, striving to augment the transparency and interpretability of AI systems. This survey paper serves as a comprehensive exploration of the state-of-the-art in XAI methods and their practical applications. We delve into a spectrum of techniques, spanning from model-agnostic approaches to interpretable machine learning models, meticulously scrutinizing their respective strengths, limitations, and real-world implications. The landscape of XAI is rich and varied, with diverse methodologies tailored to address different facets of interpretability. Model-agnostic approaches offer versatility by providing insights into model behavior across various AI architectures. In contrast, interpretable machine learning models prioritize transparency by design, offering inherent understandability at the expense of some predictive performance. Layer-wise Relevance Propagation (LRP) and attention mechanisms delve into the inner workings of neural networks, shedding light on feature importance and decision processes. Additionally, counterfactual explanations open avenues for exploring what-if scenarios, elucidating the causal relationships between input features and model outcomes. In tandem with methodological exploration, this survey scrutinizes the deployment and impact of XAI across multifarious domains. Successful case studies showcase the practical utility of transparent AI in healthcare diagnostics, financial risk assessment, criminal justice systems, and more. By elucidating these use cases, we illuminate the transformative potential of XAI in enhancing decision-making processes while fostering accountability and fairness. Nevertheless, the journey towards fully transparent AI systems is fraught with challenges and opportunities. As we traverse the current landscape of XAI, we identify pressing areas for further research and development. These include refining interpretability metrics, addressing the scalability of XAI techniques to complex models, and navigating the ethical dimensions of transparency in AI decision-making.Through this survey, we endeavor to cultivate a deeper understanding of transparency in AI decision-making, empowering stakeholders to navigate the intricate interplay between accuracy, interpretability, and ethical considerations. By fostering interdisciplinary dialogue and inspiring collaborative innovation, we aspire to catalyze future advancements in Explainable AI, ultimately paving the way towards more accountable and trustworthy AI systems.
27
TOPCU, Deniz. "How to explain a machine learning model: HbA1c classification example." Journal of Medicine and Palliative Care 4, no. 2 (March 27, 2023): 117–25. http://dx.doi.org/10.47582/jompac.1259507.
Abstract:
Aim: Machine learning tools have various applications in healthcare. However, the implementation of developed models is still limited because of various challenges. One of the most important problems is the lack of explainability of machine learning models. Explainability refers to the capacity to reveal the reasoning and logic behind the decisions made by AI systems, making it straightforward for human users to understand the process and how the system arrived at a specific outcome. The study aimed to compare the performance of different model-agnostic explanation methods using two different ML models created for HbA1c classification.
Material and Method: The H2O AutoML engine was used for the development of two ML models (Gradient boosting machine (GBM) and default random forests (DRF)) using 3,036 records from NHANES open data set. Both global and local model-agnostic explanation methods, including performance metrics, feature important analysis and Partial dependence, Breakdown and Shapley additive explanation plots were utilized for the developed models.
Results: While both GBM and DRF models have similar performance metrics, such as mean per class error and area under the receiver operating characteristic curve, they had slightly different variable importance. Local explainability methods also showed different contributions to the features.
Conclusion: This study evaluated the significance of explainable machine learning techniques for comprehending complicated models and their role in incorporating AI in healthcare. The results indicate that although there are limitations to current explainability methods, particularly for clinical use, both global and local explanation models offer a glimpse into evaluating the model and can be used to enhance or compare models.
28
Vieira, Carla Piazzon Ramos, and Luciano Antonio Digiampietri. "A study about Explainable Articial Intelligence: using decision tree to explain SVM." Revista Brasileira de Computação Aplicada 12, no. 1 (January 8, 2020): 113–21. http://dx.doi.org/10.5335/rbca.v12i1.10247.
Abstract:
The technologies supporting Artificial Intelligence (AI) have advanced rapidly over the past few years and AI is becoming a commonplace in every aspect of life like the future of self-driving cars or earlier health diagnosis. For this to occur shortly, the entire community stands in front of the barrier of explainability, an inherent problem of latest models (e.g. Deep Neural Networks) that were not present in the previous hype of AI (linear and rule-based models). Most of these recent models are used as black boxes without understanding partially or even completely how different features influence the model prediction avoiding algorithmic transparency. In this paper, we focus on how much we can understand the decisions made by an SVM Classifier in a post-hoc model agnostic approach. Furthermore, we train a tree-based model (inherently interpretable) using labels from the SVM, called secondary training data to provide explanations and compare permutation importance method to the more commonly used measures such as accuracy and show that our methods are both more reliable and meaningful techniques to use. We also outline the main challenges for such methods and conclude that model-agnostic interpretability is a key component in making machine learning more trustworthy.
29
Noviandy, Teuku Rizky, Ghalieb Mutig Idroes, Irsan Hardi, Mohd Afjal, and Samrat Ray. "A Model-Agnostic Interpretability Approach to Predicting Customer Churn in the Telecommunications Industry." Infolitika Journal of Data Science 2, no. 1 (May 27, 2024): 34–44. http://dx.doi.org/10.60084/ijds.v2i1.199.
Abstract:
Customer churn is critical for businesses across various industries, especially in the telecommunications sector, where high churn rates can significantly impact revenue and growth. Understanding the factors leading to customer churn is essential for developing effective retention strategies. Despite the predictive power of machine learning models, there is a growing demand for model interpretability to ensure trust and transparency in decision-making processes. This study addresses this gap by applying advanced machine learning models, specifically Naïve Bayes, Random Forest, AdaBoost, XGBoost, and LightGBM, to predict customer churn in a telecommunications dataset. We enhanced model interpretability using SHapley Additive exPlanations (SHAP), which provides insights into feature contributions to predictions. Here, we show that LightGBM achieved the highest performance among the models, with an accuracy of 80.70%, precision of 84.35%, recall of 90.54%, and an F1-score of 87.34%. SHAP analysis revealed that features such as tenure, contract type, and monthly charges are significant predictors of customer churn. These results indicate that combining predictive analytics with interpretability methods can provide telecom companies with actionable insights to tailor retention strategies effectively. The study highlights the importance of understanding customer behavior through transparent and accurate models, paving the way for improved customer satisfaction and loyalty. Future research should focus on validating these findings with real-world data, exploring more sophisticated models, and incorporating temporal dynamics to enhance churn prediction models' predictive power and applicability.
30
Thakur, Siddhesh, Jimit Doshi, Sung Min Ha, Gaurav Shukla, Aikaterini Kotrotsou, Sanjay Talbar, Uday Kulkarni, et al. "NIMG-40. ROBUST MODALITY-AGNOSTIC SKULL-STRIPPING IN PRESENCE OF DIFFUSE GLIOMA: A MULTI-INSTITUTIONAL STUDY." Neuro-Oncology 21, Supplement_6 (November 2019): vi170. http://dx.doi.org/10.1093/neuonc/noz175.710.
Abstract:
Abstract BACKGROUND Skull-stripping describes essential pre-processing in neuro-imaging, directly impacting subsequent analyses. Existing skull-stripping algorithms are typically developed and validated only on T1-weighted MRI scans without apparent gliomas, hence may fail when applied on neuro-oncology scans. Furthermore, most algorithms have large computational footprint and lack generalization to different acquisition protocols, limiting their clinical use. We sought to identify a practical, generalizable, robust, and accurate solution to address all these limitations. METHODS We identified multi-institutional retrospective cohorts, describing pre-operative multi-parametric MRI modalities (T1,T1Gd,T2,T2-FLAIR) with distinct acquisition protocols (e.g., slice thickness, magnet strength), varying pre-applied image-based defacing techniques, and corresponding manually-delineated ground-truth brain masks. We developed a 3D fully convolutional deep learning architecture (3D-ResUNet). Following modality co-registration to a common anatomical template, the 3D-ResUNet was trained on 314 subjects from the University of Pennsylvania (UPenn), and evaluated on 91, 152, 25, and 29 unseen subjects from UPenn, Thomas Jefferson University (TJU), Washington University (WashU), and MD Anderson (MDACC), respectively. To achieve robustness against scanner/resolution variability and utilize all modalities, we introduced a novel “modality-agnostic” training approach, which allows application of the trained model on any single modality, without requiring a pre-determined modality as input. We calculate the final brain mask for any test subject by applying our trained modality-agnostic 3D-ResUNet model on the modality with the highest resolution. RESULTS The average(±stdDev) dice similarity coefficients achieved for our novel modality-agnostic model were equal to 97.81%+0.8, 95.59%+2.0, 91.61%+1.9, and 96.05%+1.4 for the unseen data from UPenn, TJU, WashU, and MDACC, respectively. CONCLUSIONS Our novel modality-agnostic skull-stripping approach produces robust near-human performance, generalizes across acquisition protocols, image-based defacing techniques, without requiring pre-determined input modalities or depending on the availability of a specific modality. Such an approach can facilitate tool standardization for harmonized pre-processing of neuro-oncology scans for multi-institutional collaborations, enabling further data sharing and computational analyses.
31
Gunel, Kadir, and Mehmet Fatih Amasyali. "Boosting Lightweight Sentence Embeddings with Knowledge Transfer from Advanced Models: A Model-Agnostic Approach." Applied Sciences 13, no. 23 (November 22, 2023): 12586. http://dx.doi.org/10.3390/app132312586.
Abstract:
In this study, we investigate knowledge transfer between two distinct sentence embedding models: a computationally demanding, highly performant model and a lightweight model derived from word vector averaging. Our objective is to augment the representational power of the lightweight model by exploiting the sophisticated features of the robust model. Diverging from traditional knowledge distillation methods that align logits or hidden states of teacher and student models, our approach uses only the output sentence vectors of the teacher model for the alignment with the student models’s word vector representations. We implement two minimization techniques for this purpose: distance minimization and distance and perplexity minimization Our methodology uses WMT datasets for training, and the enhanced embeddings are validated via Google’s Analogy tasks and Meta’s SentEval datasets. We found that our proposed models intriguingly retained and conveyed information in a model-specific fashion.
32
Kedar, Ms Mayuri Manish. "Exploring the Effectiveness of SHAP over other Explainable AI Methods." INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT 08, no. 06 (June 6, 2024): 1–5. http://dx.doi.org/10.55041/ijsrem35556.
Abstract:
Explainable Artificial Intelligence (XAI) has emerged as a critical domain to demystify the opaque decision-making processes of machine learning models, fostering trust and understanding among users. Among various XAI methods, SHAP (SHapley Additive exPlanations) has gained prominence for its theo- retically grounded approach and practical applicability. The paper presents a comprehensive exploration of SHAP’s effectiveness compared to other promi- nent XAI methods.Methods such as LIME (Local Interpretable Model-agnostic Explanations), permutation importance, Anchors and partial dependence plots are examined for their respective strengths and limitations. Through a detailed analysis of their principles, strengths, and limitations through reviewing differ- ent research papers based on some important factors of XAI, the paper aims to provide insights into the effectiveness and suitability of these methods.The study offers valuable guidance for researchers and practitioners seeking to incorporate XAI into their AI systems. Keywords: SHAP, XAI, LIME, permutation importance, Anchors and par- tial dependence plots
33
Gilo, Daniel, and Shaul Markovitch. "A General Search-Based Framework for Generating Textual Counterfactual Explanations." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 16 (March 24, 2024): 18073–81. http://dx.doi.org/10.1609/aaai.v38i16.29764.
Abstract:
One of the prominent methods for explaining the decision of a machine-learning classifier is by a counterfactual example. Most current algorithms for generating such examples in the textual domain are based on generative language models. Generative models, however, are trained to minimize a specific loss function in order to fulfill certain requirements for the generated texts. Any change in the requirements may necessitate costly retraining, thus potentially limiting their applicability. In this paper, we present a general search-based framework for generating counterfactual explanations in the textual domain. Our framework is model-agnostic, domain-agnostic, anytime, and does not require retraining in order to adapt to changes in the user requirements. We model the task as a search problem in a space where the initial state is the classified text, and the goal state is a text in a given target class. Our framework includes domain-independent modification operators, but can also exploit domain-specific knowledge through specialized operators. The search algorithm attempts to find a text from the target class with minimal user-specified distance from the original classified object.
34
Song, Rui, Fausto Giunchiglia, Yingji Li, Mingjie Tian, and Hao Xu. "TACIT: A Target-Agnostic Feature Disentanglement Framework for Cross-Domain Text Classification." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 17 (March 24, 2024): 18999–9007. http://dx.doi.org/10.1609/aaai.v38i17.29866.
Abstract:
Cross-domain text classification aims to transfer models from label-rich source domains to label-poor target domains, giving it a wide range of practical applications. Many approaches promote cross-domain generalization by capturing domaininvariant features. However, these methods rely on unlabeled samples provided by the target domains, which renders the model ineffective when the target domain is agnostic. Furthermore, the models are easily disturbed by shortcut learning in the source domain, which also hinders the improvement of domain generalization ability. To solve the aforementioned issues, this paper proposes TACIT, a target domain agnostic feature disentanglement framework which adaptively decouples robust and unrobust features by Variational Auto-Encoders. Additionally, to encourage the separation of unrobust features from robust features, we design a feature distillation task that compels unrobust features to approximate the output of the teacher. The teacher model is trained with a few easy samples that are easy to carry potential unknown shortcuts. Experimental results verify that our framework achieves comparable results to state-of-the-art baselines while utilizing only source domain data.
35
Wang, Kewei, Yizheng Wu, Zhiyu Pan, Xingyi Li, Ke Xian, Zhe Wang, Zhiguo Cao, and Guosheng Lin. "Semi-supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 6 (March 24, 2024): 5490–98. http://dx.doi.org/10.1609/aaai.v38i6.28358.
Abstract:
Class-agnostic motion prediction methods aim to comprehend motion within open-world scenarios, holding significance for autonomous driving systems. However, training a high-performance model in a fully-supervised manner always requires substantial amounts of manually annotated data, which can be both expensive and time-consuming to obtain. To address this challenge, our study explores the potential of semi-supervised learning (SSL) for class-agnostic motion prediction. Our SSL framework adopts a consistency-based self-training paradigm, enabling the model to learn from unlabeled data by generating pseudo labels through test-time inference. To improve the quality of pseudo labels, we propose a novel motion selection and re-generation module. This module effectively selects reliable pseudo labels and re-generates unreliable ones. Furthermore, we propose two data augmentation strategies: temporal sampling and BEVMix. These strategies facilitate consistency regularization in SSL. Experiments conducted on nuScenes demonstrate that our SSL method can surpass the self-supervised approach by a large margin by utilizing only a tiny fraction of labeled data. Furthermore, our method exhibits comparable performance to weakly and some fully supervised methods. These results highlight the ability of our method to strike a favorable balance between annotation costs and performance. Code will be available at https://github.com/kwwcv/SSMP.
36
Lin, Weiping, Zhenfeng Zhuang, Lequan Yu, and Liansheng Wang. "Boosting Multiple Instance Learning Models for Whole Slide Image Classification: A Model-Agnostic Framework Based on Counterfactual Inference." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 4 (March 24, 2024): 3477–85. http://dx.doi.org/10.1609/aaai.v38i4.28135.
Abstract:
Multiple instance learning is an effective paradigm for whole slide image (WSI) classification, where labels are only provided at the bag level. However, instance-level prediction is also crucial as it offers insights into fine-grained regions of interest. Existing multiple instance learning methods either solely focus on training a bag classifier or have the insufficient capability of exploring instance prediction. In this work, we propose a novel model-agnostic framework to boost existing multiple instance learning models, to improve the WSI classification performance in both bag and instance levels. Specifically, we propose a counterfactual inference-based sub-bag assessment method and a hierarchical instance searching strategy to help to search reliable instances and obtain their accurate pseudo labels. Furthermore, an instance classifier is well-trained to produce accurate predictions. The instance embedding it generates is treated as a prompt to refine the instance feature for bag prediction. This framework is model-agnostic, capable of adapting to existing multiple instance learning models, including those without specific mechanisms like attention. Extensive experiments on three datasets demonstrate the competitive performance of our method. Code will be available at https://github.com/centurion-crawler/CIMIL.
37
Ronchetti, Franco, Facundo Quiroga, Ulises Jeremias Cornejo Fandos, Gastón Gustavo Rios, Pedro Dal Bianco, Waldo Hasperué, and Laura Lanzarini. "comparison of small sample methods for Handshape Recognition." Journal of Computer Science and Technology 23, no. 1 (April 3, 2023): e03. http://dx.doi.org/10.24215/16666038.23.e03.
Abstract:
Automatic Sign Language Translation (SLT) systems can be a great asset to improve the communication with and within deaf communities. Currently, the main issue preventing effective translation models lays in the low availability of labelled data, which hinders the use of modern deep learning models.
SLT is a complex problem that involves many subtasks, of which handshape recognition is the most important. We compare a series of models specially tailored for small datasets to improve their performance on handshape recognition tasks. We evaluate Wide-DenseNet and few-shot Prototypical Network models with and without transfer learning, and also using Model-Agnostic Meta-Learning (MAML).
Our findings indicate that Wide-DenseNet without transfer learning and Prototipical Networks with transfer learning provide the best results. Prototypical networks, particularly, are vastly superior when using less than 30 samples, while Wide-DenseNet achieves the best results with more samples. On the other hand, MAML does not improve performance in any scenario. These results can help to design better SLT models.
38
Demertzis, Konstantinos, and Lazaros Iliadis. "GeoAI: A Model-Agnostic Meta-Ensemble Zero-Shot Learning Method for Hyperspectral Image Analysis and Classification." Algorithms 13, no. 3 (March 7, 2020): 61. http://dx.doi.org/10.3390/a13030061.
Abstract:
Deep learning architectures are the most effective methods for analyzing and classifying Ultra-Spectral Images (USI). However, effective training of a Deep Learning (DL) gradient classifier aiming to achieve high classification accuracy, is extremely costly and time-consuming. It requires huge datasets with hundreds or thousands of labeled specimens from expert scientists. This research exploits the MAML++ algorithm in order to introduce the Model-Agnostic Meta-Ensemble Zero-shot Learning (MAME-ZsL) approach. The MAME-ZsL overcomes the above difficulties, and it can be used as a powerful model to perform Hyperspectral Image Analysis (HIA). It is a novel optimization-based Meta-Ensemble Learning architecture, following a Zero-shot Learning (ZsL) prototype. To the best of our knowledge it is introduced to the literature for the first time. It facilitates learning of specialized techniques for the extraction of user-mediated representations, in complex Deep Learning architectures. Moreover, it leverages the use of first and second-order derivatives as pre-training methods. It enhances learning of features which do not cause issues of exploding or diminishing gradients; thus, it avoids potential overfitting. Moreover, it significantly reduces computational cost and training time, and it offers an improved training stability, high generalization performance and remarkable classification accuracy.
39
Fan, Xinchen, Lancheng Zou, Ziwu Liu, Yanru He, Lian Zou, and Ruan Chi. "CSAC-Net: Fast Adaptive sEMG Recognition through Attention Convolution Network and Model-Agnostic Meta-Learning." Sensors 22, no. 10 (May 11, 2022): 3661. http://dx.doi.org/10.3390/s22103661.
Abstract:
Gesture recognition through surface electromyography (sEMG) provides a new method for the control algorithm of bionic limbs, which is a promising technology in the field of human–computer interaction. However, subject specificity of sEMG along with the offset of the electrode makes it challenging to develop a model that can quickly adapt to new subjects. In view of this, we introduce a new deep neural network called CSAC-Net. Firstly, we extract the time-frequency feature from the raw signal, which contains rich information. Secondly, we design a convolutional neural network supplemented by an attention mechanism for further feature extraction. Additionally, we propose to utilize model-agnostic meta-learning to adapt to new subjects and this learning strategy achieves better results than the state-of-the-art methods. By the basic experiment on CapgMyo and three ablation studies, we demonstrate the advancement of CSAC-Net.
40
Saarela, Mirka, and Lilia Geogieva. "Robustness, Stability, and Fidelity of Explanations for a Deep Skin Cancer Classification Model." Applied Sciences 12, no. 19 (September 23, 2022): 9545. http://dx.doi.org/10.3390/app12199545.
Abstract:
Skin cancer is one of the most prevalent of all cancers. Because of its being widespread and externally observable, there is a potential that machine learning models integrated into artificial intelligence systems will allow self-screening and automatic analysis in the future. Especially, the recent success of various deep machine learning models shows promise that, in the future, patients could self-analyse their external signs of skin cancer by uploading pictures of these signs to an artificial intelligence system, which runs such a deep learning model and returns the classification results. However, both patients and dermatologists, who might use such a system to aid their work, need to know why the system has made a particular decision. Recently, several explanation techniques for the deep learning algorithm’s decision-making process have been introduced. This study compares two popular local explanation techniques (integrated gradients and local model-agnostic explanations) for image data on top of a well-performing (80% accuracy) deep learning algorithm trained on the HAM10000 dataset, a large public collection of dermatoscopic images. Our results show that both methods have full local fidelity. However, the integrated gradients explanations perform better with regard to quantitative evaluation metrics (stability and robustness), while the model-agnostic method seem to provide more intuitive explanations. We conclude that there is still a long way before such automatic systems can be used reliably in practice.
41
Winder, Isabelle, and Nick Winder. "An agnostic approach to ancient landscapes." Journal of Archaeology and Ancient History, no. 9 (February 13, 2023): 1–30. http://dx.doi.org/10.33063/jaah.vi9.130.
Abstract:
We argue that the phenomenological or ‘agnostic’ approach to evolutionary systems advocated by Thomas Henry Huxley is applicable in anthropological archaeology and show how agnosticism helps defuse the tension between humanists, natural philosophers and natural historians in integrative research. We deploy problem-framing methods from policy-relevant research in a palaeoanthropological context, developing a model of complex (scale-dependent, irreversible) causality and applying it to the problem of human-landscape interaction and primate foot anatomy. We illustrate this process with a single iteration of the ‘project cycle’ focussed on human-landscape interaction. Modern humans are co-operative resilience feeders, exploiting complex causality by perturbing stable, unproductive landscapes and feeding on the fluxes of energy and resources released as they spring back. Is it possible that this resiliencefeeding is older than Homo sapiens?
42
Akhtar, Naveed, and Mohammad Amir Asim Khan Jalwana. "Rethinking Interpretation: Input-Agnostic Saliency Mapping of Deep Visual Classifiers." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 1 (June 26, 2023): 178–86. http://dx.doi.org/10.1609/aaai.v37i1.25089.
Abstract:
Saliency methods provide post-hoc model interpretation by attributing input features to the model outputs. Current methods mainly achieve this using a single input sample, thereby failing to answer input-independent inquiries about the model. We also show that input-specific saliency mapping is intrinsically susceptible to misleading feature attribution. Current attempts to use `general' input features for model interpretation assume access to a dataset containing those features, which biases the interpretation. Addressing the gap, we introduce a new perspective of input-agnostic saliency mapping that computationally estimates the high-level features attributed by the model to its outputs. These features are geometrically correlated, and are computed by accumulating model's gradient information with respect to an unrestricted data distribution. To compute these features, we nudge independent data points over the model loss surface towards the local minima associated by a human-understandable concept, e.g., class label for classifiers. With a systematic projection, scaling and refinement process, this information is transformed into an interpretable visualization without compromising its model-fidelity. The visualization serves as a stand-alone qualitative interpretation. With an extensive evaluation, we not only demonstrate successful visualizations for a variety of concepts for large-scale models, but also showcase an interesting utility of this new form of saliency mapping by identifying backdoor signatures in compromised classifiers.
43
Pruthi, Danish, Rachit Bansal, Bhuwan Dhingra, Livio Baldini Soares, Michael Collins, Zachary C. Lipton, Graham Neubig, and William W. Cohen. "Evaluating Explanations: How Much Do Explanations from the Teacher Aid Students?" Transactions of the Association for Computational Linguistics 10 (2022): 359–75. http://dx.doi.org/10.1162/tacl_a_00465.
Abstract:
Abstract While many methods purport to explain predictions by highlighting salient features, what aims these explanations serve and how they ought to be evaluated often go unstated. In this work, we introduce a framework to quantify the value of explanations via the accuracy gains that they confer on a student model trained to simulate a teacher model. Crucially, the explanations are available to the student during training, but are not available at test time. Compared with prior proposals, our approach is less easily gamed, enabling principled, automatic, model-agnostic evaluation of attributions. Using our framework, we compare numerous attribution methods for text classification and question answering, and observe quantitative differences that are consistent (to a moderate to high degree) across different student model architectures and learning strategies.1
44
Mentias, Amgad, Eric D. Peterson, Neil Keshvani, Dharam J. Kumbhani, Clyde W. Yancy, Alanna A. Morris, Larry A. Allen, et al. "Achieving Equity in Hospital Performance Assessments Using Composite Race-Specific Measures of Risk-Standardized Readmission and Mortality Rates for Heart Failure." Circulation 147, no. 15 (April 11, 2023): 1121–33. http://dx.doi.org/10.1161/circulationaha.122.061995.
Abstract:
Background: The contemporary measures of hospital performance for heart failure hospitalization and 30-day risk-standardized readmission rate (RSRR) and risk-standardized mortality rate (RSMR) are estimated using the same risk adjustment model and overall event rate for all patients. Thus, these measures are mainly driven by the care quality and outcomes for the majority racial and ethnic group, and may not adequately represent the hospital performance for patients of Black and other races. Methods: Fee-for-service Medicare beneficiaries from January 2014 to December 2019 hospitalized with heart failure were identified. Hospital-level 30-day RSRR and RSMR were estimated using the traditional race-agnostic models and the race-specific approach. The composite race-specific performance metric was calculated as the average of the RSRR/RMSR measures derived separately for each race and ethnicity group. Correlation and concordance in hospital performance for all patients and patients of Black and other races were assessed using the composite race-specific and race-agnostic metrics. Results: The study included 1 903 232 patients (75.7% White [n=1 439 958]; 14.5% Black [n=276 684]; and 9.8% other races [n=186 590]) with heart failure from 1860 hospitals. There was a modest correlation between hospital-level 30-day performance metrics for patients of White versus Black race (Pearson correlation coefficient: RSRR=0.42; RSMR=0.26). Compared with the race-agnostic RSRR and RSMR, composite race-specific metrics for all patients demonstrated stronger correlation with RSRR (correlation coefficient: 0.60 versus 0.74) and RSMR (correlation coefficient: 0.44 versus 0.51) for Black patients. Concordance in hospital performance for all patients and patients of Black race was also higher with race-specific (versus race-agnostic) metrics (RSRR=64% versus 53% concordantly high-performing; 61% versus 51% concordantly low-performing). Race-specific RSRR and RSMR metrics (versus race-agnostic) led to reclassification in performance ranking of 35.8% and 39.2% of hospitals, respectively, with better 30-day and 1-year outcomes for patients of all race groups at hospitals reclassified as high-performing. Conclusions: Among patients hospitalized with heart failure, race-specific 30-day RSMR and RSRR are more equitable in representing hospital performance for patients of Black and other races.
45
Yarlagadda, Sri Kalyan, Daniel Mas Montserrat, David Güera, Carol J. Boushey, Deborah A. Kerr, and Fengqing Zhu. "Saliency-Aware Class-Agnostic Food Image Segmentation." ACM Transactions on Computing for Healthcare 2, no. 3 (July 2021): 1–17. http://dx.doi.org/10.1145/3440274.
Abstract:
Advances in image-based dietary assessment methods have allowed nutrition professionals and researchers to improve the accuracy of dietary assessment, where images of food consumed are captured using smartphones or wearable devices. These images are then analyzed using computer vision methods to estimate energy and nutrition content of the foods. Food image segmentation, which determines the regions in an image where foods are located, plays an important role in this process. Current methods are data dependent and thus cannot generalize well for different food types. To address this problem, we propose a class-agnostic food image segmentation method. Our method uses a pair of eating scene images, one before starting eating and one after eating is completed. Using information from both the before and after eating images, we can segment food images by finding the salient missing objects without any prior information about the food class. We model a paradigm of top-down saliency that guides the attention of the human visual system based on a task to find the salient missing objects in a pair of images. Our method is validated on food images collected from a dietary study that showed promising results.
46
Sun, Chenyu, Hangwei Qian, and Chunyan Miao. "CUDC: A Curiosity-Driven Unsupervised Data Collection Method with Adaptive Temporal Distances for Offline Reinforcement Learning." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 13 (March 24, 2024): 15145–53. http://dx.doi.org/10.1609/aaai.v38i13.29437.
Abstract:
Offline reinforcement learning (RL) aims to learn an effective policy from a pre-collected dataset. Most existing works are to develop sophisticated learning algorithms, with less emphasis on improving the data collection process. Moreover, it is even challenging to extend the single-task setting and collect a task-agnostic dataset that allows an agent to perform multiple downstream tasks. In this paper, we propose a Curiosity-driven Unsupervised Data Collection (CUDC) method to expand feature space using adaptive temporal distances for task-agnostic data collection and ultimately improve learning efficiency and capabilities for multi-task offline RL. To achieve this, CUDC estimates the probability of the k-step future states being reachable from the current states, and adapts how many steps into the future that the dynamics model should predict. With this adaptive reachability mechanism in place, the feature representation can be diversified, and the agent can navigate itself to collect higher-quality data with curiosity. Empirically, CUDC surpasses existing unsupervised methods in efficiency and learning performance in various downstream offline RL tasks of the DeepMind control suite.
47
Petrescu, Livia, Cătălin Petrescu, Ana Oprea, Oana Mitruț, Gabriela Moise, Alin Moldoveanu, and Florica Moldoveanu. "Machine Learning Methods for Fear Classification Based on Physiological Features." Sensors 21, no. 13 (July 1, 2021): 4519. http://dx.doi.org/10.3390/s21134519.
Abstract:
This paper focuses on the binary classification of the emotion of fear, based on the physiological data and subjective responses stored in the DEAP dataset. We performed a mapping between the discrete and dimensional emotional information considering the participants’ ratings and extracted a substantial set of 40 types of features from the physiological data, which represented the input to various machine learning algorithms—Decision Trees, k-Nearest Neighbors, Support Vector Machine and artificial networks—accompanied by dimensionality reduction, feature selection and the tuning of the most relevant hyperparameters, boosting classification accuracy. The methodology we approached included tackling different situations, such as resolving the problem of having an imbalanced dataset through data augmentation, reducing overfitting, computing various metrics in order to obtain the most reliable classification scores and applying the Local Interpretable Model-Agnostic Explanations method for interpretation and for explaining predictions in a human-understandable manner. The results show that fear can be predicted very well (accuracies ranging from 91.7% using Gradient Boosting Trees to 93.5% using dimensionality reduction and Support Vector Machine) by extracting the most relevant features from the physiological data and by searching for the best parameters which maximize the machine learning algorithms’ classification scores.
48
Rangwala, Murtaza, Jun Liu, Kulbir Singh Ahluwalia, Shayan Ghajar, Harnaik Singh Dhami, Benjamin F. Tracy, Pratap Tokekar, and Ryan K. Williams. "DeepPaSTL: Spatio-Temporal Deep Learning Methods for Predicting Long-Term Pasture Terrains Using Synthetic Datasets." Agronomy 11, no. 11 (November 5, 2021): 2245. http://dx.doi.org/10.3390/agronomy11112245.
Abstract:
Effective management of dairy farms requires an accurate prediction of pasture biomass. Generally, estimation of pasture biomass requires site-specific data, or often perfect world assumptions to model prediction systems when field measurements or other sensory inputs are unavailable. However, for small enterprises, regular measurements of site-specific data are often inconceivable. In this study, we approach the estimation of pasture biomass by predicting sward heights across the field. A convolution based sequential architecture is proposed for pasture height predictions using deep learning. We develop a process to create synthetic datasets that simulate the evolution of pasture growth over a period of 30 years. The deep learning based pasture prediction model (DeepPaSTL) is trained on this dataset while learning the spatiotemporal characteristics of pasture growth. The architecture purely learns from the trends in pasture growth through available spatial measurements and is agnostic to any site-specific data, or climatic conditions, such as temperature, precipitation, or soil condition. Our model performs within a 12% error margin even during the periods with the largest pasture growth dynamics. The study demonstrates the potential scalability of the architecture to predict any pasture size through a quantization approach during prediction. Results suggest that the DeepPaSTL model represents a useful tool for predicting pasture growth both for short and long horizon predictions, even with missing or irregular historical measurements.
49
Christensen, Cade, Torrey Wagner, and Brent Langhals. "Year-Independent Prediction of Food Insecurity Using Classical and Neural Network Machine Learning Methods." AI 2, no. 2 (May 23, 2021): 244–60. http://dx.doi.org/10.3390/ai2020015.
Abstract:
Current food crisis predictions are developed by the Famine Early Warning System Network, but they fail to classify the majority of food crisis outbreaks with model metrics of recall (0.23), precision (0.42), and f1 (0.30). In this work, using a World Bank dataset, classical and neural network (NN) machine learning algorithms were developed to predict food crises in 21 countries. The best classical logistic regression algorithm achieved a high level of significance (p < 0.001) and precision (0.75) but was deficient in recall (0.20) and f1 (0.32). Of particular interest, the classical algorithm indicated that the vegetation index and the food price index were both positively correlated with food crises. A novel method for performing an iterative multidimensional hyperparameter search is presented, which resulted in significantly improved performance when applied to this dataset. Four iterations were conducted, which resulted in excellent 0.96 for metrics of precision, recall, and f1. Due to this strong performance, the food crisis year was removed from the dataset to prevent immediate extrapolation when used on future data, and the modeling process was repeated. The best “no year” model metrics remained strong, achieving ≥0.92 for recall, precision, and f1 while meeting a 10% f1 overfitting threshold on the test (0.84) and holdout (0.83) datasets. The year-agnostic neural network model represents a novel approach to classify food crises and outperforms current food crisis prediction efforts.
50
Yamazawa, Erika, Satoshi Takahashi, Shota Tanaka, Wataru Takahashi, Takahiro Nakamoto, Shunsaku Takayanagi, Yosuke Kitagawa, et al. "RARE-16. A NOVEL RADIOMICS MODEL DIFFERENTIATING CHORDOMA AND CHONDROSARCOMA." Neuro-Oncology 21, Supplement_6 (November 2019): vi224—vi225. http://dx.doi.org/10.1093/neuonc/noz175.939.
Abstract:
Abstract BACKGROUND Chordoma and chondrosarcoma account for the majority of skull base tumors affecting the petroclival region. Chondrosarcoma has better prognosis than chordoma; heavy particle therapy is often indicated for residual/recurrent chordoma. Preoperative, precise diagnosis of the tumor would be desirable, as it can potentially impact on the choice of a surgical approach and the aggressiveness of surgery. METHODS We conducted a radiomics study to create a machine learning model distinguishing chondrosarcoma from chordoma. We collected DICOM T2-weighted images and T1-weighted images with gadolinium (GdT1) enhancement in the consective patients of chordoma or chondrosarcoma who underwent surgery at The University of Tokyo Hospital from September of 2012 to January of 2019. We selected patients with uniform MRI images. VOI (volume of interest) was set using Monaco (https://www.elekta.com/software-solutions/treatment-management/external-beam-planning/monaco.html). Not only sematic features but also agnostic features were calculated. The original images and 8 wavelet transformed images were calculated for texture agnostic features such as Gray-Level Co-occurrence Matrix (GLCM). Features were selected by recursive feature elimination (RFE). The final model evaluation was performed by average area under the curve (AUC). RESULTS The study population included 17 chordomas and 22 chondrosarcomas in a total of 39 patients. 476 features were obtained per image sequence. The number of features per case was 476 × 2 = 952 The most accurate machine learning model was created using the extracted three features from only T2. The best AUC was 0.77 ± 0.11 in logistic regression (dataset was divided randomly into halves, average value of AUC calculated six times). CONCLUSIONS This novel machine learning model can differentiate chordoma and chondrosarcoma reasonably well. A validation study with a larger number of patients is warranted.