To see the other types of publications on this topic, follow the link: Post-hoc explainabil.
Journal articles on the topic 'Post-hoc explainabil'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 43 journal articles for your research on the topic 'Post-hoc explainabil.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Zednik, Carlos, and Hannes Boelsen. "Scientific Exploration and Explainable Artificial Intelligence." Minds and Machines 32, no. 1 (March 2022): 219–39. http://dx.doi.org/10.1007/s11023-021-09583-6.
Full textAbstract:
AbstractModels developed using machine learning are increasingly prevalent in scientific research. At the same time, these models are notoriously opaque. Explainable AI aims to mitigate the impact of opacity by rendering opaque models transparent. More than being just the solution to a problem, however, Explainable AI can also play an invaluable role in scientific exploration. This paper describes how post-hoc analytic techniques from Explainable AI can be used to refine target phenomena in medical science, to identify starting points for future investigations of (potentially) causal relationships, and to generate possible explanations of target phenomena in cognitive science. In this way, this paper describes how Explainable AI—over and above machine learning itself—contributes to the efficiency and scope of data-driven scientific research.
2
Fauvel, Kevin, Tao Lin, Véronique Masson, Élisa Fromont, and Alexandre Termier. "XCM: An Explainable Convolutional Neural Network for Multivariate Time Series Classification." Mathematics 9, no. 23 (December 5, 2021): 3137. http://dx.doi.org/10.3390/math9233137.
Full textAbstract:
Multivariate Time Series (MTS) classification has gained importance over the past decade with the increase in the number of temporal datasets in multiple domains. The current state-of-the-art MTS classifier is a heavyweight deep learning approach, which outperforms the second-best MTS classifier only on large datasets. Moreover, this deep learning approach cannot provide faithful explanations as it relies on post hoc model-agnostic explainability methods, which could prevent its use in numerous applications. In this paper, we present XCM, an eXplainable Convolutional neural network for MTS classification. XCM is a new compact convolutional neural network which extracts information relative to the observed variables and time directly from the input data. Thus, XCM architecture enables a good generalization ability on both large and small datasets, while allowing the full exploitation of a faithful post hoc model-specific explainability method (Gradient-weighted Class Activation Mapping) by precisely identifying the observed variables and timestamps of the input data that are important for predictions. We first show that XCM outperforms the state-of-the-art MTS classifiers on both the large and small public UEA datasets. Then, we illustrate how XCM reconciles performance and explainability on a synthetic dataset and show that XCM enables a more precise identification of the regions of the input data that are important for predictions compared to the current deep learning MTS classifier also providing faithful explainability. Finally, we present how XCM can outperform the current most accurate state-of-the-art algorithm on a real-world application while enhancing explainability by providing faithful and more informative explanations.
3
Roscher, R., B. Bohn, M. F. Duarte, and J. Garcke. "EXPLAIN IT TO ME – FACING REMOTE SENSING CHALLENGES IN THE BIO- AND GEOSCIENCES WITH EXPLAINABLE MACHINE LEARNING." ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences V-3-2020 (August 3, 2020): 817–24. http://dx.doi.org/10.5194/isprs-annals-v-3-2020-817-2020.
Full textAbstract:
Abstract. For some time now, machine learning methods have been indispensable in many application areas. Especially with the recent development of efficient neural networks, these methods are increasingly used in the sciences to obtain scientific outcomes from observational or simulated data. Besides a high accuracy, a desired goal is to learn explainable models. In order to reach this goal and obtain explanation, knowledge from the respective domain is necessary, which can be integrated into the model or applied post-hoc. We discuss explainable machine learning approaches which are used to tackle common challenges in the bio- and geosciences, such as limited amount of labeled data or the provision of reliable and scientific consistent results. We show that recent advances in machine learning to enhance transparency, interpretability, and explainability are helpful in overcoming these challenges.
4
Gadzinski, Gregory, and Alessio Castello. "Combining white box models, black box machines and human interventions for interpretable decision strategies." Judgment and Decision Making 17, no. 3 (May 2022): 598–627. http://dx.doi.org/10.1017/s1930297500003594.
Full textAbstract:
AbstractGranting a short-term loan is a critical decision. A great deal of research has concerned the prediction of credit default, notably through Machine Learning (ML) algorithms. However, given that their black-box nature has sometimes led to unwanted outcomes, comprehensibility in ML guided decision-making strategies has become more important. In many domains, transparency and accountability are no longer optional. In this article, instead of opposing white-box against black-box models, we use a multi-step procedure that combines the Fast and Frugal Tree (FFT) methodology of Martignon et al. (2005) and Phillips et al. (2017) with the extraction of post-hoc explainable information from ensemble ML models. New interpretable models are then built thanks to the inclusion of explainable ML outputs chosen by human intervention. Our methodology improves significantly the accuracy of the FFT predictions while preserving their explainable nature. We apply our approach to a dataset of short-term loans granted to borrowers in the UK, and show how complex machine learning can challenge simpler machines and help decision makers.
5
Shen, Yifan, Li Liu, Zhihao Tang, Zongyi Chen, Guixiang Ma, Jiyan Dong, Xi Zhang, Lin Yang, and Qingfeng Zheng. "Explainable Survival Analysis with Convolution-Involved Vision Transformer." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 2 (June 28, 2022): 2207–15. http://dx.doi.org/10.1609/aaai.v36i2.20118.
Full textAbstract:
Image-based survival prediction models can facilitate doctors in diagnosing and treating cancer patients. With the advance of digital pathology technologies, the big whole slide images (WSIs) provide increasing resolution and more details for diagnosis. However, the gigabyte-size WSIs would make most models computationally infeasible. To this end, instead of using the complete WSIs, most of existing models only use a pre-selected subset of key patches or patch clusters as input, which might fail to completely capture the patient's tumor morphology. In this work, we aim to develop a novel survival analysis model to fully utilize the complete WSI information. We show that the use of a Vision Transformer (ViT) backbone, together with convolution operations involved in it, is an effective framework to improve the prediction performance. Additionally, we present a post-hoc explainable method to identify the most salient patches and distinct morphology features, making the model more faithful and the results easier to comprehend by human users. Evaluations on two large cancer datasets show that our proposed model is more effective and has better interpretability for survival prediction.
6
Gill, Navdeep, Patrick Hall, Kim Montgomery, and Nicholas Schmidt. "A Responsible Machine Learning Workflow with Focus on Interpretable Models, Post-hoc Explanation, and Discrimination Testing." Information 11, no. 3 (February 29, 2020): 137. http://dx.doi.org/10.3390/info11030137.
Full textAbstract:
This manuscript outlines a viable approach for training and evaluating machine learning systems for high-stakes, human-centered, or regulated applications using common Python programming tools. The accuracy and intrinsic interpretability of two types of constrained models, monotonic gradient boosting machines and explainable neural networks, a deep learning architecture well-suited for structured data, are assessed on simulated data and publicly available mortgage data. For maximum transparency and the potential generation of personalized adverse action notices, the constrained models are analyzed using post-hoc explanation techniques including plots of partial dependence and individual conditional expectation and with global and local Shapley feature importance. The constrained model predictions are also tested for disparate impact and other types of discrimination using measures with long-standing legal precedents, adverse impact ratio, marginal effect, and standardized mean difference, along with straightforward group fairness measures. By combining interpretable models, post-hoc explanations, and discrimination testing with accessible software tools, this text aims to provide a template workflow for machine learning applications that require high accuracy and interpretability and that mitigate risks of discrimination.
7
Aslam, Nida, Irfan Ullah Khan, Samiha Mirza, Alanoud AlOwayed, Fatima M. Anis, Reef M. Aljuaid, and Reham Baageel. "Interpretable Machine Learning Models for Malicious Domains Detection Using Explainable Artificial Intelligence (XAI)." Sustainability 14, no. 12 (June 16, 2022): 7375. http://dx.doi.org/10.3390/su14127375.
Full textAbstract:
With the expansion of the internet, a major threat has emerged involving the spread of malicious domains intended by attackers to perform illegal activities aiming to target governments, violating privacy of organizations, and even manipulating everyday users. Therefore, detecting these harmful domains is necessary to combat the growing network attacks. Machine Learning (ML) models have shown significant outcomes towards the detection of malicious domains. However, the “black box” nature of the complex ML models obstructs their wide-ranging acceptance in some of the fields. The emergence of Explainable Artificial Intelligence (XAI) has successfully incorporated the interpretability and explicability in the complex models. Furthermore, the post hoc XAI model has enabled the interpretability without affecting the performance of the models. This study aimed to propose an Explainable Artificial Intelligence (XAI) model to detect malicious domains on a recent dataset containing 45,000 samples of malicious and non-malicious domains. In the current study, initially several interpretable ML models, such as Decision Tree (DT) and Naïve Bayes (NB), and black box ensemble models, such as Random Forest (RF), Extreme Gradient Boosting (XGB), AdaBoost (AB), and Cat Boost (CB) algorithms, were implemented and found that XGB outperformed the other classifiers. Furthermore, the post hoc XAI global surrogate model (Shapley additive explanations) and local surrogate LIME were used to generate the explanation of the XGB prediction. Two sets of experiments were performed; initially the model was executed using a preprocessed dataset and later with selected features using the Sequential Forward Feature selection algorithm. The results demonstrate that ML algorithms were able to distinguish benign and malicious domains with overall accuracy ranging from 0.8479 to 0.9856. The ensemble classifier XGB achieved the highest result, with an AUC and accuracy of 0.9991 and 0.9856, respectively, before the feature selection algorithm, while there was an AUC of 0.999 and accuracy of 0.9818 after the feature selection algorithm. The proposed model outperformed the benchmark study.
8
Mikołajczyk, Agnieszka, Michał Grochowski, and Arkadiusz Kwasigroch. "Towards Explainable Classifiers Using the Counterfactual Approach - Global Explanations for Discovering Bias in Data." Journal of Artificial Intelligence and Soft Computing Research 11, no. 1 (January 1, 2021): 51–67. http://dx.doi.org/10.2478/jaiscr-2021-0004.
Full textAbstract:
AbstractThe paper proposes summarized attribution-based post-hoc explanations for the detection and identification of bias in data. A global explanation is proposed, and a step-by-step framework on how to detect and test bias is introduced. Since removing unwanted bias is often a complicated and tremendous task, it is automatically inserted, instead. Then, the bias is evaluated with the proposed counterfactual approach. The obtained results are validated on a sample skin lesion dataset. Using the proposed method, a number of possible bias-causing artifacts are successfully identified and confirmed in dermoscopy images. In particular, it is confirmed that black frames have a strong influence on Convolutional Neural Network’s prediction: 22% of them changed the prediction from benign to malignant.
9
Kumar, Akshi, Shubham Dikshit, and Victor Hugo C. Albuquerque. "Explainable Artificial Intelligence for Sarcasm Detection in Dialogues." Wireless Communications and Mobile Computing 2021 (July 2, 2021): 1–13. http://dx.doi.org/10.1155/2021/2939334.
Full textAbstract:
Sarcasm detection in dialogues has been gaining popularity among natural language processing (NLP) researchers with the increased use of conversational threads on social media. Capturing the knowledge of the domain of discourse, context propagation during the course of dialogue, and situational context and tone of the speaker are some important features to train the machine learning models for detecting sarcasm in real time. As situational comedies vibrantly represent human mannerism and behaviour in everyday real-life situations, this research demonstrates the use of an ensemble supervised learning algorithm to detect sarcasm in the benchmark dialogue dataset, MUStARD. The punch-line utterance and its associated context are taken as features to train the eXtreme Gradient Boosting (XGBoost) method. The primary goal is to predict sarcasm in each utterance of the speaker using the chronological nature of a scene. Further, it is vital to prevent model bias and help decision makers understand how to use the models in the right way. Therefore, as a twin goal of this research, we make the learning model used for conversational sarcasm detection interpretable. This is done using two post hoc interpretability approaches, Local Interpretable Model-agnostic Explanations (LIME) and Shapley Additive exPlanations (SHAP), to generate explanations for the output of a trained classifier. The classification results clearly depict the importance of capturing the intersentence context to detect sarcasm in conversational threads. The interpretability methods show the words (features) that influence the decision of the model the most and help the user understand how the model is making the decision for detecting sarcasm in dialogues.
10
Knapič, Samanta, Avleen Malhi, Rohit Saluja, and Kary Främling. "Explainable Artificial Intelligence for Human Decision Support System in the Medical Domain." Machine Learning and Knowledge Extraction 3, no. 3 (September 19, 2021): 740–70. http://dx.doi.org/10.3390/make3030037.
Full textAbstract:
In this paper, we present the potential of Explainable Artificial Intelligence methods for decision support in medical image analysis scenarios. Using three types of explainable methods applied to the same medical image data set, we aimed to improve the comprehensibility of the decisions provided by the Convolutional Neural Network (CNN). In vivo gastral images obtained by a video capsule endoscopy (VCE) were the subject of visual explanations, with the goal of increasing health professionals’ trust in black-box predictions. We implemented two post hoc interpretable machine learning methods, called Local Interpretable Model-Agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP), and an alternative explanation approach, the Contextual Importance and Utility (CIU) method. The produced explanations were assessed by human evaluation. We conducted three user studies based on explanations provided by LIME, SHAP and CIU. Users from different non-medical backgrounds carried out a series of tests in a web-based survey setting and stated their experience and understanding of the given explanations. Three user groups (n = 20, 20, 20) with three distinct forms of explanations were quantitatively analyzed. We found that, as hypothesized, the CIU-explainable method performed better than both LIME and SHAP methods in terms of improving support for human decision-making and being more transparent and thus understandable to users. Additionally, CIU outperformed LIME and SHAP by generating explanations more rapidly. Our findings suggest that there are notable differences in human decision-making between various explanation support settings. In line with that, we present three potential explainable methods that, with future improvements in implementation, can be generalized to different medical data sets and can provide effective decision support to medical experts.
11
Vieira, Carla Piazzon Ramos, and Luciano Antonio Digiampietri. "A study about Explainable Articial Intelligence: using decision tree to explain SVM." Revista Brasileira de Computação Aplicada 12, no. 1 (January 8, 2020): 113–21. http://dx.doi.org/10.5335/rbca.v12i1.10247.
Full textAbstract:
The technologies supporting Artificial Intelligence (AI) have advanced rapidly over the past few years and AI is becoming a commonplace in every aspect of life like the future of self-driving cars or earlier health diagnosis. For this to occur shortly, the entire community stands in front of the barrier of explainability, an inherent problem of latest models (e.g. Deep Neural Networks) that were not present in the previous hype of AI (linear and rule-based models). Most of these recent models are used as black boxes without understanding partially or even completely how different features influence the model prediction avoiding algorithmic transparency. In this paper, we focus on how much we can understand the decisions made by an SVM Classifier in a post-hoc model agnostic approach. Furthermore, we train a tree-based model (inherently interpretable) using labels from the SVM, called secondary training data to provide explanations and compare permutation importance method to the more commonly used measures such as accuracy and show that our methods are both more reliable and meaningful techniques to use. We also outline the main challenges for such methods and conclude that model-agnostic interpretability is a key component in making machine learning more trustworthy.
12
Apostolopoulos, Ioannis D., Ifigeneia Athanasoula, Mpesi Tzani, and Peter P. Groumpos. "An Explainable Deep Learning Framework for Detecting and Localising Smoke and Fire Incidents: Evaluation of Grad-CAM++ and LIME." Machine Learning and Knowledge Extraction 4, no. 4 (December 6, 2022): 1124–35. http://dx.doi.org/10.3390/make4040057.
Full textAbstract:
Climate change is expected to increase fire events and activity with multiple impacts on human lives. Large grids of forest and city monitoring devices can assist in incident detection, accelerating human intervention in extinguishing fires before they get out of control. Artificial Intelligence promises to automate the detection of fire-related incidents. This study enrols 53,585 fire/smoke and normal images and benchmarks seventeen state-of-the-art Convolutional Neural Networks for distinguishing between the two classes. The Xception network proves to be superior to the rest of the CNNs, obtaining very high accuracy. Grad-CAM++ and LIME algorithms improve the post hoc explainability of Xception and verify that it is learning features found in the critical locations of the image. Both methods agree on the suggested locations, strengthening the abovementioned outcome.
13
Lossos, Christian, Simon Geschwill, and Frank Morelli. "Offenheit durch XAI bei ML-unterstützten Entscheidungen: Ein Baustein zur Optimierung von Entscheidungen im Unternehmen?" HMD Praxis der Wirtschaftsinformatik 58, no. 2 (March 3, 2021): 303–20. http://dx.doi.org/10.1365/s40702-021-00707-1.
Full textAbstract:
ZusammenfassungKünstliche Intelligenz (KI) und Machine Learning (ML) gelten gegenwärtig als probate Mittel, um betriebswirtschaftliche Entscheidungen durch mathematische Modelle zu optimieren. Allerdings werden die Technologien häufig in Form von „Black Box“-Ansätze mit entsprechenden Risiken realisiert. Der Einsatz von Offenheit kann in diesem Kontext mehr Objektivität schaffen und als Treiber für innovative Lösungen fungieren. Rationale Entscheidungen im Unternehmen dienen im Sinne einer Mittel-Zweck-Beziehung dazu, Wettbewerbsvorteile zu erlangen. Im Sinne von Governance und Compliance sind dabei regulatorische Rahmenwerke wie COBIT 2019 und gesetzliche Grundlagen wie die Datenschutz-Grundverordnung (DSGVO) zu berücksichtigen, die ihrerseits ein Mindestmaß an Transparenz einfordern. Ferner sind auch Fairnessaspekte, die durch Bias-Effekte bei ML-Systemen beeinträchtigt werden können, zu berücksichtigen. In Teilaspekten, wie z. B. bei der Modellerstellung, wird in den Bereichen der KI und des ML das Konzept der Offenheit bereits praktiziert. Das Konzept der erklärbaren KI („Explainable Artificial Intelligence“ – XAI) vermag es aber, das zugehörige Potenzial erheblich steigern. Hierzu stehen verschiedene generische Ansätze (Ante hoc‑, Design- und Post-hoc-Konzepte) sowie die Möglichkeit, diese untereinander zu kombinieren, zur Verfügung. Entsprechend müssen Chancen und Grenzen von XAI systematisch reflektiert werden. Ein geeignetes, XAI-basiertes Modell für das Fällen von Entscheidungen im Unternehmen lässt sich mit Hilfe von Heuristiken näher charakterisieren.
14
Sudars, Kaspars, Ivars Namatēvs, and Kaspars Ozols. "Improving Performance of the PRYSTINE Traffic Sign Classification by Using a Perturbation-Based Explainability Approach." Journal of Imaging 8, no. 2 (January 30, 2022): 30. http://dx.doi.org/10.3390/jimaging8020030.
Full textAbstract:
Model understanding is critical in many domains, particularly those involved in high-stakes decisions, e.g., medicine, criminal justice, and autonomous driving. Explainable AI (XAI) methods are essential for working with black-box models such as convolutional neural networks. This paper evaluates the traffic sign classifier of the Deep Neural Network (DNN) from the Programmable Systems for Intelligence in Automobiles (PRYSTINE) project for explainability. The results of explanations were further used for the CNN PRYSTINE classifier vague kernels’ compression. Then, the precision of the classifier was evaluated in different pruning scenarios. The proposed classifier performance methodology was realised by creating an original traffic sign and traffic light classification and explanation code. First, the status of the kernels of the network was evaluated for explainability. For this task, the post-hoc, local, meaningful perturbation-based forward explainable method was integrated into the model to evaluate each kernel status of the network. This method enabled distinguishing high- and low-impact kernels in the CNN. Second, the vague kernels of the classifier of the last layer before the fully connected layer were excluded by withdrawing them from the network. Third, the network’s precision was evaluated in different kernel compression levels. It is shown that by using the XAI approach for network kernel compression, the pruning of 5% of kernels leads to a 2% loss in traffic sign and traffic light classification precision. The proposed methodology is crucial where execution time and processing capacity prevail.
15
Abbas, Asmaa, Mohamed Medhat Gaber, and Mohammed M. Abdelsamea. "XDecompo: Explainable Decomposition Approach in Convolutional Neural Networks for Tumour Image Classification." Sensors 22, no. 24 (December 15, 2022): 9875. http://dx.doi.org/10.3390/s22249875.
Full textAbstract:
Of the various tumour types, colorectal cancer and brain tumours are still considered among the most serious and deadly diseases in the world. Therefore, many researchers are interested in improving the accuracy and reliability of diagnostic medical machine learning models. In computer-aided diagnosis, self-supervised learning has been proven to be an effective solution when dealing with datasets with insufficient data annotations. However, medical image datasets often suffer from data irregularities, making the recognition task even more challenging. The class decomposition approach has provided a robust solution to such a challenging problem by simplifying the learning of class boundaries of a dataset. In this paper, we propose a robust self-supervised model, called XDecompo, to improve the transferability of features from the pretext task to the downstream task. XDecompo has been designed based on an affinity propagation-based class decomposition to effectively encourage learning of the class boundaries in the downstream task. XDecompo has an explainable component to highlight important pixels that contribute to classification and explain the effect of class decomposition on improving the speciality of extracted features. We also explore the generalisability of XDecompo in handling different medical datasets, such as histopathology for colorectal cancer and brain tumour images. The quantitative results demonstrate the robustness of XDecompo with high accuracy of 96.16% and 94.30% for CRC and brain tumour images, respectively. XDecompo has demonstrated its generalization capability and achieved high classification accuracy (both quantitatively and qualitatively) in different medical image datasets, compared with other models. Moreover, a post hoc explainable method has been used to validate the feature transferability, demonstrating highly accurate feature representations.
16
Srinivasu, Parvathaneni Naga, N. Sandhya, Rutvij H. Jhaveri, and Roshani Raut. "From Blackbox to Explainable AI in Healthcare: Existing Tools and Case Studies." Mobile Information Systems 2022 (June 13, 2022): 1–20. http://dx.doi.org/10.1155/2022/8167821.
Full textAbstract:
Introduction. Artificial intelligence (AI) models have been employed to automate decision-making, from commerce to more critical fields directly affecting human lives, including healthcare. Although the vast majority of these proposed AI systems are considered black box models that lack explainability, there is an increasing trend of attempting to create medical explainable Artificial Intelligence (XAI) systems using approaches such as attention mechanisms and surrogate models. An AI system is said to be explainable if humans can tell how the system reached its decision. Various XAI-driven healthcare approaches and their performances in the current study are discussed. The toolkits used in local and global post hoc explainability and the multiple techniques for explainability pertaining the Rational, Data, and Performance explainability are discussed in the current study. Methods. The explainability of the artificial intelligence model in the healthcare domain is implemented through the Local Interpretable Model-Agnostic Explanations and Shapley Additive Explanations for better comprehensibility of the internal working mechanism of the original AI models and the correlation among the feature set that influences decision of the model. Results. The current state-of-the-art XAI-based and future technologies through XAI are reported on research findings in various implementation aspects, including research challenges and limitations of existing models. The role of XAI in the healthcare domain ranging from the earlier prediction of future illness to the disease’s smart diagnosis is discussed. The metrics considered in evaluating the model’s explainability are presented, along with various explainability tools. Three case studies about the role of XAI in the healthcare domain with their performances are incorporated for better comprehensibility. Conclusion. The future perspective of XAI in healthcare will assist in obtaining research insight in the healthcare domain.
17
Ntakolia, Charis, Christos Kokkotis, Patrik Karlsson, and Serafeim Moustakidis. "An Explainable Machine Learning Model for Material Backorder Prediction in Inventory Management." Sensors 21, no. 23 (November 27, 2021): 7926. http://dx.doi.org/10.3390/s21237926.
Full textAbstract:
Global competition among businesses imposes a more effective and low-cost supply chain allowing firms to provide products at a desired quality, quantity, and time, with lower production costs. The latter include holding cost, ordering cost, and backorder cost. Backorder occurs when a product is temporarily unavailable or out of stock and the customer places an order for future production and shipment. Therefore, stock unavailability and prolonged delays in product delivery will lead to additional production costs and unsatisfied customers, respectively. Thus, it is of high importance to develop models that will effectively predict the backorder rate in an inventory system with the aim of improving the effectiveness of the supply chain and, consequentially, the performance of the company. However, traditional approaches in the literature are based on stochastic approximation, without incorporating information from historical data. To this end, machine learning models should be employed for extracting knowledge of large historical data to develop predictive models. Therefore, to cover this need, in this study, the backorder prediction problem was addressed. Specifically, various machine learning models were compared for solving the binary classification problem of backorder prediction, followed by model calibration and a post-hoc explainability based on the SHAP model to identify and interpret the most important features that contribute to material backorder. The results showed that the RF, XGB, LGBM, and BB models reached an AUC score of 0.95, while the best-performing model was the LGBM model after calibration with the Isotonic Regression method. The explainability analysis showed that the inventory stock of a product, the volume of products that can be delivered, the imminent demand (sales), and the accurate prediction of the future demand can significantly contribute to the correct prediction of backorders.
18
Foretic, Nikola, Vladimir Pavlinovic, and Miodrag Spasic. "Differences in Specific Power Performance among Playing Positions in Top Level Female Handball." Sport Mont 20, no. 1 (February 1, 2022): 109–13. http://dx.doi.org/10.26773/smj.220219.
Full textAbstract:
Since all activities in handball should be performed as fast and as powerful as possible, power is involved in almost all players’ movements. There is an evident lack of studies that monitored this ability during matches. The study aimed to determine the situational differences in power among playing positions in top-level female handball. Variables included: body height, average game time, fastest shot, fastest sprint, and highest jump. Subjects were 227 female handball players that participated in the European handball championship 2020. Analysis of variance with the post-hoc Scheffe test was calculated. Results showed significant differences among playing positions in body height, fastest sprint and highest jump performed in real game situations. The largest differences were noticed in anthropometrics, with significant differences between back- and pivot-players on one side, and wing-players on the other. The fastest sprinting was recorded for wingers (26.5±1.12 km/h), who were significantly faster than other players. Jumping performance was most diverse among playing positions, with back-players being superior in this performance (47.24±17.61 cm, 48.44±20.71 cm, and 50.35±16.76 cm for centre-backs, leftbacks, and right-backs, respectively). Evidenced differences are explainable knowing the specific positions’ roles and typical game situations which players encounter during the match.
19
Biswas, Shreyan, Lorenzo Corti, Stefan Buijsman, and Jie Yang. "CHIME: Causal Human-in-the-Loop Model Explanations." Proceedings of the AAAI Conference on Human Computation and Crowdsourcing 10, no. 1 (October 14, 2022): 27–39. http://dx.doi.org/10.1609/hcomp.v10i1.21985.
Full textAbstract:
Explaining the behaviour of Artificial Intelligence models has become a necessity. Their opaqueness and fragility are not tolerable in high-stakes domains especially. Although considerable progress is being made in the field of Explainable Artificial Intelligence, scholars have demonstrated limits and flaws of existing approaches: explanations requiring further interpretation, non-standardised explanatory format, and overall fragility. In light of this fragmentation, we turn to the field of philosophy of science to understand what constitutes a good explanation, that is, a generalisation that covers both the actual outcome and, possibly multiple, counterfactual outcomes. Inspired by this, we propose CHIME: a human-in-the-loop, post-hoc approach focused on creating such explanations by establishing the causal features in the input. We first elicit people's cognitive abilities to understand what parts of the input the model might be attending to. Then, through Causal Discovery we uncover the underlying causal graph relating the different concepts. Finally, with such a structure, we compute the causal effects different concepts have towards a model's outcome. We evaluate the Fidelity, Coherence, and Accuracy of the explanations obtained with CHIME with respect to two state-of-the-art Computer Vision models trained on real-world image data sets. We found evidence that the explanations reflect the causal concepts tied to a model's prediction, both in terms of causal strength and accuracy.
20
Ntakolia, Charis, Dimitrios Priftis, Mariana Charakopoulou-Travlou, Ioanna Rannou, Konstantina Magklara, Ioanna Giannopoulou, Konstantinos Kotsis, et al. "An Explainable Machine Learning Approach for COVID-19’s Impact on Mood States of Children and Adolescents during the First Lockdown in Greece." Healthcare 10, no. 1 (January 13, 2022): 149. http://dx.doi.org/10.3390/healthcare10010149.
Full textAbstract:
The global spread of COVID-19 led the World Health Organization to declare a pandemic on 11 March 2020. To decelerate this spread, countries have taken strict measures that have affected the lifestyles and economies. Various studies have focused on the identification of COVID-19’s impact on the mental health of children and adolescents via traditional statistical approaches. However, a machine learning methodology must be developed to explain the main factors that contribute to the changes in the mood state of children and adolescents during the first lockdown. Therefore, in this study an explainable machine learning pipeline is presented focusing on children and adolescents in Greece, where a strict lockdown was imposed. The target group consists of children and adolescents, recruited from children and adolescent mental health services, who present mental health problems diagnosed before the pandemic. The proposed methodology imposes: (i) data collection via questionnaires; (ii) a clustering process to identify the groups of subjects with amelioration, deterioration and stability to their mood state; (iii) a feature selection process to identify the most informative features that contribute to mood state prediction; (iv) a decision-making process based on an experimental evaluation among classifiers; (v) calibration of the best-performing model; and (vi) a post hoc interpretation of the features’ impact on the best-performing model. The results showed that a blend of heterogeneous features from almost all feature categories is necessary to increase our understanding regarding the effect of the COVID-19 pandemic on the mood state of children and adolescents.
21
Antoniadi, Anna Markella, Yuhan Du, Yasmine Guendouz, Lan Wei, Claudia Mazo, Brett A. Becker, and Catherine Mooney. "Current Challenges and Future Opportunities for XAI in Machine Learning-Based Clinical Decision Support Systems: A Systematic Review." Applied Sciences 11, no. 11 (May 31, 2021): 5088. http://dx.doi.org/10.3390/app11115088.
Full textAbstract:
Machine Learning and Artificial Intelligence (AI) more broadly have great immediate and future potential for transforming almost all aspects of medicine. However, in many applications, even outside medicine, a lack of transparency in AI applications has become increasingly problematic. This is particularly pronounced where users need to interpret the output of AI systems. Explainable AI (XAI) provides a rationale that allows users to understand why a system has produced a given output. The output can then be interpreted within a given context. One area that is in great need of XAI is that of Clinical Decision Support Systems (CDSSs). These systems support medical practitioners in their clinic decision-making and in the absence of explainability may lead to issues of under or over-reliance. Providing explanations for how recommendations are arrived at will allow practitioners to make more nuanced, and in some cases, life-saving decisions. The need for XAI in CDSS, and the medical field in general, is amplified by the need for ethical and fair decision-making and the fact that AI trained with historical data can be a reinforcement agent of historical actions and biases that should be uncovered. We performed a systematic literature review of work to-date in the application of XAI in CDSS. Tabular data processing XAI-enabled systems are the most common, while XAI-enabled CDSS for text analysis are the least common in literature. There is more interest in developers for the provision of local explanations, while there was almost a balance between post-hoc and ante-hoc explanations, as well as between model-specific and model-agnostic techniques. Studies reported benefits of the use of XAI such as the fact that it could enhance decision confidence for clinicians, or generate the hypothesis about causality, which ultimately leads to increased trustworthiness and acceptability of the system and potential for its incorporation in the clinical workflow. However, we found an overall distinct lack of application of XAI in the context of CDSS and, in particular, a lack of user studies exploring the needs of clinicians. We propose some guidelines for the implementation of XAI in CDSS and explore some opportunities, challenges, and future research needs.
22
Chatterjee, Soumick, Arnab Das, Chirag Mandal, Budhaditya Mukhopadhyay, Manish Vipinraj, Aniruddh Shukla, Rajatha Nagaraja Rao, Chompunuch Sarasaen, Oliver Speck, and Andreas Nürnberger. "TorchEsegeta: Framework for Interpretability and Explainability of Image-Based Deep Learning Models." Applied Sciences 12, no. 4 (February 10, 2022): 1834. http://dx.doi.org/10.3390/app12041834.
Full textAbstract:
Clinicians are often very sceptical about applying automatic image processing approaches, especially deep learning-based methods, in practice. One main reason for this is the black-box nature of these approaches and the inherent problem of missing insights of the automatically derived decisions. In order to increase trust in these methods, this paper presents approaches that help to interpret and explain the results of deep learning algorithms by depicting the anatomical areas that influence the decision of the algorithm most. Moreover, this research presents a unified framework, TorchEsegeta, for applying various interpretability and explainability techniques for deep learning models and generates visual interpretations and explanations for clinicians to corroborate their clinical findings. In addition, this will aid in gaining confidence in such methods. The framework builds on existing interpretability and explainability techniques that are currently focusing on classification models, extending them to segmentation tasks. In addition, these methods have been adapted to 3D models for volumetric analysis. The proposed framework provides methods to quantitatively compare visual explanations using infidelity and sensitivity metrics. This framework can be used by data scientists to perform post hoc interpretations and explanations of their models, develop more explainable tools, and present the findings to clinicians to increase their faith in such models. The proposed framework was evaluated based on a use case scenario of vessel segmentation models trained on Time-of-Flight (TOF) Magnetic Resonance Angiogram (MRA) images of the human brain. Quantitative and qualitative results of a comparative study of different models and interpretability methods are presented. Furthermore, this paper provides an extensive overview of several existing interpretability and explainability methods.
23
Meng, Deyu, Hongzhi Guo, Siyu Liang, Zhibo Tian, Ran Wang, Guang Yang, and Ziheng Wang. "Effectiveness of a Hybrid Exercise Program on the Physical Abilities of Frail Elderly and Explainable Artificial-Intelligence-Based Clinical Assistance." International Journal of Environmental Research and Public Health 19, no. 12 (June 7, 2022): 6988. http://dx.doi.org/10.3390/ijerph19126988.
Full textAbstract:
Background: Due to the low physical fitness of the frail elderly, current exercise program strategies have a limited impact. Eight-form Tai Chi has a low intensity, but high effectiveness in the elderly. Inspired by it, we designed an exercise program that incorporates eight-form Tai Chi, strength, and endurance exercises, to improve physical fitness and reverse frailty in the elderly. Additionally, for the ease of use in clinical practice, machine learning simulations were used to predict the frailty status after the intervention. Methods: For 24 weeks, 150 frail elderly people completed the experiment, which comprised the eight-form Tai Chi group (TC), the strength and endurance training group (SE), and a comprehensive intervention combining both TC and SE (TCSE). The comparison of the demographic variables used one-way ANOVA for continuous data and the chi-squared test for categorical data. Two-way repeated measures analysis of variance (ANOVA) was performed to determine significant main effects and interaction effects. Eleven machine learning models were used to predict the frailty status of the elderly following the intervention. Results: Two-way repeated measures ANOVA results before the intervention, group effects of ten-meter maximum walking speed (10 m MWS), grip strength (GS), timed up and go test (TUGT), and the six-minute walk test (6 min WT) were not significant. There was a significant interaction effect of group × time in ten-meter maximum walking speed, grip strength, and the six-minute walk test. Post hoc tests showed that after 24 weeks of intervention, subjects in the TCSE group showed the greatest significant improvements in ten-meter maximum walking speed (p < 0.05) and the six-minute walk test (p < 0.05) compared to the TC group and SE group. The improvement in grip strength in the TCSE group (4.29 kg) was slightly less than that in the SE group (5.16 kg). There was neither a significant main effect nor a significant interaction effect for TUGT in subjects. The stacking model outperformed other algorithms. Accuracy and the F1-score were 67.8% and 71.3%, respectively. Conclusion: A hybrid exercise program consisting of eight-form Tai Chi and strength and endurance exercises can more effectively improve physical fitness and reduce frailty among the elderly. It is possible to predict whether an elderly person will reverse frailty following an exercise program based on the stacking model.
24
Wei, Meiqi, Deyu Meng, Hongzhi Guo, Shichun He, Zhibo Tian, Ziyi Wang, Guang Yang, and Ziheng Wang. "Hybrid Exercise Program for Sarcopenia in Older Adults: The Effectiveness of Explainable Artificial Intelligence-Based Clinical Assistance in Assessing Skeletal Muscle Area." International Journal of Environmental Research and Public Health 19, no. 16 (August 12, 2022): 9952. http://dx.doi.org/10.3390/ijerph19169952.
Full textAbstract:
Background: Sarcopenia is a geriatric syndrome characterized by decreased skeletal muscle mass and function with age. It is well-established that resistance exercise and Yi Jin Jing improve the skeletal muscle mass of older adults with sarcopenia. Accordingly, we designed an exercise program incorporating resistance exercise and Yi Jin Jing to increase skeletal muscle mass and reverse sarcopenia in older adults. Additionally, machine learning simulations were used to predict the sarcopenia status after the intervention. Method: This randomized controlled trial assessed the effects of sarcopenia in older adults. For 24 weeks, 90 older adults with sarcopenia were divided into intervention groups, including the Yi Jin Jing and resistance training group (YR, n = 30), the resistance training group (RT, n = 30), and the control group (CG, n = 30). Computed tomography (CT) scans of the abdomen were used to quantify the skeletal muscle cross-sectional area at the third lumbar vertebra (L3 SMA). Participants’ age, body mass, stature, and BMI characteristics were analyzed by one-way ANOVA and the chi-squared test for categorical data. This study explored the improvement effect of three interventions on participants’ L3 SMA, skeletal muscle density at the third lumbar vertebra (L3 SMD), skeletal muscle interstitial fat area at the third lumbar vertebra region of interest (L3 SMFA), skeletal muscle interstitial fat density at the third lumbar vertebra (L3 SMFD), relative skeletal muscle mass index (RSMI), muscle fat infiltration (MFI), and handgrip strength. Experimental data were analyzed using two-way repeated-measures ANOVA. Eleven machine learning models were trained and tested 100 times to assess the model’s performance in predicting whether sarcopenia could be reversed following the intervention. Results: There was a significant interaction in L3 SMA (p < 0.05), RSMI (p < 0.05), MFI (p < 0.05), and handgrip strength (p < 0.05). After the intervention, participants in the YR and RT groups showed significant improvements in L3 SMA, RSMI, and handgrip strength. Post hoc tests showed that the YR group (p < 0.05) yielded significantly better L3 SMA and RSMI than the RT group (p < 0.05) and CG group (p < 0.05) after the intervention. Compared with other models, the stacking model exhibits the best performance in terms of accuracy (85.7%) and F1 (75.3%). Conclusion: One hybrid exercise program with Yi Jin Jing and resistance exercise training can improve skeletal muscle area among older adults with sarcopenia. Accordingly, it is possible to predict whether sarcopenia can be reversed in older adults based on our stacking model.
25
Xie, Yibing, Nichakorn Pongsakornsathien, Alessandro Gardi, and Roberto Sabatini. "Explanation of Machine-Learning Solutions in Air-Traffic Management." Aerospace 8, no. 8 (August 12, 2021): 224. http://dx.doi.org/10.3390/aerospace8080224.
Full textAbstract:
Advances in the trusted autonomy of air-traffic management (ATM) systems are currently being pursued to cope with the predicted growth in air-traffic densities in all classes of airspace. Highly automated ATM systems relying on artificial intelligence (AI) algorithms for anomaly detection, pattern identification, accurate inference, and optimal conflict resolution are technically feasible and demonstrably able to take on a wide variety of tasks currently accomplished by humans. However, the opaqueness and inexplicability of most intelligent algorithms restrict the usability of such technology. Consequently, AI-based ATM decision-support systems (DSS) are foreseen to integrate eXplainable AI (XAI) in order to increase interpretability and transparency of the system reasoning and, consequently, build the human operators’ trust in these systems. This research presents a viable solution to implement XAI in ATM DSS, providing explanations that can be appraised and analysed by the human air-traffic control operator (ATCO). The maturity of XAI approaches and their application in ATM operational risk prediction is investigated in this paper, which can support both existing ATM advisory services in uncontrolled airspace (Classes E and F) and also drive the inflation of avoidance volumes in emerging performance-driven autonomy concepts. In particular, aviation occurrences and meteorological databases are exploited to train a machine learning (ML)-based risk-prediction tool capable of real-time situation analysis and operational risk monitoring. The proposed approach is based on the XGBoost library, which is a gradient-boost decision tree algorithm for which post-hoc explanations are produced by SHapley Additive exPlanations (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME). Results are presented and discussed, and considerations are made on the most promising strategies for evolving the human–machine interactions (HMI) to strengthen the mutual trust between ATCO and systems. The presented approach is not limited only to conventional applications but also suitable for UAS-traffic management (UTM) and other emerging applications.
26
Sadler, Sophie, Derek Greene, and Daniel Archambault. "Towards explainable community finding." Applied Network Science 7, no. 1 (December 8, 2022). http://dx.doi.org/10.1007/s41109-022-00515-6.
Full textAbstract:
AbstractThe detection of communities of nodes is an important task in understanding the structure of networks. Multiple approaches have been developed to tackle this problem, many of which are in common usage in real-world applications, such as in public health networks. However, clear insight into the reasoning behind the community labels produced by these algorithms is rarely provided. Drawing inspiration from the machine learning literature, we aim to provide post-hoc explanations for the outputs of these algorithms using interpretable features of the network. In this paper, we propose a model-agnostic methodology that identifies a set of informative features to help explain the output of a community finding algorithm. We apply it to three well-known algorithms, though the methodology is designed to generalise to new approaches. As well as identifying important features for a post-hoc explanation system, we report on the common features found made by the different algorithms and the differences between the approaches.
27
Farahani, Farzad V., Krzysztof Fiok, Behshad Lahijanian, Waldemar Karwowski, and Pamela K. Douglas. "Explainable AI: A review of applications to neuroimaging data." Frontiers in Neuroscience 16 (December 1, 2022). http://dx.doi.org/10.3389/fnins.2022.906290.
Full textAbstract:
Deep neural networks (DNNs) have transformed the field of computer vision and currently constitute some of the best models for representations learned via hierarchical processing in the human brain. In medical imaging, these models have shown human-level performance and even higher in the early diagnosis of a wide range of diseases. However, the goal is often not only to accurately predict group membership or diagnose but also to provide explanations that support the model decision in a context that a human can readily interpret. The limited transparency has hindered the adoption of DNN algorithms across many domains. Numerous explainable artificial intelligence (XAI) techniques have been developed to peer inside the “black box” and make sense of DNN models, taking somewhat divergent approaches. Here, we suggest that these methods may be considered in light of the interpretation goal, including functional or mechanistic interpretations, developing archetypal class instances, or assessing the relevance of certain features or mappings on a trained model in a post-hoc capacity. We then focus on reviewing recent applications of post-hoc relevance techniques as applied to neuroimaging data. Moreover, this article suggests a method for comparing the reliability of XAI methods, especially in deep neural networks, along with their advantages and pitfalls.
28
Vale, Daniel, Ali El-Sharif, and Muhammed Ali. "Explainable artificial intelligence (XAI) post-hoc explainability methods: risks and limitations in non-discrimination law." AI and Ethics, March 15, 2022. http://dx.doi.org/10.1007/s43681-022-00142-y.
Full text
29
Karim, Muhammad Monjurul, Yu Li, and Ruwen Qin. "Toward Explainable Artificial Intelligence for Early Anticipation of Traffic Accidents." Transportation Research Record: Journal of the Transportation Research Board, February 18, 2022, 036119812210761. http://dx.doi.org/10.1177/03611981221076121.
Full textAbstract:
Traffic accident anticipation is a vital function of Automated Driving Systems (ADS) in providing a safety-guaranteed driving experience. An accident anticipation model aims to predict accidents promptly and accurately before they occur. Existing Artificial Intelligence (AI) models of accident anticipation lack a human-interpretable explanation of their decision making. Although these models perform well, they remain a black-box to the ADS users who find it to difficult to trust them. To this end, this paper presents a gated recurrent unit (GRU) network that learns spatio-temporal relational features for the early anticipation of traffic accidents from dashcam video data. A post-hoc attention mechanism named Grad-CAM (Gradient-weighted Class Activation Map) is integrated into the network to generate saliency maps as the visual explanation of the accident anticipation decision. An eye tracker captures human eye fixation points for generating human attention maps. The explainability of network-generated saliency maps is evaluated in comparison to human attention maps. Qualitative and quantitative results on a public crash data set confirm that the proposed explainable network can anticipate an accident on average 4.57 s before it occurs, with 94.02% average precision. Various post-hoc attention-based XAI methods are then evaluated and compared. This confirms that the Grad-CAM chosen by this study can generate high-quality, human-interpretable saliency maps (with 1.23 Normalized Scanpath Saliency) for explaining the crash anticipation decision. Importantly, results confirm that the proposed AI model, with a human-inspired design, can outperform humans in accident anticipation.
30
Fleisher, Will. "Understanding, Idealization, and Explainable AI." Episteme, November 3, 2022, 1–27. http://dx.doi.org/10.1017/epi.2022.39.
Full textAbstract:
Abstract Many AI systems that make important decisions are black boxes: how they function is opaque even to their developers. This is due to their high complexity and to the fact that they are trained rather than programmed. Efforts to alleviate the opacity of black box systems are typically discussed in terms of transparency, interpretability, and explainability. However, there is little agreement about what these key concepts mean, which makes it difficult to adjudicate the success or promise of opacity alleviation methods. I argue for a unified account of these key concepts that treats the concept of understanding as fundamental. This allows resources from the philosophy of science and the epistemology of understanding to help guide opacity alleviation efforts. A first significant benefit of this understanding account is that it defuses one of the primary, in-principle objections to post hoc explainable AI (XAI) methods. This “rationalization objection” argues that XAI methods provide mere rationalizations rather than genuine explanations. This is because XAI methods involve using a separate “explanation” system to approximate the original black box system. These explanation systems function in a completely different way than the original system, yet XAI methods make inferences about the original system based on the behavior of the explanation system. I argue that, if we conceive of XAI methods as idealized scientific models, this rationalization worry is dissolved. Idealized scientific models misrepresent their target phenomena, yet are capable of providing significant and genuine understanding of their targets.
31
Yang, Chu-I., and Yi-Pei Li. "Explainable uncertainty quantifications for deep learning-based molecular property prediction." Journal of Cheminformatics 15, no. 1 (February 3, 2023). http://dx.doi.org/10.1186/s13321-023-00682-3.
Full textAbstract:
AbstractQuantifying uncertainty in machine learning is important in new research areas with scarce high-quality data. In this work, we develop an explainable uncertainty quantification method for deep learning-based molecular property prediction. This method can capture aleatoric and epistemic uncertainties separately and attribute the uncertainties to atoms present in the molecule. The atom-based uncertainty method provides an extra layer of chemical insight to the estimated uncertainties, i.e., one can analyze individual atomic uncertainty values to diagnose the chemical component that introduces uncertainty to the prediction. Our experiments suggest that atomic uncertainty can detect unseen chemical structures and identify chemical species whose data are potentially associated with significant noise. Furthermore, we propose a post-hoc calibration method to refine the uncertainty quantified by ensemble models for better confidence interval estimates. This work improves uncertainty calibration and provides a framework for assessing whether and why a prediction should be considered unreliable. Graphical Abstract
32
Lee, Minyoung, Joohyoung Jeon, and Hongchul Lee. "Explainable AI for domain experts: a post Hoc analysis of deep learning for defect classification of TFT–LCD panels." Journal of Intelligent Manufacturing, March 26, 2021. http://dx.doi.org/10.1007/s10845-021-01758-3.
Full text
33
Belle, Vaishak, and Ioannis Papantonis. "Principles and Practice of Explainable Machine Learning." Frontiers in Big Data 4 (July 1, 2021). http://dx.doi.org/10.3389/fdata.2021.688969.
Full textAbstract:
Artificial intelligence (AI) provides many opportunities to improve private and public life. Discovering patterns and structures in large troves of data in an automated manner is a core component of data science, and currently drives applications in diverse areas such as computational biology, law and finance. However, such a highly positive impact is coupled with a significant challenge: how do we understand the decisions suggested by these systems in order that we can trust them? In this report, we focus specifically on data-driven methods—machine learning (ML) and pattern recognition models in particular—so as to survey and distill the results and observations from the literature. The purpose of this report can be especially appreciated by noting that ML models are increasingly deployed in a wide range of businesses. However, with the increasing prevalence and complexity of methods, business stakeholders in the very least have a growing number of concerns about the drawbacks of models, data-specific biases, and so on. Analogously, data science practitioners are often not aware about approaches emerging from the academic literature or may struggle to appreciate the differences between different methods, so end up using industry standards such as SHAP. Here, we have undertaken a survey to help industry practitioners (but also data scientists more broadly) understand the field of explainable machine learning better and apply the right tools. Our latter sections build a narrative around a putative data scientist, and discuss how she might go about explaining her models by asking the right questions. From an organization viewpoint, after motivating the area broadly, we discuss the main developments, including the principles that allow us to study transparent models vs. opaque models, as well as model-specific or model-agnostic post-hoc explainability approaches. We also briefly reflect on deep learning models, and conclude with a discussion about future research directions.
34
Okazaki, Kotaro, and Katsumi Inoue. "Explainable Model Fusion for Customer Journey Mapping." Frontiers in Artificial Intelligence 5 (May 11, 2022). http://dx.doi.org/10.3389/frai.2022.824197.
Full textAbstract:
Due to advances in computing power and internet technology, various industrial sectors are adopting IT infrastructure and artificial intelligence (AI) technologies. Recently, data-driven predictions have attracted interest in high-stakes decision-making. Despite this, advanced AI methods are less often used for such tasks. This is because AI technology is a black box for the social systems it is meant to support; trustworthiness and fairness have not yet been established. Meanwhile in the field of marketing, strategic decision-making is a high-stakes problem that has a significant impact on business trends. For global marketing, with its diverse cultures and market environments, future decision-making is likely to focus on building consensus on the formulation of the problem itself rather than on solutions for achieving the goal. There are two important and conflicting facts: the fact that the core of domestic strategic decision-making comes down to the formulation of the problem itself, and the fact that it is difficult to realize AI technology that can achieve problem formulation. How can we resolve this difficulty with current technology? This is the main challenge for the realization of high-level human-AI systems in the marketing field. Thus, we propose customer journey mapping (CJM) automation through model-level data fusion, a process for the practical problem formulation known as explainable alignment. Using domain-specific requirements and observations as inputs, the system automatically outputs a CJM. Explainable alignment corresponds with both human and AI perspectives and in formulating the problem, thereby improving strategic decision-making in marketing. Following preprocessing to make latent variables and their dynamics transparent with latent Dirichlet allocation and a variational autoencoder, a post-hoc explanation is implemented in which a hidden Markov model and learning from an interpretation transition are combined with a long short-term memory architecture that learns sequential data between touchpoints for extracting attitude rules for CJM. Finally, we realize the application of human-AI systems to strategic decision-making in marketing with actual logs in over-the-top media services, in which the dynamic behavior of customers for CJM can be automatically extracted.
35
Hegselmann, Stefan, Christian Ertmer, Thomas Volkert, Antje Gottschalk, Martin Dugas, and Julian Varghese. "Development and validation of an interpretable 3 day intensive care unit readmission prediction model using explainable boosting machines." Frontiers in Medicine 9 (August 23, 2022). http://dx.doi.org/10.3389/fmed.2022.960296.
Full textAbstract:
BackgroundIntensive care unit (ICU) readmissions are associated with mortality and poor outcomes. To improve discharge decisions, machine learning (ML) could help to identify patients at risk of ICU readmission. However, as many models are black boxes, dangerous properties may remain unnoticed. Widely used post hoc explanation methods also have inherent limitations. Few studies are evaluating inherently interpretable ML models for health care and involve clinicians in inspecting the trained model.MethodsAn inherently interpretable model for the prediction of 3 day ICU readmission was developed. We used explainable boosting machines that learn modular risk functions and which have already been shown to be suitable for the health care domain. We created a retrospective cohort of 15,589 ICU stays and 169 variables collected between 2006 and 2019 from the University Hospital Münster. A team of physicians inspected the model, checked the plausibility of each risk function, and removed problematic ones. We collected qualitative feedback during this process and analyzed the reasons for removing risk functions. The performance of the final explainable boosting machine was compared with a validated clinical score and three commonly used ML models. External validation was performed on the widely used Medical Information Mart for Intensive Care version IV database.ResultsThe developed explainable boosting machine used 67 features and showed an area under the precision-recall curve of 0.119 ± 0.020 and an area under the receiver operating characteristic curve of 0.680 ± 0.025. It performed on par with state-of-the-art gradient boosting machines (0.123 ± 0.016, 0.665 ± 0.036) and outperformed the Simplified Acute Physiology Score II (0.084 ± 0.025, 0.607 ± 0.019), logistic regression (0.092 ± 0.026, 0.587 ± 0.016), and recurrent neural networks (0.095 ± 0.008, 0.594 ± 0.027). External validation confirmed that explainable boosting machines (0.221 ± 0.023, 0.760 ± 0.010) performed similarly to gradient boosting machines (0.232 ± 0.029, 0.772 ± 0.018). Evaluation of the model inspection showed that explainable boosting machines can be useful to detect and remove problematic risk functions.ConclusionsWe developed an inherently interpretable ML model for 3 day ICU readmission prediction that reached the state-of-the-art performance of black box models. Our results suggest that for low- to medium-dimensional datasets that are common in health care, it is feasible to develop ML models that allow a high level of human control without sacrificing performance.
36
Weber, Patrick, K. Valerie Carl, and Oliver Hinz. "Applications of Explainable Artificial Intelligence in Finance—a systematic review of Finance, Information Systems, and Computer Science literature." Management Review Quarterly, February 28, 2023. http://dx.doi.org/10.1007/s11301-023-00320-0.
Full textAbstract:
AbstractDigitalization and technologization affect numerous domains, promising advantages but also entailing risks. Hence, when decision-makers in highly-regulated domains like Finance implement these technological advances—especially Artificial Intelligence—regulators prescribe high levels of transparency, assuring the traceability of decisions for third parties. Explainable Artificial Intelligence (XAI) is of tremendous importance in this context. We provide an overview of current research on XAI in Finance with a systematic literature review screening 2,022 articles from leading Finance, Information Systems, and Computer Science outlets. We identify a set of 60 relevant articles, classify them according to the used XAI methods and goals that they aim to achieve, and provide an overview of XAI methods used in different Finance areas. Areas like risk management, portfolio optimization, and applications around the stock market are well-researched, while anti-money laundering is understudied. Researchers implement both transparent models and post-hoc explainability, while they recently favored the latter.
37
Lee, Kyungtae, Mukil V. Ayyasamy, Yangfeng Ji, and Prasanna V. Balachandran. "A comparison of explainable artificial intelligence methods in the phase classification of multi-principal element alloys." Scientific Reports 12, no. 1 (July 8, 2022). http://dx.doi.org/10.1038/s41598-022-15618-4.
Full textAbstract:
AbstractWe demonstrate the capabilities of two model-agnostic local post-hoc model interpretability methods, namely breakDown (BD) and shapley (SHAP), to explain the predictions of a black-box classification learning model that establishes a quantitative relationship between chemical composition and multi-principal element alloys (MPEA) phase formation. We trained an ensemble of support vector machines using a dataset with 1,821 instances, 12 features with low pair-wise correlation, and seven phase labels. Feature contributions to the model prediction are computed by BD and SHAP for each composition. The resulting BD and SHAP transformed data are then used as inputs to identify similar composition groups using k-means clustering. Explanation-of-clusters by features reveal that the results from SHAP agree more closely with the literature. Visualization of compositions within a cluster using Ceteris-Paribus (CP) profile plots show the functional dependencies between the feature values and predicted response. Despite the differences between BD and SHAP in variable attribution, only minor changes were observed in the CP profile plots. Explanation-of-clusters by examples show that the clusters that share a common phase label contain similar compositions, which clarifies the similar-looking CP profile trends. Two plausible reasons are identified to describe this observation: (1) In the limits of a dataset with independent and non-interacting features, BD and SHAP show promise in recognizing MPEA composition clusters with similar phase labels. (2) There is more than one explanation for the MPEA phase formation rules with respect to the set of features considered in this work.
38
Papagni, Guglielmo, Jesse de Pagter, Setareh Zafari, Michael Filzmoser, and Sabine T. Koeszegi. "Artificial agents’ explainability to support trust: considerations on timing and context." AI & SOCIETY, June 27, 2022. http://dx.doi.org/10.1007/s00146-022-01462-7.
Full textAbstract:
AbstractStrategies for improving the explainability of artificial agents are a key approach to support the understandability of artificial agents’ decision-making processes and their trustworthiness. However, since explanations are not inclined to standardization, finding solutions that fit the algorithmic-based decision-making processes of artificial agents poses a compelling challenge. This paper addresses the concept of trust in relation to complementary aspects that play a role in interpersonal and human–agent relationships, such as users’ confidence and their perception of artificial agents’ reliability. Particularly, this paper focuses on non-expert users’ perspectives, since users with little technical knowledge are likely to benefit the most from “post-hoc”, everyday explanations. Drawing upon the explainable AI and social sciences literature, this paper investigates how artificial agent’s explainability and trust are interrelated at different stages of an interaction. Specifically, the possibility of implementing explainability as a trust building, trust maintenance and restoration strategy is investigated. To this extent, the paper identifies and discusses the intrinsic limits and fundamental features of explanations, such as structural qualities and communication strategies. Accordingly, this paper contributes to the debate by providing recommendations on how to maximize the effectiveness of explanations for supporting non-expert users’ understanding and trust.
39
Kucklick, Jan-Peter, and Oliver Müller. "Tackling the Accuracy–Interpretability Trade-off: Interpretable Deep Learning Models for Satellite Image-based Real Estate Appraisal." ACM Transactions on Management Information Systems, October 10, 2022. http://dx.doi.org/10.1145/3567430.
Full textAbstract:
Deep learning models fuel many modern decision support systems, because they typically provide high predictive performance. Among other domains, deep learning is used in real-estate appraisal, where it allows to extend the analysis from hard facts only (e.g., size, age) to also consider more implicit information about the location or appearance of houses in the form of image data. However, one downside of deep learning models is their intransparent mechanic of decision making, which leads to a trade-off between accuracy and interpretability. This limits their applicability for tasks where a justification of the decision is necessary. Therefore, in this paper, we first combine different perspectives on interpretability into a multi-dimensional framework for a socio-technical perspective on explainable artificial intelligence. Second, we measure the performance gains of using multi-view deep learning which leverages additional image data (satellite images) for real estate appraisal. Third, we propose and test a novel post-hoc explainability method called Grad-Ram. This modified version of Grad-Cam mitigates the intransparency of convolutional neural networks (CNNs) for predicting continuous outcome variables. With this, we try to reduce the accuracy-interpretability trade-off of multi-view deep learning models. Our proposed network architecture outperforms traditional hedonic regression models by 34% in terms of MAE. Furthermore, we find that the used satellite images are the second most important predictor after square feet in our model and that the network learns interpretable patterns about the neighborhood structure and density.
40
Nguyen, Sam, Ryan Chan, Jose Cadena, Braden Soper, Paul Kiszka, Lucas Womack, Mark Work, et al. "Budget constrained machine learning for early prediction of adverse outcomes for COVID-19 patients." Scientific Reports 11, no. 1 (October 1, 2021). http://dx.doi.org/10.1038/s41598-021-98071-z.
Full textAbstract:
AbstractThe combination of machine learning (ML) and electronic health records (EHR) data may be able to improve outcomes of hospitalized COVID-19 patients through improved risk stratification and patient outcome prediction. However, in resource constrained environments the clinical utility of such data-driven predictive tools may be limited by the cost or unavailability of certain laboratory tests. We leveraged EHR data to develop an ML-based tool for predicting adverse outcomes that optimizes clinical utility under a given cost structure. We further gained insights into the decision-making process of the ML models through an explainable AI tool. This cohort study was performed using deidentified EHR data from COVID-19 patients from ProMedica Health System in northwest Ohio and southeastern Michigan. We tested the performance of various ML approaches for predicting either increasing ventilatory support or mortality. We performed post hoc analysis to obtain optimal feature sets under various budget constraints. We demonstrate that it is possible to achieve a significant reduction in cost at the expense of a small reduction in predictive performance. For example, when predicting ventilation, it is possible to achieve a 43% reduction in cost with only a 3% reduction in performance. Similarly, when predicting mortality, it is possible to achieve a 50% reduction in cost with only a 1% reduction in performance. This study presents a quick, accurate, and cost-effective method to evaluate risk of deterioration for patients with SARS-CoV-2 infection at the time of clinical evaluation.
41
Chen, Ruoyu, Jingzhi Li, Hua Zhang, Changchong Sheng, Li Liu, and Xiaochun Cao. "Sim2Word: Explaining Similarity with Representative Attribute Words via Counterfactual Explanations." ACM Transactions on Multimedia Computing, Communications, and Applications, September 8, 2022. http://dx.doi.org/10.1145/3563039.
Full textAbstract:
Recently, we have witnessed substantial success using the deep neural network in many tasks. While there still exists concerns about the explainability of decision-making, it is beneficial for users to discern the defects in the deployed deep models. Existing explainable models either provide the image-level visualization of attention weights or generate textual descriptions as post-hoc justifications. Different from existing models, in this paper, we propose a new interpretation method that explains the image similarity models by salience maps and attribute words. Our interpretation model contains visual salience maps generation and the counterfactual explanation generation. The former has two branches: global identity relevant region discovery and multi-attribute semantic region discovery. Branch one aims to capture the visual evidence supporting the similarity score, which is achieved by computing counterfactual feature maps. Branch two aims to discover semantic regions supporting different attributes, which helps to understand which attributes in an image might change the similarity score. Then, by fusing visual evidence from two branches, we can obtain the salience maps indicating important response evidence. The latter will generate the attribute words that best explain the similarity using the proposed erasing model. The effectiveness of our model is evaluated on the classical face verification task. Experiments conducted on two benchmarks VGG-Face2 and Celeb-A demonstrate that our model can provide convincing interpretable explanations for the similarity. Moreover, our algorithm can be applied to evidential learning cases, e.g. finding the most characteristic attributes in a set of face images and we verify its effectiveness on the VGGFace2 dataset.
42
Zini, Julia El, and Mariette Awad. "On the Explainability of Natural Language Processing Deep Models." ACM Computing Surveys, July 19, 2022. http://dx.doi.org/10.1145/3529755.
Full textAbstract:
Despite their success, deep networks are used as black-box models with outputs that are not easily explainable during the learning and the prediction phases. This lack of interpretability is significantly limiting the adoption of such models in domains where decisions are critical such as the medical and legal fields. Recently, researchers have been interested in developing methods that help explain individual decisions and decipher the hidden representations of machine learning models in general and deep networks specifically. While there has been a recent explosion of work on Ex plainable A rtificial I ntelligence ( ExAI ) on deep models that operate on imagery and tabular data, textual datasets present new challenges to the ExAI community. Such challenges can be attributed to the lack of input structure in textual data, the use of word embeddings that add to the opacity of the models and the difficulty of the visualization of the inner workings of deep models when they are trained on textual data. Lately, methods have been developed to address the aforementioned challenges and present satisfactory explanations on Natural Language Processing (NLP) models. However, such methods are yet to be studied in a comprehensive framework where common challenges are properly stated and rigorous evaluation practices and metrics are proposed. Motivated to democratize ExAI methods in the NLP field, we present in this work a survey that studies model-agnostic as well as model-specific explainability methods on NLP models. Such methods can either develop inherently interpretable NLP models or operate on pre-trained models in a post-hoc manner. We make this distinction and we further decompose the methods into three categories according to what they explain: (1) word embeddings (input-level), (2) inner workings of NLP models (processing-level) and (3) models’ decisions (output-level). We also detail the different evaluation approaches interpretability methods in the NLP field. Finally, we present a case-study on the well-known neural machine translation in an appendix and we propose promising future research directions for ExAI in the NLP field.
43
Abdelsamea, Mohammed M., Mohamed Medhat Gaber, Aliyuda Ali, Marios Kyriakou, and Shams Fawki. "A logarithmically amortising temperature effect for supervised learning of wheat solar disinfestation of rice weevil Sitophilus oryzae (Coleoptera: Curculionidae) using plastic bags." Scientific Reports 13, no. 1 (February 14, 2023). http://dx.doi.org/10.1038/s41598-023-29594-w.
Full textAbstract:
AbstractThis work investigates the effectiveness of solar heating using clear polyethylene bags against rice weevil Sitophilus oryzae (L.), which is one of the most destructive insect pests against many strategic grains such as wheat. In this paper, we aim at finding the key parameters that affect the control heating system against stored grain insects while ensuring that the wheat grain quality is maintained. We provide a new benchmark dataset, where the experimental and environmental data was collected based on fieldwork during the summer in Canada. We measure the effectiveness of the solution using a novel formula to describe the amortising temperature effect on rice weevil. We adopted different machine learning models to predict the effectiveness of our solution in reaching a lethal heating condition for insect pests, and hence measure the importance of the parameters. The performance of our machine learning models has been validated using a 10-fold cross-validation, showing a high accuracy of 99.5% with 99.01% recall, 100% precision and 99.5% F1-Score obtained by the Random Forest model. Our experimental study on machine learning with SHAP values as an eXplainable post-hoc model provides the best environmental conditions and parameters that have a significant effect on the disinfestation of rice weevils. Our findings suggest that there is an optimal medium-sized grain amount when using solar bags for thermal insect disinfestation under high ambient temperatures. Machine learning provides us with a versatile model for predicting the lethal temperatures that are most effective for eliminating stored grain insects inside clear plastic bags. Using this powerful technology, we can gain valuable information on the optimal conditions to eliminate these pests. Our model allows us to predict whether a certain combination of parameters will be effective in the treatment of insects using thermal control. We make our dataset publicly available under a Creative Commons Licence to encourage researchers to use it as a benchmark for their studies.