Dissertations / Theses on the topic 'Artificial symbol learning'

This body of work explores an emerging aspect of human-robot interaction, transparency. Socially guided machine learning has proven that highly immersive robotic behaviors have yielded better results than lesser interactive behaviors for performance and shorter training time. While other work explores this transparency in learning by demonstration using non-verbal cues to point out the importance or preference users may have towards behaviors, my work follows this argument and attempts to extend it by offering cues to the internal task representation. What I show is that task-transparency, or the ability to connect and discuss the task in a fluent way implores the user to shape and correct the learned goal in ways that may be impossible by other present day learning by demonstration methods. Additionally, some participants are shown to prefer task-transparent robots which appear to have the ability of "introspection" in which it can modify the learned goal by other methods than just demonstration.

This thesis demonstrates how the power of symbolic processing can be exploited in the learning of low level control functions. It proposes a novel hybrid architecture with a tight coupling between a variant of symbolic planning and reinforcement learning. This architecture combines the strengths of the function approximation of subsymbolic learning with the more abstract compositional nature of symbolic learning. The former is able to represent mappings of world states to actions in an accurate way. The latter allows a more rapid solution to problems by exploiting structure within the domain. A control function is learnt over time through interaction with the world. Symbols are attached to features in the functions. The symbolic attachments act as anchor points used to transform the function of a previously learnt task to that of a new task. The solution of more complex tasks is achieved through composing simpler functions, using the symbolic attachments to determine the composition. The result is used as the initial control function of the new task and then modified through further learning. This is shown to produce a significant speed up over basic reinforcement learning.

Artificial Intelligence Lab, Department of MIS, University of Arizona
Information retrieval using probabilistic techniques has attracted significant attention on the part of researchers in information and computer science over the past few decades. In the 1980s, knowledge-based techniques also made an impressive contribution to “intelligent” information retrieval and indexing. More recently, information science researchers have turned to other newer artificial-intelligence- based inductive learning techniques including neural networks, symbolic learning, and genetic algorithms. These newer techniques, which are grounded on diverse paradigms, have provided great opportunities for researchers to enhance the information processing and retrieval capabilities of current information storage and retrieval systems. In this article, we first provide an overview of these newer techniques and their use in information science research. To familiarize readers with these techniques, we present three popular methods: the connectionist Hopfield network; the symbolic ID3/ID5R; and evolution- based genetic algorithms. We discuss their knowledge representations and algorithms in the context of information retrieval. Sample implementation and testing results from our own research are also provided for each technique. We believe these techniques are promising in their ability to analyze user queries, identify users’ information needs, and suggest alternatives for search. With proper user-system interactions, these methods can greatly complement the prevailing full-text, keywordbased, probabilistic, and knowledge-based techniques.

Artificial Intelligence Lab, Department of MIS, University of Arizona
For our research, we investigated a different problem-solving scenario called game playing, which is unstructured, complex, and seldom-studied. We considered several real-life game-playing scenarios and decided on greyhound racing. The large amount of historical information involved in the search poses a challenge for both human experts and machine-learning algorithms. The questions then become: Can machine-learning techniques reduce the uncertainty in a complex game-playing scenario? Can these methods outperform human experts in prediction? Our research sought to answer these questions.

Building and discovering useful features when constructing machine learning models is the central task for the machine learning practitioner. Good features are useful not only in increasing the predictive power of a model but also in illuminating the underlying drivers of a target variable. In this research we propose a novel feature learning technique in which Symbolic regression is endowed with a ``Range Terminal'' that allows it to explore functions of the aggregate of variables over time. We test the Range Terminal on a synthetic data set and a real world data in which we predict seasonal greenness using satellite derived temperature and snow data over a portion of the Arctic. On the synthetic data set we find Symbolic regression with the Range Terminal outperforms standard Symbolic regression and Lasso regression. On the Arctic data set we find it outperforms standard Symbolic regression, fails to beat the Lasso regression, but finds useful features describing the interaction between Land Surface Temperature, Snow, and seasonal vegetative growth in the Arctic.

This thesis is structured in four parts for a total of ten chapters.

The first part, introduction and review (Chapters 1 to 4), presents an extensive state-of-the-art review of both symbolic and connectionist GI methods, that serves also to state most of the basic material needed to describe later the contributions of the thesis. These contributions constitute the contents of the rest of parts (Chapters 5 to 10).

The second part, contributions on symbolic and connectionist techniques for regular grammatical inference (Chapters 5 to 7), describes the contributions related to the theory and methods for regular GI, which include other lateral subjects such as the representation oí. finite-state machines (FSMs) in recurrent neural networks (RNNs).

The third part of the thesis, augmented regular expressions and their inductive inference, comprises Chapters 8 and 9. The augmented regular expressions (or AREs) are defined and proposed as a new representation for a subclass of CSLs that does not contain all the context-free languages but a large class of languages capable of describing patterns with symmetries and other (context-sensitive) structures of interest in pattern recognition problems.

The fourth part of the thesis just includes Chapter 10: conclusions and future research. Chapter 10 summarizes the main results obtained and points out the lines of further research that should be followed both to deepen in some of the theoretical aspects raised and to facilitate the application of the developed GI tools to real-world problems in the area of computer vision.

Le reti neurali artificiali, grazie alle nuove tecniche di Deep Learning, hanno completamente rivoluzionato il panorama tecnologico degli ultimi anni, dimostrandosi efficaci in svariati compiti di Intelligenza Artificiale e ambiti affini. Sarebbe quindi interessante analizzare in che modo e in quale misura le deep network possano sostituire le IA simboliche. Dopo gli impressionanti risultati ottenuti nel gioco del Go, come caso di studio è stato scelto il gioco del Mulino, un gioco da tavolo largamente diffuso e ampiamente studiato. È stato quindi creato il sistema completamente sub-simbolico Neural Nine Men’s Morris, che sfrutta tre reti neurali per scegliere la mossa migliore. Le reti sono state addestrate su un dataset di più di 1.500.000 coppie (stato del gioco, mossa migliore), creato in base alle scelte di una IA simbolica. Il sistema ha dimostrato di aver imparato le regole del gioco proponendo una mossa valida in più del 99% dei casi di test. Inoltre ha raggiunto un’accuratezza del 39% rispetto al dataset e ha sviluppato una propria strategia di gioco diversa da quella della IA addestratrice, dimostrandosi un giocatore peggiore o migliore a seconda dell’avversario. I risultati ottenuti in questo caso di studio mostrano che, in questo contesto, la chiave del successo nella progettazione di sistemi AI allo stato dell’arte sembra essere un buon bilanciamento tra tecniche simboliche e sub-simboliche, dando più rilevanza a queste ultime, con lo scopo di raggiungere la perfetta integrazione di queste tecnologie.

Informed Machine Learning is an umbrella term that comprises a set of methodologies in which domain knowledge is injected into a data-driven system in order to improve its level of accuracy, satisfy some external constraint, and in general serve the purposes of explainability and reliability. The said topid has been widely explored in the literature by means of many different techniques. Moving Targets is one such a technique particularly focused on constraint satisfaction: it is based on decomposition and bi-level optimization and proceeds by iteratively refining the target labels through a master step which is in charge of enforcing the constraints, while the training phase is delegated to a learner. In this work, we extend the algorithm in order to deal with semi-supervised learning and soft constraints. In particular, we focus our empirical evaluation on both regression and classification tasks involving monotonicity shape constraints. We demonstrate that our method is robust with respect to its hyperparameters, as well as being able to generalize very well while reducing the number of violations on the enforced constraints. Additionally, the method can even outperform, both in terms of accuracy and constraint satisfaction, other state-of-the-art techniques such as Lattice Models and Semantic-based Regularization with a Lagrangian Dual approach for automatic hyperparameter tuning.

Em Aprendizado de Máquina - AM não existe um único algoritmo que é sempre melhor para todos os domínios de aplicação. Na prática, diversas pesquisas mostram que Redes Neurais Artificiais - RNAs têm um 'bias' indutivo apropriado para diversos domínios. Em razão disso, RNAs têm sido aplicadas na resolução de vários problemas com desempenho satisfatório. Sistemas de AM simbólico possuem um 'bias' indutivo menos flexível do que as RNAs. Enquanto que as RNAs são capazes de aprender qualquer função, sistemas de AM simbólico geralmente aprendem conceitos que podem ser descritos na forma de hiperplanos. Por outro lado, sistemas de AM simbólico representam o conceito induzido por meio de estruturas simbólicas, as quais são geralmente compreensíveis pelos seres humanos. Assim, sistemas de AM simbólico são preferíveis quando é essencial a compreensibilidade do conceito induzido. RNAs carecem da capacidade de explicar suas decisões, uma vez que o conhecimento é codificado na forma de valores de seus pesos e 'thresholds'. Essa codificação é difícil de ser interpretada por seres humanos. Em diversos domínios de aplicação, tal como aprovação de crédito e diagnóstico médico, prover uma explicação sobre a classificação dada a um determinado caso é de crucial importância. De um modo similar, diversos usuários de sistemas de AM simbólico desejam validar o conhecimento induzido, com o objetivo de assegurar que a generalização feita pelo algoritmo é correta. Para que RNAs sejam aplicadas em um maior número de domínios, diversos pesquisadores têm proposto métodos para extrair conhecimento compreensível de RNAs. As principais contribuições desta tese são dois métodos que extraem conhecimento simbólico de RNAs. Os métodos propostos possuem diversas vantagens sobre outros métodos propostos previamente, tal como ser aplicáveis a qualquer arquitetura ou algoritmo de aprendizado de RNAs supervisionadas. O primeiro método proposto utiliza sistemas de AM simbólico para extrair conhecimento de RNAs, e o segundo método proposto estende o primeiro, combinado o conhecimento induzido por diversos sistemas de AM simbólico por meio de um Algoritmo Genético - AG. Os métodos propostos são analisados experimentalmente em diversos domínios de aplicação. Ambos os métodos são capazes de extrair conhecimento simbólico com alta fidelidade em relação à RNA treinada. Os métodos propostos são comparados com o método TREPAN, apresentando resultados promissores. TREPAN é um método bastante conhecido para extrair conhecimento de RNAs.
In Machine Learning - ML there is not a single algorithm that is the best for all application domains. In practice, several research works have shown that Artificial Neural Networks - ANNs have an appropriate inductive bias for several domains. Thus, ANNs have been applied to a number of data sets with high predictive accuracy. Symbolic ML algorithms have a less flexible inductive bias than ANNs. While ANNs can learn any input-output mapping, i.e., ANNs have the universal approximation property, symbolic ML algorithms frequently learn concepts describing them using hyperplanes. On the other hand, symbolic algorithms are needed when a good understating of the decision process is essential, since symbolic ML algorithms express the knowledge induced using symbolic structures that can be interpreted and understood by humans. ANNs lack the capability of explaining their decisions since the knowledge is encoded as real-valued weights and biases of the network. This encoding is difficult to be interpreted by humans. In several application domains, such as credit approval and medical diagnosis, providing an explanation related to the classification given to a certain case is of crucial importance. In a similar way, several users of ML algorithms desire to validate the knowledge induced, in order to assure that the generalization made by the algorithm is correct. In order to apply ANNs to a larger number of application domains, several researches have proposed methods to extract comprehensible knowledge from ANNs. The primary contribution of this thesis consists of two methods that extract symbolic knowledge, expressed as decision rules, from ANNs. The proposed methods have several advantages over previous methods, such as being applicable to any architecture and supervised learning algorithm of ANNs. The first method uses standard symbolic ML algorithm to extract knowledge from ANNs, and the second method extends the first method by combining the knowledge induced by several symbolic ML algorithms through the application of a Genetic Algorithm - GA. The proposed methods are experimentally analyzed in a number of application domains. Results show that both methods are capable to extract symbolic knowledge having high fidelity with trained ANNs. The proposed methods are compared with TREPAN, showing promising results. TREPAN is a well known method to extract knowledge from ANNs.

Many different extensions of the VAE framework have been introduced in the past. How­ ever, the vast majority of them focused on pure sub­-symbolic approaches that are not sufficient for solving generative tasks that require a form of reasoning. In this thesis, we propose the probabilistic logic VAE (PLVAE), a neuro-­symbolic deep generative model that combines the representational power of VAEs with the reasoning ability of probabilistic ­logic programming. The strength of PLVAE resides in its probabilistic ­logic prior, which provides an interpretable structure to the latent space that can be easily changed in order to apply the model to different scenarios. We provide empirical results of our approach by training PLVAE on a base task and then using the same model to generalize to novel tasks that involve reasoning with the same set of symbols.

Plusieurs méthodes ont été développées par l'Intelligence Artificielle pour reproduire certains aspects de l'intelligence humaine. Ces méthodes permettent de simuler les processus de raisonnement en s'appuyant sur les connaissances de base disponibles. Chaque méthode comporte des points forts, mais aussi des limitations. La réalisation de systèmes hybrides est une démarche courante Qui permet de combiner les points forts de chaque approche, et d'obtenir ainsi des performances plus élevées ou un champ d'application plus large. Un autre aspect très important du développement des systèmes hybrides intelligents est leur capacité d'acquérir de nouvelles connaissances à partir de plusieurs sources différentes et de les faire évoluer. Dans cette thèse, nous avons développé des recherches sur les systèmes hybrides neuro-symboliques, et en particulier sur l'acquisition incrémentale de connaissances à partir de connaissances théoriques (règles) et empiriques (exemples). Un nouveau système hybride, nommé système INSS - Incremental Neuro-Symbolic System, a été étudié et réalisé. Ce système permet le transfert de connaissances déclaratives (règles symboliques) d'un module symbolique vers un module connexionniste (réseau de neurones artificiel - RNA) à travers un convertisseur de règles en réseau. Les connaissances du réseau ainsi obtenu sont affinées par un processus d'apprentissage à partir d'exemples. Ce raffinement se fait soit par ajout de nouvelles connaissances, soit par correction des incohérences, grâce à l'utilisation d'un réseau constructif de type Cascade-Correlation. Une méthode d'extraction incrémentale de règles a été intégrée au système INSS, ainsi que des algorithmes de validation des connaissances qui ont permis de mieux coupler les modules connexionniste et symbolique. Le système d'apprentissage automatique INSS a été conçu pour l'acquisition constructive (incrémentale) de connaissances. Le système a été testé sur plusieurs applications, en utilisant des problèmes académiques et des problèmes réels (diagnostic médical, modélisation cognitive et contrôle d'un robot autonome). Les résultats montrent que le système INSS a des performances supérieures et de nombreux avantages par rapport aux autres systèmes hybrides du même type
Various Artificial Intelligence methods have been developed to reproduce intelligent human behaviour. These methods allow to reproduce some human reasoning process using the available knowledge. Each method has its advantages, but also some drawbacks. Hybrid systems combine different approaches in order to take advantage of their respective strengths. These hybrid intelligent systems also present the ability to acquire new knowledge from different sources and so to improve their application performance. This thesis presents our research in the field of hybrid neuro-symbolic systems, and in particular the study of machine learning tools used for constructive knowledge acquisition. We are interested in the automatic acquisition of theoretical knowledge (rules) and empirical knowledge (examples). We present a new hybrid system we implemented: INSS - Incremental Neuro-Symbolic System. This system allows knowledge transfer from the symbolic module to the connectionist module (Artificial Neural Network - ANN), through symbolic rule compilation into an ANN. We can refine the initial ANN knowledge through neural learning using a set of examples. The incremental ANN learning method used, the Cascade-Correlation algorithm, allows us to change or to add new knowledge to the network. Then, the system can also extract modified (or new) symbolic rules from the ANN and validate them. INSS is a hybrid machine learning system that implements a constructive knowledge acquisition method. We conclude by showing the results we obtained with this system in different application domains: ANN artificial problems(The Monk's Problems), computer aided medical diagnosis (Toxic Comas), a cognitive modelling task (The Balance Scale Problem) and autonomous robot control. The results we obtained show the improved performance of INSS and its advantages over others hybrid neuro-symbolic systems

This diploma thesis deals with the cryptanalysis of symmetric encryption algorithms. The aim of this thesis is to show different point of view on this issues. The dissimilar way, compared to the recent methods, lies in the use of the power of evolutionary principles which are in the cryptanalytic system applied with help of genetic programming. In the theoretical part the cryptography, cryptanalysis of symmetric encryption algorithms and genetic programming are described. On the ground of the obtained information a project of cryptanalytic system which uses evolutionary principles is represented. Practical part deals with implementation of symmetric encrypting algorithm, linear cryptanalysis and simulation instrument of genetic programming. The end of the thesis represents experiments together with projected cryptanalytic system which uses genetic programming and evaluates reached results.

In the recent decades, alternative notions regarding the role of symbols in intelligence in natural and artificial systems have attracted significant inter- est. The main difference of the so-called situated and embodied approaches to cognitive science from the traditional cognitivist position is that symbolic repre- sentations are viewed as resources, similar to maps used for navigation or plans for activity, instead of as transparent stand-ins in internal world models. Thus, all symbolic resources have to be interpreted and re-contextualized for use in concrete situations. In this view, one of the primary sources of such symbolic resources is language. Cognitivism views language as a vessel carrying informa- tion originally located in the processing mechanisms of the individual agents. Situated approaches, on the other hand, view language both as a communicative mechanism and as a means for the individual agents to enhance and extend their cognitive machinery, by e.g. better utilizing their attentional resources, or mod- ifying their perceptual-motor means. Taking inspiration from these ideas, and building on multi-agent models developed in other fields, the field of language evolution developed models of the emergence of shared resources for communi- cation in a community of agents. In these models, agents with various means of categorization and learning engage in communicative interactions with each other, using shared signs to refer either to pre-given meanings or entities in a situation. In order to avoid falling into the same mentalist pitfalls as cognitivism in the design of these models, such as the stipulation of an inner sphere of mean- ings for which communicative signs are mere labels, the role of communication should be viewed as one of the social coordination of behavior using physically grounded symbols. To this end, an experimental setup for language games, and a robotic model for agents which engage in such games are presented. The setup allows the agents to utilize shared symbols in the completion of a simple task, with one agent instructing another on which action to undertake. The symbols used by agents in the language games are grounded in the embodied choices presented to them by their environment, and the agents can further use the symbols created in these games for enhancing their own behavioral means. The learning mechanism of the agents is similarity-based, and uses low-level sensory data to avoid the building in of features. Experiments have shown that the establishment of a common vocabulary of labels depends on how well the instructors are trained on the task and the availability of feedback mechanisms for the exchanged labels.

(in English): The purpose of this thesis is to describe developments of research in the field of artificial intelligence, from the point of view reflecting changes in current paradigms, and to analyze contemporary tendencies. This thesis systemically places the paradigm term into contexts of theoretical sciences and it explains in what way the term is being used. Further, the thesis describes artificial intelligence and several selected components. The thesis researches the basic paradigms of artificial intelligence - the symbolic and connectionistic paradigm, and is also researching new approaches and analyzing their beginnings and important development periods. The thesis analyzes reasons that were behind these developments. In addition to questions related to technical developments, financial support of selected research played an important role. The closing part of the thesis also analyzes reasons of current artificial intelligence expansion, worries connected to this expansion, and current research trends.