Дисертації з теми "Classification based on generative models"

Creating models to generate contextual responses to input queries is a difficult problem. It is even more difficult when available data contains noise and sensitive data. Finding models or methods to handle such issues is important in order to use data for productive means.This thesis proposes a model based on a cooperating pair of Topic Models of differing tasks (LDA and GSDMM) in order to alleviate the problematic properties of data. The model is tested on a real-world dataset with these difficulties as well as a dataset without them. The goal is to 1) look at the behaviour of the different topic models to see if their topical representation of the data is of use as input or output to other models and 2) find out what properties can be alleviated as a result.The results show that topic modeling can represent the semantic information of documents well enough to produce well-behaved input data for other models, which can also deal well with large vocabularies and noisy data. The topical clustering of the response data is sufficient enough for a classification model to predict the context of the response, from which valid responses can be created.
Att skapa modeller som genererar kontextuella svar på frågor är ett svårt problem från början, någonting som blir än mer svårt när tillgänglig data innehåller både brus och känslig information. Det är både viktigt och av stort intresse att hitta modeller och metoder som kan hantera dessa svårigheter så att även problematisk data kan användas produktivt.Detta examensarbete föreslår en modell baserat på ett par samarbetande Topic Models (ämnesbaserade modeller) med skiljande ansvarsområden (LDA och GSDMM) för att underlätta de problematiska egenskaperna av datan. Modellen testas på ett verkligt dataset med dessa svårigheter samt ett dataset utan dessa. Målet är att 1) inspektera båda ämnesmodellernas beteende för att se om dessa kan representera datan på ett sådant sätt att andra modeller kan använda dessa som indata eller utdata och 2) förstå vilka av dessa svårigheter som kan hanteras som följd.Resultaten visar att ämnesmodellerna kan representera semantiken och betydelsen av dokument bra nog för att producera välartad indata för andra modeller. Denna representation kan även hantera stora ordlistor och brus i texten. Resultaten visar även att ämnesgrupperingen av responsdatan är godartad nog att användas som mål för klassificeringsmodeller sådant att korrekta meningar kan genereras som respons.

Toxic compounds, such as pesticides, are routinely tested against a range of aquatic, avian and mammalian species as part of the registration process. The need for reducing dependence on animal testing has led to an increasing interest in alternative methods such as in silico modelling. The QSAR (Quantitative Structure Activity Relationship)-based models are already in use for predicting physicochemical properties, environmental fate, eco-toxicological effects, and specific biological endpoints for a wide range of chemicals. Data plays an important role in modelling QSARs and also in result analysis for toxicity testing processes. This research addresses number of issues in predictive toxicology. One issue is the problem of data quality. Although large amount of toxicity data is available from online sources, this data may contain some unreliable samples and may be defined as of low quality. Its presentation also might not be consistent throughout different sources and that makes the access, interpretation and comparison of the information difficult. To address this issue we started with detailed investigation and experimental work on DEMETRA data. The DEMETRA datasets have been produced by the EC-funded project DEMETRA. Based on the investigation, experiments and the results obtained, the author identified a number of data quality criteria in order to provide a solution for data evaluation in toxicology domain. An algorithm has also been proposed to assess data quality before modelling. Another issue considered in the thesis was the missing values in datasets for toxicology domain. Least Square Method for a paired dataset and Serial Correlation for single version dataset provided the solution for the problem in two different situations. A procedural algorithm using these two methods has been proposed in order to overcome the problem of missing values. Another issue we paid attention to in this thesis was modelling of multi-class data sets in which the severe imbalance class samples distribution exists. The imbalanced data affect the performance of classifiers during the classification process. We have shown that as long as we understand how class members are constructed in dimensional space in each cluster we can reform the distribution and provide more knowledge domain for the classifier.

As we gain increased understanding of genetic disorders and gene regulation more focus has turned towards complex interactions. Combinations of genes or gene and environmental factors have been suggested to explain the missing heritability behind complex diseases. Furthermore, gene activation and splicing seem to be governed by a complex machinery of histone modification (HM), transcription factor (TF), and DNA sequence signals. This thesis aimed to apply and develop multivariate machine learning methods for use on such biological problems. Monte Carlo feature selection was combined with rule-based classification to identify interactions between HMs and to study the interplay of factors with importance for asthma and allergy. Firstly, publicly available ChIP-seq data (Paper I) for 38 HMs was studied. We trained a classifier for predicting exon inclusion levels based on the HMs signals. We identified HMs important for splicing and illustrated that splicing could be predicted from the HM patterns. Next, we applied a similar methodology on data from two large birth cohorts describing asthma and allergy in children (Paper II). We identified genetic and environmental factors with importance for allergic diseases which confirmed earlier results and found candidate gene-gene and gene-environment interactions. In order to interpret and present the classifiers we developed Ciruvis, a web-based tool for network visualization of classification rules (Paper III). We applied Ciruvis on classifiers trained on both simulated and real data and compared our tool to another methodology for interaction detection using classification. Finally, we continued the earlier study on epigenetics by analyzing HM and TF signals in genes with or without evidence of bidirectional transcription (Paper IV). We identified several HMs and TFs with different signals between unidirectional and bidirectional genes. Among these, the CTCF TF was shown to have a well-positioned peak 60-80 bp upstream of the transcription start site in unidirectional genes.

Cette thèse est issue du projet Européen Homo.symbiosus qui étudie les transitions d'équilibre des interactions entre l'hôte et son microbiote intestinal. Pour étudier les transitions nous suivons deux directions : la modélisation mécaniste des interactions hôte-microbiote et l'analyse de données temporelles de comptage microbien.Nous avons enrichi et simulé un modèle déterministe de la crypte intestinale grâce au schéma numérique EDK, en étudiant notamment l'impact des différents paramètres en utilisant la méthode des effets élémentaires de Morris. Ce modèle s'est avéré capable de simuler d'une part des états symbiotiques et dysbiotiques des interactions et d'autre part des scénarios de transition.En parallèle, un modèle EDO compartimental du colon inspiré de travaux existants a été développé et couplé au modèle de crypte. La thèse a contribué à l'enrichissement de la modélisation du métabolisme bactérien et à la modélisation de l'immunité innée à l'échelle de la muqueuse intestinale. Une exploration numérique nous a permis d'évaluer l'influence de l'alimentation sur l'état stationnaire du modèle et d'étudier l'effet d'un scénario pathologique en mimant une brèche de la barrière épithéliale.De plus, nous avons développé une approche d'analyse des données microbiennes visant à évaluer la déviation des écosystèmes microbiens subissant une forte perturbation de leur environnement par rapport à un état de référence. Cette méthode, basée sur une classification DMM, permet d'étudier les transitions d'équilibre de l'écosystème dans le cas de données avec peu d'individus et peu de points de temps. Par ailleurs, une méthode de classification de courbes utilisant le modèle SBM a été appliquée pour étudier l'effet de différentes perturbations de l'écosystème microbien, des résultats de cette étude ont pu être utilisés pour enrichir le modèle d'interactions hôte-microbiote
This thesis stems from the European project Homo.symbiosus, which investigates the equilibrium transitions of interactions between the host and its intestinal microbiota. To study these transitions, we pursue two directions: the mechanistic modeling of host-microbiota interactions and the analysis of temporal microbial count data.We enriched and simulated a deterministic model of the intestinal crypt using the EDK numerical scheme, particularly studying the impact of different parameters using the Morris Elementary Effects method. This model proved capable of simulating, on one hand, symbiotic and dysbiotic interaction states and, on the other hand, transition scenarios between states of dysbiosis and symbiosis.In parallel, a compartmental ODE model of the colon, inspired by existing studies, was developed and coupled with the crypt model. The thesis contributed to the enhancement of bacterial metabolism modeling and the modeling of innate immunity at the scale of the intestinal mucosa. A numerical exploration allowed us to assess the influence of diet on the steady state of the model and to study the effect of a pathological scenario by mimicking a breach in the epithelial barrier.Furthermore, we developed an approach to analyze microbial data aimed at assessing the deviation of microbial ecosystems undergoing significant environmental disturbances compared to a reference state. This method, based on DMM classification, enables the study of ecosystem equilibrium transitions in cases with few individuals and few time points. Moreover, a curve classification method using the SBM model was applied to investigate the effects of various disturbances on the microbial ecosystem; the results from this study were used to enrich the host-microbiota interaction model

The focus of this thesis is on the implementation, the evaluation and the useful application of the third dimension in software visualization. Software engineering is characterized by a complex interplay of different stakeholders that produce and use several artifacts. Software visualization is used as one mean to address this increasing complexity. It provides role- and task-specific views of artifacts that contain information about structure, behavior, and evolution of a software system in its entirety. The main potential of the third dimension is the possibility to provide multiple views in one software visualization for all three aspects. However, empirical findings concerning the role of the third dimension in software visualization are rare. Furthermore, there are only few 3D software visualizations that provide multiple views of a software system including all three aspects. Finally, the current tool support lacks of generating easy integrateable, scalable, and platform independent 2D, 2.5D, and 3D software visualizations automatically. Hence, the objective is to develop a software visualization that represents all important structural entities and relations of a software system, that can display behavioral and evolutionary aspects of a software system as well, and that can be generated automatically. In order to achieve this objective the following research methods are applied. A literature study is conducted, a software visualization generator is conceptualized and prototypically implemented, a structured approach to plan and design controlled experiments in software visualization is developed, and a controlled experiment is designed and performed to investigate the role of the third dimension in software visualization. The main contributions are an overview of the state-of-the-art in 3D software visualization, a structured approach including a theoretical model to control influence factors during controlled experiments in software visualization, an Eclipse-based generator for producing automatically role- and task-specific 2D, 2.5D, and 3D software visualizations, the controlled experiment investigating the role of the third dimension in software visualization, and the recursive disk metaphor combining the findings with focus on the structure of software including useful applications of the third dimension regarding behavior and evolution.

In some classification problems where human judgments, qualitative and imprecise data exist, uncertainty comes from fuzziness rather than randomness. Limited number of fuzzy classification approaches is available for use for these classification problems to capture the effect of fuzzy uncertainty imbedded in data. The scope of this study mainly comprises two parts: new fuzzy classification approaches based on Tanaka&rsquo
s Fuzzy Linear Regression (FLR) approach, and an improvement of an existing one, Improved Fuzzy Classifier Functions (IFCF). Tanaka&rsquo
s FLR approach is a well known fuzzy regression technique used for the prediction problems including fuzzy type of uncertainty. In the first part of the study, three alternative approaches are presented, which utilize the FLR approach for a particular customer satisfaction classification problem. A comparison of their performances and their applicability in other cases are discussed. In the second part of the study, the improved IFCF method, Nonparametric Improved Fuzzy Classifier Functions (NIFCF), is presented, which proposes to use a nonparametric method, Multivariate Adaptive Regression Splines (MARS), in clustering phase of the IFCF method. NIFCF method is applied on three data sets, and compared with Fuzzy Classifier Function (FCF) and Logistic Regression (LR) methods.

Achieving high software quality today involves manual analysis, test planning, documentation of testing strategy and test cases, and development of automated test scripts to support regression testing. This thesis is motivated by the opportunity to bridge the gap between current test automation and true test automation by investigating learning-based solutions to software testing. We present an approach that combines a trainable web component classifier, a test case description language, and a trainable test generation and execution system that can learn to generate new test cases. Training data was collected and hand-labeled across 7 systems, 95 web pages, and 17,360 elements. A total of 250 test flows were also manually hand-crafted for training purposes. Various machine learning algorithms were evaluated. Results showed that Random Forest classifiers performed well on several web component classification problems. In addition, Long Short-Term Memory neural networks were able to model and generate new valid test flows.

This thesis demonstrates that agent-based modelling offers a viable compatriot to traditional experimental methodologies for criminology scholars, that can be applied to explore the divide between micro-level criminological theory and macro-level observations of crime; and in turn, aid in the assessment of those theories which aim to describe the crime event. The following overarching research question is addressed: Are the micro-level mechanisms of the opportunity theories generatively sufficient to explain macroscopic patterns commonly observed in the empirical study of crime? Drawing on the approach of generative social science (Epstein, 1999), this thesis presents a systematic assessment of the generative sufficiency of three distinct mechanisms of offender movement, target selection and learning derived from the routine activity approach (Cohen & Felson, 1979), rational choice perspective (Clarke, 1980; Cornish & Clarke, 1986) and crime pattern theory (Brantingham & Brantingham, 1978, 1981). An agent-based model of offending is presented, in which an artificial landscape is inhabited by both potential victims and offenders who behave according to several of the key propositions of the routine activity approach, rational choice perspective and crime pattern theory. Following a computational laboratory-based approach, for each hypothetical mechanism studied, control and experimental behaviours are developed to represent the absence or presence of a proposed mechanism within the virtual population.
Thesis (PhD Doctorate)
Doctor of Philosophy (PhD)
School of Criminology and Criminal Justice
Arts, Education and Law
Full Text

Recently, Cognitive Radio (CR) has been intended as an intelligent radio endowed with cognition which can be developed by implementing Artificial Intelligence (AI) techniques. Specifically, data-driven Self-Awareness (SA) functionalities, such as detection of spectrum abnormalities, can be effectively implemented as shown by the proposed research. One important application is PHY-layer security since it is essential to establish secure wireless communications against external jamming attacks. In this framework, signals are non-stationary and features from such kind of dynamic spectrum, with multiple high sampling rate signals, are then extracted through the Stockwell Transform (ST) with dual-resolution which has been proposed and validated in this work as part of spectrum sensing techniques. Afterwards, analysis of the state-of-the-art about learning dynamic models from observed features describes theoretical aspects of Machine Learning (ML). In particular, following the recent advances of ML, learning deep generative models with several layers of non-linear processing has been selected as AI method for the proposed spectrum abnormality detection in CR for a brain-inspired, data-driven SA. In the proposed approach, the features extracted from the ST representation of the wideband spectrum are organized in a high-dimensional generalized state vector and, then, a generative model is learned and employed to detect any deviation from normal situations in the analysed spectrum (abnormal signals or behaviours). Specifically, conditional GAN (C-GAN), auxiliary classifier GAN (AC-GAN), and deep VAE have been considered as deep generative models. A dataset of a dynamic spectrum with multi-OFDM signals has been generated by using the National Instruments mm-Wave Transceiver which operates at 28 GHz (central carrier frequency) with 800 MHz frequency range. Training of the deep generative model is performed on the generalized state vector representing the mmWave spectrum with normality pattern without any malicious activity. Testing is based on new and independent data samples corresponding to abnormality pattern where the moving signal follows a different behaviour which has not been observed during training. An abnormality indicator is measured and used for the binary classification (normality hypothesis otherwise abnormality hypothesis), while the performance of the generative models is evaluated and compared through ROC curves and accuracy metrics.

L’objectiu principal d’aquesta tesi és crear un sistema CAD avançat per a qualsevol tipus de modalitat d’imatge mèdica amb altes taxes de sensibilitat i especificitat basades en tècniques d’aprenentatge profund. Més concretament, volem millorar el mètode automàtic de detecció de les regions d’interès (ROI), que són àrees de la imatge que contenen possibles teixits malalts, així com la segmentació de les troballes (delimitació de la frontera) i, en definitiva, una predicció del diagnosi més adequat (classificació). En aquesta tesi ens centrem en diversos camps, que inclouen mamografies i ecografies per diagnosticar un càncer de mama, anàlisi de lesions de la pell en imatges dermoscòpiques i inspecció del fons de la retina per evitar la retinopatia diabètica.
El objetivo principal de esta tesis es crear un sistema CAD avanzado para cualquier tipo de modalidad de imagen médica con altas tasas de sensibilidad y especificidad basadas en técnicas de aprendizaje profundo. Más concretamente, queremos mejorar el método automático de detección de las regiones de interés (ROI), que son áreas de la imagen que contienen posibles tejidos enfermos, así como la segmentación de los hallazgos (delimitación de la frontera) y, en definitiva, una predicción del diagnóstico más adecuado (clasificación). En esta tesis nos centramos en diversos campos, que incluyen mamografías y ecografías para diagnosticar un cáncer de mama, análisis de lesiones de la piel en imágenes dermoscòpiques y inspección del fondo de la retina para evitar la retinopatía diabética
The main aim of this thesis is to create an advanced CAD system for any type of medical image modality with high sensitivity and specificity rates based on deep learning techniques. More specifically, we want to improve the automatic method of detection of Regions of Interest (ROI), which are areas of the image that contain possible ill tissues, as well as segmentation of the findings (delimitation with a boundary), and ultimately, a prediction of a most suitable diagnose (classification). In this thesis, we focus on several topics including mammograms and ultrasound images to diagnose breast cancer, skin lesions analysis in dermoscopic images and retinal fundus images examination to avoid diabetic retinopathy.

The aim of the project presented in this paper was to investigate the possibility toautomatically sub-classify the superfamily of Short-chain Dehydrogenase/Reductases (SDR).This was done based on an algorithm previously designed to sub-classify the superfamily ofMedium-chain Dehydrogenase/Reductases (MDR). While the SDR-family is interesting andimportant to sub-classify there was also a focus on making the process as automatic aspossible so that future families also can be classified using the same methods.To validate the results generated it was compared to previous sub-classifications done on theSDR-family. The results proved promising and the work conducted here can be seen as a goodinitial part of a more comprehensive full investigation

The task of Phone recognition is a fundamental task in Speech recognition and often serves a critical role in bench-marking purposes. Researchers have used a variety of models used in the past to address this task, using both generative and discriminative learning approaches. Among them, generative approaches such as the use of Gaussian mixture model-based hidden Markov models are always favored because of their mathematical tractability. However, the use of generative models such as hidden Markov models and its hybrid varieties is no longer in fashion owing to a large inclination to discriminative learning approaches, which have been found to perform better. The only downside is that these approaches do not always ensure mathematical tractability or convergence guarantees as opposed to their generative counterparts. So, the research problem was to investigate whether there could be a process of augmenting the modeling capability of generative Models using a kind of neural network based architectures that could simultaneously prove mathematically tractable and expressive. Normalizing flows are a class of generative models that have been garnered a lot of attention recently in the field of density estimation and offer a method for exact likelihood computation and inference. In this project, a few varieties of Normalizing flow-based hidden Markov models were used for the task of Phone recognition on the TIMIT dataset. It was been found that these models and their mixture model varieties outperformed classical generative model varieties like Gaussian mixture models. A decision fusion approach using classical Gaussian and Normalizing flow-based mixtures showed competitive results compared to discriminative learning approaches. Further analysis based on classes of speech phones was carried out to compare the generative models used. Additionally, a study of the robustness of these algorithms to noisy speech conditions was also carried out.
Uppgiften för fonemigenkänning är en grundläggande uppgift i taligenkänning och tjänar ofta en kritisk roll i benchmarkingändamål. Forskare har använt en mängd olika modeller som använts tidigare för att hantera denna uppgift genom att använda både generativa och diskriminerande inlärningssätt. Bland dem är generativa tillvägagångssätt som användning av Gaussian-blandnings modellbaserade dolda Markov-modeller alltid föredragna på grund av deras matematiska spårbarhet. Men användningen av generativa modeller som dolda Markov-modeller och dess hybridvarianter är inte längre på mode på grund av en stor lutning till diskriminerande inlärningsmetoder, som har visat sig fungera bättre. Den enda nackdelen är att dessa tillvägagångssätt inte alltid säkerställer matematisk spårbarhet eller konvergensgarantier i motsats till deras generativa motsvarigheter. Således var forskningsproblemet att undersöka om det kan finnas en process för att förstärka modelleringsförmågan hos generativa modeller med hjälp av ett slags neurala nätverksbaserade arkitekturer som samtidigt kunde visa sig matematiskt spårbart och uttrycksfullt. Normaliseringsflöden är en klass generativa modeller som nyligen har fått mycket uppmärksamhet inom området för densitetsberäkning och erbjuder en metod för exakt sannolikhetsberäkning och slutsats. I detta projekt användes några få varianter av Normaliserande flödesbaserade dolda Markov-modeller för uppgiften att fonemigenkänna i TIMIT-datasatsen. Det visade sig att dessa modeller och deras blandningsmodellvarianter överträffade klassiska generativa modellvarianter som Gaussiska blandningsmodeller. Ett beslutssmältningsstrategi med klassiska Gaussiska och Normaliserande flödesbaserade blandningar visade konkurrenskraftiga resultat jämfört med diskriminerande inlärningsmetoder. Ytterligare analys baserat på klasser av talsignaler utfördes för att jämföra de generativa modellerna som användes. Dessutom genomfördes en studie av robustheten hos dessa algoritmer till bullriga talförhållanden.

The generalization abilities of machine learning algorithms often depend on the algorithms’ initialization, parameter settings, training sets, or feature selections. For instance, SVM classifier performance largely relies on whether the selected kernel functions are suitable for real application data. To enhance the performance of individual classifiers, this dissertation proposes classifier fusion models using computational intelligence knowledge to combine different classifiers. The first fusion model called T1FFSVM combines multiple SVM classifiers through constructing a fuzzy logic system. T1FFSVM can be improved by tuning the fuzzy membership functions of linguistic variables using genetic algorithms. The improved model is called GFFSVM. To better handle uncertainties existing in fuzzy MFs and in classification data, T1FFSVM can also be improved by applying type-2 fuzzy logic to construct a type-2 fuzzy classifier fusion model (T2FFSVM). T1FFSVM, GFFSVM, and T2FFSVM use accuracy as a classifier performance measure. AUC (the area under an ROC curve) is proved to be a better classifier performance metric. As a comparison study, AUC-based classifier fusion models are also proposed in the dissertation. The experiments on biomedical datasets demonstrate promising performance of the proposed classifier fusion models comparing with the individual composing classifiers. The proposed classifier fusion models also demonstrate better performance than many existing classifier fusion methods. The dissertation also studies one interesting phenomena in biology domain using machine learning and classifier fusion methods. That is, how protein structures and sequences are related each other. The experiments show that protein segments with similar structures also share similar sequences, which add new insights into the existing knowledge on the relation between protein sequences and structures: similar sequences share high structure similarity, but similar structures may not share high sequence similarity.

Un nombre important de modèles probabilistes connaissent une grande perte d'intérêt pour la classification avec apprentissage supervisé depuis un certain nombre d'années, tels que le Naive Bayes ou la chaîne de Markov cachée. Ces modèles, qualifiés de génératifs, sont critiqués car leur classificateur induit doit prendre en compte la loi des observations, qui peut s'avérer très complexe à apprendre quand le nombre de features de ces derniers est élevé. C'est notamment le cas en Traitement des Langues Naturelles, où les récents algorithmes convertissent des mots en vecteurs numériques de grande taille pour atteindre de meilleures performances.Au cours de cette thèse, nous montrons que tout modèle génératif peut définir son classificateur sans prendre en compte la loi des observations. Cette proposition remet en question la catégorisation connue des modèles probabilistes et leurs classificateurs induits - en classes générative et discriminante - et ouvre la voie à un grand nombre d'applications possibles. Ainsi, la chaîne de Markov cachée peut être appliquée sans contraintes à la décomposition syntaxique de textes, ou encore le Naive Bayes à l'analyse de sentiments.Nous allons plus loin, puisque cette proposition permet de calculer le classificateur d'un modèle probabiliste génératif avec des réseaux de neurones. Par conséquent, nous « neuralisons » les modèles cités plus haut ainsi qu'un grand nombre de leurs extensions. Les modèles ainsi obtenus permettant d'atteindre des scores pertinents pour diverses tâches de Traitement des Langues Naturelles tout en étant interprétable, nécessitant peu de données d'entraînement, et étant simple à mettre en production
Many probabilistic models have been neglected for classification tasks with supervised learning for several years, as the Naive Bayes or the Hidden Markov Chain. These models, called generative, are criticized because the induced classifier must learn the observations' law. This problem is too complex when the number of observations' features is too large. It is especially the case with Natural Language Processing tasks, as the recent embedding algorithms convert words in large numerical vectors to achieve better scores.This thesis shows that every generative model can define its induced classifier without using the observations' law. This proposition questions the usual categorization of the probabilistic models and classifiers and allows many new applications. Therefore, Hidden Markov Chain can be efficiently applied to Chunking and Naive Bayes to sentiment analysis.We go further, as this proposition allows to define the classifier induced from a generative model with neural network functions. We "neuralize" the models mentioned above and many of their extensions. Models so obtained allow to achieve relevant scores for many Natural Language Processing tasks while being interpretable, able to require little training data, and easy to serve

This paper presents methods and results of county-level population estimation using Landsat Thematic Mapper (TM) images of Denton County and Collin County in Texas. Landsat TM images acquired in March 2000 were classified into residential and non-residential classes using maximum likelihood classification and knowledge-based classification methods. Accuracy assessment results from the classified image produced using knowledge-based classification and traditional supervised classification (maximum likelihood classification) methods suggest that knowledge-based classification is more effective than traditional supervised classification methods. Furthermore, using randomly selected samples of census block groups, ordinary least squares (OLS) and geographically weighted regression (GWR) models were created for total population estimation. The overall accuracy of the models is over 96% at the county level. The results also suggest that underestimation normally occurs in block groups with high population density, whereas overestimation occurs in block groups with low population density.

The machine learning method deep neural networks are commonly used for artificial intelligence applications such as speech recognition, robotics, and computer vision. Deep neural networks often have very good accuracy, the downside is the complexity of the computations. To be able to use deep neural network models on devices with less computing power, such as smart-phones e.g., can the model run on the cloud and send the results to the device. This thesis will evaluate the possibility to use a smart-phone as a camera unit with Google’s open source neural network called Inception, to identify traffic signs. The thesis analyzes the possibility to move the computation to the cloud and still use the system for real-time applications, and compare it to running the image model on the edge (the device itself). The accuracy of the model, as well as how estimations of future 5G mobile networks will affect the quality of service for the system is also analyzed. The result shows that the model achieved an accuracy of 88.0 % on the "German traffic sign benchmark" data set and 97.6 % on a newly created data set (data sets of images to test the neural network model on). The total time when using this system, from sending the image to receiving the result, is > 2 s. Because of this can it not be used for any application affecting traffic safety. Estimated improvements from future 5G mobile networks could include reduced communication delay, ultra-reliable communication, and with the higher bandwidth available could the system achieve a higher capacity if that would be required e.g. sending higher quality images.

As the information flow on the Internet keeps growing it becomes increasingly easy to miss important news which does not have a mass appeal. Combating this problem calls for increasingly sophisticated information retrieval methods. Pre-trained transformer based language models have shown great generalization performance on many natural language processing tasks. This work investigates how well such a language model, Open AI’s General Pre-trained Transformer 2 model (GPT-2), generalizes to information retrieval and classification of online news articles, written in English, with the purpose of comparing this approach with the more traditional method of Term Frequency-Inverse Document Frequency (TF-IDF) vectorization. The aim is to shed light on how useful state-of-the-art transformer based language models are for the construction of personalized information retrieval systems. Using transfer learning the smallest version of GPT-2 is trained to rank and classify news articles achieving similar results to the purely TF-IDF based approach. While the average Normalized Discounted Cumulative Gain (NDCG) achieved by the GPT-2 based model was about 0.74 percentage points higher the sample size was too small to give these results high statistical certainty.
Informationsflödet på Internet fortsätter att öka vilket gör det allt lättare att missa viktiga nyheter som inte intresserar en stor mängd människor. För att bekämpa detta problem behövs allt mer sofistikerade informationssökningsmetoder. Förtränade transformermodeller har sedan ett par år tillbaka tagit över som de mest framstående neurala nätverken för att hantera text. Det här arbetet undersöker hur väl en sådan språkmodell, Open AIs General Pre-trained Transformer 2 (GPT-2), kan generalisera från att generera text till att användas för informationssökning och klassificering av texter. För att utvärdera detta jämförs en transformerbaserad modell med en mer traditionell Term Frequency- Inverse Document Frequency (TF-IDF) vektoriseringsmodell. Målet är att klargöra hur användbara förtränade transformermodeller faktiskt är i skapandet av specialiserade informationssökningssystem. Den minsta versionen av språkmodellen GPT-2 anpassas och tränas om till att ranka och klassificera nyhetsartiklar, skrivna på engelska, och uppnår liknande prestanda som den TF-IDF baserade modellen. Den GPT-2 baserade modellen hade i genomsnitt 0.74 procentenheter högre Normalized Discounted Cumulative Gain (NDCG) men provstorleken var ej stor nog för att ge dessa resultat hög statistisk säkerhet.

This thesis deals with the integration of auxiliary data knowledge into machine learning methods especially prototype based classification models. The problem of classification is diverse and evaluation of the result by using only the accuracy is not adequate in many applications. Therefore, the classification tasks are analyzed more deeply. Possibilities to extend prototype based methods to integrate extra knowledge about the data or the classification goal is presented to obtain problem adequate models. One of the proposed extensions is Generalized Learning Vector Quantization for direct optimization of statistical measurements besides the classification accuracy. But also modifying the metric adaptation of the Generalized Learning Vector Quantization for functional data, i. e. data with lateral dependencies in the features, is considered.

The process of designing a floorplan is highly iterative and requires extensive human labor. Currently, there are a number of computer programs that aid humans in floorplan design. These programs, however, are limited in their inability to fully automate the creative process. Such automation would allow a professional to quickly generate many possible floorplan solutions, greatly expediting the process. However, automating this creative process is very difficult because of the many implicit and explicit rules a model must learn in order create viable floorplans. In this paper, we propose a method of floorplan generation using two machine learning models: a sequential model that generates rooms within the floorplan, and a graph-based model that finds adjacencies between generated rooms. Each of these models can be altered such that they are each capable of producing a floorplan independently; however, we find that the combination of these models outperforms each of its pieces, as well as a statistic-based approach.

Multi-label classification consists of learning a function that is capable of mapping an object to a set of relevant labels. It has applications such as the association of genes with biological functions, semantic classification of scenes and text categorization. Traditional classification (i.e., single-label) is therefore a particular case of multi-label classification in which each object is associated with exactly one label. A successful approach to constructing classifiers is to obtain a probabilistic model of the relation between object attributes and labels. This model can then be used to classify objects, finding the most likely prediction by computing the marginal probability or the most probable explanation (MPE) of the labels given the attributes. Depending on the probabilistic models family chosen, such inferences may be intractable when the number of labels is large. Sum-Product Networks (SPN) are deep probabilistic models, that allow tractable marginal inference. Nevertheless, as with many other probabilistic models, performing MPE inference is NP- hard. Although, SPNs have already been used successfully for traditional classification tasks (i.e. single-label), there is no in-depth investigation on the use of SPNs for Multi-Label classification. In this work we investigate the use of SPNs for Multi-Label classification. We compare several algorithms for learning SPNs combined with different proposed approaches for classification. We show that SPN-based multi-label classifiers are competitive against state-of-the-art classifiers, such as Random k-Labelsets with Support Vector Machine and MPE inference on CutNets, in a collection of benchmark datasets.
A classificação Multi-Rótulo consiste em aprender uma função que seja capaz de mapear um objeto para um conjunto de rótulos relevantes. Ela possui aplicações como associação de genes com funções biológicas, classificação semântica de cenas e categorização de texto. A classificação tradicional, de rótulo único é, portanto, um caso particular da Classificação Multi-Rótulo, onde cada objeto está associado com exatamente um rótulo. Uma abordagem bem sucedida para classificação é obter um modelo probabilístico da relação entre atributos do objeto e rótulos. Esse modelo pode então ser usado para classificar objetos, encon- trando a predição mais provável por meio da probabilidade marginal ou a explicação mais provavél dos rótulos dados os atributos. Dependendo da família de modelos probabilísticos escolhidos, tais inferências podem ser intratáveis quando o número de rótulos é grande. As redes Soma-Produto (SPN, do inglês Sum Product Network) são modelos probabilísticos profundos, que permitem inferência marginal tratável. No entanto, como em muitos outros modelos probabilísticos, a inferência da explicação mais provavél é NP-difícil. Embora SPNs já tenham sido usadas com sucesso para tarefas de classificação tradicionais, não existe investigação aprofundada no uso de SPNs para classificação Multi-Rótulo. Neste trabalho, investigamos o uso de SPNs para classificação Multi-Rótulo. Comparamos vários algoritmos de aprendizado de SPNs combinados com diferentes abordagens propostos para classi- ficação. Mostramos que os classificadores Multi-Rótulos baseados em SPN são competitivos contra classificadores estado-da-arte, como Random k-Labelsets usando Máquinas de Suporte Vetorial e inferência exata da explicação mais provavél em CutNets, em uma coleção de conjuntos de dados de referência.

Network traffic exhibits a high level of variability over short periods of time. This variability impacts negatively on the performance (accuracy) of anomaly-based network Intrusion Detection Systems (IDS) that are built using predictive models in a batch-learning setup. This thesis investigates how adapting the discriminating threshold of model predictions, specifically to the evaluated traffic, improves the detection rates of these Intrusion Detection models. Specifically, this thesis studied the adaptability features of three well known Machine Learning algorithms: C5.0, Random Forest, and Support Vector Machine. The ability of these algorithms to adapt their prediction thresholds was assessed and analysed under different scenarios that simulated real world settings using the prospective sampling approach. A new dataset (STA2018) was generated for this thesis and used for the analysis. This thesis has demonstrated empirically the importance of threshold adaptation in improving the accuracy of detection models when training and evaluation (test) traffic have different statistical properties. Further investigation was undertaken to analyse the effects of feature selection and data balancing processes on a model's accuracy when evaluation traffic with different significant features were used. The effects of threshold adaptation on reducing the accuracy degradation of these models was statistically analysed. The results showed that, of the three compared algorithms, Random Forest was the most adaptable and had the highest detection rates. This thesis then extended the analysis to apply threshold adaptation on sampled traffic subsets, by using different sample sizes, sampling strategies and label error rates. This investigation showed the robustness of the Random Forest algorithm in identifying the best threshold. The Random Forest algorithm only needed a sample that was 0.05% of the original evaluation traffic to identify a discriminating threshold with an overall accuracy rate of nearly 90% of the optimal threshold.

Over the last decade a huge amount of experimental data on biological systems has been generated by modern high-throughput methods. Aided by bioinformatics, the '-omics' (genomics, transcriptomics, proteomics, metabolomics and interactomics) have listed, quantif ed and analyzed molecular components and interactions on all levels of cellular regulation. However, a comprehensive framework, that does not only list, but links all those components, is still largely missing. The biology-based but highly interdisciplinary field of systems biology aims at such a holistic understanding of complex biological systems covering the length scales from molecules to whole organisms. Spanning the length scales, it has to integrate the data from very different fields and to bring together scientists from those fields. For linking experiments and theory, hypothesis-driven research is an indispensable concept, formulating a cycle of experiment, modeling, model predictions for new experiments and, fi nally, their experimental validation as the start of the new iteration. On the hierarchy of length scales certain unique entities can be identi fied. At the nanometer scale such functional entities are molecules and at the micrometer level these are the cells. Cells can be studied in vitro as independent individuals isolated from an organism, but their interplay and communication in vivo is crucial for tissue function. Control over such regulation mechanisms is therefore a main goal of medical research. The requirements for understanding cellular interplay also illustrate the interdisciplinarity of systems biology, because chemical, physical and biological knowledge is needed simultaneously. Following the notion of cells as the basic units of life, the focus of this thesis are mathematical multi-scale models of multi-cellular systems employing the concept of individual (or agent) based modeling (IBM). This concept accounts for the entity cell and their individuality in function and space. Motivated by experimental observations, cells are represented as elastic and adhesive spheres. Their interaction is given by a model for elastic homogeneous spheres, which has been established for analysis of the elastic response of cells, plus an adhesion term. Cell movement is modeled by an equation of motion for each cell which is based on the balance of interaction, friction and active forces on the respective cell. As a fi rst step the model was carefully examined with regard to the model assumptions, namely, spherical shape, homogeneous isotropic elastic body and apriori undirected movement. The model examination included simulations of cell sorting and compression of multicellular spheroids. Cell sorting could not be achieved with only short range adhesion. However, it sorting completed with long range interactions for small cell numbers, but failed for larger aggregates. Compression dynamics of multi-cellular spheroids was apparently reproduced qualitatively by the model. But in a more detailed survey neither the time scales nor the rounding after compression could be reproduced. Based on these results, the applications consistent with the assumed simpli cations are discussed. One already established application is colony growth in two-dimensional cell cultures. In order to model cell growth and division, a two-phase model of the cell cycle was established. In a growth phase the cell doubles its volume by stochastic increments, and in a mitotic phase it divides into two daughter cells of equal volume. Additionally, control of the cell cycle by contact inhibition is included in the model. After examination of its applicability, the presented model is used for simulations of in vitro growth of mesenchymal stem cells (MSC) and subsequent cartilage formation in multi-cellular spheroids. A main factor for both processes is the oxygen concentration. Experimental results have shown, that i) MSC grow much better in vitro at low than at high oxygen concentrations and ii) the MSC progeny harvested from low oxygen culture produce higher amounts of the cartilage components aggrecan and collagen II in multicellular spheroids than the ones from high oxygen culture. In order to model these processes, IBM was extended by a stochastic model for cellular differentiation. In this model cellular differentiation is captured phenomenologically by two additional individual properties, the degree of differentiation and the lineage or cell type, which are subject to fl uctuations, that are state and environment dependent. After fitting the model parameters to the experimental results on MSC growth in monoclonal expansion cultures at low and high oxygen concentrations, the resulting simulated cell populations were used for initialization of the simulations of cartilage formation in multi-cellular spheroids. The model nicely reproduced the experimental results on growth dynamics and the observed number of functional cells in the spheroids and suggests the following explanation for the difference between the two expansion cultures: due to the stronger pre-differentiation found after expansion in high oxygen, the plasticity of these cells is smaller and less cell adopt the chondrogenic phenotype and start to produce cartilage. Moreover, the model predicts an optimal oxygen concentration for cartilage formation independent of expansion culture and a de-differentiating effect of low oxygen culture within 24h. Because all simulations comply with the concept of hypothesis-driven research and follow closely the experimental protocols, they can easily be tested and are currently used for optimization of a bioreactor for cartilage production. Cell populations are composed of individual cells and regulation of population properties is performed by individual cell, but knowledge about individual cell fates is largely missing due to the problem of single cell tracking. The IBM modeling approach used for modeling MSC growth and differentiation generically includes information of each individual cell and is therefore perfectly suited for tackling this question. Based on the validated parameter set, the model was used to generate predictions on plasticity of single cells and related population dynamics. Single cell plasticity was quantifi ed by calculating transition times into stem cell and differentiated cell states at high and low oxygen concentrations. At low oxygen the results predict a frequent exchange between all subpopulations, while at high oxygen a quasi-deterministic differentiation is found. After quantifying the plasticity of single cells at low and high oxygen concentration, the plasticity of a cell population is addressed in a simulation closely following a regeneration experiment of populations of hematopoietic progenitor cells. In the simulation the regeneration of the distribution of differentiation states in the population is monitored after selection of subpopulations of stem cells and differentiated cells. Simulated regeneration occurs on the time scales estimated from the single cell transition times except the unexpectedly fast regeneration from differentiated cells in the high oxygen environment, which favors differentiation. The latter case emphasizes the importance of single outlier cells in such system, which in this case repopulate less differentiated states with their progeny. In general, cell proliferation and regeneration behavior are in uenced by biomechanical and geometrical properties of the environment e.g. matrix stiffness or cell density. Because in the model cells are represented as physical objects, a variation of friction is linked to cell motility. The cultures of less motile cells become denser at the same size and the effects of contact inhibition of growth more pronounced. This variation of friction coe fficients allows the comparison of cultures with varying degrees of contact inhibition regarding their differentiation structure and the results suggest, that stalled proliferation is su fficient to explain the well-known differentiation effects in confl uent colonies. In addition, the composition of the simulated stem cell pool was analyzed regarding differentiation. In contrast to the established pedigree models, where stem cell can only be produced by asymmetric division, this model predicts that most of the cells in stem cell states descend from progenitor cells of intermediate differentiation states. A more detailed analysis of single cell derived clones revealed properties that could not be described by the model so far. First, a differentiation gradient was observed in larger colonies, that was the opposite of the one predicted by the model. Second, the proliferative activity turned out to depend not only on oxygen, but also to be a property of individual clones persisting over many generations. Because the relation slow growth/pre-differentiation also holds for single cell derived clones, the general model of differentiation is extended by another heritable individual property. Motivated by the decline of proliferation and differentiation in culture and the high metabolic and epigenetic activity during cell division, each division event is assumed to de-stabilize stem cell states. Consequently, in the model the cells age in terms of cell divisions determines the fl uctuations in stem cell states and the environment the mean fl uctuation strength. Including this novel concept, that links aging to growth and differentiation dynamics, into the model reproduces the experimental results regarding differentiation gradient and persistent clonal heterogeneity. The spatial differentiation pattern can largely be explained by the spatio-temporal growth pattern of the mono-clonal cell assembly: cells close to the border of the cell assembly have undergone more cell divisions than those in the interior and therefore their stem cell states are less stable. Heterogeneity of single-cell derived clones depends on the age of the first cell in the clone. When the stem cell fluctuations equal the mean fl uctuations strength, the proliferative activity passes a maximum at a certain age due to the destabilization of stem cell states. Thereafter the proliferative activity decreases, because more time is spent in non-proliferative differentiated states. Considering the number of divisions the cells have already undergone in vivo and after the initial expansion in vitro, it can be assumed that all cells have already passed this maximum. Interestingly, the model also predicts an optimal age for directed differentiation, when cells stably differentiate, but have not lost the required plasticity. According to the model, this clonal heterogeneity may be caused purely in vitro, but hypothetical simulation of in vivo aging yielded results consistent with experiments on MSC from rats of varying age. Finally, the detailed molecular regulation mechanisms in a multi-scale tissue model of liver zonation was studied, in which the key molecular components were explicitly modeled. Hence, this model resolved the intracellular regulation in higher resolution than the above considered differentiation models which had summarized the intracellular control and differentiation mechanisms by a few phenomenological, dynamical variables. The metabolic zonation of the liver is essential for many of the complex liver functions. One of the vitally important enzymes, glutamine synthetase, (GS) is only synthesized in a strictly defi ned pattern. Experimental evidence has shown that a particular pathway, the canonical wnt pathway, controls expression of the gene for GS. A model for transport, receptor dynamics and intracellular regulation mechanism has been set up for modeling the spatio-temporal formation of this pattern. It includes membrane-bound transport of the morphogen and an enzyme kinetics approach to fibeta-catenin-regulation in the interior of the cell. As an IBM this model reproduces the results of co-culture experiments in which two-dimensional arrangements of liver cells and an epithelial liver cell line give rise to different patterns of GS synthesis. The two main predictions of the model are: First, GS-synthesis requires a certain local cell number of wnt releasing cells. And second, a simple inversion of geometry explains the difference between the specifi c GS pattern found in the liver and in the co-culture experiments. Summarizing the results presented in this thesis, it can be concluded that properties such as the occurrence of memory effects and single cells pursuing fates far off the population average could be essential for biological function. Considering the role of single cells in many tissues, the use of individual based methods, that are able to take such effects into account, can be expected to be a very valuable tool for the problems of systems biology.

Many authors have shown that investors are not fully rational, as the traditional Efficient Markets Hypothesis suggests, and that investor sentiment can have an impact on stock prices. As investor sentiment is not directly measurable, different proxies have been used by researchers. In addition, progress in natural language processing has contributed to the development of new sentiment measures based on text sources obtained by news providers and social media. This work deals with a classification problem on financial news data and defines a reliable proxy for investor sentiment using both dictionary – based and supervised Machine Learning techniques. In particular, LSTMs networks have been adopted. The resulting sentiment proxies have been used as exogenous variables in the mean and variance equations of a Generalized Autoregressive Conditional Heteroskedasticity model in order to prove the existence of a relationship among them and stock returns and among them and volatility.

Managing customer credit is an important issue for each commercial bank; therefore, banks take great care when dealing with customer loans to avoid any improper decisions that can lead to loss of opportunity or financial losses. The manual estimation of customer creditworthiness has become both time- and resource-consuming. Moreover, a manual approach is subjective (dependable on the bank employee who gives this estimation), which is why devising and implementing programming models that provide loan estimations is the only way of eradicating the ‘human factor’ in this problem. This model should give recommendations to the bank in terms of whether or not a loan should be given, or otherwise can give a probability in relation to whether the loan will be returned. Nowadays, a number of models have been designed, but there is no ideal classifier amongst these models since each gives some percentage of incorrect outputs; this is a critical consideration when each percent of incorrect answer can mean millions of dollars of losses for large banks. However, the LR remains the industry standard tool for credit-scoring models development. For this purpose, an investigation is carried out on the combination of the most efficient classifiers in credit-scoring scope in an attempt to produce a classifier that exceeds each of its classifiers or components. In this work, a fusion model referred to as ‘the Classifiers Consensus Approach’ is developed, which gives a lot better performance than each of single classifiers that constitute it. The difference of the consensus approach and the majority of other combiners lie in the fact that the consensus approach adopts the model of real expert group behaviour during the process of finding the consensus (aggregate) answer. The consensus model is compared not only with single classifiers, but also with traditional combiners and a quite complex combiner model known as the ‘Dynamic Ensemble Selection’ approach. As a pre-processing technique, step data-filtering (select training entries which fits input data well and remove outliers and noisy data) and feature selection (remove useless and statistically insignificant features which values are low correlated with real quality of loan) are used. These techniques are valuable in significantly improving the consensus approach results. Results clearly show that the consensus approach is statistically better (with 95% confidence value, according to Friedman test) than any other single classifier or combiner analysed; this means that for similar datasets, there is a 95% guarantee that the consensus approach will outperform all other classifiers. The consensus approach gives not only the best accuracy, but also better AUC value, Brier score and H-measure for almost all datasets investigated in this thesis. Moreover, it outperformed Logistic Regression. Thus, it has been proven that the use of the consensus approach for credit-scoring is justified and recommended in commercial banks. Along with the consensus approach, the dynamic ensemble selection approach is analysed, the results of which show that, under some conditions, the dynamic ensemble selection approach can rival the consensus approach. The good sides of dynamic ensemble selection approach include its stability and high accuracy on various datasets. The consensus approach, which is improved in this work, may be considered in banks that hold the same characteristics of the datasets used in this work, where utilisation could decrease the level of mistakenly rejected loans of solvent customers, and the level of mistakenly accepted loans that are never to be returned. Furthermore, the consensus approach is a notable step in the direction of building a universal classifier that can fit data with any structure. Another advantage of the consensus approach is its flexibility; therefore, even if the input data is changed due to various reasons, the consensus approach can be easily re-trained and used with the same performance.

The recent advances in aerial- and satellite-based hyperspectral imaging sensor technologies have led to an increased availability of Earth's images with high spatial and spectral resolution, which opened the door to a large range of important applications. Hyperspectral imaging records detailed spectrum of the received light in each spatial position in the image, in which each pixel contains a highly detailed representation of the reflectance of the materials present on the ground, and a better characterization in terms of geometrical details. Since different substances exhibit different spectral signatures, the abundance of informative content conveyed in the hyperspectral images permits an improved characterization of different land coverage. Therefore, hyperspectral imaging emerged as a well-suited technology for accurate image classi fication in remote sensing. In spite of that, a signi ficantly increased complexity of the analysis introduces a series of challenges that need to be addressed on a serious note. In order to fully exploit the potential offered by these sensors, there is a need to develop accurate and effective models for spectral-spatial analysis of the recorded data. This thesis aims at presenting novel strategies for the analysis and classifi cation of hyperspectral remote sensing images, placing the focal point on the investigation on deep networks for the extraction and integration of spectral and spatial information. Deep learning has demonstrated cutting-edge performances in computer vision, particularly in object recognition and classi cation. It has also been successfully adopted in hyperspectral remote sensing domain as well. However, it is a very challenging task to fully utilize the massive potential of deep models in hyperspectral remote sensing applications since the number of training samples is limited which limits the representation capability of a deep model. Furthermore, the existing architectures of deep models need to be further investigated and modifi ed accordingly to better complement the joint use of spectral and spatial contents of hyperspectral images. In this thesis, we propose three different deep learning-based models to effectively represent spectral-spatial characteristics of hyperspectral data in the interest of classifi cation of remote sensing images. Our first proposed model focuses on integrating CRF and CNN into an end-to-end learning framework for classifying images. Our main contribution in this model is the introduction of a deep CRF in which the CRF parameters are computed using CNN and further optimized by adopting piecewise training. Furthermore, we address the problem of over fitting by employing data augmentation techniques and increased the size of the training samples for training deep networks. Our proposed 3DCNN-CRF model can be trained to fully exploit the usefulness of CRF in the context of classi fication by integrating it completely inside of a deep model. Considering that the separation of constituent materials and their abundances provide detailed analysis of the data, our second algorithm investigates the potential of using unmixing results in deep models to classify images. We extend an existing region based structure preserving non-negative matrix factorization method to estimate groups of spectral bands with the goal to capture subtle spectral-spatial distribution from the image. We subsequently use these important unmixing results as input to generate superpixels, which are further represented by kernel density estimated probability distribution function. Finally, these abundance information-guided superpixels are directly supplied into a deep model in which the inference is implicitly formulated as a recurrent neural network to perform the eventual classifi cation. Finally, we perform a detailed investigation on the possibilities of adopting generative adversarial models into hyperspectral image classifi cation. We present a GAN-based spectral-spatial method that primarily focuses on signifi cantly improving the multiclass classi cation ability of the discriminator of GAN models. In this context, we propose to adopt the triplet constraint property and extend it to build a useful feature embedding for remote sensing images for use in classi cation. Furthermore, our proposed Triplet- 3D-GAN model also includes feedback from discriminator's intermediate features to improve the quality of the generator's sample generation process.
Thesis (PhD Doctorate)
Doctor of Philosophy (PhD)
School of Info & Comm Tech
Science, Environment, Engineering and Technology
Full Text

Représenter l'information décrivant la musique est une activité complexe, qui implique différentes sous-tâches. Ce manuscrit de thèse porte principalement sur la musique classique et étudie comment représenter et exploiter ses informations. L'objectif principal est l'étude de stratégies de représentation et de découverte des connaissances appliquées à la musique classique, dans des domaines tels que la production de base de connaissances, la prédiction de métadonnées et les systèmes de recommandation. Nous proposons une architecture pour la gestion des métadonnées de musique à l'aide des technologies du Web Sémantique. Nous introduisons une ontologie spécialisée et un ensemble de vocabulaires contrôlés pour les différents concepts spécifiques à la musique. Ensuite, nous présentons une approche de conversion des données, afin d’aller au-delà de la pratique bibliothécaire actuellement utilisée, en s’appuyant sur des règles de mapping et sur l’interconnexion avec des vocabulaires contrôlés. Enfin, nous montrons comment ces données peuvent être exploitées. En particulier, nous étudions des approches basées sur des plongements calculés sur des métadonnées structurées, des titres et de la musique symbolique pour classer et recommander de la musique. Plusieurs applications de démonstration ont été réalisées pour tester les approches et les ressources précédentes
Representing the information about music is a complex activity that involves different sub-tasks. This thesis manuscript mostly focuses on classical music, researching how to represent and exploit its information. The main goal is the investigation of strategies of knowledge representation and discovery applied to classical music, involving subjects such as Knowledge-Base population, metadata prediction, and recommender systems. We propose a complete workflow for the management of music metadata using Semantic Web technologies. We introduce a specialised ontology and a set of controlled vocabularies for the different concepts specific to music. Then, we present an approach for converting data, in order to go beyond the librarian practice currently in use, relying on mapping rules and interlinking with controlled vocabularies. Finally, we show how these data can be exploited. In particular, we study approaches based on embeddings computed on structured metadata, titles, and symbolic music for ranking and recommending music. Several demo applications have been realised for testing the previous approaches and resources

Els mètodes de reconeixement d'accions permeten als sistemes intel·ligents reconèixer accions humanes en vídeos de la vida quotidiana. No obstant, molts mètodes de reconeixement d'accions donen taxes notables d’error de classificació degut a les grans variacions dins dels vídeos de la mateixa classe i als canvis en el punt de vista, l'escala i el fons. Per reduir la classificació incorrecta , proposem un nou mètode de representació de vídeo que captura l'evolució temporal de l'acció que succeeix en el vídeo, un nou mètode per a la segmentació de mans i un nou mètode per al reconeixement d'activitats humanes en imatges fixes.
Los métodos de reconocimiento de acciones permiten que los sistemas inteligentes reconozcan acciones humanas en videos de la vida cotidiana. No obstante, muchos métodos de reconocimiento de acciones dan tasas notables de error de clasificación debido a las grandes variaciones dentro de los videos de la misma clase y los cambios en el punto de vista, la escala y el fondo. Para reducir la clasificación errónea, Łproponemos un nuevo método de representación de video que captura la evolución temporal de la acción que ocurre en el video completo, un nuevo método para la segmentación de manos y un nuevo método para el reconocimiento de actividades humanas en imágenes fijas.
Action recognition methods enable intelligent systems to recognize human actions in daily life videos. However, many action recognition methods give noticeable misclassification rates due to the big variations within the videos of the same class, and the changes in viewpoint, scale and background. To reduce the misclassification rate, we propose a new video representation method that captures the temporal evolution of the action happening in the whole video, a new method for human hands segmentation and a new method for human activity recognition in still images.

There is considerable opportunity to develop new modelling techniques within a Geographic Information Systems (GIS) framework for the development of sustainable marine cage culture. However, the spatial data sets are often uncertain and incomplete, therefore new spatial models employing “soft computing” methods such as fuzzy logic may be more suitable. The aim of this study is to develop a model using Neuro-fuzzy techniques in a 3D GIS (Arc View 3.2) to predict coastal environmental vulnerability for Atlantic salmon cage aquaculture. A 3D hydrodynamic model (3DMOHID) coupled to a particle-tracking model is applied to study the circulation patterns, dispersion processes and residence time in Mulroy Bay, Co. Donegal Ireland, an Irish fjard (shallow fjordic system), an area of restricted exchange, geometrically complicated with important aquaculture activities. The hydrodynamic model was calibrated and validated by comparison with sea surface and water flow measurements. The model provided spatial and temporal information on circulation, renewal time, helping to determine the influence of winds on circulation patterns and in particular the assessment of the hydrographic conditions with a strong influence on the management of fish cage culture. The particle-tracking model was used to study the transport and flushing processes. Instantaneous massive releases of particles from key boxes are modelled to analyse the ocean-fjord exchange characteristics and, by emulating discharge from finfish cages, to show the behaviour of waste in terms of water circulation and water exchange. In this study the results from the hydrodynamic model have been incorporated into GIS to provide an easy-to-use graphical user interface for 2D (maps), 3D and temporal visualization (animations), for interrogation of results. v Data on the physical environment and aquaculture suitability were derived from a 3- dimensional hydrodynamic model and GIS for incorporation into the final model framework and included mean and maximum current velocities, current flow quiescence time, water column stratification, sediment granulometry, particulate waste dispersion distance, oxygen depletion, water depth, coastal protection zones, and slope. The Neuro-fuzzy classification model NEFCLASS–J, was used to develop learning algorithms to create the structure (rule base) and the parameters (fuzzy sets) of a fuzzy classifier from a set of classified training data. A total of 42 training sites were sampled using stratified random sampling from the GIS raster data layers, and the vulnerability categories for each were manually classified into four categories based on the opinions of experts with field experience and specific knowledge of the environmental problems investigated. The final products, GIS/based Neuro Fuzzy maps were achieved by combining modeled and real environmental parameters relevant to marine fin fish Aquaculture. Environmental vulnerability models, based on Neuro-fuzzy techniques, showed sensitivity to the membership shapes of the fuzzy sets, the nature of the weightings applied to the model rules, and validation techniques used during the learning and validation process. The accuracy of the final classifier selected was R=85.71%, (estimated error value of ±16.5% from Cross Validation, N=10) with a Kappa coefficient of agreement of 81%. Unclassified cells in the whole spatial domain (of 1623 GIS cells) ranged from 0% to 24.18 %. A statistical comparison between vulnerability scores and a significant product of aquaculture waste (nitrogen concentrations in sediment under the salmon cages) showed that the final model gave a good correlation between predicted environmental vi vulnerability and sediment nitrogen levels, highlighting a number of areas with variable sensitivity to aquaculture. Further evaluation and analysis of the quality of the classification was achieved and the applicability of separability indexes was also studied. The inter-class separability estimations were performed on two different training data sets to assess the difficulty of the class separation problem under investigation. The Neuro-fuzzy classifier for a supervised and hard classification of coastal environmental vulnerability has demonstrated an ability to derive an accurate and reliable classification into areas of different levels of environmental vulnerability using a minimal number of training sets. The output will be an environmental spatial model for application in coastal areas intended to facilitate policy decision and to allow input into wider ranging spatial modelling projects, such as coastal zone management systems and effective environmental management of fish cage aquaculture.

This Ph.D thesis work is dedicated to 3D facial surface analysis, processing as well as to the newly proposed 3D face recognition modality, which is based on mapping techniques. Facial surface processing and analysis is one of the most important steps for 3Dface recognition algorithms. Automatic anthropometric facial features localization also plays an important role for face localization, face expression recognition, face registration ect., thus its automatic localization is a crucial step for 3D face processing algorithms. In this work we focused on precise and rotation invariant landmarks localization, which are later used directly for face recognition. The landmarks are localized combining local surface properties expressed in terms of differential geometry tools and global facial generic model, used for face validation. Since curvatures, which are differential geometry properties, are sensitive to surface noise, one of the main contributions of this thesis is a modification of curvatures calculation method. The modification incorporates the surface noise into the calculation method and helps to control smoothness of the curvatures. Therefore the main facial points can be reliably and precisely localized (100% nose tip localization using 8 mm precision)under the influence of rotations and surface noise. The modification of the curvatures calculation method was also tested under different face model resolutions, resulting in stable curvature values. Finally, since curvatures analysis leads to many facial landmark candidates, the validation of which is time consuming, facial landmarks localization based on learning technique was proposed. The learning technique helps to reject incorrect landmark candidates with a high probability, thus accelerating landmarks localization. Face recognition using 3D models is a relatively new subject, which has been proposed to overcome shortcomings of 2D face recognition modality. However, 3Dface recognition algorithms are likely more complicated. Additionally, since 3D face models describe facial surface geometry, they are more sensitive to facial expression changes. Our contribution is reducing dimensionality of the input data by mapping3D facial models on to 2D domain using non-rigid, conformal mapping techniques. Having 2D images which represent facial models, all previously developed 2D face recognition algorithms can be used. In our work, conformal shape images of 3Dfacial surfaces were fed in to 2D2 PCA, achieving more than 86% recognition rate rank-one using the FRGC data set. The effectiveness of all the methods has been evaluated using the FRGC and Bosphorus datasets.

The field of machine learning, and, in particular, methods to improve the capability of machines to perform a wider variety of generalised tasks are among the most rapidly growing research areas in today’s world. The current applications of machine learning and artificial intelligence can be divided into many significant fields namely computer vision, data sciences, real time analytics and Natural Language Processing (NLP). All these applications are being used to help computer based systems to operate more usefully in everyday contexts. Computer vision research is currently active in a wide range of areas such as the development of autonomous vehicles, object recognition, Content Based Image Retrieval (CBIR), image segmentation and terrestrial analysis from space (i.e. crop estimation). Despite significant prior research, the area of object recognition still has many topics to be explored. This PhD thesis focuses on using advanced machine learning approaches to enable the automated recognition of fresh produce (i.e. fruits and vegetables) at supermarket self-checkouts. This type of complex classification task is one of the most recently emerging applications of advanced computer vision approaches and is a productive research topic in this field due to the limited means of representing the features and machine learning techniques for classification. Fruits and vegetables offer significant inter and intra class variance in weight, shape, size, colour and texture which makes the classification challenging. The applications of effective fruit and vegetable classification have significant importance in daily life e.g. crop estimation, fruit classification, robotic harvesting, fruit quality assessment, etc. One potential application for this fruit and vegetable classification capability is for supermarket self-checkouts. Increasingly, supermarkets are introducing self-checkouts in stores to make the checkout process easier and faster. However, there are a number of challenges with this as all goods cannot readily be sold with packaging and barcodes, for instance loose fresh items (e.g. fruits and vegetables). Adding barcodes to these types of items individually is impractical and pre-packaging limits the freedom of choice when selecting fruits and vegetables and creates additional waste, hence reducing customer satisfaction. The current situation, which relies on customers correctly identifying produce themselves leaves open the potential for incorrect billing either due to inadvertent error, or due to intentional fraudulent misclassification resulting in financial losses for the store. To address this identified problem, the main goals of this PhD work are: (a) exploring the types of visual and non-visual sensors that could be incorporated into a self-checkout system for classification of fruits and vegetables, (b) determining a suitable feature representation method for fresh produce items available at supermarkets, (c) identifying optimal machine learning techniques for classification within this context and (d) evaluating our work relative to the state-of-the-art object classification results presented in the literature. An in-depth analysis of related computer vision literature and techniques is performed to identify and implement the possible solutions. A progressive process distribution approach is used for this project where the task of computer vision based fruit and vegetables classification is divided into pre-processing and classification techniques. Different classification techniques have been implemented and evaluated as possible solution for this problem. Both visual and non-visual features of fruit and vegetables are exploited to perform the classification. Novel classification techniques have been carefully developed to deal with the complex and highly variant physical features of fruit and vegetables while taking advantages of both visual and non-visual features. The capability of classification techniques is tested in individual and ensemble manner to achieved the higher effectiveness. Significant results have been obtained where it can be concluded that the fruit and vegetables classification is complex task with many challenges involved. It is also observed that a larger dataset can better comprehend the complex variant features of fruit and vegetables. Complex multidimensional features can be extracted from the larger datasets to generalise on higher number of classes. However, development of a larger multiclass dataset is an expensive and time consuming process. The effectiveness of classification techniques can be significantly improved by subtracting the background occlusions and complexities. It is also worth mentioning that ensemble of simple and less complicated classification techniques can achieve effective results even if applied to less number of features for smaller number of classes. The combination of visual and nonvisual features can reduce the struggle of a classification technique to deal with higher number of classes with similar physical features. Classification of fruit and vegetables with similar physical features (i.e. colour and texture) needs careful estimation and hyper-dimensional embedding of visual features. Implementing rigorous classification penalties as loss function can achieve this goal at the cost of time and computational requirements. There is a significant need to develop larger datasets for different fruit and vegetables related computer vision applications. Considering more sophisticated loss function penalties and discriminative hyper-dimensional features embedding techniques can significantly improve the effectiveness of the classification techniques for the fruit and vegetables applications.

Product development process begins at a very abstract level of understanding the requirements. The data needs to be passed on the next phase of development. This happens after every stage for further development and finally a product is made. This thesis deals with the data exchange process of software development process in specific. The problem lies in handling of data in terms of redundancy and versions of the data to be handled. Also, once data passed on to next stage, the ability to exchange it in reveres order is not existent in evident forms. The results found during this thesis discusses the solutions for the problem by getting all the data at same level, in terms of its format. Having the concept ready, provides an opportunity to use this data based on our requirements. In this research, the problem of data consistency, data verification is dealt with. This data is used during the development and data merging from various sources. The concept that is formulated can be expanded to a wide variety of applications with respect to development process. If the process involves exchange of data - scalability and generalization are the main foundation concepts that are contained within the concept.

S rozvojom techník pre efektívnu akvizíciu genomických dát sa jednou z kľúčových vedeckých výziev stala interpretácia výsledkov týchto experimentov v zmysluplnom biologickom kontexte. Táto práca sa zameriava na využitie informácií ukrytých v dobre charakterizovaných transkriptomických a epigenomických dátach z verejne dostupných zdrojov pre účely takejto interpretácie. Najskôr je vytvorený integrovaný súbor dát generovaných metódami DNase-seq a ATAC-seq, ktoré kvantifikujú chromatínovú dostupnosť. Tieto údaje sú doplnené verejne dostupnými výsledkami techniky RNA-seq pre kvantitatívne hodnotenie génovej expresie a vhodne predspracované pre ďalšiu analýzu. Pripravené dáta sú následne použité na trénovanie modelov strojového učenia (klasifikátorov) s dvomi základnými cieľmi. Po prvé za účelom augmentácie metadát prislúchajúcich k jednotlivým biologickým vzorkám v trénovacom dátovom súbore pomocou predikcie nedefinovaných anotácií. Po druhé pre anotáciu zle charakterizovaných testovacích dát (nepoužitých v trénovacej fáze) za účelom overenia generalizačnej schopnosti zostavených modelov. Dosiahnuté výsledky ukazujú, že natrénované klasifikátory sú schopné zachytiť biologicky relevantné informácie, zatiaľ čo vplyv technických artefaktov je minimalizovaný. Navrhnutý prístup je preto schopný prispieť k lepšiemu pochopeniu komplexných transkriptomických a epigenomických dát, predovšetkým v oblasti onkologického výskumu.

De nombreux problèmes traités par l'IA sont des problèmes de classification de données d'entrées complexes qui doivent être séparées en différentes classes. Les fonctions transformant l'espace complexe des valeurs d'entrées en un espace plus simple, linéairement séparable, se font soit par apprentissage (réseaux convolutionels profonds), soit par projection dans un espace de haute dimension afin d'obtenir une représentation non-linéaire 'riche' des entrées puis un appariement linaire entre l'espace de haute dimension et les unités de sortie, tels qu'utilisés dans les Support Vector Machines (travaux de Vapnik 1966-1995). L'objectif de la thèse est de réaliser une architecture optimisée, générique dans un domaine d'application donné, permettant de pré-traiter des données afin de les préparer pour une classification en un minimum d'opérations. En outre, cette architecture aura pour but d'augmenter l'autonomie du modèle en lui permettant par exemple d'apprendre en continu, d'être robuste aux données corrompues ou d'identifier des données que le modèle ne pourrait pas traiter
Many issues addressed by AI involve the classification of complex input data that needs to be separated into different classes. The functions that transform the complex input values into a simpler, linearly separable space are achieved either through learning (deep convolutional networks) or by projecting into a high-dimensional space to obtain a 'rich' non-linear representation of the inputs, followed by a linear mapping between the high-dimensional space and the output units, as used in Support Vector Machines (Vapnik's work 1966-1995). The thesis aims to create an optimized, generic architecture capable of preprocessing data to prepare them for classification with minimal operations required. Additionally, this architecture aims to enhance the model's autonomy by enabling continuous learning, robustness to corrupted data, and the identification of data that the model cannot process

NetLogo has become a standard platform for agent-based simulation, yet there appears to be widespread belief that it is not suitable for large and complex models due to slow execution. Our experience does not support that belief. NetLogo programs often do run very slowly when written to minimize code length and maximize clarity, but relatively simple and easily tested changes can almost always produce major increases in execution speed. We recommend a five-step process for quantifying execution speed, identifying slow parts of code, and writing faster code. Avoiding or improving agent filtering statements can often produce dramatic speed improvements. For models with extensive initialization methods, reorganizing the setup procedure can reduce the initialization effort in simulation experiments. Programming the same behavior in a different way can sometimes provide order-of-magnitude speed increases. For models in which most agents do nothing on most time steps, discrete event simulation—facilitated by the time extension to NetLogo—can dramatically increase speed. NetLogo’s BehaviorSpace tool makes it very easy to conduct multiple-model-run experiments in parallel on either desktop or high performance cluster computers, so even quite slow models can be executed thousands of times. NetLogo also is supported by efficient analysis tools, such as BehaviorSearch and RNetLogo, that can reduce the number of model runs and the effort to set them up for (e.g.) parameterization and sensitivity analysis.

Simple random sampling (SRS) is the most commonly used sampling design in data collection. In many applications (e.g., in fisheries and medical research) quantification of the variable of interest is either time-consuming or expensive but ranking a number of sampling units, without actual measurement on them, can be done relatively easy and at low cost. In these situations, one may use rank-based sampling (RBS) designs to obtain more representative samples from the underlying population and improve the efficiency of the statistical inference. In this thesis, we study the theory and application of the finite mixture models (FMMs) under RBS designs. In Chapter 2, we study the problems of Maximum Likelihood (ML) estimation and classification in a general class of FMMs under different ranked set sampling (RSS) designs. In Chapter 3, deriving Fisher information (FI) content of different RSS data structures including complete and incomplete RSS data, we show that the FI contained in each variation of the RSS data about different features of FMMs is larger than the FI contained in their SRS counterparts. There are situations where it is difficult to rank all the sampling units in a set with high confidence. Forcing rankers to assign unique ranks to the units (as RSS) can lead to substantial ranking error and consequently to poor statistical inference. We hence focus on the partially rank-ordered set (PROS) sampling design, which is aimed at reducing the ranking error and the burden on rankers by allowing them to declare ties (partially ordered subsets) among the sampling units. Studying the information and uncertainty structures of the PROS data in a general class of distributions, in Chapter 4, we show the superiority of the PROS design in data analysis over RSS and SRS schemes. In Chapter 5, we also investigate the ML estimation and classification problems of FMMs under the PROS design. Finally, we apply our results to estimate the age structure of a short-lived fish species based on the length frequency data, using SRS, RSS and PROS designs.

The Aveston Copper and Kelly (ACK) Method has been routinely used in estimating the efficiency of the bond between the textile and cementitious matrix. This method however has a limited applicability due to the simplifying assumptions such as perfect bond. A numerical model for simulation of tensile behavior of reinforced cement-based composites is presented to capture the inefficiency of the bond mechanisms. In this approach the role of interface properties which are instrumental in the simulation of the tensile response is investigated. The model simulates the tension stiffening effect of cracked matrix, and evolution of crack spacing in tensile members. Independent experimental results obtained from literature are used to verify the model and develop composite tensile stress strain response using alkali resistant (AR) glass textile reinforced concrete. The composite stress strain response is then used with a bilinear representation of the composite obtained from the tensile stiffening model. The closed form and simplified equations for representation of flexural response are obtained and used for both back-calculation and also design. A method based on the average moment-curvature relationship in the structural design and analysis of one way and two way flexural elements using yield line analysis approaches is proposed. This comprehensive approach directly shows the interrelation of fundamental materials characterization techniques with simplified design equations for further utilization of textile reinforced concrete materials.

In this investigation, the fabrication of functionally graded structures of SS316L to Ni-based alloys were studied, using the novel technique 3D plasma metal deposition. Two Ni-based alloys were used, a heat resistance alloy Ni80-20 and the solid-solution strengthened Ni625. Different configurations were analyzed, for the Ni80-20 a hard transition and a smooth transition with a region of 50% SS316L/50% Ni80-20. Regarding the structures with Ni625, a smooth transition configuration and variations in the heat input were applied. The effect of the process parameters on the geometry of the structures and the microstructures was studied. Microstructure examinations were carried out using optical and scanning electron microscopy. In addition, microhardness analysis were made on the interfaces. In general, the smooth transition of both systems showed a gradual change in the properties. The microstructural results for the SS316L (both systems) showed an austenite matrix with δ-phase. For the mixed zone and the Ni80-20 an austenite (γ) matrix with some M7C3 precipitates and laves phase were recognized. The as-built Ni625 microstructure was composed of an austenite (γ) matrix with secondary phases laves and δ-Ni3Nb, and precipitates M7C3. The mixed zone exhibited the same phases but with changes in the morphology.