Dissertations / Theses on the topic 'Gaussian mixture models'

Thesis (MScEng (Electrical and Electronic Engineering))--University of Stellenbosch, 2010.
ENGLISH ABSTRACT: The importance of Language Identification for African languages is seeing a dramatic increase due to the development of telecommunication infrastructure and, as a result, an increase in volumes of data and speech traffic in public networks. By automatically processing the raw speech data the vital assistance given to people in distress can be speeded up, by referring their calls to a person knowledgeable in that language. To this effect a speech corpus was developed and various algorithms were implemented and tested on raw telephone speech data. These algorithms entailed data preparation, signal processing, and statistical analysis aimed at discriminating between languages. The statistical model of Gaussian Mixture Models (GMMs) were chosen for this research due to their ability to represent an entire language with a single stochastic model that does not require phonetic transcription. Language Identification for African languages using GMMs is feasible, although there are some few challenges like proper classification and accurate study into the relationship of langauges that need to be overcome. Other methods that make use of phonetically transcribed data need to be explored and tested with the new corpus for the research to be more rigorous.
AFRIKAANSE OPSOMMING: Die belang van die Taal identifiseer vir Afrika-tale is sien ’n dramatiese toename te danke aan die ontwikkeling van telekommunikasie-infrastruktuur en as gevolg ’n toename in volumes van data en spraak verkeer in die openbaar netwerke.Deur outomaties verwerking van die ruwe toespraak gegee die noodsaaklike hulp verleen aan mense in nood kan word vinniger-up ”, deur te verwys hul oproepe na ’n persoon ingelichte in daardie taal. Tot hierdie effek van ’n toespraak corpus het ontwikkel en die verskillende algoritmes is gemplementeer en getoets op die ruwe telefoon toespraak gegee.Hierdie algoritmes behels die data voorbereiding, seinverwerking, en statistiese analise wat gerig is op onderskei tussen tale.Die statistiese model van Gauss Mengsel Modelle (GGM) was gekies is vir hierdie navorsing as gevolg van hul vermo te verteenwoordig ’n hele taal met’ n enkele stogastiese model wat nodig nie fonetiese tanscription nie. Taal identifiseer vir die Afrikatale gebruik GGM haalbaar is, alhoewel daar enkele paar uitdagings soos behoorlike klassifikasie en akkurate ondersoek na die verhouding van TALE wat moet oorkom moet word.Ander metodes wat gebruik maak van foneties getranskribeerde data nodig om ondersoek te word en getoets word met die nuwe corpus vir die ondersoek te word strenger.

In this thesis, a probabilistic model for transforming a voice to sound like another specific voice is tested. The model is fully automatic and only requires some 100 training sentences from both speakers with the same acoustic content. The classical source-filter decomposition allows prosodic and spectral transformation to be performed independently. The transformations are based on a Gaussian mixture model and a transformation function suggested by Y. Stylianou. Feature vectors of the same content from the source and target speaker, aligned in time by dynamic time warping, are fitted to a GMM. The short time spectra, represented as cepstral coefficients and derived from LPC, and the pitch periods, represented as fundamental frequency estimated from the RAPT algorithm, are transformed with the same probabilistic transformation function. Several techniques of spectrum and pitch transformation were assessed in addition to some novel smoothing techniques of the fundamental frequency contour. The pitch transform was implemented on the excitation signal from the inverse LP filtering by time domain PSOLA. The transformed spectrum parameters were used in the synthesis filter with the transformed excitation as input to yield the transformed voice. A listening test was performed with the best setup from objective tests and the results indicate that it is possible to recognise the transformed voice as the target speaker with a 72 % probability. However, the synthesised voice was affected by a muffling effect due to incorrect frequency transformation and the prosody sounded somewhat robotic.

In most of state-of-the-art speech recognition systems, Gaussian mixture models (GMMs) are used to model the density of the emitting states in the hidden Markov models (HMMs). In a conventional system, the model parameters of each GMM are estimated directly and independently given the alignment. This results a large number of model parameters to be estimated, and consequently, a large amount of training data is required to fit the model. In addition, different sources of acoustic variability that impact the accuracy of a recogniser such as pronunciation variation, accent, speaker factor and environmental noise are only weakly modelled and factorized by adaptation techniques such as maximum likelihood linear regression (MLLR), maximum a posteriori adaptation (MAP) and vocal tract length normalisation (VTLN). In this thesis, we will discuss an alternative acoustic modelling approach — the subspace Gaussian mixture model (SGMM), which is expected to deal with these two issues better. In an SGMM, the model parameters are derived from low-dimensional model and speaker subspaces that can capture phonetic and speaker correlations. Given these subspaces, only a small number of state-dependent parameters are required to derive the corresponding GMMs. Hence, the total number of model parameters can be reduced, which allows acoustic modelling with a limited amount of training data. In addition, the SGMM-based acoustic model factorizes the phonetic and speaker factors and within this framework, other source of acoustic variability may also be explored. In this thesis, we propose a regularised model estimation for SGMMs, which avoids overtraining in case that the training data is sparse. We will also take advantage of the structure of SGMMs to explore cross-lingual acoustic modelling for low-resource speech recognition. Here, the model subspace is estimated from out-domain data and ported to the target language system. In this case, only the state-dependent parameters need to be estimated which relaxes the requirement of the amount of training data. To improve the robustness of SGMMs against environmental noise, we propose to apply the joint uncertainty decoding (JUD) technique that is shown to be efficient and effective. We will report experimental results on the Wall Street Journal (WSJ) database and GlobalPhone corpora to evaluate the regularisation and cross-lingual modelling of SGMMs. Noise compensation using JUD for SGMM acoustic models is evaluated on the Aurora 4 database.

A contribução original desta tese é um novo algoritmo que integra um aproximador de funções com alta eficiência amostral com aprendizagem por reforço em espaços de estados contínuos. A pesquisa completa inclui o desenvolvimento de um algoritmo online e incremental capaz de aprender por meio de uma única passada sobre os dados. Este algoritmo, chamado de Fast Incremental Gaussian Mixture Network (FIGMN) foi empregado como um aproximador de funções eficiente para o espaço de estados de tarefas contínuas de aprendizagem por reforço, que, combinado com Q-learning linear, resulta em performance competitiva. Então, este mesmo aproximador de funções foi empregado para modelar o espaço conjunto de estados e valores Q, todos em uma única FIGMN, resultando em um algoritmo conciso e com alta eficiência amostral, i.e., um algoritmo de aprendizagem por reforço capaz de aprender por meio de pouquíssimas interações com o ambiente. Um único episódio é suficiente para aprender as tarefas investigadas na maioria dos experimentos. Os resultados são analisados a fim de explicar as propriedades do algoritmo obtido, e é observado que o uso da FIGMN como aproximador de funções oferece algumas importantes vantagens para aprendizagem por reforço em relação a redes neurais convencionais.
This thesis’ original contribution is a novel algorithm which integrates a data-efficient function approximator with reinforcement learning in continuous state spaces. The complete research includes the development of a scalable online and incremental algorithm capable of learning from a single pass through data. This algorithm, called Fast Incremental Gaussian Mixture Network (FIGMN), was employed as a sample-efficient function approximator for the state space of continuous reinforcement learning tasks, which, combined with linear Q-learning, results in competitive performance. Then, this same function approximator was employed to model the joint state and Q-values space, all in a single FIGMN, resulting in a concise and data-efficient algorithm, i.e., a reinforcement learning algorithm that learns from very few interactions with the environment. A single episode is enough to learn the investigated tasks in most trials. Results are analysed in order to explain the properties of the obtained algorithm, and it is observed that the use of the FIGMN function approximator brings some important advantages to reinforcement learning in relation to conventional neural networks.

Thesis (M.S.) -- Clemson University, 2009.
Contains additional supplemental files. Title from first page of PDF file. Document formatted into pages; contains vii, 54 p. ; also includes color graphics.

Over the years analysts have used the EM algorithm to obtain maximum likelihood estimates from incomplete data for various models. The general algorithm admits several appealing properties such as strong global convergence; however, the rate of convergence is linear which in some cases may be unacceptably slow. This work is primarily concerned with applying Anderson acceleration to the EM algorithm for Gaussian mixture models (GMM) in hopes of alleviating slow convergence. As preamble we provide a review of maximum likelihood estimation and derive the EM algorithm in detail. The iterates that correspond to the GMM are then formulated and examples are provided. These examples show how faster convergence is experienced when the data are well separated, whereas much slower convergence is seen whenever the sample is poorly separated. The Anderson acceleration method is then presented, and its connection to the EM algorithm is discussed. The work is then concluded by applying Anderson acceleration to the EM algorithm which results in reducing the number of iterations required to obtain convergence.

COORDENAÇÃO DE APERFEIÇOAMENTO DO PESSOAL DE ENSINO SUPERIOR
Neste trabalho um novo modelo de mistura de distribuições é proposto, onde a estrutura da mistura é determinada por uma árvore de decisão com transição suave. Modelos baseados em mistura de distribuições são úteis para aproximar distribuições condicionais desconhecidas de dados multivariados. A estrutura em árvore leva a um modelo que é mais simples, e em alguns casos mais interpretável, do que os propostos anteriormente na literatura. Baseando-se no algoritmo de Esperança- Maximização (EM), foi derivado um estimador de quasi- máxima verossimilhança. Além disso, suas propriedades assintóticas são derivadas sob condições de regularidades. Uma estratégia de crescimento da árvore, do especifico para o geral, é também proposta para evitar possíveis problemas de identificação. Tanto a estimação quanto a estratégia de crescimento são avaliados em um experimento Monte Carlo, mostrando que a teoria ainda funciona para pequenas amostras. A habilidade de aproximação universal é ainda analisada em experimentos de simulação. Para concluir, duas aplicações com bases de dados reais são apresentadas.
In this work a new model of mixture of distributions is proposed, where the mixing structure is determined by a smooth transition tree architecture. Models based on mixture of distributions are useful in order to approximate unknown conditional distributions of multivariate data. The tree structure yields a model that is simpler, and in some cases more interpretable, than previous proposals in the literature. Based on the Expectation-Maximization (EM) algorithm a quasi-maximum likelihood estimator is derived and its asymptotic properties are derived under mild regularity conditions. In addition, a specific-to-general model building strategy is proposed in order to avoid possible identification problems. Both the estimation procedure and the model building strategy are evaluated in a Monte Carlo experiment, which give strong support for the theorydeveloped in small samples. The approximation capabilities of the model is also analyzed in a simulation experiment. Finally, two applications with real datasets are considered.

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2011.
Cataloged from PDF version of thesis.
Includes bibliographical references (p. 177-180).
Data assimilation, as presented in this thesis, is the statistical merging of sparse observational data with computational models so as to optimally improve the probabilistic description of the field of interest, thereby reducing uncertainties. The centerpiece of this thesis is the introduction of a novel such scheme that overcomes prior shortcomings observed within the community. Adopting techniques prevalent in Machine Learning and Pattern Recognition, and building on the foundations of classical assimilation schemes, we introduce the GMM-DO filter: Data Assimilation with Gaussian mixture models using the Dynamically Orthogonal field equations. We combine the use of Gaussian mixture models, the EM algorithm and the Bayesian Information Criterion to accurately approximate distributions based on Monte Carlo data in a framework that allows for efficient Bayesian inference. We give detailed descriptions of each of these techniques, supporting their application by recent literature. One novelty of the GMM-DO filter lies in coupling these concepts with an efficient representation of the evolving probabilistic description of the uncertain dynamical field: the Dynamically Orthogonal field equations. By limiting our attention to a dominant evolving stochastic subspace of the total state space, we bridge an important gap previously identified in the literature caused by the dimensionality of the state space. We successfully apply the GMM-DO filter to two test cases: (1) the Double Well Diffusion Experiment and (2) the Sudden Expansion fluid flow. With the former, we prove the validity of utilizing Gaussian mixture models, the EM algorithm and the Bayesian Information Criterion in a dynamical systems setting. With the application of the GMM-DO filter to the two-dimensional Sudden Expansion fluid flow, we further show its applicability to realistic test cases of non-trivial dimensionality. The GMMDO filter is shown to consistently capture and retain the far-from-Gaussian statistics that arise, both prior and posterior to the assimilation of data, resulting in its superior performance over contemporary filters. We present the GMM-DO filter as an efficient, data-driven assimilation scheme, focused on a dominant evolving stochastic subspace of the total state space, that respects nonlinear dynamics and captures non-Gaussian statistics, obviating the use of heuristic arguments.
by Thomas Sondergaard.
S.M.

We present limit theory for tests of homogeneity for finite mixture models. More specifically, we derive the asymptotic distribution of certain random quantities used for testing that a mixture of two distributions is in fact just a single distribution. Our methods apply to cases where the mixture component distributions come from one of a wide class of one-parameter exponential families, both continous and discrete. We consider two random quantities, one related to testing simple hypotheses, the other composite hypotheses. For simple hypotheses we consider the maximum of the standardised score process, which is itself a test statistic. For composite hypotheses we consider the maximum of the efficient score process, which is itself not a statistic (it depends on the unknown true distribution) but is asymptotically equivalent to certain common test statistics in a certain sense. We show that we can approximate both quantities with the maximum of a certain Gaussian process depending on the sample size and the true distribution of the observations, which when suitably normalised has a limiting distribution of the Gumbel extreme value type. Although the limit theory is not practically useful for computing approximate p-values, we use Monte-Carlo simulations to show that another method suggested by the theory, involving using a Studentised version of the maximum-score statistic and simulating a Gaussian process to compute approximate p-values, is remarkably accurate and uses a fraction of the computing resources that a straight Monte-Carlo approximation would.

We present limit theory for tests of homogeneity for finite mixture models. More specifically, we derive the asymptotic distribution of certain random quantities used for testing that a mixture of two distributions is in fact just a single distribution. Our methods apply to cases where the mixture component distributions come from one of a wide class of one-parameter exponential families, both continous and discrete. We consider two random quantities, one related to testing simple hypotheses, the other composite hypotheses. For simple hypotheses we consider the maximum of the standardised score process, which is itself a test statistic. For composite hypotheses we consider the maximum of the efficient score process, which is itself not a statistic (it depends on the unknown true distribution) but is asymptotically equivalent to certain common test statistics in a certain sense. We show that we can approximate both quantities with the maximum of a certain Gaussian process depending on the sample size and the true distribution of the observations, which when suitably normalised has a limiting distribution of the Gumbel extreme value type. Although the limit theory is not practically useful for computing approximate p-values, we use Monte-Carlo simulations to show that another method suggested by the theory, involving using a Studentised version of the maximum-score statistic and simulating a Gaussian process to compute approximate p-values, is remarkably accurate and uses a fraction of the computing resources that a straight Monte-Carlo approximation would.

Classificação de fluxos contínuos de dados possui muitos desafios para a comunidade de mineração de dados quando o ambiente não é estacionário. Um dos maiores desafios para a aprendizagem em fluxos contínuos de dados está relacionado com a adaptação às mudanças de conceito, as quais ocorrem como resultado da evolução dos dados ao longo do tempo. Duas formas principais de desenvolver abordagens adaptativas são os métodos baseados em conjunto de classificadores e os algoritmos incrementais. Métodos baseados em conjunto de classificadores desempenham um papel importante devido à sua modularidade, o que proporciona uma maneira natural de se adaptar a mudanças de conceito. Os algoritmos incrementais são mais rápidos e possuem uma melhor capacidade anti-ruído do que os conjuntos de classificadores, mas têm mais restrições sobre os fluxos de dados. Assim, é um desafio combinar a flexibilidade e a adaptação de um conjunto de classificadores na presença de mudança de conceito, com a simplicidade de uso encontrada em um único classificador com aprendizado incremental. Com essa motivação, nesta dissertação, propomos um algoritmo incremental, online e probabilístico para a classificação em problemas que envolvem mudança de conceito. O algoritmo é chamado IGMN-NSE e é uma adaptação do algoritmo IGMN. As duas principais contribuições da IGMN-NSE em relação à IGMN são: melhoria de poder preditivo para tarefas de classificação e a adaptação para alcançar um bom desempenho em cenários não estacionários. Estudos extensivos em bases de dados sintéticas e do mundo real demonstram que o algoritmo proposto pode rastrear os ambientes em mudança de forma muito próxima, independentemente do tipo de mudança de conceito.
Data stream classification poses many challenges for the data mining community when the environment is non-stationary. The greatest challenge in learning classifiers from data stream relates to adaptation to the concept drifts, which occur as a result of changes in the underlying concepts. Two main ways to develop adaptive approaches are ensemble methods and incremental algorithms. Ensemble method plays an important role due to its modularity, which provides a natural way of adapting to change. Incremental algorithms are faster and have better anti-noise capacity than ensemble algorithms, but have more restrictions on concept drifting data streams. Thus, it is a challenge to combine the flexibility and adaptation of an ensemble classifier in the presence of concept drift, with the simplicity of use found in a single classifier with incremental learning. With this motivation, in this dissertation we propose an incremental, online and probabilistic algorithm for classification as an effort of tackling concept drifting. The algorithm is called IGMN-NSE and is an adaptation of the IGMN algorithm. The two main contributions of IGMN-NSE in relation to the IGMN are: predictive power improvement for classification tasks and adaptation to achieve a good performance in non-stationary environments. Extensive studies on both synthetic and real-world data demonstrate that the proposed algorithm can track the changing environments very closely, regardless of the type of concept drift.

Thesis (Ph. D.)--University of Sydney, 2002.
Title from title screen (viewed Apr. 28, 2008). Submitted in fulfilment of the requirements for the degree of Doctor of Philosophy to the School of Mathematics and Statistics, Faculty of Science. Includes bibliography. Also available in print form.

Le traitement de nuages de points 3D de lidars permet aux robots mobiles autonomes terrestres de construire des modèles sémantiques de l'environnement extérieur dans lequel ils évoluent. Ces modèles sont intéressants car ils représentent des informations qualitatives, et ainsi donnent à un robot la capacité de raisonner à un niveau plus élevé d'abstraction. Le coeur d'un système de modélisation sémantique est la capacité de classifier les observations venant du capteur. Nous proposons un système de classification centré sur l'apprentissage non-supervisé. La prémière couche, la couche intermédiaire, consiste en un modèle de mélange gaussien. Ce modèle est déterminé de manière non-supervisée lors d'une étape de training. Il definit un ensemble de classes intermédiaires qui correspond à une partition fine des classes présentes dans l'environnement. La deuxième couche, la couche finale, consiste en un regroupement des classes intermédiaires dans un ensemble de classes finales qui, elles, sont interprétables dans le contexte de la tâche ciblée. Le regroupement est déterminé par un expert lors de l'étape de training, de manière supervisée, mais guidée par les classes intermédiaires. L'évaluation est basée sur deux jeux de données acquis avec de différents lidars et possédant différentes caractéristiques. L'évaluation est quantitative pour l'un des jeux de données, et qualitative pour l'autre. La concéption du système utilise la procédure standard de l'apprentissage, basée sur les étapes de training, validation et test. L'opération suit la pipeline standard de classification. Le système est simple, et ne requiert aucun pré-traitement ou post-traitement
The processing of 3D lidar point clouds enable terrestrial autonomous mobile robots to build semantic models of the outdoor environments in which they operate. Such models are interesting because they encode qualitative information, and thus provide to a robot the ability to reason at a higher level of abstraction. At the core of a semantic modelling system, lies the capacity to classify the sensor observations. We propose a two-layer classi- fication model which strongly relies on unsupervised learning. The first, intermediary layer consists of a Gaussian mixture model. This model is determined in a training step in an unsupervised manner, and defines a set of intermediary classes which is a fine-partitioned representation of the environment. The second, final layer consists of a grouping of the intermediary classes into final classes that are interpretable in a considered target task. This grouping is determined by an expert during the training step, in a process which is supervised, yet guided by the intermediary classes. The evaluation is done for two datasets acquired with different lidars and possessing different characteristics. It is done quantitatively using one of the datasets, and qualitatively using another. The system is designed following the standard learning procedure, based on a training, a validation and a test steps. The operation follows a standard classification pipeline. The system is simple, with no requirement of pre-processing or post-processing stages

Les différences entre conditions d'apprentissage et conditions de test peuvent considérablement dégrader la qualité des transcriptions produites par un système de reconnaissance automatique de la parole (RAP). L'adaptation est un moyen efficace pour réduire l'inadéquation entre les modèles du système et les données liées à un locuteur ou un canal acoustique particulier. Il existe deux types dominants de modèles acoustiques utilisés en RAP : les modèles de mélanges gaussiens (GMM) et les réseaux de neurones profonds (DNN). L'approche par modèles de Markov cachés (HMM) combinés à des GMM (GMM-HMM) a été l'une des techniques les plus utilisées dans les systèmes de RAP pendant de nombreuses décennies. Plusieurs techniques d'adaptation ont été développées pour ce type de modèles. Les modèles acoustiques combinant HMM et DNN (DNN-HMM) ont récemment permis de grandes avancées et surpassé les modèles GMM-HMM pour diverses tâches de RAP, mais l'adaptation au locuteur reste très difficile pour les modèles DNN-HMM. L'objectif principal de cette thèse est de développer une méthode de transfert efficace des algorithmes d'adaptation des modèles GMM aux modèles DNN. Une nouvelle approche pour l'adaptation au locuteur des modèles acoustiques de type DNN est proposée et étudiée : elle s'appuie sur l'utilisation de fonctions dérivées de GMM comme entrée d'un DNN. La technique proposée fournit un cadre général pour le transfert des algorithmes d'adaptation développés pour les GMM à l'adaptation des DNN. Elle est étudiée pour différents systèmes de RAP à l'état de l'art et s'avère efficace par rapport à d'autres techniques d'adaptation au locuteur, ainsi que complémentaire
Differences between training and testing conditions may significantly degrade recognition accuracy in automatic speech recognition (ASR) systems. Adaptation is an efficient way to reduce the mismatch between models and data from a particular speaker or channel. There are two dominant types of acoustic models (AMs) used in ASR: Gaussian mixture models (GMMs) and deep neural networks (DNNs). The GMM hidden Markov model (GMM-HMM) approach has been one of the most common technique in ASR systems for many decades. Speaker adaptation is very effective for these AMs and various adaptation techniques have been developed for them. On the other hand, DNN-HMM AMs have recently achieved big advances and outperformed GMM-HMM models for various ASR tasks. However, speaker adaptation is still very challenging for these AMs. Many adaptation algorithms that work well for GMMs systems cannot be easily applied to DNNs because of the different nature of these models. The main purpose of this thesis is to develop a method for efficient transfer of adaptation algorithms from the GMM framework to DNN models. A novel approach for speaker adaptation of DNN AMs is proposed and investigated. The idea of this approach is based on using so-called GMM-derived features as input to a DNN. The proposed technique provides a general framework for transferring adaptation algorithms, developed for GMMs, to DNN adaptation. It is explored for various state-of-the-art ASR systems and is shown to be effective in comparison with other speaker adaptation techniques and complementary to them

[EN] This thesis considers new applications of non-Gaussian mixtures in the framework of statistical signal processing and pattern recognition. The non-Gaussian mixtures were implemented by mixtures of independent component analyzers (ICA). The fundamental hypothesis of ICA is that the observed signals can be expressed as a linear transformation of a set of hidden variables, usually referred to as sources, which are statistically independent. This independence allows factoring the original M-dimensional probability density function (PDF) of the data as a product of one-dimensional probability densities, greatly simplifying the modeling of the data. ICA mixture models (ICAMM) provide further flexibility by alleviating the independency requirement of ICA, thus allowing the model to obtain local projections of the data without compromising its generalization capabilities. Here are explored new possibilities of ICAMM for the purposes of estimation and classification of signals. The thesis makes several contributions to the research in non-Gaussian mixtures: (i) a method for maximum-likelihood estimation of missing data, based on the maximization of the PDF of the data given the ICAMM; (ii) a method for Bayesian estimation of missing data that minimizes the mean squared error and can obtain the confidence interval of the prediction; (iii) a generalization of the sequential dependence model for ICAMM to semi-supervised or supervised learning and multiple chains of dependence, thus allowing the use of multimodal data; and (iv) introduction of ICAMM in diverse novel applications, both for estimation and for classification. The developed methods were validated via an extensive number of simulations that covered multiple scenarios. These tested the sensitivity of the proposed methods with respect to the following parameters: number of values to estimate; kinds of source distributions; correspondence of the data with respect to the assumptions of the model; number of classes in the mixture model; and unsupervised, semi-supervised, and supervised learning. The performance of the proposed methods was evaluated using several figures of merit, and compared with the performance of multiple classical and state-of-the-art techniques for estimation and classification. Aside from the simulations, the methods were also tested on several sets of real data from different types: data from seismic exploration studies; ground penetrating radar surveys; and biomedical data. These data correspond to the following applications: reconstruction of damaged or missing data from ground-penetrating radar surveys of historical walls; reconstruction of damaged or missing data from a seismic exploration survey; reconstruction of artifacted or missing electroencephalographic (EEG) data; diagnosis of sleep disorders; modeling of the brain response during memory tasks; and exploration of EEG data from subjects performing a battery of neuropsychological tests. The obtained results demonstrate the capability of the proposed methods to work on problems with real data. Furthermore, the proposed methods are general-purpose and can be used in many signal processing fields.
[ES] Esta tesis considera nuevas aplicaciones de las mezclas no Gaussianas dentro del marco de trabajo del procesado estadístico de señal y del reconocimiento de patrones. Las mezclas no Gaussianas fueron implementadas mediante mezclas de analizadores de componentes independientes (ICA). La hipótesis fundamental de ICA es que las señales observadas pueden expresarse como una transformación lineal de un grupo de variables ocultas, normalmente llamadas fuentes, que son estadísticamente independientes. Esta independencia permite factorizar la función de densidad de probabilidad (PDF) original M-dimensional de los datos como un producto de densidades unidimensionales, simplificando ampliamente el modelado de los datos. Los modelos de mezclas ICA (ICAMM) aportan una mayor flexibilidad al relajar el requisito de independencia de ICA, permitiendo que el modelo obtenga proyecciones locales de los datos sin comprometer su capacidad de generalización. Aquí se exploran nuevas posibilidades de ICAMM para los propósitos de estimación y clasificación de señales. La tesis realiza varias contribuciones a la investigación en mezclas no Gaussianas: (i) un método de estimación de datos faltantes por máxima verosimilitud, basado en la maximización de la PDF de los datos dado el ICAMM; (ii) un método de estimación Bayesiana de datos faltantes que minimiza el error cuadrático medio y puede obtener el intervalo de confianza de la predicción; (iii) una generalización del modelo de dependencia secuencial de ICAMM para aprendizaje supervisado o semi-supervisado y múltiples cadenas de dependencia, permitiendo así el uso de datos multimodales; y (iv) introducción de ICAMM en varias aplicaciones novedosas, tanto para estimación como para clasificación. Los métodos desarrollados fueron validados mediante un número extenso de simulaciones que cubrieron múltiples escenarios. Éstos comprobaron la sensibilidad de los métodos propuestos con respecto a los siguientes parámetros: número de valores a estimar; tipo de distribuciones de las fuentes; correspondencia de los datos con respecto a las suposiciones del modelo; número de clases en el modelo de mezclas; y aprendizaje supervisado, semi-supervisado y no supervisado. El rendimiento de los métodos propuestos fue evaluado usando varias figuras de mérito, y comparado con el rendimiento de múltiples técnicas clásicas y del estado del arte para estimación y clasificación. Además de las simulaciones, los métodos también fueron probados sobre varios grupos de datos de diferente tipo: datos de estudios de exploración sísmica; exploraciones por radar de penetración terrestre; y datos biomédicos. Estos datos corresponden a las siguientes aplicaciones: reconstrucción de datos dañados o faltantes de exploraciones de radar de penetración terrestre de muros históricos; reconstrucción de datos dañados o faltantes de un estudio de exploración sísmica; reconstrucción de datos electroencefalográficos (EEG) dañados o artefactados; diagnóstico de desórdenes del sueño; modelado de la respuesta del cerebro durante tareas de memoria; y exploración de datos EEG de sujetos durante la realización de una batería de pruebas neuropsicológicas. Los resultados obtenidos demuestran la capacidad de los métodos propuestos para trabajar en problemas con datos reales. Además, los métodos propuestos son de propósito general y pueden utilizarse en muchos campos del procesado de señal.
[CAT] Aquesta tesi considera noves aplicacions de barreges no Gaussianes dins del marc de treball del processament estadístic de senyal i del reconeixement de patrons. Les barreges no Gaussianes van ser implementades mitjançant barreges d'analitzadors de components independents (ICA). La hipòtesi fonamental d'ICA és que els senyals observats poden ser expressats com una transformació lineal d'un grup de variables ocultes, comunament anomenades fonts, que són estadísticament independents. Aquesta independència permet factoritzar la funció de densitat de probabilitat (PDF) original M-dimensional de les dades com un producte de densitats de probabilitat unidimensionals, simplificant àmpliament la modelització de les dades. Els models de barreges ICA (ICAMM) aporten una major flexibilitat en alleugerar el requeriment d'independència d'ICA, permetent així que el model obtinga projeccions locals de les dades sense comprometre la seva capacitat de generalització. Ací s'exploren noves possibilitats d'ICAMM pels propòsits d'estimació i classificació de senyals. Aquesta tesi aporta diverses contribucions a la recerca en barreges no Gaussianes: (i) un mètode d'estimació de dades faltants per màxima versemblança, basat en la maximització de la PDF de les dades donat l'ICAMM; (ii) un mètode d'estimació Bayesiana de dades faltants que minimitza l'error quadràtic mitjà i pot obtenir l'interval de confiança de la predicció; (iii) una generalització del model de dependència seqüencial d'ICAMM per entrenament supervisat o semi-supervisat i múltiples cadenes de dependència, permetent així l'ús de dades multimodals; i (iv) introducció d'ICAMM en diverses noves aplicacions, tant per a estimació com per a classificació. Els mètodes desenvolupats van ser validats mitjançant una extensa quantitat de simulacions que cobriren múltiples situacions. Aquestes van verificar la sensibilitat dels mètodes proposats amb respecte als següents paràmetres: nombre de valors per estimar; mena de distribucions de les fonts; correspondència de les dades amb respecte a les suposicions del model; nombre de classes del model de barreges; i aprenentatge supervisat, semi-supervisat i no-supervisat. El rendiment dels mètodes proposats va ser avaluat mitjançant diverses figures de mèrit, i comparat amb el rendiments de múltiples tècniques clàssiques i de l'estat de l'art per a estimació i classificació. A banda de les simulacions, els mètodes van ser verificats també sobre diversos grups de dades reals de diferents tipus: dades d'estudis d'exploració sísmica; exploracions de radars de penetració de terra; i dades biomèdiques. Aquestes dades corresponen a les següents aplicacions: reconstrucció de dades danyades o faltants d'estudis d'exploracions de radar de penetració de terra sobre murs històrics; reconstrucció de dades danyades o faltants en un estudi d'exploració sísmica; reconstrucció de dades electroencefalogràfiques (EEG) artefactuades o faltants; diagnosi de desordres de la son; modelització de la resposta del cervell durant tasques de memòria; i exploració de dades EEG de subjectes realitzant una bateria de tests neuropsicològics. Els resultats obtinguts han demostrat la capacitat dels mètodes proposats per treballar en problemes amb dades reals. A més, els mètodes proposats són de propòsit general i poden fer-se servir en molts camps del processament de senyal.
Safont Armero, G. (2015). New Insights in Prediction and Dynamic Modeling from Non-Gaussian Mixture Processing Methods [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/53913
TESIS

Orientador: Prof. Dr. Francisco J. Fraga
Dissertação (mestrado) - Universidade Federal do ABC, Programa de Pós-Graduação em Engenharia da Informação, 2014.
Most speaker verifcation systems are based on Gaussian mixture models and more recently on the so-called i-vector. These two methods are affected in mismatched testtrain conditions, which might be caused by vocal-efort variability, different speakingstyles or channel efects. In this work, we compared the impact of speech rate variation and room reverberation on both methods. We found that performance degradation due to variation on speech rate can be mitigated by adding fast speech samples into the training set, which decreased equal error rates for Gaussian mixture models and i-vector, respectively. Regarding reverberation, we investigated the achievements of both methods when three diferent reverberation compensation techniques are applied in order to overcome performance degradation. The results showed that having reverberant background models separated by diferent levels of reverberation can bene t both methods, with the i-vector providing the best performance in that scenario. Finally, the performance of two auditory-inspired features, mel-frequency cepstral coe ficients and the so-called modulation spectrum features, are compared in presence of room reverberation. For the speaker verifcation system considered in this work, modulation spectrum features are equally afected by reverberation time and have their performance degraded as the level of reverberation increases.

This thesis proposes a probabilistic level set method to be used in segmentation of tumors with heterogeneous intensities. It models the intensities of the tumor and surrounding tissue using Gaussian mixture models. Through a contour based initialization procedure samples are gathered to be used in expectation maximization of the mixture model parameters. The proposed method is compared against a threshold-based segmentation method using MRI images retrieved from The Cancer Imaging Archive. The cases are manually segmented and an automated testing procedure is used to find optimal parameters for the proposed method and then it is tested against the threshold-based method. Segmentation times, dice coefficients, and volume errors are compared. The evaluation reveals that the proposed method has a comparable mean segmentation time to the threshold-based method, and performs faster in cases where the volume error does not exceed 40%. The mean dice coefficient and volume error are also improved while achieving lower deviation.

The evaluation of retina structures has been of great interest because it could be used as a non-intrusive diagnosis in modern ophthalmology to detect many important eye diseases as well as cardiovascular disorders. A variety of retinal image analysis tools have been developed to assist ophthalmologists and eye diseases experts by reducing the time required in eye screening, optimising the costs as well as providing efficient disease treatment and management systems. A key component in these tools is the segmentation and quantification of retina structures. However, the imaging artefacts such as noise, intensity homogeneity and the overlapping tissue of retina structures can cause significant degradations to the performance of these automated image analysis tools. This thesis aims to provide robust and reliable automated retinal image analysis technique to allow for early detection of various retinal and other diseases. In particular, four innovative segmentation methods have been proposed, including two for retinal vessel network segmentation, two for optic disc segmentation and one for retina nerve fibre layers detection. First, three pre-processing operations are combined in the segmentation method to remove noise and enhance the appearance of the blood vessel in the image, and a Mixture of Gaussians is used to extract the blood vessel tree. Second, a graph cut segmentation approach is introduced, which incorporates the mechanism of vectors flux into the graph formulation to allow for the segmentation of very narrow blood vessels. Third, the optic disc segmentation is performed using two alternative methods: the Markov random field image reconstruction approach detects the optic disc by removing the blood vessels from the optic disc area, and the graph cut with compensation factor method achieves that using prior information of the blood vessels. Fourth, the boundaries of the retinal nerve fibre layer (RNFL) are detected by adapting a graph cut segmentation technique that includes a kernel-induced space and a continuous multiplier based max-flow algorithm. The strong experimental results of our retinal blood vessel segmentation methods including Mixture of Gaussian, Graph Cut achieved an average accuracy of 94:33%, 94:27% respectively. Our optic disc segmentation methods including Markov Random Field and Compensation Factor also achieved an average sensitivity of 92:85% and 85:70% respectively. These results obtained on several public datasets and compared with existing methods have shown that our proposed methods are robust and efficient in the segmenting retinal structures such the blood vessels and the optic disc.

The field of speech recognition has during the last decade left the re- search stage and found its way in to the public market. Most computers and mobile phones sold today support dictation and transcription in a number of chosen languages.  Swedish is often not one of them. In this thesis, which is executed on behalf of the Swedish Radio, an Automatic Speech Recognition model for Swedish is trained and the performance evaluated. The model is built using the open source toolkit Kaldi.  Two approaches of training the acoustic part of the model is investigated. Firstly, using Hidden Markov Model and Gaussian Mixture Models and secondly, using Hidden Markov Models and Deep Neural Networks. The later approach using deep neural networks is found to achieve a better performance in terms of Word Error Rate.
De senaste åren har olika tillämpningar inom människa-dator interaktion och främst taligenkänning hittat sig ut på den allmänna marknaden. Många system och tekniska produkter stöder idag tjänsterna att transkribera tal och diktera text. Detta gäller dock främst de större språken och sällan finns samma stöd för mindre språk som exempelvis svenskan. I detta examensprojekt har en modell för taligenkänning på svenska ut- vecklas. Det är genomfört på uppdrag av Sveriges Radio som skulle ha stor nytta av en fungerande taligenkänningsmodell på svenska. Modellen är utvecklad i ramverket Kaldi. Två tillvägagångssätt för den akustiska träningen av modellen är implementerade och prestandan för dessa två är evaluerade och jämförda. Först tränas en modell med användningen av Hidden Markov Models och Gaussian Mixture Models och slutligen en modell där Hidden Markov Models och Deep Neural Networks an- vänds, det visar sig att den senare uppnår ett bättre resultat i form av måttet Word Error Rate.

South Africa is currently struggling to deal with a significant poaching and livestock theft problem. This work is concerned with the detection and classification of ground based targets using radar micro- Doppler signatures to aid in the monitoring of borders, nature reserves and farmlands. The research starts of by investigating the state of the art of ground target classification. Different radar systems are investigated with respect to their ability to classify targets at different operating frequencies. Finally, a Gaussian Mixture Model Hidden Markov Model based (GMM-HMM) classification approach is presented and tested in an operational environment. The GMM-HMM method is compared to methods in the literature and is shown to achieve reasonable (up to 95%) classification accuracy, marginally outperforming existing ground target classification methods.
Dissertation (MEng)--University of Pretoria, 2017.
Electrical, Electronic and Computer Engineering
MEng
Unrestricted

U ovoj disertaciji je predstavljen model koji aproksimira pune kova-rijansne matrice u modelu gausovih mešavina (GMM) sa smanjenimbrojem parametara i izračunavanja koji su potrebni za izračunavanjeizglednosti. U predloženom modelu inverzne kovarijansne matrice suaproksimirane korišćenjem retke reprezentacije njihovih karakteri-stičnih vektora. Pored samog modela prikazan je i algoritam zaestimaciju parametara zasnovan na kriterijumu maksimizacijeizgeldnosti. Eksperimentalni rezultati na problemu prepoznavanjagovora su pokazali da predloženi model za isti nivo greške kao GMMsa upunim kovarijansnim, redukuje broj parametara za 45%.
This thesis proposes a model which approximates full covariance matrices inGaussian mixture models with a reduced number of parameters andcomputations required for likelihood evaluations. In the proposed modelinverse covariance (precision) matrices are approximated using sparselyrepresented eigenvectors. A maximum likelihood algorithm for parameterestimation and its practical implementation are presented. Experimentalresults on a speech recognition task show that while keeping the word errorrate close to the one obtained by GMMs with full covariance matrices, theproposed model can reduce the number of parameters by 45%.

A common problem when using background models to segment moving objects from video sequences is that objects cast shadow usually significantly differ from the background and therefore get detected as foreground. This causes several problems when extracting and labeling objects, such as object shape distortion and several objects merging together. The purpose of this thesis is to explore various possibilities to handle this problem.

Three methods for statistical background modeling are reviewed. All methods work on a per pixel basis, the first is based on approximating the median, the next on using Gaussian mixture models, and the last one is based on channel representation. It is concluded that all methods detect cast shadows as foreground.

A study of existing methods to handle cast shadows has been carried out in order to gain knowledge on the subject and get ideas. A common approach is to transform the RGB-color representation into a representation that separates color into intensity and chromatic components in order to determine whether or not newly sampled pixel-values are related to the background. The color spaces HSV, IHSL, CIELAB, YCbCr, and a color model proposed in the literature (Horprasert et al.) are discussed and compared for the purpose of shadow detection. It is concluded that Horprasert's color model is the most suitable for this purpose.

The thesis ends with a proposal of a method to combine background modeling using Gaussian mixture models with shadow detection using Horprasert's color model. It is concluded that, while not perfect, such a combination can be very helpful in segmenting objects and detecting their cast shadow.

In this thesis a clustering of the Stockholm county housing market has been performed using different clustering methods. Data has been derived and different geographical constraints have been used. DeSO areas (Demographic statistical areas), developed by SCB, have been used to divide the housing market in to smaller regions for which the derived variables have been calculated. Hierarchical clustering methods, SKATER and Gaussian mixture models have been applied. Methods using different kinds of geographical constraints have also been applied in an attempt to create more geographically contiguous clusters. The different methods are then compared with respect to performance and stability. The best performing method is the Gaussian mixture model EII, also known as the K-means algorithm. The most stable method when applied to bootstrapped samples is the ClustGeo-method.
I denna uppsats har en klustring av Stockholms läns bostadsmarknad genomförts med olika klustringsmetoder. Data har bearbetats och olika geografiska begränsningar har använts. DeSO (Demografiska Statistiska Områden), som utvecklats av SCB, har använts för att dela in bostadsmarknaden i mindre regioner för vilka områdesattribut har beräknats. Hierarkiska klustringsmetoder, SKATER och Gaussian mixture models har tillämpats. Metoder som använder olika typer av geografiska begränsningar har också tillämpats i ett försök att skapa mer geografiskt sammanhängande kluster. De olika metoderna jämförs sedan med avseende på kvalitet och stabilitet. Den bästa metoden, med avseende på kvalitet, är en Gaussian mixture model kallad EII, även känd som K-means. Den mest stabila metoden är ClustGeo-metoden.

Based on the assumption that returns in Commercial Real Estate are normally distributed, the Sharpe Ratio has been the standard risk-adjusted performance measure for the past several years. Research has questioned whether this assumption can be reasonably made. The Upside Potential Ratio as a risk-adjusted performance measure is an alternative to measure performance on a risk-adjusted basis but its values differ from the Sharpe Ratio's only in the assumption of skewed returns. We will provide reasonable evidence that CRE returns should not be fitted with a normal distribution and present the Gaussian Mixture Model as our choice of distribution to fit skewness. We will then use a GMM distribution to measure performance of CRE domestic markets via UPR. Additional insights will be presented by introducing an alternative risk-adjusted perfomance measure that we will call D-ratio. We will show how the UPR and the D-ratio can provide a tool-box that can be added to any existing investment strategy when identifying markets' past performance and timing of entrance. The intent of this thesis is not to provide a comprehensive framework for CRE investment decisions but to introduce statistical and mathematical tools that can serve any portfolio manager in augmenting any investment strategy already in place.

La diversité des phénotypes et des conditions observées au sein de l'espèce humaine est le résultat de multiples processus biologiques interdépendants. Cependant, dans le contexte de la médecine personnalisée et du traitement de maladies de plus en plus complexes, systématiques et hétérogènes, il est crucial de développer des approches qui capturent de manière exhaustive la complexité des mécanismes biologiques sous-jacents à la variabilité des profils biologiques. Cela s'étend du niveau individuel au niveau cellulaire, englobant les tissus et les organes. Une telle précision et une telle granularité sont essentielles pour que les cliniciens, les biologistes et les statisticiens comprennent les causes sous-jacentes de la diversité des réponses aux traitements cliniques et puissent prédire d'éventuels effets indésirables. Afin d'aborder de manière exhaustive la complexité hiérarchique et stratifiée des systèmes biologiques, nous avons considéré deux niveaux d'étude dans ce manuscrit. Au niveau de granularité le plus bas, désigné dans ce manuscrit sous le terme "endotype", nous examinons les processus conduisant aux variations observées ans les profils d'expression transcriptomiques entre individus. Notamment, pour tenir compte de la variabilité non expliquée observée entre patients affectés par la même maladie, nous introduisons une variable latente discrète. Pour identifier les sous-groupes non observés, dépendant de cette variable cachée, nous utilisons des modèles de mélange probabilistes, en supposant que chaque profil transcriptomique individuel est échantillonné à partir d'une distribution gaussienne multivariée, dont les paramètres ne peuvent pas être directement estimés dans la population générale. Ensuite, nous nous intéressons à un niveau de complexité supplémentaire, en passant en revue les méthodes canoniques permettant d'estimer la composition des tissus, souvent très hétérogènes, au sein d'un même individu. Plus précisément, nous discutons de diverses techniques de déconvolution conçues pour estimer les ratios de populations cellulaires, ces dernières contribuant en proportions inconnues au profil transcriptomique global mesuré.Nous présentons ensuite notre propre algorithme de déconvolution, nommé "DeCovarT", qui offre une précision améliorée de la délimitation de populations cellulaires fortement corrélées, en incorporant explicitement les réseaux de co-expression propres à chaque type cellulaire purifié
The diversity of phenotypes and conditions observed within the human species is driven by multiple intertwined biological processes. However, in the context of personalized medicine and the treatment of increasingly complex, systemic, and heterogeneous diseases, it is crucial to develop approaches that comprehensively capture the complexity of the biological mechanisms underlying the variability in biological profiles. This spans from the individual level to the cellular level, encompassing tissues and organs. Such granularity and precision are essential for clinicians, biologists, and statisticians to understand the underlying causes of the diversity in responses to clinical treatments and predict potential adverse effects. This manuscript primarily focuses on two biological entities of interest, namely transcriptome profiles and immune cell populations, for dissecting the diversity of disease outcomes and responses to treatment observed across individuals. The introductory section provides a comprehensive overview on the intertwined mechanisms controlling the activity and abundance of these inputs, and subsequently details standard physical methods for quantifying them in real-world conditions. To comprehensively address the intricate multi-layered organization of biological systems, we considered two distinct resolution scopes in this manuscript. At the lowest level of granularity, referred to in this manuscript as an "endotype" we examine variations in the overall bulk expression profiles across individuals. To account for the unexplained variability observed among patients sharing the same disease, we introduce an underlying latent discrete factor. To identify the unobserved subgroups characterized by this hidden variable, we employ a mixture model-based approach, assuming that each individual transcriptomic profile is sampled from a multivariate Gaussian distribution. Subsequently, we delve into a bigger layer of complexity, by integrating the cellular composition of heterogeneous tissues. Specifically, we discuss various deconvolution techniques designed to estimate the ratios of cellular populations, contributing in unknown proportions to the total observed bulk transcriptome. We then introduce an independent deconvolution algorithm, "DeCovarT", which demonstrates improved accuracy in delineating highly correlated cell types by explicitly incorporating the co-expression network structures of each purified cell type

In this master's thesis, hidden Markov models (HMM) are evaluated as a tool for forecasting movements in a currency cross. With an ever increasing electronic market, making way for more automated trading, or so called algorithmic trading, there is constantly a need for new trading strategies trying to find alpha, the excess return, in the market.

HMMs are based on the well-known theories of Markov chains, but where the states are assumed hidden, governing some observable output. HMMs have mainly been used for speech recognition and communication systems, but have lately also been utilized on financial time series with encouraging results. Both discrete and continuous versions of the model will be tested, as well as single- and multivariate input data.

In addition to the basic framework, two extensions are implemented in the belief that they will further improve the prediction capabilities of the HMM. The first is a Gaussian mixture model (GMM), where one for each state assign a set of single Gaussians that are weighted together to replicate the density function of the stochastic process. This opens up for modeling non-normal distributions, which is often assumed for foreign exchange data. The second is an exponentially weighted expectation maximization (EWEM) algorithm, which takes time attenuation in consideration when re-estimating the parameters of the model. This allows for keeping old trends in mind while more recent patterns at the same time are given more attention.

Empirical results shows that the HMM using continuous emission probabilities can, for some model settings, generate acceptable returns with Sharpe ratios well over one, whilst the discrete in general performs poorly. The GMM therefore seems to be an highly needed complement to the HMM for functionality. The EWEM however does not improve results as one might have expected. Our general impression is that the predictor using HMMs that we have developed and tested is too unstable to be taken in as a trading tool on foreign exchange data, with too many factors influencing the results. More research and development is called for.

Dans beaucoup de pays du monde, on observe une importante augmentation du nombre de personnes âgées vivant seules. Depuis quelques années, un nombre significatif de projets de recherche sur l’assistance aux personnes âgées ont vu le jour. La plupart de ces projets utilisent plusieurs modalités (vidéo, son, détection de chute, etc.) pour surveiller l'activité de la personne et lui permettre de communiquer naturellement avec sa maison "intelligente", et, en cas de danger, lui venir en aide au plus vite. Ce travail a été réalisé dans le cadre du projet ANR VERSO de recherche industrielle, Sweet-Home. Les objectifs du projet sont de proposer un système domotique permettant une interaction naturelle (par commande vocale et tactile) avec la maison, et procurant plus de sécurité à l'habitant par la détection des situations de détresse. Dans ce cadre, l'objectif de ce travail est de proposer des solutions pour la reconnaissance des sons de la vie courante dans un contexte réaliste. La reconnaissance du son fonctionnera en amont d'un système de Reconnaissance Automatique de la Parole. Les performances de celui-ci dépendent donc de la fiabilité de la séparation entre la parole et les autres sons. Par ailleurs, une bonne reconnaissance de certains sons, complétée par d'autres sources informations (détection de présence, détection de chute, etc.) permettrait de bien suivre les activités de la personne et de détecter ainsi les situations de danger. Dans un premier temps, nous nous sommes intéressés aux méthodes en provenance de la Reconnaissance et Vérification du Locuteur. Dans cet esprit, nous avons testé des méthodes basées sur GMM et SVM. Nous avons, en particulier, testé le noyau SVM-GSL (SVM GMM Supervector Linear Kernel) utilisé pour la classification de séquences. SVM-GSL est une combinaison de SVM et GMM et consiste à transformer une séquence de vecteurs de longueur arbitraire en un seul vecteur de très grande taille, appelé Super Vecteur, et utilisé en entrée d'un SVM. Les expérimentations ont été menées en utilisant une base de données créée localement (18 classes de sons, plus de 1000 enregistrements), puis le corpus du projet Sweet-Home, en intégrant notre système dans un système plus complet incluant la détection multi-canaux du son et la reconnaissance de la parole. Ces premières expérimentations ont toutes été réalisées en utilisant un seul type de coefficients acoustiques, les MFCC. Par la suite, nous nous sommes penchés sur l'étude d'autres familles de coefficients en vue d'en évaluer l'utilisabilité en reconnaissance des sons de l'environnement. Notre motivation fut de trouver des représentations plus simples et/ou plus efficaces que les MFCC. En utilisant 15 familles différentes de coefficients, nous avons également expérimenté deux approches pour transformer une séquence de vecteurs en un seul vecteur, à utiliser avec un SVM linéaire. Dans le première approche, on calcule un nombre fixe de coefficients statistiques qui remplaceront toute la séquence de vecteurs. La seconde approche (une des contributions de ce travail) utilise une méthode de discrétisation pour trouver, pour chaque caractéristique d'un vecteur acoustique, les meilleurs points de découpage permettant d'associer une classe donnée à un ou plusieurs intervalles de valeurs. La probabilité de la séquence est estimée par rapport à chaque intervalle. Les probabilités obtenues ainsi sont utilisées pour construire un seul vecteur qui remplacera la séquence de vecteurs acoustiques. Les résultats obtenus montrent que certaines familles de coefficients sont effectivement plus adaptées pour reconnaître certaines classes de sons. En effet, pour la plupart des classes, les meilleurs taux de reconnaissance ont été observés avec une ou plusieurs familles de coefficients différentes des MFCC. Certaines familles sont, de surcroît, moins complexes et comptent une seule caractéristique par fenêtre d'analyse contre 16 caractéristiques pour les MFCC
In many countries around the world, the number of elderly people living alone has been increasing. In the last few years, a significant number of research projects on elderly people monitoring have been launched. Most of them make use of several modalities such as video streams, sound, fall detection and so on, in order to monitor the activities of an elderly person, to supply them with a natural way to communicate with their “smart-home”, and to render assistance in case of an emergency. This work is part of the Industrial Research ANR VERSO project, Sweet-Home. The goals of the project are to propose a domotic system that enables a natural interaction (using touch and voice command) between an elderly person and their house and to provide them a higher safety level through the detection of distress situations. Thus, the goal of this work is to come up with solutions for sound recognition of daily life in a realistic context. Sound recognition will run prior to an Automatic Speech Recognition system. Therefore, the speech recognition’s performances rely on the reliability of the speech/non-speech separation. Furthermore, a good recognition of a few kinds of sounds, complemented by other sources of information (presence detection, fall detection, etc.) could allow for a better monitoring of the person's activities that leads to a better detection of dangerous situations. We first had been interested in methods from the Speaker Recognition and Verification field. As part of this, we have experimented methods based on GMM and SVM. We had particularly tested a Sequence Discriminant SVM kernel called SVM-GSL (SVM GMM Super Vector Linear Kernel). SVM-GSL is a combination of GMM and SVM whose basic idea is to map a sequence of vectors of an arbitrary length into one high dimensional vector called a Super Vector and used as an input of an SVM. Experiments had been carried out using a locally created sound database (containing 18 sound classes for over 1000 records), then using the Sweet-Home project's corpus. Our daily sounds recognition system was integrated into a more complete system that also performs a multi-channel sound detection and speech recognition. These first experiments had all been performed using one kind of acoustical coefficients, MFCC coefficients. Thereafter, we focused on the study of other families of acoustical coefficients. The aim of this study was to assess the usability of other acoustical coefficients for environmental sounds recognition. Our motivation was to find a few representations that are simpler and/or more effective than the MFCC coefficients. Using 15 different acoustical coefficients families, we have also experimented two approaches to map a sequence of vectors into one vector, usable with a linear SVM. The first approach consists of computing a set of a fixed number of statistical coefficients and use them instead of the whole sequence. The second one, which is one of the novel contributions of this work, makes use of a discretization method to find, for each feature within an acoustical vector, the best cut points that associates a given class with one or many intervals of values. The likelihood of the sequence is estimated for each interval. The obtained likelihood values are used to build one single vector that replaces the sequence of acoustical vectors. The obtained results show that a few families of coefficients are actually more appropriate to the recognition of some sound classes. For most sound classes, we noticed that the best recognition performances were obtained with one or many families other than MFCC. Moreover, a number of these families are less complex than MFCC. They are actually a one-feature per frame acoustical families, whereas MFCC coefficients contain 16 features per frame

As intelligent access solutions begin to dominate the world, the statistical learning methods to answer for the behavior of these needs attention, as there is no clear answer to how an algorithm could learn and predict exactly how people move. This project aims at investigating if, with the help of unsupervised learning methods, it is possible to distinguish anomalies from normal events in an access system, and if the most probable choice of cylinder to be unlocked by a user can be calculated.Given to do this is a data set of the previous events in an access system, together with the access configurations - and the algorithms that were used consisted of an auto-encoder and a probabilistic generative model.The auto-encoder managed to, with success, encode the high-dimensional data set into one of significantly lower dimension, and the probabilistic generative model, which was chosen to be a Gaussian mixture model, identified clusters in the data and assigned a measure of unexpectedness to the events.Lastly, the probabilistic generative model was used to compute the conditional probability of which the user, given all the details except which cylinder that was chosen during an event, would choose a certain cylinder. The result of this was a correct guess in 65.7 % of the cases, which can be seen as a satisfactory number for something originating from an unsupervised problem.
Allt eftersom att intelligenta åtkomstlösningar tar över i samhället, så är det nödvändigt att ägna de statistiska inlärnings-metoderna bakom dessa tillräckligt med uppmärksamhet, eftersom det inte finns något självklart svar på hur en algoritm ska kunna lära sig och förutspå människors exakta rörelsemönster.Det här projektet har som mål att, med hjälp av oövervakad inlärning, undersöka huruvida det är möjligt att urskilja anomalier från normala iakttagelser, och om den låscylinder med högst sannolikhet att en användare väljer att försöka låsa upp går att beräknda.Givet för att genomföra detta projekt är en datamängd där händelser från ett åtkomstsystem finns, tillsammans med tillhörande åtkomstkonfig-urationer. Algoritmerna som användes i projektet har bestått av en auto-encoder och en probabilistisk generativ modell.Auto-encodern lyckades, med tillfredsställande resultat, att koda det hög-dimensionella datat till ett annat med betydligt lägre dimension, och den probabilistiska generativa modellen, som valdes till en Gaussisk mixtur-modell, lyckades identifiera kluster i datat och med att tilldela varje observation ett mått på dess otrolighet.Till slut så användes den probabilistiska generativa modellen för att beräkna en villkorad sannolikhet, för vilken användaren, given alla attribut för en händelse utom just vilken låscylinder som denna försökte öppna, skulle välja.Resultatet av dessa var en korrekt gissning i 65,7 % av fallen, vilket kan ses som en tillfredställande siffra för något som härrör från ett oövervakat problem.

Apesar da ampla diversidade de línguas indígenas no Brasil, poucas pesquisas estudam estas línguas e suas relações. Inúmeros esforços têm sido dedicados a procurar similaridades entre as palavras das línguas indígenas e classificá-las em famílias de línguas. Seguindo a classificação mais aceita das línguas indígenas do Brasil, esta pesquisa propõe comparar palavras de 10 línguas indígenas brasileiras. Para isso, considera-se que estas palavras são sinais de fala e estima-se a função de distribuição de probabilidade (PDF) de cada palavra, usando um modelo de mistura de gaussianas (GMM). A PDF foi considerada um modelo para representar as palavras. Os modelos foram comparados utilizando medidas de distância para construir estruturas hierárquicas que evidenciaram possíveis relações entre as palavras. Seguindo esta linha, a hipótese levantada nesta pesquisa é que as PDFs baseadas em GMM conseguem caracterizar as palavras das línguas indígenas, permitindo o emprego de medidas de distância entre elas para estabelecer relações entre as palavras, de forma que tais relações confirmem algumas das classificações. Os parâmetros do GMM foram calculados utilizando o algoritmo Maximização da Expectância (em inglês, Expectation Maximization (EM)). A divergência Kullback Leibler (KL) foi empregada para medir semelhança entre as PDFs. Esta divergência serve de base para estabelecer as estruturas hierárquicas que ilustram as relações entre os modelos. A estimativa da PDF, baseada em GMM foi testada com o auxílio de sinais simulados, sendo possível confirmar que os parâmetros obtidos são próximos dos originais. Foram implementadas várias medidas de distância para avaliar se a semelhança entre os modelos estavam determinadas pelos modelos e não pelas medidas adotadas neste estudo. Os resultados de todas as medidas foram similares, somente foi observada alguma diferença nos agrupamentos realizados pela distância C2, por isso foi proposta como complemento da divergência KL. Estes resultados sugerem que as relações entre os modelos dependem das suas características, não das métricas de distância selecionadas no estudo e que as PDFs baseadas em GMM, conseguem fazer uma caracterização adequada das palavras. Em geral, foram observados agrupamentos entre palavras que pertenciam a línguas de um mesmo tronco linguístico, assim como se observou uma tendência a incluir línguas isoladas nos agrupamentos dos troncos linguísticos. Palavras que pertenciam a determinada língua apresentaram um comportamento padrão, sendo identificadas por esse tipo de comportamento. Embora os resultados para as palavras das línguas indígenas sejam inconclusivos, considera-se que o estudo foi útil para aumentar o conhecimento destas 10 línguas estudadas, propondo novas linhas de pesquisas dedicadas à análise destas palavras.
Although there exists a large diversity of indigenous languages in Brazil, there are few researches on these languages and their relationships. Numerous efforts have been dedicated to search for similarities among words of indigenous languages to classify them into families. Following the most accepted classification of Brazilian indigenous languages, this research proposes to compare words of 10 Brazilian indigenous languages. The words of the indigenous languages are considered speech signals and the Probability Distribution Function (PDF) of each word was estimated using the Gaussian Mixture Models (GMM). This estimation was considered a model to represent each word. The models were compared using distance measures to construct hierarchical structures that illustrate possible relationships among words. The hypothesis in this research is that the estimation of the PDF, based on GMM can characterize the words of indigenous languages, allowing the use of distance measures between the PDFs to establish relationships among the words and confirm some of the classifications. The Expectation Maximization algorithm (EM) was implemented to estimate the parameters that describe the GMM. The Kullback Leibler (KL) divergence was used to measure similarities between two PDFs. This divergence is the basis to establish the hierarchical structures that show the relationships among the models. The PDF estimation, based on GMM was tested using simulated signals, allowing confirming the useful approximation of the original parameters. Several distance measures were implemented to prove that the similarities among the models depended on the model of each word, and not on the distance measure adopted in this study. The results of all measures were similar, however, as the clustering results of the C2 distances showed some differences from the other clusters, C2 distance was proposed to complement the KL divergence. The results suggest that the relationships between models depend on their characteristics, and not on the distance measures selected in this study, and the PDFs based on GMM can properly characterize the words. In general, relations among languages that belong to the same linguistic branch were illustrated, showing a tendency to include isolated languages in groups of languages that belong to the same linguistic branches. As the GMM of some language families presents a standard behavior, it allows identifying each family. Although the results of the words of indigenous languages are inconclusive, this study is considered very useful to increase the knowledge of these types of languages and to propose new research lines directed to analyze this type of signals.

Ca2+ and calmodulin play important roles in many physiologically crucial pathways. The conformational landscape of calmodulin is intriguing. Conformational changes allow for binding target-proteins, while binding Ca2+ yields population shifts within the landscape. Thus, target-proteins become Ca2+-sensitive upon calmodulin binding. Calmodulin regulates more than 300 target-proteins, and mutations are linked to lethal disorders. The mechanisms underlying Ca2+ and target-protein binding are complex and pose interesting questions. Such questions are typically addressed with experiments which fail to provide simultaneous molecular and dynamics insights. In this thesis, questions on binding mechanisms are probed with molecular dynamics simulations together with tailored unsupervised learning and data analysis. In Paper 1, a free energy landscape estimator based on Gaussian mixture models with cross-validation was developed and used to evaluate the efficiency of regular molecular dynamics compared to temperature-enhanced molecular dynamics. This comparison revealed interesting properties of the free energy landscapes, highlighting different behaviors of the Ca2+-bound and unbound calmodulin conformational ensembles. In Paper 2, spectral clustering was used to shed light on Ca2+ and target protein binding. With these tools, it was possible to characterize differences in target-protein binding depending on Ca2+-state as well as N-terminal or C-terminal lobe binding. This work invites data-driven analysis into the field of biomolecule molecular dynamics, provides further insight into calmodulin’s Ca2+ and targetprotein binding, and serves as a stepping-stone towards a complete understanding of calmodulin’s Ca2+-dependent conformational ensembles.

QC 20180912

Trucks are designed, configured and marketed for various working environments. There lies a concern whether trucks are used as intended by the manufacturer, as usage may impact the longevity, efficiency and productivity of the trucks. In this thesis we propose a framework divided into two separate parts, that aims to extract costumers’ driving behaviours from Logged Vehicle Data (LVD) in order to a): evaluate whether they align with so-called Global Transport Application (GTA) parameters and b): evaluate the usage in terms of performance. Gaussian mixture model (GMM) is employed to cluster and classify various driving behaviors. Association rule mining was applied on the categorized clusters to validate that the usage follow GTA configuration. Furthermore, Correlation Coefficient (CC) was used to find linear relationships between usage and performance in terms of Fuel Consumption (FC). It is found that the vast majority of the trucks seemingly follow GTA parameters, thus used as marketed. Likewise, the fuel economy was found to be linearly dependent with drivers’ various performances. The LVD lacks detail, such as Global Positioning System (GPS) information, needed to capture the usage in such a way that more definitive conclusions can be drawn.

This thesis was later conducted as a scientific paper and was submit- ted to the conference of ICIMP, 2020. The publication was accepted the 23th of September (2019), and will be presented in January, 2020.

Automatic recognition of people is an active field of research with important forensic and security applications. In these applications, it is not always possible for the subject to be in close proximity to the system. Voice represents a human behavioural trait which can be used to recognise people in such situations. Automatic Speaker Verification (ASV) is the process of verifying a persons identity through the analysis of their speech and enables recognition of a subject at a distance over a telephone channel { wired or wireless. A significant amount of research has focussed on the application of Gaussian mixture model (GMM) techniques to speaker verification systems providing state-of-the-art performance. GMM's are a type of generative classifier trained to model the probability distribution of the features used to represent a speaker. Recently introduced to the field of ASV research is the support vector machine (SVM). An SVM is a discriminative classifier requiring examples from both positive and negative classes to train a speaker model. The SVM is based on margin maximisation whereby a hyperplane attempts to separate classes in a high dimensional space. SVMs applied to the task of speaker verification have shown high potential, particularly when used to complement current GMM-based techniques in hybrid systems. This work aims to improve the performance of ASV systems using novel and innovative SVM-based techniques. Research was divided into three main themes: session variability compensation for SVMs; unsupervised model adaptation; and impostor dataset selection. The first theme investigated the differences between the GMM and SVM domains for the modelling of session variability | an aspect crucial for robust speaker verification. Techniques developed to improve the robustness of GMMbased classification were shown to bring about similar benefits to discriminative SVM classification through their integration in the hybrid GMM mean supervector SVM classifier. Further, the domains for the modelling of session variation were contrasted to find a number of common factors, however, the SVM-domain consistently provided marginally better session variation compensation. Minimal complementary information was found between the techniques due to the similarities in how they achieved their objectives. The second theme saw the proposal of a novel model for the purpose of session variation compensation in ASV systems. Continuous progressive model adaptation attempts to improve speaker models by retraining them after exploiting all encountered test utterances during normal use of the system. The introduction of the weight-based factor analysis model provided significant performance improvements of over 60% in an unsupervised scenario. SVM-based classification was then integrated into the progressive system providing further benefits in performance over the GMM counterpart. Analysis demonstrated that SVMs also hold several beneficial characteristics to the task of unsupervised model adaptation prompting further research in the area. In pursuing the final theme, an innovative background dataset selection technique was developed. This technique selects the most appropriate subset of examples from a large and diverse set of candidate impostor observations for use as the SVM background by exploiting the SVM training process. This selection was performed on a per-observation basis so as to overcome the shortcoming of the traditional heuristic-based approach to dataset selection. Results demonstrate the approach to provide performance improvements over both the use of the complete candidate dataset and the best heuristically-selected dataset whilst being only a fraction of the size. The refined dataset was also shown to generalise well to unseen corpora and be highly applicable to the selection of impostor cohorts required in alternate techniques for speaker verification.

This century is surely the century of data (Donoho, 2000). Data analysis has been an emerging activity over the last few decades. High dimensional data is in particular more and more pervasive with the advance of massive data collection system, such as microarrays, satellite imagery, and financial data. However, analysis of high dimensional data is of challenge with the so called curse of dimensionality (Bellman 1961). This research dissertation presents several methodologies in the application of high dimensional data analysis. The first part discusses a joint analysis of multiple microarray gene expressions. Microarray analysis dates back to Golub et al. (1999). It draws much attention after that. One common goal of microarray analysis is to determine which genes are differentially expressed. These genes behave significantly differently between groups of individuals. However, in microarray analysis, there are thousands of genes but few arrays (samples, individuals) and thus relatively low reproducibility remains. It is natural to consider joint analyses that could combine microarrays from different experiments effectively in order to achieve improved accuracy. In particular, we present a model-based approach for better identification of differentially expressed genes by incorporating data from different studies. The model can accommodate in a seamless fashion a wide range of studies including those performed at different platforms, and/or under different but overlapping biological conditions. Model-based inferences can be done in an empirical Bayes fashion. Because of the information sharing among studies, the joint analysis dramatically improves inferences based on individual analysis. Simulation studies and real data examples are presented to demonstrate the effectiveness of the proposed approach under a variety of complications that often arise in practice. The second part is about covariance matrix estimation in high dimensional data. First, we propose a penalised likelihood estimator for high dimensional t-distribution. The student t-distribution is of increasing interest in mathematical finance, education and many other applications. However, the application in t-distribution is limited by the difficulty in the parameter estimation of the covariance matrix for high dimensional data. We show that by imposing LASSO penalty on the Cholesky factors of the covariance matrix, EM algorithm can efficiently compute the estimator and it performs much better than other popular estimators. Secondly, we propose an estimator for high dimensional Gaussian mixture models. Finite Gaussian mixture models are widely used in statistics thanks to its great flexibility. However, parameter estimation for Gaussian mixture models with high dimensionality can be rather challenging because of the huge number of parameters that need to be estimated. For such purposes, we propose a penalized likelihood estimator to specifically address such difficulties. The LASSO penalty we impose on the inverse covariance matrices encourages sparsity on its entries and therefore helps reducing the dimensionality of the problem. We show that the proposed estimator can be efficiently computed via an Expectation-Maximization algorithm. To illustrate the practical merits of the proposed method, we consider its application in model-based clustering and mixture discriminant analysis. Numerical experiments with both simulated and real data show that the new method is a valuable tool in handling high dimensional data. Finally, we present structured estimators for high dimensional Gaussian mixture models. The graphical representation of every cluster in Gaussian mixture models may have the same or similar structure, which is an important feature in many applications, such as image processing, speech recognition and gene network analysis. Failure to consider the sharing structure would deteriorate the estimation accuracy. To address such issues, we propose two structured estimators, hierarchical Lasso estimator and group Lasso estimator. An EM algorithm can be applied to conveniently solve the estimation problem. We show that when clusters share similar structures, the proposed estimator perform much better than the separate Lasso estimator.

Uncertainty in rainfall forecasts affects the level of quality and assurance for decisions made to manage water resource-based systems. However, eliminating uncertainty in a complete manner could be difficult, decision-makers thus are challenged to make decisions in the light of uncertainty. This study provides statistical models as an approach to cope with uncertainty, including: a) a statistical method relying on a Gaussian mixture (GM) model to assist in better characterize uncertainty in climate model projections and evaluate their performance in matching observations; b) a stochastic model that incorporates the El Niño–Southern Oscillation (ENSO) cycle to narrow uncertainty in seasonal rainfall forecasts; and c) a statistical approach to determine to what extent drought events forecasted using ENSO information could be utilized in the water resources decision-making process. This study also investigates the relationship between calibration and lead time on the ability to narrow the interannual uncertainty of forecasts and the associated usefulness for decision making. These objectives are demonstrated for the northwest region of Costa Rica as a case study of a developing country in Central America. This region of Costa Rica is under an increasing risk of future water shortages due to climate change, increased demand, and high variability in the bimodal cycle of seasonal rainfall. First, the GM model is shown to be a suitable approach to compare and characterize long-term projections of climate models. The GM representation of seasonal cycles is then employed to construct detailed comparison tests for climate models with respect to observed rainfall data. Three verification metrics demonstrate that an acceptable degree of predictability can be obtained by incorporating ENSO information in reducing error and interannual variability in the forecast of seasonal rainfall. The predictability of multicategory rainfall forecasts in the late portion of the wet season surpasses that in the early portion of the wet season. Later, the value of drought forecast information for coping with uncertainty in making decisions on water management is determined by quantifying the reduction in expected losses relative to a perfect forecast. Both the discrimination ability and the relative economic value of drought-event forecasts are improved by the proposed forecast method, especially after calibration. Positive relative economic value is found only for a range of scenarios of the cost-loss ratio, which indicates that the proposed forecast could be used for specific cases. Otherwise, taking actions (no-actions) is preferred as the cost-loss ratio approaches zero (one). Overall, the approach of incorporating ENSO information into seasonal rainfall forecasts would provide useful value to the decision-making process - in particular at lead times of one year ahead.

Foreground segmentation is a common first step in tracking and surveillance applications.  The purpose of foreground segmentation is to provide later stages of image processing with an indication of where interesting data can be found.  This thesis is an investigation of how foreground segmentation can be performed in two contexts: as a pre-step to trajectory tracking and as a pre-step in indoor surveillance applications.

Three methods are selected and detailed: a single Gaussian method, a Gaussian mixture model method, and a codebook method.  Experiments are then performed on typical input video using the methods.  It is concluded that the Gaussian mixture model produces the output which yields the best trajectories when used as input to the trajectory tracker.  An extension is proposed to the Gaussian mixture model which reduces shadow, improving the performance of foreground segmentation in the surveillance context.

Atualmente diversos sistemas sensoriais fornecem dados em fluxos e essas observações medidas são frequentemente de alta dimensionalidade, ou seja, o número de variáveis medidas é grande, e as observações chegam em sequência. Este é, em particular, o caso de sistemas de visão em robôs. Aprendizagem supervisionada e não-supervisionada com esses fluxos de dados é um desafio, porque o algoritmo deve ser capaz de aprender com cada observação e depois descartá-la antes de considerar a próxima, mas diversos métodos requerem todo o conjunto de dados a fim de estimar seus parâmetros e, portanto, não são adequados para aprendizagem em tempo real. Além disso, muitas abordagens sofrem com a denominada maldição da dimensionalidade (BELLMAN, 1961) e não conseguem lidar com dados de entrada de alta dimensionalidade. Para superar os problemas descritos anteriormente, este trabalho propõe um novo modelo de rede neural probabilístico e incremental, denominado Local Projection Incremental Gaussian Mixture Network (LP-IGMN), que é capaz de realizar aprendizagem perpétua com dados de alta dimensionalidade, ou seja, ele pode aprender continuamente considerando a estabilidade dos parâmetros do modelo atual e automaticamente ajustar sua topologia levando em conta a fronteira do subespaço encontrado por cada neurônio oculto. O método proposto pode encontrar o subespaço intrísico onde os dados se localizam, o qual é denominado de subespaço principal. Ortogonal ao subespaço principal, existem as dimensões que são ruidosas ou que carregam pouca informação, ou seja, com pouca variância, e elas são descritas por um único parâmetro estimado. Portanto, LP-IGMN é robusta a diferentes fontes de dados e pode lidar com grande número de variáveis ruidosas e/ou irrelevantes nos dados medidos. Para avaliar a LP-IGMN nós realizamos diversos experimentos usando conjunto de dados simulados e reais. Demonstramos ainda diversas aplicações do nosso método em tarefas de reconhecimento de imagens. Os resultados mostraram que o desempenho da LP-IGMN é competitivo, e geralmente superior, com outras abordagens do estado da arte, e que ela pode ser utilizada com sucesso em aplicações que requerem aprendizagem perpétua em espaços de alta dimensionalidade.
Nowadays several sensory systems provide data in ows and these measured observations are frequently high-dimensional, i.e., the number of measured variables is large, and the observations are arriving in a sequence. This is in particular the case of robot vision systems. Unsupervised and supervised learning with such data streams is challenging, because the algorithm should be capable of learning from each observation and then discard it before considering the next one, but several methods require the whole dataset in order to estimate their parameters and, therefore, are not suitable for online learning. Furthermore, many approaches su er with the so called curse of dimensionality (BELLMAN, 1961) and can not handle high-dimensional input data. To overcome the problems described above, this work proposes a new probabilistic and incremental neural network model, called Local Projection Incremental Gaussian Mixture Network (LP-IGMN), which is capable to perform life-long learning with high-dimensional data, i.e., it can continuously learn considering the stability of the current model's parameters and automatically adjust its topology taking into account the subspace's boundary found by each hidden neuron. The proposed method can nd the intrinsic subspace where the data lie, which is called the principal subspace. Orthogonal to the principal subspace, there are the dimensions that are noisy or carry little information, i.e., with small variance, and they are described by a single estimated parameter. Therefore, LP-IGMN is robust to di erent sources of data and can deal with large number of noise and/or irrelevant variables in the measured data. To evaluate LP-IGMN we conducted several experiments using simulated and real datasets. We also demonstrated several applications of our method in image recognition tasks. The results have shown that the LP-IGMN performance is competitive, and usually superior, with other stateof- the-art approaches, and it can be successfully used in applications that require life-long learning in high-dimensional spaces.

Le problème de la propagation non linéaire d’incertitude est crucial en astrodynamique, car tous les systèmes d’intérêt pratique, allant de la navigation à la détermination d’orbite et au suivi de cibles, impliquent des non-linéarités dans leurs modèles dynamiques et de mesure. Un sujet d’intérêt est la propagation précise d’incertitude à travers la dynamique orbitale non linéaire, une exigence fondamentale dans plusieurs applications telles que la surveillance de l’espace, la gestion du trafic spatial et la fin de vie des satellites. Étant donnée une représentation dimensionnelle finie de la fonction de densité de probabilité (pdf) de l’état initial, l’objectif est d’obtenir une représentation similaire de cette pdf à tout moment futur. Ce problème a été historiquement abordé avec des méthodes linéarisées ou des simulations de Monte Carlo (MC), toutes deux inadaptées pour satisfaire la demande d’un nombre croissant d’applications. Les méthodes linéarisées sont très performantes, mais ne peuvent pas gérer de fortes non-linéarités ou de longues fenêtres de propagation en raison de la validité locale de la linéarisation. En revanche, les méthodes MC peuvent gérer tout type de non-linéarité, mais sont trop coûteuses en termes de calcul pour toute tâche nécessitant la propagation de plusieurs pdf. Au lieu de cela, cette thèse exploite des méthodes multi-fidélité et des techniques d’algèbre différentielle (DA) pour développer des méthodes efficaces pour la propagation précise des incertitudes à travers des systèmes dynamiques non linéaires. La première méthode, appelée low-order automatic domain splitting (LOADS), représente l’incertitude avec un ensemble de polynômes de Taylor du deuxième ordre et exploite une mesure de non-linéarité basée sur la DA pour ajuster leur nombre en fonction de la dynamique locale et de la précision requise. Un modèle adaptatif de mélange Gaussien (GMM) est ensuite développé en associant chaque polynôme à un noyau pondéré pour obtenir une représentation analytique de la pdf d’état. En outre, une méthode multi-fidélité est proposée pour réduire le coût computationnel des algorithmes précédents tout en conservant une précision similaire. La méthode GMM est dans ce cas exécutée sur un modèle dynamique à faible fidélité, et seules les moyennes des noyaux sont propagées ponctuellement dans une dynamique à haute fidélité pour corriger la pdf à faible fidélité. Si les méthodes précédentes traitent de la propagation d’une incertitude initiale dans un modèle dynamique déterministe, les effets des forces mal ou non modélisées sont enfin pris en compte pour améliorer le réalisme des statistiques propagées. Dans ce cas, la méthode multi-fidélité est d’abord utilisée pour propager l’incertitude initiale dans un modèle dynamique déterministe de faible fidélité. Les propagations ponctuelles sont ensuite remplacées par une propagation polynomiale des moments de la pdf dans un système dynamique stochastique. Ces moments modélisent les effets des accélérations stochastiques sur les moyennes des noyaux, et couplés à la méthode GMM, ils fournissent une description de la pdf qui tient compte de l’incertitude initiale et des effets des forces négligées. Les méthodes proposées sont appliquées au problème de la propagation d’incertitude en orbite, et leurs performances sont évaluées dans différents régimes orbitaux. Les résultats démontrent leur efficacité pour une propagation précise de l’incertitude initiale et des effets du bruit du processus à une fraction du coût de calcul des simulations MC. La méthode LOADS est ensuite utilisée pour résoudre le problème de la détermination initiale d’orbite en exploitant les informations sur l’incertitude des mesures, et pour développer une méthode de prétraitement des données qui améliore la robustesse des algorithmes de détermination d’orbite. Ces outils sont enfin validés sur des observations réelles d’un objet en orbite de transfert géostationnaire
The problem of nonlinear uncertainty propagation (UP) is crucial in astrodynamics since all systems of practical interest, ranging from navigation to orbit determination (OD) and target tracking, involve nonlinearities in their dynamics and measurement models. One topic of interest is the accurate propagation of uncertainty through the nonlinear orbital dynamics, a fundamental requirement in several applications such as space surveillance and tracking (SST), space traffic management (STM), and end-of-life (EOL) disposal. Given a finite-dimensional representation of the probability density function (pdf) of the initial state, the main goal is to obtain a similar representation of the state pdf at any future time. This problem has been historically tackled with either linearized methods or Monte Carlo (MC) simulations, both of which are unsuitable to satisfy the demand of a rapidly growing number of applications. Linearized methods are light on computational resources, but cannot handle strong nonlinearities or long propagation windows due to the local validity of the linearization. In contrast, MC methods can handle any kind of nonlinearity, but are too computationally expensive for any task that requires the propagation of several pdfs. Instead, this thesis leverages multifidelity methods and differential algebra (DA) techniques to develop computationally efficient methods for the accurate propagation of uncertainties through nonlinear dynamical systems. The first method, named low-order automatic domain splitting (LOADS), represents the uncertainty with a set of second-order Taylor polynomials and leverages a DA-based measure of nonlinearity to adjust their number based on the local dynamics and the required accuracy. An adaptive Gaussian mixture model (GMM) method is then developed by associating each polynomial to a weighted Gaussian kernel, thus obtaining an analytical representation of the state pdf. Going further, a multifidelity method is proposed to reduce the computational cost of the former algorithms while retaining a similar accuracy. The adaptive GMM method is in this case run on a low-fidelity dynamical model, and only the expected values of the kernels are propagated point-wise in high-fidelity dynamics to compute a posteriori correction of the low-fidelity state pdf. If the former methods deal with the propagation of an initial uncertainty through a deterministic dynamical model, the effects of mismodeled or unmodeled forces are finally considered to further enhance the realism of the propagated statistics. In this case, the multifidelity GMM method is used at first to propagate the initial uncertainty through a low-fidelity, deterministic dynamical model. The point-wise propagations are then replaced with a DA-based algorithm to efficiently propagate a polynomial representation of the moments of the pdf in a stochastic dynamical system. These moments model the effects of stochastic accelerations on the deterministic kernels’ means, and coupled with the former GMM provide a description of the propagated state pdf that accounts for both the uncertainty in the initial state and the effects of neglected forces. The proposed methods are applied to the problem of orbit UP, and their performance is assessed in different orbital regimes. The results demonstrate the effectiveness of these methods in accurately propagating the initial uncertainty and the effects of process noise at a fraction of the computational cost of high-fidelity MC simulations. The LOADS method is then employed to solve the initial orbit determination (IOD) problem by exploiting the information on measurement uncertainty and to develop a preprocessing scheme aimed at improving the robustness of batch OD algorithms. These tools are finally validated on a set of real observations for an object in geostationary transfer orbit (GTO)

Model and variable selection have attracted considerable attention in areas of application where datasets usually contain thousands of variables. Variable selection is a critical step to reduce the dimension of high dimensional data by eliminating irrelevant variables. The general objective of variable selection is not only to obtain a set of cost-effective predictors selected but also to improve prediction and prediction variance. We have made several contributions to this issue through a range of advanced topics: providing a graphical view of Bayesian Variable Selection (BVS), recovering sparsity in multivariate nonparametric models and proposing a testing procedure for evaluating nonlinear interaction effect in a semiparametric model. To address the first topic, we propose a new Bayesian variable selection approach via the graphical model and the Ising model, which we refer to the ``Bayesian Ising Graphical Model'' (BIGM). There are several advantages of our BIGM: it is easy to (1) employ the single-site updating and cluster updating algorithm, both of which are suitable for problems with small sample sizes and a larger number of variables, (2) extend this approach to nonparametric regression models, and (3) incorporate graphical prior information. In the second topic, we propose a Nonnegative Garrote on a Kernel machine (NGK) to recover sparsity of input variables in smoothing functions. We model the smoothing function by a least squares kernel machine and construct a nonnegative garrote on the kernel model as the function of the similarity matrix. An efficient coordinate descent/backfitting algorithm is developed. The third topic involves a specific genetic pathway dataset in which the pathways interact with the environmental variables. We propose a semiparametric method to model the pathway-environment interaction. We then employ a restricted likelihood ratio test and a score test to evaluate the main pathway effect and the pathway-environment interaction.
Ph. D.

Cette thèse étudie les approches de classification automatique basées sur les modèles de mélange gaussiens et les critères de choix de modèles pour la classification automatique de données discrétisées. Quatorze algorithmes binned-EM et quatorze algorithmes bin-EM-CEM sont développés pour quatorze modèles de mélange gaussiens parcimonieux. Ces nouveaux algorithmes combinent les avantages des données discrétisées en termes de réduction du temps d’exécution et les avantages des modèles de mélange gaussiens parcimonieux en termes de simplification de l'estimation des paramètres. Les complexités des algorithmes binned-EM et bin-EM-CEM sont calculées et comparées aux complexités des algorithmes EM et CEM respectivement. Afin de choisir le bon modèle qui s'adapte bien aux données et qui satisfait les exigences de précision en classification avec un temps de calcul raisonnable, les critères AIC, BIC, ICL, NEC et AWE sont étendus à la classification automatique de données discrétisées lorsque l'on utilise les algorithmes binned-EM et bin-EM-CEM proposés. Les avantages des différentes méthodes proposées sont illustrés par des études expérimentales
This thesis studies the Gaussian mixture model-based clustering approaches and the criteria of model selection for binned data clustering. Fourteen binned-EM algorithms and fourteen bin-EM-CEM algorithms are developed for fourteen parsimonious Gaussian mixture models. These new algorithms combine the advantages in computation time reduction of binning data and the advantages in parameters estimation simplification of parsimonious Gaussian mixture models. The complexities of the binned-EM and the bin-EM-CEM algorithms are calculated and compared to the complexities of the EM and the CEM algorithms respectively. In order to select the right model which fits well the data and satisfies the clustering precision requirements with a reasonable computation time, AIC, BIC, ICL, NEC, and AWE criteria, are extended to binned data clustering when the proposed binned-EM and bin-EM-CEM algorithms are used. The advantages of the different proposed methods are illustrated through experimental studies

Of the many existing eukaryotic gene finding software programs, none are able to guarantee accurate identification of genomic protein coding regions and other biological signals central to pathway from DNA to the protein. Eukaryotic gene finding is difficult mainly due to noncontiguous and non-continuous nature of genes. Existing approaches are heavily dependent on the compositional statistics of the sequences they learn from and are not equally suitable for all types of sequences. This thesis firstly develops efficient digital signal processing-based methods for the identification of genomic protein coding regions, and then combines the optimum signal processing-based non-data-driven technique with an existing data-driven statistical method in a novel system demonstrating improved identification of acceptor splice sites. Most existing well-known DNA symbolic-to-numeric representations map the DNA information into three or four numerical sequences, potentially increasing the computational requirement of the sequence analyzer. Proposed mapping schemes, to be used for signal processing-based gene and exon prediction, incorporate DNA structural properties in the representation, in addition to reducing complexity in subsequent processing. A detailed comparison of all DNA representations, in terms of computational complexity and relative accuracy for the gene and exon prediction problem, reveals the newly proposed ?paired numeric? to be the best DNA representation. Existing signal processing-based techniques rely mostly on the period-3 behaviour of exons to obtain one dimensional gene and exon prediction features, and are not well equipped to capture the complementary properties of exonic / intronic regions and deal with the background noise in detection of exons at their nucleotide levels. These issues have been addressed in this thesis, by proposing six one-dimensional and three multi-dimensional signal processing-based gene and exon prediction features. All one-dimensional and multi-dimensional features have been evaluated using standard datasets such as Burset/Guigo1996, HMR195, and the GENSCAN test set. This is the first time that different gene and exon prediction features have been compared using substantial databases and using nucleotide-level metrics. Furthermore, the first investigation of the suitability of different window sizes for period-3 exon detection is performed. Finally, the optimum signal processing-based gene and exon prediction scheme from our evaluations is combined with a data-driven statistical technique for the recognition of acceptor splice sites. The proposed DSP-statistical hybrid is shown to achieve 43% reduction in false positives over WWAM, as used in GENSCAN.

The Pulse Couple Neural Network (PCNN) was developed by Eckhorn to model the observed synchronization of neural assemblies in the visual cortex of small mammals such as a cat. In this dissertation, three novel PCNN based automatic segmentation algorithms were developed to segment Magnetic Resonance Imaging (MRI) data: (a) PCNN image 'signature' based single region cropping; (b) PCNN - Kittler Illingworth minimum error thresholding and (c) PCNN -Gaussian Mixture Model - Expectation Maximization (GMM-EM) based multiple material segmentation. Among other control tests, the proposed algorithms were tested on three T2 weighted acquisition configurations comprising a total of 42 rat brain volumes, 20 T1 weighted MR human brain volumes from Harvard's Internet Brain Segmentation Repository and 5 human MR breast volumes. The results were compared against manually segmented gold standards, Brain Extraction Tool (BET) V2.1 results, published results and single threshold methods. The Jaccard similarity index was used for numerical evaluation of the proposed algorithms. Our quantitative results demonstrate conclusively that PCNN based multiple material segmentation strategies can approach a human eye's intensity delineation capability in grayscale image segmentation tasks.

Detecting human actions using a camera has many possible applications in the security industry. When a human performs an action, his/her body goes through a signature sequence of poses. To detect these pose changes and hence the activities performed, a pattern recogniser needs to be built into the video system. Due to the temporal nature of the patterns, Hidden Markov Models (HMM), used extensively in speech recognition, were investigated. Initially a gesture recognition system was built using novel features. These features were obtained by approximating the contour of the foreground object with a polygon and extracting the polygon's vertices. A Gaussian Mixture Model (GMM) was fit to the vertices obtained from a few frames and the parameters of the GMM itself were used as features for the HMM. A more practical activity detection system using a more sophisticated foreground segmentation algorithm immune to varying lighting conditions and permanent changes to the foreground was then built. The foreground segmentation algorithm models each of the pixel values using clusters and continually uses incoming pixels to update the cluster parameters. Cast shadows were identified and removed by assuming that shadow regions were less likely to produce strong edges in the image than real objects and that this likelihood further decreases after colour segmentation. Colour segmentation itself was performed by clustering together pixel values in the feature space using a gradient ascent algorithm called mean shift. More robust features in the form of mesh features were also obtained by dividing the bounding box of the binarised object into grid elements and calculating the ratio of foreground to background pixels in each of the grid elements. These features were vector quantized to reduce their dimensionality and the resulting symbols presented as features to the HMM to achieve a recognition rate of 62% for an event involving a person writing on a white board. The recognition rate increased to 80% for the "seen" person sequences, i.e. the sequences of the person used to train the models. With a fixed lighting position, the lack of a shadow removal subsystem improved the detection rate. This is because of the consistent profile of the shadows in both the training and testing sequences due to the fixed lighting positions. Even with a lower recognition rate, the shadow removal subsystem was considered an indispensable part of a practical, generic surveillance system.

Detecting human actions using a camera has many possible applications in the security industry. When a human performs an action, his/her body goes through a signature sequence of poses. To detect these pose changes and hence the activities performed, a pattern recogniser needs to be built into the video system. Due to the temporal nature of the patterns, Hidden Markov Models (HMM), used extensively in speech recognition, were investigated. Initially a gesture recognition system was built using novel features. These features were obtained by approximating the contour of the foreground object with a polygon and extracting the polygon's vertices. A Gaussian Mixture Model (GMM) was fit to the vertices obtained from a few frames and the parameters of the GMM itself were used as features for the HMM. A more practical activity detection system using a more sophisticated foreground segmentation algorithm immune to varying lighting conditions and permanent changes to the foreground was then built. The foreground segmentation algorithm models each of the pixel values using clusters and continually uses incoming pixels to update the cluster parameters. Cast shadows were identified and removed by assuming that shadow regions were less likely to produce strong edges in the image than real objects and that this likelihood further decreases after colour segmentation. Colour segmentation itself was performed by clustering together pixel values in the feature space using a gradient ascent algorithm called mean shift. More robust features in the form of mesh features were also obtained by dividing the bounding box of the binarised object into grid elements and calculating the ratio of foreground to background pixels in each of the grid elements. These features were vector quantized to reduce their dimensionality and the resulting symbols presented as features to the HMM to achieve a recognition rate of 62% for an event involving a person writing on a white board. The recognition rate increased to 80% for the "seen" person sequences, i.e. the sequences of the person used to train the models. With a fixed lighting position, the lack of a shadow removal subsystem improved the detection rate. This is because of the consistent profile of the shadows in both the training and testing sequences due to the fixed lighting positions. Even with a lower recognition rate, the shadow removal subsystem was considered an indispensable part of a practical, generic surveillance system.