Дисертації з теми "Variational Infernce"
Щоб переглянути інші типи публікацій з цієї теми, перейдіть за посиланням: Variational Infernce.
Оформте джерело за APA, MLA, Chicago, Harvard та іншими стилями
Ознайомтеся з топ-50 дисертацій для дослідження на тему "Variational Infernce".
Біля кожної праці в переліку літератури доступна кнопка «Додати до бібліографії». Скористайтеся нею – і ми автоматично оформимо бібліографічне посилання на обрану працю в потрібному вам стилі цитування: APA, MLA, «Гарвард», «Чикаго», «Ванкувер» тощо.
Також ви можете завантажити повний текст наукової публікації у форматі «.pdf» та прочитати онлайн анотацію до роботи, якщо відповідні параметри наявні в метаданих.
Переглядайте дисертації для різних дисциплін та оформлюйте правильно вашу бібліографію.
1
Rouillard, Louis. "Bridging Simulation-based Inference and Hierarchical Modeling : Applications in Neuroscience." Electronic Thesis or Diss., université Paris-Saclay, 2024. http://www.theses.fr/2024UPASG024.
Анотація:
La neuroimagerie étudie l'architecture et le fonctionnement du cerveau à l'aide de la résonance magnétique (IRM). Pour comprendre le signal complexe observé, les neuroscientifiques émettent des hypothèses sous la forme de modèles explicatifs, régis par des paramètres interprétables. Cette thèse étudie l'inférence statistique : deviner quels paramètres auraient pu produire le signal à travers le modèle.L'inférence en neuroimagerie est complexifiée par au moins trois obstacles : une grande dimensionnalité, une grande incertitude et la structure hiérarchique des données. Pour s'attaquer à ce régime, nous utlisons l'inférence variationnelle (VI), une méthode basée sur l'optimisation.Plus précisément, nous combinons l'inférence variationnelle stochastique structurée et les flux de normalisation (NF) pour concevoir des familles variationnelles expressives et adaptées à la large dimensionnalité. Nous appliquons ces techniques à l'IRM de diffusion et l'IRM fonctionnelle, sur des tâches telles que la parcellation individuelle, l'inférence de la microstructure et l'estimation du couplage directionnel. Via ces applications, nous soulignons l'interaction entre les divergences de Kullback-Leibler (KL) forward et reverse comme outils complémentaires pour l'inférence. Nous démontrons également les capacité de l'inférence variationelle automatique (AVI) comme méthode d'inférence robuste et adaptée à la large dimensionnalité, apte à relever les défis de la modélisation en neuroscienceNeuroimaging investigates the brain's architecture and function using magnetic resonance (MRI). To make sense of the complex observed signal, Neuroscientists posit explanatory models, governed by interpretable parameters. This thesis tackles statistical inference : guessing which parameters could have yielded the signal through the model.Inference in Neuroimaging is complexified by at least three hurdles : a large dimensionality, a large uncertainty, and the hierarchcial structure of data. We look into variational inference (VI) as an optimization-based method to tackle this regime.Specifically, we conbine structured stochastic VI and normalizing flows (NFs) to design expressive yet scalable variational families. We apply those techniques in diffusion and functional MRI, on tasks including individual parcellation, microstructure inference and directional coupling estimation. Through these applications, we underline the interplay between the forward and reverse Kullback-Leibler (KL) divergences as complemen-tary tools for inference. We also demonstrate the ability of automatic VI (AVI) as a reliable and scalable inference method to tackle the challenges of model-driven Neuroscience
2
Calabrese, Chris M. Eng Massachusetts Institute of Technology. "Distributed inference : combining variational inference with distributed computing." Thesis, Massachusetts Institute of Technology, 2013. http://hdl.handle.net/1721.1/85407.
Анотація:
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2013.Cataloged from PDF version of thesis.
Includes bibliographical references (pages 95-97).
The study of inference techniques and their use for solving complicated models has taken off in recent years, but as the models we attempt to solve become more complex, there is a worry that our inference techniques will be unable to produce results. Many problems are difficult to solve using current approaches because it takes too long for our implementations to converge on useful values. While coming up with more efficient inference algorithms may be the answer, we believe that an alternative approach to solving this complicated problem involves leveraging the computation power of multiple processors or machines with existing inference algorithms. This thesis describes the design and implementation of such a system by combining a variational inference implementation (Variational Message Passing) with a high-level distributed framework (Graphlab) and demonstrates that inference is performed faster on a few large graphical models when using this system.
by Chris Calabrese.
M. Eng.
3
Lawrence, Neil David. "Variational inference in probabilistic models." Thesis, University of Cambridge, 2001. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.621104.
4
Beal, Matthew James. "Variational algorithms for approximate Bayesian inference." Thesis, University College London (University of London), 2003. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.404387.
5
Wang, Pengyu. "Collapsed variational inference for computational linguistics." Thesis, University of Oxford, 2016. https://ora.ox.ac.uk/objects/uuid:13c08f60-1441-4ea5-b52f-7ffd0d7a744f.
Анотація:
Bayesian modelling is a natural fit for tasks in computational linguistics, since it can provide interpretable structures, useful prior controls, and coherent management of uncertainty. However, exact Bayesian inference is intractable for many models of practical interest. Developing both accurate and efficient approximate Bayesian inference algorithms remains a fundamental challenge, especially for the field of computational linguistics where datasets are large and growing and models consist of complex latent structures. Collapsed variational inference (CVI) is an important milestone that combines the efficiency of variational inference (VI) and the accuracy of Markov chain Monte Carlo (MCMC) (Teh et al., 2006). However, its previous applications were limited to bag-of-words models whose hidden variables are conditionally independent given the parameters, whereas in computational linguistics, the hidden variable dependencies are crucial for modelling the underlying syntactic and semantic relations. To enlarge the application domain of CVI as well as to address the above Bayesian inference challenge, we investigate the applications of collapsed variational inference to computational linguistics. In this thesis, our contributions are three-fold. First, we solve a number of inference challenges arising from the hidden variable dependencies and derive a set of new CVI algorithms for the two ubiquitous and foundational models in computational linguistics, namely hidden Markov models (HMMs) and probabilistic context free grammars. We also propose CVI for hierarchical Dirichlet process (HDP) HMMs that are Bayesian nonparametric extensions of HMMs. Second, along the way we propose a set of novel algorithmic techniques, which are generally applicable to a wide variety of probabilistic graphical models in the conjugate exponential family and computational linguistic models using non-conjugate HDP constructions. Therefore, our work represents one step in bridging the gap between increasingly richer Bayesian models in computational linguistics and recent advances in approximate Bayesian inference. Third, we empirically evaluate our proposed CVI algorithms and their stochastic versions in a range of computational linguistic tasks, such as part-of-speech induction, grammar induction and many others. Experimental results consistently demonstrate that, using our techniques for handling the hidden variable dependencies, the empirical advantages of both VI and MCMC can be combined in a much larger domain of CVI applications.
6
Mamikonyan, Arsen. "Variational inference for non-stationary distributions." Thesis, Massachusetts Institute of Technology, 2017. http://hdl.handle.net/1721.1/113125.
Анотація:
Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2017.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Cataloged from student-submitted PDF version of thesis.
Includes bibliographical references (page 49).
In this thesis, I look at multiple Variational Inference algorithm, transform Kalman Variational Bayes and Stochastic Variational Inference into streaming algorithms and try to identify if any of them work with non-stationary distributions. I conclude that Kalman Variational Bayes can do as good as any other algorithm for stationary distributions, and tracks non-stationary distributions better than any other algorithm in question.
by Arsen Mamikonyan.
M. Eng.
7
Thorpe, Matthew. "Variational methods for geometric statistical inference." Thesis, University of Warwick, 2015. http://wrap.warwick.ac.uk/74241/.
Анотація:
Estimating multiple geometric shapes such as tracks or surfaces creates significant mathematical challenges particularly in the presence of unknown data association. In particular, problems of this type have two major challenges. The first is typically the object of interest is infinite dimensional whilst data is finite dimensional. As a result the inverse problem is ill-posed without regularization. The second is the data association makes the likelihood function highly oscillatory. The focus of this thesis is on techniques to validate approaches to estimating problems in geometric statistical inference. We use convergence of the large data limit as an indicator of robustness of the methodology. One particular advantage of our approach is that we can prove convergence under modest conditions on the data generating process. This allows one to apply the theory where very little is known about the data. This indicates a robustness in applications to real world problems. The results of this thesis therefore concern the asymptotics for a selection of statistical inference problems. We construct our estimates as the minimizer of an appropriate functional and look at what happens in the large data limit. In each case we will show our estimates converge to a minimizer of a limiting functional. In certain cases we also add rates of convergence. The emphasis is on problems which contain a data association or classification component. More precisely we study a generalized version of the k-means method which is suitable for estimating multiple trajectories from unlabeled data which combines data association with spline smoothing. Another problem considered is a graphical approach to estimating the labeling of data points. Our approach uses minimizers of the Ginzburg-Landau functional on a suitably defined graph. In order to study these problems we use variational techniques and in particular I-convergence. This is the natural framework to use for studying sequences of minimization problems. A key advantage of this approach is that it allows us to deal with infinite dimensional and highly oscillatory functionals.
8
Challis, E. A. L. "Variational approximate inference in latent linear models." Thesis, University College London (University of London), 2013. http://discovery.ucl.ac.uk/1414228/.
Анотація:
Latent linear models are core to much of machine learning and statistics. Specific examples of this model class include Bayesian generalised linear models, Gaussian process regression models and unsupervised latent linear models such as factor analysis and principal components analysis. In general, exact inference in this model class is computationally and analytically intractable. Approximations are thus required. In this thesis we consider deterministic approximate inference methods based on minimising the Kullback-Leibler (KL) divergence between a given target density and an approximating `variational' density. First we consider Gaussian KL (G-KL) approximate inference methods where the approximating variational density is a multivariate Gaussian. Regarding this procedure we make a number of novel contributions: sufficient conditions for which the G-KL objective is differentiable and convex are described, constrained parameterisations of Gaussian covariance that make G-KL methods fast and scalable are presented, the G-KL lower-bound to the target density's normalisation constant is proven to dominate those provided by local variational bounding methods. We also discuss complexity and model applicability issues of G-KL and other Gaussian approximate inference methods. To numerically validate our approach we present results comparing the performance of G-KL and other deterministic Gaussian approximate inference methods across a range of latent linear model inference problems. Second we present a new method to perform KL variational inference for a broad class of approximating variational densities. Specifically, we construct the variational density as an affine transformation of independently distributed latent random variables. The method we develop extends the known class of tractable variational approximations for which the KL divergence can be computed and optimised and enables more accurate approximations of non-Gaussian target densities to be obtained.
9
Matthews, Alexander Graeme de Garis. "Scalable Gaussian process inference using variational methods." Thesis, University of Cambridge, 2017. https://www.repository.cam.ac.uk/handle/1810/278022.
Анотація:
Gaussian processes can be used as priors on functions. The need for a flexible, principled, probabilistic model of functional relations is common in practice. Consequently, such an approach is demonstrably useful in a large variety of applications. Two challenges of Gaussian process modelling are often encountered. These are dealing with the adverse scaling with the number of data points and the lack of closed form posteriors when the likelihood is non-Gaussian. In this thesis, we study variational inference as a framework for meeting these challenges. An introductory chapter motivates the use of stochastic processes as priors, with a particular focus on Gaussian process modelling. A section on variational inference reviews the general definition of Kullback-Leibler divergence. The concept of prior conditional matching that is used throughout the thesis is contrasted to classical approaches to obtaining tractable variational approximating families. Various theoretical issues arising from the application of variational inference to the infinite dimensional Gaussian process setting are settled decisively. From this theory we are able to give a new argument for existing approaches to variational regression that settles debate about their applicability. This view on these methods justifies the principled extensions found in the rest of the work. The case of scalable Gaussian process classification is studied, both for its own merits and as a case study for non-Gaussian likelihoods in general. Using the resulting algorithms we find credible results on datasets of a scale and complexity that was not possible before our work. An extension to include Bayesian priors on model hyperparameters is studied alongside a new inference method that combines the benefits of variational sparsity and MCMC methods. The utility of such an approach is shown on a variety of example modelling tasks. We describe GPflow, a new Gaussian process software library that uses TensorFlow. Implementations of the variational algorithms discussed in the rest of the thesis are included as part of the software. We discuss the benefits of GPflow when compared to other similar software. Increased computational speed is demonstrated in relevant, timed, experimental comparisons.
10
Maestrini, Luca. "On variational approximations for frequentist and bayesian inference." Doctoral thesis, Università degli studi di Padova, 2018. http://hdl.handle.net/11577/3424936.
Анотація:
Variational approximations are approximate inference techniques for complex statisticalmodels providing fast, deterministic alternatives to conventional methods that,however accurate, take much longer to run. We extend recent work concerning variationalapproximations developing and assessing some variational tools for likelihoodbased and Bayesian inference. In particular, the first part of this thesis employs a Gaussian
variational approximation strategy to handle frequentist generalized linear mixedmodels with general design random effects matrices such as those including spline basisfunctions. This method involves approximation to the distributions of random effectsvectors, conditional on the responses, via a Gaussian density. The second thread isconcerned with a particular class of variational approximations, known as mean fieldvariational Bayes, which is based upon a nonparametric product density restriction on
the approximating density. Algorithms for inference and fitting for models with elaborateresponses and structures are developed adopting the variational message passingperspective. The modularity of variational message passing is such that extensions tomodels with more involved likelihood structures and scalability to big datasets are relatively
simple. We also derive algorithms for models containing higher level randomeffects and non-normal responses, which are streamlined in support of computationalefficiency. Numerical studies and illustrations are provided, including comparisons witha Markov chain Monte Carlo benchmark.Le approssimazioni variazionali sono tecniche di inferenza approssimata per modelli statisticicomplessi che si propongono come alternative, più rapide e di tipo deterministico,a metodi tradizionali che, sebbene accurati, necessitano di maggiori tempi per l'adattamento. Vengono qui sviluppati e valutati alcuni strumenti variazionali per l'inferenzabasata sulla verosimiglianza e per l'inferenza bayesiana, estendendo dei risultati recentiin letteratura sulle approssimazioni variazionali. In particolare, la prima parte dellatesi impiega una strategia basata su un'approssimazione variazionale gaussiana per la funzione di verosimiglianza di modelli lineari generalizzati misti con matrici di disegnodegli effetti casuali generiche, includenti, per esempio, funzioni di basi spline. Questometodo consiste nell'approssimare la distribuzione del vettore degli effetti casuali,condizionatamente alle risposte, con una densità gaussiana. Il secondo filone concerneinvece una particolare classe di approssimazioni variazionali nota come mean field variational Bayes, che impone un prodotto di densità come restrizione non parametrica sulla densità approssimante. Vengono sviluppati algoritmi per l'inferenza e l'adattamento dimodelli con risposte elaborate, adottando la prospettiva del variational message passing. La modularità del variational message passing è tale da consentire estensioni amodelli con strutture di verosimiglianza più complesse e scalabilità a insiemi di dati di grandi dimensioni con relativa semplicità. Vengono inoltre derivati in forma esplicitadegli algoritmi per modelli contenenti effetti casuali su più livelli e risposte non normali,introducendo semplicazioni atte a incrementare l'efficienza computazionale. Sonoinclusi studi numerici e illustrazioni, considerando come riferimento per un confronto il metodo Markov chain Monte Carlo.
11
Houghton, Adrian James. "Variational Bayesian inference for comparison Var(1) models." Thesis, University of Newcastle Upon Tyne, 2009. http://hdl.handle.net/10443/790.
Анотація:
Suppose that we wish to determine which models in a candidate set are most likely to have given rise to a set of observed data. Then, it is well-established that, from a Bayesian viewpoint, evaluation of the marginal likelihood for each candidate is a crucial step to this end. For the purposes of model comparison, this will enable subsequent computation of both Bayes’ factors and posterior model probabilities. Given its evident significance in this area, it is thus regrettable that analytic calculation of the marginal likelihood is often not possible. To tackle this problem, one recent addition to the literature is the variational Bayesian approach. In this thesis, it is seen that variational Bayes provides efficient, accurate approximations to both the marginal likelihood and the parameter posterior distribution, conditioned on each model. In particular, the theory is applied to ranking sparse, vector autoregressive graphical models of order 1 in both the zero and non-zero mean case. That is, our primary aim is to estimate the unknown sparsity structure of the autoregressive matrix in the process. Moreover, approximate, marginal posterior information about the coefficients of this matrix is also of interest. To enable rapid exploration of higher-dimensional graphical spaces, a Metropolis-Hastings algorithm is presented so that a random walk can be made between neighbouring graphs. The scheme is then tested on both simulated and real datasets of varying dimension.
12
Zhang, Ye. "Community Detection| Fundamental Limits, Methodology, and Variational Inference." Thesis, Yale University, 2018. http://pqdtopen.proquest.com/#viewpdf?dispub=10957347.
Анотація:
Network analysis has become one of the most active research areas over the past few years. A core problem in network analysis is community detection. In this thesis, we investigate it under Stochastic Block Model and Degree-corrected Block Model from three different perspectives: 1) the minimax rates of community detection problem, 2) rate-optimal and computationally feasible algorithms, and 3) computational and theoretical guarantees of variational inference for community detection.
13
Andersson, Gabriel. "Decoding Neural Signals Associated to Cytokine Activity." Thesis, KTH, Matematik (Inst.), 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-291559.
Анотація:
The Vagus nerve has shown to play an important role regarding inflammatory diseases, regulating the production of proteins that mediate inflammation. Two important such proteins are the pro-inflammatory cytokines, TNF and IL-1β. This thesis makes use of Vagus nerve recordings, where TNF and IL-1β are subsequently injected in mice, with the aim to see if cytokine-specific information can be extracted. To this end, a type of semi-supervised learning approach is applied, where the observed waveform-data are modeled using a conditional probability distribution. The conditioning is done based on an estimate of how often each observed waveform occurs and local maxima of the conditional distribution are interpreted as candidate-waveforms to encode cytokine information. The methodology yields varying, but promising results. The occurrence of several candidate waveforms are found to increase substantially after exposure to cytokine. Difficulties obtaining coherent results are discussed, as well as different approaches for future work.Vagusnerven har visat sig spela en viktig roll beträffande inflammatoriska sjukdomar. Denna nerv reglerar produktionen av inflammatoriska protein, som de inflammationsfrämjande cytokinerna TNF och IL-1β. Detta arbete använder sig av elektroniska mätningar av Vagusnerven i möss som under tiden blir injicerade med de två cytokinerna TNF och IL-1β. Syftet med arbetet är att undersöka om det är möjligt att extrahera information om de specifika cytokinerna från Vagusnervmätningarna. För att uppnå detta designar vi en semi-vägledd lärandemetod som modellerar dem observerade vågformerna med en betingad sannolikhetsfunktion. Betingandet baseras på en uppskattning av hur ofta varje enskild vågform förekommer och lokala maximum av den betingade sannolikhetsfunktionen tolkas som möjliga kandidat-vågformer att innehålla cytokin-information. Metodiken ger varierande, men lovande resultat. Förekomsten av flertalet kandidat-vågformer har en tydlig ökning efter tidpunkten för cytokin-injektion. Vidare så diskuteras svårigheter i att uppnå konsekventa resultat för alla mätningar, samt olika möjligheter för framtida arbete inom området.
14
Ocone, Andrea. "Variational inference for Gaussian-jump processes with application in gene regulation." Thesis, University of Edinburgh, 2013. http://hdl.handle.net/1842/8280.
Анотація:
In the last decades, the explosion of data from quantitative techniques has revolutionised our understanding of biological processes. In this scenario, advanced statistical methods and algorithms are becoming fundamental to decipher the dynamics of biochemical mechanisms such those involved in the regulation of gene expression. Here we develop mechanistic models and approximate inference techniques to reverse engineer the dynamics of gene regulation, from mRNA and/or protein time series data. We start from an existent variational framework for statistical inference in transcriptional networks. The framework is based on a continuous-time description of the mRNA dynamics in terms of stochastic differential equations, which are governed by latent switching variables representing the on/off activity of regulating transcription factors. The main contributions of this work are the following. We speeded-up the variational inference algorithm by developing a method to compute a posterior approximate distribution over the latent variables using a constrained optimisation algorithm. In addition to computational benefits, this method enabled the extension to statistical inference in networks with a combinatorial model of regulation. A limitation of this framework is the fact that inference is possible only in transcriptional networks with a single-layer architecture (where a single or couples of transcription factors regulate directly an arbitrary number of target genes). The second main contribution in this work is the extension of the inference framework to hierarchical structures, such as feed-forward loop. In the last contribution we define a general structure for transcription-translation networks. This work is important since it provides a general statistical framework to model complex dynamics in gene regulatory networks. The framework is modular and scalable to realistically large systems with general architecture, thus representing a valuable alternative to traditional differential equation models. All models are embedded in a Bayesian framework; inference is performed using a variational approach and compared to exact inference where possible. We apply the models to the study of different biological systems, from the metabolism in E. coli to the circadian clock in the picoalga O. tauri.
15
Jaakkola, Tommi S. (Tommi Sakari). "Variational methods for inference and estimation in graphical models." Thesis, Massachusetts Institute of Technology, 1997. http://hdl.handle.net/1721.1/10307.
16
Sontag, David Alexander. "Cutting plane algorithms for variational inference in graphical models." Thesis, Massachusetts Institute of Technology, 2007. http://hdl.handle.net/1721.1/40327.
Анотація:
Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2007.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Includes bibliographical references (leaves 65-66).
In this thesis, we give a new class of outer bounds on the marginal polytope, and propose a cutting-plane algorithm for efficiently optimizing over these constraints. When combined with a concave upper bound on the entropy, this gives a new variational inference algorithm for probabilistic inference in discrete Markov Random Fields (MRFs). Valid constraints are derived for the marginal polytope through a series of projections onto the cut polytope. Projecting onto a larger model gives an efficient separation algorithm for a large class of valid inequalities arising from each of the original projections. As a result, we obtain tighter upper bounds on the logpartition function than possible with previous variational inference algorithms. We also show empirically that our approximations of the marginals are significantly more accurate. This algorithm can also be applied to the problem of finding the Maximum a Posteriori assignment in a MRF, which corresponds to a linear program over the marginal polytope. One of the main contributions of the thesis is to bring together two seemingly different fields, polyhedral combinatorics and probabilistic inference, showing how certain results in either field can carry over to the other.
by David Alexander Sontag.
S.M.
17
Knollmüller, Jakob [Verfasser], and Torsten [Akademischer Betreuer] Enßlin. "Metric Gaussian variational inference / Jakob Knollmüller ; Betreuer: Torsten Enßlin." München : Universitätsbibliothek der Ludwig-Maximilians-Universität, 2020. http://d-nb.info/1227839901/34.
18
Teng, Jing. "Variational filtering for bayesian inference in wireless sensor networks." Troyes, 2009. http://www.theses.fr/2009TROY0019.
Анотація:
Dans cette thèse, nous traitons les problèmes d'inférence bayésienne décentralisée dans les réseaux de capteurs sans fil (RCSF). Une approche variationnelle est proposée dans cette thèse afin de s’accommoder des contraintes énergétiques et des contraintes de transmission, inhérentes dans le cadre des RCSF. Trois applications, étroitement liées, ont été traitées: le tracking d'une seule cible, le tracking de plusieurs cibles et l’auto-localisation et le suivi de cible simultanés (SLAT). L’implémentation décentralisée de l’approche variationnelle repose sur un compromis entre la précision de l’estimation et l'efficacité énergétique. Les contributions de la thèse consistent en les points suivants: - Au niveau théorique, nous avons développé une approche variationnelle qui permet une prise en compte implicite de la propagation des erreurs d’approximation en mettant à jour les formes approximées des densités de probabilité dans un cadre non paramétrique. - Au niveau modélisation, la proposition d’un modèle de mélange continu de gaussiennes décrivant la dynamique de l’état de la cible permet un suivi efficace de trajectoires présentant des sauts brusques. Aussi, afin de minimiser la consommation de l’énergie et de respecter la bande passante limitée, un modèle d’observation binaire est utilisé. - Au niveau algorithmique, un protocole d’activation dynamique des clusters est proposé pour l’implémentation décentralisée du filtrage bayésien dans un réseau clustérisé. Un deuxième algorithme de type Dijkstra est aussi proposé afin de former les capteurs leaders d’une manière réactiveIn this thesis, we tackle the intractable Bayesian inference problems in wireless sensor networks (WSNs) by variational approximation. A general framework for variational Bayesian inference is proposed for three basic and closely related applications: single target tracking, multiple targets tracking (MTT), simultaneous sensor localization and target tracking (SLAT). The trade-off between estimation precision and energy-awareness is the primary focus for the WSN applications, leading to decentralized execution of the variational filter (VF). Contributions of the thesis consist in following points: - A VF algorithm simultaneously updates and approximates the filtering distribution, reducing the temporal dependence to one Gaussian statistic. - A general state evolution model describes the target state, allowing discrete jumps in target trajectory. - A binary proximity observation model quantifies an observation to a single bit, minimizing energy and bandwidth consumption. - A non-myopic cluster activation rule based on the prediction of VF is proposed for the proactive cluster management, which dramatically decreases hand-off operations between successive clusters. - A Dijkstra-like clustering algorithm for reactive cluster management yields optimal clustering. - An hybrid probabilistic data association and VF scheme is employed for MTT. - A distributed VF solution for SLAT on-line up-dates and refines estimates of sensor locations and target trajectory
19
Abeywardana, Sachinthaka. "Variational Inference in Generalised Hyperbolic and von Mises-Fisher Distributions." Thesis, The University of Sydney, 2015. http://hdl.handle.net/2123/16504.
Анотація:
Most real world data are skewed, contain more than the set of real numbers, and have higher probabilities of extreme events occurring compared to a normal distribution. In this thesis we explore two non-Gaussian distributions, the Generalised Hyperbolic Distribution (GHD) and, the von-Mises Fisher (vMF) Distribution. These distributions are studied in the context of 1) Regression in heavy tailed data, 2) Quantifying variance of functions with reference to finding relevant quantiles and, 3) Clustering data that lie on the surface of the sphere. Firstly, we extend Gaussian Processes (GPs) and use the Genralised Hyperbolic Processes as a prior on functions instead. This prior is more flexible than GPs and is especially able to model data that has high kurtosis. The method is based on placing a Generalised Inverse Gaussian prior over the signal variance, which yields a scalar mixture of GPs. We show how to perform inference efficiently for the predictive mean and variance, and use a variational EM method for learning. Secondly, the skewed extension of the GHD is studied with respect to quantile regression. An underlying GP prior on the quantile function is used to make the inference non-parametric, while the skewed GHD is used as the data likelihood. The skewed GHD has a single parameter alpha which states the required quantile. Similar variational methods as the first contribution is used to perform inference. Finally, vMF distributions are introduced in order to cluster spherical data. In the two previous contributions continuous scalar mixtures of Gaussians were used to make the inference process simpler. However, for clustering, a discrete number of vMF distributions are typically used. We propose a Dirichlet Process (DP) to infer the number of clusters in the spherical data setup. The framework is extended to incorporate a nested and a temporal clustering architecture. Throughout this thesis in many cases the posterior cannot be calculated in closed form. Variational Bayesian approximations are derived in this situation for efficient inference. In certain cases further lower bounding of the optimisation function is required in order to perform Variational Bayes. These bounds themselves are novel.
20
Nissilä, M. (Mauri). "Iterative receivers for digital communications via variational inference and estimation." Doctoral thesis, University of Oulu, 2008. http://urn.fi/urn:isbn:9789514286865.
Анотація:
Abstract
In this thesis, iterative detection and estimation algorithms for digital communications systems in the presence of parametric uncertainty are explored and further developed. In particular, variational methods, which have been extensively applied in other research fields such as artificial intelligence and machine learning, are introduced and systematically used in deriving approximations to the optimal receivers in various channel conditions. The key idea behind the variational methods is to transform the problem of interest into an optimization problem via an introduction of extra degrees of freedom known as variational parameters. This is done so that, for fixed values of the free parameters, the transformed problem has a simple solution, solving approximately the original problem.
The thesis contributes to the state of the art of advanced receiver design in a number of ways. These include the development of new theoretical and conceptual viewpoints of iterative turbo-processing receivers as well as a new set of practical joint estimation and detection algorithms. Central to the theoretical studies is to show that many of the known low-complexity turbo receivers, such as linear minimum mean square error (MMSE) soft-input soft-output (SISO) equalizers and demodulators that are based on the Bayesian expectation-maximization (BEM) algorithm, can be formulated as solutions to the variational optimization problem. This new approach not only provides new insights into the current designs and structural properties of the relevant receivers, but also suggests some improvements on them.
In addition, SISO detection in multipath fading channels is considered with the aim of obtaining a new class of low-complexity adaptive SISOs. As a result, a novel, unified method is proposed and applied in order to derive recursive versions of the classical Baum-Welch algorithm and its Bayesian counterpart, referred to as the BEM algorithm. These formulations are shown to yield computationally attractive soft decision-directed (SDD) channel estimators for both deterministic and Rayleigh fading intersymbol interference (ISI) channels.
Next, by modeling the multipath fading channel as a complex bandpass autoregressive (AR) process, it is shown that the statistical parameters of radio channels, such as frequency offset, Doppler spread, and power-delay profile, can be conveniently extracted from the estimated AR parameters which, in turn, may be conveniently derived via an EM algorithm. Such a joint estimator for all relevant radio channel parameters has a number of virtues, particularly its capability to perform equally well in a variety of channel conditions.
Lastly, adaptive iterative detection in the presence of phase uncertainty is investigated. As a result, novel iterative joint Bayesian estimation and symbol a posteriori probability (APP) computation algorithms, based on the variational Bayesian method, are proposed for both constant-phase channel models and dynamic phase models, and their performance is evaluated via computer simulations.
21
Cherief-Abdellatif, Badr-Eddine. "Contributions to the theoretical study of variational inference and robustness." Electronic Thesis or Diss., Institut polytechnique de Paris, 2020. http://www.theses.fr/2020IPPAG001.
Анотація:
Cette thèse de doctorat traite de l'inférence variationnelle et de la robustesse en statistique et en machine learning. Plus précisément, elle se concentre sur les propriétés statistiques des approximations variationnelles et sur la conception d'algorithmes efficaces pour les calculer de manière séquentielle, et étudie les estimateurs basés sur le Maximum Mean Discrepancy comme règles d'apprentissage qui sont robustes à la mauvaise spécification du modèle.Ces dernières années, l'inférence variationnelle a été largement étudiée du point de vue computationnel, cependant, la littérature n'a accordé que peu d'attention à ses propriétés théoriques jusqu'à très récemment. Dans cette thèse, nous étudions la consistence des approximations variationnelles dans divers modèles statistiques et les conditions qui assurent leur consistence. En particulier, nous abordons le cas des modèles de mélange et des réseaux de neurones profonds. Nous justifions également d'un point de vue théorique l'utilisation de la stratégie de maximisation de l'ELBO, un critère numérique qui est largement utilisé dans la communauté VB pour la sélection de modèle et dont l'efficacité a déjà été confirmée en pratique. En outre, l'inférence Bayésienne offre un cadre d'apprentissage en ligne attrayant pour analyser des données séquentielles, et offre des garanties de généralisation qui restent valables même en cas de mauvaise spécification des modèles et en présence d'adversaires. Malheureusement, l'inférence Bayésienne exacte est rarement tractable en pratique et des méthodes d'approximation sont généralement employées, mais ces méthodes préservent-elles les propriétés de généralisation de l'inférence Bayésienne ? Dans cette thèse, nous montrons que c'est effectivement le cas pour certains algorithmes d'inférence variationnelle (VI). Nous proposons de nouveaux algorithmes tempérés en ligne et nous en déduisons des bornes de généralisation. Notre résultat théorique repose sur la convexité de l'objectif variationnel, mais nous soutenons que notre résultat devrait être plus général et présentons des preuves empiriques à l'appui. Notre travail donne des justifications théoriques en faveur des algorithmes en ligne qui s'appuient sur des méthodes Bayésiennes approchées.Une autre question d'intérêt majeur en statistique qui est abordée dans cette thèse est la conception d'une procédure d'estimation universelle. Cette question est d'un intérêt majeur, notamment parce qu'elle conduit à des estimateurs robustes, un thème d'actualité en statistique et en machine learning. Nous abordons le problème de l'estimation universelle en utilisant un estimateur de minimisation de distance basé sur la Maximum Mean Discrepancy. Nous montrons que l'estimateur est robuste à la fois à la dépendance et à la présence de valeurs aberrantes dans le jeu de données. Nous mettons également en évidence les liens qui peuvent exister avec les estimateurs de minimisation de distance utilisant la distance L2. Enfin, nous présentons une étude théorique de l'algorithme de descente de gradient stochastique utilisé pour calculer l'estimateur, et nous étayons nos conclusions par des simulations numériques. Nous proposons également une version Bayésienne de notre estimateur, que nous étudions à la fois d'un point de vue théorique et d'un point de vue computationnelThis PhD thesis deals with variational inference and robustness. More precisely, it focuses on the statistical properties of variational approximations and the design of efficient algorithms for computing them in an online fashion, and investigates Maximum Mean Discrepancy based estimators as learning rules that are robust to model misspecification.In recent years, variational inference has been extensively studied from the computational viewpoint, but only little attention has been put in the literature towards theoretical properties of variational approximations until very recently. In this thesis, we investigate the consistency of variational approximations in various statistical models and the conditions that ensure the consistency of variational approximations. In particular, we tackle the special case of mixture models and deep neural networks. We also justify in theory the use of the ELBO maximization strategy, a model selection criterion that is widely used in the Variational Bayes community and is known to work well in practice.Moreover, Bayesian inference provides an attractive online-learning framework to analyze sequential data, and offers generalization guarantees which hold even under model mismatch and with adversaries. Unfortunately, exact Bayesian inference is rarely feasible in practice and approximation methods are usually employed, but do such methods preserve the generalization properties of Bayesian inference? In this thesis, we show that this is indeed the case for some variational inference algorithms. We propose new online, tempered variational algorithms and derive their generalization bounds. Our theoretical result relies on the convexity of the variational objective, but we argue that our result should hold more generally and present empirical evidence in support of this. Our work presents theoretical justifications in favor of online algorithms that rely on approximate Bayesian methods. Another point that is addressed in this thesis is the design of a universal estimation procedure. This question is of major interest, in particular because it leads to robust estimators, a very hot topic in statistics and machine learning. We tackle the problem of universal estimation using a minimum distance estimator based on the Maximum Mean Discrepancy. We show that the estimator is robust to both dependence and to the presence of outliers in the dataset. We also highlight the connections that may exist with minimum distance estimators using L2-distance. Finally, we provide a theoretical study of the stochastic gradient descent algorithm used to compute the estimator, and we support our findings with numerical simulations. We also propose a Bayesian version of our estimator, that we study from both a theoretical and a computational points of view
22
曾達誠 and Tat-shing Tsang. "Statistical inference on the coefficient of variation." Thesis, The University of Hong Kong (Pokfulam, Hong Kong), 2000. http://hub.hku.hk/bib/B31223503.
23
Tsang, Tat-shing. "Statistical inference on the coefficient of variation /." Hong Kong : University of Hong Kong, 2000. http://sunzi.lib.hku.hk/hkuto/record.jsp?B21903980.
24
Wang, Jiabin. "Variational Bayes inference based segmentation algorithms for brain PET-CT images." Thesis, The University of Sydney, 2012. https://hdl.handle.net/2123/29251.
Анотація:
Dual modality PET-CT imaging can provide aligned anatomical (CT) and functional (PET) images in a single scanning session, and has nowadays steadily replaced single modality PET imaging in clinical practice. The enormous number of PET-CT images produced in hospitals are currently analysed almost entirely through visual inspection on a slice-by-slice basis, which requires a high degree of skill and concentration, and is time-consuming, expensive, prone to operator bias, and unsuitable for the processing large-scale studies. Computer-aided diagnosis, where image segmentation is an essential step, would enable doctors and researchers to bypass these issues. However, most medical image segmentation methods are designed for single modality images. In this thesis, the automated segmentation of dual-modality brain PET-CT images has been comprehensively investigated by using variational learning techniques. Two novel statistical segmentation algorithms, namely the DE-VEM algorithm and PA-VEM algorithm, have been proposed to delineate brain PET-CT images into grey matter (GM), white matter (WM) and cerebrospinal fluid (CSF).
In statistical image segmentation, voxel values are usually characterised by probabilistic models, whose parameters can be estimated by using the maximum likelihood estimation, and the optimal segmentation result is regarded as the one that maximises the posterior probability. Despite of their simplicity, statistical approaches intrinsically suffer from overfitting and local convergence. In variational Bayes inference, statistical model parameters are further assumed to be random variables to improve the model's flexibility. Instead of directly estimating the posterior probability, variational learning techniques use a variational distribution to approximate the posterior probability, and thus are able to overcome the drawback of overfitting. The most widely used variational learning technique is the variational expectation maximisation (VEM) algorithm. As a natural extension of the traditional expectation maximisation (EM) algorithm, the VEM algorithm is also a two-step iterative process and still faces the risk of being trapped in a local maximum and the difficulty of incorporating prior knowledge.
Inspired by the fact that global optimisation techniques, such as the genetic algorithm, have been successfully applied to replace the EM algorithm in the maximum-likelihood estimation of probabilistic models, this research combines the differential evolution (DE) algorithm and VEM algorithm to solve the optimisation problem involved in the variational Bayes inference, and thus proposes the DE-VEM algorithm for brain PET -CT image segmentation. In this algorithm, the DE scheme is introduced to search a global solution and the VEM scheme is employed to perform a local search. Since DE is population-based global optimisation technique and has proven itself in a variety of applications with good, the DEYEM algorithm has the potential to avoid local convergence. The proposed algorithm has been compared with the YEM algorithm and the segmentation function in the statistical parametric mapping (SPM, Version 2008) package in 21 clinical brain PET -CT images. My results show that the DE-YEM algorithm outperforms the other two algorithms and can produce accurate segmentation of brain PET-CT images.
Meanwhile, to incorporate the prior anatomical information into the variational learning based brain image segmentation process, the probabilistic brain atlas is generated and used to guide the search of an optimal segmentation result through performing the YEM iteration. As a result, the probabilistic atlas based YEM (PA-YEM) algorithm is developed to allow each voxel to have an adaptable prior probability of belonging to each class. This algorithm has been compared to the segmentation functions in the SPM8 package and the EMS package, the DE-YEM algorithm, and the DEV algorithm in 21 clinical brain PET-CT images. My results demonstrate that the proposed PA-YEM algorithm can substantially improve the accuracy of segmenting brain PET -CT images.
Although this research uses the brain PET -CT images as case studies, the theoretical outcomes are generic and can be extended to the segmentation of other dual-modality medical images. The future work in this area should be focused mainly on improving the computational efficiency of variational learning based image segmentation approaches.
25
BOLZONI, MATTIA. "Variational inference and semi-parametric methods for time-series probabilistic forecasting." Doctoral thesis, Università degli Studi di Milano-Bicocca, 2021. http://hdl.handle.net/10281/313704.
Анотація:
Prevedere la probabilità di eventi futuri è un problema comune. L'approccio più utilizzato assume una struttura fissa per questa probabilità, detta modello, dipendente da variabili latenti dette parametri. Dopo aver osservato dei dati è possibile inferire una distribuzione per queste variabili non osservabili.
Il procedimento di inferenza non è sempre immediato, siccome selezionare un singolo valore per i parametri potrebbe portare a scarsi risultati, mentre approssimare una distribuzione usando MCMC potrebbe essere complicato. L'inferenza variazionale (VI) sta ricevendo una crescente attenzione come alternativa per approssimare la distribuzione a posteriori tramite un problema di ottimo. Tuttavia, VI spesso impone una struttura parametrica alla distribuzione proposta.
Il primo contributo della tesi, detto Hierarchical Variational Inference (HVI), è una metodologia che utilizza reti neurali per creare un'approssimazione semi-parametrica della distribuzione a posteriori. HVI richiede gli stessi requisiti minimi di un Metropolis-Hastings o di un Hamiltonian MCMC, per essere applicata.
Il secondo contributo è un pacchetto Python per l'inferenza variazionale su serie storiche usando modelli media-covarianza. Questo utilizza HVI e tecniche di VI standard combinate con reti neurali. I risultati sperimentali, su dati econometrici e finanziari, mostrano un consistente miglioramento della previsione usando VI, rispetto a stime puntuali dei parametri, in particolare producendo stimatori con minor variabilità.Probabilistic forecasting is a common task. The usual approach assumes a fixed structure for the outcome distribution, often called model, that depends on unseen quantities called parameters. It uses data to infer a reasonable distribution over these latent values. The inference step is not always straightforward, because single-value can lead to poor performances and overfitting while handling a proper distribution with MCMC can be challenging. Variational Inference (VI) is emerging as a viable optimisation based alternative that models the target posterior with instrumental variables called variational parameters. However, VI usually imposes a parametric structure on the proposed posterior. The thesis's first contribution is Hierarchical Variational Inference (HVI) a methodology that uses Neural Networks to create semi-parametric posterior approximations with the same minimum requirements as Metropolis-Hastings or Hamiltonian MCMC. The second contribution is a Python package to conduct VI on time-series models for mean-covariance estimate, using HVI and standard VI techniques combined with Neural Networks. Results on econometric and financial data show a consistent improvement using VI, compared to point estimate, obtaining lower variance forecasting.
26
Ban, Yutong. "Suivi multi-locuteurs avec information audio-visuel pour la perception du robot." Thesis, Université Grenoble Alpes (ComUE), 2019. http://www.theses.fr/2019GREAM017/document.
Анотація:
La perception des robots joue un rôle crucial dans l’interaction homme-robot (HRI). Le système de perception fournit les informations au robot sur l’environnement, ce qui permet au robot de réagir en consequence. Dans un scénario de conversation, un groupe de personnes peut discuter devant le robot et se déplacer librement. Dans de telles situations, les robots sont censés comprendre où sont les gens, ceux qui parlent et de quoi ils parlent. Cette thèse se concentre sur les deux premières questions, à savoir le suivi et la diarisation des locuteurs. Nous utilisons différentes modalités du système de perception du robot pour remplir cet objectif. Comme pour l’humain, l’ouie et la vue sont essentielles pour un robot dans un scénario de conversation. Les progrès de la vision par ordinateur et du traitement audio de la dernière décennie ont révolutionné les capacités de perception des robots. Dans cette thèse, nous développons les contributions suivantes : nous développons d’abord un cadre variationnel bayésien pour suivre plusieurs objets. Le cadre bayésien variationnel fournit des solutions explicites, rendant le processus de suivi très efficace. Cette approche est d’abord appliqué au suivi visuel de plusieurs personnes. Les processus de créations et de destructions sont en adéquation avec le modèle probabiliste proposé pour traiter un nombre variable de personnes. De plus, nous exploitons la complémentarité de la vision et des informations du moteur du robot : d’une part, le mouvement actif du robot peut être intégré au système de suivi visuel pour le stabiliser ; d’autre part, les informations visuelles peuvent être utilisées pour effectuer l’asservissement du moteur. Par la suite, les informations audio et visuelles sont combinées dans le modèle variationnel, pour lisser les trajectoires et déduire le statut acoustique d’une personne : parlant ou silencieux. Pour expérimenter un scenario où l’information visuelle est absente, nous essayons le modèle pour la localisation et le suivi des locuteurs basé sur l’information acoustique uniquement. Les techniques de déréverbération sont d’abord appliquées, dont le résultat est fourni au système de suivi. Enfin, une variante du modèle de suivi des locuteurs basée sur la distribution de von-Mises est proposée, celle-ci étant plus adaptée aux données directionnelles. Toutes les méthodes proposées sont validées sur des bases de données specifiques à chaque applicationRobot perception plays a crucial role in human-robot interaction (HRI). Perception system provides the robot information of the surroundings and enables the robot to give feedbacks. In a conversational scenario, a group of people may chat in front of the robot and move freely. In such situations, robots are expected to understand where are the people, who are speaking, or what are they talking about. This thesis concentrates on answering the first two questions, namely speaker tracking and diarization. We use different modalities of the robot’s perception system to achieve the goal. Like seeing and hearing for a human-being, audio and visual information are the critical cues for a robot in a conversational scenario. The advancement of computer vision and audio processing of the last decade has revolutionized the robot perception abilities. In this thesis, we have the following contributions: we first develop a variational Bayesian framework for tracking multiple objects. The variational Bayesian framework gives closed-form tractable problem solutions, which makes the tracking process efficient. The framework is first applied to visual multiple-person tracking. Birth and death process are built jointly with the framework to deal with the varying number of the people in the scene. Furthermore, we exploit the complementarity of vision and robot motorinformation. On the one hand, the robot’s active motion can be integrated into the visual tracking system to stabilize the tracking. On the other hand, visual information can be used to perform motor servoing. Moreover, audio and visual information are then combined in the variational framework, to estimate the smooth trajectories of speaking people, and to infer the acoustic status of a person- speaking or silent. In addition, we employ the model to acoustic-only speaker localization and tracking. Online dereverberation techniques are first applied then followed by the tracking system. Finally, a variant of the acoustic speaker tracking model based on von-Mises distribution is proposed, which is specifically adapted to directional data. All the proposed methods are validated on datasets according to applications
27
Hsu, Wei-Ning Ph D. Massachusetts Institute of Technology. "Unsupervised learning of disentangled representations for speech with neural variational inference models." Thesis, Massachusetts Institute of Technology, 2018. http://hdl.handle.net/1721.1/118059.
Анотація:
Thesis: S.M., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018.Cataloged from PDF version of thesis.
Includes bibliographical references (pages 121-128).
Despite recent successes in machine learning, artificial intelligence is still far from matching human intelligence in many ways. Two important aspects are transferability and amount of supervision required. Take speech recognition for example: while humans can easily adapt to a new accent without explicit supervision (i.e., ground truth transcripts for speech of a new accent), current machine learning techniques still struggle with such a scenario. We argue that an essential component of human learning is unsupervised or weakly supervised representation learning, which transforms input signals to low dimensional representations that facilitate subsequent structured learning and knowledge acquisition. In this thesis, we develop unsupervised representation learning frameworks for speech data. We start with investigating an existing variational autoencoder (VAE) model for learning latent representations, and derive novel latent space operations for speech transformation. The transformation method is applied to unsupervised domain adaptation problems, which addresses the transferability issues of supervised machine learning framework. We then extend the VAE models, and propose a novel factorized hierarchical variational autoencoder (FHVAE), which better models a generative process of sequential data, and learns not only disentangled, but also interpretable latent representations without any supervision. By leveraging the interpretability, we demonstrate that such representations can be applied to a wide range of tasks, including but not limited to: voice conversion, denoising, speaker verification, speaker invariant phonetic feature extraction, and noise invariant phonetic feature extraction. In the last part of this thesis, we examine scalability issues regarding the original FHVAE training algorithm in terms of runtime, memory, and optimization stability. Based on our analysis, we propose a hierarchical sampling algorithm for training, which enables training of FHVAE models on arbitrarily large datasets.
by Wei-Ning Hsu.
S.M.
28
Steinberg, John. "A Comparative Analysis of Bayesian Nonparametric Variational Inference Algorithms for Speech Recognition." Master's thesis, Temple University Libraries, 2013. http://cdm16002.contentdm.oclc.org/cdm/ref/collection/p245801coll10/id/216605.
Анотація:
Electrical and Computer EngineeringM.S.E.E.
Nonparametric Bayesian models have become increasingly popular in speech recognition tasks such as language and acoustic modeling due to their ability to discover underlying structure in an iterative manner. These methods do not require a priori assumptions about the structure of the data, such as the number of mixture components, and can learn this structure directly. Dirichlet process mixtures (DPMs) are a widely used nonparametric Bayesian method which can be used as priors to determine an optimal number of mixture components and their respective weights in a Gaussian mixture model (GMM). Because DPMs potentially require an infinite number of parameters, inference algorithms are needed to make posterior calculations tractable. The focus of this work is an evaluation of three of these Bayesian variational inference algorithms which have only recently become computationally viable: Accelerated Variational Dirichlet Process Mixtures (AVDPM), Collapsed Variational Stick Breaking (CVSB), and Collapsed Dirichlet Priors (CDP). To eliminate other effects on performance such as language models, a phoneme classification task is chosen to more clearly assess the viability of these algorithms for acoustic modeling. Evaluations were conducted on the CALLHOME English and Mandarin corpora, consisting of two languages that, from a human perspective, are phonologically very different. It is shown in this work that these inference algorithms yield error rates comparable to a baseline Gaussian mixture model (GMM) but with a factor of up to 20 fewer mixture components. AVDPM is shown to be the most attractive choice because it delivers the most compact models and is computationally efficient, enabling its application to big data problems.
Temple University--Theses
29
Xu, Zhen. "Using Social Dynamics to Make Individual Predictions| Variational Inference with Stochastic Kinetic Model." Thesis, State University of New York at Buffalo, 2017. http://pqdtopen.proquest.com/#viewpdf?dispub=10253123.
Анотація:
Social dynamics is concerned with the interactions of individuals and the resulting group behaviors. It models the temporal evolution of social systems via the interactions of the individuals within these systems. The availability of large-scale data in social networks and sensor networks offers an unprecedented opportunity to predict state changing events at the individual level. Examples of such events are disease infection, rumor propagation and opinion transition in elections, etc. Unlike previous research focusing on the collective effects of social systems, we want to make efficient inferences on the individual level.
Two main challenges are addressed: temporal modeling and computational complexity. The interaction pattern for each individual keeps changing over the time, i.e., an individual interacts with different individuals at different times. Second, as the number of tracked individual increases, the computational complexity grows exponentially with traditional sequential data analysis.
The contributions are: (i) leverage social networks and sensor networks data to make tractable inferences on both individual behaviors and collective effects in social dynamics. (ii) use the stochastic kinetic model to summarize dynamic interactions among individuals and simplify the state transition probabilities. (iii) propose an efficient variational inference algorithm whose complexity grows linearly with the number of tracked individuals M. Given the state space K of a single individual and the total number of time steps T, the complexity of naive brute-force approach is O(KMT) and the complexity of existing exact inference approach is O(KMT). In comparison, the complexity of the proposed algorithm is O(K 2MT). In practice, it requires several iterations to converge.
In the empirical study concerning epidemics dynamics, given wireless sensor network data collected from more than ten thousand people (M = 13,888) over three years (T = 3465), we use the proposed algorithm to track disease transmission, and predict the probability of infection for each individual (K = 2) along the time until convergence (I=5). It is more efficient than state of the art sampling methods, i.e., MCMC and particle filter, while achieving high accuracy.
30
OLOBATUYI, KEHINDE IBUKUN. "A Family of Variational Algorithms for Approximate Bayesian Inference of High-Dimensional Data." Doctoral thesis, Università degli Studi di Milano-Bicocca, 2021. http://hdl.handle.net/10281/325856.
Анотація:
L’approccio Bayesiano alle tecniche di machine-learning consente di integrare in un modello le informazioni a priori per evitare problemi di overfitting, cercando di approssimare la distribuzione a posteriori. Fornisce inoltre una metodologia coerente per la scelta fra diversi modelli alternativi, e richiede tipicamente uno sforzo computazionale considerevole, tale da rendere alcuni problemi intrattabili.
Questa tesi propone una famiglia di metodologie di tipo Variational Bayes per approssimare la complessità computazionale dell’approccio Bayesiano tramite l’utilizzo di variabili latenti, minimizzando la distanza di Kullback-Leibler tra la distribuzione a posteriori esatta e quella approssimata. Il primo capitolo riepiloga i concetti chiave dell’inferenza bayesiana e gli algoritmi di propagazione. Il secondo capitolo introduce il metodo Variational Bayes, il quale generalizza gli algoritmi di Expectation Maximization (EM) per la stima dei parametri tramite un approccio a massima verosimiglianza. Vengono inoltre discusse le approssimazioni fattorizzate per i metodi di Expectation Propagation (EP). Nei capitoli da 3 a 5 vengono derivate e testate diverse varianti dei metodi Variational Bayes per la famiglia dei Cluster Weighted Models (CWMs) e, partendo da un breve cenno storico, vengono proposte diverse nuove classi di CWM. Inizialmente viene analizzato il problema della riduzione di dimensionalità nei CWM, introducendo una nuova classe basata su t-distributed stochastic neighbor embedding (tSNE). Nel secondo lavoro viene proposto un Multinomial CWM per la classificazione multinomiale ed un Zero-inflated Poisson CWM per dati di tipo zero-inflazionato. Vengono derivati ed applicati gli algoritmi EM per la stima dei parametri, considerando tre diverse alternative per il passo di massimizzazione: Minimi Quadrati Ordinari (OLS), Minimi Quadrati Pesati Iterati (IRLS), e Discesa Stocastica del Gradiente (SGD). Per concludere, vengono testate le performance classificative dei modelli CWM utilizzando otto criteri diversi e vari Adjusted Rand Index (ARI). Nel sesto capitolo vengono proposte due varianti del metodo di Expectation Propagation per inverse models denominate EP-MCMC e EP-ADMM, applicandole a modelli bayesiani per image-reconstruction e confrontandone le performance con i metodi MCMC. Il settimo capitolo chiude la tesi, traendo le conclusioni dei lavori svolti e riassumendo i possibili sviluppi futuri.The Bayesian framework for machine learning allows the incorporation of prior knowledge into the system in a coherent manner which avoids overfitting problems but rather seeks to approximate the exact posterior and provides a principled basis for the selection of model among alternative models. Unfortunately, the computation required in Bayesian framework is usually intractable. This thesis provides a family of Variational Bayesian (VB) framework which approximates these intractable computations with latent variables by minimizing the Kullback-Leibler divergence between the exact posterior and the approximate distribution. Chapter 1 presents background materials on Bayesian inference, and propagation algorithms. Chapter 2 discusses the family of variational Bayesian theory. It generalizes the expectation maximization (EM) algorithm for learning maximum likelihood parameters. Finally, it discusses factorized approximation of Expectation propagation. Chapter 3 - 5 derive and apply the variants of Variational Bayesian to the family of cluster weighted models (CWMs). It investigates the background history of CWM and proposes new different members into the family. First, the dimensionality of CWM is explored by introducing the t-distributed stochastic neighbor embedding (tSNE) for dimensionality reduction which leads to CMWs based on tSNE for high-dimensional data. Afterwards, we propose a Multinomial CWM for multiclass classification and Zero-inflated Poisson CWM for zero-inflated data. This work derives and applies the Expectation Maximization algorithm with three different maximization step algorithms: Ordinary Least Squares (OLS), Iteratively Reweighted Least Squares (IRLS), and Stochastic Gradient Descent (SGD) to estimate the models' parameters. It finally examines the classification performance of the family of CWM by eight different information criteria and varieties of Adjusted Rand Index (ARI). Chapter 6 proposes a variant of Expectation Propagation: EP-MCMC, EP-ADMM algorithms to the inverse models. It demonstrates EP-MCMC and EP-ADMM on complex Bayesian models for image reconstruction and compares the performance to MCMC. Chapter 7 concludes with a discussion and possible future directions for optimization algorithms.
31
Burchett, Woodrow. "Improving the Computational Efficiency in Bayesian Fitting of Cormack-Jolly-Seber Models with Individual, Continuous, Time-Varying Covariates." UKnowledge, 2017. http://uknowledge.uky.edu/statistics_etds/27.
Анотація:
The extension of the CJS model to include individual, continuous, time-varying covariates relies on the estimation of covariate values on occasions on which individuals were not captured. Fitting this model in a Bayesian framework typically involves the implementation of a Markov chain Monte Carlo (MCMC) algorithm, such as a Gibbs sampler, to sample from the posterior distribution. For large data sets with many missing covariate values that must be estimated, this creates a computational issue, as each iteration of the MCMC algorithm requires sampling from the full conditional distributions of each missing covariate value. This dissertation examines two solutions to address this problem. First, I explore variational Bayesian algorithms, which derive inference from an approximation to the posterior distribution that can be fit quickly in many complex problems. Second, I consider an alternative approximation to the posterior distribution derived by truncating the individual capture histories in order to reduce the number of missing covariates that must be updated during the MCMC sampling algorithm. In both cases, the increased computational efficiency comes at the cost of producing approximate inferences. The variational Bayesian algorithms generally do not estimate the posterior variance very accurately and do not directly address the issues with estimating many missing covariate values. Meanwhile, the truncated CJS model provides a more significant improvement in computational efficiency while inflating the posterior variance as a result of discarding some of the data. Both approaches are evaluated via simulation studies and a large mark-recapture data set consisting of cliff swallow weights and capture histories.
32
Lauretig, Adam M. "Natural Language Processing, Statistical Inference, and American Foreign Policy." The Ohio State University, 2019. http://rave.ohiolink.edu/etdc/view?acc_num=osu1562147711514566.
33
Carbonetto, Peter. "New probabilistic inference algorithms that harness the strengths of variational and Monte Carlo methods." Thesis, University of British Columbia, 2009. http://hdl.handle.net/2429/11990.
Анотація:
The central objective of this thesis is to develop new algorithms for inference in probabilistic graphical models that improve upon the state-of-the-art and lend new insight into the computational nature of probabilistic inference. The four main technical contributions of this thesis are: 1) a new framework for inference in probabilistic models based on stochastic approximation, variational methods and sequential Monte Carlo is proposed that achieves significant improvements in accuracy and reductions in variance over existing Monte Carlo and variational methods, and at a comparable computational expense, 2) for many instances of the proposed approach to probabilistic inference, constraints must be imposed on the parameters, so I describe a new stochastic approximation algorithm that adopts the methodology of primal-dual interior-point methods and handles constrained optimization problems much more robustly than existing approaches, 3) a new class of conditionally-specified variational approximations based on mean field theory is described, which, when combined with sequential Monte Carlo, overcome some of the limitations imposed by conventional variational mean field approximations, and 4) I show how recent advances in variational inference can be used to implement inference and learning in a novel contingently acyclic probabilistic relational model, a model developed for the purpose of making predictions about relationships in a social network.
34
Burleigh, John Gordon. "Variation in the process of molecular evolution and its impact on phylogenetic inference /." free to MU campus, to others for purchase, 2002. http://wwwlib.umi.com/cr/mo/fullcit?p3052155.
35
Tison, Jean-Luc. "Genetic variation and inference of demographic histories in non-model species." Doctoral thesis, Stockholms universitet, Institutionen för molekylär biovetenskap, Wenner-Grens institut, 2014. http://urn.kb.se/resolve?urn=urn:nbn:se:su:diva-109896.
Анотація:
Both long-term environmental changes such as those driven by the glacial cycles and more recent anthropogenic impacts have had major effects on the past demography in wild organisms. Within species, these changes are reflected in the amount and distribution of neutral genetic variation. In this thesis, mitochondrial and microsatellite DNA was analysed to investigate how environmental and anthropogenic factors have affected genetic diversity and structure in four ecologically different animal species. Paper I describes the post-glacial recolonisation history of the speckled-wood butterfly (Pararge aegeria) in Northern Europe. A decrease in genetic diversity with latitude and a marked population structure were uncovered, consistent with a hypothesis of repeated founder events during the postglacial recolonisation. Moreover, Approximate Bayesian Computation analyses indicate that the univoltine populations in Scandinavia and Finland originate from recolonisations along two routes, one on each side of the Baltic. Paper II aimed to investigate how past sea-level rises affected the population history of the convict surgeonfish (Acanthurus triostegus) in the Indo-Pacific. Assessment of the species’ demographic history suggested a population expansion that occurred approximately at the end of the last glaciation. Moreover, the results demonstrated an overall lack of phylogeographic structure, probably due to the high dispersal rates associated with the species’ pelagic larval stage. Populations at the species’ eastern range margin were significantly differentiated from other populations, which likely is a consequence of their geographic isolation. In Paper III, we assessed the effect of human impact on the genetic variation of European moose (Alces alces) in Sweden. Genetic analyses revealed a spatial structure with two genetic clusters, one in northern and one in southern Sweden, which were separated by a narrow transition zone. Moreover, demographic inference suggested a recent population bottleneck. The inferred timing of this bottleneck coincided with a known reduction in population size in the 19th and early 20th century due to high hunting pressure. In Paper IV, we examined the effect of an indirect but well-described human impact, via environmental toxic chemicals (PCBs), on the genetic variation of Eurasian otters (Lutra lutra) in Sweden. Genetic clustering assignment revealed differentiation between otters in northern and southern Sweden, but also in the Stockholm region. ABC analyses indicated a decrease in effective population size in both northern and southern Sweden. Moreover, comparative analyses of historical and contemporary samples demonstrated a more severe decline in genetic diversity in southern Sweden compared to northern Sweden, in agreement with the levels of PCBs found.At the time of the doctoral defense, the following papers were unpublished and had a status as follows: Paper 2: Manuscript. Paper 3: Manuscript.
36
Liu, Qiang. "Inference of Spot Volatility in the presence of Infinite Variation Jumps." Thesis, University of Macau, 2018. http://umaclib3.umac.mo/record=b3952482.
37
Wedenberg, Kim, and Alexander Sjöberg. "Online inference of topics : Implementation of the topic model Latent Dirichlet Allocation using an online variational bayes inference algorithm to sort news articles." Thesis, Uppsala universitet, Institutionen för informationsteknologi, 2014. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-222429.
Анотація:
The client of the project has problems with complex queries and noisewhen querying their stream of five million news articles per day. Thisresults in much manual work when sorting and pruning the search result of their query. Instead of using direct text matching, the approachof the project was to use a topic model to describe articles in terms oftopics covered and to use this new information to sort the articles. An online version of the topic model Latent Dirichlet Allocationwas implemented using online variational Bayes inference to handlestreamed data. Using 100 dimensions, topics such as sports and politics emerged during training on a 1.7 million articles big simulatedstream. These topics were used to sort articles based on context. Theimplementation was found accurate enough to be useful for the client aswell as fast and stable enough to be a feasible solution to the problem.
38
Grullon, Dylan Emanuel Centeno. "Disentangling time constant and time dependent hidden state in time series with variational Bayesian inference." Thesis, Massachusetts Institute of Technology, 2019. https://hdl.handle.net/1721.1/124572.
Анотація:
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2019
Cataloged from student-submitted PDF version of thesis.
Includes bibliographical references (pages 85-86).
In this thesis, we design and explore a new model architecture called a Variational Bayes Recurrent Neural Network (VBRNN) for modelling time series. The VBRNN contains explicit structure to disentangle time constant and time dependent dynamics for use with compatible time series, such as those that can be modelled by differential equations with time constant parameters and time dependent state. The model consists of a Variational Bayes (VB) layer to infer time constant state, as well as a conditioned-RNN to model time dependent dynamics. The VBRNN is explored through various synthetic datasets and problems, and compared to conventional methods on these datasets. This approach demonstrates effective disentanglement, motivating future work to explore the efficacy of this mo del in real word datasets.
by Dylan Emanuel Centeno Grullon.
M. Eng.
M.Eng. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science
39
Wenzel, Florian. "Scalable Inference in Latent Gaussian Process Models." Doctoral thesis, Humboldt-Universität zu Berlin, 2020. http://dx.doi.org/10.18452/20926.
Анотація:
Latente Gauß-Prozess-Modelle (latent Gaussian process models) werden von Wissenschaftlern benutzt, um verborgenen Muster in Daten zu er- kennen, Expertenwissen in probabilistische Modelle einfließen zu lassen und um Vorhersagen über die Zukunft zu treffen. Diese Modelle wurden erfolgreich in vielen Gebieten wie Robotik, Geologie, Genetik und Medizin angewendet. Gauß-Prozesse definieren Verteilungen über Funktionen und können als flexible Bausteine verwendet werden, um aussagekräftige probabilistische Modelle zu entwickeln. Dabei ist die größte Herausforderung, eine geeignete Inferenzmethode zu implementieren. Inferenz in probabilistischen Modellen bedeutet die A-Posteriori-Verteilung der latenten Variablen, gegeben der Daten, zu berechnen. Die meisten interessanten latenten Gauß-Prozess-Modelle haben zurzeit nur begrenzte Anwendungsmöglichkeiten auf großen Datensätzen.
In dieser Doktorarbeit stellen wir eine neue effiziente Inferenzmethode für latente Gauß-Prozess-Modelle vor. Unser neuer Ansatz, den wir augmented variational inference nennen, basiert auf der Idee, eine erweiterte (augmented) Version des Gauß-Prozess-Modells zu betrachten, welche bedingt konjugiert (conditionally conjugate) ist. Wir zeigen, dass Inferenz in dem erweiterten Modell effektiver ist und dass alle Schritte des variational inference Algorithmus in geschlossener Form berechnet werden können, was mit früheren Ansätzen nicht möglich war. Unser neues Inferenzkonzept ermöglicht es, neue latente Gauß-Prozess- Modelle zu studieren, die zu innovativen Ergebnissen im Bereich der Sprachmodellierung, genetischen Assoziationsstudien und Quantifizierung der Unsicherheit in Klassifikationsproblemen führen.Latent Gaussian process (GP) models help scientists to uncover hidden structure in data, express domain knowledge and form predictions about the future. These models have been successfully applied in many domains including robotics, geology, genetics and medicine. A GP defines a distribution over functions and can be used as a flexible building block to develop expressive probabilistic models. The main computational challenge of these models is to make inference about the unobserved latent random variables, that is, computing the posterior distribution given the data. Currently, most interesting Gaussian process models have limited applicability to big data. This thesis develops a new efficient inference approach for latent GP models. Our new inference framework, which we call augmented variational inference, is based on the idea of considering an augmented version of the intractable GP model that renders the model conditionally conjugate. We show that inference in the augmented model is more efficient and, unlike in previous approaches, all updates can be computed in closed form. The ideas around our inference framework facilitate novel latent GP models that lead to new results in language modeling, genetic association studies and uncertainty quantification in classification tasks.
40
Marklund, Emil. "Bayesian inference in aggregated hidden Markov models." Thesis, Uppsala universitet, Institutionen för biologisk grundutbildning, 2015. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-243090.
Анотація:
Single molecule experiments study the kinetics of molecular biological systems. Many such studies generate data that can be described by aggregated hidden Markov models, whereby there is a need of doing inference on such data and models. In this study, model selection in aggregated Hidden Markov models was performed with a criterion of maximum Bayesian evidence. Variational Bayes inference was seen to underestimate the evidence for aggregated model fits. Estimation of the evidence integral by brute force Monte Carlo integration theoretically always converges to the correct value, but it converges in far from tractable time. Nested sampling is a promising method for solving this problem by doing faster Monte Carlo integration, but it was here seen to have difficulties generating uncorrelated samples.
41
Li, Xin. "Haplotype Inference from Pedigree Data and Population Data." Cleveland, Ohio : Case Western Reserve University, 2010. http://rave.ohiolink.edu/etdc/view?acc_num=case1259867573.
Анотація:
Thesis(Ph.D.)--Case Western Reserve University, 2010Title from PDF (viewed on 2009-12-30) Department of Electrical Engineering and Computer Science Includes abstract Includes bibliographical references and appendices Available online via the OhioLINK ETD Center
42
Michelen, Strofer Carlos Alejandro. "Machine Learning and Field Inversion approaches to Data-Driven Turbulence Modeling." Diss., Virginia Tech, 2021. http://hdl.handle.net/10919/103155.
Анотація:
There still is a practical need for improved closure models for the Reynolds-averaged Navier-Stokes (RANS) equations. This dissertation explores two different approaches for using experimental data to provide improved closure for the Reynolds stress tensor field. The first approach uses machine learning to learn a general closure model from data. A novel framework is developed to train deep neural networks using experimental velocity and pressure measurements. The sensitivity of the RANS equations to the Reynolds stress, required for gradient-based training, is obtained by means of both variational and ensemble methods. The second approach is to infer the Reynolds stress field for a flow of interest from limited velocity or pressure measurements of the same flow. Here, this field inversion is done using a Monte Carlo Bayesian procedure and the focus is on improving the inference by enforcing known physical constraints on the inferred Reynolds stress field. To this end, a method for enforcing boundary conditions on the inferred field is presented. The two data-driven approaches explored and improved upon here demonstrate the potential for improved practical RANS predictions.Doctor of Philosophy
The Reynolds-averaged Navier-Stokes (RANS) equations are widely used to simulate fluid flows in engineering applications despite their known inaccuracy in many flows of practical interest. The uncertainty in the RANS equations is known to stem from the Reynolds stress tensor for which no universally applicable turbulence model exists. The computational cost of more accurate methods for fluid flow simulation, however, means RANS simulations will likely continue to be a major tool in engineering applications and there is still a need for improved RANS turbulence modeling. This dissertation explores two different approaches to use available experimental data to improve RANS predictions by improving the uncertain Reynolds stress tensor field. The first approach is using machine learning to learn a data-driven turbulence model from a set of training data. This model can then be applied to predict new flows in place of traditional turbulence models. To this end, this dissertation presents a novel framework for training deep neural networks using experimental measurements of velocity and pressure. When using velocity and pressure data, gradient-based training of the neural network requires the sensitivity of the RANS equations to the learned Reynolds stress. Two different methods, the continuous adjoint and ensemble approximation, are used to obtain the required sensitivity. The second approach explored in this dissertation is field inversion, whereby available data for a flow of interest is used to infer a Reynolds stress field that leads to improved RANS solutions for that same flow. Here, the field inversion is done via the ensemble Kalman inversion (EKI), a Monte Carlo Bayesian procedure, and the focus is on improving the inference by enforcing known physical constraints on the inferred Reynolds stress field. To this end, a method for enforcing boundary conditions on the inferred field is presented. While further development is needed, the two data-driven approaches explored and improved upon here demonstrate the potential for improved practical RANS predictions.
43
Lienart, Thibaut. "Inference on Markov random fields : methods and applications." Thesis, University of Oxford, 2017. http://ora.ox.ac.uk/objects/uuid:3095b14c-98fb-4bda-affc-a1fa1708f628.
Анотація:
This thesis considers the problem of performing inference on undirected graphical models with continuous state spaces. These models represent conditional independence structures that can appear in the context of Bayesian Machine Learning. In the thesis, we focus on computational methods and applications. The aim of the thesis is to demonstrate that the factorisation structure corresponding to the conditional independence structure present in high-dimensional models can be exploited to decrease the computational complexity of inference algorithms. First, we consider the smoothing problem on Hidden Markov Models (HMMs) and discuss novel algorithms that have sub-quadratic computational complexity in the number of particles used. We show they perform on par with existing state-of-the-art algorithms with a quadratic complexity. Further, a novel class of rejection free samplers for graphical models known as the Local Bouncy Particle Sampler (LBPS) is explored and applied on a very large instance of the Probabilistic Matrix Factorisation (PMF) problem. We show the method performs slightly better than Hamiltonian Monte Carlo methods (HMC). It is also the first such practical application of the method to a statistical model with hundreds of thousands of dimensions. In a second part of the thesis, we consider approximate Bayesian inference methods and in particular the Expectation Propagation (EP) algorithm. We show it can be applied as the backbone of a novel distributed Bayesian inference mechanism. Further, we discuss novel variants of the EP algorithms and show that a specific type of update mechanism, analogous to the mirror descent algorithm outperforms all existing variants and is robust to Monte Carlo noise. Lastly, we show that EP can be used to help the Particle Belief Propagation (PBP) algorithm in order to form cheap and adaptive proposals and significantly outperform classical PBP.
44
Saha, Abhijoy. "A Geometric Framework for Modeling and Inference using the Nonparametric Fisher–Rao metric." The Ohio State University, 2019. http://rave.ohiolink.edu/etdc/view?acc_num=osu1562679374833421.
45
Ashton, Gregory. "Timing variations in neutron stars : models, inference and their implications for gravitational waves." Thesis, University of Southampton, 2016. https://eprints.soton.ac.uk/401822/.
Анотація:
Timing variations in pulsars, low frequency ubiquitous structure known as timing noise and sudden increases in the rotational frequency which we call glitches, provide a means to study neutron stars. Since the first observations, many models have been proposed, yet no definitive explanation has arisen. In this thesis, we aim to improve this situation by developing models of timing noise. We focus chiefly on precession models which explain periodic modulation seen in radio pulsar data. Developing models and testing them provides an opportunity to infer the elemental properties of neutron stars: evidence for long period precession has implications for the superfluid component predicted by models used to explain glitches. However, often more than one model can qualitatively explain the data, therefore we need a method to decide which model best fits the data. This is precisely the case for PSR B1828-11 which has been used as evidence for both precession and so-called magnetospheric switching. We address this confusion by applying the tools of probability theory to develop a Bayesian model comparison and find that the evidence is in favour of precession. In the second part of this thesis, we will discuss the implications of timing variations for the detection of continuous gravitational waves from neutron stars. To search for these signals, matched filtering methods are used which require a template, a guess for what the signal ‘looks like’. Timing variations, as seen in the electromagnetic signal, may also exist in the gravitational wave signal. If detected, these could provide an invaluable source of information about neutron stars. However, if not included in the template, they may mean that the gravitational wave signal is not detected in the first place. We investigate this issue for both timing noise and glitches, using electromagnetic observations to predict for what types of gravitational wave searches this may be an issue. We find that while timing noise is unlikely to be an issue for current gravitational wave searches, glitches may cause a significant problem in all-sky searches for gravitational waves from neutron stars.
46
Tong, Zhigang. "Statistical Inference for Heavy Tailed Time Series and Vectors." Thesis, Université d'Ottawa / University of Ottawa, 2017. http://hdl.handle.net/10393/35649.
Анотація:
In this thesis we deal with statistical inference related to extreme value phenomena.
Specifically, if X is a random vector with values in d-dimensional space, our goal is
to estimate moments of ψ(X) for a suitably chosen function ψ when the magnitude
of X is big. We employ the powerful tool of regular variation for random variables,
random vectors and time series to formally define the limiting quantities of interests
and construct the estimators. We focus on three statistical estimation problems: (i)
multivariate tail estimation for regularly varying random vectors, (ii) extremogram
estimation for regularly varying time series, (iii) estimation of the expected shortfall
given an extreme component under a conditional extreme value model. We establish asymptotic normality of estimators for each of the estimation problems. The theoretical findings are supported by simulation studies and the estimation procedures are applied to some financial data.
47
Cheema, Prasad. "Machine Learning for Inverse Structural-Dynamical Problems: From Bayesian Non-Parametrics, to Variational Inference, and Chaos Surrogates." Thesis, University of Sydney, 2020. https://hdl.handle.net/2123/24139.
Анотація:
To ensure that the design of a structure is both robust and efficient, engineers often investigate inverse dynamical modeling problems. In particular, there are three archetypal inverse modeling problems which arise in the context of structural engineering. These are respectively: (i) The eigenvalue assignment problem, (ii) Bayesian model updating, and (iii) Operational modal analysis. It is the intent of this dissertation to investigate all three aforementioned inverse dynamical problems within the broader context of modern machine learning advancements.
Firstly, the inverse eigenvalue assignment problem will be investigated via performing eigenvalue placement with respect to several different mass-spring systems. It will be shown that flexible, and robust inverse design analysis is possible by appealing to black box variational methods. Secondly, stochastic model updating will be explored via an in-house, physical T-tail structure. This will be addressed through the careful consideration of polynomial chaos theory, and Bayesian model updating, as a means to rapidly quantify structural uncertainties, and perform model updating between a finite element simulation, and the physical structure. Finally, the monitoring phase of a structure often represents an important and unique challenge for engineers. This dissertation will explore the notion of operational modal analysis for a cable-stayed bridge, by building upon a Bayesian non-parametric approach. This will be shown to circumvent the need for many classic thresholds, factors, and parameters which have often hindered analysis in this area.
Ultimately, this dissertation is written with the express purpose of critically assessing modern machine learning algorithms in the context of some archetypal inverse dynamical modeling problems. It is therefore hoped that this dissertation will act as a point of reference, and inspiration for further work, and future engineers in this area.
48
Miao, Yishu. "Deep generative models for natural language processing." Thesis, University of Oxford, 2017. http://ora.ox.ac.uk/objects/uuid:e4e1f1f9-e507-4754-a0ab-0246f1e1e258.
Анотація:
Deep generative models are essential to Natural Language Processing (NLP) due to their outstanding ability to use unlabelled data, to incorporate abundant linguistic features, and to learn interpretable dependencies among data. As the structure becomes deeper and more complex, having an effective and efficient inference method becomes increasingly important. In this thesis, neural variational inference is applied to carry out inference for deep generative models. While traditional variational methods derive an analytic approximation for the intractable distributions over latent variables, here we construct an inference network conditioned on the discrete text input to provide the variational distribution. The powerful neural networks are able to approximate complicated non-linear distributions and grant the possibilities for more interesting and complicated generative models. Therefore, we develop the potential of neural variational inference and apply it to a variety of models for NLP with continuous or discrete latent variables. This thesis is divided into three parts. Part I introduces a generic variational inference framework for generative and conditional models of text. For continuous or discrete latent variables, we apply a continuous reparameterisation trick or the REINFORCE algorithm to build low-variance gradient estimators. To further explore Bayesian non-parametrics in deep neural networks, we propose a family of neural networks that parameterise categorical distributions with continuous latent variables. Using the stick-breaking construction, an unbounded categorical distribution is incorporated into our deep generative models which can be optimised by stochastic gradient back-propagation with a continuous reparameterisation. Part II explores continuous latent variable models for NLP. Chapter 3 discusses the Neural Variational Document Model (NVDM): an unsupervised generative model of text which aims to extract a continuous semantic latent variable for each document. In Chapter 4, the neural topic models modify the neural document models by parameterising categorical distributions with continuous latent variables, where the topics are explicitly modelled by discrete latent variables. The models are further extended to neural unbounded topic models with the help of stick-breaking construction, and a truncation-free variational inference method is proposed based on a Recurrent Stick-breaking construction (RSB). Chapter 5 describes the Neural Answer Selection Model (NASM) for learning a latent stochastic attention mechanism to model the semantics of question-answer pairs and predict their relatedness. Part III discusses discrete latent variable models. Chapter 6 introduces latent sentence compression models. The Auto-encoding Sentence Compression Model (ASC), as a discrete variational auto-encoder, generates a sentence by a sequence of discrete latent variables representing explicit words. The Forced Attention Sentence Compression Model (FSC) incorporates a combined pointer network biased towards the usage of words from source sentence, which significantly improves the performance when jointly trained with the ASC model in a semi-supervised learning fashion. Chapter 7 describes the Latent Intention Dialogue Models (LIDM) that employ a discrete latent variable to learn underlying dialogue intentions. Additionally, the latent intentions can be interpreted as actions guiding the generation of machine responses, which could be further refined autonomously by reinforcement learning. Finally, Chapter 8 summarizes our findings and directions for future work.
49
Shringarpure, Suyash. "Statistical Methods for studying Genetic Variation in Populations." Research Showcase @ CMU, 2012. http://repository.cmu.edu/dissertations/117.
Анотація:
The study of genetic variation in populations is of great interest for the study of the evolutionary history of humans and other species. Improvement in sequencing technology has resulted in the availability of many large datasets of genetic data. Computational methods have therefore become quite important in analyzing these data. Two important problems that have been studied using genetic data are population stratification (modeling individual ancestry with respect to ancestral populations) and genetic association (finding genetic polymorphisms that affect a trait). In this thesis, we develop methods to improve our understanding of these two problems.
For the population stratification problem, we develop hierarchical Bayesian models that incorporate the evolutionary processes that are known to affect genetic variation. By developing mStruct, we show that modeling more evolutionary processes improves the accuracy of the recovered population structure. We demonstrate how nonparametric Bayesian processes can be used to address the question of choosing the optimal number of ancestral populations that describe the genetic diversity of a given sample of individuals. We also examine how sampling bias in genotyping study design can affect results of population structure analysis and propose a probabilistic framework for modeling and correcting sample selection bias.
Genome-wide association studies (GWAS) have vastly improved our understanding of many diseases. However, such studies have failed to uncover much of the variation responsible for a number of common multi-factorial diseases and complex traits. We show how artificial selection experiments on model organisms can be used to better understand the nature of genetic associations. We demonstrate using simulations that using data from artificial selection experiments improves the performance of conventional methods of performing association. We also validate our approach using semi-simulated data from an artificial selection experiment on Drosophila Melanogaster.
50
Nguyen, Trong Nghia. "Deep Learning Based Statistical Models for Business and Financial Data." Thesis, The University of Sydney, 2021. https://hdl.handle.net/2123/26944.
Анотація:
We investigate a wide range of statistical models commonly used in many business and financial econometrics applications and propose flexible ways to combine these highly interpretable models with powerful predictive models in the deep learning literature to leverage the advantages and compensate the disadvantages of each of the modelling approaches. Our approaches of utilizing deep learning techniques for financial data are different from the recently proposed deep learning-based models in the financial econometrics literature in several perspectives. First, we do not overlook well-established structures that have been successfully used in statistical modelling. We flexibly incorporate deep learning techniques to the statistical models to capture the data effects that cannot be explained by the simple linear components of those models. Our proposed modelling frameworks therefore normally include two components: a linear part to explain linear dependencies and a deep learning-based part to capture data effects rather than linearity possibly exhibited in the underlying process. Second, we do not use the neural network structures in the same fashion as they are implemented in the deep learning literature but modify those black-box methods to make them more explainable and hence improve the interpretability of the proposed models. As the results, our hybrid models not only perform better than the pure deep learning techniques in term of interpretation but also often produce more accurate out-of-sample forecasts than the counterpart statistical frameworks. Third, we propose advanced Bayesian inference methodologies to efficiently quantify the uncertainty about the model estimation and prediction. For the proposed high dimensional deep learning-based models, performing efficient Bayesian inference is extremely challenging and is often ignored in the engineer-oriented papers, which generally prefer the frequentist estimation approaches mainly due to the simplicity.