Дисертації з теми "Hierarchical Bayesian Priors"

Several recent empirical studies, particularly in the regional economic growth literature, emphasize the importance of explicitly accounting for uncertainty surrounding model specification. Standard approaches to deal with the problem of model uncertainty involve the use of Bayesian model-averaging techniques. However, Bayesian model-averaging for spatial autoregressive models suffers from severe drawbacks both in terms of computational time and possible extensions to more flexible econometric frameworks. To alleviate these problems, this paper presents two global-local shrinkage priors in the context of high-dimensional matrix exponential spatial specifications. A simulation study is conducted to evaluate the performance of the shrinkage priors. Results suggest that they perform particularly well in high-dimensional environments, especially when the number of parameters to estimate exceeds the number of observations. Moreover, we use pan-European regional economic growth data to illustrate the performance of the proposed shrinkage priors.

Shrinkage for time-varying parameter (TVP) models is investigated within a Bayesian framework, with the aim to automatically reduce time-varying Parameters to staticones, if the model is overfitting. This is achieved through placing the double gamma shrinkage prior on the process variances. An efficient Markov chain Monte Carlo scheme is devel- oped, exploiting boosting based on the ancillarity-sufficiency interweaving strategy. The method is applicable both to TVP models for univariate a swell as multivariate time series. Applications include a TVP generalized Phillips curve for EU area inflation modeling and a multivariate TVP Cholesky stochastic volatility model for joint modeling of the Returns from the DAX-30index.

We assess the relationship between model size and complexity in the time-varying parameter VAR framework via thorough predictive exercises for the Euro Area, the United Kingdom and the United States. It turns out that sophisticated dynamics through drifting coefficients are important in small data sets while simpler models tend to perform better in sizeable data sets. To combine best of both worlds, novel shrinkage priors help to mitigate the curse of dimensionality, resulting in competitive forecasts for all scenarios considered. Furthermore, we discuss dynamic model selection to improve upon the best performing individual model for each point in time.
Series: Department of Economics Working Paper Series

This thesis explores and proposes methods to improve species distribution models. Throughout this thesis, a rich class of statistical modelling techniques has been developed to address crucial and interesting issues related to the data input into these models. The overall contribution of this research is the advancement of knowledge on species distribution modelling through an increased understanding of extraneous zeros, quality of the ecological data, variable selection that incorporates ecological theory and evaluating performance of the fitted models. Though motivated by the challenge of species distribution modelling from ecology, this research is broadly relevant to many fields, including bio-security and medicine. Specifically, this research is of potential significance to researchers seeking to: identify and explain extraneous zeros; assess the quality of their data; or employ expert-informed variable selection.

We implement techniques of small area estimation (SAE) to study consumption, a welfare indicator, which is used to assess poverty in the 2003-2004 Nepal Living Standards Survey (NLSS-II) and the 2001 census. NLSS-II has detailed information of consumption, but it can give estimates only at stratum level or higher. While population variables are available for all households in the census, they do not include the information on consumption; the survey has the `population' variables nonetheless. We combine these two sets of data to provide estimates of poverty indicators (incidence, gap and severity) for small areas (wards, village development committees and districts). Consumption is the aggregate of all food and all non-food items consumed. In the welfare survey the responders are asked to recall all information about consumptions throughout the reference year. Therefore, such data are likely to be noisy, possibly due to response errors or recalling errors. The consumption variable is continuous and positively skewed, so a statistician might use a logarithmic transformation, which can reduce skewness and help meet the normality assumption required for model building. However, it could be problematic since back transformation may produce inaccurate estimates and there are difficulties in interpretations. Without using the logarithmic transformation, we develop hierarchical Bayesian models to link the survey to the census. In our models for consumption, we incorporate the `population' variables as covariates. First, we assume that consumption is noiseless, and it is modeled using three scenarios: the exponential distribution, the gamma distribution and the generalized gamma distribution. Second, we assume that consumption is noisy, and we fit the generalized beta distribution of the second kind (GB2) to consumption. We consider three more scenarios of GB2: a mixture of exponential and gamma distributions, a mixture of two gamma distributions, and a mixture of two generalized gamma distributions. We note that there are difficulties in fitting the models for noisy responses because these models have non-identifiable parameters. For each scenario, after fitting two hierarchical Bayesian models (with and without area effects), we show how to select the most plausible model and we perform a Bayesian data analysis on Nepal's poverty data. We show how to predict the poverty indicators for all wards, village development committees and districts of Nepal (a big data problem) by combining the survey data with the census. This is a computationally intensive problem because Nepal has about four million households with about four thousand households in the survey and there is no record linkage between households in the survey and the census. Finally, we perform empirical studies to assess the quality of our survey-census procedure.

In the recent years the challenge for new prior specifications and for complex hierarchical models became even more relevant in Bayesian inference. The advent of the Markov Chain Monte Carlo techniques, along with new probabilistic programming languages and new algorithms, extended the boundaries of the field, both in theoretical and applied directions. In the present thesis, we address theoretical and applied tasks. In the first part we propose a new class of prior distributions which might depend on the data and specified as a mixture between a noninformative and an informative prior. The generic prior belonging to this class provides less information than an informative prior and is more likely to not dominate the inference when the data size is small or moderate. Such a distribution is well suited for robustness tasks, especially in case of informative prior misspecification. Simulation studies within the conjugate models show that this proposal may be convenient for reducing the mean squared errors and improving the frequentist coverage. Furthermore, under mild conditions this class of distributions yields some other nice theoretical properties. In the second part of the thesis we use hierarchical Bayesian models for predicting some soccer quantities and we extend the usual match goals’ modeling strategy by including the bookmakers’ information directly in the model. Posterior predictive checks on in-sample and out-of sample data show an excellent model fit, a good model calibration and, ultimately, the possibility for building efficient betting strategies.
Negli ultimi anni la sfida per la specificazione di nuove distribuzioni a priori e per l’uso di complessi modelli gerarchici è diventata ancora più rilevante all’interno dell’inferenza Bayesiana. L’avvento delle tecniche Markov Chain Monte Carlo, insieme a nuovi linguaggi di programmazione probabilistici, ha esteso i confini del campo, sia in direzione teorica che applicata. Nella presente tesi ci dedichiamo a obiettivi teorici e applicati. Nella prima parte proponiamo una nuova classe di distribuzioni a priori che dipendono dai dati e che sono specificate tramite una mistura tra una a priori non informativa e una a priori informativa. La generica distribuzione appartenente a questa nuova classe fornisce meno informazione di una priori informativa e si candida a non dominare le conclusioni inferenziali quando la dimensione campionaria è piccola o moderata. Tale distribuzione `e idonea per scopi di robustezza, specialmente in caso di scorretta specificazione della distribuzione a priori informativa. Alcuni studi di simulazione all’interno di modelli coniugati mostrano che questa proposta può essere conveniente per ridurre gli errori quadratici medi e per migliorare la copertura frequentista. Inoltre, sotto condizioni non restrittive, questa classe di distribuzioni d`a luogo ad alcune altre interessanti proprietà teoriche. Nella seconda parte della tesi usiamo la classe dei modelli gerarchici Bayesiani per prevedere alcune grandezze relative al gioco del calcio ed estendiamo l’usuale modellazione per i goal includendo nel modello un’ulteriore informazione proveniente dalle case di scommesse. Strumenti per sondare a posteriori la bontà di adattamento del modello ai dati mettono in luce un’ottima aderenza del modello ai dati in possesso, una buona calibrazione dello stesso e suggeriscono, infine, la costruzione di efficienti strategie di scommesse per dati futuri.

We considered a non-centered parameterization of the standard random-effects model, which is based on the Cholesky decomposition of the variance-covariance matrix. The regression type structure of the non-centered parameterization allows to choose a simple, conditionally conjugate normal prior on the Cholesky factor. Based on the non-centered parameterization, we search for a parsimonious variance-covariance matrix by identifying the non-zero elements of the Cholesky factors using Bayesian variable selection methods. With this method we are able to learn from the data for each effect, whether it is random or not, and whether covariances among random effects are zero or not. An application in marketing shows a substantial reduction of the number of free elements of the variance-covariance matrix. (author's abstract)
Series: Research Report Series / Department of Statistics and Mathematics

Vector autoregressive (VAR) models are frequently used for forecasting and impulse response analysis. For both applications, shrinkage priors can help improving inference. In this paper we derive the shrinkage prior of Griffin et al. (2010) for the VAR case and its relevant conditional posterior distributions. This framework imposes a set of normally distributed priors on the autoregressive coefficients and the covariances of the VAR along with Gamma priors on a set of local and global prior scaling parameters. This prior setup is then generalized by introducing another layer of shrinkage with scaling parameters that push certain regions of the parameter space to zero. A simulation exercise shows that the proposed framework yields more precise estimates of the model parameters and impulse response functions. In addition, a forecasting exercise applied to US data shows that the proposed prior outperforms other specifications in terms of point and density predictions. (authors' abstract)
Series: Department of Economics Working Paper Series

This study demonstrates a fully Bayesian approach to multilevel/hierarchical linear growth modeling using freely available software. Further, the study incorporates informative prior distributions for fixed effect estimates using an objective approach. The objective approach uses previous sample results to form prior distributions included in subsequent samples analyses, a process referred to as Bayesian updating. Further, a method for model checking is outlined based on fit indices including information criteria (i.e., Akaike information criterion, Bayesian information criterion, and deviance information criterion) and approximate Bayes factor calculations. For this demonstration, five distinct samples of schools in the process of implementing School-Wide Positive Behavior Interventions and Supports (SWPBIS) collected from 2008 to 2013 were used with the unit of analysis being the school. First, the within-year SWPBIS fidelity growth was modeled as a function of time measured in months from initial measurement occasion. Uninformative priors were used to estimate growth parameters for the 2008-09 sample, and both uninformative and informative priors based on previous years' samples were used to model data from the 2009-10, 2010-11, 2011-12, 2012-13 samples. Bayesian estimates were also compared to maximum likelihood (ML) estimates, and reliability information is provided. Second, an additional three examples demonstrated how to include predictors into the growth model with demonstrations for: (a) the inclusion of one school-level predictor (years implementing) of SWPBIS fidelity growth, (b) several school-level predictors (relative socio-economic status, size, and geographic location), and (c) school and district predictors (sustainability factors hypothesized to be related to implementation processes) in a three-level growth model. Interestingly, Bayesian models estimated with informative prior distributions in all cases resulted in more optimal fit indices than models estimated with uninformative prior distributions.

L'estimation des abondances de population est essentielle pour comprendre les dynamiques de population, les interactions entre espèces et estimer les risques de transmission d'agents pathogènes dans les populations. Plusieurs méthodes d'échantillonnages, basées sur des hypothèses spécifiques permettent d'estimer ces abondances : les méthodes par comptages uniques, par « distance sampling », par échantillonnages successifs ou par capture marquage recapture. Nous nous sommes intéressés à l'abondance des tiques Ixodes ricinus, vecteurs de nombreux agents pathogènes. Cette abondance est classiquement estimée par le nombre de tiques capturées lors d'échantillonnages uniques réalisés sur différentes unités d'observation. Cependant, de nombreuses études remettent en cause cette hypothèse forte et suggèrent que le taux d'échantillonnage est variable selon les conditions d'échantillonnage (type de végétation,…) mais ne prennent pas en compte ce taux d'échantillonnage pour autant. A partir d'une méthode d'échantillonnage par « removal sampling » (RS), (i) nous avons montré que les conditions environnementales influençaient le taux d'échantillonnage et l'indicateur d'abondance usuel i.e. le nombre de tiques capturées lors d'un seul échantillonnage (ii) nous avons proposé une méthode pour détecter l'indicateur d'abondance, basés sur le nombre cumulé de capture, le moins soumis aux variations du taux ; (iii) par une approche Bayésienne hiérarchique, nous avons estimé simultanément l'abondance de tiques des unités d'observation et la valeur du taux d'échantillonnage en fonction du type de végétation et de l'heure d'échantillonnage. Nous avons montré que le taux d'échantillonnage sur des arbustes (entre 33,9 % et 47,4%) était significativement inférieur au taux d'échantillonnage sur des feuilles mortes (entre 53,6 % et 66,7%). De plus, nous avons montré que le modèle RS tend vers un modèle de Poisson iid lorsque la taille de la population N0 tend vers l'infini ce qui pose des problèmes d'indétermination pour estimer les paramètres N0 et τ, le taux d'échantillonnage. Nous avons également montré que (i) les estimateurs Bayésiens divergent lorsque les lois a priori sont des lois vagues ; (ii) les lois a priori β(a, b) avec a > 2 sur τ conduisaient à des estimateurs Bayésien convergents. Enfin, nous avons proposé des recommandations quant au choix des lois a priori pour τ afin d'obtenir de bonnes estimations pour N0 ou pour τ. Nous discutons de la pertinence des méthodes RS pour les tiques et des perspectives envisageables pour (i) estimer le risque acarologique représenté par la population de tiques potentiellement actives sur une unité d'observation, (ii) estimer un risque à l'échelle d'une parcelle, à savoir comment répartir l'effort d'échantillonnage entre le nombre d'unités d'observation et le nombre d'échantillonnages successifs par unités d'observation
The estimation of animal abundance is essential to understand population dynamics, species interactions and disease patterns in populations and to estimate the risk of pathogens transmission. Several sampling methods such as single counts, distance sampling, removal sampling or capture mark recapture could be used to estimate abundance. In this study, we are investigated the abundance of Ixodes ricinus ticks, which are involved in the transmission of many pathogens. Tick abundance is commonly estimated by the number of nymphs captured during a single observation (a cloth dragged on a given surface). In this case, analyses of abundance patterns assumes that the probability of detecting a tick, hence the sampling rate, remains constant across the observations. In practice, however, this assumption is often not satisfied as the sampling rate may fluctuate between observation plots. The variation of sampling rate is never taken into account in estimations of tick abundance. Using a removal sampling design (RS), (i) we showed that the sampling rate and the usual abundance indicator (based on a single drag observation per spot) were both influenced by environmental conditions ; (ii) we proposed a method to determine the abundance indicator the least influenced by sampling rate variations ; (iii) using a hierarchical Bayesian model, we estimated simultaneously the abundance and the sampling rate according the type of vegetation, and the time of sampling. The sampling rate varied between 33,9 % and 47,4 % for shrubs and 53,6 % and 66,7 % for dead leaves. In addition, we show that the RS model tends to Poisson iid model when the population size N0 tends to infinite. This result conduct to infinite estimations for N0. We show that (i) Bayesian estimators were divergent for vague prior ; (ii) β(a, b) prior for a > 2 on τ conduct to convergent estimators. Then, we proposed recommendations for prior choice for τ parameter to give good estimations of N0 or τ. We discuss the relevance of RS for ticks and the possible perspectives to (i) estimate the acarologic risk associated to all potential active ticks for given spot, (ii) estimate the risk at the larger scale, i.e. how to distribute the sampling effort between number of spot and number of consecutive sampling by spot

Cette thèse s’intéresse au potentiel et à la prévision des temps d’attente concernant le covoiturage sur un territoire en utilisant des méthodes d’apprentissage statistique. Cinq thèmes principaux sont abordés dans le présent manuscrit. Le premier présente des techniques de régression quantile afin de prédire des temps d’attente. Le deuxième détaille la construction d’un processus de travail empruntant des outils des Systèmes d’Information Géographique (SIG) afin d’exploiter pleinement les données issues du covoiturage. Dans un troisième temps nous construisons un modèle hiérarchique bayésien en vue de prédire des flux de trafic et des temps d’attente. En quatrième partie nous proposons une méthode de construction d’une loi a priori informative par transfert bayésien dans le but d’améliorer les prédictions des temps d’attente pour une situation de jeu de données court. Enfin, le dernier thème se concentre sur la mise en production et l’exploitation industrielle du modèle hiérarchique bayésien
This thesis focuses on the potential and prediction of carpooling waiting times in a territory using statistical learning methods. Five main themes are covered in this manuscript. The first presents quantile regression techniques to predict waiting times. The second details the construction of a workflow based on Geographic Information Systems (GIS) tools in order to fully leverage the carpooling data. In a third part we develop a hierarchical bayesian model in order to predict traffic flows and waiting times. In the fourth part, we propose a methodology for constructing an informative prior by bayesian transfer to improve the prediction of waiting times for a short dataset situation. Lastly, the final theme focuses on the production and industrial exploitation of the bayesian hierarchical model

Many prior distributions are suggested for variance parameters in the hierarchical model. The “Non-informative” interval of the conjugate inverse-gamma prior might cause problems. I consider three priors – conjugate inverse-gamma, log-normal and truncated normal for the variance parameters and do the numerical analysis on Gelman’s 8-schools data. Then with the posterior draws, I compare the Bayesian credible intervals of parameters using the three priors. I use predictive distributions to do predictions and then discuss the differences of the three priors suggested.
text

Hierarchical models are suitable and very natural to model many real life phenomena, where data arise in nested fashion. The use of Bayesian approach to hierarchical models has numerous advantages over the classical approach. Estimating a phenomenon with hierarchical model can be viewed as a smoothing problem, and hence while summarizing such a phenomenon via hierarchical model we do not want to undersmooth the phenomenon. That is, in most of the practical applications undersmoothing is more serious type of error than oversmoothing. So, we need an estimation approach which can guard against undersmoothing. If we can control the undersmoothing reasonably, we may get a better calibrated summary of the phenomenon we estimate. In this study, we have incorporated the aspect of smoothing in estimating the parameters of Bayesian hierarchical models. In doing so we have proposed a conservative prior for the variance component to achieve the adequate degree of smoothness while estimating the phenomenon under study. We have conducted simulation studies to decide about the appropriate values to be used for the hyperparameter while using the conservative prior to ensure the adequate degree of smoothness. We have investigated the performance of the proposed conservative prior in guarding against undersmoothing in simple normal-normal hierarchical models (random effects models for normal response) and in non-parametric regression curve estimation problem via simulation studies. We have also investigated the performance of the proposed prior compared to those of the uniform shrinkage prior and the Jeffreys' prior, with respect to both guarding against undersmoothing and the MSE of the estimated model parameters, through simulation studies.

With the development of modern data collection approaches, researchers may collect hundreds to millions of variables, yet may not need to utilize all explanatory variables available in predictive models. Hence, choosing models that consist of a subset of variables often becomes a crucial step. In linear regression, variable selection not only reduces model complexity, but also prevents over-fitting. From a Bayesian perspective, prior specification of model parameters plays an important role in model selection as well as parameter estimation, and often prevents over-fitting through shrinkage and model averaging.

We develop two novel hierarchical priors for selection and model averaging, for Generalized Linear Models (GLMs) and normal linear regression, respectively. They can be considered as "spike-and-slab" prior distributions or more appropriately "spike- and-bell" distributions. Under these priors we achieve dimension reduction, since their point masses at zero allow predictors to be excluded with positive posterior probability. In addition, these hierarchical priors have heavy tails to provide robust- ness when MLE's are far from zero.

Zellner's g-prior is widely used in linear models. It preserves correlation structure among predictors in its prior covariance, and yields closed-form marginal likelihoods which leads to huge computational savings by avoiding sampling in the parameter space. Mixtures of g-priors avoid fixing g in advance, and can resolve consistency problems that arise with fixed g. For GLMs, we show that the mixture of g-priors using a Compound Confluent Hypergeometric distribution unifies existing choices in the literature and maintains their good properties such as tractable (approximate) marginal likelihoods and asymptotic consistency for model selection and parameter estimation under specific values of the hyper parameters.

While the g-prior is invariant under rotation within a model, a potential problem with the g-prior is that it inherits the instability of ordinary least squares (OLS) estimates when predictors are highly correlated. We build a hierarchical prior based on scale mixtures of independent normals, which incorporates invariance under rotations within models like ridge regression and the g-prior, but has heavy tails like the Zeller-Siow Cauchy prior. We find this method out-performs the gold standard mixture of g-priors and other methods in the case of highly correlated predictors in Gaussian linear models. We incorporate a non-parametric structure, the Dirichlet Process (DP) as a hyper prior, to allow more flexibility and adaptivity to the data.

Dissertation

Abstract In recent years, deep neural networks have achieved extraordinary performance on supervised learning tasks. Convolutional neural networks (CNN) have vastly improved the state of the art for most computer vision tasks including object recognition and segmentation. However, their success relies on the presence of a large amount of labeled data. In contrast, relatively fewer work has been done in deep learning to handle scenarios when access to ground truth is limited, partial or completely absent. In this thesis, we propose models to handle challenging problems with limited labeled information. Our first contribution is a neural architecture that allows for the extraction of infinitely many features from an object while allowing for tractable inference. This is achieved by using the `kernel trick', that is, we express the inner product in the infinite dimensional feature space as a kernel. The kernel can either be computed exactly for single layer feedforward networks, or approximated by an iterative algorithm for deep convolutional networks. The corresponding models are referred to as stretched deep networks (SDN). We show that when the amount of training data is limited, SDNs with random weights drastically outperform fully supervised CNNs with similar architectures. While SDNs perform reasonably well for classification with limited labeled data, they can not utilize unlabeled data which is often much easier to obtain. A common approach to utilize unlabeled data is to couple the classifier with an autoencoder (or its variants) thereby minimizing reconstruction error in addition to the classification error. We discuss the limitations of decoder based architectures and propose a model that allows for the utilization of unlabeled data without the need of a decoder. This is achieved by jointly modeling the distribution of data and latent features in a manner that explicitly assigns zero probability to unobserved data. The joint probability of the data and the latent features is maximized using a two-step EM-like procedure. Depending on the task, we allow the latent features to be one-hot or real-valued vectors and define a suitable prior on the features. For instance, one-hot features correspond to class labels and are directly used for the unsupervised and semi-supervised classification tasks. For real-valued features, we use hierarchical Bayesian models as priors over the latent features. Hence, the proposed model, which we refer to as discriminative encoder (or DisCoder), is flexible in the type of latent features that it can capture. The proposed model achieves state-of-the-art performance on several challenging datasets. Having addressed the problem of utilizing unlabeled data for classification, we move to a domain where obtaining labels is a lot more expensive, that is, semantic segmentation of images. Explicitly labeling each pixel of an image with the object that the pixel belongs to, is an expensive operation, in terms of time as well as effort? Currently, only a few classes of images have been densely (pixel-level) labeled. Even among these classes, only a few images per class have pixel-level supervision. Models that rely on densely-labeled images, cannot utilize a much larger set of weakly annotated images available on the web. Moreover, these models cannot learn the segmentation masks for new classes, where there is no densely labeled data. Hence, we propose a model for utilizing weakly-labeled data for semantic segmentation of images. This is achieved by generating fake labels for each image, while simultaneously forcing the output of the CNN to satisfy the mean-field constraints imposed by a conditional random field. We show that one can enforce the CRF constraints by forcing the distribution at each pixel to be close to the distribution of its neighbors. The proposed model is very fast to train and achieves state-of-the-art performance on the popular VOC-2012 dataset for the task of weakly supervised semantic segmentation of images.

We develop statistical methods for tackling two important problems in genetic association studies. First, we propose a Bayesian approach to overcome the winner's curse in genetic studies. Second, we consider a Bayesian latent variable model for analyzing longitudinal family data with pleiotropic phenotypes. Winner's curse in genetic association studies refers to the estimation bias of the reported odds ratios (OR) for an associated genetic variant from the initial discovery samples. It is a consequence of the sequential procedure in which the estimated effect of an associated genetic marker must first pass a stringent significance threshold. We propose a hierarchical Bayes method in which a spike-and-slab prior is used to account for the possibility that the significant test result may be due to chance. We examine the robustness of the method using different priors corresponding to different degrees of confidence in the testing results and propose a Bayesian model averaging procedure to combine estimates produced by different models. The Bayesian estimators yield smaller variance compared to the conditional likelihood estimator and outperform the latter in the low power studies. We investigate the performance of the method with simulations and applications to four real data examples. Pleiotropy occurs when a single genetic factor influences multiple quantitative or qualitative phenotypes, and it is present in many genetic studies of complex human traits. The longitudinal family studies combine the features of longitudinal studies in individuals and cross-sectional studies in families. Therefore, they provide more information about the genetic and environmental factors associated with the trait of interest. We propose a Bayesian latent variable modeling approach to model multiple phenotypes simultaneously in order to detect the pleiotropic effect and allow for longitudinal and/or family data. An efficient MCMC algorithm is developed to obtain the posterior samples by using hierarchical centering and parameter expansion techniques. We apply spike and slab prior methods to test whether the phenotypes are significantly associated with the latent disease status. We compute Bayes factors using path sampling and discuss their application in testing the significance of factor loadings and the indirect fixed effects. We examine the performance of our methods via extensive simulations and apply them to the blood pressure data from a genetic study of type 1 diabetes (T1D) complications.