Thèses sur le sujet « Latent Models, Small Area Estimation »

Latent Markov Models (LMMs) are a particular class of statistical models in which a latent process is assumed. LMMs allow for the analysis of longitudinal data when the response variables measure common characteristics of interest, which are not directly observable. In LMMs the characteristics of interest, and their evolution in time, are represented by a latent process that follows a first order discrete Markov chain and units are allowed to change latent state over time. In studying LMMs, it is important to distinguish between two components: the measurement model, i.e. the conditional distribution of the response variables given the latent process, and the latent model, i.e. the distribution of the latent process. This thesis focuses on LMMs for aggregated data. It considers two fields of applications: disease mapping and small area estimation. The goal of disease mapping is the study of the geographical pattern and variation of a disease measured through counts and incidence rates. From a methodological point of view, this work extends LMMs to include a spatial pattern in the latent model. This extension allows the probability of being in a latent state and the probability to move from a latent state to another over time to be influenced by the neighbouring areas. The model is fitted within a Bayesian framework using Gibbs and Random Metropolis Hastings algorithms with augmented data that allows for a more efficient sampling of model parameters. Simulations studies are also conducted to investigate the performance of the proposed model on data generated under different settings. The model has also been applied to a data set of county specific lung cancer deaths counts in the state of Ohio, USA, during the years 1968-1988. Small area estimation (SAE) methods are used in inference for finite populations to obtain estimates of parameters of interest when domain sample sizes are too small to provide adequate precision for direct domain estimators. The second work develops a new area-level SAE method using LMMs. In particular, since area-level SAE models consider a sampling and a linking model, a LMM is used as the linking model. In a hierarchical Bayesian framework the sampling model is introduced as the highest level of hierarchy. In this context, data are considered aggregated because direct estimates are usually mean and frequencies. Under the assumption of normality the response variable, the model is estimated using a Gibbs sampling in a data augmentation context. The application field in this second work is particularly relevant: it uses yearly unemployment rates at Local Labour Market Areas level for the period 2004-2014 from the Labour Force Survey conducted by the Italian National Statistical Institute (ISTAT).

Sample surveys are widely used as a cost-effective way to collect information on variables of interest in target populations. In applications, we are generally interested in parameters such as population means, totals, and quantiles. Similar parameters for subpopulations or areas, formed by geographic areas and socio-demographic groups, are also of interest in applications. However, the sample size might be small or even zero in subpopulations due to the probability sampling and the budget limitation. There has been intensive research on how to produce reliable estimates for characteristics of interest for subpopulations for which the sample size is small or even zero. We call this line of research Small Area Estimation (SAE). In this thesis, we study the performance of a number of small area quantile estimators based on a popular unit-level model and its variations. When a finite population can be regarded as a sample from some model, we may use the whole sample from the finite population to determine the model structure with a good precision. The information can then be used to produce more reliable estimates for small areas. However, if the model assumption is wrong, the resulting estimates can be misleading and their mean squared errors can be underestimated. Therefore, it is critical to check the robustness of estimators under various model mis-specification scenarios. In this thesis, we first conduct simulation studies to investigate the performance of three small area quantile estimators in the literature. They are found not to be very robust in some likely situations. Based on these observations, we propose an approach to obtain more robust small area quantile estimators. Simulation results show that the proposed new methods have superior performance either when the error distribution in the model is non-normal or the data set contain many outliers.
Science, Faculty of
Statistics, Department of
Graduate

Thesis (Ph. D.) -- University of Maryland, College Park, 2007.
Thesis research directed by: Mathematics. Title from t.p. of PDF. Includes bibliographical references. Published by UMI Dissertation Services, Ann Arbor, Mich. Also available in paper.

The demand for reliable small area estimates derived from survey data has increased greatly in recent years due to, among other things, their growing use in formulating policies and programs, allocation of government funds, regional planning, small area business decisions and other applications. Traditional area-specific (direct) estimates may not provide acceptable precision for small areas because sample sizes are seldom large enough in many small areas of interest. This makes it necessary to borrow information across related areas through indirect estimation based on models, using auxiliary information such as recent census data and current administrative data. Methods based on models are now widely accepted. The principal focus of this thesis is the development of a flexible modeling strategy in small area estimation with demonstrations and evaluations using the 1989 United States census bureau median income dataset. This dissertation is divided into two main parts, the first part deals with development of the proposed model and comparision of this model to the standard area-level Fay-Herriot model through the empirical Bayes (EB) approach. Results from these two models are compared in terms of average relative bias, average squared relative bias, average absolute bias, average squared deviation as well as the empirical mean square error. The proposed model exhibits remarkably better performance over the standard Fay-Herriot model. The second part represents our attempt to construct a hierarchical Bayes (HB) approach to estimate parameters for the proposed model, with implementation carried out by Markov chain Monte Carlo (MCMC) techniques. MCMC was implemented via the Gibbs sampling algorithm using R software package. We used several graphical tools to assess convergence and determine the length of the burn-in period. Results from the two models are compared in terms of average relative bias, average squared relative bias and average absolute bias. Our empirical results highlight the superiority of using the proposed model over the Fay-Herriot model. However, the advantage of the proposed model comes at a price since its implementation is mildly more difficult than the Fay-Herriot model.
L'esigenza di stime affidabili per piccole aree tratte da sondaggi è cresciuta notevolmente negli ultimi anni, grazie all'aumento del loro utilizzo nella formulazione delle politiche, nella ripartizione dei fondi statali, nella pianificazione regionale, nelle applicazioni business e in altre applicazioni. Le tradizionali stime specifiche per l'area (stime dirette) potrebbero non fornire una precisione accettabile, perché la numerosità campionaria in molte delle piccole aree d'interesse potrebbe essere ridotta o nulla. Questo rende neccessario sfruttare le informazioni dalle zone simili, tramitte una stima indiretta basata sui modelli per informazioni ausiliarie come i dati dei censimenti o i dati amministrativi. I metodi basati sui modelli sono ora piuttosto diffusi. L'attenzione principale di questa tesi è sviluppare una strategia di modellazione flessibile nella stima di piccole aree, e la sua valutazione utilizzando il Censimento negli Stati Uniti sul reddito mediano, del 1989. Questa dissertazione è composta di due parti : la prima tratta lo sviluppo del modello e il confronto del modello proposto con il modello standard di Fay-Herriot tramite l'approcio di Bayes empirico. I risultati per questi due modelli sono stati confrontati in termini del bias relativo medio, del bias quadratico medio, del bias medio assoluto, della deviazione quadratica media ed inotre in termini del errore quadratico medio empirico. Il modello proposto dimostra un rendimento assai migliore rispetto al modello standard di Fay-Herriot. La seconda parte presenta il nostro tentativo di costruire un approccio di Bayes Gerarchico per la stima dei parametri del modello proposto, con l'attuazione delle tecniche di Markov Chain Monte Carlo (MCMC). MCMC è stato utilizzato tramitte l'algoritmo di campionamento Gibbs, utilizzando il software R. I risultati dai due modelli sono stati confrontati in termini di bias relativo medio, bias relativo quadratico medio e il bias assoluto medio. I nostri risultati empirici sottolineano la superiorità del modello proposto rispetto al modello Fay-Herriot. Tuttavia, il vantaggio del modello proposto è limitato visto che la sua attuazione è leggermente più complicata rispetto al modello di Fay-Herriot.

Bayesian hierarchical models are effective tools for small area estimation by pooling small datasets together. The pooling procedures allow individual areas to “borrow strength” from each other to desirably improve the estimation. This work is an extension of Nandram and Choi (2002), NC, to perform inference on finite population proportions when there exists non-identifiability of the missing pattern for nonresponse in binary survey data. We review the small-area selection model (SSM) in NC which is able to incorporate the non-identifiability. Moreover, the proposed SSM, together with the individual-area selection model (ISM), and the small-area pattern-mixture model (SPM) are evaluated by real crime data in Stasny (1991). Furthermore, the methodology is compared to ISM and SPM using simulated small area datasets. Computational issues related to the MCMC are also discussed.

Population estimation in inter-censual years has many important applications. In this research, high-resolution pan-sharpened IKONOS image, LiDAR data, and parcel data are used to estimate small-area population in the eastern part of the city of Denton, Texas. Residential buildings are extracted through object-based classification techniques supported by shape indices and spectral signatures. Three population indicators -building count, building volume and building area at block level are derived using spatial joining and zonal statistics in GIS. Linear regression and geographically weighted regression (GWR) models generated using the three variables and the census data are used to estimate population at the census block level. The maximum total estimation accuracy that can be attained by the models is 94.21%. Accuracy assessments suggest that the GWR models outperformed linear regression models due to their better handling of spatial heterogeneity. Models generated from building volume and area gave better results. The models have lower accuracy in both densely populated census blocks and sparsely populated census blocks, which could be partly attributed to the lower accuracy of the LiDAR data used.

We implement techniques of small area estimation (SAE) to study consumption, a welfare indicator, which is used to assess poverty in the 2003-2004 Nepal Living Standards Survey (NLSS-II) and the 2001 census. NLSS-II has detailed information of consumption, but it can give estimates only at stratum level or higher. While population variables are available for all households in the census, they do not include the information on consumption; the survey has the `population' variables nonetheless. We combine these two sets of data to provide estimates of poverty indicators (incidence, gap and severity) for small areas (wards, village development committees and districts). Consumption is the aggregate of all food and all non-food items consumed. In the welfare survey the responders are asked to recall all information about consumptions throughout the reference year. Therefore, such data are likely to be noisy, possibly due to response errors or recalling errors. The consumption variable is continuous and positively skewed, so a statistician might use a logarithmic transformation, which can reduce skewness and help meet the normality assumption required for model building. However, it could be problematic since back transformation may produce inaccurate estimates and there are difficulties in interpretations. Without using the logarithmic transformation, we develop hierarchical Bayesian models to link the survey to the census. In our models for consumption, we incorporate the `population' variables as covariates. First, we assume that consumption is noiseless, and it is modeled using three scenarios: the exponential distribution, the gamma distribution and the generalized gamma distribution. Second, we assume that consumption is noisy, and we fit the generalized beta distribution of the second kind (GB2) to consumption. We consider three more scenarios of GB2: a mixture of exponential and gamma distributions, a mixture of two gamma distributions, and a mixture of two generalized gamma distributions. We note that there are difficulties in fitting the models for noisy responses because these models have non-identifiable parameters. For each scenario, after fitting two hierarchical Bayesian models (with and without area effects), we show how to select the most plausible model and we perform a Bayesian data analysis on Nepal's poverty data. We show how to predict the poverty indicators for all wards, village development committees and districts of Nepal (a big data problem) by combining the survey data with the census. This is a computationally intensive problem because Nepal has about four million households with about four thousand households in the survey and there is no record linkage between households in the survey and the census. Finally, we perform empirical studies to assess the quality of our survey-census procedure.

It is a standard practice in small area estimation (SAE) to use a model-based approach to borrow information from neighboring areas or from areas with similar characteristics. However, survey data tend to have gaps, ties and outliers, and parametric models may be problematic because statistical inference is sensitive to parametric assumptions. We propose nonparametric hierarchical Bayesian models for multi-stage finite population sampling to robustify the inference and allow for heterogeneity, outliers, skewness, etc. Bayesian predictive inference for SAE is studied by embedding a parametric model in a nonparametric model. The Dirichlet process (DP) has attractive properties such as clustering that permits borrowing information. We exemplify by considering in detail two-stage and three-stage hierarchical Bayesian models with DPs at various stages. The computational difficulties of the predictive inference when the population size is much larger than the sample size can be overcome by the stick-breaking algorithm and approximate methods. Moreover, the model comparison is conducted by computing log pseudo marginal likelihood and Bayes factors. We illustrate the methodology using body mass index (BMI) data from the National Health and Nutrition Examination Survey and simulated data. We conclude that a nonparametric model should be used unless there is a strong belief in the specific parametric form of a model.

Spatial data is often aggregated into small area. It is challenging to analyse these data. Two new statistical techniques were developed to tackle these challenges. Bayesian meta-analysis models were used to unveil geographic disparities in cancer using the Australian Cancer Atlas. More flexible Bayesian Empirical Likelihood models were used to analyse more complex data including COVID19 deaths. Both methods provided new insights into the analysis of spatial data.

Thesis (M.S.) -- Worcester Polytechnic Institute.
Keywords: Poisson-Gamma Regression; MCMC; Bayesian; Small Area Estimation; Simultaneous Inference; Statistics Includes bibliographical references (p. 61-67).

Dans cette thèse, nous nous intéressons à l'estimation robuste de courbes moyennes ou totales de consommation électrique par sondage en population finie, pour l'ensemble de la population ainsi que pour des petites sous-populations, en présence ou non de courbes partiellement inobservées.En effet, de nombreuses études réalisées dans le groupe EDF, que ce soit dans une optique commerciale ou de gestion du réseau de distribution par Enedis, se basent sur l'analyse de courbes de consommation électrique moyennes ou totales, pour différents groupes de clients partageant des caractéristiques communes. L'ensemble des consommations électriques de chacun des 35 millions de clients résidentiels et professionnels Français ne pouvant être mesurées pour des raisons de coût et de protection de la vie privée, ces courbes de consommation moyennes sont estimées par sondage à partir de panels. Nous prolongeons les travaux de Lardin (2012) sur l'estimation de courbes moyennes par sondage en nous intéressant à des aspects spécifiques de cette problématique, à savoir l'estimation robuste aux unités influentes, l'estimation sur des petits domaines, et l'estimation en présence de courbes partiellement ou totalement inobservées.Pour proposer des estimateurs robustes de courbes moyennes, nous adaptons au cadre fonctionnel l'approche unifiée d'estimation robuste en sondages basée sur le biais conditionnel proposée par Beaumont (2013). Pour cela, nous proposons et comparons sur des jeux de données réelles trois approches : l'application des méthodes usuelles sur les courbes discrétisées, la projection sur des bases de dimension finie (Ondelettes ou Composantes Principales de l'Analyse en Composantes Principales Sphériques Fonctionnelle en particulier) et la troncature fonctionnelle des biais conditionnels basée sur la notion de profondeur d'une courbe dans un jeu de données fonctionnelles. Des estimateurs d'erreur quadratique moyenne instantanée, explicites et par bootstrap, sont également proposés.Nous traitons ensuite la problématique de l'estimation sur de petites sous-populations. Dans ce cadre, nous proposons trois méthodes : les modèles linéaires mixtes au niveau unité appliqués sur les scores de l'Analyse en Composantes Principales ou les coefficients d'ondelettes, la régression fonctionnelle et enfin l'agrégation de prédictions de courbes individuelles réalisées à l'aide d'arbres de régression ou de forêts aléatoires pour une variable cible fonctionnelle. Des versions robustes de ces différents estimateurs sont ensuite proposées en déclinant la démarche d'estimation robuste basée sur les biais conditionnels proposée précédemment.Enfin, nous proposons quatre estimateurs de courbes moyennes en présence de courbes partiellement ou totalement inobservées. Le premier est un estimateur par repondération par lissage temporel non paramétrique adapté au contexte des sondages et de la non réponse et les suivants reposent sur des méthodes d'imputation. Les portions manquantes des courbes sont alors déterminées soit en utilisant l'estimateur par lissage précédemment cité, soit par imputation par les plus proches voisins adaptée au cadre fonctionnel ou enfin par une variante de l'interpolation linéaire permettant de prendre en compte le comportement moyen de l'ensemble des unités de l'échantillon. Des approximations de variance sont proposées dans chaque cas et l'ensemble des méthodes sont comparées sur des jeux de données réelles, pour des scénarios variés de valeurs manquantes
In this thesis, we address the problem of robust estimation of mean or total electricity consumption curves by sampling in a finite population for the entire population and for small areas. We are also interested in estimating mean curves by sampling in presence of partially missing trajectories.Indeed, many studies carried out in the French electricity company EDF, for marketing or power grid management purposes, are based on the analysis of mean or total electricity consumption curves at a fine time scale, for different groups of clients sharing some common characteristics.Because of privacy issues and financial costs, it is not possible to measure the electricity consumption curve of each customer so these mean curves are estimated using samples. In this thesis, we extend the work of Lardin (2012) on mean curve estimation by sampling by focusing on specific aspects of this problem such as robustness to influential units, small area estimation and estimation in presence of partially or totally unobserved curves.In order to build robust estimators of mean curves we adapt the unified approach to robust estimation in finite population proposed by Beaumont et al (2013) to the context of functional data. To that purpose we propose three approaches : application of the usual method for real variables on discretised curves, projection on Functional Spherical Principal Components or on a Wavelets basis and thirdly functional truncation of conditional biases based on the notion of depth.These methods are tested and compared to each other on real datasets and Mean Squared Error estimators are also proposed.Secondly we address the problem of small area estimation for functional means or totals. We introduce three methods: unit level linear mixed model applied on the scores of functional principal components analysis or on wavelets coefficients, functional regression and aggregation of individual curves predictions by functional regression trees or functional random forests. Robust versions of these estimators are then proposed by following the approach to robust estimation based on conditional biais presented before.Finally, we suggest four estimators of mean curves by sampling in presence of partially or totally unobserved trajectories. The first estimator is a reweighting estimator where the weights are determined using a temporal non parametric kernel smoothing adapted to the context of finite population and missing data and the other ones rely on imputation of missing data. Missing parts of the curves are determined either by using the smoothing estimator presented before, or by nearest neighbours imputation adapted to functional data or by a variant of linear interpolation which takes into account the mean trajectory of the entire sample. Variance approximations are proposed for each method and all the estimators are compared to each other on real datasets for various missing data scenarios