Academic literature on the topic 'Misspecified bounds'

We show that Bayesian posteriors concentrate on the outcome distributions that approximately minimize the Kullback–Leibler divergence from the empirical distribution, uniformly over sample paths, even when the prior does not have full support. This generalizes Diaconis and Freedman's (1990) uniform convergence result to, e.g., priors that have finite support, are constrained by independence assumptions, or have a parametric form that cannot match some probability distributions. The concentration result lets us provide a rate of convergence for Berk's (1966) result on the limiting behavior of posterior beliefs when the prior is misspecified. We provide a bound on approximation errors in “anticipated‐utility” models, and extend our analysis to outcomes that are perceived to follow a Markov process.

This paper studies a Nyström-type subsampling approach to large kernel learning methods in the misspecified case, where the target function is not assumed to belong to the reproducing kernel Hilbert space generated by the underlying kernel. This case is less understood in spite of its practical importance. To model such a case, the smoothness of target functions is described in terms of general source conditions. It is surprising that almost for the whole range of the source conditions, describing the misspecified case, the corresponding learning rate bounds can be achieved with just one value of the regularization parameter. This observation allows a formulation of mild conditions under which the plain Nyström subsampling can be realized with subquadratic cost maintaining the guaranteed learning rates.

The papers collected in this issue are united in a common view that it is rational to recognize that we have a poor perception of the constraints we face when making economic decisions and hence we employ decision rules that are robust. Robustness can be interpreted in different ways but generally it implies that our decision rules should not depend critically on an exact description of these constraints but they should perform well over a prespecified range of potential variations in the assumed economic environment. So, we are interested in deriving optimal and hence rational decisions where our utility or loss function incorporates the need for robustness in the face of a misspecified model. This misspecification can involve placing simple bounds on deviations from the parameters we assume for a nominal model, or misspecified dynamics, neglected nonlinearities, time variation, or quite general arbitrary misspecification in the transfer function between the input uncertainties and the output variables in which we are ultimately interested.

This paper studies the averaging GMM estimator that combines a conservative GMM estimator based on valid moment conditions and an aggressive GMM estimator based on both valid and possibly misspecified moment conditions, where the weight is the sample analog of an infeasible optimal weight. We establish asymptotic theory on uniform approximation of the upper and lower bounds of the finite‐sample truncated risk difference between any two estimators, which is used to compare the averaging GMM estimator and the conservative GMM estimator. Under some sufficient conditions, we show that the asymptotic lower bound of the truncated risk difference between the averaging estimator and the conservative estimator is strictly less than zero, while the asymptotic upper bound is zero uniformly over any degree of misspecification. The results apply to quadratic loss functions. This uniform asymptotic dominance is established in non‐Gaussian semiparametric nonlinear models.

Datasets displaying temporal dependencies abound in science and engineering applications, with Markov models representing a simplified and popular view of the temporal dependence structure. In this paper, we consider Bayesian settings that place prior distributions over the parameters of the transition kernel of a Markov model, and seek to characterize the resulting, typically intractable, posterior distributions. We present a Probably Approximately Correct (PAC)-Bayesian analysis of variational Bayes (VB) approximations to tempered Bayesian posterior distributions, bounding the model risk of the VB approximations. Tempered posteriors are known to be robust to model misspecification, and their variational approximations do not suffer the usual problems of over confident approximations. Our results tie the risk bounds to the mixing and ergodic properties of the Markov data generating model. We illustrate the PAC-Bayes bounds through a number of example Markov models, and also consider the situation where the Markov model is misspecified.

Un nouvel algorithme est proposé et validé pour générer une échelle de temps robuste. Prévu pour une utilisation dans un essaim de nanosatellites, l'Autonomous Time Scale using the Student's T-distribution (ATST) peut traiter les anomalies subies par les horloges et les liens inter-satellites dans un environnement hostile. Les types d'anomalies traités incluent les sauts de phase, les sauts de fréquence, un bruit de mesure élevé dans certains liens et les données manquantes. En prenant la moyenne pondérée des résidus contenus dans l'équation de l'échelle de temps de base (BTSE), la contribution des satellites avec des mesures anormales est réduite pour la génération de l'échelle de temps. Les poids attribués à chaque horloge sont basés sur l'hypothèse que les résidus suivent la loi de Student.La performance de l'algorithme ATST est équivalente à celle de l'algorithme AT1 oracle, qui est une version de l'échelle de temps AT1 avec la capacité de détecter parfaitement toutes les anomalies dans des données simulées. Bien que l'algorithme n'ait pas de méthode de détection explicite, l'ATST affiche toujours un niveau de robustesse comparable à celui d'un détecteur parfait. Cependant, l'ATST est conçu pour un essaim avec de nombreuses horloges de types homogènes et est limité par une complexité numérique élevée. De plus, les anomalies sont toutes traitées de la même manière sans distinction entre les différents types d'anomalies. Malgré ces limitations identifiées, le nouvel algorithme représente une contribution prometteuse dans le domaine des échelles de temps grâce à la robustesse atteinte.Une méthode de traitement des horloges ajoutées ou retirées de l'ensemble est également proposée dans cette thèse en conjonction avec l'ATST. Cette méthode préserve la continuité de phase et de fréquence de l'échelle de temps en attribuant un poids nul aux horloges pertinentes lorsque le nombre total d'horloges est modifié. Un estimateur des moindres carrés (Least Squares, LS) est présenté pour montrer comment les mesures des liens inter-satellites peuvent être traitées en amont pour réduire le bruit de mesure et en même temps remplacer les mesures manquantes. L'estimateur LS peut être utilisé avec une méthode de détection qui élimine les mesures anormales, puis l'estimateur LS remplace les mesures supprimées par les estimations correspondantes.Cette thèse examine également l'estimation optimale de l'estimateur du maximum de vraisemblance (MLE) pour les paramètres des lois de probabilités à queues lourdes : précisément la loi de Student et la loi des mélanges gaussiens. Les améliorations obtenues en supposant correctement ces lois par rapport à l'hypothèse de la loi gaussienne sont démontrées avec les bornes de Cramér-Rao mal spécifiées (MCRB). Le MCRB dérivé confirme que les lois à queues lourdes sont meilleures pour l'estimation de la moyenne en présence de valeurs aberrantes. L'estimation des paramètres des lois à queues lourdes nécessite au moins 25 horloges pour obtenir l'erreur minimale, c'est-à-dire que l'estimateur atteigne l'efficacité asymptotique. Cette méthodologie pourra nous aider à analyser d'autres types d'anomalies suivant des lois différentes.Des propositions pour des pistes de recherche futures incluent le traitement des limitations de l'algorithme ATST concernant les types et le nombre d'horloges. Une nouvelle moyenne pour attribuer les poids en utilisant le machine learning est envisageable grâce à la compréhension des résidus du BTSE. Les anomalies transitoires peuvent être mieux traitées par le machine learning ou même avec un estimateur robuste de la fréquence des horloges sur une fenêtre de données passées. Cela est intéressant à explorer et à comparer à l'algorithme ATST, qui est proposé pour des anomalies instantanées
A new robust time scale algorithm, the Autonomous Time scale using the Student's T-distribution (ATST), has been proposed and validated using simulated clock data. Designed for use in a nanosatellite swarm, ATST addresses phase jumps, frequency jumps, anomalous measurement noise, and missing data by making a weighted average of the residuals contained in the Basic Time Scale Equation (BTSE). The weights come from an estimator that assumes the BTSE residuals are modeled by a Student's t-distribution.Despite not detecting anomalies explicitly, the ATST algorithm performs similarly to a version of the AT1 time scale that detects anomalies perfectly in simulated data. However, ATST is best for homogeneous clock types, requires a high number of clocks, adds computational complexity, and cannot necessarily differentiate anomaly types. Despite these identified limitations the robustness achieved is a promising contribution to the field of time scale algorithms.The implementation of ATST includes a method that maintains phase and frequency continuity when clocks are removed or reintroduced into the ensemble by resetting appropriate clock weights to zero. A Least Squares (LS) estimator is also presented to pre-process inter-satellite measurements, reducing noise and estimating missing data. The LS estimator is also compatible with anomaly detection which removes anomalous inter-satellite measurements because it can replace the removed measurements with their estimates.The thesis also explores optimal estimation of parameters of two heavy-tailed distributions: the Student's t and Bimodal Gaussian mixture. The Misspecified Cramér Rao Bound (MCRB) confirms that assuming heavy-tailed distributions handles outliers better compared to assuming a Gaussian distribution. We also observe that at least 25 clocks are required for asymptotic efficiency when estimating the mean of the clock residuals. The methodology also aids in analyzing other anomaly types fitting different distributions.Future research proposals include addressing ATST's limitations with diverse clock types, mitigating performance loss with fewer clocks, and exploring robust time scale generation using machine learning to weight BTSE residuals. Transient anomalies can be targeted using machine learning or even a similar method of robust estimation of clock frequencies over a window of past data. This is interesting to research and compare to the ATST algorithm that is instead proposed for instantaneous anomalies

abstract: Signal compressed using classical compression methods can be acquired using brute force (i.e. searching for non-zero entries in component-wise). However, sparse solutions require combinatorial searches of high computations. In this thesis, instead, two Bayesian approaches are considered to recover a sparse vector from underdetermined noisy measurements. The first is constructed using a Bernoulli-Gaussian (BG) prior distribution and is assumed to be the true generative model. The second is constructed using a Gamma-Normal (GN) prior distribution and is, therefore, a different (i.e. misspecified) model. To estimate the posterior distribution for the correctly specified scenario, an algorithm based on generalized approximated message passing (GAMP) is constructed, while an algorithm based on sparse Bayesian learning (SBL) is used for the misspecified scenario. Recovering sparse signal using Bayesian framework is one class of algorithms to solve the sparse problem. All classes of algorithms aim to get around the high computations associated with the combinatorial searches. Compressive sensing (CS) is a widely-used terminology attributed to optimize the sparse problem and its applications. Applications such as magnetic resonance imaging (MRI), image acquisition in radar imaging, and facial recognition. In CS literature, the target vector can be recovered either by optimizing an objective function using point estimation, or recovering a distribution of the sparse vector using Bayesian estimation. Although Bayesian framework provides an extra degree of freedom to assume a distribution that is directly applicable to the problem of interest, it is hard to find a theoretical guarantee of convergence. This limitation has shifted some of researches to use a non-Bayesian framework. This thesis tries to close this gab by proposing a Bayesian framework with a suggested theoretical bound for the assumed, not necessarily correct, distribution. In the simulation study, a general lower Bayesian Cram\'er-Rao bound (BCRB) bound is extracted along with misspecified Bayesian Cram\'er-Rao bound (MBCRB) for GN model. Both bounds are validated using mean square error (MSE) performances of the aforementioned algorithms. Also, a quantification of the performance in terms of gains versus losses is introduced as one main finding of this report.
Dissertation/Thesis
Masters Thesis Computer Engineering 2019