Dissertations / Theses on the topic 'Linear regression'

This thesis was based on the subjects of mathematical statistics and industrial economics and management in order to analyze the grades of pupils in the final year of elementary school. The purpose was to find out what variables had a statistically significant impact on pupils’ final grades so that municipalities and schools could better understand what variables are important when trying to improve the average school results. A multiple regression model was used on data, obtained from the database of Skolverket, in order to examine what variables were statistically important. The final regression model acquired through a model reduction procedure showed that mostly structural covariates such as the academic background of pupils, percentage of female pupils and the percentage with Swedish background had a statistically significant impact on the academic performances of the students. R2 adjusted of the final model was 0.5289. The multiple regression model was discussed by referencing to previous research. In addition, the strategic management performance framework known as Balanced Scorecard which was introduced by Robert S. Kaplan and David P. Norton was used to discuss relevant key performance indicators to achieve the strategic objectives of schools.
Detta examensarbete, inom ämnet för matematisk statistik och industriell ekonomi, genomfördes med syftet att analysera avgångsbetygen för år 9 i den svenska skolan. Syftet var att förstå vilka variabler som hade en statistisk signifikant påverkan på elevers avgångsbetyg, så kommuner kan förstå vilka variabler som är viktiga för att förbättra de genomsnittliga skolresultaten. En regressionsanalys utfördes, på data från Skolverket, för att se vilka variabler som var statistiskt signifikanta. Den slutgiltiga regressionsmodellen, erhållen genom iterativ reducering av variabler, visade att främst strukturella kovariat, som akademisk bakgrund hos elever, andel kvinnliga studenter och andel studenter med svensk bakgrund hade en signifikant betydelse på studenters akademiska resultat. Justerad R2 var 0.5289 för den slutgiltiga modellen. I diskussionen utvärderades modellen utifrån tidigare forskning. Vidare användes teorin om balanserat styrkort, utvecklat av Robert S. Kaplan och David P. Norton, för att diskutera relevanta nyckeltal för att uppnå strategiska mål för skolan.

Accurate rainfall prediction is a challenging task because of the complex physical processes involved. This complexity is compounded in Australia as the climate can be highly variable. Accurate rainfall prediction is immensely benecial for making informed policy, planning and management decisions, and can assist with the most sustainable operation of water resource systems. Short-term prediction of rainfall is provided by meteorological services; however, the intermediate to long-term prediction of rainfall remains challenging and contains much uncertainty. Many prediction approaches have been proposed in the literature, including statistical and computational intelligence approaches. However, finding a method to model the complex physical process of rainfall, especially in Australia where the climate is highly variable, is still a major challenge. The aims of this study are to: (a) develop an optimization based clusterwise linear regression method, (b) develop new prediction methods based on clusterwise linear regression, (c) assess the influence of geographic regions on the performance of prediction models in predicting monthly and weekly rainfall in Australia, (d) determine the combined influence of meteorological variables on rainfall prediction in Australia, and (e) carry out a comparative analysis of new and existing prediction techniques using Australian rainfall data. In this study, rainfall data with five input meteorological variables from 24 geographically diverse weather stations in Australia, over the period January 1970 to December 2014, have been taken from the Scientific Information for Land Owners (SILO). We also consider the climate zones when selecting weather stations, because Australia experiences a variety of climates due to its size. The data was divided into training and testing periods for evaluation purposes. In this study, optimization based clusterwise linear regression is modified and new prediction methods are developed for rainfall prediction. The proposed method is applied to predict monthly and weekly rainfall. The prediction performance of the clusterwise linear regression method was evaluated by comparing observed and predicted rainfall values using the performance measures: root mean squared error, the mean absolute error, the mean absolute scaled error and the Nash-Sutclie coefficient of efficiency. The proposed method is also compared with the clusterwise linear regression based on the maximum likelihood estimation, linear support vector machines for regression, support vector machines for regression with radial basis kernel function, multiple linear regression, artificial neural networks with and without hidden layer and k-nearest neighbours methods using computational results. Initially, to determine the appropriate input variables to be used in the investigation, we assessed all combinations of meteorological variables. The results confirm that single meteorological variables alone are unable to predict rainfall accurately. The prediction performance of all selected models was improved by adding the input variables in most locations. To assess the influence of geographic regions on the performance of prediction models and to compare the prediction performance of models, we trained models with the best combination of input variables and predicted monthly and weekly rainfall over the test periods. The results of this analysis confirm that the prediction performance of all selected models varied considerably with geographic regions for both weekly and monthly rainfall predictions. It is found that models have the lowest prediction error in the desert climate zone and highest in subtropical and tropical zones. The results also demonstrate that the proposed algorithm is capable of finding the patterns and trends of the observations for monthly and weekly rainfall predictions in all geographic regions. In desert, tropical and subtropical climate zones, the proposed method outperform other methods in most locations for both monthly and weekly rainfall predictions. In temperate and grassland zones the prediction performance of the proposed model is better in some locations while in the remaining locations it is slightly lower than the other models.
Doctor of Philosophy

This thesis presented a useful tool in regression. Nonparametric linear regression techniques were described in the general context of regression. A comparison of two of these techniques, kernel regression and iterative regression, showed various aspects of nonparametric linear regressors.

Este trabalho tem o objetivo de divulgar a metodologia de ponderação de modelos ou Bayesian Model Averaging (BMA) entre os pesquisadores da área agronômica e discutir suas vantagens e limitações. Com o BMA é possível combinar resultados de diferentes modelos acerca de determinada quantidade de interesse, com isso, o BMA apresenta-se como sendo uma metodologia alternativa de análise de dados frente os usuais métodos de seleção de modelos tais como o Coeficiente de Determinação Múltipla (R2 ), Coeficiente de Determinação Múltipla Ajustado (R2), Estatística de Mallows ( Cp) e Soma de Quadrados de Predição (PRESS). Vários trabalhos foram, recentemente, realizados com o objetivo de comparar o desempenho do BMA em relação aos métodos de seleção de modelos, porém, há ainda muitas situações para serem exploradas até que se possa chegar a uma conclusão geral acerca desta metodologia. Neste trabalho, o BMA foi aplicado a um conjunto de dados proveniente de um experimento agronômico. A seguir, o desempenho preditivo do BMA foi comparado com o desempenho dos métodos de seleção acima citados por meio de um estudo de simulação variando o grau de multicolinearidade e o tamanho amostral. Em cada uma dessas situações, foram utilizadas 1000 amostras geradas a partir de medidas descritivas de conjuntos de dados reais da área agronômica. O desempenho preditivo das metodologias em comparação foi medido pelo Logaritmo do Escore Preditivo (LEP). Os resultados empíricos obtidos indicaram que o BMA apresenta desempenho semelhante aos métodos usuais de seleção de modelos nas situações de multicolinearidade exploradas neste trabalho.
The objective of this work was divulge to Bayesian Model Averaging (BMA) between the researchers of the agronomy area and discuss its advantages and limitations. With the BMA is possible combine results of difeerent models about determined quantity of interest, with that, the BMA presents as being a metodology alternative of data analysis front the usual models selection approaches, for example the Coefficient of Multiple Determination (R2), Coefficient of Multiple Determination Adjusted (R2), Mallows (Cp Statistics) and Prediction Error Sum Squares (PRESS). Several works recently were carried out with the objective of compare the performance of the BMA regarding the approaches of models selection, however, there is still many situations for will be exploited to that can arrive to a general conclusion about this metodology. In this work, the BMA was applied to data originating from an agronomy experiment. It follow, the predictive performance of the BMA was compared with the performance of the approaches of selection above cited by means of a study of simulation varying the degree of multicollinearity, measured by the number of condition of the matrix standardized X'X and the number of observations in the sample. In each one of those situations, were utilized 1000 samples generated from the descriptive information of agronomy data. The predictive performance of the metodologies in comparison was measured by the Logarithm of the Score Predictive (LEP). The empirical results obtained indicated that the BMA presents similar performance to the usual approaches of selection of models in the situations of multicollinearity exploited.

In the field of linear regression analysis, a single outlier can dramatically influence ordinary least squares estimation while low-breakdown procedures such as M regression and bounded influence regression may be unable to combat a small percentage of outliers. A high-breakdown procedure such as least trimmed squares (LTS) regression can accommodate up to 50% of the data (in the limit) being outlying with respect to the general trend. Two available one-step improvement procedures based on LTS are Mallows 1-step (M1S) regression and Schweppe 1-step (S1S) regression (the current state-of-the-art method). Issues with these methods include (1) computational approximations and sub-sampling variability, (2) dramatic coefficient sensitivity with respect to very slight differences in initial values, (3) internal instability when determining the general trend and (4) performance in low-breakdown scenarios. A new high-breakdown regression procedure is introduced that addresses these issues, plus offers an insightful summary regarding the presence and structure of multivariate outliers. This proposed method blends a cluster analysis phase with a controlled bounded influence regression phase, thereby referred to as cluster-based bounded influence regression, or CBI. Representing the data space via a special set of anchor points, a collection of point-addition OLS regression estimators forms the basis of a metric used in defining the similarity between any two observations. Cluster analysis then yields a main cluster "halfset" of observations, with the remaining observations becoming one or more minor clusters. An initial regression estimator arises from the main cluster, with a multiple point addition DFFITS argument used to carefully activate the minor clusters through a bounded influence regression framework. CBI achieves a 50% breakdown point, is regression equivariant, scale equivariant and affine equivariant and distributionally is asymptotically normal. Case studies and Monte Carlo studies demonstrate the performance advantage of CBI over S1S and the other high breakdown methods regarding coefficient stability, scale estimation and standard errors. A dendrogram of the clustering process is one graphical display available for multivariate outlier detection. Overall, the proposed methodology represents advancement in the field of robust regression, offering a distinct philosophical viewpoint towards data analysis and the marriage of estimation with diagnostic summary.
Ph. D.

Consideramos um modelo de regressão linear simples, em que tanto a variável resposta como a independente estão sujeitas a censura intervalar. Como motivação utilizamos um estudo em que o objetivo é avaliar a possibilidade de previsão dos resultados de um exame audiológico comportamental a partir dos resultados de um exame audiológico eletrofisiológico. Calculamos intervalos de previsão para a variável resposta, analisamos o comportamento dos estimadores de máxima verossimilhança obtidos sob o modelo proposto e comparamos seu desempenho com aquele de estimadores obtidos de um modelo de regressão linear simples usual, no qual a censura dos dados é desconsiderada.
We consider a simple linear regression model in which both variables are interval censored. To motivate the problem we use data from an audiometric study designed to evaluate the possibility of prediction of behavioral thresholds from physiological thresholds. We develop prediction intervals for the response variable, obtain the maximum likelihood estimators of the proposed model and compare their performance with that of estimators obtained under ordinary linear regression models.

The work was intended to support the Association of mertolenga cattle breed in its breeding process and decision making, namely in modeling the cost per day of production of the male mertolenga cattle, and in identifying the variables that favor the sale of the animal as a product with a protected designation of origin (PDO) seal. The database contained information on 716 male animals, of which 54 % went to the slaughter that guarantees the PDO seal. We also had data on the cost structure production of the animals from when it enters into the CTR to slaughter, in addition to the individual characteristics of each animal, in particular, of its estimated breeding value. To obtain the cost-per-day production model, multiple linear regression models and other generalized linear models were used. For the classi cation of the animal as a PDO slaughter destination, a logistic regression model was used. When we comparing the generalized linear models tested, the multiple linear regression model was con rmed as the best technique to explain the cost per day of production. For this model, it was found that information such as weight at entry as well as di erent estimated breeding value positively in uence the cost of production. With regard to logistic regression, weight at entry, age at entry and genetic values referring to maternal capacity and calving interval are factors that enhance the animal being sold under the PDO seal; Sumário: Com o trabalho desenvolvido nesta dissertação, pretendeu-se apoiar a Associação de produtores de bovinos da raça mertolenga no seu processo de recria e nas tomadas de decisão, nomeadamente na modelação do custo por dia de produção de bovinos machos da raça mertolenga, e na identificação das variáveis que favorecem a venda do animal como um produto com selo de denominação de origem protegida (DOP). A base de dados continha a informação de 716 animais machos, dos quais 54% foram para o abate que garante o selo DOP, dados referentes _a estrutura de custo de produção dos animais desde a entrada no CTR até o abate, além das características individuais de cada animal, em particular, dos seus valores genéticos. Para obter o modelo do custo por dia de produção, utilizou-se modelos de regressão linear múltipla e outros modelos lineares generalizados. Para a classificação do animal por destino de abate DOP, utilizou-se um modelo de regressão logística. Quando se comparou os diferentes modelos lineares generalizados testados, confirmou-se o modelo de regressão linear multipla como o mais adequado para explicar o custo por dia de produção. Para este modelo, verificou-se que informações como o peso à entrada bem como diferentes valores genéticos infuenciam de forma positiva o custo de produção. No que diz respeito a regressão logística, o peso à entrada, a idade à entrada e os valores genéticos referentes à capacidade maternal e intervalo entre partos são fatores potenciadores do animal ser vendido

Durante as últimas décadas, o melhoramento genético em bovinos leiteiros no Brasil baseou-se somente na importação de material genético, resultando em ganhos genéticos de pequena magnitude para as características de interesse econômico. Dessa forma, existe a necessidade eminente de avaliações genéticas dos animais sob condições nacionais de ambiente, de maneira a se prover um aumento na produção de leite aliado à qualidade. Neste contexto, o conhecimento sobre a composição do leite é de extrema importância para o entendimento de como alguns fatores ambientais e, principalmente genéticos podem influenciar no aumento dos conteúdos de proteína (PROT), gordura (GOR) e ácidos graxos (AG) benéficos e na redução da contagem de células somáticas, visando a melhoria da qualidade nutricional deste produto. Diante disso, o objetivo desse trabalho foi predizer os teores de AG de interesse usando regressão linear bayesiana, bem como estimar componentes de variância, coeficientes de herdabilidade e comparar modelos de diferentes ordens de ajuste por meio de funções polinomiais de Legendre, sob modelos de regressão aleatória. Amostras de leite foram submetidas a análises de cromatografia gasosa e espectrometria em infravermelho médio para determinação dos ácidos graxos. A comparação dos resultados obtidos por ambos os métodos foi realizada por meio da correlação de Pearson, análise de Bland-Altman e regressão linear bayesiana e, posteriormente, equações de predição foram desenvolvidas para os ácidos graxos mirístico (C14:0) e linoléico conjugado (CLA), a partir de regressões lineares simples e múltipla bayesiana considerando-se prioris nãoinformativas e informativas. Polinômios ortogonais de Legendre de 1ª a 6ª ordens foram utilizados para o ajuste das regressões aleatórias das características. A predição dos AG por meio da aplicação da regressão linear foi viável, com erros de predição variando entre 0,01 e 4,84g por 100g de gordura para o C14:0 e 0,002 e 1,85 por 100g de gordura para o CLA, sendo neste caso os menores erros de predição obtidos quando adotada a regressão múltipla com priori não informativa. Os modelos que melhor se ajustaram para GOR, PROT, C16:0, C18:0, C18:1c9, CLA, saturados (SAT), insaturados (INSAT), monoinsaturados (MONO) e poliinsaturados (POLI) foi o de 1ª ordem, e para escore de célula somática (ESC) e C14:0 o de 2ª ordem. As estimativas de herdabilidade obtidas variaram de 0,08 a 0,11 para GOR; 0,28 a 0,35 para PROT; 0,03 a 0,22 para ECS; 0,12 a 0,31 para C16:0; 0,08 a 0,14 para C18:0; 0,24 a 0,43 para C14:0; 0,07 a 0,17 para C18:1c9; 0,13 a 0,39 para CLA; 0,14 a 0,31 para SAT; 0,04 a 0,14 para INSAT; 0,04 a 0,13 para MONO; 0,09 a 0,20 para POLI e 0,12 para PROD, nos modelos que melhor se ajustaram. Concluise que melhorias na qualidade nutricional do leite podem ser obtidas por meio da inclusão das características produtivas e do perfil de ácidos graxos em programas de seleção genética.
During the last decades, genetic improvement in dairy cattle in Brazil was based only on the importation of genetic material, resulting in small genetic gains for economic interest traits. There is a perceived need for genetic evaluation under national environment conditions to provide an increase in milk production allied to quality. In this context, the knowledge of the milk composition is very important for understanding how certain environmental factors and especially genetic factors may influence the increase in protein content (PROT), fat (FAT), beneficial fatty acids (FA) and in reducing somatic cell count, aiming to improve the nutritional quality of this product. The aim of this study was to predict the levels of interest FA using Bayesian linear regression and estimate the components of variance, coefficients of heritability and compare models with different orders of adjustment by Legendre polynomials functions, in random regression models. Milk samples were subjected to gas chromatography analysis and mid-infrared spectrometry for the determination of fatty acids. The comparison of the results obtained by both methods was performed using Pearson\'s correlation, Bland-Altman analysis and Bayesian linear regression, subsequently, prediction equations were developed for the fatty acids myristic (C14:0) and conjugated linoleic (CLA) from simple linear regressions and multiple Bayesian considering non-informative and informative priors. Legendre orthogonal polynomials from 1st to 6th orders were used to fit the random regression of the traits. That was viable the prediction of FA by applying the linear regression with prediction errors ranging from 0.01 to 4.84 g per 100 g of fat for C14:0 and 0.002 to 1.85 per 100 g of fat for CLA, in this case the smaller prediction errors obtained when adopted the multiple regression with non-informative priori. The models that best fit for FAT, PROT, C16:0, C18:0, C18:1C9, CLA, saturated (SAT), unsaturated (UNSAT), monounsaturated (MONO) and polyunsaturated (POLY) was the one of 1st order and for somatic cell scores (SCS) and C14:0 the one of 2nd order. The estimates of heritability ranged from 0.08 to 0.11 for FAT; 0.28 to 0.35 for PROT; 0.03 to 0.22 for SCS; 0.12 to 0.31 for C16:0; 0.08 to 0.14 for C18:0; 0.24 to 0.43 for C14:0; 0.07 to 0.17 for C18:1C9; 0.13 to 0.39 for CLA; 0.14 to 0.31 for SAT; 0.04 to 0.14 for UNSAT; 0.04 to 0.13 for MONO, 0.09 to 0.20 for POLY and 0.12 for PROD, in the models that best fit. We conclude that improvements in the nutritional quality of milk can be obtained through the inclusion of productive traits and fatty acid profile in genetic selection programs.

A quantificação da evapotranspiração é uma tarefa essencial para a determinação do balanço hídrico em uma bacia hidrográfica e para o estabelecimento do déficit hídrico de uma cultura. Nesse sentido, o presente trabalho aborda a análise da evapotranspiração de referência (ETo) para a região de Jaboticabal-SP. O comportamento do fenômeno na região foi estudado a partir da interpretação de dados de uma bateria de 12 lisímetros de drenagem (EToLis) e estimativas teóricas por 10 equações diferentes disponíveis na literatura. A análise estatística de correlação indica que as estimativas da ETo por equações teóricas comparadas à EToLis medida em lisímetro de drenagem não apresentaram bons índices de comparação e erro. Admitindo que a operação dos lisímetros não permitiu a determinação da ETo com boa confiabilidade, propôs-se um ajuste local das demais metodologias de estimativa da ETo, através de auto-regressão (AR) dos ruídos destas equações em comparação com uma média anual estimada pela equação de Penman-Monteith (EToPM), tomada como padrão, em períodos quinzenal e mensal. O ajuste através de regressão linear simples também foi analisado. Os resultados obtidos indicam que a radiação efetiva é a variável climática de maior importância para o estabelecimento da ETo na região. A estimativa pela equação de Penman-Monteith apresentou excelente concordância com as equações de Makkink (1957) e do balanço de energia. Os ajustes locais propostos apresentaram excelentes resultados para a maioria das equações testadas, dando-se destaque às equações da radiação solar FAO-24, de Makkink (1957), de Jensen-Haise (1963), de Camargo (1971), do balanço de radiação, de Turc (1961) e de Thornthwaite (1948). O ajuste por regressão linear simples é de mais fácil execução e apresentou excelentes resultados.
The quantification of the evapotranspiration is an essential task for the determination of the water balance in a watershed and for the establishment of the culture´s water deficit. Therefore, the present work describes the analysis of the reference evapotranspiration (ETo) for the region of Jaboticabal-SP. The phenomenon behavior in the region was studied based on the interpretation of 12 drainage lysimeters data (EToLis) and on theoretical estimates for 10 different equations available in the Literature. An statistical analysis indicated that the theoretical ETo estimates compared with the EToLis did not present good indices of comparison and error. Admitting that the lysimeters operation did not allow a reliable ETo determination, a local adjustment of the theoretical methodologies for ETo estimate was considered. An auto-regression (AR) of the noises of these equations in comparison with the annual average estimate for the Penman-Monteith equation (EToPM), taken as standard, has been performed in fortnightly and monthly periods. The adjustment through simple linear regression has also been analyzed. The obtained results indicate that the effective radiation is the most important climatic variable for the establishment of the ETo in the region. The Penman-Monteith estimate presented excellent correlation to the estimates by Makkink (1957) equation and the energy balance. The local adjustments presented excellent results for the majority of the tested equations, specially for the solar radiation FAO-24, Makkink (1957), Jensen-Haise (1963), Camargo (1971), radiation balance, Turc (1961) and Thornthwaite (1948) equations. The adjustment by simple linear regression is of easier execution and also presented excellent results.

Metamodeling is a very popular approach for the approximation of complex systems. Metamodeling techniques can be categorized according to the type of regression method employed as linear and nonlinear models. The Response Surface Methodology (RSM) is an example of linear regression. In classical RSM metamodels, parameters are estimated using the Least Squares (LS) Method. Robust regression techniques, such as Least Absolute Deviation (LAD) and M-regression, are also considered in this study due to the outliers existing in data sets. Artificial Neural Networks (ANN) and Multivariate Adaptive Regression Splines (MARS) are examples for non-linear regression technique. In this thesis these two nonlinear metamodeling techniques are constructed and their performances are compared with the performances of linear models.

In this work we investigate the use of perfect sampling methods within the context of Bayesian linear regression. We focus on inference problems related to the marginal posterior model probabilities. Model averaged inference for the response and Bayesian variable selection are considered. Perfect sampling is an alternate form of Markov chain Monte Carlo that generates exact sample points from the posterior of interest. This approach removes the need for burn-in assessment faced by traditional MCMC methods. For model averaged inference, we find the monotone Gibbs coupling from the past (CFTP) algorithm is the preferred choice. This requires the predictor matrix be orthogonal, preventing variable selection, but allowing model averaging for prediction of the response. Exploring choices of priors for the parameters in the Bayesian linear model, we investigate sufficiency for monotonicity assuming Gaussian errors. We discover that a number of other sufficient conditions exist, besides an orthogonal predictor matrix, for the construction of a monotone Gibbs Markov chain. Requiring an orthogonal predictor matrix, we investigate new methods of orthogonalizing the original predictor matrix. We find that a new method using the modified Gram-Schmidt orthogonalization procedure performs comparably with existing transformation methods, such as generalized principal components. Accounting for the effect of using an orthogonal predictor matrix, we discover that inference using model averaging for in-sample prediction of the response is comparable between the original and orthogonal predictor matrix. The Gibbs sampler is then investigated for sampling when using the original predictor matrix and the orthogonal predictor matrix. We find that a hybrid method, using a standard Gibbs sampler on the orthogonal space in conjunction with the monotone CFTP Gibbs sampler, provides the fastest computation and convergence to the posterior distribution. We conclude the hybrid approach should be used when the monotone Gibbs CFTP sampler becomes impractical, due to large backwards coupling times. We demonstrate large backwards coupling times occur when the sample size is close to the number of predictors, or when hyper-parameter choices increase model competition. The monotone Gibbs CFTP sampler should be taken advantage of when the backwards coupling time is small. For the problem of variable selection we turn to the exact version of the independent Metropolis-Hastings (IMH) algorithm. We reiterate the notion that the exact IMH sampler is redundant, being a needlessly complicated rejection sampler. We then determine a rejection sampler is feasible for variable selection when the sample size is close to the number of predictors and using Zellner’s prior with a small value for the hyper-parameter c. Finally, we use the example of simulating from the posterior of c conditional on a model to demonstrate how the use of an exact IMH view-point clarifies how the rejection sampler can be adapted to improve efficiency.

Many variable selection techniques have been developed that focus on first-order linear regression models. In some applications, such as modeling response surfaces, fitting second-order terms can improve predictive accuracy. However, the number of spurious interactions can be large leading to poor results with many methods. We focus on forward selection, describing algorithms that use the natural hierarchy existing in second-order linear regression models to limit spurious interactions. We then develop stopping rules by extending False Selection Rate methodology to these algorithms. In addition, we describe alternative estimation methods for fitting regression models including the LASSO, CART, and MARS. We also propose a general method for controlling multiple-group false selection rates, which we apply to second-order linear regression models. By estimating a separate entry level for first-order and second-order terms, we obtain equal contributions to the false selection rate from each group. We compare the methods via Monte Carlo simulation and apply them to optimizing response surface experimental designs.

This thesis considers optimization techniques with applications in assignment and generalized linear regression problems. The first part of the thesis investigates the worst-case robust counterparts of combinatorial optimization problems with least squares (LS) cost functions, where the uncertainty lies on the linear transformation of the design variables. We consider the case of ellipsoidal uncertainty, and prove that the worst case robust LS optimization problem, although NP-hard, is still amenable to convexrelaxation based on semidefinite optimization. We motivate our proposed relaxation using Lagrangian duality, and illustrate that the tightness of the Lagrange bidual relaxation is strongly dependent on the description of the feasible region of the worst-case robust LS problem. The results arising from this analysis are applicable to a broad range of assignment problems. The second part of the thesis considers combinatorial optimization problems arising specifically in the context of conference program formation. We start by arguing that both papers and reviewers can be represented as feature vectors in a suitable keyword space. This enables rigorous mathematical formulation of the conference formation process. The first problem, paper-to-session assignment, is formulated as a capacitatedk-means clustering problem. We formally prove that it is NP-hard and propose a variety of approximate solutions, ranging from alternating optimization to semidefinite relaxation. Suitable convex relaxation methods are proposed for the paper-to-reviewer assignment problem as well. Our methods are tested using real conference data for both problems, and show very promising results. In a related but distinct research direction, the third part of the thesis focuses on preference measurement applications: Review profiling, i.e., determining the reviewer’s expertise (and thus identifying the associated feature vector for the reviewer) on the basis of their past and present review preferences, or ‘bids’, is an excellent example of preference measurement. We argue that the need for robust preference measurement is apparent in modern applications. Using conjoint analysis (CA) as a basis, we propose a new statistical model for choice-based preference measurement, a part of preference analysis where data are only expressed in the form of binary choices. The model uses deterministic auxiliary variables to account for outliers and to detect the salient features that influence decisions. Contributions include conditions for statistical identifiability, derivation of the pertinent Cramér-Rao Lower Bound (CRLB), and ML consistency conditions for the proposed nonlinear model. The proposed ML approach lends itself naturally to ℓ1-type convex relaxations which are well-suited for distributed implementation, based on the alternating direction method of multipliers (ADMM). A particular decomposition is advocated which bypasses the apparent need for outlier variable communication, thus maintaining scalability. In the last part of the thesis we argue that this modeling has greater intellectual merits than preference measurement, and explain how related ideas can be put in the context of generalized linear regression models, drawing links between ℓ1-methods, stochastic convex optimization, and the field of robust statistics.

QC 20140902

This thesis examines factors that are of most statistical significance for the sales prices of apartments in the Stockholm City Centre. Factors examined are address, area, balcony, construction year, elevator, fireplace, floor number, maisonette, monthly fee, penthouse and number of rooms. On the basis of this examination, a model for predicting prices of apartments is constructed. In order to evaluate how the factors influence the price, this thesis analyses sales statistics and the mathematical method used is the multiple linear regression model. In a minor case-study and literature review, included in this thesis, the relationship between proximity to public transport and the prices of apartments in Stockholm are examined. The result of this thesis states that it is possible to construct a model, from the factors analysed, which can predict the prices of apartments in Stockholm City Centre with an explanation degree of 91% and a two million SEK confidence interval of 95%. Furthermore, a conclusion can be drawn that the model predicts lower priced apartments more accurately. In the case-study and literature review, the result indicates support for the hypothesis that proximity to public transport is positive for the price of an apartment. However, such a variable should be regarded with caution due to the purpose of the modelling, which differs between an individual application and a social economic application
Denna uppsats undersöker faktorer som är av störst statistisk signifikans för priset vid försäljning av lägenheter i Stockholms innerstad. Faktorer som undersöks är adress, yta, balkong, byggår, hiss, kakelugn, våningsnummer, etage, månadsavgift, vindsvåning och antal rum. Utifrån denna undersökning konstrueras en modell för att predicera priset på lägenheter. För att avgöra vilka faktorer som påverkar priset på lägenheter analyseras försäljningsstatistik. Den matematiska metoden som används är multipel linjär regressionsanalys. I en mindre litteratur- och fallstudie, inkluderad i denna uppsats, undersöks sambandet mellan närhet till kollektivtrafik och priset på läagenheter i Stockholm.   Resultatet av denna uppsats visar att det är möjligt att konstruera en modell, utifrån de faktorer som undersöks, som kan predicera priset på läagenheter i Stockholms innerstad med en förklaringsgrad på 91 % och ett två miljoner SEK konfidensintervall på 95 %. Vidare dras en slutsats att modellen preciderar lägenheter med ett lägre pris noggrannare. I litteratur- och fallstudien indikerar resultatet stöd för hypotesen att närhet till kollektivtrafik är positivt för priset på en lägenhet. Detta skall dock betraktas med försiktighet med anledning av syftet med modelleringen vilket skiljer sig mellan en individuell tillämpning och en samhällsekonomisk tillämpning.

在機器學習領域，我們通常有很多的特徵變量，以確定一些回應變量的行為。例如在基因測試問題，我們有數以萬計的基因用來作為特徵變量，而它們與某些疾病的關係需要被確定。沒有提供具體的知識，最簡單和基本的方法來模擬這種問題會是一個線性的模型。有很多現成的方法來解決線性回歸問題，像傳統的普通最小二乘回歸法，嶺回歸和套索回歸。設 N 為樣本數和，p 為特徵變量數，在普通的情況下，我們通常有足夠的樣本（N> P）。 在這種情況下，普通線性回歸的方法，例如嶺回歸通常會給予合理的對未來的回應變量測值的預測。隨著現代統計學的發展，我們經常會遇到高維問題（N << P），如 DNA 芯片數據的測試問題。在這些類型的高維問題中，確定特徵變量和回應變量之間的關係在沒有任何進一步的假設的情況下是相當困難的。在很多現實問題中，儘管有大量的特徵變量存在，但是完全有可能只有極少數的特徵變量和回應變量有直接關係，而大部分其他的特徵變量都是無效的。 套索和嶺回歸等傳統線性回歸在高維問題中有其局限性。套索回歸在應用於高維問題時，會因為測量噪聲的存在而表現得很糟糕，這將導致非常低的預測準確率。嶺回歸也有其明顯的局限性。它不能夠分開真正的特徵變量和無效的特徵變量。我提出的新方法的目的就是在高維線性回歸中克服以上兩種方法的局限性，從而導致更精確和穩定的預測。想法其實很簡單，與其做一個單一步驟的線性回歸，我們將回歸過程分成兩個步驟。第一步，我们棄那些預測有相關性很小或為零的特徵變量。第二步，我們應該得到一個消減過的特徵變量集，我們將用這個集和回應變量來進行嶺回歸從而得到我們需要的結果。
In the field of statistical learning, we usually have a lot of features to determine the behavior of some response. For example in gene testing problems we have lots of genes as features and their relations with certain disease need to be determined. Without specific knowledge available, the most simple and fundamental way to model this kind of problem would be a linear model. There are many existing method to solve linear regression, like conventional ordinary least squares, ridge regression and LASSO (least absolute shrinkage and selection operator). Let N denote the number of samples and p denote the number of predictors, in ordinary settings where we have enough samples (N > p), ordinary linear regression methods like ridge regression will usually give reasonable predictions for the future values of the response. In the development of modern statistical learning, it's quite often that we meet high dimensional problems (N << p), like documents classification problems and microarray data testing problems. In high-dimensional problems it is generally quite difficult to identify the relationship between the predictors and the response without any further assumptions. Despite the fact that there are many predictors for prediction, most of the predictors are actually spurious in a lot of real problems. A predictor being spurious means that it is not directly related to the response. For example in microarray data testing problems, millions of genes may be available for doing prediction, but only a few hundred genes are actually related to the target disease. Conventional techniques in linear regression like LASSO and ridge regression both have their limitations in high-dimensional problems. The LASSO is one of the "state of the art technique for sparsity recovery, but when applied to high-dimensional problems, LASSO's performance is degraded a lot due to the presence of the measurement noise, which will result in high variance prediction and large prediction error. Ridge regression on the other hand is more robust to the additive measurement noise, but has its obvious limitation of not being able to separate true predictors from spurious predictors. As mentioned previously in many high-dimensional problems a large number of the predictors could be spurious, then in these cases ridge's disability in separating spurious and true predictors will result in poor interpretability of the model as well as poor prediction performance. The new technique that I will propose in this thesis aims to accommodate for the limitations of these two methods thus resulting in more accurate and stable prediction performance in a high-dimensional linear regression problem with signicant measurement noise. The idea is simple, instead of the doing a single step regression, we divide the regression procedure into two steps. In the first step we try to identify the seemingly relevant predictors and those that are obviously spurious by calculating the uni-variant correlations between the predictors and the response. We then discard those predictors that have very small or zero correlation with the response. After the first step we should have obtained a reduced predictor set. In the second step we will perform a ridge regression between the reduced predictor set and the response, the result of this ridge regression will then be our desired output. The thesis will be organized as follows, first I will start with a literature review about the linear regression problem and introduce in details about the ridge and LASSO and explain more precisely about their limitations in high-dimensional problems. Then I will introduce my new method called supervised ridge regression and show the reasons why it should dominate the ridge and LASSO in high-dimensional problems, and some simulation results will be demonstrated to strengthen my argument. Finally I will conclude with the possible limitations of my method and point out possible directions for further investigations.
Detailed summary in vernacular field only.
Zhu, Xiangchen.
Thesis (M.Phil.)--Chinese University of Hong Kong, 2013.
Includes bibliographical references (leaves 68-69).
Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web.
Abstracts also in Chinese.
Chapter 1. --- BASICS ABOUT LINEAR REGRESSION --- p.2
Chapter 1.1 --- Introduction --- p.2
Chapter 1.2 --- Linear Regression and Least Squares --- p.2
Chapter 1.2.1 --- Standard Notations --- p.2
Chapter 1.2.2 --- Least Squares and Its Geometric Meaning --- p.4
Chapter 2. --- PENALIZED LINEAR REGRESSION --- p.9
Chapter 2.1 --- Introduction --- p.9
Chapter 2.2 --- Deficiency of the Ordinary Least Squares Estimate --- p.9
Chapter 2.3 --- Ridge Regression --- p.12
Chapter 2.3.1 --- Introduction to Ridge Regression --- p.12
Chapter 2.3.2 --- Expected Prediction Error And Noise Variance Decomposition of Ridge Regression --- p.13
Chapter 2.3.3 --- Shrinkage effects on different principal components by ridge regression --- p.18
Chapter 2.4 --- The LASSO --- p.22
Chapter 2.4.1 --- Introduction to the LASSO --- p.22
Chapter 2.4.2 --- The Variable Selection Ability and Geometry of LASSO --- p.25
Chapter 2.4.3 --- Coordinate Descent Algorithm to solve for the LASSO --- p.28
Chapter 3. --- LINEAR REGRESSION IN HIGH-DIMENSIONAL PROBLEMS --- p.31
Chapter 3.1 --- Introduction --- p.31
Chapter 3.2 --- Spurious Predictors and Model Notations for High-dimensional Linear Regression --- p.32
Chapter 3.3 --- Ridge and LASSO in High-dimensional Linear Regression --- p.34
Chapter 4. --- THE SUPERVISED RIDGE REGRESSION --- p.39
Chapter 4.1 --- Introduction --- p.39
Chapter 4.2 --- Definition of Supervised Ridge Regression --- p.39
Chapter 4.3 --- An Underlying Latent Model --- p.43
Chapter 4.4 --- Ridge LASSO and Supervised Ridge Regression --- p.45
Chapter 4.4.1 --- LASSO vs SRR --- p.45
Chapter 4.4.2 --- Ridge regression vs SRR --- p.46
Chapter 5. --- TESTING AND SIMULATION --- p.49
Chapter 5.1 --- A Simulation Example --- p.49
Chapter 5.2 --- More Experiments --- p.54
Chapter 5.2.1 --- Correlated Spurious and True Predictors --- p.55
Chapter 5.2.2 --- Insufficient Amount of Data Samples --- p.59
Chapter 5.2.3 --- Low Dimensional Problem --- p.62
Chapter 6. --- CONCLUSIONS AND DISCUSSIONS --- p.66
Chapter 6.1 --- Conclusions --- p.66
Chapter 6.2 --- References and Related Works --- p.68

一個社會經濟學的目標變量，經常存在兩種不同收集頻率的數據。由於較低頻率的一組數據通常由大型普查中所獲得，其準確度及可靠性會較高。因此較低頻率的一組數據一般會視作基準，用作對頻率較高的另一組數據進行修正。
在基準修正過程中，一般會假設調查誤差及目標數據的大小互相獨立，即「累加模型」。然而，現實中兩者通常是相關的，目標變量越大，調查誤差亦會越大，即「乘積模型」。對此問題，陳兆國及胡家浩提出了利用準線性回歸手法對乘積模型進行基準修正。在本論文中，假設調查誤差服從AR(1)模型，首先我們會示範如何利用準線性回歸手法及默認調查誤差模型進行基準數據修正。然後，運用基準預測的方式，提出一個對調查誤差模型的估計辦法。最後我們會比較兩者的表現以及一些選擇誤差模型的指引。
For a target socio-economic variable, two sources of data with different collecting frequencies may be available in survey data analysis. In general, due to the difference of sample size or the data source, two sets of data do not agree with each other. Usually, the more frequent observations are less reliable, and the less frequent observations are much more accurate. In benchmarking problem, the less frequent observations can be treated as benchmarks, and will be used to adjust the higher frequent data.
In the common benchmarking setting, the survey error and the target variable are always assumed to be independent (Additive case). However, in reality, they should be correlated (Multiplicative case). The larger the variable, the larger the survey error. To deal with this problem, Chen and Wu (2006) proposed a regression method called quasi-linear regression for the multiplicative case. In this paper, by assuming the survey error to be an AR(1) model, we will demonstrate the benchmarking procedure using default error model for the quasi-linear regression. Also an error modelling procedure using benchmark forecast method will be proposed. Finally, we will compare the performance of the default error model with the fitted error model.
Detailed summary in vernacular field only.
Detailed summary in vernacular field only.
Luk, Wing Pan.
Thesis (M.Phil.)--Chinese University of Hong Kong, 2012.
Includes bibliographical references (leaves 56-57).
Abstracts also in Chinese.
Chapter 1 --- Introduction --- p.1
Chapter 1.1 --- Recent Development For Benchmarking Methods --- p.2
Chapter 1.2 --- Multiplicative Case And Benchmarking Problem --- p.3
Chapter 2 --- Benchmarking With Quasi-linear Regression --- p.8
Chapter 2.1 --- Iterative Procedure For Quasi-linear Regression --- p.9
Chapter 2.2 --- Prediction Using Default Value φ --- p.16
Chapter 2.3 --- Performance Of Using Default Error Model --- p.17
Chapter 3 --- Estimation Of φ Via BM Forecasting method --- p.26
Chapter 3.1 --- Benchmark Forecasting Method --- p.26
Chapter 3.2 --- Performance Of Benchmark Forecasting Method --- p.28
Chapter 4 --- Benchmarking By The Estimated Value --- p.34
Chapter 4.1 --- Benchmarking With The Estimated Error Model --- p.35
Chapter 4.2 --- Performance Of Using Estimated Error Model --- p.36
Chapter 4.3 --- Suggestions For Selecting Error Model --- p.45
Chapter 5 --- Fitting AR(1) Model For Non-AR(1) Error --- p.47
Chapter 5.1 --- Settings For Non-AR(1) Model --- p.47
Chapter 5.2 --- Simulation Studies --- p.48
Chapter 6 --- An Illustrative Example: The Canada Total Retail Trade Se-ries --- p.50
Chapter 7 --- Conclusion --- p.54
Bibliography --- p.56

Thesis (Ph. D.)--University of Wisconsin--Madison, 1989.
Typescript. Vita. eContent provider-neutral record in process. Description based on print version record. Includes bibliographical references (leaves 101-104).