Добірка наукової літератури з теми "Multitask regression"

The study of multitask learning algorithms is one of very important issues. This paper proposes a least-square regularized regression algorithm for multi-task learning with hypothesis space being the union of a sequence of Hilbert spaces. The algorithm consists of two steps of selecting the optimal Hilbert space and searching for the optimal function. We assume that the distributions of different tasks are related to a set of transformations under which any Hilbert space in the hypothesis space is norm invariant. We prove that under the above assumption the optimal prediction function of every task is in the same Hilbert space. Based on this result, a pivotal error decomposition is founded, which can use samples of related tasks to bound excess error of the target task. We obtain an upper bound for the sample error of related tasks, and based on this bound, potential faster learning rates are obtained compared to single-task learning algorithms.

Different energy systems are closely connected with each other in industrial-park integrated energy system (IES). The energy demand forecasting has important impact on IES dispatching and planning. This paper proposes an approach of short-term energy forecasting for electricity, heat, and gas by employing deep multitask learning whose structure is constructed by deep belief network (DBN) and multitask regression layer. The DBN can extract abstract and effective characteristics in an unsupervised fashion, and the multitask regression layer above the DBN is used for supervised prediction. Then, subject to condition of practical demand and model integrity, the whole energy forecasting model is introduced, including preprocessing, normalization, input properties, training stage, and evaluating indicator. Finally, the validity of the algorithm and the accuracy of the energy forecasts for an industrial-park IES system are verified through the simulations using actual operating data from load system. The positive results turn out that the deep multitask learning has great prospects for load forecast.

Vital for Space Situational Awareness, Initial Orbit Determination (IOD) may be used to initialize object tracking and associate observations with a tracked satellite. Classical IOD algorithms provide only a point solution and are sensitive to noisy measurements and to certain target-observer geometry. This work examines the ability of a Multivariate GPR (MV-GPR) to accurately perform IOD and quantify the associated uncertainty. Given perfect test inputs, MV-GPR performs comparably to a simpler multitask learning GPR algorithm and the classical Gauss–Gibbs IOD in terms of prediction accuracy. It significantly outperforms the multitask learning GPR algorithm in uncertainty quantification due to the direct handling of output dimension correlations. A moment-matching algorithm provides an analytic solution to the input noise problem under certain assumptions. The algorithm is adapted to the MV-GPR formulation and shown to be an effective tool to accurately quantify the added input uncertainty. This work shows that the MV-GPR can provide a viable solution with quantified uncertainty which is robust to observation noise and traditionally challenging orbit-observer geometries.

Various factors make stock market forecasting difficult and arduous. Single-task learning models fail to achieve good results because they ignore the correlation between multiple related tasks. Multitask learning methods can capture the cross-correlation among subtasks and achieve a satisfactory learning effect by training all tasks simultaneously. With this motivation, we assume that the related tasks are close enough to share a common model whereas having their own independent models. Based on this hypothesis, we propose a multitask learning least squares support vector regression (MTL-LS-SVR) algorithm, and an extension, EMTL-LS-SVR. Theoretical analysis shows that these models can be converted to linear systems. A Krylov-Cholesky algorithm is introduced to determine the optimal solutions of the models. We tested the proposed models by applying them to forecasts of the Chinese stock market index trend and the stock prices of five stated-owned banks. The experimental results demonstrate their validity.

Given the impact of renewable sources in the overall energy production, accurate predictions are becoming essential, with machine learning becoming a very important tool in this context. In many situations, the prediction problem can be divided into several tasks, more or less related between them but each with its own particularities. Multitask learning (MTL) aims to exploit this structure, training several models at the same time to improve on the results achievable either by a common model or by task-specific models. In this paper, we show how an MTL approach based on support vector regression can be applied to the prediction of photovoltaic and wind energy, problems where tasks can be defined according to different criteria. As shown experimentally with three different datasets, the MTL approach clearly outperforms the results of the common and specific models for photovoltaic energy, and are at the very least quite competitive for wind energy.

Les dispositifs d'imagerie cérébrale peuvent donner un aperçu de l'activité neuronale à plusieurs endroits et points dans le temps. En pratique, les études d'imagerie cérébrales sont généralement menées pour plusieurs personnes suivant le même protocole expérimental. L'inférence des régions actives du cerveau est un problème inverse mal posé qui ne peut être résolu qu'en ajoutant des hypothèses a priori sur les solutions. Plusieurs hypothèses préalables ont été poursuivies dans la littérature, comme la favorisation des solutions parcimonieuses ou la résolution du problème pour plusieurs sujets à la fois. Cependant, aucune ne profite de la géométrie spatiale du problème. Le but de cette thèse est d'exploiter au maximum les aspects multisujets, spatiaux et temporels des données de magnétoencéphalographie pour améliorer le conditionnement du problème inverse. À cette fin, nos contributions s'articulent autour de trois axes : le transport optimal (OT), la régression multi-tâches parcimonieuse et les séries temporelles. En effet, la capacité de l'OT à mesurer les disparités spatiales entre les distributions le rend très bien adapté à la comparaison et l'aggrégation des cartes d'activation neurales en fonction de leur forme et de leur emplacement sur la surface du cortex cérébral. Pour des raisons numériques, on utilise la formulation entropique du transport optimal, qui, selon nous, comporte deux pièces manquantes importantes. D'un point de vue théorique, elle n'a aucune expression analytique à ce jour, et d'un point de vue pratique, l'entropie conduit à une augmentation significative de la variance, phénomène connu sous le nom de biais entropique. Nous complétons ce puzzle en étudiant les Gaussiennes multivariées pour lesquelles nous découvrons une forme close de l'OT entropique et proposons des algorithmes débiaisés pour calculer des barycentres de transport optimal rapides et précis. Ensuite, nous définissons une pénalité multitâche basé sur l'OT et des pénalités de parcimonie pour résoudre le problème inverse pour plusieurs sujets afin de promouvoir des solutions cohérentes sur le plan spatial. Nos résultats sur des données réelles mettent en évidence les avantages de l'utilisation de l'OT comme régularisation par rapport aux pénalités de régression multitâches classiques. Enfin, nous proposons une nouvelle divergence pour comparer et moyenner des données spatio-temporelles basée sur un alignement temporel entre des observations spatialement similaires, le tout via un algorithme rapide et adapté aux GPUs
Brain imaging devices can provide a glimpse at neural activity in multiple spatial locations and time points. Moreover, neuroimaging studies are usually conducted for multiple individuals undergoing the same experimental protocol. Inferring the underlying sources is a challenging inverse problem that can only be tackled by biasing the solutions with prior domain knowledge. Several prior hypotheses have been pursued in the literature such as promoting sparse over dense solutions or solving the problem for multiple subjects at once. However, none take advantage of the particular spatial geometry of the problem. The purpose of this thesis is to exploit the multi-subject, spatial and temporal aspects of magneto-encephalography data as much as possible to improve the conditioning of the inverse problem. To that end, our contributions revolve around three axes: optimal transport (OT), sparse multi-task regression and time series. Indeed, the ability of OT to capture spatial disparities between measures makes it very well suited to compare and average neural activation patterns based on their shape and location over the cortical surface of the brain. For the sake of scalability, we take advantage of the entropic formulation of optimal transport, which we argue has two important missing pieces. From a theoretical perspective, it has no closed form analytical expressions, and from a practical perspective, entropy leads to a significant increase in variance known as "entropic bias". We complete this puzzle by studying multivariate Gaussians for which we uncover an entropic OT closed form and propose "debiased" algorithms to compute fast and accurate optimal transport barycenters. Second, we define a multi-task prior based on OT and sparse penalties to jointly solve the inverse problem for multiple subjects to promote spatially coherent solutions. Our real data experiments highlight the benefits of using OT as a prior over classical multi-task regression penalties. Finally, we propose a loss function to compare and average spatio-temporal data that computes temporal alignments across spatially similar observations of the data via a fast GPU friendly algorithm

Les simulateurs déterministes de neutronique pour les réacteurs nucléaires suivent aujourd'hui majoritairement un schéma multi-échelles à deux étapes. Au cours d'un calcul dit « réseau », la physique est finement résolue au niveau des motifs élémentaires du réacteur (assemblages de combustible) ; puis, ces motifs sont mis en contact dans un calcul dit « cœur », où la configuration globale est calculée de manière plus grossière. La communication entre ces deux codes se fait de manière différée par le transfert de données physiques, dont les plus importantes se nomment « sections efficaces homogénéisées » (notées ci-après HXS) et peuvent être représentées par des fonctions multivariées. Leur utilisation différée et leur dépendance à des conditions physiques variables imposent un schéma de type tabulation-interpolation : les HXS sont précalculées dans une large gamme de situations, stockées, puis approximées dans le code cœur à partir de ces données afin de correspondre à un état bien précis du réacteur. Dans un contexte d'augmentation de la finesse des simulations, les outils mathématiques actuellement utilisés pour cette étape d'approximation montrent aujourd'hui leurs limites ; la problématique de cette thèse est ainsi de leur trouver des remplaçants, capables de rendre l'interpolation des HXS plus précise, plus économe en données et en espace de stockage, et tout aussi rapide. Tout l'arsenal du machine learning, de l'approximation fonctionnelle, etc, peut être mis à contribution pour traiter ce problème.Afin de trouver un modèle d'approximation adapté au problème, l'on a commencé par une analyse des jeux de données générés pour cette thèse : corrélations entre les HXS, allure de leurs dépendances, dimension linéaire, etc. Ce dernier point s'est révélé particulièrement fructueux : les jeux de HXS s'avèrent être d'une très faible dimension effective, ce qui permet de simplifier grandement leur approximation. En particulier, l'on a développé une méthodologie innovante basée sur l'Empirical Interpolation Method (EIM), capable de remplacer la majorité des appels au code réseau par des extrapolations d'un petit volume de données, et de réduire le stockage des HXS d'un ou deux ordres de grandeur - le tout occasionnant une perte de précision négligeable. Pour conserver les avantages d'une telle méthodologie tout en répondant à la totalité de la problématique de thèse, l'on s'est ensuite tourné vers un puissant modèle de machine learning épousant la même structure de faible dimension : les processus gaussiens multi-sorties (MOGP). Procédant par étapes depuis les modèles gaussiens les plus simples (GP mono-sorties) jusqu'à de plus complexes, l'on a montré que ces outils sont pleinement adaptés au problème considéré, et permettent des gains majeurs par rapport à l'existant. De nombreux choix de modélisation ont été discutés et comparés ; les modèles ont été adaptés à des données de très grande taille, requérant une optimisation de leur implémentation ; et les fonctionnalités nouvelles qu'ils offrent ont été expérimentées, notamment la prédiction d'incertitudes et l'apprentissage actif.Enfin, un travail théorique a été accompli sur la famille de modèles étudiées - le Linear Model of Co-regionalisation (LMC) - afin d'éclairer certaines zones d'ombre de leur théorie encore jeune. Cette réflexion a mené à la définition d'un nouveau modèle, le PLMC, qui a été implémenté, optimisé et testé sur de nombreux jeux de données réelles et synthétiques. Plus simple que ses concurrents, ce modèle s'est aussi révélé autant voire plus précis et rapide, et doté de plusieurs fonctionnalités exclusives, mises à profit durant la thèse.Ce travail ouvre de multiples perspectives pour la simulation neutronique. Doté de modèles d'apprentissage puissants et flexibles, l'on peut envisager des évolutions importantes des codes : propagation systématique des incertitudes, correction de diverses approximations, prise en compte de davantage de variables…
Today, most deterministic neutronics simulators for nuclear reactors follow a two-step multi-scale scheme. In a so-called “lattice” calculation, the physics is finely resolved at the level of the elementary reactor pattern (fuel assemblies); these tiles are then brought into contact in a so-called “core” calculation, where the overall configuration is calculated more coarsely. Communication between these two codes is realized by the deferred transfer of physical data, the most important of which are called “homogenized cross sections” (hereafter referred to as HXS) and can be represented by multivariate functions. Their deferred use and dependence on variable physical conditions call for a tabulation-interpolation scheme: HXS are precalculated in a wide range of situations, stored, then approximated in the core code from the stored values to correspond to a specific reactor state. In a context of increasing simulation finesse, the mathematical tools currently used for this approximation stage are now showing their limitations. The aim of this thesis is to find replacements for them, capable of making HXS interpolation more accurate, more economical in terms of data and storage space, and just as fast. The whole arsenal of machine learning, functional approximation, etc., can be put at use to tackle this problem.In order to find a suitable approximation model, we began by analyzing the datasets generated for this thesis: correlations between HXS's, shapes of their dependencies, linear dimension, etc. This last point proved particularly fruitful: HXS sets turn out to be of very low effective dimension, which greatly simplifies their approximation. In particular, we leveraged this fact to develop an innovative methodology based on the Empirical Interpolation Method (EIM), capable of replacing the majority of lattice code calls by extrapolations from a small volume of data, and reducing HXS storage by one or two orders of magnitude - all with a negligible loss of accuracy.To retain the advantages of such a methodology while addressing the full scope of the thesis problem, we then turned to a powerful machine learning model matching the same low-dimensional structure: multi-output Gaussian processes (MOGPs). Proceeding step by step from the simplest Gaussian models (single-output GPs) to most complex ones, we showed that these tools are fully adapted to the problem under consideration, and offer major gains over current HXS interpolation routines. Numerous modeling choices were discussed and compared; models were adapted to very large data, requiring some optimization of their implementation; and the new functionalities which they offer were tested, notably uncertainty prediction and active learning.Finally, theoretical work was carried out on the studied family of models - the Linear Model of Co-regionalisation (LMC) - in order to shed light on certain grey areas in their still young theory. This led to the definition of a new model, the PLMC, which was implemented, optimized and tested on numerous real and synthetic data sets. Simpler than its competitors, this model has also proved to be just as accurate and fast if not more so, and holds a number of exclusive functionalities that were put to good use during the thesis.This work opens up many new prospects for neutronics simulation. Equipped with powerful and flexible learning models, it is possible to envisage significant evolutions for deterministic codes: systematic propagation of uncertainties, correction of various approximations, taking into account of more variables

This article describes solutions to couple of problems: CMU-MOSEI database preprocessing to improve data quality and bimodal multitask classification of emotions and sentiments. With the help of experimental studies, representative features for acoustic and linguistic information are identified among pretrained neural networks with Transformer architecture. The most representative features for the analysis of emotions and sentiments are EmotionHuBERT and RoBERTa for audio and text modalities respectively. The article establishes a baseline for bimodal multitask recognition of sentiments and emotions – 63.2% and 61.3%, respectively, measured with macro F-score. Experiments were conducted with different approaches to combining modalities – concatenation and multi-head attention. The most effective architecture of neural network with early concatenation of audio and text modality and late multi-head attention for emotions and sentiments recognition is proposed. The proposed neural network is combined with logistic regression, which achieves 63.5% and 61.4% macro F-score by bimodal (audio and text) multitasking recognition of 3 sentiment classes and 6 emotion binary classes.

This work addresses the person re-identification problem, which consists on matching images of individuals captured by multiple and non-overlapping surveillance cameras. Works from literature tackle this problem proposing robust feature descriptors and matching functions, where the latter is responsible to assign the correct identity for individuals and is the focus of this work. Specifically, we propose two matching methods: the Kernel MBPLS and the Kernel X-CRC. The Kernel MBPLS is a nonlinear regression model that is scalable with respect to the number of cameras and allows the inclusion of additional labelled information (e.g., attributes). Differently, the Kernel X-CRC is a nonlinear and multitask matching function that can be used jointly with subspace learning approaches to boost the matching rates. We present an extensive experimental evaluation of both approaches in four datasets (VIPeR, PRID450S, WARD and Market-1501). Experimental results demonstrate that the Kernel MBPLS and the Kernel X-CRC outperforms approaches from literature. Furthermore, we show that the Kernel X-CRC can be successfuly applied in large-scale and multiple cameras datasets.

The use of cellphones and tablets becomes everyday more frequent at schools and universities, not only in spare time, but also during class. If cellphones are commonly used in class for purposes unrelated to the discipline, it is likely that students may be distracted during lectures or activities, as they often overestimate their ability to multitask which could eventually lead to academic underperformance. The aim of this research is to reveal the relationship between actual time spent by mobile phone users and metacognitive awareness and academic performance. Academic performance will be obtained through the self-report data. Data is planned to collect 100 students who continue their education at Gaziantep University Nizip Education Faculty. As a demographic data, the relationship between age gender and grade level and the amount of mobile phone usage time as well as the relationship between metacognitive awareness and mobile phone usage time will be analyzed by regression analysis. All analyses will be made using SPSS for Windows. First, in order to find significant differences between the average academic performance of subgroups of the sample, independent t-tests and ANOVA analyses will be performed. A hierarchical regression was performed next to verify if there is any statistically significant relationship between cellphone usage and academic performance. The author expecting that there is a negative relationship between academic performance and cell phone usage time. Also there is a negative correlation between metacognitive awareness and actual cell phone usage time. The results showed that there was no correlation between cell phone usage and metacognitive awareness. Also there was no correlation between exam scores and metacognitive awareness. This study showed that there was a strong and positive correlation between self-efficacy and metacognitive awareness.