Dissertationen zum Thema „Multimodal data processing“
Geben Sie eine Quelle nach APA, MLA, Chicago, Harvard und anderen Zitierweisen an
Machen Sie sich mit Top-17 Dissertationen für die Forschung zum Thema "Multimodal data processing" bekannt.
Neben jedem Werk im Literaturverzeichnis ist die Option "Zur Bibliographie hinzufügen" verfügbar. Nutzen Sie sie, wird Ihre bibliographische Angabe des gewählten Werkes nach der nötigen Zitierweise (APA, MLA, Harvard, Chicago, Vancouver usw.) automatisch gestaltet.
Sie können auch den vollen Text der wissenschaftlichen Publikation im PDF-Format herunterladen und eine Online-Annotation der Arbeit lesen, wenn die relevanten Parameter in den Metadaten verfügbar sind.
Sehen Sie die Dissertationen für verschiedene Spezialgebieten durch und erstellen Sie Ihre Bibliographie auf korrekte Weise.
Cadène, Rémi. „Deep Multimodal Learning for Vision and Language Processing“. Electronic Thesis or Diss., Sorbonne université, 2020. http://www.theses.fr/2020SORUS277.
Der volle Inhalt der QuelleDigital technologies have become instrumental in transforming our society. Recent statistical methods have been successfully deployed to automate the processing of the growing amount of images, videos, and texts we produce daily. In particular, deep neural networks have been adopted by the computer vision and natural language processing communities for their ability to perform accurate image recognition and text understanding once trained on big sets of data. Advances in both communities built the groundwork for new research problems at the intersection of vision and language. Integrating language into visual recognition could have an important impact on human life through the creation of real-world applications such as next-generation search engines or AI assistants.In the first part of this thesis, we focus on systems for cross-modal text-image retrieval. We propose a learning strategy to efficiently align both modalities while structuring the retrieval space with semantic information. In the second part, we focus on systems able to answer questions about an image. We propose a multimodal architecture that iteratively fuses the visual and textual modalities using a factorized bilinear model while modeling pairwise relationships between each region of the image. In the last part, we address the issues related to biases in the modeling. We propose a learning strategy to reduce the language biases which are commonly present in visual question answering systems
Lizarraga, Gabriel M. „A Neuroimaging Web Interface for Data Acquisition, Processing and Visualization of Multimodal Brain Images“. FIU Digital Commons, 2018. https://digitalcommons.fiu.edu/etd/3855.
Der volle Inhalt der QuelleGimenes, Gabriel Perri. „Advanced techniques for graph analysis: a multimodal approach over planetary-scale data“. Universidade de São Paulo, 2015. http://www.teses.usp.br/teses/disponiveis/55/55134/tde-26062015-105026/.
Der volle Inhalt der QuelleAplicações como comércio eletrônico, redes de computadores, redes sociais e biologia (interação proteica), entre outras, levaram a produção de dados que podem ser representados como grafos à escala planetária { podendo possuir milhões de nós e bilhões de arestas. Tais aplicações apresentam problemas desafiadores quando a tarefa consiste em usar as informações contidas nos grafos para auxiliar processos de tomada de decisão através da descoberta de padrões não triviais e potencialmente utéis. Para processar esses grafos em busca de padrões, tanto pesquisadores como a indústria tem usado recursos de processamento distribuído organizado em clusters computacionais. Entretanto, a construção e manutenção desses clusters pode ser complexa, trazendo tanto problemas técnicos como financeiros que podem ser proibitivos em diversos casos. Por isso, torna-se desejável a capacidade de se processar grafos em larga escala usando somente um nó computacional. Para isso, foram desenvolvidos processos e algoritmos seguindo três abordagens diferentes, visando a definição de um arcabouço de análise capaz de revelar padrões, compreensão e auxiliar na tomada de decisão sobre grafos em escala planetária.
Rabhi, Sara. „Optimized deep learning-based multimodal method for irregular medical timestamped data“. Electronic Thesis or Diss., Institut polytechnique de Paris, 2022. http://www.theses.fr/2022IPPAS003.
Der volle Inhalt der QuelleThe wide adoption of Electronic Health Records in hospitals’ information systems has led to the definition of large databases grouping various types of data such as textual notes, longitudinal medical events, and tabular patient information. However, the records are only filled during consultations or hospital stays that depend on the patient’s state, and local habits. A system that can leverage the different types of data collected at different time scales is critical for reconstructing the patient’s health trajectory, analyzing his history, and consequently delivering more adapted care.This thesis work addresses two main challenges of medical data processing: learning to represent the sequence of medical observations with irregular elapsed time between consecutive visits and optimizing the extraction of medical events from clinical notes. Our main goal is to design a multimodal representation of the patient’s health trajectory to solve clinical prediction problems. Our first work built a framework for modeling irregular medical time series to evaluate the importance of considering the time gaps between medical episodes when representing a patient’s health trajectory. To that end, we conducted a comparative study of sequential neural networks and irregular time representation techniques. The clinical objective was to predict retinopathy complications for type 1 diabetes patients in the French database CaRéDIAB (Champagne Ardenne Réseau Diabetes) using their history of HbA1c measurements. The study results showed that the attention-based model combined with the soft one-hot representation of time gaps led to AUROC score of 88.65% (specificity of 85.56%, sensitivity of 83.33%), an improvement of 4.3% when compared to the LSTM-based model. Motivated by these results, we extended our framework to shorter multivariate time series and predicted in-hospital mortality for critical care patients of the MIMIC-III dataset. The proposed architecture, HiTT, improved the AUC score by 5% over the Transformer baseline. In the second step, we focused on extracting relevant medical information from clinical notes to enrich the patient’s health trajectories. Particularly, Transformer-based architectures showed encouraging results in medical information extraction tasks. However, these complex models require a large, annotated corpus. This requirement is hard to achieve in the medical field as it necessitates access to private patient data and high expert annotators. To reduce annotation cost, we explored active learning strategies that have been shown to be effective in tasks such as text classification, information extraction, and speech recognition. In addition to existing methods, we defined a Hybrid Weighted Uncertainty Sampling active learning strategy that takes advantage of the contextual embeddings learned by the Transformer-based approach to measuring the representativeness of samples. A simulated study using the i2b2-2010 challenge dataset showed that our proposed metric reduces the annotation cost by 70% to achieve the same score as passive learning. Lastly, we combined multivariate medical time series and medical concepts extracted from clinical notes of the MIMIC-III database to train a multimodal transformer-based architecture. The test results of the in-hospital mortality task showed an improvement of 5.3% when considering additional text data. This thesis contributes to patient health trajectory representation by alleviating the burden of episodic medical records and the manual annotation of free-text notes
Dean, David Brendan. „Synchronous HMMs for audio-visual speech processing“. Thesis, Queensland University of Technology, 2008. https://eprints.qut.edu.au/17689/3/David_Dean_Thesis.pdf.
Der volle Inhalt der QuelleDean, David Brendan. „Synchronous HMMs for audio-visual speech processing“. Queensland University of Technology, 2008. http://eprints.qut.edu.au/17689/.
Der volle Inhalt der QuelleOuenniche, Kaouther. „Multimodal deep learning for audiovisual production“. Electronic Thesis or Diss., Institut polytechnique de Paris, 2023. http://www.theses.fr/2023IPPAS020.
Der volle Inhalt der QuelleWithin the dynamic landscape of television content, the critical need to automate the indexing and organization of archives has emerged as a paramount objective. In response, this research explores the use of deep learning techniques to automate the extraction of diverse metadata from television archives, improving their accessibility and reuse.The first contribution of this research revolves around the classification of camera motion types. This is a crucial aspect of content indexing as it allows for efficient categorization and retrieval of video content based on the visual dynamics it exhibits. The novel approach proposed employs 3D convolutional neural networks with residual blocks, a technique inspired by action recognition methods. A semi-automatic approach for constructing a reliable camera motion dataset from publicly available videos is also presented, minimizing the need for manual intervention. Additionally, the creation of a challenging evaluation dataset, comprising real-life videos shot with professional cameras at varying resolutions, underlines the robustness and generalization power of the proposed technique, achieving an average accuracy rate of 94%.The second contribution centers on the demanding task of Video Question Answering. In this context, we explore the effectiveness of attention-based transformers for facilitating grounded multimodal learning. The challenge here lies in bridging the gap between the visual and textual modalities and mitigating the quadratic complexity of transformer models. To address these issues, a novel framework is introduced, which incorporates a lightweight transformer and a cross-modality module. This module leverages cross-correlation to enable reciprocal learning between text-conditioned visual features and video-conditioned textual features. Furthermore, an adversarial testing scenario with rephrased questions highlights the model's robustness and real-world applicability. Experimental results on benchmark datasets, such as MSVD-QA and MSRVTT-QA, validate the proposed methodology, with an average accuracy of 45% and 42%, respectively, which represents notable improvements over existing approaches.The third contribution of this research addresses the multimodal video captioning problem, a critical aspect of content indexing. The introduced framework incorporates a modality-attention module that captures the intricate relationships between visual and textual data using cross-correlation. Moreover, the integration of temporal attention enhances the model's ability to produce meaningful captions, considering the temporal dynamics of video content. Our work also incorporates an auxiliary task employing a contrastive loss function, which promotes model generalization and a deeper understanding of inter-modal relationships and underlying semantics. The utilization of a transformer architecture for encoding and decoding significantly enhances the model's capacity to capture interdependencies between text and video data. The research validates the proposed methodology through rigorous evaluation on the MSRVTT benchmark,viachieving BLEU4, ROUGE, and METEOR scores of 0.4408, 0.6291 and 0.3082, respectively. In comparison to state-of-the-art methods, this approach consistently outperforms, with performance gains ranging from 1.21% to 1.52% across the three metrics considered.In conclusion, this manuscript offers a holistic exploration of deep learning-based techniques to automate television content indexing, addressing the labor-intensive and time-consuming nature of manual indexing. The contributions encompass camera motion type classification, VideoQA, and multimodal video captioning, collectively advancing the state of the art and providing valuable insights for researchers in the field. These findings not only have practical applications for content retrieval and indexing but also contribute to the broader advancement of deep learning methodologies in the multimodal context
Bernardi, Dario. „A feasibility study on pairinga smartwatch and a mobile devicethrough multi-modal gestures“. Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-254387.
Der volle Inhalt der QuelleParkoppling är processen för att etablera en anslutning mellan två personliga enheter. Även om den processen rent intuitivt verkar väldigt enkel så är det en utmaning att göra det säkert på grund av en uppsjö olika attackvektorer och användbarhets-relaterade problem. Faktum är att angripare kanske vill spionera på kommunikationen mellan enheterna för att samla information, eller skada enheten. Dessutom kvarstår problemet att erbjuda användaren ett simpelt och användarvänligt sätt att parkoppla enheter som håller en hög nivå av säkerhet. På grund av mängden av olika enheter och parkopplingsscenarier är det helt enkelt inte möjligt att skapa ett enskilt säkert sätt att parkoppla enheter på.I den här uppsatsen studerar vi genomförbarheten av ett nytt parkopplingsschema baserat på kombinerade rörelser, nämligen en målande rörelse supportat av data från accelerometern. I synnerhet kan en användare parkoppla en smart klocka på sin handled med en mobiltelefon genom att måla med sitt finger på mobiltelefonens skärm. För ändamålet utvecklar vi en mobilapplikation för smarta klocka och mobiltelefoner för att testa och processa inhämtad data som support för ett säkert engagemangsbaserat protokoll. Utöver det genomförde vi ett antal experiment för att verifiera om synkroniserade rörelser har tydliga liknelser i jämförelse med icke synkroniserade rörelser.Resultatet visade att det är genomförbart att implementera ett sådant system vilket också erbjuder användaren ett naturligt sätt att genomföra en säker parkoppling. Detta innovativa system kan komma att användas av ett stort antal mobila enheter (t.ex. smarta klockor, mobiltelefoner, surfplattor etc) i olika scenarion.
Mozaffari, Maaref Mohammad Hamed. „A Real-Time and Automatic Ultrasound-Enhanced Multimodal Second Language Training System: A Deep Learning Approach“. Thesis, Université d'Ottawa / University of Ottawa, 2020. http://hdl.handle.net/10393/40477.
Der volle Inhalt der QuelleBenmoussat, Mohammed Seghir. „Hyperspectral imagery algorithms for the processing of multimodal data : application for metal surface inspection in an industrial context by means of multispectral imagery, infrared thermography and stripe projection techniques“. Thesis, Aix-Marseille, 2013. http://www.theses.fr/2013AIXM4347/document.
Der volle Inhalt der QuelleThe work presented in this thesis deals with the quality control and inspection of industrial metallic surfaces. The purpose is the generalization and application of hyperspectral imagery methods for multimodal data such as multi-channel optical images and multi-temporal thermographic images. In the first application, data cubes are built from multi-component images to detect surface defects within flat metallic parts. The best performances are obtained with multi-wavelength illuminations in the visible and near infrared ranges, and detection using spectral angle mapper with mean spectrum as a reference. The second application turns on the use of thermography imaging for the inspection of nuclear metal components to detect surface and subsurface defects. A 1D approach is proposed based on using the kurtosis to select 1 principal component (PC) from the first PCs obtained after reducing the original data cube with the principal component analysis (PCA) algorithm. The proposed PCA-1PC method gives good performances with non-noisy and homogeneous data, and SVD with anomaly detection algorithms gives the most consistent results and is quite robust to perturbations such as inhomogeneous background. Finally, an approach based on fringe analysis and structured light techniques in case of deflectometric recordings is presented for the inspection of free-form metal surfaces. After determining the parameters describing the sinusoidal stripe patterns, the proposed approach consists in projecting a list of phase-shifted patterns and calculating the corresponding phase-images. Defect location is based on detecting and analyzing the stripes within the phase-images
Neumann, Markus. „Automatic multimodal real-time tracking for image plane alignment in interventional Magnetic Resonance Imaging“. Phd thesis, Université de Strasbourg, 2014. http://tel.archives-ouvertes.fr/tel-01038023.
Der volle Inhalt der QuelleMuliukov, Artem. „Étude croisée des cartes auto-organisatrices et des réseaux de neurones profonds pour l'apprentissage multimodal inspiré du cerveau“. Electronic Thesis or Diss., Université Côte d'Azur, 2024. https://intranet-theses.unice.fr/2024COAZ4008.
Der volle Inhalt der QuelleCortical plasticity is one of the main features that enable our capability to learn and adapt in our environment. Indeed, the cerebral cortex has the ability to self-organize itself through two distinct forms of plasticity: the structural plasticity and the synaptic plasticity. These mechanisms are very likely at the basis of an extremely interesting characteristic of the human brain development: the multimodal association. The brain uses spatio-temporal correlations between several modalities to structure the data and create sense from observations. Moreover, biological observations show that one modality can activate the internal representation of another modality when both are correlated. To model such a behavior, Edelman and Damasio proposed respectively the Reentry and the Convergence Divergence Zone frameworks where bi-directional neural communications can lead to both multimodal fusion (convergence) and inter-modal activation (divergence). Nevertheless, these frameworks do not provide a computational model at the neuron level, and only few works tackle this issue of bio-inspired multimodal association which is yet necessary for a complete representation of the environment especially when targeting autonomous and embedded intelligent systems. In this doctoral project, we propose to pursue the exploration of brain-inspired computational models of self-organization for multimodal unsupervised learning in neuromorphic systems. These neuromorphic architectures get their energy-efficient from the bio-inspired models they support, and for that reason we only consider in our work learning rules based on local and distributed processing
Boscaro, Anthony. „Analyse multimodale et multicritères pour l'expertise et la localisation de défauts dans les composants électriques modernes“. Thesis, Bourgogne Franche-Comté, 2017. http://www.theses.fr/2017UBFCK014/document.
Der volle Inhalt der QuelleThe purpose of this manuscript is to exhibit the research work solving the issue of data processing stem from defect localization techniques. This step being decisive in the failure analysis process, scientists have to harness data coming from light emission and laser techniques. Nevertheless, this analysis process is sequential and only depends on the expert’s decision. This factor leads to a not quantified probability of localization. Consequently to solve these issues, a multimodaland multicriteria analysis has been developped, taking advantage of the heterogeneous and complementary nature of light emission and laser probing techniques. This kind of process is based on advanced level tools such as signal/image processing and data fusion. The final aim being to provide a quantitive and qualitative decision help for the experts.The first part of this manuscript is dedicated to the description of the entire process for 1D and 2D data enhancement. Thereafter, the spatio-temporal analysis of laser probing waveforms will be tackled. Finally, the last part highlights the decision support brought by data fusion
„Graph-based approaches for multimodal brain imaging data analysis“. Tulane University, 2021.
Den vollen Inhalt der Quelle findenChen, I.-Wei, und 陳弈暐. „An Integrated Electrocardiography and Photoplethysmography Signal Processing System Based on Ensemble Empirical Mode Decomposition Method for Multimodal Physiological Data Monitoring“. Thesis, 2018. http://ndltd.ncl.edu.tw/handle/yk4fna.
Der volle Inhalt der QuelleFürbach, Radek. „Metody lokalizace rozdílů v různých modálitách malířských děl“. Master's thesis, 2013. http://www.nusl.cz/ntk/nusl-328544.
Der volle Inhalt der QuelleNouri, Golmaei Sara. „Improving the Performance of Clinical Prediction Tasks by using Structured and Unstructured Data combined with a Patient Network“. Thesis, 2021. http://dx.doi.org/10.7912/C2/41.
Der volle Inhalt der QuelleWith the increasing availability of Electronic Health Records (EHRs) and advances in deep learning techniques, developing deep predictive models that use EHR data to solve healthcare problems has gained momentum in recent years. The majority of clinical predictive models benefit from structured data in EHR (e.g., lab measurements and medications). Still, learning clinical outcomes from all possible information sources is one of the main challenges when building predictive models. This work focuses mainly on two sources of information that have been underused by researchers; unstructured data (e.g., clinical notes) and a patient network. We propose a novel hybrid deep learning model, DeepNote-GNN, that integrates clinical notes information and patient network topological structure to improve 30-day hospital readmission prediction. DeepNote-GNN is a robust deep learning framework consisting of two modules: DeepNote and patient network. DeepNote extracts deep representations of clinical notes using a feature aggregation unit on top of a state-of-the-art Natural Language Processing (NLP) technique - BERT. By exploiting these deep representations, a patient network is built, and Graph Neural Network (GNN) is used to train the network for hospital readmission predictions. Performance evaluation on the MIMIC-III dataset demonstrates that DeepNote-GNN achieves superior results compared to the state-of-the-art baselines on the 30-day hospital readmission task. We extensively analyze the DeepNote-GNN model to illustrate the effectiveness and contribution of each component of it. The model analysis shows that patient network has a significant contribution to the overall performance, and DeepNote-GNN is robust and can consistently perform well on the 30-day readmission prediction task. To evaluate the generalization of DeepNote and patient network modules on new prediction tasks, we create a multimodal model and train it on structured and unstructured data of MIMIC-III dataset to predict patient mortality and Length of Stay (LOS). Our proposed multimodal model consists of four components: DeepNote, patient network, DeepTemporal, and score aggregation. While DeepNote keeps its functionality and extracts representations of clinical notes, we build a DeepTemporal module using a fully connected layer stacked on top of a one-layer Gated Recurrent Unit (GRU) to extract the deep representations of temporal signals. Independent to DeepTemporal, we extract feature vectors of temporal signals and use them to build a patient network. Finally, the DeepNote, DeepTemporal, and patient network scores are linearly aggregated to fit the multimodal model on downstream prediction tasks. Our results are very competitive to the baseline model. The multimodal model analysis reveals that unstructured text data better help to estimate predictions than temporal signals. Moreover, there is no limitation in applying a patient network on structured data. In comparison to other modules, the patient network makes a more significant contribution to prediction tasks. We believe that our efforts in this work have opened up a new study area that can be used to enhance the performance of clinical predictive models.