Journal articles on the topic 'Variational autoencoder (VAE)'
To see the other types of publications on this topic, follow the link: Variational autoencoder (VAE).
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Variational autoencoder (VAE).'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Zemouri, Ryad. "Semi-Supervised Adversarial Variational Autoencoder." Machine Learning and Knowledge Extraction 2, no. 3 (September 6, 2020): 361–78. http://dx.doi.org/10.3390/make2030020.
Full textAbstract:
We present a method to improve the reconstruction and generation performance of a variational autoencoder (VAE) by injecting an adversarial learning. Instead of comparing the reconstructed with the original data to calculate the reconstruction loss, we use a consistency principle for deep features. The main contributions are threefold. Firstly, our approach perfectly combines the two models, i.e., GAN and VAE, and thus improves the generation and reconstruction performance of the VAE. Secondly, the VAE training is done in two steps, which allows to dissociate the constraints used for the construction of the latent space on the one hand, and those used for the training of the decoder. By using this two-step learning process, our method can be more widely used in applications other than image processing. While training the encoder, the label information is integrated to better structure the latent space in a supervised way. The third contribution is to use the trained encoder for the consistency principle for deep features extracted from the hidden layers. We present experimental results to show that our method gives better performance than the original VAE. The results demonstrate that the adversarial constraints allow the decoder to generate images that are more authentic and realistic than the conventional VAE.
2
Kamata, Hiromichi, Yusuke Mukuta, and Tatsuya Harada. "Fully Spiking Variational Autoencoder." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 6 (June 28, 2022): 7059–67. http://dx.doi.org/10.1609/aaai.v36i6.20665.
Full textAbstract:
Spiking neural networks (SNNs) can be run on neuromorphic devices with ultra-high speed and ultra-low energy consumption because of their binary and event-driven nature. Therefore, SNNs are expected to have various applications, including as generative models being running on edge devices to create high-quality images. In this study, we build a variational autoencoder (VAE) with SNN to enable image generation. VAE is known for its stability among generative models; recently, its quality advanced. In vanilla VAE, the latent space is represented as a normal distribution, and floating-point calculations are required in sampling. However, this is not possible in SNNs because all features must be binary time series data. Therefore, we constructed the latent space with an autoregressive SNN model, and randomly selected samples from its output to sample the latent variables. This allows the latent variables to follow the Bernoulli process and allows variational learning. Thus, we build the Fully Spiking Variational Autoencoder where all modules are constructed with SNN. To the best of our knowledge, we are the first to build a VAE only with SNN layers. We experimented with several datasets, and confirmed that it can generate images with the same or better quality compared to conventional ANNs. The code is available at https://github.com/kamata1729/FullySpikingVAE.
3
Chow, Yuen Ler, Shantanu Singh, Anne E. Carpenter, and Gregory P. Way. "Predicting drug polypharmacology from cell morphology readouts using variational autoencoder latent space arithmetic." PLOS Computational Biology 18, no. 2 (February 25, 2022): e1009888. http://dx.doi.org/10.1371/journal.pcbi.1009888.
Full textAbstract:
A variational autoencoder (VAE) is a machine learning algorithm, useful for generating a compressed and interpretable latent space. These representations have been generated from various biomedical data types and can be used to produce realistic-looking simulated data. However, standard vanilla VAEs suffer from entangled and uninformative latent spaces, which can be mitigated using other types of VAEs such as β-VAE and MMD-VAE. In this project, we evaluated the ability of VAEs to learn cell morphology characteristics derived from cell images. We trained and evaluated these three VAE variants—Vanilla VAE, β-VAE, and MMD-VAE—on cell morphology readouts and explored the generative capacity of each model to predict compound polypharmacology (the interactions of a drug with more than one target) using an approach called latent space arithmetic (LSA). To test the generalizability of the strategy, we also trained these VAEs using gene expression data of the same compound perturbations and found that gene expression provides complementary information. We found that the β-VAE and MMD-VAE disentangle morphology signals and reveal a more interpretable latent space. We reliably simulated morphology and gene expression readouts from certain compounds thereby predicting cell states perturbed with compounds of known polypharmacology. Inferring cell state for specific drug mechanisms could aid researchers in developing and identifying targeted therapeutics and categorizing off-target effects in the future.
4
Nugroho, Herminarto, Meredita Susanty, Ade Irawan, Muhamad Koyimatu, and Ariana Yunita. "Fully Convolutional Variational Autoencoder For Feature Extraction Of Fire Detection System." Jurnal Ilmu Komputer dan Informasi 13, no. 1 (March 14, 2020): 9. http://dx.doi.org/10.21609/jiki.v13i1.761.
Full textAbstract:
This paper proposes a fully convolutional variational autoencoder (VAE) for features extraction from a large-scale dataset of fire images. The dataset will be used to train the deep learning algorithm to detect fire and smoke. The features extraction is used to tackle the curse of dimensionality, which is the common issue in training deep learning with huge datasets. Features extraction aims to reduce the dimension of the dataset significantly without losing too much essential information. Variational autoencoders (VAEs) are powerfull generative model, which can be used for dimension reduction. VAEs work better than any other methods available for this purpose because they can explore variations on the data in a specific direction.
5
Zhu, Jinlin, Muyun Jiang, and Zhong Liu. "Fault Detection and Diagnosis in Industrial Processes with Variational Autoencoder: A Comprehensive Study." Sensors 22, no. 1 (December 29, 2021): 227. http://dx.doi.org/10.3390/s22010227.
Full textAbstract:
This work considers industrial process monitoring using a variational autoencoder (VAE). As a powerful deep generative model, the variational autoencoder and its variants have become popular for process monitoring. However, its monitoring ability, especially its fault diagnosis ability, has not been well investigated. In this paper, the process modeling and monitoring capabilities of several VAE variants are comprehensively studied. First, fault detection schemes are defined in three distinct ways, considering latent, residual, and the combined domains. Afterwards, to conduct the fault diagnosis, we first define the deep contribution plot, and then a deep reconstruction-based contribution diagram is proposed for deep domains under the fault propagation mechanism. In a case study, the performance of the process monitoring capability of four deep VAE models, namely, the static VAE model, the dynamic VAE model, and the recurrent VAE models (LSTM-VAE and GRU-VAE), has been comparatively evaluated on the industrial benchmark Tennessee Eastman process. Results show that recurrent VAEs with a deep reconstruction-based diagnosis mechanism are recommended for industrial process monitoring tasks.
6
Takahashi, Hiroshi, Tomoharu Iwata, Yuki Yamanaka, Masanori Yamada, and Satoshi Yagi. "Variational Autoencoder with Implicit Optimal Priors." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 5066–73. http://dx.doi.org/10.1609/aaai.v33i01.33015066.
Full textAbstract:
The variational autoencoder (VAE) is a powerful generative model that can estimate the probability of a data point by using latent variables. In the VAE, the posterior of the latent variable given the data point is regularized by the prior of the latent variable using Kullback Leibler (KL) divergence. Although the standard Gaussian distribution is usually used for the prior, this simple prior incurs over-regularization. As a sophisticated prior, the aggregated posterior has been introduced, which is the expectation of the posterior over the data distribution. This prior is optimal for the VAE in terms of maximizing the training objective function. However, KL divergence with the aggregated posterior cannot be calculated in a closed form, which prevents us from using this optimal prior. With the proposed method, we introduce the density ratio trick to estimate this KL divergence without modeling the aggregated posterior explicitly. Since the density ratio trick does not work well in high dimensions, we rewrite this KL divergence that contains the high-dimensional density ratio into the sum of the analytically calculable term and the lowdimensional density ratio term, to which the density ratio trick is applied. Experiments on various datasets show that the VAE with this implicit optimal prior achieves high density estimation performance.
7
Rákos, Olivér, Szilárd Aradi, Tamás Bécsi, and Zsolt Szalay. "Compression of Vehicle Trajectories with a Variational Autoencoder." Applied Sciences 10, no. 19 (September 26, 2020): 6739. http://dx.doi.org/10.3390/app10196739.
Full textAbstract:
The perception and prediction of the surrounding vehicles’ trajectories play a significant role in designing safe and optimal control strategies for connected and automated vehicles. The compression of trajectory data and the drivers’ strategic behavior’s classification is essential to communicate in vehicular ad-hoc networks (VANETs). This paper presents a Variational Autoencoder (VAE) solution to solve the compression problem, and as an added benefit, it also provides classification information. The input is the time series of vehicle positions along actual real-world trajectories obtained from a dataset containing highway measurements, which also serves as the target. During training, the autoencoder learns to compress and decompress this data and produces a small, few element context vector that can represent vehicle behavior in a probabilistic manner. The experiments show how the size of this context vector affects the performance of the method. The method is compared to other approaches, namely, Bidirectional LSTM Autoencoder and Sparse Convolutional Autoencoder. According to the results, the Sparse Autoencoder fails to converge to the target for the specific tasks. The Bidirectional LSTM Autoencoder could provide the same performance as the VAE, though only with double context vector length, proving that the compression capability of the VAE is better. The Support Vector Machine method is used to prove that the context vector can be used for maneuver classification for lane changing behavior. The utilization of this method, considering neighboring vehicles, can be extended for maneuver prediction using a wider, more complex network structure.
8
Zacherl, Johannes, Philipp Frank, and Torsten A. Enßlin. "Probabilistic Autoencoder Using Fisher Information." Entropy 23, no. 12 (December 6, 2021): 1640. http://dx.doi.org/10.3390/e23121640.
Full textAbstract:
Neural networks play a growing role in many scientific disciplines, including physics. Variational autoencoders (VAEs) are neural networks that are able to represent the essential information of a high dimensional data set in a low dimensional latent space, which have a probabilistic interpretation. In particular, the so-called encoder network, the first part of the VAE, which maps its input onto a position in latent space, additionally provides uncertainty information in terms of variance around this position. In this work, an extension to the autoencoder architecture is introduced, the FisherNet. In this architecture, the latent space uncertainty is not generated using an additional information channel in the encoder but derived from the decoder by means of the Fisher information metric. This architecture has advantages from a theoretical point of view as it provides a direct uncertainty quantification derived from the model and also accounts for uncertainty cross-correlations. We can show experimentally that the FisherNet produces more accurate data reconstructions than a comparable VAE and its learning performance also apparently scales better with the number of latent space dimensions.
9
Elbattah, Mahmoud, Colm Loughnane, Jean-Luc Guérin, Romuald Carette, Federica Cilia, and Gilles Dequen. "Variational Autoencoder for Image-Based Augmentation of Eye-Tracking Data." Journal of Imaging 7, no. 5 (May 3, 2021): 83. http://dx.doi.org/10.3390/jimaging7050083.
Full textAbstract:
Over the past decade, deep learning has achieved unprecedented successes in a diversity of application domains, given large-scale datasets. However, particular domains, such as healthcare, inherently suffer from data paucity and imbalance. Moreover, datasets could be largely inaccessible due to privacy concerns, or lack of data-sharing incentives. Such challenges have attached significance to the application of generative modeling and data augmentation in that domain. In this context, this study explores a machine learning-based approach for generating synthetic eye-tracking data. We explore a novel application of variational autoencoders (VAEs) in this regard. More specifically, a VAE model is trained to generate an image-based representation of the eye-tracking output, so-called scanpaths. Overall, our results validate that the VAE model could generate a plausible output from a limited dataset. Finally, it is empirically demonstrated that such approach could be employed as a mechanism for data augmentation to improve the performance in classification tasks.
10
Cao, Shichen, Jingjing Li, Kenric P. Nelson, and Mark A. Kon. "Coupled VAE: Improved Accuracy and Robustness of a Variational Autoencoder." Entropy 24, no. 3 (March 18, 2022): 423. http://dx.doi.org/10.3390/e24030423.
Full textAbstract:
We present a coupled variational autoencoder (VAE) method, which improves the accuracy and robustness of the model representation of handwritten numeral images. The improvement is measured in both increasing the likelihood of the reconstructed images and in reducing divergence between the posterior and a prior latent distribution. The new method weighs outlier samples with a higher penalty by generalizing the original evidence lower bound function using a coupled entropy function based on the principles of nonlinear statistical coupling. We evaluated the performance of the coupled VAE model using the Modified National Institute of Standards and Technology (MNIST) dataset and its corrupted modification C-MNIST. Histograms of the likelihood that the reconstruction matches the original image show that the coupled VAE improves the reconstruction and this improvement is more substantial when seeded with corrupted images. All five corruptions evaluated showed improvement. For instance, with the Gaussian corruption seed the accuracy improves by 1014 (from 10−57.2 to 10−42.9) and robustness improves by 1022 (from 10−109.2 to 10−87.0). Furthermore, the divergence between the posterior and prior distribution of the latent distribution is reduced. Thus, in contrast to the β-VAE design, the coupled VAE algorithm improves model representation, rather than trading off the performance of the reconstruction and latent distribution divergence.
11
Choong, Jun Jin, Xin Liu, and Tsuyoshi Murata. "Optimizing Variational Graph Autoencoder for Community Detection with Dual Optimization." Entropy 22, no. 2 (February 7, 2020): 197. http://dx.doi.org/10.3390/e22020197.
Full textAbstract:
Variational Graph Autoencoder (VGAE) has recently gained traction for learning representations on graphs. Its inception has allowed models to achieve state-of-the-art performance for challenging tasks such as link prediction, rating prediction, and node clustering. However, a fundamental flaw exists in Variational Autoencoder (VAE)-based approaches. Specifically, merely minimizing the loss of VAE increases the deviation from its primary objective. Focusing on Variational Graph Autoencoder for Community Detection (VGAECD) we found that optimizing the loss using the stochastic gradient descent often leads to sub-optimal community structure especially when initialized poorly. We address this shortcoming by introducing a dual optimization procedure. This procedure aims to guide the optimization process and encourage learning of the primary objective. Additionally, we linearize the encoder to reduce the number of learning parameters. The outcome is a robust algorithm that outperforms its predecessor.
12
Sun, Ying, Lang Li, Yang Ding, Jiabao Bai, and Xiangning Xin. "Image Compression Algorithm Based On Variational Autoencoder." Journal of Physics: Conference Series 2066, no. 1 (November 1, 2021): 012008. http://dx.doi.org/10.1088/1742-6596/2066/1/012008.
Full textAbstract:
Abstract Variational Autoencoder (VAE), as a kind of deep hidden space generation model, has achieved great success in performance in recent years, especially in image generation. This paper aims to study image compression algorithms based on variational autoencoders. This experiment uses the image quality evaluation measurement model, because the image super-resolution algorithm based on interpolation is the most direct and simple method to change the image resolution. In the experiment, the first step of the whole picture is transformed by the variational autoencoder, and then the actual coding is applied to the complete coefficient. Experimental data shows that after encoding using the improved encoding method of the variational autoencoder, the number of bits required for the encoding symbol stream required for transmission or storage in the traditional encoding method is greatly reduced, and symbol redundancy is effectively avoided. The experimental results show that the image research algorithm using variational autoencoder for image 1, image 2, and image 3 reduces the time by 3332, 2637, and 1470 bit respectively compared with the traditional image research algorithm of self-encoding. In the future, people will introduce deep convolutional neural networks to optimize the generative adversarial network, so that the generative adversarial network can obtain better convergence speed and model stability.
13
Yang, FengLei, Fei Liu, and ShanShan Liu. "Collaborative Filtering Based on a Variational Gaussian Mixture Model." Future Internet 13, no. 2 (February 1, 2021): 37. http://dx.doi.org/10.3390/fi13020037.
Full textAbstract:
Collaborative filtering (CF) is a widely used method in recommendation systems. Linear models are still the mainstream of collaborative filtering research methods, but non-linear probabilistic models are beyond the limit of linear model capacity. For example, variational autoencoders (VAEs) have been extensively used in CF, and have achieved excellent results. Aiming at the problem of the prior distribution for the latent codes of VAEs in traditional CF is too simple, which makes the implicit variable representations of users and items too poor. This paper proposes a variational autoencoder that uses a Gaussian mixture model for latent factors distribution for CF, GVAE-CF. On this basis, an optimization function suitable for GVAE-CF is proposed. In our experimental evaluation, we show that the recommendation performance of GVAE-CF outperforms the previously proposed VAE-based models on several popular benchmark datasets in terms of recall and normalized discounted cumulative gain (NDCG), thus proving the effectiveness of the algorithm.
14
Wu, Hanwei, and Markus Flierl. "Vector Quantization-Based Regularization for Autoencoders." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 6380–87. http://dx.doi.org/10.1609/aaai.v34i04.6108.
Full textAbstract:
Autoencoders and their variations provide unsupervised models for learning low-dimensional representations for downstream tasks. Without proper regularization, autoencoder models are susceptible to the overfitting problem and the so-called posterior collapse phenomenon. In this paper, we introduce a quantization-based regularizer in the bottleneck stage of autoencoder models to learn meaningful latent representations. We combine both perspectives of Vector Quantized-Variational AutoEncoders (VQ-VAE) and classical denoising regularization methods of neural networks. We interpret quantizers as regularizers that constrain latent representations while fostering a similarity-preserving mapping at the encoder. Before quantization, we impose noise on the latent codes and use a Bayesian estimator to optimize the quantizer-based representation. The introduced bottleneck Bayesian estimator outputs the posterior mean of the centroids to the decoder, and thus, is performing soft quantization of the noisy latent codes. We show that our proposed regularization method results in improved latent representations for both supervised learning and clustering downstream tasks when compared to autoencoders using other bottleneck structures.
15
Ok, Changwon, Geonseok Lee, and Kichun Lee. "Informative Language Encoding by Variational Autoencoders Using Transformer." Applied Sciences 12, no. 16 (August 9, 2022): 7968. http://dx.doi.org/10.3390/app12167968.
Full textAbstract:
In natural language processing (NLP), Transformer is widely used and has reached the state-of-the-art level in numerous NLP tasks such as language modeling, summarization, and classification. Moreover, a variational autoencoder (VAE) is an efficient generative model in representation learning, combining deep learning with statistical inference in encoded representations. However, the use of VAE in natural language processing often brings forth practical difficulties such as a posterior collapse, also known as Kullback–Leibler (KL) vanishing. To mitigate this problem, while taking advantage of the parallelization of language data processing, we propose a new language representation model as the integration of two seemingly different deep learning models, which is a Transformer model solely coupled with a variational autoencoder. We compare the proposed model with previous works, such as a VAE connected with a recurrent neural network (RNN). Our experiments with four real-life datasets show that implementation with KL annealing mitigates posterior collapses. The results also show that the proposed Transformer model outperforms RNN-based models in reconstruction and representation learning, and that the encoded representations of the proposed model are more informative than other tested models.
16
Sarkar, Arindam, Nikhil Mehta, and Piyush Rai. "Graph Representation Learning via Ladder Gamma Variational Autoencoders." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 5604–11. http://dx.doi.org/10.1609/aaai.v34i04.6013.
Full textAbstract:
We present a probabilistic framework for community discovery and link prediction for graph-structured data, based on a novel, gamma ladder variational autoencoder (VAE) architecture. We model each node in the graph via a deep hierarchy of gamma-distributed embeddings, and define each link probability via a nonlinear function of the bottom-most layer's embeddings of its associated nodes. In addition to leveraging the representational power of multiple layers of stochastic variables via the ladder VAE architecture, our framework offers the following benefits: (1) Unlike existing ladder VAE architectures based on real-valued latent variables, the gamma-distributed latent variables naturally result in non-negativity and sparsity of the learned embeddings, and facilitate their direct interpretation as membership of nodes into (possibly multiple) communities/topics; (2) A novel recognition model for our gamma ladder VAE architecture allows fast inference of node embeddings; and (3) The framework also extends naturally to incorporate node side information (features and/or labels). Our framework is also fairly modular and can leverage a wide variety of graph neural networks as the VAE encoder. We report both quantitative and qualitative results on several benchmark datasets and compare our model with several state-of-the-art methods.
17
A Samah, Azurah, Siti Nurul Aqilah Ahmad, Hairudin Abdul Majid, Zuraini Ali Shah, Haslina Hashim, Nuraina Syaza Azman, Nur Sabrina Azmi, and Dewi Nasien. "Classification of Attention Deficit Hyperactivity Disorder using Variational Autoencoder." International Journal of Innovative Computing 11, no. 2 (October 31, 2021): 81–87. http://dx.doi.org/10.11113/ijic.v11n2.352.
Full textAbstract:
Attention Deficit Hyperactivity Disorder (ADHD) categorize as one of the typical neurodevelopmental and mental disorders. Over the years, researchers have identified ADHD as a complicated disorder since it is not directly tested with a standard medical test such as a blood or urine test on the early-stage diagnosis. Apart from the physical symptoms of ADHD, clinical data of ADHD patients show that most of them have learning problems. Therefore, functional Magnetic Resonance Imaging (fMRI) is considered the most suitable method to determine functional activity in the brain region to understand brain disorders of ADHD. One of the ways to diagnose ADHD is by using deep learning techniques, which can increase the accuracy of predicting ADHD using the fMRI dataset. Past attempts of classifying ADHD based on functional connectivity coefficient using the Deep Neural Network (DNN) result in 95% accuracy. As Variational Autoencoder (VAE) is the most popular in extracting high-level data, this model is applied in this study. This study aims to enhance the performance of VAE to increase the accuracy in classifying ADHD using fMRI data based on functional connectivity analysis. The preprocessed fMRI dataset is used for decomposition to find the region of interest (ROI), followed by Independent Component Analysis (ICA) that calculates the correlation between brain regions and creates functional connectivity matrices for each subject. As a result, the VAE model achieved an accuracy of 75% on classifying ADHD.
18
Mujkic, Esma, Mark P. Philipsen, Thomas B. Moeslund, Martin P. Christiansen, and Ole Ravn. "Anomaly Detection for Agricultural Vehicles Using Autoencoders." Sensors 22, no. 10 (May 10, 2022): 3608. http://dx.doi.org/10.3390/s22103608.
Full textAbstract:
The safe in-field operation of autonomous agricultural vehicles requires detecting all objects that pose a risk of collision. Current vision-based algorithms for object detection and classification are unable to detect unknown classes of objects. In this paper, the problem is posed as anomaly detection instead, where convolutional autoencoders are applied to identify any objects deviating from the normal pattern. Training an autoencoder network to reconstruct normal patterns in agricultural fields makes it possible to detect unknown objects by high reconstruction error. Basic autoencoder (AE), vector-quantized variational autoencoder (VQ-VAE), denoising autoencoder (DAE) and semisupervised autoencoder (SSAE) with a max-margin-inspired loss function are investigated and compared with a baseline object detector based on YOLOv5. Results indicate that SSAE with an area under the curve for precision/recall (PR AUC) of 0.9353 outperforms other autoencoder models and is comparable to an object detector with a PR AUC of 0.9794. Qualitative results show that SSAE is capable of detecting unknown objects, whereas the object detector is unable to do so and fails to identify known classes of objects in specific cases.
19
Luchnikov, Ilia A., Alexander Ryzhov, Pieter-Jan Stas, Sergey N. Filippov, and Henni Ouerdane. "Variational Autoencoder Reconstruction of Complex Many-Body Physics." Entropy 21, no. 11 (November 7, 2019): 1091. http://dx.doi.org/10.3390/e21111091.
Full textAbstract:
Thermodynamics is a theory of principles that permits a basic description of the macroscopic properties of a rich variety of complex systems from traditional ones, such as crystalline solids, gases, liquids, and thermal machines, to more intricate systems such as living organisms and black holes to name a few. Physical quantities of interest, or equilibrium state variables, are linked together in equations of state to give information on the studied system, including phase transitions, as energy in the forms of work and heat, and/or matter are exchanged with its environment, thus generating entropy. A more accurate description requires different frameworks, namely, statistical mechanics and quantum physics to explore in depth the microscopic properties of physical systems and relate them to their macroscopic properties. These frameworks also allow to go beyond equilibrium situations. Given the notably increasing complexity of mathematical models to study realistic systems, and their coupling to their environment that constrains their dynamics, both analytical approaches and numerical methods that build on these models show limitations in scope or applicability. On the other hand, machine learning, i.e., data-driven, methods prove to be increasingly efficient for the study of complex quantum systems. Deep neural networks, in particular, have been successfully applied to many-body quantum dynamics simulations and to quantum matter phase characterization. In the present work, we show how to use a variational autoencoder (VAE)—a state-of-the-art tool in the field of deep learning for the simulation of probability distributions of complex systems. More precisely, we transform a quantum mechanical problem of many-body state reconstruction into a statistical problem, suitable for VAE, by using informationally complete positive operator-valued measure. We show, with the paradigmatic quantum Ising model in a transverse magnetic field, that the ground-state physics, such as, e.g., magnetization and other mean values of observables, of a whole class of quantum many-body systems can be reconstructed by using VAE learning of tomographic data for different parameters of the Hamiltonian, and even if the system undergoes a quantum phase transition. We also discuss challenges related to our approach as entropy calculations pose particular difficulties.
20
Pham, Tuan-Anh, Jong-Hoon Lee, and Choong-Shik Park. "MST-VAE: Multi-Scale Temporal Variational Autoencoder for Anomaly Detection in Multivariate Time Series." Applied Sciences 12, no. 19 (October 7, 2022): 10078. http://dx.doi.org/10.3390/app121910078.
Full textAbstract:
In IT monitoring systems, anomaly detection plays a vital role in detecting and alerting unexpected behaviors timely to system operators. With the growth of signal data in both volumes and dimensions during operation, unsupervised learning turns out to be a great solution to trigger anomalies thanks to the feasibility of working well with unlabeled data. In recent years, autoencoder, an unsupervised learning technique, has gained much attention because of its robustness. Autoencoder first compresses input data to lower-dimensional latent representation, which obtains normal patterns, then the compressed data are reconstructed back to the input form to detect abnormal data. In this paper, we propose a practical unsupervised learning approach using Multi-Scale Temporal convolutional kernels with Variational AutoEncoder (MST-VAE) for anomaly detection in multivariate time series data. Our key observation is that combining short-scale and long-scale convolutional kernels to extract various temporal information of the time series can enhance the model performance. Extensive empirical studies on five real-world datasets demonstrate that MST-VAE can outperform baseline methods in effectiveness and efficiency.
21
Kopf, Andreas, Vincent Fortuin, Vignesh Ram Somnath, and Manfred Claassen. "Mixture-of-Experts Variational Autoencoder for clustering and generating from similarity-based representations on single cell data." PLOS Computational Biology 17, no. 6 (June 30, 2021): e1009086. http://dx.doi.org/10.1371/journal.pcbi.1009086.
Full textAbstract:
Clustering high-dimensional data, such as images or biological measurements, is a long-standing problem and has been studied extensively. Recently, Deep Clustering has gained popularity due to its flexibility in fitting the specific peculiarities of complex data. Here we introduce the Mixture-of-Experts Similarity Variational Autoencoder (MoE-Sim-VAE), a novel generative clustering model. The model can learn multi-modal distributions of high-dimensional data and use these to generate realistic data with high efficacy and efficiency. MoE-Sim-VAE is based on a Variational Autoencoder (VAE), where the decoder consists of a Mixture-of-Experts (MoE) architecture. This specific architecture allows for various modes of the data to be automatically learned by means of the experts. Additionally, we encourage the lower dimensional latent representation of our model to follow a Gaussian mixture distribution and to accurately represent the similarities between the data points. We assess the performance of our model on the MNIST benchmark data set and challenging real-world tasks of clustering mouse organs from single-cell RNA-sequencing measurements and defining cell subpopulations from mass cytometry (CyTOF) measurements on hundreds of different datasets. MoE-Sim-VAE exhibits superior clustering performance on all these tasks in comparison to the baselines as well as competitor methods.
22
Posch, Stefan, Clemens Gößnitzer, Andreas B. Ofner, Gerhard Pirker, and Andreas Wimmer. "Modeling Cycle-to-Cycle Variations of a Spark-Ignited Gas Engine Using Artificial Flow Fields Generated by a Variational Autoencoder." Energies 15, no. 7 (March 23, 2022): 2325. http://dx.doi.org/10.3390/en15072325.
Full textAbstract:
A deeper understanding of the physical nature of cycle-to-cycle variations (CCV) in internal combustion engines (ICE) as well as reliable simulation strategies to predict these CCV are indispensable for the development of modern highly efficient combustion engines. Since the combustion process in ICE strongly depends on the turbulent flow field in the cylinder and, for spark-ignited engines, especially around the spark plug, the prediction of CCV using computational fluid dynamics (CFD) is limited to the modeling of turbulent flows. One possible way to determine CCV is by applying large eddy simulation (LES), whose potential in this field has already been shown despite its drawback of requiring considerable computational time and resources. This paper presents a novel strategy based on unsteady Reynolds-averaged Navier–Stokes (uRANS) CFD in combination with variational autoencoders (VAEs). A VAE is trained with flow field data from presimulated cycles at a specific crank angle. Then, the VAE can be used to generate artificial flow fields that serve to initialize new CFD simulations of the combustion process. With this novel approach, a high number of individual cycles can be simulated in a fraction of the time that LES needs for the same amount of cycles. Since the VAE is trained on data from presimulated cycles, the physical information of the cycles is transferred to the generated artificial cycles.
23
Ma, Peirong, and Xiao Hu. "A Variational Autoencoder with Deep Embedding Model for Generalized Zero-Shot Learning." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 07 (April 3, 2020): 11733–40. http://dx.doi.org/10.1609/aaai.v34i07.6844.
Full textAbstract:
Generalized zero-shot learning (GZSL) is a challenging task that aims to recognize not only unseen classes unavailable during training, but also seen classes used at training stage. It is achieved by transferring knowledge from seen classes to unseen classes via a shared semantic space (e.g. attribute space). Most existing GZSL methods usually learn a cross-modal mapping between the visual feature space and the semantic space. However, the mapping model learned only from the seen classes will produce an inherent bias when used in the unseen classes. In order to tackle such a problem, this paper integrates a deep embedding network (DE) and a modified variational autoencoder (VAE) into a novel model (DE-VAE) to learn a latent space shared by both image features and class embeddings. Specifically, the proposed model firstly employs DE to learn the mapping from the semantic space to the visual feature space, and then utilizes VAE to transform both original visual features and the features obtained by the mapping into latent features. Finally, the latent features are used to train a softmax classifier. Extensive experiments on four GZSL benchmark datasets show that the proposed model significantly outperforms the state of the arts.
24
Situ, Haozhen, and Zhimin He. "Machine learning distributions of quantum ansatz with hierarchical structure." International Journal of Modern Physics B 34, no. 20 (July 15, 2020): 2050196. http://dx.doi.org/10.1142/s0217979220501969.
Full textAbstract:
Machine learning techniques can help to represent and solve quantum systems. Learning measurement outcome distribution of quantum ansatz is useful for characterization of near-term quantum computing devices. In this work, we use the popular unsupervised machine learning model, variational autoencoder (VAE), to reconstruct the measurement outcome distribution of quantum ansatz. The number of parameters in the VAE are compared with the number of measurement outcomes. The numerical results show that VAE can efficiently learn the measurement outcome distribution with few parameters. The influence of entanglement on the task is also revealed.
25
Samanta, Soumitra, Steve O’Hagan, Neil Swainston, Timothy J. Roberts, and Douglas B. Kell. "VAE-Sim: A Novel Molecular Similarity Measure Based on a Variational Autoencoder." Molecules 25, no. 15 (July 29, 2020): 3446. http://dx.doi.org/10.3390/molecules25153446.
Full textAbstract:
Molecular similarity is an elusive but core “unsupervised” cheminformatics concept, yet different “fingerprint” encodings of molecular structures return very different similarity values, even when using the same similarity metric. Each encoding may be of value when applied to other problems with objective or target functions, implying that a priori none are “better” than the others, nor than encoding-free metrics such as maximum common substructure (MCSS). We here introduce a novel approach to molecular similarity, in the form of a variational autoencoder (VAE). This learns the joint distribution p(z|x) where z is a latent vector and x are the (same) input/output data. It takes the form of a “bowtie”-shaped artificial neural network. In the middle is a “bottleneck layer” or latent vector in which inputs are transformed into, and represented as, a vector of numbers (encoding), with a reverse process (decoding) seeking to return the SMILES string that was the input. We train a VAE on over six million druglike molecules and natural products (including over one million in the final holdout set). The VAE vector distances provide a rapid and novel metric for molecular similarity that is both easily and rapidly calculated. We describe the method and its application to a typical similarity problem in cheminformatics.
26
Chen, Fang, Tao Zhang, and Ruilin Liu. "An Active Learning Method Based on Variational Autoencoder and DBSCAN Clustering." Computational Intelligence and Neuroscience 2021 (July 30, 2021): 1–11. http://dx.doi.org/10.1155/2021/9952596.
Full textAbstract:
Active learning is aimed to sample the most informative data from the unlabeled pool, and diverse clustering methods have been applied to it. However, the distance-based clustering methods usually cannot perform well in high dimensions and even begin to fail. In this paper, we propose a new active learning method combined with variational autoencoder (VAE) and density-based spatial clustering of applications with noise (DBSCAN). It overcomes the difficulty of distance representation in high dimensions and prevents the distance concentration phenomenon from occurring in the computational learning literature with respect to high-dimensional p-norms. Finally, we compare our method with four common active learning methods and two other clustering algorithms combined with VAE on three datasets. The results demonstrate that our approach achieves competitive performance, and it is a new batch mode active learning algorithm designed for neural networks with a relatively small query batch size.
27
Valarezo, Añazco, Lopez Rivera, Hyemin Park, Nahyeon Park, and Tae-Seong Kim. "Human activities recognition with a single writs IMU via a Variational Autoencoder and android deep recurrent neural nets." Computer Science and Information Systems 17, no. 2 (2020): 581–97. http://dx.doi.org/10.2298/csis190920005v.
Full textAbstract:
Human Activity Recognition (HAR) is an active research field because of its versatility towards various application areas such as healthcare and lifecare. In this study, a novel HAR system is proposed based on an autoencoder for denoising and Recurrent Neural Network (RNN) for classification with a single Inertial Measurement Unit (IMU) located on a dominant wrist. A Variational Autoencoder (VAE) is built to denoise IMU signals which improves HAR by Android Deep RNN. Evaluating our VAE and Android Deep RNN HAR system is done in two ways. First, the system is tested on a PC using discrete epochs of activities of daily living. Our results show that VAE improves Signal-to-Noise Ratio (SNR) of the IMU signal from 8.78 to 17.26 dB. In turn, HAR improves from 89.29% to 95.11% in F1-score and from 90.38% to 95.47% in accuracy. Secondly, the system is tested on an Android device (i.e., smartphone) using continuous activity signals. This is done by transferring the PC HAR system to an Android HAR App (i.e., Android Deep RNN). We have achieved 86.13% and 95.09% in accuracy without and with VAE respectively. Our results demonstrate that HAR can be achieved in real-time on a standalone smart device with a single IMU for lifelogging services.
28
Dairi, Abdelkader, Fouzi Harrou, Ying Sun, and Sofiane Khadraoui. "Short-Term Forecasting of Photovoltaic Solar Power Production Using Variational Auto-Encoder Driven Deep Learning Approach." Applied Sciences 10, no. 23 (November 25, 2020): 8400. http://dx.doi.org/10.3390/app10238400.
Full textAbstract:
The accurate modeling and forecasting of the power output of photovoltaic (PV) systems are critical to efficiently managing their integration in smart grids, delivery, and storage. This paper intends to provide efficient short-term forecasting of solar power production using Variational AutoEncoder (VAE) model. Adopting the VAE-driven deep learning model is expected to improve forecasting accuracy because of its suitable performance in time-series modeling and flexible nonlinear approximation. Both single- and multi-step-ahead forecasts are investigated in this work. Data from two grid-connected plants (a 243 kW parking lot canopy array in the US and a 9 MW PV system in Algeria) are employed to show the investigated deep learning models’ performance. Specifically, the forecasting outputs of the proposed VAE-based forecasting method have been compared with seven deep learning methods, namely recurrent neural network, Long short-term memory (LSTM), Bidirectional LSTM, Convolutional LSTM network, Gated recurrent units, stacked autoencoder, and restricted Boltzmann machine, and two commonly used machine learning methods, namely logistic regression and support vector regression. The results of this investigation demonstrate the satisfying performance of deep learning techniques to forecast solar power and point out that the VAE consistently performed better than the other methods. Also, results confirmed the superior performance of deep learning models compared to the two considered baseline machine learning models.
29
Mahmud, Mohammad Sultan, Joshua Zhexue Huang, and Xianghua Fu. "Variational Autoencoder-Based Dimensionality Reduction for High-Dimensional Small-Sample Data Classification." International Journal of Computational Intelligence and Applications 19, no. 01 (March 2020): 2050002. http://dx.doi.org/10.1142/s1469026820500029.
Full textAbstract:
Classification problems in which the number of features (dimensions) is unduly higher than the number of samples (observations) is an essential research and application area in a variety of domains, especially in computational biology. It is also known as a high-dimensional small-sample-size (HDSSS) problem. Various dimensionality reduction methods have been developed, but they are not potent with the small-sample-sized high-dimensional datasets and suffer from overfitting and high-variance gradients. To overcome the pitfalls of sample size and dimensionality, this study employed variational autoencoder (VAE), which is a dynamic framework for unsupervised learning in recent years. The objective of this study is to investigate a reliable classification model for high-dimensional and small-sample-sized datasets with minimal error. Moreover, it evaluated the strength of different architectures of VAE on the HDSSS datasets. In the experiment, six genomic microarray datasets from Kent Ridge Biomedical Dataset Repository were selected, and several choices of dimensions (features) were applied for data preprocessing. Also, to evaluate the classification accuracy and to find a stable and suitable classifier, nine state-of-the-art classifiers that have been successful for classification tasks in high-dimensional data settings were selected. The experimental results demonstrate that the VAE can provide superior performance compared to traditional methods such as PCA, fastICA, FA, NMF, and LDA in terms of accuracy and AUROC.
30
Iatrou, Miltiadis, Christos Karydas, Xanthi Tseni, and Spiros Mourelatos. "Representation Learning with a Variational Autoencoder for Predicting Nitrogen Requirement in Rice." Remote Sensing 14, no. 23 (November 25, 2022): 5978. http://dx.doi.org/10.3390/rs14235978.
Full textAbstract:
The scope of this research was to provide rice growers with optimal N-rate recommendations through precision agriculture applications. To achieve this goal, a prediction rice yield model was constructed, based on soil data, remote sensing data (optical and radar), climatic data, and farming practices. The dataset was collected from a rice crop surface of 89.2 ha cultivated continuously for a 5-year period and was analyzed with machine learning (ML) systems. A variational autoencoder (VAE) for reconstructing the input data of the prediction model was applied, resulting in MAE of 0.6 tn/ha, with an average yield for the study fields and period measured at 9.6 tn/ha. VAE learns the original input data representation and transforms them in a latent feature space, so that the anomalies and the discrepancies of the data are reduced. The reconstructed data by VAE provided a more sophisticated and detailed ML model, improving our knowledge about the various correlations between soil, N management parameters, and yield. Both optical and radar imagery and the climatic data were found to be of high importance for the model, as indicated by the application of XAI (explainable artificial intelligence) techniques. The new model was applied in the 2022 rice cultivation in the study fields, resulting in an average yield increase of 4.32% compared to the 5 previous years of experimentation.
31
Liang, Sen, Zhi-ze Zhou, Yu-dong Guo, Xuan Gao, Ju-yong Zhang, and Hu-jun Bao. "Facial landmark disentangled network with variational autoencoder." Applied Mathematics-A Journal of Chinese Universities 37, no. 2 (June 2022): 290–305. http://dx.doi.org/10.1007/s11766-022-4589-0.
Full textAbstract:
AbstractLearning disentangled representation of data is a key problem in deep learning. Specifically, disentangling 2D facial landmarks into different factors (e.g., identity and expression) is widely used in the applications of face reconstruction, face reenactment and talking head et al.. However, due to the sparsity of landmarks and the lack of accurate labels for the factors, it is hard to learn the disentangled representation of landmarks. To address these problem, we propose a simple and effective model named FLD-VAE to disentangle arbitrary facial landmarks into identity and expression latent representations, which is based on a Variational Autoencoder framework. Besides, we propose three invariant loss functions in both latent and data levels to constrain the invariance of representations during training stage. Moreover, we implement an identity preservation loss to further enhance the representation ability of identity factor. To the best of our knowledge, this is the first work to end-to-end disentangle identity and expression factors simultaneously from one single facial landmark.
32
Min, Yongzhi, and Yaxing Li. "Self-Supervised Railway Surface Defect Detection with Defect Removal Variational Autoencoders." Energies 15, no. 10 (May 13, 2022): 3592. http://dx.doi.org/10.3390/en15103592.
Full textAbstract:
In railway surface defect detection applications, supervised deep learning methods suffer from the problems of insufficient defect samples and an imbalance between positive and negative samples. To overcome these problems, we propose a lightweight two-stage architecture including the railway cropping network (RC-Net) and defects removal variational autoencoder (DR-VAE), which requires only normal samples for training to achieve defect detection. First, we design a simple and effective RC-Net to extract railway surfaces accurately from railway inspection images. Second, the DR-VAE is proposed for background reconstruction of railway surface images to detect defects by self-supervised learning. Specifically, during the training process, DR-VAE contains a defect random mask module (D-RM) to generate self-supervised signals and uses a structural similarity index measure (SSIM) as pixel loss. In addition, the decoder of DR-VAE also acts as a discriminator to implement introspective adversarial training. In the inference stage, we reduce the random error of reconstruction by introducing a distribution capacity attenuation factor, and finally use the residuals of the original and reconstructed images to achieve segmentation of the defects. The experiments, including core parameter exploration and comparison with other models, indicate that the model can achieve a high detection accuracy.
33
Yan, Xiaoan, Yadong Xu, Daoming She, and Wan Zhang. "Reliable Fault Diagnosis of Bearings Using an Optimized Stacked Variational Denoising Auto-Encoder." Entropy 24, no. 1 (December 24, 2021): 36. http://dx.doi.org/10.3390/e24010036.
Full textAbstract:
Variational auto-encoders (VAE) have recently been successfully applied in the intelligent fault diagnosis of rolling bearings due to its self-learning ability and robustness. However, the hyper-parameters of VAEs depend, to a significant extent, on artificial settings, which is regarded as a common and key problem in existing deep learning models. Additionally, its anti-noise capability may face a decline when VAE is used to analyze bearing vibration data under loud environmental noise. Therefore, in order to improve the anti-noise performance of the VAE model and adaptively select its parameters, this paper proposes an optimized stacked variational denoising autoencoder (OSVDAE) for the reliable fault diagnosis of bearings. Within the proposed method, a robust network, named variational denoising auto-encoder (VDAE), is, first, designed by integrating VAE and a denoising auto-encoder (DAE). Subsequently, a stacked variational denoising auto-encoder (SVDAE) architecture is constructed to extract the robust and discriminative latent fault features via stacking VDAE networks layer on layer, wherein the important parameters of the SVDAE model are automatically determined by employing a novel meta-heuristic intelligent optimizer known as the seagull optimization algorithm (SOA). Finally, the extracted latent features are imported into a softmax classifier to obtain the results of fault recognition in rolling bearings. Experiments are conducted to validate the effectiveness of the proposed method. The results of analysis indicate that the proposed method not only can achieve a high identification accuracy for different bearing health conditions, but also outperforms some representative deep learning methods.
34
Kameoka, Hirokazu, Li Li, Shota Inoue, and Shoji Makino. "Supervised Determined Source Separation with Multichannel Variational Autoencoder." Neural Computation 31, no. 9 (September 2019): 1891–914. http://dx.doi.org/10.1162/neco_a_01217.
Full textAbstract:
This letter proposes a multichannel source separation technique, the multichannel variational autoencoder (MVAE) method, which uses a conditional VAE (CVAE) to model and estimate the power spectrograms of the sources in a mixture. By training the CVAE using the spectrograms of training examples with source-class labels, we can use the trained decoder distribution as a universal generative model capable of generating spectrograms conditioned on a specified class index. By treating the latent space variables and the class index as the unknown parameters of this generative model, we can develop a convergence-guaranteed algorithm for supervised determined source separation that consists of iteratively estimating the power spectrograms of the underlying sources, as well as the separation matrices. In experimental evaluations, our MVAE produced better separation performance than a baseline method.
35
Kostiuk, Yevhen, Mykola Lukashchuk, Alexander Gelbukh, and Grigori Sidorov. "Prior latent distribution comparison for the RNN Variational Autoencoder in low-resource language modeling." Journal of Intelligent & Fuzzy Systems 42, no. 5 (March 31, 2022): 4541–49. http://dx.doi.org/10.3233/jifs-219243.
Full textAbstract:
Probabilistic Bayesian methods are widely used in the machine learning domain. Variational Autoencoder (VAE) is a common architecture for solving the Language Modeling task in a self-supervised way. VAE consists of a concept of latent variables inside the model. Latent variables are described as a random variable that is fit by the data. Up to now, in the majority of cases, latent variables are considered normally distributed. The normal distribution is a well-known distribution that can be easily included in any pipeline. Moreover, the normal distribution is a good choice when the Central Limit Theorem (CLT) holds. It makes it effective when one is working with i.i.d. (independent and identically distributed) random variables. However, the conditions of CLT in Natural Language Processing are not easy to check. So, the choice of distribution family is unclear in the domain. This paper studies the priors selection impact of continuous distributions in the Low-Resource Language Modeling task with VAE. The experiment shows that there is a statistical difference between the different priors in the encoder-decoder architecture. We showed that family distribution hyperparameter is important in the Low-Resource Language Modeling task and should be considered for the model training.
36
Lee, Changro. "Data augmentation using a variational autoencoder for estimating property prices." Property Management 39, no. 3 (February 5, 2021): 408–18. http://dx.doi.org/10.1108/pm-09-2020-0057.
Full textAbstract:
PurposePrior studies on the application of deep-learning techniques have focused on enhancing computation algorithms. However, the amount of data is also a key element when attempting to achieve a goal using a quantitative approach, which is often underestimated in practice. The problem of sparse sales data is well known in the valuation of commercial properties. This study aims to expand the limited data available to exploit the capability inherent in deep learning techniques.Design/methodology/approachThe deep learning approach is used. Seoul, the capital of South Korea is selected as a case study area. Second, data augmentation is performed for properties with low trade volume in the market using a variational autoencoder (VAE), which is a generative deep learning technique. Third, the generated samples are added into the original dataset of commercial properties to alleviate data insufficiency. Finally, the accuracy of the price estimation is analyzed for the original and augmented datasets to assess the model performance.FindingsThe results using the sales datasets of commercial properties in Seoul, South Korea as a case study show that the augmented dataset by a VAE consistently shows higher accuracy of price estimation for all 30 trials, and the capabilities inherent in deep learning techniques can be fully exploited, promoting the rapid adoption of artificial intelligence skills in the real estate industry.Originality/valueAlthough deep learning-based algorithms are gaining popularity, they are likely to show limited performance when data are insufficient. This study suggests an alternative approach to overcome the lack of data problem in property valuation.
37
Yao, Dingding, Jiale Zhao, Longbiao Cheng, Junfeng Li, Xiaodong Li, Xiaochao Guo, and Yonghong Yan. "An individualization approach for head-related transfer function in arbitrary directions based on deep learning." JASA Express Letters 2, no. 6 (June 2022): 064401. http://dx.doi.org/10.1121/10.0011575.
Full textAbstract:
This paper provides an individualization approach for head-related transfer function (HRTF) in arbitrary directions based on deep learning by utilizing dual-autoencoder architecture to establish the relationship between HRTF magnitude spectrum and arbitrarily given direction and anthropometric parameters. In this architecture, one variational autoencoder (VAE) is utilized to extract interpretable and exploitable features of full-space HRTF spectra, while another autoencoder (AE) is employed for feature embedding of corresponding directions and anthropometric parameters. A deep neural networks model is finally trained to establish the relationship between these representative features. Experimental results show that the proposed method outperforms state-of-the-art methods in terms of spectral distortion.
38
Balne, Sridevi, and Chiranjeevi Manike. "An Exploration: Alzheimer’s Disease Classification Based on Spectral Matching of Shape Features." Ingénierie des systèmes d information 27, no. 5 (October 31, 2022): 791–97. http://dx.doi.org/10.18280/isi.270512.
Full textAbstract:
In this paper, we use structural deformations to classify dementia patients. Firstly, surface meshes are recovered from MRI segmented hippocampal, and node-to-node interactions between all the surface meshes are constructed using a spectral matching approach. Then, to learn the low-dimensional feature representation, an enhanced version of the variational auto-encoder (VAE) is given to the vertex coordinates of the surface meshes. We describe a new strategy for increasing variational autoencoder performance (VAE). We designed a generative adversarial training (GAN) technique to train the VAE to generate realistic medical images and apply the deep feature consistency principle, ensuring that the VAE output and its related input images have identical features. A discriminator with a SoftMax layer is concurrently trained to distinguish people with Alzheimer's from healthy people. Studies on the ADNI dataset show that the proposed method can distinguish normal people from early AD/NC and AD/EMCI classes with low computational time and higher accuracy that outperforms the support vector machine (SVM) baseline approach. All the simulation results are carried out with the Anaconda tool.
39
Thorne, Ben, Lloyd Knox, and Karthik Prabhu. "A generative model of galactic dust emission using variational autoencoders." Monthly Notices of the Royal Astronomical Society 504, no. 2 (April 12, 2021): 2603–13. http://dx.doi.org/10.1093/mnras/stab1011.
Full textAbstract:
ABSTRACT Emission from the interstellar medium can be a significant contaminant of measurements of the intensity and polarization of the cosmic microwave background (CMB). For planning CMB observations, and for optimizing foreground-cleaning algorithms, a description of the statistical properties of such emission can be helpful. Here, we examine a machine learning approach to inferring the statistical properties of dust from observational data. In particular, we apply a type of neural network called a variational autoencoder (VAE) to maps of the intensity of emission from interstellar dust as inferred from Planck sky maps and demonstrate its ability to (i) simulate new samples with similar summary statistics as the training set, (ii) provide fits to emission maps withheld from the training set, and (iii) produce constrained realizations. We find VAEs are easier to train than another popular architecture: that of generative adversarial networks, and are better suited for use in Bayesian inference.
40
Duan, Yitong, Lei Wang, Qizhong Zhang, and Jian Li. "FactorVAE: A Probabilistic Dynamic Factor Model Based on Variational Autoencoder for Predicting Cross-Sectional Stock Returns." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 4 (June 28, 2022): 4468–76. http://dx.doi.org/10.1609/aaai.v36i4.20369.
Full textAbstract:
As an asset pricing model in economics and finance, factor model has been widely used in quantitative investment. Towards building more effective factor models, recent years have witnessed the paradigm shift from linear models to more flexible nonlinear data-driven machine learning models. However, due to low signal-to-noise ratio of the financial data, it is quite challenging to learn effective factor models. In this paper, we propose a novel factor model, FactorVAE, as a probabilistic model with inherent randomness for noise modeling. Essentially, our model integrates the dynamic factor model (DFM) with the variational autoencoder (VAE) in machine learning, and we propose a prior-posterior learning method based on VAE, which can effectively guide the learning of model by approximating an optimal posterior factor model with future information. Particularly, considering that risk modeling is important for the noisy stock data, FactorVAE can estimate the variances from the distribution over the latent space of VAE, in addition to predicting returns. The experiments on the real stock market data demonstrate the effectiveness of FactorVAE, which outperforms various baseline methods.
41
Wei, Y., J. P. Levesque, C. J. Hansen, M. E. Mauel, and G. A. Navratil. "A dimensionality reduction algorithm for mapping tokamak operational regimes using a variational autoencoder (VAE) neural network." Nuclear Fusion 61, no. 12 (November 18, 2021): 126063. http://dx.doi.org/10.1088/1741-4326/ac3296.
Full textAbstract:
Abstract A variational autoencoder (VAE) is a type of unsupervised neural network which is able to learn meaningful data representations in a reduced dimensional space. We present an application of VAE in identifying the operational stability boundary of tokamak plasma discharges. This model was implemented using a dataset of over 3000 discharges from the high beta tokamak-extended pulse (HBT-EP) device. We found the VAE model to be capable of forming a continuous low-dimensional operational space map and identifying the operational boundaries using a specified warning time window. By projecting the operational parameters onto the same reduced space, this provides an intuitive way for the machine operator or an automated control system to perform disruption avoidance using a relevant control actuator as a discharge approaches a boundary. Pre-programmed GPU control experiments were conducted to demonstrate this control technique using HBT-EP’s saddle control coils as a horizontal position actuator, showing the ability to avoid the oncoming disruptive event and extend the duration of the discharge.
42
Fan, Junjun, Wai Keung Wong, Jiajun Wen, Can Gao, Dongmei Mo, and Zhihui Lai. "Fabric Defect Detection Using Deep Convolution Neural Network." AATCC Journal of Research 8, no. 1_suppl (September 2021): 143–50. http://dx.doi.org/10.14504/ajr.8.s1.18.
Full textAbstract:
Fabric defect detection plays an increasingly important role in the industrial automation application for fabric production, but how to detect defects rapidly and accurately is still challenging. In this study, we propose a powerful fabric defect detection method using a hybrid of convolutional neural network (CNN) and variational autoencoder (VAE). The convolutional layers are used for extracting fabric image pattern features and the variational autoencoder is used for modeling the latent characteristics and inferring a reconstruction. The defect positions can be detected by the differences between the original image and the reconstruction image. The proposed method is validated on public patterned fabric datasets. The experimental results demonstrate that the proposed model can achieve outstanding performance in both image level and pixel level defect detection.
43
Li, Bo, Zhengxing Sun, and Yuqi Guo. "SuperVAE: Superpixelwise Variational Autoencoder for Salient Object Detection." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 8569–76. http://dx.doi.org/10.1609/aaai.v33i01.33018569.
Full textAbstract:
Image saliency detection has recently witnessed rapid progress due to deep neural networks. However, there still exist many important problems in the existing deep learning based methods. Pixel-wise convolutional neural network (CNN) methods suffer from blurry boundaries due to the convolutional and pooling operations. While region-based deep learning methods lack spatial consistency since they deal with each region independently. In this paper, we propose a novel salient object detection framework using a superpixelwise variational autoencoder (SuperVAE) network. We first use VAE to model the image background and then separate salient objects from the background through the reconstruction residuals. To better capture semantic and spatial contexts information, we also propose a perceptual loss to take advantage from deep pre-trained CNNs to train our SuperVAE network. Without the supervision of mask-level annotated data, our method generates high quality saliency results which can better preserve object boundaries and maintain the spatial consistency. Extensive experiments on five wildly-used benchmark datasets show that the proposed method achieves superior or competitive performance compared to other algorithms including the very recent state-of-the-art supervised methods.
44
Sun, Ruoqi, Chen Huang, Hengliang Zhu, and Lizhuang Ma. "Mask-aware photorealistic facial attribute manipulation." Computational Visual Media 7, no. 3 (April 28, 2021): 363–74. http://dx.doi.org/10.1007/s41095-021-0219-7.
Full textAbstract:
AbstractThe technique of facial attribute manipulation has found increasing application, but it remains challenging to restrict editing of attributes so that a face’s unique details are preserved. In this paper, we introduce our method, which we call a mask-adversarial autoencoder (M-AAE). It combines a variational autoencoder (VAE) and a generative adversarial network (GAN) for photorealistic image generation. We use partial dilated layers to modify a few pixels in the feature maps of an encoder, changing the attribute strength continuously without hindering global information. Our training objectives for the VAE and GAN are reinforced by supervision of face recognition loss and cycle consistency loss, to faithfully preserve facial details. Moreover, we generate facial masks to enforce background consistency, which allows our training to focus on the foreground face rather than the background. Experimental results demonstrate that our method can generate high-quality images with varying attributes, and outperforms existing methods in detail preservation.
45
Ai, Dongmei, Yuduo Wang, Xiaoxin Li, and Hongfei Pan. "Colorectal Cancer Prediction Based on Weighted Gene Co-Expression Network Analysis and Variational Auto-Encoder." Biomolecules 10, no. 9 (August 20, 2020): 1207. http://dx.doi.org/10.3390/biom10091207.
Full textAbstract:
An effective feature extraction method is key to improving the accuracy of a prediction model. From the Gene Expression Omnibus (GEO) database, which includes 13,487 genes, we obtained microarray gene expression data for 238 samples from colorectal cancer (CRC) samples and normal samples. Twelve gene modules were obtained by weighted gene co-expression network analysis (WGCNA) on 173 samples. By calculating the Pearson correlation coefficient (PCC) between the characteristic genes of each module and colorectal cancer, we obtained a key module that was highly correlated with CRC. We screened hub genes from the key module by considering module membership, gene significance, and intramodular connectivity. We selected 10 hub genes as a type of feature for the classifier. We used the variational autoencoder (VAE) for 1159 genes with significantly different expressions and mapped the data into a 10-dimensional representation, as another type of feature for the cancer classifier. The two types of features were applied to the support vector machines (SVM) classifier for CRC. The accuracy was 0.9692 with an AUC of 0.9981. The result shows a high accuracy of the two-step feature extraction method, which includes obtaining hub genes by WGCNA and a 10-dimensional representation by variational autoencoder (VAE).
46
Lee, Je-Yeol, and Sang-Il Choi . "Improvement of Learning Stability of Generative Adversarial Network Using Variational Learning." Applied Sciences 10, no. 13 (June 30, 2020): 4528. http://dx.doi.org/10.3390/app10134528.
Full textAbstract:
In this paper, we propose a new network model using variational learning to improve the learning stability of generative adversarial networks (GAN). The proposed method can be easily applied to improve the learning stability of GAN-based models that were developed for various purposes, given that the variational autoencoder (VAE) is used as a secondary network while the basic GAN structure is maintained. When the gradient of the generator vanishes in the learning process of GAN, the proposed method receives gradient information from the decoder of the VAE that maintains gradient stably, so that the learning processes of the generator and discriminator are not halted. The experimental results of the MNIST and the CelebA datasets verify that the proposed method improves the learning stability of the networks by overcoming the vanishing gradient problem of the generator, and maintains the excellent data quality of the conventional GAN-based generative models.
47
Zhang, Li, Xing Chen, and Jun Yin. "Prediction of Potential miRNA–Disease Associations Through a Novel Unsupervised Deep Learning Framework with Variational Autoencoder." Cells 8, no. 9 (September 6, 2019): 1040. http://dx.doi.org/10.3390/cells8091040.
Full textAbstract:
The important role of microRNAs (miRNAs) in the formation, development, diagnosis, and treatment of diseases has attracted much attention among researchers recently. In this study, we present an unsupervised deep learning model of the variational autoencoder for MiRNA–disease association prediction (VAEMDA). Through combining the integrated miRNA similarity and the integrated disease similarity with known miRNA–disease associations, respectively, we constructed two spliced matrices. These matrices were applied to train the variational autoencoder (VAE), respectively. The final predicted association scores between miRNAs and diseases were obtained by integrating the scores from the two trained VAE models. Unlike previous models, VAEMDA can avoid noise introduced by the random selection of negative samples and reveal associations between miRNAs and diseases from the perspective of data distribution. Compared with previous methods, VAEMDA obtained higher area under the receiver operating characteristics curves (AUCs) of 0.9118, 0.8652, and 0.9091 ± 0.0065 in global leave-one-out cross validation (LOOCV), local LOOCV, and five-fold cross validation, respectively. Further, the AUCs of VAEMDA were 0.8250 and 0.8237 in global leave-one-disease-out cross validation (LODOCV), and local LODOCV, respectively. In three different types of case studies on three important diseases, the results showed that most of the top 50 potentially associated miRNAs were verified by databases and the literature.
48
Gupta, Abhishek, Ahmed Shaharyar Khwaja, Alagan Anpalagan, Ling Guan, and Bala Venkatesh. "Policy-Gradient and Actor-Critic Based State Representation Learning for Safe Driving of Autonomous Vehicles." Sensors 20, no. 21 (October 22, 2020): 5991. http://dx.doi.org/10.3390/s20215991.
Full textAbstract:
In this paper, we propose an environment perception framework for autonomous driving using state representation learning (SRL). Unlike existing Q-learning based methods for efficient environment perception and object detection, our proposed method takes the learning loss into account under deterministic as well as stochastic policy gradient. Through a combination of variational autoencoder (VAE), deep deterministic policy gradient (DDPG), and soft actor-critic (SAC), we focus on uninterrupted and reasonably safe autonomous driving without steering off the track for a considerable driving distance. Our proposed technique exhibits learning in autonomous vehicles under complex interactions with the environment, without being explicitly trained on driving datasets. To ensure the effectiveness of the scheme over a sustained period of time, we employ a reward-penalty based system where a negative reward is associated with an unfavourable action and a positive reward is awarded for favourable actions. The results obtained through simulations on DonKey simulator show the effectiveness of our proposed method by examining the variations in policy loss, value loss, reward function, and cumulative reward for ‘VAE+DDPG’ and ‘VAE+SAC’ over the learning process.
49
Dai, Mengxi, Dezhi Zheng, Rui Na, Shuai Wang, and Shuailei Zhang. "EEG Classification of Motor Imagery Using a Novel Deep Learning Framework." Sensors 19, no. 3 (January 29, 2019): 551. http://dx.doi.org/10.3390/s19030551.
Full textAbstract:
Successful applications of brain-computer interface (BCI) approaches to motor imagery (MI) are still limited. In this paper, we propose a classification framework for MI electroencephalogram (EEG) signals that combines a convolutional neural network (CNN) architecture with a variational autoencoder (VAE) for classification. The decoder of the VAE generates a Gaussian distribution, so it can be used to fit the Gaussian distribution of EEG signals. A new representation of input was developed by combining the time, frequency, and channel information from the EEG signal, and the CNN-VAE method was designed and optimized accordingly for this form of input. In this network, the classification of the extracted CNN features is performed via the deep network VAE. Our framework, with an average kappa value of 0.564, outperforms the best classification method in the literature for BCI Competition IV dataset 2b with a 3% improvement. Furthermore, using our own dataset, the CNN-VAE framework also yields the best performance for both three-electrode and five-electrode EEGs and achieves the best average kappa values 0.568 and 0.603, respectively. Our results show that the proposed CNN-VAE method raises performance to the current state of the art.
50
Lin, Weiwei, Tai Ma, Zeqing Zhang, Xiaofan Li, and Xingsi Xue. "Variational Autoencoder for Zero-Shot Recognition of Bai Characters." Wireless Communications and Mobile Computing 2022 (July 4, 2022): 1–8. http://dx.doi.org/10.1155/2022/2717322.
Full textAbstract:
When talking about Bai nationality, people are impressed by its long history and the language it has created. However, since fewer people of the young generation learn the traditional language, the glorious Bai culture becomes less known, making understanding Bai characters difficult. Based on the highly precise character recognition model for Bai characters, the paper is aimed at helping people read books written in Bai characters so as to popularize the culture. To begin with, a data set is built with the support of Bai culture fans and experts. However, the data set is not large enough as knowledge in this respect is limited. This makes the deep learning model less accurate since it lacks sufficient data. The popular zero-shot learning (ZSL) is adopted to overcome the insufficiency of data sets. We use Chinese characters as the seen class, Bai characters as the unseen class, and the number of strokes as the attribute to construct the ZSL format data set. However, the existing ZSL methods ignore the character structure information, so a generation method based on variational autoencoder (VAE) is put forward, which can automatically capture the character structure information. Experimental results show that the method facilitates the recognition of Bai characters and makes it more precise.