Academic literature on the topic 'Ensemble neural noise'

Information theory is a powerful tool for analyzing complex systems. In many areas of neuroscience, it is now possible to gather data from large ensembles of neural variables (e.g., data from many neurons, genes, or voxels). The individual variables can be analyzed with information theory to provide estimates of information shared between variables (forming a network between variables), or between neural variables and other variables (e.g., behavior or sensory stimuli). However, it can be difficult to (1) evaluate if the ensemble is significantly different from what would be expected in a purely noisy system and (2) determine if two ensembles are different. Herein, we introduce relatively simple methods to address these problems by analyzing ensembles of information sources. We demonstrate how an ensemble built of mutual information connections can be compared to null surrogate data to determine if the ensemble is significantly different from noise. Next, we show how two ensembles can be compared using a randomization process to determine if the sources in one contain more information than the other. All code necessary to carry out these analyses and demonstrations are provided.

Research in sound classification and recognition is rapidly advancing in the field of pattern recognition. One important area in this field is environmental sound recognition, whether it concerns the identification of endangered species in different habitats or the type of interfering noise in urban environments. Since environmental audio datasets are often limited in size, a robust model able to perform well across different datasets is of strong research interest. In this paper, ensembles of classifiers are combined that exploit six data augmentation techniques and four signal representations for retraining five pre-trained convolutional neural networks (CNNs); these ensembles are tested on three freely available environmental audio benchmark datasets: (i) bird calls, (ii) cat sounds, and (iii) the Environmental Sound Classification (ESC-50) database for identifying sources of noise in environments. To the best of our knowledge, this is the most extensive study investigating ensembles of CNNs for audio classification. The best-performing ensembles are compared and shown to either outperform or perform comparatively to the best methods reported in the literature on these datasets, including on the challenging ESC-50 dataset. We obtained a 97% accuracy on the bird dataset, 90.51% on the cat dataset, and 88.65% on ESC-50 using different approaches. In addition, the same ensemble model trained on the three datasets managed to reach the same results on the bird and cat datasets while losing only 0.1% on ESC-50. Thus, we have managed to create an off-the-shelf ensemble that can be trained on different datasets and reach performances competitive with the state of the art.

To overcome the weakness of generic neural networks (NNs) ensemble for prediction intervals (PIs) construction, a novel Map-Reduce framework-based distributed NN ensemble consisting of several local Gaussian granular NN (GGNNs) is proposed in this study. Each local network is weighted according to its contribution to the ensemble model. The weighted coefficient is estimated by evaluating the performance of the constructed PIs from each local network. A new evaluation principle is reported with the consideration of the predicting indices. To estimate the modelling uncertainty and the data noise simultaneously, the Gaussian granular is introduced to the numeric NNs. The constructed PIs can then be calculated by the variance of output distribution of each local NN, i.e., the summation of the model uncertainty variance and the data noise variance. To verify the effectiveness of the proposed model, a series of prediction experiments, including two classical time series with additive noise and two industrial time series, are carried out here. The results indicate that the proposed distributed GGNNs ensemble exhibits a good performance for PIs construction.

This paper presents the applicability of artificial neural networks for 24 hour ahead solar power generation forecasting of a 20 kW photovoltaic system, the developed forecasting is suitable for a reliable Microgrid energy management. In total four neural networks were proposed, namely: multi-layred perceptron, radial basis function, recurrent and a neural network ensemble consisting in ensemble of bagged networks. Forecasting reliability of the proposed neural networks was carried out in terms forecasting error performance basing on statistical and graphical methods. The experimental results showed that all the proposed networks achieved an acceptable forecasting accuracy. In term of comparison the neural network ensemble gives the highest precision forecasting comparing to the conventional networks. In fact, each network of the ensemble over-fits to some extent and leads to a diversity which enhances the noise tolerance and the forecasting generalization performance comparing to the conventional networks.

Noise signal analysis method is widely available for gearbox bevel gear fault detection. However, the noise from the gearbox is usually concealed by background noise, which leads to poor efficiency analysis. This paper reports an ensemble empirical mode decomposition (EEMD) and neural network method for bevel gear fault detection. To extract useful signal, EEMD algorithm was firstly applied to get rid of the background noise. Characteristics from a group of discriminating defect status were then chosen to build the eigenvector. Finally, the eigenvector was imported into a back propagation (BP) neural network classifier for defect diagnosis automatically. Experimental results show that the proposed approach is capable for signal denoising and providing distinguishing characteristics of founded fault. The developed method is an accurate approach to detect fault for tested bevel gear.

Electrocardiogram (ECG) is a documentation of the electrical activities of the heart. It is used to identify a number of cardiac faults such as arrhythmias, AF etc. Quite often the ECG gets corrupted by various kinds of artifacts, thus in order to gain correct information from them, they must first be denoised. This paper presents a novel approach for the filtering of low frequency artifacts of ECG signals by using Complete Ensemble Empirical Mode Decomposition (CEED) and Neural Networks, which removes most of the constituent noise while assuring no loss of information in terms of the morphology of the ECG signal. The contribution of the method lies in the fact that it combines the advantages of both EEMD and ANN. The use of CEEMD ensures that the Neural Network does not get over fitted. It also significantly helps in building better predictors at individual frequency levels. The proposed method is compared with other state-of-the-art methods in terms of Mean Square Error (MSE), Signal to Noise Ratio (SNR) and Correlation Coefficient. The results show that the proposed method has better performance as compared to other state-of-the-art methods for low frequency artifacts removal from EEG.

In this letter, we present a new hierarchical clustering approach based on the evolutionary process of Amari's dynamical neural field model. Dynamical neural field theory provides a theoretical framework macroscopically describing the activity of neuron ensemble. Based on it, our clustering approach is essentially close to the neurophysiological nature of perception. It is also computationally stable, insensitive to noise, flexible, and tractable for data with complex structure. Some examples are given to show the feasibility.

In order to enhance weak signals in strong noise background, a weak signal enhancement method based on EMDNN (neural network-assisted empirical mode decomposition) is proposed. This method combines CEEMD (complementary ensemble empirical mode decomposition), GAN (generative adversarial networks) and LSTM (long short-term memory), it enhances the efficiency of selecting effective natural mode components in empirical mode decomposition, thus the SNR (signal-noise ratio) is improved. It can also reconstruct and enhance weak signals. The experimental results show that the SNR of this method is improved from 4.1 to 6.2, and the weak signal is clearly recovered.

[Truncated abstract] The present study investigated whether any of the characteristics of the compound action potential (CAP) waveform or the spectrum of the neural noise (SNN) recorded from the cochlea, could be used to examine abnormal spike generation in the type I primary afferent neurones, possibly due to pathologies leading to abnormal hearing such as tinnitus or tone decay. It was initially hypothesised that the CAP waveform and SNN contained components produced by the local action currents generated at the peripheral ends of the type I primary afferent neurones, and that changes in these local action currents occurred due to changes in the membrane potential of these neurones. It was further hypothesised that the lateral olivo-cochlear system (LOCS) efferent neurones regulate the membrane potential of the primary afferent dendrites to maintain normal action potential generation, where instability in the membrane potential might lead to abnormal primary afferent firing, and possibly one form of tinnitus. We had hoped that the activity of the LOCS efferent neurones could be observed through secondary changes in the CAP waveform and SNN, resulting from changes in the membrane potential of the primary afferent neurones. The origins of the neural activity generating the CAP waveform and SNN peaks, and the effects of the LOCS on the CAP and SNN were experimentally investigated in guinea pigs using lesions in the auditory system, transient ischemia and asphyxia, focal and systemic temperature changes, and pharmacological manipulations of different regions along the auditory pathway. ... Therefore, the CAP and SNN are altered by changes in the propagation of the action potential along the primary afferent neurones, by changes in the morphology of the tissues surrounding the cochlear nerve, and by changes in the time course of the action currents. If the CAP waveform is not altered, the amplitude of the 1kHz speak in the spontaneous SNN can be used as an objective measure of the spontaneous firing rate of the cochlear neurones. However, because the SNN contains a complex mixture of neural activity from all cochlear neurones, and the amplitude of the spontaneous SNN is variable, it would be difficult to use the spontaneous SNN alone as a differential diagnostic test of cochlear nerve pathologies. To record extratympanic electrocochleography (ET ECochG) from humans, a custom-designed, inexpensive, low-noise, optically isolated biological amplifier was built. Furthermore, a custom-designed extratympanic active electrode and ear canal indifferent electrode were designed, which increased the signal-to-noise ratio of the ECochG recording by a factor of 2, decreasing the overall recording time by 75%. The human and guinea pig CAP waveforms recorded in the present study appeared similar, suggesting that the origins of the human and guinea pig CAP waveforms were the same, and that experimental manipulations of the guinea pig CAP waveform can be used to diagnose the cause of abnormal human ECochG waveforms in cases of cochlear nerve pathologies.

Aquesta tesi tracta d'algoritmes i fenòmens complexos en xarxes.

En la primera part s'estudia un model de neurones estocàstiques inter-comunicades mitjançant potencials d'acció. Proposem una tècnica de modelització a escala mesoscòpica i estudiem una transició de fase en un acoblament crític entre les neurones. Derivem una regla de plasticitat sinàptica local que fa que la xarxa s'auto-organitzi en el punt crític.

Seguidament tractem el problema d'inferència aproximada en xarxes probabilístiques mitjançant un algorisme que corregeix la solució obtinguda via belief propagation en grafs cíclics basada en una expansió en sèries. Afegint termes de correcció que corresponen a cicles generals en la xarxa, s'obté el resultat exacte. Introduïm i analitzem numèricament una manera de truncar aquesta sèrie.

Finalment analizem la interacció social en una comunitat d'Internet caracteritzant l'estructura de la xarxa d'usuaris, els fluxes de discussió en forma de comentaris i els patrons de temps de reacció davant una nova notícia.
This thesis is about algorithms and complex phenomena in networks.

In the first part we study a network model of stochastic spiking neurons. We propose a modelling technique based on a mesoscopic description level and show the presence of a phase transition around a critical coupling strength. We derive a local plasticity which drives the network towards the critical point.

We then deal with approximate inference in probabilistic networks. We develop an algorithm which corrects the belief propagation solution for loopy graphs based on a loop series expansion. By adding correction terms, one for each "generalized loop" in the network, the exact result is recovered. We introduce and analyze numerically a particular way of truncating the series.

Finally, we analyze the social interaction of an Internet community by characterizing the structure of the network of users, their discussion threads and the temporal patterns of reaction times to a new post.

Data mining (DM) is the process of finding patterns and relationships in databases. The breakthrough in computer technologies triggered a massive growth in data collected and maintained by organisations. In many applications, these data arrive continuously in large volumes as a sequence of instances known as a data stream. Mining these data is known as stream data mining. Due to the large amount of data arriving in a data stream, each record is normally expected to be processed only once. Moreover, this process can be carried out on different sites in the organisation simultaneously making the problem distributed in nature. Distributed stream data mining poses many challenges to the data mining community including scalability and coping with changes in the underlying concept over time. In this thesis, the author hypothesizes that learning classifier systems (LCSs) - a class of classification algorithms - have the potential to work efficiently in distributed stream data mining. LCSs are an incremental learner, and being evolutionary based they are inherently adaptive. However, they suffer from two main drawbacks that hinder their use as fast data mining algorithms. First, they require a large population size, which slows down the processing of arriving instances. Second, they require a large number of parameter settings, some of them are very sensitive to the nature of the learning problem. As a result, it becomes difficult to choose a right setup for totally unknown problems. The aim of this thesis is to attack these two problems in LCS, with a specific focus on UCS - a supervised evolutionary learning classifier system. UCS is chosen as it has been tested extensively on classification tasks and it is the supervised version of XCS, a state of the art LCS. In this thesis, the architectural design for a distributed stream data mining system will be first introduced. The problems that UCS should face in a distributed data stream task are confirmed through a large number of experiments with UCS and the proposed architectural design. To overcome the problem of large population sizes, the idea of using a Neural Network to represent the action in UCS is proposed. This new system - called NLCS { was validated experimentally using a small fixed population size and has shown a large reduction in the population size needed to learn the underlying concept in the data. An adaptive version of NLCS called ANCS is then introduced. The adaptive version dynamically controls the population size of NLCS. A comprehensive analysis of the behaviour of ANCS revealed interesting patterns in the behaviour of the parameters, which motivated an ensemble version of the algorithm with 9 nodes, each using a different parameter setting. In total they cover all patterns of behaviour noticed in the system. A voting gate is used for the ensemble. The resultant ensemble does not require any parameter setting, and showed better performance on all datasets tested. The thesis concludes with testing the ANCS system in the architectural design for distributed environments proposed earlier. The contributions of the thesis are: (1) reducing the UCS population size by an order of magnitude using a neural representation; (2) introducing a mechanism for adapting the population size; (3) proposing an ensemble method that does not require parameter setting; and primarily (4) showing that the proposed LCS can work efficiently for distributed stream data mining tasks.

Contexte La connectomique, ou la cartographie des connexions neuronales, est un champ de recherche des neurosciences évoluant rapidement, promettant des avancées majeures en ce qui concerne la compréhension du fonctionnement cérébral. La formation de circuits neuronaux en réponse à des stimuli environnementaux est une propriété émergente du cerveau. Cependant, la connaissance que nous avons de la nature précise de ces réseaux est encore limitée. Au niveau du cortex visuel, qui est l’aire cérébrale la plus étudiée, la manière dont les informations se transmettent de neurone en neurone est une question qui reste encore inexplorée. Cela nous invite à étudier l’émergence des microcircuits en réponse aux stimuli visuels. Autrement dit, comment l’interaction entre un stimulus et une assemblée cellulaire est-elle mise en place et modulée? Méthodes En réponse à la présentation de grilles sinusoïdales en mouvement, des ensembles neuronaux ont été enregistrés dans la couche II/III (aire 17) du cortex visuel primaire de chats anesthésiés, à l’aide de multi-électrodes en tungstène. Des corrélations croisées ont été effectuées entre l’activité de chacun des neurones enregistrés simultanément pour mettre en évidence les liens fonctionnels de quasi-synchronie (fenêtre de ± 5 ms sur les corrélogrammes croisés corrigés). Ces liens fonctionnels dévoilés indiquent des connexions synaptiques putatives entre les neurones. Par la suite, les histogrammes peri-stimulus (PSTH) des neurones ont été comparés afin de mettre en évidence la collaboration synergique temporelle dans les réseaux fonctionnels révélés. Enfin, des spectrogrammes dépendants du taux de décharges entre neurones ou stimulus-dépendants ont été calculés pour observer les oscillations gamma dans les microcircuits émergents. Un indice de corrélation (Rsc) a également été calculé pour les neurones connectés et non connectés. Résultats Les neurones liés fonctionnellement ont une activité accrue durant une période de 50 ms contrairement aux neurones fonctionnellement non connectés. Cela suggère que les connexions entre neurones mènent à une synergie de leur inter-excitabilité. En outre, l’analyse du spectrogramme dépendant du taux de décharge entre neurones révèle que les neurones connectés ont une plus forte activité gamma que les neurones non connectés durant une fenêtre d’opportunité de 50ms. L’activité gamma de basse-fréquence (20-40 Hz) a été associée aux neurones à décharge régulière (RS) et l’activité de haute fréquence (60-80 Hz) aux neurones à décharge rapide (FS). Aussi, les neurones fonctionnellement connectés ont systématiquement un Rsc plus élevé que les neurones non connectés. Finalement, l’analyse des corrélogrammes croisés révèle que dans une assemblée neuronale, le réseau fonctionnel change selon l’orientation de la grille. Nous démontrons ainsi que l’intensité des relations fonctionnelles dépend de l’orientation de la grille sinusoïdale. Cette relation nous a amené à proposer l’hypothèse suivante : outre la sélectivité des neurones aux caractères spécifiques du stimulus, il y a aussi une sélectivité du connectome. En bref, les réseaux fonctionnels «signature » sont activés dans une assemblée qui est strictement associée à l’orientation présentée et plus généralement aux propriétés des stimuli. Conclusion Cette étude souligne le fait que l’assemblée cellulaire, plutôt que le neurone, est l'unité fonctionnelle fondamentale du cerveau. Cela dilue l'importance du travail isolé de chaque neurone, c’est à dire le paradigme classique du taux de décharge qui a été traditionnellement utilisé pour étudier l'encodage des stimuli. Cette étude contribue aussi à faire avancer le débat sur les oscillations gamma, en ce qu'elles surviennent systématiquement entre neurones connectés dans les assemblées, en conséquence d’un ajout de cohérence. Bien que la taille des assemblées enregistrées soit relativement faible, cette étude suggère néanmoins une intrigante spécificité fonctionnelle entre neurones interagissant dans une assemblée en réponse à une stimulation visuelle. Cette étude peut être considérée comme une prémisse à la modélisation informatique à grande échelle de connectomes fonctionnels.
Background ‘Connectomics’— the mapping of neural connections, is a rapidly advancing field in neurosciences and it promises significant insights into the brain functioning. The formation of neuronal circuits in response to the sensory environment is an emergent property of the brain; however, the knowledge about the precise nature of these sub-networks is still limited. Even at the level of the visual cortex, which is the most studied area in the brain, how sensory inputs are processed between its neurons, is a question yet to be completely explored. Heuristically, this invites an investigation into the emergence of micro-circuits in response to a visual input — that is, how the intriguing interplay between a stimulus and a cell assembly is engineered and modulated? Methods Neuronal assemblies were recorded in response to randomly presented drifting sine-wave gratings in the layer II/III (area 17) of the primary visual cortex (V1) in anaesthetized cats using tungsten multi-electrodes. Cross-correlograms (CCGs) between simultaneously recorded neural activities were computed to reveal the functional links between neurons that were indicative of putative synaptic connections between them. Further, the peristimulus time histograms (PSTH) of neurons were compared to divulge the epochal synergistic collaboration in the revealed functional networks. Thereafter, perievent spectrograms were computed to observe the gamma oscillations in emergent microcircuits. Noise correlation (Rsc) was calculated for the connected and unconnected neurons within these microcircuits. Results The functionally linked neurons collaborate synergistically with augmented activity in a 50-ms window of opportunity compared with the functionally unconnected neurons suggesting that the connectivity between neurons leads to the added excitability between them. Further, the perievent spectrogram analysis revealed that the connected neurons had an augmented power of gamma activity compared with the unconnected neurons in the emergent 50-ms window of opportunity. The low-band (20-40 Hz) gamma activity was linked to the regular-spiking (RS) neurons, whereas the high-band (60-80 Hz) activity was related to the fast-spiking (FS) neurons. The functionally connected neurons systematically displayed higher Rsc compared with the unconnected neurons in emergent microcircuits. Finally, the CCG analysis revealed that there is an activation of a salient functional network in an assembly in relation to the presented orientation. Closely tuned neurons exhibited more connections than the distantly tuned neurons. Untuned assemblies did not display functional linkage. In short, a ‘signature’ functional network was formed between neurons comprising an assembly that was strictly related to the presented orientation. Conclusion Indeed, this study points to the fact that a cell-assembly is the fundamental functional unit of information processing in the brain, rather than the individual neurons. This dilutes the importance of a neuron working in isolation, that is, the classical firing rate paradigm that has been traditionally used to study the encoding of a stimulus. This study also helps to reconcile the debate on gamma oscillations in that they systematically originate between the connected neurons in assemblies. Though the size of the recorded assemblies in the current investigation was relatively small, nevertheless, this study shows the intriguing functional specificity of interacting neurons in an assembly in response to a visual input. One may form this study as a premise to computationally infer the functional connectomes on a larger scale.

The main goal of this project was to define and evaluate a new unsupervised deep learning approach that can differentiate between normal and anomalous intervals of signals like the electrical activity of the heart (ECG). Denoising autoencoders based on recurrent neural networks with gated recurrent units were used for the semantic encoding of such time frames. A subsequent cluster analysis conducted in the code space served as the decision mechanism labelling samples as anomalies or normal intervals, respectively. The cluster ensemble method called cluster-based similarity partitioning proved itself well suited for this task when used in combination with density-based spatial clustering of applications with noise. The best performing system reached an adjusted Rand index of 0.11 on real-world ECG signals labelled by medical experts. This corresponds to a precision and recall regarding the detection task of around 0.72. The new general approach outperformed several state-of-the-art outlier recognition methods and can be applied to all kinds of (medical) time series data. It can serve as a basis for more specific detectors that work in an unsupervised fashion or that are partially guided by medical experts.

Accurate prediction of the energy consumption is critical for energy-efficient production systems. However, the majority of existing prediction models aim at providing only point predictions and can be affected by uncertainties in the model parameters and input data. In this paper, a prediction model that generates prediction intervals (PIs) for estimating energy consumption of a milling machine is proposed. PIs are used to provide information on the confidence in the prediction by accounting for the uncertainty in both the model parameters and the noise in the input variables. An ensemble model of neural networks (NNs) is used to estimate PIs. A k-nearest-neighbors (k-nn) approach is applied to identify similar patterns between training and testing sets to increase the accuracy of the results by using local information from the closest patterns of the training sets. Finally, a case study that uses a dataset obtained by machining 18 parts through face-milling, contouring, slotting and pocketing, spiraling, and drilling operations is presented. Of these six operations, the case study focuses on face milling to demonstrate the effectiveness of the proposed energy prediction model.

Abstract Experiments are performed on a turbulent swirling flame placed inside a vertical tube whose fundamental acoustic mode becomes unstable at higher powers and equivalence ratios. The power, equivalence ratio, fuel composition and boundary condition of this tube are varied and, at each operating point, the combustion noise is recorded. In addition, short acoustic pulses at the fundamental frequency are supplied to the tube with a loudspeaker and the decay rates of subsequent acoustic oscillations are measured. This quantifies the linear stability of the system at every operating point. Using this data for training, we show that it is possible for a Bayesian ensemble of neural networks to predict the decay rate from a 300 millisecond sample of the (un-pulsed) combustion noise and therefore forecast impending thermoacoustic instabilities. We also show that it is possible to recover the equivalence ratio and power of the flame from these noise snippets, confirming our hypothesis that combustion noise indeed provides a fingerprint of the combustor’s internal state. Furthermore, the Bayesian nature of our algorithm enables principled estimates of uncertainty in our predictions, a reassuring feature that prevents it from making overconfident extrapolations. We use the techniques of permutation importance and integrated gradients to understand which features in the combustion noise spectra are crucial for accurate predictions and how they might influence the prediction. This study serves as a first step towards establishing interpretable and Bayesian machine learning techniques as tools to discover informative relationships in combustor data and thereby build trustworthy, robust and reliable combustion diagnostics.

Abstract The purpose of this study is to establish a model for the diagnosis of multiple micro-punch failures. The punch is assumed to a rigid body structure with a small change in the stiffness during the piercing process and its diameter is varied between Ø0.8–1.2 mm. Thus, the wearing trend of multiple punches in the piercing process and source of the interfered signals make it extremely difficult to analyze. The two major challenges that affect punch failure estimation are the poor signal-to-noise ratio within the factory environment and the rigid body mode disturbance in the signal. To acquire the vibratory signals of the piercing motion, uniaxial accelerometers were outfitted in the vertical direction on the progressive die. Since the piercing process is a series of highly nonlinear transient processes, the Ensemble Empirical Mode Decomposition (EEMD) is adopted as a decoupling operation tool for this kind of non-stationary signal. Furthermore, the dimension-reduced process can be manipulated by Intrinsic Mode Function (IMF) and a function representative of the feature is selected as the input for neural network model training. The training target is the most direct relationship with the product quality, the selected models are multi-layer perceptron and the back-propagation neural network (BPNN) of the error inversion algorithm. The artificial intelligence failure diagnosis of the piercing process is also realized.

Relation extraction has been widely studied to extract new relational facts from open corpus. Previous relation extraction methods are faced with the problem of wrong labels and noisy data, which substantially decrease the performance of the model. In this paper, we propose an ensemble neural network model - Adaptive Boosting LSTMs with Attention, to more effectively perform relation extraction. Specifically, our model first employs the recursive neural network LSTMs to embed each sentence. Then we import attention into LSTMs by considering that the words in a sentence do not contribute equally to the semantic meaning of the sentence. Next via adaptive boosting, we build strategically several such neural classifiers. By ensembling multiple such LSTM classifiers with adaptive boosting, we could build a more effective and robust joint ensemble neural networks based relation extractor. Experiment results on real dataset demonstrate the superior performance of the proposed model, improving F1-score by about 8% compared to the state-of-the-art models.