Journal articles on the topic 'Parsimonious Neural Networks'
To see the other types of publications on this topic, follow the link: Parsimonious Neural Networks.
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Parsimonious Neural Networks.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Valsecchi, Cecile, Viviana Consonni, Roberto Todeschini, Marco Emilio Orlandi, Fabio Gosetti, and Davide Ballabio. "Parsimonious Optimization of Multitask Neural Network Hyperparameters." Molecules 26, no. 23 (November 30, 2021): 7254. http://dx.doi.org/10.3390/molecules26237254.
Full textAbstract:
Neural networks are rapidly gaining popularity in chemical modeling and Quantitative Structure–Activity Relationship (QSAR) thanks to their ability to handle multitask problems. However, outcomes of neural networks depend on the tuning of several hyperparameters, whose small variations can often strongly affect their performance. Hence, optimization is a fundamental step in training neural networks although, in many cases, it can be very expensive from a computational point of view. In this study, we compared four of the most widely used approaches for tuning hyperparameters, namely, grid search, random search, tree-structured Parzen estimator, and genetic algorithms on three multitask QSAR datasets. We mainly focused on parsimonious optimization and thus not only on the performance of neural networks, but also the computational time that was taken into account. Furthermore, since the optimization approaches do not directly provide information about the influence of hyperparameters, we applied experimental design strategies to determine their effects on the neural network performance. We found that genetic algorithms, tree-structured Parzen estimator, and random search require on average 0.08% of the hours required by grid search; in addition, tree-structured Parzen estimator and genetic algorithms provide better results than random search.
2
WANG, NING, MENG JOO ER, XIAN-YAO MENG, and XIANG LI. "AN ONLINE SELF-ORGANIZING SCHEME FOR PARSIMONIOUS AND ACCURATE FUZZY NEURAL NETWORKS." International Journal of Neural Systems 20, no. 05 (October 2010): 389–403. http://dx.doi.org/10.1142/s0129065710002486.
Full textAbstract:
In this paper, an online self-organizing scheme for Parsimonious and Accurate Fuzzy Neural Networks (PAFNN), and a novel structure learning algorithm incorporating a pruning strategy into novel growth criteria are presented. The proposed growing procedure without pruning not only simplifies the online learning process but also facilitates the formation of a more parsimonious fuzzy neural network. By virtue of optimal parameter identification, high performance and accuracy can be obtained. The learning phase of the PAFNN involves two stages, namely structure learning and parameter learning. In structure learning, the PAFNN starts with no hidden neurons and parsimoniously generates new hidden units according to the proposed growth criteria as learning proceeds. In parameter learning, parameters in premises and consequents of fuzzy rules, regardless of whether they are newly created or already in existence, are updated by the extended Kalman filter (EKF) method and the linear least squares (LLS) algorithm, respectively. This parameter adjustment paradigm enables optimization of parameters in each learning epoch so that high performance can be achieved. The effectiveness and superiority of the PAFNN paradigm are demonstrated by comparing the proposed method with state-of-the-art methods. Simulation results on various benchmark problems in the areas of function approximation, nonlinear dynamic system identification and chaotic time-series prediction demonstrate that the proposed PAFNN algorithm can achieve more parsimonious network structure, higher approximation accuracy and better generalization simultaneously.
3
LEVI, REGEV, EYTAN RUPPIN, YOSSI MATIAS, and JAMES A. REGGIA. "FREQUENCY-SPATIAL TRANSFORMATION: A PROPOSAL FOR PARSIMONIOUS INTRA-CORTICAL COMMUNICATION." International Journal of Neural Systems 07, no. 05 (November 1996): 591–98. http://dx.doi.org/10.1142/s0129065796000579.
Full textAbstract:
This work examines a neural network model of a cortical module, where neurons are organized on a 2-dimensional sheet and are connected with higher probability to their spatial neighbors. Motivated by recent findings that cortical neurons have a resonant peak in their impedance magnitude function, we present a frequency-spatial transformation scheme that is schematically described as follows: An external input signal, applied to a small input subset of the neurons, spreads along the network. Due to a stochastic component in the dynamics of the neurons, the frequency of the spreading signal decreases as it propagates through the network. Depending on the input signal frequency, different neural assemblies will hence fire at their specific resonance frequency. We show analytically that the resulting frequency-spatial transformation is well-formed; an injective, fixed, mapping is obtained. Extensive numerical simulations demonstrate that a homogeneous, well-formed transformation may also be obtained in neural networks with cortical-like “Mexican-hat” connectivity. We hypothesize that a frequency-spatial transformation may serve as a basis for parsimonious cortical communication.
4
Tian, Ye, Yue-Ping Xu, Zongliang Yang, Guoqing Wang, and Qian Zhu. "Integration of a Parsimonious Hydrological Model with Recurrent Neural Networks for Improved Streamflow Forecasting." Water 10, no. 11 (November 14, 2018): 1655. http://dx.doi.org/10.3390/w10111655.
Full textAbstract:
This study applied a GR4J model in the Xiangjiang and Qujiang River basins for rainfall-runoff simulation. Four recurrent neural networks (RNNs)—the Elman recurrent neural network (ERNN), echo state network (ESN), nonlinear autoregressive exogenous inputs neural network (NARX), and long short-term memory (LSTM) network—were applied in predicting discharges. The performances of models were compared and assessed, and the best two RNNs were selected and integrated with the lumped hydrological model GR4J to forecast the discharges; meanwhile, uncertainties of the simulated discharges were estimated. The generalized likelihood uncertainty estimation method was applied to quantify the uncertainties. The results show that the LSTM and NARX better captured the time-series dynamics than the other RNNs. The hybrid models improved the prediction of high, median, and low flows, particularly in reducing the bias of underestimation of high flows in the Xiangjiang River basin. The hybrid models reduced the uncertainty intervals by more than 50% for median and low flows, and increased the cover ratios for observations. The integration of a hydrological model with a recurrent neural network considering long-term dependencies is recommended in discharge forecasting.
5
Zhang, Byoung-Tak, Peter Ohm, and Heinz Mühlenbein. "Evolutionary Induction of Sparse Neural Trees." Evolutionary Computation 5, no. 2 (June 1997): 213–36. http://dx.doi.org/10.1162/evco.1997.5.2.213.
Full textAbstract:
This paper is concerned with the automatic induction of parsimonious neural networks. In contrast to other program induction situations, network induction entails parametric learning as well as structural adaptation. We present a novel representation scheme called neural trees that allows efficient learning of both network architectures and parameters by genetic search. A hybrid evolutionary method is developed for neural tree induction that combines genetic programming and the breeder genetic algorithm under the unified framework of the minimum description length principle. The method is successfully applied to the induction of higher order neural trees while still keeping the resulting structures sparse to ensure good generalization performance. Empirical results are provided on two chaotic time series prediction problems of practical interest.
6
Morchid, Mohamed. "Parsimonious memory unit for recurrent neural networks with application to natural language processing." Neurocomputing 314 (November 2018): 48–64. http://dx.doi.org/10.1016/j.neucom.2018.05.081.
Full text
7
Wang, Ning, Meng Joo Er, and Xianyao Meng. "A fast and accurate online self-organizing scheme for parsimonious fuzzy neural networks." Neurocomputing 72, no. 16-18 (October 2009): 3818–29. http://dx.doi.org/10.1016/j.neucom.2009.05.006.
Full text
8
Yang, Da Lin, Wei Dong Yang, and Zhu Zhang. "Online Adaptive Fuzzy Neural Identification of a Piezoelectric Tube Actuator System." Applied Mechanics and Materials 275-277 (January 2013): 915–24. http://dx.doi.org/10.4028/www.scientific.net/amm.275-277.915.
Full textAbstract:
A coupled actuator-flap-circuit system model and its online identification are presented. The coupled system consists of a piezoelectric tube actuator, a trailing-edge flap, and a series R-L-C circuit. The properties of the coupled system are examined using a Mach-scaled rotor simulation on hovering state. According to the high nonlinear hysteretic characteristics of the coupled system, the generalized dynamic fuzzy neural networks (GD-FNN) implementing Takagi-Sugeno-Kang (TSK) fuzzy systems based on extended ellipsoidal radial basis function (EBF) neural network is used to identify the coupled system. The structures and parameters are adaptive adjusted during the learning process, and don’t need too much expert experiences. Simulation studies show that the piezoelectric tube actuator has high authority with a broad frequency bandwidth, satisfies the requirements for helicopter vibration reduction; GD-FNN has a high learning speed, the final networks have a parsimonious network structure and generalize well, possessing broad application prospects in the helicopter vibration reduction.
9
GALVÃO, ROBERTO KAWAKAMI HARROP, and TAKASHI YONEYAMA. "Improving the Discriminatory Capabilities of a Neural Classifier by Using a Biased-Wavelet Layer." International Journal of Neural Systems 09, no. 03 (June 1999): 167–74. http://dx.doi.org/10.1142/s0129065799000150.
Full textAbstract:
In the context of wavelet neural networks (WNN's), two modifications to the basic training algorithms are proposed, namely the introduction of a bias component in the wavelets and the adoption of a weight decay policy. A problem of ECG segment classification is used for illustration purposes. Results suggest that bias improves the discriminatory capabilities of the WNN, which is also compared favourably to a conventional perceptron classifier. The use of weight decay during training, followed by pruning, resulted in a more parsimonious network, which also turned out to be a more conservative classifier. The knowledge embedded in the wavelet layer is interpreted with basis on the concept of super-wavelets.
10
Gerber, B. S., T. G. Tape, R. S. Wigton, and P. S. Heckerling. "Selection of Predictor Variables for Pneumonia Using Neural Networks and Genetic Algorithms." Methods of Information in Medicine 44, no. 01 (2005): 89–97. http://dx.doi.org/10.1055/s-0038-1633927.
Full textAbstract:
Summary Background: Artificial neural networks (ANN) can be used to select sets of predictor variable that incorporate nonlinear interactions between variables. We used a genetic algorithm, with selection based on maximizing network accuracy and minimizing network input-layer cardinality, to evolve parsimonious sets of variables for predicting community-acquired pneumonia among patients with respiratory complaints. Methods: ANN were trained on data from 1044 patients in a training cohort, and were applied to 116 patients in a testing cohort. Chromosomes with binary genes representing input-layer variables were operated on by crossover recombination, mutation, and probabilistic selection based on a fitness function incorporating both network accuracy and input-layer cardinality. Results: The genetic algorithm evolved best 10-variable sets that discriminated pneumonia in the training cohort (ROC areas, 0.838 for selection based on average cross entropy (ENT); 0.954 for selection based on ROC area (ROC)), and in the testing cohort (ROC areas, 0.847 for ENT selection; 0.963 for ROC selection), with no significant differences between cohorts. Best variable sets based on the genetic algorithm using ROC selection discriminated pneumonia more accurately than variable sets based on stepwise neural networks (ROC areas, 0.954 versus 0.879, p = 0.030), or stepwise logistic regression (ROC areas, 0.954 versus 0.830, p = 0.000). Variable sets of lower cardinalities were also evolved, which also accurately discriminated pneumonia. Conclusion: Variable sets derived using a genetic algorithm for neural networks accurately discriminated pneumonia from other respiratory conditions, and did so with greater accuracy than variables derived using stepwise neural networks or logistic regression in some cases.
11
Seguin, Caio, Martijn P. van den Heuvel, and Andrew Zalesky. "Navigation of brain networks." Proceedings of the National Academy of Sciences 115, no. 24 (May 30, 2018): 6297–302. http://dx.doi.org/10.1073/pnas.1801351115.
Full textAbstract:
Understanding the mechanisms of neural communication in large-scale brain networks remains a major goal in neuroscience. We investigated whether navigation is a parsimonious routing model for connectomics. Navigating a network involves progressing to the next node that is closest in distance to a desired destination. We developed a measure to quantify navigation efficiency and found that connectomes in a range of mammalian species (human, mouse, and macaque) can be successfully navigated with near-optimal efficiency (>80% of optimal efficiency for typical connection densities). Rewiring network topology or repositioning network nodes resulted in 45–60% reductions in navigation performance. We found that the human connectome cannot be progressively randomized or clusterized to result in topologies with substantially improved navigation performance (>5%), suggesting a topological balance between regularity and randomness that is conducive to efficient navigation. Navigation was also found to (i) promote a resource-efficient distribution of the information traffic load, potentially relieving communication bottlenecks, and (ii) explain significant variation in functional connectivity. Unlike commonly studied communication strategies in connectomics, navigation does not mandate assumptions about global knowledge of network topology. We conclude that the topology and geometry of brain networks are conducive to efficient decentralized communication.
12
MÖLLER, RALF. "VISUAL HOMING IN ANALOG HARDWARE." International Journal of Neural Systems 09, no. 05 (October 1999): 383–89. http://dx.doi.org/10.1142/s0129065799000368.
Full textAbstract:
Insects of several species rely on visual landmarks for returning to important locations in their environment, The "average landmark vector model" is a parsimonious model which reproduces some aspects of the visual homing behavior of bees and ants, To gain insights in the structure and complexity of the neural apparatus that might underly the navigational capabilities of these animals, the average landmark vector model was implemented in analog hardware and used to control a mobile robot. The experiments demonstrate that the apparently complex task of visual homing might be realized by simple and mostly peripheral neural circuits in insect brains.
13
WEI, H. L., and S. A. BILLINGS. "GENERALIZED CELLULAR NEURAL NETWORKS (GCNNs) CONSTRUCTED USING PARTICLE SWARM OPTIMIZATION FOR SPATIO-TEMPORAL EVOLUTIONARY PATTERN IDENTIFICATION." International Journal of Bifurcation and Chaos 18, no. 12 (December 2008): 3611–24. http://dx.doi.org/10.1142/s0218127408022585.
Full textAbstract:
Particle swarm optimization (PSO) is introduced to implement a new constructive learning algorithm for training generalized cellular neural networks (GCNNs) for the identification of spatio-temporal evolutionary (STE) systems. The basic idea of the new PSO-based learning algorithm is to successively approximate the desired signal by progressively pursuing relevant orthogonal projections. This new algorithm will thus be referred to as the orthogonal projection pursuit (OPP) algorithm, which is in mechanism similar to the conventional projection pursuit approach. A novel two-stage hybrid training scheme is proposed for constructing a parsimonious GCNN model. In the first stage, the orthogonal projection pursuit algorithm is applied to adaptively and successively augment the network, where adjustable parameters of the associated units are optimized using a particle swarm optimizer. The resultant network model produced at the first stage may be redundant. In the second stage, a forward orthogonal regression (FOR) algorithm, aided by mutual information estimation, is applied to refine and improve the initially trained network. The effectiveness and performance of the proposed method is validated by applying the new modeling framework to a spatio-temporal evolutionary system identification problem.
14
Alcorta, Candace Storey, and Richard Sosis. "Why ritual works: A rejection of the by-product hypothesis." Behavioral and Brain Sciences 29, no. 6 (December 2006): 613–14. http://dx.doi.org/10.1017/s0140525x06009344.
Full textAbstract:
We argue that ritual is not a by-product as Boyer & Lienard (B&L) claim, but rather an evolved adaptation for social communication that facilitates non-agonistic social interactions among non-kin. We review the neurophysiological effects of ritual and propose neural structures and networks beyond the cortical-striato-pallidal-thalamic circuit (CSPT) likely to be implicated in ritual. The adaptationist approach to ritual offers a more parsimonious model for understanding these effects as well as the findings B&L present.
15
Notarangelo, Nicla Maria, Kohin Hirano, Raffaele Albano, and Aurelia Sole. "Transfer Learning with Convolutional Neural Networks for Rainfall Detection in Single Images." Water 13, no. 5 (February 24, 2021): 588. http://dx.doi.org/10.3390/w13050588.
Full textAbstract:
Near real-time rainfall monitoring at local scale is essential for urban flood risk mitigation. Previous research on precipitation visual effects supports the idea of vision-based rain sensors, but tends to be device-specific. We aimed to use different available photographing devices to develop a dense network of low-cost sensors. Using Transfer Learning with a Convolutional Neural Network, the rainfall detection was performed on single images taken in heterogeneous conditions by static or moving cameras without adjusted parameters. The chosen images encompass unconstrained verisimilar settings of the sources: Image2Weather dataset, dash-cams in the Tokyo Metropolitan area and experiments in the NIED Large-scale Rainfall Simulator. The model reached a test accuracy of 85.28% and an F1 score of 0.86. The applicability to real-world scenarios was proven with the experimentation with a pre-existing surveillance camera in Matera (Italy), obtaining an accuracy of 85.13% and an F1 score of 0.85. This model can be easily integrated into warning systems to automatically monitor the onset and end of rain-related events, exploiting pre-existing devices with a parsimonious use of economic and computational resources. The limitation is intrinsic to the outputs (detection without measurement). Future work concerns the development of a CNN based on the proposed methodology to quantify the precipitation intensity.
16
Gultepe, Eren, Mehran Kamkarhaghighi, and Masoud Makrehchi. "Document classification using convolutional neural networks with small window sizes and latent semantic analysis." Web Intelligence 18, no. 3 (September 30, 2020): 239–48. http://dx.doi.org/10.3233/web-200445.
Full textAbstract:
A parsimonious convolutional neural network (CNN) for text document classification that replicates the ease of use and high classification performance of linear methods is presented. This new CNN architecture can leverage locally trained latent semantic analysis (LSA) word vectors. The architecture is based on parallel 1D convolutional layers with small window sizes, ranging from 1 to 5 words. To test the efficacy of the new CNN architecture, three balanced text datasets that are known to perform exceedingly well with linear classifiers were evaluated. Also, three additional imbalanced datasets were evaluated to gauge the robustness of the LSA vectors and small window sizes. The new CNN architecture consisting of 1 to 4-grams, coupled with LSA word vectors, exceeded the accuracy of all linear classifiers on balanced datasets with an average improvement of 0.73%. In four out of the total six datasets, the LSA word vectors provided a maximum classification performance on par with or better than word2vec vectors in CNNs. Furthermore, in four out of the six datasets, the new CNN architecture provided the highest classification performance. Thus, the new CNN architecture and LSA word vectors could be used as a baseline method for text classification tasks.
17
Ghaseminejad, Ali, and Venkatesh Uddameri. "Physics-inspired integrated space–time artificial neural networks for regional groundwater flow modeling." Hydrology and Earth System Sciences 24, no. 12 (December 3, 2020): 5759–79. http://dx.doi.org/10.5194/hess-24-5759-2020.
Full textAbstract:
Abstract. An integrated space–time artificial neural network (ANN) model inspired by the governing groundwater flow equation was developed to test whether a single ANN is capable of modeling regional groundwater flow systems. Model-independent entropy measures and random forest (RF)-based feature selection procedures were used to identify suitable inputs for ANNs. L2 regularization, five-fold cross-validation, and an adaptive stochastic gradient descent (ADAM) algorithm led to a parsimonious ANN model for a 30 691 km2 agriculturally intensive area in the Ogallala Aquifer of Texas. The model testing at 38 independent wells during the 1956–2008 calibration period showed no overfitting issues and highlighted the model's ability to capture both the observed spatial dependence and temporal variability. The forecasting period (2009–2015) was marked by extreme climate variability in the region and served to evaluate the extrapolation capabilities of the model. While ANN models are universal interpolators, the model was able to capture the general trends and provide groundwater level estimates that were better than using historical means. Model sensitivity analysis indicated that pumping was the most sensitive process. Incorporation of spatial variability was more critical than capturing temporal persistence. The use of the standardized precipitation–evapotranspiration index (SPEI) as a surrogate for pumping was generally adequate but was unable to capture the heterogeneous groundwater extraction preferences of farmers under extreme climate conditions.
18
Moreni, Mael, Jerome Theau, and Samuel Foucher. "Train Fast While Reducing False Positives: Improving Animal Classification Performance Using Convolutional Neural Networks." Geomatics 1, no. 1 (January 15, 2021): 34–49. http://dx.doi.org/10.3390/geomatics1010004.
Full textAbstract:
The combination of unmanned aerial vehicles (UAV) with deep learning models has the capacity to replace manned aircrafts for wildlife surveys. However, the scarcity of animals in the wild often leads to highly unbalanced, large datasets for which even a good detection method can return a large amount of false detections. Our objectives in this paper were to design a training method that would reduce training time, decrease the number of false positives and alleviate the fine-tuning effort of an image classifier in a context of animal surveys. We acquired two highly unbalanced datasets of deer images with a UAV and trained a Resnet-18 classifier using hard-negative mining and a series of recent techniques. Our method achieved sub-decimal false positive rates on two test sets (1 false positive per 19,162 and 213,312 negatives respectively), while training on small but relevant fractions of the data. The resulting training times were therefore significantly shorter than they would have been using the whole datasets. This high level of efficiency was achieved with little tuning effort and using simple techniques. We believe this parsimonious approach to dealing with highly unbalanced, large datasets could be particularly useful to projects with either limited resources or extremely large datasets.
19
Campozano, Lenin, Leandro Robaina, Luis Felipe Gualco, Luis Maisincho, Marcos Villacís, Thomas Condom, Daniela Ballari, and Carlos Páez. "Parsimonious Models of Precipitation Phase Derived from Random Forest Knowledge: Intercomparing Logistic Models, Neural Networks, and Random Forest Models." Water 13, no. 21 (October 28, 2021): 3022. http://dx.doi.org/10.3390/w13213022.
Full textAbstract:
The precipitation phase (PP) affects the hydrologic cycle which in turn affects the climate system. A lower ratio of snow to rain due to climate change affects timing and duration of the stream flow. Thus, more knowledge about the PP occurrence and drivers is necessary and especially important in cities dependent on water coming from glaciers, such as Quito, the capital of Ecuador (2.5 million inhabitants), depending in part on the Antisana glacier. The logistic models (LM) of PP rely only on air temperature and relative humidity to predict PP. However, the processes related to PP are far more complex. The aims of this study were threefold: (i) to compare the performance of random forest (RF) and artificial neural networks (ANN) to derive PP in relation to LM; (ii) to identify the main drivers of PP occurrence using RF; and (iii) to develop LM using meteorological drivers derived from RF. The results show that RF and ANN outperformed LM in predicting PP in 8 out of 10 metrics. RF indicated that temperature, dew point temperature, and specific humidity are more important than wind or radiation for PP occurrence. With these predictors, parsimonious and efficient models were developed showing that data mining may help in understanding complex processes and complements expert knowledge.
20
Zhang, Byoung-Tak, and Heinz Mühlenbein. "Balancing Accuracy and Parsimony in Genetic Programming." Evolutionary Computation 3, no. 1 (March 1995): 17–38. http://dx.doi.org/10.1162/evco.1995.3.1.17.
Full textAbstract:
Genetic programming is distinguished from other evolutionary algorithms in that it uses tree representations of variable size instead of linear strings of fixed length. The flexible representation scheme is very important because it allows the underlying structure of the data to be discovered automatically. One primary difficulty, however, is that the solutions may grow too big without any improvement of their generalization ability. In this article we investigate the fundamental relationship between the performance and complexity of the evolved structures. The essence of the parsimony problem is demonstrated empirically by analyzing error landscapes of programs evolved for neural network synthesis. We consider genetic programming as a statistical inference problem and apply the Bayesian model-comparison framework to introduce a class of fitness functions with error and complexity terms. An adaptive learning method is then presented that automatically balances the model-complexity factor to evolve parsimonious programs without losing the diversity of the population needed for achieving the desired training accuracy. The effectiveness of this approach is empirically shown on the induction of sigma-pi neural networks for solving a real-world medical diagnosis problem as well as benchmark tasks.
21
Niv, Yael, Daphna Joel, Isaac Meilijson, and Eytan Ruppin. "Evolution of Reinforcement Learning in Uncertain Environments: A Simple Explanation for Complex Foraging Behaviors." Adaptive Behavior 10, no. 1 (January 1, 2002): 5–24. http://dx.doi.org/10.1177/1059-712302-010001-01.
Full textAbstract:
Reinforcement learning is a fundamental process by which organisms learn to achieve goals from their interactions with the environment. Using evolutionary computation techniques we evolve (near-)optimal neuronal learning rules in a simple neural network model of reinforcement learning in bumblebees foraging for nectar. The resulting neural networks exhibit efficient reinforcement learning, allowing the bees to respond rapidly to changes in reward contingencies. The evolved synaptic plasticity dynamics give rise to varying exploration/exploitation levels and to the well-documented choice strategies of risk aversion and probability matching. Additionally, risk aversion is shown to emerge even when bees are evolved in a completely risk-less environment. In contrast to existing theories in economics and game theory, risk-averse behavior is shown to be a direct consequence of (near-)optimal reinforcement learning, without requiring additional assumptions such as the existence of a nonlinear subjective utility function for rewards. Our results are corroborated by a rigorous mathematical analysis, and their robustness in real-world situations is supported by experiments in a mobile robot. Thus we provide a biologically founded, parsimonious, and novel explanation for risk aversion and probability matching.
22
Lai, Sue Ling, Ming Liu, Kuo Cheng Kuo, and Ray Chang. "Energy Consumption Forecasting in Hong Kong Using ARIMA and Artificial Neural Networks Models ." Applied Mechanics and Materials 672-674 (October 2014): 2085–97. http://dx.doi.org/10.4028/www.scientific.net/amm.672-674.2085.
Full textAbstract:
There have been considerable efforts contributed to the development of effective energy demand forecast models due to its critical role for economic development and environmental protection. This study focused on the adoption of artificial neural network (ANN) and autoregressive integrated moving average (ARIMA) models for energy consumption forecasting in Hong Kong over the period of 1975-2010. Four predictors were considered, including population, GDP, exports, and total visitor arrivals. The results show most ANN models demonstrate acceptable forecast accuracy when single predictor is considered. The best single input model is the case with GDP as predictor. Population and exports are the next proper single inputs. The model with total visitor arrivals as sole predictor does not perform satisfactorily. This indicates that tourism development demonstrates a different pattern from that of energy consumption. In addition, the forecast accuracy of ANN does not improve considerably as the number of predictors increase. Findings imply that with the ANN approach, choosing appropriate predictors is more important than increasing the number of predictors. On the other hand, ARIMA generates forecasts as accurate as some good cases by ANN. Results suggest that ARIMA is not only a parsimonious but effective approach for energy consumption forecasting in Hong Kong.
23
Luo, Xiaoliang, Brett D. Roads, and Bradley C. Love. "The Costs and Benefits of Goal-Directed Attention in Deep Convolutional Neural Networks." Computational Brain & Behavior 4, no. 2 (February 12, 2021): 213–30. http://dx.doi.org/10.1007/s42113-021-00098-y.
Full textAbstract:
AbstractPeople deploy top-down, goal-directed attention to accomplish tasks, such as finding lost keys. By tuning the visual system to relevant information sources, object recognition can become more efficient (a benefit) and more biased toward the target (a potential cost). Motivated by selective attention in categorisation models, we developed a goal-directed attention mechanism that can process naturalistic (photographic) stimuli. Our attention mechanism can be incorporated into any existing deep convolutional neural networks (DCNNs). The processing stages in DCNNs have been related to ventral visual stream. In that light, our attentional mechanism incorporates top-down influences from prefrontal cortex (PFC) to support goal-directed behaviour. Akin to how attention weights in categorisation models warp representational spaces, we introduce a layer of attention weights to the mid-level of a DCNN that amplify or attenuate activity to further a goal. We evaluated the attentional mechanism using photographic stimuli, varying the attentional target. We found that increasing goal-directed attention has benefits (increasing hit rates) and costs (increasing false alarm rates). At a moderate level, attention improves sensitivity (i.e. increases $d^{\prime }$ d ′ ) at only a moderate increase in bias for tasks involving standard images, blended images and natural adversarial images chosen to fool DCNNs. These results suggest that goal-directed attention can reconfigure general-purpose DCNNs to better suit the current task goal, much like PFC modulates activity along the ventral stream. In addition to being more parsimonious and brain consistent, the mid-level attention approach performed better than a standard machine learning approach for transfer learning, namely retraining the final network layer to accommodate the new task.
24
Yang, Genevieve J., John D. Murray, Xiao-Jing Wang, David C. Glahn, Godfrey D. Pearlson, Grega Repovs, John H. Krystal, and Alan Anticevic. "Functional hierarchy underlies preferential connectivity disturbances in schizophrenia." Proceedings of the National Academy of Sciences 113, no. 2 (December 23, 2015): E219—E228. http://dx.doi.org/10.1073/pnas.1508436113.
Full textAbstract:
Schizophrenia may involve an elevated excitation/inhibition (E/I) ratio in cortical microcircuits. It remains unknown how this regulatory disturbance maps onto neuroimaging findings. To address this issue, we implemented E/I perturbations within a neural model of large-scale functional connectivity, which predicted hyperconnectivity following E/I elevation. To test predictions, we examined resting-state functional MRI in 161 schizophrenia patients and 164 healthy subjects. As predicted, patients exhibited elevated functional connectivity that correlated with symptom levels, and was most prominent in association cortices, such as the fronto-parietal control network. This pattern was absent in patients with bipolar disorder (n = 73). To account for the pattern observed in schizophrenia, we integrated neurobiologically plausible, hierarchical differences in association vs. sensory recurrent neuronal dynamics into our model. This in silico architecture revealed preferential vulnerability of association networks to E/I imbalance, which we verified empirically. Reported effects implicate widespread microcircuit E/I imbalance as a parsimonious mechanism for emergent inhomogeneous dysconnectivity in schizophrenia.
25
Iqbal, Abdullah, Nektarios A. Valous, Da-Wen Sun, and Paul Allen. "Parsimonious classification of binary lacunarity data computed from food surface images using kernel principal component analysis and artificial neural networks." Meat Science 87, no. 2 (February 2011): 107–14. http://dx.doi.org/10.1016/j.meatsci.2010.08.014.
Full text
26
Toth, E., and L. Brandimarte. "Prediction of local scour depth at bridge piers under clear-water and live-bed conditions: comparison of literature formulae and artificial neural networks." Journal of Hydroinformatics 13, no. 4 (January 18, 2011): 812–24. http://dx.doi.org/10.2166/hydro.2011.065.
Full textAbstract:
The scouring effect of the flowing water around bridge piers may undermine the stability of the structure, leading to extremely high direct and indirect costs and, in extreme cases, the loss of human lives. The use of Artificial Neural Network (ANN) models has been recently proposed in the literature for estimating the maximum scour depth around bridge piers: this study aims at further investigating the potentiality of the ANN approach and, in particular, at analysing the influence of the experimental setting (laboratory or field data) and of the sediment transport mode (clear water or live bed) on the prediction performances. A large database of both field and laboratory observations has been collected from the literature for predicting the maximum local scour depth as a function of a parsimonious set of variables characterizing the flow, the sediments and the pier. Neural networks with an increasing degree of specialization have been implemented – using different subsets of the calibration data in the training phase – and validated over an external validation dataset. The results confirm that the ANN scour depths' predictions outperform the estimates obtained by empirical formulae conventionally used in the literature and in the current engineering practice, and demonstrate the importance of taking into account the differences in the type of available data – laboratory or field data – and the sediment transport mode – clear water or live bed conditions.
27
RIBEIRO, BERNARDETE, AMÂNDIO MARQUES, JORGE HENRIQUES, and MANUEL ANTUNES. "CHOOSING REAL-TIME PREDICTORS FOR VENTRICULAR ARRHYTHMIA DETECTION." International Journal of Pattern Recognition and Artificial Intelligence 21, no. 08 (December 2007): 1249–63. http://dx.doi.org/10.1142/s0218001407005934.
Full textAbstract:
The risk of developing life-threatening ventricular arrhythmias in patients with structural heart disease is higher with increased occurrence of premature ventricular complex (PVC). Therefore, reliable detection of these arrhythmias is a challenge for a cardiovascular diagnosis system. While early diagnosis is critical, the task of its automatic detection and classification becomes crucial. Therefore, the underlying models should be efficient, albeit ensuring robustness. Although neural networks (NN) have proven successful in this setting, we show that kernel-based learning algorithms achieve superior performance. In particular, recently developed sparse Bayesian methods, such as, Relevance Vector Machines (RVM), present a parsimonious solution when compared with Support Vector Machines (SVM), yet revealing competitive accuracy. This can lead to significant reduction in the computational complexity of the decision function, thereby making RVM more suitable for real-time applications.
28
Venkatesh, S., S. Gopal, and K. Kannan. "Effectiveness of Partition and Graph Theoretic Clustering Algorithms for Multiple Source Partial Discharge Pattern Classification Using Probabilistic Neural Network and Its Adaptive Version: A Critique Based on Experimental Studies." Journal of Electrical and Computer Engineering 2012 (2012): 1–19. http://dx.doi.org/10.1155/2012/479696.
Full textAbstract:
Partial discharge (PD) is a major cause of failure of power apparatus and hence its measurement and analysis have emerged as a vital field in assessing the condition of the insulation system. Several efforts have been undertaken by researchers to classify PD pulses utilizing artificial intelligence techniques. Recently, the focus has shifted to the identification of multiple sources of PD since it is often encountered in real-time measurements. Studies have indicated that classification of multi-source PD becomes difficult with the degree of overlap and that several techniques such as mixed Weibull functions, neural networks, and wavelet transformation have been attempted with limited success. Since digital PD acquisition systems record data for a substantial period, the database becomes large, posing considerable difficulties during classification. This research work aims firstly at analyzing aspects concerning classification capability during the discrimination of multisource PD patterns. Secondly, it attempts at extending the previous work of the authors in utilizing the novel approach of probabilistic neural network versions for classifying moderate sets of PD sources to that of large sets. The third focus is on comparing the ability of partition-based algorithms, namely, the labelled (learning vector quantization) and unlabelled (K-means) versions, with that of a novel hypergraph-based clustering method in providing parsimonious sets of centers during classification.
29
Champion, Kathleen, Bethany Lusch, J. Nathan Kutz, and Steven L. Brunton. "Data-driven discovery of coordinates and governing equations." Proceedings of the National Academy of Sciences 116, no. 45 (October 21, 2019): 22445–51. http://dx.doi.org/10.1073/pnas.1906995116.
Full textAbstract:
The discovery of governing equations from scientific data has the potential to transform data-rich fields that lack well-characterized quantitative descriptions. Advances in sparse regression are currently enabling the tractable identification of both the structure and parameters of a nonlinear dynamical system from data. The resulting models have the fewest terms necessary to describe the dynamics, balancing model complexity with descriptive ability, and thus promoting interpretability and generalizability. This provides an algorithmic approach to Occam’s razor for model discovery. However, this approach fundamentally relies on an effective coordinate system in which the dynamics have a simple representation. In this work, we design a custom deep autoencoder network to discover a coordinate transformation into a reduced space where the dynamics may be sparsely represented. Thus, we simultaneously learn the governing equations and the associated coordinate system. We demonstrate this approach on several example high-dimensional systems with low-dimensional behavior. The resulting modeling framework combines the strengths of deep neural networks for flexible representation and sparse identification of nonlinear dynamics (SINDy) for parsimonious models. This method places the discovery of coordinates and models on an equal footing.
30
Mapuwei, Tichaona W., Oliver Bodhlyera, and Henry Mwambi. "Univariate Time Series Analysis of Short-Term Forecasting Horizons Using Artificial Neural Networks: The Case of Public Ambulance Emergency Preparedness." Journal of Applied Mathematics 2020 (May 1, 2020): 1–11. http://dx.doi.org/10.1155/2020/2408698.
Full textAbstract:
This study examined the applicability of artificial neural network models in modelling univariate time series ambulance demand for short-term forecasting horizons in Zimbabwe. Bulawayo City Councils’ ambulance services department was used as a case study. Two models, feed-forward neural network (FFNN) and seasonal autoregressive integrated moving average, (SARIMA) were developed using monthly historical data from 2010 to 2017 and compared against observed data for 2018. The mean absolute error (MAE), root mean square error (RMSE), and paired sample t-test were used as performance measures. Calculated performance measures for FFNN were MAE (94.0), RMSE (137.19), and the test statistic value p=0.493(>0.05) whilst corresponding values for SARIMA were 105.71, 125.28, and p=0.005(<0.05), respectively. Findings of this study suggest that the FFNN model is inclined to value estimation whilst the SARIMA model is directional with a linear pattern over time. Based on the performance measures, the parsimonious FFNN model was selected to predict short-term annual ambulance demand. Demand forecasts with FFNN for 2019 reflected the expected general trends in Bulawayo. The forecasts indicate high demand during the months of January, March, September, and December. Key ambulance logistic activities such as vehicle servicing, replenishment of essential equipment and drugs, staff training, leave days scheduling, and mock drills need to be planned for April, June, and July when low demand is anticipated. This deliberate planning strategy would avoid a dire situation whereby ambulances are available but without adequate staff, essential drugs, and equipment to respond to public emergency calls.
31
Tao, Xiao-ling, Zi-yi Liu, and Chang-song Yang. "An Efficient Network Security Situation Assessment Method Based on AE and PMU." Wireless Communications and Mobile Computing 2021 (September 13, 2021): 1–9. http://dx.doi.org/10.1155/2021/1173065.
Full textAbstract:
Network security situation assessment (NSSA) is an important and effective active defense technology in the field of network security situation awareness. By analyzing the historical network security situation awareness data, NSSA can evaluate the network security threat and analyze the network attack stage, thus fully grasping the overall network security situation. With the rapid development of 5G, cloud computing, and Internet of things, the network environment is increasingly complex, resulting in diversity and randomness of network threats, which directly determine the accuracy and the universality of NSSA methods. Meanwhile, the indicator data is characterized by large scale and heterogeneity, which seriously affect the efficiency of the NSSA methods. In this paper, we design a new NSSA method based on the autoencoder (AE) and parsimonious memory unit (PMU). In our novel method, we first utilize an AE-based data dimensionality reduction method to process the original indicator data, thus effectively removing the redundant part of the indicator data. Subsequently, we adopt a PMU deep neural network to achieve accurate and efficient NSSA. The experimental results demonstrate that the accuracy and efficiency of our novel method are both greatly improved.
32
Canolty, Ryan T., Charles F. Cadieu, Kilian Koepsell, Karunesh Ganguly, Robert T. Knight, and Jose M. Carmena. "Detecting event-related changes of multivariate phase coupling in dynamic brain networks." Journal of Neurophysiology 107, no. 7 (April 1, 2012): 2020–31. http://dx.doi.org/10.1152/jn.00610.2011.
Full textAbstract:
Oscillatory phase coupling within large-scale brain networks is a topic of increasing interest within systems, cognitive, and theoretical neuroscience. Evidence shows that brain rhythms play a role in controlling neuronal excitability and response modulation (Haider B, McCormick D. Neuron 62: 171–189, 2009) and regulate the efficacy of communication between cortical regions (Fries P. Trends Cogn Sci 9: 474–480, 2005) and distinct spatiotemporal scales (Canolty RT, Knight RT. Trends Cogn Sci 14: 506–515, 2010). In this view, anatomically connected brain areas form the scaffolding upon which neuronal oscillations rapidly create and dissolve transient functional networks (Lakatos P, Karmos G, Mehta A, Ulbert I, Schroeder C. Science 320: 110–113, 2008). Importantly, testing these hypotheses requires methods designed to accurately reflect dynamic changes in multivariate phase coupling within brain networks. Unfortunately, phase coupling between neurophysiological signals is commonly investigated using suboptimal techniques. Here we describe how a recently developed probabilistic model, phase coupling estimation (PCE; Cadieu C, Koepsell K Neural Comput 44: 3107–3126, 2010), can be used to investigate changes in multivariate phase coupling, and we detail the advantages of this model over the commonly employed phase-locking value (PLV; Lachaux JP, Rodriguez E, Martinerie J, Varela F. Human Brain Map 8: 194–208, 1999). We show that the N-dimensional PCE is a natural generalization of the inherently bivariate PLV. Using simulations, we show that PCE accurately captures both direct and indirect (network mediated) coupling between network elements in situations where PLV produces erroneous results. We present empirical results on recordings from humans and nonhuman primates and show that the PCE-estimated coupling values are different from those using the bivariate PLV. Critically on these empirical recordings, PCE output tends to be sparser than the PLVs, indicating fewer significant interactions and perhaps a more parsimonious description of the data. Finally, the physical interpretation of PCE parameters is straightforward: the PCE parameters correspond to interaction terms in a network of coupled oscillators. Forward modeling of a network of coupled oscillators with parameters estimated by PCE generates synthetic data with statistical characteristics identical to empirical signals. Given these advantages over the PLV, PCE is a useful tool for investigating multivariate phase coupling in distributed brain networks.
33
NUNES DE CASTRO, LEANDRO, and FERNANDO J. VON ZUBEN. "AUTOMATIC DETERMINATION OF RADIAL BASIS FUNCTIONS: AN IMMUNITY-BASED APPROACH." International Journal of Neural Systems 11, no. 06 (December 2001): 523–35. http://dx.doi.org/10.1142/s0129065701000941.
Full textAbstract:
The appropriate operation of a radial basis function (RBF) neural network depends mainly upon an adequate choice of the parameters of its basis functions. The simplest approach to train an RBF network is to assume fixed radial basis functions defining the activation of the hidden units. Once the RBF parameters are fixed, the optimal set of output weights can be determined straightforwardly by using a linear least squares algorithm, which generally means reduction in the learning time as compared to the determination of all RBF network parameters using supervised learning. The main drawback of this strategy is the requirement of an efficient algorithm to determine the number, position, and dispersion of the RBFs. The approach proposed here is inspired by models derived from the vertebrate immune system, that will be shown to perform unsupervised cluster analysis. The algorithm is introduced and its performance is compared to that of the random, k-means center selection procedures and other results from the literature. By automatically defining the number of RBF centers, their positions and dispersions, the proposed method leads to parsimonious solutions. Simulation results are reported concerning regression and classification problems.
34
Chapuy, Bjoern, Chip Stewart, Timothy Wood, Andrew Dunford, Kirsty Wienand, Gad Getz, and Margaret A. Shipp. "Validation of the Genetically-Defined DLBCL Subtypes and Generation of a Parsimonious Probabilistic Classifier." Blood 134, Supplement_1 (November 13, 2019): 920. http://dx.doi.org/10.1182/blood-2019-131250.
Full textAbstract:
Diffuse large B-cell lymphoma (DLBCL) is a clinically and molecularly heterogeneous disease with recognized transcriptional subtypes associated with normal cells of origin, activated B-cell (ABC) and germinal center B-cell (GCB) tumors. Emerging data suggested that additional heterogeneity existed, prompting us to comprehensively characterize genomic signatures of 304 newly diagnosed DLBCLs from patients treated with state-of-the-art therapy. We integrated recurrent mutations, somatic copy number alterations (SCNAs) and structural variants (SVs) and identified 5 genetically distinct DLBCL clusters (C1- C5 DLBCLs; Chapuy, Stewart, Dunford, et al. Nat Med 2018). Specifically, we identified two genetically distinct ABC subtypes, including favorable-risk C1 DLBCLs with features of extrafollicular origin and alterations also seen in transformed marginal zone lymphomas (NOTCH2 and NF-κB pathway member mutations and BCL6 SVs). Unfavorable-risk C5 ABC DLBCLs harbored frequent 18q/BCL2 copy gain and co-occurring CD79B and MYD88L265P mutations. We also identified two genetically distinct GCB subtypes, including unfavorable-risk C3 DLBCLs with frequent BCL2 SVs, mutations in chromatin-modifying enzymes (CREBBP, MLL2, EZH2) and BCR/PI3K signaling pathway members (including inactivating PTEN mutations and copy loss). Favorable-risk C4 GCB DLBCLs had frequent mutations in core and linker histones and signaling intermediates (SGK1, BRAF and STAT3). Additionally, we identified an ABC/GCB-independent subtype, C2 DLBCLs, characterized by frequent bi-allelic TP53 inactivation, 9p21.23/CDKN2A copy loss and associated genomic instability reflected in recurrent SCNAs, increased genome doublings and a distinct outcome following induction therapy. A next step in utilizing the characterized genetic substructure was to confirm it in an independent series and develop a molecular classifier that allows prospective identification of C1-C5 DLBCLs. To this end, we accessed whole exome sequencing, copy number and SV data from a recent cohort of newly diagnosed DLBCLs (39 tumor-normal pairs, 462 tumor-only samples; Schmitz et al. NEJM 2018). All samples were re-analyzed using our mutational and SCNA pipelines and our newly generated tumor-only algorithm (Chapuy, Stewart, Dunford, et al. Nat Med 2018) to avoid batch effects and harmonize the datasets. SVs were used as reported. Purity and ploidy were inferred using ABSOLUTE and samples with missing data or low purity were removed. For the combined cohort (579 samples), we assessed our previously characterized 158 genetic drivers (Chapuy, Stewart, Dunford, et al. Nat Med 2018) and confirmed equal distribution of their marginal frequencies (R=0.88, p=1.5e-51), excluding batch effects. Next, we applied non-negative matrix factorization (NNF) consensus clustering to the combined dataset (158 genetic drivers vs. 579 tumors) and confirmed the C1-C5 DLBCL genetic clusters. Notably, tumors from both series contributed at comparable frequencies to the respective C1-C5 DLBCLs. We also noted an enrichment of the alternative genetic labels from Schmitz et al. in 3 of our C1-C5 DLBCL subtypes (B2N in C1 DLBCLs, p<0.0001; EZB in C3 DLBCLs, p<0.0001; MCD in C5 DLBCLs, p<0.0001). These data confirmed the identity of the C1-C5 DLBCL clusters in an independent cohort. Next, we developed a molecular classifier that prospectively identified C1-C5 DLBCLs using a minimum number of easy-to-measure features. The NMF-defined classes of the combined cohort were used as gold-standard training and validation datasets. We tested different models for classification and selected an artificial neural network approach which provides accurate classification of individual samples and well-calibrated confidence metrics. To minimize potential overtraining, we developed a reduced input feature set of the 22 most discriminating features, constructed confidence metrics for each sample and trained an ensemble of Feed-Forward Neural Networks via 10-fold cross validation. With this approach, our classifier had 84% accuracy for the total set and 94% accuracy for the high-confidence samples (70% of all samples). The newly developed parsimonious classifier will allow prospective identification of the independently confirmed C1-C5 DLBCL subtypes in newly diagnosed patients, a necessity for clinical application. Disclosures Getz: Pharmacyclics: Research Funding; IBM: Research Funding; MuTect, ABSOLTUE, MutSig and POLYSOLVER: Patents & Royalties: MuTect, ABSOLTUE, MutSig and POLYSOLVER. Shipp:BMS: Consultancy, Honoraria, Membership on an entity's Board of Directors or advisory committees, Research Funding; Merck & Co.: Honoraria, Membership on an entity's Board of Directors or advisory committees, Research Funding; AstraZeneca: Honoraria, Membership on an entity's Board of Directors or advisory committees; Bayer: Research Funding; Gilead Sciences: Honoraria, Membership on an entity's Board of Directors or advisory committees; Takeda Pharmaceuticals: Honoraria, Membership on an entity's Board of Directors or advisory committees.
35
Kim, Taehwan, and Tülay Adalı. "Approximation by Fully Complex Multilayer Perceptrons." Neural Computation 15, no. 7 (July 1, 2003): 1641–66. http://dx.doi.org/10.1162/089976603321891846.
Full textAbstract:
We investigate the approximation ability of a multi layer perceptron (MLP) network when it is extended to the complex domain. The main challenge for processing complex data with neural networks has been the lack of bounded and analytic complex nonlinear activation functions in the complex domain, as stated by Liouville's theorem. To avoid the conflict between the boundedness and the analyticity of a nonlinear complex function in the complex domain, a number of ad hoc MLPs that include using two real-valued MLPs, one processing the real part and the other processing the imaginary part, have been traditionally employed. However, since nonanalytic functions do not meet the Cauchy-Riemann conditions, they render themselves into degenerative backpropagation algorithms that compromise the efficiency of nonlinear approximation and learning in the complex vector field. A number of elementary transcendental functions (ETFs) derivable from the entire exponential functionez that are analytic are defined as fully complex activation functions and are shown to provide a parsimonious structure for processing data in the complex domain and address most of the shortcomings of the traditional approach. The introduction of ETFs, however, raises a new question in the approximation capability of this fully complex MLP. In this letter, three proofs of the approximation capability of the fully complex MLP are provided based on the characteristics of singularity among ETFs. First, the fully complex MLPs with continuous ETFs over a compact set in the complex vector field are shown to be the universal approximator of any continuous complex mappings. The complex universal approximation theorem extends to bounded measurable ETFs possessing a removable singularity. Finally, it is shown that the output of complex MLPs using ETFs with isolated and essential singularities uniformly converges to any nonlinear mapping in the deleted annulus of singularity nearest to the origin.
36
Forghanparast, Farhang, and Ghazal Mohammadi. "Using Deep Learning Algorithms for Intermittent Streamflow Prediction in the Headwaters of the Colorado River, Texas." Water 14, no. 19 (September 22, 2022): 2972. http://dx.doi.org/10.3390/w14192972.
Full textAbstract:
Predicting streamflow in intermittent rivers and ephemeral streams (IRES), particularly those in climate hotspots such as the headwaters of the Colorado River in Texas, is a necessity for all planning and management endeavors associated with these ubiquitous and valuable surface water resources. In this study, the performance of three deep learning algorithms, namely Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM), and Self-Attention LSTM models, were evaluated and compared against a baseline Extreme Learning Machine (ELM) model for monthly streamflow prediction in the headwaters of the Texas Colorado River. The predictive performance of the models was assessed over the entire range of flow as well as for capturing the extreme hydrologic events (no-flow events and extreme floods) using a suite of model evaluation metrics. According to the results, the deep learning algorithms, especially the LSTM-based models, outperformed the ELM with respect to all evaluation metrics and offered overall higher accuracy and better stability (more robustness against overfitting). Unlike its deep learning counterparts, the simpler ELM model struggled to capture important components of the IRES flow time-series and failed to offer accurate estimates of the hydrologic extremes. The LSTM model (K.G.E. > 0.7, R2 > 0.75, and r > 0.85), with better evaluation metrics than the ELM and CNN algorithm, and competitive performance to the SA–LSTM model, was identified as an appropriate, effective, and parsimonious streamflow prediction tool for the headwaters of the Colorado River in Texas.
37
Wolff, J. Gerard. "How the SP System May Promote Sustainability in Energy Consumption in IT Systems." Sustainability 13, no. 8 (April 20, 2021): 4565. http://dx.doi.org/10.3390/su13084565.
Full textAbstract:
The SP System (SPS), referring to the SP Theory of Intelligence and its realisation as the SP Computer Model, has the potential to reduce demands for energy from IT, especially in AI applications and in the processing of big data, in addition to reductions in CO2 emissions when the energy comes from the burning of fossil fuels. The biological foundations of the SPS suggest that with further development, the SPS may approach the extraordinarily low (20 W)energy demands of the human brain. Some of these savings may arise in the SPS because, like people, the SPS may learn usable knowledge from a single exposure or experience. As a comparison, deep neural networks (DNNs) need many repetitions, with much consumption of energy, for the learning of one concept. Another potential saving with the SPS is that like people, it can incorporate old learning in new. This contrasts with DNNs where new learning wipes out old learning (‘catastrophic forgetting’). Other ways in which the mature SPS is likely to prove relatively parsimonious in its demands for energy arise from the central role of information compression (IC) in the organisation and workings of the system: by making data smaller, there is less to process; because the efficiency of searching for matches between patterns can be improved by exploiting probabilities that arise from the intimate connection between IC and probabilities; and because, with SPS-derived ’Model-Based Codings’ of data, there can be substantial reductions in the demand for energy in transmitting data from one place to another.
38
Kaiser, E., J. N. Kutz, and S. L. Brunton. "Sparse identification of nonlinear dynamics for model predictive control in the low-data limit." Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 474, no. 2219 (November 2018): 20180335. http://dx.doi.org/10.1098/rspa.2018.0335.
Full textAbstract:
Data-driven discovery of dynamics via machine learning is pushing the frontiers of modelling and control efforts, providing a tremendous opportunity to extend the reach of model predictive control (MPC). However, many leading methods in machine learning, such as neural networks (NN), require large volumes of training data, may not be interpretable, do not easily include known constraints and symmetries, and may not generalize beyond the attractor where models are trained. These factors limit their use for the online identification of a model in the low-data limit, for example following an abrupt change to the system dynamics. In this work, we extend the recent sparse identification of nonlinear dynamics (SINDY) modelling procedure to include the effects of actuation and demonstrate the ability of these models to enhance the performance of MPC, based on limited, noisy data. SINDY models are parsimonious, identifying the fewest terms in the model needed to explain the data, making them interpretable and generalizable. We show that the resulting SINDY-MPC framework has higher performance, requires significantly less data, and is more computationally efficient and robust to noise than NN models, making it viable for online training and execution in response to rapid system changes. SINDY-MPC also shows improved performance over linear data-driven models, although linear models may provide a stopgap until enough data is available for SINDY. SINDY-MPC is demonstrated on a variety of dynamical systems with different challenges, including the chaotic Lorenz system, a simple model for flight control of an F8 aircraft, and an HIV model incorporating drug treatment.
39
Toth, E. "Catchment classification based on characterisation of streamflow and precipitation time series." Hydrology and Earth System Sciences 17, no. 3 (March 15, 2013): 1149–59. http://dx.doi.org/10.5194/hess-17-1149-2013.
Full textAbstract:
Abstract. The formulation of objective procedures for the delineation of homogeneous groups of catchments is a fundamental issue in both operational and research hydrology. For assessing catchment similarity, a variety of hydrological information may be considered; in this paper, gauged sites are characterised by a set of streamflow signatures that include a representation, albeit simplified, of the properties of fine time-scale flow series and in particular of the dynamic components of the data, in order to keep into account the sequential order and the stochastic nature of the streamflow process. The streamflow signatures are provided in input to a clustering algorithm based on unsupervised SOM neural networks, obtaining groups of catchments with a clear hydrological distinctiveness, as highlighted by the identification of the main patterns of the input variables in the different classes and the interpretation of their interrelations. In addition, even if no geographical, morphological nor climatological information is provided in input to the SOM network, the clusters exhibit an overall consistency as far as location, altitude and precipitation regime are concerned. In order to assign ungauged sites to such groups, the catchments are represented through a parsimonious set of morphometric and pluviometric variables, including also indexes that attempt to synthesise the variability and correlation properties of the precipitation time series, thus providing information on the type of weather forcing that is specific to each basin. Following a principal components analysis, needed for synthesizing and better understanding the morpho-pluviometric catchment properties, a discriminant analysis finally assigns the ungauged catchments, through a leave-one-out cross validation, to one of the above identified hydrologic response classes. The approach delivers a quite satisfactory identification of the membership of ungauged catchments to the streamflow-based classes, since the comparison of the two cluster sets shows a misclassification rate of around 20%. Overall results indicate that the inclusion of information on the properties of the fine time-scale streamflow and rainfall time series may be a promising way for better representing the hydrologic and climatic character of the study catchments.
40
Toth, E. "Catchment classification based on characterisation of streamflow and precipitation time-series." Hydrology and Earth System Sciences Discussions 9, no. 9 (September 26, 2012): 10805–28. http://dx.doi.org/10.5194/hessd-9-10805-2012.
Full textAbstract:
Abstract. The formulation of objective procedures for the delineation of homogeneous groups of catchments is a fundamental issue in both operational and research hydrology. For assessing catchment similarity, a variety of hydrological information may be considered; in this paper, gauged sites are characterised by a set of streamflow signatures that include a representation, albeit simplified, of the properties of fine time-scale flow series and in particular of the dynamic components of the data, in order to keep into account the sequential order and the stochastic nature of the streamflow process. The streamflow signatures are provided in input to a clustering algorithm based on unsupervised SOM neural networks, providing an overall reasonable grouping of catchments on the basis of their hydrological response. In order to assign ungauged sites to such groups, the catchments are represented through a parsimonious set of morphometric and pluviometric variables, including also indexes that attempt to synthesize the variability and correlation properties of the precipitation time-series, thus providing information on the type of weather forcing that is specific to each basin. Following a principal components analysis, needed for synthesizing and better understanding the morpho-pluviometric catchment properties, a discriminant analysis finally classifies the ungauged catchments, through a leave-one-out cross-validation, to one of the above identified hydrologic response classes. The approach delivers quite satisfactory results for ungauged catchments, since the comparison of the two cluster sets shows an acceptable overlap. Overall results indicate that the inclusion of information on the properties of the fine time-scale streamflow and rainfall time-series may be a promising way for better representing the hydrologic and climatic character of the study catchments.
41
Pratama, Mahardhika, Meng Joo Er, Xiang Li, Richard J. Oentaryo, Edwin Lughofer, and Imam Arifin. "Data driven modeling based on dynamic parsimonious fuzzy neural network." Neurocomputing 110 (June 2013): 18–28. http://dx.doi.org/10.1016/j.neucom.2012.11.013.
Full text
42
Ajeel, Nidhal Khaleel. "Predicting Future Ranked Statistics and Recorded Values for Some Statistical Distributions." Webology 18, Special Issue 04 (September 30, 2021): 364–84. http://dx.doi.org/10.14704/web/v18si04/web18135.
Full textAbstract:
Regional frequency analysis (AFR) brings together a variety of statistical methods aimed at predicting the behavior of extreme hydrological variables at ungauged sites. Regression techniques, geostatistical methods and classification are among the statistical tools frequently encountered in the literature. Methodologies based on these tools lead to regional models that offer a simple, but very useful description of the relationship between extreme hydrological variables and physiometeorological characteristics of a site. These regional models then make it possible to predict the behavior of variables of interest at places where no hydrological information is available. These methods are generally based on restrictive theoretical assumptions, including linearity and normality. These do not reflect the reality of natural phenomena. The general objectives of this paper are to identify the methods affected by these hypotheses, evaluate their impacts and propose improvements aimed at obtaining more realistic and fairer representations. Projection pursuit regression is a non-parametric method similar to generalized additive models and artificial neural networks that are considered in AFR to take into account the non-linearity of hydrological processes. In a comparative study, this paper shows that regression with revealing directions makes it possible to obtain more parsimonious models while preserving the same predictive power as the other nonparametric methods. Canonical Correlation Analysis (ACC) is used to create neighborhoods within which a model (e.g. multiple regression) is used to predict hydrologic variables at ungagged sites on the other hand, ACC strongly depends on the assumptions of normality and linearity. A new methodology for delineating neighborhoods is proposed in this paper and uses revealing direction regression to predict a reference point representing hydrological and physiometeorological information that is relevant to these groupings. The results show that the new methodology generalizes that of ACC, improves the homogeneity of neighborhoods and leads to better performance. In AFR, kriging techniques on transformed spaces are suggested in order to predict extreme hydrological variables. However, a transformation is required so that the hydrological variables of interest derive approximately from a multidimensional normal distribution. This transformation introduces a bias and leads to suboptimal predictions. Solutions have been proposed, but have not been tested in AFR. This paper proposes the approach of spatial copulas and shows that this approach provides satisfactory solutions to the problems encountered with kriging techniques. Max-stable processes are a theoretical formalization of spatial extremes and correspond to a more faithful representation of hydrological processes on the other hand; their characterization of extreme dependence poses technical problems which slow down their adoption. In this paper, the approximate Bayesian calculus is examined as a solution. The results of a simulation study show that the approximate Bayesian computation is superior to the standard approach of compound likelihood. In addition, this approach is more appropriate in order to take into account specification errors.
43
Winkielman, Piotr, and Andrzej Nowak. "Dynamics of cognition-emotion interface: Coherence breeds familiarity and liking, and does it fast." Behavioral and Brain Sciences 28, no. 2 (April 2005): 222–23. http://dx.doi.org/10.1017/s0140525x05510045.
Full textAbstract:
We present a dynamical model of interaction between recognition memory and affect, focusing on the phenomenon of “warm glow of familiarity.” In our model, both familiarity and affect reflect quick monitoring of coherence in an attractor neural network. This model parsimoniously explains a variety of empirical phenomena, including mere-exposure and beauty-in-averages effects, and the speed of familiarity and affect judgments.
44
Eslami, Payman, Kihyo Jung, Daewon Lee, and Amir Tjolleng. "Predicting tanker freight rates using parsimonious variables and a hybrid artificial neural network with an adaptive genetic algorithm." Maritime Economics & Logistics 19, no. 3 (August 2017): 538–50. http://dx.doi.org/10.1057/mel.2016.1.
Full text
45
Tran, Minh-Quan, Ahmed S. Zamzam, Phuong H. Nguyen, and Guus Pemen. "Multi-Area Distribution System State Estimation Using Decentralized Physics-Aware Neural Networks." Energies 14, no. 11 (May 24, 2021): 3025. http://dx.doi.org/10.3390/en14113025.
Full textAbstract:
The development of active distribution grids requires more accurate and lower computational cost state estimation. In this paper, the authors investigate a decentralized learning-based distribution system state estimation (DSSE) approach for large distribution grids. The proposed approach decomposes the feeder-level DSSE into subarea-level estimation problems that can be solved independently. The proposed method is decentralized pruned physics-aware neural network (D-P2N2). The physical grid topology is used to parsimoniously design the connections between different hidden layers of the D-P2N2. Monte Carlo simulations based on one-year of load consumption data collected from smart meters for a three-phase distribution system power flow are developed to generate the measurement and voltage state data. The IEEE 123-node system is selected as the test network to benchmark the proposed algorithm against the classic weighted least squares and state-of-the-art learning-based DSSE approaches. Numerical results show that the D-P2N2 outperforms the state-of-the-art methods in terms of estimation accuracy and computational efficiency.
46
Taylor, Jordan A., Laura L. Hieber, and Richard B. Ivry. "Feedback-dependent generalization." Journal of Neurophysiology 109, no. 1 (January 1, 2013): 202–15. http://dx.doi.org/10.1152/jn.00247.2012.
Full textAbstract:
Generalization provides a window into the representational changes that occur during motor learning. Neural network models have been integral in revealing how the neural representation constrains the extent of generalization. Specifically, two key features are thought to define the pattern of generalization. First, generalization is constrained by the properties of the underlying neural units; with directionally tuned units, the extent of generalization is limited by the width of the tuning functions. Second, error signals are used to update a sensorimotor map to align the desired and actual output, with a gradient-descent learning rule ensuring that the error produces changes in those units responsible for the error. In prior studies, task-specific effects in generalization have been attributed to differences in neural tuning functions. Here we ask whether differences in generalization functions may arise from task-specific error signals. We systematically varied visual error information in a visuomotor adaptation task and found that this manipulation led to qualitative differences in generalization. A neural network model suggests that these differences are the result of error feedback processing operating on a homogeneous and invariant set of tuning functions. Consistent with novel predictions derived from the model, increasing the number of training directions led to specific distortions of the generalization function. Taken together, the behavioral and modeling results offer a parsimonious account of generalization that is based on the utilization of feedback information to update a sensorimotor map with stable tuning functions.
47
Jędrzejewski, Arkadiusz, Grzegorz Marcjasz, and Rafał Weron. "Importance of the Long-Term Seasonal Component in Day-Ahead Electricity Price Forecasting Revisited: Parameter-Rich Models Estimated via the LASSO." Energies 14, no. 11 (June 2, 2021): 3249. http://dx.doi.org/10.3390/en14113249.
Full textAbstract:
Recent studies suggest that decomposing a series of electricity spot prices into a trend-seasonal and a stochastic component, modeling them independently, and then combining their forecasts can yield more accurate predictions than an approach in which the same parsimonious regression or neural network-based model is calibrated to the prices themselves. Here, we show that significant accuracy gains can also be achieved in the case of parameter-rich models estimated via the least absolute shrinkage and selection operator (LASSO). Moreover, we provide insights as to the order of applying seasonal decomposition and variance stabilizing transformations before model calibration, and propose two well-performing forecast averaging schemes that are based on different approaches for modeling the long-term seasonal component.
48
Xiong, Lihua, Kieran M. O’Connor, and Shenglian Guo. "Comparison of three updating schemes using artificial neural network in flow forecasting." Hydrology and Earth System Sciences 8, no. 2 (April 30, 2004): 247–55. http://dx.doi.org/10.5194/hess-8-247-2004.
Full textAbstract:
Abstract. Three updating schemes using artificial neural network (ANN) in flow forecasting are compared in terms of model efficiency. The first is the ANN model in the simulation mode plus an autoregressive (AR) model. For the ANN model in the simulation model, the input includes the observed rainfall and the previously estimated discharges, while the AR model is used to forecast the flow simulation errors of the ANN model. The second one is the ANN model in the updating mode, i.e. the ANN model uses the observed discharge directly together with the observed rainfall as the input. In this scheme, the weights of the ANN model are obtained by optimisation and then kept fixed in the procedure of flow forecasting. The third one is also the ANN model in the updating mode; however, the weights of the ANN model are no longer fixed but updated at each time step by the backpropagation method using the latest forecast error of the ANN model. These three updating schemes are tested for flow forecasting on ten catchments and it is found that the third updating scheme is more effective than the other two in terms of their efficiency in flow forecasting. Moreover, compared to the first updating scheme, the third scheme is more parsimonious in terms of the number of parameters, since the latter does not need any additional correction model. In conclusion, this paper recommends the ANN model with the backpropagation method, which updates the weights of ANN at each time step according to the latest forecast error, for use in real-time flow forecasting. Keywords: artificial neural network (ANN), updating, flow forecasting, backpropagation method
49
Mathonsi, Thabang, and Terence L. van Zyl. "A Statistics and Deep Learning Hybrid Method for Multivariate Time Series Forecasting and Mortality Modeling." Forecasting 4, no. 1 (December 22, 2021): 1–25. http://dx.doi.org/10.3390/forecast4010001.
Full textAbstract:
Hybrid methods have been shown to outperform pure statistical and pure deep learning methods at forecasting tasks and quantifying the associated uncertainty with those forecasts (prediction intervals). One example is Exponential Smoothing Recurrent Neural Network (ES-RNN), a hybrid between a statistical forecasting model and a recurrent neural network variant. ES-RNN achieves a 9.4% improvement in absolute error in the Makridakis-4 Forecasting Competition. This improvement and similar outperformance from other hybrid models have primarily been demonstrated only on univariate datasets. Difficulties with applying hybrid forecast methods to multivariate data include (i) the high computational cost involved in hyperparameter tuning for models that are not parsimonious, (ii) challenges associated with auto-correlation inherent in the data, as well as (iii) complex dependency (cross-correlation) between the covariates that may be hard to capture. This paper presents Multivariate Exponential Smoothing Long Short Term Memory (MES-LSTM), a generalized multivariate extension to ES-RNN, that overcomes these challenges. MES-LSTM utilizes a vectorized implementation. We test MES-LSTM on several aggregated coronavirus disease of 2019 (COVID-19) morbidity datasets and find our hybrid approach shows consistent, significant improvement over pure statistical and deep learning methods at forecast accuracy and prediction interval construction.
50
Yland, Jennifer J., Taiyao Wang, Zahra Zad, Sydney K. Willis, Tanran R. Wang, Amelia K. Wesselink, Tammy Jiang, Elizabeth E. Hatch, Lauren A. Wise, and Ioannis Ch Paschalidis. "Predictive models of pregnancy based on data from a preconception cohort study." Human Reproduction 37, no. 3 (January 13, 2022): 565–76. http://dx.doi.org/10.1093/humrep/deab280.
Full textAbstract:
Abstract STUDY QUESTION Can we derive adequate models to predict the probability of conception among couples actively trying to conceive? SUMMARY ANSWER Leveraging data collected from female participants in a North American preconception cohort study, we developed models to predict pregnancy with performance of ∼70% in the area under the receiver operating characteristic curve (AUC). WHAT IS KNOWN ALREADY Earlier work has focused primarily on identifying individual risk factors for infertility. Several predictive models have been developed in subfertile populations, with relatively low discrimination (AUC: 59–64%). STUDY DESIGN, SIZE, DURATION Study participants were female, aged 21–45 years, residents of the USA or Canada, not using fertility treatment, and actively trying to conceive at enrollment (2013–2019). Participants completed a baseline questionnaire at enrollment and follow-up questionnaires every 2 months for up to 12 months or until conception. We used data from 4133 participants with no more than one menstrual cycle of pregnancy attempt at study entry. PARTICIPANTS/MATERIALS, SETTING, METHODS On the baseline questionnaire, participants reported data on sociodemographic factors, lifestyle and behavioral factors, diet quality, medical history and selected male partner characteristics. A total of 163 predictors were considered in this study. We implemented regularized logistic regression, support vector machines, neural networks and gradient boosted decision trees to derive models predicting the probability of pregnancy: (i) within fewer than 12 menstrual cycles of pregnancy attempt time (Model I), and (ii) within 6 menstrual cycles of pregnancy attempt time (Model II). Cox models were used to predict the probability of pregnancy within each menstrual cycle for up to 12 cycles of follow-up (Model III). We assessed model performance using the AUC and the weighted-F1 score for Models I and II, and the concordance index for Model III. MAIN RESULTS AND THE ROLE OF CHANCE Model I and II AUCs were 70% and 66%, respectively, in parsimonious models, and the concordance index for Model III was 63%. The predictors that were positively associated with pregnancy in all models were: having previously breastfed an infant and using multivitamins or folic acid supplements. The predictors that were inversely associated with pregnancy in all models were: female age, female BMI and history of infertility. Among nulligravid women with no history of infertility, the most important predictors were: female age, female BMI, male BMI, use of a fertility app, attempt time at study entry and perceived stress. LIMITATIONS, REASONS FOR CAUTION Reliance on self-reported predictor data could have introduced misclassification, which would likely be non-differential with respect to the pregnancy outcome given the prospective design. In addition, we cannot be certain that all relevant predictor variables were considered. Finally, though we validated the models using split-sample replication techniques, we did not conduct an external validation study. WIDER IMPLICATIONS OF THE FINDINGS Given a wide range of predictor data, machine learning algorithms can be leveraged to analyze epidemiologic data and predict the probability of conception with discrimination that exceeds earlier work. STUDY FUNDING/COMPETING INTEREST(S) The research was partially supported by the U.S. National Science Foundation (under grants DMS-1664644, CNS-1645681 and IIS-1914792) and the National Institutes for Health (under grants R01 GM135930 and UL54 TR004130). In the last 3 years, L.A.W. has received in-kind donations for primary data collection in PRESTO from FertilityFriend.com, Kindara.com, Sandstone Diagnostics and Swiss Precision Diagnostics. L.A.W. also serves as a fibroid consultant to AbbVie, Inc. The other authors declare no competing interests. TRIAL REGISTRATION NUMBER N/A.