Bibliografie tematiche / Approximate identity neural networks

Indice

Articoli di riviste
Tesi
Libri
Capitoli di libri
Atti di convegni

Letteratura scientifica selezionata sul tema "Approximate identity neural networks"

Autore: Grafiati

Pubblicato: 4 giugno 2021

Ultima modifica: 1 febbraio 2022

Cita una fonte nei formati APA, MLA, Chicago, Harvard e in molti altri stili

Scegli il tipo di fonte:

Consulta la lista di attuali articoli, libri, tesi, atti di convegni e altre fonti scientifiche attinenti al tema "Approximate identity neural networks".

Accanto a ogni fonte nell'elenco di riferimenti c'è un pulsante "Aggiungi alla bibliografia". Premilo e genereremo automaticamente la citazione bibliografica dell'opera scelta nello stile citazionale di cui hai bisogno: APA, MLA, Harvard, Chicago, Vancouver ecc.

Puoi anche scaricare il testo completo della pubblicazione scientifica nel formato .pdf e leggere online l'abstract (il sommario) dell'opera se è presente nei metadati.

Articoli di riviste sul tema "Approximate identity neural networks"

Moon, Sunghwan. "ReLU Network with Bounded Width Is a Universal Approximator in View of an Approximate Identity." Applied Sciences 11, no. 1 (January 4, 2021): 427. http://dx.doi.org/10.3390/app11010427.

Testo completo

Abstract (sommario):

Deep neural networks have shown very successful performance in a wide range of tasks, but a theory of why they work so well is in the early stage. Recently, the expressive power of neural networks, important for understanding deep learning, has received considerable attention. Classic results, provided by Cybenko, Barron, etc., state that a network with a single hidden layer and suitable activation functions is a universal approximator. A few years ago, one started to study how width affects the expressiveness of neural networks, i.e., a universal approximation theorem for a deep neural network with a Rectified Linear Unit (ReLU) activation function and bounded width. Here, we show how any continuous function on a compact set of Rnin,nin∈N can be approximated by a ReLU network having hidden layers with at most nin+5 nodes in view of an approximate identity.

Gli stili APA, Harvard, Vancouver, ISO e altri

Funahashi, Ken-Ichi. "Approximate realization of identity mappings by three-layer neural networks." Electronics and Communications in Japan (Part III: Fundamental Electronic Science) 73, no. 11 (1990): 61–68. http://dx.doi.org/10.1002/ecjc.4430731107.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Zainuddin, Zarita, and Saeed Panahian Fard. "The Universal Approximation Capabilities of Cylindrical Approximate Identity Neural Networks." Arabian Journal for Science and Engineering 41, no. 8 (March 4, 2016): 3027–34. http://dx.doi.org/10.1007/s13369-016-2067-9.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Turchetti, C., M. Conti, P. Crippa, and S. Orcioni. "On the approximation of stochastic processes by approximate identity neural networks." IEEE Transactions on Neural Networks 9, no. 6 (1998): 1069–85. http://dx.doi.org/10.1109/72.728353.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Conti, M., and C. Turchetti. "Approximate identity neural networks for analog synthesis of nonlinear dynamical systems." IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications 41, no. 12 (1994): 841–58. http://dx.doi.org/10.1109/81.340846.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Fard, Saeed Panahian, and Zarita Zainuddin. "Almost everywhere approximation capabilities of double Mellin approximate identity neural networks." Soft Computing 20, no. 11 (July 2, 2015): 4439–47. http://dx.doi.org/10.1007/s00500-015-1753-y.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Panahian Fard, Saeed, та Zarita Zainuddin. "The universal approximation capabilities of double 2 $$\pi $$ π -periodic approximate identity neural networks". Soft Computing 19, № 10 (6 вересня 2014): 2883–90. http://dx.doi.org/10.1007/s00500-014-1449-8.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Panahian Fard, Saeed, and Zarita Zainuddin. "Analyses for L p [a, b]-norm approximation capability of flexible approximate identity neural networks." Neural Computing and Applications 24, no. 1 (October 8, 2013): 45–50. http://dx.doi.org/10.1007/s00521-013-1493-9.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

DiMattina, Christopher, and Kechen Zhang. "How to Modify a Neural Network Gradually Without Changing Its Input-Output Functionality." Neural Computation 22, no. 1 (January 2010): 1–47. http://dx.doi.org/10.1162/neco.2009.05-08-781.

Testo completo

Abstract (sommario):

It is generally unknown when distinct neural networks having different synaptic weights and thresholds implement identical input-output transformations. Determining the exact conditions for structurally distinct yet functionally equivalent networks may shed light on the theoretical constraints on how diverse neural circuits might develop and be maintained to serve identical functions. Such consideration also imposes practical limits on our ability to uniquely infer the structure of underlying neural circuits from stimulus-response measurements. We introduce a biologically inspired mathematical method for determining when the structure of a neural network can be perturbed gradually while preserving functionality. We show that for common three-layer networks with convergent and nondegenerate connection weights, this is possible only when the hidden unit gains are power functions, exponentials, or logarithmic functions, which are known to approximate the gains seen in some biological neurons. For practical applications, our numerical simulations with finite and noisy data show that continuous confounding of parameters due to network functional equivalence tends to occur approximately even when the gain function is not one of the aforementioned three types, suggesting that our analytical results are applicable to more general situations and may help identify a common source of parameter variability in neural network modeling.

Gli stili APA, Harvard, Vancouver, ISO e altri

Germani, S., G. Tosti, P. Lubrano, S. Cutini, I. Mereu, and A. Berretta. "Artificial Neural Network classification of 4FGL sources." Monthly Notices of the Royal Astronomical Society 505, no. 4 (June 24, 2021): 5853–61. http://dx.doi.org/10.1093/mnras/stab1748.

Testo completo

Abstract (sommario):

ABSTRACT The Fermi-LAT DR1 and DR2 4FGL catalogues feature more than 5000 gamma-ray sources of which about one fourth are not associated with already known objects, and approximately one third are associated with blazars of uncertain nature. We perform a three-category classification of the 4FGL DR1 and DR2 sources independently, using an ensemble of Artificial Neural Networks (ANNs) to characterize them based on the likelihood of being a Pulsar (PSR), a BL Lac type blazar (BLL) or a Flat Spectrum Radio Quasar (FSRQ). We identify candidate PSR, BLL, and FSRQ among the unassociated sources with approximate equipartition among the three categories and select 10 classification outliers as potentially interesting for follow-up studies.

Gli stili APA, Harvard, Vancouver, ISO e altri

Più fonti

Tesi sul tema "Approximate identity neural networks"

Ling, Hong. "Implementation of Stochastic Neural Networks for Approximating Random Processes." Master's thesis, Lincoln University. Environment, Society and Design Division, 2007. http://theses.lincoln.ac.nz/public/adt-NZLIU20080108.124352/.

Testo completo

Abstract (sommario):

Artificial Neural Networks (ANNs) can be viewed as a mathematical model to simulate natural and biological systems on the basis of mimicking the information processing methods in the human brain. The capability of current ANNs only focuses on approximating arbitrary deterministic input-output mappings. However, these ANNs do not adequately represent the variability which is observed in the systems natural settings as well as capture the complexity of the whole system behaviour. This thesis addresses the development of a new class of neural networks called Stochastic Neural Networks (SNNs) in order to simulate internal stochastic properties of systems. Developing a suitable mathematical model for SNNs is based on canonical representation of stochastic processes or systems by means of Karhunen-Loève Theorem. Some successful real examples, such as analysis of full displacement field of wood in compression, confirm the validity of the proposed neural networks. Furthermore, analysis of internal workings of SNNs provides an in-depth view on the operation of SNNs that help to gain a better understanding of the simulation of stochastic processes by SNNs.

Gli stili APA, Harvard, Vancouver, ISO e altri

Garces, Freddy. "Dynamic neural networks for approximate input- output linearisation-decoupling of dynamic systems." Thesis, University of Reading, 2000. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.368662.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Li, Yingzhen. "Approximate inference : new visions." Thesis, University of Cambridge, 2018. https://www.repository.cam.ac.uk/handle/1810/277549.

Testo completo

Abstract (sommario):

Nowadays machine learning (especially deep learning) techniques are being incorporated to many intelligent systems affecting the quality of human life. The ultimate purpose of these systems is to perform automated decision making, and in order to achieve this, predictive systems need to return estimates of their confidence. Powered by the rules of probability, Bayesian inference is the gold standard method to perform coherent reasoning under uncertainty. It is generally believed that intelligent systems following the Bayesian approach can better incorporate uncertainty information for reliable decision making, and be less vulnerable to attacks such as data poisoning. Critically, the success of Bayesian methods in practice, including the recent resurgence of Bayesian deep learning, relies on fast and accurate approximate Bayesian inference applied to probabilistic models. These approximate inference methods perform (approximate) Bayesian reasoning at a relatively low cost in terms of time and memory, thus allowing the principles of Bayesian modelling to be applied to many practical settings. However, more work needs to be done to scale approximate Bayesian inference methods to big systems such as deep neural networks and large-scale dataset such as ImageNet. In this thesis we develop new algorithms towards addressing the open challenges in approximate inference. In the first part of the thesis we develop two new approximate inference algorithms, by drawing inspiration from the well known expectation propagation and message passing algorithms. Both approaches provide a unifying view of existing variational methods from different algorithmic perspectives. We also demonstrate that they lead to better calibrated inference results for complex models such as neural network classifiers and deep generative models, and scale to large datasets containing hundreds of thousands of data-points. In the second theme of the thesis we propose a new research direction for approximate inference: developing algorithms for fitting posterior approximations of arbitrary form, by rethinking the fundamental principles of Bayesian computation and the necessity of algorithmic constraints in traditional inference schemes. We specify four algorithmic options for the development of such new generation approximate inference methods, with one of them further investigated and applied to Bayesian deep learning tasks.

Gli stili APA, Harvard, Vancouver, ISO e altri

Liu, Leo M. Eng Massachusetts Institute of Technology. "Acoustic models for speech recognition using Deep Neural Networks based on approximate math." Thesis, Massachusetts Institute of Technology, 2015. http://hdl.handle.net/1721.1/100633.

Testo completo

Abstract (sommario):

Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2015. This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections. Cataloged from student-submitted PDF version of thesis. Includes bibliographical references (pages 81-83). Deep Neural Networks (DNNs) are eective models for machine learning. Unfortunately, training a DNN is extremely time-consuming, even with the aid of a graphics processing unit (GPU). DNN training is especially slow for tasks with large datasets. Existing approaches for speeding up the process involve parallelizing the Stochastic Gradient Descent (SGD) algorithm used to train DNNs. Those approaches do not guarantee the same results as normal SGD since they introduce non-trivial changes into the algorithm. A new approach for faster training that avoids signicant changes to SGD is to use low-precision hardware. The low-precision hardware is faster than a GPU, but it performs arithmetic with 1% error. In this arithmetic, 98 + 2 = 99:776 and 10 * 10 = 100:863. This thesis determines whether DNNs would still be able to produce state-of-the-art results using this low-precision arithmetic. To answer this question, we implement an approximate DNN that uses the low-precision arithmetic and evaluate it on the TIMIT phoneme recognition task and the WSJ speech recognition task. For both tasks, we nd that acoustic models based on approximate DNNs perform as well as ones based on conventional DNNs; both produce similar recognition error rates. The approximate DNN is able to match the conventional DNN only if it uses Kahan summations to preserve precision. These results show that DNNs can run on low-precision hardware without the arithmetic causing any loss in recognition ability. The low-precision hardware is therefore a suitable approach for speeding up DNN training. by Leo Liu. M. Eng.

Gli stili APA, Harvard, Vancouver, ISO e altri

Scotti, Andrea. "Graph Neural Networks and Learned Approximate Message Passing Algorithms for Massive MIMO Detection." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-284500.

Testo completo

Abstract (sommario):

Massive multiple-input and multiple-output (MIMO) is a method to improvethe performance of wireless communication systems by having a large numberof antennas at both the transmitter and the receiver. In the fifth-generation(5G) mobile communication system, Massive MIMO is a key technology toface the increasing number of mobile users and satisfy user demands. At thesame time, recovering the transmitted information in a massive MIMO uplinkreceiver requires more computational complexity when the number of transmittersincreases. Indeed, the optimal maximum likelihood (ML) detector hasa complexity exponentially increasing with the number of transmitters. Therefore,one of the main challenges in the field is to find the best sub-optimalMIMO detection algorithm according to the performance/complexity tradeoff.In this work, all the algorithms are empirically evaluated for large MIMOsystems and higher-order modulations.Firstly, we show how MIMO detection can be represented by a MarkovRandom Field (MRF) and addressed by the loopy belief propagation (LBP)algorithm to approximately solve the equivalent MAP (maximum a posteriori)inference problem. Then, we propose a novel algorithm (BP-MMSE) thatstarts from the minimum mean square error (MMSE) solution and updates theprior in each iteration with the LBP belief. To avoid the complexity of computingMMSE, we use Graph Neural Networks (GNNs) to learn a messagepassingalgorithm that solves the inference task on the same graph.To further reduce the complexity of message-passing algorithms, we recallhow in the large system limit, approximate message passing (AMP), a lowcomplexity iterative algorithm, can be derived from LBP to solve MIMO detectionfor i.i.d. Gaussian channels. Then, we show numerically how AMPwith damping (DAMP) can be robust to low/medium correlation among thechannels. To conclude, we propose a low complexity deep neural iterativescheme (Pseudo-MMNet) for solvingMIMOdetection in the presence of highlycorrelated channels at the expense of online training for each channel realization.Pseudo-MMNet is based on MMNet algorithm presented in [24] (in turnbased on AMP) and it significantly reduces the online training complexity thatmakes MMNet far from realistic implementations. Massiv MIMO (multiple-input and multiple-output) är en metod som förbättrarprestandan i trådlösa kommunikationssystem genom att ett stort antal antenneranvänds i både sändare och mottagare. I den femte generationens (5G)mobila kommunikationssystem är Massiv MIMO en mycket viktig teknologiför att möta det växande antalet mobilanvändare och tillgodose användarnasbehov. Samtidigt ökar beräkningskomplexiteten för att återfinna den överfördainformationen i en trådlös Massiv MIMO-upplänk när antalet antenner ökar.Faktum är att den optimala ML-detektorn (maximum likelihood) har en beräkningskomplexitetsom ökar exponentiellt med antalet sändare. En av huvudutmaningarnainom detta område är därför att hitta den bästa suboptimalaMIMO-detekteringsalgoritmen med hänsyn till både prestanda och komplexitet.I detta arbete visar vi hur MIMO-detektering kan representeras av ett MarkovRandom Field (MRF) och använder loopy belief-fortplantning (LBP) föratt lösa det motsvarande MAP-slutledningsproblemet (maximum a posteriori).Vi föreslår sedan en ny algoritm (BP-MMSE) som kombinerar LBP ochMMSE (minimum mean square error) för att lösa problemet vid högre modulationsordningarsom QAM-16 (kvadratamplitudsmodulation) och QAM-64.För att undvika komplexiteten med att beräkna MMSE så använder vi oss avgraf neurala nätverk (GNN) för att lära en message-passing algoritm som löserslutledningsproblemet med samma graf. En message-passing algoritm måstegiven en komplett graf utbyta kvadraten av antalet noder meddelanden. För attminska message-passing algoritmers beräkningskomplexitet vet vi att approximativmessage-passing (AMP) kan härledas från LBP i gränsvärdet av storasystem för att lösa MIMO-detektering med oberoende och likafördelade (i.i.d)Gaussiska kanaler. Vi visar sedan hur AMP med dämpning (DAMP) kan vararobust med låg- till mellan-korrelerade kanaler.Avslutningsvis föreslår vi en iterativ djup neuralt nätverk algoritm medlåg beräkningskomplexitet (Pseudo-MMNet) för att lösa MIMO-detektering ikanaler med hög korrelation på bekostnad av online-träning för varje realiseringav kanalen. Pseudo-MMNet är baserad på MMnet som presenteras i [23](istället för AMP) och minskar signifikant online-träningskomplexiteten somgör MMNet orealistisk att använda. Alla föreslagna algoritmer är empirisktutvärderade för stora MIMO-system och högre ordningar av modulation.

Gli stili APA, Harvard, Vancouver, ISO e altri

Gaur, Yamini. "Exploring Per-Input Filter Selection and Approximation Techniques for Deep Neural Networks." Thesis, Virginia Tech, 2019. http://hdl.handle.net/10919/90404.

Testo completo

Abstract (sommario):

We propose a dynamic, input dependent filter approximation and selection technique to improve the computational efficiency of Deep Neural Networks. The approximation techniques convert 32 bit floating point representation of filter weights in neural networks into smaller precision values. This is done by reducing the number of bits used to represent the weights. In order to calculate the per-input error between the trained full precision filter weights and the approximated weights, a metric called Multiplication Error (ME) has been chosen. For convolutional layers, ME is calculated by subtracting the approximated filter weights from the original filter weights, convolving the difference with the input and calculating the grand-sum of the resulting matrix. For fully connected layers, ME is calculated by subtracting the approximated filter weights from the original filter weights, performing matrix multiplication between the difference and the input and calculating the grand-sum of the resulting matrix. ME is computed to identify approximated filters in a layer that result in low inference accuracy. In order to maintain the accuracy of the network, these filters weights are replaced with the original full precision weights. Prior work has primarily focused on input independent (static) replacement of filters to low precision weights. In this technique, all the filter weights in the network are replaced by approximated filter weights. This results in a decrease in inference accuracy. The decrease in accuracy is higher for more aggressive approximation techniques. Our proposed technique aims to achieve higher inference accuracy by not approximating filters that generate high ME. Using the proposed per-input filter selection technique, LeNet achieves an accuracy of 95.6% with 3.34% drop from the original accuracy value of 98.9% for truncating to 3 bits for the MNIST dataset. On the other hand upon static filter approximation, LeNet achieves an accuracy of 90.5% with 8.5% drop from the original accuracy. The aim of our research is to potentially use low precision weights in deep learning algorithms to achieve high classification accuracy with less computational overhead. We explore various filter approximation techniques and implement a per-input filter selection and approximation technique that selects the filters to approximate during run-time. Master of Science Deep neural networks, just like the human brain can learn important information about the data provided to them and can classify a new input based on the labels corresponding to the provided dataset. Deep learning technology is heavily employed in devices using computer vision, image and video processing and voice detection. The computational overhead incurred in the classification process of DNNs prohibits their use in smaller devices. This research aims to improve network efficiency in deep learning by replacing 32 bit weights in neural networks with less precision weights in an input-dependent manner. Trained neural networks are numerically robust. Different layers develop tolerance to minor variations in network parameters. Therefore, differences induced by low-precision calculations fall well within tolerance limit of the network. However, for aggressive approximation techniques like truncating to 3 and 2 bits, inference accuracy drops severely. We propose a dynamic technique that during run-time, identifies the approximated filters resulting in low inference accuracy for a given input and replaces those filters with the original filters to achieve high inference accuracy. The proposed technique has been tested for image classification on Convolutional Neural Networks. The datasets used are MNIST and CIFAR-10. The Convolutional Neural Networks used are 4-layered CNN, LeNet-5 and AlexNet.

Gli stili APA, Harvard, Vancouver, ISO e altri

Dumlupinar, Taha. "Approximate Analysis And Condition Assesment Of Reinforced Concrete T-beam Bridges Using Artificial Neural Networks." Master's thesis, METU, 2008. http://etd.lib.metu.edu.tr/upload/3/12609732/index.pdf.

Testo completo

Abstract (sommario):

In recent years, artificial neural networks (ANNs) have been employed for estimation and prediction purposes in many areas of civil/structural engineering. In this thesis, multilayered feedforward backpropagation algorithm is used for the approximate analysis and calibration of RC T-beam bridges and modeling of bridge ratings of these bridges. Currently bridges are analyzed using a standard FEM program. However, when a large population of bridges is concerned, such as the one considered in this project (Pennsylvania T-beam bridge population), it is impractical to carry out FEM analysis of all bridges in the population due to the fact that development and analysis of every single bridge requires considerable time as well as effort. Rapid and acceptably approximate analysis of bridges seems to be possible using ANN approach. First part of the study describes the application of neural network (NN) systems in developing the relationships between bridge parameters and bridge responses. The NN models are trained using some training data that are obtainedfrom finite-element analyses and that contain bridge parameters as inputs and critical responses as outputs. In the second part, ANN systems are used for the calibration of the finite element model of a typical RC T-beam bridge -the Manoa Road Bridge from the Pennsylvania&rsquo s T-beam bridge population - based on field test data. Manual calibration of these models are extremely time consuming and laborious. Therefore, a neural network- based method is developed for easy and practical calibration of these models. The ANN model is trained using some training data that are obtained from finite-element analyses and that contain modal and displacement parameters as inputs and structural parameters as outputs. After the training is completed, fieldmeasured data set is fed into the trained ANN model. Then, FE model is updated with the predicted structural parameters from the ANN model. In the final part, Neural Networks (NNs) are used to model the bridge ratings of RC T-beam bridges based on bridge parameters. Bridge load ratings are calculated more accurately by taking into account the actual geometry and detailing of the T-beam bridges. Then, ANN solution is developed to easily compute bridge load ratings.

Gli stili APA, Harvard, Vancouver, ISO e altri

Tornstad, Magnus. "Evaluating the Practicality of Using a Kronecker-Factored Approximate Curvature Matrix in Newton's Method for Optimization in Neural Networks." Thesis, KTH, Skolan för teknikvetenskap (SCI), 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-275741.

Testo completo

Abstract (sommario):

For a long time, second-order optimization methods have been regarded as computationally inefficient and intractable for solving the optimization problem associated with deep learning. However, proposed in recent research is an adaptation of Newton's method for optimization in which the Hessian is approximated by a Kronecker-factored approximate curvature matrix, known as KFAC. This work aims to assess its practicality for use in deep learning. Benchmarks were performed using abstract, binary, classification problems, as well as the real-world Boston Housing regression problem, and both deep and shallow network architectures were employed. KFAC was found to offer great savings in computational complexity compared to a naive approximate second-order implementation using the Gauss Newton matrix. Comparing performance in deep and shallow networks, the loss convergence of both stochastic gradient descent (SGD) and KFAC showed a dependency upon network architecture, where KFAC tended to converge quicker in deep networks, and SGD tended to converge quicker in shallow networks. The study concludes that KFAC can perform well in deep learning, showing competitive loss minimization versus basic SGD, but that it can be sensitive to initial weigths. This sensitivity could be remedied by allowing the first steps to be taken by SGD, in order to set KFAC on a favorable trajectory. Andra ordningens optimeringsmetoder have länge ansetts vara beräkningsmässigt ineffektiva för att lösa optimeringsproblemet inom djup maskininlärning. En alternativ optimiseringsstrategi som använder en Kronecker-faktoriserad approximativ Hessian (KFAC) i Newtons metod för optimering, har föreslagits i tidigare studier. Detta arbete syftar till att utvärdera huruvida metoden är praktisk att använda i djup maskininlärning. Test körs på abstrakta, binära, klassificeringsproblem, samt ett verkligt regressionsproblem: Boston Housing data. Studien fann att KFAC erbjuder stora besparingar i tidskopmlexitet jämfört med när en mer naiv implementation med Gauss-Newton matrisen används. Vidare visade sig losskonvergensen hos både stokastisk gradient descent (SGD) och KFAC beroende av nätverksarkitektur: KFAC tenderade att konvergera snabbare i djupa nätverk, medan SGD tenderade att konvergera snabbare i grunda nätverk. Studien drar slutsatsen att KFAC kan prestera väl för djup maskininlärning jämfört med en grundläggande variant av SGD. KFAC visade sig dock kunna vara mycket känslig för initialvikter. Detta problem kunde lösas genom att låta de första stegen tas av SGD så att KFAC hamnade på en gynnsam bana.

Gli stili APA, Harvard, Vancouver, ISO e altri

Hanselmann, Thomas. "Approximate dynamic programming with adaptive critics and the algebraic perceptron as a fast neural network related to support vector machines." University of Western Australia. School of Electrical, Electronic and Computer Engineering, 2003. http://theses.library.uwa.edu.au/adt-WU2004.0005.

Testo completo

Abstract (sommario):

[Truncated abstract. Please see the pdf version for the complete text. Also, formulae and special characters can only be approximated here. Please see the pdf version of this abstract for an accurate reproduction.] This thesis treats two aspects of intelligent control: The first part is about long-term optimization by approximating dynamic programming and in the second part a specific class of a fast neural network, related to support vector machines (SVMs), is considered. The first part relates to approximate dynamic programming, especially in the framework of adaptive critic designs (ACDs). Dynamic programming can be used to find an optimal decision or control policy over a long-term period. However, in practice it is difficult, and often impossible, to calculate a dynamic programming solution, due to the 'curse of dimensionality'. The adaptive critic design framework addresses this issue and tries to find a good solution by approximating the dynamic programming process for a stationary environment. In an adaptive critic design there are three modules, the plant or environment to be controlled, a critic to estimate the long-term cost and an action or controller module to produce the decision or control strategy. Even though there have been many publications on the subject over the past two decades, there are some points that have had less attention. While most of the publications address the training of the critic, one of the points that has not received systematic attention is training of the action module.¹ Normally, training starts with an arbitrary, hopefully stable, decision policy and its long-term cost is then estimated by the critic. Often the critic is a neural network that has to be trained, using a temporal difference and Bellman's principle of optimality. Once the critic network has converged, a policy improvement step is carried out by gradient descent to adjust the parameters of the controller network. Then the critic is retrained again to give the new long-term cost estimate. However, it would be preferable to focus more on extremal policies earlier in the training. Therefore, the Calculus of Variations is investigated to discard the idea of using the Euler equations to train the actor. However, an adaptive critic formulation for a continuous plant with a short-term cost as an integral cost density is made and the chain rule is applied to calculate the total derivative of the short-term cost with respect to the actor weights. This is different from the discrete systems, usually used in adaptive critics, which are used in conjunction with total ordered derivatives. This idea is then extended to second order derivatives such that Newton's method can be applied to speed up convergence. Based on this, an almost concurrent actor and critic training was proposed. The equations are developed for any non-linear system and short-term cost density function and these were tested on a linear quadratic regulator (LQR) setup. With this approach the solution to the actor and critic weights can be achieved in only a few actor-critic training cycles. Some other, more minor issues, in the adaptive critic framework are investigated, such as the influence of the discounting factor in the Bellman equation on total ordered derivatives, the target interpretation in backpropagation through time as moving and fixed targets, the relation between simultaneous recurrent networks and dynamic programming is stated and a reinterpretation of the recurrent generalized multilayer perceptron (GMLP) as a recurrent generalized finite impulse MLP (GFIR-MLP) is made. Another subject in this area that is investigated, is that of a hybrid dynamical system, characterized as a continuous plant and a set of basic feedback controllers, which are used to control the plant by finding a switching sequence to select one basic controller at a time. The special but important case is considered when the plant is linear but with some uncertainty in the state space and in the observation vector, and a quadratic cost function. This is a form of robust control, where a dynamic programming solution has to be calculated. &sup1Werbos comments that most treatment of action nets or policies either assume enumerative maximization, which is good only for small problems, except for the games of Backgammon or Go [1], or, gradient-based training. The latter is prone to difficulties with local minima due to the non-convex nature of the cost-to-go function. With incremental methods, such as backpropagation through time, calculus of variations and model-predictive control, the dangers of non-convexity of the cost-to-go function with respect to the control is much less than the with respect to the critic parameters, when the sampling times are small. Therefore, getting the critic right has priority. But with larger sampling times, when the control represents a more complex plan, non-convexity becomes more serious.

Gli stili APA, Harvard, Vancouver, ISO e altri

Malfatti, Guilherme Meneguzzi. "Técnicas de agrupamento de dados para computação aproximativa." reponame:Biblioteca Digital de Teses e Dissertações da UFRGS, 2017. http://hdl.handle.net/10183/169096.

Testo completo

Abstract (sommario):

Dois dos principais fatores do aumento da performance em aplicações single-thread – frequência de operação e exploração do paralelismo no nível das instruções – tiveram pouco avanço nos últimos anos devido a restrições de potência. Neste contexto, considerando a natureza tolerante a imprecisões (i.e.: suas saídas podem conter um nível aceitável de ruído sem comprometer o resultado final) de muitas aplicações atuais, como processamento de imagens e aprendizado de máquina, a computação aproximativa torna-se uma abordagem atrativa. Esta técnica baseia-se em computar valores aproximados ao invés de precisos que, por sua vez, pode aumentar o desempenho e reduzir o consumo energético ao custo de qualidade. No atual estado da arte, a forma mais comum de exploração da técnica é através de redes neurais (mais especificamente, o modelo Multilayer Perceptron), devido à capacidade destas estruturas de aprender funções arbitrárias e aproximá-las. Tais redes são geralmente implementadas em um hardware dedicado, chamado acelerador neural. Contudo, essa execução exige uma grande quantidade de área em chip e geralmente não oferece melhorias suficientes que justifiquem este espaço adicional. Este trabalho tem por objetivo propor um novo mecanismo para fazer computação aproximativa, baseado em reúso aproximativo de funções e trechos de código. Esta técnica agrupa automaticamente entradas e saídas de dados por similaridade, armazena-os em uma tabela em memória controlada via software. A partir disto, os valores quantizados podem ser reutilizados através de uma busca a essa tabela, onde será selecionada a saída mais apropriada e desta forma a execução do trecho de código será substituído. A aplicação desta técnica é bastante eficaz, sendo capaz de alcançar uma redução, em média, de 97.1% em Energy-Delay-Product (EDP) quando comparado a aceleradores neurais. Two of the major drivers of increased performance in single-thread applications - increase in operation frequency and exploitation of instruction-level parallelism - have had little advances in the last years due to power constraints. In this context, considering the intrinsic imprecision-tolerance (i.e., outputs may present an acceptable level of noise without compromising the result) of many modern applications, such as image processing and machine learning, approximate computation becomes a promising approach. This technique is based on computing approximate instead of accurate results, which can increase performance and reduce energy consumption at the cost of quality. In the current state of the art, the most common way of exploiting the technique is through neural networks (more specifically, the Multilayer Perceptron model), due to the ability of these structures to learn arbitrary functions and to approximate them. Such networks are usually implemented in a dedicated neural accelerator. However, this implementation requires a large amount of chip area and usually does not offer enough improvements to justify this additional cost. The goal of this work is to propose a new mechanism to address approximate computation, based on approximate reuse of functions and code fragments. This technique automatically groups input and output data by similarity and stores this information in a sofware-controlled memory. Based on these data, the quantized values can be reused through a search to this table, in which the most appropriate output will be selected and, therefore, execution of the original code will be replaced. Applying this technique is effective, achieving an average 97.1% reduction in Energy-Delay-Product (EDP) when compared to neural accelerators.

Gli stili APA, Harvard, Vancouver, ISO e altri

Più fonti

Libri sul tema "Approximate identity neural networks"

Snail, Mgebwi Lavin. The antecedens [sic] and the emergence of the black consciousness movement in South Africa: Its ideology and organisation. München: Akademischer Verlag, 1993.

Cerca il testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Butz, Martin V., and Esther F. Kutter. Brain Basics from a Computational Perspective. Oxford University Press, 2017. http://dx.doi.org/10.1093/acprof:oso/9780198739692.003.0007.

Testo completo

Abstract (sommario):

This chapter provides a crude overview of current knowledge in neuroscience about the human nervous system and its functionality. The distinction between the peripheral and central nervous systems is introduced. Next, brain anatomy is introduced, as well as nerve cells and the information processing principles that unfold in biological neural networks. Moreover, brain modules are covered, including their interconnected communication. With modularizations and wiring systematicities in mind, functional and structural systematicities are surveyed, including neural homunculi, cortical columnar structures, and the six-layered structure of the cerebral cortex. Finally, different available brain imaging techniques are contrasted. In conclusion, evidence is surveyed that suggests that the brain can be viewed as a highly modularized predictive processing system, which maintains internal activity and produces internal structures for the purpose of maintaining bodily needs in approximate homeostasis.

Gli stili APA, Harvard, Vancouver, ISO e altri

Bindemann, Markus, ed. Forensic Face Matching. Oxford University Press, 2021. http://dx.doi.org/10.1093/oso/9780198837749.001.0001.

Testo completo

Abstract (sommario):

Person identification at passport control, at borders, in police investigations, and in criminal trials relies critically on the identity verification of people via image-to-image or person-to-image comparison. While this task is known as ‘facial image comparison’ in forensic settings, it has been studied as ‘unfamiliar face matching’ in cognitive science. This book brings together expertise from practitioners, and academics in psychology and law, to draw together what is currently known about these tasks. It explains the problem of identity impostors and how within-person variability and between-person similarity, due to factors such as image quality, lighting direction, and view, affect identification. A framework to develop a cognitive theory of face matching is offered. The face-matching abilities of untrained lay observers, facial reviewers, facial examiners, and super-recognizers are analysed and contrasted. Individual differences between observers, learning and training for face recognition and face matching, and personnel selection are reviewed. The admissibility criteria of evidence from face matching in legal settings are considered, focusing on aspects such as the requirement of relevance, the prohibition on evidence of opinion, and reliability. Key concepts relevant to automatic face recognition algorithms at airports and in police investigations are explained, such as deep convolutional neural networks, biometrics, and human–computer interaction. Finally, new security threats in the form of hyper-realistic mask disguises are considered, including the impact these have on person identification in applied and laboratory settings.

Gli stili APA, Harvard, Vancouver, ISO e altri

Capitoli di libri sul tema "Approximate identity neural networks"

Fard, Saeed Panahian, and Zarita Zainuddin. "Toroidal Approximate Identity Neural Networks Are Universal Approximators." In Neural Information Processing, 135–42. Cham: Springer International Publishing, 2014. http://dx.doi.org/10.1007/978-3-319-12637-1_17.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Zainuddin, Zarita, and Saeed Panahian Fard. "Double Approximate Identity Neural Networks Universal Approximation in Real Lebesgue Spaces." In Neural Information Processing, 409–15. Berlin, Heidelberg: Springer Berlin Heidelberg, 2012. http://dx.doi.org/10.1007/978-3-642-34475-6_49.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Panahian Fard, Saeed, and Zarita Zainuddin. "The Universal Approximation Capabilities of Mellin Approximate Identity Neural Networks." In Advances in Neural Networks – ISNN 2013, 205–13. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013. http://dx.doi.org/10.1007/978-3-642-39065-4_26.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Panahian Fard, Saeed, and Zarita Zainuddin. "Universal Approximation by Generalized Mellin Approximate Identity Neural Networks." In Proceedings of the 4th International Conference on Computer Engineering and Networks, 187–94. Cham: Springer International Publishing, 2015. http://dx.doi.org/10.1007/978-3-319-11104-9_22.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Fard, Saeed Panahian, and Zarita Zainuddin. "The Universal Approximation Capability of Double Flexible Approximate Identity Neural Networks." In Lecture Notes in Electrical Engineering, 125–33. Cham: Springer International Publishing, 2013. http://dx.doi.org/10.1007/978-3-319-01766-2_15.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Panahian Fard, Saeed, and Zarita Zainuddin. "On the Universal Approximation Capability of Flexible Approximate Identity Neural Networks." In Emerging Technologies for Information Systems, Computing, and Management, 201–7. New York, NY: Springer New York, 2013. http://dx.doi.org/10.1007/978-1-4614-7010-6_23.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Hanif, Muhammad Abdullah, Muhammad Usama Javed, Rehan Hafiz, Semeen Rehman, and Muhammad Shafique. "Hardware–Software Approximations for Deep Neural Networks." In Approximate Circuits, 269–88. Cham: Springer International Publishing, 2018. http://dx.doi.org/10.1007/978-3-319-99322-5_13.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Choi, Jungwook, and Swagath Venkataramani. "Approximate Computing Techniques for Deep Neural Networks." In Approximate Circuits, 307–29. Cham: Springer International Publishing, 2018. http://dx.doi.org/10.1007/978-3-319-99322-5_15.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Ishibuchi, H., and H. Tanaka. "Approximate Pattern Classification Using Neural Networks." In Fuzzy Logic, 225–36. Dordrecht: Springer Netherlands, 1993. http://dx.doi.org/10.1007/978-94-011-2014-2_22.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Bai, Xuerui, Jianqiang Yi, and Dongbin Zhao. "Approximate Dynamic Programming for Ship Course Control." In Advances in Neural Networks – ISNN 2007, 349–57. Berlin, Heidelberg: Springer Berlin Heidelberg, 2007. http://dx.doi.org/10.1007/978-3-540-72383-7_41.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Atti di convegni sul tema "Approximate identity neural networks"

Zainuddin, Zarita, and Saeed Panahian Fard. "Spherical approximate identity neural networks are universal approximators." In 2014 10th International Conference on Natural Computation (ICNC). IEEE, 2014. http://dx.doi.org/10.1109/icnc.2014.6975812.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Fard Panahian, Saeed, and Zarita Zainuddin. "Universal Approximation Property of Weighted Approximate Identity Neural Networks." In The 5th International Conference on Computer Engineering and Networks. Trieste, Italy: Sissa Medialab, 2015. http://dx.doi.org/10.22323/1.259.0007.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Panahian Fard, Saeed, and Zarita Zainuddin. "The Universal Approximation Capabilities of 2pi-Periodic Approximate Identity Neural Networks." In 2013 International Conference on Information Science and Cloud Computing Companion (ISCC-C). IEEE, 2013. http://dx.doi.org/10.1109/iscc-c.2013.147.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Fard, Saeed Panahian. "Solving Universal Approximation Problem by Hankel Approximate Identity Neural Networks in Function Spaces." In The fourth International Conference on Information Science and Cloud Computing. Trieste, Italy: Sissa Medialab, 2016. http://dx.doi.org/10.22323/1.264.0031.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Zainuddin, Zarita, та Saeed Panahian Fard. "Approximation of multivariate 2π-periodic functions by multiple 2π-periodic approximate identity neural networks based on the universal approximation theorems". У 2015 11th International Conference on Natural Computation (ICNC). IEEE, 2015. http://dx.doi.org/10.1109/icnc.2015.7377957.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Ahmadian, M. T., and A. Mobini. "Online Prediction of Plate Deformations Under External Forces Using Neural Networks." In ASME 2006 International Mechanical Engineering Congress and Exposition. ASMEDC, 2006. http://dx.doi.org/10.1115/imece2006-15844.

Testo completo

Abstract (sommario):

Recently online prediction of plate deformations in modern systems have been considered by many researchers, common standard methods are highly time consuming and powerful processors are needed for online computation of deformations. Artificial neural networks have capability to develop complex, nonlinear functional relationships between input and output patterns based on limited data. A good trained network could predict output data very fast with acceptable accuracy. This paper describes the application of an artificial neural network to identify deformation pattern of a four-side clamped plate under external loads. In this paper the distributed loads are approximated by a set of concentrated loads. An artificial neural network is designed to predict plate deformation pattern under external forces. Results indicate a well trained artificial neural network reveals an extremely fast convergence and a high degree of accuracy in the process of predicting deformation pattern of plates. Additionally this paper represents application of neural network in inverse problem. This part illustrates the capability of neural networks in identification of plate external loads based on plate deformations. Load identification has many applications in identification of real loads in machineries for design and development.

Gli stili APA, Harvard, Vancouver, ISO e altri

Mao, X., V. Joshi, T. P. Miyanawala, and Rajeev K. Jaiman. "Data-Driven Computing With Convolutional Neural Networks for Two-Phase Flows: Application to Wave-Structure Interaction." In ASME 2018 37th International Conference on Ocean, Offshore and Arctic Engineering. American Society of Mechanical Engineers, 2018. http://dx.doi.org/10.1115/omae2018-78425.

Testo completo

Abstract (sommario):

Fluctuating wave force on a bluff body is of great significance in many offshore and marine engineering applications. We present a Convolutional Neural Network (CNN) based data-driven computing to predict the unsteady wave forces on bluff bodies due to the free-surface wave motion. For the full-order modeling and high-fidelity data generation, the air-water interface for such wave-body problems must be captured accurately for a broad range of physical and geometric parameters. Originated from the thermodynamically consistent theories, the physically motivated Allen-Cahn phase-field method has many advantages over other interface capturing techniques such as level-set and volume-of-fluid methods. The Allen-Cahn equation is solved in the mass-conservative form by imposing a Lagrange multiplier technique. While a tremendous amount of wave-body interaction data is generated in offshore engineering via both CFD simulations and experiments, the results are generally underutilized. Design space exploration and flow control of such practical scenarios are still time-consuming and expensive. An alternative to semi-analytical modeling, CNN is a class of deep neural network for solving inverse problems which is efficient in parametric data-driven computation and can use the domain knowledge. It establishes a model with arbitrarily generated model parameters, makes predictions using the model and existing input parametric settings, and adjusts the model parameters according to the error between the predictions and existing results. The computational cost of this prediction process, compared with high-fidelity CFD simulation, is significantly reduced, which makes CNN an accessible tool in design and optimization problems. In this study, CNN-based data-driven computing is utilized to predict the wave forces on bluff bodies with different geometries and distances to the free surface. The discrete convolution process with a non-linear rectification is employed to approximate the mapping between the bluff-body shape, the distance to the free-surface and the fluid forces. The wave-induced fluid forces on bluff bodies of different shapes and submergences are predicted by the trained CNN. Finally, a convergence study is performed to identify the effective hyper-parameters of the CNN such as the convolution kernel size, the number of kernels and the learning rate. Overall, the proposed CNN-based approximation procedure has a profound impact on the parametric design of bluff bodies experiencing wave loads.

Gli stili APA, Harvard, Vancouver, ISO e altri

Li, Longyuan, Junchi Yan, Xiaokang Yang, and Yaohui Jin. "Learning Interpretable Deep State Space Model for Probabilistic Time Series Forecasting." In Twenty-Eighth International Joint Conference on Artificial Intelligence {IJCAI-19}. California: International Joint Conferences on Artificial Intelligence Organization, 2019. http://dx.doi.org/10.24963/ijcai.2019/402.

Testo completo

Abstract (sommario):

Probabilistic time series forecasting involves estimating the distribution of future based on its history, which is essential for risk management in downstream decision-making. We propose a deep state space model for probabilistic time series forecasting whereby the non-linear emission model and transition model are parameterized by networks and the dependency is modeled by recurrent neural nets. We take the automatic relevance determination (ARD) view and devise a network to exploit the exogenous variables in addition to time series. In particular, our ARD network can incorporate the uncertainty of the exogenous variables and eventually helps identify useful exogenous variables and suppress those irrelevant for forecasting. The distribution of multi-step ahead forecasts are approximated by Monte Carlo simulation. We show in experiments that our model produces accurate and sharp probabilistic forecasts. The estimated uncertainty of our forecasting also realistically increases over time, in a spontaneous manner.

Gli stili APA, Harvard, Vancouver, ISO e altri

Sen, Sanchari, Swagath Venkataramani, and Anand Raghunathan. "Approximate computing for spiking neural networks." In 2017 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 2017. http://dx.doi.org/10.23919/date.2017.7926981.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Xu, Xiangrui, Yaqin Lee, Yunlong Gao, and Cao Yuan. "Adding identity numbers to deep neural networks." In Automatic Target Recognition and Navigation, edited by Hanyu Hong, Jianguo Liu, and Xia Hua. SPIE, 2020. http://dx.doi.org/10.1117/12.2540293.

Testo completo

Gli stili APA, Harvard, Vancouver, ISO e altri

Offriamo sconti su tutti i piani premium per gli autori le cui opere sono incluse in raccolte letterarie tematiche. Contattaci per ottenere un codice promozionale unico!