Journal articles: 'Neural Tensor Network'

1

Gao, Yuan, Laurence T. Yang, Dehua Zheng, Jing Yang, and Yaliang Zhao. "Quantized Tensor Neural Network." ACM/IMS Transactions on Data Science 2, no. 4 (November 30, 2021): 1–18. http://dx.doi.org/10.1145/3491255.

Full text

Abstract:

Tensor network as an effective computing framework for efficient processing and analysis of high-dimensional data has been successfully applied in many fields. However, the performance of traditional tensor networks still cannot match the strong fitting ability of neural networks, so some data processing algorithms based on tensor networks cannot achieve the same excellent performance as deep learning models. To further improve the learning ability of tensor network, we propose a quantized tensor neural network in this article (QTNN), which integrates the advantages of neural networks and tensor networks, namely, the powerful learning ability of neural networks and the simplicity of tensor networks. The QTNN model can be further regarded as a generalized multilayer nonlinear tensor network, which can efficiently extract low-dimensional features of the data while maintaining the original structure information. In addition, to more effectively represent the local information of data, we introduce multiple convolution layers in QTNN to extract the local features. We also develop a high-order back-propagation algorithm for training the parameters of QTNN. We conducted classification experiments on multiple representative datasets to further evaluate the performance of proposed models, and the experimental results show that QTNN is simpler and more efficient while compared to the classic deep learning models.

APA, Harvard, Vancouver, ISO, and other styles

2

MURTHY, GARIMELLA RAMA. "MULTI/INFINITE DIMENSIONAL NEURAL NETWORKS, MULTI/INFINITE DIMENSIONAL LOGIC THEORY." International Journal of Neural Systems 15, no. 03 (June 2005): 223–35. http://dx.doi.org/10.1142/s0129065705000190.

Full text

Abstract:

A mathematical model of an arbitrary multi-dimensional neural network is developed and a convergence theorem for an arbitrary multi-dimensional neural network represented by a fully symmetric tensor is stated and proved. The input and output signal states of a multi-dimensional neural network/logic gate are related through an energy function, defined over the fully symmetric tensor (representing the connection structure of a multi-dimensional neural network). The inputs and outputs are related such that the minimum/maximum energy states correspond to the output states of the logic gate/neural network realizing a logic function. Similarly, a logic circuit consisting of the interconnection of logic gates, represented by a block symmetric tensor, is associated with a quadratic/higher degree energy function. Infinite dimensional logic theory is discussed through the utilization of infinite dimension/order tensors.

APA, Harvard, Vancouver, ISO, and other styles

3

Feng, Yu, Xianfeng Xu, and Yun Meng. "Short-Term Load Forecasting with Tensor Partial Least Squares-Neural Network." Energies 12, no. 6 (March 14, 2019): 990. http://dx.doi.org/10.3390/en12060990.

Full text

Abstract:

Short-term load forecasting is very important for power systems. The load is related to many factors which compose tensors. However, tensors cannot be input directly into most traditional forecasting models. This paper proposes a tensor partial least squares-neural network model (TPN) to forecast the power load. The model contains a tensor decomposition outer model and a nonlinear inner model. The outer model extracts common latent variables of tensor input and vector output and makes the residuals less than the threshold by iteration. The inner model determines the relationship between the latent variable matrix and the output by using a neural network. This model structure can preserve the information of tensors and the nonlinear features of the system. Three classical models, partial least squares (PLS), least squares support vector machine (LSSVM) and neural network (NN), are selected to compare the forecasting results. The results show that the proposed model is efficient for short-term load and daily load peak forecasting. Compared to PLS, LSSVM and NN, the TPN has the best forecasting accuracy.

APA, Harvard, Vancouver, ISO, and other styles

4

Xu, Zenglin. "Tensor Networks Meet Neural Networks." Journal of Physics: Conference Series 2278, no. 1 (May 1, 2022): 012003. http://dx.doi.org/10.1088/1742-6596/2278/1/012003.

Full text

Abstract:

Abstract As a simulation of the human cognitive system, deep neural networks have achieved great success in many machine learning tasks and are the main driving force of the current development of artificial intelligence. On the other hand, tensor networks as an approximation of quantum many-body systems in quantum physics are applied to quantum physics, statistical physics, quantum chemistry and machine learning. This talk will first give a brief introduction to neural networks and tensor networks, and then discuss the cross-field research between deep neural networks and tensor networks, such as network compression and knowledge fusion, including our recent work on tensor neural networks. Finally, this talk will also discuss the connection to quantum machine learning.

APA, Harvard, Vancouver, ISO, and other styles

5

Sobolev, Konstantin, Dmitry Ermilov, Anh-Huy Phan, and Andrzej Cichocki. "PARS: Proxy-Based Automatic Rank Selection for Neural Network Compression via Low-Rank Weight Approximation." Mathematics 10, no. 20 (October 14, 2022): 3801. http://dx.doi.org/10.3390/math10203801.

Full text

Abstract:

Low-rank matrix/tensor decompositions are promising methods for reducing the inference time, computation, and memory consumption of deep neural networks (DNNs). This group of methods decomposes the pre-trained neural network weights through low-rank matrix/tensor decomposition and replaces the original layers with lightweight factorized layers. A main drawback of the technique is that it demands a great amount of time and effort to select the best ranks of tensor decomposition for each layer in a DNN. This paper proposes a Proxy-based Automatic tensor Rank Selection method (PARS) that utilizes a Bayesian optimization approach to find the best combination of ranks for neural network (NN) compression. We observe that the decomposition of weight tensors adversely influences the feature distribution inside the neural network and impairs the predictability of the post-compression DNN performance. Based on this finding, a novel proxy metric is proposed to deal with the abovementioned issue and to increase the quality of the rank search procedure. Experimental results show that PARS improves the results of existing decomposition methods on several representative NNs, including ResNet-18, ResNet-56, VGG-16, and AlexNet. We obtain a 3× FLOP reduction with almost no loss of accuracy for ILSVRC-2012ResNet-18 and a 5.5× FLOP reduction with an accuracy improvement for ILSVRC-2012 VGG-16.

APA, Harvard, Vancouver, ISO, and other styles

6

Wang, Xuezhong, Maolin Che, and Yimin Wei. "Tensor neural network models for tensor singular value decompositions." Computational Optimization and Applications 75, no. 3 (January 20, 2020): 753–77. http://dx.doi.org/10.1007/s10589-020-00167-1.

Full text

APA, Harvard, Vancouver, ISO, and other styles

7

Zhan, Tianming, Bo Song, Yang Xu, Minghua Wan, Xin Wang, Guowei Yang, and Zebin Wu. "SSCNN-S: A Spectral-Spatial Convolution Neural Network with Siamese Architecture for Change Detection." Remote Sensing 13, no. 5 (February 27, 2021): 895. http://dx.doi.org/10.3390/rs13050895.

Full text

Abstract:

In this paper, a spectral-spatial convolution neural network with Siamese architecture (SSCNN-S) for hyperspectral image (HSI) change detection (CD) is proposed. First, tensors are extracted in two HSIs recorded at different time points separately and tensor pairs are constructed. The tensor pairs are then incorporated into the spectral-spatial network to obtain two spectral-spatial vectors. Thereafter, the Euclidean distances of the two spectral-spatial vectors are calculated to represent the similarity of the tensor pairs. We use a Siamese network based on contrastive loss to train and optimize the network so that the Euclidean distance output by the network describes the similarity of tensor pairs as accurately as possible. Finally, the values obtained by inputting all tensor pairs into the trained model are used to judge whether a pixel belongs to the change area. SSCNN-S aims to transform the problem of HSI CD into a problem of similarity measurement for tensor pairs by introducing the Siamese network. The network used to extract tensor features in SSCNN-S combines spectral and spatial information to reduce the impact of noise on CD. Additionally, a useful four-test scoring method is proposed to improve the experimental efficiency instead of taking the mean value from multiple measurements. Experiments on real data sets have demonstrated the validity of the SSCNN-S method.

APA, Harvard, Vancouver, ISO, and other styles

8

Hayashi, Kohei. "Exploring Unexplored Tensor Network Decompositions for Convolutional Neural Networks." Brain & Neural Networks 29, no. 4 (December 5, 2022): 193–201. http://dx.doi.org/10.3902/jnns.29.193.

Full text

APA, Harvard, Vancouver, ISO, and other styles

9

Ling, Julia, Andrew Kurzawski, and Jeremy Templeton. "Reynolds averaged turbulence modelling using deep neural networks with embedded invariance." Journal of Fluid Mechanics 807 (October 18, 2016): 155–66. http://dx.doi.org/10.1017/jfm.2016.615.

Full text

Abstract:

There exists significant demand for improved Reynolds-averaged Navier–Stokes (RANS) turbulence models that are informed by and can represent a richer set of turbulence physics. This paper presents a method of using deep neural networks to learn a model for the Reynolds stress anisotropy tensor from high-fidelity simulation data. A novel neural network architecture is proposed which uses a multiplicative layer with an invariant tensor basis to embed Galilean invariance into the predicted anisotropy tensor. It is demonstrated that this neural network architecture provides improved prediction accuracy compared with a generic neural network architecture that does not embed this invariance property. The Reynolds stress anisotropy predictions of this invariant neural network are propagated through to the velocity field for two test cases. For both test cases, significant improvement versus baseline RANS linear eddy viscosity and nonlinear eddy viscosity models is demonstrated.

APA, Harvard, Vancouver, ISO, and other styles

10

Hameed, Marawan Gamal Abdel, Marzieh S. Tahaei, Ali Mosleh, and Vahid Partovi Nia. "Convolutional Neural Network Compression through Generalized Kronecker Product Decomposition." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 1 (June 28, 2022): 771–79. http://dx.doi.org/10.1609/aaai.v36i1.19958.

Full text

Abstract:

Modern Convolutional Neural Network (CNN) architectures, despite their superiority in solving various problems, are generally too large to be deployed on resource constrained edge devices. In this paper, we reduce memory usage and floating-point operations required by convolutional layers in CNNs. We compress these layers by generalizing the Kronecker Product Decomposition to apply to multidimensional tensors, leading to the Generalized Kronecker Product Decomposition (GKPD). Our approach yields a plug-and-play module that can be used as a drop-in replacement for any convolutional layer. Experimental results for image classification on CIFAR-10 and ImageNet datasets using ResNet, MobileNetv2 and SeNet architectures substantiate the effectiveness of our proposed approach. We find that GKPD outperforms state-of-the-art decomposition methods including Tensor-Train and Tensor-Ring as well as other relevant compression methods such as pruning and knowledge distillation.

APA, Harvard, Vancouver, ISO, and other styles

11

Huang, Jian-Hui, Jiu-Ming Huang, Ai-Ping Li, and Yong-Zhi Tong. "Knowledge Reasoning Based on Neural Tensor Network." ITM Web of Conferences 12 (2017): 04004. http://dx.doi.org/10.1051/itmconf/20171204004.

Full text

APA, Harvard, Vancouver, ISO, and other styles

12

Wu, Bijiao, Dingheng Wang, Guangshe Zhao, Lei Deng, and Guoqi Li. "Hybrid tensor decomposition in neural network compression." Neural Networks 132 (December 2020): 309–20. http://dx.doi.org/10.1016/j.neunet.2020.09.006.

Full text

APA, Harvard, Vancouver, ISO, and other styles

13

Hong, Wenxing, Wenjing Xu, Jianwei Qi, and Yang Weng. "Neural Tensor Network for Multi- Label Classification." IEEE Access 7 (2019): 96936–41. http://dx.doi.org/10.1109/access.2019.2930206.

Full text

APA, Harvard, Vancouver, ISO, and other styles

14

Vanchurin, Vitaly. "The World as a Neural Network." Entropy 22, no. 11 (October 26, 2020): 1210. http://dx.doi.org/10.3390/e22111210.

Full text

Abstract:

We discuss a possibility that the entire universe on its most fundamental level is a neural network. We identify two different types of dynamical degrees of freedom: “trainable” variables (e.g., bias vector or weight matrix) and “hidden” variables (e.g., state vector of neurons). We first consider stochastic evolution of the trainable variables to argue that near equilibrium their dynamics is well approximated by Madelung equations (with free energy representing the phase) and further away from the equilibrium by Hamilton–Jacobi equations (with free energy representing the Hamilton’s principal function). This shows that the trainable variables can indeed exhibit classical and quantum behaviors with the state vector of neurons representing the hidden variables. We then study stochastic evolution of the hidden variables by considering D non-interacting subsystems with average state vectors, x¯1, …, x¯D and an overall average state vector x¯0. In the limit when the weight matrix is a permutation matrix, the dynamics of x¯μ can be described in terms of relativistic strings in an emergent D+1 dimensional Minkowski space-time. If the subsystems are minimally interacting, with interactions that are described by a metric tensor, and then the emergent space-time becomes curved. We argue that the entropy production in such a system is a local function of the metric tensor which should be determined by the symmetries of the Onsager tensor. It turns out that a very simple and highly symmetric Onsager tensor leads to the entropy production described by the Einstein–Hilbert term. This shows that the learning dynamics of a neural network can indeed exhibit approximate behaviors that were described by both quantum mechanics and general relativity. We also discuss a possibility that the two descriptions are holographic duals of each other.

APA, Harvard, Vancouver, ISO, and other styles

15

Kwon, Kyuahn, and Jaeyong Chung. "Reducing Parameters of Neural Networks via Recursive Tensor Approximation." Electronics 11, no. 2 (January 11, 2022): 214. http://dx.doi.org/10.3390/electronics11020214.

Full text

Abstract:

Large-scale neural networks have attracted much attention for surprising results in various cognitive tasks such as object detection and image classification. However, the large number of weight parameters in the complex networks can be problematic when the models are deployed to embedded systems. In addition, the problems are exacerbated in emerging neuromorphic computers, where each weight parameter is stored within a synapse, the primary computational resource of the bio-inspired computers. We describe an effective way of reducing the parameters by a recursive tensor factorization method. Applying the singular value decomposition in a recursive manner decomposes a tensor that represents the weight parameters. Then, the tensor is approximated by algorithms minimizing the approximation error and the number of parameters. This process factorizes a given network, yielding a deeper, less dense, and weight-shared network with good initial weights, which can be fine-tuned by gradient descent.

APA, Harvard, Vancouver, ISO, and other styles

16

Huang, Qiuyuan, Li Deng, Dapeng Wu, Chang Liu, and Xiaodong He. "Attentive Tensor Product Learning." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 1344–51. http://dx.doi.org/10.1609/aaai.v33i01.33011344.

Full text

Abstract:

This paper proposes a novel neural architecture — Attentive Tensor Product Learning (ATPL) — to represent grammatical structures of natural language in deep learning models. ATPL exploits Tensor Product Representations (TPR), a structured neural-symbolic model developed in cognitive science, to integrate deep learning with explicit natural language structures and rules. The key ideas of ATPL are: 1) unsupervised learning of role-unbinding vectors of words via the TPR-based deep neural network; 2) the use of attention modules to compute TPR; and 3) the integration of TPR with typical deep learning architectures including long short-term memory and feedforward neural networks. The novelty of our approach lies in its ability to extract the grammatical structure of a sentence by using role-unbinding vectors, which are obtained in an unsupervised manner. Our ATPL approach is applied to 1) image captioning, 2) part of speech (POS) tagging, and 3) constituency parsing of a natural language sentence. The experimental results demonstrate the effectiveness of the proposed approach in all these three natural language processing tasks.

APA, Harvard, Vancouver, ISO, and other styles

17

Chu, Weimeng, Shunan Wu, Xiao He, Yufei Liu, and Zhigang Wu. "Deep learning-based inertia tensor identification of the combined spacecraft." Proceedings of the Institution of Mechanical Engineers, Part G: Journal of Aerospace Engineering 234, no. 7 (February 7, 2020): 1356–66. http://dx.doi.org/10.1177/0954410020904555.

Full text

Abstract:

The identification accuracy of inertia tensor of combined spacecraft, which is composed by a servicing spacecraft and a captured target, could be easily affected by the measurement noise of angular rate. Due to frequently changing operating environments of combined spacecraft in space, the measurement noise of angular rate can be very complex. In this paper, an inertia tensor identification approach based on deep learning method is proposed to improve the ability of identifying inertia tensor of combined spacecraft in the presence of complex measurement noise. A deep neural network model for identification is constructed and trained by enough training data and a designed learning strategy. To verify the identification performance of the proposed deep neural network model, two testing set with different ranks of measure noises are used for simulation tests. Comparison tests are also delivered among the proposed deep neural network model, recursive least squares identification method, and tradition deep neural network model. The comparison results show that the proposed deep neural network model yields a more accurate and stable identification performance for inertia tensor of combined spacecraft in changeable and complex operating environments.

APA, Harvard, Vancouver, ISO, and other styles

18

Liu, Xiushan, Chun Shan, Qin Zhang, Jun Cheng, and Peng Xu. "Compressed Wavelet Tensor Attention Capsule Network." Security and Communication Networks 2021 (April 16, 2021): 1–12. http://dx.doi.org/10.1155/2021/9949204.

Full text

Abstract:

Texture classification plays an important role for various computer vision tasks. Depending upon the powerful feature extraction capability, convolutional neural network (CNN)-based texture classification methods have attracted extensive attention. However, there still exist many challenges, such as the extraction of multilevel texture features and the exploration of multidirectional relationships. To address the problem, this paper proposes the compressed wavelet tensor attention capsule network (CWTACapsNet), which integrates multiscale wavelet decomposition, tensor attention blocks, and quantization techniques into the framework of capsule neural network. Specifically, the multilevel wavelet decomposition is in charge of extracting multiscale spectral features in frequency domain; in addition, the tensor attention blocks explore the multidimensional dependencies of convolutional feature channels, and the quantization techniques make the computational storage complexities be suitable for edge computing requirements. The proposed CWTACapsNet provides an efficient way to explore spatial domain features, frequency domain features, and their dependencies which are useful for most texture classification tasks. Furthermore, CWTACapsNet benefits from quantization techniques and is suitable for edge computing applications. Experimental results on several texture datasets show that the proposed CWTACapsNet outperforms the state-of-the-art texture classification methods not only in accuracy but also in robustness.

APA, Harvard, Vancouver, ISO, and other styles

19

McConkey, R., E. Yee, and F. S. Lien. "Deep structured neural networks for turbulence closure modeling." Physics of Fluids 34, no. 3 (March 2022): 035110. http://dx.doi.org/10.1063/5.0083074.

Full text

Abstract:

Despite well-known limitations of Reynolds-averaged Navier–Stokes (RANS) simulations, this methodology remains the most widely used tool for predicting many turbulent flows due to computational efficiency. Machine learning is a promising approach to improve the accuracy of RANS simulations. One major area of improvement is using machine learning models to represent the complex relationship between the mean flow field gradients and the Reynolds stress tensor. In the present work, modifications to improve the stability of previous optimal eddy viscosity approaches for RANS simulations are presented and evaluated. The optimal eddy viscosity is reformulated with a non-negativity constraint, which promotes numerical stability. We demonstrate that the new formulation of the optimal eddy viscosity improves conditioning of RANS equations for a periodic hills test case. To demonstrate the suitability of this proportional/orthogonal tensor decomposition for use in a physics-informed data-driven turbulence closure, we use two neural networks (structured on this specific tensor decomposition, which is incorporated as an inductive bias into the network design) to predict the newly reformulated linear and non-linear parts of the Reynolds stress tensor. Injecting these network model predictions for the Reynolds stresses into RANS simulation improves predictions of the velocity field, even when compared to a sophisticated (state of the art) physics-based turbulence closure model. Finally, we apply shapley additive explanations values to obtain insights from the learned representation for inner workings of the neural network used to predict the optimal eddy viscosity from the input feature data.

APA, Harvard, Vancouver, ISO, and other styles

20

Khamparia, Aditya, Deepak Gupta, Nhu Gia Nguyen, Ashish Khanna, Babita Pandey, and Prayag Tiwari. "Sound Classification Using Convolutional Neural Network and Tensor Deep Stacking Network." IEEE Access 7 (2019): 7717–27. http://dx.doi.org/10.1109/access.2018.2888882.

Full text

APA, Harvard, Vancouver, ISO, and other styles

21

Yin, Miao, Huy Phan, Xiao Zang, Siyu Liao, and Bo Yuan. "BATUDE: Budget-Aware Neural Network Compression Based on Tucker Decomposition." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 8 (June 28, 2022): 8874–82. http://dx.doi.org/10.1609/aaai.v36i8.20869.

Full text

Abstract:

Model compression is very important for the efficient deployment of deep neural network (DNN) models on resource-constrained devices. Among various model compression approaches, high-order tensor decomposition is particularly attractive and useful because the decomposed model is very small and fully structured. For this category of approaches, tensor ranks are the most important hyper-parameters that directly determine the architecture and task performance of the compressed DNN models. However, as an NP-hard problem, selecting optimal tensor ranks under the desired budget is very challenging and the state-of-the-art studies suffer from unsatisfied compression performance and timing-consuming search procedures. To systematically address this fundamental problem, in this paper we propose BATUDE, a Budget-Aware TUcker DEcomposition-based compression approach that can efficiently calculate optimal tensor ranks via one-shot training. By integrating the rank selecting procedure to the DNN training process with a specified compression budget, the tensor ranks of the DNN models are learned from the data and thereby bringing very significant improvement on both compression ratio and classification accuracy for the compressed models. The experimental results on ImageNet dataset show that our method enjoys 0.33% top-5 higher accuracy with 2.52X less computational cost as compared to the uncompressed ResNet-18 model. For ResNet-50, the proposed approach enables 0.37% and 0.55% top-5 accuracy increase with 2.97X and 2.04X computational cost reduction, respectively, over the uncompressed model.

APA, Harvard, Vancouver, ISO, and other styles

22

Jain, Aabha, and Neha Sharma. "Accelerated AI Inference at CNN-Based Machine Vision in ASICs: A Design Approach." ECS Transactions 107, no. 1 (April 24, 2022): 5165–74. http://dx.doi.org/10.1149/10701.5165ecst.

Full text

Abstract:

Machine Vision (MV) has become an increasingly popular technology in Industry 4.0 for advanced manufacturing processes. Specially, assembly lines require fast MV of the machine objects, assembly-functions, and operations to improve accuracy and quality. In order to achieve this, MV deployed customized Application Specific Integrated Circuits (ASICs). These ASICs designed to run Convolution Neural Network (CNNs) and perform AI inference. The processing needs high performance computing (HPC) of a large number of tensors with least energy spending in the ASICs. In the paper, we propose a design approach for efficient tensor processing in tensor cores for accelerated AI inference at CNN based MV in ASICs. The design approach deploys the small DNN (Deep Learning Neural Network) model SqueezeNext along with application of quantization, fast arithmetic reduction, tensor cores aware tuning, pruning, and fusion for the efficient AI inference in ASICs. Successful implementation of proposed design can provide a competitive advantage to industries in terms of quality and cost of product along with time saving in manufacturing process.

APA, Harvard, Vancouver, ISO, and other styles

23

Cho, Hyungmin. "RiSA: A Reinforced Systolic Array for Depthwise Convolutions and Embedded Tensor Reshaping." ACM Transactions on Embedded Computing Systems 20, no. 5s (October 31, 2021): 1–20. http://dx.doi.org/10.1145/3476984.

Full text

Abstract:

Depthwise convolutions are widely used in convolutional neural networks (CNNs) targeting mobile and embedded systems. Depthwise convolution layers reduce the computation loads and the number of parameters compared to the conventional convolution layers. Many deep neural network (DNN) accelerators adopt an architecture that exploits the high data-reuse factor of DNN computations, such as a systolic array. However, depthwise convolutions have low data-reuse factor and under-utilize the processing elements (PEs) in systolic arrays. In this paper, we present a DNN accelerator design called RiSA, which provides a novel mechanism that boosts the PE utilization for depthwise convolutions on a systolic array with minimal overheads. In addition, the PEs in systolic arrays can be efficiently used only if the data items ( tensors ) are arranged in the desired layout. Typical DNN accelerators provide various types of PE interconnects or additional modules to flexibly rearrange the data items and manage data movements during DNN computations. RiSA provides a lightweight set of tensor management tasks within the PE array itself that eliminates the need for an additional module for tensor reshaping tasks. Using this embedded tensor reshaping, RiSA supports various DNN models, including convolutional neural networks and natural language processing models while maintaining a high area efficiency. Compared to Eyeriss v2, RiSA improves the area and energy efficiency for MobileNet-V1 inference by 1.91× and 1.31×, respectively.

APA, Harvard, Vancouver, ISO, and other styles

24

Kim, Han-joon, and Pureum Lim. "A Tensor Space Model-Based Deep Neural Network for Text Classification." Applied Sciences 11, no. 20 (October 18, 2021): 9703. http://dx.doi.org/10.3390/app11209703.

Full text

Abstract:

Most text classification systems use machine learning algorithms; among these, naïve Bayes and support vector machine algorithms adapted to handle text data afford reasonable performance. Recently, given developments in deep learning technology, several scholars have used deep neural networks (recurrent and convolutional neural networks) to improve text classification. However, deep learning-based text classification has not greatly improved performance compared to that of conventional algorithms. This is because a textual document is essentially expressed as a vector (only), albeit with word dimensions, which compromises the inherent semantic information, even if the vector is (appropriately) transformed to add conceptual information. To solve this ‘loss of term senses’ problem, we develop a concept-driven deep neural network based upon our semantic tensor space model. The semantic tensor used for text representation features a dependency between the term and the concept; we use this to develop three deep neural networks for text classification. We perform experiments using three standard document corpora, and we show that our proposed methods are superior to both traditional and more recent learning methods.

APA, Harvard, Vancouver, ISO, and other styles

25

Noskova, E. S., I. E. Zakharov, Y. N. Shkandybin, and S. G. Rykovanov. "Towards energy-efficient neural network calculations." Computer Optics 46, no. 1 (February 2022): 160–66. http://dx.doi.org/10.18287/2412-6179-co-914.

Full text

Abstract:

Nowadays, the problem of creating high-performance and energy-efficient hardware for Artificial Intelligence tasks is very acute. The most popular solution to this problem is the use of Deep Learning Accelerators, such as GPUs and Tensor Processing Units to run neural networks. Recently, NVIDIA has announced the NVDLA project, which allows one to design neural network accelerators based on an open-source code. This work describes a full cycle of creating a prototype NVDLA accelerator, as well as testing the resulting solution by running the resnet-50 neural network on it. Finally, an assessment of the performance and power efficiency of the prototype NVDLA accelerator when compared to the GPU and CPU is provided, the results of which show the superiority of NVDLA in many characteristics.

APA, Harvard, Vancouver, ISO, and other styles

26

Oymak, Samet, and Mahdi Soltanolkotabi. "Learning a deep convolutional neural network via tensor decomposition." Information and Inference: A Journal of the IMA 10, no. 3 (February 1, 2021): 1031–71. http://dx.doi.org/10.1093/imaiai/iaaa042.

Full text

Abstract:

Abstract In this paper, we study the problem of learning the weights of a deep convolutional neural network. We consider a network where convolutions are carried out over non-overlapping patches. We develop an algorithm for simultaneously learning all the kernels from the training data. Our approach dubbed deep tensor decomposition (DeepTD) is based on a low-rank tensor decomposition. We theoretically investigate DeepTD under a realizable model for the training data where the inputs are chosen i.i.d. from a Gaussian distribution and the labels are generated according to planted convolutional kernels. We show that DeepTD is sample efficient and provably works as soon as the sample size exceeds the total number of convolutional weights in the network.

APA, Harvard, Vancouver, ISO, and other styles

27

Clark, Stephen R. "Unifying neural-network quantum states and correlator product states via tensor networks." Journal of Physics A: Mathematical and Theoretical 51, no. 13 (February 23, 2018): 135301. http://dx.doi.org/10.1088/1751-8121/aaaaf2.

Full text

APA, Harvard, Vancouver, ISO, and other styles

28

He, Zhen, Shaobing Gao, Liang Xiao, Daxue Liu, and Hangen He. "Multimedia Data Modelling Using Multidimensional Recurrent Neural Networks." Symmetry 10, no. 9 (September 1, 2018): 370. http://dx.doi.org/10.3390/sym10090370.

Full text

Abstract:

Modelling the multimedia data such as text, images, or videos usually involves the analysis, prediction, or reconstruction of them. The recurrent neural network (RNN) is a powerful machine learning approach to modelling these data in a recursive way. As a variant, the long short-term memory (LSTM) extends the RNN with the ability to remember information for longer. Whilst one can increase the capacity of LSTM by widening or adding layers, additional parameters and runtime are usually required, which could make learning harder. We therefore propose a Tensor LSTM where the hidden states are tensorised as multidimensional arrays (tensors) and updated through a cross-layer convolution. As parameters are spatially shared within the tensor, we can efficiently widen the model without extra parameters by increasing the tensorised size; as deep computations of each time step are absorbed by temporal computations of the time series, we can implicitly deepen the model with little extra runtime by delaying the output. We show by experiments that our model is well-suited for various multimedia data modelling tasks, including text generation, text calculation, image classification, and video prediction.

APA, Harvard, Vancouver, ISO, and other styles

29

Fonseca, Ricardo, Oscar Guarnizo, Diego Suntaxi, Alfonso Cadiz, and Werner Creixell. "Convolutional Neural Network Feature Extraction Using Covariance Tensor Decomposition." IEEE Access 9 (2021): 66646–60. http://dx.doi.org/10.1109/access.2021.3076033.

Full text

APA, Harvard, Vancouver, ISO, and other styles

30

Choi, Hyunsoo, Junghoon Jang, and Saebyeok Lee. "Document Feature Composition using Event-based Neural Tensor Network." KIISE Transactions on Computing Practices 25, no. 8 (August 31, 2019): 407–11. http://dx.doi.org/10.5626/ktcp.2019.25.8.407.

Full text

APA, Harvard, Vancouver, ISO, and other styles

31

Qiaoben, You, Zheng Wang, Jianguo Li, Yinpeng Dong, Yu-Gang Jiang, and Jun Zhu. "Composite Binary Decomposition Networks." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 4747–54. http://dx.doi.org/10.1609/aaai.v33i01.33014747.

Full text

Abstract:

Binary neural networks have great resource and computing efficiency, while suffer from long training procedure and non-negligible accuracy drops, when comparing to the fullprecision counterparts. In this paper, we propose the composite binary decomposition networks (CBDNet), which first compose real-valued tensor of each layer with a limited number of binary tensors, and then decompose some conditioned binary tensors into two low-rank binary tensors, so that the number of parameters and operations are greatly reduced comparing to the original ones. Experiments demonstrate the effectiveness of the proposed method, as CBDNet can approximate image classification network ResNet-18 using 5.25 bits, VGG-16 using 5.47 bits, DenseNet-121 using 5.72 bits, object detection networks SSD300 using 4.38 bits, and semantic segmentation networks SegNet using 5.18 bits, all with minor accuracy drops.1

APA, Harvard, Vancouver, ISO, and other styles

32

Harada, Akira, Shota Nishikawa, and Shoichi Yamada. "Deep Learning of the Eddington Tensor in Core-collapse Supernova Simulation." Astrophysical Journal 925, no. 2 (January 31, 2022): 117. http://dx.doi.org/10.3847/1538-4357/ac3998.

Full text

Abstract:

Abstract We trained deep neural networks (DNNs) as a function of the neutrino energy density, flux, and the fluid velocity to reproduce the Eddington tensor for neutrinos obtained in our first-principles core-collapse supernova simulation. Although the moment method, which is one of the most popular approximations for neutrino transport, requires a closure relation, none of the analytical closure relations commonly employed in the literature capture all aspects of the neutrino angular distribution in momentum space. In this paper, we develop a closure relation by using DNNs that take the neutrino energy density, flux, and the fluid velocity as the inputs and the Eddington tensor as the output. We consider two kinds of DNNs: a conventional DNN, named a component-wise neural network (CWNN), and a tensor-basis neural network (TBNN). We find that the diagonal component of the Eddington tensor is better reproduced by the DNNs than the M1 closure relation, especially for low to intermediate energies. For the off-diagonal component, the DNNs agree better with the Boltzmann solver than the M1 closure relation at large radii. In the comparison between the two DNNs, the TBNN displays slightly better performance than the CWNN. With these new closure relations at hand, based on DNNs that well reproduce the Eddington tensor at much lower costs, we have opened up a new possibility for the moment method.

APA, Harvard, Vancouver, ISO, and other styles

33

Lyu, Shengfei, and Jiaqi Liu. "Convolutional Recurrent Neural Networks for Text Classification." Journal of Database Management 32, no. 4 (October 2021): 65–82. http://dx.doi.org/10.4018/jdm.2021100105.

Full text

Abstract:

Recurrent neural network (RNN) and convolutional neural network (CNN) are two prevailing architectures used in text classification. Traditional approaches combine the strengths of these two networks by straightly streamlining them or linking features extracted from them. In this article, a novel approach is proposed to maintain the strengths of RNN and CNN to a great extent. In the proposed approach, a bi-directional RNN encodes each word into forward and backward hidden states. Then, a neural tensor layer is used to fuse bi-directional hidden states to get word representations. Meanwhile, a convolutional neural network is utilized to learn the importance of each word for text classification. Empirical experiments are conducted on several datasets for text classification. The superior performance of the proposed approach confirms its effectiveness.

APA, Harvard, Vancouver, ISO, and other styles

34

Yang, Ying, Chengyang Zhang, and Huaixin Cao. "Approximating Ground States by Neural Network Quantum States." Entropy 21, no. 1 (January 17, 2019): 82. http://dx.doi.org/10.3390/e21010082.

Full text

Abstract:

Motivated by the Carleo’s work (Science, 2017, 355: 602), we focus on finding the neural network quantum statesapproximation of the unknown ground state of a given Hamiltonian H in terms of the best relative error and explore the influences of sum, tensor product, local unitary of Hamiltonians on the best relative error. Besides, we illustrate our method with some examples.

APA, Harvard, Vancouver, ISO, and other styles

35

Seeliger, K., L. Ambrogioni, Y. Güçlütürk, L. M. van den Bulk, U. Güçlü, and M. A. J. van Gerven. "End-to-end neural system identification with neural information flow." PLOS Computational Biology 17, no. 2 (February 4, 2021): e1008558. http://dx.doi.org/10.1371/journal.pcbi.1008558.

Full text

Abstract:

Neural information flow (NIF) provides a novel approach for system identification in neuroscience. It models the neural computations in multiple brain regions and can be trained end-to-end via stochastic gradient descent from noninvasive data. NIF models represent neural information processing via a network of coupled tensors, each encoding the representation of the sensory input contained in a brain region. The elements of these tensors can be interpreted as cortical columns whose activity encodes the presence of a specific feature in a spatiotemporal location. Each tensor is coupled to the measured data specific to a brain region via low-rank observation models that can be decomposed into the spatial, temporal and feature receptive fields of a localized neuronal population. Both these observation models and the convolutional weights defining the information processing within regions are learned end-to-end by predicting the neural signal during sensory stimulation. We trained a NIF model on the activity of early visual areas using a large-scale fMRI dataset recorded in a single participant. We show that we can recover plausible visual representations and population receptive fields that are consistent with empirical findings.

APA, Harvard, Vancouver, ISO, and other styles

36

Zhou, Lingxiao, Shuaichao Zhang, Jingru Yu, and Xiqun Chen. "Spatial–Temporal Deep Tensor Neural Networks for Large-Scale Urban Network Speed Prediction." IEEE Transactions on Intelligent Transportation Systems 21, no. 9 (September 2020): 3718–29. http://dx.doi.org/10.1109/tits.2019.2932038.

Full text

APA, Harvard, Vancouver, ISO, and other styles

37

TJANDRA, Andros, Sakriani SAKTI, and Satoshi NAKAMURA. "Recurrent Neural Network Compression Based on Low-Rank Tensor Representation." IEICE Transactions on Information and Systems E103.D, no. 2 (February 1, 2020): 435–49. http://dx.doi.org/10.1587/transinf.2019edp7040.

Full text

APA, Harvard, Vancouver, ISO, and other styles

38

XING, Ling, Mei HE, Qiang MA, and Min ZHU. "Multi-semantic audio classification method based on tensor neural network." Journal of Computer Applications 32, no. 10 (May 23, 2013): 2895–98. http://dx.doi.org/10.3724/sp.j.1087.2012.02895.

Full text

APA, Harvard, Vancouver, ISO, and other styles

39

Pandey, Sandeep Kumar, Hanumant Singh Shekhawat, and S. R. M. Prasanna. "Attention gated tensor neural network architectures for speech emotion recognition." Biomedical Signal Processing and Control 71 (January 2022): 103173. http://dx.doi.org/10.1016/j.bspc.2021.103173.

Full text

APA, Harvard, Vancouver, ISO, and other styles

40

Li Tianqi, 李天琪, 谭海 Tan Hai, 戴激光 Dai Jiguang, 杜阳 Du Yang, and 王杨 Wang Yang. "Road Extraction Method Combining Convolutional Neural Network and Tensor Voting." Laser & Optoelectronics Progress 57, no. 20 (2020): 201019. http://dx.doi.org/10.3788/lop57.201019.

Full text

APA, Harvard, Vancouver, ISO, and other styles

41

Zhang, Xiaoying, and Yangde Zhang. "Outlier detection based on the neural network for tensor estimation." Biomedical Signal Processing and Control 13 (September 2014): 148–56. http://dx.doi.org/10.1016/j.bspc.2014.04.005.

Full text

APA, Harvard, Vancouver, ISO, and other styles

42

Xu, Xiaowen, Qiang Wu, Shuo Wang, Ju Liu, Jiande Sun, and Andrzej Cichocki. "Whole Brain fMRI Pattern Analysis Based on Tensor Neural Network." IEEE Access 6 (2018): 29297–305. http://dx.doi.org/10.1109/access.2018.2815770.

Full text

APA, Harvard, Vancouver, ISO, and other styles

43

Malathi, M., and P. Sinthia. "Brain Tumour Segmentation Using Convolutional Neural Network with Tensor Flow." Asian Pacific Journal of Cancer Prevention 20, no. 7 (July 1, 2019): 2095–101. http://dx.doi.org/10.31557/apjcp.2019.20.7.2095.

Full text

APA, Harvard, Vancouver, ISO, and other styles

44

Zhou, Mingyi, Yipeng Liu, Zhen Long, Longxi Chen, and Ce Zhu. "Tensor rank learning in CP decomposition via convolutional neural network." Signal Processing: Image Communication 73 (April 2019): 12–21. http://dx.doi.org/10.1016/j.image.2018.03.017.

Full text

APA, Harvard, Vancouver, ISO, and other styles

45

Wang, Xuezhong, Maolin Che, and Yimin Wei. "Neural network approach for solving nonsingular multi-linear tensor systems." Journal of Computational and Applied Mathematics 368 (April 2020): 112569. http://dx.doi.org/10.1016/j.cam.2019.112569.

Full text

APA, Harvard, Vancouver, ISO, and other styles

46

Naidu, D. J. Samatha, and T. Mahammad Rafi. "HANDWRITTEN CHARACTER RECOGNITION USING CONVOLUTIONAL NEURAL NETWORKS." International Journal of Computer Science and Mobile Computing 10, no. 8 (August 30, 2021): 41–45. http://dx.doi.org/10.47760/ijcsmc.2021.v10i08.007.

Full text

Abstract:

Handwritten character Recognition is one of the active area of research where deep neural networks are been utilized. Handwritten character Recognition is a challenging task because of many reasons. The Primary reason is different people have different styles of handwriting. The secondary reason is there are lot of characters like capital letters, small letters & special symbols. In existing were immense research going on the field of handwritten character recognition system has been design using fuzzy logic and created on VLSI(very large scale integrated)structure. To Recognize the tamil characters they have use neural networks with the Kohonen self-organizing map(SOM) which is an unsupervised neural networks. In proposed system this project design a image segmentation based hand written character recognition system. The convolutional neural network is the current state of neural network which has wide application in fields like image, video recognition. The system easily identify or easily recognize text in English languages and letters, digits. By using Open cv for performing image processing and having tensor flow for training the neural network. To develop this concept proposing the innovative method for offline handwritten characters. detection using deep neural networks using python programming language.

APA, Harvard, Vancouver, ISO, and other styles

47

Klus, Stefan, and Patrick Gelß. "Tensor-Based Algorithms for Image Classification." Algorithms 12, no. 11 (November 9, 2019): 240. http://dx.doi.org/10.3390/a12110240.

Full text

Abstract:

Interest in machine learning with tensor networks has been growing rapidly in recent years. We show that tensor-based methods developed for learning the governing equations of dynamical systems from data can, in the same way, be used for supervised learning problems and propose two novel approaches for image classification. One is a kernel-based reformulation of the previously introduced multidimensional approximation of nonlinear dynamics (MANDy), the other an alternating ridge regression in the tensor train format. We apply both methods to the MNIST and fashion MNIST data set and show that the approaches are competitive with state-of-the-art neural network-based classifiers.

APA, Harvard, Vancouver, ISO, and other styles

48

Wang, Weiping, Feng Zhang, Xi Luo, and Shigeng Zhang. "PDRCNN: Precise Phishing Detection with Recurrent Convolutional Neural Networks." Security and Communication Networks 2019 (October 29, 2019): 1–15. http://dx.doi.org/10.1155/2019/2595794.

Full text

Abstract:

Through well-designed counterfeit websites, phishing induces online users to visit forged web pages to obtain their private sensitive information, e.g., account number and password. Existing antiphishing approaches are mostly based on page-related features, which require to crawl content of web pages as well as accessing third-party search engines or DNS services. This not only leads to their low efficiency in detecting phishing but also makes them rely on network environment and third-party services heavily. In this paper, we propose a fast phishing website detection approach called PDRCNN that relies only on the URL of the website. PDRCNN neither needs to retrieve content of the target website nor uses any third-party services as previous approaches do. It encodes the information of an URL into a two-dimensional tensor and feeds the tensor into a novelly designed deep learning neural network to classify the original URL. We first use a bidirectional LSTM network to extract global features of the constructed tensor and give all string information to each character in the URL. After that, we use a CNN to automatically judge which characters play key roles in phishing detection, capture the key components of the URL, and compress the extracted features into a fixed length vector space. By combining the two types of networks, PDRCNN achieves better performance than just using either one of them. We built a dataset containing nearly 500,000 URLs which are obtained through Alexa and PhishTank. Experimental results show that PDRCNN achieves a detection accuracy of 97% and an AUC value of 99%, which is much better than state-of-the-art approaches. Furthermore, the recognition process is very fast: on the trained PDRCNN model, the average per URL detection time only cost 0.4 ms.

APA, Harvard, Vancouver, ISO, and other styles

49

Hari Krishna, Yaram, Kanagala Bharath Kumar, Dasari Maharshi, and J. Amudhavel. "Image Processing and Restriction of Video Downloads Using Cloud." International Journal of Engineering & Technology 7, no. 2.32 (May 31, 2018): 327. http://dx.doi.org/10.14419/ijet.v7i2.32.15705.

Full text

Abstract:

Flower image classification using deep learning and convolutional neural network (CNN) based on machine learning in Tensor flow. Tensor flow IDE is used to implement machine learning algorithms. Flower image processing is based on supervised learning which detects the parameters of image. Parameters of the image were compared by decision algorithms. These images are classified by neurons in convolutional neural network. Video processing based on machine learning is used in restriction of downloading the videos by preventing the second response from the server and enabling the debugging of the video by removing the request from the user.

APA, Harvard, Vancouver, ISO, and other styles

50

Xu, Shaofu, and Weiwen Zou. "Optical tensor core architecture for neural network training based on dual-layer waveguide topology and homodyne detection." Chinese Optics Letters 19, no. 8 (2021): 082501. http://dx.doi.org/10.3788/col202119.082501.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Journal articles on the topic 'Neural Tensor Network'

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles