Journal articles on the topic 'Structured sparsity model'
To see the other types of publications on this topic, follow the link: Structured sparsity model.
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Structured sparsity model.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Niu, Wei, Mengshu Sun, Zhengang Li, Jou-An Chen, Jiexiong Guan, Xipeng Shen, Yanzhi Wang, Sijia Liu, Xue Lin, and Bin Ren. "RT3D: Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 10 (May 18, 2021): 9179–87. http://dx.doi.org/10.1609/aaai.v35i10.17108.
Full textAbstract:
Mobile devices are becoming an important carrier for deep learning tasks, as they are being equipped with powerful, high-end mobile CPUs and GPUs. However, it is still a challenging task to execute 3D Convolutional Neural Networks (CNNs) targeting for real-time performance, besides high inference accuracy. The reason is more complex model structure and higher model dimensionality overwhelm the available computation/storage resources on mobile devices. A natural way may be turning to deep learning weight pruning techniques. However, the direct generalization of existing 2D CNN weight pruning methods to 3D CNNs is not ideal for fully exploiting mobile parallelism while achieving high inference accuracy. This paper proposes RT3D, a model compression and mobile acceleration framework for 3D CNNs, seamlessly integrating neural network weight pruning and compiler code generation techniques. We propose and investigate two structured sparsity schemes i.e., the vanilla structured sparsity and kernel group structured (KGS) sparsity that are mobile acceleration friendly. The vanilla sparsity removes whole kernel groups, while KGS sparsity is a more fine-grained structured sparsity that enjoys higher flexibility while exploiting full on-device parallelism. We propose a reweighted regularization pruning algorithm to achieve the proposed sparsity schemes. The inference time speedup due to sparsity is approaching the pruning rate of the whole model FLOPs (floating point operations). RT3D demonstrates up to 29.1x speedup in end-to-end inference time comparing with current mobile frameworks supporting 3D CNNs, with moderate 1%~1.5% accuracy loss. The end-to-end inference time for 16 video frames could be within 150 ms, when executing representative C3D and R(2+1)D models on a cellphone. For the first time, real-time execution of 3D CNNs is achieved on off-the-shelf mobiles.
2
Sun, Jun, Qidong Chen, Jianan Sun, Tao Zhang, Wei Fang, and Xiaojun Wu. "Graph-structured multitask sparsity model for visual tracking." Information Sciences 486 (June 2019): 133–47. http://dx.doi.org/10.1016/j.ins.2019.02.043.
Full text
3
Ruan, Xiaofeng, Yufan Liu, Bing Li, Chunfeng Yuan, and Weiming Hu. "DPFPS: Dynamic and Progressive Filter Pruning for Compressing Convolutional Neural Networks from Scratch." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 3 (May 18, 2021): 2495–503. http://dx.doi.org/10.1609/aaai.v35i3.16351.
Full textAbstract:
Filter pruning is a commonly used method for compressing Convolutional Neural Networks (ConvNets), due to its friendly hardware supporting and flexibility. However, existing methods mostly need a cumbersome procedure, which brings many extra hyper-parameters and training epochs. This is because only using sparsity and pruning stages cannot obtain a satisfying performance. Besides, many works do not consider the difference of pruning ratio across different layers. To overcome these limitations, we propose a novel dynamic and progressive filter pruning (DPFPS) scheme that directly learns a structured sparsity network from Scratch. In particular, DPFPS imposes a new structured sparsity-inducing regularization specifically upon the expected pruning parameters in a dynamic sparsity manner. The dynamic sparsity scheme determines sparsity allocation ratios of different layers and a Taylor series based channel sensitivity criteria is presented to identify the expected pruning parameters. Moreover, we increase the structured sparsity-inducing penalty in a progressive manner. This helps the model to be sparse gradually instead of forcing the model to be sparse at the beginning. Our method solves the pruning ratio based optimization problem by an iterative soft-thresholding algorithm (ISTA) with dynamic sparsity. At the end of the training, we only need to remove the redundant parameters without other stages, such as fine-tuning. Extensive experimental results show that the proposed method is competitive with 11 state-of-the-art methods on both small-scale and large-scale datasets (i.e., CIFAR and ImageNet). Specifically, on ImageNet, we achieve a 44.97% pruning ratio of FLOPs by compressing ResNet-101, even with an increase of 0.12% Top-5 accuracy. Our pruned models and codes are released at https://github.com/taoxvzi/DPFPS.
4
Wu, Hao, Shu Li, Yingpin Chen, and Zhenming Peng. "Seismic impedance inversion using second-order overlapping group sparsity with A-ADMM." Journal of Geophysics and Engineering 17, no. 1 (November 22, 2019): 97–116. http://dx.doi.org/10.1093/jge/gxz094.
Full textAbstract:
Abstract The anisotropic total variation with overlapping group sparsity (ATV_OGS) regularisation term is an improvement on the anisotropic total variation (ATV) regularisation term. It has been employed successfully in seismic impedance inversion as it can enhance the boundary information and relieve the staircase effect by exploring the structured sparsity of seismic impedance. However, because ATV_OGS constrains only the structured sparsity of the impedance's first-order difference and ignores the structured sparsity of the second-order difference, the staircase effect still occurs in an inversion result based on ATV_OGS. To further fit the structured sparsity of the impedance's second-order gradients, we introduce the overlapping group sparsity into the second-order difference of the impedance and propose a novel second-order ATV with overlapping group sparsity (SATV_OGS) seismic impedance inversion method. The proposed method reduces the interference of the large amplitude noise and further mitigates the staircase effect of the ATV_OGS. Furthermore, the accelerated alternating direction method of multipliers (A-ADMM) framework applied to this novel method. It can increase the efficiency of inversion. The experiments are carried out on a general model data and field data. Based on the experimental results, the proposed method can obtain higher resolution impedance than some impedance inversion methods based on total variation.
5
Zhu, Zijiang, Junshan Li, Yi Hu, and Xiaoguang Deng. "Research on Age Estimation Algorithm Based on Structured Sparsity." International Journal of Pattern Recognition and Artificial Intelligence 33, no. 06 (April 21, 2019): 1956006. http://dx.doi.org/10.1142/s0218001419560068.
Full textAbstract:
In order to solve the inaccuracy of age estimation dataset and the imbalance of age distribution, this paper proposes an age estimation model based on the structured sparse learning. Firstly, the Multi-label representation of facial images is performed by age, and the age estimation model is trained by solving the model matrix. Finally, the correlation with all age labels is calculated according to the facial images and age estimation model to be tested, and the most correlated age is taken as the predicted age. This paper sets up a series of verification experiments, and analyzes the structured sparse age estimation model from several perspectives. The proposed algorithm has achieved good results in the evaluation of indexes such as the mean absolute error, accumulation index curve and convergence rate, and has designed the demo system to put the model into use. Facts prove that the age estimation model proposed in this paper may achieve a good estimation effect.
6
Zhang, Lingli. "Total variation with modified group sparsity for CT reconstruction under low SNR." Journal of X-Ray Science and Technology 29, no. 4 (July 27, 2021): 645–62. http://dx.doi.org/10.3233/xst-200833.
Full textAbstract:
BACKGROUND AND OBJECTIVE: Since the stair artifacts may affect non-destructive testing (NDT) and diagnosis in the later stage, an applicable model is desperately needed, which can deal with the stair artifacts and preserve the edges. However, the classical total variation (TV) algorithm only considers the sparsity of the gradient transformed image. The objective of this study is to introduce and test a new method based on group sparsity to address the low signal-to-noise ratio (SNR) problem. METHODS: This study proposes a weighted total variation with overlapping group sparsity model. This model combines the Gaussian kernel and overlapping group sparsity into TV model denoted as GOGS-TV, which considers the structure sparsity of the image to be reconstructed to deal with the stair artifacts. On one hand, TV is the accepted commercial algorithm, and it can work well in many situations. On the other hand, the Gaussian kernel can associate the points around each pixel. Quantitative assessments are implemented to verify this merit. RESULTS: Numerical simulations are performed to validate the presented method, compared with the classical simultaneous algebraic reconstruction technique (SART) and the state-of-the-art TV algorithm. It confirms the significantly improved SNR of the reconstruction images both in suppressing the noise and preserving the edges using new GOGS-TV model. CONCLUSIONS: The proposed GOGS-TV model demonstrates its advantages to reduce stair artifacts especially in low SNR reconstruction because this new model considers both the sparsity of the gradient image and the structured sparsity. Meanwhile, the Gaussian kernel is utilized as a weighted factor that can be adapted to the global distribution.
7
Ou, Weihua, and Wenjun Xiao. "Structured sparsity model with spatial similarity regularisation for semantic feature selection." International Journal of Advanced Media and Communication 7, no. 2 (2017): 138. http://dx.doi.org/10.1504/ijamc.2017.085941.
Full text
8
Xiao, Wenjun, and Weihua Ou. "Structured sparsity model with spatial similarity regularisation for semantic feature selection." International Journal of Advanced Media and Communication 7, no. 2 (2017): 138. http://dx.doi.org/10.1504/ijamc.2017.10006892.
Full text
9
Ma, Xiaolong, Fu-Ming Guo, Wei Niu, Xue Lin, Jian Tang, Kaisheng Ma, Bin Ren, and Yanzhi Wang. "PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-Time Execution on Mobile Devices." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 5117–24. http://dx.doi.org/10.1609/aaai.v34i04.5954.
Full textAbstract:
Model compression techniques on Deep Neural Network (DNN) have been widely acknowledged as an effective way to achieve acceleration on a variety of platforms, and DNN weight pruning is a straightforward and effective method. There are currently two mainstreams of pruning methods representing two extremes of pruning regularity: non-structured, fine-grained pruning can achieve high sparsity and accuracy, but is not hardware friendly; structured, coarse-grained pruning exploits hardware-efficient structures in pruning, but suffers from accuracy drop when the pruning rate is high. In this paper, we introduce PCONV, comprising a new sparsity dimension, – fine-grained pruning patterns inside the coarse-grained structures. PCONV comprises two types of sparsities, Sparse Convolution Patterns (SCP) which is generated from intra-convolution kernel pruning and connectivity sparsity generated from inter-convolution kernel pruning. Essentially, SCP enhances accuracy due to its special vision properties, and connectivity sparsity increases pruning rate while maintaining balanced workload on filter computation. To deploy PCONV, we develop a novel compiler-assisted DNN inference framework and execute PCONV models in real-time without accuracy compromise, which cannot be achieved in prior work. Our experimental results show that, PCONV outperforms three state-of-art end-to-end DNN frameworks, TensorFlow-Lite, TVM, and Alibaba Mobile Neural Network with speedup up to 39.2 ×, 11.4 ×, and 6.3 ×, respectively, with no accuracy loss. Mobile devices can achieve real-time inference on large-scale DNNs.
10
Javanmardi, Mohammadreza, Amir Hossein Farzaneh, and Xiaojun Qi. "A Robust Structured Tracker Using Local Deep Features." Electronics 9, no. 5 (May 20, 2020): 846. http://dx.doi.org/10.3390/electronics9050846.
Full textAbstract:
Deep features extracted from convolutional neural networks have been recently utilized in visual tracking to obtain a generic and semantic representation of target candidates. In this paper, we propose a robust structured tracker using local deep features (STLDF). This tracker exploits the deep features of local patches inside target candidates and sparsely represents them by a set of templates in the particle filter framework. The proposed STLDF utilizes a new optimization model, which employs a group-sparsity regularization term to adopt local and spatial information of the target candidates and attain the spatial layout structure among them. To solve the optimization model, we propose an efficient and fast numerical algorithm that consists of two subproblems with the close-form solutions. Different evaluations in terms of success and precision on the benchmarks of challenging image sequences (e.g., OTB50 and OTB100) demonstrate the superior performance of the STLDF against several state-of-the-art trackers.
11
Fan, Bo, Xiaoli Zhou, Shuo Chen, Zhijie Jiang, and Yongqiang Cheng. "Sparse Bayesian Perspective for Radar Coincidence Imaging with Model Errors." Mathematical Problems in Engineering 2020 (April 21, 2020): 1–12. http://dx.doi.org/10.1155/2020/9202654.
Full textAbstract:
Sparsity-driven methods are commonly applied to reconstruct targets in radar coincidence imaging (RCI), where the reference matrix needs to be computed precisely and the prior knowledge of the accurate imaging model is essential. Unfortunately, the existence of model errors in practical RCI applications is common, which defocuses the reconstructed image considerably. Accordingly, this paper aims to formulate a unified framework for sparsity-driven RCI with model errors based on the sparse Bayesian approach. Firstly, a parametric joint sparse reconstruction model is built to describe the RCI when perturbed by model errors. The structured sparse Bayesian prior is then assigned to this model, after which the structured sparse Bayesian autofocus (SSBA) algorithm is proposed in the variational Bayesian expectation maximization (VBEM) framework; this solution jointly realizes sparse imaging and model error calibration. Simulation results demonstrate that the proposed algorithm can both calibrate the model errors and obtain a well-focused target image with high reconstruction accuracy.
12
Lobacheva, Ekaterina, Nadezhda Chirkova, Alexander Markovich, and Dmitry Vetrov. "Structured Sparsification of Gated Recurrent Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 4989–96. http://dx.doi.org/10.1609/aaai.v34i04.5938.
Full textAbstract:
One of the most popular approaches for neural network compression is sparsification — learning sparse weight matrices. In structured sparsification, weights are set to zero by groups corresponding to structure units, e. g. neurons. We further develop the structured sparsification approach for the gated recurrent neural networks, e. g. Long Short-Term Memory (LSTM). Specifically, in addition to the sparsification of individual weights and neurons, we propose sparsifying the preactivations of gates. This makes some gates constant and simplifies an LSTM structure. We test our approach on the text classification and language modeling tasks. Our method improves the neuron-wise compression of the model in most of the tasks. We also observe that the resulting structure of gate sparsity depends on the task and connect the learned structures to the specifics of the particular tasks.
13
Mao, Jiachen, Huanrui Yang, Ang Li, Hai Li, and Yiran Chen. "TPrune." ACM Transactions on Cyber-Physical Systems 5, no. 3 (July 2021): 1–22. http://dx.doi.org/10.1145/3446640.
Full textAbstract:
The invention of Transformer model structure boosts the performance of Neural Machine Translation (NMT) tasks to an unprecedented level. Many previous works have been done to make the Transformer model more execution-friendly on resource-constrained platforms. These researches can be categorized into three key fields: Model Pruning, Transfer Learning, and Efficient Transformer Variants. The family of model pruning methods are popular for their simplicity in practice and promising compression rate and have achieved great success in the field of convolution neural networks (CNNs) for many vision tasks. Nonetheless, previous Transformer pruning works did not perform a thorough model analysis and evaluation on each Transformer component on off-the-shelf mobile devices. In this work, we analyze and prune transformer models at the line-wise granularity and also implement our pruning method on real mobile platforms. We explore the properties of all Transformer components as well as their sparsity features, which are leveraged to guide Transformer model pruning. We name our whole Transformer analysis and pruning pipeline as TPrune. In TPrune, we first propose Block-wise Structured Sparsity Learning (BSSL) to analyze Transformer model property. Then, based on the characters derived from BSSL, we apply Structured Hoyer Square (SHS) to derive the final pruned models. Comparing with the state-of-the-art Transformer pruning methods, TPrune is able to achieve a higher model compression rate with less performance degradation. Experimental results show that our pruned models achieve 1.16×–1.92× speedup on mobile devices with 0%–8% BLEU score degradation compared with the original Transformer model.
14
Ravishankar, Saiprasad, Anna Ma, and Deanna Needell. "Analysis of fast structured dictionary learning." Information and Inference: A Journal of the IMA 9, no. 4 (November 19, 2019): 785–811. http://dx.doi.org/10.1093/imaiai/iaz028.
Full textAbstract:
Abstract Sparsity-based models and techniques have been exploited in many signal processing and imaging applications. Data-driven methods based on dictionary and sparsifying transform learning enable learning rich image features from data and can outperform analytical models. In particular, alternating optimization algorithms have been popular for learning such models. In this work, we focus on alternating minimization for a specific structured unitary sparsifying operator learning problem and provide a convergence analysis. While the algorithm converges to the critical points of the problem generally, our analysis establishes under mild assumptions, the local linear convergence of the algorithm to the underlying sparsifying model of the data. Analysis and numerical simulations show that our assumptions hold for standard probabilistic data models. In practice, the algorithm is robust to initialization.
15
Kuske, Jan, and Stefania Petra. "Performance Bounds For Co-/Sparse Box Constrained Signal Recovery." Analele Universitatii "Ovidius" Constanta - Seria Matematica 27, no. 1 (March 1, 2019): 79–106. http://dx.doi.org/10.2478/auom-2019-0005.
Full textAbstract:
Abstract The recovery of structured signals from a few linear measurements is a central point in both compressed sensing (CS) and discrete tomography. In CS the signal structure is described by means of a low complexity model e.g. co-/sparsity. The CS theory shows that any signal/image can be undersampled at a rate dependent on its intrinsic complexity. Moreover, in such undersampling regimes, the signal can be recovered by sparsity promoting convex regularization like ℓ1- or total variation (TV-) minimization. Precise relations between many low complexity measures and the sufficient number of random measurements are known for many sparsity promoting norms. However, a precise estimate of the undersampling rate for the TV seminorm is still lacking. We address this issue by: a) providing dual certificates testing uniqueness of a given cosparse signal with bounded signal values, b) approximating the undersampling rates via the statistical dimension of the TV descent cone and c) showing empirically that the provided rates also hold for tomographic measurements.
16
Zhang, Chengjin, Zehao Wang, Qiang An, Shiyong Li, Ahmad Hoorfar, and Chenxiao Kou. "Clustering-Driven DGS-Based Micro-Doppler Feature Extraction for Automatic Dynamic Hand Gesture Recognition." Sensors 22, no. 21 (November 5, 2022): 8535. http://dx.doi.org/10.3390/s22218535.
Full textAbstract:
We propose in this work a dynamic group sparsity (DGS) based time-frequency feature extraction method for dynamic hand gesture recognition (HGR) using millimeter-wave radar sensors. Micro-Doppler signatures of hand gestures show both sparse and structured characteristics in time-frequency domain, but previous study only focus on sparsity. We firstly introduce the structured prior when modeling the micro-Doppler signatures in this work to further enhance the features of hand gestures. The time-frequency distributions of dynamic hand gestures are first modeled using a dynamic group sparse model. A DGS-Subspace Pursuit (DGS-SP) algorithm is then utilized to extract the corresponding features. Finally, the support vector machine (SVM) classifier is employed to realize the dynamic HGR based on the extracted group sparse micro-Doppler features. The experiment shows that the proposed method achieved 3.3% recognition accuracy improvement over the sparsity-based method and has a better recognition accuracy than CNN based method in small dataset.
17
Xie, Zhonghua, Lingjun Liu, and Cui Yang. "An Entropy-Based Algorithm with Nonlocal Residual Learning for Image Compressive Sensing Recovery." Entropy 21, no. 9 (September 17, 2019): 900. http://dx.doi.org/10.3390/e21090900.
Full textAbstract:
Image recovery from compressive sensing (CS) measurement data, especially noisy data has always been challenging due to its implicit ill-posed nature, thus, to seek a domain where a signal can exhibit a high degree of sparsity and to design an effective algorithm have drawn increasingly more attention. Among various sparsity-based models, structured or group sparsity often leads to more powerful signal reconstruction techniques. In this paper, we propose a novel entropy-based algorithm for CS recovery to enhance image sparsity through learning the group sparsity of residual. To reduce the residual of similar packed patches, the group sparsity of residual is described by a Laplacian scale mixture (LSM) model, therefore, each singular value of the residual of similar packed patches is modeled as a Laplacian distribution with a variable scale parameter, to exploit the benefits of high-order dependency among sparse coefficients. Due to the latent variables, the maximum a posteriori (MAP) estimation of the sparse coefficients cannot be obtained, thus, we design a loss function for expectation–maximization (EM) method based on relative entropy. In the frame of EM iteration, the sparse coefficients can be estimated with the denoising-based approximate message passing (D-AMP) algorithm. Experimental results have shown that the proposed algorithm can significantly outperform existing CS techniques for image recovery.
18
Peng, Houwen, Bing Li, Rongrong Ji, Weiming Hu, Weihua Xiong, and Congyan Lang. "Salient Object Detection via Low-Rank and Structured Sparse Matrix Decomposition." Proceedings of the AAAI Conference on Artificial Intelligence 27, no. 1 (June 30, 2013): 796–802. http://dx.doi.org/10.1609/aaai.v27i1.8591.
Full textAbstract:
Salient object detection provides an alternative solution to various image semantic understanding tasks such as object recognition, adaptive compression and image retrieval. Recently, low-rank matrix recovery (LR) theory has been introduced into saliency detection, and achieves impressed results. However, the existing LR-based models neglect the underlying structure of images, and inevitably degrade the associated performance. In this paper, we propose a Low-rank and Structured sparse Matrix Decomposition (LSMD) model for salient object detection. In the model, a tree-structured sparsity-inducing norm regularization is firstly introduced to provide a hierarchical description of the image structure to ensure the completeness of the extracted salient object. The similarity of saliency values within the salient object is then guaranteed by the $\ell _\infty$-norm. Finally, high-level priors are integrated to guide the matrix decomposition and enhance the saliency detection. Experimental results on the largest public benchmark database show that our model outperforms existing LR-based approaches and other state-of-the-art methods, which verifies the effectiveness and robustness of the structure cues in our model.
19
Li, Zun, and Michael Wellman. "Structure Learning for Approximate Solution of Many-Player Games." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 02 (April 3, 2020): 2119–27. http://dx.doi.org/10.1609/aaai.v34i02.5586.
Full textAbstract:
Games with many players are difficult to solve or even specify without adopting structural assumptions that enable representation in compact form. Such structure is generally not given and will not hold exactly for particular games of interest. We introduce an iterative structure-learning approach to search for approximate solutions of many-player games, assuming only black-box simulation access to noisy payoff samples. Our first algorithm, K-Roles, exploits symmetry by learning a role assignment for players of the game through unsupervised learning (clustering) methods. Our second algorithm, G3L, seeks sparsity by greedy search over local interactions to learn a graphical game model. Both algorithms use supervised learning (regression) to fit payoff values to the learned structures, in compact representations that facilitate equilibrium calculation. We experimentally demonstrate the efficacy of both methods in reaching quality solutions and uncovering hidden structure, on both perfectly and approximately structured game instances.
20
TEWARSON, RP. "A review of computational techniques in flow Network models." MAUSAM 36, no. 4 (April 6, 2022): 441–46. http://dx.doi.org/10.54302/mausam.v36i4.2052.
Full textAbstract:
Several computational algorithms which have been found useful in the computer simulation of flow network problems are discussed. The emphasis is on accurate, fast and low-cost methods for handling large structured problems. In the case of underdetermined problems, the implementation of smoothing and related techniques that yield a class of desirable solutions is described. It is shown how the model structure has been utilized in the solution process to save computer storage and run time. How the accuracy of the numerical schemes has been improved by the use of splines without disturbing the sparsity structure of the given system is also discussed.
21
Bhandari, Ramesh, and Sharad Kumar Ghimire. "Performance Analysis of Structured Matrix Decomposition with Contour Based Spatial Prior for Extracting Salient Object from Complex Scene." Journal of the Institute of Engineering 15, no. 2 (July 31, 2019): 133–40. http://dx.doi.org/10.3126/jie.v15i2.27658.
Full textAbstract:
Automatically extracting most conspicuous object from an image is useful and important for many computer vision related tasks. Performance of several applications such as object segmentation, image classification based on salient object and content based image editing in computer vision can be improved using this technique. In this research work, performance of structured matrix decomposition with contour based spatial prior is analyzed for extracting salient object from the complex scene. To separate background and salient object, structured matrix decomposition model based on low rank matrix recovery theory is used along with two structural regularizations. Tree structured sparsity inducing regularization is used to capture image structure and to enforce the same object to assign similar saliency values. And, Laplacian regularization is used to enlarge the gap between background part and salient object part. In addition to structured matrix decomposition model, general high level priors along with biologically inspired contour based spatial prior is integrated to improve the performance of saliency related tasks. The performance of the proposed method is evaluated on two demanding datasets, namely, ICOSEG and PASCAL-S for complex scene images. For PASCAL-S dataset precision recall curve of proposed method starts from 0.81 and follows top and right-hand border more than structured matrix decomposition which starts from 0.79. Similarly, structural similarity index score, which is 0.596654 and 0.394864 without using contour based spatial prior and 0.720875 and 0.568001 using contour based spatial prior for ICOSEG and PASCAL-S datasets shows improved result.
22
Yang, Qing, Jiachen Mao, Zuoguan Wang, and “Helen” Li Hai. "Dynamic Regularization on Activation Sparsity for Neural Network Efficiency Improvement." ACM Journal on Emerging Technologies in Computing Systems 17, no. 4 (June 30, 2021): 1–16. http://dx.doi.org/10.1145/3447776.
Full textAbstract:
When deploying deep neural networks in embedded systems, it is crucial to decrease the model size and computational complexity for improving the execution speed and efficiency. In addition to conventional compression techniques, e.g., weight pruning and quantization, removing unimportant activations can also dramatically reduce the amount of data communication and the computation cost. Unlike weight parameters, the pattern of activations is directly related to input data and thereby changes dynamically. To regulate the dynamic activation sparsity (DAS), in this work, we propose a generic low-cost approach based on winners-take-all (WTA) dropout technique. The network enhanced by the proposed WTA dropout, namely DASNet , features structured activation sparsity with an improved sparsity level. Compared to the static feature map pruning methods, DASNets provide better computation cost reduction. The WTA dropout technique can be easily applied in deep neural networks without incurring additional training variables. More importantly, DASNet can be seamlessly integrated with other compression techniques, such as weight pruning and quantization, without compromising accuracy. Our experiments on various networks and datasets present significant runtime speedups with negligible accuracy losses.
23
Xu, Runxin, Fuli Luo, Chengyu Wang, Baobao Chang, Jun Huang, Songfang Huang, and Fei Huang. "From Dense to Sparse: Contrastive Pruning for Better Pre-trained Language Model Compression." Proceedings of the AAAI Conference on Artificial Intelligence 36, no. 10 (June 28, 2022): 11547–55. http://dx.doi.org/10.1609/aaai.v36i10.21408.
Full textAbstract:
Pre-trained Language Models (PLMs) have achieved great success in various Natural Language Processing (NLP) tasks under the pre-training and fine-tuning paradigm. With large quantities of parameters, PLMs are computation-intensive and resource-hungry. Hence, model pruning has been introduced to compress large-scale PLMs. However, most prior approaches only consider task-specific knowledge towards downstream tasks, but ignore the essential task-agnostic knowledge during pruning, which may cause catastrophic forgetting problem and lead to poor generalization ability. To maintain both task-agnostic and task-specific knowledge in our pruned model, we propose ContrAstive Pruning (CAP) under the paradigm of pre-training and fine-tuning. It is designed as a general framework, compatible with both structured and unstructured pruning. Unified in contrastive learn- ing, CAP enables the pruned model to learn from the pre-trained model for task-agnostic knowledge, and fine-tuned model for task-specific knowledge. Besides, to better retain the performance of the pruned model, the snapshots (i.e., the intermediate models at each pruning iteration) also serve as effective supervisions for pruning. Our extensive experiments show that adopting CAP consistently yields significant improvements, especially in extremely high sparsity scenarios. With only 3% model parameters reserved (i.e., 97% sparsity), CAP successfully achieves 99.2% and 96.3% of the original BERT performance in QQP and MNLI tasks. In addition, our probing experiments demonstrate that the model pruned by CAP tends to achieve better generalization ability.
24
Green, Spence, Marie-Catherine de Marneffe, and Christopher D. Manning. "Parsing Models for Identifying Multiword Expressions." Computational Linguistics 39, no. 1 (March 2013): 195–227. http://dx.doi.org/10.1162/coli_a_00139.
Full textAbstract:
Multiword expressions lie at the syntax/semantics interface and have motivated alternative theories of syntax like Construction Grammar. Until now, however, syntactic analysis and multiword expression identification have been modeled separately in natural language processing. We develop two structured prediction models for joint parsing and multiword expression identification. The first is based on context-free grammars and the second uses tree substitution grammars, a formalism that can store larger syntactic fragments. Our experiments show that both models can identify multiword expressions with much higher accuracy than a state-of-the-art system based on word co-occurrence statistics. We experiment with Arabic and French, which both have pervasive multiword expressions. Relative to English, they also have richer morphology, which induces lexical sparsity in finite corpora. To combat this sparsity, we develop a simple factored lexical representation for the context-free parsing model. Morphological analyses are automatically transformed into rich feature tags that are scored jointly with lexical items. This technique, which we call a factored lexicon, improves both standard parsing and multiword expression identification accuracy.
25
Neelamani, Ramesh (Neelsh), Christine E. Krohn, Jerry R. Krebs, Justin K. Romberg, Max Deffenbaugh, and John E. Anderson. "Efficient seismic forward modeling using simultaneous random sources and sparsity." GEOPHYSICS 75, no. 6 (November 2010): WB15—WB27. http://dx.doi.org/10.1190/1.3509470.
Full textAbstract:
The high cost of simulating densely sampled seismic forward modeling data arises from activating sources one at a time in sequence. To increase efficiency, one could leverage recent innovations in seismic field-data acquisition and activate several (e.g., 2–6) sources simultaneously during modeling. However, such approaches would suffer from degraded data quality because of the interference between the model’s responses to the simultaneous sources. Two new efficient simultaneous-source modeling approaches are proposed that rely on the novel tandem use of randomness and sparsity to construct almost noise-free model response to individual sources. In each approach, the first step is to measure the model’s cumulative response with all sources activated simultaneously using randomly scaled band-limited impulses or continuous band-limited random-noise waveforms. In the second step, the model response to each individual sourceis estimated from the cumulative receiver measurement by exploiting knowledge of the random source waveforms and the sparsity of the model response to individual sources in a known transform domain (e.g., curvelet domain). The efficiency achievable by the approaches is primarily governed by the sparsity of the model response. By invoking results from the field of compressive sensing, theoretical bounds are provided that assert that the approaches would need less modeling time for sparser (i.e., simpler or more structured) model responses. A simulated modeling example is illustrated that shows that data collected with as many as 8192 sources activated simultaneously can be separated into the 8192 individual source gathers with data quality comparable to that obtained when the sources were activated sequentially. The proposed approaches could also dramatically improve seismic field-data acquisition efficiency if the source signatures actually probing the earth can be measured accurately.
26
Wang, Li, Ali Mohammad-Djafari, Nicolas Gac, and Mircea Dumitru. "Bayesian 3D X-Ray Computed Tomography with a Hierarchical Prior Model for Sparsity in Haar Transform Domain." Entropy 20, no. 12 (December 16, 2018): 977. http://dx.doi.org/10.3390/e20120977.
Full textAbstract:
In this paper, a hierarchical prior model based on the Haar transformation and an appropriate Bayesian computational method for X-ray CT reconstruction are presented. Given the piece-wise continuous property of the object, a multilevel Haar transformation is used to associate a sparse representation for the object. The sparse structure is enforced via a generalized Student-t distribution ( S t g ), expressed as the marginal of a normal-inverse Gamma distribution. The proposed model and corresponding algorithm are designed to adapt to specific 3D data sizes and to be used in both medical and industrial Non-Destructive Testing (NDT) applications. In the proposed Bayesian method, a hierarchical structured prior model is proposed, and the parameters are iteratively estimated. The initialization of the iterative algorithm uses the parameters of the prior distributions. A novel strategy for the initialization is presented and proven experimentally. We compare the proposed method with two state-of-the-art approaches, showing that our method has better reconstruction performance when fewer projections are considered and when projections are acquired from limited angles.
27
Zhang, Kai Song, Luo Zhong, and Xuan Ya Zhang. "Image Restoration via Group l2,1 Norm-Based Structural Sparse Representation." International Journal of Pattern Recognition and Artificial Intelligence 32, no. 04 (December 13, 2017): 1854008. http://dx.doi.org/10.1142/s0218001418540083.
Full textAbstract:
Sparse representation has recently been extensively studied in the field of image restoration. Many sparsity-based approaches enforce sparse coding on patches with certain constraints. However, extracting structural information is a challenging task in the field image restoration. Motivated by the fact that structured sparse representation (SSR) method can capture the inner characteristics of image structures, which helps in finding sparse representations of nonlinear features or patterns, we propose the SSR approach for image restoration. Specifically, a generalized model is developed using structured restraint, namely, the group [Formula: see text]-norm of the coefficient matrix is introduced in the traditional sparse representation with respect to minimizing the differences within classes and maximizing the differences between classes for sparse representation, and its applications with image restoration are also explored. The sparse coefficients of SSR are obtained through iterative optimization approach. Experimental results have shown that the proposed SSR technique can significantly deliver the reconstructed images with high quality, which manifest the effectiveness of our approach in both peak signal-to-noise ratio performance and visual perception.
28
Xu, Caibin, Zhibo Yang, and Mingxi Deng. "Weighted Structured Sparse Reconstruction-Based Lamb Wave Imaging Exploiting Multipath Edge Reflections in an Isotropic Plate." Sensors 20, no. 12 (June 21, 2020): 3502. http://dx.doi.org/10.3390/s20123502.
Full textAbstract:
Lamb wave-based structural health monitoring techniques have the ability to scan a large area with relatively few sensors. Lamb wave imaging is a signal processing strategy that generates an image for locating scatterers according to the received Lamb waves. This paper presents a Lamb wave imaging method, which is formulated as a weighted structured sparse reconstruction problem. A dictionary is constructed by an analytical Lamb wave scattering model and an edge reflection prediction technique, which is used to decompose the experimental scattering signals under the constraint of weighted structured sparsity. The weights are generated from the correlation coefficients between the scattering signals and the predicted ones. Simulation and experimental results from an aluminum plate verify the effectiveness of the present method, which can generate images with sparse pixel values even with very limited number of sensors.
29
Wu, Yinjun, Jingchao Ni, Wei Cheng, Bo Zong, Dongjin Song, Zhengzhang Chen, Yanchi Liu, Xuchao Zhang, Haifeng Chen, and Susan B. Davidson. "Dynamic Gaussian Mixture based Deep Generative Model For Robust Forecasting on Sparse Multivariate Time Series." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 1 (May 18, 2021): 651–59. http://dx.doi.org/10.1609/aaai.v35i1.16145.
Full textAbstract:
Forecasting on sparse multivariate time series (MTS) aims to model the predictors of future values of time series given their incomplete past, which is important for many emerging applications. However, most existing methods process MTS’s individually, and do not leverage the dynamic distributions underlying the MTS’s, leading to sub-optimal results when the sparsity is high. To address this challenge, we propose a novel generative model, which tracks the transition of latent clusters, instead of isolated feature representations, to achieve robust modeling. It is characterized by a newly designed dynamic Gaussian mixture distribution, which captures the dynamics of clustering structures, and is used for emitting time series. The generative model is parameterized by neural networks. A structured inference network is also designed for enabling inductive analysis. A gating mechanism is further introduced to dynamically tune the Gaussian mixture distributions. Extensive experimental results on a variety of real-life datasets demonstrate the effectiveness of our method.
30
Sun, Xin, Zenghui Song, Junyu Dong, Yongbo Yu, Claudia Plant, and Christian Böhm. "Network Structure and Transfer Behaviors Embedding via Deep Prediction Model." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 5041–48. http://dx.doi.org/10.1609/aaai.v33i01.33015041.
Full textAbstract:
Network-structured data is becoming increasingly popular in many applications. However, these data present great challenges to feature engineering due to its high non-linearity and sparsity. The issue on how to transfer the link-connected nodes of the huge network into feature representations is critical. As basic properties of the real-world networks, the local and global structure can be reflected by dynamical transfer behaviors from node to node. In this work, we propose a deep embedding framework to preserve the transfer possibilities among the network nodes. We first suggest a degree-weight biased random walk model to capture the transfer behaviors of the network. Then a deep embedding framework is introduced to preserve the transfer possibilities among the nodes. A network structure embedding layer is added into the conventional Long Short-Term Memory Network to utilize its sequence prediction ability. To keep the local network neighborhood, we further perform a Laplacian supervised space optimization on the embedding feature representations. Experimental studies are conducted on various real-world datasets including social networks and citation networks. The results show that the learned representations can be effectively used as features in a variety of tasks, such as clustering, visualization and classification, and achieve promising performance compared with state-of-the-art models.
31
Zhao, Yue, and Jianbo Su. "New Sparse Facial Feature Description Model Based on Salience Evaluation of Regions and Features." International Journal of Pattern Recognition and Artificial Intelligence 29, no. 05 (July 9, 2015): 1556007. http://dx.doi.org/10.1142/s0218001415560078.
Full textAbstract:
Some regions (or blocks) and their affiliated features of face images are normally of more importance for face recognition. However, the variety of feature contributions, which exerts different saliency on recognition, is usually ignored. This paper proposes a new sparse facial feature description model based on salience evaluation of regions and features, which not only considers the contributions of different face regions, but also distinguishes that of different features in the same region. Specifically, the structured sparse learning scheme is employed as the salience evaluation method to encourage sparsity at both the group and individual levels for balancing regions and features. Therefore, the new facial feature description model is obtained by combining the salience evaluation method with region-based features. Experimental results show that the proposed model achieves better performance with much lower feature dimensionality.
32
Severa, William, Ojas Parekh, Conrad D. James, and James B. Aimone. "A Combinatorial Model for Dentate Gyrus Sparse Coding." Neural Computation 29, no. 1 (January 2017): 94–117. http://dx.doi.org/10.1162/neco_a_00905.
Full textAbstract:
The dentate gyrus forms a critical link between the entorhinal cortex and CA3 by providing a sparse version of the signal. Concurrent with this increase in sparsity, a widely accepted theory suggests the dentate gyrus performs pattern separation—similar inputs yield decorrelated outputs. Although an active region of study and theory, few logically rigorous arguments detail the dentate gyrus’s (DG) coding. We suggest a theoretically tractable, combinatorial model for this action. The model provides formal methods for a highly redundant, arbitrarily sparse, and decorrelated output signal.To explore the value of this model framework, we assess how suitable it is for two notable aspects of DG coding: how it can handle the highly structured grid cell representation in the input entorhinal cortex region and the presence of adult neurogenesis, which has been proposed to produce a heterogeneous code in the DG. We find tailoring the model to grid cell input yields expansion parameters consistent with the literature. In addition, the heterogeneous coding reflects activity gradation observed experimentally. Finally, we connect this approach with more conventional binary threshold neural circuit models via a formal embedding.
33
Kazemi, Nasser. "Automatic blind deconvolution with Toeplitz-structured sparse total least squares." GEOPHYSICS 83, no. 6 (November 1, 2018): V345—V357. http://dx.doi.org/10.1190/geo2018-0136.1.
Full textAbstract:
Given the noise-corrupted seismic recordings, blind deconvolution simultaneously solves for the reflectivity series and the wavelet. Blind deconvolution can be formulated as a fully perturbed linear regression model and solved by the total least-squares (TLS) algorithm. However, this algorithm performs poorly when the data matrix is a structured matrix and ill-conditioned. In blind deconvolution, the data matrix has a Toeplitz structure and is ill-conditioned. Accordingly, we develop a fully automatic single-channel blind-deconvolution algorithm to improve the performance of the TLS method. The proposed algorithm, called Toeplitz-structured sparse TLS, has no assumptions about the phase of the wavelet. However, it assumes that the reflectivity series is sparse. In addition, to reduce the model space and the number of unknowns, the algorithm benefits from the structural constraints on the data matrix. Our algorithm is an alternating minimization method and uses a generalized cross validation function to define the optimum regularization parameter automatically. Because the generalized cross validation function does not require any prior information about the noise level of the data, our approach is suitable for real-world applications. We validate the proposed technique using synthetic examples. In noise-free data, we achieve a near-optimal recovery of the wavelet and the reflectivity series. For noise-corrupted data with a moderate signal-to-noise ratio (S/N), we found that the algorithm successfully accounts for the noise in its model, resulting in a satisfactory performance. However, the results deteriorate as the S/N and the sparsity level of the data are decreased. We also successfully apply the algorithm to real data. The real-data examples come from 2D and 3D data sets of the Teapot Dome seismic survey.
34
Chang, David, Woo Suk Hong, and Richard Andrew Taylor. "Generating contextual embeddings for emergency department chief complaints." JAMIA Open 3, no. 2 (July 1, 2020): 160–66. http://dx.doi.org/10.1093/jamiaopen/ooaa022.
Full textAbstract:
Abstract Objective We learn contextual embeddings for emergency department (ED) chief complaints using Bidirectional Encoder Representations from Transformers (BERT), a state-of-the-art language model, to derive a compact and computationally useful representation for free-text chief complaints. Materials and methods Retrospective data on 2.1 million adult and pediatric ED visits was obtained from a large healthcare system covering the period of March 2013 to July 2019. A total of 355 497 (16.4%) visits from 65 737 (8.9%) patients were removed for absence of either a structured or unstructured chief complaint. To ensure adequate training set size, chief complaint labels that comprised less than 0.01%, or 1 in 10 000, of all visits were excluded. The cutoff threshold was incremented on a log scale to create seven datasets of decreasing sparsity. The classification task was to predict the provider-assigned label from the free-text chief complaint using BERT, with Long Short-Term Memory (LSTM) and Embeddings from Language Models (ELMo) as baselines. Performance was measured as the Top-k accuracy from k = 1:5 on a hold-out test set comprising 5% of the samples. The embedding for each free-text chief complaint was extracted as the final 768-dimensional layer of the BERT model and visualized using t-distributed stochastic neighbor embedding (t-SNE). Results The models achieved increasing performance with datasets of decreasing sparsity, with BERT outperforming both LSTM and ELMo. The BERT model yielded Top-1 accuracies of 0.65 and 0.69, Top-3 accuracies of 0.87 and 0.90, and Top-5 accuracies of 0.92 and 0.94 on datasets comprised of 434 and 188 labels, respectively. Visualization using t-SNE mapped the learned embeddings in a clinically meaningful way, with related concepts embedded close to each other and broader types of chief complaints clustered together. Discussion Despite the inherent noise in the chief complaint label space, the model was able to learn a rich representation of chief complaints and generate reasonable predictions of their labels. The learned embeddings accurately predict provider-assigned chief complaint labels and map semantically similar chief complaints to nearby points in vector space. Conclusion Such a model may be used to automatically map free-text chief complaints to structured fields and to assist the development of a standardized, data-driven ontology of chief complaints for healthcare institutions.
35
Aghamiry, Hossein S., and Ali Gholami. "Interval-Q estimation and compensation: An adaptive dictionary-learning approach." GEOPHYSICS 83, no. 4 (July 1, 2018): V233—V242. http://dx.doi.org/10.1190/geo2017-0001.1.
Full textAbstract:
A seismic trace corresponding to an anelastic layered earth model can be sparsely represented by a structured dictionary of properly attenuated wavelets, through a nonstationary sparse deconvolution. The sparseness of the coefficients (reflectivity series), however, considerably decreases when the wavelets are incorrectly modeled due to an incorrect [Formula: see text] model. Mathematically, the wavelets, as the elements of the dictionary, are nonlinearly related to the earth quality factor [Formula: see text] (the inverse of the attenuation coefficient). A parametric dictionary-learning strategy enables interval-[Formula: see text] estimation and compensation by training the dictionary atoms from the input trace adaptively to provide a sparse representation of it. We assumed a piecewise [Formula: see text] model by dividing the dictionary elements into several groups, each containing several wavelets whose temporal supports are close to each other and can be described by a single [Formula: see text]-value. The dictionary is learned iteratively where at each iteration only one group of the wavelets is optimized by searching for the corresponding optimum [Formula: see text]-value, leading to an iterative construction of the [Formula: see text] model. The main advantages of our method for interval-[Formula: see text] estimation are its stability because it performs in a forward-modeling manner and its accuracy because of the resolved interferences by the sparsity constraint. Our method is tested on synthetic and field data sets, and the results that we obtained demonstrate the stability and accuracy of the method for interval-[Formula: see text] estimation and compensation.
36
Zhao, Liang, Yuyang Gao, Jieping Ye, Feng Chen, Yanfang Ye, Chang-Tien Lu, and Naren Ramakrishnan. "Spatio-Temporal Event Forecasting Using Incremental Multi-Source Feature Learning." ACM Transactions on Knowledge Discovery from Data 16, no. 2 (April 30, 2022): 1–28. http://dx.doi.org/10.1145/3464976.
Full textAbstract:
The forecasting of significant societal events such as civil unrest and economic crisis is an interesting and challenging problem which requires both timeliness, precision, and comprehensiveness. Significant societal events are influenced and indicated jointly by multiple aspects of a society, including its economics, politics, and culture. Traditional forecasting methods based on a single data source find it hard to cover all these aspects comprehensively, thus limiting model performance. Multi-source event forecasting has proven promising but still suffers from several challenges, including (1) geographical hierarchies in multi-source data features, (2) hierarchical missing values, (3) characterization of structured feature sparsity, and (4) difficulty in model’s online update with incomplete multiple sources. This article proposes a novel feature learning model that concurrently addresses all the above challenges. Specifically, given multi-source data from different geographical levels, we design a new forecasting model by characterizing the lower-level features’ dependence on higher-level features. To handle the correlations amidst structured feature sets and deal with missing values among the coupled features, we propose a novel feature learning model based on an N th-order strong hierarchy and fused-overlapping group Lasso. An efficient algorithm is developed to optimize model parameters and ensure global optima. More importantly, to enable the model update in real time, the online learning algorithm is formulated and active set techniques are leveraged to resolve the crucial challenge when new patterns of missing features appear in real time. Extensive experiments on 10 datasets in different domains demonstrate the effectiveness and efficiency of the proposed models.
37
Yang, Li, Zhezhi He, and Deliang Fan. "Harmonious Coexistence of Structured Weight Pruning and Ternarization for Deep Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 6623–30. http://dx.doi.org/10.1609/aaai.v34i04.6138.
Full textAbstract:
Deep convolutional neural network (DNN) has demonstrated phenomenal success and been widely used in many computer vision tasks. However, its enormous model size and high computing complexity prohibits its wide deployment into resource limited embedded system, such as FPGA and mGPU. As the two most widely adopted model compression techniques, weight pruning and quantization compress DNN model through introducing weight sparsity (i.e., forcing partial weights as zeros) and quantizing weights into limited bit-width values, respectively. Although there are works attempting to combine the weight pruning and quantization, we still observe disharmony between weight pruning and quantization, especially when more aggressive compression schemes (e.g., Structured pruning and low bit-width quantization) are used. In this work, taking FPGA as the test computing platform and Processing Elements (PE) as the basic parallel computing unit, we first propose a PE-wise structured pruning scheme, which introduces weight sparsification with considering of the architecture of PE. In addition, we integrate it with an optimized weight ternarization approach which quantizes weights into ternary values ({-1,0,+1}), thus converting the dominant convolution operations in DNN from multiplication-and-accumulation (MAC) to addition-only, as well as compressing the original model (from 32-bit floating point to 2-bit ternary representation) by at least 16 times. Then, we investigate and solve the coexistence issue between PE-wise Structured pruning and ternarization, through proposing a Weight Penalty Clipping (WPC) technique with self-adapting threshold. Our experiment shows that the fusion of our proposed techniques can achieve the best state-of-the-art ∼21× PE-wise structured compression rate with merely 1.74%/0.94% (top-1/top-5) accuracy degradation of ResNet-18 on ImageNet dataset.
38
Wang, Lei, Wei Kong, and Shuaiqun Wang. "Detecting genetic associations with brain imaging phenotypes in Alzheimer’s disease via a novel structured KCCA approach." Journal of Bioinformatics and Computational Biology 19, no. 04 (May 4, 2021): 2150012. http://dx.doi.org/10.1142/s0219720021500128.
Full textAbstract:
Neuroimaging genetics has become an important research topic since it can reveal complex associations between genetic variants (i.e. single nucleotide polymorphisms (SNPs) and the structures or functions of the human brain. However, existing kernel mapping is difficult to directly use the sparse representation method in the kernel feature space, which makes it difficult for most existing sparse canonical correlation analysis (SCCA) methods to be directly promoted in the kernel feature space. To bridge this gap, we adopt a novel alternating projected gradient approach, gradient KCCA (gradKCCA) model to develop a powerful model for exploring the intrinsic associations among genetic markers, imaging quantitative traits (QTs) of interest. Specifically, this model solves kernel canonical correlation (KCCA) with an additional constraint that projection directions have pre-images in the original data space, a sparsity-inducing variant of the model is achieved through controlling the [Formula: see text]-norm of the preimages of the projection directions. We evaluate this model using Alzheimer’s disease Neuroimaging Initiative (ADNI) cohort to discover the relationships among SNPs from Alzheimer’s disease (AD) risk gene APOE, imaging QTs extracted from structural magnetic resonance imaging (MRI) scans. Our results show that the algorithm not only outperforms the traditional KCCA method in terms of Root Mean Square Error (RMSE) and Correlation Coefficient (CC) but also identify the meaningful and relevant biomarkers of SNPs (e.g. rs157594 and rs405697), which are positively related to right Postcentral and right SupraMarginal brain regions in this study. Empirical results indicate its promising capability in revealing biologically meaningful neuroimaging genetics associations and improving the disease-related mechanistic understanding of AD.
39
Du, Yinhao, Kun Fan, Xi Lu, and Cen Wu. "Integrating Multi–Omics Data for Gene-Environment Interactions." BioTech 10, no. 1 (January 29, 2021): 3. http://dx.doi.org/10.3390/biotech10010003.
Full textAbstract:
Gene-environment (G×E) interaction is critical for understanding the genetic basis of complex disease beyond genetic and environment main effects. In addition to existing tools for interaction studies, penalized variable selection emerges as a promising alternative for dissecting G×E interactions. Despite the success, variable selection is limited in terms of accounting for multidimensional measurements. Published variable selection methods cannot accommodate structured sparsity in the framework of integrating multiomics data for disease outcomes. In this paper, we have developed a novel variable selection method in order to integrate multi-omics measurements in G×E interaction studies. Extensive studies have already revealed that analyzing omics data across multi-platforms is not only sensible biologically, but also resulting in improved identification and prediction performance. Our integrative model can efficiently pinpoint important regulators of gene expressions through sparse dimensionality reduction, and link the disease outcomes to multiple effects in the integrative G×E studies through accommodating a sparse bi-level structure. The simulation studies show the integrative model leads to better identification of G×E interactions and regulators than alternative methods. In two G×E lung cancer studies with high dimensional multi-omics data, the integrative model leads to an improved prediction and findings with important biological implications.
40
Pérez, Eduardo, Jan Kirchhof, Fabian Krieg, and Florian Römer. "Subsampling Approaches for Compressed Sensing with Ultrasound Arrays in Non-Destructive Testing." Sensors 20, no. 23 (November 25, 2020): 6734. http://dx.doi.org/10.3390/s20236734.
Full textAbstract:
Full Matrix Capture is a multi-channel data acquisition method which enables flexible, high resolution imaging using ultrasound arrays. However, the measurement time and data volume are increased considerably. Both of these costs can be circumvented via compressed sensing, which exploits prior knowledge of the underlying model and its sparsity to reduce the amount of data needed to produce a high resolution image. In order to design compression matrices that are physically realizable without sophisticated hardware constraints, structured subsampling patterns are designed and evaluated in this work. The design is based on the analysis of the Cramér–Rao Bound of a single scatterer in a homogeneous, isotropic medium. A numerical comparison of the point spread functions obtained with different compression matrices and the Fast Iterative Shrinkage/Thresholding Algorithm shows that the best performance is achieved when each transmit event can use a different subset of receiving elements and each receiving element uses a different section of the echo signal spectrum. Such a design has the advantage of outperforming other structured patterns to the extent that suboptimal selection matrices provide a good performance and can be efficiently computed with greedy approaches.
41
Bi, Chuan-Xing, Feng-Min Zhang, Xiao-Zheng Zhang, Yong-Bin Zhang, and Rong Zhou. "Sound field reconstruction using block sparse Bayesian learning equivalent source method." Journal of the Acoustical Society of America 151, no. 4 (April 2022): 2378–90. http://dx.doi.org/10.1121/10.0010103.
Full textAbstract:
Nearfield acoustic holography based on the compressed sensing theory can realize the accurate reconstruction of sound fields with fewer measurement points on the premise that an appropriate sparse basis is obtained. However, for different types of sound sources, the appropriate sparse bases are diverse and should be constructed elaborately. In this paper, a block sparse Bayesian learning (SBL) equivalent source method is proposed for realizing the reconstruction of the sound fields radiated by different types of sources, including the spatially sparse sources, the spatially extended sources, and the mixed ones of the above two, without the elaborate construction of the sparse basis. The proposed method constructs a block sparse equivalent source model and promotes a block sparse solution by imposing a structured prior on the equivalent source model and estimating the posterior of the model by using the SBL, which can achieve the accurate reconstruction of the radiated sound fields of different types of sources simply by adjusting the block size. Numerical simulation and experimental results demonstrate the validity and superiority of the proposed method, and the effects of two key parameters, the block size, and sparsity pruning threshold value are investigated through simulations.
42
Li, Zhen, Baojun Zhao, and Wenzheng Wang. "An Efficient Spectral Feature Extraction Framework for Hyperspectral Images." Remote Sensing 12, no. 23 (December 4, 2020): 3967. http://dx.doi.org/10.3390/rs12233967.
Full textAbstract:
Extracting diverse spectral features from hyperspectral images has become a hot topic in recent years. However, these models are time consuming for training and test and suffer from a poor discriminative ability, resulting in low classification accuracy. In this paper, we design an effective feature extracting framework for the spectra of hyperspectral data. We construct a structured dictionary to encode spectral information and apply learning machine to map coding coefficients. To reduce training and testing time, the sparsity constraint is replaced by a block-diagonal constraint to accelerate the iteration, and an efficient extreme learning machine is employed to fit the spectral characteristics. To optimize the discriminative ability of our model, we first add spectral convolution to extract abundant spectral information. Then, we design shared constraints for subdictionaries so that the common features of subdictionaries can be expressed more effectively, and the discriminative and reconstructive ability of dictionary will be improved. The experimental results on diverse databases show that the proposed feature extraction framework can not only greatly reduce the training and testing time, but also lead to very competitive accuracy performance compared with deep learning models.
43
Liu, Zhao-Yang, and Sheng-Jun Huang. "Active Sampling for Open-Set Classification without Initial Annotation." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 4416–23. http://dx.doi.org/10.1609/aaai.v33i01.33014416.
Full textAbstract:
Open-set classification is a common problem in many real world tasks, where data is collected for known classes, and some novel classes occur at the test stage. In this paper, we focus on a more challenging case where the data examples collected for known classes are all unlabeled. Due to the high cost of label annotation, it is rather important to train a model with least labeled data for both accurate classification on known classes and effective detection of novel classes. Firstly, we propose an active learning method by incorporating structured sparsity with diversity to select representative examples for annotation. Then a latent low-rank representation is employed to simultaneously perform classification and novel class detection. Also, the method along with a fast optimization solution is extended to a multi-stage scenario, where classes occur and disappear in batches at each stage. Experimental results on multiple datasets validate the superiority of the proposed method with regard to different performance measures.
44
Ma, Yuekun, Yun Liu, Dezheng Zhang, Jiye Zhang, He Liu, and Yonghong Xie. "A Multigranularity Text Driven Named Entity Recognition CGAN Model for Traditional Chinese Medicine Literatures." Computational Intelligence and Neuroscience 2022 (September 24, 2022): 1–11. http://dx.doi.org/10.1155/2022/1495841.
Full textAbstract:
Recognition of Traditional Chinese Medicine (TCM) entities from different types of literature is challenging research, which is the foundation for extracting a large amount of TCM knowledge existing in unstructured texts into structured formats. The lack of large-scale annotated data makes unsatisfactory application of conventional deep learning models in TCM text knowledge extraction. Some other unsupervised methods rely on other auxiliary data, such as domain dictionaries. We propose a multigranularity text-driven NER model based on Conditional Generation Adversarial Network (MT-CGAN) to implement TCM NER with small-scale annotated corpus. In the model, a multigranularity text features encoder (MTFE) is designed to extract rich semantic and grammatical information from multiple dimensions of TCM texts. By differentiating the conditional constraints of the generator and discriminator of MT-CGAN, the synchronization between the generated tag labs and the named entities is guaranteed. Furthermore, seeds of different TCM text types are introduced into our model to improve the precision of NER. We compare our method with other baseline methods to illustrate the effectiveness of our method on 4 kinds of gold-standard datasets. The experiment results show that the standard precision, recall, and F1 score of our method are higher than the state-of-the-art methods by 0.24∼8.97%, 0.89∼12.74%, and 0.01∼10.84%. MT-CGAN is able to extract entities from different types of TCM literature effectively. Our experimental results indicate that the proposed approach has a clear advantage in processing TCM texts with more entity types, higher sparsity, less regular features, and a small-scale corpus.
45
Lu, Lyujian, Saad Elbeleidy, Lauren Zoe Baker, and Hua Wang. "Learning Multi-Modal Biomarker Representations via Globally Aligned Longitudinal Enrichments." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 01 (April 3, 2020): 817–24. http://dx.doi.org/10.1609/aaai.v34i01.5426.
Full textAbstract:
Alzheimer's Disease (AD) is a chronic neurodegenerative disease that severely impacts patients' thinking, memory and behavior. To aid automatic AD diagnoses, many longitudinal learning models have been proposed to predict clinical outcomes and/or disease status, which, though, often fail to consider missing temporal phenotypic records of the patients that can convey valuable information of AD progressions. Another challenge in AD studies is how to integrate heterogeneous genotypic and phenotypic biomarkers to improve diagnosis prediction. To cope with these challenges, in this paper we propose a longitudinal multi-modal method to learn enriched genotypic and phenotypic biomarker representations in the format of fixed-length vectors that can simultaneously capture the baseline neuroimaging measurements of the entire dataset and progressive variations of the varied counts of follow-up measurements over time of every participant from different biomarker sources. The learned global and local projections are aligned by a soft constraint and the structured-sparsity norm is used to uncover the multi-modal structure of heterogeneous biomarker measurements. While the proposed objective is clearly motivated to characterize the progressive information of AD developments, it is a nonsmooth objective that is difficult to efficiently optimize in general. Thus, we derive an efficient iterative algorithm, whose convergence is rigorously guaranteed in mathematics. We have conducted extensive experiments on the Alzheimer's Disease Neuroimaging Initiative (ADNI) data using one genotypic and two phenotypic biomarkers. Empirical results have demonstrated that the learned enriched biomarker representations are more effective in predicting the outcomes of various cognitive assessments. Moreover, our model has successfully identified disease-relevant biomarkers supported by existing medical findings that additionally warrant the correctness of our method from the clinical perspective.
46
Sarkar, Arindam, Nikhil Mehta, and Piyush Rai. "Graph Representation Learning via Ladder Gamma Variational Autoencoders." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 5604–11. http://dx.doi.org/10.1609/aaai.v34i04.6013.
Full textAbstract:
We present a probabilistic framework for community discovery and link prediction for graph-structured data, based on a novel, gamma ladder variational autoencoder (VAE) architecture. We model each node in the graph via a deep hierarchy of gamma-distributed embeddings, and define each link probability via a nonlinear function of the bottom-most layer's embeddings of its associated nodes. In addition to leveraging the representational power of multiple layers of stochastic variables via the ladder VAE architecture, our framework offers the following benefits: (1) Unlike existing ladder VAE architectures based on real-valued latent variables, the gamma-distributed latent variables naturally result in non-negativity and sparsity of the learned embeddings, and facilitate their direct interpretation as membership of nodes into (possibly multiple) communities/topics; (2) A novel recognition model for our gamma ladder VAE architecture allows fast inference of node embeddings; and (3) The framework also extends naturally to incorporate node side information (features and/or labels). Our framework is also fairly modular and can leverage a wide variety of graph neural networks as the VAE encoder. We report both quantitative and qualitative results on several benchmark datasets and compare our model with several state-of-the-art methods.
47
Wang, Meiling, Xiaohai He, Zhao Zhang, Luping Liu, Linbo Qing, and Yan Liu. "Dual-process system based on mixed semantic fusion for Chinese medical knowledge-based question answering." Mathematical Biosciences and Engineering 20, no. 3 (2023): 4912–39. http://dx.doi.org/10.3934/mbe.2023228.
Full textAbstract:
<abstract><p>Chinese medical knowledge-based question answering (cMed-KBQA) is a vital component of the intelligence question-answering assignment. Its purpose is to enable the model to comprehend questions and then deduce the proper answer from the knowledge base. Previous methods solely considered how questions and knowledge base paths were represented, disregarding their significance. Due to entity and path sparsity, the performance of question and answer cannot be effectively enhanced. To address this challenge, this paper presents a structured methodology for the cMed-KBQA based on the cognitive science dual systems theory by synchronizing an observation stage (System 1) and an expressive reasoning stage (System 2). System 1 learns the question's representation and queries the associated simple path. Then System 2 retrieves complicated paths for the question from the knowledge base by using the simple path provided by System 1. Specifically, System 1 is implemented by the entity extraction module, entity linking module, simple path retrieval module, and simple path-matching model. Meanwhile, System 2 is performed by using the complex path retrieval module and complex path-matching model. The public CKBQA2019 and CKBQA2020 datasets were extensively studied to evaluate the suggested technique. Using the metric average F1-score, our model achieved 78.12% on CKBQA2019 and 86.60% on CKBQA2020.</p></abstract>
48
Sun, Le, Qihao Cheng, and Zhiguo Chen. "Hyperspectral Image Super-Resolution Method Based on Spectral Smoothing Prior and Tensor Tubal Row-Sparse Representation." Remote Sensing 14, no. 9 (April 29, 2022): 2142. http://dx.doi.org/10.3390/rs14092142.
Full textAbstract:
Due to the limited hardware conditions, hyperspectral image (HSI) has a low spatial resolution, while multispectral image (MSI) can gain higher spatial resolution. Therefore, derived from the idea of fusion, we reconstructed HSI with high spatial resolution and spectral resolution from HSI and MSI and put forward an HSI Super-Resolution model based on Spectral Smoothing prior and Tensor tubal row-sparse representation, termed SSTSR. Foremost, nonlocal priors are applied to refine the super-resolution task into reconstructing each nonlocal clustering tensor. Then per nonlocal cluster tensor is decomposed into two sub tensors under the tensor t-prodcut framework, one sub-tensor is called tersor dictionary and the other is called tensor coefficient. Meanwhile, in the process of dictionary learning and sparse coding, spectral smoothing constraint is imposed on the tensor dictionary, and L1,1,2 norm based tubal row-sparse regularizer is enforced on the tensor coefficient to enhance the structured sparsity. With this model, the spatial similarity and spectral similarity of the nonlocal cluster tensor are fully utilized. Finally, the alternating direction method of multipliers (ADMM) was employed to optimize the solution of our method. Experiments on three simulated datasets and one real dataset show that our approach is superior to many advanced HSI super-resolution methods.
49
Boutin, Victor, Angelo Franciosini, Franck Ruffier, and Laurent Perrinet. "Effect of Top-Down Connections in Hierarchical Sparse Coding." Neural Computation 32, no. 11 (November 2020): 2279–309. http://dx.doi.org/10.1162/neco_a_01325.
Full textAbstract:
Hierarchical sparse coding (HSC) is a powerful model to efficiently represent multidimensional, structured data such as images. The simplest solution to solve this computationally hard problem is to decompose it into independent layer-wise subproblems. However, neuroscientific evidence would suggest interconnecting these subproblems as in predictive coding (PC) theory, which adds top-down connections between consecutive layers. In this study, we introduce a new model, 2-layer sparse predictive coding (2L-SPC), to assess the impact of this interlayer feedback connection. In particular, the 2L-SPC is compared with a hierarchical Lasso (Hi-La) network made out of a sequence of independent Lasso layers. The 2L-SPC and a 2-layer Hi-La networks are trained on four different databases and with different sparsity parameters on each layer. First, we show that the overall prediction error generated by 2L-SPC is lower thanks to the feedback mechanism as it transfers prediction error between layers. Second, we demonstrate that the inference stage of the 2L-SPC is faster to converge and generates a refined representation in the second layer compared to the Hi-La model. Third, we show that the 2L-SPC top-down connection accelerates the learning process of the HSC problem. Finally, the analysis of the emerging dictionaries shows that the 2L-SPC features are more generic and present a larger spatial extension.
50
Li, Liangliang, Xianpeng Wang, Jinmei Shi, and Xiang Lan. "Real-Valued Weighted Subspace Fitting Algorithm for DOA Estimation with Block Sparse Recovery." Mathematical Problems in Engineering 2023 (January 12, 2023): 1–13. http://dx.doi.org/10.1155/2023/7199603.
Full textAbstract:
In this paper, the problem of direction-of-arrival (DOA) estimation for strictly noncircular sources under the condition of unknown mutual coupling is concerned, and then a robust real-valued weighted subspace fitting (WSF) algorithm is proposed via block sparse recovery. Inspired by noncircularity, the real-valued coupled extended array output with double array aperture is first structured via exploiting the real-valued conversion. Then, an efficient real-valued block extended sparse recovery model is constructed by performing the parameterized decoupling operation to avoid the unknown mutual coupling and noncircular phase effects. Thereafter, the WSF framework is investigated to recover the real-valued block sparse matrix, where the spectrum of real-valued NC MUSIC-like is utilized to design a weighted matrix for strengthening the solutions sparsity. Eventually, DOA estimation is achieved based on the support set of the reconstructed block sparse matrix. Owing to the combination of noncircularity, parametrized decoupling thought, and reweighted strategy, the proposed method not only effectively achieves high-precision estimation, but also efficiently reduces the computational complexity. Plenty of simulation results demonstrate the effectiveness and efficiency of the proposed method.