Academic literature on the topic 'Structured sparsity model'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the lists of relevant articles, books, theses, conference reports, and other scholarly sources on the topic 'Structured sparsity model.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Journal articles on the topic "Structured sparsity model"
1
Niu, Wei, Mengshu Sun, Zhengang Li, Jou-An Chen, Jiexiong Guan, Xipeng Shen, Yanzhi Wang, Sijia Liu, Xue Lin, and Bin Ren. "RT3D: Achieving Real-Time Execution of 3D Convolutional Neural Networks on Mobile Devices." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 10 (May 18, 2021): 9179–87. http://dx.doi.org/10.1609/aaai.v35i10.17108.
Full textAbstract:
Mobile devices are becoming an important carrier for deep learning tasks, as they are being equipped with powerful, high-end mobile CPUs and GPUs. However, it is still a challenging task to execute 3D Convolutional Neural Networks (CNNs) targeting for real-time performance, besides high inference accuracy. The reason is more complex model structure and higher model dimensionality overwhelm the available computation/storage resources on mobile devices. A natural way may be turning to deep learning weight pruning techniques. However, the direct generalization of existing 2D CNN weight pruning methods to 3D CNNs is not ideal for fully exploiting mobile parallelism while achieving high inference accuracy. This paper proposes RT3D, a model compression and mobile acceleration framework for 3D CNNs, seamlessly integrating neural network weight pruning and compiler code generation techniques. We propose and investigate two structured sparsity schemes i.e., the vanilla structured sparsity and kernel group structured (KGS) sparsity that are mobile acceleration friendly. The vanilla sparsity removes whole kernel groups, while KGS sparsity is a more fine-grained structured sparsity that enjoys higher flexibility while exploiting full on-device parallelism. We propose a reweighted regularization pruning algorithm to achieve the proposed sparsity schemes. The inference time speedup due to sparsity is approaching the pruning rate of the whole model FLOPs (floating point operations). RT3D demonstrates up to 29.1x speedup in end-to-end inference time comparing with current mobile frameworks supporting 3D CNNs, with moderate 1%~1.5% accuracy loss. The end-to-end inference time for 16 video frames could be within 150 ms, when executing representative C3D and R(2+1)D models on a cellphone. For the first time, real-time execution of 3D CNNs is achieved on off-the-shelf mobiles.
2
Sun, Jun, Qidong Chen, Jianan Sun, Tao Zhang, Wei Fang, and Xiaojun Wu. "Graph-structured multitask sparsity model for visual tracking." Information Sciences 486 (June 2019): 133–47. http://dx.doi.org/10.1016/j.ins.2019.02.043.
Full text
3
Ruan, Xiaofeng, Yufan Liu, Bing Li, Chunfeng Yuan, and Weiming Hu. "DPFPS: Dynamic and Progressive Filter Pruning for Compressing Convolutional Neural Networks from Scratch." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 3 (May 18, 2021): 2495–503. http://dx.doi.org/10.1609/aaai.v35i3.16351.
Full textAbstract:
Filter pruning is a commonly used method for compressing Convolutional Neural Networks (ConvNets), due to its friendly hardware supporting and flexibility. However, existing methods mostly need a cumbersome procedure, which brings many extra hyper-parameters and training epochs. This is because only using sparsity and pruning stages cannot obtain a satisfying performance. Besides, many works do not consider the difference of pruning ratio across different layers. To overcome these limitations, we propose a novel dynamic and progressive filter pruning (DPFPS) scheme that directly learns a structured sparsity network from Scratch. In particular, DPFPS imposes a new structured sparsity-inducing regularization specifically upon the expected pruning parameters in a dynamic sparsity manner. The dynamic sparsity scheme determines sparsity allocation ratios of different layers and a Taylor series based channel sensitivity criteria is presented to identify the expected pruning parameters. Moreover, we increase the structured sparsity-inducing penalty in a progressive manner. This helps the model to be sparse gradually instead of forcing the model to be sparse at the beginning. Our method solves the pruning ratio based optimization problem by an iterative soft-thresholding algorithm (ISTA) with dynamic sparsity. At the end of the training, we only need to remove the redundant parameters without other stages, such as fine-tuning. Extensive experimental results show that the proposed method is competitive with 11 state-of-the-art methods on both small-scale and large-scale datasets (i.e., CIFAR and ImageNet). Specifically, on ImageNet, we achieve a 44.97% pruning ratio of FLOPs by compressing ResNet-101, even with an increase of 0.12% Top-5 accuracy. Our pruned models and codes are released at https://github.com/taoxvzi/DPFPS.
4
Wu, Hao, Shu Li, Yingpin Chen, and Zhenming Peng. "Seismic impedance inversion using second-order overlapping group sparsity with A-ADMM." Journal of Geophysics and Engineering 17, no. 1 (November 22, 2019): 97–116. http://dx.doi.org/10.1093/jge/gxz094.
Full textAbstract:
Abstract The anisotropic total variation with overlapping group sparsity (ATV_OGS) regularisation term is an improvement on the anisotropic total variation (ATV) regularisation term. It has been employed successfully in seismic impedance inversion as it can enhance the boundary information and relieve the staircase effect by exploring the structured sparsity of seismic impedance. However, because ATV_OGS constrains only the structured sparsity of the impedance's first-order difference and ignores the structured sparsity of the second-order difference, the staircase effect still occurs in an inversion result based on ATV_OGS. To further fit the structured sparsity of the impedance's second-order gradients, we introduce the overlapping group sparsity into the second-order difference of the impedance and propose a novel second-order ATV with overlapping group sparsity (SATV_OGS) seismic impedance inversion method. The proposed method reduces the interference of the large amplitude noise and further mitigates the staircase effect of the ATV_OGS. Furthermore, the accelerated alternating direction method of multipliers (A-ADMM) framework applied to this novel method. It can increase the efficiency of inversion. The experiments are carried out on a general model data and field data. Based on the experimental results, the proposed method can obtain higher resolution impedance than some impedance inversion methods based on total variation.
5
Zhu, Zijiang, Junshan Li, Yi Hu, and Xiaoguang Deng. "Research on Age Estimation Algorithm Based on Structured Sparsity." International Journal of Pattern Recognition and Artificial Intelligence 33, no. 06 (April 21, 2019): 1956006. http://dx.doi.org/10.1142/s0218001419560068.
Full textAbstract:
In order to solve the inaccuracy of age estimation dataset and the imbalance of age distribution, this paper proposes an age estimation model based on the structured sparse learning. Firstly, the Multi-label representation of facial images is performed by age, and the age estimation model is trained by solving the model matrix. Finally, the correlation with all age labels is calculated according to the facial images and age estimation model to be tested, and the most correlated age is taken as the predicted age. This paper sets up a series of verification experiments, and analyzes the structured sparse age estimation model from several perspectives. The proposed algorithm has achieved good results in the evaluation of indexes such as the mean absolute error, accumulation index curve and convergence rate, and has designed the demo system to put the model into use. Facts prove that the age estimation model proposed in this paper may achieve a good estimation effect.
6
Zhang, Lingli. "Total variation with modified group sparsity for CT reconstruction under low SNR." Journal of X-Ray Science and Technology 29, no. 4 (July 27, 2021): 645–62. http://dx.doi.org/10.3233/xst-200833.
Full textAbstract:
BACKGROUND AND OBJECTIVE: Since the stair artifacts may affect non-destructive testing (NDT) and diagnosis in the later stage, an applicable model is desperately needed, which can deal with the stair artifacts and preserve the edges. However, the classical total variation (TV) algorithm only considers the sparsity of the gradient transformed image. The objective of this study is to introduce and test a new method based on group sparsity to address the low signal-to-noise ratio (SNR) problem. METHODS: This study proposes a weighted total variation with overlapping group sparsity model. This model combines the Gaussian kernel and overlapping group sparsity into TV model denoted as GOGS-TV, which considers the structure sparsity of the image to be reconstructed to deal with the stair artifacts. On one hand, TV is the accepted commercial algorithm, and it can work well in many situations. On the other hand, the Gaussian kernel can associate the points around each pixel. Quantitative assessments are implemented to verify this merit. RESULTS: Numerical simulations are performed to validate the presented method, compared with the classical simultaneous algebraic reconstruction technique (SART) and the state-of-the-art TV algorithm. It confirms the significantly improved SNR of the reconstruction images both in suppressing the noise and preserving the edges using new GOGS-TV model. CONCLUSIONS: The proposed GOGS-TV model demonstrates its advantages to reduce stair artifacts especially in low SNR reconstruction because this new model considers both the sparsity of the gradient image and the structured sparsity. Meanwhile, the Gaussian kernel is utilized as a weighted factor that can be adapted to the global distribution.
7
Ou, Weihua, and Wenjun Xiao. "Structured sparsity model with spatial similarity regularisation for semantic feature selection." International Journal of Advanced Media and Communication 7, no. 2 (2017): 138. http://dx.doi.org/10.1504/ijamc.2017.085941.
Full text
8
Xiao, Wenjun, and Weihua Ou. "Structured sparsity model with spatial similarity regularisation for semantic feature selection." International Journal of Advanced Media and Communication 7, no. 2 (2017): 138. http://dx.doi.org/10.1504/ijamc.2017.10006892.
Full text
9
Ma, Xiaolong, Fu-Ming Guo, Wei Niu, Xue Lin, Jian Tang, Kaisheng Ma, Bin Ren, and Yanzhi Wang. "PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-Time Execution on Mobile Devices." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 5117–24. http://dx.doi.org/10.1609/aaai.v34i04.5954.
Full textAbstract:
Model compression techniques on Deep Neural Network (DNN) have been widely acknowledged as an effective way to achieve acceleration on a variety of platforms, and DNN weight pruning is a straightforward and effective method. There are currently two mainstreams of pruning methods representing two extremes of pruning regularity: non-structured, fine-grained pruning can achieve high sparsity and accuracy, but is not hardware friendly; structured, coarse-grained pruning exploits hardware-efficient structures in pruning, but suffers from accuracy drop when the pruning rate is high. In this paper, we introduce PCONV, comprising a new sparsity dimension, – fine-grained pruning patterns inside the coarse-grained structures. PCONV comprises two types of sparsities, Sparse Convolution Patterns (SCP) which is generated from intra-convolution kernel pruning and connectivity sparsity generated from inter-convolution kernel pruning. Essentially, SCP enhances accuracy due to its special vision properties, and connectivity sparsity increases pruning rate while maintaining balanced workload on filter computation. To deploy PCONV, we develop a novel compiler-assisted DNN inference framework and execute PCONV models in real-time without accuracy compromise, which cannot be achieved in prior work. Our experimental results show that, PCONV outperforms three state-of-art end-to-end DNN frameworks, TensorFlow-Lite, TVM, and Alibaba Mobile Neural Network with speedup up to 39.2 ×, 11.4 ×, and 6.3 ×, respectively, with no accuracy loss. Mobile devices can achieve real-time inference on large-scale DNNs.
10
Javanmardi, Mohammadreza, Amir Hossein Farzaneh, and Xiaojun Qi. "A Robust Structured Tracker Using Local Deep Features." Electronics 9, no. 5 (May 20, 2020): 846. http://dx.doi.org/10.3390/electronics9050846.
Full textAbstract:
Deep features extracted from convolutional neural networks have been recently utilized in visual tracking to obtain a generic and semantic representation of target candidates. In this paper, we propose a robust structured tracker using local deep features (STLDF). This tracker exploits the deep features of local patches inside target candidates and sparsely represents them by a set of templates in the particle filter framework. The proposed STLDF utilizes a new optimization model, which employs a group-sparsity regularization term to adopt local and spatial information of the target candidates and attain the spatial layout structure among them. To solve the optimization model, we propose an efficient and fast numerical algorithm that consists of two subproblems with the close-form solutions. Different evaluations in terms of success and precision on the benchmarks of challenging image sequences (e.g., OTB50 and OTB100) demonstrate the superior performance of the STLDF against several state-of-the-art trackers.
Dissertations / Theses on the topic "Structured sparsity model"
1
Tillander, Annika. "Classification models for high-dimensional data with sparsity patterns." Doctoral thesis, Stockholms universitet, Statistiska institutionen, 2013. http://urn.kb.se/resolve?urn=urn:nbn:se:su:diva-95664.
Full textAbstract:
Today's high-throughput data collection devices, e.g. spectrometers and gene chips, create information in abundance. However, this poses serious statistical challenges, as the number of features is usually much larger than the number of observed units. Further, in this high-dimensional setting, only a small fraction of the features are likely to be informative for any specific project. In this thesis, three different approaches to the two-class supervised classification in this high-dimensional, low sample setting are considered. There are classifiers that are known to mitigate the issues of high-dimensionality, e.g. distance-based classifiers such as Naive Bayes. However, these classifiers are often computationally intensive and therefore less time-consuming for discrete data. Hence, continuous features are often transformed into discrete features. In the first paper, a discretization algorithm suitable for high-dimensional data is suggested and compared with other discretization approaches. Further, the effect of discretization on misclassification probability in high-dimensional setting is evaluated. Linear classifiers are more stable which motivate adjusting the linear discriminant procedure to high-dimensional setting. In the second paper, a two-stage estimation procedure of the inverse covariance matrix, applying Lasso-based regularization and Cuthill-McKee ordering is suggested. The estimation gives a block-diagonal approximation of the covariance matrix which in turn leads to an additive classifier. In the third paper, an asymptotic framework that represents sparse and weak block models is derived and a technique for block-wise feature selection is proposed. Probabilistic classifiers have the advantage of providing the probability of membership in each class for new observations rather than simply assigning to a class. In the fourth paper, a method is developed for constructing a Bayesian predictive classifier. Given the block-diagonal covariance matrix, the resulting Bayesian predictive and marginal classifier provides an efficient solution to the high-dimensional problem by splitting it into smaller tractable problems. The relevance and benefits of the proposed methods are illustrated using both simulated and real data.Med dagens teknik, till exempel spektrometer och genchips, alstras data i stora mängder. Detta överflöd av data är inte bara till fördel utan orsakar även vissa problem, vanligtvis är antalet variabler (p) betydligt fler än antalet observation (n). Detta ger så kallat högdimensionella data vilket kräver nya statistiska metoder, då de traditionella metoderna är utvecklade för den omvända situationen (p<n). Dessutom är det vanligtvis väldigt få av alla dessa variabler som är relevanta för något givet projekt och styrkan på informationen hos de relevanta variablerna är ofta svag. Därav brukar denna typ av data benämnas som gles och svag (sparse and weak). Vanligtvis brukar identifiering av de relevanta variablerna liknas vid att hitta en nål i en höstack. Denna avhandling tar upp tre olika sätt att klassificera i denna typ av högdimensionella data. Där klassificera innebär, att genom ha tillgång till ett dataset med både förklaringsvariabler och en utfallsvariabel, lära en funktion eller algoritm hur den skall kunna förutspå utfallsvariabeln baserat på endast förklaringsvariablerna. Den typ av riktiga data som används i avhandlingen är microarrays, det är cellprov som visar aktivitet hos generna i cellen. Målet med klassificeringen är att med hjälp av variationen i aktivitet hos de tusentals gener (förklaringsvariablerna) avgöra huruvida cellprovet kommer från cancervävnad eller normalvävnad (utfallsvariabeln). Det finns klassificeringsmetoder som kan hantera högdimensionella data men dessa är ofta beräkningsintensiva, därav fungera de ofta bättre för diskreta data. Genom att transformera kontinuerliga variabler till diskreta (diskretisera) kan beräkningstiden reduceras och göra klassificeringen mer effektiv. I avhandlingen studeras huruvida av diskretisering påverkar klassificeringens prediceringsnoggrannhet och en mycket effektiv diskretiseringsmetod för högdimensionella data föreslås. Linjära klassificeringsmetoder har fördelen att vara stabila. Nackdelen är att de kräver en inverterbar kovariansmatris och vilket kovariansmatrisen inte är för högdimensionella data. I avhandlingen föreslås ett sätt att skatta inversen för glesa kovariansmatriser med blockdiagonalmatris. Denna matris har dessutom fördelen att det leder till additiv klassificering vilket möjliggör att välja hela block av relevanta variabler. I avhandlingen presenteras även en metod för att identifiera och välja ut blocken. Det finns också probabilistiska klassificeringsmetoder som har fördelen att ge sannolikheten att tillhöra vardera av de möjliga utfallen för en observation, inte som de flesta andra klassificeringsmetoder som bara predicerar utfallet. I avhandlingen förslås en sådan Bayesiansk metod, givet den blockdiagonala matrisen och normalfördelade utfallsklasser. De i avhandlingen förslagna metodernas relevans och fördelar är visade genom att tillämpa dem på simulerade och riktiga högdimensionella data.
2
Vinyes, Marina. "Convex matrix sparsity for demixing with an application to graphical model structure estimation." Thesis, Paris Est, 2018. http://www.theses.fr/2018PESC1130/document.
Full textAbstract:
En apprentissage automatique on a pour but d'apprendre un modèle, à partir de données, qui soit capable de faire des prédictions sur des nouvelles données (pas explorées auparavant). Pour obtenir un modèle qui puisse se généraliser sur les nouvelles données, et éviter le sur-apprentissage, nous devons restreindre le modèle. Ces restrictions sont généralement une connaissance a priori de la structure du modèle. Les premières approches considérées dans la littérature sont la régularisation de Tikhonov et plus tard le Lasso pour induire de la parcimonie dans la solution. La parcimonie fait partie d'un concept fondamental en apprentissage automatique. Les modèles parcimonieux sont attrayants car ils offrent plus d'interprétabilité et une meilleure généralisation (en évitant le sur-apprentissage) en induisant un nombre réduit de paramètres dans le modèle. Au-delà de la parcimonie générale et dans de nombreux cas, les modèles sont structurellement contraints et ont une représentation simple de certains éléments fondamentaux, comme par exemple une collection de vecteurs, matrices ou tenseurs spécifiques. Ces éléments fondamentaux sont appelés atomes. Dans ce contexte, les normes atomiques fournissent un cadre général pour estimer ce type de modèles. périodes de modèles. Le but de cette thèse est d'utiliser le cadre de parcimonie convexe fourni par les normes atomiques pour étudier une forme de parcimonie matricielle. Tout d'abord, nous développons un algorithme efficace basé sur les méthodes de Frank-Wolfe et qui est particulièrement adapté pour résoudre des problèmes convexes régularisés par une norme atomique. Nous nous concentrons ensuite sur l'estimation de la structure des modèles graphiques gaussiens, où la structure du modèle est encodée dans la matrice de précision et nous étudions le cas avec des variables manquantes. Nous proposons une formulation convexe avec une approche algorithmique et fournissons un résultat théorique qui énonce les conditions nécessaires pour récupérer la structure souhaitée. Enfin, nous considérons le problème de démixage d'un signal en deux composantes ou plus via la minimisation d’une somme de normes ou de jauges, encodant chacune la structure a priori des composants à récupérer. En particulier, nous fournissons une garantie de récupération exacte dans le cadre sans bruit, basée sur des mesures d'incohérenceThe goal of machine learning is to learn a model from some data that will make accurate predictions on data that it has not seen before. In order to obtain a model that will generalize on new data, and avoid overfitting, we need to restrain the model. These restrictions are usually some a priori knowledge of the structure of the model. First considered approaches included a regularization, first ridge regression and later Lasso regularization for inducing sparsity in the solution. Sparsity, also known as parsimony, has emerged as a fundamental concept in machine learning. Parsimonious models are appealing since they provide more interpretability and better generalization (avoid overfitting) through the reduced number of parameters. Beyond general sparsity and in many cases, models are constrained structurally so they have a simple representation in terms of some fundamental elements, consisting for example of a collection of specific vectors, matrices or tensors. These fundamental elements are called atoms. In this context, atomic norms provide a general framework for estimating these sorts of models. The goal of this thesis is to use the framework of convex sparsity provided by atomic norms to study a form of matrix sparsity. First, we develop an efficient algorithm based on Frank-Wolfe methods that is particularly adapted to solve problems with an atomic norm regularization. Then, we focus on the structure estimation of Gaussian graphical models, where the structure of the graph is encoded in the precision matrix and study the case with unobserved variables. We propose a convex formulation with an algorithmic approach and provide a theoretical result that states necessary conditions for recovering the desired structure. Finally, we consider the problem of signal demixing into two or more components via the minimization of a sum of norms or gauges, encoding each a structural prior on the corresponding components to recover. In particular, we provide general exact recovery guarantees in the noiseless setting based on incoherence measures
3
Smith, Chandler B. "Sparsity Constrained Inverse Problems - Application to Vibration-based Structural Health Monitoring." ScholarWorks @ UVM, 2019. https://scholarworks.uvm.edu/graddis/1143.
Full textAbstract:
Vibration-based structural health monitoring (SHM) seeks to detect, quantify, locate, and prognosticate damage by processing vibration signals measured while the structure is operational. The basic premise of vibration-based SHM is that damage will affect the stiffness, mass or energy dissipation properties of the structure and in turn alter its measured dynamic characteristics. In order to make SHM a practical technology it is necessary to perform damage assessment using only a minimum number of permanently installed sensors. Deducing damage at unmeasured regions of the structural domain requires solving an inverse problem that is underdetermined and(or) ill-conditioned. In addition, the effects of local damage on global vibration response may be overshadowed by the effects of modelling error, environmental changes, sensor noise, and unmeasured excitation. These theoretical and practical challenges render the damage identification inverse problem ill-posed, and in some cases unsolvable with conventional inverse methods.
This dissertation proposes and tests a novel interpretation of the damage identification inverse problem. Since damage is inherently local and strictly reduces stiffness and(or) mass, the underdetermined inverse problem can be made uniquely solvable by either imposing sparsity or non-negativity on the solution space. The goal of this research is to leverage this concept in order to prove that damage identification can be performed in practical applications using significantly less measurements than conventional inverse methods require. This dissertation investigates two sparsity inducing methods, L1-norm optimization and the non-negative least squares, in their application to identifying damage from eigenvalues, a minimal sensor-based feature that results in an underdetermined inverse problem. This work presents necessary conditions for solution uniqueness and a method to quantify the bounds on the non-unique solution space. The proposed methods are investigated using a wide range of numerical simulations and validated using a four-story lab-scale frame and a full-scale 17 m long aluminum truss. The findings of this study suggest that leveraging the attributes of both L1-norm optimization and non-negative constrained least squares can provide significant improvement over their standalone applications and over other existing methods of damage detection.
4
Kim, Yookyung. "Compressed Sensing Reconstruction Using Structural Dependency Models." Diss., The University of Arizona, 2012. http://hdl.handle.net/10150/238613.
Full textAbstract:
Compressed sensing (CS) theory has demonstrated that sparse signals can be reconstructed from far fewer measurements than suggested by the Nyquist sampling theory. CS has received great attention recently as an alternative to the current paradigm of sampling followed by compression. Initial CS operated under the implicit assumption that the sparsity domain coefficients are independently distributed. Recent results, however, show that exploiting statistical dependencies in sparse signals improves the recovery performance of CS. This dissertation proposes model-based CS reconstruction techniques. Statistical dependency models for several CS problems are proposed and incorporated into different CS algorithms. These models allow incorporation of a priori information into the CS reconstruction problems. Firstly, we propose the use of a Bayes least squares-Gaussian scale mixtures (BLS-GSM) model for CS recovery of natural images. The BLS-GSM model is able to exploit dependencies inherent in wavelet coefficients. This model is incorporated into several recent CS algorithms. The resulting methods significantly reduce reconstruction errors and/or the number of measurements required to obtain a desired reconstruction quality, when compared to state-of-the-art model-based CS methods in the literature. The model-based CS reconstruction techniques are then extended to video. In addition to spatial dependencies, video sequences exhibit significant temporal dependencies as well. In this dissertation, a model for jointly exploiting spatial and temporal dependencies in video CS is also proposed. The proposed method enforces structural self-similarity of image blocks within each frame as well as across neighboring frames. By sparsely representing collections of similar blocks, dominant image structures are retained while noise and incoherent undersampling artifacts are eliminated. A new video CS algorithm which incorporates this model is then proposed. The proposed algorithm iterates between enforcement of the self-similarity model and consistency with measurements. By enforcing measurement consistency in residual domain, sparsity is increased and CS reconstruction performance is enhanced. The proposed approach exhibits superior subjective image quality and significantly improves peak-signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM).Finally, a model-based CS framework is proposed for super resolution (SR) reconstruction. The SR reconstruction is formulated as a CS problem and a self-similarity model is incorporated into the reconstruction. The proposed model enforces similarity of collections of blocks through shrinkage of their transform-domain coefficients. A sharpening operation is performed in transform domain to emphasize edge recovery. The proposed method is compared with state-of-the-art SR techniques and provides high-quality SR images, both quantitatively and subjectively.
5
McGrady, Christopher Dwain. "Linking Rheological and Processing Behavior to Molecular Structure in Sparsely-Branched Polyethylenes Using Constitutive Relationships." Diss., Virginia Tech, 2009. http://hdl.handle.net/10919/37924.
Full textAbstract:
This dissertation works towards the larger objective of identifying and assessing the key features of molecular structure that lead to desired polymer processing performance with an ultimate goal of being able to tailor-make specific macromolecules that yield the desired processing response. A series of eight well-characterized, high-density polyethylene (HDPE) resins, with varying degrees of sparse long chain branching (LCB) content, is used to study the effect of both LCB content and distribution on the rheological and commercial processing response using the Pom-pom constitutive relationship. A flow instability known as ductile failure in extensional flow required the development a novel technique known as encapsulation in order to carry out shear-free rheological characterization.
Ductile failure prevents the rheological measurement of transient stress growth at higher strains for certain strain-hardening materials. This reduces the accuracy of nonlinear parameters for constitutive equations fit from transient stress growth data, as well as their effectiveness in modeling extensionally driven processes such as film casting. An experimental technique to overcome ductile failure called encapsulation in which the material that undergoes ductile failure is surrounded by a resin that readily deforms homogeneously at higher strains is introduced. A simple parallel model is shown to calculate the viscosity of the core material.
The effect of sparse long chain branching, LCB, on the film-casting process is analyzed at various drawdown ratios. A full rheological characterization in both shear and shear-free flows is also presented. At low drawdown ratios, the low-density polyethylenes, LDPE, exhibited the least degree of necking at distances less than the HDPE frostline. The sparsely-branched HDPE resins films had similar final film-widths that were larger than those of the linear HDPE. As the drawdown ratio was increased, film width profiles separated based on branching level. Small amounts of LCB were found to reduce the amount of necking at intermediate drawdown ratios. At higher drawdown ratios, the sparsely-branched HDPE resins of lower LCB had content film-widths that mimicked that of the linear HDPE, while the sparsely-branched HDPE resins of higher LCB content retained a larger film width. Molecular structural analysis via the Pom-pom constitutive model suggested that branching that was distributed across a larger range of backbone lengths serve to improve resistance to necking. As the drawdown ratio increased, the length of the backbones dominating the response decreased, so that the linear chains were controlling the necking behavior of the sparsely-branched resins of lower LCB content while remaining in branched regime for higher LCB content HDPEs. Other processing variables such as shear viscosity magnitude, extrudate swell, and non-isothermal processing conditions were eliminated as contributing factors to the differences in the film width profile.
The effect of sparse long chain branching, LCB, on the shear step-strain relaxation modulus is analyzed using a series of eight well-characterized, high-density polyethylene (HDPE) resins. The motivation for this work is in assessing the ability of step-strain flows to provide specific information about a material's branching architecture. Fundamental to this goal is proving the validity of relaxation moduli data at times shorter than the onset of time-strain separability. Strains of 1% to 1250% are imposed on materials with LCB content ranging from zero to 3.33 LCB per 10,000 carbon atoms. All materials are observed to obey time-strain separation beyond some characteristic time, Ï k. The presence of LCB is observed to increase the value of Ï k relative to the linear resin. Furthermore, the amount of LCB content is seen to correlate positively with increasing Ï k. The behavior of the relaxation modulus at times shorter than Ï k is investigated by an analysis of the enhancement seen in the linear relaxation modulus, G0(t), as a function of strain and LCB content. This enhancement is seen to 1) increase with increasing strain in all resins, 2) be significantly larger in the sparsely-branched HDPE resins relative to the linear HDPE resin, and 3) increase in magnitude with increasing LCB content. The shape and smoothness of the damping function is investigated to rule out the presence of wall-slip and material rupture during testing. The finite rise time to impose the desired strain is carefully monitored and compared to the Rouse relaxation time of the linear HDPE resins studied. Sparse LCB is found to increase the magnitude of the relaxation modulus at short times relative to the linear resin. It is shown that these differences are due to variations in the material architecture, specifically LCB content, and not because of mechanical anomalies.Ph. D.
6
Yan, Enxu. "Sublinear-Time Learning and Inference for High-Dimensional Models." Research Showcase @ CMU, 2018. http://repository.cmu.edu/dissertations/1207.
Full textAbstract:
Across domains, the scale of data and complexity of models have both been increasing greatly in the recent years. For many models of interest, tractable learning and inference without access to expensive computational resources have become challenging. In this thesis, we approach efficient learning and inference through the leverage of sparse structures inherent in the learning objective, which allows us to develop algorithms sublinear in the size of parameters without compromising the accuracy of models. In particular, we address the following three questions for each problem of interest: (a) how to formulate model estimation as an optimization problem with tractable sparse structure, (b) how to efficiently, i.e. in sublinear time, search, maintain, and utilize the sparse structures during training and inference, (c) how to guarantee fast convergence of our optimization algorithm despite its greedy nature? By answering these questions, we develop state-of-the-art algorithms in varied domains. Specifically, in the extreme classification domain, we utilizes primal and dual sparse structures to develop greedy algorithms of complexity sublinear in the number of classes, which obtain state-of-the-art accuracies on several benchmark data sets with one or two orders of magnitude speedup over existing algorithms. We also apply the primal-dual-sparse theory to develop a state-of-the-art trimming algorithm for Deep Neural Networks, which sparsifies neuron connections of a DNN with a task-dependent theoretical guarantee, which results in models of smaller storage cost and faster inference speed. When it comes to structured prediction problems (i.e. graphical models) with inter-dependent outputs, we propose decomposition methods that exploit sparse messages to decompose a structured learning problem of large output domains into factorwise learning modules amenable to sublineartime optimization methods, leading to practically much faster alternatives to existing learning algorithms. The decomposition technique is especially effective when combined with search data structures, such as those for Maximum Inner-Product Search (MIPS), to improve the learning efficiency jointly. Last but not the least, we design novel convex estimators for a latent-variable model by reparameterizing it as a solution of sparse support in an exponentially high-dimensional space, and approximate it with a greedy algorithm, which yields the first polynomial-time approximation method for the Latent-Feature Models and Generalized Mixed Regression without restrictive data assumptions.
7
Rösmann, Christoph [Verfasser], Torsten [Akademischer Betreuer] Bertram, and Martin [Gutachter] Mönnigmann. "Time-optimal nonlinear model predictive control : Direct transcription methods with variable discretization and structural sparsity exploitation / Christoph Rösmann ; Gutachter: Martin Mönnigmann ; Betreuer: Torsten Bertram." Dortmund : Universitätsbibliothek Dortmund, 2019. http://d-nb.info/1199106364/34.
Full text
8
Roulet, Vincent. "On the geometry of optimization problems and their structure." Thesis, Paris Sciences et Lettres (ComUE), 2017. http://www.theses.fr/2017PSLEE069/document.
Full textAbstract:
Dans de nombreux domaines tels que l’apprentissage statistique, la recherche opérationnelle ou encore la conception de circuits, une tâche est modélisée par un jeu de paramètres que l’on cherche à optimiser pour prendre la meilleure décision possible. Mathématiquement, le problème revient à minimiser une fonction de l’objectif recherché par des algorithmes itératifs. Le développement de ces derniers dépend alors de la géométrie de la fonction ou de la structure du problème. Dans une première partie, cette thèse étudie comment l’acuité d’une fonction autour de ses minima peut être exploitée par le redémarrage d’algorithmes classiques. Les schémas optimaux sont présentés pour des problèmes convexes généraux. Ils nécessitent cependant une description complète de la fonction, ce qui est rarement disponible. Des stratégies adaptatives sont donc développées et prouvées être quasi-optimales. Une analyse spécifique est ensuite conduite pour les problèmes parcimonieux qui cherchent des représentations compressées des variables du problème. Leur géométrie conique sous-jacente, qui décrit l’acuité de la fonction de l’objectif, se révèle contrôler à la fois la performance statistique du problème et l’efficacité des procédures d’optimisation par une seule quantité. Une seconde partie est dédiée aux problèmes d’apprentissage statistique. Ceux-ci effectuent une analyse prédictive de données à l’aide d’un large nombre d’exemples. Une approche générique est présentée pour à la fois résoudre le problème de prédiction et le simplifier en groupant soit les variables, les exemples ou les tâches. Des méthodes algorithmiques systématiques sont développées en analysant la géométrie induite par une partition des données. Une analyse théorique est finalement conduite lorsque les variables sont groupées par analogie avec les méthodes parcimonieusesIn numerous fields such as machine learning, operational research or circuit design, a task is modeled by a set of parameters to be optimized in order to take the best possible decision. Formally, the problem amounts to minimize a function describing the desired objective with iterative algorithms. The development of these latter depends then on the characterization of the geometry of the function or the structure of the problem. In a first part, this thesis studies how sharpness of a function around its minimizers can be exploited by restarting classical algorithms. Optimal schemes are presented for general convex problems. They require however a complete description of the function that is rarely available. Adaptive strategies are therefore developed and shown to achieve nearly optimal rates. A specific analysis is then carried out for sparse problems that seek for compressed representation of the variables of the problem. Their underlying conic geometry, that describes sharpness of the objective, is shown to control both the statistical performance of the problem and the efficiency of dedicated optimization methods by a single quantity. A second part is dedicated to machine learning problems. These perform predictive analysis of data from large set of examples. A generic framework is presented to both solve the prediction problem and simplify it by grouping either features, samples or tasks. Systematic algorithmic approaches are developed by analyzing the geometry induced by partitions of the data. A theoretical analysis is then carried out for grouping features by analogy to sparse methods
9
Kolar, Mladen. "Uncovering Structure in High-Dimensions: Networks and Multi-task Learning Problems." Research Showcase @ CMU, 2013. http://repository.cmu.edu/dissertations/229.
Full textAbstract:
Extracting knowledge and providing insights into complex mechanisms underlying noisy high-dimensional data sets is of utmost importance in many scientific domains. Statistical modeling has become ubiquitous in the analysis of high dimensional functional data in search of better understanding of cognition mechanisms, in the exploration of large-scale gene regulatory networks in hope of developing drugs for lethal diseases, and in prediction of volatility in stock market in hope of beating the market. Statistical analysis in these high-dimensional data sets is possible only if an estimation procedure exploits hidden structures underlying data.
This thesis develops flexible estimation procedures with provable theoretical guarantees for uncovering unknown hidden structures underlying data generating process. Of particular interest are procedures that can be used on high dimensional data sets where the number of samples n is much smaller than the ambient dimension p. Learning in high-dimensions is difficult due to the curse of dimensionality, however, the special problem structure makes inference possible. Due to its importance for scientific discovery, we put emphasis on consistent structure recovery throughout the thesis. Particular focus is given to two important problems, semi-parametric estimation of networks and feature selection in multi-task learning.
10
Todeschini, Adrien. "Probabilistic and Bayesian nonparametric approaches for recommender systems and networks." Thesis, Bordeaux, 2016. http://www.theses.fr/2016BORD0237/document.
Full textAbstract:
Nous proposons deux nouvelles approches pour les systèmes de recommandation et les réseaux. Dans la première partie, nous donnons d’abord un aperçu sur les systèmes de recommandation avant de nous concentrer sur les approches de rang faible pour la complétion de matrice. En nous appuyant sur une approche probabiliste, nous proposons de nouvelles fonctions de pénalité sur les valeurs singulières de la matrice de rang faible. En exploitant une représentation de modèle de mélange de cette pénalité, nous montrons qu’un ensemble de variables latentes convenablement choisi permet de développer un algorithme espérance-maximisation afin d’obtenir un maximum a posteriori de la matrice de rang faible complétée. L’algorithme résultant est un algorithme à seuillage doux itératif qui adapte de manière itérative les coefficients de réduction associés aux valeurs singulières. L’algorithme est simple à mettre en œuvre et peut s’adapter à de grandes matrices. Nous fournissons des comparaisons numériques entre notre approche et de récentes alternatives montrant l’intérêt de l’approche proposée pour la complétion de matrice à rang faible. Dans la deuxième partie, nous présentons d’abord quelques prérequis sur l’approche bayésienne non paramétrique et en particulier sur les mesures complètement aléatoires et leur extension multivariée, les mesures complètement aléatoires composées. Nous proposons ensuite un nouveau modèle statistique pour les réseaux creux qui se structurent en communautés avec chevauchement. Le modèle est basé sur la représentation du graphe comme un processus ponctuel échangeable, et généralise naturellement des modèles probabilistes existants à structure en blocs avec chevauchement au régime creux. Notre construction s’appuie sur des vecteurs de mesures complètement aléatoires, et possède des paramètres interprétables, chaque nœud étant associé un vecteur représentant son niveau d’affiliation à certaines communautés latentes. Nous développons des méthodes pour simuler cette classe de graphes aléatoires, ainsi que pour effectuer l’inférence a posteriori. Nous montrons que l’approche proposée peut récupérer une structure interprétable à partir de deux réseaux du monde réel et peut gérer des graphes avec des milliers de nœuds et des dizaines de milliers de connectionsWe propose two novel approaches for recommender systems and networks. In the first part, we first give an overview of recommender systems and concentrate on the low-rank approaches for matrix completion. Building on a probabilistic approach, we propose novel penalty functions on the singular values of the low-rank matrix. By exploiting a mixture model representation of this penalty, we show that a suitably chosen set of latent variables enables to derive an expectation-maximization algorithm to obtain a maximum a posteriori estimate of the completed low-rank matrix. The resulting algorithm is an iterative soft-thresholded algorithm which iteratively adapts the shrinkage coefficients associated to the singular values. The algorithm is simple to implement and can scale to large matrices. We provide numerical comparisons between our approach and recent alternatives showing the interest of the proposed approach for low-rank matrix completion. In the second part, we first introduce some background on Bayesian nonparametrics and in particular on completely random measures (CRMs) and their multivariate extension, the compound CRMs. We then propose a novel statistical model for sparse networks with overlapping community structure. The model is based on representing the graph as an exchangeable point process, and naturally generalizes existing probabilistic models with overlapping block-structure to the sparse regime. Our construction builds on vectors of CRMs, and has interpretable parameters, each node being assigned a vector representing its level of affiliation to some latent communities. We develop methods for simulating this class of random graphs, as well as to perform posterior inference. We show that the proposed approach can recover interpretable structure from two real-world networks and can handle graphs with thousands of nodes and tens of thousands of edges
Book chapters on the topic "Structured sparsity model"
1
Karimi, Amir-Hossein, Julius von Kügelgen, Bernhard Schölkopf, and Isabel Valera. "Towards Causal Algorithmic Recourse." In xxAI - Beyond Explainable AI, 139–66. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-031-04083-2_8.
Full textAbstract:
AbstractAlgorithmic recourse is concerned with aiding individuals who are unfavorably treated by automated decision-making systems to overcome their hardship, by offering recommendations that would result in a more favorable prediction when acted upon. Such recourse actions are typically obtained through solving an optimization problem that minimizes changes to the individual’s feature vector, subject to various plausibility, diversity, and sparsity constraints. Whereas previous works offer solutions to the optimization problem in a variety of settings, they critically overlook real-world considerations pertaining to the environment in which recourse actions are performed.The present work emphasizes that changes to a subset of the individual’s attributes may have consequential down-stream effects on other attributes, thus making recourse a fundamcausal problem. Here, we model such considerations using the framework of structural causal models, and highlight pitfalls of not considering causal relations through examples and theory. Such insights allow us to reformulate the optimization problem to directly optimize for minimally-costly recourse over a space of feasible actions (in the form of causal interventions) rather than optimizing for minimally-distant “counterfactual explanations”. We offer both the optimization formulations and solutions to deterministic and probabilistic recourse, on an individualized and sub-population level, overcoming the steep assumptive requirements of offering recourse in general settings. Finally, using synthetic and semi-synthetic experiments based on the German Credit dataset, we demonstrate how such methods can be applied in practice under minimal causal assumptions.
2
Carson, Dean B., Doris A. Carson, Per Axelsson, Peter Sköld, and Gabriella Sköld. "Disruptions and Diversions: The Demographic Consequences of Natural Disasters in Sparsely Populated Areas." In The Demography of Disasters, 81–99. Cham: Springer International Publishing, 2020. http://dx.doi.org/10.1007/978-3-030-49920-4_5.
Full textAbstract:
Abstract The Eight Ds model (Carson and Carson 2014) explains the unique characteristics of human and economic geography for sparsely populated areas (SPAs) as disconnected, discontinuous, diverse, detailed, dynamic, distant, dependent and delicate. According to the model, SPAs are subject to dramatic changes in demographic characteristics that result from both identifiable black swan events and less apparent tipping points in longer-term processes of demographic change (Carson et al. 2011). The conceptual foundations for this assertion are clear. Populations in SPAs can experience large and long-term impacts on the overall demographic structure as a result of decisions by a relatively small number of people. High levels of migration and mobility cause constant shifts in the demographic profile and prime SPAs to adapt to many different demographic states (Carson and Carson 2014). The Northern Territory of Australia, for example, experienced previously unseen waves of pre-retirement aged migrants in the past decade or so (Martel et al. 2013) as evidence of detailed but important changes to past trends. However, while dramatic demographic changes are conceptually possible and occasionally observable, there have been few attempts to examine the conditions under which such changes are likely to occur or not to occur. This is an important question particularly in relation to black swan events such as natural disasters because effective disaster management policy and planning is at least partially dependent on understanding who is affected and in what ways (Bird et al. 2013).
3
Deng, Zhaoxian, and Zhiqiang Zeng. "Multi-View Subspace Clustering by Combining ℓ2,p-Norm and Multi-Rank Minimization of Tensors." In Frontiers in Artificial Intelligence and Applications. IOS Press, 2022. http://dx.doi.org/10.3233/faia220020.
Full textAbstract:
In this article, based on the self-represented multi-view subspace clustering framework, we propose a new clustering model. Based on the assumption that different features can be linearly represented by data mapped to different subspaces, multiview subspace learning methods take advantage of the complementary and consensus informations between various kind of views of the data can boost the clustering performance. We search for the tensor with the lowest rank and then extract the frontal slice of it to establish a well-structured affinity matrix. Based on the tensor singular value decomposition (t-SVD), our low-rank constraint can be achieved. We impose the ℓ2,p-norm to flexibly control the sparsity of the error matrix, making it more robust to noise, which will enhance the robustness of our clustering model. With combining ℓ2,p-norm and tensor multi-rank minimization, the proposed Multi-view Subspace Clustering(MVSC) model can effectively perform clustering with multiple data resources. We test our model on one real-world spoon dataset and several publicly availabe datasets. Extensive evaluation methods have proved that our model is effective and efficient.
4
Scheir, Peter, Peter Prettenhofer, Stefanie N. Lindstaedt, and Chiara Ghidini. "An Associative and Adaptive Network Model For Information Retrieval In The Semantic Web." In Advances in Semantic Web and Information Systems, 309–44. IGI Global, 2010. http://dx.doi.org/10.4018/978-1-60566-992-2.ch014.
Full textAbstract:
While it is agreed that semantic enrichment of resources would lead to better search results, at present the low coverage of resources on the web with semantic information presents a major hurdle in realizing the vision of search on the Semantic Web. To address this problem, this chapter investigates how to improve retrieval performance in settings where resources are sparsely annotated with semantic information. Techniques from soft computing are employed to find relevant material that was not originally annotated with the concepts used in a query. The authors present an associative retrieval model for the Semantic Web and evaluate if and to which extent the use of associative retrieval techniques increases retrieval performance. In addition, the authors present recent work on adapting the network structure based on relevance feedback by the user to further improve retrieval effectiveness. The evaluation of new retrieval paradigms - such as retrieval based on technology for the Semantic Web - presents an additional challenge since no off-the-shelf test corpora exist. Hence, this chapter gives a detailed description of the approach taken to evaluate the information retrieval service the authors have built.
5
Cui, Jin, Yifei Zou, and Siyuan Zhang. "Using MRNet to Predict Lunar Rock Categories Detected by Chang’e 5 Probe." In Advances in Transdisciplinary Engineering. IOS Press, 2022. http://dx.doi.org/10.3233/atde220491.
Full textAbstract:
China’s Chang’e 5 mission has been a remarkable success, with the Chang’e 5 lander traveling on the Oceanus Procellarum to collect images of the lunar surface. Over the past half century, people have brought back some lunar rock samples, but its quantity does not meet the need for research. Under current circumstances, people still mainly rely on the analysis of rocks on the lunar surface through the detection of lunar rover. The Oceanus Procellarum, chosen by Chang’e 5 mission, contains various kinds of rock species. Therefore, we first applied to the National Astronomical Observatories of the China under the Chinese Academy of Sciences for the Navigation and Terrain Camera (NaTeCam) of the lunar surface image, and established a lunar surface rock image data set CE5ROCK. The data set contains 100 images, which randomly divided into training, validation and test set. Experimental results show that the identification accuracy testing on convolutional neural network (CNN) models like AlexNet or MobileNet is about to 40.0%. In order to make full use of the global information in Moon images, this paper proposes the MRNet (MoonRockNet) network architecture. The encoding structure of the network uses VGG16 for feature extraction, and the decoding part adds dilated convolution and commonly used U-Net structure on the original VGG16 decoding structure, which is more conducive to identify more refined but more sparsely distributed types of lunar rocks. We have conducted extensive experiments on the established CE5ROCK data set, and the experimental results show that MRNet can achieve more accurate rock type identification, and outperform other existing mainstream algorithms in the identification performance.
Conference papers on the topic "Structured sparsity model"
1
Zhao, Chen, Jian Zhang, Siwei Ma, Ruiqin Xiong, and Wen Gao. "A dual structured-sparsity model for compressive-sensed video reconstruction." In 2015 Visual Communications and Image Processing (VCIP). IEEE, 2015. http://dx.doi.org/10.1109/vcip.2015.7457804.
Full text
2
Cai, Xiao, Feiping Nie, Weidong Cai, and Heng Huang. "New Graph Structured Sparsity Model for Multi-label Image Annotations." In 2013 IEEE International Conference on Computer Vision (ICCV). IEEE, 2013. http://dx.doi.org/10.1109/iccv.2013.104.
Full text
3
Feng, Fangchen, and Matthieu Kowalski. "Hybrid model and structured sparsity for under-determined convolutive audio source separation." In ICASSP 2014 - 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2014. http://dx.doi.org/10.1109/icassp.2014.6854893.
Full text
4
Wang, Zheng, Feiping Nie, Lai Tian, Rong Wang, and Xuelong Li. "Discriminative Feature Selection via A Structured Sparse Subspace Learning Module." In Twenty-Ninth International Joint Conference on Artificial Intelligence and Seventeenth Pacific Rim International Conference on Artificial Intelligence {IJCAI-PRICAI-20}. California: International Joint Conferences on Artificial Intelligence Organization, 2020. http://dx.doi.org/10.24963/ijcai.2020/416.
Full textAbstract:
In this paper, we first propose a novel Structured Sparse Subspace Learning S^3L module to address the long-standing subspace sparsity issue. Elicited by proposed module, we design a new discriminative feature selection method, named Subspace Sparsity Discriminant Feature Selection S^2DFS which enables the following new functionalities: 1) Proposed S^2DFS method directly joints trace ratio objective and structured sparse subspace constraint via L2,0-norm to learn a row-sparsity subspace, which improves the discriminability of model and overcomes the parameter-tuning trouble with comparison to the methods used L2,1-norm regularization; 2) An alternative iterative optimization algorithm based on the proposed S^3L module is presented to explicitly solve the proposed problem with a closed-form solution and strict convergence proof. To our best knowledge, such objective function and solver are first proposed in this paper, which provides a new though for the development of feature selection methods. Extensive experiments conducted on several high-dimensional datasets demonstrate the discriminability of selected features via S^2DFS with comparison to several related SOTA feature selection methods. Source matlab code: https://github.com/StevenWangNPU/L20-FS.
5
Xu, Jie, Cheng Deng, Xinbo Gao, Dinggang Shen, and Heng Huang. "Predicting Alzheimer's Disease Cognitive Assessment via Robust Low-Rank Structured Sparse Model." In Twenty-Sixth International Joint Conference on Artificial Intelligence. California: International Joint Conferences on Artificial Intelligence Organization, 2017. http://dx.doi.org/10.24963/ijcai.2017/542.
Full textAbstract:
Alzheimer's disease (AD) is a neurodegenerative disorder with slow onset, which could result in the deterioration of the duration of persistent neurological dysfunction. How to identify the informative longitudinal phenotypic neuroimaging markers and predict cognitive measures are crucial to recognize AD at early stage. Many existing models related imaging measures to cognitive status using regression models, but they did not take full consideration of the interaction between cognitive scores. In this paper, we propose a robust low-rank structured sparse regression method (RLSR) to address this issue. The proposed model simultaneously selects effective features and learns the underlying structure between cognitive scores by utilizing novel mixed structured sparsity inducing norms and low-rank approximation. In addition, an efficient algorithm is derived to solve the proposed non-smooth objective function with proved convergence. Empirical studies on cognitive data of the ADNI cohort demonstrate the superior performance of the proposed method.
6
Zhu, Xiaotian, Wengang Zhou, and Houqiang Li. "Improving Deep Neural Network Sparsity through Decorrelation Regularization." In Twenty-Seventh International Joint Conference on Artificial Intelligence {IJCAI-18}. California: International Joint Conferences on Artificial Intelligence Organization, 2018. http://dx.doi.org/10.24963/ijcai.2018/453.
Full textAbstract:
Modern deep learning models usually suffer high complexity in model size and computation when transplanted to resource constrained platforms. To this end, many works are dedicated to compressing deep neural networks. Adding group LASSO regularization is one of the most effective model compression methods since it generates structured sparse networks. We investigate the deep neural networks trained by group LASSO constraint and observe that even with strong sparsity regularization imposed, there still exists substantial filter correlation among the convolution filters, which is undesired for a compact neural network. We propose to suppress such correlation with a new kind of constraint called decorrelation regularization, which explicitly forces the network to learn a set of less correlated filters. The experiments on CIFAR10/100 and ILSVRC2012 datasets show that when combined our decorrelation regularization with group LASSO, the correlation between filters could be effectively weakened, which increases the sparsity of the resulting model and leads to better compressing performance.
7
Niu, Yue, and Hongjie Zhang. "A Self-Aggregated Hierarchical Topic Model for Short Texts." In 2nd International Conference on Machine Learning, IOT and Blockchain (MLIOB 2021). Academy and Industry Research Collaboration Center (AIRCC), 2021. http://dx.doi.org/10.5121/csit.2021.111212.
Full textAbstract:
With the growth of the internet, short texts such as tweets from Twitter, news titles from the RSS, or comments from Amazon have become very prevalent. Many tasks need to retrieve information hidden from the content of short texts. So ontology learning methods are proposed for retrieving structured information. Topic hierarchy is a typical ontology that consists of concepts and taxonomy relations between concepts. Current hierarchical topic models are not specially designed for short texts. These methods use word co-occurrence to construct concepts and general-special word relations to construct taxonomy topics. But in short texts, word cooccurrence is sparse and lacking general-special word relations. To overcome this two problems and provide an interpretable result, we designed a hierarchical topic model which aggregates short texts into long documents and constructing topics and relations. Because long documents add additional semantic information, our model can avoid the sparsity of word cooccurrence. In experiments, we measured the quality of concepts by topic coherence metric on four real-world short texts corpus. The result showed that our topic hierarchy is more interpretable than other methods.
8
Fan, Mingyu, Xiaojun Chang, Xiaoqin Zhang, Di Wang, and Liang Du. "Top-k Supervise Feature Selection via ADMM for Integer Programming." In Twenty-Sixth International Joint Conference on Artificial Intelligence. California: International Joint Conferences on Artificial Intelligence Organization, 2017. http://dx.doi.org/10.24963/ijcai.2017/228.
Full textAbstract:
Recently, structured sparsity inducing based feature selection has become a hot topic in machine learning and pattern recognition. Most of the sparsity inducing feature selection methods are designed to rank all features by certain criterion and then select the k top ranked features, where k is an integer. However, the k top features are usually not the top k features and therefore maybe a suboptimal result. In this paper, we propose a novel supervised feature selection method to directly identify the top k features. The new method is formulated as a classic regularized least squares regression model with two groups of variables. The problem with respect to one group of the variables turn out to be a 0-1 integer programming, which had been considered very hard to solve. To address this, we utilize an efficient optimization method to solve the integer programming, which first replaces the discrete 0-1 constraints with two continuous constraints and then utilizes the alternating direction method of multipliers to optimize the equivalent problem. The obtained result is the top subset with k features under the proposed criterion rather than the subset of k top features. Experiments have been conducted on benchmark data sets to show the effectiveness of proposed method.
9
Li, Yanyu, Pu Zhao, Geng Yuan, Xue Lin, Yanzhi Wang, and Xin Chen. "Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization." In Thirty-First International Joint Conference on Artificial Intelligence {IJCAI-22}. California: International Joint Conferences on Artificial Intelligence Organization, 2022. http://dx.doi.org/10.24963/ijcai.2022/449.
Full textAbstract:
Neural architecture search (NAS) and network pruning are widely studied efficient AI techniques, but not yet perfect. NAS performs exhaustive candidate architecture search, incurring tremendous search cost. Though (structured) pruning can simply shrink model dimension, it remains unclear how to decide the per-layer sparsity automatically and optimally. In this work, we revisit the problem of layer-width optimization and propose Pruning-as-Search (PaS), an end-to-end channel pruning method to search out desired sub-network automatically and efficiently. Specifically, we add a depth-wise binary convolution to learn pruning policies directly through gradient descent. By combining the structural reparameterization and PaS, we successfully searched out a new family of VGG-like and lightweight networks, which enable the flexibility of arbitrary width with respect to each layer instead of each stage. Experimental results show that our proposed architecture outperforms prior arts by around 1.0% top-1 accuracy under similar inference speed on ImageNet-1000 classification task. Furthermore, we demonstrate the effectiveness of our width search on complex tasks including instance segmentation and image translation. Code and models are released.
10
Liu, Huan, Qinghua Zheng, Minnan Luo, Dingwen Zhang, Xiaojun Chang, and Cheng Deng. "How Unlabeled Web Videos Help Complex Event Detection?" In Twenty-Sixth International Joint Conference on Artificial Intelligence. California: International Joint Conferences on Artificial Intelligence Organization, 2017. http://dx.doi.org/10.24963/ijcai.2017/564.
Full textAbstract:
The lack of labeled exemplars is an important factor that makes the task of multimedia event detection (MED) complicated and challenging. Utilizing artificially picked and labeled external sources is an effective way to enhance the performance of MED. However, building these data usually requires professional human annotators, and the procedure is too time-consuming and costly to scale. In this paper, we propose a new robust dictionary learning framework for complex event detection, which is able to handle both labeled and easy-to-get unlabeled web videos by sharing the same dictionary. By employing the lq-norm based loss jointly with the structured sparsity based regularization, our model shows strong robustness against the substantial noisy and outlier videos from open source. We exploit an effective optimization algorithm to solve the proposed highly non-smooth and non-convex problem. Extensive experiment results over standard datasets of TRECVID MEDTest 2013 and TRECVID MEDTest 2014 demonstrate the effectiveness and superiority of the proposed framework on complex event detection.
Reports on the topic "Structured sparsity model"
1
Yu, Guoshen, Guillermo Sapiro, and Stephane Mallat. Solving Inverse Problems with Piecewise Linear Estimators: From Gaussian Mixture Models to Structured Sparsity. Fort Belvoir, VA: Defense Technical Information Center, June 2010. http://dx.doi.org/10.21236/ada540722.
Full text
2
Rahmani, Mehran, Xintong Ji, and Sovann Reach Kiet. Damage Detection and Damage Localization in Bridges with Low-Density Instrumentations Using the Wave-Method: Application to a Shake-Table Tested Bridge. Mineta Transportation Institute, September 2022. http://dx.doi.org/10.31979/mti.2022.2033.
Full textAbstract:
This study presents a major development to the wave method, a methodology used for structural identification and monitoring. The research team tested the method for use in structural damage detection and damage localization in bridges, the latter being a challenging task. The main goal was to assess capability of the improved method by applying it to a shake-table-tested prototype bridge with sparse instrumentation. The bridge was a 4-span reinforced concrete structure comprising two columns at each bent (6 columns total) and a flat slab. It was tested to failure using seven biaxial excitations at its base. Availability of a robust and verified method, which can work with sparse recording stations, can be valuable for detecting damage in bridges soon after an earthquake. The proposed method in this study includes estimating the shear (cS) and the longitudinal (cL) wave velocities by fitting an equivalent uniform Timoshenko beam model in impulse response functions of the recorded acceleration response. The identification algorithm is enhanced by adding the model’s damping ratio to the unknown parameters, as well as performing the identification for a range of initial values to avoid early convergence to a local minimum. Finally, the research team detect damage in the bridge columns by monitoring trends in the identified shear wave velocities from one damaging event to another. A comprehensive comparison between the reductions in shear wave velocities and the actual observed damages in the bridge columns is presented. The results revealed that the reduction of cS is generally consistent with the observed distribution and severity of damage during each biaxial motion. At bents 1 and 3, cS is consistently reduced with the progression of damage. The trends correctly detected the onset of damage at bent 1 during biaxial 3, and damage in bent 3 during biaxial 4. The most significant reduction was caused by the last two biaxial motions in bents 1 and 3, also consistent with the surveyed damage. In bent 2 (middle bent), the reduction trend in cS was relatively minor, correctly showing minor damage at this bent. Based on these findings, the team concluded that the enhanced wave method presented in this study was capable of detecting damage in the bridge and identifying the location of the most severe damage. The proposed methodology is a fast and inexpensive tool for real-time or near real-time damage detection and localization in similar bridges, especially those with sparsely deployed accelerometers.