Journal articles on the topic 'CNN embedding networks'
To see the other types of publications on this topic, follow the link: CNN embedding networks.
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'CNN embedding networks.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
David, Merlin Susan, and Shini Renjith. "Comparison of word embeddings in text classification based on RNN and CNN." IOP Conference Series: Materials Science and Engineering 1187, no. 1 (September 1, 2021): 012029. http://dx.doi.org/10.1088/1757-899x/1187/1/012029.
Full textAbstract:
Abstract This paper presents a comparison of word embeddings in text classification using RNN and CNN. In the field of image classification, deep learning methods like as RNN and CNN have shown to be popular. CNN is most popular model among deep learning techniques in the field of NLP because of its simplicity and parallelism, even if the dataset is huge. Word embedding techniques employed are GloVe and fastText. Use of different word embeddings showed a major difference in the accuracy of the models. When it comes to embedding of rare words, GloVe can sometime perform poorly. Inorder to tackle this issue, fastText method is used. Deep neural networks with fastText showed a remarkable improvement in the accuracy than GloVe. But fastText took some time to train when compared to GloVe. Further, the accuracy was improved by minimizing the batch size. Finally we concluded that the word embeddings have a huge impact on the performance of text classification models
2
Rhanoui, Maryem, Mounia Mikram, Siham Yousfi, and Soukaina Barzali. "A CNN-BiLSTM Model for Document-Level Sentiment Analysis." Machine Learning and Knowledge Extraction 1, no. 3 (July 25, 2019): 832–47. http://dx.doi.org/10.3390/make1030048.
Full textAbstract:
Document-level sentiment analysis is a challenging task given the large size of the text, which leads to an abundance of words and opinions, at times contradictory, in the same document. This analysis is particularly useful in analyzing press articles and blog posts about a particular product or company, and it requires a high concentration, especially when the topic being discussed is sensitive. Nevertheless, most existing models and techniques are designed to process short text from social networks and collaborative platforms. In this paper, we propose a combination of Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) models, with Doc2vec embedding, suitable for opinion analysis in long texts. The CNN-BiLSTM model is compared with CNN, LSTM, BiLSTM and CNN-LSTM models with Word2vec/Doc2vec embeddings. The Doc2vec with CNN-BiLSTM model was applied on French newspapers articles and outperformed the other models with 90.66% accuracy.
3
Selvarajah, Jarashanth, and Ruwan Nawarathna. "Identifying Tweets with Personal Medication Intake Mentions using Attentive Character and Localized Context Representations." JUCS - Journal of Universal Computer Science 28, no. 12 (December 28, 2022): 1312–29. http://dx.doi.org/10.3897/jucs.84130.
Full textAbstract:
Individuals with health anomalies often share their experiences on social media sites, such as Twitter, which yields an abundance of data on a global scale. Nowadays, social media data constitutes a leading source to build drug monitoring and surveillance systems. However, a proper assessment of such data requires discarding mentions which do not express drug-related personal health experiences. We automate this process by introducing a novel deep learning model. The model includes character-level and word-level embeddings, embedding-level attention, convolu- tional neural networks (CNN), bidirectional gated recurrent units (BiGRU), and context-aware attentions. An embedding for a word is produced by integrating both word-level and character-level embeddings using an embedding-level attention mechanism, which selects the salient features from both embeddings without expanding dimensionality. The resultant embedding is further analyzed by three CNN layers independently, where each extracts unique n-grams. BiGRUs followed by attention layers further process the outputs from each CNN layer. Besides, the resultant embedding is also encoded by a BiGRU with attention. Our model is developed to cope with the intricate attributes inherent to tweets such as vernacular texts, descriptive medical phrases, frequently misspelt words, abbreviations, short messages, and others. All these four outputs are summed and sent to a softmax classifier. We built a dataset by incorporating tweets from two benchmark datasets designed for the same objective to evaluate the performance. Our model performs substantially better than existing models, including several customized Bidirectional Encoder Representations from Transformers (BERT) models with an F1-score of 0.772.
4
Tang, Weixuan, Bin Li, Shunquan Tan, Mauro Barni, and Jiwu Huang. "CNN-Based Adversarial Embedding for Image Steganography." IEEE Transactions on Information Forensics and Security 14, no. 8 (August 2019): 2074–87. http://dx.doi.org/10.1109/tifs.2019.2891237.
Full text
5
Zheng, Zhedong, Liang Zheng, and Yi Yang. "A Discriminatively Learned CNN Embedding for Person Reidentification." ACM Transactions on Multimedia Computing, Communications, and Applications 14, no. 1 (January 16, 2018): 1–20. http://dx.doi.org/10.1145/3159171.
Full text
6
Wang, Rong, Cong Tian, and Lin Yan. "Malware Detection Using CNN via Word Embedding in Cloud Computing Infrastructure." Scientific Programming 2021 (September 11, 2021): 1–7. http://dx.doi.org/10.1155/2021/8381550.
Full textAbstract:
The Internet of Things (IoT), cloud, and fog computing paradigms provide a powerful large-scale computing infrastructure for a variety of data and computation-intensive applications. These cutting-edge computing infrastructures, however, are nevertheless vulnerable to serious security and privacy risks. One of the most important countermeasures against cybersecurity threats is intrusion detection and prevention systems, which monitor devices, networks, and systems for malicious activity and policy violations. The detection and prevention systems range from antivirus software to hierarchical systems that monitor the traffic of whole backbone networks. At the moment, the primary defensive solutions are based on malware feature extraction. Most known feature extraction algorithms use byte N-gram patterns or binary strings to represent log files or other static information. The information taken from program files is expressed using word embedding (GloVe) and a new feature extraction method proposed in this article. As a result, the relevant vector space model (VSM) will incorporate more information about unknown programs. We utilize convolutional neural network (CNN) to analyze the feature maps represented by word embedding and apply Softmax to fit the probability of a malicious program. Eventually, we consider a program to be malicious if the probability is greater than 0.5; otherwise, it is a benign program. Experimental result shows that our approach achieves a level of accuracy higher than 98%.
7
Liu, Han, Jun Li, Lin He, and Yu Wang. "Superpixel-Guided Layer-Wise Embedding CNN for Remote Sensing Image Classification." Remote Sensing 11, no. 2 (January 17, 2019): 174. http://dx.doi.org/10.3390/rs11020174.
Full textAbstract:
Irregular spatial dependency is one of the major characteristics of remote sensing images, which brings about challenges for classification tasks. Deep supervised models such as convolutional neural networks (CNNs) have shown great capacity for remote sensing image classification. However, they generally require a huge labeled training set for the fine tuning of a deep neural network. To handle the irregular spatial dependency of remote sensing images and mitigate the conflict between limited labeled samples and training demand, we design a superpixel-guided layer-wise embedding CNN (SLE-CNN) for remote sensing image classification, which can efficiently exploit the information from both labeled and unlabeled samples. With the superpixel-guided sampling strategy for unlabeled samples, we can achieve an automatic determination of the neighborhood covering for a spatial dependency system and thus adapting to real scenes of remote sensing images. In the designed network, two types of loss costs are combined for the training of CNN, i.e., supervised cross entropy and unsupervised reconstruction cost on both labeled and unlabeled samples, respectively. Our experimental results are conducted with three types of remote sensing data, including hyperspectral, multispectral, and synthetic aperture radar (SAR) images. The designed SLE-CNN achieves excellent classification performance in all cases with a limited labeled training set, suggesting its good potential for remote sensing image classification.
8
Kim, Jaeyoung, Hanhoon Park, and Jong-Il Park. "CNN-based image steganalysis using additional data embedding." Multimedia Tools and Applications 79, no. 1-2 (October 31, 2019): 1355–72. http://dx.doi.org/10.1007/s11042-019-08251-3.
Full text
9
Li, Yue, Hongqi Wang, Liqun Yu, Sarah Yvonne Cooper, and Jing-Yan Wang. "Query-Specific Deep Embedding of Content-Rich Network." Computational Intelligence and Neuroscience 2020 (August 25, 2020): 1–11. http://dx.doi.org/10.1155/2020/5943798.
Full textAbstract:
In this paper, we propose to embed a content-rich network for the purpose of similarity searching for a query node. In this network, besides the information of the nodes and edges, we also have the content of each node. We use the convolutional neural network (CNN) to represent the content of each node and then use the graph convolutional network (GCN) to further represent the node by merging the representations of its neighboring nodes. The GCN output is further fed to a deep encoder-decoder model to convert each node to a Gaussian distribution and then convert back to its node identity. The dissimilarity between the two nodes is measured by the Wasserstein distance between their Gaussian distributions. We define the nodes of the network to be positives if they are relevant to the query node and negative if they are irrelevant. The labeling of the positives/negatives is based on an upper bound and a lower bound of the Wasserstein distances between the candidate nodes and the query nodes. We learn the parameters of CNN, GCN, encoder-decoder model, Gaussian distributions, and the upper bound and lower bounds jointly. The learning problem is modeled as a minimization problem to minimize the losses of node identification, network structure preservation, positive/negative query-specific relevance-guild distance, and model complexity. An iterative algorithm is developed to solve the minimization problem. We conducted experiments over benchmark networks, especially innovation networks, to verify the effectiveness of the proposed method and showed its advantage over the state-of-the-art methods.
10
Li, Na, Deyun Zhou, Jiao Shi, Mingyang Zhang, Tao Wu, and Maoguo Gong. "Deep Fully Convolutional Embedding Networks for Hyperspectral Images Dimensionality Reduction." Remote Sensing 13, no. 4 (February 15, 2021): 706. http://dx.doi.org/10.3390/rs13040706.
Full textAbstract:
Due to the superior spatial–spectral extraction capability of the convolutional neural network (CNN), CNN shows great potential in dimensionality reduction (DR) of hyperspectral images (HSIs). However, most CNN-based methods are supervised while the class labels of HSIs are limited and difficult to obtain. While a few unsupervised CNN-based methods have been proposed recently, they always focus on data reconstruction and are lacking in the exploration of discriminability which is usually the primary goal of DR. To address these issues, we propose a deep fully convolutional embedding network (DFCEN), which not only considers data reconstruction but also introduces the specific learning task of enhancing feature discriminability. DFCEN has an end-to-end symmetric network structure that is the key for unsupervised learning. Moreover, a novel objective function containing two terms—the reconstruction term and the embedding term of a specific task—is established to supervise the learning of DFCEN towards improving the completeness and discriminability of low-dimensional data. In particular, the specific task is designed to explore and preserve relationships among samples in HSIs. Besides, due to the limited training samples, inherent complexity and the presence of noise in HSIs, a preprocessing where a few noise spectral bands are removed is adopted to improve the effectiveness of unsupervised DFCEN. Experimental results on three well-known hyperspectral datasets and two classifiers illustrate that the low dimensional features of DFCEN are highly separable and DFCEN has promising classification performance compared with other DR methods.
11
Luo, Yuan, Yu Cheng, Özlem Uzuner, Peter Szolovits, and Justin Starren. "Segment convolutional neural networks (Seg-CNNs) for classifying relations in clinical notes." Journal of the American Medical Informatics Association 25, no. 1 (August 31, 2017): 93–98. http://dx.doi.org/10.1093/jamia/ocx090.
Full textAbstract:
Abstract We propose Segment Convolutional Neural Networks (Seg-CNNs) for classifying relations from clinical notes. Seg-CNNs use only word-embedding features without manual feature engineering. Unlike typical CNN models, relations between 2 concepts are identified by simultaneously learning separate representations for text segments in a sentence: preceding, concept1, middle, concept2, and succeeding. We evaluate Seg-CNN on the i2b2/VA relation classification challenge dataset. We show that Seg-CNN achieves a state-of-the-art micro-average F-measure of 0.742 for overall evaluation, 0.686 for classifying medical problem–treatment relations, 0.820 for medical problem–test relations, and 0.702 for medical problem–medical problem relations. We demonstrate the benefits of learning segment-level representations. We show that medical domain word embeddings help improve relation classification. Seg-CNNs can be trained quickly for the i2b2/VA dataset on a graphics processing unit (GPU) platform. These results support the use of CNNs computed over segments of text for classifying medical relations, as they show state-of-the-art performance while requiring no manual feature engineering.
12
Wang, Ying, Enrico Coiera, and Farah Magrabi. "Using convolutional neural networks to identify patient safety incident reports by type and severity." Journal of the American Medical Informatics Association 26, no. 12 (August 28, 2019): 1600–1608. http://dx.doi.org/10.1093/jamia/ocz146.
Full textAbstract:
Abstract Objective To evaluate the feasibility of a convolutional neural network (CNN) with word embedding to identify the type and severity of patient safety incident reports. Materials and Methods A CNN with word embedding was applied to identify 10 incident types and 4 severity levels. Model training and validation used data sets (n_type = 2860, n_severity = 1160) collected from a statewide incident reporting system. Generalizability was evaluated using an independent hospital-level reporting system. CNN architectures were examined by varying layer size and hyperparameters. Performance was evaluated by F score, precision, recall, and compared to binary support vector machine (SVM) ensembles on 3 testing data sets (type/severity: n_benchmark = 286/116, n_original = 444/4837, n_independent = 6000/5950). Results A CNN with 6 layers was the most effective architecture, outperforming SVMs with better generalizability to identify incidents by type and severity. The CNN achieved high F scores (> 85%) across all test data sets when identifying common incident types including falls, medications, pressure injury, and aggression. When identifying common severity levels (medium/low), CNN outperformed SVMs, improving F scores by 11.9%–45.1% across all 3 test data sets. Discussion Automated identification of incident reports using machine learning is challenging because of a lack of large labelled training data sets and the unbalanced distribution of incident classes. The standard classification strategy is to build multiple binary classifiers and pool their predictions. CNNs can extract hierarchical features and assist in addressing class imbalance, which may explain their success in identifying incident report types. Conclusion A CNN with word embedding was effective in identifying incidents by type and severity, providing better generalizability than SVMs.
13
Zhou, Zhili, Meimin Wang, Yi Cao, and Yuecheng Su. "CNN Feature-Based Image Copy Detection with Contextual Hash Embedding." Mathematics 8, no. 7 (July 17, 2020): 1172. http://dx.doi.org/10.3390/math8071172.
Full textAbstract:
As one of the important techniques for protecting the copyrights of digital images, content-based image copy detection has attracted a lot of attention in the past few decades. The traditional content-based copy detection methods usually extract local hand-crafted features and then quantize these features to visual words by the bag-of-visual-words (BOW) model to build an inverted index file for rapid image matching. Recently, deep learning features, such as the features derived from convolutional neural networks (CNN), have been proven to outperform the hand-crafted features in many applications of computer vision. However, it is not feasible to directly apply the existing global CNN features for copy detection, since they are usually sensitive to partial content-discarded attacks, such as copping and occlusion. Thus, we propose a local CNN feature-based image copy detection method with contextual hash embedding. We first extract the local CNN features from images and then quantize them to visual words to construct an index file. Then, as the BOW quantization process decreases the discriminability of these features to some extent, a contextual hash sequence is captured from a relatively large region surrounding each CNN feature and then is embedded into the index file to improve the feature’s discriminability. Extensive experimental results demonstrate that the proposed method achieves a superior performance compared to the related works in the copy detection task.
14
Li, Jun, Guimin Huang, Jianheng Chen, and Yabing Wang. "Dual CNN for Relation Extraction with Knowledge-Based Attention and Word Embeddings." Computational Intelligence and Neuroscience 2019 (July 14, 2019): 1–10. http://dx.doi.org/10.1155/2019/6789520.
Full textAbstract:
Relation extraction is the underlying critical task of textual understanding. However, the existing methods currently have defects in instance selection and lack background knowledge for entity recognition. In this paper, we propose a knowledge-based attention model, which can make full use of supervised information from a knowledge base, to select an entity. We also design a method of dual convolutional neural networks (CNNs) considering the word embedding of each word is restricted by using a single training tool. The proposed model combines a CNN with an attention mechanism. The model inserts the word embedding and supervised information from the knowledge base into the CNN, performs convolution and pooling, and combines the knowledge base and CNN in the full connection layer. Based on these processes, the model not only obtains better entity representations but also improves the performance of relation extraction with the help of rich background knowledge. The experimental results demonstrate that the proposed model achieves competitive performance.
15
K A, Shirien, Neethu George, and Surekha Mariam Varghese. "Descriptive Answer Script Grading System using CNN-BiLSTM Network." International Journal of Recent Technology and Engineering 9, no. 5 (January 30, 2021): 139–44. http://dx.doi.org/10.35940/ijrte.e5212.019521.
Full textAbstract:
Descriptive answer script assessment and rating program is an automated framework to evaluate the answer scripts correctly. There are several classification schemes in which a piece of text is evaluated on the basis of spelling, semantics and meaning. But, lots of these aren’t successful. Some of the models available to rate the response scripts include Simple Long Short Term Memory (LSTM), Deep LSTM. In addition to that Convolution Neural Network and Bi-directional LSTM is considered here to refine the result. The model uses convolutional neural networks and bidirectional LSTM networks to learn local information of words and capture long-term dependency information of contexts on the Tensorflow and Keras deep learning framework. The embedding semantic representation of texts can be used for computing semantic similarities between pieces of texts and to grade them based on the similarity score. The experiment used methods for data optimization, such as data normalization and dropout, and tested the model on an Automated Student Evaluation Short Response Scoring, a commonly used public dataset. By comparing with the existing systems, the proposed model has achieved the state-of-the-art performance and achieves better results in the accuracy of the test dataset.
16
Yao, Jinliang, Chenrui Wang, Chuang Hu, and Xiaoxi Huang. "Chinese Spam Detection Using a Hybrid BiGRU-CNN Network with Joint Textual and Phonetic Embedding." Electronics 11, no. 15 (August 3, 2022): 2418. http://dx.doi.org/10.3390/electronics11152418.
Full textAbstract:
The proliferation of spam in China has a negative impact on internet users’ experiences online. Existing methods for detecting spam are primarily based on machine learning. However, it has been discovered that these methods are susceptible to adversarial textual spam that has frequently been imperceptibly modified by spammers. Spammers continually modify their strategies to circumvent spam detection systems. Text with Chinese homophonic substitution may be easily understood by users according to its context. Currently, spammers widely use homophonic substitution to break down spam identification systems on the internet. To address these issues, we propose a Bidirectional Gated Recurrent Unit (BiGRU)–Text Convolutional Neural Network (TextCNN) hybrid model with joint embedding for detecting Chinese spam. Our model effectively uses phonetic information and combines the advantages of parameter sharing from TextCNN with long-term memory from BiGRU. The experimental results on real-world datasets show that our model resists homophone noise to some extent and outperforms mainstream deep learning models. We also demonstrate the generality of joint textual and phonetic embedding, which is applicable to other deep learning networks in Chinese spam detection tasks.
17
Tao, Yue, Zhiwei Jia, Runze Ma, and Shugong Xu. "TRIG: Transformer-Based Text Recognizer with Initial Embedding Guidance." Electronics 10, no. 22 (November 13, 2021): 2780. http://dx.doi.org/10.3390/electronics10222780.
Full textAbstract:
Scene text recognition (STR) is an important bridge between images and text, attracting abundant research attention. While convolutional neural networks (CNNS) have achieved remarkable progress in this task, most of the existing works need an extra module (context modeling module) to help CNN to capture global dependencies to solve the inductive bias and strengthen the relationship between text features. Recently, the transformer has been proposed as a promising network for global context modeling by self-attention mechanism, but one of the main short-comings, when applied to recognition, is the efficiency. We propose a 1-D split to address the challenges of complexity and replace the CNN with the transformer encoder to reduce the need for a context modeling module. Furthermore, recent methods use a frozen initial embedding to guide the decoder to decode the features to text, leading to a loss of accuracy. We propose to use a learnable initial embedding learned from the transformer encoder to make it adaptive to different input images. Above all, we introduce a novel architecture for text recognition, named TRansformer-based text recognizer with Initial embedding Guidance (TRIG), composed of three stages (transformation, feature extraction, and prediction). Extensive experiments show that our approach can achieve state-of-the-art on text recognition benchmarks.
18
Quan, Changqin, Zhiwei Luo, and Song Wang. "A Hybrid Deep Learning Model for Protein–Protein Interactions Extraction from Biomedical Literature." Applied Sciences 10, no. 8 (April 13, 2020): 2690. http://dx.doi.org/10.3390/app10082690.
Full textAbstract:
The exponentially increasing size of biomedical literature and the limited ability of manual curators to discover protein–protein interactions (PPIs) in text has led to delays in keeping PPI databases updated with the current findings. The state-of-the-art text mining methods for PPI extraction are primarily based on deep learning (DL) models, and the performance of a DL-based method is mainly affected by the architecture of DL models and the feature embedding methods. In this study, we compared different architectures of DL models, including convolutional neural networks (CNN), long short-term memory (LSTM), and hybrid models, and proposed a hybrid architecture of a bidirectional LSTM+CNN model for PPI extraction. Pretrained word embedding and shortest dependency path (SDP) embedding are fed into a two-embedding channel model, such that the model is able to model long-distance contextual information and can capture the local features and structure information effectively. The experimental results showed that the proposed model is superior to the non-hybrid DL models, and the hybrid CNN+Bidirectional LSTM model works well for PPI extraction. The visualization and comparison of the hidden features learned by different DL models further confirmed the effectiveness of the proposed model.
19
Hua, Lei, and Chanqin Quan. "A Shortest Dependency Path Based Convolutional Neural Network for Protein-Protein Relation Extraction." BioMed Research International 2016 (2016): 1–9. http://dx.doi.org/10.1155/2016/8479587.
Full textAbstract:
The state-of-the-art methods for protein-protein interaction (PPI) extraction are primarily based on kernel methods, and their performances strongly depend on the handcraft features. In this paper, we tackle PPI extraction by using convolutional neural networks (CNN) and propose a shortest dependency path based CNN (sdpCNN) model. The proposed method(1)only takes the sdp and word embedding as input and(2)could avoid bias from feature selection by using CNN. We performed experiments on standard Aimed and BioInfer datasets, and the experimental results demonstrated that our approach outperformed state-of-the-art kernel based methods. In particular, by tracking the sdpCNN model, we find that sdpCNN could extract key features automatically and it is verified that pretrained word embedding is crucial in PPI task.
20
Li, Yifu, Ran Jin, and Yuan Luo. "Classifying relations in clinical narratives using segment graph convolutional and recurrent neural networks (Seg-GCRNs)." Journal of the American Medical Informatics Association 26, no. 3 (December 27, 2018): 262–68. http://dx.doi.org/10.1093/jamia/ocy157.
Full textAbstract:
Abstract We propose to use segment graph convolutional and recurrent neural networks (Seg-GCRNs), which use only word embedding and sentence syntactic dependencies, to classify relations from clinical notes without manual feature engineering. In this study, the relations between 2 medical concepts are classified by simultaneously learning representations of text segments in the context of sentence syntactic dependency: preceding, concept1, middle, concept2, and succeeding segments. Seg-GCRN was systematically evaluated on the i2b2/VA relation classification challenge datasets. Experiments show that Seg-GCRN attains state-of-the-art micro-averaged F-measure for all 3 relation categories: 0.692 for classifying medical treatment–problem relations, 0.827 for medical test–problem relations, and 0.741 for medical problem–medical problem relations. Comparison with the previous state-of-the-art segment convolutional neural network (Seg-CNN) suggests that adding syntactic dependency information helps refine medical word embedding and improves concept relation classification without manual feature engineering. Seg-GCRN can be trained efficiently for the i2b2/VA dataset on a GPU platform.
21
Hu, Donghui, Qiang Shen, Shengnan Zhou, Xueliang Liu, Yuqi Fan, and Lina Wang. "Adaptive Steganalysis Based on Selection Region and Combined Convolutional Neural Networks." Security and Communication Networks 2017 (2017): 1–9. http://dx.doi.org/10.1155/2017/2314860.
Full textAbstract:
Digital image steganalysis is the art of detecting the presence of information hiding in carrier images. When detecting recently developed adaptive image steganography methods, state-of-art steganalysis methods cannot achieve satisfactory detection accuracy, because the adaptive steganography methods can adaptively embed information into regions with rich textures via the guidance of distortion function and thus make the effective steganalysis features hard to be extracted. Inspired by the promising success which convolutional neural network (CNN) has achieved in the fields of digital image analysis, increasing researchers are devoted to designing CNN based steganalysis methods. But as for detecting adaptive steganography methods, the results achieved by CNN based methods are still far from expected. In this paper, we propose a hybrid approach by designing a region selection method and a new CNN framework. In order to make the CNN focus on the regions with complex textures, we design a region selection method by finding a region with the maximal sum of the embedding probabilities. To evolve more diverse and effective steganalysis features, we design a new CNN framework consisting of three separate subnets with independent structure and configuration parameters and then merge and split the three subnets repeatedly. Experimental results indicate that our approach can lead to performance improvement in detecting adaptive steganography.
22
Yan, Zhenguo, and Yue Wu. "A Neural N-Gram Network for Text Classification." Journal of Advanced Computational Intelligence and Intelligent Informatics 22, no. 3 (May 20, 2018): 380–86. http://dx.doi.org/10.20965/jaciii.2018.p0380.
Full textAbstract:
Convolutional Neural Networks (CNNs) effectively extract local features from input data. However, CNN based on word embedding and convolution layers displays poor performance in text classification tasks when compared with traditional baseline methods. We address this problem and propose a model named NNGN that simplifies the convolution layer in the CNN by replacing it with a pooling layer that extracts n-gram embedding in a simpler way and obtains document representations via linear computation. We implement two settings in our model to extract n-gram features. In the first setting, which we refer to as seq-NNGN, we consider word order within each n-gram. In the second setting, BoW-NNGN, we do not consider word order. We compare the performance of these settings in different classification tasks with those of other models. The experimental results show that our proposed model achieves better performance than state-of-the-art models.
23
Fesseha, Awet, Shengwu Xiong, Eshete Derb Emiru, Moussa Diallo, and Abdelghani Dahou. "Text Classification Based on Convolutional Neural Networks and Word Embedding for Low-Resource Languages: Tigrinya." Information 12, no. 2 (January 25, 2021): 52. http://dx.doi.org/10.3390/info12020052.
Full textAbstract:
This article studies convolutional neural networks for Tigrinya (also referred to as Tigrigna), which is a family of Semitic languages spoken in Eritrea and northern Ethiopia. Tigrinya is a “low-resource” language and is notable in terms of the absence of comprehensive and free data. Furthermore, it is characterized as one of the most semantically and syntactically complex languages in the world, similar to other Semitic languages. To the best of our knowledge, no previous research has been conducted on the state-of-the-art embedding technique that is shown here. We investigate which word representation methods perform better in terms of learning for single-label text classification problems, which are common when dealing with morphologically rich and complex languages. Manually annotated datasets are used here, where one contains 30,000 Tigrinya news texts from various sources with six categories of “sport”, “agriculture”, “politics”, “religion”, “education”, and “health” and one unannotated corpus that contains more than six million words. In this paper, we explore pretrained word embedding architectures using various convolutional neural networks (CNNs) to predict class labels. We construct a CNN with a continuous bag-of-words (CBOW) method, a CNN with a skip-gram method, and CNNs with and without word2vec and FastText to evaluate Tigrinya news articles. We also compare the CNN results with traditional machine learning models and evaluate the results in terms of the accuracy, precision, recall, and F1 scoring techniques. The CBOW CNN with word2vec achieves the best accuracy with 93.41%, significantly improving the accuracy for Tigrinya news classification.
24
Wu, Pin, Xuting Chang, Yang Yang, and Xiaoqiang Li. "BASN—Learning Steganography with a Binary Attention Mechanism." Future Internet 12, no. 3 (February 27, 2020): 43. http://dx.doi.org/10.3390/fi12030043.
Full textAbstract:
Secret information sharing through image carriers has aroused much research attention in recent years with images’ growing domination on the Internet and mobile applications. The technique of embedding secret information in images without being detected is called image steganography. With the booming trend of convolutional neural networks (CNN), neural-network-automated tasks have been embedded more deeply in our daily lives. However, a series of wrong labeling or bad captioning on the embedded images has left a trace of skepticism and finally leads to a self-confession like exposure. To improve the security of image steganography and minimize task result distortion, models must maintain the feature maps generated by task-specific networks being irrelative to any hidden information embedded in the carrier. This paper introduces a binary attention mechanism into image steganography to help alleviate the security issue, and, in the meantime, increase embedding payload capacity. The experimental results show that our method has the advantage of high payload capacity with little feature map distortion and still resist detection by state-of-the-art image steganalysis algorithms.
25
Jin, Zhujun, Yu Yang, Yuling Chen, and Yuwei Chen. "IAS-CNN: Image adaptive steganalysis via convolutional neural network combined with selection channel." International Journal of Distributed Sensor Networks 16, no. 3 (March 2020): 155014772091100. http://dx.doi.org/10.1177/1550147720911002.
Full textAbstract:
Steganography is conducive to communication security, but the abuse of steganography brings many potential dangers. And then, steganalysis plays an important role in preventing the abuse of steganography. Nowadays, steganalysis based on deep learning generally has a large number of parameters, and its pertinence to adaptive steganography algorithms is weak. In this article, we propose a lightweight convolutional neural network named IAS-CNN which targets to image adaptive steganalysis. To solve the limitation of manually designing residual extraction filters, we adopt the method of self-learning filter in the network. That is, a high-pass filter in spatial rich model is applied to initialize the weights of the first layer and then these weights are updated through the backpropagation of the network. In addition, the knowledge of selection channel is incorporated into IAS-CNN to enhance residuals in regions that have a high probability for steganography by inputting embedding probability maps into IAS-CNN. Also, IAS-CNN is designed as a lightweight network to reduce the consumption of resources and improve the speed of processing. Experimental results show that IAS-CNN performs well in steganalysis. IAS-CNN not only has similar performance with YedroudjNet in S-UNIWARD steganalysis but also has fewer parameters and convolutional computations.
26
SIMON, HYELLAMADA, BENSON YUSUF BAHA, and ETEMI JOSHUA GARBA. "A MULTI-PLATFORM APPROACH USING HYBRID DEEP LEARNING MODELS FOR AUTOMATIC DETECTION OF HATE SPEECH ON SOCIAL MEDIA." BIMA JOURNAL OF SCIENCE AND TECHNOLOGY (2536-6041) 6, no. 02 (August 30, 2022): 77–90. http://dx.doi.org/10.56892/bimajst.v6i02.358.
Full textAbstract:
Hate speech on online social networks is a general problem across social media platforms that has the potential of causing physical harm to the society. The growing number of hateful comments on the Internet and the rate at which tweets and posts are published per second on social media make it a challenging task to manually identify and remove the hateful commentsfrom such posts. Although numerous publications have proposed machine learning approaches to detect hate speech and other antisocial online behaviours without concentrating on blocking the hate speech from being published on social media. Similarly, prior publications on deep learning and multi-platform approaches did not work on the topic of detecting hate speech in Englishlanguage comments on Twitter and Facebook. This paper proposed a deep learning approach based on a hybrid of convolutional neural network (CNN) and long short-term memory (LSTM) with pre-trained GloVe words embedding to automatically detect and block hate speech on multiple social media platforms including Twitter and Facebook. Thus, datasets were collected from Twitter and Facebook which were annotated as hateful and non-hateful. A set of features were extracted from the datasets based on word embedding mechanism, and the word embeddings were fed into our deep learning framework. The experiment was carried out as a three independent tasks approach. The results show that our hybrid CNN-LSTM approach in Task 1 achieved an f1-score of 0.91, Task 2 obtained an f1-score of 0.92, and Task 3 achieved an f1-score of 0.87. Thus, there is outstanding performance in classifying text as Hate speech or non-hate speech in all the considered metrics. Based on the findings, we conclude that hatespeech can be detected and blocked on social media before it can reach the public.
27
Sun, Fengmei, and Yi Zuo. "Autonomous Classification and Decision-Making Support of Citizen E-Petitions Based on Bi-LSTM-CNN." Mathematical Problems in Engineering 2022 (September 16, 2022): 1–17. http://dx.doi.org/10.1155/2022/9451108.
Full textAbstract:
The increasing number of e-petition services requires accurate calculation methods to perform rapid and automated delivery. Automated text classification significantly reduces the burden of manual sorting, improving service efficiency. Moreover, existing text classification methods focus on improving sole models with an insufficient exploration of hybrid models. Moreover, existing research lacks combinatorial model selection schemes that yield satisfactory performance for petition classification applications. To address these issues, we propose a hybrid deep-learning classification model that can accurately classify the responsible department of a petition. First, e-petitions were collected from the Chinese bulletin board system and then cleaned, segmented, and tokenized into a sequence of words. Second, we employed the word2vec model to pretrain an embedding matrix based on the e-petition corpus. An embedding matrix maps words into vectors. Finally, a hybridized classifier based on convolutional neural networks (CNN) and bidirectional long short-term memory (Bi-LSTM) is proposed to extract features from the title and body of the petition. Compared with baseline models such as CNN, Bi-LSTM, and Bi-LSTM-CNN, the weighted F1 score of the proposed model is improved by 5.82%, 4.31%, and 1.58%, respectively. Furthermore, the proposed automated petition classification decision support system is available on the e-petition website and can be used to accurately deliver petitions and conduct citizen opinion analysis.
28
Wang, Hanqing, Jiaolong Yang, Wei Liang, and Xin Tong. "Deep Single-View 3D Object Reconstruction with Visual Hull Embedding." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 8941–48. http://dx.doi.org/10.1609/aaai.v33i01.33018941.
Full textAbstract:
3D object reconstruction is a fundamental task of many robotics and AI problems. With the aid of deep convolutional neural networks (CNNs), 3D object reconstruction has witnessed a significant progress in recent years. However, possibly due to the prohibitively high dimension of the 3D object space, the results from deep CNNs are often prone to missing some shape details. In this paper, we present an approach which aims to preserve more shape details and improve the reconstruction quality. The key idea of our method is to leverage object mask and pose estimation from CNNs to assist the 3D shape learning by constructing a probabilistic singleview visual hull inside of the network. Our method works by first predicting a coarse shape as well as the object pose and silhouette using CNNs, followed by a novel 3D refinement CNN which refines the coarse shapes using the constructed probabilistic visual hulls. Experiment on both synthetic data and real images show that embedding a single-view visual hull for shape refinement can significantly improve the reconstruction quality by recovering more shapes details and improving shape consistency with the input image.
29
Cong, Kai, Tao Li, Beibei Li, Zhan Gao, Yanbin Xu, and Fei Gao. "KGDetector: Detecting Chinese Sensitive Information via Knowledge Graph-Enhanced BERT." Security and Communication Networks 2022 (May 19, 2022): 1–9. http://dx.doi.org/10.1155/2022/4656837.
Full textAbstract:
The Bidirectional Encoder Representations from Transformers (BERT) technique has been widely used in detecting Chinese sensitive information. However, existing BERT-based frameworks usually fail to emphasize key entities in the texts that contribute significantly to knowledge inference. To meet this gap, we propose a BERT and knowledge graph-based novel framework to detect Chinese sensitive information (named KGDetector). Specifically, we first train a pretrained knowledge graph-based Chinese entity embedding model to characterize entities in the Chinese textual inputs. Finally, we propose an effective framework KGDetector to detect Chinese sensitive information, which employs the knowledge graph-based embedding model and the CNN classification model. Extensive experiments on our crafted Chinese sensitive information dataset demonstrate that KGDetector can effectively detect Chinese sensitive information, outperforming existing baseline frameworks.
30
Lee, Jae-Eun, Young-Ho Seo, and Dong-Wook Kim. "Convolutional Neural Network-Based Digital Image Watermarking Adaptive to the Resolution of Image and Watermark." Applied Sciences 10, no. 19 (September 29, 2020): 6854. http://dx.doi.org/10.3390/app10196854.
Full textAbstract:
Digital watermarking has been widely studied as a method of protecting the intellectual property rights of digital images, which are high value-added contents. Recently, studies implementing these techniques with neural networks have been conducted. This paper also proposes a neural network to perform a robust, invisible blind watermarking for digital images. It is a convolutional neural network (CNN)-based scheme that consists of pre-processing networks for both host image and watermark, a watermark embedding network, an attack simulation for training, and a watermark extraction network to extract watermark whenever necessary. It has three peculiarities for the application aspect: The first is the host image resolution’s adaptability. This is to apply the proposed method to any resolution of the host image and is performed by composing the network without using any resolution-dependent layer or component. The second peculiarity is the adaptability of the watermark information. This is to provide usability of any user-defined watermark data. It is conducted by using random binary data as the watermark and is changed each iteration during training. The last peculiarity is the controllability of the trade-off relationship between watermark invisibility and robustness against attacks, which provides applicability for different applications requiring different invisibility and robustness. For this, a strength scaling factor for watermark information is applied. Besides, it has the following structural or in-training peculiarities. First, the proposed network is as simple as the most profound path consists of only 13 CNN layers, which is through the pre-processing network, embedding network, and extraction network. The second is that it maintains the host’s resolution by increasing the resolution of a watermark in the watermark pre-processing network, which is to increases the invisibility of the watermark. Also, the average pooling is used in the watermark pre-processing network to properly combine the binary value of the watermark data with the host image, and it also increases the invisibility of the watermark. Finally, as the loss function, the extractor uses mean absolute error (MAE), while the embedding network uses mean square error (MSE). Because the extracted watermark information consists of binary values, the MAE between the extracted watermark and the original one is more suitable for balanced training between the embedder and the extractor. The proposed network’s performance is confirmed through training and evaluation that the proposed method has high invisibility for the watermark (WM) and high robustness against various pixel-value change attacks and geometric attacks. Each of the three peculiarities of this scheme is shown to work well with the experimental results. Besides, it is exhibited that the proposed scheme shows good performance compared to the previous methods.
31
Rajasekar, Devaraj, and Lourdusamy Robert. "Unsupervised Word Embedding with Ensemble Deep Learning for Twitter Rumor Identification." Revue d'Intelligence Artificielle 36, no. 5 (December 23, 2022): 769–76. http://dx.doi.org/10.18280/ria.360515.
Full textAbstract:
In social networks, rumor identification is a major problem. The structural data in a topic is applied to derive useful attributes for rumor identification. Most standard rumor identification methods concentrate on local structural attributes, ignoring the global structural attributes that exist between the source tweet and its responses. To tackle this issue, a Source-Replies relation Graph (SR-graph) has been built to develop an Ensemble Graph Convolutional neural Net (EGCN) with a Nodes Proportion Allocation Mechanism (NPAM) which identifies the rumor. But, the word vectors were trained by the standard word-embedding model which does not increase the accuracy for large Twitter databases. To solve this problem, an unsupervised word-embedding method is needed for large Twitter corpora. As a result, the Twitter word-embedded EGCN (T-EGCN) model is proposed in this article, which uses unsupervised learning-based word embedding to find rumors in huge Twitter databases. Initially, the latent contextual semantic correlation and co-occurrence statistical attributes among words in tweets are extracted. Then, to create a rumor attribute vector of tweets, these word embeddings are concatenated with the GloVe model's word attribute vectors, Twitter-specific attributes, and n-gram attributes. Further, the EGCN is trained by using this attribute vector to identify rumors in a huge Twitter database. Finally, the testing results exhibit that the T-EGCN achieves 87.56% accuracy, whereas the RNN, GCN, PGNN, EGCN, and BiLSTM-CNN attain 65.38%, 68.41%, 75.04%, 81.87%, and 86.12%, respectively for rumor identification.
32
Dalyan, Tuğba, Hakan Ayral, and Özgür Özdemir. "A Comprehensive Study of Learning Approaches for Author Gender Identification." Information Technology and Control 51, no. 3 (September 23, 2022): 429–45. http://dx.doi.org/10.5755/j01.itc.51.3.29907.
Full textAbstract:
In recent years, author gender identification is an important yet challenging task in the fields of information retrieval and computational linguistics. In this paper, different learning approaches are presented to address the problem of author gender identification for Turkish articles. First, several classification algorithms are applied to the list of representations based on different paradigms: fixed-length vector representations such as Stylometric Features (SF), Bag-of-Words (BoW) and distributed word/document embeddings such as Word2vec, fastText and Doc2vec. Secondly, deep learning architectures, Convolution Neural Network (CNN), Recurrent Neural Network (RNN), special kinds of RNN such as Long-Short Term Memory (LSTM) and Gated Recurrent Unit (GRU), C-RNN, Bidirectional LSTM (bi-LSTM), Bidirectional GRU (bi-GRU), Hierarchical Attention Networks and Multi-head Attention (MHA) are designated and their comparable performances are evaluated. We conducted a variety of experiments and achieved outstanding empirical results. To conclude, ML algorithms with BoW have promising results. fast-Text is also probably suitable between embedding models. This comprehensive study contributes to literature utilizing different learning approaches based on several ways of representations. It is also first important attempt to identify author gender applying SF, fastText and DNN architectures to the Turkish language.
33
Hemalatha, J., S. Geetha, Sekar Mohan, and S. Nivetha. "An Efficient Steganalysis of Medical Images by Using Deep Learning Based Discrete Scalable Alex Net Convolutionary Neural Networks Classifier." Journal of Medical Imaging and Health Informatics 11, no. 10 (October 1, 2021): 2667–74. http://dx.doi.org/10.1166/jmihi.2021.3858.
Full textAbstract:
Steganalysis is the technique that tries to beat steganography by detecting and removing secret information. Steganalysis involves the detection of a message embedded in a picture. Deep Learning (DL) advances have offered alternative approaches to many difficult issues, including the field of image steganalysis using deep-learning architecture based on convolutionary neural networks (CNN). In recent years, many CNN architectures have been established that have enhanced the exact identification of steganographic images. This work presents a novel architecture which involves a preprocessing stage using histogram equalization and adaptive recursive median filter banks to reduce image noise, a feature extraction stage using shearlet multilinear local embedding methods and then finally the classification can be done by using the discrete scalable Alex NET CNN classifier. Performance was evaluated on the RGB-BMP Steganalysis Dataset with different experimental setups. To prove the effectiveness of the suggested algorithm it can be compared with the other existing methodologies. This work improves classification accuracies on all other existing algorithms over test data.
34
Li, Huang, and Ji. "Bearing Fault Diagnosis with a Feature Fusion Method Based on an Ensemble Convolutional Neural Network and Deep Neural Network." Sensors 19, no. 9 (April 30, 2019): 2034. http://dx.doi.org/10.3390/s19092034.
Full textAbstract:
Rolling bearings are the core components of rotating machinery. Their health directly affects the performance, stability and life of rotating machinery. To prevent possible damage, it is necessary to detect the condition of rolling bearings for fault diagnosis. With the rapid development of intelligent fault diagnosis technology, various deep learning methods have been applied in fault diagnosis in recent years. Convolution neural networks (CNN) have shown high performance in feature extraction. However, the pooling operation of CNN can lead to the loss of much valuable information and the relationship between the whole and the part may be ignored. In this study, we proposed CNNEPDNN, a novel bearing fault diagnosis model based on ensemble deep neural network (DNN) and CNN. We firstly trained CNNEPDNN model. Each of its local networks was trained with different training datasets. The CNN used vibration sensor signals as the input, whereas the DNN used nine time-domain statistical features from bearing vibration sensor signals as the input. Each local network of CNNEPDNN extracted different features from its own trained dataset, thus we fused features with different discrimination for fault recognition. CNNEPDNN was tested under 10 fault conditions based on the bearing data from Bearing Data Center of Case Western Reserve University (CWRU). To evaluate the proposed model, four aspects were analyzed: convergence speed of training loss function, test accuracy, F-Score and the feature clustering result by t-distributed stochastic neighbor embedding (t-SNE) visualization. The training loss function of the proposed model converged more quickly than the local models under different loads. The test accuracy of the proposed model is better than that of CNN, DNN and BPNN. The F-Score value of the model is higher than that of CNN model, and the feature clustering effect of the proposed model was better than that of CNN.
35
Liang, Liqian, Congyan Lang, Zun Li, Jian Zhao, Tao Wang, and Songhe Feng. "Seeing Crucial Parts: Vehicle Model Verification via a Discriminative Representation Model." ACM Transactions on Multimedia Computing, Communications, and Applications 18, no. 1s (February 28, 2022): 1–22. http://dx.doi.org/10.1145/3474596.
Full textAbstract:
Widely used surveillance cameras have promoted large amounts of street scene data, which contains one important but long-neglected object: the vehicle. Here we focus on the challenging problem of vehicle model verification. Most previous works usually employ global features (e.g., fully connected features) to further perform vehicle-level deep metric learning (e.g., triplet-based network). However, we argue that it is noteworthy to investigate the distinctiveness of local features and consider vehicle-part-level metric learning by reducing the intra-class variance as much as possible. In this article, we introduce a simple yet powerful deep model—the enforced intra-class alignment network (EIA-Net)—which can learn a more discriminative image representation by localizing key vehicle parts and jointly incorporating two distance metrics: vehicle-level embedding and vehicle-part-sensitive embedding. For learning features, we propose an effective feature extraction module that is composed of two components: the regional proposal network (RPN)-based network and part-based CNN. The RPN is used to define key vehicle regions and aggregate local features on these regions, whereas part-based CNN offers supplementary global features for the RPN-based network. The fusion features learned by feature extraction module are cast into the deep metric learning module. Especially, we derived an enforced intra-class alignment loss by re-utilizing key vehicle part information to enhance reducing intra-class variance. Furthermore, we modify the coupled cluster loss to model the vehicle-level embedding by enlarging the inter-class variance while shortening intra-class variance. Extensive experiments over benchmark datasets VehicleID and CompCars have shown that the proposed EIA-Net significantly outperforms the state-of-the-art approaches for vehicle model verification. Furthermore, we also conduct comprehensive experiments on vehicle re-identification datasets (i.e., VehicleID and VeRi776) to validate the generalization ability effectiveness of our proposed method.
36
KLUNGPORNKUN, Mongkud, and Peerapon VATEEKUL. "Hierarchical Text Categorization Using Level Based Neural Networks of Word Embedding Sequences with Sharing Layer Information." Walailak Journal of Science and Technology (WJST) 16, no. 2 (October 18, 2018): 121–31. http://dx.doi.org/10.48048/wjst.2019.4145.
Full textAbstract:
In text corpora, it is common to categorize each document to a predefined class hierarchy, which is usually a tree. One of the most widely-used approaches is a level-based strategy that induces a multiclass classifier for each class level independently. However, all prior attempts did not utilize information from its parent level and employed a bag of words rather than considered a sequence of words. In this paper, we present a novel level-based hierarchical text categorization with a strategy called “sharing layer information” For each class level, a neural network is constructed, where its input is a sequence of word embedding vectors generated from Convolutional Neural Networks (CNN). Also, a training strategy to avoid imbalance issues is proposed called “the balanced resampling with mini-batch training” Furthermore, a label correction strategy is proposed to conform the predicted results from all networks on different class levels. The experiment was conducted on 2 standard benchmarks: WIPO and Wiki comparing to a top-down based SVM framework with TF-IDF inputs called “HR-SVM.” The results show that the proposed model can achieved the highest accuracy in terms of micro F1 and outperforms the baseline in the top levels in terms of macro F1.
37
Tian, Juan, and Yingxiang Li. "Convolutional Neural Networks for Steganalysis via Transfer Learning." International Journal of Pattern Recognition and Artificial Intelligence 33, no. 02 (October 24, 2018): 1959006. http://dx.doi.org/10.1142/s0218001419590067.
Full textAbstract:
Recently, a large number of studies have shown that Convolutional Neural Networks are effective for learning features automatically for steganalysis. This paper uses the transfer learning method to help the training of CNNs for steganalysis. First, a Gaussian high-pass filter is designed for pretreatment of the images, that can enhance the weak stego noise in the stegos. Then, the classical Inception-V3 model is improved, and the improved network is used for steganalysis through the method of transfer learning. In order to test the effectiveness of the developed model, two spatial domain content-adaptive steganographic algorithms WOW and S-UNIWARD are used. The results imply that the proposed CNN achieves a better performance at low embedding rates compared with the SRM with ensemble classifiers and the SPAM implemented with a Gaussian SVM on BOSSbase. Finally, a steganalysis system based on the trained model was designed. Through experiments, the generalization ability of the system was tested and discussed.
38
Oliveira, D. A. B. "AUGMENTING DATA USING GAUSSIAN MIXTURE EMBEDDING FOR IMPROVING LAND COVER SEGMENTATION." ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences IV-3/W2-2020 (October 29, 2020): 71–76. http://dx.doi.org/10.5194/isprs-annals-iv-3-w2-2020-71-2020.
Full textAbstract:
Abstract. The use of convolutional neural networks improved greatly data synthesis in the last years and have been widely used for data augmentation in scenarios where very imbalanced data is observed, such as land cover segmentation. Balancing the proportion of classes for training segmentation models can be very challenging considering that samples where all classes are reasonably represented might constitute a small portion of a training set, and techniques for augmenting this small amount of data such as rotation, scaling and translation might be not sufficient for efficient training. In this context, this paper proposes a methodology to perform data augmentation from few samples to improve the performance of CNN-based land cover semantic segmentation. First, we estimate the latent data representation of selected training samples by means of a mixture of Gaussians, using an encoder-decoder CNN. Then, we change the latent embedding used to generate the mixture parameters, at random and in training time, to generate new mixture models slightly different from the original. Finally, we compute the displacement maps between the original and the modified mixture models, and use them to elastically deform the original images, creating new realistic samples out of the original ones. Our disentangled approach allows the spatial modification of displacement maps to preserve objects where deformation is undesired, like buildings and cars, where geometry is highly discriminant. With this simple pipeline, we managed to augment samples in training time, and improve the overall performance of two basal semantic segmentation CNN architectures for land cover semantic segmentation.
39
Iklima, Zendi, Trie Maya Kadarina, and Muhammad Hafidz Ibnu Hajar. "Sentiment classification of delta robot trajectory control using word embedding and convolutional neural network." Indonesian Journal of Electrical Engineering and Computer Science 26, no. 1 (April 1, 2022): 211. http://dx.doi.org/10.11591/ijeecs.v26.i1.pp211-220.
Full textAbstract:
Sentiment classification (SC) is an important research field in natural language processing (NLP) <span lang="EN-US">that classifying, extracting and recognizing subjective information from unstructured text, including opinions, evaluations, emotions, and attitudes. Human-robot interaction (HRI) also involves natural language processing, knowledge representation, and reasoning by utilizing deep learning, cognitive science, and robotics. However, sentiment classification for HRI is rarely implemented, especially to navigate a robot using the Indonesian Language which semantically dynamics when written in text. This paper proposes a sentiment classification of Bahasa Indonesia that supports the delta robot to move in particular trajectory directions. Navigation commands of the delta robot were vectorized using a word embedding method containing two-dimensional matrices to propose the classifier pattern such as convolutional neural network (CNN). The result compared the particular architecture of CNN, GloVe-CNN, and Word2Vec-CNN. As a classifier method, CNN models trained, validated, and tested with higher accuracy are 98.97% and executed in less than a minute. The classifier produces four navigation labels: right means </span><em><span lang="EN-US">'kanan'</span></em><span lang="EN-US">, left means </span><em><span lang="EN-US">'kiri</span></em><span lang="EN-US">', top means </span><em><span lang="EN-US">'atas</span></em><span lang="EN-US">', bottom means </span><em><span lang="EN-US">'bawah</span></em><span lang="EN-US">', and multiplier factor. The classifier result is utilized to transform any navigation commands into direction along with end-effector coordinates.</span>
40
Xue, Feng, Weizhong Yan, Tianyi Wang, Hao Huang, and Bojun Feng. "Deep anomaly detection for industrial systems: a case study." Annual Conference of the PHM Society 12, no. 1 (November 3, 2020): 8. http://dx.doi.org/10.36001/phmconf.2020.v12i1.1186.
Full textAbstract:
We explore the use of deep neural networks for anomaly detection of industrial systems where the data are multivariate time series measurements. We formulate the problem as a self-supervised learning where data under normal operation is used to train a deep neural network autoregressive model, i.e., use a window of time series data to predict future data values. The aim of such a model is to learn to represent the system dynamic behavior under normal conditions, while expect higher model vs. measurement discrepancies under faulty conditions. In real world applications, many control settings are categorical in nature. In this paper, vector embedding and joint losses are employed to deal with such situations. Both LSTM and CNN based deep neural network backbones are studied on the Secure Water Treatment (SWaT) testbed datasets. Also, Support Vector Data Description (SVDD) method is adapted to such anomaly detection settings with deep neural networks. Evaluation methods and results are discussed based on the SWaT dataset along with potential pitfalls.
41
Chen, Changfeng, and Qiang Li. "A Multimodal Music Emotion Classification Method Based on Multifeature Combined Network Classifier." Mathematical Problems in Engineering 2020 (August 1, 2020): 1–11. http://dx.doi.org/10.1155/2020/4606027.
Full textAbstract:
Aiming at the shortcomings of single network classification model, this paper applies CNN-LSTM (convolutional neural networks-long short-term memory) combined network in the field of music emotion classification and proposes a multifeature combined network classifier based on CNN-LSTM which combines 2D (two-dimensional) feature input through CNN-LSTM and 1D (single-dimensional) feature input through DNN (deep neural networks) to make up for the deficiencies of original single feature models. The model uses multiple convolution kernels in CNN for 2D feature extraction, BiLSTM (bidirectional LSTM) for serialization processing and is used, respectively, for audio and lyrics single-modal emotion classification output. In the audio feature extraction, music audio is finely divided and the human voice is separated to obtain pure background sound clips; the spectrogram and LLDs (Low Level Descriptors) are extracted therefrom. In the lyrics feature extraction, the chi-squared test vector and word embedding extracted by Word2vec are, respectively, used as the feature representation of the lyrics. Combining the two types of heterogeneous features selected by audio and lyrics through the classification model can improve the classification performance. In order to fuse the emotional information of the two modals of music audio and lyrics, this paper proposes a multimodal ensemble learning method based on stacking, which is different from existing feature-level and decision-level fusion methods, the method avoids information loss caused by direct dimensionality reduction, and the original features are converted into label results for fusion, effectively solving the problem of feature heterogeneity. Experiments on million song dataset show that the audio classification accuracy of the multifeature combined network classifier in this paper reaches 68%, and the lyrics classification accuracy reaches 74%. The average classification accuracy of the multimodal reaches 78%, which is significantly improved compared with the single-modal.
42
Shi, Dongxian, Ming Xu, Ting Wu, and Liang Kou. "Intrusion Detecting System Based on Temporal Convolutional Network for In-Vehicle CAN Networks." Mobile Information Systems 2021 (September 23, 2021): 1–13. http://dx.doi.org/10.1155/2021/1440259.
Full textAbstract:
In recent years, deep learning theories, such as Recurrent Neural Networks (RNN) and Convolutional Neural Networks (CNN), have been applied as effective methods for intrusion detection in the vehicle CAN network. However, the existing RNNs realize detection by establishing independent models for each CAN ID, which are unable to learn the potential characteristics of different IDs well, and have relatively complicated model structure and high calculation time cost. CNNs can achieve rapid detection by learning the characteristics of normal and attack CAN ID sequences and exhibit good performance, but the current methods do not locate abnormal points in the sequence. To solve the above problems, this paper proposes an in-vehicle CAN network intrusion detection model based on Temporal Convolutional Network, which is called Temporal Convolutional Network-Based Intrusion Detection System (TCNIDS). In TCNIDS, the CAN ID is serialized into a natural language sequence and a word vector is constructed for each CAN ID through the word embedding coding method to reduce the data dimension. At the same time, TCNIDS uses the parameterized Relu method to improve the temporal convolutional network, which can better learn the potential features of the normal sequence. The TCNIDS model has a simple structure and realizes the point anomaly detection at the message level by predicting the future sequence of normal CAN data and setting the probability strategy. The experimental results show that the overall detection rate, false alarm rate, and accuracy rate of TCNIDS under fuzzy attack, spoofing attack, and DoS attack are higher than those of the traditional temporal convolutional network intrusion detection model.
43
Liu, Feng, Xuan Zhou, Xuehu Yan, Yuliang Lu, and Shudong Wang. "Image Steganalysis via Diverse Filters and Squeeze-and-Excitation Convolutional Neural Network." Mathematics 9, no. 2 (January 19, 2021): 189. http://dx.doi.org/10.3390/math9020189.
Full textAbstract:
Steganalysis is a method to detect whether the objects contain secret messages. With the popularity of deep learning, using convolutional neural networks (CNNs), steganalytic schemes have become the chief method of combating steganography in recent years. However, the diversity of filters has not been fully utilized in the current research. This paper constructs a new effective network with diverse filter modules (DFMs) and squeeze-and-excitation modules (SEMs), which can better capture the embedding artifacts. As the essential parts, combining three different scale convolution filters, DFMs can process information diversely, and the SEMs can enhance the effective channels out from DFMs. The experiments presented that our CNN is effective against content-adaptive steganographic schemes with different payloads, such as S-UNIWARD and WOW algorithms. Moreover, some state-of-the-art methods are compared with our approach to demonstrate the outstanding performance.
44
Asim, Muhammad Nabeel, Muhammad Imran Malik, Christoph Zehe, Johan Trygg, Andreas Dengel, and Sheraz Ahmed. "MirLocPredictor: A ConvNet-Based Multi-Label MicroRNA Subcellular Localization Predictor by Incorporating k-Mer Positional Information." Genes 11, no. 12 (December 9, 2020): 1475. http://dx.doi.org/10.3390/genes11121475.
Full textAbstract:
MicroRNAs (miRNA) are small noncoding RNA sequences consisting of about 22 nucleotides that are involved in the regulation of almost 60% of mammalian genes. Presently, there are very limited approaches for the visualization of miRNA locations present inside cells to support the elucidation of pathways and mechanisms behind miRNA function, transport, and biogenesis. MIRLocator, a state-of-the-art tool for the prediction of subcellular localization of miRNAs makes use of a sequence-to-sequence model along with pretrained k-mer embeddings. Existing pretrained k-mer embedding generation methodologies focus on the extraction of semantics of k-mers. However, in RNA sequences, positional information of nucleotides is more important because distinct positions of the four nucleotides define the function of an RNA molecule. Considering the importance of the nucleotide position, we propose a novel approach (kmerPR2vec) which is a fusion of positional information of k-mers with randomly initialized neural k-mer embeddings. In contrast to existing k-mer-based representation, the proposed kmerPR2vec representation is much more rich in terms of semantic information and has more discriminative power. Using novel kmerPR2vec representation, we further present an end-to-end system (MirLocPredictor) which couples the discriminative power of kmerPR2vec with Convolutional Neural Networks (CNNs) for miRNA subcellular location prediction. The effectiveness of the proposed kmerPR2vec approach is evaluated with deep learning-based topologies (i.e., Convolutional Neural Networks (CNN) and Recurrent Neural Network (RNN)) and by using 9 different evaluation measures. Analysis of the results reveals that MirLocPredictor outperform state-of-the-art methods with a significant margin of 18% and 19% in terms of precision and recall.
45
Alshazly, Hammam, Christoph Linse, Erhardt Barth, and Thomas Martinetz. "Handcrafted versus CNN Features for Ear Recognition." Symmetry 11, no. 12 (December 8, 2019): 1493. http://dx.doi.org/10.3390/sym11121493.
Full textAbstract:
Ear recognition is an active research area in the biometrics community with the ultimate goal to recognize individuals effectively from ear images. Traditional ear recognition methods based on handcrafted features and conventional machine learning classifiers were the prominent techniques during the last two decades. Arguably, feature extraction is the crucial phase for the success of these methods due to the difficulty in designing robust features to cope with the variations in the given images. Currently, ear recognition research is shifting towards features extracted by Convolutional Neural Networks (CNNs), which have the ability to learn more specific features robust to the wide image variations and achieving state-of-the-art recognition performance. This paper presents and compares ear recognition models built with handcrafted and CNN features. First, we experiment with seven top performing handcrafted descriptors to extract the discriminating ear image features and then train Support Vector Machines (SVMs) on the extracted features to learn a suitable model. Second, we introduce four CNN based models using a variant of the AlexNet architecture. The experimental results on three ear datasets show the superior performance of the CNN based models by 22%. To further substantiate the comparison, we perform visualization of the handcrafted and CNN features using the t-distributed Stochastic Neighboring Embedding (t-SNE) visualization technique and the characteristics of features are discussed. Moreover, we conduct experiments to investigate the symmetry of the left and right ears and the obtained results on two datasets indicate the existence of a high degree of symmetry between the ears, while a fair degree of asymmetry also exists.
46
Khan, S., I. Ali, F. Ghaffar, and Q. Mazhar-ul-Haq. "Classification of Macromolecules Based on Amino Acid Sequences Using Deep Learning." Engineering, Technology & Applied Science Research 12, no. 6 (December 1, 2022): 9491–95. http://dx.doi.org/10.48084/etasr.5230.
Full textAbstract:
The classification of amino acids and their sequence analysis plays a vital role in life sciences and is a challenging task. Deep learning models have well-established frameworks for solving a broad spectrum of complex learning problems compared to traditional machine learning techniques. This article uses and compares state-of-the-art deep learning models like Convolution Neural Networks (CNNs), Long Short-Term Memory (LSTM), and Gated Recurrent Units (GRU) to solve macromolecule classification problems using amino acid sequences. The CNN extracts features from amino acid sequences, which are treated as vectors with the use of word embedding. These vectors are fed to the above-mentioned models to train robust classifiers. The results show that word2vec as embedding combined with VGG-16 performs better than LSTM and GRU. The proposed approach gets an error rate of 1.5%.
47
Yu, Zeping, Rui Cao, Qiyi Tang, Sen Nie, Junzhou Huang, and Shi Wu. "Order Matters: Semantic-Aware Neural Networks for Binary Code Similarity Detection." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 01 (April 3, 2020): 1145–52. http://dx.doi.org/10.1609/aaai.v34i01.5466.
Full textAbstract:
Binary code similarity detection, whose goal is to detect similar binary functions without having access to the source code, is an essential task in computer security. Traditional methods usually use graph matching algorithms, which are slow and inaccurate. Recently, neural network-based approaches have made great achievements. A binary function is first represented as an control-flow graph (CFG) with manually selected block features, and then graph neural network (GNN) is adopted to compute the graph embedding. While these methods are effective and efficient, they could not capture enough semantic information of the binary code. In this paper we propose semantic-aware neural networks to extract the semantic information of the binary code. Specially, we use BERT to pre-train the binary code on one token-level task, one block-level task, and two graph-level tasks. Moreover, we find that the order of the CFG's nodes is important for graph similarity detection, so we adopt convolutional neural network (CNN) on adjacency matrices to extract the order information. We conduct experiments on two tasks with four datasets. The results demonstrate that our method outperforms the state-of-art models.
48
Dong, Shuyuan. "An Integrated Method of Rolling Bearing Fault Diagnosis Based on Convolutional Neural Network Optimized by Sparrow Optimization Algorithm." Scientific Programming 2022 (July 15, 2022): 1–16. http://dx.doi.org/10.1155/2022/6234169.
Full textAbstract:
Intending to solve the problems including poor self-adaptive ability and generalization ability of the traditional categorizing method under big data, a parameter-optimized Convolutional Neural Network (CNN) based on Sparrow Search Algorithm (SSA) is proposed in this research. Initially, the raw data regarding a series of bearing vibration signals are processed with Fast Fourier Transform (FFT) and Continuous Wavelet Transform (CWT) to attain groups of time-frequency maps. Then, Locally Linear Embedding (LLE) and linear normalization are introduced to make these maps proper for the input of CNN. Next, the preprocessed data sets are utilized as training and testing samples for CNN, and the accuracy rate of the testing is considered as the fitness of SSA, which is used to search for optimal parameter combinations for CNN by SAA. Meanwhile, the construction of the CNN is determined by experience and other previous researches. Finally, an NN-based defect diagnosis model for bearings will be constructed after the SAA has determined the appropriate parameters. The model’s accuracy rate may reach 99.4 percent after repeated testing using samples, which is significantly superior to the classic fault detection approach and the fault diagnostic method based solely on shallow networks. This experimental result demonstrates that the suggested strategy may significantly increase the model’s self-adaptive feature extraction capacity and accuracy rate, implying a higher performance in defect diagnosis in the presence of huge data.
49
Hasan, Asif, Tripti Sharma, Azizuddin Khan, and Mohammed Hasan Ali Al-Abyadh. "Analysing Hate Speech against Migrants and Women through Tweets Using Ensembled Deep Learning Model." Computational Intelligence and Neuroscience 2022 (April 10, 2022): 1–8. http://dx.doi.org/10.1155/2022/8153791.
Full textAbstract:
Twitter’s popularity has exploded in the previous few years, making it one of the most widely used social media sites. As a result of this development, the strategies described in this study are now more beneficial. Additionally, there has been an increase in the number of people who express their views in demeaning ways to others. As a result, hate speech has piqued interest in the subject of sentiment analysis, which has developed various algorithms for detecting emotions in social networks using intuitive means. This paper proposes the deep learning model to classify the sentiments in two separate analyses. In the first analysis, the tweets are classified based on the hate speech against the migrants and the women. In the second analysis, the detection is performed using a deep learning model to organise whether the hate speech is performed by a single or a group of users. During the text analysis, word embedding is implemented using the combination of deep learning models such as BiLSTM, CNN, and MLP. These models are integrated with word embedding methods such as inverse glove (global vector), document frequency (TF-IDF), and transformer-based embedding.
50
Wu, Minghu, Leming Guo, Rui Chen, Wanyin Du, Juan Wang, Min Liu, Xiangbin Kong, and Jing Tang. "Improved YOLOX Foreign Object Detection Algorithm for Transmission Lines." Wireless Communications and Mobile Computing 2022 (October 20, 2022): 1–10. http://dx.doi.org/10.1155/2022/5835693.
Full textAbstract:
It is quite simple for foreign objects to attach themselves to transmission line corridors because of the wide variety of laying and the complex, changing environment. If these foreign objects are not found and removed in a timely manner, they can have a significant impact on the transmission lines’ ability to operate safely. Due to the problem of poor accuracy of foreign object identification in transmission line image inspection, we provide an improved YOLOX technique for detection of foreign objects in transmission lines. The method improves the YOLOX target detection network by first using Atrous Spatial Pyramid Pooling to increase sensitivity to foreign objects of different scales, then by embedding Convolutional Block Attention Module to increase model recognition accuracy, and finally by using GIoU loss to further optimize. The testing findings show that the enhanced YOLOX network has a mAP improvement of around 4.24% over the baseline YOLOX network. The target detection SSD, Faster R-CNN, YOLOv5, and YOLOV7 networks have improved less than this. The effectiveness and superiority of the algorithm are proven.