Статті в журналах з теми "Graph attention network (GAT)"
Щоб переглянути інші типи публікацій з цієї теми, перейдіть за посиланням: Graph attention network (GAT).
Оформте джерело за APA, MLA, Chicago, Harvard та іншими стилями
Ознайомтеся з топ-50 статей у журналах для дослідження на тему "Graph attention network (GAT)".
Біля кожної праці в переліку літератури доступна кнопка «Додати до бібліографії». Скористайтеся нею – і ми автоматично оформимо бібліографічне посилання на обрану працю в потрібному вам стилі цитування: APA, MLA, «Гарвард», «Чикаго», «Ванкувер» тощо.
Також ви можете завантажити повний текст наукової публікації у форматі «.pdf» та прочитати онлайн анотацію до роботи, якщо відповідні параметри наявні в метаданих.
Переглядайте статті в журналах для різних дисциплін та оформлюйте правильно вашу бібліографію.
1
Wu, Nan, and Chaofan Wang. "Ensemble Graph Attention Networks." Transactions on Machine Learning and Artificial Intelligence 10, no. 3 (June 12, 2022): 29–41. http://dx.doi.org/10.14738/tmlai.103.12399.
Анотація:
Graph neural networks have demonstrated its success in many applications on graph-structured data. Many efforts have been devoted to elaborating new network architectures and learning algorithms over the past decade. The exploration of applying ensemble learning techniques to enhance existing graph algorithms have been overlooked. In this work, we propose a simple generic bagging-based ensemble learning strategy which is applicable to any backbone graph models. We then propose two ensemble graph neural network models – Ensemble-GAT and Ensemble-HetGAT by applying the ensemble strategy to the graph attention network (GAT), and a heterogeneous graph attention network (HetGAT). We demonstrate the effectiveness of the proposed ensemble strategy on GAT and HetGAT through comprehensive experiments with four real-world homogeneous graph datasets and three real-world heterogeneous graph datasets on node classification tasks. The proposed Ensemble-GAT and Ensemble-HetGAT outperform the state-of-the-art graph neural network and heterogeneous graph neural network models on most of the benchmark datasets. The proposed ensemble strategy also alleviates the over-smoothing problem in GAT and HetGAT.
2
Verma, Atul Kumar, Rahul Saxena, Mahipal Jadeja, Vikrant Bhateja, and Jerry Chun-Wei Lin. "Bet-GAT: An Efficient Centrality-Based Graph Attention Model for Semi-Supervised Node Classification." Applied Sciences 13, no. 2 (January 7, 2023): 847. http://dx.doi.org/10.3390/app13020847.
Анотація:
Graph Neural Networks (GNNs) have witnessed great advancement in the field of neural networks for processing graph datasets. Graph Convolutional Networks (GCNs) have outperformed current models/algorithms in accomplishing tasks such as semi-supervised node classification, link prediction, and graph classification. GCNs perform well even with a very small training dataset. The GCN framework has evolved to Graph Attention Model (GAT), GraphSAGE, and other hybrid frameworks. In this paper, we effectively usd the network centrality approach to select nodes from the training set (instead of a traditional random selection), which is fed into GCN (and GAT) to perform semi-supervised node classification tasks. This allows us to take advantage of the best positional nodes in the network. Based on empirical analysis, we choose the betweenness centrality measure for selecting the training nodes. We also mathematically justify why our proposed technique offers better training. This novel training technique is used to analyze the performance of GCN and GAT models on five benchmark networks—Cora, Citeseer, PubMed, Wiki-CS, and Amazon Computers. In GAT implementations, we obtain improved classification accuracy compared to the other state-of-the-art GCN-based methods. Moreover, to the best of our knowledge, the results obtained for Citeseer, Wiki- CS, and Amazon Computer datasets are the best compared to all the existing node classification methods.
3
Lu, Shengfu, Jiaming Kang, Jinyu Zhang, and Mi Li. "Assessment method of depressive disorder level based on graph attention network." ITM Web of Conferences 45 (2022): 01039. http://dx.doi.org/10.1051/itmconf/20224501039.
Анотація:
This paper presents an approach to predict the depression self-rating scale of Patient Health Questions-9 (PHQ-9) values from pupil-diameter data based on the graph attention network (GAT). The pupil diameter signal was derived from the eye information collected synchronously while the subjects were viewing the virtual reality emotional scene, and then the scores of PHQ-9 depression self-rating scale were collected for depression level. The chebyshev distance based GAT (Chebyshev-GAT) was constructed by extracting pupil-diameter change rate, emotional bandwidth, information entropy and energy, and their statistical distribution. The results show that, the error (MAE and SMRE)of the prediction results using Chebyshev-GAT is smaller then the traditional regression prediction model.
4
Xiang, Zhijie, Weijia Gong, Zehui Li, Xue Yang, Jihua Wang, and Hong Wang. "Predicting Protein–Protein Interactions via Gated Graph Attention Signed Network." Biomolecules 11, no. 6 (May 28, 2021): 799. http://dx.doi.org/10.3390/biom11060799.
Анотація:
Protein–protein interactions (PPIs) play a key role in signal transduction and pharmacogenomics, and hence, accurate PPI prediction is crucial. Graph structures have received increasing attention owing to their outstanding performance in machine learning. In practice, PPIs can be expressed as a signed network (i.e., graph structure), wherein the nodes in the network represent proteins, and edges represent the interactions (positive or negative effects) of protein nodes. PPI predictions can be realized by predicting the links of the signed network; therefore, the use of gated graph attention for signed networks (SN-GGAT) is proposed herein. First, the concept of graph attention network (GAT) is applied to signed networks, in which “attention” represents the weight of neighbor nodes, and GAT updates the node features through the weighted aggregation of neighbor nodes. Then, the gating mechanism is defined and combined with the balance theory to obtain the high-order relations of protein nodes to improve the attention effect, making the attention mechanism follow the principle of “low-order high attention, high-order low attention, different signs opposite”. PPIs are subsequently predicted on the Saccharomyces cerevisiae core dataset and the Human dataset. The test results demonstrate that the proposed method exhibits strong competitiveness.
5
Yuan, Hong, Jing Huang, and Jin Li. "Protein-ligand binding affinity prediction model based on graph attention network." Mathematical Biosciences and Engineering 18, no. 6 (2021): 9148–62. http://dx.doi.org/10.3934/mbe.2021451.
Анотація:
<abstract> <p>Estimating the binding affinity between proteins and drugs is very important in the application of structure-based drug design. Currently, applying machine learning to build the protein-ligand binding affinity prediction model, which is helpful to improve the performance of classical scoring functions, has attracted many scientists' attention. In this paper, we have developed an affinity prediction model called GAT-Score based on graph attention network (GAT). The protein-ligand complex is represented by a graph structure, and the atoms of protein and ligand are treated in the same manner. Two improvements are made to the original graph attention network. Firstly, a dynamic feature mechanism is designed to enable the model to deal with bond features. Secondly, a virtual super node is introduced to aggregate node-level features into graph-level features, so that the model can be used in the graph-level regression problems. PDBbind database v.2018 is used to train the model. Finally, the performance of GAT-Score was tested by the scheme $C_s$ (Core set as the test set) and <italic>CV</italic> (Cross-Validation). It has been found that our results are better than most methods from machine learning models with traditional molecular descriptors.</p> </abstract>
6
Jing, Weipeng, Xianyang Song, Donglin Di, and Houbing Song. "geoGAT: Graph Model Based on Attention Mechanism for Geographic Text Classification." ACM Transactions on Asian and Low-Resource Language Information Processing 20, no. 5 (September 30, 2021): 1–18. http://dx.doi.org/10.1145/3434239.
Анотація:
In the area of geographic information processing, there are few researches on geographic text classification. However, the application of this task in Chinese is relatively rare. In our work, we intend to implement a method to extract text containing geographical entities from a large number of network texts. The geographic information in these texts is of great practical significance to transportation, urban and rural planning, disaster relief, and other fields. We use the method of graph convolutional neural network with attention mechanism to achieve this function. Graph attention networks (GAT) is an improvement of graph convolutional neural networks (GCN). Compared with GCN, the advantage of GAT is that the attention mechanism is proposed to weight the sum of the characteristics of adjacent vertices. In addition, We construct a Chinese dataset containing geographical classification from multiple datasets of Chinese text classification. The Macro-F Score of the geoGAT we used reached 95% on the new Chinese dataset.
7
Liu, Yiwen, Tao Wen, and Zhenning Wu. "Motion Artifact Detection Based on Regional–Temporal Graph Attention Network from Head Computed Tomography Images." Electronics 13, no. 4 (February 10, 2024): 724. http://dx.doi.org/10.3390/electronics13040724.
Анотація:
Artifacts are the main cause of degradation in CT image quality and diagnostic accuracy. Because of the complex texture of CT images, it is a challenging task to automatically detect artifacts from limited image samples. Recently, graph convolutional networks (GCNs) have achieved great success and shown promising results in medical imaging due to their powerful learning ability. However, GCNs do not take the attention mechanism into consideration. To overcome their limitations, we propose a novel Regional–Temporal Graph Attention Network for motion artifact detection from computed tomography images (RT-GAT). In this paper, head CT images are viewed as a heterogeneous graph by taking regional and temporal information into consideration, and the graph attention network is utilized to extract the features of the constructed graph. Then, the feature vector is input into the classifier to detect the motion artifacts. The experimental results demonstrate that our proposed RT-GAT method outperforms the state-of-the-art methods on a real-world CT dataset.
8
Huang, Ling, Xing-Xing Liu, Shu-Qiang Huang, Chang-Dong Wang, Wei Tu, Jia-Meng Xie, Shuai Tang, and Wendi Xie. "Temporal Hierarchical Graph Attention Network for Traffic Prediction." ACM Transactions on Intelligent Systems and Technology 12, no. 6 (December 31, 2021): 1–21. http://dx.doi.org/10.1145/3446430.
Анотація:
As a critical task in intelligent traffic systems, traffic prediction has received a large amount of attention in the past few decades. The early efforts mainly model traffic prediction as the time-series mining problem, in which the spatial dependence has been largely ignored. As the rapid development of deep learning, some attempts have been made in modeling traffic prediction as the spatio-temporal data mining problem in a road network, in which deep learning techniques can be adopted for modeling the spatial and temporal dependencies simultaneously. Despite the success, the spatial and temporal dependencies are only modeled in a regionless network without considering the underlying hierarchical regional structure of the spatial nodes, which is an important structure naturally existing in the real-world road network. Apart from the challenge of modeling the spatial and temporal dependencies like the existing studies, the extra challenge caused by considering the hierarchical regional structure of the road network lies in simultaneously modeling the spatial and temporal dependencies between nodes and regions and the spatial and temporal dependencies between regions. To this end, this article proposes a new Temporal Hierarchical Graph Attention Network (TH-GAT). The main idea lies in augmenting the original road network into a region-augmented network, in which the hierarchical regional structure can be modeled. Based on the region-augmented network, the region-aware spatial dependence model and the region-aware temporal dependence model can be constructed, which are two main components of the proposed TH-GAT model. In addition, in the region-aware spatial dependence model, the graph attention network is adopted, in which the importance of a node to another node, of a node to a region, of a region to a node, and of a region to another region, can be captured automatically by means of the attention coefficients. Extensive experiments are conducted on two real-world traffic datasets, and the results have confirmed the superiority of the proposed TH-GAT model.
9
Song, Kyungwoo, Yohan Jung, Dongjun Kim, and Il-Chul Moon. "Implicit Kernel Attention." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 11 (May 18, 2021): 9713–21. http://dx.doi.org/10.1609/aaai.v35i11.17168.
Анотація:
Attention computes the dependency between representations, and it encourages the model to focus on the important selective features. Attention-based models, such as Transformer and graph attention network (GAT), are widely utilized for sequential data and graph-structured data. This paper suggests a new interpretation and generalized structure of the attention in Transformer and GAT. For the attention in Transformer and GAT, we derive that the attention is a product of two parts: 1) the RBF kernel to measure the similarity of two instances and 2) the exponential of L2 norm to compute the importance of individual instances. From this decomposition, we generalize the attention in three ways. First, we propose implicit kernel attention with an implicit kernel function instead of manual kernel selection. Second, we generalize L2 norm as the Lp norm. Third, we extend our attention to structured multi-head attention. Our generalized attention shows better performance on classification, translation, and regression tasks.
10
Zheng, Jing, Ziren Gao, Jingsong Ma, Jie Shen, and Kang Zhang. "Deep Graph Convolutional Networks for Accurate Automatic Road Network Selection." ISPRS International Journal of Geo-Information 10, no. 11 (November 11, 2021): 768. http://dx.doi.org/10.3390/ijgi10110768.
Анотація:
The selection of road networks is very important for cartographic generalization. Traditional artificial intelligence methods have improved selection efficiency but cannot fully extract the spatial features of road networks. However, current selection methods, which are based on the theory of graphs or strokes, have low automaticity and are highly subjective. Graph convolutional networks (GCNs) combine graph theory with neural networks; thus, they can not only extract spatial information but also realize automatic selection. Therefore, in this study, we adopted GCNs for automatic road network selection and transformed the process into one of node classification. In addition, to solve the problem of gradient vanishing in GCNs, we compared and analyzed the results of various GCNs (GraphSAGE and graph attention networks [GAT]) by selecting small-scale road networks under different deep architectures (JK-Nets, ResNet, and DenseNet). Our results indicate that GAT provides better selection of road networks than other models. Additionally, the three abovementioned deep architectures can effectively improve the selection effect of models; JK-Nets demonstrated more improvement with higher accuracy (88.12%) than other methods. Thus, our study shows that GCN is an appropriate tool for road network selection; its application in cartography must be further explored.
11
Liu, Tong, and Bojun Liu. "Next basket recommendation based on graph attention network and transformer." Journal of Physics: Conference Series 2303, no. 1 (July 1, 2022): 012023. http://dx.doi.org/10.1088/1742-6596/2303/1/012023.
Анотація:
Abstract To solve the problem that when using recurrent neural network will forget the previously learned basket information and cannot reflect the user’s real purchase intention. A next basket recommendation based on graph attention network (GAT) and transformer model is proposed. Construct the item-basket relationship graph and use GAT to learn the basket interactive characteristics, then model the user interest representation, and finally obtain the item probability distribution through deep neural network. The experimental results on two real world datasets show that the proposed model outperforms state-of-the-art existing basket recommendation models. Through ablation experiments and hyperparameter experiments, the effectiveness of each module and the influence of model parameters are proved.
12
Zhu, Taomei, Maria Jesus Lopez Boada, and Beatriz Lopez Boada. "Adaptive Graph Attention and Long Short-Term Memory-Based Networks for Traffic Prediction." Mathematics 12, no. 2 (January 12, 2024): 255. http://dx.doi.org/10.3390/math12020255.
Анотація:
While the increased availability of traffic data is allowing us to better understand urban mobility, research on data-driven and predictive modeling is also providing new methods for improving traffic management and reducing congestion. In this paper, we present a hybrid predictive modeling architecture, namely GAT-LSTM, by incorporating graph attention (GAT) and long short-term memory (LSTM) networks for handling traffic prediction tasks. In this architecture, GAT networks capture the spatial dependencies of the traffic network, LSTM networks capture the temporal correlations, and the Dayfeature component incorporates time and external information (such as day of the week, extreme weather conditions, holidays, etc.). A key attention block is designed to integrate GAT, LSTM, and the Dayfeature components as well as learn and assign weights to these different components within the architecture. This method of integration is proven effective at improving prediction accuracy, as shown by the experimental results obtained with the PeMS08 open dataset, and the proposed model demonstrates state-of-the-art performance in these experiments. Furthermore, the hybrid model demonstrates adaptability to dynamic traffic conditions, different prediction horizons, and various traffic networks.
13
Deng, Xuan, Cheng Zhang, Jian Shi, and Zizhao Wu. "PU-GAT: Point cloud upsampling with graph attention network." Graphical Models 130 (December 2023): 101201. http://dx.doi.org/10.1016/j.gmod.2023.101201.
14
Zhao, Yanna, Gaobo Zhang, Changxu Dong, Qi Yuan, Fangzhou Xu, and Yuanjie Zheng. "Graph Attention Network with Focal Loss for Seizure Detection on Electroencephalography Signals." International Journal of Neural Systems 31, no. 07 (May 18, 2021): 2150027. http://dx.doi.org/10.1142/s0129065721500271.
Анотація:
Automatic seizure detection from electroencephalogram (EEG) plays a vital role in accelerating epilepsy diagnosis. Previous researches on seizure detection mainly focused on extracting time-domain and frequency-domain features from single electrodes, while paying little attention to the positional correlations between different EEG channels of the same subject. Moreover, data imbalance is common in seizure detection scenarios where the duration of nonseizure periods is much longer than the duration of seizures. To cope with the two challenges, a novel seizure detection method based on graph attention network (GAT) is presented. The approach acts on graph-structured data and takes the raw EEG data as input. The positional relationship between different EEG signals is exploited by GAT. The loss function of the GAT model is redefined using the focal loss to tackle data imbalance problem. Experiments are conducted on the CHB-MIT dataset. The accuracy, sensitivity and specificity of the proposed method are 98.89[Formula: see text], 97.10[Formula: see text] and 99.63[Formula: see text], respectively.
15
Pu, S., Y. Song, Y. Chen, Y. Li, J. Zhang, Q. Lin, X. Zhu, et al. "HYPERSPECTRAL IMAGE CLASSIFICATION WITH LOCALIZED SPECTRAL FILTERING-BASED GRAPH ATTENTION NETWORK." ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences V-3-2022 (May 17, 2022): 155–61. http://dx.doi.org/10.5194/isprs-annals-v-3-2022-155-2022.
Анотація:
Abstract. Graph-based deep learning has been proved a promising approach that has an apparent superiority for learning graph data and modeling spatial topological relations between features. In particular, graph attention networks (GATs) are good at efficiently processing the graph-structured hyperspectral data by leveraging masked self-attention layers to address the known shortcomings of previous frameworks based on graph convolutions or their approximations. In this study, we proposed a novel approach that combines localized spectral filtering and GAT for the hyperspectral image classification task. First, we conducted unsupervised t-SNE (t-distributed stochastic neighbor embedding) manifold learning-based feature dimensionality reduction to create localized hyperspectral data cubes. Then, these feature cubes combined with localized adjacent matrices were fed into a shallow graph attention network in a supervised learning manner. Finally, we obtained credible classification results and promising classification performance in distinguishing diversified land covers through reducing the possible redundancy of spectral information and enhancing the expression of local spatial-spectral information. Experiments on two real hyperspectral data sets (that is, Indian Pines-A (IA) and Huanghekou (HH) data sets) demonstrated that the presented approach offers promising classification performance, that is, the GAT using t-SNE acquires superior performance than that of using PCA (principal component analysis), and also proves the great importance of combining spatial- and spectral information for hyperspectral image classification.
16
Wang, Renping, Shun Li, Enhao Tang, Sen Lan, Yajing Liu, Jing Yang, Shizhen Huang, and Hailong Hu. "SH-GAT: Software-hardware co-design for accelerating graph attention networks on FPGA." Electronic Research Archive 32, no. 4 (2024): 2310–22. http://dx.doi.org/10.3934/era.2024105.
Анотація:
<abstract><p>Graph convolution networks (GCN) have demonstrated success in learning graph structures; however, they are limited in inductive tasks. Graph attention networks (GAT) were proposed to address the limitations of GCN and have shown high performance in graph-based tasks. Despite this success, GAT faces challenges in hardware acceleration, including: 1) The GAT algorithm has difficulty adapting to hardware; 2) challenges in efficiently implementing Sparse matrix multiplication (SPMM); and 3) complex addressing and pipeline stall issues due to irregular memory accesses. To this end, this paper proposed SH-GAT, an FPGA-based GAT accelerator that achieves more efficient GAT inference. The proposed approach employed several optimizations to enhance GAT performance. First, this work optimized the GAT algorithm using split weights and softmax approximation to make it more hardware-friendly. Second, a load-balanced SPMM kernel was designed to fully leverage potential parallelism and improve data throughput. Lastly, data preprocessing was performed by pre-fetching the source node and its neighbor nodes, effectively addressing pipeline stall and complexly addressing issues arising from irregular memory access. SH-GAT was evaluated on the Xilinx FPGA Alveo U280 accelerator card with three popular datasets. Compared to existing CPU, GPU, and state-of-the-art (SOTA) FPGA-based accelerators, SH-GAT can achieve speedup by up to 3283$ \times $, 13$ \times $, and 2.3$ \times $.</p></abstract>
17
Lin, Yu-Chen, Chia-Hung Wang, and Yu-Cheng Lin. "GAT TransPruning: progressive channel pruning strategy combining graph attention network and transformer." PeerJ Computer Science 10 (April 23, 2024): e2012. http://dx.doi.org/10.7717/peerj-cs.2012.
Анотація:
Recently, large-scale artificial intelligence models with billions of parameters have achieved good results in experiments, but their practical deployment on edge computing platforms is often subject to many constraints because of their resource requirements. These models require powerful computing platforms with a high memory capacity to store and process the numerous parameters and activations, which makes it challenging to deploy these large-scale models directly. Therefore, model compression techniques are crucial role in making these models more practical and accessible. In this article, a progressive channel pruning strategy combining graph attention network and transformer, namely GAT TransPruning, is proposed, which uses the graph attention networks (GAT) and the attention of transformer mechanism to determine the channel-to-channel relationship in large networks. This approach ensures that the network maintains its critical functional connections and optimizes the trade-off between model size and performance. In this study, VGG-16, VGG-19, ResNet-18, ResNet-34, and ResNet-50 are used as large-scale network models with the CIFAR-10 and CIFAR-100 datasets for verification and quantitative analysis of the proposed progressive channel pruning strategy. The experimental results reveal that the accuracy rate only drops by 6.58% when the channel pruning rate is 89% for VGG-19/CIFAR-100. In addition, the lightweight model inference speed is 9.10 times faster than that of the original large model. In comparison with the traditional channel pruning schemes, the proposed progressive channel pruning strategy based on the GAT and Transformer cannot only cut out the insignificant weight channels and effectively reduce the model size, but also ensure that the performance drop rate of its lightweight model is still the smallest even under high pruning ratio.
18
Bian, Chen, Xiu-Juan Lei, and Fang-Xiang Wu. "GATCDA: Predicting circRNA-Disease Associations Based on Graph Attention Network." Cancers 13, no. 11 (May 25, 2021): 2595. http://dx.doi.org/10.3390/cancers13112595.
Анотація:
CircRNAs (circular RNAs) are a class of non-coding RNA molecules with a closed circular structure. CircRNAs are closely related to the occurrence and development of diseases. Due to the time-consuming nature of biological experiments, computational methods have become a better way to predict the interactions between circRNAs and diseases. In this study, we developed a novel computational method called GATCDA utilizing a graph attention network (GAT) to predict circRNA–disease associations with disease symptom similarity, network similarity, and information entropy similarity for both circRNAs and diseases. GAT learns representations for nodes on a graph by an attention mechanism, which assigns different weights to different nodes in a neighborhood. Considering that the circRNA–miRNA–mRNA axis plays an important role in the generation and development of diseases, circRNA–miRNA interactions and disease–mRNA interactions were adopted to construct features, in which mRNAs were related to 88% of miRNAs. As demonstrated by five-fold cross-validation, GATCDA yielded an AUC value of 0.9011. In addition, case studies showed that GATCDA can predict unknown circRNA–disease associations. In conclusion, GATCDA is a useful method for exploring associations between circRNAs and diseases.
19
Zhao, Mengmeng, Haipeng Peng, Lixiang Li, and Yeqing Ren. "Graph Attention Network and Informer for Multivariate Time Series Anomaly Detection." Sensors 24, no. 5 (February 26, 2024): 1522. http://dx.doi.org/10.3390/s24051522.
Анотація:
Time series anomaly detection is very important to ensure the security of industrial control systems (ICSs). Many algorithms have performed well in anomaly detection. However, the performance of most of these algorithms decreases sharply with the increase in feature dimension. This paper proposes an anomaly detection scheme based on Graph Attention Network (GAT) and Informer. GAT learns sequential characteristics effectively, and Informer performs excellently in long time series prediction. In addition, long-time forecasting loss and short-time forecasting loss are used to detect multivariate time series anomalies. Short-time forecasting is used to predict the next time value, and long-time forecasting is employed to assist the short-time prediction. We conduct a large number of experiments on industrial control system datasets SWaT and WADI. Compared with most advanced methods, we achieve competitive results, especially on higher-dimensional datasets. Moreover, the proposed method can accurately locate anomalies and realize interpretability.
20
Tanvir, Raihanul Bari, Md Mezbahul Islam, Masrur Sobhan, Dongsheng Luo, and Ananda Mohan Mondal. "MOGAT: A Multi-Omics Integration Framework Using Graph Attention Networks for Cancer Subtype Prediction." International Journal of Molecular Sciences 25, no. 5 (February 28, 2024): 2788. http://dx.doi.org/10.3390/ijms25052788.
Анотація:
Accurate cancer subtype prediction is crucial for personalized medicine. Integrating multi-omics data represents a viable approach to comprehending the intricate pathophysiology of complex diseases like cancer. Conventional machine learning techniques are not ideal for analyzing the complex interrelationships among different categories of omics data. Numerous models have been suggested using graph-based learning to uncover veiled representations and network formations unique to distinct types of omics data to heighten predictions regarding cancers and characterize patients’ profiles, amongst other applications aimed at improving disease management in medical research. The existing graph-based state-of-the-art multi-omics integration approaches for cancer subtype prediction, MOGONET, and SUPREME, use a graph convolutional network (GCN), which fails to consider the level of importance of neighboring nodes on a particular node. To address this gap, we hypothesize that paying attention to each neighbor or providing appropriate weights to neighbors based on their importance might improve the cancer subtype prediction. The natural choice to determine the importance of each neighbor of a node in a graph is to explore the graph attention network (GAT). Here, we propose MOGAT, a novel multi-omics integration approach, leveraging GAT models that incorporate graph-based learning with an attention mechanism. MOGAT utilizes a multi-head attention mechanism to extract appropriate information for a specific sample by assigning unique attention coefficients to neighboring samples. Based on our knowledge, our group is the first to explore GAT in multi-omics integration for cancer subtype prediction. To evaluate the performance of MOGAT in predicting cancer subtypes, we explored two sets of breast cancer data from TCGA and METABRIC. Our proposed approach, MOGAT, outperforms MOGONET by 32% to 46% and SUPREME by 2% to 16% in cancer subtype prediction in different scenarios, supporting our hypothesis. Our results also showed that GAT embeddings provide a better prognosis in differentiating the high-risk group from the low-risk group than raw features.
21
Lv, Shaoqing, Jungang Dong, Chichi Wang, Xuanhong Wang, and Zhiqiang Bao. "RB-GAT: A Text Classification Model Based on RoBERTa-BiGRU with Graph ATtention Network." Sensors 24, no. 11 (May 24, 2024): 3365. http://dx.doi.org/10.3390/s24113365.
Анотація:
With the development of deep learning, several graph neural network (GNN)-based approaches have been utilized for text classification. However, GNNs encounter challenges when capturing contextual text information within a document sequence. To address this, a novel text classification model, RB-GAT, is proposed by combining RoBERTa-BiGRU embedding and a multi-head Graph ATtention Network (GAT). First, the pre-trained RoBERTa model is exploited to learn word and text embeddings in different contexts. Second, the Bidirectional Gated Recurrent Unit (BiGRU) is employed to capture long-term dependencies and bidirectional sentence information from the text context. Next, the multi-head graph attention network is applied to analyze this information, which serves as a node feature for the document. Finally, the classification results are generated through a Softmax layer. Experimental results on five benchmark datasets demonstrate that our method can achieve an accuracy of 71.48%, 98.45%, 80.32%, 90.84%, and 95.67% on Ohsumed, R8, MR, 20NG and R52, respectively, which is superior to the existing nine text classification approaches.
22
Lei, Zengyu, Caiming Zhang, Yunyang Xu, and Xuemei Li. "DR-GAT: Dynamic routing graph attention network for stock recommendation." Information Sciences 654 (January 2024): 119833. http://dx.doi.org/10.1016/j.ins.2023.119833.
23
Wan, Qizhi, Changxuan Wan, Keli Xiao, Kun Lu, Chenliang Li, Xiping Liu, and Dexi Liu. "Dependency Structure-Enhanced Graph Attention Networks for Event Detection." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 17 (March 24, 2024): 19098–106. http://dx.doi.org/10.1609/aaai.v38i17.29877.
Анотація:
Existing models on event detection share three-fold limitations, including (1) insufficient consideration of the structures between dependency relations, (2) limited exploration of the directed-edge semantics, and (3) issues in strengthening the event core arguments. To tackle these problems, we propose a dependency structure-enhanced event detection framework. In addition to the traditional token dependency parsing tree, denoted as TDG, our model considers the dependency edges in it as new nodes and constructs a dependency relation graph (DRG). DRG allows the embedding representations of dependency relations to be updated as nodes rather than edges in a graph neural network. Moreover, the levels of core argument nodes in the two graphs are adjusted by dependency relation types in TDG to enhance their status. Subsequently, the two graphs are further encoded and jointly trained in graph attention networks (GAT). Importantly, we design an interaction strategy of node embedding for the two graphs and refine the attention coefficient computational method to encode the semantic meaning of directed edges. Extensive experiments are conducted to validate the effectiveness of our method, and the results confirm its superiority over the state-of-the-art baselines. Our model outperforms the best benchmark with the F1 score increased by 3.5 and 3.4 percentage points on ACE2005 English and Chinese corpus.
24
Yang, Wu-Lue, Xiao-Ze Chen, and Xu-Hua Yang. "Semisupervised Classification with High-Order Graph Learning Attention Neural Network." Mathematical Problems in Engineering 2021 (December 7, 2021): 1–10. http://dx.doi.org/10.1155/2021/3911137.
Анотація:
At present, the graph neural network has achieved good results in the semisupervised classification of graph structure data. However, the classification effect is greatly limited in those data without graph structure, incomplete graph structure, or noise. It has no high prediction accuracy and cannot solve the problem of the missing graph structure. Therefore, in this paper, we propose a high-order graph learning attention neural network (HGLAT) for semisupervised classification. First, a graph learning module based on the improved variational graph autoencoder is proposed, which can learn and optimize graph structures for data sets without topological graph structure and data sets with missing topological structure and perform regular constraints on the generated graph structure to make the optimized graph structure more reasonable. Then, in view of the shortcomings of graph attention neural network (GAT) that cannot make full use of the graph high-order topology structure for node classification and graph structure learning, we propose a graph classification module that extends the attention mechanism to high-order neighbors, in which attention decays according to the increase of neighbor order. HGLAT performs joint optimization on the two modules of graph learning and graph classification and performs semisupervised node classification while optimizing the graph structure, which improves the classification performance. On 5 real data sets, by comparing 8 classification methods, the experiment shows that HGLAT has achieved good classification results on both a data set with graph structure and a data set without graph structure.
25
Cai, Fengze, Qiang Hu, Renjie Zhou, and Neal Xiong. "REEGAT: RoBERTa Entity Embedding and Graph Attention Networks Enhanced Sentence Representation for Relation Extraction." Electronics 12, no. 11 (May 27, 2023): 2429. http://dx.doi.org/10.3390/electronics12112429.
Анотація:
Relation extraction is one of the most important intelligent information extraction technologies, which can be used to construct and optimize services in intelligent communication systems (ICS). One issue with the existing relation extraction approaches is that they use one-sided sentence embedding as their final prediction vector, which degrades relation extraction performance. The innovative relation extraction model REEGAT (RoBERTa Entity Embedding and Graph Attention networks enhanced sentence representation) that we present in this paper, incorporates the concept of enhanced word embedding from graph neural networks. The model first uses RoBERTa to obtain word embedding and PyTorch embedding to obtain relation embedding. Then, the multi-headed attention mechanism in GAT (graph attention network) is introduced to weight the word embedding and relation embedding to enrich further the meaning conveyed by the word embedding. Finally, the entity embedding component is used to obtain sentence representation by pooling the word embedding from GAT and the entity embedding from named entity recognition. The weighted and pooled word embedding contains more relational information to alleviate the one-sided problem of sentence representation. The experimental findings demonstrate that our model outperforms other standard methods.
26
Cui, Wei, Xin He, Meng Yao, Ziwei Wang, Yuanjie Hao, Jie Li, Weijie Wu, et al. "Knowledge and Spatial Pyramid Distance-Based Gated Graph Attention Network for Remote Sensing Semantic Segmentation." Remote Sensing 13, no. 7 (March 30, 2021): 1312. http://dx.doi.org/10.3390/rs13071312.
Анотація:
The pixel-based semantic segmentation methods take pixels as recognitions units, and are restricted by the limited range of receptive fields, so they cannot carry richer and higher-level semantics. These reduce the accuracy of remote sensing (RS) semantic segmentation to a certain extent. Comparing with the pixel-based methods, the graph neural networks (GNNs) usually use objects as input nodes, so they not only have relatively small computational complexity, but also can carry richer semantic information. However, the traditional GNNs are more rely on the context information of the individual samples and lack geographic prior knowledge that reflects the overall situation of the research area. Therefore, these methods may be disturbed by the confusion of “different objects with the same spectrum” or “violating the first law of geography” in some areas. To address the above problems, we propose a remote sensing semantic segmentation model called knowledge and spatial pyramid distance-based gated graph attention network (KSPGAT), which is based on prior knowledge, spatial pyramid distance and a graph attention network (GAT) with gating mechanism. The model first uses superpixels (geographical objects) to form the nodes of a graph neural network and then uses a novel spatial pyramid distance recognition algorithm to recognize the spatial relationships. Finally, based on the integration of feature similarity and the spatial relationships of geographic objects, a multi-source attention mechanism and gating mechanism are designed to control the process of node aggregation, as a result, the high-level semantics, spatial relationships and prior knowledge can be introduced into a remote sensing semantic segmentation network. The experimental results show that our model improves the overall accuracy by 4.43% compared with the U-Net Network, and 3.80% compared with the baseline GAT network.
27
Cao, Hailin, Wang Zhu, Wenjuan Feng, and Jin Fan. "Robust Beamforming Based on Graph Attention Networks for IRS-Assisted Satellite IoT Communications." Entropy 24, no. 3 (February 24, 2022): 326. http://dx.doi.org/10.3390/e24030326.
Анотація:
Satellite communication is expected to play a vital role in realizing Internet of Remote Things (IoRT) applications. This article considers an intelligent reflecting surface (IRS)-assisted downlink low Earth orbit (LEO) satellite communication network, where IRS provides additional reflective links to enhance the intended signal power. We aim to maximize the sum-rate of all the terrestrial users by jointly optimizing the satellite’s precoding matrix and IRS’s phase shifts. However, it is difficult to directly acquire the instantaneous channel state information (CSI) and optimal phase shifts of IRS due to the high mobility of LEO and the passive nature of reflective elements. Moreover, most conventional solution algorithms suffer from high computational complexity and are not applicable to these dynamic scenarios. A robust beamforming design based on graph attention networks (RBF-GAT) is proposed to establish a direct mapping from the received pilots and dynamic network topology to the satellite and IRS’s beamforming, which is trained offline using the unsupervised learning approach. The simulation results corroborate that the proposed RBF-GAT approach can achieve more than 95% of the performance provided by the upper bound with low complexity.
28
Yang, Xiaohui, Hailong Ma, and Miao Wang. "Research on Rumor Detection Based on a Graph Attention Network With Temporal Features." International Journal of Data Warehousing and Mining 19, no. 2 (March 2, 2023): 1–17. http://dx.doi.org/10.4018/ijdwm.319342.
Анотація:
The higher-order and temporal characteristics of tweet sequences are often ignored in the field of rumor detection. In this paper, a new rumor detection method (T-BiGAT) is proposed to capture the temporal features between tweets by combining a graph attention network (GAT) and gated recurrent neural network (GRU). First, timestamps are calculated for each tweet within the same event. On the premise of the same timestamp, two different propagation subgraphs are constructed according to the response relationship between tweets. Then, GRU is used to capture intralayer dependencies between sibling nodes in the subtree; global features of each subtree are extracted using an improved GAT. Furthermore, GRU is reused to capture the temporal dependencies of individual subgraphs at different timestamps. Finally, weights are assigned to the global feature vectors of different timestamp subtrees for aggregation, and a mapping function is used to classify the aggregated vectors.
29
Cao, Ruifen, Chuan He, Pijing Wei, Yansen Su, Junfeng Xia, and Chunhou Zheng. "Prediction of circRNA-Disease Associations Based on the Combination of Multi-Head Graph Attention Network and Graph Convolutional Network." Biomolecules 12, no. 7 (July 2, 2022): 932. http://dx.doi.org/10.3390/biom12070932.
Анотація:
Circular RNAs (circRNAs) are covalently closed single-stranded RNA molecules, which have many biological functions. Previous experiments have shown that circRNAs are involved in numerous biological processes, especially regulatory functions. It has also been found that circRNAs are associated with complex diseases of human beings. Therefore, predicting the associations of circRNA with disease (called circRNA-disease associations) is useful for disease prevention, diagnosis and treatment. In this work, we propose a novel computational approach called GGCDA based on the Graph Attention Network (GAT) and Graph Convolutional Network (GCN) to predict circRNA-disease associations. Firstly, GGCDA combines circRNA sequence similarity, disease semantic similarity and corresponding Gaussian interaction profile kernel similarity, and then a random walk with restart algorithm (RWR) is used to obtain the preliminary features of circRNA and disease. Secondly, a heterogeneous graph is constructed from the known circRNA-disease association network and the calculated similarity of circRNAs and diseases. Thirdly, the multi-head Graph Attention Network (GAT) is adopted to obtain different weights of circRNA and disease features, and then GCN is employed to aggregate the features of adjacent nodes in the network and the features of the nodes themselves, so as to obtain multi-view circRNA and disease features. Finally, we combined a multi-layer fully connected neural network to predict the associations of circRNAs with diseases. In comparison with state-of-the-art methods, GGCDA can achieve AUC values of 0.9625 and 0.9485 under the results of fivefold cross-validation on two datasets, and AUC of 0.8227 on the independent test set. Case studies further demonstrate that our approach is promising for discovering potential circRNA-disease associations.
30
Yağci, Mehmet Yavuz, and Muhammed Ali Aydin. "EA-GAT: Event aware graph attention network on cyber-physical systems." Computers in Industry 159-160 (August 2024): 104097. http://dx.doi.org/10.1016/j.compind.2024.104097.
31
Zhang, Yuhang, Yaoqun Xu, and Yu Zhang. "A Graph Neural Network Node Classification Application Model with Enhanced Node Association." Applied Sciences 13, no. 12 (June 15, 2023): 7150. http://dx.doi.org/10.3390/app13127150.
Анотація:
This study combines the present stage of the node classification problem with the fact that there is frequent noise in the graph structure of the graph convolution calculation, which can lead to the omission of some of the actual edge relations between nodes and the appearance of numerous isolated nodes. In this paper, we propose the graph neural network model ENode-GAT for improving the accuracy of small sample node classification using the method of external referencing of similar word nodes, combined with Graph Convolutional Neural Network (GCN), Graph Attention Network (GAT), and the early stop algorithm. In order to demonstrate the applicability of the model, this paper employs two distinct types of node datasets for its investigations. The first is the Cora dataset, which is widely used in node classification at this time, and the second is a small-sample Stock dataset created by Eastern Fortune’s stock prospectus of the Science and Technology Board (STB). The experimental results demonstrate that the ENode-GAT model proposed in this paper obtains 85.1% classification accuracy on the Cora dataset and 85.3% classification accuracy on the Stock dataset, with certain classification advantages. It also verifies the future applicability of the model to the fields of stock classification, tender document classification, news classification, and government announcement classification, among others.
32
Ye, Haonan, and Xiao Luo. "Cascading Failure Analysis on Shanghai Metro Networks: An Improved Coupled Map Lattices Model Based on Graph Attention Networks." International Journal of Environmental Research and Public Health 19, no. 1 (December 25, 2021): 204. http://dx.doi.org/10.3390/ijerph19010204.
Анотація:
Analysis of the robustness and vulnerability of metro networks has great implications for public transport planning and emergency management, particularly considering passengers’ dynamic behaviors. This paper presents an improved coupled map lattices (CMLs) model based on graph attention networks (GAT) to study the cascading failure process of metro networks. The proposed model is applied to the Shanghai metro network using the automated fare collection (AFC) data, and the passengers’ dynamic behaviors are simulated by GAT. The quantitative cascading failure analysis shows that Shanghai metro network is robust to random attacks, but fragile to intentional attacks. Moreover, there is an approximately normal distribution between instant cascading failure speed and time step and the perturbation in a station which leads to steady state is approximately a constant. The result shows that a station surrounded by other densely distributed stations can trigger cascading failure faster and the cascading failure triggered by low-level accidents will spread in a short time and disappear quickly. This study provides an effective reference for dynamic safety evaluation and emergency management in metro networks.
33
Li, Yansheng, Ruixian Chen, Yongjun Zhang, Mi Zhang, and Ling Chen. "Multi-Label Remote Sensing Image Scene Classification by Combining a Convolutional Neural Network and a Graph Neural Network." Remote Sensing 12, no. 23 (December 7, 2020): 4003. http://dx.doi.org/10.3390/rs12234003.
Анотація:
As one of the fundamental tasks in remote sensing (RS) image understanding, multi-label remote sensing image scene classification (MLRSSC) is attracting increasing research interest. Human beings can easily perform MLRSSC by examining the visual elements contained in the scene and the spatio-topological relationships of these visual elements. However, most of existing methods are limited by only perceiving visual elements but disregarding the spatio-topological relationships of visual elements. With this consideration, this paper proposes a novel deep learning-based MLRSSC framework by combining convolutional neural network (CNN) and graph neural network (GNN), which is termed the MLRSSC-CNN-GNN. Specifically, the CNN is employed to learn the perception ability of visual elements in the scene and generate the high-level appearance features. Based on the trained CNN, one scene graph for each scene is further constructed, where nodes of the graph are represented by superpixel regions of the scene. To fully mine the spatio-topological relationships of the scene graph, the multi-layer-integration graph attention network (GAT) model is proposed to address MLRSSC, where the GAT is one of the latest developments in GNN. Extensive experiments on two public MLRSSC datasets show that the proposed MLRSSC-CNN-GNN can obtain superior performance compared with the state-of-the-art methods.
34
Mu, Jichong, Jihong Ouyang, Yachen Yao, and Zongxiao Ren. "Span-Prototype Graph Based on Graph Attention Network for Nested Named Entity Recognition." Electronics 12, no. 23 (November 23, 2023): 4753. http://dx.doi.org/10.3390/electronics12234753.
Анотація:
Named entity recognition, a fundamental task in natural language processing, faces challenges related to the sequence labeling framework widely used when dealing with nested entities. The span-based method transforms nested named entity recognition into span classification tasks, which makes it an efficient way to deal with overlapping entities. However, too much overlap among spans may confuse the model, leading to inaccurate classification performance. Moreover, the entity mentioned in the training dataset contains rich information about entities, which are not fully utilized. So, in this paper, a span-prototype graph is constructed to improve span representation and increase its distinction. In detail, we utilize the entity mentions in the training dataset to create a prototype for each entity category and add prototype loss to adapt the span to its similar prototype. Then, we feed prototypes and span into a graph attention network (GAT), enabling span to automatically learn from different prototypes, which integrate the information about entities into the span representation. Experiments on three common nested named entity recognition datasets, including ACE2004, ACE2005, and GENIA, show that the proposed method achieves 87.28%, 85.97%, and 79.74% F1 scores on ACE2004, ACE2005, and GENIA, respectively, performing better than baselines.
35
Yang, Xiaohui, Hailong Ma, and Miao Wang. "Rumor Detection with Bidirectional Graph Attention Networks." Security and Communication Networks 2022 (January 18, 2022): 1–13. http://dx.doi.org/10.1155/2022/4840997.
Анотація:
In order to extract the relevant features of rumors effectively, this paper proposes a novel rumor detection model with bidirectional graph attention network on the basis of constructing a directed graph, named P-BiGAT. Firstly, this model builds the propagation tree and diffusion tree through the tweet comment and reposting relationship. Secondly, the improved graph attention network (GAT) is used to extract the propagation feature and the diffusion feature through two different directions, and the multihead attention mechanism is used to extract the semantic information of the source tweet. Finally, the propagation feature, diffusion feature, and semantic information representation of the source tweet are connected together through a fully connected layer, and the mapping function is used to determine the authenticity of the information. In addition, this paper also proposes a new node update method and applies it to the model in order to select neighbor node information effectively. Specifically, it can select the neighbor information node with larger weight to update the node according to the weight of the neighbor node. The results of the experiment show that the model is better than the baseline method of comparison in accuracy, precision, recall, and F1 measure on the public datasets.
36
Ji, Cunmei, Zhihao Liu, Yutian Wang, Jiancheng Ni, and Chunhou Zheng. "GATNNCDA: A Method Based on Graph Attention Network and Multi-Layer Neural Network for Predicting circRNA-Disease Associations." International Journal of Molecular Sciences 22, no. 16 (August 7, 2021): 8505. http://dx.doi.org/10.3390/ijms22168505.
Анотація:
Circular RNAs (circRNAs) are a new class of endogenous non-coding RNAs with covalent closed loop structure. Researchers have revealed that circRNAs play an important role in human diseases. As experimental identification of interactions between circRNA and disease is time-consuming and expensive, effective computational methods are an urgent need for predicting potential circRNA–disease associations. In this study, we proposed a novel computational method named GATNNCDA, which combines Graph Attention Network (GAT) and multi-layer neural network (NN) to infer disease-related circRNAs. Specially, GATNNCDA first integrates disease semantic similarity, circRNA functional similarity and the respective Gaussian Interaction Profile (GIP) kernel similarities. The integrated similarities are used as initial node features, and then GAT is applied for further feature extraction in the heterogeneous circRNA–disease graph. Finally, the NN-based classifier is introduced for prediction. The results of fivefold cross validation demonstrated that GATNNCDA achieved an average AUC of 0.9613 and AUPR of 0.9433 on the CircR2Disease dataset, and outperformed other state-of-the-art methods. In addition, case studies on breast cancer and hepatocellular carcinoma showed that 20 and 18 of the top 20 candidates were respectively confirmed in the validation datasets or published literature. Therefore, GATNNCDA is an effective and reliable tool for discovering circRNA–disease associations.
37
Baul, Sudipto, Khandakar Tanvir Ahmed, Joseph Filipek, and Wei Zhang. "omicsGAT: Graph Attention Network for Cancer Subtype Analyses." International Journal of Molecular Sciences 23, no. 18 (September 6, 2022): 10220. http://dx.doi.org/10.3390/ijms231810220.
Анотація:
The use of high-throughput omics technologies is becoming increasingly popular in all facets of biomedical science. The mRNA sequencing (RNA-seq) method reports quantitative measures of more than tens of thousands of biological features. It provides a more comprehensive molecular perspective of studied cancer mechanisms compared to traditional approaches. Graph-based learning models have been proposed to learn important hidden representations from gene expression data and network structure to improve cancer outcome prediction, patient stratification, and cell clustering. However, these graph-based methods cannot rank the importance of the different neighbors for a particular sample in the downstream cancer subtype analyses. In this study, we introduce omicsGAT, a graph attention network (GAT) model to integrate graph-based learning with an attention mechanism for RNA-seq data analysis. The multi-head attention mechanism in omicsGAT can more effectively secure information of a particular sample by assigning different attention coefficients to its neighbors. Comprehensive experiments on The Cancer Genome Atlas (TCGA) breast cancer and bladder cancer bulk RNA-seq data and two single-cell RNA-seq datasets validate that (1) the proposed model can effectively integrate neighborhood information of a sample and learn an embedding vector to improve disease phenotype prediction, cancer patient stratification, and cell clustering of the sample and (2) the attention matrix generated from the multi-head attention coefficients provides more useful information compared to the sample correlation-based adjacency matrix. From the results, we can conclude that some neighbors play a more important role than others in cancer subtype analyses of a particular sample based on the attention coefficient.
38
Wu, Xingping, Qiheng Yuan, Chunlei Zhou, Xiang Chen, Donghai Xuan, and Jinwei Song. "Carbon emissions forecasting based on temporal graph transformer-based attentional neural network." Journal of Computational Methods in Sciences and Engineering 24, no. 3 (June 17, 2024): 1405–21. http://dx.doi.org/10.3233/jcm-247139.
Анотація:
In the field of electric carbon, the mapping relationship between carbon emission flow calculation and power flow calculation was studied by combining techniques such as current trajectory tracking, carbon flow trajectory analysis, power system flow calculation methods, and electric network analysis theory. By delving into the mechanism between these two factors, a better understanding of the correlation between them can be achieved. In addition, by using time series data, graph attention neural networks (GNN), distributed computing technology, and spatiotemporal computing engines, carbon emission fluctuations can be decomposed and a high-frequency “energy-electricity-carbon” integrated dynamic emission factor can be achieved. Through the spatiotemporal distribution patterns of this dynamic factor in multiple dimensions, the carbon emissions from key industries in cities can be accurately calculated. In this paper, the LSTM-GAT model is used as the core to construct a key carbon emission prediction model for cities. The study focuses on the power plant, chemical industry, steel, transportation industry, and construction industry, which are high energy-consuming industries with an annual electricity consumption of more than 100 million kWh in a major city of China. By analyzing the entire life cycle from power generation to electricity consumption and conducting current flow analysis, monthly, weekly, and daily carbon emission calculations were performed. Additionally, other factors such as the industrial development index, GDP, coverage area of power generation enterprises, regional population, size, and type of power-consuming units were included in the comprehensive calculation to build a measurement system. By conducting experiments and analyzing historical data, we have found that the LSTM-GAT model outperforms the single models of GCN, GAT, LSTM, GRU, and RNN in terms of lower error values and higher accuracy. The LSTM-GAT model is better suited for predicting carbon emissions and related indicators with an accuracy rate of 89.5%. Our predictions show that the carbon emissions will exhibit a slow growth trend in the future, while the carbon emission intensity will decrease. This information can provide a scientific basis for government decision-making.
39
Alothali, Eiman, Motamen Salih, Kadhim Hayawi, and Hany Alashwal. "Bot-MGAT: A Transfer Learning Model Based on a Multi-View Graph Attention Network to Detect Social Bots." Applied Sciences 12, no. 16 (August 13, 2022): 8117. http://dx.doi.org/10.3390/app12168117.
Анотація:
Twitter, as a popular social network, has been targeted by different bot attacks. Detecting social bots is a challenging task, due to their evolving capacity to avoid detection. Extensive research efforts have proposed different techniques and approaches to solving this problem. Due to the scarcity of recently updated labeled data, the performance of detection systems degrades when exposed to a new dataset. Therefore, semi-supervised learning (SSL) techniques can improve performance, using both labeled and unlabeled examples. In this paper, we propose a framework based on the multi-view graph attention mechanism using a transfer learning (TL) approach, to predict social bots. We called the framework ‘Bot-MGAT’, which stands for bot multi-view graph attention network. The framework used both labeled and unlabeled data. We used profile features to reduce the overheads of the feature engineering. We executed our experiments on a recent benchmark dataset that included representative samples of social bots with graph structural information and profile features only. We applied cross-validation to avoid uncertainty in the model’s performance. Bot-MGAT was evaluated using graph SSL techniques: single graph attention networks (GAT), graph convolutional networks (GCN), and relational graph convolutional networks (RGCN). We compared Bot-MGAT to related work in the field of bot detection. The results of Bot-MGAT with TL outperformed, with an accuracy score of 97.8%, an F1 score of 0.9842, and an MCC score of 0.9481.
40
Wei, Pengfei, Bi Zeng, and Wenxiong Liao. "Joint intent detection and slot filling with wheel-graph attention networks." Journal of Intelligent & Fuzzy Systems 42, no. 3 (February 2, 2022): 2409–20. http://dx.doi.org/10.3233/jifs-211674.
Анотація:
Intent detection and slot filling are recognized as two very important tasks in a spoken language understanding (SLU) system. In order to model these two tasks at the same time, many joint models based on deep neural networks have been proposed recently and archived excellent results. In addition, graph neural network has made good achievements in the field of vision. Therefore, we combine these two advantages and propose a new joint model with a wheel-graph attention network (Wheel-GAT), which is able to model interrelated connections directly for single intent detection and slot filling. To construct a graph structure for utterances, we create intent nodes, slot nodes, and directed edges. Intent nodes can provide utterance-level semantic information for slot filling, while slot nodes can also provide local keyword information for intent detection. The two tasks promote each other and carry out end-to-end training at the same time. Experiments show that our proposed approach is superior to multiple baselines on ATIS and SNIPS datasets. Besides, we also demonstrate that using bi-directional encoder representation from transformer (BERT) model further boosts the performance of the SLU task.
41
Zhao, Mingxiu, Jing Zhang, Qin Li, Junzheng Yang, Estevao Siga, and Tianchi Zhang. "GAT-ABiGRU Based Prediction Model for AUV Trajectory." Applied Sciences 14, no. 10 (May 15, 2024): 4184. http://dx.doi.org/10.3390/app14104184.
Анотація:
Autonomous underwater vehicles (AUVs) are critical components of current maritime operations. However, because of the complicated marine environment, AUVs are at significant risk of being lost, and such losses significantly impact the continuity and safety of aquatic activities. This article suggests a methodology for forecasting the trajectory of lost autonomous underwater vehicles (AUVs) based on GAT-ABiGRU. Firstly, the time-series data of the AUV are transformed into a graph structure to represent the dependencies between data points. Secondly, a graph attention network is utilized to capture the spatial features of the trajectory data, while an attention-based bidirectional gated recurrent unit network learns the temporal features of the trajectory data; finally, the predicted drift trajectory is obtained. The findings show that the GAT-ABiGRU model outperforms previous trajectory prediction models, is highly accurate and robust in drift trajectory prediction, and presents a new method for forecasting the trajectory of wrecked AUVs.
42
Umair, Muhammad, Iftikhar Alam, Atif Khan, Inayat Khan, Niamat Ullah, and Mohammad Yusuf Momand. "N-GPETS: Neural Attention Graph-Based Pretrained Statistical Model for Extractive Text Summarization." Computational Intelligence and Neuroscience 2022 (November 22, 2022): 1–14. http://dx.doi.org/10.1155/2022/6241373.
Анотація:
The extractive summarization approach involves selecting the source document’s salient sentences to build a summary. One of the most important aspects of extractive summarization is learning and modelling cross-sentence associations. Inspired by the popularity of Transformer-based Bidirectional Encoder Representations (BERT) pretrained linguistic model and graph attention network (GAT) having a sophisticated network that captures intersentence associations, this research work proposes a novel neural model N-GPETS by combining heterogeneous graph attention network with BERT model along with statistical approach using TF-IDF values for extractive summarization task. Apart from sentence nodes, N-GPETS also works with different semantic word nodes of varying granularity levels that serve as a link between sentences, improving intersentence interaction. Furthermore, proposed N-GPETS becomes more improved and feature-rich by integrating graph layer with BERT encoder at graph initialization step rather than employing other neural network encoders such as CNN or LSTM. To the best of our knowledge, this work is the first attempt to combine the BERT encoder and TF-IDF values of the entire document with a heterogeneous attention graph structure for the extractive summarization task. The empirical outcomes on benchmark news data sets CNN/DM show that the proposed model N-GPETS gets favorable results in comparison with other heterogeneous graph structures employing the BERT model and graph structures without the BERT model.
43
Chen, Yang, Weibing Wan, Jimi Hu, Yuxuan Wang, and Bo Huang. "Complex Causal Extraction of Fusion of Entity Location Sensing and Graph Attention Networks." Information 13, no. 8 (July 31, 2022): 364. http://dx.doi.org/10.3390/info13080364.
Анотація:
At present, there is no uniform definition of annotation schemes for causal extraction, and existing methods are limited by the dependence of relations on long spans, which makes complex sentences such as multi-causal relations and nested causal relations difficult to extract. To solve these problems, a head-to-tail entity annotation method is proposed, which can express the complete semantics of complex causal relations and clearly describe the boundaries of entities. Based on this, a causal model, RPA-GCN (relation position and attention-graph convolutional networks), is constructed, incorporating GAT (graph attention network) and entity location perception. The attention layer is combined with a dependency tree to enhance the model’s ability to perceive relational features, and a bi-directional graph convolutional network is constructed to further capture the deep interaction information between entities and relationships. Finally, the classifier iteratively predicts the relationship of each word pair in the sentence and analyzes all causal pairs in the sentence by a scoring function. Experiments on SemEval 2010 task 8 and the Altlex dataset show that our proposed method has significant advantages in solving complex causal extraction compared to state-of-the-art methods.
44
Zhou, Hang, Weikun Wang, Jiayun Jin, Zengwei Zheng, and Binbin Zhou. "Graph Neural Network for Protein–Protein Interaction Prediction: A Comparative Study." Molecules 27, no. 18 (September 19, 2022): 6135. http://dx.doi.org/10.3390/molecules27186135.
Анотація:
Proteins are the fundamental biological macromolecules which underline practically all biological activities. Protein–protein interactions (PPIs), as they are known, are how proteins interact with other proteins in their environment to perform biological functions. Understanding PPIs reveals how cells behave and operate, such as the antigen recognition and signal transduction in the immune system. In the past decades, many computational methods have been developed to predict PPIs automatically, requiring less time and resources than experimental techniques. In this paper, we present a comparative study of various graph neural networks for protein–protein interaction prediction. Five network models are analyzed and compared, including neural networks (NN), graph convolutional neural networks (GCN), graph attention networks (GAT), hyperbolic neural networks (HNN), and hyperbolic graph convolutions (HGCN). By utilizing the protein sequence information, all of these models can predict the interaction between proteins. Fourteen PPI datasets are extracted and utilized to compare the prediction performance of all these methods. The experimental results show that hyperbolic graph neural networks tend to have a better performance than the other methods on the protein-related datasets.
45
Shao, Yingzhao, Yunsong Li, Li Li, Yuanle Wang, Yuchen Yang, Yueli Ding, Mingming Zhang, Yang Liu, and Xiangqiang Gao. "RANet: Relationship Attention for Hyperspectral Anomaly Detection." Remote Sensing 15, no. 23 (November 30, 2023): 5570. http://dx.doi.org/10.3390/rs15235570.
Анотація:
Hyperspectral anomaly detection (HAD) is of great interest for unknown exploration. Existing methods only focus on local similarity, which may show limitations in detection performance. To cope with this problem, we propose a relationship attention-guided unsupervised learning with convolutional autoencoders (CAEs) for HAD, called RANet. First, instead of only focusing on the local similarity, RANet, for the first time, pays attention to topological similarity by leveraging the graph attention network (GAT) to capture deep topological relationships embedded in a customized incidence matrix from absolutely unlabeled data mixed with anomalies. Notably, the attention intensity of GAT is self-adaptively controlled by adjacency reconstruction ability, which can effectively reduce human intervention. Next, we adopt an unsupervised CAE to jointly learn with the topological relationship attention to achieve satisfactory model performance. Finally, on the basis of background reconstruction, we detect anomalies by the reconstruction error. Extensive experiments on hyperspectral images (HSIs) demonstrate that our proposed RANet outperforms existing fully unsupervised methods.
46
Gao, Yunmeng, Liang Zhao, Jin Du, and Junnan Wang. "Spatial-temporal Traffic Flow Prediction Model Based on the GAT and BiGRU." Journal of Physics: Conference Series 2589, no. 1 (September 1, 2023): 012024. http://dx.doi.org/10.1088/1742-6596/2589/1/012024.
Анотація:
Abstract Real-time and accurate traffic flow prediction is crucial for improving the safety, stability, and efficiency of intelligent transportation system. Considering that traffic flow prediction methods rarely analyze from the perspective of the road network, in this paper, a spatial-temporal traffic flow prediction model based on the combination of graph attention network (GAT) and bidirectional gated recurrent unit (BiGRU) neural network is proposed. Firstly, GAT is used to analyze the complex topology of the road network, effectively obtaining the spatial features of the road network. Secondly, BiGRU is used to learn the dynamic changes of traffic flow data, effectively obtaining the temporal features. Thirdly, the obtained spatial-temporal features are output by the fully connected layer to complete the prediction of future traffic flow. Finally, the model is validated and evaluated on the California highway dataset. The experimental results show that the accuracy of GAT-BiGRU model is better than other benchmark models in predicting future traffic flows transformation, especially in long-term prediction.
47
Xu, Dawei, Qing Liu, Liehuang Zhu, Zhonghua Tan, Feng Gao, and Jian Zhao. "GCNRDM: A Social Network Rumor Detection Method Based on Graph Convolutional Network in Mobile Computing." Wireless Communications and Mobile Computing 2021 (October 8, 2021): 1–11. http://dx.doi.org/10.1155/2021/1690669.
Анотація:
Mobile computing is a new technology emerging with the development of mobile communication, Internet, database, distributed computing, and other technologies. Mobile computing technology will enable computers or other information intelligent terminal devices to realize data transmission and resource sharing in the wireless environment. Its role is to bring useful, accurate, and timely information to any customer at anytime, anywhere, and to change the way people live and work. In mobile computing environment, a lot of Internet rumors hidden among the huge amounts of information communication network can cause harm to society and people’s life; this paper proposes a model of social network rumor detection based on convolution networks, the use of adjacency matrix between the nodes represent user and the relationship between the constructions of social network topology. We use a high-order graph neural network (K-GNN) to extract the rumor posting features. At the same time, the graph attention network (GAT) is used to extract the association features of other nodes of the network topology. The experimental results show that the method of the detection model in this paper improves the accuracy of prediction classification compared with deep learning methods such as RNN, GRU, and attention mechanism. The innovation of the paper proposes a rumor detection model based on the graph convolutional network, which lies in considering the propagation structure among users. It has a strong practical value.
48
Peng, Feifei, Wei Lu, Wenxia Tan, Kunlun Qi, Xiaokang Zhang, and Quansheng Zhu. "Multi-Output Network Combining GNN and CNN for Remote Sensing Scene Classification." Remote Sensing 14, no. 6 (March 18, 2022): 1478. http://dx.doi.org/10.3390/rs14061478.
Анотація:
Scene classification is an active research area in the remote sensing (RS) domain. Some categories of RS scenes, such as medium residential and dense residential scenes, would contain the same type of geographical objects but have various spatial distributions among these objects. The adjacency and disjointness relationships among geographical objects are normally neglected by existing RS scene classification methods using convolutional neural networks (CNNs). In this study, a multi-output network (MopNet) combining a graph neural network (GNN) and a CNN is proposed for RS scene classification with a joint loss. In a candidate RS image for scene classification, superpixel regions are constructed through image segmentation and are represented as graph nodes, while graph edges between nodes are created according to the spatial adjacency among corresponding superpixel regions. A training strategy of a jointly learning CNN and GNN is adopted in the MopNet. Through the message propagation mechanism of MopNet, spatial and topological relationships imbedded in the edges of graphs are employed. The parameters of the CNN and GNN in MopNet are updated simultaneously with the guidance of a joint loss via the backpropagation mechanism. Experimental results on the OPTIMAL-31 and aerial image dataset (AID) datasets show that the proposed MopNet combining a graph convolutional network (GCN) or graph attention network (GAT) and ResNet50 achieves state-of-the-art accuracy. The overall accuracy obtained on OPTIMAL-31 is 96.06% and those on AID are 95.53% and 97.11% under training ratios of 20% and 50%, respectively. Spatial and topological relationships imbedded in RS images are helpful for improving the performance of scene classification.
49
Cui, Wei, Yuanjie Hao, Xing Xu, Zhanyun Feng, Huilin Zhao, Cong Xia, and Jin Wang. "Remote Sensing Scene Graph and Knowledge Graph Matching with Parallel Walking Algorithm." Remote Sensing 14, no. 19 (September 29, 2022): 4872. http://dx.doi.org/10.3390/rs14194872.
Анотація:
In deep neural network model training and prediction, due to the limitation of GPU memory and computing resources, massive image data must be cropped into limited-sized samples. Moreover, in order to improve the generalization ability of the model, the samples need to be randomly distributed in the experimental area. Thus, the background information is often incomplete or even missing. On this condition, a knowledge graph must be applied to the semantic segmentation of remote sensing. However, although a single sample contains only a limited number of geographic categories, the combinations of geographic objects are diverse and complex in different samples. Additionally, the involved categories of geographic objects often span different classification system branches. Therefore, existing studies often directly regard all the categories involved in the knowledge graph as candidates for specific sample segmentation, which leads to high computation cost and low efficiency. To address the above problems, a parallel walking algorithm based on cross modality information is proposed for the scene graph—knowledge graph matching (PWGM). The algorithm uses a graph neural network to map the visual features of the scene graph into the semantic space of the knowledge graph through anchors and designs a parallel walking algorithm of the knowledge graph that takes into account the visual features of complex scenes. Based on the algorithm, we propose a semantic segmentation model for remote sensing. The experiments demonstrate that our model improves the overall accuracy by 3.7% compared with KGGAT (which is a semantic segmentation model using a knowledge graph and graph attention network (GAT)), by 5.1% compared with GAT and by 13.3% compared with U-Net. Our study not only effectively improves the recognition accuracy and efficiency of remote sensing objects, but also offers useful exploration for the development of deep learning from a data-driven to a data-knowledge dual drive.
50
Zhan, Huixin, Kun Zhang, Keyi Lu, and Victor S. Sheng. "Measuring the Privacy Leakage via Graph Reconstruction Attacks on Simplicial Neural Networks (Student Abstract)." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 13 (June 26, 2023): 16380–81. http://dx.doi.org/10.1609/aaai.v37i13.27050.
Анотація:
In this paper, we measure the privacy leakage via studying whether graph representations can be inverted to recover the graph used to generate them via graph reconstruction attack (GRA). We propose a GRA that recovers a graph's adjacency matrix from the representations via a graph decoder that minimizes the reconstruction loss between the partial graph and the reconstructed graph. We study three types of representations that are trained on the graph, i.e., representations output from graph convolutional network (GCN), graph attention network (GAT), and our proposed simplicial neural network (SNN) via a higher-order combinatorial Laplacian. Unlike the first two types of representations that only encode pairwise relationships, the third type of representation, i.e., SNN outputs, encodes higher-order interactions (e.g., homological features) between nodes. We find that the SNN outputs reveal the lowest privacy-preserving ability to defend the GRA, followed by those of GATs and GCNs, which indicates the importance of building more private representations with higher-order node information that could defend the potential threats, such as GRAs.