Journal articles on the topic 'Transfer learning (TL)'
To see the other types of publications on this topic, follow the link: Transfer learning (TL).
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Transfer learning (TL).'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Nishida, Satoshi, Yusuke Nakano, Antoine Blanc, Naoya Maeda, Masataka Kado, and Shinji Nishimoto. "Brain-Mediated Transfer Learning of Convolutional Neural Networks." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 5281–88. http://dx.doi.org/10.1609/aaai.v34i04.5974.
Full textAbstract:
The human brain can effectively learn a new task from a small number of samples, which indicates that the brain can transfer its prior knowledge to solve tasks in different domains. This function is analogous to transfer learning (TL) in the field of machine learning. TL uses a well-trained feature space in a specific task domain to improve performance in new tasks with insufficient training data. TL with rich feature representations, such as features of convolutional neural networks (CNNs), shows high generalization ability across different task domains. However, such TL is still insufficient in making machine learning attain generalization ability comparable to that of the human brain. To examine if the internal representation of the brain could be used to achieve more efficient TL, we introduce a method for TL mediated by human brains. Our method transforms feature representations of audiovisual inputs in CNNs into those in activation patterns of individual brains via their association learned ahead using measured brain responses. Then, to estimate labels reflecting human cognition and behavior induced by the audiovisual inputs, the transformed representations are used for TL. We demonstrate that our brain-mediated TL (BTL) shows higher performance in the label estimation than the standard TL. In addition, we illustrate that the estimations mediated by different brains vary from brain to brain, and the variability reflects the individual variability in perception. Thus, our BTL provides a framework to improve the generalization ability of machine-learning feature representations and enable machine learning to estimate human-like cognition and behavior, including individual variability.
2
Yu, Fuchao, Xianchao Xiu, and Yunhui Li. "A Survey on Deep Transfer Learning and Beyond." Mathematics 10, no. 19 (October 3, 2022): 3619. http://dx.doi.org/10.3390/math10193619.
Full textAbstract:
Deep transfer learning (DTL), which incorporates new ideas from deep neural networks into transfer learning (TL), has achieved excellent success in computer vision, text classification, behavior recognition, and natural language processing. As a branch of machine learning, DTL applies end-to-end learning to overcome the drawback of traditional machine learning that regards each dataset individually. Although some valuable and impressive general surveys exist on TL, special attention and recent advances in DTL are lacking. In this survey, we first review more than 50 representative approaches of DTL in the last decade and systematically summarize them into four categories. In particular, we further divide each category into subcategories according to models, functions, and operation objects. In addition, we discuss recent advances in TL in other fields and unsupervised TL. Finally, we provide some possible and exciting future research directions.
3
Cho, Seong Hee, Seokgoo Kim, and Joo-Ho Choi. "Transfer Learning-Based Fault Diagnosis under Data Deficiency." Applied Sciences 10, no. 21 (November 3, 2020): 7768. http://dx.doi.org/10.3390/app10217768.
Full textAbstract:
In the fault diagnosis study, data deficiency, meaning that the fault data for the training are scarce, is often encountered, and it may deteriorate the performance of the fault diagnosis greatly. To solve this issue, the transfer learning (TL) approach is employed to exploit the neural network (NN) trained in another (source) domain where enough fault data are available in order to improve the NN performance of the real (target) domain. While there have been similar attempts of TL in the literature to solve the imbalance issue, they were about the sample imbalance between the source and target domain, whereas the present study considers the imbalance between the normal and fault data. To illustrate this, normal and fault datasets are acquired from the linear motion guide, in which the data at high and low speeds represent the real operation (target) and maintenance inspection (source), respectively. The effect of data deficiency is studied by reducing the number of fault data in the target domain, and comparing the performance of TL, which exploits the knowledge of the source domain and the ordinary machine learning (ML) approach without it. By examining the accuracy of the fault diagnosis as a function of imbalance ratio, it is found that the lower bound and interquartile range (IQR) of the accuracy are improved greatly by employing the TL approach. Therefore, it can be concluded that TL is truly more effective than the ordinary ML when there is a large imbalance between the fault and normal data, such as smaller than 0.1.
4
Li, Yuyang, Bolin Fu, Xidong Sun, Donglin Fan, Yeqiao Wang, Hongchang He, Ertao Gao, Wen He, and Yuefeng Yao. "Comparison of Different Transfer Learning Methods for Classification of Mangrove Communities Using MCCUNet and UAV Multispectral Images." Remote Sensing 14, no. 21 (November 2, 2022): 5533. http://dx.doi.org/10.3390/rs14215533.
Full textAbstract:
Mangrove-forest classification by using deep learning algorithms has attracted increasing attention but remains challenging. The current studies on the transfer classification of mangrove communities between different regions and different sensors are especially still unclear. To fill the research gap, this study developed a new deep-learning algorithm (encoder–decoder with mixed depth-wise convolution and cascade upsampling, MCCUNet) by modifying the encoder and decoder sections of the DeepLabV3+ algorithm and presented three transfer-learning strategies, namely frozen transfer learning (F-TL), fine-tuned transfer learning (Ft-TL), and sensor-and-phase transfer learning (SaP-TL), to classify mangrove communities by using the MCCUNet algorithm and high-resolution UAV multispectral images. This study combined the deep-learning algorithms with recursive feature elimination and principal component analysis (RFE–PCA), using a high-dimensional dataset to map and classify mangrove communities, and evaluated their classification performance. The results of this study showed the following: (1) The MCCUNet algorithm outperformed the original DeepLabV3+ algorithm for classifying mangrove communities, achieving the highest overall classification accuracy (OA), i.e., 97.24%, in all scenarios. (2) The RFE–PCA dimension reduction improved the classification performance of deep-learning algorithms. The OA of mangrove species from using the MCCUNet algorithm was improved by 7.27% after adding dimension-reduced texture features and vegetation indices. (3) The Ft-TL strategy enabled the algorithm to achieve better classification accuracy and stability than the F-TL strategy. The highest improvement in the F1–score of Spartina alterniflora was 19.56%, using the MCCUNet algorithm with the Ft-TL strategy. (4) The SaP-TL strategy produced better transfer-learning classifications of mangrove communities between images of different phases and sensors. The highest improvement in the F1–score of Aegiceras corniculatum was 19.85%, using the MCCUNet algorithm with the SaP-TL strategy. (5) All three transfer-learning strategies achieved high accuracy in classifying mangrove communities, with the mean F1–score of 84.37%~95.25%.
5
Chen, Muzi. "Analysis on Transfer Learning Models and Applications in Natural Language Processing." Highlights in Science, Engineering and Technology 16 (November 10, 2022): 446–52. http://dx.doi.org/10.54097/hset.v16i.2609.
Full textAbstract:
Assumptions have been established that many machine learning algorithms expect the training data and the testing data to share the same feature space or distribution. Thus, transfer learning (TL) rises due to the tolerance of the different feature spaces and the distribution of data. It is an optimization to improve performance from task to task. This paper includes the basic knowledge of transfer learning and summarizes some relevant experimental results of popular applications using transfer learning in the natural language processing (NLP) field. The mathematical definition of TL is briefly mentioned. After that, basic knowledge including the different categories of TL, and the comparison between TL and traditional machine learning models is introduced. Then, some applications which mainly focus on question answering, cyberbullying detection, and sentiment analysis will be presented. Other applications will also be briefly introduced such as Named Entity Recognition (NER), Intent Classification, and Cross-Lingual Learning, etc. For each application, this study provides reference on transfer learning models for related researches.
6
Li, Zhichao, and Jinwei Dong. "A Framework Integrating DeeplabV3+, Transfer Learning, Active Learning, and Incremental Learning for Mapping Building Footprints." Remote Sensing 14, no. 19 (September 22, 2022): 4738. http://dx.doi.org/10.3390/rs14194738.
Full textAbstract:
Convolutional neural network (CNN)-based remote sensing (RS) image segmentation has become a widely used method for building footprint mapping. Recently, DeeplabV3+, an advanced CNN architecture, has shown satisfactory performance for building extraction in different urban landscapes. However, it faces challenges due to the large amount of labeled data required for model training and the extremely high costs associated with the annotation of unlabelled data. These challenges encouraged us to design a framework for building footprint mapping with fewer labeled data. In this context, the published studies on RS image segmentation are reviewed first, with a particular emphasis on the use of active learning (AL), incremental learning (IL), transfer learning (TL), and their integration for reducing the cost of data annotation. Based on the literature review, we defined three candidate frameworks by integrating AL strategies (i.e., margin sampling, entropy, and vote entropy), IL, TL, and DeeplabV3+. They examine the efficacy of AL, the efficacy of IL in accelerating AL performance, and the efficacy of both IL and TL in accelerating AL performance, respectively. Additionally, these frameworks enable the iterative selection of image tiles to be annotated, training and evaluation of DeeplabV3+, and quantification of the landscape features of selected image tiles. Then, all candidate frameworks were examined using WHU aerial building dataset as it has sufficient (i.e., 8188) labeled image tiles with representative buildings (i.e., various densities, areas, roof colors, and shapes of the building). The results support our theoretical analysis: (1) all three AL strategies reduced the number of image tiles by selecting the most informative image tiles, and no significant differences were observed in their performance; (2) image tiles with more buildings and larger building area were proven to be informative for the three AL strategies, which were prioritized during the data selection process; (3) IL can expedite model training by accumulating knowledge from chosen labeled tiles; (4) TL provides a better initial learner by incorporating knowledge from a pre-trained model; (5) DeeplabV3+ incorporated with IL, TL, and AL has the best performance in reducing the cost of data annotation. It achieved good performance (i.e., mIoU of 0.90) using only 10–15% of the sample dataset; DeeplabV3+ needs 50% of the sample dataset to realize the equivalent performance. The proposed frameworks concerning DeeplabV3+ and the results imply that integrating TL, AL, and IL in human-in-the-loop building extraction could be considered in real-world applications, especially for building footprint mapping.
7
Jeon, Ho-Kun, Seungryong Kim, Jonathan Edwin, and Chan-Su Yang. "Sea Fog Identification from GOCI Images Using CNN Transfer Learning Models." Electronics 9, no. 2 (February 11, 2020): 311. http://dx.doi.org/10.3390/electronics9020311.
Full textAbstract:
This study proposes an approaching method of identifying sea fog by using Geostationary Ocean Color Imager (GOCI) data through applying a Convolution Neural Network Transfer Learning (CNN-TL) model. In this study, VGG19 and ResNet50, pre-trained CNN models, are used for their high identification performance. The training and testing datasets were extracted from GOCI images for the area of coastal regions of the Korean Peninsula for six days in March 2015. With varying band combinations and changing whether Transfer Learning (TL) is applied, identification experiments were executed. TL enhanced the performance of the two models. Training data of CNN-TL showed up to 96.3% accuracy in matching, both with VGG19 and ResNet50, identically. Thus, it is revealed that CNN-TL is effective for the detection of sea fog from GOCI imagery.
8
Xin, Baogui, and Wei Peng. "Prediction for Chaotic Time Series-Based AE-CNN and Transfer Learning." Complexity 2020 (September 16, 2020): 1–9. http://dx.doi.org/10.1155/2020/2680480.
Full textAbstract:
It has been a hot and challenging topic to predict the chaotic time series in the medium-to-long term. We combine autoencoders and convolutional neural networks (AE-CNN) to capture the intrinsic certainty of chaotic time series. We utilize the transfer learning (TL) theory to improve the prediction performance in medium-to-long term. Thus, we develop a prediction scheme for chaotic time series-based AE-CNN and TL named AE-CNN-TL. Our experimental results show that the proposed AE-CNN-TL has much better prediction performance than any one of the following: AE-CNN, ARMA, and LSTM.
9
Wang, Peng (Edward), and Matthew Russell. "Domain Adversarial Transfer Learning for Generalized Tool Wear Prediction." Annual Conference of the PHM Society 12, no. 1 (November 3, 2020): 8. http://dx.doi.org/10.36001/phmconf.2020.v12i1.1137.
Full textAbstract:
Given its demonstrated ability in analyzing and revealing patterns underlying data, Deep Learning (DL) has been increasingly investigated to complement physics-based models in various aspects of smart manufacturing, such as machine condition monitoring and fault diagnosis, complex manufacturing process modeling, and quality inspection. However, successful implementation of DL techniques relies greatly on the amount, variety, and veracity of data for robust network training. Also, the distributions of data used for network training and application should be identical to avoid the internal covariance shift problem that reduces the network performance applicability. As a promising solution to address these challenges, Transfer Learning (TL) enables DL networks trained on a source domain and task to be applied to a separate target domain and task. This paper presents a domain adversarial TL approach, based upon the concepts of generative adversarial networks. In this method, the optimizer seeks to minimize the loss (i.e., regression or classification accuracy) across the labeled training examples from the source domain while maximizing the loss of the domain classifier across the source and target data sets (i.e., maximizing the similarity of source and target features). The developed domain adversarial TL method has been implemented on a 1-D CNN backbone network and evaluated for prediction of tool wear propagation, using NASA's milling dataset. Performance has been compared to other TL techniques, and the results indicate that domain adversarial TL can successfully allow DL models trained on certain scenarios to be applied to new target tasks.
10
Minami, Shunya, Song Liu, Stephen Wu, Kenji Fukumizu, and Ryo Yoshida. "A General Class of Transfer Learning Regression without Implementation Cost." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 10 (May 18, 2021): 8992–99. http://dx.doi.org/10.1609/aaai.v35i10.17087.
Full textAbstract:
We propose a novel framework that unifies and extends existing methods of transfer learning (TL) for regression. To bridge a pretrained source model to the model on a target task, we introduce a density-ratio reweighting function, which is estimated through the Bayesian framework with a specific prior distribution. By changing two intrinsic hyperparameters and the choice of the density-ratio model, the proposed method can integrate three popular methods of TL: TL based on cross-domain similarity regularization, a probabilistic TL using the density-ratio estimation, and fine-tuning of pretrained neural networks. Moreover, the proposed method can benefit from its simple implementation without any additional cost; the regression model can be fully trained using off-the-shelf libraries for supervised learning in which the original output variable is simply transformed to a new output variable. We demonstrate its simplicity, generality, and applicability using various real data applications.
11
Rodríguez, Eva, Pol Valls, Beatriz Otero, Juan José Costa, Javier Verdú, Manuel Alejandro Pajuelo, and Ramon Canal. "Transfer-Learning-Based Intrusion Detection Framework in IoT Networks." Sensors 22, no. 15 (July 27, 2022): 5621. http://dx.doi.org/10.3390/s22155621.
Full textAbstract:
Cyberattacks in the Internet of Things (IoT) are growing exponentially, especially zero-day attacks mostly driven by security weaknesses on IoT networks. Traditional intrusion detection systems (IDSs) adopted machine learning (ML), especially deep Learning (DL), to improve the detection of cyberattacks. DL-based IDSs require balanced datasets with large amounts of labeled data; however, there is a lack of such large collections in IoT networks. This paper proposes an efficient intrusion detection framework based on transfer learning (TL), knowledge transfer, and model refinement, for the effective detection of zero-day attacks. The framework is tailored to 5G IoT scenarios with unbalanced and scarce labeled datasets. The TL model is based on convolutional neural networks (CNNs). The framework was evaluated to detect a wide range of zero-day attacks. To this end, three specialized datasets were created. Experimental results show that the proposed TL-based framework achieves high accuracy and low false prediction rate (FPR). The proposed solution has better detection rates for the different families of known and zero-day attacks than any previous DL-based IDS. These results demonstrate that TL is effective in the detection of cyberattacks in IoT environments.
12
Fu, Chenbo, Yongli Zheng, Yi Liu, Qi Xuan, and Guanrong Chen. "NES-TL: Network Embedding Similarity-Based Transfer Learning." IEEE Transactions on Network Science and Engineering 7, no. 3 (July 1, 2020): 1607–18. http://dx.doi.org/10.1109/tnse.2019.2942341.
Full text
13
Ullah, Naeem, Javed Ali Khan, Mohammad Sohail Khan, Wahab Khan, Izaz Hassan, Marwa Obayya, Noha Negm, and Ahmed S. Salama. "An Effective Approach to Detect and Identify Brain Tumors Using Transfer Learning." Applied Sciences 12, no. 11 (June 2, 2022): 5645. http://dx.doi.org/10.3390/app12115645.
Full textAbstract:
Brain tumors are considered one of the most serious, prominent and life-threatening diseases globally. Brain tumors cause thousands of deaths every year around the globe because of the rapid growth of tumor cells. Therefore, timely analysis and automatic detection of brain tumors are required to save the lives of thousands of people around the globe. Recently, deep transfer learning (TL) approaches are most widely used to detect and classify the three most prominent types of brain tumors, i.e., glioma, meningioma and pituitary. For this purpose, we employ state-of-the-art pre-trained TL techniques to identify and detect glioma, meningioma and pituitary brain tumors. The aim is to identify the performance of nine pre-trained TL classifiers, i.e., Inceptionresnetv2, Inceptionv3, Xception, Resnet18, Resnet50, Resnet101, Shufflenet, Densenet201 and Mobilenetv2, by automatically identifying and detecting brain tumors using a fine-grained classification approach. For this, the TL algorithms are evaluated on a baseline brain tumor classification (MRI) dataset, which is freely available on Kaggle. Additionally, all deep learning (DL) models are fine-tuned with their default values. The fine-grained classification experiment demonstrates that the inceptionresnetv2 TL algorithm performs better and achieves the highest accuracy in detecting and classifying glioma, meningioma and pituitary brain tumors, and hence it can be classified as the best classification algorithm. We achieve 98.91% accuracy, 98.28% precision, 99.75% recall and 99% F-measure values with the inceptionresnetv2 TL algorithm, which out-performs the other DL algorithms. Additionally, to ensure and validate the performance of TL classifiers, we compare the efficacy of the inceptionresnetv2 TL algorithm with hybrid approaches, in which we use convolutional neural networks (CNN) for deep feature extraction and a Support Vector Machine (SVM) for classification. Similarly, the experiment’s results show that TL algorithms, and inceptionresnetv2 in particular, out-perform the state-of-the-art DL algorithms in classifying brain MRI images into glioma, meningioma, and pituitary. The hybrid DL approaches used in the experiments are Mobilnetv2, Densenet201, Squeeznet, Alexnet, Googlenet, Inceptionv3, Resnet50, Resnet18, Resnet101, Xception, Inceptionresnetv3, VGG19 and Shufflenet.
14
Cheng, Qiao, Xiangke Wang, Yifeng Niu, and Lincheng Shen. "Reusing Source Task Knowledge via Transfer Approximator in Reinforcement Transfer Learning." Symmetry 11, no. 1 (December 29, 2018): 25. http://dx.doi.org/10.3390/sym11010025.
Full textAbstract:
Transfer Learning (TL) has received a great deal of attention because of its ability to speed up Reinforcement Learning (RL) by reusing learned knowledge from other tasks. This paper proposes a new transfer learning framework, referred to as Transfer Learning via Artificial Neural Network Approximator (TL-ANNA). It builds an Artificial Neural Network (ANN) transfer approximator to transfer the related knowledge from the source task into the target task and reuses the transferred knowledge with a Probabilistic Policy Reuse (PPR) scheme. Specifically, the transfer approximator maps the state of the target task symmetrically to states of the source task with a certain mapping rule, and activates the related knowledge (components of the action-value function) of the source task as the input of the ANNs; it then predicts the quality of the actions in the target task with the ANNs. The target learner uses the PPR scheme to bias the RL with the suggested action from the transfer approximator. In this way, the transfer approximator builds a symmetric knowledge path between the target task and the source task. In addition, two mapping rules for the transfer approximator are designed, namely, Full Mapping Rule and Group Mapping Rule. Experiments performed on the RoboCup soccer Keepaway task verified that the proposed transfer learning methods outperform two other transfer learning methods in both jumpstart and time to threshold metrics and are more robust to the quality of source knowledge. In addition, the TL-ANNA with the group mapping rule exhibits slightly worse performance than the one with the full mapping rule, but with less computation and space cost when appropriate grouping method is used.
15
Azkue, Markel, Mattin Lucu, Egoitz Martinez-Laserna, and Iosu Aizpuru. "Calendar Ageing Model for Li-Ion Batteries Using Transfer Learning Methods." World Electric Vehicle Journal 12, no. 3 (September 6, 2021): 145. http://dx.doi.org/10.3390/wevj12030145.
Full textAbstract:
Getting accurate lifetime predictions for a particular cell chemistry remains a challenging process, largely dependent on time and cost-intensive experimental battery testing. This paper proposes a transfer learning (TL) method to develop LIB ageing models, which allow for the leveraging of experimental laboratory testing data previously obtained for a different cell technology. The TL method is implemented through Neural Networks models, using LiNiMnCoO2/C laboratory ageing data as a baseline model. The obtained TL model achieves an 1.01% overall error for a broad range of operating conditions, using for retraining only two experimental ageing tests of LiFePO4/C cells.
16
Adama, David Ada, Ahmad Lotfi, and Robert Ranson. "A Survey of Vision-Based Transfer Learning in Human Activity Recognition." Electronics 10, no. 19 (October 2, 2021): 2412. http://dx.doi.org/10.3390/electronics10192412.
Full textAbstract:
Human activity recognition (HAR) and transfer learning (TL) are two broad areas widely studied in computational intelligence (CI) and artificial intelligence (AI) applications. Much effort has been put into developing suitable solutions to advance the current performance of existing systems. However, challenges are facing the existing methods of HAR. In HAR, the variations in data required in HAR systems pose challenges to many existing solutions. The type of sensory information used could play an important role in overcoming some of these challenges. Vision-based information in 3D acquired using RGB-D cameras is one type. Furthermore, with the successes encountered in TL, HAR stands to benefit from TL to address challenges to existing methods. Therefore, it is important to review the current state-of-the-art related to both areas. This paper presents a comprehensive survey of vision-based HAR using different methods with a focus on the incorporation of TL in HAR methods. It also discusses the limitations, challenges and possible future directions for more research.
17
Lv, Ying, Bofeng Zhang, Xiaodong Yue, and Zhikang Xu. "A Novel Ensemble Strategy Based on Determinantal Point Processes for Transfer Learning." Mathematics 10, no. 23 (November 23, 2022): 4409. http://dx.doi.org/10.3390/math10234409.
Full textAbstract:
Transfer learning (TL) hopes to train a model for target domain tasks by using knowledge from different but related source domains. Most TL methods focus more on improving the predictive performance of the single model across domains. Since domain differences cannot be avoided, the knowledge from the source domain to obtain the target domain is limited. Therefore, the transfer model has to predict out-of-distribution (OOD) data in the target domain. However, the prediction of the single model is unstable when dealing with the OOD data, which can easily cause negative transfer. To solve this problem, we propose a parallel ensemble strategy based on Determinantal Point Processes (DPP) for transfer learning. In this strategy, we first proposed an improved DPP sampling to generate training subsets with higher transferability and diversity. Second, we use the subsets to train the base models. Finally, the base models are fused using the adaptability of subsets. To validate the effectiveness of the ensemble strategy, we couple the ensemble strategy into traditional TL models and deep TL models and evaluate the transfer performance models on text and image data sets. The experiment results show that our proposed ensemble strategy can significantly improve the performance of the transfer model.
18
Alzubaidi, Laith, Mohammed A. Fadhel, Omran Al-Shamma, Jinglan Zhang, J. Santamaría, Ye Duan, and Sameer R. Oleiwi. "Towards a Better Understanding of Transfer Learning for Medical Imaging: A Case Study." Applied Sciences 10, no. 13 (June 29, 2020): 4523. http://dx.doi.org/10.3390/app10134523.
Full textAbstract:
One of the main challenges of employing deep learning models in the field of medicine is a lack of training data due to difficulty in collecting and labeling data, which needs to be performed by experts. To overcome this drawback, transfer learning (TL) has been utilized to solve several medical imaging tasks using pre-trained state-of-the-art models from the ImageNet dataset. However, there are primary divergences in data features, sizes, and task characteristics between the natural image classification and the targeted medical imaging tasks. Therefore, TL can slightly improve performance if the source domain is completely different from the target domain. In this paper, we explore the benefit of TL from the same and different domains of the target tasks. To do so, we designed a deep convolutional neural network (DCNN) model that integrates three ideas including traditional and parallel convolutional layers and residual connections along with global average pooling. We trained the proposed model against several scenarios. We utilized the same and different domain TL with the diabetic foot ulcer (DFU) classification task and with the animal classification task. We have empirically shown that the source of TL from the same domain can significantly improve the performance considering a reduced number of images in the same domain of the target dataset. The proposed model with the DFU dataset achieved F1-score value of 86.6% when trained from scratch, 89.4% with TL from a different domain of the targeted dataset, and 97.6% with TL from the same domain of the targeted dataset.
19
Macias, Edwar, Jose Lopez Vicario, Javier Serrano, Jose Ibeas, and Antoni Morell. "Transfer Learning Improving Predictive Mortality Models for Patients in End-Stage Renal Disease." Electronics 11, no. 9 (April 30, 2022): 1447. http://dx.doi.org/10.3390/electronics11091447.
Full textAbstract:
Deep learning is becoming a fundamental piece in the paradigm shift from evidence-based to data-based medicine. However, its learning capacity is rarely exploited when working with small data sets. Through transfer learning (TL), information from a source domain is transferred to a target one to enhance a learning task in such domain. The proposed TL mechanisms are based on sample and feature space augmentation. Thus, deep autoencoders extract complex representations for the data in the TL approach. Their latent representations, the so-called codes, are handled to transfer information among domains. The transfer of samples is carried out by computing a latent space mapping matrix that links codes from both domains for later reconstruction. The feature space augmentation is based on the computation of the average of the most similar codes from one domain. Such an average augments the features in a target domain. The proposed framework is evaluated in the prediction of mortality in patients in end-stage renal disease, transferring information related to the mortality of patients with acute kidney injury from the massive database MIMIC-III. Compared to other TL mechanisms, the proposed approach improves 6–11% in previous mortality predictive models. The integration of TL approaches into learning tasks in pathologies with data volume issues could encourage the use of data-based medicine in a clinical setting.
20
Liu, Zeyu, Meng Jiang, and Tengfei Luo. "Leverage electron properties to predict phonon properties via transfer learning for semiconductors." Science Advances 6, no. 45 (November 2020): eabd1356. http://dx.doi.org/10.1126/sciadv.abd1356.
Full textAbstract:
Electron properties are usually easier to obtain than phonon properties. The ability to leverage electron properties to help predict phonon properties can thus greatly benefit materials by design for applications like thermoelectrics and electronics. Here, we demonstrate the ability of using transfer learning (TL), where knowledge learned from training machine learning models on electronic bandgaps of 1245 semiconductors is transferred to improve the models, trained using only 124 data, for predicting various phonon properties (phonon bandgap, group velocity, and heat capacity). Compared to directly trained models, TL reduces the mean absolute errors of prediction by 65, 14, and 54% respectively, for the three phonon properties. The TL models are further validated using several semiconductors outside of the 1245 database. Results also indicate that TL can leverage not-so-accurate proxy properties, as long as they encode composition-property relation, to improve models for target properties, a notable feature to materials informatics in general.
21
Wang, GongMing, JunFei Qiao, Jing Bi, WenJing Li, and MengChu Zhou. "TL-GDBN: Growing Deep Belief Network With Transfer Learning." IEEE Transactions on Automation Science and Engineering 16, no. 2 (April 2019): 874–85. http://dx.doi.org/10.1109/tase.2018.2865663.
Full text
22
Zhang, Jun, Ke Yan, Qingcai Chen, and Bin Liu. "PreRBP-TL: prediction of species-specific RNA-binding proteins based on transfer learning." Bioinformatics 38, no. 8 (February 17, 2022): 2135–43. http://dx.doi.org/10.1093/bioinformatics/btac106.
Full textAbstract:
Abstract Motivation RNA-binding proteins (RBPs) play crucial roles in post-transcriptional regulation. Accurate identification of RBPs helps to understand gene expression, regulation, etc. In recent years, some computational methods were proposed to identify RBPs. However, these methods fail to accurately identify RBPs from some specific species with limited data, such as bacteria. Results In this study, we introduce a computational method called PreRBP-TL for identifying species-specific RBPs based on transfer learning. The weights of the prediction model were initialized by pretraining with the large general RBP dataset and then fine-tuned with the small species-specific RPB dataset by using transfer learning. The experimental results show that the PreRBP-TL achieves better performance for identifying the species-specific RBPs from Human, Arabidopsis, Escherichia coli and Salmonella, outperforming eight state-of-the-art computational methods. It is anticipated PreRBP-TL will become a useful method for identifying RBPs. Availability and implementation For the convenience of researchers to identify RBPs, the web server of PreRBP-TL was established, freely available at http://bliulab.net/PreRBP-TL. Supplementary information Supplementary data are available at Bioinformatics online.
23
Niu, Shuteng, Yushan Jiang, Bowen Chen, Jian Wang, Yongxin Liu, and Houbing Song. "Cross-Modality Transfer Learning for Image-Text Information Management." ACM Transactions on Management Information Systems 13, no. 1 (March 31, 2022): 1–14. http://dx.doi.org/10.1145/3464324.
Full textAbstract:
In the past decades, information from all kinds of data has been on a rapid increase. With state-of-the-art performance, machine learning algorithms have been beneficial for information management. However, insufficient supervised training data is still an adversity in many real-world applications. Therefore, transfer learning (TF) was proposed to address this issue. This article studies a not well investigated but important TL problem termed cross-modality transfer learning (CMTL). This topic is closely related to distant domain transfer learning (DDTL) and negative transfer. In general, conventional TL disciplines assume that the source domain and the target domain are in the same modality. DDTL aims to make efficient transfers even when the domains or the tasks are entirely different. As an extension of DDTL, CMTL aims to make efficient transfers between two different data modalities, such as from image to text. As the main focus of this study, we aim to improve the performance of image classification by transferring knowledge from text data. Previously, a few CMTL algorithms were proposed to deal with image classification problems. However, most existing algorithms are very task specific, and they are unstable on convergence. There are four main contributions in this study. First, we propose a novel heterogeneous CMTL algorithm, which requires only a tiny set of unlabeled target data and labeled source data with associate text tags. Second, we introduce a latent semantic information extraction method to connect the information learned from the image data and the text data. Third, the proposed method can effectively handle the information transfer across different modalities (text-image). Fourth, we examined our algorithm on a public dataset, Office-31. It has achieved up to 5% higher classification accuracy than “non-transfer” algorithms and up to 9% higher than existing CMTL algorithms.
24
Gautam, Vinay, Naresh K. Trivedi, Aman Singh, Heba G. Mohamed, Irene Delgado Noya, Preet Kaur, and Nitin Goyal. "A Transfer Learning-Based Artificial Intelligence Model for Leaf Disease Assessment." Sustainability 14, no. 20 (October 20, 2022): 13610. http://dx.doi.org/10.3390/su142013610.
Full textAbstract:
The paddy crop is the most essential and consumable agricultural produce. Leaf disease impacts the quality and productivity of paddy crops. Therefore, tackling this issue as early as possible is mandatory to reduce its impact. Consequently, in recent years, deep learning methods have been essential in identifying and classifying leaf disease. Deep learning is used to observe patterns in disease in crop leaves. For instance, organizing a crop’s leaf according to its shape, size, and color is significant. To facilitate farmers, this study proposed a Convolutional Neural Networks-based Deep Learning (CNN-based DL) architecture, including transfer learning (TL) for agricultural research. In this study, different TL architectures, viz. InceptionV3, VGG16, ResNet, SqueezeNet, and VGG19, were considered to carry out disease detection in paddy plants. The approach started with preprocessing the leaf image; afterward, semantic segmentation was used to extract a region of interest. Consequently, TL architectures were tuned with segmented images. Finally, the extra, fully connected layers of the Deep Neural Network (DNN) are used to classify and identify leaf disease. The proposed model was concerned with the biotic diseases of paddy leaves due to fungi and bacteria. The proposed model showed an accuracy rate of 96.4%, better than state-of-the-art models with different variants of TL architectures. After analysis of the outcomes, the study concluded that the anticipated model outperforms other existing models.
25
Ge, Feng-Xiang, Yanyu Bai, Mengjia Li, Guangping Zhu, and Jingwei Yin. "Label distribution-guided transfer learning for underwater source localization." Journal of the Acoustical Society of America 151, no. 6 (June 2022): 4140–49. http://dx.doi.org/10.1121/10.0011741.
Full textAbstract:
Underwater source localization by deep neural networks (DNNs) is challenging since training these DNNs generally requires a large amount of experimental data and is computationally expensive. In this paper, label distribution-guided transfer learning (LD-TL) for underwater source localization is proposed, where a one-dimensional convolutional neural network (1D-CNN) is pre-trained with the simulation data generated by an underwater acoustic propagation model and then fine-tuned with a very limited amount of experimental data. In particular, the experimental data for fine-tuning the pre-trained 1D-CNN are labeled with label distribution vectors instead of one-hot encoded vectors. Experimental results show that the performance of underwater source localization with a very limited amount of experimental data is significantly improved by the proposed LD-TL.
26
Gao, Mingyu, Dawei Qi, Hongbo Mu, and Jianfeng Chen. "A Transfer Residual Neural Network Based on ResNet-34 for Detection of Wood Knot Defects." Forests 12, no. 2 (February 11, 2021): 212. http://dx.doi.org/10.3390/f12020212.
Full textAbstract:
In recent years, due to the shortage of timber resources, it has become necessary to reduce the excessive consumption of forest resources. Non-destructive testing technology can quickly find wood defects and effectively improve wood utilization. Deep learning has achieved significant results as one of the most commonly used methods in the detection of wood knots. However, compared with convolutional neural networks in other fields, the depth of deep learning models for the detection of wood knots is still very shallow. This is because the number of samples marked in the wood detection is too small, which limits the accuracy of the final prediction of the results. In this paper, ResNet-34 is combined with transfer learning, and a new TL-ResNet34 deep learning model with 35 convolution depths is proposed to detect wood knot defects. Among them, ResNet-34 is used as a feature extractor for wood knot defects. At the same time, a new method TL-ResNet34 is proposed, which combines ResNet-34 with transfer learning. After that, the wood knot defect dataset was applied to TL-ResNet34 for testing. The results show that the detection accuracy of the dataset trained by TL-ResNet34 is significantly higher than that of other methods. This shows that the final prediction accuracy of the detection of wood knot defects can be improved by TL-ResNet34.
27
Ikeda, Satoru, Hitoshi Kono, Kaori Watanabe, and Hidekazu Suzuki. "Body Calibration: Automatic Inter-Task Mapping between Multi-Legged Robots with Different Embodiments in Transfer Reinforcement Learning." Actuators 11, no. 5 (May 21, 2022): 140. http://dx.doi.org/10.3390/act11050140.
Full textAbstract:
Machine learning algorithms are effective in realizing the programming of robots that behave autonomously for various tasks. For example, reinforcement learning (RL) does not require supervision or data sets; the RL agent explores solutions by itself. However, RL requires a long learning time, particularly for actual robot learning situations. Transfer learning (TL) in RL has been proposed to address this limitation. TL realizes fast adaptation and decreases the problem-solving time by utilizing the knowledge of the policy, value function, and Q-function from RL. Taylor proposed TL using inter-task mapping that defines the correspondence between the state and action between the source and target domains. Inter-task mapping is defined based on human intuition and experience; therefore, the effect of TL may not be obtained. The difference in robot shapes for TL is similar to the cognition in the modification of human body composition, and automatic inter-task mapping can be performed by referring to the body representation that is assumed to be stored in the human brain. In this paper, body calibration is proposed, which refers to the physical expression in the human brain. It realizes automatic inter-task mapping by acquiring data modeled on a body diagram that illustrates human body composition and posture. The proposed method is evaluated in a TL situation from a computer simulation of RL to actual robot control with a multi-legged robot.
28
Sharma, Gaurav, Carlo Colantuoni, Loyal A. Goff, Elana J. Fertig, and Genevieve Stein-O’Brien. "projectR: an R/Bioconductor package for transfer learning via PCA, NMF, correlation and clustering." Bioinformatics 36, no. 11 (March 13, 2020): 3592–93. http://dx.doi.org/10.1093/bioinformatics/btaa183.
Full textAbstract:
Abstract Motivation Dimension reduction techniques are widely used to interpret high-dimensional biological data. Features learned from these methods are used to discover both technical artifacts and novel biological phenomena. Such feature discovery is critically importent in analysis of large single-cell datasets, where lack of a ground truth limits validation and interpretation. Transfer learning (TL) can be used to relate the features learned from one source dataset to a new target dataset to perform biologically driven validation by evaluating their use in or association with additional sample annotations in that independent target dataset. Results We developed an R/Bioconductor package, projectR, to perform TL for analyses of genomics data via TL of clustering, correlation and factorization methods. We then demonstrate the utility TL for integrated data analysis with an example for spatial single-cell analysis. Availability and implementation projectR is available on Bioconductor and at https://github.com/genesofeve/projectR. Contact gsteinobrien@jhmi.edu or ejfertig@jhmi.edu Supplementary information Supplementary data are available at Bioinformatics online.
29
Zhang, Kai, Guanghua Xu, Xiaowei Zheng, Huanzhong Li, Sicong Zhang, Yunhui Yu, and Renghao Liang. "Application of Transfer Learning in EEG Decoding Based on Brain-Computer Interfaces: A Review." Sensors 20, no. 21 (November 5, 2020): 6321. http://dx.doi.org/10.3390/s20216321.
Full textAbstract:
The algorithms of electroencephalography (EEG) decoding are mainly based on machine learning in current research. One of the main assumptions of machine learning is that training and test data belong to the same feature space and are subject to the same probability distribution. However, this may be violated in EEG processing. Variations across sessions/subjects result in a deviation of the feature distribution of EEG signals in the same task, which reduces the accuracy of the decoding model for mental tasks. Recently, transfer learning (TL) has shown great potential in processing EEG signals across sessions/subjects. In this work, we reviewed 80 related published studies from 2010 to 2020 about TL application for EEG decoding. Herein, we report what kind of TL methods have been used (e.g., instance knowledge, feature representation knowledge, and model parameter knowledge), describe which types of EEG paradigms have been analyzed, and summarize the datasets that have been used to evaluate performance. Moreover, we discuss the state-of-the-art and future development of TL for EEG decoding. The results show that TL can significantly improve the performance of decoding models across subjects/sessions and can reduce the calibration time of brain–computer interface (BCI) systems. This review summarizes the current practical suggestions and performance outcomes in the hope that it will provide guidance and help for EEG research in the future.
30
Oblinger, Daniel. "Toward a Computational Model of Transfer." AI Magazine 32, no. 2 (March 16, 2011): 126. http://dx.doi.org/10.1609/aimag.v32i2.2337.
Full textAbstract:
This article focuses on a broad framing of the DARPA Transfer Learning Program research and an assessment of its progress, limitations, and challenges, from an admittedly personal but DARPA-influenced perspective. I will focus on a broad framing of TL that that will allow us to talk about this body of work as a whole, and use this to look towards work yet to be done in this area. I will consider both indicated application areas for transfer learning, as well as indicated future research challenges. With each of these I will also venture assessment of the "ripeness" of each of these subareas for follow on work-of course this assessment will be a very personal estimation based on the effort and progress made during the TL program.
31
Pachón, César G., Dora M. Ballesteros, and Diego Renza. "Fake Banknote Recognition Using Deep Learning." Applied Sciences 11, no. 3 (January 30, 2021): 1281. http://dx.doi.org/10.3390/app11031281.
Full textAbstract:
Recently, some state-of-the-art works have used deep learning-based architectures, specifically convolutional neural networks (CNNs), for banknote recognition and counterfeit detection with promising results. However, it is not clear which design strategy is more appropriate (custom or by transfer learning) in terms of classifier performance and inference times for massive data applications. This paper presents a comparison of the two design strategies in various types of architecture. For the transfer learning (TL) strategy, the most appropriate freezing points in CNN architectures (sequential, residual and Inception) are identified. In addition, a custom model based on an AlexNet-type sequential CNN is proposed. Both the TL and the custom models were trained and compared using a Colombian banknote dataset. According to the results, ResNet18 achieved the best accuracy, with 100%. On the other hand, the network with the shortest inference times was the proposed custom network, since its performance is up to 6.48-times faster in CPU and 16.29-times faster in GPU than the inference time with the models by transfer learning.
32
Ee, Ken-ji, Ahmad Fakhri Bin Ab. Nasir, Anwar P. P. Abdul Majeed, Mohd Azraai Mohd Razman, and Nur Hafieza Ismail. "The Animal Classification: An Evaluation of Different Transfer Learning Pipeline." MEKATRONIKA 3, no. 1 (June 17, 2021): 27–31. http://dx.doi.org/10.15282/mekatronika.v3i1.6680.
Full textAbstract:
The animal classification system is a technology to classify the animal class (type) automatically and useful in many applications. There are many types of learning models applied to this technology recently. Nonetheless, it is worth noting that the extraction of the features and the classification of the animal features is non-trivial, particularly in the deep learning approach for a successful animal classification system. The use of Transfer Learning (TL) has been demonstrated to be a powerful tool in the extraction of essential features. However, the employment of such a method towards animal classification applications are somewhat limited. The present study aims to determine a suitable TL-conventional classifier pipeline for animal classification. The VGG16 and VGG19 were used in extracting features and then coupled with either k-Nearest Neighbour (k-NN) or Support Vector Machine (SVM) classifier. Prior to that, a total of 4000 images were gathered consisting of a total of five classes which are cows, goats, buffalos, dogs, and cats. The data was split into the ratio of 80:20 for train and test. The classifiers hyper parameters are tuned by the Grids Search approach that utilises the five-fold cross-validation technique. It was demonstrated from the study that the best TL pipeline identified is the VGG16 along with an optimised SVM, as it was able to yield an average classification accuracy of 0.975. The findings of the present investigation could facilitate animal classification application, i.e. for monitoring animals in wildlife.
33
Parthiban, Shree Nandhini, Palaniswamy Amudha, and Subramaniam Pillai Sivakumari. "Exploitation of Advanced Deep Learning Methods and Feature Modeling for Air Quality Prediction." Revue d'Intelligence Artificielle 36, no. 6 (December 31, 2022): 959–67. http://dx.doi.org/10.18280/ria.360618.
Full textAbstract:
Air pollution is a major issue because Particulate Matter (PM) has a substantially higher effect on human health than other pollutants. Air Quality (AQ) prediction has become critical recently to take action to reduce pollution. This research introduces a unique methodology for assessing the effectiveness of PM10 and PM2.5. Enhanced spatial, temporal sequence-Improved Sparse Auto Encoder with Deep Learning (EISAE-DL) has been proposed to predict AQ affected by the prolonged dependency of air pollution congregation. However, Long Short-Term Memory (LSTM) used in EISAE-DL has suffered from the learning of a long-term dependent sequence of the training dataset. In addition, it is hard to create very reliable AQ forecasts at higher periodic frequencies, such as daily, weekly, or even monthly. This paper proposes Transfer learning (TL) in a Stacked Bidirectional and Unidirectional LSTM to solve the learning issue in LSTM for long-term dependencies. So, EISAE-DL with TL and modified LSTM model is named as EISAE-Deep Transfer Learning (EISAE-DTL). TL with a modified structure can handle large-size datasets effectively. However, training time is increased more than twice for non-transfer learning way of modeling due to TL, Wasserstein Distance-based adversarial learning is proposed in EISAE-DTL to decrease the variances among AQ data collected from any two sites. The proposed work is named EISAE- Enhanced DTL (EISAE-EDTL). The developed EISAE-DTL and EISAE-EDTL models are compared and analyzed with the performance of existing algorithms EISAE-DL, ISAE-DL, TL-BLSTM, MMSL, and ST-DNN. The experimental findings demonstrate the accuracy, precision, sensitivity, specificity, Area Under Curve (AUC), and Matthew's correlation coefficient of the proposed model performs admirably and improves present condition approaches.
34
Fan, Luqin, Jing Zhang, Yu He, Ying Liu, Tao Hu, and Heng Zhang. "Optimal Scheduling of Microgrid Based on Deep Deterministic Policy Gradient and Transfer Learning." Energies 14, no. 3 (January 23, 2021): 584. http://dx.doi.org/10.3390/en14030584.
Full textAbstract:
Microgrid has flexible composition, a complex operation mechanism, and a large amount of data while operating. However, optimization methods of microgrid scheduling do not effectively accumulate and utilize the scheduling knowledge at present. This paper puts forward a microgrid optimal scheduling method based on Deep Deterministic Policy Gradient (DDPG) and Transfer Learning (TL). This method uses Reinforcement Learning (RL) to learn the scheduling strategy and accumulates the corresponding scheduling knowledge. Meanwhile, the DDPG model is introduced to extend the microgrid scheduling strategy action from the discrete action space to the continuous action space. On this basis, this paper holds that a microgrid optimal scheduling TL algorithm on the strength of the actual supply and demand similarity is proposed with a purpose of making use of the existing scheduling knowledge effectively. The simulation results indicate that this paper can provide optimal scheduling strategy for microgrid with complex operation mechanism flexibly and efficiently through the effective accumulation of scheduling knowledge and the utilization of scheduling knowledge through TL.
35
Bizzego, Andrea, Giulio Gabrieli, and Gianluca Esposito. "Deep Neural Networks and Transfer Learning on a Multivariate Physiological Signal Dataset." Bioengineering 8, no. 3 (March 6, 2021): 35. http://dx.doi.org/10.3390/bioengineering8030035.
Full textAbstract:
While Deep Neural Networks (DNNs) and Transfer Learning (TL) have greatly contributed to several medical and clinical disciplines, the application to multivariate physiological datasets is still limited. Current examples mainly focus on one physiological signal and can only utilise applications that are customised for that specific measure, thus it limits the possibility of transferring the trained DNN to other domains. In this study, we composed a dataset (n=813) of six different types of physiological signals (Electrocardiogram, Electrodermal activity, Electromyogram, Photoplethysmogram, Respiration and Acceleration). Signals were collected from 232 subjects using four different acquisition devices. We used a DNN to classify the type of physiological signal and to demonstrate how the TL approach allows the exploitation of the efficiency of DNNs in other domains. After the DNN was trained to optimally classify the type of signal, the features that were automatically extracted by the DNN were used to classify the type of device used for the acquisition using a Support Vector Machine. The dataset, the code and the trained parameters of the DNN are made publicly available to encourage the adoption of DNN and TL in applications with multivariate physiological signals.
36
Halpern, Diane F., and Milton D. Hakel. "Learning That Lasts a Lifetime: Teaching for Long-Term Retention and Transfer." New Directions for Teaching and Learning 2002, no. 89 (2002): 3–7. http://dx.doi.org/10.1002/tl.42.
Full text
37
Shapiee, Muhammad Nur Aiman, Muhammad Ar Rahim Ibrahim, Muhammad Amirul Abdullah, Rabiu Muazu Musa, Noor Azuan Abu Osman, Anwar P.P Abdul Majeed, and Mohd Azraai Mohd Razman. "The Classification of Skateboarding Tricks : A Transfer Learning and Machine Learning Approach." MEKATRONIKA 2, no. 2 (October 27, 2020): 1–12. http://dx.doi.org/10.15282/mekatronika.v2i2.6683.
Full textAbstract:
The skateboarding scene has arrived at new statures, particularly with its first appearance at the now delayed Tokyo Summer Olympic Games. Hence, attributable to the size of the game in such competitive games, progressed creative appraisal approaches have progressively increased due consideration by pertinent partners, particularly with the enthusiasm of a more goal-based assessment. This study purposes for classifying skateboarding tricks, specifically Frontside 180, Kickflip, Ollie, Nollie Front Shove-it, and Pop Shove-it over the integration of image processing, Trasnfer Learning (TL) to feature extraction enhanced with tradisional Machine Learning (ML) classifier. A male skateboarder performed five tricks every sort of trick consistently and the YI Action camera captured the movement by a range of 1.26 m. Then, the image dataset were features built and extricated by means of three TL models, and afterward in this manner arranged to utilize by k-Nearest Neighbor (k-NN) classifier. The perception via the initial experiments showed, the MobileNet, NASNetMobile, and NASNetLarge coupled with optimized k-NN classifiers attain a classification accuracy (CA) of 95%, 92% and 90%, respectively on the test dataset. Besides, the result evident from the robustness evaluation showed the MobileNet+k-NN pipeline is more robust as it could provide a decent average CA than other pipelines. It would be demonstrated that the suggested study could characterize the skateboard tricks sufficiently and could, over the long haul, uphold judges decided for giving progressively objective-based decision.
38
Lim, Wee Sheng, Ahmad Fakhri Ab. Nasir, Mohd Azraai Mohd Razman, Anwar P. P. Abdul Majeed, Nur Shazwani Kamarudin, and M. Zulfahmi Toh. "Traffic Sign Classification Using Transfer Learning: An Investigation of Feature-Combining Model." MEKATRONIKA 3, no. 2 (July 30, 2021): 37–41. http://dx.doi.org/10.15282/mekatronika.v3i2.7346.
Full textAbstract:
The traffic sign classification system is a technology to help drivers to recognise the traffic sign hence reducing the accident. Many types of learning models have been applied to this technology recently. However, the deployment of learning models is unknown and shown to be non-trivial towards image classification and object detection. The implementation of Transfer Learning (TL) has been demonstrated to be a powerful tool in the extraction of essential features as well as can save lots of training time. Besides, the feature-combining model exhibited great performance in the TL method in many applications. Nonetheless, the utilisation of such methods towards traffic sign classification applications are not yet being evaluated. The present study aims to exploit and investigate the effectiveness of transfer learning feature-combining models, particularly to classify traffic signs. The images were gathered from GTSRB dataset which consists of 10 different types of traffic signs i.e. warning, stop, repair, not enter, traffic light, turn right, speed limit (80km/s), speed limit (50km/s), speed limit (60km/s), and turn left sign board. A total of 7000 images were then split to 70:30 for train and test ratio using a stratified method. The VGG16 and VGG19 TL-features models were used to combine with two classifiers, Random Forest (RF) and Neural Network. In summary, six different pipelines were trained and tested. From the results obtained, the best pipeline was VGG16+VGG19 with RF classifier, which was able to yield an average classification accuracy of 0.9838. The findings showed that the feature-combining model successfully classifies the traffic signs much better than the single TL-feature model. The investigation would be useful for traffic signs classification applications i.e. for ADAS systems
39
Gulzar, Yonis. "Fruit Image Classification Model Based on MobileNetV2 with Deep Transfer Learning Technique." Sustainability 15, no. 3 (January 19, 2023): 1906. http://dx.doi.org/10.3390/su15031906.
Full textAbstract:
Due to the rapid emergence and evolution of AI applications, the utilization of smart imaging devices has increased significantly. Researchers have started using deep learning models, such as CNN, for image classification. Unlike the traditional models, which require a lot of features to perform well, CNN does not require any handcrafted features to perform well. It uses numerous filters, which extract required features from images automatically for classification. One of the issues in the horticulture industry is fruit classification, which requires an expert with a lot of experience. To overcome this issue an automated system is required which can classify different types of fruits without the need for any human effort. In this study, a dataset of a total of 26,149 images of 40 different types of fruits was used for experimentation. The training and test set were randomly recreated and divided into the ratio of 3:1. The experiment introduces a customized head of five different layers into MobileNetV2 architecture. The classification layer of the MobileNetV2 model is replaced by the customized head, which produced the modified version of MobileNetV2 called TL-MobileNetV2. In addition, transfer learning is used to retain the pre-trained model. TL-MobileNetV2 achieves an accuracy of 99%, which is 3% higher than MobileNetV2, and the equal error rate of TL-MobileNetV2 is just 1%. Compared to AlexNet, VGG16, InceptionV3, and ResNet, the accuracy is better by 8, 11, 6, and 10%, respectively. Furthermore, the TL-MobileNetV2 model obtained 99% precision, 99% for recall, and a 99% F1-score. It can be concluded that transfer learning plays a big part in achieving better results, and the dropout technique helps to reduce the overfitting in transfer learning.
40
Sanagala, Skandha S., Andrew Nicolaides, Suneet K. Gupta, Vijaya K. Koppula, Luca Saba, Sushant Agarwal, Amer M. Johri, Manudeep S. Kalra, and Jasjit S. Suri. "Ten Fast Transfer Learning Models for Carotid Ultrasound Plaque Tissue Characterization in Augmentation Framework Embedded with Heatmaps for Stroke Risk Stratification." Diagnostics 11, no. 11 (November 15, 2021): 2109. http://dx.doi.org/10.3390/diagnostics11112109.
Full textAbstract:
Background and Purpose: Only 1–2% of the internal carotid artery asymptomatic plaques are unstable as a result of >80% stenosis. Thus, unnecessary efforts can be saved if these plaques can be characterized and classified into symptomatic and asymptomatic using non-invasive B-mode ultrasound. Earlier plaque tissue characterization (PTC) methods were machine learning (ML)-based, which used hand-crafted features that yielded lower accuracy and unreliability. The proposed study shows the role of transfer learning (TL)-based deep learning models for PTC. Methods: As pertained weights were used in the supercomputer framework, we hypothesize that transfer learning (TL) provides improved performance compared with deep learning. We applied 11 kinds of artificial intelligence (AI) models, 10 of them were augmented and optimized using TL approaches—a class of Atheromatic™ 2.0 TL (AtheroPoint™, Roseville, CA, USA) that consisted of (i–ii) Visual Geometric Group-16, 19 (VGG16, 19); (iii) Inception V3 (IV3); (iv–v) DenseNet121, 169; (vi) XceptionNet; (vii) ResNet50; (viii) MobileNet; (ix) AlexNet; (x) SqueezeNet; and one DL-based (xi) SuriNet-derived from UNet. We benchmark 11 AI models against our earlier deep convolutional neural network (DCNN) model. Results: The best performing TL was MobileNet, with accuracy and area-under-the-curve (AUC) pairs of 96.10 ± 3% and 0.961 (p < 0.0001), respectively. In DL, DCNN was comparable to SuriNet, with an accuracy of 95.66% and 92.7 ± 5.66%, and an AUC of 0.956 (p < 0.0001) and 0.927 (p < 0.0001), respectively. We validated the performance of the AI architectures with established biomarkers such as greyscale median (GSM), fractal dimension (FD), higher-order spectra (HOS), and visual heatmaps. We benchmarked against previously developed Atheromatic™ 1.0 ML and showed an improvement of 12.9%. Conclusions: TL is a powerful AI tool for PTC into symptomatic and asymptomatic plaques.
41
Yu, Ping, Hongwei Zhao, Ming Hu, Hui Yan, Xiaozhong Geng, Hanlin Chen, and Dejin Chu. "QoC-Driven MEC Transfer System Framework in Wireless Networks." Wireless Communications and Mobile Computing 2022 (August 19, 2022): 1–14. http://dx.doi.org/10.1155/2022/7863972.
Full textAbstract:
In this paper, we propose a Heterogeneous MEC System Framework based on Transfer Learning (HMECSF-TL), which uses convolutional neural network (CNN) to process few training samples. In view of the time-varying network environment and the limited end devices resources, the HMECSF-TL framework uses transfer learning (TL) technology to optimize the CNN model and jointly optimizes the allocation of computing resources and communication resources, which is beneficial to achieve the dual goals of extending the use time of end devices and improving the speed and the accuracy of image classification. We first introduce the Quality of Content (QoC)-driven MEC transfer system architecture of cloud-edge-end. The cloud server uses the existing image dataset to train the general neural network model in advance and transfer the general model to the edge servers, and then the edge servers deploy the local models to the end devices to form the personalized models. Then, considering the time-varying situation of the network environment, in order to get the updated model faster and better, we present the process of collaborative optimization of model between the edge sever and multiple end devices, using an edge server as an example. Considering the limited resources of the end devices, we propose a joint optimization of energy and latency with the goal of minimizing offloading cost, in order to rapidly improve the speed and the accuracy of image classification with few training samples under the premise of rational resource allocation and verify the performance of the framework experimentally. Simulation results show that the proposed HMECSF-TL framework outperforms the benchmark strategy without TL in terms of reducing the model training time and improving the image classification accuracy, as well as reducing the offloading cost.
42
Mahendra Kumar, Jothi Letchumy, Mamunur Rashid, Rabiu Muazu Musa, Mohd Azraai Mohd Razman, Norizam Sulaiman, Rozita Jailani, and Anwar P.P. Abdul Majeed. "The classification of EEG-based winking signals: a transfer learning and random forest pipeline." PeerJ 9 (March 31, 2021): e11182. http://dx.doi.org/10.7717/peerj.11182.
Full textAbstract:
Brain Computer-Interface (BCI) technology plays a considerable role in the control of rehabilitation or peripheral devices for stroke patients. This is particularly due to their inability to control such devices from their inherent physical limitations after such an attack. More often than not, the control of such devices exploits electroencephalogram (EEG) signals. Nonetheless, it is worth noting that the extraction of the features and the classification of the signals is non-trivial for a successful BCI system. The use of Transfer Learning (TL) has been demonstrated to be a powerful tool in the extraction of essential features. However, the employment of such a method towards BCI applications, particularly in regard to EEG signals, are somewhat limited. The present study aims to evaluate the effectiveness of different TL models in extracting features for the classification of wink-based EEG signals. The extracted features are classified by means of fine-tuned Random Forest (RF) classifier. The raw EEG signals are transformed into a scalogram image via Continuous Wavelet Transform (CWT) before it was fed into the TL models, namely InceptionV3, Inception ResNetV2, Xception and MobileNet. The dataset was divided into training, validation, and test datasets, respectively, via a stratified ratio of 60:20:20. The hyperparameters of the RF models were optimised through the grid search approach, in which the five-fold cross-validation technique was adopted. The optimised RF classifier performance was compared with the conventional TL-based CNN classifier performance. It was demonstrated from the study that the best TL model identified is the Inception ResNetV2 along with an optimised RF pipeline, as it was able to yield a classification accuracy of 100% on both the training and validation dataset. Therefore, it could be established from the study that a comparable classification efficacy is attainable via the Inception ResNetV2 with an optimised RF pipeline. It is envisaged that the implementation of the proposed architecture to a BCI system would potentially facilitate post-stroke patients to lead a better life quality.
43
Lu, Heng, Lei Ma, Xiao Fu, Chao Liu, Zhi Wang, Min Tang, and Naiwen Li. "Landslides Information Extraction Using Object-Oriented Image Analysis Paradigm Based on Deep Learning and Transfer Learning." Remote Sensing 12, no. 5 (February 25, 2020): 752. http://dx.doi.org/10.3390/rs12050752.
Full textAbstract:
How to acquire landslide disaster information quickly and accurately has become the focus and difficulty of disaster prevention and relief by remote sensing. Landslide disasters are generally featured by sudden occurrence, proposing high demand for emergency data acquisition. The low-altitude Unmanned Aerial Vehicle (UAV) remote sensing technology is widely applied to acquire landslide disaster data, due to its convenience, high efficiency, and ability to fly at low altitude under cloud. However, the spectrum information of UAV images is generally deficient and manual interpretation is difficult for meeting the need of quick acquisition of emergency data. Based on this, UAV images of high-occurrence areas of landslide disaster in Wenchuan County and Baoxing County in Sichuan Province, China were selected for research in the paper. Firstly, the acquired UAV images were pre-processed to generate orthoimages. Subsequently, multi-resolution segmentation was carried out to obtain image objects, and the barycenter of each object was calculated to generate a landslide sample database (including positive and negative samples) for deep learning. Next, four landslide feature models of deep learning and transfer learning, namely Histograms of Oriented Gradients (HOG), Bag of Visual Word (BOVW), Convolutional Neural Network (CNN), and Transfer Learning (TL) were compared, and it was found that the TL model possesses the best feature extraction effect, so a landslide extraction method based on the TL model and object-oriented image analysis (TLOEL) was proposed; finally, the TLOEL method was compared with the object-oriented nearest neighbor classification (NNC) method. The research results show that the accuracy of the TLOEL method is higher than the NNC method, which can not only achieve the edge extraction of large landslides, but also detect and extract middle and small landslides accurately that are scatteredly distributed.
44
Zhai, Naiju, and Xiaofeng Zhou. "Temperature Prediction of Heating Furnace Based on Deep Transfer Learning." Sensors 20, no. 17 (August 19, 2020): 4676. http://dx.doi.org/10.3390/s20174676.
Full textAbstract:
Heating temperature is very important in the process of billet production, and it directly affects the quality of billet. However, there is no direct method to measure billet temperature, so we need to accurately predict the temperature of each heating zone in the furnace in order to approximate the billet temperature. Due to the complexity of the heating process, it is difficult to accurately predict the temperature of each heating zone and each heating zone sensor datum to establish a model, which will increase the cost of calculation. To solve these two problems, a two-layer transfer learning framework based on a temporal convolution network (TL-TCN) is proposed for the first time, which transfers the knowledge learned from the source heating zone to the target heating zone. In the first layer, the TCN model is built for the source domain data, and the self-transfer learning method is used to optimize the TCN model to obtain the basic model, which improves the prediction accuracy of the source domain. In the second layer, we propose two frameworks: one is to generate the target model directly by using fine-tuning, and the other is to generate the target model by using generative adversarial networks (GAN) for domain adaption. Case studies demonstrated that the proposed TL-TCN framework achieves state-of-the-art prediction results on each dataset, and the prediction errors are significantly reduced. Consistent results applied to each dataset indicate that this framework is the most advanced method to solve the above problem under the condition of limited samples.
45
Yu, Zhipeng, Jianghai Zhao, Yucheng Wang, Linglong He, and Shaonan Wang. "Surface EMG-Based Instantaneous Hand Gesture Recognition Using Convolutional Neural Network with the Transfer Learning Method." Sensors 21, no. 7 (April 5, 2021): 2540. http://dx.doi.org/10.3390/s21072540.
Full textAbstract:
In recent years, surface electromyography (sEMG)-based human–computer interaction has been developed to improve the quality of life for people. Gesture recognition based on the instantaneous values of sEMG has the advantages of accurate prediction and low latency. However, the low generalization ability of the hand gesture recognition method limits its application to new subjects and new hand gestures, and brings a heavy training burden. For this reason, based on a convolutional neural network, a transfer learning (TL) strategy for instantaneous gesture recognition is proposed to improve the generalization performance of the target network. CapgMyo and NinaPro DB1 are used to evaluate the validity of our proposed strategy. Compared with the non-transfer learning (non-TL) strategy, our proposed strategy improves the average accuracy of new subject and new gesture recognition by 18.7% and 8.74%, respectively, when up to three repeated gestures are employed. The TL strategy reduces the training time by a factor of three. Experiments verify the transferability of spatial features and the validity of the proposed strategy in improving the recognition accuracy of new subjects and new gestures, and reducing the training burden. The proposed TL strategy provides an effective way of improving the generalization ability of the gesture recognition system.
46
Zhou, Shuo, Wenwen Li, Christopher Cox, and Haiping Lu. "Side Information Dependence as a Regularizer for Analyzing Human Brain Conditions across Cognitive Experiments." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 04 (April 3, 2020): 6957–64. http://dx.doi.org/10.1609/aaai.v34i04.6179.
Full textAbstract:
The increasing of public neuroimaging datasets opens a door to analyzing homogeneous human brain conditions across datasets by transfer learning (TL). However, neuroimaging data are high-dimensional, noisy, and with small sample sizes. It is challenging to learn a robust model for data across different cognitive experiments and subjects. A recent TL approach minimizes domain dependence to learn common cross-domain features, via the Hilbert-Schmidt Independence Criterion (HSIC). Inspired by this approach and the multi-source TL theory, we propose a Side Information Dependence Regularization (SIDeR) learning framework for TL in brain condition decoding. Specifically, SIDeR simultaneously minimizes the empirical risk and the statistical dependence on the domain side information, to reduce the theoretical generalization error bound. We construct 17 brain decoding TL tasks using public neuroimaging data for evaluation. Comprehensive experiments validate the superiority of SIDeR over ten competing methods, particularly an average improvement of 15.6% on the TL tasks with multi-source experiments.
47
Yaqub, Muhammad, Feng Jinchao, Shahzad Ahmed, Kaleem Arshid, Muhammad Atif Bilal, Muhammad Pervez Akhter, and Muhammad Sultan Zia. "GAN-TL: Generative Adversarial Networks with Transfer Learning for MRI Reconstruction." Applied Sciences 12, no. 17 (September 2, 2022): 8841. http://dx.doi.org/10.3390/app12178841.
Full textAbstract:
Generative adversarial networks (GAN), which are fueled by deep learning, are an efficient technique for image reconstruction using under-sampled MR data. In most cases, the performance of a particular model’s reconstruction must be improved by using a substantial proportion of the training data. However, gathering tens of thousands of raw patient data for training the model in actual clinical applications is difficult because retaining k-space data is not customary in the clinical process. Therefore, it is imperative to increase the generalizability of a network that was created using a small number of samples as quickly as possible. This research explored two unique applications based on deep learning-based GAN and transfer learning. Seeing as MRI reconstruction procedures go for brain and knee imaging, the proposed method outperforms current techniques in terms of signal-to-noise ratio (PSNR) and structural similarity index (SSIM). As compared to the results of transfer learning for the brain and knee, using a smaller number of training cases produced superior results, with acceleration factor (AF) 2 (for brain PSNR (39.33); SSIM (0.97), for knee PSNR (35.48); SSIM (0.90)) and AF 4 (for brain PSNR (38.13); SSIM (0.95), for knee PSNR (33.95); SSIM (0.86)). The approach that has been described would make it easier to apply future models for MRI reconstruction without necessitating the acquisition of vast imaging datasets.
48
Peng, DunLu, YinRui Wang, Cong Liu, and Zhang Chen. "TL-NER: A Transfer Learning Model for Chinese Named Entity Recognition." Information Systems Frontiers 22, no. 6 (June 4, 2019): 1291–304. http://dx.doi.org/10.1007/s10796-019-09932-y.
Full text
49
Li, Jun, Yongbao Liu, and Qijie Li. "Generative adversarial network and transfer-learning-based fault detection for rotating machinery with imbalanced data condition." Measurement Science and Technology 33, no. 4 (January 11, 2022): 045103. http://dx.doi.org/10.1088/1361-6501/ac3945.
Full textAbstract:
Abstract Intelligent fault diagnosis achieves tremendous success in machine fault diagnosis because of its outstanding data-driven capability. However, the severely imbalanced dataset in practical scenarios of industrial rotating machinery is still a big challenge for the development of intelligent fault diagnosis methods. In this paper, we solve this issue by constructing a novel deep learning model incorporated with a transfer learning (TL) method based on the time-generative adversarial network (Time-GAN) and efficient-net models. Firstly, the proposed model, called Time-GAN-TL, extends the imbalanced fault diagnosis of rolling bearings using time-series GAN. Secondly, balanced vibration signals are converted into two-dimensional images for training and classification by implementing the efficient-net into the transfer learning method. Finally, the proposed method is validated using two types of rolling bearing experimental data. The high-precision diagnosis results of the transfer learning experiments and the comparison with other representative fault diagnosis classification methods reveal the efficiency, reliability, and generalization performance of the presented model.
50
Zhou, Jianmin, Xiaotong Yang, and Jiahui Li. "Deep Residual Network Combined with Transfer Learning Based Fault Diagnosis for Rolling Bearing." Applied Sciences 12, no. 15 (August 3, 2022): 7810. http://dx.doi.org/10.3390/app12157810.
Full textAbstract:
Fault diagnosis of rolling bearings is significant for mechanical equipment operation and maintenance. Presently, the deep convolutional neural network (CNN) is increasingly used for fault diagnosis of rolling bearings, but CNN has challenges with incomplete training and lengthy training times. This paper proposes a residual network combined with the transfer learning (ResNet-TL) based diagnosis method for rolling bearing, which can preprocess the one-dimensional data of vibration signals into image data. Then, the transfer learning theory in parameter transfer is applied to the training of the network model, and the ResNet34 network is pre-trained and re-trained; the image data are selected to be the inputs of the fault diagnosis model. The experimental validation of the rolling bearing fault dataset collected from the practical bench and Case Western Reserve University shows the superiority of the ResNet34-TL model compared with other classification models.