Journal articles on the topic 'Pytorch model'
To see the other types of publications on this topic, follow the link: Pytorch model.
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Pytorch model.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Meshram, Anshuja Anand. "Review on Different Software Tools for Deep Learning." International Journal for Research in Applied Science and Engineering Technology 10, no. 1 (January 31, 2022): 565–71. http://dx.doi.org/10.22214/ijraset.2022.39873.
Full textAbstract:
Abstract: Deep Learning Applications are being applied in various domains in recent years. Training a deep learning model is a very time consuming task. But, many open source frameworks are available to simplify this task. In this review paper we have discussed the features of some popular open source software tools available for deep learning along with their advantages and disadvantages. Software tools discussed in this paper are Tensorflow, Keras, Pytorch, Microsoft Cognitive Toolkit (CNTK). Keywords: Deep Learning, Frameworks, Open Source, Tensorflow, Pytorch, Keras, CNTK
2
Munjal, Rohan, Sohaib Arif, Frank Wendler, and Olfa Kanoun. "Comparative Study of Machine-Learning Frameworks for the Elaboration of Feed-Forward Neural Networks by Varying the Complexity of Impedimetric Datasets Synthesized Using Eddy Current Sensors for the Characterization of Bi-Metallic Coins." Sensors 22, no. 4 (February 9, 2022): 1312. http://dx.doi.org/10.3390/s22041312.
Full textAbstract:
A suitable framework for the development of artificial neural networks is important because it decides the level of accuracy, which can be reached for a certain dataset and increases the certainty about the reached classification results. In this paper, we conduct a comparative study for the performance of four frameworks, Keras with TensorFlow, Pytorch, TensorFlow, and Cognitive Toolkit (CNTK), for the elaboration of neural networks. The number of neurons in the hidden layer of the neural networks is varied from 8 to 64 to understand its effect on the performance metrics of the frameworks. A test dataset is synthesized using an analytical model and real measured impedance spectra by an eddy current sensor coil on EUR 2 and TRY 1 coins. The dataset has been extended by using a novel method based on interpolation technique to create datasets with different difficulty levels to replicate the scenario with a good imitation of EUR 2 coins and to investigate the limit of the prediction accuracy. It was observed that the compared frameworks have high accuracy performance for a lower level of difficulty in the dataset. As the difficulty in the dataset is raised, there was a drop in the accuracy of CNTK and Keras with TensorFlow depending upon the number of neurons in the hidden layers. It was observed that CNTK has the overall worst accuracy performance with an increase in the difficulty level of the datasets. Therefore, the major comparison was confined to Pytorch and TensorFlow. It was observed for Pytorch and TensorFlow with 32 and 64 neurons in hidden layers that there is a minor drop in the accuracy with an increase in the difficulty level of the dataset and was above 90% until both the coins were 80% closer to each other in terms of electrical and magnetic properties. However, Pytorch with 32 neurons in the hidden layer has a reduction in model size by 70% and 16.3% and predicts the class, 73.6% and 15.6% faster in comparison to TensorFlow and Pytorch with 64 neurons.
3
Kyrylenko, O. M. "Development of a method of re-identification of a person." Optoelectronic Information-Power Technologies 41, no. 1 (May 2, 2022): 25–32. http://dx.doi.org/10.31649/1681-7893-2021-41-1-25-32.
Full textAbstract:
The review of OSNet neural network architecture is made for the purpose of training of own models of re-identification of the person. The structure of the neural network was also considered. Existing data sets for model training are investigated. Models were trained using PyTorch. The obtained own models were tested on the validation databases Market-1501 and DukeMTMC-reID. The results of learning neural network models are presented. The results are obtained in comparison with existing analogues.
4
Pobeda, Sergey, M. Chernyh, F. Makarenko, and Konstantin Zolnikov. "Creation of a behavioral model of a LDMOS transistor based on an artificial MLP neural network and its description in Verilog-A language." Modeling of systems and processes 14, no. 2 (July 26, 2021): 28–34. http://dx.doi.org/10.12737/2219-0767-2021-14-2-28-34.
Full textAbstract:
The article deals with the creation of a behavioral model of lateral metal oxide transistors (LDMOS) based on a neural network of the multilayer percep-tron type. The model is identified using a backpropa-gation algorithm. Demonstrated the process of creating an ANN model using Pytorch, a machine learning framework for the Python language, with subsequent transfer to the standard analog circuit modeling lan-guage Verilog-A.
5
Belozerov, Ilya Andreevich, and Vladimir Anatolievich Sudakov. "Investigation of machine learning models for medical image segmentation." Keldysh Institute Preprints, no. 37 (2022): 1–15. http://dx.doi.org/10.20948/prepr-2022-37.
Full textAbstract:
On the example of X-ray images of human lungs, the analysis and construction of models of semantic segmentation of computer vision is carried out. The paper explores various approaches to medical image processing, comparing methods for implementing deep learning models and evaluating them. 5 models of neural networks have been developed to perform the segmentation task, implemented using such well-known libraries as: TensorFlow and PyTorch. The model with the best performance can be used to build a system for automatic segmentation of various images of patients and calculate the characteristics of their organs.
6
Dalskov, Anders, Daniel Escudero, and Marcel Keller. "Secure Evaluation of Quantized Neural Networks." Proceedings on Privacy Enhancing Technologies 2020, no. 4 (October 1, 2020): 355–75. http://dx.doi.org/10.2478/popets-2020-0077.
Full textAbstract:
AbstractWe investigate two questions in this paper: First, we ask to what extent “MPC friendly” models are already supported by major Machine Learning frameworks such as TensorFlow or PyTorch. Prior works provide protocols that only work on fixed-point integers and specialized activation functions, two aspects that are not supported by popular Machine Learning frameworks, and the need for these specialized model representations means that it is hard, and often impossible, to use e.g., TensorFlow to design, train and test models that later have to be evaluated securely. Second, we ask to what extent the functionality for evaluating Neural Networks already exists in general-purpose MPC frameworks. These frameworks have received more scrutiny, are better documented and supported on more platforms. Furthermore, they are typically flexible in terms of the threat model they support. In contrast, most secure evaluation protocols in the literature are targeted to a specific threat model and their implementations are only a “proof-of-concept”, making it very hard for their adoption in practice. We answer both of the above questions in a positive way:We observe that the quantization techniques supported by both TensorFlow, PyTorch and MXNet can provide models in a representation that can be evaluated securely; and moreover, that this evaluation can be performed by a general purpose MPC framework. We perform extensive benchmarks to understand the exact trade-offs between different corruption models, network sizes and efficiency. These experiments provide an interesting insight into cost between active and passive security, as well as honest and dishonest majority. Our work shows then that the separating line between existing ML frameworks and existing MPC protocols may be narrower than implicitly suggested by previous works.
7
Duan, Bo, Zhengmin Xu, Lili Pan, Wenxia Chen, and Zhongwei Qiao. "Prediction of Hearing Prognosis of Large Vestibular Aqueduct Syndrome Based on the PyTorch Deep Learning Model." Journal of Healthcare Engineering 2022 (April 13, 2022): 1–6. http://dx.doi.org/10.1155/2022/4814577.
Full textAbstract:
In order to compare magnetic resonance imaging (MRI) findings of patients with large vestibular aqueduct syndrome (LVAS) in the stable hearing loss (HL) group and the fluctuating HL group, this paper provides reference for clinicians' early intervention. From January 2001 to January 2016, patients with hearing impairment diagnosed as LVAS in infancy in the Department of Otorhinolaryngology, Head and Neck Surgery, Children's Hospital of Fudan University were collected and divided into the stable HL group (n = 29) and the fluctuating HL group (n = 30). MRI images at initial diagnosis were collected, and various deep learning neural network training models were established based on PyTorch to classify and predict the two series. Vgg16_bn, vgg19_bn, and ResNet18, convolutional neural networks (CNNs) with fewer layers, had favorable effects for model building, with accs of 0.9, 0.8, and 0.85, respectively. ResNet50, a CNN with multiple layers and an acc of 0.54, had relatively poor effects. The GoogLeNet-trained model performed best, with an acc of 0.98. We conclude that deep learning-based radiomics can assist doctors in accurately predicting LVAS patients to classify them into either fluctuating or stable HL types and adopt differentiated treatment methods.
8
Li, Yaxin, Wei Jin, Han Xu, and Jiliang Tang. "DeepRobust: a Platform for Adversarial Attacks and Defenses." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 18 (May 18, 2021): 16078–80. http://dx.doi.org/10.1609/aaai.v35i18.18017.
Full textAbstract:
DeepRobust is a PyTorch platform for generating adversarial examples and building robust machine learning models for different data domains. Users can easily evaluate the attack performance against different defense methods with DeepRobust and get performance analyzing visualization. In this paper, we introduce the functions of DeepRobust with detailed instructions. We believe that DeepRobust is a useful tool to measure deep learning model robustness and to find the suitable countermeasures against adversarial attacks. The platform is kept updated and can be found at https://github.com/DSE-MSU/DeepRobust. More details of instruction can be found in the documentation at https://deeprobust.readthedocs.io/en/latest/.
9
How, Chun Kit, Ismail Mohd Khairuddin, Mohd Azraai Mohd Razman, Anwar P. P. Abdul Majeed, and Wan Hasbullah Mohd Isa. "Development of Audio-Visual Speech Recognition using Deep-Learning Technique." MEKATRONIKA 4, no. 1 (June 27, 2022): 88–95. http://dx.doi.org/10.15282/mekatronika.v4i1.8625.
Full textAbstract:
Deep learning is a technique with artificial intelligent (AI) that simulate humans’ learning behavior. Audio-visual speech recognition is important for the listener understand the emotions behind the spoken words truly. In this thesis, two different deep learning models, Convolutional Neural Network (CNN) and Deep Neural Network (DNN), were developed to recognize the speech’s emotion from the dataset. Pytorch framework with torchaudio library was used. Both models were given the same training, validation, testing, and augmented datasets. The training will be stopped when the training loop reaches ten epochs, or the validation loss function does not improve for five epochs. At the end, the highest accuracy and lowest loss function of CNN model in the training dataset are 76.50% and 0.006029 respectively, meanwhile the DNN model achieved 75.42% and 0.086643 respectively. Both models were evaluated using confusion matrix. In conclusion, CNN model has higher performance than DNN model, but needs to improvise as the accuracy of testing dataset is low and the loss function is high.
10
McAllister, Dianna, Mauro Mendez, Ariana Bermúdez, and Pascal Tyrrell. "Visualization of Layers Within a Convolutional Neural Network Using Gradient Activation Maps." Journal of Undergraduate Life Sciences 14, no. 1 (December 31, 2020): 6. http://dx.doi.org/10.33137/juls.v14i1.35833.
Full textAbstract:
Introduction: Convolutional neural networks (CNNs) are machine learning tools that have great potential in the field of medical imaging. However, it is often regarded as a “black box” as the process that is used by the machine to acquire a result is not transparent. It would be valuable to find a method to be able to understand how the machine comes to its decision. Therefore, the purpose of this study is to examine how effective gradient-weighted class activation mapping (grad-CAM) visualizations are for certain layers in a CNN-based dental x-ray artifact prediction model.
Methods: To tackle this project, Python code using PyTorch trained a CNN to classify dental plates as unusable or usable depending on the presence of artifacts. Furthermore, Python using PyTorch was also used to overlay grad-CAM visualizations on the given input images for various layers within the model. One image with seventeen different overlays of artifacts was used in this study.
Results: In earlier layers, the model appeared to focus on general features such as lines and edges of the teeth, while in later layers, the model was more interested in detailed aspects of the image. All images that contained artifacts resulted in the model focusing on more detailed areas of the image rather than the artifacts themselves. Whereas the images without artifacts resulted in the model focusing on the visualization of areas that surrounded the teeth.
Discussion and Conclusion: As subsequent layers examined more detailed aspects of the image as shown by the grad-CAM visualizations, they provided better insight into how the model processes information when it is making its classifications. Since all the images with artifacts showed similar trends in the visualizations of the various layers, it provides evidence to suggest that the location and size of the artifact does not affect the model’s pattern recognition and image classification.
11
Sankaran, Anush, Olivier Mastropietro, Ehsan Saboori, Yasser Idris, Davis Sawyer, MohammadHossein AskariHemmat, and Ghouthi Boukli Hacene. "Deeplite NeutrinoTM: A BlackBox Framework for Constrained Deep Learning Model Optimization." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 17 (May 18, 2021): 15166–74. http://dx.doi.org/10.1609/aaai.v35i17.17780.
Full textAbstract:
Designing deep learning-based solutions is becoming a race for training deeper models with a greater number of layers. While a large-size deeper model could provide competitive accuracy, it creates a lot of logistical challenges and unreasonable resource requirements during development and deployment. This has been one of the key reasons for deep learning models not being excessively used in various production environments, especially in edge devices. There is an immediate requirement for optimizing and compressing these deep learning models, to enable on-device intelligence. In this research, we introduce a black-box framework, Deeplite Neutrino^{TM} for production-ready optimization of deep learning models. The framework provides an easy mechanism for the end-users to provide constraints such as a tolerable drop in accuracy or target size of the optimized models, to guide the whole optimization process. The framework is easy to include in an existing production pipeline and is available as a Python Package, supporting PyTorch and Tensorflow libraries. The optimization performance of the framework is shown across multiple benchmark datasets and popular deep learning models. Further, the framework is currently used in production and the results and testimonials from several clients are summarized.
12
Zhou, Yuqian, Jianbo Jiao, Haibin Huang, Yang Wang, Jue Wang, Honghui Shi, and Thomas Huang. "When AWGN-Based Denoiser Meets Real Noises." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 07 (April 3, 2020): 13074–81. http://dx.doi.org/10.1609/aaai.v34i07.7009.
Full textAbstract:
Discriminative learning based image denoisers have achieved promising performance on synthetic noises such as Additive White Gaussian Noise (AWGN). The synthetic noises adopted in most previous work are pixel-independent, but real noises are mostly spatially/channel-correlated and spatially/channel-variant. This domain gap yields unsatisfied performance on images with real noises if the model is only trained with AWGN. In this paper, we propose a novel approach to boost the performance of a real image denoiser which is trained only with synthetic pixel-independent noise data dominated by AWGN. First, we train a deep model that consists of a noise estimator and a denoiser with mixed AWGN and Random Value Impulse Noise (RVIN). We then investigate Pixel-shuffle Down-sampling (PD) strategy to adapt the trained model to real noises. Extensive experiments demonstrate the effectiveness and generalization of the proposed approach. Notably, our method achieves state-of-the-art performance on real sRGB images in the DND benchmark among models trained with synthetic noises. Codes are available at https://github.com/yzhouas/PD-Denoising-pytorch.
13
Villanueva-Domingo, Pablo, Francisco Villaescusa-Navarro, Daniel Anglés-Alcázar, Shy Genel, Federico Marinacci, David N. Spergel, Lars Hernquist, Mark Vogelsberger, Romeel Dave, and Desika Narayanan. "Inferring Halo Masses with Graph Neural Networks." Astrophysical Journal 935, no. 1 (August 1, 2022): 30. http://dx.doi.org/10.3847/1538-4357/ac7aa3.
Full textAbstract:
Abstract Understanding the halo–galaxy connection is fundamental in order to improve our knowledge on the nature and properties of dark matter. In this work, we build a model that infers the mass of a halo given the positions, velocities, stellar masses, and radii of the galaxies it hosts. In order to capture information from correlations among galaxy properties and their phase space, we use Graph Neural Networks (GNNs), which are designed to work with irregular and sparse data. We train our models on galaxies from more than 2000 state-of-the-art simulations from the Cosmology and Astrophysics with MachinE Learning Simulations project. Our model, which accounts for cosmological and astrophysical uncertainties, is able to constrain the masses of the halos with a ∼0.2 dex accuracy. Furthermore, a GNN trained on a suite of simulations is able to preserve part of its accuracy when tested on simulations run with a different code that utilizes a distinct subgrid physics model, showing the robustness of our method. The PyTorch Geometric implementation of the GNN is publicly available on GitHub (https://github.com/PabloVD/HaloGraphNet).
14
Yu, Qinhan. "Animal Image Classifier Based on Convolutional Neural Network." SHS Web of Conferences 144 (2022): 03017. http://dx.doi.org/10.1051/shsconf/202214403017.
Full textAbstract:
In modern society, there are dogs and cats around people, as well as rare wild animals living in nature. The relationship between human beings and animals is getting closer and closer. The rapid development of machine learning and deep learning technology has been widely used in the academic field. Aiming at the problem of animal image classification, this paper uses Pytorch to learn about 10,000 pictures containing cats, dogs, and wild animals (tiger, lion, etc.) based on the research algorithm of convolutional neural network in the field of image classification. And a convolutional neural network model that can realize the animal image classifier is established and optimized, so that the model can efficiently classify cats, dogs and wildlife pictures. The results show that the accuracy of the two models is above 90%, and the model loss ranges from 0.706 to 0.061, and 0.807 to 0.051, respectively, showing the characteristics of good model fitting effect and strong optimization ability. Meanwhile, The accuracy of the model can be increased by properly increasing the number of full connection layers. Therefore, by constructing the convolutional neural network, the accurate detection of national ecological protection animal images can be realized.
15
Zhang, Lisa, Pouria Fewzee, and Charbel Feghali. "AI education matters." AI Matters 7, no. 3 (September 2021): 18–20. http://dx.doi.org/10.1145/3511322.3511327.
Full textAbstract:
We introduce a Model AI Assignment (Neller et al., 2021) where students combine various techniques from a deep learning course to build a denoising autoencoder (Shen, Mueller, Barzilay, & Jaakkola, 2020) for news headlines. Students then use this denoising autoencoder to query similar headlines, and interpolate between headlines. Building this denoising autoencoder requires students to apply many course concepts, including data augmentation, word and sentence embeddings, autoencoders, recurrent neural networks, sequence-to-sequence networks, and temperature. As such, this assignment can be ideal as a final assessment that synthesizes many topics. This assignment is written in PyTorch, uses the torchtext package, and is intended to be completed on the Google Colab platform.
16
Huang, Xiang. "Personalized Travel Route Recommendation Model of Intelligent Service Robot Using Deep Learning in Big Data Environment." Journal of Robotics 2022 (January 29, 2022): 1–8. http://dx.doi.org/10.1155/2022/7778592.
Full textAbstract:
Aiming at the problems that the traditional model is difficult to extract information features, difficult to learn deep knowledge, and cannot automatically and effectively obtain features, which leads to the problem of low recommendation accuracy, this paper proposes a personalized tourism route recommendation model of intelligent service robot using deep learning in a big data environment. Firstly, by crawling the relevant website data, obtain the basic information data and comment the text data of tourism service items, as well as the basic information data, and comment the text data of users and preprocess them, such as data cleaning. Then, a neural network model based on the self-attention mechanism is proposed, in which the data features are obtained by the Gaussian kernel function and node2vec model, and the self-attention mechanism is used to capture the long-term and short-term preferences of users. Finally, the processed data is input into the trained recommendation model to generate a personalized tourism route recommendation scheme. The experimental analysis of the proposed model based on Pytorch deep learning framework shows that its Pre@10, Rec@10 values are 88% and 83%, respectively, and the mean square error is 1.537, which are better than other comparison models and closer to the real tourist route of the tourists.
17
Emmitt, Joshua, Sina Masoud-Ansari, Rebecca Phillipps, Stacey Middleton, Jennifer Graydon, and Simon Holdaway. "Machine learning for stone artifact identification: Distinguishing worked stone artifacts from natural clasts using deep neural networks." PLOS ONE 17, no. 8 (August 10, 2022): e0271582. http://dx.doi.org/10.1371/journal.pone.0271582.
Full textAbstract:
Stone artifacts are often the most abundant class of objects found in archaeological sites but their consistent identification is limited by the number of experienced analysts available. We report a machine learning based technology for stone artifact identification as part of a solution to the lack of such experts directed at distinguishing worked stone objects from naturally occurring lithic clasts. Three case study locations from Egypt, Australia, and New Zealand provide a data set of 6769 2D images, 3868 flaked artifact and 2901 rock images used to train and test a machine learning model based on an openly available PyTorch implementation of Faster R-CNN ResNet 50. Results indicate 100% agreement between the model and original human derived classifications, a better performance than the results achieved independently by two human analysts who reassessed the 2D images available to the machine learning model. Machine learning neural networks provide the potential to consistently assess the composition of large archaeological assemblages composed of objects modified in a variety of ways.
18
Schorr, Christian, Payman Goodarzi, Fei Chen, and Tim Dahmen. "Neuroscope: An Explainable AI Toolbox for Semantic Segmentation and Image Classification of Convolutional Neural Nets." Applied Sciences 11, no. 5 (March 3, 2021): 2199. http://dx.doi.org/10.3390/app11052199.
Full textAbstract:
Trust in artificial intelligence (AI) predictions is a crucial point for a widespread acceptance of new technologies, especially in sensitive areas like autonomous driving. The need for tools explaining AI for deep learning of images is thus eminent. Our proposed toolbox Neuroscope addresses this demand by offering state-of-the-art visualization algorithms for image classification and newly adapted methods for semantic segmentation of convolutional neural nets (CNNs). With its easy to use graphical user interface (GUI), it provides visualization on all layers of a CNN. Due to its open model-view-controller architecture, networks generated and trained with Keras and PyTorch are processable, with an interface allowing extension to additional frameworks. We demonstrate the explanation abilities provided by Neuroscope using the example of traffic scene analysis.
19
Hong-Jie Zhang, Hong-Jie Zhang, Guo-Jun Lin Hong-Jie Zhang, Tian-Tian Chen Guo-Jun Lin, Shun-Yong Zhou Tian-Tian Chen, and Hong-Rong Jing Shun-Yong Zhou. "Facial Expression Recognition Based on Improved Residual Block Network." 電腦學刊 33, no. 4 (August 2022): 159–68. http://dx.doi.org/10.53106/199115992022083304013.
Full textAbstract:
<p>Facial expression recognition is widely used, but there are some problems such as complex scenes, lack of data sets and low recognition rate. In this paper, we construct a new network model and name it RNFC. The RNFC network adopts 6 improved residual blocks to extract features. Features are passed into the fully connected layer by flattening the data, and Dropout techniques are introduced between the fully connected layers to prevent overfitting of the model. Based on the pytorch framework, we use a cross-entropy loss function to improve the training speed of the network. And perform denoising and enhancement pre-processing on the FER2013 dataset. The RNFC network is trained and tested on the pretreated FER2013. It has a higher recognition rate than classical networks such as VGGnet19 and ResNet18.</p> <p> </p>
20
Hu, JianSheng, JunJie Ma, Bin Xiao, and Rui Zhang. "Improved Lightweight YOLOv3 model for Target Detection Algorithm." Journal of Physics: Conference Series 2370, no. 1 (November 1, 2022): 012029. http://dx.doi.org/10.1088/1742-6596/2370/1/012029.
Full textAbstract:
When detecting small objects in interior situations, the classic object detection algorithm performs poorly in terms of real-time detection task and high precision detection task. This paper suggests an optimized tiny-YOLOv3-Shufflenetv2 light-weight model based on indoor scenes. The scheme adopts the fusion light-weight network which combines ShuffleNetv2 and YOLOv3, it reduces the complexity of the model to meet the lightweight requirements while ensuring good detection results for deployment to mobile robots. Also in this paper, an indoor small target object dataset, indoor-2022, is created to improve and optimize the model for the data images. YOLOv3, YOLOv3-Shufflenetv2, and tiny-YOLOv3-Shufflenetv2 are trained and tested on the indoor-2022 small target dataset in the Pytorch framework. The experimental findings indicate that in the indoor-2022 dataset. Compared with the single YOLOv3 model for object detection tasks, the fusion improved model used in this article improves the recognition ability of small objects in indoor images, With a 10-fold reduction in model size and a 4-fold increase in detection speed, only results in 1.6% reduction in the mean accuracy (mAP), and the comparison experiments with the current stage of traditional target detection algorithms validate the proposed tiny-YOLOv3-Shufflenetv2 model is verified to be superior and feasible. The optimized model in this article reduces mannequin parameters and model size while additionally ensuring the accuracy and velocity of inspection, and meets the requirements for deployment on indoor mobile robots.
21
Zhang, Yaolong, Junfan Xia, and Bin Jiang. "REANN: A PyTorch-based end-to-end multi-functional deep neural network package for molecular, reactive, and periodic systems." Journal of Chemical Physics 156, no. 11 (March 21, 2022): 114801. http://dx.doi.org/10.1063/5.0080766.
Full textAbstract:
In this work, we present a general purpose deep neural network package for representing energies, forces, dipole moments, and polarizabilities of atomistic systems. This so-called recursively embedded atom neural network model takes advantages of both the physically inspired atomic descriptor based neural networks and the message-passing based neural networks. Implemented in the PyTorch framework, the training process is parallelized on both the central processing unit and the graphics processing unit with high efficiency and low memory in which all hyperparameters can be optimized automatically. We demonstrate the state-of-the-art accuracy, high efficiency, scalability, and universality of this package by learning not only energies (with or without forces) but also dipole moment vectors and polarizability tensors in various molecular, reactive, and periodic systems. An interface between a trained model and LAMMPs is provided for large scale molecular dynamics simulations. We hope that this open-source toolbox will allow for future method development and applications of machine learned potential energy surfaces and quantum-chemical properties of molecules, reactions, and materials.
22
Kari, Dariush, and Andrew C. Singer. "Underwater acoustic localization using a modular differentiable model for acoustic wave propagation." Journal of the Acoustical Society of America 151, no. 4 (April 2022): A234. http://dx.doi.org/10.1121/10.0011171.
Full textAbstract:
Model-based algorithms for underwater acoustic localization are highly dependent upon environmental knowledge that is rarely available. Moreover, model-based localization usually involves optimization in high-dimensional spaces to find the best candidate for source localization. On the other hand, data-driven methods deliver poor generalization performance in data starved scenarios. Therefore, in order to improve generalization, we propose a modular, deep learning-based architecture that inherently learns the multipath structure, while learning the environmental properties from the training data. Furthermore, since ReLU-based fully connected networks cannot capture the high-frequency contents of the signals of interest properly, we use sinusoidal activations. Although the model needs an estimate of the carrier frequency as a hyper-parameter, it is observed that it can tolerate deviations about the true frequency. This model maps the source and receiver locations to received signals and is then used in a gradient descent manner exploiting the automatic differentiation toolbox in PyTorch, to find the source location. To evaluate the performance of the algorithm, we use Bellhop to generate data for a given set of environmental parameters. We show that our method outperforms matched-field processing and a deep fully connected network without a modular structure.
23
Choi, Hyeonseong, and Jaehwan Lee. "Efficient Use of GPU Memory for Large-Scale Deep Learning Model Training." Applied Sciences 11, no. 21 (November 4, 2021): 10377. http://dx.doi.org/10.3390/app112110377.
Full textAbstract:
To achieve high accuracy when performing deep learning, it is necessary to use a large-scale training model. However, due to the limitations of GPU memory, it is difficult to train large-scale training models within a single GPU. NVIDIA introduced a technology called CUDA Unified Memory with CUDA 6 to overcome the limitations of GPU memory by virtually combining GPU memory and CPU memory. In addition, in CUDA 8, memory advise options are introduced to efficiently utilize CUDA Unified Memory. In this work, we propose a newly optimized scheme based on CUDA Unified Memory to efficiently use GPU memory by applying different memory advise to each data type according to access patterns in deep learning training. We apply CUDA Unified Memory technology to PyTorch to see the performance of large-scale learning models through the expanded GPU memory. We conduct comprehensive experiments on how to efficiently utilize Unified Memory by applying memory advises when performing deep learning. As a result, when the data used for deep learning are divided into three types and a memory advise is applied to the data according to the access pattern, the deep learning execution time is reduced by 9.4% compared to the default Unified Memory.
24
Bayrakdar, Ibrahim S., Kaan Orhan, Özer Çelik, Elif Bilgir, Hande Sağlam, Fatma Akkoca Kaplan, Sinem Atay Görür, Alper Odabaş, Ahmet Faruk Aslan, and Ingrid Różyło-Kalinowska. "A U-Net Approach to Apical Lesion Segmentation on Panoramic Radiographs." BioMed Research International 2022 (January 15, 2022): 1–7. http://dx.doi.org/10.1155/2022/7035367.
Full textAbstract:
The purpose of the paper was the assessment of the success of an artificial intelligence (AI) algorithm formed on a deep-convolutional neural network (D-CNN) model for the segmentation of apical lesions on dental panoramic radiographs. A total of 470 anonymized panoramic radiographs were used to progress the D-CNN AI model based on the U-Net algorithm (CranioCatch, Eskisehir, Turkey) for the segmentation of apical lesions. The radiographs were obtained from the Radiology Archive of the Department of Oral and Maxillofacial Radiology of the Faculty of Dentistry of Eskisehir Osmangazi University. A U-Net implemented with PyTorch model (version 1.4.0) was used for the segmentation of apical lesions. In the test data set, the AI model segmented 63 periapical lesions on 47 panoramic radiographs. The sensitivity, precision, and F1-score for segmentation of periapical lesions at 70% IoU values were 0.92, 0.84, and 0.88, respectively. AI systems have the potential to overcome clinical problems. AI may facilitate the assessment of periapical pathology based on panoramic radiographs.
25
Wang, Yang, Bohai Yi, and Kexin Yu. "Exploration and research about key technologies concerning Deep Learning models targeting Mobile Terminals." Journal of Physics: Conference Series 2303, no. 1 (July 1, 2022): 012086. http://dx.doi.org/10.1088/1742-6596/2303/1/012086.
Full textAbstract:
Abstract With the rapid development of artificial intelligence and the popularity of mobile devices, mobile deep learning model technology has become a research hotspot in recent years. This paper studies the realization of the mobile terminal deep learning model from the optimization techniques of the deep learning model and the framework of deep learning. This review sorts out the model optimization techniques of pruning, quantization, and model knowledge distillation of deep learning models, and analyzes the lightweight deep learning models and deep learning frameworks suitable for mobile terminals. From the perspective of deep learning model compression, this paper provides multi-granularity pruning, pruning combined with batch normalization factor and filter correlation, joint dynamic pruning, pruning based on cross-entropy; a multi-module feature training method based on knowledge distillation, and an optimized model training strategy based on self-distillation; local quantization, exponential quantization. From the perspective of directly adopting a deep learning framework, this paper compares four different frameworks (Caffe/Caffe2, TensorFlow, Keras and Pytorch) introduced by different companies. The benefits of two other frameworks, TensorFlow Lite and FeatherCNN, are also mentioned. From the perspective of lightweight deep learning model design, this paper analyzes the design of three lightweight models such as SqueezeNet, MobileNet, and ShuffleNet, and compares their performance parameters such as accuracy gap and calculation speed with conventional models such as AlexNet and GoogleNet. Finally, the paper looks ahead to future directions in the field and what the authors believe are important ideas that may help inspire new ideas.
26
Lee, Pilhyeon, Youngjung Uh, and Hyeran Byun. "Background Suppression Network for Weakly-Supervised Temporal Action Localization." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 07 (April 3, 2020): 11320–27. http://dx.doi.org/10.1609/aaai.v34i07.6793.
Full textAbstract:
Weakly-supervised temporal action localization is a very challenging problem because frame-wise labels are not given in the training stage while the only hint is video-level labels: whether each video contains action frames of interest. Previous methods aggregate frame-level class scores to produce video-level prediction and learn from video-level action labels. This formulation does not fully model the problem in that background frames are forced to be misclassified as action classes to predict video-level labels accurately. In this paper, we design Background Suppression Network (BaS-Net) which introduces an auxiliary class for background and has a two-branch weight-sharing architecture with an asymmetrical training strategy. This enables BaS-Net to suppress activations from background frames to improve localization performance. Extensive experiments demonstrate the effectiveness of BaS-Net and its superiority over the state-of-the-art methods on the most popular benchmarks – THUMOS'14 and ActivityNet. Our code and the trained model are available at https://github.com/Pilhyeon/BaSNet-pytorch.
27
Wang, Tiange, Ruijie Jiang, YuLun (Elain) Lin, Kyle Monahan, Douglas Leaffer, Stephen Doroff, and Brian Tracey. "Scalable Machine Learning Approach to Classifying Transportation Noise at Two Urban Sites in Greater Boston, Massachusetts." INTER-NOISE and NOISE-CON Congress and Conference Proceedings 263, no. 1 (August 1, 2021): 4962–74. http://dx.doi.org/10.3397/in-2021-2907.
Full textAbstract:
The goal of this study was to characterize transportation noise by vehicle class in two urban communities, to inform studies of transport noise and ultra-fine particulates. Data were collected from April to September 2016 (150 days) of continuous recording in each urban community using high-resolution microphones. Training data was created for airplanes, trucks/buses, and train events by manual listening and extraction of audio files. Digital signal processing using STFT and Hanning windowing was performed in MATLAB, creating audio spectrograms with varying frequency: log vs linear frequency scales, and 4K vs 20K max frequency. For each of the four spectrogram sets, a neural net model using PyTorch was trained via a compute cluster. Initial results for a multi-class model provide an accuracy of 85%. Comparison between a selection of frequency scales and expanding to longer time periods is ongoing. Validation with airport transport logs and local bus and train schedules will be presented.
28
Ravi, Niranjan, and Mohamed El-Sharkawy. "Real-Time Embedded Implementation of Improved Object Detector for Resource-Constrained Devices." Journal of Low Power Electronics and Applications 12, no. 2 (April 13, 2022): 21. http://dx.doi.org/10.3390/jlpea12020021.
Full textAbstract:
Artificial intelligence (A.I.) has revolutionised a wide range of human activities, including the accelerated development of autonomous vehicles. Self-navigating delivery robots are recent trends in A.I. applications such as multitarget object detection, image classification, and segmentation to tackle sociotechnical challenges, including the development of autonomous driving vehicles, surveillance systems, intelligent transportation, and smart traffic monitoring systems. In recent years, object detection and its deployment on embedded edge devices have seen a rise in interest compared to other perception tasks. Embedded edge devices have limited computing power, which impedes the deployment of efficient detection algorithms in resource-constrained environments. To improve on-board computational latency, edge devices often sacrifice performance, creating the need for highly efficient A.I. models. This research examines existing loss metrics and their weaknesses, and proposes an improved loss metric that can address the bounding box regression problem. Enhanced metrics were implemented in an ultraefficient YOLOv5 network and tested on the targeted datasets. The latest version of the PyTorch framework was incorporated in model development. The model was further deployed using the ROS 2 framework running on NVIDIA Jetson Xavier NX, an embedded development platform, to conduct the experiment in real time.
29
Shahriari, Mostafa, Rudolf Ramler, and Lukas Fischer. "How Do Deep-Learning Framework Versions Affect the Reproducibility of Neural Network Models?" Machine Learning and Knowledge Extraction 4, no. 4 (October 5, 2022): 888–911. http://dx.doi.org/10.3390/make4040045.
Full textAbstract:
In the last decade, industry’s demand for deep learning (DL) has increased due to its high performance in complex scenarios. Due to the DL method’s complexity, experts and non-experts rely on blackbox software packages such as Tensorflow and Pytorch. The frameworks are constantly improving, and new versions are released frequently. As a natural process in software development, the released versions contain improvements/changes in the methods and their implementation. Moreover, versions may be bug-polluted, leading to the model performance decreasing or stopping the model from working. The aforementioned changes in implementation can lead to variance in obtained results. This work investigates the effect of implementation changes in different major releases of these frameworks on the model performance. We perform our study using a variety of standard datasets. Our study shows that users should consider that changing the framework version can affect the model performance. Moreover, they should consider the possibility of a bug-polluted version before starting to debug source code that had an excellent performance before a version change. This also shows the importance of using virtual environments, such as Docker, when delivering a software product to clients.
30
Yang, Baohua, Jifeng Ma, Xia Yao, Weixing Cao, and Yan Zhu. "Estimation of Leaf Nitrogen Content in Wheat Based on Fusion of Spectral Features and Deep Features from Near Infrared Hyperspectral Imagery." Sensors 21, no. 2 (January 17, 2021): 613. http://dx.doi.org/10.3390/s21020613.
Full textAbstract:
Nitrogen is an important indicator for monitoring wheat growth. The rapid development and wide application of non-destructive detection provide many approaches for estimating leaf nitrogen content (LNC) in wheat. Previous studies have shown that better results have been obtained in the estimation of LNC in wheat based on spectral features. However, the lack of automatically extracted features leads to poor universality of the estimation model. Therefore, a feature fusion method for estimating LNC in wheat by combining spectral features with deep features (spatial features) was proposed. The deep features were automatically obtained with a convolutional neural network model based on the PyTorch framework. The spectral features were obtained using spectral information including position features (PFs) and vegetation indices (VIs). Different models based on feature combination for evaluating LNC in wheat were constructed: partial least squares regression (PLS), gradient boosting decision tree (GBDT), and support vector regression (SVR). The results indicate that the model based on the fusion feature from near-ground hyperspectral imagery has good estimation effect. In particular, the estimation accuracy of the GBDT model is the best (R2 = 0.975 for calibration set, R2 = 0.861 for validation set). These findings demonstrate that the approach proposed in this study improved the estimation performance of LNC in wheat, which could provide technical support in wheat growth monitoring.
31
Lee, Jaewoo, and Daniel Kifer. "Scaling up Differentially Private Deep Learning with Fast Per-Example Gradient Clipping." Proceedings on Privacy Enhancing Technologies 2021, no. 1 (January 1, 2021): 128–44. http://dx.doi.org/10.2478/popets-2021-0008.
Full textAbstract:
AbstractRecent work on Renyi Differential Privacy has shown the feasibility of applying differential privacy to deep learning tasks. Despite their promise, however, differentially private deep networks often lag far behind their non-private counterparts in accuracy, showing the need for more research in model architectures, optimizers, etc. One of the barriers to this expanded research is the training time — often orders of magnitude larger than training non-private networks. The reason for this slowdown is a crucial privacy-related step called “per-example gradient clipping” whose naive implementation undoes the benefits of batch training with GPUs. By analyzing the back-propagation equations we derive new methods for per-example gradient clipping that are compatible with auto-differeniation (e.g., in Py-Torch and TensorFlow) and provide better GPU utilization. Our implementation in PyTorch showed significant training speed-ups (by factors of 54x - 94x for training various models with batch sizes of 128). These techniques work for a variety of architectural choices including convolutional layers, recurrent networks, attention, residual blocks, etc.
32
Al-Tameemi, Maad Issa, Ammar A. Hasan, and Bashra Kadhim Oleiwi. "Design and implementation monitoring robotic system based on you only look once model using deep learning technique." IAES International Journal of Artificial Intelligence (IJ-AI) 12, no. 1 (March 1, 2023): 106. http://dx.doi.org/10.11591/ijai.v12.i1.pp106-113.
Full textAbstract:
<span lang="EN-US">The need for robotics systems has become an urgent necessity in various fields, especially in video surveillance and live broadcasting systems. The main goal of this work is to design and implement a rover robotic monitoring system based on raspberry pi 4 model B to control this overall system and display a live video by using a webcam (USB camera) as well as using you only look once algorithm-version five (YOLOv5) to detect, recognize and display objects in real-time. This deep learning algorithm is highly accurate and fast and is implemented by Python, OpenCV, PyTorch codes and the Context Object Detection Task (COCO) 2020 dataset. This robot can move in all directions and in different places especially in undesirable places to transmit live video with a moving camera and process it by the YOLOv5 model. Also, the robot system can receive images, videos, or YouTube links and process them with YOLOv5. Raspberry Pi is controlled remotely by connecting to the network through Wi-Fi locally or publicly using the internet with a remote desktop connection application. The results were very satisfactory and proved the high-performance efficiency of the robot.</span>
33
Naik, Shivam, Khurram Azeem Hashmi, Alain Pagani, Marcus Liwicki, Didier Stricker, and Muhammad Zeshan Afzal. "Investigating Attention Mechanism for Page Object Detection in Document Images." Applied Sciences 12, no. 15 (July 26, 2022): 7486. http://dx.doi.org/10.3390/app12157486.
Full textAbstract:
Page object detection in scanned document images is a complex task due to varying document layouts and diverse page objects. In the past, traditional methods such as Optical Character Recognition (OCR)-based techniques have been employed to extract textual information. However, these methods fail to comprehend complex page objects such as tables and figures. This paper addresses the localization problem and classification of graphical objects that visually summarize vital information in documents. Furthermore, this work examines the benefit of incorporating attention mechanisms in different object detection networks to perform page object detection on scanned document images. The model is designed with a Pytorch-based framework called Detectron2. The proposed pipelines can be optimized end-to-end and exhaustively evaluated on publicly available datasets such as DocBank, PublayNet, and IIIT-AR-13K. The achieved results reflect the effectiveness of incorporating the attention mechanism for page object detection in documents.
34
Rampášek, Ladislav, Daniel Hidru, Petr Smirnov, Benjamin Haibe-Kains, and Anna Goldenberg. "Dr.VAE: improving drug response prediction via modeling of drug perturbation effects." Bioinformatics 35, no. 19 (March 8, 2019): 3743–51. http://dx.doi.org/10.1093/bioinformatics/btz158.
Full textAbstract:
Abstract Motivation Individualized drug response prediction is a fundamental part of personalized medicine for cancer. Great effort has been made to discover biomarkers or to develop machine learning methods for accurate drug response prediction in cancers. Incorporating prior knowledge of biological systems into these methods is a promising avenue to improve prediction performance. High-throughput cell line assays of drug-induced transcriptomic perturbation effects are a prior knowledge that has not been fully incorporated into a drug response prediction model yet. Results We introduce a unified probabilistic approach, Drug Response Variational Autoencoder (Dr.VAE), that simultaneously models both drug response in terms of viability and transcriptomic perturbations. Dr.VAE is a deep generative model based on variational autoencoders. Our experimental results showed Dr.VAE to do as well or outperform standard classification methods for 23 out of 26 tested Food and Drug Administration-approved drugs. In a series of ablation experiments we showed that the observed improvement of Dr.VAE can be credited to the incorporation of drug-induced perturbation effects with joint modeling of treatment sensitivity. Availability and implementation Processed data and software implementation using PyTorch (Paszke et al., 2017) are available at: https://github.com/rampasek/DrVAE. Supplementary information Supplementary data are available at Bioinformatics online.
35
Hou, Liang, Chunhua Chen, Shaojie Wang, Yongjun Wu, and Xiu Chen. "Multi-Object Detection Method in Construction Machinery Swarm Operations Based on the Improved YOLOv4 Model." Sensors 22, no. 19 (September 26, 2022): 7294. http://dx.doi.org/10.3390/s22197294.
Full textAbstract:
To handle the problem of low detection accuracy and missed detection caused by dense detection objects, overlapping, and occlusions in the scenario of complex construction machinery swarm operations, this paper proposes a multi-object detection method based on the improved YOLOv4 model. Firstly, the K-means algorithm is used to initialize the anchor boxes to improve the learning efficiency of the depth features of construction machinery objects. Then, the pooling operation is replaced with dilated convolution to solve the problem that the pooling layer reduces the resolution of feature maps and causes a high missed detection rate. Finally, focus loss is introduced to optimize the loss function of YOLOv4 to improve the imbalance of positive and negative samples during the model training process. To verify the effectiveness of the above optimizations, the proposed method is verified on the Pytorch platform with a self-build dataset. The experimental results show that the mean average precision(mAP) of the improved YOLOv4 model for multi-object detection of construction machinery can reach 97.03%, which is 2.16% higher than that of the original YOLOv4 detection network. Meanwhile, the detection speed is 31.11 fps, and it is reduced by only 0.59 fps, still meeting the real-time requirements. The research lays a foundation for environment perception of construction machinery swarm operations and promotes the unmanned and intelligent development of construction machinery swarm operations.
36
Rongrong, Shan, Ma Zhenyu, Ye Hong, Lin Zhenxing, Qiu Gongming, Ge Chengyu, Lu Yang, and Yu Kun. "Fault Diagnosis Method of Distribution Equipment Based on Hybrid Model of Robot and Deep Learning." Journal of Robotics 2022 (April 14, 2022): 1–11. http://dx.doi.org/10.1155/2022/9742815.
Full textAbstract:
In view of the poor effect of most fault diagnosis methods on the intelligent recognition of equipment images, a fault diagnosis method of distribution equipment based on the hybrid model of robot and deep learning is proposed to reduce the dependence on manpower and realize efficient intelligent diagnosis. Firstly, the robot is used to collect the on-site state images of distribution equipment to build the image information database of distribution equipment. At the same time, the robot background is used as the comprehensive database data analysis platform to optimize the sample quality of the database. Then, the massive infrared images are segmented based on chroma saturation brightness space to distinguish the defective equipment images, and the defective equipment areas are extracted from the images by OTSU method. Finally, the residual network is used to improve the region-based fully convolutional networks (R-FCN) algorithm, and the improved R-FCN algorithm trained by the online hard example mining method is used for fault feature learning. The fault type, grade, and location of distribution equipment are obtained through fault criterion analysis. The experimental analysis of the proposed method based on PyTorch platform shows that the fault diagnosis time and accuracy are about 5.5 s and 92.06%, respectively, which are better than other comparison methods and provide a certain theoretical basis for the automatic diagnosis of power grid equipment.
37
Ari, Tugba, Hande Sağlam, Hasan Öksüzoğlu, Orhan Kazan, İbrahim Şevki Bayrakdar, Suayip Burak Duman, Özer Çelik, et al. "Automatic Feature Segmentation in Dental Periapical Radiographs." Diagnostics 12, no. 12 (December 7, 2022): 3081. http://dx.doi.org/10.3390/diagnostics12123081.
Full textAbstract:
While a large number of archived digital images make it easy for radiology to provide data for Artificial Intelligence (AI) evaluation; AI algorithms are more and more applied in detecting diseases. The aim of the study is to perform a diagnostic evaluation on periapical radiographs with an AI model based on Convoluted Neural Networks (CNNs). The dataset includes 1169 adult periapical radiographs, which were labelled in CranioCatch annotation software. Deep learning was performed using the U-Net model implemented with the PyTorch library. The AI models based on deep learning models improved the success rate of carious lesion, crown, dental pulp, dental filling, periapical lesion, and root canal filling segmentation in periapical images. Sensitivity, precision and F1 scores for carious lesion were 0.82, 0.82, and 0.82, respectively; sensitivity, precision and F1 score for crown were 1, 1, and 1, respectively; sensitivity, precision and F1 score for dental pulp, were 0.97, 0.87 and 0.92, respectively; sensitivity, precision and F1 score for filling were 0.95, 0.95, and 0.95, respectively; sensitivity, precision and F1 score for the periapical lesion were 0.92, 0.85, and 0.88, respectively; sensitivity, precision and F1 score for root canal filling, were found to be 1, 0.96, and 0.98, respectively. The success of AI algorithms in evaluating periapical radiographs is encouraging and promising for their use in routine clinical processes as a clinical decision support system.
38
Shamsutdinova, T. M. "Problems and Prospects for the Application of Neural Networks for the Sphere of Education." Open Education 26, no. 6 (November 23, 2022): 4–10. http://dx.doi.org/10.21686/1818-4243-2022-6-4-10.
Full textAbstract:
The purpose of the study is to examine the theory and practice of using neural networks in education, to develop the concept of a neural network adaptive learning environment, and to implement a neural network model of one of the subsystems of this environment (using a model example of creating an adaptive educational trajectory).Materials and methods. The study includes a review of bibliographic sources on the application of neural networks in the field of education. It also includes modeling the structure of a neural network adaptive learning environment. The PyTorch library was used to programmatically implement the neural network.Results. The prospects for the use of neural networks in the field of education are considered, including various tasks of recognition, diagnostics, classification, clustering, forecasting, optimization, etc. A structural model of an adaptive learning NeuroSmart environment is created. This environment includes a number of subsystems for solving the following tasks: biometric identification of the student; determining starting level; choosing an adaptive learning path; determining the current level of competency development; automated verification of students’ work using recognition technology; analysis of the final result of training; monitoring information security incidents in an electronic course, etc. In order to study the possibilities and problems of applying neural network models to the tasks of constructing student adaptive learning trajectories, a model example of a neural network was created. This example illustrates the possibility of using a neural network to select further nodes of the educational trajectory based on the available data on the current learning parameters in an electronic course. To implement the neural network, the PyTorch deep learning library and Pandas library modules were used. SGD, Adam, Rprop were used as an optimizer to perform gradient descent steps. For each of the optimizers, a computer study of the stability of the neural network was carried out by varying the following parameters: the learning rate coefficient, the number of neurons in the hidden layer, and the number of training epochs.Conclusion. It can be assumed that the next stage in the evolution of the use of neural network technologies in the field of education will be their integration into complex multi-component Smart systems capable of automatically supporting student learning at all stages of the implementation of their personal educational trajectory. Obviously, the practical implementation of complex neural network systems of this level is a very difficult task and can be solved so far only at the level of individual subsystems. There are a number of problems associated with computer simulation of the educational environment based on neural network models: the question of the optimal structure of the neural network (for example, the number of neurons and layers in the network) has not been studied enough, there are no clear criteria for the optimality of the adaptive educational trajectory. Nevertheless, it should be noted that the task of developing new forms and technologies of personalized e-learning is in great demand, which makes modeling based on neural networks especially relevant.
39
Gully-Santiago, Michael, and Caroline V. Morley. "An Interpretable Machine-learning Framework for Modeling High-resolution Spectroscopic Data*." Astrophysical Journal 941, no. 2 (December 1, 2022): 200. http://dx.doi.org/10.3847/1538-4357/aca0a2.
Full textAbstract:
Abstract Comparison of échelle spectra to synthetic models has become a computational statistics challenge, with over 10,000 individual spectral lines affecting a typical cool star échelle spectrum. Telluric artifacts, imperfect line lists, inexact continuum placement, and inflexible models frustrate the scientific promise of these information-rich data sets. Here we debut an interpretable machine-learning framework blasé that addresses these and other challenges. The semiempirical approach can be viewed as “transfer learning”—first pretraining models on noise-free precomputed synthetic spectral models, then learning the corrections to line depths and widths from whole-spectrum fitting to an observed spectrum. The auto-differentiable model employs back-propagation, the fundamental algorithm empowering modern deep learning and neural networks. Here, however, the 40,000+ parameters symbolize physically interpretable line profile properties such as amplitude, width, location, and shape, plus radial velocity and rotational broadening. This hybrid data-/model-driven framework allows joint modeling of stellar and telluric lines simultaneously, a potentially transformative step forward for mitigating the deleterious telluric contamination in the near-infrared. The blasé approach acts as both a deconvolution tool and semiempirical model. The general-purpose scaffolding may be extensible to many scientific applications, including precision radial velocities, Doppler imaging, chemical abundances for Galactic archeology, line veiling, magnetic fields, and remote sensing. Its sparse-matrix architecture and GPU acceleration make blasé fast. The open-source PyTorch-based code blase includes tutorials, Application Programming Interface documentation, and more. We show how the tool fits into the existing Python spectroscopy ecosystem, demonstrate a range of astrophysical applications, and discuss limitations and future extensions.
40
Hitchcock, James A., Markus Hundertmark, Daniel Foreman-Mackey, Etienne Bachelet, Martin Dominik, Rachel Street, and Yiannis Tsapras. "PyTorchDIA: a flexible, GPU-accelerated numerical approach to Difference Image Analysis." Monthly Notices of the Royal Astronomical Society 504, no. 3 (April 21, 2021): 3561–79. http://dx.doi.org/10.1093/mnras/stab1114.
Full textAbstract:
ABSTRACT We present a GPU-accelerated numerical approach for fast kernel and differential background solutions. The model image proposed in the Bramich Difference Image Analysis (DIA) algorithm is analogous to a very simple convolutional neural network (CNN), with a single convolutional filter (i.e. the kernel) and an added scalar bias (i.e. the differential background). Here, we do not solve for the discrete pixel array in the classical, analytical linear least-squares sense. Instead, by making use of PyTorch tensors (GPU compatible multidimensional matrices) and associated deep learning tools, we solve for the kernel via an inherently massively parallel optimization. By casting the DIA problem as a GPU-accelerated optimization that utilizes automatic differentiation tools, our algorithm is both flexible to the choice of scalar objective function, and can perform DIA on astronomical data sets at least an order of magnitude faster than its classical analogue. More generally, we demonstrate that tools developed for machine learning can be used to address generic data analysis and modelling problems.
41
Gao, Mingyu, Fei Wang, Peng Song, Junyan Liu, and DaWei Qi. "BLNN: Multiscale Feature Fusion-Based Bilinear Fine-Grained Convolutional Neural Network for Image Classification of Wood Knot Defects." Journal of Sensors 2021 (August 17, 2021): 1–18. http://dx.doi.org/10.1155/2021/8109496.
Full textAbstract:
Wood defects are quickly identified from an optical image based on deep learning methodology, which effectively improves the wood utilization. The traditional neural network technique is unemployed for the wood defect detection of optical image used, which results from a long training time, low recognition accuracy, and nonautomatic extraction of defect image features. In this paper, a wood knot defect detection model (so-called BLNN) combined deep learning is reported. Two subnetworks composed of convolutional neural networks are trained by Pytorch. By using the feature extraction capabilities of the two subnetworks and combining the bilinear join operation, the fine-grained features of the image are obtained. The experimental results show that the accuracy has reached up 99.20%, and the training time is obviously reduced with the speed of defect detection about 0.0795 s/image. It indicates that BLNN has the ability to improve the accuracy of defect recognition and has a potential application in the detection of wood knot defects.
42
Wu, Hao, and Zhi Zhou. "Using Convolution Neural Network for Defective Image Classification of Industrial Components." Mobile Information Systems 2021 (September 11, 2021): 1–8. http://dx.doi.org/10.1155/2021/9092589.
Full textAbstract:
Computer vision provides effective solutions in many imaging relation problems, including automatic image segmentation and classification. Artificially trained models can be employed to tag images and identify objects spontaneously. In large-scale manufacturing, industrial cameras are utilized to take constant images of components for several reasons. Due to the limitations caused by motion, lens distortion, and noise, some defective images are captured, which are to be identified and separated. One common way to address this problem is by looking into these images manually. However, this solution is not only very time-consuming but is also inaccurate. The paper proposes a deep learning-based artificially intelligent system that can quickly train and identify faulty images. For this purpose, a pretrained convolution neural network based on the PyTorch framework is employed to extract discriminating features from the dataset, which is then used for the classification task. In order to eliminate the chances of overfitting, the proposed model also employed Dropout technology to adjust the network. The experimental study reveals that the system can precisely classify the normal and defective images with an accuracy of over 91%.
43
Shu, Juan, Yu Li, Sheng Wang, Bowei Xi, and Jianzhu Ma. "Disease gene prediction with privileged information and heteroscedastic dropout." Bioinformatics 37, Supplement_1 (July 1, 2021): i410—i417. http://dx.doi.org/10.1093/bioinformatics/btab310.
Full textAbstract:
Abstract Motivation Recently, machine learning models have achieved tremendous success in prioritizing candidate genes for genetic diseases. These models are able to accurately quantify the similarity among disease and genes based on the intuition that similar genes are more likely to be associated with similar diseases. However, the genetic features these methods rely on are often hard to collect due to high experimental cost and various other technical limitations. Existing solutions of this problem significantly increase the risk of overfitting and decrease the generalizability of the models. Results In this work, we propose a graph neural network (GNN) version of the Learning under Privileged Information paradigm to predict new disease gene associations. Unlike previous gene prioritization approaches, our model does not require the genetic features to be the same at training and test stages. If a genetic feature is hard to measure and therefore missing at the test stage, our model could still efficiently incorporate its information during the training process. To implement this, we develop a Heteroscedastic Gaussian Dropout algorithm, where the dropout probability of the GNN model is determined by another GNN model with a mirrored GNN architecture. To evaluate our method, we compared our method with four state-of-the-art methods on the Online Mendelian Inheritance in Man dataset to prioritize candidate disease genes. Extensive evaluations show that our model could improve the prediction accuracy when all the features are available compared to other methods. More importantly, our model could make very accurate predictions when >90% of the features are missing at the test stage. Availability and implementation Our method is realized with Python 3.7 and Pytorch 1.5.0 and method and data are freely available at: https://github.com/juanshu30/Disease-Gene-Prioritization-with-Privileged-Information-and-Heteroscedastic-Dropout.
44
Yan, Shipeng, Songyang Zhang, and Xuming He. "A Dual Attention Network with Semantic Embedding for Few-Shot Learning." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 9079–86. http://dx.doi.org/10.1609/aaai.v33i01.33019079.
Full textAbstract:
Despite recent success of deep neural networks, it remains challenging to efficiently learn new visual concepts from limited training data. To address this problem, a prevailing strategy is to build a meta-learner that learns prior knowledge on learning from a small set of annotated data. However, most of existing meta-learning approaches rely on a global representation of images and a meta-learner with complex model structures, which are sensitive to background clutter and difficult to interpret. We propose a novel meta-learning method for few-shot classification based on two simple attention mechanisms: one is a spatial attention to localize relevant object regions and the other is a task attention to select similar training data for label prediction. We implement our method via a dual-attention network and design a semantic-aware meta-learning loss to train the meta-learner network in an end-to-end manner. We validate our model on three few-shot image classification datasets with extensive ablative study, and our approach shows competitive performances over these datasets with fewer parameters. For facilitating the future research, code and data split are available: https://github.com/tonysy/STANet-PyTorch
45
Rhinehart, Tessa, Samuel Lapp, and Justin Kitzes. "Identifying and building on the current state of bioacoustics software." Journal of the Acoustical Society of America 151, no. 4 (April 2022): A27. http://dx.doi.org/10.1121/10.0010544.
Full textAbstract:
Bioacoustics is a powerful and increasingly commonly used tool for terrestrial and marine biological assessments. As the scale of bioacoustic data collection has increased, techniques for processing these data have diversified. However, with analysis methods rapidly evolving and dozens of analysis software packages already available, it is challenging to identify which software, if any, meets a particular researcher’s needs. We reviewed bioacoustics software to identify packages aimed at or used by bioacoustics researchers in ecology. We compiled descriptions of the function of 65 stable or actively developed software packages used for bioacoustics analyses. Of these, 59 were free or open-source packages. In addition, we developed free, open-source Python software, OpenSoundscape, that addresses gaps in available software. OpenSoundscape simplifies the process of creating flexible, scalable deep learning algorithms for bioacoustic analysis. It can be used to train binary or multiclass convolutional neural networks with any PyTorch-implemented model structure (e.g., ResNet50, Inception v3). Researchers can easily customize its spectrogram preprocessing and data augmentation routines to improve model performance. OpenSoundscape also includes modules to work with annotated acoustic data, apply additional signal processing algorithms, perform acoustic localization, and “open the black box” of deep learning using Grad-CAM.
46
Zhu, Anfu, Shuaihao Chen, Fangfang Lu, Congxiao Ma, and Fengrui Zhang. "Recognition Method of Tunnel Lining Defects Based on Deep Learning." Wireless Communications and Mobile Computing 2021 (September 30, 2021): 1–12. http://dx.doi.org/10.1155/2021/9070182.
Full textAbstract:
The defect identification of tunnel lining is a task with a lot of tasks and time-consuming work, and currently, it mainly relies on manual operation. This paper takes the ground-penetrating radar image of the internal defects of the lining as the research object, and chooses the popular VGG16, ResNet34 convolutional neural network (CNN) to build the automatic recognition model for comparative study, and proposes an improved ResNet34 defect-recognition model. In this paper, SGD and Adam training algorithms are used to update network parameters, and the PyTorch depth framework is used to train the network. The test results show that the ResNet34 network has faster convergence speed, higher accuracy rate, and shorter training time than the VGG16 network. The ResNet34 network using the Adam algorithm can achieve 99.08% accuracy. The improved ResNet34 network can achieve an accuracy of 99.25%, and at the same, reduce the parameter amount by 4.22% compared with the ResNet34 network, which can better identify defects in the lining. The research in this paper shows that the deep learning method can provide new ideas for the identification of tunnel lining defects.
47
Keser, Gaye, Ibrahim Sevki Bayrakdar, Filiz Namdar Pekiner, Özer Çelik, and Kaan Orhan. "A deep learning approach for masseter muscle segmentation on ultrasonography." Journal of Ultrasonography 22, no. 91 (October 1, 2022): 204–8. http://dx.doi.org/10.15557/jou.2022.0034.
Full textAbstract:
Aim: Deep learning algorithms have lately been used for medical image processing, and they have showed promise in a range of applications. The purpose of this study was to develop and test computer-based diagnostic tools for evaluating masseter muscle segmentation on ultrasonography images. Materials and methods: A total of 388 anonymous adult masseter muscle retrospective ultrasonographic images were evaluated. The masseter muscle was labeled on ultrasonography images using the polygonal type labeling method with the CranioCatch labeling program (CranioCatch, Eskişehir, Turkey). All images were re-checked and verified by Oral and Maxillofacial Radiology experts. This data set was divided into training (n = 312), verification (n = 38) and test (n = 38) sets. In the study, an artificial intelligence model was developed using PyTorch U-Net architecture, which is a deep learning approach. Results: In our study, the artificial intelligence deep learning model known as U-net provided the detection and segmentation of all test images, and when the success rate in the estimation of the images was evaluated, the F1, sensitivity and precision results of the model were 1.0, 1.0 and 1.0, respectively. Conclusion: Artificial intelligence shows promise in automatic segmentation of masseter muscle on ultrasonography images. This strategy can aid surgeons, radiologists, and other medical practitioners in reducing diagnostic time.
48
Gokhale, Angelina, Mandaar B. Pande, and Dhanya Pramod. "Implementation of a quantum transfer learning approach to image splicing detection." International Journal of Quantum Information 18, no. 05 (August 2020): 2050024. http://dx.doi.org/10.1142/s0219749920500240.
Full textAbstract:
In this paper, we present an implementation of quantum transfer learning to blind and passive detection of image splicing forgeries. Though deep learning models are becoming increasingly popular for various computer vision use cases, they depend on powerful classical machines and GPUs for dealing with complex problem solving and also to reduce the time taken for computation. The quantum computing research community has demonstrated elegant solutions to complex use cases in deep learning and computer vision for reducing storage space and increasing the accuracy of results compared to those obtained on a classical computer. We extend the quantum transfer learning approach formerly applied to image classification, for solving the growing problem of image manipulation, specifically, image splicing detection. A hybrid model is built using the ResNet50 pre-trained classical deep learning network and a quantum variational circuit to classify spliced versus authentic images. We present a comparative empirical study of classical versus quantum transfer learning approach using Xanadu’s pennylane quantum simulator and the pytorch deep learning framework. The model was also evaluated on the actual quantum processor ibmqx2 provided by IBM. Results obtained by execution on the quantum processor ([Formula: see text]%, [Formula: see text]%) and simulator ([Formula: see text]%, [Formula: see text]%) showed improvements in comparison to those obtained from classical computers ([Formula: see text]%, [Formula: see text]%).
49
Jakab, Balázs, Boudewijn van Leeuwen, and Zalán Tobak. "Detection of Plastic Greenhouses Using High Resolution Rgb Remote Sensing Data and Convolutional Neural Network." Journal of Environmental Geography 14, no. 1-2 (April 1, 2021): 38–46. http://dx.doi.org/10.2478/jengeo-2021-0004.
Full textAbstract:
Abstract Agricultural production in greenhouses shows a rapid growth in many parts of the world. This form of intensive farming requires a large amount of water and fertilizers, and can have a severe impact on the environment. The number of greenhouses and their location is important for applications like spatial planning, environmental protection, agricultural statistics and taxation. Therefore, with this study we aim to develop a methodology to detect plastic greenhouses in remote sensing data using machine learning algorithms. This research presents the results of the use of a convolutional neural network for automatic object detection of plastic greenhouses in high resolution remotely sensed data within a GIS environment with a graphical interface to advanced algorithms. The convolutional neural network is trained with manually digitized greenhouses and RGB images downloaded from Google Earth. The ArcGIS Pro geographic information system provides access to many of the most advanced python-based machine learning environments like Keras – TensorFlow, PyTorch, fastai and Scikit-learn. These libraries can be accessed via a graphical interface within the GIS environment. Our research evaluated the results of training and inference of three different convolutional neural networks. Experiments were executed with many settings for the backbone models and hyperparameters. The performance of the three models in terms of detection accuracy and time required for training was compared. The model based on the VGG_11 backbone model (with dropout) resulted in an average accuracy of 79.2% with a relatively short training time of 90 minutes, the much more complex DenseNet121 model was trained in 16.5 hours and showed a result of 79.1%, while the ResNet18 based model showed an average accuracy of 83.1% with a training time of 3.5 hours.
50
Chung, Yang-Hoon, Young-Seob Jeong, Gati Lother Martin, Min Seo Choi, You Jin Kang, Misoon Lee, Ana Cho, Bon Sung Koo, Sung Hwan Cho, and Sang Hyun Kim. "Prediction of blood pressure changes associated with abdominal pressure changes during robotic laparoscopic low abdominal surgery using deep learning." PLOS ONE 17, no. 6 (June 6, 2022): e0269468. http://dx.doi.org/10.1371/journal.pone.0269468.
Full textAbstract:
Background Intraoperative hypertension and blood pressure (BP) fluctuation are known to be associated with negative patient outcomes. During robotic lower abdominal surgery, the patient’s abdominal cavity is filled with CO2, and the patient’s head is steeply positioned toward the floor (Trendelenburg position). Pneumoperitoneum and the Trendelenburg position together with physiological alterations during anesthesia, interfere with predicting BP changes. Recently, deep learning using recurrent neural networks (RNN) was shown to be effective in predicting intraoperative BP. A model for predicting BP rise was designed using RNN under special scenarios during robotic laparoscopic surgery and its accuracy was tested. Methods Databases that included adult patients (over 19 years old) undergoing low abdominal da Vinci robotic surgery (ovarian cystectomy, hysterectomy, myomectomy, prostatectomy, and salpingo-oophorectomy) at Soonchunhyang University Bucheon Hospital from October 2018 to March 2021 were used. An RNN-based model was designed using Python3 language with the PyTorch packages. The model was trained to predict whether hypertension (20% increase in the mean BP from baseline) would develop within 10 minutes after pneumoperitoneum. Results Eight distinct datasets were generated and the predictive power was compared. The macro-average F1 scores of the datasets ranged from 68.18% to 72.33%. It took only 3.472 milliseconds to obtain 39 prediction outputs. Conclusions A prediction model using the RNN may predict BP rises during robotic laparoscopic surgery.