Dissertations / Theses on the topic 'Binary neural networks (BNN)'
To see the other types of publications on this topic, follow the link: Binary neural networks (BNN).
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 39 dissertations / theses for your research on the topic 'Binary neural networks (BNN).'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse dissertations / theses on a wide variety of disciplines and organise your bibliography correctly.
1
Simons, Taylor Scott. "High-Speed Image Classification for Resource-Limited Systems Using Binary Values." BYU ScholarsArchive, 2021. https://scholarsarchive.byu.edu/etd/9097.
Full textAbstract:
Image classification is a memory- and compute-intensive task. It is difficult to implement high-speed image classification algorithms on resource-limited systems like FPGAs and embedded computers. Most image classification algorithms require many fixed- and/or floating-point operations and values. In this work, we explore the use of binary values to reduce the memory and compute requirements of image classification algorithms. Our objective was to implement these algorithms on resource-limited systems while maintaining comparable accuracy and high speeds. By implementing high-speed image classification algorithms on resource-limited systems like embedded computers, FPGAs, and ASICs, automated visual inspection can be performed on small low-powered systems. Industries like manufacturing, medicine, and agriculture can benefit from compact, high-speed, low-power visual inspection systems. Tasks like defect detection in manufactured products and quality sorting of harvested produce can be performed cheaper and more quickly. In this work, we present ECO Jet Features, an algorithm adapted to use binary values for visual inspection. The ECO Jet Features algorithm ran 3.7x faster than the original ECO Features algorithm on embedded computers. It also allowed the algorithm to be implemented on an FPGA, achieving 78x speedup over full-sized desktop systems, using a fraction of the power and space. We reviewed Binarized Neural Nets (BNNs), neural networks that use binary values for weights and activations. These networks are particularly well suited for FPGA implementation and we compared and contrasted various FPGA implementations found throughout the literature. Finally, we combined the deep learning methods used in BNNs with the efficiency of Jet Features to make Neural Jet Features. Neural Jet Features are binarized convolutional layers that are learned through deep learning and learn classic computer vision kernels like the Gaussian and Sobel kernels. These kernels are efficiently computed as a group and their outputs can be reused when forming output channels. They performed just as well as BNN convolutions on visual inspection tasks and are more stable when trained on small models.
2
Braga, AntoÌ‚nio de PaÌdua. "Design models for recursive binary neural networks." Thesis, Imperial College London, 1995. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.336442.
Full text
3
Redkar, Shrutika. "Deep Learning Binary Neural Network on an FPGA." Digital WPI, 2017. https://digitalcommons.wpi.edu/etd-theses/407.
Full textAbstract:
In recent years, deep neural networks have attracted lots of attentions in the field of computer vision and artificial intelligence. Convolutional neural network exploits spatial correlations in an input image by performing convolution operations in local receptive fields. When compared with fully connected neural networks, convolutional neural networks have fewer weights and are faster to train. Many research works have been conducted to further reduce computational complexity and memory requirements of convolutional neural networks, to make it applicable to low-power embedded applications. This thesis focuses on a special class of convolutional neural network with only binary weights and activations, referred as binary neural networks. Weights and activations for convolutional and fully connected layers are binarized to take only two values, +1 and -1. Therefore, the computations and memory requirement have been reduced significantly. The proposed architecture of binary neural networks has been implemented on an FPGA as a real time, high speed, low power computer vision platform. Only on-chip memories are utilized in the FPGA design. The FPGA implementation is evaluated using the CIFAR-10 benchmark and achieved a processing speed of 332,164 images per second for CIFAR-10 dataset with classification accuracy of about 86.06%.
4
Ezzadeen, Mona. "Conception d'un circuit dédié au calcul dans la mémoire à base de technologie 3D innovante." Electronic Thesis or Diss., Aix-Marseille, 2022. http://theses.univ-amu.fr.lama.univ-amu.fr/221212_EZZADEEN_955e754k888gvxorp699jljcho_TH.pdf.
Full textAbstract:
Avec le développement de l'internet des objets et de l'intelligence artificielle, le "déluge de données" est une réalité, poussant au développement de systèmes de calcul efficaces énergétiquement. Dans ce contexte, en effectuant le calcul directement à l'intérieur ou à proximité des mémoires, le paradigme de l'in/near-memory-computing (I/NMC) semble être une voie prometteuse. En effet, les transferts de données entre les mémoires et les unités de calcul sont très énergivores. Cependant, les classiques mémoires Flash souffrent de problèmes de miniaturisation et ne semblent pas facilement adaptées à l'I/NMC. Ceci n'est pas le cas de nouvelles technologies mémoires émergentes comme les ReRAM. Ces dernières souffrent cependant d'une variabilité importante, et nécessitent l'utilisation d'un transistor d'accès par bit (1T1R) pour limiter les courants de fuite, dégradant ainsi leur densité. Dans cette thèse, nous nous proposons de résoudre ces deux défis. Tout d'abord, l'impact de la variabilité des ReRAM sur les opérations de lecture et de calcul en mémoire est étudié, et de nouvelles techniques de calculs booléens robustes et à faible impact surfacique sont développées. Dans le contexte des réseaux de neurones, de nouveaux accélérateurs neuromorphiques à base de ReRAM sont proposés et caractérisés, visant une bonne robustesse face à la variabilité, un bon parallélisme et une efficacité énergétique élevée. Dans un deuxième temps, pour résoudre les problèmes de densité d'intégration, une nouvelle technologie de cube mémoire 3D à base de ReRAM 1T1R est proposée, pouvant à la fois être utilisée en tant que mémoire de type NOR 3D dense qu'en tant qu'accélérateur pour l'I/NMCWith the advent of edge devices and artificial intelligence, the data deluge is a reality, making energy-efficient computing systems a must-have. Unfortunately, classical von Neumann architectures suffer from the high cost of data transfers between memories and processing units. At the same time, CMOS scaling seems more and more challenging and costly to afford, limiting the chips' performance due to power consumption issues.In this context, bringing the computation directly inside or near memories (I/NMC) seems an appealing solution. However, data-centric applications require an important amount of non-volatile storage, and modern Flash memories suffer from scaling issues and are not very suited for I/NMC. On the other hand, emerging memory technologies such as ReRAM present very appealing memory performances, good scalability, and interesting I/NMC features. However, they suffer from variability issues and from a degraded density integration if an access transistor per bitcell (1T1R) is used to limit the sneak-path currents. This thesis work aims to overcome these two challenges. First, the variability impact on read and I/NMC operations is assessed and new robust and low-overhead ReRAM-based boolean operations are proposed. In the context of neural networks, new ReRAM-based neuromorphic accelerators are developed and characterized, with an emphasis on good robustness against variability, good parallelism, and high energy efficiency. Second, to resolve the density integration issues, an ultra-dense 3D 1T1R ReRAM-based Cube and its architecture are proposed, which can be used as a 3D NOR memory as well as a low overhead and energy-efficient I/NMC accelerator
5
Kennedy, John V. "The design of a scalable and application independent platform for binary neural networks." Thesis, University of York, 1998. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.323503.
Full text
6
Li, Guo. "Neural network for optimization of binary computer-generated hologram with printing model /." Online version of thesis, 1995. http://hdl.handle.net/1850/12234.
Full text
7
Medvedieva, S. O., I. V. Bogach, V. A. Kovenko, С. О. Медведєва, І. В. Богач, and В. А. Ковенко. "Neural networks in Machine learning." Thesis, ВНТУ, 2019. http://ir.lib.vntu.edu.ua//handle/123456789/24788.
Full textAbstract:
В статті розглянуті основи роботи з нейронними мережами, особливу увагу приділено моделі мережі під назвою «перцептрон», запровадженої Френком Розенблаттом. До того ж було розкрито тему найпоширеніших мов програмування, що дозволяють втілити нейронні мережі у життя, шляхом створення програмного забезпечення, пов`язаного з ними.The paper covers the basic principles of Neural Networks’ work. Special attention is paid to Frank Rosenblatt’s model of the network called “perceptron”. In addition, the article touches upon the main programming languages used to write software for Neural Networks.
8
Wilson, Brittany Michelle. "Evaluating and Improving the SEU Reliability of Artificial Neural Networks Implemented in SRAM-Based FPGAs with TMR." BYU ScholarsArchive, 2020. https://scholarsarchive.byu.edu/etd/8619.
Full textAbstract:
Artificial neural networks (ANNs) are used in many types of computing applications. Traditionally, ANNs have been implemented in software, executing on CPUs and even GPUs, which capitalize on the parallelizable nature of ANNs. More recently, FPGAs have become a target platform for ANN implementations due to their relatively low cost, low power, and flexibility. Some safety-critical applications could benefit from ANNs, but these applications require a certain level of reliability. SRAM-based FPGAs are sensitive to single-event upsets (SEUs), which can lead to faults and errors in execution. However there are techniques that can mask such SEUs and thereby improve the overall design reliability. This thesis evaluates the SEU reliability of neural networks implemented in SRAM-based FPGAs and investigates mitigation techniques against upsets for two case studies. The first was based on the LeNet-5 convolutional neural network and was used to test an implementation with both fault injection and neutron radiation experiments, demonstrating that our fault injection experiments could accurately evaluate SEU reliability of the networks. SEU reliability was improved by selectively applying TMR to the most critical layers of the design, achieving a 35% improvement reliability at an increase in 6.6% resources. The second was an existing neural network called BNN-PYNQ. While the base design was more sensitive to upsets than the CNN previous tested, the TMR technique improved the reliability by approximately 7× in fault injection experiments.
9
Mealey, Thomas C. "Binary Recurrent Unit: Using FPGA Hardware to Accelerate Inference in Long Short-Term Memory Neural Networks." University of Dayton / OhioLINK, 2018. http://rave.ohiolink.edu/etdc/view?acc_num=dayton1524402925375566.
Full text
10
Strandberg, Rickard, and Johan Låås. "A comparison between Neural networks, Lasso regularized Logistic regression, and Gradient boosted trees in modeling binary sales." Thesis, KTH, Optimeringslära och systemteori, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-252556.
Full textAbstract:
The primary purpose of this thesis is to predict whether or not a customer will make a purchase from a specific item category. The historical data is provided by the Nordic online-based IT-retailer Dustin. The secondary purpose is to evaluate how well a fully connected feed forward neural network performs as compared to Lasso regularized logistic regression and gradient boosted trees (XGBoost) on this task. This thesis finds XGBoost to be superior to the two other methods in terms of prediction accuracy, as well as speed.Det primära syftet med denna uppsats är att förutsäga huruvida en kund kommer köpa en specifik produkt eller ej. Den historiska datan tillhandahålls av den Nordiska internet-baserade IT-försäljaren Dustin. Det sekundära syftet med uppsatsen är att evaluera hur väl ett djupt neuralt nätverk presterar jämfört med Lasso regulariserad logistisk regression och gradient boostade träd (GXBoost). Denna uppsats fann att XGBoost presterade bättre än de två andra metoderna i såväl träffsäkerhet, som i hastighet.
11
Holesovsky, Ondrej. "Compact ConvNets with Ternary Weights and Binary Activations." Thesis, KTH, Robotik, perception och lärande, RPL, 2017. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-216389.
Full textAbstract:
Compact architectures, ternary weights and binary activations are two methods suitable for making neural networks more efficient. We introduce a) a dithering binary activation which improves accuracy of ternary weight networks with binary activations by randomizing quantization error, and b) a method of implementing ternary weight networks with binary activations using binary operations. Despite these new approaches, training a compact SqueezeNet architecture with ternary weights and full precision activations on ImageNet degrades classification accuracy significantly more than when training a less compact architecture the same way. Therefore ternary weights in their current form cannot be called the best method for reducing network size. However, the effect of weight decay on ternary weight network training should be investigated more in order to have more certainty in this finding.Kompakta arkitekturer, ternära vikter och binära aktiveringar är två metoder som är lämpliga för att göra neurala nätverk effektivare. Vi introducerar a) en dithering binär aktivering som förbättrar noggrannheten av ternärviktsnätverk med binära aktiveringar genom randomisering av kvantiseringsfel, och b) en metod för genomförande ternärviktsnätverk med binära aktiveringar med användning av binära operationer. Trots dessa nya metoder, att träna en kompakt SqueezeNet-arkitektur med ternära vikter och fullprecisionaktiveringar på ImageNet försämrar klassificeringsnoggrannheten betydligt mer än om man tränar en mindre kompakt arkitektur på samma sätt. Därför kan ternära vikter i deras nuvarande form inte kallas bästa sättet att minska nätverksstorleken. Emellertid, effekten av weight decay på träning av ternärviktsnätverk bör undersökas mer för att få större säkerhet i detta resultat.
12
Bergtold, Jason Scott. "Advances in Applied Econometrics: Binary Discrete Choice Models, Artificial Neural Networks, and Asymmetries in the FAST Multistage Demand System." Diss., Virginia Tech, 2004. http://hdl.handle.net/10919/27266.
Full textAbstract:
The dissertation examines advancements in the methods and techniques used in the field of econometrics. These advancements include: (i) a re-examination of the underlying statistical foundations of statistical models with binary dependent variables. (ii) using feed-forward backpropagation artificial neural networks for modeling dichotomous choice processes, and (iii) the estimation of unconditional demand elasticities using the flexible multistage demand system with asymmetric partitions and fixed effects across time.
The first paper re-examines the underlying statistical foundations of statistical models with binary dependent variables using the probabilistic reduction approach. This re-examination leads to the development of the Bernoulli Regression Model, a family of statistical models arising from conditional Bernoulli distributions. The paper provides guidelines for specifying and estimating a Bernoulli Regression Model, as well as, methods for generating and simulating conditional binary choice processes. Finally, the Multinomial Regression Model is presented as a direct extension.
The second paper empirically compares the out-of-sample predictive capabilities of artificial neural networks to binary logit and probit models. To facilitate this comparison, the statistical foundations of dichotomous choice models and feed-forward backpropagation artificial neural networks (FFBANNs) are re-evaluated. Using contingent valuation survey data, the paper shows that FFBANNs provide an alternative to the binary logit and probit models with linear index functions. Direct comparisons between the models showed that the FFBANNs performed marginally better than the logit and probit models for a number of within-sample and out-of-sample performance measures, but in the majority of cases these differences were not statistically significant. In addition, guidelines for modeling contingent valuation survey data and techniques for estimating median WTP measures using FFBANNs are examined.
The third paper estimates a set of unconditional price and expenditure elasticities for 49 different processed food categories using scanner data and the flexible and symmetric translog (FAST) multistage demand system. Due to the use of panel data and the presence of heterogeneity across time, temporal fixed effects were incorporated into the model. Overall, estimated price elasticities are larger, in absolute terms, than previous estimates. The use of disaggregated product groupings, scanner data, and the estimation of unconditional elasticities likely accounts for these differences.Ph. D.
13
Sibanda, Wilbert. "Comparative study of neural networks and design of experiments to the classification of HIV status / Wilbert Sibanda." Thesis, North West University, 2013. http://hdl.handle.net/10394/13179.
Full textAbstract:
This research addresses the novel application of design of experiment, artificial neural net-works and logistic regression to study the effect of demographic characteristics on the risk of acquiring HIV infection among the antenatal clinic attendees in South Africa. The annual antenatal HIV survey is the only major national indicator for HIV prevalence in South Africa. This is a vital technique to understand the changes in the HIV epidemic over time. The annual antenatal clinic data contains the following demographic characteristics for each pregnant woman; age (herein called mother's age), partner's age (herein father's age), population group (race), level of education, gravidity (number of pregnancies), parity (number of children born), HIV and syphilis status. This project applied a screening design of experiment technique to rank the effects of individual demographic characteristics on the risk of acquiring an HIV infection. There are a various screening design techniques such as fractional or full factorial and Plackett-Burman designs. In this work, a two-level fractional factorial design was selected for the purposes of screening. In addition to screening designs, this project employed response surface methodologies (RSM) to estimate interaction and quadratic effects of demographic characteristics using a central composite face-centered and a Box-Behnken design. Furthermore, this research presents the novel application of multi-layer perceptron’s (MLP) neural networks to model the demographic characteristics of antenatal clinic attendees. A review report was produced to study the application of neural networks to modelling HIV/AIDS around the world. The latter report is important to enhance our understanding of the extent to which neural networks have been applied to study the HIV/AIDS pandemic. Finally, a binary logistic regression technique was employed to benchmark the results obtained by the design of experiments and neural networks methodologies. The two-level fractional factorial design demonstrated that HIV prevalence was highly sensitive to changes in the mother's age (15-55 years) and level of her education (Grades 0-13). The central composite face centered and Box-Behnken designs employed to study the individual and interaction effects of demographic characteristics on the spread of HIV in South Africa, demonstrated that HIV status of an antenatal clinic attendee was highly sensitive to changes in pregnant mother's age and her educational level. In addition, the interaction of the mother's age with other demographic characteristics was also found to be an important determinant of the risk of acquiring an HIV infection. Furthermore, the central composite face centered and Box-Behnken designs illustrated that, individual-ally the pregnant mother's parity and her partner's age had no marked effect on her HIV status. However, the pregnant woman’s parity and her male partner’s age did show marked effects on her HIV status in “two way interactions with other demographic characteristics”. The multilayer perceptron (MLP) sensitivity test also showed that the age of the pregnant woman had the greatest effect on the risk of acquiring an HIV infection, while her gravidity and syphilis status had the lowest effects. The outcome of the MLP modelling produced the same results obtained by the screening and response surface methodologies. The binary logistic regression technique was compared with a Box-Behnken design to further elucidate the differential effects of demographic characteristics on the risk of acquiring HIV amongst pregnant women. The two methodologies indicated that the age of the pregnant woman and her level of education had the most profound effects on her risk of acquiring an HIV infection. To facilitate the comparison of the performance of the classifiers used in this study, a receiver operating characteristics (ROC) curve was applied. Theoretically, an ROC analysis provides tools to select optimal models and to discard suboptimal ones independent from the cost context or the classification distribution. SAS Enterprise MinerTM was employed to develop the required receiver-of-characteristics (ROC) curves. To validate the results obtained by the above classification methodologies, a credit scoring add-on in SAS Enterprise MinerTM was used to build binary target scorecards comprised of HIV positive and negative datasets for probability determination. The process involved grouping variables using weights-of-evidence (WOE), prior to performing a logistic regression to produce predicted probabilities. The process of creating bins for the scorecard enables the study of the inherent relationship between demographic characteristics and an in-dividual’s HIV status. This technique increases the understanding of the risk ranking ability of the scorecard method, while offering an added advantage of being predictive.
14
Al-Shammaa, Mohammed. "Granular computing approach for intelligent classifier design." Thesis, Brunel University, 2016. http://bura.brunel.ac.uk/handle/2438/13686.
Full textAbstract:
Granular computing facilitates dealing with information by providing a theoretical framework to deal with information as granules at different levels of granularity (different levels of specificity/abstraction). It aims to provide an abstract explainable description of the data by forming granules that represent the features or the underlying structure of corresponding subsets of the data. In this thesis, a granular computing approach to the design of intelligent classification systems is proposed. The proposed approach is employed for different classification systems to investigate its efficiency. Fuzzy inference systems, neural networks, neuro-fuzzy systems and classifier ensembles are considered to evaluate the efficiency of the proposed approach. Each of the considered systems is designed using the proposed approach and classification performance is evaluated and compared to that of the standard system. The proposed approach is based on constructing information granules from data at multiple levels of granularity. The granulation process is performed using a modified fuzzy c-means algorithm that takes classification problem into account. Clustering is followed by a coarsening process that involves merging small clusters into large ones to form a lower granularity level. The resulted granules are used to build each of the considered binary classifiers in different settings and approaches. Granules produced by the proposed granulation method are used to build a fuzzy classifier for each granulation level or set of levels. The performance of the classifiers is evaluated using real life data sets and measured by two classification performance measures: accuracy and area under receiver operating characteristic curve. Experimental results show that fuzzy systems constructed using the proposed method achieved better classification performance. In addition, the proposed approach is used for the design of neural network classifiers. Resulted granules from one or more granulation levels are used to train the classifiers at different levels of specificity/abstraction. Using this approach, the classification problem is broken down into the modelling of classification rules represented by the information granules resulting in more interpretable system. Experimental results show that neural network classifiers trained using the proposed approach have better classification performance for most of the data sets. In a similar manner, the proposed approach is used for the training of neuro-fuzzy systems resulting in similar improvement in classification performance. Lastly, neural networks built using the proposed approach are used to construct a classifier ensemble. Information granules are used to generate and train the base classifiers. The final ensemble output is produced by a weighted sum combiner. Based on the experimental results, the proposed approach has improved the classification performance of the base classifiers for most of the data sets. Furthermore, a genetic algorithm is used to determine the combiner weights automatically.
15
Kindbom, Hannes. "LSTM vs Random Forest for Binary Classification of Insurance Related Text." Thesis, KTH, Matematisk statistik, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-252748.
Full textAbstract:
The field of natural language processing has received increased attention lately, but less focus is put on comparing models, which differ in complexity. This thesis compares Random Forest to LSTM, for the task of classifying a message as question or non-question. The comparison was done by training and optimizing the models on historic chat data from the Swedish insurance company Hedvig. Different types of word embedding were also tested, such as Word2vec and Bag of Words. The results demonstrated that LSTM achieved slightly higher scores than Random Forest, in terms of F1 and accuracy. The models’ performance were not significantly improved after optimization and it was also dependent on which corpus the models were trained on. An investigation of how a chatbot would affect Hedvig’s adoption rate was also conducted, mainly by reviewing previous studies about chatbots’ effects on user experience. The potential effects on the innovation’s five attributes, relative advantage, compatibility, complexity, trialability and observability were analyzed to answer the problem statement. The results showed that the adoption rate of Hedvig could be positively affected, by improving the first two attributes. The effects a chatbot would have on complexity, trialability and observability were however suggested to be negligible, if not negative.Det vetenskapliga området språkteknologi har fått ökad uppmärksamhet den senaste tiden, men mindre fokus riktas på att jämföra modeller som skiljer sig i komplexitet. Den här kandidatuppsatsen jämför Random Forest med LSTM, genom att undersöka hur väl modellerna kan användas för att klassificera ett meddelande som fråga eller icke-fråga. Jämförelsen gjordes genom att träna och optimera modellerna på historisk chattdata från det svenska försäkringsbolaget Hedvig. Olika typer av word embedding, så som Word2vec och Bag of Words, testades också. Resultaten visade att LSTM uppnådde något högre F1 och accuracy än Random Forest. Modellernas prestanda förbättrades inte signifikant efter optimering och resultatet var också beroende av vilket korpus modellerna tränades på. En undersökning av hur en chattbot skulle påverka Hedvigs adoption rate genomfördes också, huvudsakligen genom att granska tidigare studier om chattbotars effekt på användarupplevelsen. De potentiella effekterna på en innovations fem attribut, relativ fördel, kompatibilitet, komplexitet, prövbarhet and observerbarhet analyserades för att kunna svara på frågeställningen. Resultaten visade att Hedvigs adoption rate kan påverkas positivt, genom att förbättra de två första attributen. Effekterna en chattbot skulle ha på komplexitet, prövbarhet och observerbarhet ansågs dock vara försumbar, om inte negativ.
16
Nguyen, Thanh Le Vi. "Local Binary Pattern based algorithms for the discrimination and detection of crops and weeds with similar morphologies." Thesis, Edith Cowan University, Research Online, Perth, Western Australia, 2020. https://ro.ecu.edu.au/theses/2359.
Full textAbstract:
In cultivated agricultural fields, weeds are unwanted species that compete with the crop plants for nutrients, water, sunlight and soil, thus constraining their growth. Applying new real-time weed detection and spraying technologies to agriculture would enhance current farming practices, leading to higher crop yields and lower production costs. Various weed detection methods have been developed for Site-Specific Weed Management (SSWM) aimed at maximising the crop yield through efficient control of weeds. Blanket application of herbicide chemicals is currently the most popular weed eradication practice in weed management and weed invasion. However, the excessive use of herbicides has a detrimental impact on the human health, economy and environment. Before weeds are resistant to herbicides and respond better to weed control strategies, it is necessary to control them in the fallow, pre-sowing, early post-emergent and in pasture phases. Moreover, the development of herbicide resistance in weeds is the driving force for inventing precision and automation weed treatments. Various weed detection techniques have been developed to identify weed species in crop fields, aimed at improving the crop quality, reducing herbicide and water usage and minimising environmental impacts.
In this thesis, Local Binary Pattern (LBP)-based algorithms are developed and tested experimentally, which are based on extracting dominant plant features from camera images to precisely detecting weeds from crops in real time. Based on the efficient computation and robustness of the first LBP method, an improved LBP-based method is developed based on using three different LBP operators for plant feature extraction in conjunction with a Support Vector Machine (SVM) method for multiclass plant classification. A 24,000-image dataset, collected using a testing facility under simulated field conditions (Testbed system), is used for algorithm training, validation and testing. The dataset, which is published online under the name “bccr-segset”, consists of four subclasses: background, Canola (Brassica napus), Corn (Zea mays), and Wild radish (Raphanus raphanistrum). In addition, the dataset comprises plant images collected at four crop growth stages, for each subclass. The computer-controlled Testbed is designed to rapidly label plant images and generate the “bccr-segset” dataset. Experimental results show that the classification accuracy of the improved LBP-based algorithm is 91.85%, for the four classes.
Due to the similarity of the morphologies of the canola (crop) and wild radish (weed) leaves, the conventional LBP-based method has limited ability to discriminate broadleaf crops from weeds. To overcome this limitation and complex field conditions (illumination variation, poses, viewpoints, and occlusions), a novel LBP-based method (denoted k-FLBPCM) is developed to enhance the classification accuracy of crops and weeds with similar morphologies. Our contributions include (i) the use of opening and closing morphological operators in pre-processing of plant images, (ii) the development of the k-FLBPCM method by combining two methods, namely, the filtered local binary pattern (LBP) method and the contour-based masking method with a coefficient k, and (iii) the optimal use of SVM with the radial basis function (RBF) kernel to precisely identify broadleaf plants based on their distinctive features. The high performance of this k-FLBPCM method is demonstrated by experimentally attaining up to 98.63% classification accuracy at four different growth stages for all classes of the “bccr-segset” dataset.
To evaluate performance of the k-FLBPCM algorithm in real-time, a comparison analysis between our novel method (k-FLBPCM) and deep convolutional neural networks (DCNNs) is conducted on morphologically similar crops and weeds. Various DCNN models, namely VGG-16, VGG-19, ResNet50 and InceptionV3, are optimised, by fine-tuning their hyper-parameters, and tested. Based on the experimental results on the “bccr-segset” dataset collected from the laboratory and the “fieldtrip_can_weeds” dataset collected from the field under practical environments, the classification accuracies of the DCNN models and the k-FLBPCM method are almost similar. Another experiment is conducted by training the algorithms with plant images obtained at mature stages and testing them at early stages. In this case, the new k-FLBPCM method outperformed the state-of-the-art CNN models in identifying small leaf shapes of canola-radish (crop-weed) at early growth stages, with an order of magnitude lower error rates in comparison with DCNN models. Furthermore, the execution time of the k-FLBPCM method during the training and test phases was faster than the DCNN counterparts, with an identification time difference of approximately 0.224ms per image for the laboratory dataset and 0.346ms per image for the field dataset. These results demonstrate the ability of the k-FLBPCM method to rapidly detect weeds from crops of similar appearance in real time with less data, and generalize to different size plants better than the CNN-based methods.
17
Gardner, Angelica. "Stronger Together? An Ensemble of CNNs for Deepfakes Detection." Thesis, Linnéuniversitetet, Institutionen för datavetenskap och medieteknik (DM), 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:lnu:diva-97643.
Full textAbstract:
Deepfakes technology is a face swap technique that enables anyone to replace faces in a video, with highly realistic results. Despite its usefulness, if used maliciously, this technique can have a significant impact on society, for instance, through the spreading of fake news or cyberbullying. This makes the ability of deepfakes detection a problem of utmost importance. In this paper, I tackle the problem of deepfakes detection by identifying deepfakes forgeries in video sequences. Inspired by the state-of-the-art, I study the ensembling of different machine learning solutions built on convolutional neural networks (CNNs) and use these models as objects for comparison between ensemble and single model performances. Existing work in the research field of deepfakes detection suggests that escalated challenges posed by modern deepfake videos make it increasingly difficult for detection methods. I evaluate that claim by testing the detection performance of four single CNN models as well as six stacked ensembles on three modern deepfakes datasets. I compare various ensemble approaches to combine single models and in what way their predictions should be incorporated into the ensemble output. The results I found was that the best approach for deepfakes detection is to create an ensemble, though, the ensemble approach plays a crucial role in the detection performance. The final proposed solution is an ensemble of all available single models which use the concept of soft (weighted) voting to combine its base-learners’ predictions. Results show that this proposed solution significantly improved deepfakes detection performance and substantially outperformed all single models.
18
Le, Thu Anh. "An Exploration of the Word2vec Algorithm: Creating a Vector Representation of a Language Vocabulary that Encodes Meaning and Usage Patterns in the Vector Space Structure." Thesis, University of North Texas, 2016. https://digital.library.unt.edu/ark:/67531/metadc849728/.
Full textAbstract:
This thesis is an exloration and exposition of a highly efficient shallow neural network algorithm called word2vec, which was developed by T. Mikolov et al. in order to create vector representations of a language vocabulary such that information about the meaning and usage of the vocabulary words is encoded in the vector space structure. Chapter 1 introduces natural language processing, vector representations of language vocabularies, and the word2vec algorithm. Chapter 2 reviews the basic mathematical theory of deterministic convex optimization. Chapter 3 provides background on some concepts from computer science that are used in the word2vec algorithm: Huffman trees, neural networks, and binary cross-entropy. Chapter 4 provides a detailed discussion of the word2vec algorithm itself and includes a discussion of continuous bag of words, skip-gram, hierarchical softmax, and negative sampling. Finally, Chapter 5 explores some applications of vector representations: word categorization, analogy completion, and language translation assistance.
19
Левчук, Святослав Богданович. "Інтелектуальна система мерчандайзингу. Детекція та розпізнавання асортименту." Master's thesis, Київ, 2018. https://ela.kpi.ua/handle/123456789/23987.
Full textAbstract:
Магістерська дисертація: 126 с., 47 рис., 30 табл., 2 додатки, 31 джерело.
Об'єктом дослідження є методи мерчандайзингу товарів в торгових точках.
Предметом дослідження є методи класифікації товарів на полицях в торгових точках.
Мета роботи – розробка інтелектуальної системи мерчендайзингу, яка дозволить зменшити використання людського ресурсу та максимально оптимізувати процес мерчендайзингу за рахунок автоматичного моніторингу наявності товару на полицях та розробка системи класифікації товарів як складової частини системи мерчиндайзингу для аналізу товарів на полиці відносно планограм магазину.
В роботі розглянуто і проаналізовано сучасні системи мерчандайзингу та іх недоліки, також, розглядаються існуючі методи класифікації.
Запропоновано метод класифікації товарів в магазині з спеціально розробленою згортковою нейронною мережею, який побудовано на основі методів з використанням згорткових нейронних мереж, з нелінійними класифікаторами та адаптивним методом оптимізації.
Інтелектуальна система мерчандайзингу та система класифікації асортименту реалізовані за допомогою мови програмування Python з використанням СУБД MySql. Результати даної роботи рекомендується використовувати для моніторингу якості викладки товарів на полицях та контролю наповненості полиць у торгових точках.Master thesis explanatory note: 126 p., 47 fig., 30 tab., 2 appendices, 31 sources. The object of research – intelligent merchandising system. The subject of research – classification methods of goods on shelves in stores. The purpose of the work is to develop an intelligent merchandising system that will reduce the use of human resources and maximize the process of merchandising through automatic monitoring of the availability of goods on shelves and to develop of goods classification system as a part of a merchandising system for the analysis of goods on the shelf in relation to the store planograms. In the work, modern merchandising systems and their shortcomings are considered and analyzed, as well as existing classification methods are considered. Goods classification method with specially developed convolutional neural network, which is constructed on the basis of methods using convolutional neural networks, with nonlinear classifiers and an adaptive optimization method is proposed. Intelligent merchandising system and assortment classification system are implemented using Python programming language with MySql DB. The results of this work are recommended for monitoring the compliance with the planogram and availiability of the goods on shelves in stores.
20
Singh, Gurpreet. "Statistical Modeling of Dynamic Risk in Security Systems." Thesis, KTH, Matematisk statistik, 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-273599.
Full textAbstract:
Big data has been used regularly in finance and business to build forecasting models. It is, however, a relatively new concept in the security industry. This study predicts technology related alarm codes that will sound in the coming 7 days at location $L$ by observing the past 7 days. Logistic regression and neural networks are applied to solve this problem. Due to the problem being of a multi-labeled nature logistic regression is applied in combination with binary relevance and classifier chains. The models are trained on data that has been labeled with two separate methods, the first method labels the data by only observing location $L$. The second considers $L$ and $L$'s surroundings. As the problem is multi-labeled the labels are likely to be unbalanced, thus a resampling technique, SMOTE, and random over-sampling is applied to increase the frequency of the minority labels. Recall, precision, and F1-score are calculated to evaluate the models. The results show that the second labeling method performs better for all models and that the classifier chains and binary relevance model performed similarly. Resampling the data with the SMOTE technique increases the macro average F1-scores for the binary relevance and classifier chains models, however, the neural networks performance decreases. The SMOTE resampling technique also performs better than random over-sampling. The neural networks model outperforms the other two models on all methods and achieves the highest F1-score.Big data har använts regelbundet inom ekonomi för att bygga prognosmodeller, det är dock ett relativt nytt koncept inom säkerhetsbranschen. Denna studie förutsäger vilka larmkoder som kommer att låta under de kommande 7 dagarna på plats $L$ genom att observera de senaste 7 dagarna. Logistisk regression och neurala nätverk används för att lösa detta problem. Eftersom att problemet är av en multi-label natur tillämpas logistisk regression i kombination med binary relevance och classifier chains. Modellerna tränas på data som har annoterats med två separata metoder. Den första metoden annoterar datan genom att endast observera plats $L$ och den andra metoden betraktar $L$ och $L$:s omgivning. Eftersom problemet är multi-labeled kommer annoteringen sannolikt att vara obalanserad och därför används resamplings metoden, SMOTE, och random over-sampling för att öka frekvensen av minority labels. Recall, precision och F1-score mättes för att utvärdera modellerna. Resultaten visar att den andra annoterings metoden presterade bättre för alla modeller och att classifier chains och binary relevance presterade likartat. Binary relevance och classifier chains modellerna som tränades på datan som använts sig av resamplings metoden SMOTE gav ett högre macro average F1-score, dock sjönk prestationen för neurala nätverk. Resamplings metoden SMOTE presterade även bättre än random over-sampling. Neurala nätverksmodellen överträffade de andra två modellerna på alla metoder och uppnådde högsta F1-score.
21
Hedar, Sara. "Applying Machine Learning Methods to Predict the Outcome of Shots in Football." Thesis, Uppsala universitet, Avdelningen för systemteknik, 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-414774.
Full textAbstract:
The thesis investigates a publicly available dataset which covers morethan three million events in football matches. The aim of the study isto train machine learning models capable of modeling the relationshipbetween a shot event and its outcome. That is, to predict if a footballshot will result in a goal or not. By representing the shot indifferent ways, the aim is to draw conclusion regarding what elementsof a shot allows for a good prediction of its outcome. The shotrepresentation was varied both by including different numbers of eventspreceding the shot and by varying the set of features describing eachevent.The study shows that the performance of the machine learning modelsbenefit from including events preceding the shot. The highestpredictive performance was achieved by a long short-term memory neuralnetwork trained on the shot event and six events preceding the shot.The features which were found to have the largest positive impact onthe shot events were the precision of the event, the position on thefield and how the player was in contact with the ball. The size of thedataset was also evaluated and the results suggest that it issufficiently large for the size of the networks evaluated.
22
Teixeira, Alex Fernandes Rocha. "Identificação de uma coluna de destilação de metanol-água através de modelos paramétricos e redes neurais artificiais." Universidade Federal de Alagoas, 2011. http://www.repositorio.ufal.br/handle/riufal/1195.
Full textAbstract:
This work presents a black box identification for a continuous methanol-water distillation column setting in open loop and closed loop response. Step changes and Pseudo-Random Binary Signal (PRBS) disturbance were used to excite the plant. The mathematical models candidates to identify were the Artificial Neural Networks (ANN) and the parametric models: ARX(autoregressive with exogenous inputs ), ARMAX (AutoRegressive Moving Average with eXogenous inputs ), OE(Output Error) and the Box-Jenkins (BJ)structure. The closed loop configuration was the R-V. The results showed that for the bottom loop, the best response were given by BJ, OE and RNA for both open and closed loop response. For the top closed loop, the best responses were also given by BJ, OE and RNA while in open loop condition, the RNA was the one that gave satisfactory outcome. It was verified that the pseudo-random binary signal was a good choice of excitation signal in identification for both open loop and closed dynamic systems.Foi realizado neste trabalho identificação caixa preta do processo de destilação Metanol-Água nas configurações malha aberta e malha fechada, utilizando como sinais de perturbação a função degrau e o Sinal Binário Pseudo-Aleatório (PRBS) para excitar a planta. Os modelos matemáticos candidatos a identificação foram as Redes Neurais Artificiais (RNA), e os modelos paramétricos discretos lineares autorregressivo com entradas externas (ARX do inglês AutoRegressive with eXogenous Inputs), autorregressivo com média móvel e entradas exógenas (ARMAX do inglês AutoRegressive Moving Average with eXogenous Inputs), modelo do tipo erro na saída (OE do inglês Output Error) e a estrutura Box-Jenkins (BJ). Com a disposição dos modelos, foram comparados quais dos modelos matemáticos candidatos à identificação melhor representa o processo coluna de destilação metanol-água. Comparou-se qual configuração do processo no ensaio de identificação para geração de dados apresenta mais vantagens, se em malha aberta ou em malha fechada, nas condições e metodologias utilizadas. Constatou-se a funcionalidade do sinal binário pseudo-aleatório como uma boa opção de excitação na identificação em malha aberta e fechada para sistemas dinâmicos.
23
Alamgir, Nyma. "Computer vision based smoke and fire detection for outdoor environments." Thesis, Queensland University of Technology, 2020. https://eprints.qut.edu.au/201654/1/Nyma_Alamgir_Thesis.pdf.
Full textAbstract:
Surveillance Video-based detection of outdoor smoke and fire has been a challenging task due to the chaotic variations of shapes, movement, colour, texture, and density. This thesis contributes to the advancement of the contemporary efforts of smoke and fire detection by proposing novel technical methods and their possible integration into a complete fire safety model. The novel contributions of this thesis include an efficient feature calculation method combining local and global texture properties, the development of deep learning-based models and a conceptual framework to incorporate weather information in the fire safety model for improved accuracy in fire prediction and detection.
24
Santos, Cinara de Jesus. "Avaliação do uso de classificadores para verificação de atendimento a critérios de seleção em programas sociais." Universidade Federal de Juiz de Fora (UFJF), 2017. https://repositorio.ufjf.br/jspui/handle/ufjf/5582.
Full textAbstract:
Submitted by isabela.moljf@hotmail.com (isabela.moljf@hotmail.com) on 2017-08-15T12:01:50Z
No. of bitstreams: 1
cinaradejesussantos.pdf: 4566569 bytes, checksum: bddc2ea97276541c0a8ad30a371102d1 (MD5)Approved for entry into archive by Adriana Oliveira (adriana.oliveira@ufjf.edu.br) on 2017-08-15T12:02:54Z (GMT) No. of bitstreams: 1 cinaradejesussantos.pdf: 4566569 bytes, checksum: bddc2ea97276541c0a8ad30a371102d1 (MD5)
Made available in DSpace on 2017-08-15T12:02:54Z (GMT). No. of bitstreams: 1 cinaradejesussantos.pdf: 4566569 bytes, checksum: bddc2ea97276541c0a8ad30a371102d1 (MD5) Previous issue date: 2017-03-07
Classificadores são separadores de grupos que mediante determinadas características organiza os dados agrupando elementos que apresentem traços semelhantes, o que permite reconhecimento de padrões e identificação de elementos que não se encaixam. Esse procedimento de classificação e separação pode ser observado em processos do cotidiano como exames (clínicos ou por imagem), separadores automáticos de grãos na agroindústria, identificador de probabilidades, reconhecedores de caracteres, identificação biométrica - digital, íris, face, etc. O estudo aqui proposto utiliza uma base de dados do Ministério do Desenvolvimento Social e Combate a Fome (MDS), contendo informações sobre beneficiários do Programa Bolsa Família (PBF), onde contamos com registros descritores do ambiente domiciliar, grau de instrução dos moradores do domicílio assim como o uso de serviços de saúde pelos mesmos e informações de cunho financeiro (renda e gastos das famílias). O foco deste estudo não visa avaliar o PBF, mas o comportamento de classificadores aplicados sobre bases de caráter social, pois estas apresentam certas particularidades. Sobre as variáveis que descrevem uma família como beneficiária ou não do PBF, testamos três algoritmos classificadores - regressão logística, árvore binária de decisão e rede neural artificial em múltiplas camadas. O desempenho destes processos foi medido a partir de métricas decorrentes da chamada matriz de confusão. Como os erros e acertos de uma classe n˜ao s˜ao os complementares da outra classe é de suma importância que ambas sejam corretamente identificadas. Um desempenho satisfatório para ambas as classes em um mesmo cenário não foi alçado - a identificação do grupo minoritário apresentou baixa eficiência mesmo com reamostragem seguida de reaplicação dos três processos classificatórios escolhidos, o que aponta para a necessidade de novos experimentos.
Classifiers are group separators that, by means of certain characteristics, organize the data by grouping elements that present similar traits, which allows pattern recognition and the identification of elements that do not fit. Classification procedures can be used in everyday processes such as clinical or imaging exams, automatic grain separators in agribusiness, probability identifiers, character recognition, biometric identification by thumbprints, iris, face, etc. This study uses a database of the Ministry of Social Development and Fight against Hunger (MDS), containing information on beneficiaries of the Bolsa Fam´ılia Program (PBF). The data describe the home environment, the level of education of the residents of the household, their use of public health services, and some financial information (income and expenses of families). The focus of this study is not to evaluate the PBF, but to analyze the performance of the classifiers when applied to bases of social character, since these have certain peculiarities. We have tested three classification algorithms - logistic regression, binary decision trees and artificial neural networks. The performance of these algorithms was measured by metrics computed from the so-called confusion matrix. As the probabilities of right and wrong classifications of a class are not complementary, it is of the utmost importance that both are correctly identified. A good evaluation could not be archive for both classes in a same scenario was not raised - the identification of the minority group showed low efficiency even with resampling followed by reapplication of the three classificatory processes chosen, which points to the need for new experiments.
25
Nguyen, Minh Ha Information Technology & Electrical Engineering Australian Defence Force Academy UNSW. "Cooperative coevolutionary mixture of experts : a neuro ensemble approach for automatic decomposition of classification problems." Awarded by:University of New South Wales - Australian Defence Force Academy. School of Information Technology and Electrical Engineering, 2006. http://handle.unsw.edu.au/1959.4/38752.
Full textAbstract:
Artificial neural networks have been widely used for machine learning and optimization. A neuro ensemble is a collection of neural networks that works cooperatively on a problem. In the literature, it has been shown that by combining several neural networks, the generalization of the overall system could be enhanced over the separate generalization ability of the individuals. Evolutionary computation can be used to search for a suitable architecture and weights for neural networks. When evolutionary computation is used to evolve a neuro ensemble, it is usually known as evolutionary neuro ensemble. In most real-world problems, we either know little about these problems or the problems are too complex to have a clear vision on how to decompose them by hand. Thus, it is usually desirable to have a method to automatically decompose a complex problem into a set of overlapping or non-overlapping sub-problems and assign one or more specialists (i.e. experts, learning machines) to each of these sub-problems. An important feature of neuro ensemble is automatic problem decomposition. Some neuro ensemble methods are able to generate networks, where each individual network is specialized on a unique sub-task such as mapping a subspace of the feature space. In real world problems, this is usually an important feature for a number of reasons including: (1) it provides an understanding of the decomposition nature of a problem; (2) if a problem changes, one can replace the network associated with the sub-space where the change occurs without affecting the overall ensemble; (3) if one network fails, the rest of the ensemble can still function in their sub-spaces; (4) if one learn the structure of one problem, it can potentially be transferred to other similar problems. In this thesis, I focus on classification problems and present a systematic study of a novel evolutionary neuro ensemble approach which I call cooperative coevolutionary mixture of experts (CCME). Cooperative coevolution (CC) is a branch of evolutionary computation where individuals in different populations cooperate to solve a problem and their fitness function is calculated based on their reciprocal interaction. The mixture of expert model (ME) is a neuro ensemble approach which can generate networks that are specialized on different sub-spaces in the feature space. By combining CC and ME, I have a powerful framework whereby it is able to automatically form the experts and train each of them. I show that the CCME method produces competitive results in terms of generalization ability without increasing the computational cost when compared to traditional training approaches. I also propose two different mechanisms for visualizing the resultant decomposition in high-dimensional feature spaces. The first mechanism is a simple one where data are grouped based on the specialization of each expert and a color-map of the data records is visualized. The second mechanism relies on principal component analysis to project the feature space onto lower dimensions, whereby decision boundaries generated by each expert are visualized through convex approximations. I also investigate the regularization effect of learning by forgetting on the proposed CCME. I show that learning by forgetting helps CCME to generate neuro ensembles of low structural complexity while maintaining their generalization abilities. Overall, the thesis presents an evolutionary neuro ensemble method whereby (1) the generated ensemble generalizes well; (2) it is able to automatically decompose the classification problem; and (3) it generates networks with small architectures.
26
Narayanan, Arun. "Computational auditory scene analysis and robust automatic speech recognition." The Ohio State University, 2014. http://rave.ohiolink.edu/etdc/view?acc_num=osu1401460288.
Full text
27
Beneš, Jiří. "Unární klasifikátor obrazových dat." Master's thesis, Vysoké učení technické v Brně. Fakulta elektrotechniky a komunikačních technologií, 2021. http://www.nusl.cz/ntk/nusl-442432.
Full textAbstract:
The work deals with an introduction to classification algorithms. It then divides classifiers into unary, binary and multi-class and describes the different types of classifiers. The work compares individual classifiers and their areas of use. For unary classifiers, practical examples and a list of used architectures are given in the work. The work contains a chapter focused on the comparison of the effects of hyper parameters on the quality of unary classification for individual architectures. Part of the submission is a practical example of reimplementation of the unary classifier.
28
Faula, Yannick. "Extraction de caractéristiques sur des images acquises en contexte mobile : Application à la reconnaissance de défauts sur ouvrages d’art." Thesis, Lyon, 2020. http://www.theses.fr/2020LYSEI077.
Full textAbstract:
Le réseau ferroviaire français dispose d’une infrastructure de grande ampleur qui se compose de nombreux ouvrages d’art. Ces derniers subissent les dégradations du temps et du trafic et font donc l’objet d’une surveillance périodique pour détecter l’apparition de défauts. Aujourd’hui, cette inspection se fait en grande partie, visuellement par des opérateurs experts. Plusieurs entreprises testent de nouveaux vecteurs d’acquisition photo comme le drone, destinés à la surveillance des ouvrages de génie civil. Dans cette thèse, l’objectif principal est de développer un système capable de détecter, localiser et enregistrer d’éventuels défauts de l’ouvrage. Un grand défi est de détecter des défauts sous-pixels comme les fissures en temps réel pour améliorer l’acquisition. Pour cela, une analyse par seuillage local a été conçue pour traiter de grandes images. Cette analyse permet d’extraire des points d’intérêts (Points FLASH: Fast Local Analysis by threSHolding) où une ligne droite peut se faufiler. La mise en relation intelligente de ces points permet de détecter et localiser les fissures fines. Les résultats de détection de fissures de surfaces altérées issues d'images d'ouvrages d'art démontrent de meilleures performances en temps de calcul et robustesse que les algorithmes existants. En amont de l'étape de détection, il est nécessaire de s’assurer que les images acquises soient de bonne qualité pour réaliser le traitement. Une mauvaise mise au point ou un flou de bougé sont à bannir. Nous avons développé une méthode réutilisant les calculs de la détection en extrayant des mesures de Local Binary Patterns (LBP) afin de vérifier la qualité en temps réel. Enfin, pour réaliser une acquisition permettant une reconstruction photogrammétrique, les images doivent avoir un recouvrement suffisant. Notre algorithme, réutilisant les points d’intérêts de la détection, permet un appariement simple entre deux images sans passer par des algorithmes de type RANSAC. Notre méthode est invariante en rotation, translation et à une certaine plage de changements d’échelle. Après l’acquisition, sur les images de qualité optimale, il est possible d'employer des méthodes plus coûteuses en temps comme les réseaux de neurones à convolution. Ces derniers bien qu'incapables d’assurer une détection de fissures en temps réel peuvent être utilisés pour détecter certains types d’avaries. Cependant, le manque de données impose la constitution de notre propre jeu de données. A l'aide d'approches de classification indépendante (classifieurs SVM one-class), nous avons développé un système flexible capable d’évoluer dans le temps, de détecter puis de classifier les différents types de défauts. Aucun système de ce type n’apparaît dans la littérature. Les travaux réalisés sur l’extraction de caractéristiques sur des images pour la détection de défauts pourront être utiles dans d’autres applications telles que la navigation de véhicules intelligents ou le word-spottingThe french railway network has a huge infrastructure which is composed of many civil engineering structures. These suffer from degradation of time and traffic and they are subject to a periodic monitoring in order to detect appearance of defects. At the moment, this inspection is mainly done visually by monitoring operators. Several companies test new vectors of photo acquisition like the drone, designed for civil engineering monitoring. In this thesis, the main goal is to develop a system able to detect, localize and save potential defects of the infrastructure. A huge issue is to detect sub-pixel defects like cracks in real time for improving the acquisition. For this task, a local analysis by thresholding is designed for treating large images. This analysis can extract some points of interest (FLASH points: Fast Local Analysis by threSHolding) where a straight line can sneak in. The smart spatial relationship of these points allows to detect and localise fine cracks. The results of the crack detection on concrete degraded surfaces coming from images of infrastructure show better performances in time and robustness than the state-of-art algorithms. Before the detection step, we have to ensure the acquired images have a sufficient quality to make the process. A bad focus or a movement blur are prohibited. We developed a method reusing the preceding computations to assess the quality in real time by extracting Local Binary Pattern (LBP) values. Then, in order to make an acquisition for photogrammetric reconstruction, images have to get a sufficient overlapping. Our algorithm, reusing points of interest of the detection, can make a simple matching between two images without using algorithms as type RANSAC. Our method has invariance in rotation, translation and scale range. After the acquisition, with images with optimal quality, it is possible to exploit methods more expensive in time like convolution neural networks. These are not able to detect cracks in real time but can detect other kinds of damages. However, the lack of data requires the constitution of our database. With approaches of independent classification (classifier SVM one-class), we developed a dynamic system able to evolve in time, detect and then classify the different kinds of damages. No system like ours appears in the literature for the defect detection on civil engineering structure. The implemented works on feature extraction on images for damage detection will be used in other applications as smart vehicle navigation or word spotting
29
Murach, Thomas. "Monoscopic Analysis of H.E.S.S. Phase II Data on PSR B1259–63/LS 2883." Doctoral thesis, Humboldt-Universität zu Berlin, 2017. http://dx.doi.org/10.18452/18484.
Full textAbstract:
Cherenkov-Teleskope sind in der Lage, das schwache Cherenkovlicht aus Teilchenschauern zu detektieren, die von kosmischen Teilchen mit Energien von ca. 100 GeV bis 100 TeV in der Erdatmosphäre initiiert werden. Das Ziel ist die Detektion von Cherenkovlicht aus Schauern, die von Gammastrahlen erzeugt wurden, der größte Teil der Schauer stammt jedoch von geladenen Teilchen. Im Jahr 2012 wurde das H.E.S.S.-Observatorium in Namibia, bis dahin bestehend aus vier Teleskopen mit 100 m²-Spiegeln, um ein fünftes Teleskop mit einer Spiegelfläche von ca. 600 m² ergänzt. Aufgrund der großen Spiegelfläche besitzt dieses Teleskop die niedrigste Energieschwelle aller Teleskope dieser Art. In dieser Dissertation wird ein schneller Algorithmus namens MonoReco präsentiert, der grundlegende Eigenschaften der Gammastrahlen wie ihre Energien und Richtungen rekonstruieren kann. Dieser Algorithmus kann weiterhin unterscheiden, ob Schauer von Gammastrahlen oder von geladenen Teilchen der kosmischen Strahlung initiiert wurden. Diese Aufgaben werden mit mithilfe von künstlichen neuronalen Netzwerken erfüllt, welche ausschließlich die Momente der Intensitätsverteilungen in der Kamera des neuen Teleskops analysieren. Eine Energieschwelle von 59 GeV und Richtungsauflösungen von 0.1°-0.3° werden erreicht. Das Energiebias liegt bei wenigen Prozent, die Energieauflösung bei 20-30%. Unter anderem mit dem MonoReco-Algorithmus wurden Daten, die in der Zeit um das Periastron des Binärsystems PSR B1259-63/LS 2883 im Jahre 2014 genommen wurden, analysiert. Es handelt sich hierbei um einen Neutronenstern, der sich in einem 3,4-Jahres-Orbit um einen massereichen Stern mit einer den Stern umgebenden Scheibe aus Gas und Plasmen befindet. Zum ersten Mal konnte H.E.S.S. das Gammastrahlenspektrum dieses Systems bei Energien unterhalb von 200 GeV messen. Weiterhin wurde bei erstmaligen Beobachtungen zur Zeit des Periastrons ein lokales Flussminimum gemessen. Sowohl vor dem ersten als auch nach dem zweiten Transit des Neutronensterns durch die Scheibe wurden hohe Flüsse gemessen. Im zweiten Fall wurden Beobachtungen erstmals zeitgleich mit dem Fermi-LAT-Experiment durchgeführt, das wiederholt sehr hohe Flüsse in diesem Teil des Orbits messen konnte. Ein Vergleich der gemessenen Flüsse mit Vorhersagen eines leptonischen Modells zeigt gute Übereinstimmungen.Cherenkov telescopes can detect the faint Cherenkov light emitted by air showers that were initiated by cosmic particles with energies between approximately 100 GeV and 100 TeV in the Earth's atmosphere. Aiming for the detection of Cherenkov light emitted by gamma ray-initiated air showers, the vast majority of all detected showers are initiated by charged cosmic rays. In 2012 the H.E.S.S. observatory, until then comprising four telescopes with 100 m² mirrors each, was extended by adding a much larger fifth telescope with a very large mirror area of 600 m². Due to the large mirror area, this telescope has the lowest energy threshold of all telescopes of this kind. In this dissertation, a fast algorithm called MonoReco is presented that can reconstruct fundamental properties of the primary gamma rays like their direction or their energy. Furthermore, this algorithm can distinguish between air showers initiated either by gamma rays or by charged cosmic rays. Those tasks are accomplished with the help of artificial neural networks, which analyse moments of the intensity distributions in the camera of the new telescope exclusively. The energy threshold is 59 GeV and angular resolutions of 0.1°-0.3° are achieved. The energy reconstruction bias is at the level of a few percent, the energy resolution is at the level of 20-30%. Data taken around the 2014 periastron passage of the gamma-ray binary PSR B1259-63/LS 2883 were analysed with, among others, the MonoReco algorithm. This binary system comprises a neutron star in a 3.4 year orbit around a massive star with a circumstellar disk consisting of gas and plasma. For the first time the gamma-ray spectrum of this system could be measured by H.E.S.S. down to below 200 GeV. Furthermore, a local flux minimum could be measured during unprecedented measurements at the time of periastron. High fluxes were measured both before the first and after the second transit of the neutron star through the disk. In the second case measurements could be performed for the first time contemporaneously with the Fermi-LAT experiment, which has repeatedly detected very high fluxes at this part of the orbit. A good agreement between measured fluxes and predictions of a leptonic model is found.
30
"Applications of neural networks in the binary classification problem." 1997. http://library.cuhk.edu.hk/record=b5889310.
Full textAbstract:
by Chan Pak Kei, Bernard.Thesis (M.Phil.)--Chinese University of Hong Kong, 1997.
Includes bibliographical references (leaves 125-127).
Chapter 1 --- Introduction --- p.10
Chapter 1.1 --- Overview --- p.10
Chapter 1.2 --- Classification Approaches --- p.11
Chapter 1.3 --- The Use of Neural Network --- p.12
Chapter 1.4 --- Motivations --- p.14
Chapter 1.5 --- Organization of Thesis --- p.16
Chapter 2 --- Related Work --- p.19
Chapter 2.1 --- Overview --- p.19
Chapter 2.2 --- Neural Network --- p.20
Chapter 2.2.1 --- Backpropagation Feedforward Neural Network --- p.20
Chapter 2.2.2 --- Training of a Backpropagation Feedforward Neural Network --- p.22
Chapter 2.2.3 --- Single Hidden-layer Model --- p.27
Chapter 2.2.4 --- Data Preprocessing --- p.27
Chapter 2.3 --- Fuzzy Sets --- p.29
Chapter 2.3.1 --- Fuzzy Linear Regression Analysis --- p.29
Chapter 2.4 --- Network Architecture Altering Algorithms --- p.31
Chapter 2.4.1 --- Pruning Algorithms --- p.32
Chapter 2.4.2 --- Constructive/Growing Algorithms --- p.35
Chapter 2.5 --- Summary --- p.38
Chapter 3 --- Hybrid Classification Systems --- p.39
Chapter 3.1 --- Overview --- p.39
Chapter 3.2 --- Literature Review --- p.41
Chapter 3.2.1 --- Fuzzy Linear Regression(FLR) with Fuzzy Interval Analysis --- p.41
Chapter 3.3 --- Data Sample and Methodology --- p.44
Chapter 3.4 --- Hybrid Model --- p.46
Chapter 3.4.1 --- Construction of Model --- p.46
Chapter 3.5 --- Experimental Results --- p.50
Chapter 3.5.1 --- Experimental Results on Breast Cancer Database --- p.50
Chapter 3.5.2 --- Experimental Results on Synthetic Data --- p.53
Chapter 3.6 --- Conclusion --- p.55
Chapter 4 --- Searching for Suitable Network Size Automatically --- p.59
Chapter 4.1 --- Overview --- p.59
Chapter 4.2 --- Literature Review --- p.61
Chapter 4.2.1 --- Pruning Algorithm --- p.61
Chapter 4.2.2 --- Constructive Algorithms (Growing) --- p.66
Chapter 4.2.3 --- Integration of methods --- p.67
Chapter 4.3 --- Methodology and Approaches --- p.68
Chapter 4.3.1 --- Growing --- p.68
Chapter 4.3.2 --- Combinations of Growing and Pruning --- p.69
Chapter 4.4 --- Experimental Results --- p.75
Chapter 4.4.1 --- Breast-Cancer Cytology Database --- p.76
Chapter 4.4.2 --- Tic-Tac-Toe Database --- p.82
Chapter 4.5 --- Conclusion --- p.89
Chapter 5 --- Conclusion --- p.91
Chapter 5.1 --- Recall of Thesis Objectives --- p.91
Chapter 5.2 --- Summary of Achievements --- p.92
Chapter 5.2.1 --- Data Preprocessing --- p.92
Chapter 5.2.2 --- Network Size --- p.93
Chapter 5.3 --- Future Works --- p.94
Chapter A --- Experimental Results of Ch3 --- p.95
Chapter B --- Experimental Results of Ch4 --- p.112
Bibliography --- p.125
31
TANG, CHI-HUAN, and 唐其煥. "Low-cost Design and Implementation for Binary Convolutional Neural Networks." Thesis, 2018. http://ndltd.ncl.edu.tw/handle/c5aa76.
Full textAbstract:
碩士國立高雄應用科技大學
電子工程系
106
In recent years, deep learning has been one of the most popular subject in academia and widely used in many fields such as computer vision, image classification, motion recognition, voice recognition, and big-data analysis tasks. Although the larger neural network architecture can improve accuracy obviously, the cost of memory usage, power consumption and time consumption also increase. How to use memory and speed effectively to achieve a certain accuracy has been the most popular subject in recent years. In the first part of this thesis, we will introduce the development of convolution neural network in recent years, and then we will introduce and explore the diversification of binary neural network. Finally, we will focus on Deep Residual Network and propose our method to improved XNOR-Net. By adjusting Deep Residual Network basic structure, increasing the possible of input layer and replacing more simply bit counter than multiplier, we can simplify large network architecture and increase accuracy than previous network greatly. The experimental results demonstrate that our design achieves the same performances in memory usage as XNOR-Net. Moreover, it can dramatically increase accuracy in Cifar10/Cifar100 datasets, and achieve the good accuracy result than other binary neural network paper.
32
"Approaches to the implementation of binary relation inference network." Chinese University of Hong Kong, 1994. http://library.cuhk.edu.hk/record=b5888221.
Full textAbstract:
by C.W. Tong.Thesis (M.Phil.)--Chinese University of Hong Kong, 1994.
Includes bibliographical references (leaves 96-98).
Chapter 1 --- Introduction --- p.1
Chapter 1.1 --- The Availability of Parallel Processing Machines --- p.2
Chapter 1.1.1 --- Neural Networks --- p.5
Chapter 1.2 --- Parallel Processing in the Continuous-Time Domain --- p.6
Chapter 1.3 --- Binary Relation Inference Network --- p.10
Chapter 2 --- Binary Relation Inference Network --- p.12
Chapter 2.1 --- Binary Relation Inference Network --- p.12
Chapter 2.1.1 --- Network Structure --- p.14
Chapter 2.2 --- Shortest Path Problem --- p.17
Chapter 2.2.1 --- Problem Statement --- p.17
Chapter 2.2.2 --- A Binary Relation Inference Network Solution --- p.18
Chapter 3 --- A Binary Relation Inference Network Prototype --- p.21
Chapter 3.1 --- The Prototype --- p.22
Chapter 3.1.1 --- The Network --- p.22
Chapter 3.1.2 --- Computational Element --- p.22
Chapter 3.1.3 --- Network Response Time --- p.27
Chapter 3.2 --- Improving Response --- p.29
Chapter 3.2.1 --- Removing Feedback --- p.29
Chapter 3.2.2 --- Selecting Minimum with Diodes --- p.30
Chapter 3.3 --- Speeding Up the Network Response --- p.33
Chapter 3.4 --- Conclusion --- p.35
Chapter 4 --- VLSI Building Blocks --- p.36
Chapter 4.1 --- The Site --- p.37
Chapter 4.2 --- The Unit --- p.40
Chapter 4.2.1 --- A Minimum Finding Circuit --- p.40
Chapter 4.2.2 --- A Tri-state Comparator --- p.44
Chapter 4.3 --- The Computational Element --- p.45
Chapter 4.3.1 --- Network Performances --- p.46
Chapter 4.4 --- Discussion --- p.47
Chapter 5 --- A VLSI Chip --- p.48
Chapter 5.1 --- Spatial Configuration --- p.49
Chapter 5.2 --- Layout --- p.50
Chapter 5.2.1 --- Computational Elements --- p.50
Chapter 5.2.2 --- The Network --- p.52
Chapter 5.2.3 --- I/O Requirements --- p.53
Chapter 5.2.4 --- Optional Modules --- p.53
Chapter 5.3 --- A Scalable Design --- p.54
Chapter 6 --- The Inverse Shortest Paths Problem --- p.57
Chapter 6.1 --- Problem Statement --- p.59
Chapter 6.2 --- The Embedded Approach --- p.63
Chapter 6.2.1 --- The Formulation --- p.63
Chapter 6.2.2 --- The Algorithm --- p.65
Chapter 6.3 --- Implementation Results --- p.66
Chapter 6.4 --- Other Implementations --- p.67
Chapter 6.4.1 --- Sequential Machine --- p.67
Chapter 6.4.2 --- Parallel Machine --- p.68
Chapter 6.5 --- Discussion --- p.68
Chapter 7 --- Closed Semiring Optimization Circuits --- p.71
Chapter 7.1 --- Transitive Closure Problem --- p.72
Chapter 7.1.1 --- Problem Statement --- p.72
Chapter 7.1.2 --- Inference Network Solutions --- p.73
Chapter 7.2 --- Closed Semirings --- p.76
Chapter 7.3 --- Closed Semirings and the Binary Relation Inference Network --- p.79
Chapter 7.3.1 --- Minimum Spanning Tree --- p.80
Chapter 7.3.2 --- VLSI Implementation --- p.84
Chapter 7.4 --- Conclusion --- p.86
Chapter 8 --- Conclusions --- p.87
Chapter 8.1 --- Summary of Achievements --- p.87
Chapter 8.2 --- Future Work --- p.89
Chapter 8.2.1 --- VLSI Fabrication --- p.89
Chapter 8.2.2 --- Network Robustness --- p.90
Chapter 8.2.3 --- Inference Network Applications --- p.91
Chapter 8.2.4 --- Architecture for the Bellman-Ford Algorithm --- p.91
Bibliography --- p.92
Appendices --- p.99
Chapter A --- Detailed Schematic --- p.99
Chapter A.1 --- Schematic of the Inference Network Structures --- p.99
Chapter A.1.1 --- Unit with Self-Feedback --- p.99
Chapter A.1.2 --- Unit with Self-Feedback Removed --- p.100
Chapter A.1.3 --- Unit with a Compact Minimizer --- p.100
Chapter A.1.4 --- Network Modules --- p.100
Chapter A.2 --- Inference Network Interface Circuits --- p.100
Chapter B --- Circuit Simulation and Layout Tools --- p.107
Chapter B.1 --- Circuit Simulation --- p.107
Chapter B.2 --- VLSI Circuit Design --- p.110
Chapter B.3 --- VLSI Circuit Layout --- p.111
Chapter C --- The Conjugate-Gradient Descent Algorithm --- p.113
Chapter D --- Shortest Path Problem on MasPar --- p.115
33
Srinivas, Suraj. "Learning Compact Architectures for Deep Neural Networks." Thesis, 2017. http://etd.iisc.ernet.in/2005/3581.
Full textAbstract:
Deep neural networks with millions of parameters are at the heart of many state of the art computer vision models. However, recent works have shown that models with much smaller number of parameters can often perform just as well. A smaller model has the advantage of being faster to evaluate and easier to store - both of which are crucial for real-time and embedded applications. While prior work on compressing neural networks have looked at methods based on sparsity, quantization and factorization of neural network layers, we look at the alternate approach of pruning neurons.
Training Neural Networks is often described as a kind of `black magic', as successful training requires setting the right hyper-parameter values (such as the number of neurons in a layer, depth of the network, etc ). It is often not clear what these values should be, and these decisions often end up being either ad-hoc or driven through extensive experimentation. It would be desirable to automatically set some of these hyper-parameters for the user so as to minimize trial-and-error. Combining this objective with our earlier preference for smaller models, we ask the following question - for a given task, is it possible to come up with small neural network architectures automatically? In this thesis, we propose methods to achieve the same.
The work is divided into four parts. First, given a neural network, we look at the problem of identifying important and unimportant neurons. We look at this problem in a data-free setting, i.e; assuming that the data the neural network was trained on, is not available. We propose two rules for identifying wasteful neurons and show that these suffice in such a data-free setting. By removing neurons based on these rules, we are able to reduce model size without significantly affecting accuracy.
Second, we propose an automated learning procedure to remove neurons during the process of training. We call this procedure ‘Architecture-Learning’, as this automatically discovers the optimal width and depth of neural networks. We empirically show that this procedure is preferable to trial-and-error based Bayesian Optimization procedures for selecting neural network architectures.
Third, we connect ‘Architecture-Learning’ to a popular regularize called ‘Dropout’, and propose a novel regularized which we call ‘Generalized Dropout’. From a Bayesian viewpoint, this method corresponds to a hierarchical extension of the Dropout algorithm. Empirically, we observe that Generalized Dropout corresponds to a more flexible version of Dropout, and works in scenarios where Dropout fails.
Finally, we apply our procedure for removing neurons to the problem of removing weights in a neural network, and achieve state-of-the-art results in scarifying neural networks.
34
Lian, Chi-Li, and 連崔立. "Using Probabilistic Neural Networks and Binary Sequence Algorithm to Build Financial Prediction Models - A Case of the Electronic Industry in Taiwan." Thesis, 2008. http://ndltd.ncl.edu.tw/handle/3t54vc.
Full textAbstract:
碩士國立臺北科技大學
工業工程與管理研究所
96
This research attempts to use probabilistic neural networks(PNN) and binary sequence algorithm(BSA) to build financial prediction models, regard listed company as the research object, take three annual financial materials of company. The main purpose to build this financial prediction models, lie in finding the potential financial crisis inside enterprises ahead of time, offer investors and electronic industry one to consult alert news by this. This research is divided into two stages and built the model, the first stage is to use two kinds of data type and four kinds of period to build financial classification model, elect the best model to produce the classifying value, The second stage is to rise from these classifying value prediction pattern through BSA, predicting via these prediction patterns. Looked by the real example result, with appropriate prediction pattern, can offer better prediction result.
35
Lin, Zhouhan. "Deep neural networks for natural language processing and its acceleration." Thèse, 2019. http://hdl.handle.net/1866/23438.
Full textAbstract:
Cette thèse par article comprend quatre articles qui contribuent au domaine de l'apprentissage profond, en particulier à l'accélération de l’apprentissage par le biais de réseaux à faible précision et à l'application de réseaux de neurones profonds au traitement du langage naturel.
Dans le premier article, nous étudions un schéma d’entraînement de réseau de neurones qui élimine la plupart des multiplications en virgule flottante. Cette approche consiste à binariser ou à ternariser les poids dans la propagation en avant et à quantifier les états cachés dans la propagation arrière, ce qui convertit les multiplications en changements de signe et en décalages binaires. Les résultats expérimentaux sur des jeux de données de petite à moyenne taille montrent que cette approche produit des performances encore meilleures que l’approche standard de descente de gradient stochastique, ouvrant la voie à un entraînement des réseaux de neurones rapide et efficace au niveau du matériel.
Dans le deuxième article, nous avons proposé un mécanisme structuré d’auto-attention d’enchâssement de phrases qui extrait des représentations interprétables de phrases sous forme matricielle. Nous démontrons des améliorations dans 3 tâches différentes: le profilage de l'auteur, la classification des sentiments et l'implication textuelle. Les résultats expérimentaux montrent que notre modèle génère un gain en performance significatif par rapport aux autres méthodes d’enchâssement de phrases dans les 3 tâches.
Dans le troisième article, nous proposons un modèle hiérarchique avec graphe de calcul dynamique, pour les données séquentielles, qui apprend à construire un arbre lors de la lecture de la séquence. Le modèle apprend à créer des connexions de saut adaptatives, ce qui facilitent l'apprentissage des dépendances à long terme en construisant des cellules récurrentes de manière récursive. L’entraînement du réseau peut être fait soit par entraînement supervisée en donnant des structures d’arbres dorés, soit par apprentissage par renforcement. Nous proposons des expériences préliminaires dans 3 tâches différentes: une nouvelle tâche d'évaluation de l'expression mathématique (MEE), une tâche bien connue de la logique propositionnelle et des tâches de modélisation du langage. Les résultats expérimentaux montrent le potentiel de l'approche proposée.
Dans le quatrième article, nous proposons une nouvelle méthode d’analyse par circonscription utilisant les réseaux de neurones. Le modèle prédit la structure de l'arbre d'analyse en prédisant un scalaire à valeur réelle, soit la distance syntaxique, pour chaque position de division dans la phrase d'entrée. L'ordre des valeurs relatives de ces distances syntaxiques détermine ensuite la structure de l'arbre d'analyse en spécifiant l'ordre dans lequel les points de division seront sélectionnés, en partitionnant l'entrée de manière récursive et descendante. L’approche proposée obtient une performance compétitive sur le jeu de données Penn Treebank et réalise l’état de l’art sur le jeu de données Chinese Treebank.This thesis by article consists of four articles which contribute to the field of deep learning, specifically in the acceleration of training through low-precision networks, and the application of deep neural networks on natural language processing. In the first article, we investigate a neural network training scheme that eliminates most of the floating-point multiplications. This approach consists of binarizing or ternarizing the weights in the forward propagation and quantizing the hidden states in the backward propagation, which converts multiplications to sign changes and binary shifts. Experimental results on datasets from small to medium size show that this approach result in even better performance than standard stochastic gradient descent training, paving the way to fast, hardware-friendly training of neural networks. In the second article, we proposed a structured self-attentive sentence embedding that extracts interpretable sentence representations in matrix form. We demonstrate improvements on 3 different tasks: author profiling, sentiment classification and textual entailment. Experimental results show that our model yields a significant performance gain compared to other sentence embedding methods in all of the 3 tasks. In the third article, we propose a hierarchical model with dynamical computation graph for sequential data that learns to construct a tree while reading the sequence. The model learns to create adaptive skip-connections that ease the learning of long-term dependencies through constructing recurrent cells in a recursive manner. The training of the network can either be supervised training by giving golden tree structures, or through reinforcement learning. We provide preliminary experiments in 3 different tasks: a novel Math Expression Evaluation (MEE) task, a well-known propositional logic task, and language modelling tasks. Experimental results show the potential of the proposed approach. In the fourth article, we propose a novel constituency parsing method with neural networks. The model predicts the parse tree structure by predicting a real valued scalar, named syntactic distance, for each split position in the input sentence. The order of the relative values of these syntactic distances then determine the parse tree structure by specifying the order in which the split points will be selected, recursively partitioning the input, in a top-down fashion. Our proposed approach was demonstrated with competitive performance on Penn Treebank dataset, and the state-of-the-art performance on Chinese Treebank dataset.
36
Gurioli, Gianmarco. "Adaptive Regularisation Methods under Inexact Evaluations for Nonconvex Optimisation and Machine Learning Applications." Doctoral thesis, 2021. http://hdl.handle.net/2158/1238314.
Full textAbstract:
The major aim of this research thesis is to handle two main challenges arising when solving unconstrained optimisation problems with second-order methods: the reduction of the per-iteration cost and the stochastic analysis of the resulting non- deterministic algorithms. This is motivated by the fact that second-order procedures can be more efficient than first-order ones on badly scaled and ill-conditioned problems, since they seem to potentially take advantage of curvature information to easier escape from saddle points, being more robust to the choice of hyperparameters and the parameters tuning, but at the price of a more expensive per-iteration cost, due to the computation of Hessian-vector products. Furthermore, the effort of reducing such a cost with inexact function and/or derivatives evaluations, that have to fulfill suitable accuracy requirements, leads to non-deterministic variants of the methods, that have to be supported by a stochastic complexity analysis. The thesis builds on a particular class of second-order globally convergent methods based on the Adaptive Cubic Regularisation (ARC) framework, motivated by the fact that its complexity, in terms of the worst-case number of iterations to reach a first-order critical point, has been proved to be optimal. To this purpose, the design, analysis and development of novel variants of ARC methods, employing inexact derivatives and/or function. evaluations, are investigated. To start with, a suitable reference version of the ARC method is firstly introduced, obtained by merging existing basic forms of ARC algorithms, in order to set the general background on adaptive cubic regularisation. Having set the scene, we then cope with the need of introducing inexactness in function and derivatives computations while conserving optimal complexity. After setting the finite-sum minimisation framework, this starts with the employment of inexact Hessian information, adaptively chosen, before moving on to an extended framework based on function estimates and approximate derivatives evaluations. The stochastic complexity analysis of the presented frameworks is thus performed. Finally, numerical tests within the context of supervised learning are reported, ranging from popular machine learning datasets to a real-life machine learning industrial application related to the parametric design of centrifugal pumps.
37
Croon, Dennis Gerardus. "The outperformance of the semantic learning machine, against commonly used algorithms, for binary and multi-class medical image classification: combined with the usage of feature extraction by several convolutional neural networks." Master's thesis, 2020. http://hdl.handle.net/10362/103901.
Full textAbstract:
Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced AnalyticsExtensive recent research has shown the importance of innovation in medical healthcare, with a focus on Pneumonia. It is vital and lifesaving to predict Pneumonia cases as fast as possible and preferably in advance of the symptoms. An online database source managed to gather Pneumonia-specific image data, with not just the presence of the infection, but also the nature of it, divided in bacterial- and viral infection. The first achievement is extracting valuable information from the X-Ray image datasets. Using several ImageNet pre-trained CNNs, knowledge can be gained from images and transferred to numeric arrays. This, both binary and multi-class classification data, requires a sophisticated prediction algorithm that recognizes X-Ray image patterns. Multiple, recently performed experiments show promising results about the innovative Semantic Learning Machine (SLM) that is essentially a geometric semantic hill climber for feedforward Neural Networks. This SLM is based on a derivation of the Geometric Semantic Genetic Programming (GSGP) mutation operator for real-value semantics. To prove the outperformance of the binary and multi-class SLM in general, a selection of commonly used algorithms is necessary in this research. A comprehensive hyperparameter optimization is performed for commonly used algorithms for those kinds of real-life problems, such as: Random Forest, Support Vector Machine, KNearestNeighbors and Neural Networks. The results of the SLM are promising for the Pneumonia application but could be used for all types of predictions based on images in combination with the CNN feature extractions.
Uma extensa pesquisa recente mostrou a importância da inovação na assistência médica, com foco na pneumonia. É vital e salva-vidas prever os casos de pneumonia o mais rápido possível e, de preferência, antes dos sintomas. Uma fonte on-line conseguiu coletar dados de imagem específicos da pneumonia, identificando não apenas a presença da infecção, mas também seu tipo, bacteriana ou viral. A primeira conquista é extrair informações valiosas dos conjuntos de dados de imagem de raios-X. Usando várias CNNs pré-treinadas da ImageNet, é possível obter conhecimento das imagens e transferi-las para matrizes numéricas. Esses dados de classificação binários e multi-classe requerem um sofisticado algoritmo de predição que reconhece os padrões de imagem de raios-X. Vários experimentos realizados recentemente mostram resultados promissores sobre a inovadora Semantic Learning Machine (SLM), que é essencialmente um hill climber semântico geométrico para feedforward neural network. Esse SLM é baseado em uma derivação do operador de mutação da Geometric Semantic Genetic Programming (GSGP) para valor-reais semânticos. Para provar o desempenho superior do SLM binário e multi-classe em geral, é necessária uma seleção de algoritmos mais comuns na pesquisa. Uma otimização abrangente dos hiperparâmetros é realizada para algoritmos comumente utilizados para esses tipos de problemas na vida real, como Random Forest, Support Vector Machine,K-Nearest Neighbors and Neural Networks. Os resultados do SLM são promissores para o aplicativo pneumonia, mas podem ser usados para todos os tipos de previsões baseadas em imagens em combinação com as extrações de recursos da CNN.
38
Silva, André de Vasconcelos Santos. "Sparse distributed representations as word embeddings for language understanding." Master's thesis, 2018. http://hdl.handle.net/10071/18245.
Full textAbstract:
Word embeddings are vector representations of words that capture semantic and syntactic
similarities between them. Similar words tend to have closer vector representations in a N
dimensional space considering, for instance, Euclidean distance between the points associated
with the word vector representations in a continuous vector space. This property, makes word
embeddings valuable in several Natural Language Processing tasks, from word analogy and
similarity evaluation to the more complex text categorization, summarization or translation tasks.
Typically state of the art word embeddings are dense vector representations, with low
dimensionality varying from tens to hundreds of floating number dimensions, usually obtained
from unsupervised learning on considerable amounts of text data by training and optimizing an
objective function of a neural network.
This work presents a methodology to derive word embeddings as binary sparse vectors, or word
vector representations with high dimensionality, sparse representation and binary features (e.g.
composed only by ones and zeros). The proposed methodology tries to overcome some
disadvantages associated with state of the art approaches, namely the size of corpus needed for
training the model, while presenting comparable evaluations in several Natural Language
Processing tasks.
Results show that high dimensionality sparse binary vectors representations, obtained from a
very limited amount of training data, achieve comparable performances in similarity and
categorization intrinsic tasks, whereas in analogy tasks good results are obtained only for nouns
categories. Our embeddings outperformed eight state of the art word embeddings in word
similarity tasks, and two word embeddings in categorization tasks.A designação word embeddings refere-se a representações vetoriais das palavras que capturam as similaridades semânticas e sintáticas entre estas. Palavras similares tendem a ser representadas por vetores próximos num espaço N dimensional considerando, por exemplo, a distância Euclidiana entre os pontos associados a estas representações vetoriais num espaço vetorial contínuo. Esta propriedade, torna as word embeddings importantes em várias tarefas de Processamento Natural da Língua, desde avaliações de analogia e similaridade entre palavras, às mais complexas tarefas de categorização, sumarização e tradução automática de texto. Tipicamente, as word embeddings são constituídas por vetores densos, de dimensionalidade reduzida. São obtidas a partir de aprendizagem não supervisionada, recorrendo a consideráveis quantidades de dados, através da otimização de uma função objetivo de uma rede neuronal. Este trabalho propõe uma metodologia para obter word embeddings constituídas por vetores binários esparsos, ou seja, representações vetoriais das palavras simultaneamente binárias (e.g. compostas apenas por zeros e uns), esparsas e com elevada dimensionalidade. A metodologia proposta tenta superar algumas desvantagens associadas às metodologias do estado da arte, nomeadamente o elevado volume de dados necessário para treinar os modelos, e simultaneamente apresentar resultados comparáveis em várias tarefas de Processamento Natural da Língua. Os resultados deste trabalho mostram que estas representações, obtidas a partir de uma quantidade limitada de dados de treino, obtêm performances consideráveis em tarefas de similaridade e categorização de palavras. Por outro lado, em tarefas de analogia de palavras apenas se obtém resultados consideráveis para a categoria gramatical dos substantivos. As word embeddings obtidas com a metodologia proposta, e comparando com o estado da arte, superaram a performance de oito word embeddings em tarefas de similaridade, e de duas word embeddings em tarefas de categorização de palavras.
39
Venables, Anne. "Ecological and biological modeling for natural resource management: applications to wetland classification and evaluation." Thesis, 2014. https://vuir.vu.edu.au/25869/.
Full textAbstract:
The goal of wetland assessment is to identify and quantify the condition of wetlands, taking into account the presences of threats likely to impact the services and functions the wetlands provide. There are a wide variety of methods available for undertaking wetland assessment; most rely on data collection across a broad range of attributes at wetland sites to gauge wetland condition. This thesis examines the practice of wetland assessment in West Gippsland, south-eastern Australia and it investigates the contribution, and potencies, of component biological, chemical, hydrological and physical data inputs, individually and collectively, to the identification of high social, economic and environmental value wetlands in the region. A systematic analysis using statistics and data-mining techniques was undertaken of the inventory data for 163 representative wetlands to discover pertinent relationships between the values of different site characteristics and the classification of high-value wetlands. Binary logistic regression and neural networks were used to build models mimicking the wetland assessment process, and an assessment of their abilities to do so was conducted. The influences of two wetland classification schemes: Corrick and Norman (1980) scheme, and Ecological Vegetation Classes (EVCs), on the naming of high-value wetlands were also investigated.