Dissertations / Theses on the topic 'DECISION TRESS'
To see the other types of publications on this topic, follow the link: DECISION TRESS.
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 dissertations / theses for your research on the topic 'DECISION TRESS.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse dissertations / theses on a wide variety of disciplines and organise your bibliography correctly.
1
Kustra, Rafal. "Soft decision trees." Thesis, National Library of Canada = Bibliothèque nationale du Canada, 1997. http://www.collectionscanada.ca/obj/s4/f2/dsk2/ftp04/mq28745.pdf.
Full text
2
Máša, Petr. "Finding Optimal Decision Trees." Doctoral thesis, Vysoká škola ekonomická v Praze, 2006. http://www.nusl.cz/ntk/nusl-456.
Full textAbstract:
Rozhodovácí stromy jsou rozšířenou technikou pro popis dat. Používají se často teké pro predikace. Zajímavým problémemje, že konkrétní distribuce může být popsána jedním či více rozhodovacími stromy.Obvykle nás zajímá co nejjednodušší rozhodovací strom(který budeme nazývat též optimální rozhodovací strom).Tato práce navrhuje rozšíření prořezávácí fáze algoritmů pro rozhodovací stromytak, aby umožňovala více prořezávání. V práci byly zkoumány teoretické i praktické vlastnosti tohoto rozšířeného algoritmu. Jako hlavní teoretický výsledek bylo dokázano, že pro jistou třídu distribucí nalezne algoritmus optimální rozhodovací strom(tj.nejmenší rozhodovací strom, který reprezentuje danou distribuci). V praktických testech bylo zkoumáno, jak je schopen algoritmus rekonstruovat známý strom z dat. Zajímalo nás, zdali dosáhne naše rozšíření zlepšení v počtu správně rekonstruovaných stromů zejména v případě, že data jsou dodatečně velká ( z hlediska počtu záznamů). Tato doměnka byla potvrzena praktickými testy. Obdobný výsledek byl před několika lety prokázán pro Bayesovské sítě. Algoritmus navržený v této disertační práci je polynomiální v počtu listů stromu, který je výstupem hladového algoritmu pro růst stromů, což je vylepšení oproti jednoduchému algoritmu prohledávání všech možných stromů, který je exponenciální.
3
Minguillón, Alfonso Julià. "On cascading small decision trees." Doctoral thesis, Universitat Autònoma de Barcelona, 2002. http://hdl.handle.net/10803/3027.
Full textAbstract:
Aquesta tesi tracta sobre la utilització d'arbres de decisió petits per a la classificació i la mineria de dades. La idea intuïtiva darrera d'aquesta tesi és que una seqüència d'arbres de decisió petits pot rendir millor que un arbre de decisió gran, reduint tan el cost d'entrenament com el d'explotació.El nostre primer objectiu va ser desenvolupar un sistema capaç de reconèixer diferents tipus d'elements presents en un document com ara el fons, text, línies horitzontals i verticals, dibuixos esquemàtics i imatges. Aleshores, cada element pot ser tractat d'acord a les seves característiques. Per exemple, el fons s'elimina i no és processat, mentre que les altres regions serien comprimides usant l'algorisme apropiat, JPEG amb pèrdua per a les imatges i un mètode sense pèrdua per a la resta, per exemple. Els primers experiments usant arbres de decisió varen mostrar que els arbres de decisió construïts eren massa grans i que patien de sobre-entrenament. Aleshores, vàrem tractar d'aprofitar la redundància espacial present en les imatges, utilitzant una aproximació de resolució múltiple: si un bloc gran no pot ser correctament classificat, trencar-lo en quatre sub-blocs i repetir el procés recursivament per a cada sub-bloc, usant tot el coneixement que s'hagi calculat amb anterioritat. Els blocs que no poden ser processats per una mida de bloc donada s'etiqueten com a "mixed", pel que la paraula progressiu pren sentit: una primera versió de poca resolució de la imatge classificada és obtinguda amb el primer classificador, i és refinada pel segon, el tercer, etc., fins que una versió final és obtinguda amb l'últim classificador del muntatge. De fet, l'ús de l'esquema progressiu porta a l'ús d'arbres de decisió més petits, ja que ja no cal un classificador complex. En lloc de construir un classificador gran i complex per a classificar tot el conjunt d'entrenament, només provem de resoldre la part més fàcil del problema de classificació, retardant la resta per a un segon classificador, etc.
La idea bàsica d'aquesta tesi és, doncs, un compromís entre el cost i la precisió sota una restricció de confiança. Una primera classificació es efectuada a baix cost; si un element és classificat amb una confiança elevada, s'accepta, i si no ho és, es rebutja i s'efectua una segona classificació, etc. És bàsicament, una variació del paradigma de "cascading", on un primer classificador s'usa per a calcular informació addicional per a cada element d'entrada, que serà usada per a millorar la precisió de classificació d'un segon classificador, etc. El que presentem en aquesta tesi és, bàsicament, una extensió del paradigma de "cascading" i una avaluació empírica exhaustiva dels paràmetres involucrats en la creació d'arbres de decisió progressius. Alguns aspectes teòrics relacionats als arbres de decisió progressius com la complexitat del sistema, per exemple, també són tractats.
This thesis is about using small decision trees for classification and data mining. The intuitive idea behind this thesis is that a sequence of small decision trees may perform better than a large decision tree, reducing both training and exploitation costs.
Our first goal was to develop a system capable to recognize several kinds of elements present in a document such as background, text, horizontal and vertical lines, line drawings and images. Then, each element would be treated accordingly to its characteristics. For example, background regions would be removed and not processed at all, while the other regions would be compressed using an appropriate algorithm, the lossy JPEG standard operation mode for images and a lossless method for the rest, for instance. Our first experiments using decision trees showed that the decision trees we built were too large and they suffered from overfitting. Then, we tried to take advantage of spatial redundancy present in images, using a multi-resolution approach: if a large block cannot be correctly classified, split it in four subblocks and repeat the process recursively for each subblock, using all previous computed knowledge about such block. Blocks that could not be processed at a given block size were labeled as mixed, so the word progressive came up: a first low resolution version of the classified image is obtained with the first classifier, and it is refined by the second one, the third one, etc, until a final version is obtained with the last classifier in the ensemble. Furthermore, the use of the progressive scheme yield to the use of smaller decision trees, as we no longer need a complex classifier. Instead of building a large and complex classifier for classifying the whole input training set, we only try to solve the easiest part of the classification problem, delaying the rest for a second classifier, and so.
The basic idea in this thesis is, therefore, a trade-off between cost and accuracy under a confidence constraint. A first classification is performed at a low cost; if an element is classified with a high confidence, it is accepted, and if not, it is rejected and a second classification is performed, and so. It is, basically, a variation of the cascading paradigm, where a first classifier is used to compute additional information from each input sample, information that will be used to improve classification accuracy by a second classifier, and so on. What we present in this thesis, basically, is an extension of the cascading paradigm and an exhaustive empirical evaluation of the parameters involved in the creation of progressive decision trees. Some basic theoretical issues related to progressive decision trees such as system complexity, for example, are also addressed.
Esta tesis trata sobre la utilización de árboles de decisión pequeños para la clasificación y la minería de datos. La idea intuitiva detrás de esta tesis es que una secuencia de árboles de decisión pequeños puede rendir mejor que un árbol de decisión grande, reduciendo tanto el coste de entrenamiento como el de explotación.
Nuestro primer objetivo fue desarrollar un sistema capaz de reconocer diferentes tipos de elementos presentes en un documento, como el fondo, texto, líneas horizontales y verticales, dibujos esquemáticos y imágenes. Entonces, cada elemento puede ser tratado de acuerdo a sus características. Por ejemplo, el fondo se elimina y no se procesa, mientras que las otras regiones serían comprimidas usando el algoritmo apropiado, JPEG con pérdida para las imágenes y un método sin pérdida para el resto, por ejemplo. Los primeros experimentos usando árboles de decisión mostraron que los árboles de decisión construidos eran demasiado grandes y que sufrían de sobre-entrenamiento. Entonces, se trató de aprovechar la redundancia espacial presente en las imágenes, utilizando una aproximación de resolución múltiple: si un bloque grande no puede ser correctamente clasificado, romperlo en cuatro sub-bloques y repetir el proceso recursivamente para cada sub-bloque, usando todo el conocimiento que se haya calculado con anterioridad. Los bloques que no pueden ser procesados para una medida de bloque dada se etiquetan como "mixed", por lo que la palabra progresivo toma sentido: una primera versión de poca resolución de la imagen clasificada se obtiene con el primer clasificador, y se refina por el segundo, el tercero, etc., hasta que una versión final es obtenida con el último clasificador del montaje. De hecho, el uso del esquema progresivo lleva al uso de árboles de decisión más pequeños, ya que ya no es necesario un clasificador complejo. En lugar de construir un clasificador grande y complejo para clasificar todo el conjunto de entrenamiento, sólo tratamos de resolver la parte más fácil del problema de clasificación, retardando el resto para un segundo clasificador, etc.
La idea básica de esta tesis es, entonces, un compromiso entre el coste y la precisión bajo una restricción de confianza. Una primera clasificación es efectuada a bajo coste; si un elemento es clasificado con una confianza elevada, se acepta, y si no lo es, se rechaza y se efectúa una segunda clasificación, etc. Es básicamente, una variación del paradigma de "cascading", donde un primer clasificador se usa para calcular información adicional para cada elemento de entrada, que será usada para mejorar la precisión de clasificación de un segundo clasificador, etc. Lo que presentamos en esta tesis es, básicamente, una extensión del paradigma de "cascading" y una evaluación empírica exhaustiva de los parámetros involucrados en la creación de árboles de decisión progresivos. Algunos aspectos teóricos relacionados con los árboles de decisión progresivos como la complejidad del sistema, por ejemplo, también son tratados.
4
Pisetta, Vincent. "New Insights into Decision Trees Ensembles." Thesis, Lyon 2, 2012. http://www.theses.fr/2012LYO20018/document.
Full textAbstract:
Les ensembles d’arbres constituent à l’heure actuelle l’une des méthodes d’apprentissage statistique les plus performantes. Toutefois, leurs propriétés théoriques, ainsi que leurs performances empiriques restent sujettes à de nombreuses questions. Nous proposons dans cette thèse d’apporter un nouvel éclairage à ces méthodes. Plus particulièrement, après avoir évoqué les aspects théoriques actuels (chapitre 1) de trois schémas ensemblistes principaux (Forêts aléatoires, Boosting et Discrimination Stochastique), nous proposerons une analyse tendant vers l’existence d’un point commun au bien fondé de ces trois principes (chapitre 2). Ce principe tient compte de l’importance des deux premiers moments de la marge dans l’obtention d’un ensemble ayant de bonnes performances. De là, nous en déduisons un nouvel algorithme baptisé OSS (Oriented Sub-Sampling) dont les étapes sont en plein accord et découlent logiquement du cadre que nous introduisons. Les performances d’OSS sont empiriquement supérieures à celles d’algorithmes en vogue comme les Forêts aléatoires et AdaBoost. Dans un troisième volet (chapitre 3), nous analysons la méthode des Forêts aléatoires en adoptant un point de vue « noyau ». Ce dernier permet d’améliorer la compréhension des forêts avec, en particulier la compréhension et l’observation du mécanisme de régularisation de ces techniques. Le fait d’adopter un point de vue noyau permet d’améliorer les Forêts aléatoires via des méthodes populaires de post-traitement comme les SVM ou l’apprentissage de noyaux multiples. Ceux-ci démontrent des performances nettement supérieures à l’algorithme de base, et permettent également de réaliser un élagage de l’ensemble en ne conservant qu’une petite partie des classifieurs le composantDecision trees ensembles are among the most popular tools in machine learning. Nevertheless, their theoretical properties as well as their empirical performances are subject to strong investigation up to date. In this thesis, we propose to shed light on these methods. More precisely, after having described the current theoretical aspects of three main ensemble schemes (chapter 1), we give an analysis supporting the existence of common reasons to the success of these three principles (chapter 2). This last takes into account the two first moments of the margin as an essential ingredient to obtain strong learning abilities. Starting from this rejoinder, we propose a new ensemble algorithm called OSS (Oriented Sub-Sampling) whose steps are in perfect accordance with the point of view we introduce. The empirical performances of OSS are superior to the ones of currently popular algorithms such as Random Forests and AdaBoost. In a third chapter (chapter 3), we analyze Random Forests adopting a “kernel” point of view. This last allows us to understand and observe the underlying regularization mechanism of these kinds of methods. Adopting the kernel point of view also enables us to improve the predictive performance of Random Forests using popular post-processing techniques such as SVM and multiple kernel learning. In conjunction with random Forests, they show greatly improved performances and are able to realize a pruning of the ensemble by conserving only a small fraction of the initial base learners
5
Wickramarachchi, Darshana Chitraka. "Oblique decision trees in transformed spaces." Thesis, University of Canterbury. Mathematics and Statistics, 2015. http://hdl.handle.net/10092/11051.
Full textAbstract:
Decision trees (DTs) play a vital role in statistical modelling. Simplicity and interpretability of the solution structure have made the method popular in a wide range of disciplines. In data classification problems, DTs recursively partition the feature space into disjoint sub-regions until each sub-region becomes homogeneous with respect to a particular class. Axis parallel splits, the simplest form of splits, partition the feature space parallel to feature axes. However, for some problem domains DTs with axis parallel splits can produce complicated boundary structures. As an alternative, oblique splits are used to partition the feature space potentially simplifying the boundary structure. Various approaches have been explored to find optimal oblique splits. One approach is based on optimisation techniques. This is considered the benchmark approach, however, its major limitation is that the tree induction algorithm is computationally expensive. On the other hand, split finding approaches based on heuristic arguments have gained popularity and have made improvements on benchmark methods. This thesis proposes a methodology to induce oblique decision trees in transformed spaces based on a heuristic argument.
As the first goal of the thesis, a new oblique decision tree algorithm, called HHCART (\underline{H}ouse\underline{H}older \underline{C}lassification and \underline{R}egression \underline{T}ree) is proposed. The proposed algorithm utilises a series of Householder matrices to reflect the training data at each non-terminal node during the tree construction. Householder matrices are constructed using the eigenvectors from each classes' covariance matrix. Axis parallel splits in the reflected (or transformed) spaces provide an efficient way of finding oblique splits in the original space. Experimental results show that the accuracy and size of the HHCART trees are comparable with some benchmark methods in the literature. The appealing features of HHCART is that it can handle both qualitative and quantitative features in the same oblique split, conceptually simple and computationally efficient.
Data mining applications often come with massive example sets and inducing oblique DTs for such example sets often consumes considerable time. HHCART is a serial computing memory resident algorithm which may be ineffective when handling massive example sets. As the second goal of the thesis parallel computing and disk resident versions of the HHCART algorithm are presented so that HHCART can be used irrespective of the size of the problem.
HHCART is a flexible algorithm and the eigenvectors defining Householder matrices can be replaced by other vectors deemed effective in oblique split finding. The third endeavour of this thesis explores this aspect of HHCART. HHCART can be used with other vectors in order to improve classification results. For example, a normal vector of the angular bisector, introduced in the Geometric Decision Tree (GDT) algorithm, is used to construct the Householder reflection matrix. The proposed method produces better results than GDT for some problem domains. In the second case, \textit{Class Representative Vectors} are introduced and used to construct Householder reflection matrices. The results of this experiment show that these oblique trees produce classification results competitive with those achieved with some benchmark decision trees.
DTs are constructed using two approaches, namely: top-down and bottom-up. HHCART is a top-down tree, which is the most common approach. As the fourth idea of the thesis, the concept of HHCART is used to induce a new DT, HHBUT, using the bottom-up approach. The bottom-up approach performs cluster analysis prior to the tree building to identify the terminal nodes. The use of the Bayesian Information Criterion (BIC) to determine the number of clusters leads to accurate and compact trees when compared with Cross Validation (CV) based bottom-up trees. We suggest that HHBUT is a good alternative to the existing bottom-up tree especially when the number of examples is much higher than the number of features.
6
Han, Qian. "Mining Shared Decision Trees between Datasets." Wright State University / OhioLINK, 2010. http://rave.ohiolink.edu/etdc/view?acc_num=wright1274807201.
Full text
7
Parkhe, Vidyamani. "Randomized decision trees for data mining." [Florida] : State University System of Florida, 2000. http://etd.fcla.edu/etd/uf/2000/ane5962/thesis.pdf.
Full textAbstract:
Thesis (M.S.)--University of Florida, 2000.Title from first page of PDF file. Document formatted into pages; contains vi, 54 p.; also contains graphics. Vita. Includes bibliographical references (p. 52-53).
8
Boujari, Tahereh. "Instance-based ontology alignment using decision trees." Thesis, Linköpings universitet, Institutionen för datavetenskap, 2012. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-84918.
Full textAbstract:
Using ontologies is a key technology in the semantic web. The semantic web helps people to store their data on the web, build vocabularies, and has written rules for handling these data and also helps the search engines to distinguish between the information they want to access in web easier. In order to use multiple ontologies created by different experts we need matchers to find the similar concepts in them to use it to merge these ontologies. Text based searches use the string similarity functions to find the equivalent concepts inside ontologies using their names.This is the method that is used in lexical matchers. But a global standard for naming the concepts in different research area does not exist or has not been used. The same name may refer to different concepts while different names may describe the same concept. To solve this problem we can use another approach for calculating the similarity value between concepts which is used in structural and constraint-based matchers. It uses relations between concepts, synonyms and other information that are stored in the ontologies. Another category for matchers is instance-based that uses additional information like documents related to the concepts of ontologies, the corpus, to calculate the similarity value for the concepts. Decision trees in the area of data mining are used for different kind of classification for different purposes. Using decision trees in an instance-based matcher is the main concept of this thesis. The results of this implemented matcher using the C4.5 algorithm are discussed. The matcher is also compared to other matchers. It also is used for combination with other matchers to get a better result.
9
Lee, Hong, and 李匡. "Model-based decision trees for ranking data." Thesis, The University of Hong Kong (Pokfulam, Hong Kong), 2010. http://hub.hku.hk/bib/B45149707.
Full text
10
Beck, Jason. "Implementation and Experimentation with C4.5 Decision Trees." Honors in the Major Thesis, University of Central Florida, 2007. http://digital.library.ucf.edu/cdm/ref/collection/ETH/id/1157.
Full textAbstract:
This item is only available in print in the UCF Libraries. If this is your Honors Thesis, you can help us make it available online for use by researchers around the world by following the instructions on the distribution consent form at http://library.ucf.edu/Systems/DigitalInitiatives/DigitalCollections/InternetDistributionConsentAgreementForm.pdf You may also contact the project coordinator, Kerri Bottorff, at kerri.bottorff@ucf.edu for more information.Bachelors
Engineering and Computer Science
Computer Engineering
11
Badr, Bashar. "Implementation of decision trees for embedded systems." Thesis, Loughborough University, 2014. https://dspace.lboro.ac.uk/2134/14711.
Full textAbstract:
This research work develops real-time incremental learning decision tree solutions suitable for real-time embedded systems by virtue of having both a defined memory requirement and an upper bound on the computation time per training vector. In addition, the work provides embedded systems with the capabilities of rapid processing and training of streamed data problems, and adopts electronic hardware solutions to improve the performance of the developed algorithm. Two novel decision tree approaches, namely the Multi-Dimensional Frequency Table (MDFT) and the Hashed Frequency Table Decision Tree (HFTDT) represent the core of this research work. Both methods successfully incorporate a frequency table technique to produce a complete decision tree. The MDFT and HFTDT learning methods were designed with the ability to generate application specific code for both training and classification purposes according to the requirements of the targeted application. The MDFT allows the memory architecture to be specified statically before learning takes place within a deterministic execution time. The HFTDT method is a development of the MDFT where a reduction in the memory requirements is achieved within a deterministic execution time. The HFTDT achieved low memory usage when compared to existing decision tree methods and hardware acceleration improved the performance by up to 10 times in terms of the execution time.
12
Silva, Jesús, Palma Hugo Hernández, Núẽz William Niebles, Alex Ruiz-Lazaro, and Noel Varela. "Natural Language Explanation Model for Decision Trees." Institute of Physics Publishing, 2020. http://hdl.handle.net/10757/652131.
Full textAbstract:
This study describes a model of explanations in natural language for classification decision trees. The explanations include global aspects of the classifier and local aspects of the classification of a particular instance. The proposal is implemented in the ExpliClas open source Web service [1], which in its current version operates on trees built with Weka and data sets with numerical attributes. The feasibility of the proposal is illustrated with two example cases, where the detailed explanation of the respective classification trees is shown.
13
Ravi, Sumved Reddy. "Naturally Generated Decision Trees for Image Classification." Thesis, Virginia Tech, 2021. http://hdl.handle.net/10919/104884.
Full textAbstract:
Image classification has been a pivotal area of research in Deep Learning, with a vast body
of literature working to tackle the problem, constantly striving to achieve higher accuracies.
This push to reach achieve greater prediction accuracy however, has further exacerbated the
black box phenomenon which is inherent of neural networks, and more for so CNN style deep
architectures. Likewise, it has lead to the development of highly tuned methods, suitable only
for a specific data sets, requiring significant work to alter given new data. Although these
models are capable of producing highly accurate predictions, we have little to no ability to
understand the decision process taken by a network to reach a conclusion. This factor poses a
difficulty in use cases such as medical diagnostics tools or autonomous vehicles, which require
insight into prediction reasoning to validate a conclusion or to debug a system. In essence,
modern applications which utilize deep networks are able to learn to produce predictions, but
lack interpretability and a deeper understanding of the data. Given this key point, we look
to decision trees, opposite in nature to deep networks, with a high level of interpretability
but a low capacity for learning. In our work we strive to merge these two techniques as a
means to maintain the capacity for learning while providing insight into the decision process.
More importantly, we look to expand the understanding of class relationships through a tree
architecture. Our ultimate goal in this work is to create a technique able to automatically
create a visual feature based knowledge hierarchy for class relations, applicable broadly to
any data set or combination thereof. We maintain these goals in an effort to move away
from specific systems and instead toward artificial general intelligence (AGI). AGI requires
a deeper understanding over a broad range of information, and more so the ability to learn
new information over time. In our work we embed networks of varying sizes and complexity
within decision trees on a node level, where each node network is responsible for selecting
the next branch path in the tree. Each leaf node represents a single class and all parent
and ancestor nodes represent groups of classes. We designed the method such that classes
are reasonably grouped by their visual features, where parent and ancestor nodes represent
hidden super classes. Our work aims to introduce this method as a small step towards
AGI, where class relations are understood through an automatically generated decision tree
(representing a class hierarchy), capable of accurate image classification.Master of Science
Many modern day applications make use of deep networks for image classification. Often these networks are incredibly complex in architecture, and applicable only for specific tasks and data. Standard approaches use just a neural network to produce predictions. However, the internal decision process of the network remains a black box due to the nature of the technique. As more complex human related applications, such as medical image diagnostic tools or autonomous driving software, are being created, they require an understanding of reasoning behind a prediction. To provide this insight into the prediction reasoning, we propose a technique which merges decision trees and deep networks. Tested on the MNIST image data set we were able to achieve an accuracy over 99.0%. We were also able to achieve an accuracy over 73.0% on the CIFAR-10 image data set. Our method is found to create decision trees that are easily understood and are reasonably capable of image classification.
14
Tadros, Alexandre. "Topological recursive fitting trees : A framework for interpretable regression extending decision trees." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-272130.
Full textAbstract:
Many real-world machine learning applications need interpretation of an algorithm output. The simplicity of some of the most fundamental machine learning algorithms for regression, such as linear regression or decision trees, facilitates interpretation. However, they fall short when facing complex (e.g. high-dimensional, nonlinear) relationships between variables. Several approaches like artificial neural networks and bagging or boosting variants of decision trees have been able to overcome this issue but at the cost of interpretation. We propose a framework called Topological Recursive Fitting (TRF) in which a decision tree is learned based on topological properties of the data. We expect the tree structure of our approach to enable interpretation while achieving comparable performance to previously mentioned blackbox methods. Results show that TRF can achieve comparable performance with those methods, even if confirmation on a more significant number of datasets should be initiated.Många verkliga maskininlärningsapplikationer behöver tolkning av resultatet från en algoritm. Enkelheten i några av de mest grundläggande maskininlärningsalgoritmerna för regression, såsom linjär regression eller beslutsträd, underlättar tolkningen. Men det räcker inte till när de möter komplexa (t.ex. högdimensionella, icke-linjära) förhållanden mellan variabler. Flera tillvägagångssätt som konstgjorda neurala nätverk och bagging eller att öka varianter av beslutsträd har kunnat övervinna denna fråga, men på bekostnad av tolkningen. Vi föreslår ett ramverk som heter Topologiskt Rekursiva Anpassning (Topological Recursive Fitting, TRF) där ett beslutsträd lärs utifrån datats topologiska egenskaper. Vi förväntar oss att trädstrukturen i vårt tillvägagångssätt möjliggör tolkning och samtidigt uppnår jämförbar prestanda med tidigare nämnda black-box-metoder. Resultaten visar att TRF kan uppnå jämförbar prestanda med dessa metoder, även om bekräftelse på ett mer betydande antal datamängder bör initieras.
15
Zhong, Mingyu. "AN ANALYSIS OF MISCLASSIFICATION RATES FOR DECISION TREES." Doctoral diss., University of Central Florida, 2007. http://digital.library.ucf.edu/cdm/ref/collection/ETD/id/2496.
Full textAbstract:
The decision tree is a well-known methodology for classification and regression. In this dissertation, we focus on the minimization of the misclassification rate for decision tree classifiers. We derive the necessary equations that provide the optimal tree prediction, the estimated risk of the tree's prediction, and the reliability of the tree's risk estimation. We carry out an extensive analysis of the application of Lidstone's law of succession for the estimation of the class probabilities. In contrast to existing research, we not only compute the expected values of the risks but also calculate the corresponding reliability of the risk (measured by standard deviations). We also provide an explicit expression of the k-norm estimation for the tree's misclassification rate that combines both the expected value and the reliability. Furthermore, our proposed and proven theorem on k-norm estimation suggests an efficient pruning algorithm that has a clear theoretical interpretation, is easily implemented, and does not require a validation set. Our experiments show that our proposed pruning algorithm produces accurate trees quickly that compares very favorably with two other well-known pruning algorithms, CCP of CART and EBP of C4.5. Finally, our work provides a deeper understanding of decision trees.Ph.D.
School of Electrical Engineering and Computer Science
Engineering and Computer Science
Electrical Engineering PhD
16
Olsson, Magnus. "Behavior Trees for decision-making in Autonomous Driving." Thesis, KTH, Skolan för datavetenskap och kommunikation (CSC), 2016. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-183060.
Full textAbstract:
This degree project investigates the suitability of using Behavior Trees (BT) as an architecture for the behavioral layer in autonomous driving. BTs originate from video game development but have received attention in robotics research the past couple of years. This project also includes implementation of a simulated traffic environment using the Unity3D engine, where the use of BTs is evaluated and compared to an implementation using finite-state machines (FSM). After the initial implementation, the simulation along with the control architectures were extended with additional behaviors in four steps. The different versions were evaluated using software maintainability metrics (Cyclomatic complexity and Maintainability index) in order to extrapolate and reason about more complex implementations as would be required in a real autonomous vehicle. It is concluded that as the AI requirements scale and grow more complex, the BTs likely become substantially more maintainable than FSMs and hence may prove a viable alternative for autonomous driving.
17
Jenhani, Ilyes. "From possibilistic similarity measures to possibilistic decision trees." Thesis, Artois, 2010. http://www.theses.fr/2010ARTO0402/document.
Full textAbstract:
Cette thèse traite deux problèmes importants dans les domaine de l'apprentissage automatique et du raisonnement dans l'incertain : comment évaluer une relation de similarité entre deux informations incertaines et comment assurer la classification \`a partir de données incertaines. Notre première principale contribution est de proposer une approche, appelée arbre de décision possibiliste, qui permet de construire des arbres de décision à partir de données d'apprentissage imparfaites. Plus précisément, elle traite des données caractérisées par des classes incertaines o\`u l'incertitude est modélisée avec la théorie des possibilités quantitative. Nous avons développé trois approches d'arbres de décision possibilistes. Pour chacune des approches, nous avons été confrontés à résoudre plusieurs problèmes pour pouvoir construire des arbres de décision possibilistes, tels que, comment définir une mesure de sélection d'attributs quand les classes sont représentes par des distributions de possibilité, comment trouver les critères d'arrêt et comment les feuilles vont être étiquetées dans ce contexte incertain. La première approche, appelée arbre de décision possibiliste basée sur la non- spécificité, utilise le concept de non-spécificité relatif à la théorie des possibilités dans la définition de sa mesure de sélection d'attributs. Cette approche maintient les distributions de possibilité durant toutes les étapes de la procédure de construction et ce particulièrement, au moment de l'évaluation de la quantité d'information apportée par chaque attribut. En revanche, la deuxième et la troisième approches, appelées arbre de décision possibiliste basé sur la similarité et arbre de décision possibiliste basé sur le clustering, éliminent automatiquement les distributions de possibilité dans leurs mesures de sélection d'attributs. Cette stratégie a permis d'étendre le ratio de gain et, par conséquent, d'étendre l'algorithme C4.5 pour qu'il puisse traiter des données libellées par des classes possibilistes. Ces deux dernières approches sont principalement basées sur le concept de similarité entre les distributions de possibilité étudié dans la thèse.La deuxième principale contribution de cette thèse concerne l'analyse des mesures de similarité en théorie des possibilités. En effet, un challenge important était de fournir une analyse des mesures de similarité possibiliste conduite par les propriétés qu'elles doivent satisfaire. Après avoir montré le rôle important de la notion d'incohérence dans l'évaluation de la similarité en théorie des possibilités, une nouvelle mesure, appelée affinité de l'information a été proposée. Cette mesure satisfait plusieurs propriétés que nous avons établies. A la fin de cette thèse, nous avons proposé des expérimentations pour comparer et montrer la faisabilité des approches d'arbres de décision possibilistes que nous avons développéesThis thesis concerns two important issues in machine learning and reasoning under uncertainty: how to evaluate a similarity relation between two uncertain pieces of information, and how to perform classification from uncertain data. Our first main contribution is to propose a so-called possibilistic decision tree which allows to induce decision trees from training data afflicted with imperfection. More precisely, it handles training data characterized by uncertain class labels where uncertainty is modeled within the quantitative possibility theory framework. We have developed three possibilistic decision tree approaches. For each approach, we were faced and solved typical questions for inducing possibilistic decision trees such as how to define an attribute selection measure when classes are represented by possibility distributions, how to find the stopping criteria and how leaves should be labeled in such uncertain context. The first approach, so-called, non-specificity-based possibilistic decision tree uses the concept of non-specificity relative to possibility theory in its attribute selection measure component. This approach keeps up the possibility distributions within all the stages of the building procedure and especially when evaluating the informativeness of the attributes in the attribute selection step. Conversely, the second and the third approaches, so-called similarity-based possibilistic decision tree and clustering-based possibilistic decision tree, automatically, get rid of possibility distributions in their attribute selection measure. This strategy has allowed them to extend the gain ratio criterion and hence to extend the C4.5 algorithm to handle possibilistic labeled data. These two possibilistic decision tree approaches are mainly based on the concept of similarity between possibility distributions.This latter issue constitutes our second main contribution in this thesis. In fact, an important challenge was to provide a property-based analysis of possibilistic similarity measures. After showing the important role that inconsistency could play in assessing possibilistic similarity, a new inconsistency-based possibilistic similarity measure, so-called information affinity has been proposed. This measure satisfies a set of natural properties that we have established. Finally, we have conducted experiments to show the feasibility and to compare the different possibilistic decision tree approaches developed in this thesis
18
Waldhauser, Christoph, and Ronald Hochreiter. "Shaking the trees: Abilities and Capabilities of Regression and Decision Trees for Political Science." edp sciences, 2017. http://dx.doi.org/10.1051/itmconf/20171400009.
Full textAbstract:
When committing to quantitative political science, a researcher has a wealth
of methods to choose from. In this paper we compare the established method of analyzing
roll call data using W-NOMINATE scores to a data-driven supervised machine learning
method: Regression and Decision Trees (RDTs). To do this, we defined two scenarios,
one pertaining to an analytical goal, the other being aimed at predicting unknown voting
behavior. The suitability of both methods is measured in the dimensions of consistency,
tolerance towards misspecification, prediction quality and overall variability. We find that
RDTs are at least as suitable as the established method, at lower computational expense
and are more forgiving with respect to misspecification.
19
Gerdes, Mike. "Predictive Health Monitoring for Aircraft Systems using Decision Trees." Licentiate thesis, Linköpings universitet, Fluida och mekatroniska system, 2014. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-105843.
Full textAbstract:
Unscheduled aircraft maintenance causes a lot problems and costs for aircraft operators. This is due to the fact that aircraft cause significant costs if flights have to be delayed or canceled and because spares are not always available at any place and sometimes have to be shipped across the world. Reducing the number of unscheduled maintenance is thus a great costs factor for aircraft operators. This thesis describes three methods for aircraft health monitoring and prediction; one method for system monitoring, one method for forecasting of time series and one method that combines the two other methods for one complete monitoring and prediction process. Together the three methods allow the forecasting of possible failures. The two base methods use decision trees for decision making in the processes and genetic optimization to improve the performance of the decision trees and to reduce the need for human interaction. Decision trees have the advantage that the generated code can be fast and easily processed, they can be altered by human experts without much work and they are readable by humans. The human readability and modification of the results is especially important to include special knowledge and to remove errors, which the automated code generation produced.
20
Twala, Bhekisipho. "Effective techniques for handling incomplete data using decision trees." Thesis, Open University, 2005. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.418465.
Full text
21
PEREIRA, FELIPE DE ALBUQUERQUE MELLO. "A FRAMEWORK FOR GENERATING BINARY SPLITS IN DECISION TREES." PONTIFÍCIA UNIVERSIDADE CATÓLICA DO RIO DE JANEIRO, 2018. http://www.maxwell.vrac.puc-rio.br/Busca_etds.php?strSecao=resultado&nrSeq=35783@1.
Full textAbstract:
PONTIFÍCIA UNIVERSIDADE CATÓLICA DO RIO DE JANEIROCONSELHO NACIONAL DE DESENVOLVIMENTO CIENTÍFICO E TECNOLÓGICO
Nesta dissertação é apresentado um framework para desenvolver critérios de split para lidar com atributos nominais multi-valorados em árvores de decisão. Critérios gerados por este framework podem ser implementados para rodar em tempo polinomial no número de classes e valores, com garantia teórica de produzir um split próximo do ótimo. Apresenta-se também um estudo experimental, utilizando datasets reais, onde o tempo de execução e acurácia de métodos oriundos do framework são avaliados.
In this dissertation we propose a framework for designing splitting criteria for handling multi-valued nominal attributes for decision trees. Criteria derived from our framework can be implemented to run in polynomial time in the number of classes and values, with theoretical guarantee of producing a split that is close to the optimal one. We also present an experimental study, using real datasets, where the running time and accuracy of the methods obtained from the framework are evaluated.
22
Gangadhara, Kanthi, and Dubbaka Sai Anusha Reddy. "Comparing Compound and Ordinary Diversity measures Using Decision Trees." Thesis, Högskolan i Borås, Institutionen Handels- och IT-högskolan, 2011. http://urn.kb.se/resolve?urn=urn:nbn:se:hb:diva-20385.
Full textAbstract:
An ensemble of classifiers succeeds in improving the accuracy of the whole when thecomponent classifiers are both diverse and accurate. Diversity is required to ensure that theclassifiers make uncorrelated errors. Theoretical and experimental approaches from previousresearch show very low correlation between ensemble accuracy and diversity measure.Introducing Proposed Compound diversity functions by Albert Hung-Ren KO and RobertSabourin, (2009), by combining diversities and performances of individual classifiers exhibitstrong correlations between the diversities and accuracy. To be consistent with existingarguments compound diversity of measures are evaluated and compared with traditionaldiversity measures on different problems. Evaluating diversity of errors and comparison withmeasures are significant in this study. The results show that compound diversity measuresare better than ordinary diversity measures. However, the results further explain evaluation ofdiversity of errors on available data.Program: Magisterutbildning i informatik
23
Manninger, Mátyás. "Optimizations for uncertainty prediction on streams with decision trees." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-235490.
Full textAbstract:
Many companies use machine learning to make predictions that are critical for their everyday operations. In many cases, knowing the uncertainty of the predictions can be crucial, as decisions are made based on calculated risk. Most uncertainty estimation methods are not designed for an online setting and cannot handle unbounded data streams or concept drift. This study proposes optimizations to two decision tree based algorithms that are specially designed to accommodate the online case. One is conformal prediction and the other is onlineQRF. Two new parameters are introduced for conformal prediction to exchange speed and accuracy. The replacement of the histograms at the leaves of the trees for onlineQRF algorithm with online quantile sketches is also proposed. These modifications are then tested on public datasets. The empirical results are analyzed in terms of speed and accuracy. The two parameters for conformal prediction did not have a significant effect on the algorithm and did not improve it in a meaningful way. Changing the data aggregation method at the leaves for onlineQRF reduced the prediction time significantly and opened up for further improvements in accuracy. This empirical study shows an improvement to a state-of-the-art online machine learning algorithm that could be adopted throughout many industries.Många företag använder maskininlärning för att göra prediktioner som är kritiska för deras vardagliga verksamhet. I många fall kan kunska- pen om prediktionernas osäkerhet vara avgörande, eftersom beslut fattas utifrån beräknad risk. De flesta osäkerhetsestimeringsmetoder- na är inte utformade för strömmande data och kan inte hantera obe- gränsade dataströmmar eller datadrift. Denna studie föreslår optime- ringar till två beslutsträdsbaserade algoritmer som är speciellt utfor- made för att tillgodose osäkerhetsprediktioner för strömmande data. En av dem är conformal prediction medan den andra är onlineQRF. Två nya parametrar introduceras för conformal prediction för att by- ta ut hastighet och noggrannhet. Utbytet av histogrammen vid trä- dens löv för onlineQRF-algoritmen med online kvantilskisser föreslås också. De två parametrarna för conformal prediction hade ingen sig- nifikant effekt på algoritmen och förbättrade den inte på något me- ningsfullt sätt. Ersättningen av aggregationsmetoden vid löven för on- lineQRF minskade förutsägelsestiden signifikant och öppnade upp för ytterligare förbättringar i noggrannhet. Denna empiriska studie visar en förbättring av en state-of-the-art online maskininlärningsalgoritm som kunde vidtas genom många branscher.
24
Svantesson, David. "Implementing Streaming Parallel Decision Trees on Graphic Processing Units." Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-230953.
Full textAbstract:
Decision trees have long been a prevalent area within machine learning. With streaming data environments as well as large datasets becoming increasingly common, researchers have developed decision tree algorithms adapted to streaming data. One such algorithm is SPDT, which approaches the streaming data problem by making use of workers on a network combined with a dynamic histogram approximation of the data. There exist several implementations for decision trees on GPU, but those are uncommon in a streaming data setting. In this research, conducted at RISE SICS, the possibilities of accelerating the SPDT algorithm on GPU is investigated. An implementation is successfully created using the CUDA platform. The implementation uses a set number of data samples per layer to better fit the GPU platform. Experiments were conducted to investigate the impact on both accuracy and speed. It is found that the GPU implementation performs as well as the CPU implementation in terms of accuracy, suggesting that using small subsets of the data in each layer is sufficient for making accurate split decisions. The GPU implementation is found to be up to 113 times faster than the reference Scala CPU implementation for one of the tested datasets, and 13 times faster on average over all the tested datasets. Weak parts of the implementation are identified, and further improvements are suggested to increase both accuracy and runtime performance.Beslutsträd har länge varit ett betydande område inom maskininlärning. Strömmandedata och stora dataset har blivit allt vanligare, vilket har lett till att forskare utvecklat algoritmer för beslutsträd anpassade till dessa miljöer. En av dessa algoritmer är SPDT. Denna algoritm använder sig av flera arbetare i ett nätverk kombinerat med en dynamisk histogram-representation av data. Det existerar flera implementationer av beslutsträd på grafikkort, men inte många för strömmande data. I detta forskningsarbete, utfört på RISE SICS, undersöks möjligheten att snabba upp SPDT genom att accelerera beräkningar med hjälp av grafikkort. En lyckad implementation skriven i CUDA beskrivs. Implementationen anpassar sig till grafikkortsplattformen genom att använda sig utav ett bestämt antal datapunkter per lager. Experiment som undersöker effekten på noggrannhet och hastighet har genomförts. Resultaten visar att GPU-implementationen presterar lika väl som CPU-implementationen vad gäller noggrannhet, vilket påvisar att användandet av en mindre del av data i varje lager är tillräckligt för goda resultat. GPU-implementationen är upp till 113 gånger snabbare jämfört med en existerande CPU-implementation skriven i Scala, och är i medel 13 gånger snabbare. Svagheter i implementationen identifieras, och vidare förbättringar till implementationen föreslås för att förbättra både noggrannhet och hastighetsprestanda.
25
Kirkby, Richard Brendon. "Improving Hoeffding Trees." The University of Waikato, 2008. http://hdl.handle.net/10289/2568.
Full textAbstract:
Modern information technology allows information to be collected at a far greater rate than ever before. So fast, in fact, that the main problem is making sense of it all. Machine learning offers promise of a solution, but the field mainly focusses on achieving high accuracy when data supply is limited. While this has created sophisticated classification algorithms, many do not cope with increasing data set sizes. When the data set sizes get to a point where they could be considered to represent a continuous supply, or data stream, then incremental classification algorithms are required. In this setting, the effectiveness of an algorithm cannot simply be assessed by accuracy alone. Consideration needs to be given to the memory available to the algorithm and the speed at which data is processed in terms of both the time taken to predict the class of a new data sample and the time taken to include this sample in an incrementally updated classification model. The Hoeffding tree algorithm is a state-of-the-art method for inducing decision trees from data streams. The aim of this thesis is to improve this algorithm. To measure improvement, a comprehensive framework for evaluating the performance of data stream algorithms is developed. Within the framework memory size is fixed in order to simulate realistic application scenarios. In order to simulate continuous operation, classes of synthetic data are generated providing an evaluation on a large scale. Improvements to many aspects of the Hoeffding tree algorithm are demonstrated. First, a number of methods for handling continuous numeric features are compared. Second, tree prediction strategy is investigated to evaluate the utility of various methods. Finally, the possibility of improving accuracy using ensemble methods is explored. The experimental results provide meaningful comparisons of accuracy and processing speeds between different modifications of the Hoeffding tree algorithm under various memory limits. The study on numeric attributes demonstrates that sacrificing accuracy for space at the local level often results in improved global accuracy. The prediction strategy shown to perform best adaptively chooses between standard majority class and Naive Bayes prediction in the leaves. The ensemble method investigation shows that combining trees can be worthwhile, but only when sufficient memory is available, and improvement is less likely than in traditional machine learning. In particular, issues are encountered when applying the popular boosting method to streams.
26
Kyper, Eric S. "An information criterion for use in predictive data mining /." View online ; access limited to URI, 2006. http://0-wwwlib.umi.com.helin.uri.edu/dissertations/dlnow/3225319.
Full text
27
Bogdan, Vukobratović. "Hardware Acceleration of Nonincremental Algorithms for the Induction of Decision Trees and Decision Tree Ensembles." Phd thesis, Univerzitet u Novom Sadu, Fakultet tehničkih nauka u Novom Sadu, 2017. https://www.cris.uns.ac.rs/record.jsf?recordId=102520&source=NDLTD&language=en.
Full textAbstract:
The thesis proposes novel full decision tree and decision tree ensembleinduction algorithms EFTI and EEFTI, and various possibilities for theirimplementations are explored. The experiments show that the proposed EFTIalgorithm is able to infer much smaller DTs on average, without thesignificant loss in accuracy, when compared to the top-down incremental DTinducers. On the other hand, when compared to other full tree inductionalgorithms, it was able to produce more accurate DTs, with similar sizes, inshorter times. Also, the hardware architectures for acceleration of thesealgorithms (EFTIP and EEFTIP) are proposed and it is shown in experimentsthat they can offer substantial speedups.У овоj дисертациjи, представљени су нови алгоритми EFTI и EEFTI заформирање стабала одлуке и њихових ансамбала неинкременталномметодом, као и разне могућности за њихову имплементациjу.Експерименти показуjу да jе предложени EFTI алгоритам у могућностида произведе драстично мања стабла без губитка тачности у односу напостојеће top-down инкременталне алгоритме, а стабла знатно већетачности у односу на постојеће неинкременталне алгоритме. Такође супредложене хардверске архитектуре за акцелерацију ових алгоритама(EFTIP и EEFTIP) и показано је да је уз помоћ ових архитектура могућеостварити знатна убрзања.
U ovoj disertaciji, predstavljeni su novi algoritmi EFTI i EEFTI zaformiranje stabala odluke i njihovih ansambala neinkrementalnommetodom, kao i razne mogućnosti za njihovu implementaciju.Eksperimenti pokazuju da je predloženi EFTI algoritam u mogućnostida proizvede drastično manja stabla bez gubitka tačnosti u odnosu napostojeće top-down inkrementalne algoritme, a stabla znatno većetačnosti u odnosu na postojeće neinkrementalne algoritme. Takođe supredložene hardverske arhitekture za akceleraciju ovih algoritama(EFTIP i EEFTIP) i pokazano je da je uz pomoć ovih arhitektura mogućeostvariti znatna ubrzanja.
28
Basgalupp, Márcio Porto. "LEGAL-Tree: um algoritmo genético multi-objetivo para indução de árvores de decisão." Universidade de São Paulo, 2010. http://www.teses.usp.br/teses/disponiveis/55/55134/tde-12052010-165344/.
Full textAbstract:
Dentre as diversas tarefas em que os algoritmos evolutivos têm sido empregados, a indução de regras e de árvores de decisão tem se mostrado uma abordagem bastante atrativa em diversos domínios de aplicação. Algoritmos de indução de árvores de decisão representam uma das técnicas mais populares em problemas de classificação. Entretanto, os algoritmos tradicionais de indução apresentam algumas limitações, pois, geralmente, usam uma estratégia gulosa, top down e com particionamento recursivo para a construção das árvores. Esses fatores degradam a qualidade dos dados, os quais podem gerar regras estatisticamente não significativas. Este trabalho propõe o algoritmo LEGAL-Tree, uma nova abordagem baseada em algoritmos genéticos para indução de árvores de decisão. O algoritmo proposto visa evitar a estratégia gulosa e a convergência para ótimos locais. Para isso, esse algoritmo adota uma abordagem multi-objetiva lexicográfica. Nos experimentos realizados sobre bases de dados de diversos problemas de classificação, a função de fitness de LEGAL-Tree considera as duas medidas mais comuns para avaliação das árvores de decisão: acurácia e tamanho da árvore. Os resultados obtidos mostraram que LEGAL-Tree teve um desempenho equivalente ao algoritmo SimpleCart (implementação em Java do algoritmo CART) e superou o tradicional algoritmo J48 (implementação em Java do algoritmo C4.5), além de ter superado também o algoritmo evolutivo GALE. A principal contribuição de LEGAL-Tree não foi gerar árvores com maior acurácia preditiva, mas sim gerar árvores menores e, portanto, mais compreensíveis ao usuário do que as outras abordagens, mantendo a acurácia preditiva equivalente. Isso mostra que LEGAL-Tree obteve sucesso na otimização lexicográfica de seus objetivos, uma vez que a idéia era justamente dar preferência às árvores menores (em termos de número de nodos) quando houvesse equivalência de acuráciaAmong the several tasks evolutionary algorithms have been successfully employed, the induction of classification rules and decision trees has been shown to be a relevant approach for several application domains. Decision tree induction algorithms represent one of the most popular techniques for dealing with classification problems. However, conventionally used decision trees induction algorithms present limitations due to the strategy they usually implement: recursive top-down data partitioning through a greedy split evaluation. The main problem with this strategy is quality loss during the partitioning process, which can lead to statistically insignificant rules. In this thesis we propose the LEGAL-Tree algorithm, a new GA-based algorithm for decision tree induction. The proposed algorithm aims to prevent the greedy strategy and to avoid converging to local optima. For such, it is based on a lexicographic multi-objective approach. In the experiments which were run in several classification problems, LEGAL-Tree\'s fitness function considers two of the most common measures to evaluate decision trees: accuracy and tree size. Results show that LEGAL-Tree performs similarly to SimpleCart (CART Java implementation) and outperforms J48 (C4.5 Java implementation) and the evolutionary algorithm GALE. LEGAL-Tree\'s main contribution is not to generate trees with the highest predictive accuracy possible, but to provide smaller (and thus more comprehensible) trees, keeping a competitive accuracy rate. LEGAL-Tree is able to provide both comprehensible and accurate trees, which shows that the lexicographic fitness evaluation is successful since its goal is to prioritize smaller trees (fewer number of nodes) when there is equivalency in terms of accuracy
29
Tsang, Pui-kwan Smith, and 曾沛坤. "Efficient decision tree building algorithms for uncertain data." Thesis, The University of Hong Kong (Pokfulam, Hong Kong), 2008. http://hub.hku.hk/bib/B41290719.
Full text
30
Tsang, Pui-kwan Smith. "Efficient decision tree building algorithms for uncertain data." Click to view the E-thesis via HKUTO, 2008. http://sunzi.lib.hku.hk/hkuto/record/B41290719.
Full text
31
Roberts, Lucas R. "Variable Selection and Decision Trees: The DiVaS and ALoVaS Methods." Diss., Virginia Tech, 2014. http://hdl.handle.net/10919/70878.
Full textAbstract:
In this thesis we propose a novel modification to Bayesian decision tree methods. We provide a
historical survey of the statistics and computer science research in decision trees. Our approach
facilitates covariate selection explicitly in the model, something not present in previous research.
We define a transformation that allows us to use priors from linear models to facilitate covariate
selection in decision trees. Using this transform, we modify many common approaches to variable
selection in the linear model and bring these methods to bear on the problem of explicit covariate
selection in decision tree models. We also provide theoretical guidelines, including a theorem,
which gives necessary and sufficient conditions for consistency of decision trees in infinite dimensional
spaces. Our examples and case studies use both simulated and real data cases with moderate
to large numbers of covariates. The examples support the claim that our approach is to be preferred
in large dimensional datasets. Moreover, our approach shown here has, as a special case, the model
known as Bayesian CART.Ph. D.
32
Revend, War. "Predicting House Prices on the Countryside using Boosted Decision Trees." Thesis, KTH, Matematisk statistik, 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-279849.
Full textAbstract:
This thesis intends to evaluate the feasibility of supervised learning models for predicting house prices on the countryside of South Sweden. It is essential for mortgage lenders to have accurate housing valuation algorithms and the current model offered by Booli is not accurate enough when evaluating residence prices on the countryside. Different types of boosted decision trees were implemented to address this issue and their performances were compared to traditional machine learning methods. These different types of supervised learning models were implemented in order to find the best model with regards to relevant evaluation metrics such as root-mean-squared error (RMSE) and mean absolute percentage error (MAPE). The implemented models were ridge regression, lasso regression, random forest, AdaBoost, gradient boosting, CatBoost, XGBoost, and LightGBM. All these models were benchmarked against Booli's current housing valuation algorithms which are based on a k-NN model. The results from this thesis indicated that the LightGBM model is the optimal one as it had the best overall performance with respect to the chosen evaluation metrics. When comparing the LightGBM model to the benchmark, the performance was overall better, the LightGBM model had an RMSE score of 0.330 compared to 0.358 for the Booli model, indicating that there is a potential of using boosted decision trees to improve the predictive accuracy of residence prices on the countryside.Denna uppsats ämnar utvärdera genomförbarheten hos olika övervakade inlärningsmodeller för att förutse huspriser på landsbygden i Södra Sverige. Det är viktigt för bostadslånsgivare att ha noggranna algoritmer när de värderar bostäder, den nuvarande modellen som Booli erbjuder har dålig precision när det gäller värderingar av bostäder på landsbygden. Olika typer av boostade beslutsträd implementerades för att ta itu med denna fråga och deras prestanda jämfördes med traditionella maskininlärningsmetoder. Dessa olika typer av övervakad inlärningsmodeller implementerades för att hitta den bästa modellen med avseende på relevanta prestationsmått som t.ex. root-mean-squared error (RMSE) och mean absolute percentage error (MAPE). De övervakade inlärningsmodellerna var ridge regression, lasso regression, random forest, AdaBoost, gradient boosting, CatBoost, XGBoost, and LightGBM. Samtliga algoritmers prestanda jämförs med Boolis nuvarande bostadsvärderingsalgoritm, som är baserade på en k-NN modell. Resultatet från denna uppsats visar att LightGBM modellen är den optimala modellen för att värdera husen på landsbygden eftersom den hade den bästa totala prestandan med avseende på de utvalda utvärderingsmetoderna. LightGBM modellen jämfördes med Booli modellen där prestandan av LightGBM modellen var i överlag bättre, där LightGBM modellen hade ett RMSE värde på 0.330 jämfört med Booli modellen som hade ett RMSE värde på 0.358. Vilket indikerar att det finns en potential att använda boostade beslutsträd för att förbättra noggrannheten i förutsägelserna av huspriser på landsbygden.
33
KINSEY, MICHAEL LOY. "PRIVACY PRESERVING INDUCTION OF DECISION TREES FROM GEOGRAPHICALLY DISTRIBUTED DATABASES." University of Cincinnati / OhioLINK, 2005. http://rave.ohiolink.edu/etdc/view?acc_num=ucin1123855448.
Full text
34
Au, Manix. "Automatic State Construction using Decision Trees for Reinforcement Learning Agents." Thesis, Queensland University of Technology, 2005. https://eprints.qut.edu.au/15965/1/Manix_Au_Thesis.pdf.
Full textAbstract:
Reinforcement Learning (RL) is a learning framework in which an agent learns a policy from continual interaction with the environment. A policy is a mapping from states to actions. The agent receives rewards as feedback on the actions performed. The objective of RL is to design autonomous agents to search for the policy that maximizes the expectation of the cumulative reward.
When the environment is partially observable, the agent cannot determine the states with certainty. These states are called hidden in the literature. An agent that relies exclusively on the current observations will not always find the optimal policy. For example, a mobile robot needs to remember the number of doors went by in order to reach a specific door, down a corridor of identical doors.
To overcome the problem of partial observability, an agent uses both current and past (memory) observations to construct an internal state representation, which is treated
as an abstraction of the environment.
This research focuses on how features of past events are extracted with variable granularity regarding the internal state construction. The project introduces a new method that applies Information Theory and decision tree technique to derive a tree structure, which represents the state and the policy. The relevance, of a candidate feature, is assessed by the Information Gain Ratio ranking with respect to the cumulative expected reward.
Experiments carried out on three different RL tasks have shown that our variant of the U-Tree (McCallum, 1995) produces a more robust state representation and faster learning. This better performance can be explained by the fact that the Information Gain Ratio exhibits a lower variance in return prediction than the Kolmogorov-Smirnov statistical test used in the original U-Tree algorithm.
35
Au, Manix. "Automatic State Construction using Decision Trees for Reinforcement Learning Agents." Queensland University of Technology, 2005. http://eprints.qut.edu.au/15965/.
Full textAbstract:
Reinforcement Learning (RL) is a learning framework in which an agent learns a policy from continual interaction with the environment. A policy is a mapping from states to actions. The agent receives rewards as feedback on the actions performed. The objective of RL is to design autonomous agents to search for the policy that maximizes the expectation of the cumulative reward. When the environment is partially observable, the agent cannot determine the states with certainty. These states are called hidden in the literature. An agent that relies exclusively on the current observations will not always find the optimal policy. For example, a mobile robot needs to remember the number of doors went by in order to reach a specific door, down a corridor of identical doors. To overcome the problem of partial observability, an agent uses both current and past (memory) observations to construct an internal state representation, which is treated as an abstraction of the environment. This research focuses on how features of past events are extracted with variable granularity regarding the internal state construction. The project introduces a new method that applies Information Theory and decision tree technique to derive a tree structure, which represents the state and the policy. The relevance, of a candidate feature, is assessed by the Information Gain Ratio ranking with respect to the cumulative expected reward. Experiments carried out on three different RL tasks have shown that our variant of the U-Tree (McCallum, 1995) produces a more robust state representation and faster learning. This better performance can be explained by the fact that the Information Gain Ratio exhibits a lower variance in return prediction than the Kolmogorov-Smirnov statistical test used in the original U-Tree algorithm.
36
Sinsel, Erik W. "Ensemble learning for ranking interesting attributes." Morgantown, W. Va. : [West Virginia University Libraries], 2005. https://eidr.wvu.edu/etd/documentdata.eTD?documentid=4400.
Full textAbstract:
Thesis (M.S.)--West Virginia University, 2005.Title from document title page. Document formatted into pages; contains viii, 81 p. : ill. Includes abstract. Includes bibliographical references (p. 72-74).
37
Alotaibi, Reem Moteab Sultan. "Learning versatile decision trees : towards context-awareness and multi-label classification." Thesis, University of Bristol, 2016. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.723447.
Full text
38
Kerbs, Robert W. "The Extraction of Classification Rules and Decision Trees from Independence Diagrams." NSUWorks, 2001. http://nsuworks.nova.edu/gscis_etd/630.
Full textAbstract:
Databases are growing exponentially in many application domains. Timely construction of models that represent identified patterns and regularities in the data facilitate the prediction of future events based upon past performance. Data mining can promote this process through various model building techniques. The goal is to create models that intuitively represent the data and perhaps aid in the discovery of new knowledge.
Most data mining methods rely upon either fully-automated information-theoretic or statistical algorithms. Typically, these algorithms are non-interactive, hide the model derivation process from the user, require the assistance of a domain expert, are application-specific, and may not clearly translate detected relationships.
This paper proposes a visual data mining algorithm, BLUE, as an alternative to present data mining techniques. BLUE visually supports the processes of classification and prediction by combining two visualization methods. The first consists of a modification to independence diagrams, called BIDS, allowing for the examination of pairs of categorical attributes in relational databases. The second uses decision trees to provide a global context from which a model can be constructed. Classification rules are extracted from the decision trees to assist in concept representation. BLUE uses the abilities of the human visual system to detect patterns and regularities in images. The algorithm employs a mechanism that permits the user to interactively backtrack to previously visited nodes to guide and explore the creation of the model. As a decision tree is induced, classification rules are simultaneously extracted. Experimental results show that BLUE produces models that are more comprehensible when compared with alternative methods. These experimental results lend support for future studies in visual data mining.
39
Ahmed, Istiak. "An ensemble learning approach based on decision trees and probabilistic argumentation." Thesis, Umeå universitet, Institutionen för datavetenskap, 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:umu:diva-175967.
Full textAbstract:
This research discusses a decision support system that includes different machine learning approaches (e.g. ensemble learning, decision trees) and a symbolic reasoning approach (e.g. argumentation). The purpose of this study is to define an ensemble learning algorithm based on formal argumentation and decision trees. Using a decision tree algorithmas a base learning algorithm and an argumentation framework as a decision fusion technique of an ensemble architecture, the proposed system produces outcomes. The introduced algorithm is a hybrid ensemble learning approach based on a formal argumentation-based method. It is evaluated with sample data sets (e.g. an open-access data set and an extracted data set from ultrasound images) and it provides satisfactory outcomes. This study approaches the problem that is related to an ensemble learning algorithm and a formal argumentation approach. A probabilistic argumentation framework is implemented as a decision fusion in an ensemble learning approach. An open-access library is also developed for the user. The generic version of the library can be used in different purposes.
40
Gerdes, Mike. "Health Monitoring for Aircraft Systems using Decision Trees and Genetic Evolution." Diss., Aircraft Design and Systems Group (AERO), Department of Automotive and Aeronautical Engineering, Hamburg University of Applied Sciences, 2019. http://d-nb.info/1202830382.
Full textAbstract:
Reducing unscheduled maintenance is important for aircraft operators. There are significant costs if flights must be delayed or cancelled, for example, if spares are not available and have to be shipped across the world. This thesis describes three methods of aircraft health condition monitoring and prediction; one for system monitoring, one for forecasting and one combining the two other methods for a complete monitoring and prediction process. Together, the three methods allow organizations to forecast possible failures. The first two use decision trees for decision-making and genetic optimization to improve the performance of the decision trees and to reduce the need for human interaction. Decision trees have several advantages: the generated code is quickly and easily processed, it can be altered by human experts without much work, it is readable by humans, and it requires few resources for learning and evaluation. The readability and the ability to modify the results are especially important; special knowledge can be gained and errors produced by the automated code generation can be removed. A large number of data sets is needed for meaningful predictions. This thesis uses two data sources: first, data from existing aircraft sensors, and second, sound and vibration data from additionally installed sensors. It draws on methods from the field of big data and machine learning to analyse and prepare the data sets for the prediction process.
41
Andriyashin, Anton. "Stock picking via nonsymmetrically pruned binary decision trees with reject option." Doctoral thesis, Humboldt-Universität zu Berlin, Wirtschaftswissenschaftliche Fakultät, 2010. http://dx.doi.org/10.18452/16248.
Full textAbstract:
Die Auswahl von Aktien ist ein Gebiet der Finanzanalyse, die von speziellem Interesse sowohl für viele professionelle Investoren als auch für Wissenschaftler ist. Empirische Untersuchungen belegen, dass Aktienerträge vorhergesagt werden können. Während verschiedene Modellierungstechniken zur Aktienselektion eingesetzt werden könnten, analysiert diese Arbeit die meist verbreiteten Methoden, darunter allgemeine Gleichgewichtsmodelle und Asset Pricing Modelle; parametrische, nichtparametrische und semiparametrische Regressionsmodelle; sowie beliebte Black-Box Klassifikationsmethoden. Aufgrund vorteilhafter Eigenschaften binärer Klassifikationsbäume, wie zum Beispiel einer herausragenden Interpretationsmöglichkeit von Entscheidungsregeln, wird der Kern des Handelsalgorithmus unter Verwendung dieser modernen, nichtparametrischen Methode konstruiert. Die optimale Größe des Baumes wird als der entscheidende Faktor für die Vorhersageperformance von Klassifikationsbäumen angesehen. Während eine Vielfalt alternativer populärer Bauminduktions- und Pruningtechniken existiert, die in dieser Studie kritisch gewürdigt werden, besteht eines der Hauptanliegen dieser Arbeit in einer neuartigen Methode asymmetrischen Baumprunings mit Abweisungsoption. Diese Methode wird als Best Node Selection (BNS) bezeichnet. Eine wichtige inverse Fortpflanzungseigenschaft der BNS wird bewiesen. Diese eröffnet eine einfache Möglichkeit, um die Suche der optimalen Baumgröße in der Praxis zu implementieren. Das traditionelle costcomplexity Pruning zeigt eine ähnliche Performance hinsichtlich der Baumgenauigkeit verglichen mit beliebten alternativen Techniken, und es stellt die Standard Pruningmethode für viele Anwendungen dar. Die BNS wird mit cost-complexity Pruning empirisch verglichen, indem zwei rekursive Portfolios aus DAX-Aktien zusammengestellt werden. Vorhersagen über die Performance für jede einzelne Aktie werden von Entscheidungsbäumen gemacht, die aktualisiert werden, sobald neue Marktinformationen erhältlich sind. Es wird gezeigt, dass die BNS der traditionellen Methode deutlich überlegen ist, und zwar sowohl gemäß den Backtesting Ergebnissen als auch nach dem Diebold-Marianto Test für statistische Signifikanz des Performanceunterschieds zwischen zwei Vorhersagemethoden. Ein weiteres neuartiges Charakteristikum dieser Arbeit liegt in der Verwendung individueller Entscheidungsregeln für jede einzelne Aktie im Unterschied zum traditionellen Zusammenfassen lernender Muster. Empirische Daten in Form individueller Entscheidungsregeln für einen zufällig ausgesuchten Zeitpunkt in der Überprüfungsreihe rechtfertigen diese Methode.Stock picking is the field of financial analysis that is of particular interest for many professional investors and researchers. There is a lot of research evidence supporting the fact that stock returns can effectively be forecasted. While various modeling techniques could be employed for stock price prediction, a critical analysis of popular methods including general equilibrium and asset pricing models; parametric, non- and semiparametric regression models; and popular black box classification approaches is provided. Due to advantageous properties of binary classification trees including excellent level of interpretability of decision rules, the trading algorithm core is built employing this modern nonparametric method. Optimal tree size is believed to be the crucial factor of forecasting performance of classification trees. While there exists a set of widely adopted alternative tree induction and pruning techniques, which are critically examined in the study, one of the main contributions of this work is a novel methodology of nonsymmetrical tree pruning with reject option called Best Node Selection (BNS). An important inverse propagation property of BNS is proven that provides an easy way to implement the search for the optimal tree size in practice. Traditional cost-complexity pruning shows similar performance in terms of tree accuracy when assessed against popular alternative techniques, and it is the default pruning method for many applications. BNS is compared with costcomplexity pruning empirically by composing two recursive portfolios out of DAX30 stocks. Performance forecasts for each of the stocks are provided by constructed decision trees that are updated when new market information becomes available. It is shown that BNS clearly outperforms the traditional approach according to the backtesting results and the Diebold-Mariano test for statistical significance of the performance difference between two forecasting methods. Another novel feature of this work is the use of individual decision rules for each stock as opposed to pooling of learning samples, which is done traditionally. Empirical data in the form of provided individual decision rules for a randomly selected time point in the backtesting set justify this approach.
42
Rangwala, Maimuna H. "Empirical investigation of decision tree extraction from neural networks." Ohio : Ohio University, 2006. http://www.ohiolink.edu/etd/view.cgi?ohiou1151608193.
Full text
43
Bremner, Alexandra P. "Localised splitting criteria for classification and regression trees." Thesis, Bremner, Alexandra P. (2004) Localised splitting criteria for classification and regression trees. PhD thesis, Murdoch University, 2004. https://researchrepository.murdoch.edu.au/id/eprint/440/.
Full textAbstract:
This thesis presents a modification of existing entropy-based splitting criteria for classification and regression trees. Trees are typically grown using splitting criteria that choose optimal splits without taking future splits into account. This thesis examines localised splitting criteria that are based on local averaging in regression trees or local proportions in classification trees. The use of a localised criterion is motivated by the fact that future splits result in leaves that contain local observations, and hence local deviances provide a better approximation of the deviance of the fully grown tree. While most recent research has focussed on tree-averaging techniques that are aimed at taking a moderately successful splitting criterion and improving its predictive power, this thesis concentrates on improving the splitting criterion.
Use of a localised splitting criterion captures local structures and enables later splits to capitalise on the placement of earlier splits when growing a tree. Using the localised splitting criterion results in much simpler trees for pure interaction data (data with no main effects) and can produce trees with fewer errors and lower residual mean deviances than those produced using a global splitting criterion when applied to real data sets with strong interaction effects.
The superiority of the localised splitting criterion can persist when multiple trees are grown and averaged using simple methods. Although a single tree grown using the localised splitting criterion can outperform tree averaging using the global criterion, generally improvements in predictive performance are achieved by utilising the localised splitting criterion's property of detecting local discontinuities and averaging over sets of trees grown by placing splits where the deviance is locally minimal. Predictive performance improves further when the degree of localisation of the splitting criterion is randomly selected and weighted randomisation is used with locally minimal deviances to produce sets of trees to average over. Although state of the art methods quickly average very large numbers of trees, thus making the performance of the splitting criterion less critical, predictive performance when the localised criterion is used in bagging indicates that different splitting methods warrant investigation.
The localised splitting criterion is most useful for growing one tree or a small number of trees to examine structure in the data. Structurally different trees can be obtained by simply splitting the data where the localised splitting criterion is locally optimal.
44
Bremner, Alexandra P. "Localised splitting criteria for classification and regression trees /." Access via Murdoch University Digital Theses Project, 2004. http://wwwlib.murdoch.edu.au/adt/browse/view/adt-MU20040606.142949.
Full text
45
Barros, Rodrigo Coelho. "On the automatic design of decision-tree induction algorithms." Universidade de São Paulo, 2013. http://www.teses.usp.br/teses/disponiveis/55/55134/tde-21032014-144814/.
Full textAbstract:
Decision-tree induction is one of the most employed methods to extract knowledge from data. There are several distinct strategies for inducing decision trees from data, each one presenting advantages and disadvantages according to its corresponding inductive bias. These strategies have been continuously improved by researchers over the last 40 years. This thesis, following recent breakthroughs in the automatic design of machine learning algorithms, proposes to automatically generate decision-tree induction algorithms. Our proposed approach, namely HEAD-DT, is based on the evolutionary algorithms paradigm, which improves solutions based on metaphors of biological processes. HEAD-DT works over several manually-designed decision-tree components and combines the most suitable components for the task at hand. It can operate according to two different frameworks: i) evolving algorithms tailored to one single data set (specific framework); and ii) evolving algorithms from multiple data sets (general framework). The specific framework aims at generating one decision-tree algorithm per data set, so the resulting algorithm does not need to generalise beyond its target data set. The general framework has a more ambitious goal, which is to generate a single decision-tree algorithm capable of being effectively applied to several data sets. The specific framework is tested over 20 UCI data sets, and results show that HEAD-DTs specific algorithms outperform algorithms like CART and C4.5 with statistical significance. The general framework, in turn, is executed under two different scenarios: i) designing a domain-specific algorithm; and ii) designing a robust domain-free algorithm. The first scenario is tested over 35 microarray gene expression data sets, and results show that HEAD-DTs algorithms consistently outperform C4.5 and CART in different experimental configurations. The second scenario is tested over 67 UCI data sets, and HEAD-DTs algorithms were shown to be competitive with C4.5 and CART. Nevertheless, we show that HEAD-DT is prone to a special case of overfitting when it is executed under the second scenario of the general framework, and we point to possible alternatives for solving this problem. Finally, we perform an extensive experiment for evaluating the best single-objective fitness function for HEAD-DT, combining 5 classification performance measures with three aggregation schemes. We evaluate the 15 fitness functions in 67 UCI data sets, and the best of them are employed to generate algorithms tailored to balanced and imbalanced data. Results show that the automatically-designed algorithms outperform CART and C4.5 with statistical significance, indicating that HEAD-DT is also capable of generating custom algorithms for data with a particular kind of statistical profileÁrvores de decisão são amplamente utilizadas como estratégia para extração de conhecimento de dados. Existem muitas estratégias diferentes para indução de árvores de decisão, cada qual com suas vantagens e desvantagens tendo em vista seu bias indutivo. Tais estratégias têm sido continuamente melhoradas por pesquisadores nos últimos 40 anos. Esta tese, em sintonia com recentes descobertas no campo de projeto automático de algoritmos de aprendizado de máquina, propõe a geração automática de algoritmos de indução de árvores de decisão. A abordagem proposta, chamada de HEAD-DT, é baseada no paradigma de algoritmos evolutivos. HEAD-DT evolui componentes de árvores de decisão que foram manualmente codificados e os combina da forma mais adequada ao problema em questão. HEAD-DT funciona conforme dois diferentes frameworks: i) evolução de algoritmos customizados para uma única base de dados (framework específico); e ii) evolução de algoritmos a partir de múltiplas bases (framework geral). O framework específico tem por objetivo gerar um algoritmo por base de dados, de forma que o algoritmo projetado não necessite de poder de generalização que vá além da base alvo. O framework geral tem um objetivo mais ambicioso: gerar um único algoritmo capaz de ser efetivamente executado em várias bases de dados. O framework específico é testado em 20 bases públicas da UCI, e os resultados mostram que os algoritmos específicos gerados por HEAD-DT apresentam desempenho preditivo significativamente melhor do que algoritmos como CART e C4.5. O framework geral é executado em dois cenários diferentes: i) projeto de algoritmo específico a um domínio de aplicação; e ii) projeto de um algoritmo livre-de-domínio, robusto a bases distintas. O primeiro cenário é testado em 35 bases de expressão gênica, e os resultados mostram que o algoritmo gerado por HEAD-DT consistentemente supera CART e C4.5 em diferentes configurações experimentais. O segundo cenário é testado em 67 bases de dados da UCI, e os resultados mostram que o algoritmo gerado por HEAD-DT é competitivo com CART e C4.5. No entanto, é mostrado que HEAD-DT é vulnerável a um caso particular de overfitting quando executado sobre o segundo cenário do framework geral, e indica-se assim possíveis soluções para tal problema. Por fim, é realizado uma análise detalhada para avaliação de diferentes funções de fitness de HEAD-DT, onde 5 medidas de desempenho são combinadas com três esquemas de agregação. As 15 versões são avaliadas em 67 bases da UCI e as melhores versões são utilizadas para geração de algoritmos customizados para bases balanceadas e desbalanceadas. Os resultados mostram que os algoritmos gerados por HEAD-DT apresentam desempenho preditivo significativamente melhor que CART e C4.5, em uma clara indicação que HEAD-DT também é capaz de gerar algoritmos customizados para certo perfil estatístico dos dados de classificação
46
Sendur, Zeynel. "Text Document Categorization by Machine Learning." Scholarly Repository, 2008. http://scholarlyrepository.miami.edu/oa_theses/209.
Full textAbstract:
Because of the explosion of digital and online text information, automatic organization of documents has become a very important research area. There are mainly two machine learning approaches to enhance the organization task of the digital documents. One of them is the supervised approach, where pre-defined category labels are assigned to documents based on the likelihood suggested by a training set of labeled documents; and the other one is the unsupervised approach, where there is no need for human intervention or labeled documents at any point in the whole process. In this thesis, we concentrate on the supervised learning task which deals with document classification. One of the most important tasks of information retrieval is to induce classifiers capable of categorizing text documents. The same document can belong to two or more categories and this situation is referred by the term multi-label classification. Multi-label classification domains have been encountered in diverse fields. Most of the existing machine learning techniques which are in multi-label classification domains are extremely expensive since the documents are characterized by an extremely large number of features. In this thesis, we are trying to reduce these computational costs by applying different types of algorithms to the documents which are characterized by large number of features. Another important thing that we deal in this thesis is to have the highest possible accuracy when we have the high computational performance on text document categorization.
47
Summad, Emad. "A Monte-Carlo approach to tool selection for sheet metal punching and nibbling." Thesis, Durham University, 2001. http://etheses.dur.ac.uk/4137/.
Full textAbstract:
Selecting the best set of tools to produce certain geometrical shapes/features in sheet metal punching is one of the problems that has a great effect on product development time, cost and achieved quality. The trend nowadays is, where at all possible, to limit design to the use of standard tools. Such an option makes the problem of selecting the appropriate set of tools even more complex, especially when considering that sheet metal features can have a wide range of complex shapes. Another dimension of complexity is limited tool rack capacity. Thus, an inappropriate tool selection strategy will lead to punching inefficiency and may require frequent stopping of the machine and replacing the required tools, which is a rather expensive and time consuming exercise. This work demonstrates that the problem of selecting the best set of tools is actually a process of searching an explosive decision tree. The difficulty in searching such types of decision trees is that intermediate decisions do not necessarily reflect the total cost implication of carrying out such a decision. A new approach to solve such a complex optimisation problem using the Monte Carlo Simulation Methods has been introduced in this thesis. The aim of the present work was to establish the use of Monte Carlo methods as an "assumptions or rule free" baseline or benchmark for the assessment of search strategies. A number of case studies are given, where the feasibility of Monte Carlo Simulation Methods as an efficient and viable method to optimise such a complex optimisation problem is demonstrated. The use of a Monte Carlo approach for selecting the best set of punching tools, showed an interesting point, that is, the effect of dominant "one-to-one" feature/tool matches on the efficiency of the search. This naturally led on to the need of a search methodology that will be more efficient than the application of the Monte Carlo method alone. This thesis presents some interesting speculations for a hybrid approach to tool selection to achieve a better solution than the use of the Monte Carlo method alone to achieve the optimum solution in a shorter time.
48
MacDonald, Calum Angus. "The development of an objective methodology for the prediction of helicopter pilot workload." Thesis, Glasgow Caledonian University, 2001. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.340607.
Full text
49
Màrquez, Lluís. "Part-of-speech Tagging: A Machine Learning Approach based on Decision Trees." Doctoral thesis, Universitat Politècnica de Catalunya, 1999. http://hdl.handle.net/10803/6663.
Full textAbstract:
The study and application of general Machine Learning (ML) algorithms to theclassical ambiguity problems in the area of Natural Language Processing (NLP) isa currently very active area of research. This trend is sometimes called NaturalLanguage Learning. Within this framework, the present work explores the applicationof a concrete machine-learning technique, namely decision-tree induction, toa very basic NLP problem, namely part-of-speech disambiguation (POS tagging).Its main contributions fall in the NLP field, while topics appearing are addressedfrom the artificial intelligence perspective, rather from a linguistic point of view.A relevant property of the system we propose is the clear separation betweenthe acquisition of the language model and its application within a concrete disambiguationalgorithm, with the aim of constructing two components which are asindependent as possible. Such an approach has many advantages. For instance, thelanguage models obtained can be easily adapted into previously existing taggingformalisms; the two modules can be improved and extended separately; etc.As a first step, we have experimentally proven that decision trees (DT) providea flexible (by allowing a rich feature representation), efficient and compact wayfor acquiring, representing and accessing the information about POS ambiguities.In addition to that, DTs provide proper estimations of conditional probabilities fortags and words in their particular contexts. Additional machine learning techniques,based on the combination of classifiers, have been applied to address some particularweaknesses of our tree-based approach, and to further improve the accuracy in themost difficult cases.As a second step, the acquired models have been used to construct simple,accurate and effective taggers, based on diiferent paradigms. In particular, wepresent three different taggers that include the tree-based models: RTT, STT, andRELAX, which have shown different properties regarding speed, flexibility, accuracy,etc. The idea is that the particular user needs and environment will define whichis the most appropriate tagger in each situation. Although we have observed slightdifferences, the accuracy results for the three taggers, tested on the WSJ test benchcorpus, are uniformly very high, and, if not better, they are at least as good asthose of a number of current taggers based on automatic acquisition (a qualitativecomparison with the most relevant current work is also reported.Additionally, our approach has been adapted to annotate a general Spanishcorpus, with the particular limitation of learning from small training sets. A newtechnique, based on tagger combination and bootstrapping, has been proposed toaddress this problem and to improve accuracy. Experimental results showed thatvery high accuracy is possible for Spanish tagging, with a relatively low manualeffort. Additionally, the success in this real application has confirmed the validity of our approach, and the validity of the previously presented portability argumentin favour of automatically acquired taggers.
50
Valente, Lorenzo. "Reconstruction of non-prompt charmed baryon Λc with boosted decision trees technique." Bachelor's thesis, Alma Mater Studiorum - Università di Bologna, 2020. http://amslaurea.unibo.it/21033/.
Full textAbstract:
L'esperimento ALICE studia la fisica dell'interazione forte a estreme densità di energia attraverso la collisione di ioni pesanti.
In tali condizioni è possibile la formazione dello stato della materia chiamato plasma di quark e gluoni. A causa della ridotta vita media di tale stato, lo studio è molto complesso ed è pertanto possibile condurlo solo in modo indiretto sulla base delle modalità di raffreddamento e dalle particelle rilasciate nel processo. Uno dei principali metodi d’indagine è lo studio di adroni contenenti quark pesanti (charm e beauty) e di come queste particelle, prodotte nei primi stadi della collisione, interagiscono con questo stato della materia. L'obiettivo della tesi è la ricostruzione del barione charmato Λc e la distinzione del segnale non-prompt da quello prompt attraverso la tecnica dei Boosted Decision Trees. L'analisi è stata condotta attraverso l'approccio di analisi multivariata in cui è possibile considerare le proprietà di più eventi contemporaneamente, ricavando il maggior numero di informazioni attraverso tecniche di machine learning.