Дисертації з теми "Cascade neural networks"

Thesis (M.S.)--University of New Orleans, 2003.
Title from electronic submission form. "A thesis ... in partial fulfillment of the requirements for the degree of Master of Science in the Department of Electrical Engineering"--Thesis t.p. Vita. Includes bibliographical references.

Most neural network learning algorithms cannot use knowledge other than what is provided in the training data. Initialized using random weights, they cannot use prior knowledge such as knowledge stored in previously trained networks. This manuscript thesis addresses this problem. It contains a literature review of the relevant static and constructive neural network learning algorithms and of the recent research on transfer of knowledge across neural networks. Manuscript 1 describes a new algorithm, named knowledge-based cascade-correlation (KBCC), which extends the cascade-correlation learning algorithm to allow it to use prior knowledge. This prior knowledge can be provided as, but is not limited to, previously trained neural networks. The manuscript also contains a set of experiments that shows how KBCC is able to reduce its learning time by automatically selecting the appropriate prior knowledge to reuse. Manuscript 2 shows how KBCC speeds up learning on a realistic large problem of vowel recognition.

Ellipsometry is a well known optical technique used for the characterization of reflective surfaces in study and films between two media. It is based on measuring the change in the state of polarization that occurs as a beam of polarized light is reflected from or transmitted through the film. Measuring this change can be used to calculate parameters of a single layer film such as the thickness and the refractive index. However, extracting these parameters of interest requires significant numerical processing due to the noninvertible equations. Typically, this is done using least squares solving methods which are slow and adversely affected by local minima in the solvable surface. This thesis describes the development and implementation of a new technique using only Artificial Neural Networks (ANN) to calculate thin film parameters. The new method offers a speed in the orders of magnitude faster than preceding methods and convergence to local minima is completely eliminated.

Modeling of nonlinear systems based on special Hammerstein forms has been considered. In Hammerstein system modeling a static nonlinearity is connected to a dynamic linearity in cascade form. Fundamental contributions of this work are: 1) Introduction of Bezier curve nonlinearity representations
2) Introduction of B-Spline curve nonlinearity representations instead of polynomials in cascade modeling. As a result, local control in nonlinear system modeling is achieved. Thus, unexpected variations of the output can be modeled more closely. As an important demonstration case, a model is developed and named as Middle East Technical University Neural Networks and Cascade Model (METU-NN-C). Application examples are chosen by considering the Near-Earth space processes, which are important for navigation, telecommunication and many other technical applications. It is demonstrated that the models developed based on the contributions of this work are especially more accurate under disturbed conditions, which are quantified by considering Space Weather parameters. Examples include forecasting of Total Electron Content (TEC), and mapping
estimation of joint angle of simple forced pendulum
estimation of joint angles of spring loaded inverted double pendulum with forced table
identification of Van der Pol oscillator
and identification of speakers. The operation performance results of the International Reference Ionosphere (IRI-2001), METU Neural Networks (METU-NN) and METU-NN-C models are compared qualitatively and quantitatively. As a numerical example, in forecasting the TEC by using the METU-NN-C having Bezier curves in nonlinearity representation, the average absolute error is 1.11 TECu. The new cascade models are shown to be promising for system designers and operators.

Master theses deals with Constructive Neural newtorks. First part describes neural networks and coresponding mathematical models. Furher, it shows basic algorithms for learning neural networks and desribes basic constructive algotithms and their modifications. The second part deals with implementation details of selected algorithms and provides their comparision. Further comparision with backpropagation algorithm is provided.

This thesis extends the use of neural-network-based model reference adaptive control to systems that occur as cascades. In general, these systems are not feedback linearizable. The approach taken is that of approximate feedback linearization of upper subsystems whilst treating the lower-subsystem states as virtual actuators. Similarly, lower-subsystems are also feedback linearized. Typically, approximate inverses are used for linearization purposes. Model error arising from the use of an approximate inverse is minimized using a neural-network as an adaptive element. Incorrect adaptation due to (virtual) actuator saturation and dynamics is avoided using the Pseudocontrol Hedging method. Using linear approximate inverses and linear reference models generally result in large desired pseudocontrol for large external commands. Even if the provided external command is feasible (null-controllable), there is no guarantee that the reference model trajectory is feasible. In order to mitigate this, nonlinear reference models based on nested-saturation methods are used to constrain the evolution of the reference model and thus the plant states. The method presented in this thesis lends itself to the inner-outer loop control of air vehicles, where the inner-loop controls attitude dynamics and the outer-loop controls the translational dynamics of the vehicle. The outer-loop treats the closed loop attitude dynamics as an actuator. Adaptation to uncertainty in the attitude, as well as the translational dynamics, is introduced, thus minimizing the effects of model error in all six degrees of freedom and leading to more accurate position tracking. A pole-placement approach is used to choose compensator gains for the tracking error dynamics. This alleviates timescale separation requirements, allowing the outer loop bandwidth to be closer to that of the inner loop, thus increasing position tracking performance. A poor model of the attitude dynamics and a basic kinematics model is shown to be sufficient for accurate position tracking. In particular, the inner-outer loop method was used to control an unmanned helicopter and has subsequently been applied to a ducted-fan, a fixed-wing aircraft that transitions in and out of hover, and a full-scale rotorcraft. Experimental flight test results are also provided for a subset of these vehicles.

Engineering design often requires the optimisation of multiple objectives, and becomes significantly more difficult and time consuming when the response surfaces are multimodal, rather than unimodal. A surrogate model, also known as a metamodel, can be used to replace expensive computer simulations, accelerating single and multi-objective optimisation and the exploration of new design concepts. The main research focus of this work is to investigate the use of a neural network surrogate model to improve optimisation of multimodal surfaces. Several significant contributions derive from evaluating the Cascade Correlation neural network as the basis of a surrogate model. The contributions to the neural network community ultimately outnumber those to the optimisation community. The effects of training this surrogate on multimodal test functions are explored. The Cascade Correlation neural network is shown to map poorly such response surfaces. A hypothesis for this weakness is formulated and tested. A new subdivision technique is created that addresses this problem; however, this new technique requires excessively large datasets upon which to train. The primary conclusion of this work is that Cascade Correlation neural networks form an unreliable basis for a surrogate model, despite successes reported in the literature. A further contribution of this work is the enhancement of an open source optimisation toolkit, achieved by the first integration of a truly multi-objective optimisation algorithm.

Visando apoiar o desenvolvimento de controladores para servoposicionadores pneumáticos, é apresentada no presente trabalho uma proposta de um procedimento baseado no uso de redes neurais para a determinação de modelos matemáticos precisos que possam ser aplicados tanto para a simulação do seu comportamento dinâmico quanto na estrutura de controladores que utilizam estratégias baseadas em modelos. No âmbito do trabalho, esse procedimento foi testado por meio de sua aplicação na identificação das forças de atrito e da relação pressão/vazão mássica nos orifícios de controle da servoválvula (que consistem nas principais não-linearidades envolvidas em tais sistemas). Além disso, determinou-se através de redes neurais a relação inversa entre as vazões desejadas e o sinal de controle da servoválvula (difeomorfismo), a qual é aplicada em técnicas de controle baseadas em modelos. Visando validar o procedimento de modelagem proposto, foram realizadas simulações em malha aberta e malha fechada, cujos resultados são comparados com os de experimentos realizados em uma bancada de testes. Com o intuito de comprovar sua eficácia em aplicações de controle, o modelo baseado em redes neurais foi utilizado no desenvolvimento de um controlador não-linear sintetizado de acordo com uma estratégia em cascata (a qual foi já testada em outros trabalhos, mostrando resultados satisfatórios quando aplicada ao controle de servoposicionadores pneumáticos). No entanto, essa estratégia apresenta dificuldades de implantação em decorrência das dificuldades associadas ao processo de identificação dos parâmetros do sistema, que são especialmente trabalhosos neste caso. As características de estabilidade em malha fechada foram analisadas por meio do segundo método Lyapunov. Os resultados experimentais em malha fechada obtidos atestam a eficácia da estratégia de controle proposta.
The development of a precise positioning system has motivated several researches in the pneumatic systems control area to overcome the problems caused by these nonlinearities, by appropriate feedback control algorithms. In this work it is proposed a methodology based on neural networks to achieve accurate mathematical models that can be used in simulation as in controllers techniques based on models. This methodology was tested through its application in identifying the phenomenon of friction and the relationship pressure/mass flow through servo valve orifices control holes. Furthermore, using neural networks, the inverse relationship between the desired flow rates and control signal of servo valve (diffeomorphism), which is applied in various control techniques based on models, was determined. To evaluate the proposed modeling methodology, simulations were done in open and closed loop, and the results were compared with experiments conducted on a real pneumatic servo positioning system. A neural network based model was used to develop a nonlinear controller according to a cascade strategy with friction compensation (which has been tested on other studies showing satisfactory results when applied to pneumatic servo positioning control). The cascade control strategy, despite showing a good performance in trajectory tracking, presents significant difficulties in implementation due mainly to difficulties associated with the system parameters identification process, which are especially expensive. The characteristics of the closed loop stability were analyzed by Lyapunov method. The experimental results obtained in closed loop attest the efficiency of the proposed control strategy.

No presente trabalho, é realizada a modelagem e identificação de um serovoposicionador hidráulico de uma bancada de testes. As expressões analíticas tradicionalmente utilizadas em uma estratégia em cascata aplicada ao controle de trajetória de posição são obtidas. A estratégia em questão utiliza, conjuntamente, a linearização por realimentação como lei de controle do subsistema hidráulico e a lei de controle de Slotine e Li no subsistema mecânico. Com base na mesma estratégia, um controlador em cascata neural é proposto. Em tal controlador, a função analítica que representa o mapa inverso, presente na linearização por realimentação, e a função de compensação de atrito utilizada na lei de Slotine e Li são substituídas por funções constituidas por meio de redes neurais de perceptrons de múltiplas camadas. Essas redes neurais têm como entradas os estados do sistema e também a temperatura do fluido hidráulico. O novo controlador é apresentado em uma versão onde as redes neurais são aplicadas sem modificações on-line e em outra, onde são apresentadas leis de controle adaptativo para as mesmas. A prova de estabilidade do sistema em malha fechada é apresentada em ambos os casos. Resultados experimentais do controle de seguimento de trajetórias de posição em diferentes temperaturas do fluido hidráulico são apresentados. Esses resultados demonstram a maior efetividade do controlador proposto em relação aos controladores clássicos PID e PID+feefforward e ao controlador em cascata com funções analíticas fixas. Os experimentos são realizados em duas situações: quando não ocorrem variações paramétricas importantes no sistema, onde é utilizado o controlador em cascata neural fixo e quando ocorrem essas variações, onde se utiliza o controlador em cascata neural adaptativo.
In this work, the modeling and identification of a hydraulic actuator testing setup are performed and the analytical expressions that are used in a cascade control strategy applyied in a position trajectory tracking control are designed. Such cascade strategy uses the feedback linearization control law in the hydraulical subsystem and the Slotine and Li control law in the mechanical one. Based on this cascade strategy, a neural cascade controller is proposed, for which the analytical function used as inversion set in the feedback linearization control law and the friction function compensation of the Slotine and Li control law are replaced by multi layer perceptrons neural networks where the inputs are the states of the system and the hydraulic fluid temperature. The novel controller is introduced in two different aproachs: the first one where the neural networks do not have on-line modifications and the second one where adaptive control laws are proposed. For both of them the stability proof of the closed-loop system is presented. Experimental results about some position tracking controls performed in different fluid temperature are showed. The results show that the novel controller is more efective than the classical PID, PID+feedforward and the traditional analytical cascade controller. The experiments are performed in two different setups: considering the system without importants parametric variations where is applied the non adaptive cascade neural controller and in the presence of parametric variations where is applied the adaptive cascade neural controller.

Parsing floorplans have been a problem in automatic document analysis for long and have up until recent years been approached with algorithmic methods. With the rise of convolutional neural networks (CNN), this problem too has seen an upswing in performance. In this thesis the task is to recover, as accurately as possible, spatial and geometric information from floorplans. This project builds around instance segmentation models like Cascade Mask R-CNN to extract the bulk of information from a floorplan image. To complement the segmentation, a new style of using keypoint-CNN is presented to find precise locations of corners. These are then combined in a post-processing step to give the resulting segmentation. The resulting segmentation scores exceed the current baseline of the CubiCasa5k floorplan dataset with a mean IoU of 72.7% compared to 57.5%. Further, the mean IoU for individual classes is also improved for almost every class. It is also shown that Cascade Mask R-CNN is better suited than Mask R-CNN for this task.

Life long learning is a machine learning technique that deals with learning sequential tasks over time. It seeks to transfer knowledge from previous learning tasks to new learning tasks in order to increase generalization performance and learning speed. Real-time learning environments in which many agents are participating may provide learning opportunities but they are spread out in time and space outside of the geographical scope of a single learning agent. This research seeks to provide an algorithm and framework for life long learning among a network of agents in a sparse real-time learning environment. This work will utilize the robust knowledge representation of neural networks, and make use of both functional and representational knowledge transfer to accomplish this task. A new generative life long learning algorithm utilizing cascade correlation and reverberating pseudo-rehearsal and incorporating a method for merging divergent life long learning paths will be implemented.
Ph.D.
Doctorate
Electrical Engineering and Computing
Engineering and Computer Science
Computer Engineering

Les méthodes à l’état de l’art de la reconnaissance de l’écriture sont fondées sur des réseaux de neurones récurrents (RNN) à cellules LSTM ayant des performances remarquables. Dans cette thèse, nous proposons deux nouveaux principes la vérification lexicale et la génération de cohorte afin d’attaquer les problèmes de la reconnaissance de l’écriture : i) le problème des grands lexiques et des décodages dirigés par le lexique ii) la problématique de combinaison de modèles optiques pour une meilleure reconnaissance iii) la nécessité de constituer de très grands ensembles de données étiquetées dans un contexte d’apprentissage profond. La vérification lexicale est une alternative aux décodages dirigés par le lexique peu étudiée à cause des faibles performances des modèles optiques historiques (HMM). Nous montrons dans cette thèse qu’elle constitue une alternative intéressante aux approches dirigées par le lexique lorsqu’elles s’appuient sur des modèles optiques très performants comme les RNN LSTM. La génération de cohorte permet de générer facilement et rapidement un grand nombre de réseaux récurrents complémentaires en un seul apprentissage. De ces deux techniques nous construisons et proposons un nouveau schéma de cascade pour la reconnaissance de mots isolés, une nouvelle combinaison au niveau ligne LV-ROVER et une nouvelle stratégie d’auto-apprentissage de RNN LSTM pour la reconnaissance de mots isolés. La cascade proposée permet de combiner avec la vérification lexicale des milliers de réseaux et atteint des résultats à l’état de l’art pour les bases Rimes et IAM. LV-ROVER a une complexité réduite par rapport à l’algorithme original ROVER et permet de combiner des centaines de réseaux sans modèle de langage tout en dépassant l’état de l’art pour la reconnaissance de lignes sur le jeu de donnéesRimes. Notre stratégie d’auto-apprentissage permet d’apprendre à partir d’un seul réseau BLSTM et sans paramètres grâce à la cohorte et la vérification lexicale, elle montre d’excellents résultats sur les bases Rimes et IAM
State-of-the-art methods for handwriting recognition are based on LSTM recurrent neural networks (RNN) which achieve high performance recognition. In this thesis, we propose the lexicon verification and the cohort generation as two new building blocs to tackle the problem of handwriting recognition which are : i) the large vocabulary problem and the use of lexicon driven methods ii) the combination of multiple optical models iii) the need for large labeled dataset for training RNN. The lexicon verification is an alternative to the lexicon driven decoding process and can deal with lexicons of 3 millions words. The cohort generation is a method to get easily and quickly a large number of complementary recurrent neural networks extracted from a single training. From these two new techniques we build and propose a new cascade scheme for isolated word recognition, a new line level combination LV-ROVER and a new self-training strategy to train LSTM RNN for isolated handwritten words recognition. The proposed cascade combines thousands of LSTM RNN with lexicon verification and achieves state-of-the art word recognition performance on the Rimes and IAM datasets. The Lexicon Verified ROVER : LV-ROVER, has a reduce complexity compare to the original ROVER algorithm and combine hundreds of recognizers without language models while achieving state of the art for handwritten line text on the RIMES dataset. Our self-training strategy use both labeled and unlabeled data with the unlabeled data being self-labeled by its own lexicon verified predictions. The strategy enables self-training with a single BLSTM and show excellent results on the Rimes and Iam datasets

Image recognition is the process of classifying a pattern in an image into one of a number of stored classes. It is used in such diverse applications as medical screening, quality control in manufacture and military target recognition. An image recognition system is called shift invariant if a shift of the pattern in the input image produces a proportional shift in the output, meaning that both the class and location of the object in the image are identified. The work presented in this thesis considers a cascade of linear shift invariant optical processors, or correlators, separated by fields of point non-lineari ties, called the cascaded correlator. This is introduced as a method of providing parallel, shiftinvariant, non-linear pattern recognition in a system that can learn in the manner of neural networks. It is shown that if a neural network is constrained to give overall shift invariance, the resulting structure is a cascade of correlators, meaning that the cascaded correlator is the only architecture which will provide fully shift invariant pattern recognition. The issues of training of such a non-linear system are discussed in neural network terms, and the non-linear decisions of the system are investigated. By considering digital simulations of a two-stage system, it is shown that the cascaded correlator is superior to linear filtering for both discrimination and tolerance to image distortion. This is shown for theoretical images and in real-world applications based on fault identification in can manufacture. The cascaded correlator has also been proven as an optical system by implementation in a joint transform correlator architecture. By comparing simulated and optical results, the resulting practical errors are analysed and compensated. It is shown that the optical implementation produces results similar to those of the simulated system, meaning that it is possible to provide a highly non-linear decision using robust parallel optical processing techniques.

The endurance of an aircraft can be increased in the presence of failures by utilising flight control systems that are tolerant to failures. Such systems are known as fault tolerant flight control systems (FTFCS). FTFCS can be implemented by developing failure detection, identification and accommodation (FDIA) schemes. Two of the major types of failures in an aircraft system are the sensor and actuator failures. In this research, a sensor failure detection, identification and accommodation (SFDIA); and an actuator failure detection, identification and accommodation (AFDIA) schemes are developed. These schemes are developed using the artificial neural network (ANN). A number of techniques can be found in the literature that address FDIA in aircraft systems. These techniques are, for example, Kalman filters, fuzzy logic and ANN. This research uses the fully connected cascade (FCC) neural network (NN) for the development of the SFDIA and AFDIA schemes. Based on the study presented in the literature, this NN architecture is compact and efficient in comparison to the multi-layer perceptron (MLP) NN, which is a popular choice for NN applications. This is the first reported instance of the use of the FCC NN for fault tolerance applications, especially in the aerospace domain. For this research, the X-Plane 9 flight simulator is used for data collection and as a test bed. This simulator is well known for its realistic simulations and is certified by the Federal Aviation Administration (FAA) for pilot training. The developed SFDIA scheme adds endurance to an aircraft in the presence of failures in the aircraft pitch, roll and yaw rate gyro sensors. The SFDIA scheme is able to replace a faulty gyro sensor with a FCC NN based estimate, with as few as 2 neurons. In total, 105 failure experiments were conducted, out of which only 1 went undetected. In the developed AFDIA scheme, a FCC NN based roll controller is employed, which uses just 5 neurons. This controller can adapt on-line to the post failure dynamics of the aircraft following a 66\% loss of wing surface. With 66\% of the wing surface missing, the NN based roll controller is able to maintain flight. This is a remarkable display of endurance by the AFDIA scheme, following such a severe failure. The results presented in this research validate the use of FCC NNs for SFDIA and AFDIA applications.

Two classes of recurrent neural network models are compared in this report, simple recurrent networks (SRNs) and sequential cascaded networks (SCNs) which are first- and second-order networks respectively. The comparison is aimed at describing and analysing the behaviour of the networks such that the differences between them become clear. A theoretical analysis, using techniques from dynamic systems theory (DST), shows that the second-order network has more possibilities in terms of dynamical behaviours than the first-order network. It also revealed that the second order network could interpret its context with an input-dependent function in the output nodes. The experiments were based on training with backpropagation (BP) and an evolutionary algorithm (EA) on the AnBn-grammar which requires the ability to count. This analysis revealed some differences between the two training-regimes tested and also between the performance of the two types of networks. The EA was found to be far more reliable than BP in this domain. Another important finding from the experiments was that although the SCN had more possibilities than the SRN in how it could solve the problem, these were not exploited in the domain tested in this project

The project is focused on face detection in video. Firstly, it contains a summary of basic color models. Secondly, you can find the description and comparison of the basic methods for detection of human skin with a practical example of implementation of parametric detector. Thirdly, a theoretical basis for face detection and face tracking in a video containing a list of basic concepts and methods of this issue follows. Greater emphasis is placed on the description of machine learning algorithm AdaBoost and description of the possible application of the Kalman filter for the purpose of face tracking. Design, implementation and testing of library accomplished within the master thesis are listed in the final part of this thesis.

Den klassiska approachen till navigering innefattar att agenten håller en intern representativ modell av omgivningen. Denna approach har emellertid många nackdelar, speciellt för dynamiska miljöer. En modernare approach är att förlita sig på den faktiska omgivningen istället för en modell av denna. Detta arbete presenterar en undersökning av navigeringsproblemet och hur väl det löses av agenter vars kontrollmekanismer utgörs artificiella neurala nätverk. Tillförlitligheten hos de två neurala arkitekturerna Extended sequential cascaded network och Self-organized recurrent network bestäms genom experiment. Det visas i experimenten att Extended sequential cascaded network är den mest tillförlitliga arkitekturen av de två när navigeringsproblemet skall angripas. Det visas även att Extended sequential cascaded network tränar fram ett helt reaktivt beteende i samtliga experiment. Slutsatsen som kan dras av detta är att svåra problem inte alltid kräver avancerade arkitekturer för att lösastillfredsställande.

Дисертацію присвячено розробці моделей та алгоритмів аналізу фінансового стану та прогнозування ризику банкрутства підприємств та банків в умовах невизначеності, неповної та недостовірної інформації на прикладі економіки України. Проаналізовано класичні статистичні методи прогнозування ризику банкрутства підприємств на основі методів багатовимірного дискримінантного аналізу, зокрема метод Альтмана. Виявлено його недоліки та недоцільність використання в умовах економіки України, оскільки він базується на використанні достовірної інформації про стан підприємств. Тому в роботі обгрунтовано використання для прогнозування ризику банкрутства в умовах неповноти та невизначеності нечітких нейронних мереж (НММ) з виведеннями Мамдані та Цукамото. В дисертації розроблено базу правил для вирішення задачі аналізу фінансового стану та прогнозування ризику банкрутства підприємств в умовах невизначеності для нейромереж Мамдані та Цукомото. Оскільки загальний розмір повної бази нечітких правил великий, що не дає можливості її навчання за короткий час, запропоновано спосіб скорочення розмірів бази правил та її наглядне представлення шляхом використання бальних оцінок. Розроблено алгоритми прогнозування ризику банкрутства підприємств з використанням ННМ Мамдані та Цукамото. Далі в роботі розглянуто нео-фаззі каскадні мережі для аналізу фінансового стану та прогнозуванню ризику банкрутства підприємств в умовах невизначеності. Їх особливостями є відсутність бази правил висновку, а також те, що функції належності фіксовані і не потребують навчання, навчаються лише лінійні параметри – ваги зв’язків ННМ. Тому ці мережі мають прискорену збіжність навчання в порівнянні з ННМ з висновками Мамдані та Цукамото. Проведено експериментальні дослідження запропонованих моделей та алгоритмів для прогнозування ризику банкрутства підприємств України та порівняльний аналіз з класичними методами. Результати експериментів показали, що точність прогнозування ризику банкрутства складає методом Альтмана - 68-70%, матричним методом - 80%, нео-фазі каскадною нейромережею - 87%, а ННМ Мамдані та Цукамото -88-90 %. В роботі також було досліджено проблему прогнозування ризику банкрутства в банківській сфері України в умовах невизначеності. Для вирішення цієї проблеми запропоновано використання ННМ TSK та ANFIS. Проведено експериментальні дослідження ефективності використання ННМ для прогнозування ризику банкрутства банків та порівняння зі статистичними моделями ARIMA, logit-model та probit–model, а також із нечітким МГУА. В результаті експериментів встановлено, що найбільшу точність прогнозування забезпечує використання ННМ TSK (2%) та нечіткого МГУА (4%), тоді як статистичні моделі мають точність: logit-model - 16%, probit –model - 14%) та ARIMA - 18%. В процесі експериментів також було визначено адекватні фінансово-економічні показники банків для прогнозування ризику банкрутства.
The thesis is devoted to the development of models and algorithms for analysis of financial state and forecasting of bankruptcy risk of enterprises and banks in condition of uncertainty, incomplete and unreliable information on the example of the Ukrainian economy. Classical statistical methods for predicting the risk of bankruptcy on the basis of multivariate discriminant analysis, in particular the method of Altman, are analyzed. It revealed its deficiencies and inappropriateness of its use in Ukraine's economy, since it is based on the use of reliable information on the state enterprises. Therefore, the use of fuzzy neural networks (FNN) with the conclusions Mamdani and Tsukamoto to forecast the risk of bankruptcy in the conditions of incompleteness and uncertainty is entirely justified. In the thesis rule base is developed for solving the problem of financial analysis and forecasting the risk of bankruptcy of enterprises for neural networks Mamdani and Tsukamoto. Since the total size of the comprehensive fuzzy rule base is great that does not allow its training in a short time, a method of reducing the size of the rule base and its visual representation through the use of scores is suggested. Algorithms for predicting the risk of bankruptcy of enterprises with FNN Mamdani and Tsukamoto are developed. Further in the paper the cascade neo-fuzzy network (CNFN) for predicting the risk of bankruptcy in condition of uncertainty is suggested. Its features is the absence of the rule base, as well as the fact that the membership functions are fixed and does not need training. Therefore, these networks have accelerated the convergence of training compared with FNN Mamdani and Tsukamoto. Experimental studies of the proposed models and algorithms for the forecasting of the risk of bankruptcy in Ukraine and comparative analysis with classical methods are presented. The experimental results showed that the accuracy of predicting the bankruptcy risk by Altmana- by 68- 70%, matrix method - 80%, cascade neo-fuzzy neural network - 87% and FNN Mamdanі and Tsukamoto - 88-90%. The paper also studied the problem of forecasting the risk of bankruptcy in the banking sector of Ukraine in conditions of uncertainty. To solve this problem using FNN TSK and ANFIS is proposed. Experimental research of effectiveness of using FNN to predict the risk of bank failures and comparison with statistical models ARIMA, logit-model, probit-model and fuzzy GMDH are presented. The experiment established that the greatest prediction accuracy allows the use of FNN TSK (2%) and fuzzy GMDH (4%), while the statistical models: logit-model - 16%, probit-model - 14% and ARIMA - 18%. During the experiments adequate financial and economic indicators of banks to predict the risk of bankruptcy were determined.
Диссертация посвящена разработке моделей и алгоритмов анализа финансового состояния и прогнозирования риска банкротства предприятий и банков в условиях неопределенности, неполной и недостоверной информации на примере экономики Украины. Проанализированы классические статистические методы прогнозирования риска банкротства предприятий на основе методов многомерного дискриминантного анализа, в частности метод Альтмана. Выявлено его недостатки и нецелесообразность использования в условиях экономики Украины, поскольку он базируется на использовании достоверной информации о состоянии предприятий. Поэтому в работе обосновано использование для прогнозирования риска банкротства в условиях неполноты и неопределенности нечетких нейронных сетей (ННС) с выводами Мамдани и Цукамото. В дисертации разработана база правил для решения задачи анализа финансового состояния и прогнозирования риска банкротства предприятий в условиях неопределенности для нейросетей Мамдани и Цукамото. Поскольку общий размер полной базы нечетких правил большой, что не дает возможности ее обучения за короткое время, предложен способ сокращения размеров базы правил и ее наглядное представление путем использования балльных оценок. Разработаны алгоритмы прогнозирования риска банкротства предприятий с использованием ННС Мамдани и Цукамото. Далее в работе рассмотрены каскадные нео-фаззи сети для прогнозирования риска банкротства предприятий в условиях неопределенности. Их особенностями является отсутствие базы правил вывода, а также то, что функции принадлежностей фиксированные и не нуждаются в обучении, обучаются лишь линейные параметры – веса связей ННС. Поэтому эти сети имеют ускоренную сходимость обучения в сравнении с ННС Мамдани и Цукамото. Проведены экспериментальные исследования предложенных моделей и алгоритмов для прогнозирования риска банкротства предприятий Украины и сравнительный анализ с классическими методами. Результаты экспериментов показали, что точность прогнозирования риска банкротства составляет методом Альтмана - 68-70%, матричным методом - 80%, нео-фаззи каскадной нейросетью - 87%, а ННМ Мамдани и Цукамото -88-90 %. В работе также была исследована проблема прогнозирования риска банкротства в банковской сфере Украины в условиях неопределенности. Для решения этой проблемы предложено использование ННС TSK и ANFIS. Проведены экспериментальные исследования эффективности использования ННС для прогнозирования риска банкротства банков и сравнение со статистическими моделями ARIMA, logit-model и probit–model, а также с нечетким МГУА. В результате экспериментов установлено, что самую большую точность прогнозирования обеспечивает использование ННМ TSK (2%) и нечеткий МГУА (4%), тогда как статистические модели имеют точность: logit-model - 16%, probit–model - 14% и ARIMA - 18%. В процессе экспериментов были также определены адекватные финансово-экономические показатели банков для прогнозирования риска банкротства.

A otimização estrutural, utilizando ferramentas computacionais é um grande campo de pesquisa na atualidade. Os métodos utilizados, dependendo da complexidade do problema, demandam um grande custo computacional, e por isso vem sendo avaliandas várias técnicas para diminuí-lo. Uma delas é o emprego de técnicas de aproximação de análises, dentre as quais destacam-se as redes neurais, que combinadas aos métodos de otimização e de análises clássicos conseguem bons resultados e reduzem significativamente o tempo de processamento. O emprego dos compósitos laminados como material estrutural vem crescendo nos últimos tempos, incentivado pela suas excelentes propriedades mecânicas e baixo peso. Em consenso com todo o esforço científico dedicado a essa área, o presente trabalho visa a implementação de uma ferramenta computacional capaz de otimizar estruturas complexas fabricadas com tais materiais, a um baixo custo computacional. Com isto em mente, é desenvolvido um sistema de otimização, aproveitando módulos implementados previamente para a análise estática linear e não linear através do método dos elementos finitos (MEF), e o módulo de otimização por algoritmos genéticos. Serão desenvolvidos os módulos de análise modal, para otimizar também estruturas com critérios baseados em freqüências e modos, e o modulo de redes neurais de tipo perceptron para aproximações das análises feitas através do MEF. Alguns exemplos são apresentados para demonstrar que bons resultados são obtidos com a utilização de redes neurais artificiais, cujo treinamento permite poupar tempo computacional proveniente do grande número de análises usualmente necessárias no processo de otimização.
Structural optimization using computational tools has become a major research field in recent years. Methods commonly used in structural analysis and optimization may demand considerable computational cost, depending on the problem complexity. Therefore, many techniques have been evaluated in order to diminish such impact. Among these various techniques, artificial neural networks may be considered as one of the main alternatives, when combined with classic analysis and optimization methods, to reduce the computational effort without affecting the final solution quality. Use of laminated composite structures has been continuously growing in the last decades due to the excellent mechanical properties and low weight characterizing these materials. Taken into account the increasing scientific effort in the different topics of this area, the aim of the present work is the formulation and implementation of a computational code to optimize manufactured complex laminated structures with a relatively low computational cost by combining the Finite Element Method (FEM) for structural analysis, Genetic Algorithms (GA) for structural optimization and Artificial Neural Networks (ANN) to approximate the finite element solutions. The modules for linear and geometrically non-linear static finite element analysis and for optimize laminated composite plates and shells, using GA, were previously implemented. Here, the finite element module is extended to analyze dynamic responses to optimize problems based in frequencies and modal criteria, and a module with perceptron ANN is added to approximate finite element analyses. Several examples are presented to show the effectiveness of ANN to approximate solutions obtained using the FEM and to reduce significatively the computational cost.

Weather radar imagery is important for several remote sensing applications including tracking of storm fronts and radar echo classification. In particular, tracking of precipitation events is useful for both forecasting and classification of rain/non-rain events since non-rain events usually appear to be static compared to rain events. Recent weather radar imaging-based forecasting approaches [3] consider that precipitation events can be modeled as a combination of localized functions using Radial Basis Function Neural Networks (RBFNNs). Tracking of rain events can be performed by tracking the parameters of these localized functions. The RBFNN-based techniques used in forecasting are not only computationally expensive, but also moderately effective in modeling small size precipitation events. In this thesis, an existing RBFNN technique [3] was implemented to verify its computational efficiency and forecasting effectiveness. The feasibility of modeling precipitation events using RBFNN effectively was evaluated, and several modifications to the existing technique have been proposed.

The aim of this master thesis is to design and implementation in language C++ a system for the detection, tracking and classification of vehicles from streams or records from traffic cameras. The system runs on the platform Robot Operating System and uses the OpenCV, FFmpeg, TensorFlow and Keras libraries. For detection is used cascade classifier, for tracking Kalman filter and for classification of the convolutional neural network. Success rate for detection is 91.93 %, tracking 81.94 % and classification 63.72 %. This system is part of a comprehensive system, that can moreover calibrate video and measure of vehicles speed. The resulting system can be used for traffic analysis.

The aim of this master thesis is to design and implement a system for the detection, tracking and classification of vehicles from streams or records from traffic cameras in language C++. The system runs on the platform Robot Operating System and uses the OpenCV, FFmpeg, TensorFlow and Keras libraries. For detection cascade classifier is used, for tracking Kalman filter and for classification of the convolutional neural network. Out of a total of 627 cars, 479 were tracked correctly. From this number 458 were classified (trucks or lorries not included). The resulting system can be used for traffic analysis.

The thesis deals with the use of neuronal networks for the control of telecommunication network elements. The aim of the thesis is to create a simulation model of network element with switching array with memory, in which the optimization kontrol switching array is solved by means of the neural network. All source code is created in integrated environment MATLAB. To training are used feed-forward backpropagation network. Miss achieve satisfactory result mistakes. Work apposite decision procedure given to problem and it is possible on ni tie up in an effort to find optimum solving.

This thesis seeks to address current problems encountered when training classifiers within the framework of cascades of boosted ensembles (CoBE). At present, a signifi- cant challenge facing this framework are inordinate classifier training runtimes. In some cases, it can take days or weeks (Viola and Jones, 2004; Verschae et al., 2008) to train a classifier. The protracted training runtimes are an obstacle to the wider use of this framework (Brubaker et al., 2006). They also hinder the process of producing effective object detection applications and make the testing of new theories and algorithms, as well as verifications of others research, a considerable challenge (McCane and Novins, 2003). An additional shortcoming of the CoBE framework is its limited ability to train clas- sifiers incrementally. Presently, the most reliable method of integrating new dataset in- formation into an existing classifier, is to re-train a classifier from beginning using the combined new and old datasets. This process is inefficient. It lacks scalability and dis- cards valuable information learned in previous training. To deal with these challenges, this thesis extends on the research by Barczak et al. (2008), and presents alternative CoBE frameworks for training classifiers. The alterna- tive frameworks reduce training runtimes by an order of magnitude over common CoBE frameworks and introduce additional tractability to the process. They achieve this, while preserving the generalization ability of their classifiers. This research also introduces a new framework for incrementally training CoBE clas- sifiers and shows how this can be done without re-training classifiers from beginning. However, the incremental framework for CoBEs has some limitations. Although it is able to improve the positive detection rates of existing classifiers, currently it is unable to lower their false detection rates.

Plusieurs méthodes ont été développées par l'Intelligence Artificielle pour reproduire certains aspects de l'intelligence humaine. Ces méthodes permettent de simuler les processus de raisonnement en s'appuyant sur les connaissances de base disponibles. Chaque méthode comporte des points forts, mais aussi des limitations. La réalisation de systèmes hybrides est une démarche courante Qui permet de combiner les points forts de chaque approche, et d'obtenir ainsi des performances plus élevées ou un champ d'application plus large. Un autre aspect très important du développement des systèmes hybrides intelligents est leur capacité d'acquérir de nouvelles connaissances à partir de plusieurs sources différentes et de les faire évoluer. Dans cette thèse, nous avons développé des recherches sur les systèmes hybrides neuro-symboliques, et en particulier sur l'acquisition incrémentale de connaissances à partir de connaissances théoriques (règles) et empiriques (exemples). Un nouveau système hybride, nommé système INSS - Incremental Neuro-Symbolic System, a été étudié et réalisé. Ce système permet le transfert de connaissances déclaratives (règles symboliques) d'un module symbolique vers un module connexionniste (réseau de neurones artificiel - RNA) à travers un convertisseur de règles en réseau. Les connaissances du réseau ainsi obtenu sont affinées par un processus d'apprentissage à partir d'exemples. Ce raffinement se fait soit par ajout de nouvelles connaissances, soit par correction des incohérences, grâce à l'utilisation d'un réseau constructif de type Cascade-Correlation. Une méthode d'extraction incrémentale de règles a été intégrée au système INSS, ainsi que des algorithmes de validation des connaissances qui ont permis de mieux coupler les modules connexionniste et symbolique. Le système d'apprentissage automatique INSS a été conçu pour l'acquisition constructive (incrémentale) de connaissances. Le système a été testé sur plusieurs applications, en utilisant des problèmes académiques et des problèmes réels (diagnostic médical, modélisation cognitive et contrôle d'un robot autonome). Les résultats montrent que le système INSS a des performances supérieures et de nombreux avantages par rapport aux autres systèmes hybrides du même type
Various Artificial Intelligence methods have been developed to reproduce intelligent human behaviour. These methods allow to reproduce some human reasoning process using the available knowledge. Each method has its advantages, but also some drawbacks. Hybrid systems combine different approaches in order to take advantage of their respective strengths. These hybrid intelligent systems also present the ability to acquire new knowledge from different sources and so to improve their application performance. This thesis presents our research in the field of hybrid neuro-symbolic systems, and in particular the study of machine learning tools used for constructive knowledge acquisition. We are interested in the automatic acquisition of theoretical knowledge (rules) and empirical knowledge (examples). We present a new hybrid system we implemented: INSS - Incremental Neuro-Symbolic System. This system allows knowledge transfer from the symbolic module to the connectionist module (Artificial Neural Network - ANN), through symbolic rule compilation into an ANN. We can refine the initial ANN knowledge through neural learning using a set of examples. The incremental ANN learning method used, the Cascade-Correlation algorithm, allows us to change or to add new knowledge to the network. Then, the system can also extract modified (or new) symbolic rules from the ANN and validate them. INSS is a hybrid machine learning system that implements a constructive knowledge acquisition method. We conclude by showing the results we obtained with this system in different application domains: ANN artificial problems(The Monk's Problems), computer aided medical diagnosis (Toxic Comas), a cognitive modelling task (The Balance Scale Problem) and autonomous robot control. The results we obtained show the improved performance of INSS and its advantages over others hybrid neuro-symbolic systems

This thesis aims at proposing vehicle license plate detection and recognition algorithms, suitable for vehicle re-identification. Simple urban traffic analysis system is also proposed. Multiple stages of this system was developed and tested. Specifically - vehicle detection, license plate detection and recognition. Vehicle detection is based on background substraction method, which results in an average hit rate of ~92%. License plate detection is done by cascade classifiers and achieves an average hit rate of 81.92% and precision rate of 94.42%. License plate recognition based on Template matching results in an average precission rate of 60.55%. Therefore the new license plate recognition method based on license plate scanning using the sliding window principle and neural network recognition was introduced. Neural network achieves a precision rate of 64.47% for five input features. Low precision rate of neural network is caused by small amount of training sample for some specific license plate characters.

Tese de doutoramento em Engenharia Eletrotécnica e de Computadores, no ramo de especialização em Automação e Robótica, apresentada à Faculdade de Ciências e Tecnologia da Universidade de Coimbra
Esta tese concentra-se em reconhecimento de padrões, com particular ênfase para o con ito de escolha entre capacidade de generalização e custo computacional, a m de fornecer suporte para aplicações em tempo real. Neste contexto são apresentadas contribuições metodológicas e analíticas para a abordagem de dois tipos de datasets: balanceados e desbalanceados. Um dataset é denominado balanceado quando há um número aproximadamente igual de observações entre as classes, enquanto datasets que têm números desiguais de observações entre as classes são denominados desbalanceados, tal como ocorre no caso de detecção de objetos baseada em imagem. Para datasets balanceados é adoptado o perceptrão multicamada (MLP) como classi cador, uma vez que tal modelo é um aproximador universal, ou seja MLPs podem aproximar qualquer conjunto de dados. Portanto, ao invés de propor novos modelos de classi cadores, esta tese concentra-se no desenvolvimento e análise de novos métodos de treinamento para MLP, de forma a melhorar a sua capacidade de generalização através do estudo de quatro abordagens diferentes: maximização da margem de classi cação, redundância, regularização, e transdução. A idéia é explorar novos métodos de treino para MLP com vista a obter classi cadores não-lineares mais rápidos que o usual SVM com kernel não-linear, mas com capacidade de generalização similar. Devido à sua função de decisão, o SVM com kernel não-linear exige um esforço computacional elevado quando o número de vetores de suporte é grande. No contexto dos datasets desbalanceados, adotou-se classi cadores em cascata, já que tal modelo pode ser visto como uma árvore de decisão degenerativa que realiza rejeições em cascata, mantendo o tempo de processamento adequado para aplicações em tempo real. Tendo em conta que conjuntos de classi cadores são susceptíveis a ter alta dimensão VC, que pode levar ao over- tting dos dados de treino, foram deduzidos limites para a capacidade de generalização dos classi cadores em cascata, a m de suportar a aplicação do princípio da minimização do risco estrutural (SRM). Esta tese também apresenta contribuições na seleção de características e dados de treinamento, devido à forte in uência que o pre-processamento dos dados tem sobre o reconhecimento de padrões. Os métodos propostos nesta tese foram validados em vários datasets do banco de dados da UCI. Alguns resultados experimentais já podem ser consultados em três revistas da ISI, outros foram submetidos a duas revistas e ainda estão em processo de revisão. No entanto, o estudo de caso desta tese é limitado à detecção e classi cação de peões.
This thesis focuses on pattern recognition, with particular emphasis on the trade o between generalization capability and computational cost, in order to provide support for on-the- y applications. Within this context, two types of datasets are analyzed: balanced and unbalanced. A dataset is categorized as balanced when there are approximately equal numbers of observations in the classes, while unbalanced datasets have unequal numbers of observations in the classes, such as occurs in case of imagebased object detection. For balanced datasets it is adopted the multilayer perceptron (MLP) as classi er, since such model is a universal approximator, i.e. MLPs can t any dataset. Therefore, rather than proposing new classi er models, this thesis focuses on developing and analysing new training methods for MLP, in order to improve its generalization capability by exploiting four di erent approaches: maximization of the classi cation margin, redundancy, regularization, and transduction. The idea is to exploit new training methods for MLP aiming at an nonlinear classi er faster than the usual SVM with nonlinear kernel, but with similar generalization capability. Note that, due to its decision function, the SVM with nonlinear kernel requires a high computational e ort when the number of support vectors is big. For unbalanced datasets it is adopted the cascade classi er scheme, since such model can be seen as a degenerate decision tree that performs sequential rejection, keeping the processing time suitable for on-the- y applications. Taking into account that classi er ensembles are likely to have high VC dimension, which may lead to over- tting the training data, it were derived generalization bounds for cascade classi ers, in order to support the application of structural risk minimization (SRM) principle. This thesis also presents contributions on feature and data selection, due to the strong in uence that data pre-processing has on pattern recognition. The methods proposed in this thesis were validated through experiments on several UCI benchmark datasets. Some experimental results can be found in three ISI journals, others has been already submitted to two ISI journals, and are under review. However, the case study of this thesis is limited to pedestrian detection and classi cation.

碩士
國立臺北科技大學
能源與冷凍空調工程系
106
Three methods are applied in this study: linear regression, backpropagation network and cascade forward backpropagation network. The power consumption models, before cleaning the condenser, are established by using these three methods. We collect data of the chillers after cleaning condenser, then simulate the power consumption of the chillers before cleaning by using these three models. After that, the simulate results and improvement of performance are analyzed and compared by using three methods under the same baseline. In this study, the model established by cascade forward backpropagation network is more accurate and have less error than the models established by linear regression and backpropagation network. In the result, we realized that the effects of energy saving after we cleaned the condenser in case 1 is about 5.1% and in case 2 is 3.77%. In case 3, the results show that replacing hydrocarbon refrigerant R-290 can get about 25% of energy saving. Generally, using neural network to do simulation will get high accuracy. Cascade forward backpropagation network can consider the original input so that can make the results more accurate.

Evolution of neural networks, or neuroevolution, has been a successful approach to many low-level control problems such as pole balancing, vehicle control, and collision warning. However, certain types of problems — such as those involving strategic decision-making — have remained difficult to solve. This dissertation proposes the hypothesis that such problems are difficult because they are fractured: The correct action varies discontinuously as the agent moves from state to state. To evaluate this hypothesis, a method for measuring fracture using the concept of function variation of optimal policies is proposed. This metric is used to evaluate a popular neuroevolution algorithm, NEAT, empirically on a set of fractured problems. The results show that (1) NEAT does not usually perform well on such problems, and (2) the reason is that NEAT does not usually generate local decision regions, which would be useful in constructing a fractured decision boundary. To address this issue, two neuroevolution algorithms that model local decision regions are proposed: RBF-NEAT, which biases structural search by adding basis-function nodes, and Cascade-NEAT, which constrains structural search by constructing cascaded topologies. These algorithms are compared to NEAT on a set of fractured problems, demonstrating that this approach can improve performance significantly. A meta-level algorithm, SNAP-NEAT, is then developed to combine the strengths of NEAT, RBF-NEAT, and Cascade-NEAT. An evaluation in a set of benchmark problems shows that it is possible to achieve good performance even when it is not known a priori whether a problem is fractured or not. A final empirical comparison of these methods demonstrates that they can scale up to real-world tasks like keepaway and half-field soccer. These results shed new light on why constructive neuroevolution algorithms have difficulty in certain domains and illustrate how bias and constraint can be used to improve performance. Thus, this dissertation shows how neuroevolution can be scaled up from learning low-level control to learning strategic decision-making problems.
text

碩士
國立中央大學
資訊工程學系
106
In recent years, thanks to the development of CNN (convolutional neural network), researchers have made great progress on face detection and face recognition. Many unique and novel network structures have been proposed to solve different face detection or recognition problems. To use which network structure depends on the application, for example, we only need to perform face recognition on an image with only one face at customs. However, in monitoring or access control system, we need to perform face detection first to find where faces are and then recognize every faces. We propose a CNN structure which combines face detection and face recognition. We use the RPN structure from Faster R-CNN to propose candidate regions which may be faces. We then use a coarse-to-fine cascaded CNN to check each candidate regions and filter out the regions which are not faces. By using RPN structure instead of using sliding widow to propose candidate region, we can avoid checking regions in every sizes and at every places one by one. The system needs only 0.08 seconds with RPN structure, compared to 0.18 seconds with sliding window method, we get better execution speed, and the detection capability remains nearly the same. After finishing face detection, we then use FaceNet to extract features for recognition. Due to the definition of the loss function, the distance between two feature vectors extracted from two facial images can reflect the similarity of the two facial images. That is, we can recognize faces by only calculate the distance between feature vectors without using any complex classifiers, which allows us to use the same recognition system in different situations. The recognition accuracy of the proposed method can reach 97%, which is slightly lower than the methods that need to be retrained. However, considering the convenience of using the same recognition system without retraining, we think it’s still a great deal.

碩士
國立暨南國際大學
電機工程學系
105
Facial gender classification based on deep convolutional neural networks (DCNNs) cascaded with discrete wavelet transform (DWT) is a work in biometrics by means of the deep learning (DL) architecture. The applied DL architecture is convolutional neural networks (CNNs) which is appropriate to image processing. Biometrics is a subject of pattern recognition while deep learning is a novel machine learning architecture, which is considered the most powerful classifier among the state-of-the-art. The architecture of this work is that three specified channels of CNNs connected in parallel are cascaded with three frequency bands of DWT, respectively. The DWT process extracts critical features from input faces by which the number of kernel of the CNNs can be reduced. This will reduce the training time and the impact of random. Adience 3D dataset is applied as the benchmark.

碩士
國立臺北科技大學
能源與冷凍空調工程系
107
Manual method currently used is not efficient, resulting in wasted large amounts of energy. If we can use optimal chiller loading method to meet the system requirements , the total power consumption of the chillers will be minimized. This study uses the cascade forward backpropagation network to establish chiller power consumption models, which considers the operating constrains of each chillers. Simulated annealing is also integrated while satisfying cooling load conditions to optimize chiller loading.The simulation result show that integrating cascade forward backpropagation network and simulated annealing between 95% and 55% cooling loads improved power saving compared to manual method load distribution, saving maximum total power consumption by approximately 19% in75% load. Compared with the application of cascade forward backpropagation network integrated with genetic algorithm, the maximum error is about 3.4%, The results of the two algorithms are not much different, but the calculation time difference is about more than 1 minute.

碩士
國立臺灣科技大學
機械工程系
87
Chemical-mechanical planarization is a technique that used to polish the surface of a wafer and make it planner. In the planarization process, the major factors such as material remove rate, non-unfority and wafer surface reside stress will be changed with the process parameters set up. It will inference the yield rate of CMP process. In this reserch both of the 3D ocmputer simulation model of CMP, as well as a Neural Network was used to simulate and predict the resoult of CMP, so that a 3D polish mechanism model of CMP was constructed by CAD. This model was used to analyze and simulate the polish conditions. In the same time based on the theory of Neural Network which is the cascade correction learning model, with the same parameters used in the 3D mechanism model such as revolution rate of the carrier, back pressure of carieer, revolution rate of plan and characteristics of slurry etc. as the input parameters of the Neural Network, to training this Network, and form a smart Neural Network. According to both CAD model and Neural Network model, can simulate the CMP process, and optimize the input parameters for the real CMP operation.

碩士
國立交通大學
資訊管理研究所
94
Previous researchs claimed that to forecast the index movement by using the neural network, massive amount of waiting time was normally mandatory. The situation gets worse if larger training sets and more hidden units come into the iteration. Thisphenomenon explains why Backprop neural network takes time in learning process. In this paper, we propose a modified version of Backprop neural network, a cascade-correlation neural network, to simplify the configuration of experiment and shorten the running time for deriving the forcast result. Experiment shows that under the same working environment, cascade-correlation neural network outperforms Backprop neural network. Profits for cascade correlationneural network, especially, exhibit a steady increase through the tesing period, while profits for Backprop neural network flucturates, despite the fact that they both earn positive profits.

碩士
國立中興大學
機械工程學系
86
"類神經網路"以其良好之非線性映射能力，己經被廣泛的應用於系統模式 建立、模糊隸屬函數調整、文字辨識及複雜之非線性系統控制 ?等方面 。本論文主要目的乃是希望將類神經網路應用於極非線性之射出成型機料 管系統鑑別與精密溫度控制上。為了對料管系統做更完整之鑑別，本研究 嘗試在類神經網路EBP(誤差逆傳遞學習法則)架構之下，由一階逼進著手 ，在簡單的原則下，以最陡負梯度法則及參數上下限設定的方式，針對每 一次權值更新，找出增進模式或批次模式相對應之最佳學習參數 及學習 慣量 參數值，使得網路能在最少的次數下收斂，且能有不錯的重現性， 電腦模擬證明本研究所提出之新法則在收斂速度與學習成功率上，皆較其 他方法優異。在本研究之第二部分，我們先將所發展之新類神經網路學習 法則應用於射出成型機料管系統之模擬上，實際實驗結果證明，本研究所 提出之新方法確實可對複雜之料管系統做有效的鑑別。完成之類神經網路 料管模式則進一步用來做為類神經網路式參數自調PID控制器之依據，實 際實驗結果亦證明此參數自調PID控制器能將料管溫度控制於 +0.5~-0.5 度內 Artificial neural networks, with its high learning and nonlinear mapping ability, have been successfully applied to many system identification and control problems. The goal of this thesis is to apply the neural network techniques to the system identification and precise temperature control of the extremely nonlinear injection molding barrel system.In order to complete the system identification work as accuracy as possible, we propose a new and efficient multilayer neural network learning algorithm first. In this new learning algorithm (Cascade steepest descendant learning algorithm) the steepest descendant method is used to search the optimal learning constant ηand momentum term αfor each weights updating process. The well known Delta learning rule is then employed to modify the connecting weights in terms of the optimal ηand α. Computer simulations show that the proposed new algorithm outmatches other learning algorithms both in converging speed and success rate.In the second part of this research, we first apply the new neural network learning algorithm to the identification of the injection molding barrel system. Real experiment results demonstrate that the new algorithm can precisely identify the complicate barrel system. Further more, a self-tuning PID controller based on the trained neural network barrel model for precise temperature control is developed. Real experiments show that the proposed self-tuning PID controller can precisely control the barrel temperature within 0.5 degree.