Tesi sul tema "Speech reinforcement"

This dissertation describes the development of an evaluation tool that can be used by an acoustican during the design stage of enclosures used for unassisted speech. Enclosures include lecture theatres, lecture halls and speech auditoriums. The tool is designed to enable Acousticians to be able to manipulate various acoustical parameters such as the geometry and the materials or construction selection to gauge the impact on speech performance. The tool can also be used to evaluate the performance of speech privacy within spaces using the Speech Transmission Index. Computer simulation tools have a number of advantages over existing methods such as physical scale models for this type of evaluation. Typical advantages are in the elimination of the difficult selection of materials with appropriate scale model acoustic performance, resolution of air absorption at scale model frequencies, reduced cost in development of the model, no storage space problems, ease of modifying and duplicating the model. Scale models also present difficulties in measuring some of the indices such as Speech Transmission Index. Whilst equipment can be purchased for the measurement of STI, scale model equivalents and the impact of the change in frequencies and modulations have not been researched or published.Currently, there are only two methods of evaluating the Speech Transmission of an enclosure: Build a full size enclosure and test; or simulate mathematically to derive the performance. At the time this thesis was commenced there were no commercial simulation programs available that could derive Speech Transmission Index information. The evaluation tool has been implemented as a computer program, based on IBM PC type computers running Microsoft WINDOWS 3.1 or later. The implementation uses the image method for the 'ray trace' algorithm. This basic image method utilises the enhancements made by a number of authors. In particular the Transformation Matrix method and homogenous coordinates have been used to improve the speed of the algorithm. Pre-computation of mutually invisible planes allows trimming the number of possible combination of rays that need to be computed. Results of physical measurement from two case studies have been compared to results of the simulation. Good correlation between the simulations and the case studies were achieved for the Speech Transmission Index and RASTI values. The accuracy of the simulation,in terms of decay based indices, is limited by the lack of sufficient tail to the calculated number of rays. Further research and implementation of hybrid techniques utilising both the image method and more traditional ray-tracing algorithms to improve the quality of the calculated decay data are required. Investigation of techniques used in photo-realism 'ray-tracing' may result in far more realistic data which is the basic input to the Speech Transmission Index calculations.

This dissertation describes the development of an evaluation tool that can be used by an acoustican during the design stage of enclosures used for unassisted speech. Enclosures include lecture theatres, lecture halls and speech auditoriums. The tool is designed to enable Acousticians to be able to manipulate various acoustical parameters such as the geometry and the materials or construction selection to gauge the impact on speech performance. The tool can also be used to evaluate the performance of speech privacy within spaces using the Speech Transmission Index. Computer simulation tools have a number of advantages over existing methods such as physical scale models for this type of evaluation. Typical advantages are in the elimination of the difficult selection of materials with appropriate scale model acoustic performance, resolution of air absorption at scale model frequencies, reduced cost in development of the model, no storage space problems, ease of modifying and duplicating the model. Scale models also present difficulties in measuring some of the indices such as Speech Transmission Index. Whilst equipment can be purchased for the measurement of STI, scale model equivalents and the impact of the change in frequencies and modulations have not been researched or published.
Currently, there are only two methods of evaluating the Speech Transmission of an enclosure: Build a full size enclosure and test; or simulate mathematically to derive the performance. At the time this thesis was commenced there were no commercial simulation programs available that could derive Speech Transmission Index information. The evaluation tool has been implemented as a computer program, based on IBM PC type computers running Microsoft WINDOWS 3.1 or later. The implementation uses the image method for the 'ray trace' algorithm. This basic image method utilises the enhancements made by a number of authors. In particular the Transformation Matrix method and homogenous coordinates have been used to improve the speed of the algorithm. Pre-computation of mutually invisible planes allows trimming the number of possible combination of rays that need to be computed. Results of physical measurement from two case studies have been compared to results of the simulation. Good correlation between the simulations and the case studies were achieved for the Speech Transmission Index and RASTI values. The accuracy of the simulation,in terms of decay based indices, is limited by the lack of sufficient tail to the calculated number of rays. Further research and implementation of hybrid techniques utilising both the image method and more traditional ray-tracing algorithms to improve the quality of the calculated decay data are required. Investigation of techniques used in photo-realism 'ray-tracing' may result in far more realistic data which is the basic input to the Speech Transmission Index calculations.

Deficits in conversation skills can be one barrier to developing and maintaining relationships for individuals with autism spectrum disorder (ASD). Individuals with ASD may deter conversation partners if they do not stay on topic or if they dwell on topics. Several interventions have been identified in targeting the reduction of problematic (off-topic or perseverative) speech, and withheld attention for its occurrence. In addition to leveraging attention as a reinforcer, one study provided signaled access to preferred topics contingent on talking about non-perseverative or therapist-selected topics. Despite showing clear improvements in on-topic speech and stimulus control of preferred topics, little is known about the additive effects of including contingent access to preferred topics. A free operant assessment was used to evaluate participant preference for including access to preferred topics. The results indicated that participants preferred the proposed intervention with access to a leisure item.

This case study evaluated the effects of differential reinforcement of other behavior (DRO) on scripting in a four year-old child with Autism Spectrum Disorder, Obsessive Compulsive Disorder and Celiac Disease. The overall goal was to show that DRO as the only independent variable could reduce scripting in a child with autism. A vibrator was set to vibrate every six minutes to indicate the end of each interval during intervention and the behavior was measured using a partial-interval time sampling method during the two hour in-home private Applied Behavior Analysis session over a two month period. An A-BC-C design demonstrated that DRO successfully decreased scripting behavior in the child with autism. A dependent paired samples t-test was used to compare the rates of scripting during the first three days of baseline and last three days of intervention. Results demonstrated a 29% decrease in scripting behavior. This result extends previous research that showed DRO, within a combined intervention, could be effective in decreasing scripting of adolescents with autism.

This research is motivated by the benefits that knowledge regarding early development in infants may provide to different fields of science. In particular, early sensorimotor exploration behaviors are studied in the framework of developmental robotics. The main objective is about understanding the role of motor constraint awareness and imitative behaviors during sensorimotor exploration. Particular emphasis is placed on prelinguistic vocal development because during this stage infants start to master the motor systems that will later allow them to pronounce their first words. Previous works have demonstrated that goal-directed intrinsically motivated sensorimotor exploration is an essential element for sensorimotor control learning. Moreover, evidence coming from biological sciences strongly suggests that knowledge acquisition is shaped by the environment in which an agent is embedded and the embodiment of the agent itself, including developmental processes that shape what can be learned and when. In this dissertation, we firstly provide a collection of theoretical evidence that supports the relevance of our study. Starting from concepts of cognitive and developmental sciences, we arrived al the conclusion that spoken language, i.e., early \/ocal development, must be studied asan embodied and situated phenomena. Considering a synthetic approach allow us to use robots and realistic simulators as artifacts to study natural cognitive phenomena. In this work, we adopta toy example to test our cognitive architectures and a speech synthesizer that mimics the mechanisms by which humans produce speech. Next, we introduce a mechanism to endow embodied agents with motor constraint awareness. lntrinsic motivation has been studied as an importan! element to explain the emergence of structured developmental stages during early vocal development. However, previous studies failed to acknowledge the constraints imposed by the embodiment and situatedness, al sensory, motor, cognitive and social levels. We assume that during the onset of sensorimotor exploratory behaviors, motor constraints are unknown to the developmental agent. Thus, the agent must discover and learn during exploration what !hose motor constraints are. The agent is endowed with a somesthetic system based on tactile information. This system generales a sensor signal indicating if a motor configuration was reached or not. This information is later used to create a somesthetic model to predict constraint violations. Finally, we propase to include social reinforcement during exploration. Sorne works studying early vocal development have shown that environmental speech shapes the sensory space explored during babbling. More generally, imitative behaviors have been demonstrated to be crucial for early development in children as they constraint the search space.during sensorimotor exploration. Therefore, based on early interactions of infants and caregivers we proposed an imitative mechanism to reinforce intrinsically motivated sensorimotor exploration with relevan! sensory units. Thus, we modified the constraints aware sensorimotor exploration architecture to include a social instructor, expert in sensor units relevant to communication, which interacts with the developmental agent. lnteraction occurs when the learner production is ·enough' similar to one relevan! to communication. In that case, the instructor perceives this similitude and reformulates with the relevan! sensor unit. When the learner perceives an utterance by the instructor, it attempts to imitate it. In general, our results suggest that somesthetic senses and social reinforcement contribute to achieving better results during intrinsically motivated exploration. Achieving lest redundant exploration, decreasing exploration and evaluation errors, as well as showing a clearer picture of developmental transitions.
La motivación principal de este trabajo es la magnitud que las contribuciones al conocimiento en relación al desarrollo infantil pueden aportar a diferentes campos de la ciencia. Particularmente, este trabajo se enfoca en el estudio de los comportamientos de autoexploración sensorimotora en un marco robótico e inspirado en el campo de la psicología del desarrollo. Nuestro objetivo principal es entender el papel que juegan las restricciones motoras y los reflejos imitativos durante la exploración espontánea observada en infantes. Así mismo, este trabajo hace especial énfasis en el desarrollo vocal-auditivo en infantes, que les provee con las herramientas que les permitirán producir sus primeras palabras. Trabajos anteriores han demostrado que los comportamientos de autoexploración sensorimotora en niños, la cual ocurre en gran medida por motivaciones intrínsecas, es un elemento importante para aprender a controlar su cuerpo con tal de alcanzar estados sensoriales específicos. Además, evidencia obtenida de estudios biológicos sugiere tajantemente que la adquisición de conocimiento es regulada por el ambiente en el cual un agente cognitivo se desenvuelve y por el cuerpo del agente per se. Incluso, los procesos de desarrollo que ocurren a nivel físico, cognitivo y social también regulan que es aprendido y cuando esto es aprendido. La primera parte de este trabajo provee al lector con la evidencia teórica y práctica que demuestran la relevancia de esta investigación. Recorriendo conceptos que van desde las ciencias cognitivas y del desarrollo, llegamos a la conclusión de que el lenguaje, y por tanto el habla, deben ser estudiados como fenómenos cognitivos que requieren un cuerpo físico y además un ambiente propicio para su existencia. En la actualidad los sistemas robóticos, reales y simulados, pueden ser considerados como elementos para el estudio de los fenómenos cognitivos naturales. En este trabajo consideramos un ejemplo simple para probar las arquitecturas cognitivas que proponemos, y posteriormente utilizamos dichas arquitecturas con un sintetizador de voz similar al mecanismo humano de producción del habla. Como primera contribución de este trabajo proponemos introducir un mecanismo para construir robots capaces de considerar sus propias restricciones motoras durante la etapa de autoexploración sensorimotora. Ciertos mecanismos de motivación intrínseca para exploración sensorimotora han sido estudiados como posibles conductores de las trayectorias de desarrollo observadas durante el desarrollo temprano del habla. Sin embargo, en previos estudios no se consideró o que este desarrollo está a delimitado por restricciones debido al ambiente, al cuerpo físico, y a las capacidades sensoriales, motoras y cognitivas. En nuestra arquitectura, asumimos que un agente artificial no cuenta con conocimiento de sus limitantes motoras, y por tanto debe descubrirlas durante la etapa de autoexploración. Para tal efecto, el agente es proveído de un sistema somatosensorial que le indica cuando una configuración motora viola las restricciones impuestas por el propio cuerpo. Finalmente, como segunda parte de nuestra contribución proponemos incluir un mecanismo para reforzar el aprendizaje durante la autoexploración. Estudios anteriores demostraron que el ambiente lingüístico en que se desarrolla un infante, o un agente artificial, condiciona sus producciones vocales durante la autoexploración o balbuceo. En este trabajo nos enfocamos en el estudio de episodios de imitación que ocurren durante el desarrollo temprano de un agente. Basados en estudios sobre la interacción entre madres e hijos durante la etapa pre lingüística, proponemos un mecanismo para reforzar el aprendizaje durante la autoexploración con unidades sensoriales relevantes. Entonces, a partir de la arquitectura con autoconocimiento de restricciones motores, construimos una arquitectura que incluye un instructor experto en control sensorimotor. Las interacciones entre el aprendiz y el experto ocurren cuando el aprendiz produce una unidad sensorial relevante para la comunicación durante la autoexploración. En este caso, el experto percibe esta similitud y responde reformulando la producción del aprendiz como la unidad relevante. Cuando el aprendiz percibe una acción del experto, inmediatamente intenta imitarlo. Los resultados presentados en este trabajo sugieren que, los sistemas somatosensoriales, y el reforzamiento social contribuyen a lograr mejores resultados durante la etapa de autoexploración sensorimotora motivada intrínsecamente. En este sentido, se logra una exploración menos redundante, los errores de exploración y evaluación disminuyen, y por último se obtiene una imagen más nítida de las transiciones entre etapas del desarrollo.
La motivació principal d'aquest treball és la magnitud que les contribucions al coneixement en relació al desenvolupament infantil poden aportar a diferents camps de la ciència. Particularment, aquest treball s'enfoca en l'estudi dels comportaments d’autoexploració sensorimotora en un marc robòtic i inspirat en el camp de la psicologia del desenvolupament. El nostre objectiu principal és entendre el paper que juguen les restriccions motores i els reflexos imitatius durant l’exploració espontània observada en infants. Així mateix, aquest treball fa especial èmfasi en el desenvolupament vocal-auditiu en infants, que els proveeix amb les eines que els permetran produir les seves primeres paraules. Treballs anteriors han demostrat que els comportaments d'autoexploració sensorimotora en nens, la qual ocorre en gran mesura per motivacions intrínseques, és un element important per aprendre a controlar el seu cos per tal d'assolir estats sensorials específics. A més, evidencies obtingudes d'estudis biològics suggereixen que l’adquisició de coneixement és regulada per l'ambient en el qual un agent cognitiu es desenvolupa i pel cos de l'agent per se. Fins i tot, els processos de desenvolupament que ocorren a nivell físic, cognitiu i social també regulen què és après i quan això ès après. La primera part d'aquest treball proveeix el lector amb les evidencies teòrica i pràctica que demostren la rellevància d'aquesta investigació. Recorrent conceptes que van des de les ciències cognitives i del desenvolupament, vam arribar a la conclusió que el llenguatge, i per tant la parla, han de ser estudiats com a fenòmens cognitius que requereixen un cos físic i a més un ambient propici per a la seva existència. En l'actualitat els sistemes robòtics, reals i simulats, poden ser considerats com a elements per a l'estudi dels fenòmens cognitius naturals. En aquest treball considerem un exemple simple per provar les arquitectures cognitives que proposem, i posteriorment utilitzem aquestes arquitectures amb un sintetitzador de veu similar al mecanisme humà de producció de la parla. Com a primera contribució d'aquest treball proposem introduir un mecanisme per construir robots capaços de considerar les seves pròpies restriccions motores durant l'etapa d'autoexploració sensorimotora. Certs mecanismes de motivació intrínseca per exploració sensorimotora han estat estudiats com a possibles conductors de les trajectòries de desenvolupament observades durant el desenvolupament primerenc de la parla. No obstant això, en previs estudis no es va considerar que aquest desenvolupament és delimitat per restriccions a causa de l'ambient, el cos físic, i les capacitats sensorials, motores i cognitives. A la nostra arquitectura, assumim que un agent artificial no compta amb coneixement dels seus limitants motors, i per tant ha de descobrir-los durant l'etapa d'autoexploració. Per a tal efecte, l'agent és proveït d'un sistema somatosensorial que li indica quan una configuració motora viola les restriccions imposades pel propi cos. Finalment, com a segona part de la nostra contribució proposem incloure un mecanisme per reforçar l'aprenentatge durant l'autoexploració. Estudis anteriors han demostrat que l'ambient lingüísticstic en què es desenvolupa un infant, o un agent artificial, condiciona les seves produccions vocals durant l'autoexploració o balboteig. En aquest treball ens enfoquem en l'estudi d'episodis d’imitació que ocorren durant el desenvolupament primerenc d'un agent. Basats en estudis sobre la interacció entre mares i fills durant l'etapa prelingüística, proposem un mecanisme per reforçar l'aprenentatge durant l'autoexploració amb unitats sensorials rellevants. Aleshores, a partir de l'arquitectura amb autoconeixement de restriccions motors, vam construir una arquitectura que inclou un instructor expert en control sensorimotor. Les interaccions entre l'aprenent i l'expert, ocorren quan una producció sensorial de l'aprenent durant l'autoexploració és similar a una unitat sensorial rellevant per a la comunicació. En aquest cas, l'expert percep aquesta similitud i respon reformulant la producció de l'aprenent com la unitat rellevant. Quan l'aprenent percep una acció de l'expert, immediatament intenta imitar-lo. Els resultats presentats en aquest treball suggereixen que els sistemes somatosensorials i el reforçament social contribueixen a aconseguir millors resultats durant l'etapa d'autoexploració sensorimotora motivada intrínsecament. En aquest sentit, s'aconsegueix una exploració menys redundant, els errors d’exploració i avaluació disminueixen, i finalment s’obté una imatge més nítida de les transicions entre etapes del desenvolupament

The overall aim of the current research was to investigate the corticostriatal system in developmental language impairment (DLI) at the behavioral and neuroanatomical levels. Two groups of young adults, one with DLI (N = 25) and the other without (N = 23), participated in the behavioral study. A sample of procedural learning and reinforcement learning (RL) tasks was selected. Each task represents a unique aspect of procedural memory, and learning processes during these tasks have been linked, at least partially, to the functionality of the corticostriatal system. Findings showed that individuals with DLI demonstrated relatively poor performance on different aspects of procedural learning and on RL. Correlation results provide further evidence for a close relationship between individual differences in implicit learning and individual differences in language. These results implicate an abnormal corticostriatal system in DLI. In the structural imaging study, two subgroups of participants from the first study, one with DLI (n = 10) and the other without (n = 10), were matched on age, gender, and handedness. Conventional magnetic resonance imaging (MRI) and diffusion tensor imaging (DTI) were used to investigate the subcortical components of the corticostriatal system in individuals with DLI. Results showed pathological enlargement in the bilateral putamen, the right globus pallidus, and the bilateral nucleus accumbens of individuals with DLI. In addition, the DLI group revealed decreased FA in the globus pallidus and in the thalamus, indicating abnormal white matter integrity in the two subcortical regions. These imaging results underpin the behavioral results, showing corticostriatal abnormalities in DLI at both macrostructural and microstructural levels. In addition to subcortical regions, the four cerebral lobes were also included for an exploratory analysis. Findings showed that individuals with DLI had global diffusion abnormalities in cerebral white matters in the absence of volumetric alterations, and these abnormalities were closely associated with impaired language performance. The results support a role of white matter integrity in language function. In conclusion, individuals with DLI have an abnormal corticostriatal system, which may lead to compromise of a wide variety of cognitive learning, including procedural learning, RL, and certain aspects of language learning.

La quantité de diffusion de signaux de parole dans les habitacles automobiles est de plus en plus importante : télécommunications, radio, système de navigation... Cependant, malgré les efforts et les avancées mécaniques, beaucoup de bruits persistent au sein de l'habitacle dégradant fortement l'intelligibilité de ces signaux de parole. L'objectif de cette thèse est alors de développer des outils de renforcement de la parole visant à traiter les signaux avant leur dégradation afin d'assurer une bonne intelligibilité dans le bruit des habitacles automobiles. Une approche de renforcement de la parole très performante consiste à utiliser un égaliseur fréquentiel afin d’optimiser un critère d’intelligibilité : le Speech Intelligibility Index (SII). Pour faciliter l'optimisation, les méthodes actuelles se basent sur des approximations du critère. De plus, en concentrant l'énergie spectrale du signal dans des zones où l'oreille est plus sensible, ces méthodes augmentent le volume perçu ce qui peut détériorer l'expérience utilisateur. Ainsi, en plus de proposer une méthode de résolution exacte du problème de maximisation du SII, nos travaux proposent d’introduire et étudier l'influence d'une nouvelle contrainte perceptive maintenant les signaux à leur niveau perçu. La popularisation des approches d’apprentissage automatique pousse à apprendre les traitements de renforcement de la parole à partir d’exemples naturellement produits dans le bruit (parole Lombard), ou en sur-articulant (parole claire). Les travaux actuels ne parviennent pas à obtenir des gains d’intelligibilité aussi significatifs qu’avec les modifications naturelles et nous pensons que la négligence de nombreux aspects temporels pourrait en être partiellement responsable. Nos travaux proposent donc d’approfondir ces approches en exploitant des modèles d’apprentissage et des pré-traitements adaptés aux séquences temporelles longues. Nous proposons aussi une nouvelle modélisation des modifications du débit de la parole directement intégrable dans l’apprentissage machine ce qui n'avait jamais été fait auparavant
Speech is nowadays present in a number of in-car applications ranging from hands-free communications, radio programs to speech synthesis messages from the various car devices.However, despite the steady car manufacturing progress, significant noise still remains in the car interior that leads to a loss of intelligibility of speech signals. The PhD work aims at developping speech reinforcement tools in order to process the signals before they are played in a noisy in-car environment.A highly effective speech reinforcement approach is to use a frequency equalizer to optimize an intelligibility criterion : the Speech Intelligibility Index (SII). To facilitate optimization, current methods are based on approximations of the criterion. In addition, by concentrating the spectral energy of the signal in areas where the ear is more sensitive, these methods increase the perceived volume which can deteriorate the user experience. Thus, in addition to proposing an exact method of solving the SII maximization problem, our work proposes to introduce and study the influence of a new perceptual constraint in order to maintain the signals at their perceived level.The popularization of machine learning approaches pushes to learn speech reinforcement processings from examples naturally produced in noise (Lombard speech), or by over-articulation (clear speech). Current work fails to achieve intelligibility gains as significant as with natural modification, and we believe that the many temporal aspects neglect may be partially responsible. Our work therefore proposes to deepen these approaches by exploiting learning models and pre-processings adapted to long duration sequences. We also propose a new modeling of the speech rate modifications that directly fits in the machine learning model which had never been done before

Vehicles driving in the urban settings experience substantial disturbances from neighbouring vehicles and traffic signals. Deviating from the optimal trajectory causes excessive fuel consumption and delay. This research proposes a novel adaptive driving-speed-control algorithm using a Reinforcement Learning (Q-learning) approach. The proposed algorithm can respond to the prevailing traffic conditions and traffic-controls conditions at signalised intersection environment and provides the control vehicle with target driving-speeds to achieve fuel savings. The micro-simulation results confirm the effectiveness of the algorithm by significantly and consistently reducing the fuel consumption of the control vehicle under varying driving environments.

This study on the enhancement of high-speed flywheel energy storage is to investigate composite materials that are suitable for high-speed, high-energy density for energy storage and/or energy recovery. The main motivation of the study is to explore the application of the flywheel in the aviation industry for recovering some of the energy that is currently being lost at the wheel brakes of an aircraft due to the high temperature developed in the brake stack as a result of landing, frequent brake applications during taxiing in or out of heavy traffic airports and rejected take-off. Lamina and laminate mechanical properties of materials suitable for flywheel high-speed energy storage were investigated. Design and optimum stress analysis were used to determine the shape factor, maximum stress and energy density for a flywheel with a constant stress disk and a constant thickness rim. Analytical studies along with the use of the CADEC-online software were used to evaluate the lamina and laminate properties. This study found that the use of hybrid composite material with higher strength (based on first ply failure strength) and lower density and lower elastic moduli for the disk than the rim material will yield high-speed and high-energy density. The materials designed based on the results from this study show outperformance compared to previous published results of standard flywheel material combinations. The safe rotational velocity and energy density were found to be 166,000 RPM and 2.73 MJ/kg respectively. Therefore, results from this study will contribute to aiding further development of the flywheel that has recently re-emerged as a promising application for energy storage due to significant improvements in composite materials and technology. Further study on flywheel energy recovery from aircraft brakes revealed that more than half of the energy dissipated at the wheel brake as heat could be recovered and converted to some useful form. In this way, the operating life of the brakes can be prolonged. The total additional weight to the aircraft was found to be less than 0.2% of the maximum take-off weight. This additional weight can be offset by reducing the design payload while ensuring that the structural efficiency of the aircraft is not altered. It was also found that applying this method of flywheel energy recovery to active commercial Boeing-777 aircraft will result in savings equivalent to the annual carbon emission of a 6 MW fossil fuel power plant. This will also contribute to the aviation industry climate change mitigation.

Empirical studies show that similar patterns emerge from a large number of different biological systems. For example, the group size distributions of several fish species and house sparrows all follow power law distributions with an exponential truncation. Networks built by ant colonies, slime mold and those are designed by engineers resemble each other in terms of structure and transportation efficiency. Based on the investigation of experimental data, we propose a variety of simple stochastic models to unravel the underlying mechanisms which lead to the collective phenomena in different systems. All the mechanisms employed in these models are rooted in the concept of selective reinforcement. In some systems the reinforcement can build optimal solutions for biological problem solving. This thesis consists of five papers. In the first three papers, I collaborate with biologists to look into group formation in house sparrows  and the movement decisions of damsel fish.  In the last two articles, I look at how shortest paths and networks are  constructed by slime molds and pheromone laying ants, as well as studying  speed-accuracy tradeoffs in slime molds' decision making. The general goal of the study is to better understand how macro level patterns and behaviors emerges from micro level interactions in both spatial and non-spatial biological systems. With the combination of mathematical modeling and experimentation, we are able to reproduce the macro level patterns in the studied biological systems and predict behaviors of the systems using minimum number of parameters.

Le moulage RTM à haute cadence est un procédé de fabrication composite prometteur qui satisfait les exigences de l’industrie automobile pour produire des pièces structurelles complexes avec un temps de cycle court. Cependant, les réductions de temps de cycle sont un véritable défi. Dans ce procédé, une résine est injectée avec mélange en dans la cavité d’un moule contenant un renfort fibreux. Ce flux de résine réactive génère des schémas d’écoulement complexes et des couplages thermo-chimio-rhéologiques forts. En raison de la grande sensibilité de la résine et des temps de cycle serrés, la prédiction de la stratégie d’injection optimale est très difficile et très coûteuse à mener expérimentalement. Le travail réalisé a donc poursuivi deux objectifs: 1. Identifier et quantifier expérimentalement les mécanismes influençant le procédé RTM réactif avec mélange en tête et 2. Développer une méthode de simulation numérique en vue d’introduire les mécanismes identifiés dans le logiciel industriel PAM COMPOSITE développé par ESI Group. L’identification et la quantification des mécanismes ont été réalisées grâce à des investigations expérimentales et numériques. Un nouveau montage expérimental a été développé pour l’étude du mécanisme de stockage de résines intra-mèche grâce à des observations aux échelles macro- et microscopiques. De plus, une méthode numérique a été développée pour simuler l’écoulement réactif de la résine dans des matériaux à simple et à double échelle de porosité. Cette méthode a permis d’étudier les mécanismes locaux difficiles à mesurer expérimentalement et de préparer le transfert vers le logiciel industriel d’ESI
High Speed Resin Transfer Molding (RTM) is a promising composite manufacturing process fitting automotive industry requirements to produce complex structural parts with a perspective of short cycle times. However, cycle time reductions are a real challenge. In this process, a resin mixed on-line with curing agents is injected in the cavity of a mold containing a fibrous reinforcement. This flow of reactive resin generates acomplex flow pattern and strong thermo-chemo rheological couplings. Due to the high sensitivity of the resin cure, and the tight cycle times, prediction of the optimal injection strategy is very difficult and very expensive to conduct experimentally. In this context, two goals where followed in this work: 1. Identify and quantify experimentally the mechanisms, related to the process or to the reinforcement, influencing the reactive RTMprocess with on-line mixing and 2. Develop a numerical simulation method in a view of introducing the identified mechanisms in the industrial software PAMCOMPOSITE developed by ESI Group. Identification and quantification of the mechanisms were realized thanks to experimental investigations and numerical simulations. A new experimental setup has been developed for the investigation of the mechanism of intra-tow resin storage through macro-scale and micro-scale observations. Additionally, a numerical method has been developed to simulate the reactive flow of a resin in single and dual scale porous materials. This method allowed both to investigate local mechanisms difficult to study experimentally and prepare the transfer to the industrial software of ESI

The present work deals with the behavior of ancient railway sub-structure in France. A statistical study was firstly undertaken on problems occurred in the whole ancient French railway network. The analysis evidenced the particular importance of sub-grade quality for the performance of the sub-structure and the track geometry. Afterwards, an ancient railway line in the West of France was investigated. The analysis showed that the degradation speed of this line was correlated with different parameters such as the nature of sub-grades and the thickness of different layers. An increase trend of degradation speed with the increase in interlayer thickness was identified. The interlayer has a positive impact since it reduces the train-induced stress applied to the sub-grade. The hydro-mechanical behavior of interlayer soil under different conditions (water content, fines content, stress, number of cycles) was investigated. A set of triaxial tests and infiltration tests were performed for this purpose. By analyzing the shear strength properties, the permanent axial strain and the resilient modulus of interlayer soil, we found that the water content and the fines content must be considered together. Adding more fines into the interlayer presents a positive impact under unsaturated conditions thanks to the suction effect, but a negative impact under saturated conditions. The infiltration column tests with drying/wetting cycles showed that the hydraulic conductivity of interlayer soil is governed by fines fraction but did not change significantly with fines content. In order to study the mechanism of interlayer creation and mud pumping, a physical model of 550 mm inner diameter was developed. Soil samples representing the ancient French railway substructure with a ballast layer overlying an artificial silt layer (mixture of crushed sand and kaolin were tested. The effects of monotonic and cyclic loadings, water content and dry unit mass of sub-soil were investigated. It was found that the pore water pressure developed in the sub-soil and the sub-soil stiffness are the key factors for the migration of fine particles or the creation of interlayer/mud pumping. Water is the necessary condition, but it is the soil compressibility that governs the phenomenon to occur

Le présent travail porte sur le comportement des plates-formes ferroviaires anciennes en France. Tout d'abord, une étude statistique a été menée sur les problèmes survenus dans l'ensemble du réseau ferroviaire français. L'analyse montre l'importance particulière de la qualité du sol support pour la performance de la sous-structure et pour la tenue géométrique des voies. Ensuite, une ligne ferroviaire ancienne située à l'Ouest de la France a été étudiée spécifiquement. Les analyses montrent que la vitesse de dégradation de cette ligne est en corrélation avec les différents paramètres tels que la nature de sol support, l'épaisseur des couches de la sous-structure. Une tendance d'augmentation de la vitesse de dégradation avec la diminution de l'épaisseur de la couche intermédiaire est identifiée. Cette couche as un impact positif puisqu'elle réduit les contraintes appliquées au sol support. Le comportement hydromécanique du sol de la couche intermédiaire dans des conditions différentes (teneur en eau, teneur en particules fines, charge, nombre de cycles) a été étudié. Des essais triaxiaux et des essais de colonne d'infiltration ont été réalisés à cette fin. En analysant les propriétés de résistance au cisaillement, la déformation axiale permanente et le module réversible, on a constaté que les effets de la teneur en eau et de la teneur en fines doivent être pris en compte ensemble. Une augmentation de teneur en fines dans la couche intermédiaire présente un impact positif à l'état non saturé grâce à l'effet de la succion, mais un impact négatif à l'état saturé. Les essais de colonne d'infiltration avec des cycles de séchage/humidification ont montré que la conductivité hydraulique du sol est gouvernée par la fraction de fines et qu'elle ne change pas significativement avec la teneur en fines. Afin d'étudier les mécanismes de la création de la couche intermédiaire et de remontée boueuse, un modèle physique de 550 mm de diamètre intérieur a été développé. Des échantillons de sol qui représentent la sous-structure ferroviaire ancienne avec une couche de ballast posée sur une couche de limon artificielle (mélange de sable concassé et du kaolin) ont été testés. Les effets des charges monotones et cycliques, de la teneur en eau et de la masse volumique sèche du sol support ont été étudiés. Il a été constaté que la pression interstitielle développée dans le sol support et la rigidité du sol support sont des facteurs clés pour la migration des particules fines ou la création de la couche intermédiaire/la remontée boueuse. L'eau est la condition nécessaire, mais c'est la compressibilité du sol support qui gouverne le phénomène à se produire
The present work deals with the behavior of ancient railway sub-structure in France. A statistical study was firstly undertaken on problems occurred in the whole ancient French railway network. The analysis evidenced the particular importance of sub-grade quality for the performance of the sub-structure and the track geometry. Afterwards, an ancient railway line in the West of France was investigated. The analysis showed that the degradation speed of this line was correlated with different parameters such as the nature of sub-grades and the thickness of different layers. An increase trend of degradation speed with the increase in interlayer thickness was identified. The interlayer has a positive impact since it reduces the train-induced stress applied to the sub-grade. The hydro-mechanical behavior of interlayer soil under different conditions (water content, fines content, stress, number of cycles) was investigated. A set of triaxial tests and infiltration tests were performed for this purpose. By analyzing the shear strength properties, the permanent axial strain and the resilient modulus of interlayer soil, we found that the water content and the fines content must be considered together. Adding more fines into the interlayer presents a positive impact under unsaturated conditions thanks to the suction effect, but a negative impact under saturated conditions. The infiltration column tests with drying/wetting cycles showed that the hydraulic conductivity of interlayer soil is governed by fines fraction but did not change significantly with fines content. In order to study the mechanism of interlayer creation and mud pumping, a physical model of 550 mm inner diameter was developed. Soil samples representing the ancient French railway substructure with a ballast layer overlying an artificial silt layer (mixture of crushed sand and kaolin were tested. The effects of monotonic and cyclic loadings, water content and dry unit mass of sub-soil were investigated. It was found that the pore water pressure developed in the sub-soil and the sub-soil stiffness are the key factors for the migration of fine particles or the creation of interlayer/mud pumping. Water is the necessary condition, but it is the soil compressibility that governs the phenomenon to occur

Šiame darbe buvo pasiūlyti dirbtinio intelekto pritaikymo būdai bei metodai, kompiuterinio žaidimo personažui, dviejų personažų koviniam žaidimui. Pasiūlytais būdais bei metodais buvo bandoma pritaikyti dirbtinį intelektą kompiuterinio žaidimo personažui, šio tipo žaidimui. Buvo tiriama ar pasliūlyti būdai ir metodai tinka šiai kompiuterinių žaidimų klasei. Taip pat buvo tiriamas agento mokymosi greitis, nuo ko jis priklauso bei ieškomi faktoriai, kurie lemia tokio tipo žaidimo personažams sparčiau mokintis ir naudingiau naudotis įgytomis žiniomis. Atrastus ir ištirtus faktorius buvo stengiamasi suderinti taip, kad mokomas personažas balansuotų tarp greitai besimokančio ir tinkamai besinaudojančio išmoktomis žiniomis.
In this work was offered an adjustments of techniques and methods of artificial intelligence for computer player, in two-players combat games. There was tried to adapt the artificial intelligence to the computer player, with those proposed techniques and methods in this type of games. Also was investigated whether those proposed techniques and methods suits for this class of computer games. In this work also was analyzed agent learning speed, on what depends it and explored the factors that determines the rapid learning speed and efficient use of agent knowledge. Discovered and investigated factors were fitted to the rapid learning speed and efficient use of agent knowledge.

Speeding remains a significant contributing factor to road trauma internationally, despite increasingly sophisticated speed management strategies being adopted around the world. Increases in travel speed are associated with increases in crash risk and crash severity. As speed choice is a voluntary behaviour, driver perceptions are important to our understanding of speeding and, importantly, to designing effective behavioural countermeasures. The four studies conducted in this program of research represent a comprehensive approach to examining psychosocial influences on driving speeds in two countries that are at very different levels of road safety development: Australia and China. Akers’ social learning theory (SLT) was selected as the theoretical framework underpinning this research and guided the development of key research hypotheses. This theory was chosen because of its ability to encompass psychological, sociological, and criminological perspectives in understanding behaviour, each of which has relevance to speeding. A mixed-method design was used to explore the personal, social, and legal influences on speeding among car drivers in Queensland (Australia) and Beijing (China). Study 1 was a qualitative exploration, via focus group interviews, of speeding among 67 car drivers recruited from south east Queensland. Participants were assigned to groups based on their age and gender, and additionally, according to whether they self-identified as speeding excessively or rarely. This study aimed to elicit information about how drivers conceptualise speeding as well as the social and legal influences on driving speeds. The findings revealed a wide variety of reasons and circumstances that appear to be used as personal justifications for exceeding speed limits. Driver perceptions of speeding as personally and socially acceptable, as well as safe and necessary were common. Perceptions of an absence of danger associated with faster driving speeds were evident, particularly with respect to driving alone. An important distinction between the speed-based groups related to the attention given to the driving task. Rare speeders expressed strong beliefs about the need to be mindful of safety (self and others) while excessive speeders referred to the driving task as automatic, an absent-minded endeavour, and to speeding as a necessity in order to remain alert and reduce boredom. For many drivers in this study, compliance with speed limits was expressed as discretionary rather than mandatory. Social factors, such as peer and parental influence were widely discussed in Study 1 and perceptions of widespread community acceptance of speeding were noted. In some instances, the perception that ‘everybody speeds’ appeared to act as one rationale for the need to raise speed limits. Self-presentation, or wanting to project a positive image of self was noted, particularly with respect to concealing speeding infringements from others to protect one’s image as a trustworthy and safe driver. The influence of legal factors was also evident. Legal sanctions do not appear to influence all drivers to the same extent. For instance, fear of apprehension appeared to play a role in reducing speeding for many, although previous experiences of detection and legal sanctions seemed to have had limited influence on reducing speeding among some drivers. Disregard for sanctions (e.g., driving while suspended), fraudulent demerit point use, and other strategies to avoid detection and punishment were widely and openly discussed. In Study 2, 833 drivers were recruited from roadside service stations in metropolitan and regional locations in Queensland. A quantitative research strategy assessed the relative contribution of personal, social, and legal factors to recent and future self-reported speeding (i.e., frequency of speeding and intentions to speed in the future). Multivariate analyses examining a range of factors drawn from SLT revealed that factors including self-identity (i.e., identifying as someone who speeds), favourable definitions (attitudes) towards speeding, personal experiences of avoiding detection and punishment for speeding, and perceptions of family and friends as accepting of speeding were all significantly associated with greater self-reported speeding. Study 3 was an exploratory, qualitative investigation of psychosocial factors associated with speeding among 35 Chinese drivers who were recruited from the membership of a motoring organisation and a university in Beijing. Six focus groups were conducted to explore similar issues to those examined in Study 1. The findings of Study 3 revealed many similarities with respect to the themes that arose in Australia. For example, there were similarities regarding personal justifications for speeding, such as the perception that posted limits are unreasonably low, the belief that individual drivers are able to determine safe travel speeds according to personal comfort with driving fast, and the belief that drivers possess adequate skills to control a vehicle at high speed. Strategies to avoid detection and punishment were also noted, though they appeared more widespread in China and also appeared, in some cases, to involve the use of a third party, a topic that was not reported by Australian drivers. Additionally, higher perceived enforcement tolerance thresholds were discussed by Chinese participants. Overall, the findings indicated perceptions of a high degree of community acceptance of speeding and a perceived lack of risk associated with speeds that were well above posted speed limits. Study 4 extended the exploratory research phase in China with a quantitative investigation involving 299 car drivers recruited from car washes in Beijing. Results revealed a relatively inexperienced sample with less than 5 years driving experience, on average. One third of participants perceived that the certainty of penalties when apprehended was low and a similar proportion of Chinese participants reported having previously avoided legal penalties when apprehended for speeding. Approximately half of the sample reported that legal penalties for speeding were ‘minimally to not at all’ severe. Multivariate analyses revealed that past experiences of avoiding detection and punishment for speeding, as well as favourable attitudes towards speeding, and perceptions of strong community acceptance of speeding were most strongly associated with greater self-reported speeding in the Chinese sample. Overall, the results of this research make several important theoretical contributions to the road safety literature. Akers’ social learning theory was found to be robust across cultural contexts with respect to speeding; similar amounts of variance were explained in self-reported speeding in the quantitative studies conducted in Australia and China. Historically, SLT was devised as a theory of deviance and posits that deviance and conformity are learned in the same way, with the balance of influence stemming from the ways in which behaviour is rewarded and punished (Akers, 1998). This perspective suggests that those who speed and those who do not are influenced by the same mechanisms. The inclusion of drivers from both ends of the ‘speeding spectrum’ in Study 1 provided an opportunity to examine the wider utility of SLT across the full range of the behaviour. One may question the use of a theory of deviance to investigate speeding, a behaviour that could, arguably, be described as socially acceptable and prevalent. However, SLT seemed particularly relevant to investigating speeding because of its inclusion of association, imitation, and reinforcement variables which reflect the breadth of factors already found to be potentially influential on driving speeds. In addition, driving is a learned behaviour requiring observation, guidance, and practice. Thus, the reinforcement and imitation concepts are particularly relevant to this behaviour. Finally, current speed management practices are largely enforcement-based and rely on the principles of behavioural reinforcement captured within the reinforcement component of SLT. Thus, the application of SLT to a behaviour such as speeding offers promise in advancing our understanding of the factors that influence speeding, as well as extending our knowledge of the application of SLT. Moreover, SLT could act as a valuable theoretical framework with which to examine other illegal driving behaviours that may not necessarily be seen as deviant by the community (e.g., mobile phone use while driving). This research also made unique contributions to advancing our understanding of the key components and the overall structure of Akers’ social learning theory. The broader SLT literature is lacking in terms of a thorough structural understanding of the component parts of the theory. For instance, debate exists regarding the relevance of, and necessity for including broader social influences in the model as captured by differential association. In the current research, two alternative SLT models were specified and tested in order to better understand the nature and extent of the influence of differential association on behaviour. Importantly, the results indicated that differential association was able to make a unique contribution to explaining self-reported speeding, thereby negating the call to exclude it from the model. The results also demonstrated that imitation was a discrete theoretical concept that should also be retained in the model. The results suggest a need to further explore and specify mechanisms of social influence in the SLT model. In addition, a novel approach was used to operationalise SLT variables by including concepts drawn from contemporary social psychological and deterrence-based research to enhance and extend the way that SLT variables have traditionally been examined. Differential reinforcement was conceptualised according to behavioural reinforcement principles (i.e., positive and negative reinforcement and punishment) and incorporated concepts of affective beliefs, anticipated regret, and deterrence-related concepts. Although implicit in descriptions of SLT, little research has, to date, made use of the broad range of reinforcement principles to understand the factors that encourage or inhibit behaviour. This approach has particular significance to road user behaviours in general because of the deterrence-based nature of many road safety countermeasures. The concept of self-identity was also included in the model and was found to be consistent with the definitions component of SLT. A final theoretical contribution was the specification and testing of a full measurement model prior to model testing using structural equation modelling. This process is recommended in order to reduce measurement error by providing an examination of the psychometric properties of the data prior to full model testing. Despite calls for such work for a number of decades, the current work appears to be the only example of a full measurement model of SLT. There were also a number of important practical implications that emerged from this program of research. Firstly, perceptions regarding speed enforcement tolerance thresholds were highlighted as a salient influence on driving speeds in both countries. The issue of enforcement tolerance levels generated considerable discussion among drivers in both countries, with Australian drivers reporting lower perceived tolerance levels than Chinese drivers. It was clear that many drivers used the concept of an enforcement tolerance in determining their driving speed, primarily with the desire to drive faster than the posted speed limit, yet remaining within a speed range that would preclude apprehension by police. The quantitative results from Studies 2 and 4 added support to these qualitative findings. Together, the findings supported previous research and suggested that a travel speed may not be seen as illegal until that speed reaches a level over the prescribed enforcement tolerance threshold. In other words, the enforcement tolerance appears to act as a ‘de facto’ speed limit, replacing the posted limit in the minds of some drivers. The findings from the two studies conducted in China (Studies 2 and 4) further highlighted the link between perceived enforcement tolerances and a ‘de facto’ speed limit. Drivers openly discussed driving at speeds that were well above posted speed limits and some participants noted their preference for driving at speeds close to ‘50% above’ the posted limit. This preference appeared to be shaped by the perception that the same penalty would be imposed if apprehended, irrespective of what speed they travelling (at least up to 50% above the limit). Further research is required to determine whether the perceptions of Chinese drivers are mainly influenced by the Law of the People’s Republic of China or by operational practices. Together, the findings from both studies in China indicate that there may be scope to refine enforcement tolerance levels, as has happened in other jurisdictions internationally over time, in order to reduce speeding. Any attempts to do so would likely be assisted by the provision of information about the legitimacy and purpose of speed limits as well as risk factors associated with speeding because these issues were raised by Chinese participants in the qualitative research phase. Another important practical implication of this research for speed management in China is the way in which penalties are determined. Chinese drivers described perceptions of unfairness and a lack of transparency in the enforcement system because they were unsure of the penalty that they would receive if apprehended. Steps to enhance the perceived certainty and consistency of the system to promote a more equitable approach to detection and punishment would appear to be welcomed by the general driving public and would be more consistent with the intended theoretical (deterrence) basis that underpins the current speed enforcement approach. The use of mandatory, fixed penalties may assist in this regard. In many countries, speeding attracts penalties that are dependent on the severity of the offence. In China, there may be safety benefits gained from the introduction of a similar graduated scale of speeding penalties and fixed penalties might also help to address the issue of uncertainty about penalties and related perceptions of unfairness. Such advancements would be in keeping with the principles of best practice for speed management as identified by the World Health Organisation. Another practical implication relating to legal penalties, and applicable to both cultural contexts, relates to the issues of detection and punishment avoidance. These two concepts appeared to strongly influence speeding in the current samples. In Australia, detection avoidance strategies reported by participants generally involved activities that are not illegal (e.g., site learning and remaining watchful for police vehicles). The results from China were similar, although a greater range of strategies were reported. The most common strategy reported in both countries for avoiding detection when speeding was site learning, or familiarisation with speed camera locations. However, a range of illegal practices were also described by Chinese drivers (e.g., tampering with or removing vehicle registration plates so as to render the vehicle unidentifiable on camera and use of in-vehicle radar detectors). With regard to avoiding punishment when apprehended, a range of strategies were reported by drivers from both countries, although a greater range of strategies were reported by Chinese drivers. As the results of the current research indicated that detection avoidance was strongly associated with greater self-reported speeding in both samples, efforts to reduce avoidance opportunities are strongly recommended. The practice of randomly scheduling speed camera locations, as is current practice in Queensland, offers one way to minimise site learning. The findings of this research indicated that this practice should continue. However, they also indicated that additional strategies are needed to reduce opportunities to evade detection. The use of point-to-point speed detection (also known as sectio

碩士
國立交通大學
電機工程學系
106
Speech enhancement to cancel the noises in the environment has been an important topic in speech signal processing. Koizumi's research group proposed deep-neural-network-based reinforcement learning (DNN-RL) to enhance the speech in accordance with the speech quality. Their method is said to utilize limited training data efficiently and is better than the DNN-mapping method. Hence, we propose three optimization techniques to further boost the performance. First, we propose two procedures to define the actions and make a comparison between other number of templates. Second, we extend the one-level DNN-RL which yields the best speech quality to a two-level DNN-RL to separately enhance the high-frequency and low-frequency regions. Last, the Monte Carlo method is combined with the proposed DNN-RLs to ensure the stability of algorithm. To evaluate these three optimization techniques, experiments are designed to measure the difference of speech intelligibility and speech quality under different noise condition. Judging from the experiment results, the proposed procedures of defining actions has higher speech quality scores than the original procedure while the number of actions barely influences the speech quality. Also, the two-level DNN-RL produces better speech quality than the one-level DNN-RL. Last, DNN-RL combined with the Monte Carlo method benefits speech quality. Future work to combine the optimized DNN-RL method into the regression-based speech enhancement method is expected to produce a better result.

Two studies were conducted on the effects of different reinforcement contingencies on the emission of verbal operants by preschoolers. Six participants, 3 females and 3 males, 3- to 4-years old, were selected to participate in Experiment I. Six participants, 5 females and 1 male, 2- to 4-years-old, were selected for Experiment II. In Experiment I, the effects of contingent tokens versus contingent adult attention were tested on the number of tacts emitted in three different experimental settings, using an alternating treatment design. In Experiment II, the effects of contingent tokens versus contingent adult attention were tested on the number of tacts per minute and the percentage of peer-to-peer conversational units. The results from both experiments showed that these participants emitted tacts more frequently with contingent social attention than with contingent tokens. In addition, in Experiment II, peer-to-peer conversational units were low when adult attention was available and increased when adult attention was withheld. Implications of these results include, tacts are maintained specifically by social reinforcers, not simply generalized conditioned reinforcers (i.e., tokens). Thus, special attention must be paid when selecting reinforcers for teaching tacts so that the desired function is taught. Moreover, deprivation of adult-attention appears to function as a motivating operation for enhancing the value of attention from peers.

博士
國立中正大學
電機工程研究所
93
In recent years, the advanced communications technologies supply more and more network bandwidth. However, Internet users increase rapidly result in the network bandwidth is exhausted. When too many users are present in the Internet cause performance degrades. This situation is called congestion. Low throughput, high packet loss rate and high transmit delay result from congestion. The problem of insufficient bandwidth can be improved by way of enhancing the congestion control mechanism so that it can work more efficiently. Traditional methods for congestion control always monitor the queue length, on which the source rate depends. This paper is meant to explore the proposed congestion control with cooperative reinforcement learning (RL) scheme that differ from control method of AIMD, to adapt to the variant network environment. The RL scheme, mainly implemented by artificial neural networks (ANNs), consists of two subsystems: the expectation-return predictor is a long-term policy evaluator and the other is a short-term action selector, which is composed of an action-value evaluator and a stochastic action selector. In this research, we divide the study of proposed congestion control into three applications. The first application applies a RL scheme to congestion control in multimedia networks. The proposed RL scheme receives reinforcement signals generated by an immediate reward evaluator and takes the best action in the sense of state value evaluation to control source rates in consideration of system performance. The second application is the study for an adaptive multi-agent RL scheme on solving congestion control problems on dynamic high-speed networks. After receiving cooperative reinforcement signals generated by a cooperative fuzzy reward evaluator, the proposed cooperative multi-agent congestion control can learn to take correct actions adaptively under time-varying environments. The last one is the study for a cooperative congestion control for multimedia networks based on learning approach. In order to make the best of link utilization, a cooperative fuzzy reward evaluator provides cooperative reinforcement signals based on game theory are included. Simulation results have shown that these proposed approaches can increase system utilization and decrease packet losses simultaneously.

碩士
國立臺灣大學
機械工程學研究所
106
Industrial automation has become an important issue in recent years, more and more manipulators are applied to manufactories. Instead of manpower, industries prefer automated machine which has more advantages. For example, industrial automation can provide a high productivity, allowing the company to run a manufacturing plant for 24 hours every day. Besides, it has high safety when the plant is in an extreme environment. While artificial intelligence becomes a growing trend, some technologies are applied to robotics to improve the control policies or performance in unknown environment. This research focuses on trajectory optimization and dynamic model compensation. Without directly controlling each joint motor, trajectory command is sent according to the given form, which contains the via points of the trajectory. This research deals with energy and time optimization by using reinforcement learning. Designing the actor-critic agent and reward contains energy/time consumption and obstacle/torque constraints. The actions will change the position and speed of via points, the aim is to get the action of the highest reward. In the experimental stage, the dynamic model exists some differences between simulation and reality. To ensure the availability of simulation, this research uses neural networks to compensate the model.

Decision making – committing to a single action from a plethora of viable alternatives – is a necessity for all motile creatures, each moving a single body to many possible destinations. Some decisions are better than others. For example, to a rat deciding between one path that will bring it to a piece of cheese and another that will bring it to the jaws of a cat, there is a clear reason for the rat to prefer one choice over the other. Two criteria for adjusting decision making for optimal outcome are to make decisions as accurately as possible – choose the course of action most likely to result in the preferred outcome – but also to decide as fast as possible. Because these criteria often conflict, decision making has an inherent “speed-accuracy tradeoff”. Presented here is a computational neural model of decision making, which incorporates neurobiological design principles that optimize this tradeoff via reward-guided transfers of control between two sensory processing systems with different speed/accuracy characteristics. This model incorporates anatomical and physiological evidence that dopamine, the key neurotransmitter in reinforcement learning, has varying effects in different sub-regions of the basal ganglia, a subcortical structure that interfaces with the neocortex to control behavior. Based on the observed differences between these sub-regions, the model proposes that gradual adaptations of synaptic links by reinforcement learning signals lead to rapid changes in the speed and accuracy of decision making, by assigning control of behavior to alternative cortical representations. Chapter one draws conceptual links from experimental data to the design of the proposed model. Chapter two applies the model to speed-accuracy tradeoffs and habit formation by simulating forced-choice paradigms. Several robust behavioral phenomena are replicated. By isolating reinforcement learning factors that control the speed and depth of habit formation, the model can help explain why all substances that strongly and synergistically affect such factors share a high potential for habit formation, or habit abatement. To illustrate such potential applications of the current model, chapter three investigates effects of varying model parameters in accord with the known neurochemical effects of some major habit-forming substances, such as cocaine and ethanol.

This dissertation examines the impact of active shooters upon patrons attending large outdoor events. There has been a spike in shooters targeting densely populated spaces in recent years, to include open-air venues. The 2019 Gilroy Garlic Festival was selected for modeling replication using AnyLogic software to test various experiments designed to reduce casualties in the event of an active shooter situation. Through achievement of validation to produce identical outcomes of the real-world Gilroy Garlic Festival shooting, the researcher established a reliable foundational model for experimental purposes. This active shooter research project identifies the need for rapid response efforts to neutralize the shooter(s) as quickly as possible to minimize casualties. Key findings include the importance of armed officers patrolling event grounds to reduce response time, the need for adequate exits during emergency evacuations, incorporation of modern technology to identify the shooter’s location, and applicability of a 1:548 police to patron ratio.