Дисертації з теми "Photonic Neural Network"

Photonic Integrated Circuits (PICs) based on silicon photonics have received great interest due to the low loss caused by the high-refractive-index contrast and the complementary metal-oxide semiconductor compatibility. The need for high-density, high-yield, low-cost, low-power consumption, and large-scale on-chip photonic integration requires the technologies to further minimize the size while exhibiting high performance. Moreover, the fast development and expansion of silicon photonics devices for different applications and functionalities require effective design approaches to optimize the device performance while reducing the design complexity. In this thesis, several fundamental components for PICs are presented as the building blocks for advanced photonic circuits. To test the effectiveness of the design, Mach–Zehnder interferometers are simulated and fabricated on a Silicon-on-Insulator (SOI) platform, which shows a good agreement between the experimental and simulation results. Moreover, compact vertical grating couplers with broad optical bandwidth are studied. Experimental results show the compact size and the light coupling capabilities. Multimode Interference (MMI) splitter acts as one critical component in PICs. However, the minimum requirement of mid-to-mid channel spacing to avoid crosstalk limits the MMI size to be further reduced and thus limits the component density in the photonic integration. To solve this problem, a compact SOI MMI power splitter based on optical strip barriers is presented to achieve high crosstalk reduction. Three different MMI power splitters are designed and simulated with an ultra-small device footprint, high uniformity, while maintaining a low insertion loss of 0.4dB. Inverse design methods with different optimization algorithms are utilized to design compact and high-performance PIC components. Firstly, a sequential least-squares programming algorithm is introduced to inverse design a waveguide crossing. This gradient-based algorithm is suitable for simple structures with fewer parameters, or a good starting point can be obtained from experience or physical theories. Secondly, a novel dynamic iterative batch optimization method is presented in the thesis to design a high-performance segmented mode expander. In the simulation, the optimized structure achieves a coupling efficiency of 81% for TE polarization at the wavelength of 1550nm. It also shows a simulated transmission loss of lower than -1.137dB within 60nm bandwidth. This approach paves the way for the rapid design of PIC components with a compact footprint. Additionally, a Direct Binary Search (DBS) algorithm is introduced for designing pixel-like structures with binary-value-represented topology patterns, where a 3dB beam splitter is used in the design. DBS algorithm can be utilized to generate a high-quality dataset used for deep learning acceleration methods. To solve the time-efficiency and non-scalable issues of conventional inverse design methods, a neural network-based inverse design approach is presented and applied on the design of a wavelength demultiplexer structure. The method solves the data domain shift problem that existed in the conventional tandem network architecture and improves the prediction accuracy with a 99% validation accuracy. It also shows high stability and robustness to the quantity and quality of training data. The demonstrated wavelength demultiplexer has an ultra-compact footprint of 2.6×2.6μm2, a high transmission efficiency with a transmission loss of -2dB, and a low crosstalk around -7dB simultaneously.

Aujourd'hui, la plupart des ordinateurs sont encore basés sur des concepts développés il y a plus de 60 ans par Alan Turing et John von Neumann. Cependant, ces ordinateurs numériques ont déjà commencé à atteindre certaines limites physiques via la technologie de la microélectronique au silicium (dissipation, vitesse, limites d'intégration, consommation d'énergie). Des approches alternatives, plus puissantes, plus efficaces et moins consommatrices d'énergie, constituent depuis plusieurs années un enjeu scientifique majeur. Beaucoup de ces approches s'inspirent naturellement du cerveau humain, dont les principes opérationnels sont encore loin d'être compris. Au début des années 2000, la communauté scientifique s'est aperçue qu'une modification du réseau neuronal récurrent (RNN), plus simple et maintenant appelée Reservoir Computing (RC), est parfois plus efficace pour certaines fonctionnalités, et est un nouveau paradigme de calcul qui s'inspire du cerveau. Sa structure est assez semblable aux concepts classiques de RNN, présentant généralement trois parties: une couche d'entrée pour injecter l'information dans un système dynamique non-linéaire (Write-In), une seconde couche où l'information d'entrée est projetée dans un espace de grande dimension (appelé réservoir dynamique) et une couche de sortie à partir de laquelle les informations traitées sont extraites par une fonction dite de lecture-sortie. Dans l'approche RC, la procédure d'apprentissage est effectuée uniquement dans la couche de sortie, tandis que la couche d'entrée et la couche réservoir sont fixées de manière aléatoire, ce qui constitue l'originalité principale du RC par rapport aux méthodes RNN. Cette fonctionnalité permet d'obtenir plus d'efficacité, de rapidité, de convergence d'apprentissage, et permet une mise en œuvre expérimentale. Cette thèse de doctorat a pour objectifs d'implémenter pour la première fois le RC photoniques en utilisant des dispositifs de télécommunication. Notre mise en œuvre expérimentale est basée sur un système dynamique non linéaire à retard, qui repose sur un oscillateur électro-optique (EO) avec une modulation de phase différentielle. Cet oscillateur EO a été largement étudié dans le contexte de la cryptographie optique du chaos. La dynamique présentée par de tels systèmes est en effet exploitée pour développer des comportements complexes dans un espace de phase à dimension infinie, et des analogies avec la dynamique spatio-temporelle (tels que les réseaux neuronaux) sont également trouvés dans la littérature. De telles particularités des systèmes à retard ont conforté l'idée de remplacer le RNN traditionnel (généralement difficile à concevoir technologiquement) par une architecture à retard d'EO non linéaire. Afin d'évaluer la puissance de calcul de notre approche RC, nous avons mis en œuvre deux tests de reconnaissance de chiffres parlés (tests de classification) à partir d'une base de données standard en intelligence artificielle (TI-46 et AURORA-2), et nous avons obtenu des performances très proches de l'état de l'art tout en établissant un nouvel état de l'art en ce qui concerne la vitesse de classification. Notre approche RC photonique nous a en effet permis de traiter environ 1 million de mots par seconde, améliorant la vitesse de traitement de l'information d'un facteur supérieur à ~3
Nowadays most of computers are still based on concepts developed more than 60 years ago by Alan Turing and John von Neumann. However, these digital computers have already begun to reach certain physical limits of their implementation via silicon microelectronics technology (dissipation, speed, integration limits, energy consumption). Alternative approaches, more powerful, more efficient and with less consume of energy, have constituted a major scientific issue for several years. Many of these approaches naturally attempt to get inspiration for the human brain, whose operating principles are still far from being understood. In this line of research, a surprising variation of recurrent neural network (RNN), simpler, and also even sometimes more efficient for features or processing cases, has appeared in the early 2000s, now known as Reservoir Computing (RC), which is currently emerging new brain-inspired computational paradigm. Its structure is quite similar to the classical RNN computing concepts, exhibiting generally three parts: an input layer to inject the information into a nonlinear dynamical system (Write-In), a second layer where the input information is projected in a space of high dimension called dynamical reservoir and an output layer from which the processed information is extracted through a so-called Read-Out function. In RC approach the learning procedure is performed in the output layer only, while the input and reservoir layer are randomly fixed, being the main originality of RC compared to the RNN methods. This feature allows to get more efficiency, rapidity and a learning convergence, as well as to provide an experimental implementation solution. This PhD thesis is dedicated to one of the first photonic RC implementation using telecommunication devices. Our experimental implementation is based on a nonlinear delayed dynamical system, which relies on an electro-optic (EO) oscillator with a differential phase modulation. This EO oscillator was extensively studied in the context of the optical chaos cryptography. Dynamics exhibited by such systems are indeed known to develop complex behaviors in an infinite dimensional phase space, and analogies with space-time dynamics (as neural network ones are a kind of) are also found in the literature. Such peculiarities of delay systems supported the idea of replacing the traditional RNN (usually difficult to design technologically) by a nonlinear EO delay architecture. In order to evaluate the computational power of our RC approach, we implement two spoken digit recognition tests (classification tests) taken from a standard databases in artificial intelligence TI-46 and AURORA-2, obtaining results very close to state-of-the-art performances and establishing state-of-the-art in classification speed. Our photonic RC approach allowed us to process around of 1 million of words per second, improving the information processing speed by a factor ~3

For many challenging problems where the mathematical description is not explicitly defined, artificial intelligence methods appear to be much more robust compared to traditional algorithms. Such methods share the common property of learning from examples in order to “explore” the problem to solve. Then, they generalize these examples to new and unseen input signals. The reservoir computing paradigm is a bio-inspired approach drawn from the theory of artificial Recurrent Neural Networks (RNNs) to process time-dependent data. This machine learning method was proposed independently by several research groups in the early 2000s. It has enabled a breakthrough in analog information processing, with several experiments demonstrating state-of-the-art performance for a wide range of hard nonlinear tasks. These tasks include for instance dynamic pattern classification, grammar modeling, speechrecognition, nonlinear channel equalization, detection of epileptic seizures, robot control, timeseries prediction, brain-machine interfacing, power system monitoring, financial forecasting, or handwriting recognition. A Reservoir Computer (RC) is composed of three different layers. There is first the neural network itself, called “reservoir”, which consists of a large number of internal variables (i.e. reservoir states) all interconnected together to exchange information. The internal dynamics of such a system, driven by a function of the inputs and the former reservoir states, is thus extremely rich. Through an input layer, a time-dependent input signal is applied to all the internal variables to disturb the neural network dynamics. Then, in the output layer, all these reservoir states are processed, often by taking a linear combination thereof at each time-step, to compute the output signal. Let us note that the presence of a non-linearity somewhere in the system is essential to reach high performance computing on nonlinear tasks. The principal novelty of the reservoir computing paradigm was to propose an RNN where most of the connection weights are generated randomly, except for the weights adjusted to compute the output signal from a linear combination of the reservoir states. In addition, some global parameters can be tuned to get the best performance, depending on the reservoir architecture and on the task. This simple and easy process considerably decreases the training complexity compared to traditional RNNs, for which all the weights needed to be optimized. RC algorithms can be programmed using modern traditional processors. But these electronic processors are better suited to digital processing for which a lot of transistors continuously need to be switched on and off, leading to higher power consumption. As we can intuitively understand, processors with hardware directly dedicated to RC operations – in otherwords analog bio-inspired processors – could be much more efficient regarding both speed and power consumption. Based on the same idea of high speed and low power consumption, the last few decades have seen an increasing use of coherent optics in the transport of information thanks to its high bandwidth and high power efficiency advantages. In order to address the future challenge of high performance, high speed, and power efficient nontrivial computing, it is thus natural to turn towards optical implementations of RCs using coherent light. Over the last few years, several physical implementations of RCs using optics and (opto)electronics have been successfully demonstrated. In the present PhD thesis, the reservoirs are based on a large coherently driven linear passive fiber cavity. The internal states are encoded by time-multiplexing in the cavity. Each reservoir state is therefore processed sequentially. This reservoir architecture exhibits many qualities that were either absent or not simultaneously present in previous works: we can perform analog optical signal processing; the easy tunability of each key parameter achieves the best operating point for each task; the system is able to reach a strikingly weak noise floor thanks to the absence of active elements in the reservoir itself; a richer dynamics is provided by operating in coherent light, as the reservoir states are encoded in both the amplitude and the phase of the electromagnetic field; high power efficiency is obtained as a result of the passive nature and simplicity of the setup. However, it is important to note that at this stage we have only obtained low optical power consumption for the reservoir itself. We have not tried to minimize the overall power consumption, including all control electronics. The first experiment reported in chapter 4 uses a quadratic non-linearity on each reservoir state in the output layer. This non-linearity is provided by a readout photodiode since it produces a current proportional to the intensity of the light. On a number of benchmark tasks widely used in the reservoir computing community, the error rates demonstrated with this RC architecture – both in simulation and experimentally – are, to our knowledge, the lowest obtained so far. Furthermore, the analytic model describing our experiment is also of interest, asit constitutes a very simple high performance RC algorithm. The setup reported in chapter 4 requires offline digital post-processing to compute its output signal by summing the weighted reservoir states at each time-step. In chapter 5, we numerically study a realistic model of an optoelectronic “analog readout layer” adapted on the setup presented in chapter 4. This readout layer is based on an RLC low-pass filter acting as an integrator over the weighted reservoir states to autonomously generate the RC output signal. On three benchmark tasks, we obtained very good simulation results that need to be confirmed experimentally in the future. These promising simulation results pave the way for standalone high performance physical reservoir computers.The RC architecture presented in chapter 5 is an autonomous optoelectronic implementation able to electrically generate its output signal. In order to contribute to the challenge of all-optical computing, chapter 6 highlights the possibility of processing information autonomously and optically using an RC based on two coherently driven passive linear cavities. The first one constitutes the reservoir itself and pumps the second one, which acts as an optical integrator onthe weighted reservoir states to optically generate the RC output signal after sampling. A sine non-linearity is implemented on the input signal, whereas both the reservoir and the readout layer are kept linear. Let us note that, because the non-linearity in this system is provided by a Mach-Zehnder modulator on the input signal, the input signal of this RC configuration needs to be an electrical signal. On the contrary, the RC implementation presented in chapter 5 processes optical input signals, but its output is electrical. We obtained very good simulation results on a single task and promising experimental results on two tasks. At the end of this chapter, interesting perspectives are pointed out to improve the performance of this challenging experiment. This system constitutes the first autonomous photonic RC able to optically generate its output signal.
Doctorat en Sciences de l'ingénieur et technologie
info:eu-repo/semantics/nonPublished

Les réseaux photoniques à haute performance peuvent être considérés comme des supports pour les futurs systèmes de calcul. Contrairement à l'électronique, les systèmes photoniques offrent des avantages intéressants, par exemple la possibilité de réaliser des réseaux complètement parallèles. Récemment, les réseaux de neurones ont attiré l'attention de la communauté photonique. L'une des difficultés les plus importantes, en matière de réseaux photoniques parallèles à grande échelle, est la réalisation des connexions. La diffraction est exploitée ici comme méthode pour traiter les connexions entre les nœuds (couplage) dans les réseaux de neurones optiques. Dans cette thèse, nous étudions l'extensibilité d'un couplage diffractif en détails de la façon suivante :Tout d'abord, nous commençons par une introduction générale à propos de l'intelligence artificielle, de l'apprentissage machine, des réseaux de neurones artificiels et des réseaux de neurones photoniques. Lors de la conception d'un réseau neuronal fonctionnel, les règles de l'apprentissage machine sont des éléments essentiels pour optimiser une configuration et ainsi obtenir une faible erreur du système, donc les règles de l'apprentissage sont introduites (chapitre 1). Nous étudions les concepts fondamentaux du couplage diffractif dans notre réservoir spatio-temporel. Dans ce cas, la théorie de la diffraction est expliquée. Nous utilisons un schéma analytique pour fournir les limites en termes de taille des réseaux diffractifs, qui font partie intégrante de notre réseau neuronal photonique (chapitre 2). Les concepts du couplage diffractif sont étudiés expérimentalement dans deux expériences différentes afin de confirmer les limites obtenues analytiquement, et pour déterminer le nombre maximum de nœuds pouvant être couplés dans le réseau photonique (Chapitre 3). Les simulations numériques d'une telle expérience sont basées sur deux schémas différents pour calculer numériquement la taille maximale du réseau, qui approche une surface de 100 mm2 (chapitre 4). Enfin, l'ensemble du réseau neuronal photonique est démontré. Nous concevons un réservoir spatialement étendu sur 900 nœuds. En conséquence, notre système généralise la prédiction pour la séquence chaotique de Mackey-Glass (chapitre 5)
Photonic networks with high performance can be considered as substrates for future computing systems. In comparison with electronics, photonic systems have substantial privileges, for instance the possibility of a fully parallel implementation of networks. Recently, neural networks have moved into the center of attention of the photonic community. One of the most important requirements for parallel large-scale photonic networks is to realize the connectivities. Diffraction is considered as a method to process the connections between the nodes (coupling) in optical neural networks. In the current thesis, we evaluate the scalability of a diffractive coupling in more details as follow:First, we begin with a general introductions for artificial intelligence, machine learning, artificial neural network and photonic neural networks. To establish a working neural network, learning rules are an essential part to optimize a configuration for obtaining a low error from the system, hence learning rules are introduced (Chapter 1). We investigate the fundamental concepts of diffractive coupling in our spatio-temporal reservoir. In that case, theory of diffraction is explained. We use an analytical scheme to provide the limits for the size of diffractive networks which is a part of our photonic neural network (Chapter 2). The concepts of diffractive coupling are investigated experimentally by two different experiments to confirm the analytical limits and to obtain maximum number of nodes which can be coupled in the photonic network (Chapter 3). Numerical simulations for such an experimental setup is modeled in two different schemes to obtain the maximum size of network numerically, which approaches a surface of 100 mm2 (Chapter 4). Finally, the complete photonic neural network is demonstrated. We design a spatially extended reservoir for 900 nodes. Consequently, our system generalizes the prediction for the chaotic Mackey–Glass sequence (Chapter 5)

This manuscript investigates the use of microring resonators to create all-optical reservoir-computing networks implemented in silicon photonics. Artificial neural networks and reservoir-computing are promising applications for integrated photonics, as they could make use of the bandwidth and the intrinsic parallelism of optical signals. This work mainly illustrates two aspects: the modelling of photonic integrated circuits and the experimental results obtained with all-optical devices. The modelling of photonic integrated circuits is examined in detail, both concerning fundamental theory and from the point of view of numerical simulations. In particular, the simulations focus on the nonlinear effects present in integrated optical cavities, which increase the inherent complexity of their optical response. Toward this objective, I developed a new numerical tool, precise, which can simulate arbitrary circuits, taking into account both linear propagation and nonlinear effects. The experimental results concentrate on the use of SCISSORs and a single microring resonator as reservoirs and the complex perceptron scheme. The devices have been extensively tested with logical operations, achieving bit error rates of less than 10^−5 at 16 Gbps in the case of the complex perceptron. Additionally, an in-depth explanation of the experimental setup and the description of the manufactured designs are provided. The achievements reported in this work mark an encouraging first step in the direction of the development of novel networks that employ the full potential of all-optical devices.

Thesis: Ph. D., Massachusetts Institute of Technology, Department of Physics, 2017.
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 163-175).
This thesis focuses on a wide range of contemporary topics in modern electromagnetics and technology including topologically protected one-way modes, integrated photonic LIDAR, and optical neural networks. First, we numerically investigate large Chern numbers in photonic crystals and explore their origin from simultaneously gapping multiple band degeneracies. Following this, we perform microwave transmission measurements in the bulk and at the edge of ferrimagnetic photonic crystals. Bandgaps with large Chern numbers of 2, 3, and 4 are present in the experimental results 'which show excellent agreement with theory. We measure the mode profiles and Fourier transform them to produce dispersion relations of the edge modes, whose number and direction match our Chern number calculations. We use these waveguides to realize reflectionless power splitters and outline their application to general one-way circuits. Next we create a new chip-scale LIDAR architecture in analogy to planar RF lenses. Instead of relying upon many continuously tuned thermal phase shifters to implement nonmechanical beam steering, we use aplanatic lenses excited in their focal plane feeding ID gratings to generate discrete beams. We design devices which support up to 128 resolvable points in-plane and 80 resolvable points out-of-plane, which are currently being fabricated and tested. These devices have many advantages over conventional optical phased arrays including greatly increased optical output power and decreased electrical power for in-plane beamforming. Finally we explore a new approach for implementing convolutional neural networks through an integrated photonics circuit consisting of Mach-Zehnder Interferometers, optical delay lines, and optical nonlinearity units. This new platform, should be able to perform the order of a thousand inferences per second, at [mu]J power levels per inference, with the nearest state of the art ASIC and GPU competitors operating 30 times slower and requiring three orders of magnitude more power.
by Scott Alexander Skirlo.
Ph. D.

In this thesis a novel method for controlling a manipulator with arbitrary number of Degrees of freedom is proposed, the proposed method has the main advantages of two common controllers, the simplicity of PID controller and the robustness and accuracy of adaptive controller. The controller architecture is based on an Artificial Neural Network (ANN) and a PID controller.

The controller has the ability of solving inverse dynamics and inverse kinematics of robot with two separate Artificial Neural Networks. Since the ANN is learning the system parameters by itself the structure of controller can easily be changed to

improve the performance of robot.

The proposed controller can be implemented on a FPGA board to control the robot in real-time or the response of the ANN can be calculated offline and be reconstructed by controller using a lookup table. Error between the desired trajectory path and the path of the robot converges to zero rapidly and as the robot performs its tasks the controller learns the robot parameters and generates better control signal. The performance of controller is tested in simulation and on a real manipulator with satisfactory results.

Interest in nanophotonics continues to grow as integrated optics provides an affordable platform for areas like telecommunications, quantum information processing, and biosensing. Designing and characterizing integrated photonics components and circuits, however, remains a major bottleneck. This is especially true when complex circuits or devices are required to study a particular phenomenon.To address this challenge, this work develops and experimentally validates a novel machine learning design framework for nanophotonic devices that is both practical and intuitive. As case studies, artificial neural networks are trained to model strip waveguides, integrated chirped Bragg gratings, and microring resonators using a small number of simple input and output parameters relevant to designers. Once trained, the models significantly decrease the computational cost relative to traditional design methodologies. To illustrate the power of the new design paradigm, both forward and inverse design tools enabled by the new design paradigm are demonstrated. These tools are directly used to design and fabricate several integrated Bragg grating devices and ring resonator filters. The method's predictions match the experimental measurements well and do not require any post-fabrication training adjustments.

Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018.
This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.
Cataloged from student-submitted PDF version of thesis.
Includes bibliographical references (page 73).
Investigating the development of neuronal networks can help us to identify new therapies and treatments for conditions that affect the brain, such as autism and Alzheimer's disease. Two-photon calcium imaging has been a powerful tool for the investigation of the development of neuronal networks. However, one of the major challenges of working with two-photon calcium images is processing the large data sets, which often requires manual analysis by a skilled researcher. Here, we introduce a machine learning (ML) pipeline for the analysis of two-photon calcium image sequences. This semi-autonomous ML pipeline includes proposed methods for automatically identifying neurons, signal extraction, signal processing, event detection, feature extraction, and analysis. We run our ML pipeline on a dataset of two-photon calcium image sequences extracted by our team. This dataset includes two-photon calcium image sequences of spontaneous network activity from primary cortical cultures of Mecp2-deficient and wild-type mice. Loss-of-function mutation in the MECP2 gene, causes 95% of Rett syndrome cases and some cases of autism. We evaluate our ML pipeline using this dataset. Our ML pipeline reduces the time required to analyze two-photon calcium images from over 10 minutes to about 30 seconds per sample. Our goal is to accelerate the analysis of neuronal network function to aid in our understanding of neurological disorders and the identification of novel therapeutic targets.
by Raoul-Emil Roger Khouri.
M. Eng.

Reservoir computing est un ensemble de techniques permettant de simplifierl’utilisation des réseaux de neurones artificiels. Les réalisations expérimentales,notamment optiques, de ce concept ont montré des performances proches de l’étatde l’art ces dernières années. La vitesse élevée des expériences optiques ne permetpas d’y intervenir en temps réel avec un ordinateur standard. Dans ce travail, nousutilisons une carte de logique programmable (Field-Programmable Gate Array, ouFPGA) très rapide afin d’interagir avec l’expérience en temps réel, ce qui permetde développer de nouvelles fonctionnalités.Quatre expériences ont été réalisées dans ce cadre. La première visait à implé-menter un algorithme de online training, permettant d’optimiser les paramètresdu réseau de neurones en temps réel. Nous avons montré qu’un tel système étaitcapable d’accomplir des tâches réalistes dont les consignes variaient au cours dutemps.Le but de la deuxième expérience était de créer un reservoir computer optiquepermettant l’optimisation de ses poids d’entrée suivant l’algorithme de backpropaga-tion through time. L’expérience a montré que cette idée était tout à fait réalisable,malgré les quelques difficultés techniques rencontrées. Nous avons testé le systèmeobtenu sur des tâches complexes (au-delà des capacités de reservoir computers clas-siques) et avons obtenu des résultats proches de l’état de l’art.Dans la troisième expérience nous avons rebouclé notre reservoir computer op-tique sur lui-même afin de pouvoir générer des séries temporelles de façon autonome.Le système a été testé avec succès sur des séries périodiques et des attracteurs chao-tiques. L’expérience nous a également permis de mettre en évidence les effets debruit expérimental dans les systèmes rebouclés.La quatrième expérience, bien que numérique, visait le développement d’unecouche de sortie analogique. Nous avons pu vérifier que la méthode de onlinetraining, développée précédemment, était robuste contre tous les problèmes expéri-mentaux étudiés. Par conséquent, nous avons toutes les informations pour réalisercette idée expérimentalement.Finalement, durant les derniers mois de ma thèse, j’ai effectué un stage dont lebut était d’appliquer mes connaissance en programmation de FPGA et réseaux deneurones artificiels à un problème concret en imagerie cardiovasculaire. Nous avonsdéveloppé un programme capable d’analyser les images en temps réel, convenablepour des applications cliniques.
Doctorat en Sciences
info:eu-repo/semantics/nonPublished

L'hippocampe, structure cérébrale située dans le lobe temporal, est au coeur de la gestion de nombreuses fonctions cognitives comme l'encodage des informations spatiotemporelles ou encore la mémoire épisodique. A l'heure actuelle, l'hippocampe est étudié via de nombreuses méthodes notamment l'imagerie de fluorescence qui, utilisée sur des animaux éveillés, permet d'accéder au fonctionnement du réseau neuronal. Malgré cela, une sous-région : le gyrus denté a encore un rôle mal élucidé car profondément enfoui dans le cerveau. Son étude permettrait d'apporter de nouveaux éléments sur le fonctionnement de l'hippocampe. De part sa profondeur d’environs 1 mm, son imagerie demeure très difficile. En effet, la diffusion ainsi que les aberrations optiques introduites par les couches successives de matière dégradent fortement la qualité d'imagerie. Pourtant l'optique adaptative, une technique héritée de l'astronomie, pourrait changer cela. En l'intégrant à un microscope bi-photon, il serait possible de compenser les aberrations optiques introduites par le cerveau et ainsi d'arriver à effectuer l'imagerie in vivo du gyrus denté. Durant ma thèse, j'ai donc travaillé à la conception complète tant du point de vue matériel que logiciel d'un microscope bi-photon adapté à l'imagerie in vivo et équipé d'un dispositif de correction de front d'onde. J'ai également développé une méthode d'optimisation prometteuse basée sur l'approche modale de la correction des aberrations optiques couplée à l'utilisation d'une métrique adaptée à l'imagerie non-linéaire en profondeur. Enfin, j'ai pu appliquer cette méthode dans des conditions in vitro et in vivo permettant de montrer son efficacité
The hippocampus, a cortical structure located in the temporal lobe, is at the heart of the management of many cognitive functions such as spatiotemporal information encoding or episodic memory. At present, the hippocampus is studied through many methods including fluorescence imaging, and used on awake animals, allows access for the study of the neural network function. Despite this, a sub-region: the dentate gyrus has still a poorly elucidated role because it is deeply buried in the brain. His study would bring new elements on the hippocampus functioning. Due to its depth of about 1 mm, its imagery remains very difficult. Indeed, scattering as well as optical aberrations introduced by the successive layers of matter strongly degrade the imaging quality. Yet adaptive optics, a technique inherited from astronomy, could change that. By integrating it into a bi-photon microscope, it would be possible to compensate optical aberrations introduced by the brain and thus to achieve the in vivo imaging of the dentate gyrus. During my PhD, I worked on the complete design both in hardware and software of a bi-photon microscope suitable for in vivo imaging and equipped with a wavefront correction device. I also developed a promising optimization method based on the modal approach of optical aberration correction coupled with the use of a metric adapted to nonlinear depth imaging. Finally, I was able to apply this method in in vitro and in vivo conditions to show its effectiveness

Le LHC (Large Hadron Collider) fournit aux expériences du CERN (Laboratoire Européen pour la Physique des Particules) des collisions proton-proton avec une énergie de 7 TeV dans le centre de masse depuis fin Mars 2010. Le LHC a en particulier été conçu pour permettre la recherche du boson de Higgs, particule prédite par le modèle standard encore jamais observée à ce jour, dans toute la gamme de masse où il est attendu. Ce travail de thèse est une contribution à la recherche du boson de Higgs dans CMS (Compact Muon Solenoid), l'un des quatre grands détecteurs placés auprès du LHC, et développe plusieurs outils qui permettent la mesure des bruits de fonds et l'amélioration du potentiel de découverte. Un nouvel outil de récupération des photons émis par les leptons dans l'état final de la désintégration H --> ZZ(*) ->4$\ll (\ll= e\ mu)$ a été développé dans cette thèse. Cette méthode permet la récupération d'un nombre variable de photons par événements, donne une meilleure performance que la méthode précédemment utilisée dans CMS et permet l'amélioration de la résolution sur la masse des bosons Z0 et du boson de Higgs, ainsi qu'un gain de 5% sur la significance d'une observation du boson de Higgs dans ce canal. La deuxième partie de cette thèse traite de l'étude des bruits de fond et de la recherche d'un boson de Higgs léger (110 < mH < 140 GeV) dans le canal H --> $\gamma\gamma$. Un nouvel outil de discrimination $\gamma/\pi^i0$ à l'aide d'un réseau de neurone a été mis au point pour le rejet des photons provenant de la désintégration des $\pi^0$ produits copieusement dans les jets de QCD. Les performances du réseau de neurone sont examinées dans le détail. Le réseau de neurone est alors utilisé comme variable "template" permettant la mesure du processus $\gamma$+X à partir des données avec 10 nb−1 de luminosité intégrée dans CMS. La mesure du processus $\gamma\gamma+X$ est aussi préparée à partir de la simulation dans l'hypothèse d'une luminosité intégrée de 10 pb−1. La prise en compte des effets cinématiques aux ordres supérieurs, nécessaire pour la prédiction la plus précise possible du signal H -> $\gamma\gamma$ et du bruit de fond, est effectuée dans cette thèse par la méthode de repondération, pour le gg -> H $\gamma\gamma$ processus au NNLO et pour la première fois pour le processus $\gamma\gamma$ +X au NLO, dans les deux cas à l'aide de distributions doublement différentielles. Les outils de repondération et de discrimination $\gamma/\pi^0$ sont ensuite intégrés dans l'analyse pour améliorer la sensibilité de CMS à la recherche du boson de Higgs dans le canal H->$\gamma\gamma$ dans le modèle standard et au-delà, grâce à une paramétrisation effective développée par des phénoménologues avec lesquels nous avons travaillé.

Le Modèle Standard de la physique des particules explique avec succès les données expérimentales. L'origine de la masse des bosons W et Z est expliquée à l'aide du mécanisme de Higgs qui permet de briser la symétrie de jauge de l'interaction électro-faible. Cependant ce mécanisme prédit l'existence d'une particule, appelée le boson de Higgs, qui n'a pas été observée pour l'instant. Cette particule est recherchée au LHC en particulier dans les expériences ATLAS et CMS. Les premiers résultats utilisant les données du LHC permettent d'exclure, avec un niveau de confiance de 95%, un boson de Higgs qui aurait la section efficace du Modèle Standard entre 128 et 600 GeV/c² et les résultats plus anciens du LEP ont exclu un boson de Higgs plus léger que 114.4 GeV/c². Dans l'intervalle de masse restant, le canal de désintégration du Higgs en deux photons est le canal idéal pour la recherche du boson de Higgs car, malgré son faible rapport d'embranchement (environ quelques pour mille) et grâce à son état final clair, il permet d'obtenir une résonance de faible largeur dans le spectre de masse invariante des événements di-photons. La manière dont un photon est reconstruit dans CMS sera d'abord décrite et la compréhension de cette reconstruction avec les premières données du LHC présentée. Du fait de la faible largeur de la résonance du boson de Higgs à basse masse, un grand intérêt doit être porté à la résolution sur l'énergie des photons. C'est pourquoi, nous étudierons les corrections apportées à cette énergie. Ensuite, comme les pions neutres qui se désintègrent en deux photons sont le principal bruit de fond aux photons dans les données, nous verrons comment utiliser la forme du dépôt d'énergie dans le calorimètre électromagnétique de CMS à l'aide d'un réseau de neurones artificiels pour discriminer ces pions neutres des vrais photons. La chromodynamique quantique est la source d'un large nombre d'événements di-photons qui forment la majorité du bruit de fond à la désintégration du boson de Higgs. La mesure de la section efficace de ces processus et de leur cinématique aide aussi à la compréhension du Modèle Standard. La possibilité d'utiliser le pouvoir discriminant du réseau de neurones pour mesurer le nombre d'événements diphotons dans les données, a été étudiée. Les mésons neutres sont aussi un bruit de fond pour les photons issus de la désintégration du boson de Higgs. L'amélioration de l'identification à l'aide d'une coupure sur la variable de sortie du réseau de neurones a donc été évaluée : la conséquence de cette amélioration en termes de limite sera présentée sur le premier 1.6fb¹ des données de 2011 enregistrées par l'expérience CM
The Standard Model of particle physics successfully explains the majority of experimental high energy physics data. The masses of the W and Z, the vector bosons of the electroweak theory, are explained with a spontaneous breaking of the gauge symmetry. This symmetry breaking is performed, using the Higgs mechanism, by introducing a new scalar field, whose quantum, the Higgs boson, is intensively searched at LHC. Theoretical considerations suggest that the mass of the Higgs boson should be lower than 1 TeV/c² and the fit of precision electroweak measurements constrains the Higgs boson mass to be less than 158 GeV/c². Direct searches at LEP have excluded the Higgs boson with masses lower than 114.4 GeV/c², and direct searches at the Tevatron have led to an exclusion of masses between 147 and 180 GeV/c². The fit of precision electroweak measurements constrains the Higgs boson mass to be less than 158 GeV/c² (all these limits are at the 95% confidence level). The photon reconstruction in CMS is detailed in this thesisand its understanding with the first LHC data will be shown. Because of the narrow Higgs resonance, a particular attention as to be put on the photon energy resolution. Neutral pions decaying in two photons are the main background to the prompt photons: the possibility of using a neural network based on shower shape in ECAL is studied. These neutral mesons are also one important background to the photons from Higgs boson decay. The improvement of the photon identification, thanks to a cut on the neural network output, is evaluated: the result in term of limits for the first 1.6fb¹ of 2011 data is presented

On assiste à un essor spectaculaire des méthodes optiques pour suivre l’activité de populations neuronales in vivo. Ceci a permis de mettre en évidence des motifs remarquables d’organisation fonctionnelle à l’échelle mésoscopique impliqués dans de nombreuses fonctions cérébrales physiopathologiques. Cette thèse vise à mettre en place les outils permettant de disséquer les circuits sous-tendant ces motifs remarquables selon une approche expérimentale basée uniquement sur la microscopie optique. Plus particulièrement, ces outils ont été optimisés pour décrire la région CA1 de l’hippocampe adulte et le « barrel cortex » au cours du développement. En effet, deux motifs remarquables ont récemment été mis en évidence dans ces structures, les assemblées neuronales de CA1 adulte impliquées d’une part dans des processus de mémorisation et les neurones Hubs du cortex en développement et d’autre part participant au développement postnatal des circuits neuronaux. Dans ce contexte, nous avons développé un nouveau paradigme expérimental combinant imagerie calcique biphotonique in vivo, photostimulation par illumination holographique et analyse mathématique. Nous avons optimisé le choix et la co-expression de la sonde calcique et de l’opsine dans nos conditions expérimentales, et calibré leur utilisation dans les neurones de différentes structures cérébrales. De plus, nous avons conçu et assemblé un nouveau microscope à deux voies d’excitation, une pour l’imagerie calcique et l’autre pour la photostimulation holographique in vivo. Cette nouvelle approche expérimentale est en cours de validation sur les neurones Hubs à forte connectivité du « barrel cortex » en développement
Over The last five years we have observed a huge improvement of optical methods to monitor the activity of neuronal populations in vivo. With these new approaches, remarkable patterns of functional organization at the mesoscopic scale that are involved in many pathophysiological brain functions were highlighted. This thesis aims to develop tools allowing us to dissect the circuits underlying these remarkable patterns according to an experimental approach based on all optical microscopy. These tools have been optimized to describe the functional organization of CA1 neurons in the adult hippocampus as well as in the barrel cortex during development. Two remarkable patterns have recently been identified in these structures, first, adult CA1 neural assemblies involved in memory processes and second, Hub cortical neurons that shape neuronal circuit during development. We have developed a new experimental paradigm combining in vivo two photon calcium imaging, holography photostimulation and mathematical analysis. We optimized the choice and co-expression of calcium probe (GCaMP6s) and opsin (Chronos and ChR2H134R) in our experimental conditions and calibrated their use in neurons of different brain structures. In addition, we designed and assembled a new two-path excitation microscope, one for calcium imaging and the other for in vivo holography photostimulation. This new experimental approach is being validated on Hub neurons with high connectivity in the developing barrel cortex

Le Modèle Standard de la physique des particules explique avec succès les données expérimentales. L'origine de la masse des bosons W et Z est expliquée à l'aide du mécanisme de Higgs qui permet de briser la symétrie de jauge de l'interaction électro-faible. Cependant ce mécanisme prédit l'existence d'une particule, appelée le boson de Higgs, qui n'a pas été observée pour l'instant. Cette particule est recherchée au LHC en particulier dans les expériences ATLAS et CMS. Les premiers résultats utilisant les données du LHC permettent d'exclure, avec un niveau de confiance de 95%, un boson de Higgs qui aurait la section efficace du Modèle Standard entre 128 et 600 GeV/c$^2$ et les résultats plus anciens du LEP ont exclu un boson de Higgs plus léger que 114.4 GeV/c$^2$. Dans l'intervalle de masse restant, le canal de désintégration du Higgs en deux photons est le canal idéal pour la recherche du boson de Higgs car, malgré son faible rapport d'embranchement (environ quelques pour mille) et grâce à son état final clair, il permet d'obtenir une résonance de faible largeur dans le spectre de masse invariante des événements di-photons. La manière dont un photon est reconstruit dans CMS sera d'abord décrite et la compréhension de cette reconstruction avec les premières données du LHC présentée. Du fait de la faible largeur de la résonance du boson de Higgs à basse masse, un grand intérêt doit être porté à la résolution sur l'énergie des photons. C'est pourquoi, nous étudierons les corrections apportées à cette énergie. Ensuite, comme les pions neutres qui se désintègrent en deux photons sont le principal bruit de fond aux photons dans les données, nous verrons comment utiliser la forme du dépôt d'énergie dans le calorimètre électromagnétique de CMS à l'aide d'un réseau de neurones artificiels pour discriminer ces pions neutres des vrais photons. La chromodynamique quantique est la source d'un large nombre d'événements di-photons qui forment la majorité du bruit de fond à la désintégration du boson de Higgs. La mesure de la section efficace de ces processus et de leur cinématique aide aussi à la compréhension du Modèle Standard. La possibilité d'utiliser le pouvoir discriminant du réseau de neurones pour mesurer le nombre d'événements diphotons dans les données, a été étudiée. Les mésons neutres sont aussi un bruit de fond pour les photons issus de la désintégration du boson de Higgs. L'amélioration de l'identification à l'aide d'une coupure sur la variable de sortie du réseau de neurones a donc été évaluée : la conséquence de cette amélioration en termes de limite sera présentée sur le premier 1.6fb$^1$ des données de 2011 enregistrées par l'expérience CMS.

ABSTRACT In the modern day , we are witnessing two complementary trends, exponential growth in data and shrinking of chip size. The Data is approaching to 44 zettabytes by 2020 and the chips are now available with 10nm technology. The hyperconnectivity between machine-to-machine and humanto- machine creates multi-dimensional data which is more complex. Our thesis addresses the quantum meta layer abstraction which provides the interface to the Application layer to design quantum and classical algorithms. The first part of the thesis addresses the quantum algorithms and second part address classical algorithms running on top of quantum meta layer. In the first part of our thesis we explored quantum stochastic algorithm for ranking Quantum Webpages, analogous to the classical Google PageRank. The architecture is a six-waveguide photonic lattice that runs finely-tuned quantum stochastic walk. The evolution of density matrix solves the ranking of quantum webpages. We force the photon stochastic walk for quantum PageRank by matching the entries of Google matrix with parameters of the Kossakowski-Lindblad master equation. We have done extensive simulation to observe the density matrix evolution with different parameter settings. We have used noise in the Kossakowski-Lindblad master equation to break the symmetry (reciprocity) property of quantum system, which helps in distinguishable measurement of the quantum PageRank. We next propose a new quantum deep learning with photonic lattice waveguide as a feedforward neural network. The proposed deep photonic neural network uses the quantum properties for learning. The hidden layers of our deep photonic neural network can be designed to learn object representation and mentains the quantum quantum properties for longer time for optimal learning. The second part of the thesis discusses the data based learning. We have used data graph method which captures the system representation. The proposed data graph model captures and encodes the data efficiently and then the data graph is updated and trained with new data to provide efficient predictions. The model retains the previously learned knowledge by transfer learning and improves it with new training. The proposed method is highly adaptive and scalable for different real-time scenarios. Data graph models the system where every node (object) is associated with data and if two objects are related then they are linked with a data edge. The proposed algorithm is an incremental algorithm which learns hidden objects and hidden relationships through the data pattern over time and updates the model accordingly. We have used algebraic graph transformation methods to trigger the mutation of the Data Graph. This new updated Data Graph behaves differently for the data it observes. We explore more into machine learning algorithms and have proposed a complete framework to predict the state of the system based on the system parameters. We have proposed the discretization of the data points using the symbol algebra and used Bayesian machine learning algorithm to select the best model to represent the new data. Symbol algebra provides unified language platform to different sensor data and it can process both, the discrete and continuous data. The portability of unified language platform in processing heterogeneous and homogeneous data increases the hypotheses space and Bayesian machine learning gets more degrees of freedom in choosing the best model with high measure of confidence level in the predicted state.

Ever since the beginning of the notion behind the term 'cloud computing,' i.e., to share the pro- cessing and storage capabilities of a centralized system, there has been a signi cant increase in the availability of raw data. The challenges faced (e.g., high latency, storage limitations, channel band- width limitations, downtime) while processing such data in a cloud framework gave birth to edge computing where the idea is to push the computation to the edge of the network. Edge computing is a distributed computing paradigm, which off loads cloud because of performing data processing near to the source. For real-time applications (e.g., autonomous vehicles, air tra c control systems) where the latency is of prime concern, the deployment of Deep Neural Networks (DNNs) on the cloud would not be a feasible option. This is because of substantial inference time, enormous memory requirements, and numerous CPUs & GPUs, which translates to large power consumption. This difficulty in latency can be overcome by deploying DNN models on edge devices. Edge devices typically cannot handle a large DNN because of power and memory constraints. This lack of power and size restricts the need for small yet efficient implementation of DNN on edge devices. Promising results have been shown by employing the Extreme Learning Machine (ELM) in terms of faster training and high accuracy for Multilayer Perceptron (MLP) in applications such as object detection, recognition, and tracking. MLP being an instance of DNN could be a viable option to be deployed on edge devices. This motivates the need for analog implementation of MLP because of its characterizing fea- tures of low power and small size overcome the issues discussed above. In this work, a novel way of realizing the ELM framework on a single hidden layer feed-forward neural network (SLFN) is presented based on a Multiple-Input Floating Gate MOS (MIFGMOS) operational transconduc- tance amplifier (OTA). A multiple-input version of FGMOS called MIFGMOS is a device which because of its lossless charge sharing based voltage summation operation, dissipates meager power. The ability of a programmable threshold voltage and weighted summation of input gate voltage makes MIFGMOS an ideal device for emulation of biological neurons while working in the sub- threshold region for low power operation. Also, being able to serve as an analog memory in the form of statically stored charges renders the use of an input layer synaptic weights arrangement inessential. From the perspective of an analog neural network framework, the use of MIFGMOS improves areal density substantially. The transconductance curve of the employed OTA resembles a highly non-linear activation function (Sigmoid in this case). The slope and maximum level of the transconductance curve which are the tunable parameters of our setup serve as variability among activations. The proposed system has been implemented using 65nm Complementary Metal Oxide Semi- conductor (CMOS) process technology. The working principle of the implemented system has been veri ed by employing it for regression and classi cation tasks such as MNIST digit recognition.

Next generation wireless systems are envisioned to provide a variety of services with a wide range of performance requirements. Particularly, demand for high-mobility use cases involving high-speed trains, UAVs/drones, and aeroplanes is increasing. Also, wireless spectrum in the millimeter wave band (e.g., 28-60 GHz) is used to meet the growing bandwidth requirements. Communication in high-mobility and high-carrier frequency scenarios is challenging as it involves high Doppler shifts. Widely used modulation schemes such as orthogonal frequency division multiplexing (OFDM) perform poorly in such high-Doppler scenarios. Orthogonal time frequency space (OTFS) is a recently proposed modulation scheme which is robust to high Doppler shifts. It operates in the delay-Doppler domain and converts a high-Doppler channel into an almost static channel. In this thesis, we focus on the design of OTFS transceivers using deep neural networks (DNNs). The key contributions in the thesis can be summarized into three parts: 1) design of a low-complexity DNN architecture for OTFS signal detection, 2) design of a multi-DNN architecture for delay-Doppler channel training and detection, along with IQ imbalance (IQI) compensation, and 3) bit error rate (BER) analysis of OTFS in the presence of imperfect channel state information (CSI). First, we consider a DNN architecture in which each information symbol multiplexed in the delay-Doppler (DD) grid is associated with a separate DNN. The considered symbol-level DNN has fewer parameters to learn compared to a full DNN that takes into account all symbols in an OTFS frame jointly, and therefore has less complexity. When the noise model deviates from the standard i.i.d. Gaussian model (e.g., non-Gaussian noise with t-distribution) the proposed symbol-DNN detection is found to outperform maximum-likelihood (ML) detection, because of the ability of the DNN to learn the distribution. A similar performance advantage is observed in MIMO-OTFS systems where the noise across multiple received antennas are correlated. Next, we propose a multi-DNN transceiver architecture for DD channel training and detection, along with IQI compensation. The proposed transceiver learns the DD channel over a spatial coherence interval and detects the information symbols using a single DNN trained for this purpose at the receiver. The proposed transceiver also learns the IQ imbalances present in the transmitter and receiver and effectively compensates them. The transmit IQI compensation is realized using a single DNN at the transmitter which learns and provides a compensating modulation alphabet without explicitly estimating the transmit gain and phase imbalances. The receive IQI imbalance compensation is realized using two DNNs at the receiver, one DNN for explicit estimation of receive gain and phase imbalances and another DNN for compensation. Simulation results show that the proposed DNN-based architecture provides very good performance. Finally, we analyze the effect of imperfect CSI on the BER performance of OTFS. We carry out the BER analysis when a mismatched ML detector is used, i.e., when an estimated channel matrix is used for detection in place of the true channel matrix. We derive an exact expression for the pairwise error probability (PEP) using the characteristic function of the decision statistic. Using the PEP, an upper bound on the BER is obtained. Our results show that the BER bound is tight at high SNR values. We also obtain the decision rule for the true ML detector in the presence of imperfect CSI, which takes into account the channel estimation error statistics. We quantify the performance gap between the true ML detector and the mismatched ML detector through simulations.

Graph neural networks (GNNs) have become very popular for processing and analyzing graph-structured data in the last few years. Using message passing as their basic building blocks that aggregate information from neighborhoods, GNN architectures learn low-dimensional graph-level or node-level embeddings useful for several downstream machine learning tasks. In this thesis, we focus on GNN architectures that perform parallel neighborhood aggregations (in short, referred to as PA-GNNs) for two tasks, namely, graph classification and link prediction. Such architectures have a natural advantage of reduced training and inference time as neighborhood aggregation is done before training, unlike GNNs that perform the neighborhood aggregation sequentially (referred to as SA-GNNs) during training. Thus, the runtime of SA-GNNs depends on the number of edges in the graph. In the first part of the thesis, we propose a generic model for GNNs with parallel neighborhood aggregation and theoretically characterize the discriminative power of PA-GNN models to discriminate between two non-isomorphic graphs. We provide conditions under which they are provably powerful as the well-known Weisfeiler-Lehman graph isomorphism test. We then specialize the generic PA-GNN model and propose a GNN architecture, which we call SPIN: simple and parallel graph isomorphism network. Next to the theoretical characterization of the developed PA-GNN model, we also present numerical experiments on a diverse variety of real-world benchmark graph classification datasets related to social networks, chemical molecular data, and brain networks. The proposed model achieves state-of-the-art performance with reduced training and inference time. In the second part of the thesis, we propose a computational method for drug repurposing by presenting a deep learning model that captures the complex interactions between the drugs, diseases, genes, and anatomies in a large-scale interactome with over 1.4 million connections. Specifically, we propose a PA-GNN based drug repurposing architecture, which we call GDRnet, to screen a large drug database of clinically approved drugs and predict potential drugs that can be repurposed for novel diseases. GNN-based machine learning models are a natural choice for computational drug repurposing because of their ability to capture the underlying structural information in such complex biological networks. While the proposed PA-GNN architecture is computationally attractive, we present results from numerical experiments to show the efficacy of GNNs for drug repurposing. We also provide numerical experimental results on drug repurposing for coronavirus diseases (including COVID-19), where many of the drugs predicted by the proposed model are considered as the mainstay treatment.

Human speech is one of many acoustic signals we perceive, which carries linguistic and paralinguistic (e.g., speaker identity, emotional state) information. Speech acoustics are produced as a result of different temporally overlapping gestures of speech articulators (such as lips, tongue tip, tongue body, tongue dorsum, velum, and larynx), each of which regulates constriction in different parts of the vocal tract. Estimating speech acoustic representations from articulatory movements is known as articulatory- to-acoustic forward (AAF) mapping i.e., articulatory speech synthesis. While estimating articulatory movements back from the speech acoustics is known as acoustic-to-articulatory inverse (AAI) mapping. These acoustic- articulatory mapping functions are known to be complex and nonlinear. The complexity of this mapping depends on a number of factors. These include the kind of representations used in the acoustic and articulatory spaces. Typically these representations capture both linguistic and paralinguistic aspects in speech. How each of these aspects contributes to the complexity of the mapping is unknown. These representations and, in turn, the acoustic-articulatory mapping are affected by the speaking rate as well. The nature and quality of the mapping vary across speakers. Thus, the complexity of mapping also depends on the amount of data from a speaker as well as the number of speakers used in learning the mapping function. Further, how the language variations impact the mapping requires detailed investigation. This thesis analyzes a few of such factors in detail and develops neural-network based models to learn mapping functions robust to many of these factors. Electromagnetic articulography (EMA) sensor data has been used directly in the past as articulatory representations for learning the acoustic-articulatory mapping function. In this thesis, we address the problem of optimal EMA sensor placement such that the air-tissue boundaries as seen in the mid-sagittal plane of the real-time magnetic resonance imaging (rtMRI) are reconstructed with minimum error. Following optimal sensor placement work, acoustic-articulatory data was collected using EMA from 41 subjects with speech stimuli in English and Indian native languages (Hindi, Kannada, Tamil, and Telugu), resulting in a total of ∼23 hours of data, used in this thesis. Representations from raw waveform are also learned for AAI task using convolutional and bidirectional long short term memory neural networks (CNN-BLSTM), where the learned filters of CNN are found to be similar to those used for computing Mel-frequency cepstral coefficients (MFCCs), typically used for AAI task. In order to examine the extent to which a representation having only the linguistic information can recover articulatory representations, we replace MFCC vectors with one-hot encoded vectors representing phonemes, which were further modified to remove the time duration of each phoneme and keep only phoneme sequence. Experiments with phoneme sequence using attention network achieve an AAI performance that is identical to that using phoneme with timing information, while there is a drop in performance compared to that using MFCC. Experiments to examine variation in speaking rate reveal that the errors in estimating the vertical motion of tongue articulators from acoustics with fast speaking rate are significantly higher than those with slow speaking rate. In order to reduce the demand for data from a speaker, low resource AAI is proposed using a transfer learning approach. Further, we show that AAI can be modeled to learn acoustic-articulatory mappings of multiple speakers through a single AAI model rather than building separate speaker-specific models. This is achieved by conditioning an AAI model with speaker embeddings, which benefits AAI in seen and unseen speaker evaluations. Finally, we show the benefit of estimated articulatory representations in voice conversion applications. Experiments revealed that articulatory representations estimated from speaker-independent AAI preserve linguistic information and suppress speaker-dependent factors. These articulatory representations (from an unseen speaker and language) are used to drive target speaker-specific AAF to synthesis speech, which preserves linguistic information and the target speaker’s voice characteristics.

碩士
國立中山大學
應用數學系研究所
107
The main goal of this study is to classify brain single photon emission computed tomography(SPECT) images using convolutional neural network with stacking which is a method of ensemble learning.We use the five representative images in the Digital Imaging and Communications in Medicine(DICOM) storage format selected by the doctor, and the people who have done Tc-99m TRODAT-1 brain SPECT were grouped by dichotomy (normal and abnormal) and trichotomy (normal, minor and abnormal). We predict the severity of Parkinson''s disease by using two-stage prediction. Firstly, VGG-based reduction and transfer learning methods and the two different grouping ways (dichotomy and trichotomy) are used to form four different convolutional neural networks for prediction. Then, the previous prediction values are integrated as the input of logistic regression. The logistic regression model is used to predict the degree of disease. We use accuracy, precision, and recall as criteria for our model, and Grad-CAM is used to present the judgment basis of the convolutional neural network in the classification. This result can be developed into a system that is an auxiliary for the doctors diagnosing the Parkinson''s disease and assist rookie doctors in diagnostic training.

Speech enhancement is at the heart of many applications such as speech com- munication, automatic speech recognition, hearing aids etc. In this work, we consider the speech enhancement under the framework of multiple deep neural network (DNN) system. DNNs have been extensively used in speech enhance- ment due to its ability to capture complex variations in the input data. As a natural extension, researchers have used variants of a network with multi- ple DNNs for speech enhancement. Input data could be clustered to train each DNN or train all the DNNs jointly without any clustering. In this work, we pro- pose clustering methods for training multiple DNN systems and its variants for speech enhancement. One of the proposed works involves grouping phonemes into broad classes and training separate DNN for each class. Such an approach is found to perform better than single DNN based speech enhancement. However, it relies on phoneme information which may not be available for all corpora. Hence, we propose a hard expectation-maximization (EM) based task speci c clustering method, which, automatically determines clusters without relying on the knowledge of speech units. The idea is to redistribute the data points among multiple DNNs such that it enables better speech enhancement. The experimen- tal results show that the hard EM based clustering performs better than the single DNN based speech enhancement and provides results similar to that of the broad phoneme class based approach.

There has been a giant leap in technological advancement with the introduction of graphene and its remarkable properties after 2005. Since the inception of graphene, the new class of materials called 2D materials are actively being focused on for their potential use case. The recent introduction of magnetism in 2D materials has sparked a new interest among researchers due to the potential use of magnetic properties in spintronics, which is highly admired in storage devices. The extensive library of newly predicted or even synthesized 2D materials made it impossible to screen them experimentally. Therefore, theoretical and computational tools like Density Functional Theory (DFT), Monte Carlo and Molecular dynamics simulations have been the tool of choice for high-throughput screening and insight finding. Even though computational methods worked well, but they generally demand substantial computational resources. The expanding grasp of machine learning algorithms has been overreaching for material engineering. The idea to club ML algorithms with the rising 2D crystal structures and their DFT calculated properties along with other material data has enabled us to create predictive models encompassing underlying physics using machine learning which can screen the materials much faster with relatively similar accuracies in limited resources. Many materials have been investigated using machine learning algorithms to predict their properties, such as crystal structures, curie temperatures, bandgaps, Fermi energies, and charge density wave phases. In this work, we use a graph-based neural network model (CGCNN) and several highly customized hybrid ML models to identify the magnetic materials from three different databases with heavily skewed data topology. We have employed several supervised ML algorithms to determine how accurate they are in predicting the magnetic state or the amount of anisotropy using the crystal structure as the only source of information. A further effort to develop a complementary regression model for the prediction of magnetic anisotropy

This thesis is primarily focused on building a systematic theory for spatio-temporal memories based on unsupervised learning and exploring its applications. Also, contributions are made in the area of supervised learning techniques. The self-organizing map (SOM) is a biologically inspired unsupervised learning paradigm which finds numerous applications in learning, clustering and recalling spatial input patterns. SOM is also used as a software tool for visualizing high-dimensional data space by projecting it onto a lower dimensional space, typically two-dimensions and can be considered as a vector quantization (VQ) technique. Since the time SOM was originally published, many variants of it have appeared and used extensively in engineering applications. A notable variant is the neural gas (NGAS) algorithm. Though the SOM and its variants work well in practice, the learning rules in the basic SOM are heuristic. Also, the techniques are well suited for clustering and recall of spatial patterns. However, if the data is spatio-temporal, these techniques fail to learn the temporal dynamics along with the spatial proximities in the input data space. The traditional approach for learning spatio-temporal patterns is to incorporate time on the output space of a SOM along with heuristic update rules that work well in practice. Inspired by the pioneering work of Alan Turing, who used reaction-diffusion equations to explain spatial pattern formation, we develop an analogous theoretical model for building a spatio-temporal memory to learn and recall temporal patterns. Using coupled reaction-diffusion equations, we develop a mathematical theory from first principles for constructing a spatio-temporal SOM (STSOM) and derive an update rule for learning based on the gradient of a potential function. The temporal plasticity effect observed during recall in response to the input dynamics is mathematically quantified. We develop a systematic theory to reconstruct missing samples in a time series using a spatio-temporal memory. Towards this goal, a novel algorithm to estimate the Markov order of a given input sequence is proposed. The efficacy of the reconstruction technique is tested on synthetic and real data sets to validate the theory. Several applications require classification of the data by prioritizing some of the input attributes. The basic SOM and its variants consider the entire input without any priority over the attributes. We propose an unsupervised learning algorithm called the priority based soft vector quantization feature map (PSVQFM) and an architecture that prioritizes attributes of the input data towards clustering. We also present an analysis on the misclassification error and prove that the proposed algorithm is asymptotically optimal. Though the PSVQFM is not directly linked to spatio-temporal memories, the proposed algorithm can be folded within a spatio-temporal memory architecture that incorporates priority based attributes within spatio-temporal sequences. Learning rate is a crucial parameter governing the convergence rate of any learning algorithm. Most of the learning algorithms based on the stochastic gradient descent (SGD) method depend on heuristic choices of the learning rate. In this thesis, we derive improved bounds on the learning rate of SGD based adaptive learning algorithms by analyzing the largest eigenvalue of the Hessian matrix from first principles. This idea can be explored for both supervised and unsupervised learning algorithms based on SGD. We have also contributed novel ideas and techniques for supervised learning algorithms, summarized below. We proposed an artificial neural network (ANN) architecture based on the back-propagation algorithm that non-linearly transforms a given probability density function (pdf) into a desired pdf. We provide an estimate on the number of hidden neurons based on the input statistics. This approach is useful for density estimation problems. The Hessian based bounds on learning rates for algorithms based on SGD and the supervised learning technique for pdf transformation are relegated to the Appendices to keep the flow of the thesis coherent.

The convolutional neural networks (CNNs) have become the most successful models for many pattern recognition problems in the areas of computer vision, speech, text and others. One concern about CNNs has always been their need for large amount of training data, large computational re- sources and long training time. In this regard the transfer learning is a technique that can address this concern of inefficient CNN training through reuse of pretrained networks (CNNs). In this thesis we discuss transfer learning in CNNs where the transfer is from multiple source CNNs and done at subnetwork levels. The subnetwork multisource transfer is attempted for the fi rst time and hence we begin by showing the effectiveness of such a transfer. We consider subnetworks at various granularities for the transfer. These granularities begin at a whole network-level then pro-ceed to layer-level and further fi lter-level. In order to realize this kind of transfer we create a set called bank of weight fi lters (BWF) which is a repository of the pretrained subnetworks that are used as candidates for transfer. Through extensive simulations we show that subnetwork level transfer, implemented through random selection from a BWF, is elective and is also efficient in terms of training time. We also present experimental results to show that subnetwork level transfer learning is efficient in terms of the amount of training data needed. It is seen that fi lter-level transfer learning is as effective as the whole-network-level transfer which is the conventional transfer learning used with CNNs. We then show the usefulness of the fi lter-level multisource transfer for the cases of transfer from natural to non-natural (hand drawn sketches) image datasets and transfer across different CNN architectures (having different number of layers, fi lter dimensions etc.). We also discuss transfer from CNNs trained on high-resolution images to the CNNs needed for the low-resolution im- ages and vice-versa. In the multisource transfer of prelearnt weights discussed above, the transferred weights have to be fi netuned to achieve the same accuracy as that of a CNN trained from scratch. It is certainly more bene cfiial and efficient if the fi netuning of transferred weights can be completely avoided. For this, we conceptualize we conceptualize what we call a fi lter-tree which represents the complete feature generation entity that is learnt by a CNN and propose that the a filter-tree represents a subnetwork that can be used for transfer without finetuning. Similar to BWF we create a repository of pre-learnt fllter-trees called bank of filter-trees (BFT) to realize the transfer using fi lter-trees. Through experiments we show that transfer using BFT (where the transferred weights are held fixed and are not fi netunes) has performance that is on par with training from scratch, which is the best achievable performance. The selection of the subnetworks from BWF or BFT so far for all experiments was done uniformly randomly. For the sake of completion we introduce a method that can result in informed choice of fi lters from a BFT. We propose a learnable auxilliary layer called choice layer whose learnt weights give an idea of the importance/utility of different the subnetwork (fi lter-trees here) in the BFT for the target task. We show that when the random choice from BFT does not achieve the best possible accuracy, the choice layer based method can achieve it.

An inverse problem in Electromagnetics (EM) refers to the process of reconstructing the physical system by processing the measured data of its electromagnetic properties. Inverse problems are typically ill-posed, and this makes them far more challenging than the typically well-posed forward problem. The solution of such inverse problems finds applications in nondestructive testing and evaluation, biomedical imaging, geophysical exploration etc. This thesis addresses some inverse problems specific to the area of electromagnetics, arising in three different scenarios. The first problem is 3-D quantitative imaging primarily targeted towards bio-medical applications. The task is to retrieve the dielectric properties, location and the shape of an unknown object from the measured scattered field. The unknown object is modeled by discretization into several voxels, with each voxel having its own dielectric property. As the inverse problem is non-linear, typically an iterative optimization process is adopted, and a forward problem needs to be solved at every iteration. The total time for reconstruction depends on the forward solver time and the number of iterations. In many cases, the number of unknowns to be reconstructed is prohibitively large. Further, the non-convergence or false-convergence of the optimization process presents its own challenge. This thesis proposes two methodologies to solve these challenges. In the first approach a multilevel methodology is proposed where voxels are hierarchically decomposed into smaller voxels based on an appropriate indicator, leading to a non-uniform multilevel voxel structure aimed at reducing the eventual number of unknowns to be solved for, also enabling faster convergence. In the second approach, a two-stage framework is proposed comprising of Machine Learning classification followed by optimization (ML-OPT). The first stage generates an appropriate adaptive grid for the optimization process and provides a suitable initial guess aiding convergence to the global minima. This approach is aimed at detecting breast tumors where the optimization algorithm can aim for higher resolution in the suspected tumor region, while using lower resolution elsewhere. The second problem is in the domain of high-speed circuits and is focused on synthesis of transmission line physical parameters given the desired electrical parameters like characteristic impedance and propagation constant. A forward solver is used to train Neural network for several different configurations for analysis and an optimization algorithm is used for synthesis. The third problem is focused on finding the source of radiation in an electronic system e.g. an automotive ECU, given the measured field at the antenna in the radiated emissions setup. The source of radiation can be from common mode current on the cable harness or from the Design Under Test (DUT). A method based on Huygens box is proposed to quantify the radiation from cable and DUT at each frequency. On each cell of the Huygens box the value of electric field computed at the observation point taking the Electric Current (J) and Magnetic Current (M) on that cell as sources and this information on the Huygens box is used to quantify the radiation. Some part of the presented work is used via technology-transfer at Simyog Technology Pvt. Ltd., an IISc incubated startup, to develop a simulation software called Compliance-scope which allows the hardware designer to predict the EMI/EMC performance of electronics modules from an early design stage.