Dissertations / Theses on the topic 'Spatial Hearing'

Thesis (M.S.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1995.
Includes bibliographical references (leaves 60-62).
by Keith Dana Martin.
M.S.

Small timing differences occur when sounds reach one ear before the other, creating interaural phase differences (IPDs). The phase-locked activity in the auditory nerve can, at low frequencies, preserve IPDs. IPDs are used for localising and separating sounds from different directions. Chapters 3, 5, and 6 report three studies of the independent effects of age and sensorineural hearing loss on the temporal processing of sound that aids spatial hearing. Chapters 2 and 4 describe two supporting methodological studies. Chapter 2 compared the duration of training required for stable IPD-discrimination thresholds for two stimulus presentation procedures. The procedure requiring the least training was adopted for subsequent studies. Age and hearing loss are related and both may affect sensitivity to IPDs. Chapter 3 demonstrated that hearing loss, regardless of listener age, is related to poorer sensitivity to IPDs in the temporal fine structure (TFS), but not in the temporal envelope. Chapter 3 also showed that age, independent of hearing loss, is related to poorer envelope-IPD sensitivity at low modulation rates, and somewhat poorer TFS-IPD sensitivity. In Chapter 5, listener age and IPD sensitivity were both compared to subcortical neural phase locking measured through the frequency-following response (FFR). Phase coherence in the envelope-FFR at 145 Hz modulation and in the TFS-FFR deteriorated with age, suggesting less precise phase locking in old age. However, age-related changes to IPD sensitivity were not strongly related to age-related changes in FFR phase coherence. IPD sensitivity declines may be predominantly caused by deterioration of binaural processing independent of subcortical phase locking. Chapter 4 showed that electrodes at the mastoids recorded TFS-FFR generated earlier in the auditory pathway than electrodes from the nape of the neck to forehead, which recorded FFR generated later in the brainstem. However, these electrode montages did not reveal different age- or hearing-loss-related FFR deficits in Chapter 5. Chapter 6 determined whether hearing loss affected the ability to use TFS IPDs to achieve better speech perception. On average, old hearing-impaired listeners gained a small, but significant, benefit from a lateral separation of the speech sources. Replacing the TFS with binaurally in-phase sine waves (removing the TFS IPDs) significantly reduced the benefit of lateral separation. How much a listener benefitted from intact TFS IPDs in speech perception was strongly related to the extent of their hearing loss at low frequencies and their monaural processing of TFS, but not to their ability to discriminate IPDs. In general, this thesis shows that low-frequency hearing loss is associated with poor sensitivity to TFS IPDs and the ability to benefit from them when sounds are laterally separated. The thesis also shows that old age can reduce sensitivity to IPDs and weaken subcortical temporal coding. Although only partly related, these effects are likely to cause problems for old individuals in challenging listening environments.

Dynamic-range compression is used in hearing devices to reduce the wide range of environmental sound levels into a range better suited to the capability of the impaired ear. Its use is motivated by the fact that the healthy ear itself performs this function, but this natural compression is typically reduced or lost with sensorineural hearing loss. This thesis explores how dynamic-range compression influences aspects of spatial hearing that play an important role in everyday listening. Spatial hearing largely relies on comparing information from the two ears. The first two experiments investigated how spatial perception is affected when compression is applied independently at each ear, as occurs in traditional bilateral hearing-device fittings. This was found to have a variety of possib le adverse effects, such as altering the perceived position of sounds and making them appear more spatially diffuse. The effects are explained in terms of changes to the underlying acoustic cues. Some modern hearing devices incorporate a wireless link, allowing compression to be synchronized across the ears. The third experiment investigated how this might provide an advantage when listening to speech in the presence of a spatially separated noise. It was found that a small to moderate benefit was obtained, compared to unlinked compression, and that this was realized th rough changes to the monaural signal at the ear that had the more favourable ratio of speech-to-noise energy. The fourth experiment tested whether the natural compression that occurs within the healthy cochlea directly affects the use of the relative level difference between the two ears as a spatial cue. Contrary to the experimental hypothesis, it was found that the potency of this cue changes little as the overall sound intensity is varied over a wide range, raising interesting questions about how this cue is evaluated at a neural level.

Doutoramento em Música
This thesis explores the possibilities of spatial hearing in relation to sound perception, and presents three acousmatic compositions based on a musical aesthetic that emphasizes this relation in musical discourse. The first important characteristic of these compositions is the exclusive use of sine waves and other time invariant sound signals. Even though these types of sound signals present no variations in time, it is possible to perceive pitch, loudness, and tone color variations as soon as they move in space due to acoustic processes involved in spatial hearing. To emphasize the perception of such variations, this thesis proposes to divide a tone in multiple sound units and spread them in space using several loudspeakers arranged around the listener. In addition to the perception of sound attribute variations, it is also possible to create rhythm and texture variations that depend on how sound units are arranged in space. This strategy permits to overcome the so called "sound surrogacy" implicit in acousmatic music, as it is possible to establish cause-effect relations between sound movement and the perception of sound attribute, rhythm, and texture variations. Another important consequence of using sound fragmentation together with sound spatialization is the possibility to produce diffuse sound fields independently from the levels of reverberation of the room, and to create sound spaces with a certain spatial depth without using any kind of artificial sound delay or reverberation.
Esta tese explora as possibilidades da Audição Espacial em relação à percepção do som e apresenta três composições acusmáticas baseadas numa estética musical que enfatiza esta relação e a incorpora como uma parte do seu discurso musical. A primeira característica importante destas composições é a utilização exclusiva de sinusóides e de outros sinais sonoros invariáveis no tempo. Embora estes tipos de sinais não apresentem variações no tempo, é possível percepcionar variações de altura, intensidade e timbre assim que estes se movem no espaço, devido aos processos acústicos envolvidos na audição espacial. Para enfatizar a percepção destas variações, esta tese propõe dividir um som em múltiplas unidades e espalhá-las no espaço utilizando vários monitores dispostos à volta da plateia. Além da percepção de variações de características do som, também é possível criar variações de ritmo e de textura que dependem de como os sons são dispostos no espaço. Esta estratégia permite superar o problema de “sound surrogacy” implícito na música acusmática, uma vez que é possível estabelecer relações causa-efeito entre o movimento do som e a percepção de variações de características do som, variações do ritmo e textura. Outra consequênça importante da utilização da fragmentação com a espacialização do som é a possibilidade de criar campos sonoros difusos, independentemente dos níveis de reverberação da sala, e de criar espaços sonoros com uma certa profundidade, sem utilizar nenhum tipo de delay ou reverberação artificiais.

This thesis provides an overview of work conducted to investigate human spatial hearing in situations involving multiple concurrent sound sources. Much is known about spatial hearing with single sound sources, including the acoustic cues to source location and the accuracy of localisation under different conditions. However, more recently interest has grown in the behaviour of listeners in more complex environments. Concurrent sound sources pose a particularly difficult problem for the auditory system, as their identities and locations must be extracted from a common set of sensory receptors and shared computational machinery. It is clear that humans have a rich perception of their auditory world, but just how concurrent sounds are processed, and how accurately, are issues that are poorly understood. This work attempts to fill a gap in our understanding by systematically examining spatial resolution with multiple sound sources. A series of psychophysical experiments was conducted on listeners with normal hearing to measure performance in spatial localisation and discrimination tasks involving more than one source. The general approach was to present sources that overlapped in both frequency and time in order to observe performance in the most challenging of situations. Furthermore, the role of two primary sets of location cues in concurrent source listening was probed by examining performance in different spatial dimensions. The binaural cues arise due to the separation of the two ears, and provide information about the lateral position of sound sources. The spectral cues result from location-dependent filtering by the head and pinnae, and allow vertical and front-rear auditory discrimination. Two sets of experiments are described that employed relatively simple broadband noise stimuli. In the first of these, two-point discrimination thresholds were measured using simultaneous noise bursts. It was found that the pair could be resolved only if a binaural difference was present; spectral cues did not appear to be sufficient. In the second set of experiments, the two stimuli were made distinguishable on the basis of their temporal envelopes, and the localisation of a designated target source was directly examined. Remarkably robust localisation was observed, despite the simultaneous masker, and both binaural and spectral cues appeared to be of use in this case. Small but persistent errors were observed, which in the lateral dimension represented a systematic shift away from the location of the masker. The errors can be explained by interference in the processing of the different location cues. Overall these experiments demonstrated that the spatial perception of concurrent sound sources is highly dependent on stimulus characteristics and configurations. This suggests that the underlying spatial representations are limited by the accuracy with which acoustic spatial cues can be extracted from a mixed signal. Three sets of experiments are then described that examined spatial performance with speech, a complex natural sound. The first measured how well speech is localised in isolation. This work demonstrated that speech contains high-frequency energy that is essential for accurate three-dimensional localisation. In the second set of experiments, spatial resolution for concurrent monosyllabic words was examined using similar approaches to those used for the concurrent noise experiments. It was found that resolution for concurrent speech stimuli was similar to resolution for concurrent noise stimuli. Importantly, listeners were limited in their ability to concurrently process the location-dependent spectral cues associated with two brief speech sources. In the final set of experiments, the role of spatial hearing was examined in a more relevant setting containing concurrent streams of sentence speech. It has long been known that binaural differences can aid segregation and enhance selective attention in such situations. The results presented here confirmed this finding and extended it to show that the spectral cues associated with different locations can also contribute. As a whole, this work provides an in-depth examination of spatial performance in concurrent source situations and delineates some of the limitations of this process. In general, spatial accuracy with concurrent sources is poorer than with single sound sources, as both binaural and spectral cues are subject to interference. Nonetheless, binaural cues are quite robust for representing concurrent source locations, and spectral cues can enhance spatial listening in many situations. The findings also highlight the intricate relationship that exists between spatial hearing, auditory object processing, and the allocation of attention in complex environments.

Thesis (M.Sc.Eng.) PLEASE NOTE: Boston University Libraries did not receive an Authorization To Manage form for this thesis or dissertation. It is therefore not openly accessible, though it may be available by request. If you are the author or principal advisor of this work and would like to request open access for it, please contact us at open-help@bu.edu. Thank you.
Perception of sound laterality (left-right angle) is mediated by both interaural time differences (ITD) and interaural level differences (ILD). Previous localization studies in anechoic settings consistently show that low-frequency ITDs dominate perception of source laterality. However, reverberant energy differentially degrades ITDs and ILDs; the effects of room reflections on the perceptual weight given to ITDs and ILDs are not well understood. Here, we tested the hypothesis that high-frequency envelope ITD cues are important for spatial judgments in reverberant rooms by measuring the perceived laterality of high-pass, low-pass and broadband sounds. Results show that when ILD cues and ITD envelope cues are both available, reverberant energy has the smallest effect on localization of high-pass stimuli. When ILD cues are set to zero, localization of high-pass stimuli with strong envelopes (i.e. click trains and speech tokens) is also minimally affected by reverberant energy; however, as envelope modulation is reduced, subjects show increasing localization bias, responding towards the center. Moreover, for stimuli with strong envelopes, subjects with better modulation detection sensitivity are affected less by the addition of reverberant energy. These results suggest that, in contrast to in anechoic space, high-frequency envelope ITD cues influence localization in reverberant settings.
2031-01-01

This thesis provides an overview of work conducted to investigate human spatial hearing in situations involving multiple concurrent sound sources. Much is known about spatial hearing with single sound sources, including the acoustic cues to source location and the accuracy of localisation under different conditions. However, more recently interest has grown in the behaviour of listeners in more complex environments. Concurrent sound sources pose a particularly difficult problem for the auditory system, as their identities and locations must be extracted from a common set of sensory receptors and shared computational machinery. It is clear that humans have a rich perception of their auditory world, but just how concurrent sounds are processed, and how accurately, are issues that are poorly understood. This work attempts to fill a gap in our understanding by systematically examining spatial resolution with multiple sound sources. A series of psychophysical experiments was conducted on listeners with normal hearing to measure performance in spatial localisation and discrimination tasks involving more than one source. The general approach was to present sources that overlapped in both frequency and time in order to observe performance in the most challenging of situations. Furthermore, the role of two primary sets of location cues in concurrent source listening was probed by examining performance in different spatial dimensions. The binaural cues arise due to the separation of the two ears, and provide information about the lateral position of sound sources. The spectral cues result from location-dependent filtering by the head and pinnae, and allow vertical and front-rear auditory discrimination. Two sets of experiments are described that employed relatively simple broadband noise stimuli. In the first of these, two-point discrimination thresholds were measured using simultaneous noise bursts. It was found that the pair could be resolved only if a binaural difference was present; spectral cues did not appear to be sufficient. In the second set of experiments, the two stimuli were made distinguishable on the basis of their temporal envelopes, and the localisation of a designated target source was directly examined. Remarkably robust localisation was observed, despite the simultaneous masker, and both binaural and spectral cues appeared to be of use in this case. Small but persistent errors were observed, which in the lateral dimension represented a systematic shift away from the location of the masker. The errors can be explained by interference in the processing of the different location cues. Overall these experiments demonstrated that the spatial perception of concurrent sound sources is highly dependent on stimulus characteristics and configurations. This suggests that the underlying spatial representations are limited by the accuracy with which acoustic spatial cues can be extracted from a mixed signal. Three sets of experiments are then described that examined spatial performance with speech, a complex natural sound. The first measured how well speech is localised in isolation. This work demonstrated that speech contains high-frequency energy that is essential for accurate three-dimensional localisation. In the second set of experiments, spatial resolution for concurrent monosyllabic words was examined using similar approaches to those used for the concurrent noise experiments. It was found that resolution for concurrent speech stimuli was similar to resolution for concurrent noise stimuli. Importantly, listeners were limited in their ability to concurrently process the location-dependent spectral cues associated with two brief speech sources. In the final set of experiments, the role of spatial hearing was examined in a more relevant setting containing concurrent streams of sentence speech. It has long been known that binaural differences can aid segregation and enhance selective attention in such situations. The results presented here confirmed this finding and extended it to show that the spectral cues associated with different locations can also contribute. As a whole, this work provides an in-depth examination of spatial performance in concurrent source situations and delineates some of the limitations of this process. In general, spatial accuracy with concurrent sources is poorer than with single sound sources, as both binaural and spectral cues are subject to interference. Nonetheless, binaural cues are quite robust for representing concurrent source locations, and spectral cues can enhance spatial listening in many situations. The findings also highlight the intricate relationship that exists between spatial hearing, auditory object processing, and the allocation of attention in complex environments.

The purpose of this study was to ascertain whether there was indeed a relationship between processing strategy (temporal or spatial processing) and internal speech. The task utilized for the study, was constructed from the tasks used by O'Connor and Hermeiin (1973a), Conrad (1979), and Booth(1982). It was then administered to 71 hearing impaired and 74 hearing subjects. The groups were matched according to vocabulary level, rather than chronological age. Results strongly indicate that there is indeed a relationship between temporal processing and internal speech for hearing threshold level groups. Age was shown to be a determining factor for choice of processing strategy for both hearing and hearing impaired groups. Severity of hearing loss was not found to have an effect on processing strategy. Profoundly deaf children, and hearing children did not demonstrate a preferred processing strategy. It would seem necessary to continue to investigate the relationship between processing strategy and internal speech. Perhaps looking at individuals rather than heterogeneous groups would yield additional information.
Education, Faculty of
Educational and Counselling Psychology, and Special Education (ECPS), Department of
Graduate

In this thesis I attempt to explain mechanisms of neuronal coding in the auditory system as a form of adaptation to statistics of natural stereo sounds. To this end I analyse recordings of real-world auditory environments and construct novel statistical models of these data. I further compare regularities present in natural stimuli with known, experimentally observed neuronal mechanisms of spatial hearing. In a more general perspective, I use binaural auditory system as a starting point to consider the notion of function implemented by sensory neurons. In particular I argue for two, closely-related tenets: 1. The function of sensory neurons can not be fully elucidated without understanding statistics of natural stimuli they process. 2. Function of sensory representations is determined by redundancies present in the natural sensory environment. I present the evidence in support of the first tenet by describing and analysing marginal statistics of natural binaural sound. I compare observed, empirical distributions with knowledge from reductionist experiments. Such comparison allows to argue that the complexity of the spatial hearing task in the natural environment is much higher than analytic, physics-based predictions. I discuss the possibility that early brain stem circuits such as LSO and MSO do not \"compute sound localization\" as is often being claimed in the experimental literature. I propose that instead they perform a signal transformation, which constitutes the first step of a complex inference process. To support the second tenet I develop a hierarchical statistical model, which learns a joint sparse representation of amplitude and phase information from natural stereo sounds. I demonstrate that learned higher order features reproduce properties of auditory cortical neurons, when probed with spatial sounds. Reproduced aspects were hypothesized to be a manifestation of a fine-tuned computation specific to the sound-localization task. Here it is demonstrated that they rather reflect redundancies present in the natural stimulus. Taken together, results presented in this thesis suggest that efficient coding is a strategy useful for discovering structures (redundancies) in the input data. Their meaning has to be determined by the organism via environmental feedback.

In binaural hearing aids, several beamforming algorithms can be used. These beamformers aim to enhance the target speech signal and preserve the binaural cues of the target source (e.g. with constraints on the target). However, the binaural cues of the other directional sources as well the background noise are often lost after processing. This affects the global impression of the acoustic scene, and it limits the perceptual separation of the sources by the hearing aids users. To help the hearing aids users to localize all the sound sources, it is important to keep the binaural cues of all directional sources and the background noise. Therefore, this work is devoted to find the best trade-off between the noise/interferers reduction and the cues preservations not only for the directional interferers but also for the background noise based on selection and mixing processes. In this thesis, some classification decision algorithms, which are based on different criteria such as the power, the power difference, and the coherence, are proposed to complete the selection and mixing processes. Simulations are completed using recorded signals provided by a hearing aid manufacturer to validate the performance of the proposed algorithm under different realistic acoustic scenarios. After detailed testing using different complex acoustic scenarios and different beamforming configurations, the results indicate that some of the proposed classification decision algorithms show good promise, in particular the classification decision algorithm based on coherence.

The experiments reported in this thesis investigated the bases of the difficulties that older adults report when trying to listen to what one person is saying when many other people are speaking at the same time. Experiments 1–4 examined the roles of voluntary and involuntary attention in a spatial listening task for speech among young normally-hearing listeners. When talkers started speaking one at a time, listeners could hear out a target phrase that was less intense than overlapping masker phrases. When talkers started speaking in pairs, listeners could attend to a less intense target phrase only when told in advance who to listen for, where they would speak from, or when they would speak. The distracting effect of the onset of a competing talker was effective over a broad time window. Experiment 5 investigated the relationships between performance on the spatial listening task and several predictors of performance among young and older normally-hearing adults. Poorer performance was related to self-reported difficulties with listening in everyday situations, poorer hearing sensitivity, and poorer performance on visual and auditory tasks of attention requiring fast speed of processing. Experiment 6 examined brain activity associated with successful performance on the spatial listening task using magneto-encephalography. Differences in cortical activity were identified at moments when attention had to be sustained on the target phrase, or when listeners had to resist distraction from the onset of a new masker phrase. Amplitudes, and/or latencies, of differences in brain activity arising in regions associated with attentional processes were related to performance. The results suggest that skills in attention contribute to the ability to listen successfully in multi-talker environments. Age-related difficulties with listening in those environments may arise due to a specific reduction in the ability to resist distraction or a general reduction in the speed at which information can be processed.

In bats, the directivity patterns for reception are shaped by the surface geometry of the pinnae. Since many bat species are capable of large ear deformations, these beampatterns can be time-variant. To investigate this time-variance using numerical methods, a digital model that is capable of representing the pinna geometry during the entire deformation cycle has been developed.

Due to large deformations and occlusions, some of the surfaces relevant to sound diffraction may not be visible and the geometry of the entire pinna has to be computed from limited data. This has been achieved by combining a complete digital model of the pinna in one position with time-variant sparse sets of three dimensional landmark data. The landmark positions were estimated using stereo vision methods. A finite element model based on elasticity was constructed from CT scans of the pinna post mortem. This elastic model was deformed to provide a good fit to the positions of the landmarks and retain values of smoothness and surface energy comparable to life. This model was able to handle ratios of data to degrees of freedom around 1:5000 and still effect life-like deformations with an acceptable goodness of fit.
Master of Science

This research was designed to investigate the fundamental nature in which blind people utilize audible cues to attend to their surroundings. Knowledge on how blind people respond to external spatial stimuli is expected to assist in development of better tools for helping people with visual disabilities navigate their environment. There was also interest in determining how blind people compare to sighted people in auditory localization tasks. The ability of sighted individuals, blindfolded individuals, and blind individuals in localizing spatial auditory targets was assessed. An acoustic display board allowed the researcher to provide multiple sound presentations to the subjects. The subjects’ responses in localization tasks were measured using a combination of kinematic head tracking and eye tracking hardware. Data was collected and analyzed to determine the ability of the groups in localizing spatial auditory targets. Significant differences were found among the three groups in spatial localization error and temporal patterns.

Objectifs : Mesurer le niveau de handicap perceptif secondaire à une surdité unilatérale par approches psychoacoustique, questionnaires de qualité de vie et IRMf. Ce travail repose sur deux études qui étudient un lien entre : 1) les performances psychoacoustiques et le ressenti de qualité de vie et 2) les performances psychoacoustiques et le niveau de réorganisation centrale. Matériels et méthodes : Pour l'étude 1, 49 sujets adultes présentant une surdité unilatérale (UHL) et 11 contrôles ont été recrutés. Ils ont été évalués en psychoacoustique par le test FrMatrix en champ libre et dans trois conditions et ont répondu aux questionnaires de qualité de vie le Speech, Spatial and Quality of Hearing Scale (SSQ) et le Glasgow Health Status Inventory (GHSI). Pour l'étude 2, 15 sujets UHL et 15 contrôles ont été recrutés. Outre les tests psychoacoustiques de l'étude 1, les sujets ont écouté activement des sons naturels en IRMf. Résultats : L'étude 1 retrouve une corrélation significative (rho = 0.38) entre le score global au SSQ et le test FrMatrix lorsque le signal est envoyé à l'oreille déficiente et le bruit à l'oreille saine (condition dichotique). L'étude 2 démontre une corrélation significative entre le niveau de réorganisation et le FrMatrix (rho = 0.66) en condition dichotique. Conclusion : La surdité unilatérale engendre un niveau de réorganisation corticale qui s'accompagne d'une baisse significative des performances psychoacoustiques et de la qualité de vie. Ce travail souligne donc la nécessité d'une prise en charge thérapeutique des sujets présentant une surdité unilatérale
Objectives: We evaluated the consequences of unilateral hearing loss (UHL) for speech recognition in spatially separated competing noise, quality of life and brain activity via fMRI. In the first study we investigated the relationship between speech recognition and quality of life (Study 1), and in the second between speech recognition and the reorganization of brain activity. Materials and methods: In the first study 49 UHL subjects and 11 controls were recruited. Speech recognition in three speech and noise configurations was assessed with the FrMatrix test and quality of life with the Speech Spatial and Quality of Hearing Scale (SSQ) and the Glasgow Health Status Inventory. In the second study, 14 UHL and 14 matched controls were recruited. They underwent the same speech recognition testing as in the first study and also participated in a fMRI experiment that involved actively listening to naturals sounds. Results: The first study showed a significant correlation (r = 0.38) between SSQ scores and speech recognition where the competing noise was send to the better ear and the noise contralaterally (dichotic). The second study showed a significant correlation (r = 0.66) between speech recognition scores (dichotic listening situation)and brain reorganization. Conclusion: Unilateral hearing loss induces brain reorganization that significantly correlates with deficits in speech recognition in noise and overall quality of life. This work highlights the need for greater attention to improve the understanding and rehabilitation of unilateral hearing loss

Advisers: Terry Takahashi and Richard Marrocco. xiv, 152 p.
Neurons in the barn owl's inferior colliculus (IC) derive their spatial receptive fields (RF) from two auditory cues: interaural time difference (ITD) and interaural level difference (ILD). ITD serves to restrict a RF in azimuth but the precise role of ILD was, up to this point, unclear. Filtering by the ears and head insures that each spatial location is associated with a unique combination of frequency-specific ILD values (i.e., an ILD spectrum). We isolated the effect of ILD spectra using virtual sound sources in which ITD was held fixed for all spatial locations while ILD spectra were allowed to vary normally. A cell's response to these stimuli reflects the contribution of ILD to spatial tuning, referred to as an “ILD-alone RF”. In a sample of 34 cells, individual ILD-alone RFs were distributed and amorphous, but consistently showed that the ILD spectrum is facilatory at the cell's best location and inhibitory above and/or below. Prior results have suggested that an IC cell's spatial specificity is generated by summing inputs which are narrowly tuned to frequency and selective for both ILD and ITD. Based on this premise, we present a developmental model which, when trained solely on a cell's true spatial RF, reproduces both the cell's true RF and its ILD-alone RF. According to the model, the connectivity between a space-tuned IC cell and its frequency-specific inputs develops subject to two constraints: the cell must be excited by ILD spectra from the cell's best location and inhibited by spectra from locations above and below but along the vertical strip defined by the best ITD. To assess how frequency-specific inputs are integrated to form restricted spatial RFs, we measured the responses of 47 space-tuned IC cells to pure tones at varying ILDs and frequencies. ILD tuning varied with frequency. Further, pure-tone responses, summed according to the head-related filters, accounted for 56 percent of the variance in broadband ILD-alone RFs. Modelling suggests that, with broadband sounds, cells behave as though they are linearly summing their inputs, but when testing with pure tones, non-linearities arise. This dissertation includes unpublished co-authored materials.

The main focus of the research presented within this thesis is, as the title suggests, binaural spatialization. Binaural technology and, especially, the binaural recording technique are not particu-larly recent. Nevertheless, the interest in this technology has lately become substantial due to the increase in the calculation power of personal computers, which started to allow the complete and accurate real-time simulation of three-dimensional sound-fields over headphones. The goals of this body of research have been determined in order to provide elements of novelty and of contribution to the state of the art in the field of binaural spatialization. A brief summary of these is found in the following list: • The development and implementation of a binaural spatialization technique with Distance Simulation, based on the individual simulation of the distance cues and Binaural Reverb, in turn based on the weighted mix between the signals convolved with the different HRIR and BRIR sets; • The development and implementation of a characterization process for modifying a BRIR set in order to simulate different environments with different characteristics in terms of frequency response and reverb time; • The creation of a real-time and offline binaural spatialization application, imple-menting the techniques cited in the previous points, and including a set of multichannel(and Ambisonics)-to-binaural conversion tools. • The performance of a perceptual evaluation stage to verify the effectiveness, realism, and quality of the techniques developed, and • The application and use of the developed tools within both scientific and artistic “case studies”. In the following chapters, sections, and subsections, the research performed between January 2006 and March 2010 will be described, outlining the different stages before, during, and after the development of the software platform, analysing the results of the perceptual evaluations and drawing conclusions that could, in the future, be considered the starting point for new and innovative research projects.

Sound localisation is one of the most representative function of the auditory system and, as such, it has been extensively investigated across species. Spatial hearing can be dramatically altered across the life span, yet research in humans have highlighted the remarkable capacity of the brain to adapt to changes of listening conditions, such as temporary ear plugging or long lasting hearing impairments. Although several investigations have examined accommodation to altered auditory cues (Chapter 1), a common theoretical framework seems to lack and a number of questions remain open. This limits the possibility to translate our current knowledge into concrete clinical applications for individuals who experience spatial hearing difficulties after hearing loss. The current dissertation reflects the attempt to answer specific questions regarding the process of sound localisation. The first study (Chapter 2) aimed to investigate the relation between different reference frames in spatial hearing, namely egocentric and allocentric sound representation. We studies this topic in the context of a learning paradigm, assessing to what extent localisation of single sounds in simulated monaural hearing (unilateral ear plugging) can improve following an audio-visual spatial hearing training focused on egocentric sound processing vs allocentric sound processing. An untrained group was also included in the study. We found that localisation performance in the horizontal plane improved specifically in the side ipsilateral to the ear-plug for all groups. Yet, the trained groups showed a qualitatively different change of performance after four days of multisensory ego/allocentric training compared to the untrained group, providing initial evidence of the possible role of allocentric coding in acoustic space re-learning. These results further highlight the importance of including a test-retest group in paradigms of sound localisation training. The second study (Chapter 3) focused on a specific aspect of the phenomenological experience of spatial hearing, namely the subjective confidence about the perceived sound position. We examined the relation between objective localisation accuracy and subjective certainty while participants localised sounds in two different listening conditions – binaural or simulated monaural hearing. Results showed that overall subjective certainty on sound position decreased in the altered listening condition (unilateral ear-plugging). In simulated monaural hearing, localisation accuracy and spatial confidence dissociated. For instance, there were trials in which participants were accurate, but felt uncertain, and trials in which they were less accurate but expressed higher ratings of spatial confidence on sound position. Furthermore, subjective confidence increased as a function of time within the testing block, and it was related to the spatial distribution of the perceived sound-source position. The third study (Chapter 4) exploited magnetoencephalography (MEG) to study the dynamics of the cortical network implied in active sound localisation. We implemented a novel apparatus to study sound localisation in MEG with real sounds in external space, and collected behavioural and subjective responses (i.e., accuracy and confidence, as in Study 2) during this altered listening condition. Results showed that participants were able to perceive the spatial difference between the positions of stimulation, thus proving the reliability of our novel setting for the study of spatial hearing in MEG. MEG data highlight a distributed bilateral cortical network involved in active sound localisation, which emerged shortly after stimulus presentation (100—125 ms). The network comprise the classical dorsal auditory pathway plus other cortical regions usually underestimated in previous literature – most notably, regions in the central sulcus/precentral gyrus possibly involved in head movements. Connectivity analysis revealed different patterns of neural coupling, as a function of frequency band. In particular, coherence in high gamma revealed significant connections involving the parietal cortex and the posterior superior temporal cortex. In the final chapter (Chapter 5), I summarise the main findings of the three studies, discuss their implications and outline potential future directions.

Sound localisation is one of the most representative function of the auditory system and, as such, it has been extensively investigated across species. Spatial hearing can be dramatically altered across the life span, yet research in humans have highlighted the remarkable capacity of the brain to adapt to changes of listening conditions, such as temporary ear plugging or long lasting hearing impairments. Although several investigations have examined accommodation to altered auditory cues (Chapter 1), a common theoretical framework seems to lack and a number of questions remain open. This limits the possibility to translate our current knowledge into concrete clinical applications for individuals who experience spatial hearing difficulties after hearing loss. The current dissertation reflects the attempt to answer specific questions regarding the process of sound localisation. The first study (Chapter 2) aimed to investigate the relation between different reference frames in spatial hearing, namely egocentric and allocentric sound representation. We studies this topic in the context of a learning paradigm, assessing to what extent localisation of single sounds in simulated monaural hearing (unilateral ear plugging) can improve following an audio-visual spatial hearing training focused on egocentric sound processing vs allocentric sound processing. An untrained group was also included in the study. We found that localisation performance in the horizontal plane improved specifically in the side ipsilateral to the ear-plug for all groups. Yet, the trained groups showed a qualitatively different change of performance after four days of multisensory ego/allocentric training compared to the untrained group, providing initial evidence of the possible role of allocentric coding in acoustic space re-learning. These results further highlight the importance of including a test-retest group in paradigms of sound localisation training. The second study (Chapter 3) focused on a specific aspect of the phenomenological experience of spatial hearing, namely the subjective confidence about the perceived sound position. We examined the relation between objective localisation accuracy and subjective certainty while participants localised sounds in two different listening conditions – binaural or simulated monaural hearing. Results showed that overall subjective certainty on sound position decreased in the altered listening condition (unilateral ear-plugging). In simulated monaural hearing, localisation accuracy and spatial confidence dissociated. For instance, there were trials in which participants were accurate, but felt uncertain, and trials in which they were less accurate but expressed higher ratings of spatial confidence on sound position. Furthermore, subjective confidence increased as a function of time within the testing block, and it was related to the spatial distribution of the perceived sound-source position. The third study (Chapter 4) exploited magnetoencephalography (MEG) to study the dynamics of the cortical network implied in active sound localisation. We implemented a novel apparatus to study sound localisation in MEG with real sounds in external space, and collected behavioural and subjective responses (i.e., accuracy and confidence, as in Study 2) during this altered listening condition. Results showed that participants were able to perceive the spatial difference between the positions of stimulation, thus proving the reliability of our novel setting for the study of spatial hearing in MEG. MEG data highlight a distributed bilateral cortical network involved in active sound localisation, which emerged shortly after stimulus presentation (100—125 ms). The network comprise the classical dorsal auditory pathway plus other cortical regions usually underestimated in previous literature – most notably, regions in the central sulcus/precentral gyrus possibly involved in head movements. Connectivity analysis revealed different patterns of neural coupling, as a function of frequency band. In particular, coherence in high gamma revealed significant connections involving the parietal cortex and the posterior superior temporal cortex. In the final chapter (Chapter 5), I summarise the main findings of the three studies, discuss their implications and outline potential future directions.

Les principales techniques de spatialisation sonores sont pour la plupart orientées vers la reproduction de sources ponctuelles, l’étendue reste un sujet peu exploré. Cette thèse défend l’idée que les sources étendues sont pourtant des objets expressifs pouvant significativement contribuer à la richesse des pratiques de spatialisation, notamment en musique. Nous décomposerons cette thèse en trois hypothèses. La première, auditive, postulera que les sources étendues sont pertinentes perceptivement. C’est-à-dire qu’elles offrent la possibilité de faire varier de nouveaux attributs sonores et que l’auditeur est sensible à ces variations. La seconde, analytique, proposera que la polarisation des techniques de spatialisation les plus courantes vers la source ponctuelle n’est qu’arbitraire et que d’autres modèles de sources peuvent être considérés. La troisième, opérationnelle, suggèrera qu’il est possible de créer des outils permettant aux compositeurs de manier et de s’approprier les sources étendues à des fins musicales.Pour valider ces hypothèses, nous formaliserons les propriétés auditives et musicales de ces sources puis nous proposerons des méthodes concrètes pour les analyser et les synthétiser. Ces travaux seront considérés dans le cadre de la bibliothèque HOA, un ensemble d’outils de spatialisation de bas niveau que nous avons spécialement fondé à des fins d’expérimentations. Nous décrirons les spécificités de cette bibliothèque. Nous verrons notamment comment son architecture et ses différents modules permettent d’ouvrir l’ambisonie à de nouvelles pratiques éloignées du concept de ponctualité
Mainstream spatialization techniques are often oriented towards the reproduction of point sources; extension remains a relatively unexplored topic. This thesis advocates that extended sources are yet expressive objects that could contribute to the richness of spatialization practices, especially in the field of music. We’ll decompose this thesis in three hypotheses. A perceptive one, who postulates that extended sources are perceptually relevant, i.e., that they offer the possibility of varying new sound attributes and that the listener is sensitive to these variations. An analytical one, who proposes that the most common spatialization techniques focus to point sources is arbitrary and that other source’s models can be considered. And an operational one, who suggests that it’s possible to create tools for composers so they can handle and musicalize extended objects. To confirm these hypotheses, we’ll formalize the auditory and musical properties of extended sources and we’ll propose concrete methods for their analysis and synthesis. This work will be considered as part of the HOA library, a set of low-level spatialization tools we’ve founded for the purpose of experimentation. We’ll describe the specificities of this library and see how its architecture and its different modules allow the generalization of ambisonics to new practices away of punctuality

L'objectif de cette thèse était d'étudier la modalité auditive dans le syndrome de négligence spatiale unilatérale (NSU) que ce soit dans l'évaluation ou dans la prise en charge. Notre première hypothèse défendue était que, bien que la NSU touche aussi la modalité auditive, des différences dues aux spécificités de chaque modalité existent entre les symptômes auditifs et visuels de ce syndrome neuropsychologique. En conséquence, notre seconde hypothèse était que les spécificités propres à la modalité auditive pourraient permettre de pallier les déficits attentionnels rencontrés en modalité visuelle et, donc, faire de cette modalité un support efficace pour la rééducation. Concernant l'évaluation des symptômes auditifs, les mécanismes d'orientation de l'attention en modalité auditive ainsi que la latéralisation de sons ont été explorés chez des patients atteints de NSU. Ces études ont souligné des difficultés dans les deux domaines dans la NSU. Concernant la réhabilitation, la place de l'audition, et plus particulièrement de la musique, dans les rééducations proposées aux patients atteints de NSU a été explorée. Enfin, un programme de rééducation basé sur la pratique musicale a été créé et testé auprès d'une patiente atteinte de NSU chronique. Celui-ci a montré des bénéfices à long-terme sur les signes visuels de la NSU mais également dans la vie quotidienne. Les résultats de ces études vont dans le sens des hypothèses et soulignent l'importance de prendre en compte la modalité auditive dans la pratique clinique que ce soit lors de l'évaluation du syndrome de NSU ou lors de sa réhabilitation
The aim of this thesis was to focus on hearing in the evaluation and the rehabilitation of unilateral spatial neglect (USN) syndrome. USN not only affects the visual, but also the auditory modality. The first hypothesis of this thesis was that, due to the specificities of each modality, differences exist between the auditory and visual symptoms of USN. Consequently, the second hypothesis was that the specificities of the auditory modality could permit to compensate the attentional deficits that occur in the visual modality and, thus, make this modality an efficient tool for the rehabilitation. Concerning the evaluation of the auditory symptoms, orienting attention mechanisms in the auditory modality as well as sound lateralisation were explored with patients with USN. These studies showed patients with USN to have difficulties in both. Concerning the rehabilitation, the place of hearing, and in particular of music, in the re-educations in patients with USN was explored. Finally, a program based on music practice was developed and tested with a patient with a chronic USN. This patient showed long-term benefits on USN visual signs and also on daily activities. The results of these studies are consistent with the hypotheses and underline the importance of considering hearing in clinical practice either for the USN evaluation or for its rehabilitation

Les principales techniques de spatialisation sonores sont pour la plupart orientées vers la reproduction de sources ponctuelles, l’étendue reste un sujet peu exploré. Cette thèse défend l’idée que les sources étendues sont pourtant des objets expressifs pouvant significativement contribuer à la richesse des pratiques de spatialisation, notamment en musique. Nous décomposerons cette thèse en trois hypothèses. La première, auditive, postulera que les sources étendues sont pertinentes perceptivement. C’est-à-dire qu’elles offrent la possibilité de faire varier de nouveaux attributs sonores et que l’auditeur est sensible à ces variations. La seconde, analytique, proposera que la polarisation des techniques de spatialisation les plus courantes vers la source ponctuelle n’est qu’arbitraire et que d’autres modèles de sources peuvent être considérés. La troisième, opérationnelle, suggèrera qu’il est possible de créer des outils permettant aux compositeurs de manier et de s’approprier les sources étendues à des fins musicales.Pour valider ces hypothèses, nous formaliserons les propriétés auditives et musicales de ces sources puis nous proposerons des méthodes concrètes pour les analyser et les synthétiser. Ces travaux seront considérés dans le cadre de la bibliothèque HOA, un ensemble d’outils de spatialisation de bas niveau que nous avons spécialement fondé à des fins d’expérimentations. Nous décrirons les spécificités de cette bibliothèque. Nous verrons notamment comment son architecture et ses différents modules permettent d’ouvrir l’ambisonie à de nouvelles pratiques éloignées du concept de ponctualité
Mainstream spatialization techniques are often oriented towards the reproduction of point sources; extension remains a relatively unexplored topic. This thesis advocates that extended sources are yet expressive objects that could contribute to the richness of spatialization practices, especially in the field of music. We’ll decompose this thesis in three hypotheses. A perceptive one, who postulates that extended sources are perceptually relevant, i.e., that they offer the possibility of varying new sound attributes and that the listener is sensitive to these variations. An analytical one, who proposes that the most common spatialization techniques focus to point sources is arbitrary and that other source’s models can be considered. And an operational one, who suggests that it’s possible to create tools for composers so they can handle and musicalize extended objects. To confirm these hypotheses, we’ll formalize the auditory and musical properties of extended sources and we’ll propose concrete methods for their analysis and synthesis. This work will be considered as part of the HOA library, a set of low-level spatialization tools we’ve founded for the purpose of experimentation. We’ll describe the specificities of this library and see how its architecture and its different modules allow the generalization of ambisonics to new practices away of punctuality

Thesis (Ph.D)--Psychology, Georgia Institute of Technology, 2010.
Committee Chair: Walker, Bruce N.; Committee Member: Corballis, Paul M.; Committee Member: Corso, Gregory M.; Committee Member: Folds, Dennis J.; Committee Member: Houtsma, Adrianus J. M. Part of the SMARTech Electronic Thesis and Dissertation Collection.

In the world of ICT, strategies for innovation and development are increasingly focusing on applications that require spatial representation and real-time interaction with and within 3D media environments. One of the major challenges that such applications have to address is user-centricity, reflecting e.g. on developing complexity-hiding services so that people can personalize their own delivery of services. In these terms, multimodal interfaces represent a key factor for enabling an inclusive use of the new technology by everyone. In order to achieve this, multimodal realistic models that describe our environment are needed, and in particular models that accurately describe the acoustics of the environment and communication through the auditory modality. Examples of currently active research directions and application areas include 3DTV and future internet, 3D visual-sound scene coding, transmission and reconstruction and teleconferencing systems, to name but a few. The concurrent presence of multimodal senses and activities make multimodal virtual environments potentially flexible and adaptive, allowing users to switch between modalities as needed during the continuously changing conditions of use situation. Augmentation through additional modalities and sensory substitution techniques are compelling ingredients for presenting information non-visually, when the visual bandwidth is overloaded, when data are visually occluded, or when the visual channel is not available to the user (e.g., for visually impaired people). Multimodal systems for the representation of spatial information will largely benefit from the implementation of audio engines that have extensive knowledge of spatial hearing and virtual acoustics. Models for spatial audio can provide accurate dynamic information about the relation between the sound source and the surrounding environment, including the listener and his/her body which acts as an additional filter. Indeed, this information cannot be substituted by any other modality (i.e., visual or tactile). Nevertheless, today's spatial representation of audio within sonification tends to be simplistic and with poor interaction capabilities, being multimedia systems currently focused on graphics processing mostly, and integrated with simple stereo or multi-channel surround-sound. On a much different level lie binaural rendering approaches based on headphone reproduction, taking into account that possible disadvantages (e.g. invasiveness, non-flat frequency responses) are counterbalanced by a number of desirable features. Indeed, these systems might control and/or eliminate reverberation and other acoustic effects of the real listening space, reduce background noise, and provide adaptable and portable audio displays, which are all relevant aspects especially in enhanced contexts. Most of the binaural sound rendering techniques currently exploited in research rely on the use of Head-Related Transfer Functions (HRTFs), i.e. peculiar filters that capture the acoustic effects of the human head and ears. HRTFs allow loyal simulation of the audio signal that arrives at the entrance of the ear canal as a function of the sound source's spatial position. HRTF filters are usually presented under the form of acoustic signals acquired on dummy heads built according to mean anthropometric measurements. Nevertheless, anthropometric features of the human body have a key role in HRTF shaping: several studies have attested how listening to non-individual binaural sounds results in evident localization errors. On the other hand, individual HRTF measurements on a significant number of subjects result both time- and resource-expensive. Several techniques for synthetic HRTF design have been proposed during the last two decades and the most promising one relies on structural HRTF models. In this revolutionary approach, the most important effects involved in spatial sound perception (acoustic delays and shadowing due to head diffraction, reflections on pinna contours and shoulders, resonances inside the ear cavities) are isolated and modeled separately with a corresponding filtering element. HRTF selection and modeling procedures can be determined by physical interpretation: parameters of each rendering blocks or selection criteria can be estimated from real and simulated data and related to anthropometric geometries. Effective personal auditory displays represent an innovative breakthrough for a plethora of applications and structural approach can also allow for effective scalability depending on the available computational resources or bandwidth. Scenes with multiple highly realistic audiovisual objects are easily managed exploiting parallelism of increasingly ubiquitous GPUs (Graphics Processing Units). Building individual headphone equalization with perceptually robust inverse filtering techniques represents a fundamental step towards the creation of personal virtual auditory displays (VADs). To this regard, several examples might benefit from these considerations: multi-channel downmix over headphones, personal cinema, spatial audio rendering in mobile devices, computer-game engines and individual binaural audio standards for movie and music production. This thesis presents a family of approaches that overcome the current limitations of headphone-based 3D audio systems, aiming at building personal auditory displays through structural binaural audio models for an immersive sound reproduction. The resulting models allow for an interesting form of content adaptation and personalization, since they include parameters related to the user's anthropometry in addition to those related to the sound sources and the environment. The covered research directions converge to a novel framework for synthetic HRTF design and customization that combines the structural modeling paradigm with other HRTF selection techniques (inspired by non-individualized HRTF selection procedures) and represents the main novel contribution of this thesis: the Mixed Structural Modeling (MSM) approach considers the global HRTF as a combination of structural components, which can be chosen to be either synthetic or recorded components. In both cases, customization is based on individual anthropometric data, which are used to either fit the model parameters or to select a measured/simulated component within a set of available responses. The definition and experimental validation of the MSM approach addresses several pivotal issues towards the acquisition and delivery of binaural sound scenes and designing guidelines for personalized 3D audio virtual environments holding the potential of novel forms of customized communication and interaction with sound and music content. The thesis also presents a multimodal interactive system which is used to conduct subjective test on multi-sensory integration in virtual environments. Four experimental scenarios are proposed in order to test the capabilities of auditory feedback jointly to tactile or visual modalities. 3D audio feedback related to user’s movements during simple target following tasks is tested as an applicative example of audio-visual rehabilitation system. Perception of direction of footstep sounds interactively generated during walking and provided through headphones highlights how spatial information can clarify the semantic congruence between movement and multimodal feedback. A real time, physically informed audio-tactile interactive system encodes spatial information in the context of virtual map presentation with particular attention to orientation and mobility (O&M) learning processes addressed to visually impaired people. Finally, an experiment analyzes the haptic estimation of size of a virtual 3D object (a stair-step) whereas the exploration is accompanied by a real-time generated auditory feedback whose parameters vary as a function of the height of the interaction point. The collected data from these experiments suggest that well-designed multimodal feedback, exploiting 3D audio models, can definitely be used to improve performance in virtual reality and learning processes in orientation and complex motor tasks, thanks to the high level of attention, engagement, and presence provided to the user. The research framework, based on the MSM approach, serves as an important evaluation tool with the aim of progressively determining the relevant spatial attributes of sound for each application domain. In this perspective, such studies represent a novelty in the current literature on virtual and augmented reality, especially concerning the use of sonification techniques in several aspects of spatial cognition and internal multisensory representation of the body. This thesis is organized as follows. An overview of spatial hearing and binaural technology through headphones is given in Chapter 1. Chapter 2 is devoted to the Mixed Structural Modeling formalism and philosophy. In Chapter 3, topics in structural modeling for each body component are studied, previous research and two new models, i.e. near-field distance dependency and external-ear spectral cue, are presented. Chapter 4 deals with a complete case study of the mixed structural modeling approach and provides insights about the main innovative aspects of such modus operandi. Chapter 5 gives an overview of number of a number of proposed tools for the analysis and synthesis of HRTFs. System architectural guidelines and constraints are discussed in terms of real-time issues, mobility requirements and customized audio delivery. In Chapter 6, two case studies investigate the behavioral importance of spatial attribute of sound and how continuous interaction with virtual environments can benefit from using spatial audio algorithms. Chapter 7 describes a set of experiments aimed at assessing the contribution of binaural audio through headphones in learning processes of spatial cognitive maps and exploration of virtual objects. Finally, conclusions are drawn and new research horizons for further work are exposed in Chapter 8.
Il settore dell'Information and Communications Technology (ICT) sta investendo in strategie di innovazione e sviluppo sempre più rivolte ad applicazioni capaci di interazione complesse grazie alla rappresentazione spaziale in ambienti virtuali multimodali capaci di rispettare i vincoli di tempo reale. Una delle principali sfide da affrontare riguarda la centralità dell'utente, che si riflette, ad esempio, sullo sviluppo di servizi la cui complessità tecnologica viene nascosta al destinatario, e la cui offerta di servizi sia personalizzabile dall’utente e per l’utente. Per queste ragioni , le interfacce multimodali rappresentano un elemento chiave per consentire un uso diffuso di queste nuove tecnologie. Per raggiungere questo obiettivo è necessario ottenere dei modelli multimodali realistici che siano capaci di descrivere l’ambiente circostante, e in particolare modelli che sappiano rappresentare accuratamente l'acustica dell'ambiente e la trasmissione di informazione attraverso la modalità uditiva. Alcuni esempi di aree applicative e direzioni di ricerca attive nella comunità scientifica internazionale includono 3DTV e internet del futuro , codifica, trasmissione e ricostruzione della scena 3D video e audio e sistemi di teleconferenza , per citarne solo alcuni. La presenza concomitante di più modalità sensoriali e la loro integrazione rendono gli ambienti virtuali multimodali potenzialmente flessibili e adattabili, permettendo agli utenti di passare dall’una all’altra modalità in base alle necessità dettata dalle mutevoli condizioni di utilizzo di tali sistemi. Modalità sensoriali aumentata attraverso altri sensi e tecniche di sostituzione sensoriale sono elementi essenziali per la veicolazione dell’informazioni non visivamente, quando, ad esempio, il canale visivo è sovraccaricato, quando i dati sono visivamente ostruiti, o quando il canale visivo non è disponibile per l'utente (ad esempio, per le persone non vedenti). I sistemi multimodali per la rappresentazione delle informazioni spaziali beneficano sicuramente della realizzazione di motori audio che possiedano una conoscenza approfondita degli aspetti legati alla percezione spaziale e all’acustica virtuale. I modelli per il rendering di audio spazializzato sono in grado di fornire accurate informazioni dinamiche sulla relazione tra la sorgente sonora e l'ambiente circostante , compresa l'interazione del corpo dell’ascoltatore che agisce da ulteriore filtraggio acustico. Queste informazioni non possono essere sostituite da altre modalità (ad esempio quella visiva o tattile). Tuttavia , la rappresentazione spaziale del suono nei feedback acustici tende ad essere, al giorno d’oggi, semplicistica e con scarse capacità di interazione, questo perchè i sistemi multimediali attualmente si focalizzano per lo più sull’elaborazione grafica, e si accontentano di semplici tecnologie stereofoniche o surround multicanale per il rendering del suono. Il rendering binaurale riprodotto in cuffia rappresenta un approccio avveniristico, tenendo conto che i possibili svantaggi (es. invasività , risposte in frequenza non piane) possono essere man mano gestiti e controbilanciati da una serie di desiderabili caratteristiche. Questi sistemi sono caratterizzati dalla possibilità di controllare e/o eliminare il riverbero e altri effetti acustici dello spazio di ascolto circostante, di ridurre il rumore di fondo e fornire dei display audio adattabili e portatili, tutti aspetti rilevanti soprattutto in contesti di innovazione. La maggior parte delle tecniche di rendering binaurale impiegate oggigiorno in ricerca si basano sull'uso di Head Related Transfer Functions (HRTFs), vale a dire di filtri particolari che catturano gli effetti acustici di testa, busto e orecchie dell’ascoltatore. Le HRTF permettono una simulazione fedele del segnale audio che si presenta all'ingresso del canale uditivo in funzione della posizione spaziale della sorgente sonora. I filtri basati su HRTF sono generalmente presentati sotto forma di segnali acustici misurati a partire da una testa di manichino costruito secondo misurazioni antropometriche medie. Tuttavia, le caratteristiche antropometriche individuali hanno un ruolo fondamentale nel determinare le HRTF: diversi studi hanno riscontrato come l’ascolto di audio binaurale non individuale produce errori di localizzazione evidenti . D'altra parte , le misurazioni individuali di HRTF su un numero significativo di soggetti richiedono un impiego di risorse e tempo non trascurabili. Sono state proposte negli ultimi due decenni diverse tecniche per il design di HRTF sintetiche e tra le più promettente vi è quella che utilizza i modelli strutturali di HRTF. In questo approccio rivoluzionario, gli effetti più importanti coinvolti nella percezione spaziale del suono (i ritardi acustici e le ombre acustiche ad opera della diffrazione attorno alla testa, le riflessioni sui contorni dell’orecchio esterno e sulle spalle, le risonanze all'interno delle cavità dell’orecchio) sono isolati e modellati separatamente nell’elemento filtrante corrispondente. La selezione di HRTF non individuali e queste procedure di modellazione possono essere entrambe analizzate con una interpretazione fisica: i parametri di ogni blocco di rendering o i criteri di selezione possono venir stimati dalla relazione tra dati reali e simulati e antropometria dell’ascoltatore. La realizzazione di efficaci display uditivi personali rappresenta un notevole passo in avanti per numerose applicazioni; l’approccio strutturale consente una intrinseca scalabilità a seconda delle risorse computazionali o della larghezza di banda disponibili. Scene altamente realistiche con più oggetti audiovisivi riescono ad essere gestite sfruttando il parallelismo della Graphics Processing Unit (GPU) sempre più onnipresenti. Ottenere un equalizzazione individuale delle cuffie con tecniche di filtraggio inverso che siano percettivamente robuste costituisce un passo fondamentale verso la creazione di display uditivi virtuali personali. A titolo d’esempio, vengono di seguito riportate alcune aree applicative che possono trarre beneficio da queste considerazioni: riproduzione multi canale in cuffia, rendering spaziale del suono in dispositivi mobile, motori di rendering per computer-game e standard audio binaurali individuali per film e produzione musicale. Questa tesi presenta una famiglia di approcci in grado di superare gli attuali limiti dei sistemi di audio 3D in cuffia, con l’obiettivo di realizzare display uditivi personali attraverso modelli strutturali per l’audio binaurale volti ad una riproduzione immersiva del suono. I modelli che ne derivano permettono adattamento e personalizzazione di contenuti, grazie alla gestione dei parametri relativi all’antropometria dell'utente oltre a quelli relativi alle sorgenti sonore nell'ambiente . Le direzioni di ricerca intraprese convergono verso una metodologia per la progettazione e personalizzazione di HRTF sintetiche che unisce il paradigma di modellazione strutturale con altre tecniche di selezione per HRTF (ispirate a procedure di selezione non-individuali di HRTF) e rappresenta il principale contributo di questa tesi: l’ approccio a modellazione strutturale mista( MSM ) che considera la HRTF globale come una combinazione di elementi strutturali, che possono essere scelti tra componenti sia sintetiche che registrate. In entrambi i casi, la personalizzazione si basa su dati antropometrici individuali, utilizzati per adattare sia i parametri del modello sia per selezionare un componente simulato o misurato, tra un insieme di risposte all’impulso disponibili. La definizione e la validazione sperimentale dell'approccio a MSM affronta alcune questioni cruciali riguarda l'acquisizione e il rendering di scene acustiche binaurali, definendo alcune linee guida di progettazione per ambienti virtuali personali che utilizzano l’audio 3D e che possiedono nuove forme di comunicazione su misura e di interazione con contenuti sonori e musicali. In questa tesi viene anche presentato un sistema interattivo multimodale utilizzato per condurre test soggettivi sull’integrazione multisensoriale in ambienti virtuali. Vengono proposti quattro scenari sperimentali al fine di testare le funzionalità di un feedback sonoro integrato a modalità tattili o visive. (i) Un feedback con audio 3D legato ai movimenti dell'utente durante una semplice attività di inseguimento di un bersaglio viene presentato come un esempio applicativo di sistema riabilitativo audiovisivo. (ii) La percezione della direzione sonora dei passi interattivamente generati in cuffia durante la camminata evidenzia come l'informazione spaziale sia in grado di mettere in luce la congruenza semantica tra movimento e feedback multimodale. (iii) Un sistema audio tattile interattivo e real-time sintetizza l'informazione spaziale di mappe virtuali per l’educazione all’orientamento e alla mobilità (O&M) rivolta a persone non vedenti. (iv) Un ultimo esperimento analizza la stima tattile delle dimensioni di un oggetto virtuale 3D (un gradino), mentre l'esplorazione è accompagnata da un feedback sonoro generato in tempo reale i cui parametri variano in funzione dell’altezza del punto di interazione aptico. I dati raccolti da questi esperimenti suggeriscono che feedback multimodali che sfruttano correttamente modelli di audio 3D, possono essere utilizzati per migliorare la navigazione nella realtà virtuale, l’orientamento e l’apprendimento di azioni motorie complesse, grazie all'alto livello di attenzione, impegno e immersività fornito all'utente. La metodologia di ricerca, basata sull'approccio a MSM, rappresenta un importante strumento di valutazione per determinare progressivamente i principali attributi spaziali del suono in relazione a ciascun dominio applicativo. In questa prospettiva, tali studi rappresentano una novità nella letteratura scientifica corrente che ha come principale argomento di indagine la realtà virtuale e aumentata, soprattutto per quanto riguarda l'uso di tecniche di sonicazione legate alla cognizione spaziale e alla rappresentazione multisensoriale interna del corpo . Questa tesi è organizzata come segue. Un’introduzione e una panoramica sulla percezione spaziale del suono e sulle tecnologie binaurali in cuffia sono fornite nel Capitolo 1. Il Capitolo 2 è dedicato al formalismo sulla modellazione strutturale mista e sua corrispondente filosofia di ricerca. Nel Capitolo 3 vengono presentati i modelli strutturali relativi ad ogni parte del corpo, risultanti da precedenti ricerche. Due nuove proposte di modello di testa e orecchio approfondiscono rispettivamente la dipendenza dalla distanza nel near-field e le informazioni spettrali fornite dall’orecchio esterno per la localizzazione verticale del suono. Il Capitolo 4 si occupa di un caso di studio completo riguardante l'approccio a modellazione strutturale mista, fornendo degli approfondimenti riguardanti i principali aspetti innovativi di tale modus operandi. Il Capitolo 5 fornisce una panoramica di strumenti sviluppati per l'analisi e la sintesi di HRTF. Inoltre linee guida per il design di ambienti di realtà virtuale vengono discussi in termini di problematiche riguardanti vincoli di tempo reali, requisiti per la mobilità e personalizzazione del segnale audio. Nel Capitolo 6, attraverso due casi di studio viene approfondita l'importanza dell'attributo spaziale del suono nel comportamento dell’ascoltatore e come la continua interazione in ambienti virtuali possa utilizzare con successo algoritmi per l’audio spaziale. Il Capitolo 7 descrive una serie di esperimenti volti a valutare il contributo dell’audio binaurale in cuffia in processi di apprendimento di mappe cognitive spaziali e nell'esplorazione di oggetti virtuali. Infine, il Capitolo 8 apre a nuovi orizzonti per futuri lavori di ricerca.

Bilaterally independent (mismatched) hearing aids cannot replicate the natural timing and level cues between ears, and hence, may result in negative consequences for localization and speech perception in spatially-separated noise performance. Five gain reduction patterns were used to evaluate the impact of bilaterally mismatched gain reduction schemes on localization and speech perception performance in noise, compared to an unaltered bilaterally linear time-invariant amplification scheme (reference scheme), in which audibility was optimized. The bilaterally mismatched gain reduction schemes were later matched (synchronized) between ears to explore the possibility of restoring the deteriorated performance due to the mismatched schemes. Sound quality and listening-effort ratings among different gain reduction patterns were assessed, as well as the relationship between self-reported localization ability in daily life and measured localization performance in a laboratory setting. Twenty-four bilateral hearing aid users were enrolled in this study and tested in a virtual environment with insert earphones. The results indicated that bilaterally mismatched gain reduction schemes had a negative impact on localization, compared to the reference scheme; whereas matching gain reduction schemes between ears improved the deteriorated localization performance. In contrast, the use of bilaterally mismatched gain reductions did not negatively impact the speech perception performance in noise. Matching the gain reduction scheme between ears actually resulted in reduced speech perception performance, compared to the mismatched gain reductions. Self-reported localization abilities were not found to be strongly related to the measured localization performance in this study. Finally, these five different gain reduction patterns did not result in significantly different overall sound quality ratings and listening-effort ratings for hearing aid users. However, the use of gain reductions (mismatched or matched) reduced the perceived noise intrusiveness, compared to the use of reference schemes. It is unclear why there was a discrepancy between the results of the localization and speech perception performance in the present study. It is likely that hearing-impaired listeners do not use binaural cues in the localization task in the same manner as in the speech perception task.

Les prothèses auditives ont pour but de faire recouvrir les principales capacités auditives, au premier rang de laquelle : l’intelligibilité de la parole. Cela est assuré principalement par deux tâches : compenser la perte auditive et réduire le niveau de bruit. La réduction de bruit et la compensation de perte auditive sont effectuées l’une à la suite de l’autre. Or, toutes deux ont des objectifs antagonistes et introduisent des artefacts néfastes à l’appréhension d’une scène sonore complexe dans sa globalité. Dans un premier temps, nous unifions le formalisme sous-jacent aux algorithmes de débruitage et de compensation de perte de sorte à développer une solution explicite au problème dans son ensemble, pour une scène sonore composée d’une source de parole et d’un bruit ambiant. Dans un second temps, nous nous employons à mieux préserver les indices de localisation interauraux pour toutes les directions de l’espace. Pour cela, nous développons trois méthodes basées sur des approximations du terme de coût associé à la préservation de la fonction de transfert interaurale. Enfin, nous élargissons notre modèle de scène sonore à plusieurs sources de parole et du bruit ambiant. Le contexte des prothèses auditives rend ce cas difficile à traiter du fait du nombre réduit de microphones. Nous proposons d’exploiter la propriété de parcimonie de la parole dans le domaine temps-fréquence pour dépasser cet obstacle
Hearing aids are designed to restore the essential abilities of hearing, the most important of which is speech intelligibility. This is achieved mainly through two functions: compensating for hearing loss and reducing the noise level. Noise reduction and hearing loss compensation are performed one after the other. However, both have antagonistic objectives and introduce artifacts that are detrimental to the apprehension of a complex auditory scene in its entirety. In a first step, we unify the formalism underlying the denoising and loss compensation algorithms in order to develop an explicit solution to the problem as a whole, for an auditory scene composed of one speech source and an ambient noise. In a second step, we focus on a better preservation of the interaural localization cues for all spatial directions. For this purpose, we develop three methods based on approximations of the cost function related to the interaural transfer function preservation. Finally, we extend the auditory scene model to several speech sources and ambient noise. The context of hearing aids makes this case difficult to handle due to the small number of microphones. We propose to exploit the sparsity property of speech in the time-frequency domain to overcome this obstacle

abstract: This study consisted of several related projects on dynamic spatial hearing by both human and robot listeners. The first experiment investigated the maximum number of sound sources that human listeners could localize at the same time. Speech stimuli were presented simultaneously from different loudspeakers at multiple time intervals. The maximum of perceived sound sources was close to four. The second experiment asked whether the amplitude modulation of multiple static sound sources could lead to the perception of auditory motion. On the horizontal and vertical planes, four independent noise sound sources with 60° spacing were amplitude modulated with consecutively larger phase delay. At lower modulation rates, motion could be perceived by human listeners in both cases. The third experiment asked whether several sources at static positions could serve as "acoustic landmarks" to improve the localization of other sources. Four continuous speech sound sources were placed on the horizontal plane with 90° spacing and served as the landmarks. The task was to localize a noise that was played for only three seconds when the listener was passively rotated in a chair in the middle of the loudspeaker array. The human listeners were better able to localize the sound sources with landmarks than without. The other experiments were with the aid of an acoustic manikin in an attempt to fuse binaural recording and motion data to localize sounds sources. A dummy head with recording devices was mounted on top of a rotating chair and motion data was collected. The fourth experiment showed that an Extended Kalman Filter could be used to localize sound sources in a recursive manner. The fifth experiment demonstrated the use of a fitting method for separating multiple sounds sources.
Dissertation/Thesis
Doctoral Dissertation Speech and Hearing Science 2015

Despite the importance of auditory selective attention in everyday communication, the cortical mechanisms underlying the amazing ability of human brain to focus on a sound stimulus and suppress others are not well understood. Recent studies have led to the hypothesis that alpha band oscillation (8-14Hz) is a neural signature of multimodal spatial attention. Research in multiple sensory domains has shown that alpha synchronizes in the hemisphere contralateral to unattended stimuli and desynchronizes on the hemisphere contralateral to attended stimuli, suggesting it is a marker of an inhibition process for filtering out unattended stimuli. However, further research is needed to understand the possible functional role of these alpha oscillations as well as their correlation with other cortical activity. Moreover, it is not clear whether different forms of auditory attention employ different cortical mechanisms, mediated through different brain networks. This study aims to combine brain stimulation methods (transcranial Direct/Alternative Current Stimulation) with electrophysiological measurements of electroencephalography (EEG) to measure and interpret the underlying cortical activity during different forms of auditory selective attention. More specifically, there are four studies, each of which employs behavioral tasks to test specific hypotheses. First, we studied alpha oscillatory activity during auditory spatial attention. Second, we compared and contrast cortical activity during auditory spatial and non-spatial attention. Third, we used brain stimulation to see if we can show a causal relationship between alpha oscillation and selective auditory attention performance. Lastly, we applied the existing results on alpha power to use it as a quantitative biomarker to indicate the level of spatial attention network engagement. Our results contributed to the growing body of knowledge about how the brain employs auditory selective attention for effective communication.
2021-06-04T00:00:00Z

碩士
銘傳大學
商品設計學系碩士班
100
Philosopher Lorenzi Okun indicated that: "the human eye will be brought into the world, the ears will bring the world of mankind. " Obviously the hearing is an important way of human to receive external information, but also affect ability to communicate. Hearing impairment is bound to affect the lives learning and communication efficiency, but will also lead to a decline in the quality of life and the interpersonal relationship. Hearing impaired will make inconvenience of life and safety concerns, therefore, how to design and enhance the hearing impaired to improve usability to enhance their welfare should be an important issue. The objective of this study is to investigate the effect of orientation of the sound source, distance, frequency and source of clues to the impaired spatial perception. The results of experiment one indicated that the frequency of a sound, the distance of the sound source, the ears of the advantages of the sound source position and the subjects are significant to determine the sound source position tolerance,also it has two-way interaction of the sound frequency × sound source and sound frequency × sound distance. Experiment two indicated significant of stimulate sound time and stimulate the subjects was affect to impact of sound frequencies before the auditory threshold after hearing threshold tolerance, In addition, the experiment found that the two way interaction of stimulate the auditory type x audio rendering time. The results can guidance the design of the hearing impaired products to enhance the life of the well-being of the hearing-impaired. Example: the design of communications equipment, medical audiometry, calibration, broadcaster, alarms, doorbells, GPS navigation, mobile phones and parking sensor design.

碩士
國立交通大學
電信工程研究所
106
A binaural noise reduction algorithm for hearing aids is proposed in this thesis. The proposed algorithm suppresses the directional interferences and diffuse noise using the interaural time difference (ITD), interaural level difference (ILD) and interaural coherence cues via a time frequency (T-F) masking process. The averaged simulation results in the anechoic chamber show that the proposed algorithm can produce 27 dB SDR improvement, which is 2.5 dB lower than the bilateral IBM method and 17 dB higher than the MVDR beamforming method, when the interferences are located from the center more than 30 degrees. Simulations in low reverberant environments (the RT60 is approximately 0.21 sec) show the proposed algorithm can have improvement of 20 dB in SDR, which is 14 dB higher than the MVDR beamforming method. In moderate reverberant environments (the RT60 is approximately 0.63 sec), the proposed algorithm can produce improvement of 6 dB in SDR, which is 6 dB higher than the MVDR beamforming method. Furthermore, the proposed algorithm is evaluated under various specifications of filter banks for investigating dominant factors to boost the de-noise performance. Experiment results suggest that a higher resolution in low frequencies (approximately less than 1000 Hz) and a higher overlapping ratio of the filters are beneficial to de-noise performance in terms of the SDR measure.

In this thesis I attempt to explain mechanisms of neuronal coding in the auditory system as a form of adaptation to statistics of natural stereo sounds. To this end I analyse recordings of real-world auditory environments and construct novel statistical models of these data. I further compare regularities present in natural stimuli with known, experimentally observed neuronal mechanisms of spatial hearing. In a more general perspective, I use binaural auditory system as a starting point to consider the notion of function implemented by sensory neurons. In particular I argue for two, closely-related tenets: 1. The function of sensory neurons can not be fully elucidated without understanding statistics of natural stimuli they process. 2. Function of sensory representations is determined by redundancies present in the natural sensory environment. I present the evidence in support of the first tenet by describing and analysing marginal statistics of natural binaural sound. I compare observed, empirical distributions with knowledge from reductionist experiments. Such comparison allows to argue that the complexity of the spatial hearing task in the natural environment is much higher than analytic, physics-based predictions. I discuss the possibility that early brain stem circuits such as LSO and MSO do not \"compute sound localization\" as is often being claimed in the experimental literature. I propose that instead they perform a signal transformation, which constitutes the first step of a complex inference process. To support the second tenet I develop a hierarchical statistical model, which learns a joint sparse representation of amplitude and phase information from natural stereo sounds. I demonstrate that learned higher order features reproduce properties of auditory cortical neurons, when probed with spatial sounds. Reproduced aspects were hypothesized to be a manifestation of a fine-tuned computation specific to the sound-localization task. Here it is demonstrated that they rather reflect redundancies present in the natural stimulus. Taken together, results presented in this thesis suggest that efficient coding is a strategy useful for discovering structures (redundancies) in the input data. Their meaning has to be determined by the organism via environmental feedback.

abstract: Two groups of cochlear implant (CI) listeners were tested for sound source localization and for speech recognition in complex listening environments. One group (n=11) wore bilateral CIs and, potentially, had access to interaural level difference (ILD) cues, but not interaural timing difference (ITD) cues. The second group (n=12) wore a single CI and had low-frequency, acoustic hearing in both the ear contralateral to the CI and in the implanted ear. These `hearing preservation' listeners, potentially, had access to ITD cues but not to ILD cues. At issue in this dissertation was the value of the two types of information about sound sources, ITDs and ILDs, for localization and for speech perception when speech and noise sources were separated in space. For Experiment 1, normal hearing (NH) listeners and the two groups of CI listeners were tested for sound source localization using a 13 loudspeaker array. For the NH listeners, the mean RMS error for localization was 7 degrees, for the bilateral CI listeners, 20 degrees, and for the hearing preservation listeners, 23 degrees. The scores for the two CI groups did not differ significantly. Thus, both CI groups showed equivalent, but poorer than normal, localization. This outcome using the filtered noise bands for the normal hearing listeners, suggests ILD and ITD cues can support equivalent levels of localization. For Experiment 2, the two groups of CI listeners were tested for speech recognition in noise when the noise sources and targets were spatially separated in a simulated `restaurant' environment and in two versions of a `cocktail party' environment. At issue was whether either CI group would show benefits from binaural hearing, i.e., better performance when the noise and targets were separated in space. Neither of the CI groups showed spatial release from masking. However, both groups showed a significant binaural advantage (a combination of squelch and summation), which also maintained separation of the target and noise, indicating the presence of some binaural processing or `unmasking' of speech in noise. Finally, localization ability in Experiment 1 was not correlated with binaural advantage in Experiment 2.
Dissertation/Thesis
Ph.D. Speech and Hearing Science 2013

Listening to speech with competing speech in the background is challenging and becomes harder with age. Three experiments examined the auditory and cognitive aspects of auditory spatial attention in conditions in which the location of the target was uncertain. In all experiments, word identification was measured for target sentences presented with two competitor sentences. On each trial, the three sentences were presented with one from each of three spatially separated loudspeakers. A priori cues specified the location and identity callsign of the target. In Experiments I and II, sentences were also presented in conditions of simulated spatial separation achieved with the precedence effect. Participants were younger and older adults with normal hearing sensitivity below 4 kHz. For both age groups, the contributions of richer acoustic cues (those present when there was real spatial separation, but absent when there was simulated spatial separation) were most pronounced when the target occurred at “unlikely” spatial listening locations, suggesting that both age groups benefit similarly from richer acoustical cues. In Experiment II, the effect of time between the callsign cue and target word on word identification was investigated. Four timing conditions were tested: the original sentences (which contained about 300 ms of filler speech between the callsign cue and the onset of the target words), or modified sentences with silent pauses of 0, 150, or 300 ms replacing the filler speech. For targets presented from unlikely locations, word identification was better for all listeners when there was more time between the callsign cue and key words, suggesting that time is needed to switch spatial attention. In Experiment III, the effects of single and multiple switches of attention were investigated. The key finding was that, whereas both age groups performed similarly in conditions requiring a single switch of attention, the performance of older, but not younger listeners, was reduced when multiple switches of spatial attention were required. This finding suggests that difficulties disengaging attention may contribute to the listening difficulties of older adults. In conclusion, cognitive and auditory factors contributing to auditory spatial attention appear to operate similarly for all listeners in relatively simple situations, and age-related differences are observed in more complex situations.

A deficiência auditiva afecta milhões de pessoas em todo o mundo, originando vários problemas, nomeadamente a nível psicossocial, que comprometem a qualidade de vida do indivíduo. A deficiência auditiva influencia o comportamento, particularmente ao dificultar a comunicação. Com o avanço tecnológico, os produtos de apoio, em particular os aparelhos auditivos e o implante coclear, melhoram essa qualidade de vida, através da melhoria da comunicação. Com as escalas de avaliação determinamos o modo como a deficiência auditiva influencia a vida diária, com ou sem amplificação, e de que forma afecta o desempenho psicossocial, emocional ou profissional do indivíduo, sendo esta informação importante para determinar a necessidade e o sucesso de amplificação, independentemente do tipo e grau da deficiência auditiva. O objectivo do presente estudo foi a tradução e adaptação para a cultura portuguesa da escala The Speech, Spatial and Qualities of Hearing Scale (SSQ), desenvolvida por Stuart Gatehouse e William Noble em 2004. Este trabalho foi realizado nos centros auditivos da Widex Portugal. Após os procedimentos de tradução e retroversão, a versão portuguesa foi testada em 12 indivíduos, com idades compreendidas entre os 36 anos e os 80 anos, dos quais 6 utilizavam prótese auditiva há mais de um ano, um utilizava prótese há menos de um ano e 5 nunca tinham utilizado. Com a tradução e adaptação cultural para o Português Europeu do “Questionário sobre as Qualidades Espaciais do Discurso – SSQ”, contribuímos para uma melhor avaliação dos indivíduos que estejam, ou venham a estar, a cumprir programas de reabilitação auditiva.

The Short-Time Target Cancellation (STTC) algorithm, developed as part of this dissertation research, is a “Cocktail Party Problem” processor that can boost speech intelligibility for a target talker from a specified “look” direction, while suppressing the intelligibility of competing talkers. The algorithm holds promise for both automatic speech recognition and assistive listening device applications. The STTC algorithm operates on a frame-by-frame basis, leverages the computational efficiency of the Fast Fourier Transform (FFT), and is designed to run in real time. Notably, performance in objective measures of speech intelligibility and sound source segregation is comparable to that of the Ideal Binary Mask (IBM) and Ideal Ratio Mask (IRM). Because the STTC algorithm computes a time-frequency mask that can be applied independently to both the left and right signals, binaural cues for spatial hearing, including Interaural Time Differences (ITDs), Interaural Level Differences (ILDs) and spectral cues, can be preserved in potential hearing aid applications. A minimalist design for a proposed STTC Assistive Listening Device (ALD), consisting of six microphones embedded in the frame of a pair of eyeglasses, is presented and evaluated using virtual room acoustics and both objective and behavioral measures. The results suggest that the proposed STTC ALD can provide a significant speech intelligibility benefit in complex auditory scenes comprised of multiple spatially separated talkers.
2020-10-22T00:00:00Z

Normal-hearing listeners can locate sound sources, using binaural cues for azimuth angle. These binaural differences in the timing and intensity of sound arriving at the two ears, interaural time differences (ITDs) and interaural intensity differences (IIDs), also support selective listening in multi-talker environments. Auditory-brainstem neurons of the medial superior olive (MSO) and lateral superior olive (LSO) encode ITD in the envelope of sound (ITDENV) and in the temporal fine structure of low-frequency sound (ITDTFS); LSO neurons encode IID. Bilateral-cochlear-implant (bCI) listeners generally receive only IID and ITDENV. Experimental bCI pulse-bursts overcome adaptation, and convey electrical ITDTFS. Improving the understanding of mechanisms for ITD sensitivity can help bCI developers convey acoustic ITDTFS. In this dissertation, models for auditory-brainstem neurons are developed that explain human ability to detect small differences in ITD, as neuronal and MSO population mechanisms. Promoting binaural-coincidence detection and limiting backpropagation, model MSO ion-channels set resting potentials that reproduce dendritic and somatic KLT activation, somatic Na+ inactivation, and a lower amount of axonal Na+ inactivation. Sensitivity to ITDTFS in moderately fast and very fast model MSO neurons collectively match physiological data from 150 to 2000 Hz. The best-ITD (the ITD of highest spike rate) can be made contralateral-leading, by contralateral inhibition of moderate speed, or by asymmetric axon location, leveraging dendritic filtering. Leveraging standard binaural-display models, neuronal populations based on these model MSO neurons match normal-hearing human discrimination thresholds for ITDTFS in sine tones from 39 to 1500 Hz. Adaptation before binaural interaction helps model MSO neurons glimpse the ITDTFS of sound direct from a source, before reflected sound arrives from different directions. With inputs from adapting model spherical bushy cells, a moderately fast model MSO neuron reproduces in vivo responses to amplitude-modulated binaural beats, with a frequency-dependent emphasis of rising vs. peak sound-pressure for ITDTFS encoding, which reflects human ITD detection and reverberation times in outdoor environments. Distinct populations of model LSO neurons, spanning the range of electrical membrane impedance as a function of frequency in LSO neurons, collectively reflect discrimination thresholds for ITDENV in transposed tones across carrier frequency (4-10 kHz) and modulation rate (32-800 Hz).
2022-09-28T00:00:00Z

Dans le cas de perte auditive, la localisation spatiale est amoindrie et vient entraver la compréhension de la parole et ce, malgré le port de prothèses auditives. La présente étude modifie la forme de l’oreille externe d’individus à l’aide de silicone afin d’induire des changements aux indices spectraux (HRTFs), similaires à ceux causés par des prothèses auditives, et d’explorer les mécanismes perceptifs (visuel, spectral, ou tactile) permettant d’alterner d’un nouvel ensemble à l’ensemble originel de HRTFs une fois les prothèses enlevées. Les résultats démontrent que les participants s’adaptent aux nouveaux HRTFs à l’intérieur de quatre séances d’entraînement. Dès le retrait des prothèses, les participants reviennent à leur performance originale. Il n’est pas possible de conclure avec les données présentes si le changement d’un ensemble de HRTFs à un autre est influencé par un des mécanismes de rétroaction perceptuelle étudié. L’adaptation aux prothèses perdure jusqu’à quatre semaines après leur retrait.
Spatial hearing contributes greatly to speech understanding in noisy environments. Hearing aids disturb all of the acoustic cues necessary for accurate sound localization and thus negate some of their benefits. This study addressed behavioral adaptation to changes in auditory spatial cues caused by changes similar to those induced by hearing aids. Spectral cues (HRTFs) were distorted by changing the shape of the outer ear with silicon molds. The present experiment was aimed at determining the perceptual modalities (visual, spectral or tactile) that might enable the switch from the modified to the original HRTFs once the molds were removed. Results indicate that participants were able to adapt within four training sessions. Participants immediately showed accurate sound localization when ear molds were removed. It was not possible to conclude whether the perceptual feedbacks had a major impact on the choice of the correct set of HRTFs to use. Adaptation to the modified HRTFs lasted weeks after their removal.

Об’єктом дослідження є стан просторового орієнтування в учнів зі зниженим слухом. Предмет дослідження – процес розвитку просторового орієнтування в учнів початкових класів зі зниженим слухом засобами адаптивного фізичного виховання. Мета дослідження – теоретично обґрунтувати й експериментально перевірити ефективність використання засобів адаптивного фізичного виховання для розвитку просторового орієнтування в учнів початкових класів зі зниженим слухом в умовах спеціального закладу освіти. Магістерська робота складається зі вступу, трьох розділів, висновків та списку використаних джерел. У вступі обґрунтована актуальність дослідження, визначена мета та завдання дослідження. У першому розділі висвітлено теоретичні аспекти використання засобів адаптивного фізичного виховання для дітей зі зниженим слухом, дано визначення основних понять, розкрито сучасні підходи до розвитку рухової сфери дітей зі зниженим слухом. У другому розділі висвітлено організацію процесу розвитку просторового орієнтування у дітей молодшого шкільного віку зі зниженим слухом, визначено стан розвитку просторового орієнтування дітей означеної нозології з метою подальшого використання отриманих результатів в методиці проведення занять з використанням засобів адаптивного фізичного виховання. У третьому розділі обґрунтовано експериментальне дослідження ефективності використання засобів адаптивного фізичного виховання для розвитку просторового орієнтування в учнів зі зниженим слухом, висвітлено розроблений комплекс засобів адаптивного фізичного виховання для розвитку просторового орієнтування корекційно-оздоровчої спрямованості та проведено аналіз результативності формувальної частини експерименту. У висновках узагальнено основні результати магістерського дослідження.
The object of research is the state of spatial orientation in students with a reduced hearing. Subject of research – the process of development of spatial orientation in students of elementary classes with reduced hearing by means of adaptive physical education. The purpose of the study is theoretically to substantiate and experimentally check the efficiency of using means of adaptive physical education for the development of spatial orientation in students of elementary schools with reduced earning in a special educational institution. Master’s work consists of entry, three sections, conclusions and list of sources used. In the introduction, the relevance of the study is substantiated, the purpose and objectives of the study are determined. In the first section, the theoretical aspects of the use of adaptive physical education for children with reduced hearing are highlighted, the main concepts are defined, modern approaches to the development of the motor sphere of children with reduced hearing are revealed. In the second section, the organization of the development of spatial orientation in children of junior school age with reduced hearing is determined, the state of development of spatial orientation of children is indicated by the nosology in order to further use the results in the method of adaptive physical education. In the third section, an experimental study of the efficiency of the use of adaptive physical education for the development of spatial orientation in students with reduced hearing, a complex of means of adaptive physical education for the development of the spatial orientation of correctional and recreational orientation is based and analyzes the effectiveness of the molding part of the experiment. The conclusions are summarized the main results of the master’s study.

H διδακτορική διατριβή μελετά ζητήματα που αφορούν την ενσωμάτωση υπολογιστικών μοντέλων ακοής για την μοντελοποίηση και επεξεργασία ηχητικών σηματών για την βέλτιστη αναπαραγωγή τους σε χώρους με αντήχηση καθώς και την κωδικοποίηση ηχητικών δεδομένων. Το κύριο μέρος της διατριβής επικεντρώθηκε στην μοντελοποίηση των αντιληπτικά σημαντικών αλλοιώσεων λόγω αντήχησης, με την βοήθεια κατάλληλα οριζόμενων μόνο-ωτικών και διαφορικών ενδο-καναλικών παραμέτρων και την απεικόνιση τους με τη βοήθεια χρονο-συχνοτικών 2Δ αναπαραστάσεων. Ο λεπτομερής εντοπισμός των αλλοιώσεων στα ηχητικά σήματα μέσω του προτεινόμενου Δείκτη Επικάλυψης λόγω Αντήχησης (ΔΕΑ) διαμόρφωσε κατάλληλη μεθοδολογία ανάλυσης-σύνθεσης, για την καταστολή της αντήχησης σε συγκεκριμένες χρονο-συχνοτικές περιοχές. Το κύριο πλεονέκτημα της προτεινόμενης, εξαρτώμενης του σήματος, μεθοδολογίας είναι ότι επιτυγχάνεται η καταστολή των, με σχετική καθυστέρηση, παραμορφώσεων λόγω αντήχησης σε μια μεγαλύτερη κλίμακα, δεδομένου ότι μόνο οι αντιληπτικά σημαντικές περιοχές του σήματος επηρεάζονται από την επεξεργασία. Επιπλέον, αναζητήθηκε η δυνατότητα ανάλυσης των ηχητικών δεδομένων με βάση τις εσωτερικές τους αναπαραστάσεις (όπως δηλαδή τις παρέχει το υπολογιστικό μοντέλο ακοής) με εφαρμογή στην περιοχή της κωδικοποίησης σημάτων. Ο προτεινόμενος μη-ομοιόμορφος κβαντιστής πραγματοποιεί τη διαδικασία της κβάντισης χρονο-συχνοτικά με κατάλληλη οδήγηση από το υπολογιστικό μοντέλο ακοής, εξασφαλίζοντας καλύτερη υποκειμενική ηχητική ποιότητα, σε σχέση με ένα ομοιόμορφο PCM κβαντιστή. Χρησιμοποιώντας τη βασική λειτουργία του μη-ομοιόμορφου κβαντιστή, υλοποιήθηκε ενά κριτήριο αξιολόγησης ηχητικών δεδομένων, όπου σε αντίθεση με καθιερώμενα κριτήρια (όπως το Noise to Mask Ration, NMR) επιτελεί τις λειτουργίες του στο πεδίο χρόνου-συχνότητας και παρέχει τη δυνατότητα εντοπισμού της υποκειμενικά σημαντικής παραμόρφωσης με βάση την χρονική εξέλιξη του σήματος.
The dissertation studies issues concerning the integration of computational auditory models for modeling and processing of audio signals for optimal reproduction in reverberant spaces as well as topics related to audio coding. Based on the theoretical framework analysis that was established, the necessity of a signal-dependent approach was underlined for modeling the perceptually-relevant effects of reverberation. The main part of the dissertation thesis was focused on describing the perceptually-relevant alterations due to reverberation, based on appropriate defined monaural and differential inter-channel parameters and also their representation with well-defined time-frequency 2D maps. The detailed localization of alterations due to reverberation in the acoustic signals via the proposed Reverberation Masking Index (RMI) introduced an analysis-synthesis methodology for the compensation of reverberation in perceptually-significant time-frequency regions incorporating also, well-established digital signal processing techniques. The main advantage of the proposed signal-dependent methodology is that the suppression of reverberant tails can be achieved on a larger scale under practical conditions, since only perceptually significant regions of the signal are affected after processing. Additionally, the proposed framework complements the more traditional system-dependent inverse filtering methods, enabling novel and efficient signal processing schemes to evolve for room dereverberation applications. The thesis examines also the feasibility of the acoustic signal analysis based on the internal representations provided by the computational auditory model, applicable in the area of audio coding. The proposed non-uniform quantizer operates in the time-frequency domain, where a novel quantization process is driven by the computational auditory model, thus enabling an overall better perceptual quality with respect to uniform PCM quantizer. Considering the fundamental operation of the novel non-uniform quantizer, a criterion for audio quality evaluation was proposed, where contrary to well-established criteria (i.e., Noise to Mask Ratio, NMR) its potential structure performs in the time-frequency domain and provides the detailed localization of perceptually-important distortions based on the input signal’s evolution.