Tesis sobre el tema "Audio immersif"

L’audiodescripció (AD) ja no és considerada un servei exclusiu d’accessibilitat, dedicat a una petita fracció d’individus, sinó que s’entén com un servei inclusiu, dirigit no només a les persones amb discapacitats sensorials, sinó també a altres audiències, com els ancians o els nens. És probable que la demanda d’aquest servei creixi en un futur pròxim. Encara que en el mercat s’observa un interès en augmentar cada cop més la oferta de productes audiovisuals accessibles, no s’hauria de deixar de banda la qualitat de l’experiència de l’usuari. Emmarcada dins el model inclusiu d’accessibilitat, aquesta tesi doctoral té com a objectiu explorar l’impacte de l’AD en la resposta emocional del seu públic objectiu – usuaris cecs i amb baixa visió (CiBV). Per a això, es van realitzar dos estudis de recerca, un centrat en l’estil de AD i l’altre en el el tipus de veu d’AD. La justificació darrere de la selecció d’aquests paràmetres es va relacionar amb els resultats de la investigació anterior. Es van establir dos objectius principals d’investigació. El primer, en relació amb l’estil d’AD, es va centrar en l’estudi de la recepció de dos estils d’AD – estàndard i creatiu – per als usuaris CiBV. El segon, vinculat al tipus de veu d’AD, va implicar l’estudi de la recepció de dos tipus de veu d’AD – humana i sintètica – per a usuaris CiBV per a dos gèneres: ficció i documental. L’aspecte innovador d’aquesta tesi doctoral és l’enfocament metodològic adoptat. La tesi segueix una metodologia centrada en l’usuari, però no es limita a informar sobre les opinions dels usuaris, sinó que pretén mesurar la seva resposta emocional als estímuls presentats. L’eina d’investigació que s’utilitza per mesurar l’experiència de l’usuari és el qüestionari “Independent Television Commission Sense of Presence Inventory”, un dels cinc qüestionaris canònics utilitzats per mesurar la presència. Dos elements addicionals – interès i confusió – s’afegeixen perquè els usuaris expressin la seva resposta emocional als estímuls mostrats. Els resultats indiquen que no només els guions d’AD, creatius o estàndard, sinó també els tipus de veu d’AD tenen un efecte directe en la recepció de la pel·lícula i, per tant, en l’experiència dels usuaris. Quan es tracta de l’estil d’AD, els resultats mostren que l’AD creativa, en comparació amb l’AD estàndard, va produir nivells més alts de presència en tots els participants. En general, l’estil creatiu semblava ser més natural, especialment per als participants amb pèrdua de visió recent. No obstant això, va resultar ser més confús que l’estil estàndard, possiblement perquè els participants no estaven acostumats a la terminologia cinematogràfica inclosa en el guió o no esperaven escoltar vocabulari col·loquial. Tot i això, quan es va preguntar explícitament quin estil d’AD – creatiu o estàndard – prefereien, la majoria dels participants va optar per l’AD creativa. Quan es tracta del tipus de veu d’AD, l’AD amb veu humana, en comparació amb l’AD amb veu sintètica (TTS AD1), va provocar nivells significativament més alts de presència per la ficció. Els nivells de presència per al documental van ser similars, sense diferències estadísticament significatives, en relació amb el tipus de veu d’AD. La ficció amb AD narrada per una veu humana va ser avaluada pels participants com a més interessant i menys confusa en comparació amb la ficció narrada per una veu sintètica. En el cas del documental, els nivells d’interès i confusió eren comparables per a les dues veus. Quan se’ls va preguntar quina veu preferien per a un gènere donat, els usuaris van mostrar preferència per l’AD narrada per una persona per a la ficció i cap preferència entre l’AD amb veu humana i AD amb veu sintètica per al documental. En general, les troballes suggereixen que quan es narra adequadament, tant en termes d’estil com de veu, l’AD pot facilitar que les audiències CyBV tinguin una experiència audiovisual més atractiva. S’espera que aquesta tesi doctoral sigui un primer un pas útil per entendre els vincles entre el producte d’AD i el context de l’usuari, i per a poder dur a terme investigacions més profundes en el futur.
La audiodescripción (AD) ya no está considerada como un servicio exclusivo de accesibilidad, dedicado a una pequeña fracción de individuos, sino como un servicio inclusivo, dirigido no sólo a personas con discapacidades sensoriales, sino también a otras audiencias, como los ancianos o los niños. Es probable que la demanda de este servicio crezca en un futuro cercano. Aunque en el mercado se observa un interés en aumentar cada vez más la oferta de productos audiovisuales accesibles, no se debería dejar de lado la calidad de la experiencia del usuario. Enmarcada dentro del modelo inclusivo de accesibilidad, esta tesis doctoral tiene como objetivo explorar el impacto de la AD en la respuesta emocional de su público objetivo – usuarios ciegos y con baja visión (CyBV). Para ello, se realizaron dos estudios de investigación, uno centrado en el estilo de AD y el otro en la entrega vocal de AD. La justificación detrás de la selección de estos parámetros se relacionó con los resultados de investigación anterior. Se establecieron dos objetivos principales de investigación. La primera, en relación con el estilo de AD, se centró en estudiar la recepción de dos estilos de AD – estándar y creativo – por los usuarios CyBV. La segunda, vinculada a la entrega vocal de AD, implicó el estudio de recepción de dos tipos de voces de AD – humana y sintética – por usuarios CyBV para dos géneros: ficción y documental. El aspecto novedoso de esta tesis doctoral es el enfoque metodológico adoptado. La tesis sigue una metodología centrada en el usuario, pero no se limita a informar sobre las opiniones de los usuarios, sino que pretende medir su respuesta emocional a los estímulos presentados. La herramienta de investigación que se utiliza para medir la experiencia del usuario es el cuestionario de la Independent Television Commission Sense of Presence Inventory, que es uno de los cinco cuestionarios canónicos utilizados para medir la presencia. Dos elementos adicionales – interés y confusión – se añaden para que los usuarios expresen su respuesta emocional a los estímulos presentados. Los resultados indican que no solo los guiones de AD, creativos o estándar, sino también las voces de entrega de AD tienen un efecto directo en la recepción de la película y, por lo tanto, en la experiencia de los usuarios. Cuando se trata del estilo de la AD, los resultados muestran que la AD creativa, en comparación con la AD estándar, produjo mayores niveles de presencia para todos los participantes. En general, el estilo creativo parecía más natural, especialmente a los participantes con pérdida de visión reciente. Sin embargo, resultó ser más confuso que el estilo estándar, posiblemente porque los participantes no estaban acostumbrados a la terminología cinematográfica incluida en el guión o no esperaban escuchar vocabulario coloquial. A pesar de eso, cuando se preguntó explícitamente qué estilo de AD – creativo o estándar – preferían, la mayoría de los participantes optó por la AD creativa. En cuanto al tipo de voz de AD, la AD narrada por una persona, en comparación con la AD narrada por una voz sintética (TTS AD1), provocó niveles significativamente más altos de presencia para la ficción. Los niveles de presencia para el documental fueron similares, sin diferencias estadísticamente significativas en relación con el tipo de voz de la AD. La ficción con AD narrada por una voz humana fue evaluada por los participantes como más interesante y menos confusa en comparación con la AD para ficción narrada mediante síntesis de voz. En el caso del documental, los niveles de interés y confusión eran similares para ambas voces. Cuando se les preguntó qué voz prefierían para un género dado, los usuarios mostraron una preferencia por la AD narrada por una persona para la ficción y ninguna preferencia entre la AD con voz humana y AD con voz sintética para el documental. En general, los resultados sugieren que cuando se narra adecuadamente, tanto en términos de estilo como de voz, la AD puede aumentar las posibilidades de que las audiencias CyBV tengan una experiencia audiovisual más atractiva. Se espera que esta tesis doctoral sea un primer paso para entender los vínculos entre el producto de la AD y el contexto del usuario, y abrir el camino a investigaciones más profundas en el futuro.
Audio description (AD) is no longer seen as an exclusive accessibility service dedicated to a small fraction of individuals, but as an inclusive service, addressing not only the needs of people with sensory impairments, but also those of other audiences, including the elderly or children. The demand for this service is likely to grow in the near future. Although a drive for quantity of accessible audiovisual products can be observed on the market, the issue of the quality of user experience should not be overlooked. Framed within the inclusive model of accessibility, this PhD thesis aims to explore the impact of AD on the emotional response of its target audiences – blind and visually impaired (B/VIP) users. To this end, two research studies were carried out: one focusing on AD style and the other on AD vocal delivery. The rationale behind selecting these parameters was related to the findings of previous research conducted in the field. Two main research objectives were set. The first one, concerning AD style, centred around studying the reception of two AD styles – standard and creative – by B/VIP users. The second one, linked to AD vocal delivery, involved studying the reception of two AD voice types – human and synthetic – by B/VIP users for two genres: fiction and documentary. The novel aspect of this PhD thesis is the methodological approach adopted. It follows a user-centric methodology, not limiting itself to reporting on users’ opinions, but also aiming to measure their emotional response to a stimuli presented. The research tool that is used to gauge user experience is the Independent Television Commission Sense of Presence Inventory questionnaire, one of five canonical questionnaires used for measuring presence. Two additional items – interest and confusion – are added for users to report their emotional response to the stimuli shown. The results indicate that not only AD scripts – creative or standard – but also AD delivery voices have a direct effect on the reception of a film, and therefore on users’ experience. When it comes to AD style, the results show that creative AD, compared to standard AD, yielded higher levels of presence for all participants. Overall, the creative AD style seemed more natural, especially to participants with recent sight loss. However, it turned out to be more confusing than standard AD, possibly because the participants were not used to cinematic terminology included in the script or did not expect to hear unsavoury vocabulary. Nevertheless, when explicitly asked which AD style – creative or standard – they prefer, the majority of participants opted for creative AD. When it comes to AD vocal delivery, AD narrated by a human, compared to text-to-speech (TTS) AD, prompted significantly higher levels of presence for fiction. Presence rates for documentary were similar, with no statistically significant differences in relation to AD voice type. Fiction with a human voice AD was assessed by participants as being more interesting and less confusing than fiction with TTS AD. In the case of documentary, the levels of interest and confusion were comparable for both voice types. When asked which voice type they prefer for a given genre, participants showed a preference for AD read by a human for fiction, but no preference between a human voice AD and TTS AD for documentary. Overall, the findings suggest that when properly delivered, both in terms of style and voice, AD may increase the chances of B/VIP audiences having a more engaging viewing experience. It is hoped that this PhD thesis will act as a useful stepping stone towards understanding the links between the AD product and user context, and towards conducting further research in this field.

L'utilisation de réponses impulsionnelles spatiales de salles (spatial room impulse response, SRIR) dans la reproduction d'effets de réverbération de salle tri-dimensionnels connaît aujourd'hui une réelle démocratisation grâce à la commercialisation répandue d'antennes sphériques de microphones (spherical microphone array, SMA) et à une capacité de calcul numérique en croissance continue. Ces SRIR peuvent reproduire des effets de réverbération spatialisés sur des dispositifs immersifs ("surround-sound") à travers des convolutions multicanal de plus en plus performantes. De cette utilisation découle naturellement une demande pour des techniques d'analyse et de traitement non seulement capables d'assurer une reproduction fidèle, mais qui pourraient éventuellement aussi servir à contrôler différentes modifications de la SRIR de façon plus créative que réaliste. Dans ce contexte, l'objectif principal de cette thèse est de développer un environnement complet d'analyse, de traitement, et de manipulation temps-fréquence-espace de SRIR. Les outils d'analyse doivent mener à une modélisation approfondie permettant ensuite un traitement de la mesure vis-à-vis de ses limitations intrinsèques (conditions de mesure, accumulation de bruit de fond, etc.) ainsi qu'une capacité à faire évoluer certaines caractéristiques de l'effet de réverbération décrit par la SRIR. Ces caractéristiques peuvent être tout à fait objectives, c'est-à-dire explicitement reliées à différents paramètres du modèle, ou alors plutôt informées par une connaissance de la perception humaine de l'acoustique des salles. Les aspects théoriques de ce projet de recherche sont présentés en deux parties principales. Dans un premier temps, le modèle de signal de SRIR sous-jacent est décrit en s'inspirant directement des approches historiques dans les domaines de la réverbération artificielle et le traitement de SMA, tout en y proposant plusieurs extensions. Le modèle de signal est alors exploité afin de définir les méthodes d'analyse qui forment le noyau du cadre de traitement-manipulation final. Ces méthodes se focalisent particulièrement sur (a) l'identification du "temps de mélange" décrivant le moment de transition entre les premières réflexions et la réverbération tardive, (b) la génération d'une cartographie temps-espace des premières réflexions, et (c) l'estimation des paramètres régissant la décroissance exponentielle de l'enveloppe d'énergie de la réverbération tardive, à la fois en fréquence et en direction. La définition d'une procédure de génération de représentations directionnelles de SRIR (directional room impulse response, DRIR) est aussi nécessaire pour pouvoir prendre en compte la dépendance directionnelle de ces propriétés. En seconde partie, les paramètres de modélisation explicités par les méthodes d'analyse sont exploités à des fins soit de traitement (c'est-à-dire tenter de corriger certaines des limitations inhérentes au processus de mesure par SMA), soit de manipulation ou de modification plus créative de la SRIR. Deux méthodes de traitement sont développées en particulier : (1) une procedure d'atténuation de bruits non stationnaires agissant directement sur les signaux de mesure par balayages de fréquence exponentiels (exponential sweep method, ESM) répétés, et (2) une technique de débruitage basée sur une extrapolation et une resynthèse de la queue de réverbération tardive. Les descriptions théoriques ainsi complétées, les principales méthodes d'analyse ainsi que la génération de DRIR et le débruitage sont sujets à une série de tests de validation au cours desquels des SRIR simulées sont utilisées afin d'évaluer la performance, les limitations, et la paramétrisation des différentes techniques. Ces sous-études permettent à chaque méthode d'être vérifiée individuellement, et donnent un aperçu détaillé du fonctionnement interne des outils d'analyse. Enfin, une vue d'ensemble de l'environnement d'analyse-traitement-manipulation est obtenue [...]
The use of spatial room impulse responses (SRIR) for the reproduction of three-dimensional reverberation effects through multi-channel convolution over immersive surround-sound loudspeaker systems has become commonplace within the last few years, thanks in large part to the commercial availability of various spherical microphone arrays (SMA) as well as a constant increase in computing power. This use has in turn created a demand for analysis and treatment techniques not only capable of ensuring the faithful reproduction of the measured reverberation effect, but which could also be used to control various modifications of the SRIR in a more "creative" approach, as is often encountered in the production of immersive musical performances and installations. Within this context, the principal objective of the current thesis is the definition of a complete space-time-frequency framework for the analysis, treatment, and manipulation of SRIRs. The analysis tools should lead to an in-depth model allowing for measurements to first be treated with respect to their inherent limitations (measurement conditions, background noise, etc.), as well as offering the ability to modify different characteristics of the final reverberation effect described by the SRIR. These characteristics can be either completely objective, even physical, or otherwise informed by knowledge of human auditory perception with regard to room acoustics. The theoretical work in this research project is therefore presented in two main parts. First, the underlying SRIR signal model is described, heavily inspired by the historical approaches from the fields of artificial reverberation synthesis and SMA signal processing, while at the same time (incrementally) extending both. The signal model is then used to define the analysis methods that form the core of the final framework; these focus particularly on (a) identifying the "mixing time" that defines the moment of transition between the early reflection and late reverberation regimes, (b) obtaining a space-time cartography of the early reflections, and (c) estimating the frequency- and direction-dependent properties of the late reverberation's exponential energy decay envelope. In order to account for the directional dependence of these properties, a procedure for generating directional SRIR representations (i.e. directional room impulse responses, DRIR) that guarantee the preservation of certain fundamental reverberation properties must also be defined. In the second part, the model parameters made explicit by the analysis methods are exploited in order to either treat (i.e. attempt to correct some of the inevitable limitations inherent to the SMA measurement process) or more creatively manipulate and modify the SRIR. Two treatment methods in particular are developed in this thesis: (1) a pre-analysis procedure acting directly on repeated exponential sweep method (ESM) SMA measurement signals in an attempt to simultaneously increase the resulting SRIR's signal-to-noise ratio (SNR) while reducing its vulnerability to non-stationary noise events, and (2) a post-analysis denoising technique based on replacing the SRIR's background noise floor with a resynthesized extrapolation of the late reverberation tail. The theoretical descriptions thus complete, the main analysis methods as well as the DRIR generation and the denoising treatment procedures are then subjected to a series of validation tests, wherein simulated SRIRs (or parts thereof) are used to evaluate the performance, discuss the limitations, and parameterize the implementation of the different techniques. These sub-studies allow each method to be individually verified, resulting in a comprehensive investigation into the inner workings of the analysis toolbox (as well as the denoising process). Finally, to provide a concluding overview of the complete analysis-treatment-manipulation framework, similar studies are carried out using examples of real-world [...]

The development and improvement of 3-D immersive audio is gaining momentum through the growing interest in virtual reality. Possible applications reach from recreating real world environments to immersive concerts and performances to exploiting big data acoustically. To improve the immersive experience several measures can be taken. The recording of the sound field, the spatialization and the development of the loudspeaker arrays are some of the greatest challenges. In this thesis, these challenges for improving immersive audio will be explored. First, there will be a short introduction about 3D audio and a review about the state of the art technology and research. Next, the thesis will provide an introduction to 3D loudspeaker arrays and describe the systems used during this research. Furthermore, the development of a new 16-element 3rd order sound field microphone will be described. Afterwards, different spatial audio algorithms such as higher order ambisonics, wave field synthesis and vector based amplitude panning will be described, analyzed and compared. For each spatialization algorithm, the quality of soundfield reproduction will be quantified using listener perception tests for clarity and sound source localization.
Master of Science

In immersive audio environments with distributed microphones, Independent Component Analysis (ICA) can be applied to uncover signals from a mixture of other signals and noise, such as in a cocktail party recording. ICA algorithms have been developed for instantaneous source mixtures and convolutional source mixtures. While ICA for instantaneous mixtures works when no delays exist between the signals in each mixture, distributed microphone recordings typically result various delays of the signals over the recorded channels. The convolutive ICA algorithm should account for delays; however, it requires many parameters to be set and often has stability issues. This thesis introduces the Channel Aligned FastICA (CAICA), which requires knowledge of the source distance to each microphone, but does not require knowledge of noise sources. Furthermore, the CAICA is combined with Time Frequency Masking (TFM), yielding even better SOI extraction even in low SNR environments. Simulations were conducted for ranking experiments tested the performance of three algorithms: Weighted Beamforming (WB), CAICA, CAICA with TFM. The Closest Microphone (CM) recording is used as a reference for all three. Statistical analyses on the results demonstrated superior performance for the CAICA with TFM. The algorithms were applied to experimental recordings to support the conclusions of the simulations. These techniques can be deployed in mobile platforms, used in surveillance for capturing human speech and potentially adapted to biomedical fields.

The complex relationship between array gain patterns and microphone distributions limits the application of traditional optimization algorithms on irregular arrays, which show enhanced beamforming performance for human speech capture in immersive environments. This work analyzes the relationship between irregular microphone geometries and spatial filtering performance with statistical methods. Novel geometry descriptors are developed to capture the properties of irregular microphone distributions showing their impact on array performance. General guidelines and optimization methods for regular and irregular array design are proposed in immersive (near-field) environments to obtain superior beamforming ability for speech applications. Optimization times are greatly reduced through the objective function rules using performance-based geometric descriptions of microphone distributions that circumvent direct array gain computations over the space of interest. In addition, probabilistic descriptions of acoustic scenes are introduced to incorporate various levels of prior knowledge for the source distribution. To verify the effectiveness of the proposed optimization methods, simulated gain patterns and real SNR results of the optimized arrays are compared to corresponding traditional regular arrays and arrays obtained from direct exhaustive searching methods. Results show large SNR enhancements for the optimized arrays over arbitrary randomly generated arrays and regular arrays, especially at low microphone densities. The rapid convergence and acceptable processing times observed during the experiments establish the feasibility of proposed optimization methods for array geometry design in immersive environments where rapid deployment is required with limited knowledge of the acoustic scene, such as in mobile platforms and audio surveillance applications.

The Generalized Sidelobe Canceller is an adaptive algorithm for optimally estimating the parameters for beamforming, the signal processing technique of combining data from an array of sensors to improve SNR at a point in space. This work focuses on the algorithm’s application to widely-separated microphone arrays with irregular distributions used for human voice capture. Methods are presented for improving the performance of the algorithm’s blocking matrix, a stage that creates a noise reference for elimination, by proposing a stochastic model for amplitude correction and enhanced use of cross correlation for phase correction and time-difference of arrival estimation via a correlation coefficient threshold. This correlation technique is also applied to a multilateration algorithm for an efficient method of explicit target tracking. In addition, the underlying microphone array geometry is studied with parameters and guidelines for evaluation proposed. Finally, an analysis of the stability of the system is performed with respect to its adaptation parameters.

Al giorno d'oggi si registra un grande interesse verso i sistemi di telecon- ferenza multimediale a seguito della crescente richiesta di comunicazioni effi- cienti e dello sviluppo di tecniche avanzate per il processamento digitale dei segnali. Un sistema di teleconferenza dovrebbe fornire una rappresentazione realistica del campo sonoro e visivo, consentendo una comunicazione natu- rale tra i partecipanti dislocati ovunque nel mondo come fossero nella stessa stanza. In questo contesto, sono stati sviluppati molti sistemi, a partire da applicazioni basate su PC pensate per comunicazioni tra singoli utenti no a sistemi complessi dotati di ampi schermi che riproducono la stanza remota come fosse il proseguimento della stanza locale. Nei sistemi di teleconferenza è possibile ridurre l'eco indesiderata dovuta all'accoppiamento tra l'altoparlante e il microfono usando un cancellatore d'eco acustica (AEC). In presenza di più di un partecipante, per la localizza- zione del parlatore devono essere presi in considerazione sistemi multicanale. Possono essere ottenute prestazioni più realistiche già con sistemi stereofonici, poichè gli ascoltatori hanno a disposizione informazioni spaziali che aiutano ad identi care la posizione del parlatore. Tuttavia, deve essere impiegato un maggior numero di ltri adattativi e la relazione lineare esistente tra i due canali generati dalla stessa sorgente causa problemi aggiuntivi: la soluzione dell'algoritmo adattativo non è unica e dipende dalla posizione del parla- tore nella stanza di trasmissione che non è stazionaria, causando possibili problemi di convergenza. In aggiunta, la scelta dell'algoritmo adattativo di- venta estremamente importante perchè le prestazioni dipendono dal numero di condizionamento del segnale d'ingresso che è molto alto nello scenario multicanale. In questa tesi, vengono presentati contributi innovativi per la cancellazione d'eco acustica stereofonica basati sul fenomeno della \missing- fundamental". L'innovazione delle soluzioni è legata alla grande riduzione della coerenza tra i canali del segnale stereo che si riesce ad ottenere senza al- terare la qualità dell'audio e la percezione stereofonica. Inoltre, viene discussa una soluzione per migliorare la velocità di convergenza dei filtri adattativi basata su un metodo per la variazione del passo d'adattamento: l'approccio è applicato alla cancellazione d'eco acustica stereofonica ma in realtà può essere usato per generici algoritmi adattativi. Contestualmente, si è assistito ad un crescente interesse nel progetto di si- stemi che forniscono una riproduzione dei suoni la più realistica possibile così che l'ascoltatore non si accorge che sono stati prodotti artifi cialmente poichè è immerso nella scena audio virtuale circondato da un elevato numero di altoparlanti. I sistemi convezionali sono progettati per massimizzare la senzazione acustica in una speci ca posizione dell'ambiente d'ascolto, il cosiddetto sweet spot. Inoltre, non è possibile ottenere una corretta localiz- zazione della sorgente con un numero limitato di altoparlanti. Quindi, sono stati condotti diversi studi sull'ottimizzazione di questi sistemi, concentrando l'attenzione su nuove tecniche di registrazione e riproduzione, ovvero la Wave Field Analysis (WFA) e la Wave Field Synthesis (WFS). La prima è una tec- nica di registrazione del campo sonoro basata su array di microfoni e la seconda consente la sintesi del campo sonoro attraverso array di altoparlanti. Per utilizzare queste tecniche in scenari reali (ad esempio, sistemi di telecon- ferenza, cinema, home theatre) è necessario applicare algoritmi multicanale per il processamento digitale dei segnali, già sviluppati per sistemi tradizio- nali. Questo porta all'introduzione della Wave Domain Adaptive Filtering (WDAF), ovvero una generalizzazione spazio-temporale dell'algoritmo adat- tativo Fast Least Mean Squares, consentendo una considerevole riduzione della complessità computazionale. In questa tesi vengono discusse soluzioni efficienti per un'implementazione in tempo reale e possibili approssimazioni di fase delle funzioni guida usate per gestire gli altoparlanti. Inoltre, vengono presentati un approccio per la WDAF basato sulla struttura Weighted-Overlap-Add e una tecnica per il puntamento digitale dei arrays lineari basata sulla WFS: l'obiettivo di questi studi è quello di applicare questi concetti in scenari reali, come nel caso di un sistema di teleconferenza. Infatti, le suddette tecniche per la riproduzione audio immersiva possono essere sfruttate per migliorare le prestazioni di si- stemi di teleconferenza a grandezza naturale, combinando requisiti temporali e spaziali. Inoltre, risultano necessari algoritmi di riproduzione audio per migliorare la qualità audio percepita così da rendere più piacevole l'ambiente d'ascolto tenendo conto di alcune caratteristiche proprie dell'ambiente. Più speci ficata- mente, l'equalizzazione rappresenta uno strumento potente capace di gestire le irregolarità della risposta in frequenza: un equalizzatore può compensare il posizionamento del parlatore e le caratteristiche della stanza d'ascolto e può essere applicato in un sistema di teleconferenza per rendere la comunicazione la più naturale possibile. In questo lavoro vengono discusse la valutazione di un equalizzatore multipunto e una soluzione mixed-phase con un ritardo di gruppo della stanza adeguatamente progettato.
Nowadays, there is a large interest towards multimedia teleconferencing sys- tems as a consequence of the increasing requirement for efficent communica- tions and the development of advanced digital signal processing techniques. A teleconferencing system should provide a realistic representation of visual and sound fields, allowing a natural communication among participants any- where in the world as they were all in the same room. In this context, a lot of systems have been developed ranging from PC-based applications, thought for single users communications, up to complex systems provided with large video screens playing the remote room as it were a continuum of the local room. In teleconferencing systems the undesired echo due to coupling between the loudspeaker and the microphone can be reduced using an acoustic echo can- celer (AEC). In the presence of more than one participant, multichannel systems have to be taken into consideration for speaker localization. More realistic performance can be already obtained through stereophonic systems since listeners have spatial information that helps to identify the speaker position. Anyway, more adaptive lters have to be used and the linear rela- tionship existing between the two channels generated from the same source brings some additional problems: the solution of the adaptive algorithm is not unique and depends on the speaker position in the transmission room which is not stationary, causing possible convergence problems. Moreover, the choice of the adaptive algorithm becomes extremely important because the performance depends on the condition number of the input signal which is very high in the multichannel scenario. In this thesis novel contributions for stereophonic acoustic echo cancellation are given based on the \missing- fundamental" phenomenon. The novelty of the solutions is related to the great interchannel coherence reduction obtained without a ecting speech quality and stereo perception. Moreover, a solution for improving the con- vergence speed of adaptive lters is discussed based on a variable step-size method: the approach is applied to stereophonic acoustic echo cancellation but, actually, it can be used for generic adaptive algorithms. Contextually, there has been an increasing interest in the design of systems providing a reproduction of sounds as realistic as possible so that the lis- tener does not notice that they have been produced arti cially since he is immersed in the virtual audio scene surrounded by a large number of loud- speakers. Conventional systems are designed to obtain the optimal acoustic sensation in a particular position of the listening environment, i.e., the so called sweet spot. Furthermore, it is impossible to achieve a correct source localization with a limited number of loudspeakers. Hence, several research e orts have been made in the optimization of these systems, focusing on new recording and reproduction techniques, i.e., Wave Field Analysis (WFA) and Wave Field Synthesis (WFS). The former is a sound eld recording tech- nique based on microphone arrays and the latter allows sound eld synthesis through loudspeakers arrays. At the aim of using these techniques in real world applications (e.g., teleconferencing systems, cinemas, home theatres) it is necessary to apply multichannel digital signal processing algorithms, already developed for traditional systems. This led to the introduction of Wave Domain Adaptive Filtering (WDAF), a spatio-temporal generalization of Fast Least Mean Squares adaptive algorithm, allowing a considerable re- duction of the computational complexity. Efficient solutions for real time implementation and possible phase approx- imations of the driving functions used in order to manage the loudspeakers are discussed in this thesis. Furthermore, a Weighted-Overlap-Add-based (WOLA-based) approach for WDAF and a WFS-based digital pointing of line arrays are presented: the objective of these studies is that of apply- ing these concepts in real scenarios, such as a teleconferencing system. In- deed, the aforementioned immersive audio reproduction techniques can be exploited for enhancing the performance of life-sized teleconferencing sys- tems, combining temporal and spatial requirements. Furthermore, audio rendering algorithms are needed to improve the perceived audio quality in order to make the listening environment more pleasant by taking into account some speci c features of the environment. More specifically, equalization represents a powerful tool capable of dealing with the frequency response irregularities: an equalizer can compensates for speaker placement and listening room characteristics and it can be applied in a tele-conferencing system to make the communication the most natural as possible. The evaluation of a multipoint equalizer and a mixed-phase solution with a suitably designed room group delay are discussed in this work.

La tesi si propone di studiare la percezione del senso di immersività nella musica in formato binaurale, con l’obiettivo di migliorare l’esperienza d’ascolto e metterla a confronto con la musica ascoltata in formato stereofonico. Attraverso un’analisi delle basi della fisica del suono e della psicoacustica, il saggio vuole illustrare quali sono le tecnologie che permettono alla musica immersiva di essere prodotta. A tal proposito verranno presi in esame sia la storia delle tecniche di riproduzione sonora, che gli strumenti attualmente disponibili, per permettere alla comunità scientifica di avere un quadro completo sulle risorse odierne. Il binomio tra musica e spatial audio è in continua crescita con il passare del tempo, per questo il testo si propone anche di evidenziare quali sono i principali problemi di ricerca e gli standard necessari sui quali concentrare eventuali prossimi lavori. Il saggio si conclude con un esperimento che mette a confronto due differenti tipologie di mix di una canzone: una versione stereofonica e una versione in formato binaurale, processata tramite la Digital Audio Workstation Logic Pro X e la Dolby Atmos Production Suite. I risultati porteranno delle evidenze interessanti a favore della versione in formato binaurale e diverse suggestioni sugli sviluppi futuri.
The thesis aims to study the perception of the sense of immersion in music in binaural format, with the target of improving the listening experience and comparing it with the music listened to in stereophonic format. Through an analysis of the basics of the physics of sound and psychoacoustics, the essay wants to illustrate which technologies allow immersive music to be produced. In this regard, both the history of sound reproduction techniques and the tools currently available will be examined to allow the scientific community to have a complete picture of today's resources. The combination of music and spatial audio is constantly growing with the passage of the years, which is why the text also aims to highlight the main research problems and the necessary standards on which to focus any future works. The essay ends with an experiment that compares two different types of song mixes: a stereo version and a binaural format version, processed through the Digital Audio Workstation Logic Pro X and the Dolby Atmos Production Suite. The results will bring interesting evidence in favor of the binaural format version and various suggestions on future developments.

This thesis describes parameter field spatialization, a novel technique for creating and controlling spatial sound patterns formed along a surface, when working with sound synthesis and signal processing in a computer-music environment. Its main purpose is the creation of spatially-dynamic immersive sources in electroacoustic composition. This technique generates multiple decorrelated signals from a given sound process definition, which when spatialized at different locations can create a single enveloping auditory event with large width and heigth. By modulating parameters of the sound process differently for each signal it is possible to create a spatial surface pattern. The modulation signals are generated based on a mathematical model which assumes a surface encompassing all the loudspeakers and describes an abstract pattern in this surface through a mathematical function of time and surface coordinates, called parameter field. This research investigates whether parameter field spatialization can successfully create and precisely control spatial surface patterns, and how these patterns can be made into a compositional parameter in computer music. The technique was implemented in ImmLib, a software library for the SuperCollider audio-synthesis environment. Several specific examples of the combination of parameter fields with sound processes were investigated from a perceptual point of view, in listening sessions using two loudspeaker systems, one spherical and the other a vertical rectangular grid. A group of composers was invited to use the software for their own work, producing three pieces presented in public, which were analysed regarding the use of the technique. From this practical work findings relating to the most effective strategies regarding the use of parameter fields were outlined.

Den här studien fokuserade på begreppet flow och hur detta tillstånd påverkas av två olika sorters ljudläggnigar. Flow är i studiens sammanhang en kondition som spelaren befinner sig i när den är bekant med spelets mekaniker och är medveten om dess mål och hinder. Genom att ta in information med hjälp av olika medel som exempelvis ljudeffekter och animationer kan spelaren göra snabba och reaktiva beslut. Tidigare forskning visade hur många audivitva källor av immersion påverkar flow negativt, som exempelvis ambienser. Samtidigt visade forskning hur en spelare som befinner sig i ett tillstånd av immersion lättare kommer in i ett ’flow’-tillstånd. För att förstå denna relation mellan flow och immersion, samt hur ljud kan påverka dessa tillstånd skapades ett enkelt ’first-person shooter’ spel med två olika ljudläggningar. Genom insamlad data kunde man se hur ljudläggningarna påverkade spelarens flow mycket annorlunda.

Virtual Reality (VR) can potentially transport the user to another world. Outside of VR, musical soundtrack is usually placed outside of the scene, referred to as non-diegetic sound. In VR, this could potentially break immersion. Other ways to implement music have to be tested. A test was created consisting of three scenes with a wide selection of “listening modes”, or musical configurations. The listening modes ranged from non-diegetic stereo music via headphones to diegetic, played from speakers inside the VR spaces. 10 respondents played through the scenes in VR, experiencing every listening mode. Respondents then replied to a questionnaire gathering their thoughts on their experience. Results showed that immersion improved the more the experience corresponded to expectations from outside of VR. Non-diegetic listening modes were considered less immersive than diegetic listening modes. This study lays a basic foundation for further research on music in VR with initial guidelines for proper implementation.

Mid/side processing techniques are commonly used in the music recording industry to widen the stereo image to create a more enveloping listening experience. Since the gaming industry is now in need of better audio solutions to stay on par with the recent visual advances intechnology; these mid/side techniques could potentially be a useful tool for sound designers to use. In this study, an experiment was conducted where 16 participants were asked to play 4 scenarios with different audio settings meant to enhance envelopment in different ways. After each scenario the participants were asked to rate their preference and perceived envelopment followed by a short survey after all 4 scenarios were completed. The quantitative data showed very little evidence suggesting the mid/side processing to be neither perceived more enveloping nor more preferred than the other versions, except for a group with gamers that played games less than 6 hours per week. The qualitative data on the other hand, showed hints at the mid/side version having envelopment as its defining attribute along with it making the sound design more exciting and making some sounds more powerful. The main problem with the mid/side technique seems to be that it has to exclude in-game spatialization for the widened stereo image to be perceived as enveloping. However, if it is applied on sounds that do not need to be spatialized then it might be able to improve the perceived envelopment of those sounds.

Virtual Reality is one of the next big things in gaming, more and more games delivering an immersive VR-experience are popping up. Words such as immersion and presence has quickly become buzzwords that’s often used to describe a VR-game or experience. This interactive simulation of reality is literally turning people’s heads. The crowd pleaser, the ability to look around in 360-degrees, is however casting a shadow on the aural aspect. This study focused on this problem in relation to audio narrative. We examined which differences we could identify between a purely diegetic audio narrative and one utilizing a mix between diegetic and non-diegetic sound. How to grab the player’s attention and guide them to places in order for them to progress in the story. By spatializing audio using HRTF, we tested this dilemma through a game comparison with the help of soundscapes by R. Murray Schafer and auditory hierarchy by David Sonnenschein, as well as inspiration from Actor Network Theory. In our game comparison we found that while the synthesized sound, non-diegetic, ensured that the sound grabs the player’s attention, the risk of breaking the player’s immersion also increases.

This report covers the creation and implementation of a 3D audio application using FMOD Ex API. The report will also cover a walkthrough of the basic principles of 3D and surround audio, examples of other uses of 3D audio, a comparison between available technologies today, both software and hardware and finally the result of the implementation of the 3D sound environment software, both server and client. The application was created to explore the use of 3D audio in immersive environments. There was no application like this available when this project was conducted. An inductive approach along with a form of rapid application development and scenario creation was used to achieve the results presented in this report. The implementation resulted in a working client and server software which is able to create a 3D sound environment. Based on a user evaluation the software proved to be quite successful. With the help of the implementation the user, or operator, can now create a sound environment for another user, or a listener. The environment is created and designed by the operator using the client side of the implementation and later played through the server side which is connected to a 4.1 speaker system. The operator can steer and add sounds from the client to an active environment and the listener can experience the change in real time. This project was conducted as a bachelor thesis in computer science at Mälardalens University in Västerås, Sweden.

Denna undersökning syftar till att studera förhållandet mellan ljud, bild och spelare i en TV- och datorspelssituation. Med hjälp av fokusgruppdiskussioner berörs frågor på det audiovisuella området, där deltagarna är aktiva spelare som har tillryggalagt ett mycket stort antal speltimmar som de bygger sina kunskaper på. Här framkommer att ljud och musik fyller viktiga funktioner för spelarens möjlighet att interagera och leva sig in i spelet och att varje spel har en unik ljud- eller musikmiljö som spelarna tycks förhålla sig till på olika sätt beroende på varför man spelar och om man spelar ensam eller tillsammans med andra, det vill säga singelplayer eller multiplayer. Informanternas kunskaper utgör styrkan i denna undersökning men det vore önskvärt att utöka studien med fler fokusgruppdiskussioner samt djupintervjuer med utvalda informanter för att fördjupa materialet. Som förslag på vidare forskning vore det intressant att studera användandet av virtuella headsets där en stereoskopisk 3D bild skapas som spårar huvudets rörelser, för att undersöka om detta kan ha en inverkan på hur vi uppfattar ljudet i ett spel.

Submitted by Filipe dos Santos (fsantos@pucsp.br) on 2018-12-04T11:46:19Z No. of bitstreams: 1 Felipe Antunes de Oliveira Rodrigues.pdf: 2502573 bytes, checksum: b9be7ad020151fc8cbad343711414219 (MD5)
Made available in DSpace on 2018-12-04T11:46:19Z (GMT). No. of bitstreams: 1 Felipe Antunes de Oliveira Rodrigues.pdf: 2502573 bytes, checksum: b9be7ad020151fc8cbad343711414219 (MD5) Previous issue date: 2018-09-24
Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES
This dissertation has the objective of analyzing the experience of immersive audio in digital games, as perceived by the player as a listener and agent in the soundscape of virtual reality. The first chapter seeks to briefly examine and define the concepts of immersion and presence, in order to allow a deeper study of immersive audio to be possible. It also presents considerations about the coupling devices through which immersion in digital games is possible, in particular headphones, so common in the gamer routine. The second chapter is dedicated to the analysis of sound elements found in the virtual reality of digital games through the crossing of concepts and theories by authors such as Schafer, Meneguette, Huiberts, Droumeva among others. By doing so it seeks a comprehensive understanding of what can typically be heard in this medium in order to stimulate the immersive experience. The chapter also deals with problems and solutions that might interfere in the immersion in positive or negative ways, in order to clarify the relevance of the minutiae with respect to immersive audio. The third chapter approaches the subject from another angle, seeking an understanding of how the immersive experience of audio em digital games presents itself to the player. Next, analysis of of the game can hear the soundscape in the game are made. At the end of the chapter the concepts are reviewed, aiming to presenting an understanding of how the player is given the immersive experience of audio in digital games. The fourth and last chapter consists of a case study, in order to demonstrate, in a practical way, the immersive experience of audio studied so far
Esta dissertação tem como objetivo a análise da experiência do áudio imersivo em jogos digitais, como percebida pelo jogador enquanto ouvinte e agente na paisagem sonora da realidade virtual. O primeiro capítulo busca examinar e definir brevemente os conceitos de imersão e presença, de forma a permitir que um estudo mais aprofundado sobre o áudio imersivo seja possível. Também apresenta considerações sobre aparatos de acoplamento através dos quais a imersão em jogos digitais é possível, em particular os fones de ouvido, tão comuns na rotina gamer. O segundo capítulo é dedicado a análise dos elementos sonoros encontrados na realidade virtual dos jogos digitais através do cruzamento de conceitos de teorias de autores como Schafer, Meneguette, Huiberts, Droumeva entre outros. Busca-se assim um entendimento abrangente daquilo que tipicamente pode ser ouvido nessa mídia de forma a estimular a experiência imersiva. O capítulo também trata de problemas e soluções que possam interferir na imersão de forma positiva ou negativa, de forma a clarificar a relevância das minúcias no que diz respeito ao áudio imersivo. O terceiro capítulo aborda o assunto de outro ângulo, buscando um entendimento das formas como o áudio é apresentado ao jogador através de convenções e escolhas de design. Em seguida são feitas análises das diferentes formas como o jogador pode ouvir a paisagem sonora do jogo. Ao fim do capítulo são revisados os conceitos estudados, visando apresentar um entendimento de como se dá para o jogador a experiência imersiva do áudio em jogos digitais. O quarto e último capítulo consiste em um estudo de caso, com a finalidade de demonstrar, de forma prática, a experiência imersiva do áudio, estudada até então

This study is an evaluation of online annotated readers developed for first grade students enrolled in Chinese immersion. The electronic readers were created to provide additional input to immersion students, who had little time in class for Chinese character reinforcement. The students accessed online readers from their homes and took assessments before and after each reader to test for improved character comprehension. In addition, students were divided into treatment and control groups. The treatment group had annotated electronic readers with audio and games. Conversely, the control group did not have annotations but audio was included. Results demonstrate a significant difference between preliminary and post-assessments, suggesting that students comprehended more characters after reading. No significant differences were detected between the control (non-annotated) and treatment (annotated) groups. Additional data collected from parent surveys provide useful demographics about subjects' socio-cultural and language variables as well as highlight parental desires for more support and help-aides. Computer Assisted Language Learning (CALL) in relationship to young immersion students learning Chinese is also discussed. Results suggest that online annotated readers can be an important resource for students who have limited instructional time in the classroom and little opportunity to receive help at home.

I nuvarande forskning så existerar det ett research gap inom ljudproduktion och ljuddesign för Virtual Reality-applikationer. Det finns en mängd teknisk forskning om uppspelning av spatialiserat ljud för virtual reality med tekniker som binauralt ljud, HRTF (head-related transfer functions) och ambisonics. Allt detta är av högsta grad relevant för denna undersökning och kommer att diskuteras kring samt redogöras för i senare del av artikeln, men kommer inte att ta en central del av undersökningen. Snarare är det avsaknaden av metoder och processer för det kreativa skapandet och implementeringen av ljud för Virtual Reality som intresserar den här undersökningen. Denna artikel tar ett initiativ för att framhäva vikten av en genomtänkt och väl utförd ljuddesign för Virtual Reality- applikationer samt redogöra för tankesätt och metoder som kan appliceras i en audiell designprocess. Detta genom analys utav fenomenen immersion och närvaro. Med denna analys som grund översätts dessa sedan till metoder och nyckelord som främjar att stödja skapandet av applikationers audiella innehåll. Undersökningen har mynnat ut i ett designobjekt som existerar som ett exempel för denna process och redogörs med en grundlig dokumentation. Artikeln avslutas med en jämförande studie där detta designobjekt placeras tillsammans med andra designexempel från marknaden i ett antecknande portfolio. Detta för att undersöka likheter mellan dessa exempel utifrån de nyckelord som kommer att framföras samt redogöras för i artikeln. Förhoppningen är att denna process kan tydliggöra för ljudets roll i att skapa en immersiv upplevelse samt inspirera andra ljuddesigners att applicera dessa metoder i dennes egna designprocess.
In present research there exists a research gap within the field of sound production and sound design for virtual reality-applications. There is a ton of technical research to be found such as the deliverance of spatial audio for virtual reality through techniques like binaural audio, HRTF (head-related transfer functions) and ambisonics. All of these are highly relevant for this research and will be discussed in a later part of the article however they won't take a central part of the research process. It is rather the lack of methods and processes for the creative design and implementation of sound for virtual reality that this research finds interesting. This article takes an initiative to shed light on the importance of a well thought- through and properly executed sound design for Virtual Reality applications and gives an account for the thought process as well as methods that can be applied to an audio design process. This is done through an analysis of the phenomena known as immersion and presence. With the analysis as groundwork, the explanations of these phenomena then get translated to methods and keywords that support the creation of audio content for the given application. This research has formed the basis for a design object that exists as an example for this process and has been thoroughly documented along the way. The article ends with a comparative research study where this design object is placed together with other design examples from the general market in an annotated portfolio. The reason for this is to point out similarities through the lens of keywords that will be presented and explained in this article. This process sets out to clarify the role sound has in the creation of an immersive experience as well as to inspire other sound designers to apply these methods in their own design processes.

Based on previous research on spatial attributes in foley and the concept that auditory immersion in first-person perspective computer games is enhanced by believable sound effects, this study explores if there is a connection between stereo foley and the believability of the first-person perspective, and regardless, if there is a preference to either mono or stereo foley. An interactive listening test was created in unreal engine 4, where 20 subjects, all considered gamers, played three levels that differed visually and in auditory content. In these levels, subjects auditioned two versions of avatar-related foley sounds. One version was mono, the other stereo. The test prompted the subjects to complete two tasks for each level, whereupon the foley version changed upon completion of the first task. The subjects then answered questions in between each level, regarding the foley version. They were asked to rate believability and choose a preference, as well as provide motivations for their choices. The quantitative data showed next no evidence that either mono or stereo was generally perceived as more believable or preferred. However, the qualitative data indicates that the majority of players tend to prefer and rate stereo foley as more believable in certain game environments. Furthermore, the data indicates that some subjects prefer a sensory replication of reality in foley. It is also shown that preference for stereo width vary between subjects and therefore argued that there cannot be a perfect standardized setting for stereo foley.

This thesis expands on previous research and designs of avatar-based child interview training software. The goal of the thesis was to identify requirements, identify technologies and evaluate the likelihood of acceptance of a distribution ready software that would enhance role-play training exercises commonly used for child interview training. After identifying the requirements needed to create this type of application the needed technologies for solving those requirements were identified and one prototype and two production ready applications were developed. The production ready versions were distributed in an official capacity through AvBIT Labs Ab. Each version was evaluated using the technological acceptance model (TAM) in order to determine likelihood of acceptance in relevant industries. The TAM survey, USE survey and correspondence with experts were used to evaluate missing requirements and the likelihood of software acceptance. The research conducted in this thesis directly contributed to the founding of AvBIT Labs AB and the distribution of the AvBIT application to both governmental and non-governmental organizations, seeking to enhance their child interview training, throughout Europe.

The purpose of this thesis is to propose an alternative type of interface for mobile software being used while walking or running. Our work addresses the problem of visual user interfaces for mobile software be- ing potentially unsafe for pedestrians, and not being very immersive when used for location-based games. In addition, location-based games and applications can be dif- ficult to develop when directly interfacing with the sensors used to track the user’s location. These problems need to be addressed because portable computing devices are be- coming a popular tool for navigation, playing games, and accessing the internet while walking. This poses a safety problem for mobile users, who may be paying too much attention to their device to notice and react to hazards in their environment. The difficulty of developing location-based games and other location-aware applications may significantly hinder the prevalence of applications that explore new interaction techniques for ubiquitous computing. We created the TREC toolkit to address the issues with tracking sensors while developing location-based games and applications. We have developed functional location-based applications with TREC to demonstrate the amount of work that can be saved by using this toolkit.In order to have a safer and more immersive alternative to visual interfaces, we have developed ambient audio interfaces for use with mobile applications. Ambient audio uses continuous streams of sound over headphones to present information to mobile users without distracting them from walking safely. In order to test the effectiveness of ambient audio, we ran a study to compare ambient audio with handheld visual interfaces in a location-based game. We compared players’ ability to safely navigate the environment, their sense of immersion in the game, and their performance at the in-game tasks. We found that ambient audio was able to significantly increase players’ safety and sense of immersion compared to a visual interface, while players performed signifi- cantly better at the game tasks when using the visual interface. This makes ambient audio a legitimate alternative to visual interfaces for mobile users when safety and immersion are a priority.
Thesis (Master, Computing) -- Queen's University, 2012-01-31 23:35:28.946

碩士
中華大學
資訊管理學系
95
Multiplayer online games are the trend of the day. A recent innovation in multiplayer online game is the addition of technology that allows players to communicate with each using VoIP rather than typed text. Many of multiplayer online games have already support voice communication for players such as Microsoft Xbox Live® [1]. However, it merely provides voice channels for players to chat. This paper proposes a use of VoIP conference, which it provides an immersive audio communication in multiplayer online game. The mix of voices from different sound sources according to player’s location in the virtual game space. For each user, the audios are attenuated according to distance from the listener. This provides a better use of VoIP function and players can also have better enjoyment in the game. This paper is based on the multiplayer online game with VoIP architecture from Chen’s thesis in 2005[8].

In spite of the increasing interest that video games have gained over the last decades, there has not yet been an academic effort to identify trend-like patterns in the creation of sound for video game characters. Such studies would be beneficial for amateur sound designers and indie game designers by enabling them to better understand the strategies that may coerce the player into diving and pleasantly navigating within the game world. The search for answers to this research gap entailed a path of research, which unfolded into six chapters. The Introduction (Chapter 1) describes the research gap and the mixed methods research that was used, while Chapter 2 contextualizes it. Chapter 3 seeks to understand the problem at the theoretical level, examining the concepts of Perception, Emotion and Immersion, and Chapter 4 delves into the research problem, aiming at evincing the importance of character soundprint for the acoustic ecology of the game. Chapter 5 confirms and further develops the previous research findings by investing in an in-depth analysis of two instrumental case studies. Finally, the Conclusion (Chapter 6) levels up the discussion, hinting at the new paths for game audio that Virtual Reality and 3-D audio will lead to. The dissertation offers a variety of research outputs that were essential for grounding the research hypothesis and which will hopefully be useful in the future: analytical graphs resulting from an online survey; a table systematizing the way different musical structures influence emotion; a table resulting from the direct observation of popular characters’ soundprints in a universe of 30 games; a table summarizing the soundprint types from patterns identified in the previous table; a table for structuring game analysis; and several tables analysing the case studies Non-Player Characters’ and Player Characters’s soundprints. Even though it is not an exact science, the analysis of video game characters showed it is possible to discern different soundprint types. The research further evinced that, along with the remaining structures of the game, sound and soundprints assist with the immersion process of the players, enhancing their opportunities for identifying with the characters and finding their own place in the game world.
Apesar do interesse crescente que os videojogos têm vindo a conquistar nas últimas décadas, não se verificou ainda um esforço académico para a identificação de padrões de tendência na criação de som para as personagens de videojogos, o que beneficiaria sound designers amadores e game designers indie, pois permitiria um melhor entendimento das estratégias que provocam a imersão do jogador no universo do jogo. A busca de respostas para esta lacuna de investigação implicou um caminho de investigação que se dividiu em seis capítulos. O Capítulo 1 (Introdução) descreve a lacuna de investigação e o método de investigação misto utilizado, enquanto o Capítulo 2 a contextualiza. O Capítulo 3 procura entender o problema a nível teórico, analisando os conceitos de Perceção, Emoção e Imersão, enquanto o Capítulo 4 se centra no problema de investigação, procurando evidenciar a importância do soundprint das personagens para a ecologia acústica do jogo. O Capítulo 5 confirma os resultados da pesquisa e vai para além deles, investindo numa análise aprofundada de dois estudos de caso instrumentais. Por fim, a Conclusão (Capítulo 6) eleva a discussão a um nível superior, indicando os novos caminhos que, com a Realidade Virtual e o áudio 3-D, o áudio para videojogos virá a trilhar. A dissertação oferece um conjunto diversificado de resultados de investigação que foram essenciais para a fundamentação da hipótese de trabalho e poderão vir a ser úteis para pesquisa futura: gráficos analíticos de respostas a um inquérito online; uma tabela de sistematização da forma como diferentes estruturas musicais influenciam a emoção; uma tabela resultante da observação direta de soundprints de personagens populares num universo de 30 jogos; uma tabela resumindo os tipos de soundprint identificados na tabela anterior; uma tabela de estruturação de análise de jogos; e várias tabelas de análise do soundprint dos NPCs (personagens não jogáveis) e das personagens do jogador dos casos de estudo. Apesar de não ser uma ciência exata, a análise das personagens de vídeo jogos revelou que é possível classificar diferentes tipos de soundprint e que, em conjunto com as restantes estruturas do jogo, o som e os soundprints ajudam ao processo de imersão dos jogadores, aumentando as possibilidades de se identificarem com as personagens e encontrarem o seu lugar no mundo do jogo.

The degree of diffusion of hypercomplex algebras in adaptive and non-adaptive filtering research topics is growing faster and faster. The performance of hypercomplex adaptive filters has been widely experimented during the last decade. Quaternion filters, in particular, have been utilized in systems where the signals to be processed have some form of correlation. Besides correlation, the debate today concerns the usefulness and the benefits of representing multidimensional systems by means of these complicated mathematical structures and the criterions of choice between one algebra or another. One of the goals of this work is to discuss whether the choice of a certain algebra in the description of a problem/environment can play a significant role and determine an adaptive filter performance. That said, adaptive filtering can be expanded to new numerical systems and unseen sides of physical problems can be highlighted thanks to the mathematical properties of such hypercomplex algebras. Each algebra has its own rules and calculation outcomes may not be compatible from one algebra to another. However, such peculiarities diversify algebras in a way that each of them fits specific geometrical/physical problems. The bulk of study and experiments presented in this work was carried out in a 3-Dimensional (3D) audio context. 3D audio is the new frontier in audio technology and it is quickly taking place in many applications, from cinema to virtual reality, audio surveillance and video games. The large amount of data requires fast and compact solutions for signal processing. With this aim in view, research is moving towards the exploration of hypercomplex algebras in order to find a non-redundant and compact form for the representation of 3D sound fields without loss of information. Quaternion sound fields are currently under investigation and this thesis presents some recent results concerning the integration of hypercomplex (quaternion) adaptive signal processing into a 3D audio environment.

Lo spazio destinato alla performance musicale costituisce la cornice e il contenuto dell'esperienza degli ascoltatori. L'ambiente acustico impone continue negoziazioni che differiscono in base al ruolo e alla posizione dell'ascoltatore come compositore, esecutore o membro del pubblico. Obiettivo della mia ricerca è indagare l'acustica di uno spazio performativo, il Teatro del Maggio Musicale Fiorentino di Firenze, seguendo due percorsi complementari, entrambi basati su un modello digitale interattivo. Il primo offre un’esperienza audiovisiva in cui l'utente può esplorare virtualmente la sala grande del teatro scegliendo tra le riproduzioni binaurali di 13 diverse posizioni di ascolto. Il secondo riguarda la percezione sonora e visiva di un'esecuzione della romanza “Una furtiva lagrima” dall'opera di Donizetti L'elisir d'amore. L'utente, attraverso l’utilizzo di ambisonics, video a 360 ° e realtà virtuale, può fruire questa performance e il contributo acustico della sala da tre diverse posizioni nel teatro: sul palco, nella buca dell'orchestra e in platea.
 Per la realizzazione di tali esperienze di virtual acoustics si è svolto uno studio interdisciplinare afferente alle Digital Humanities che ha riguardato: la percezione sonora; l’acustica; l’architettura; il contesto storico-spettacolare dei luoghi della performance musicale attraverso un’indagine musicologica del rapporto tra opere e spazio; e le pratiche di restituzione digitale in contesti immersivi del patrimonio culturale immateriale.

Since the success of James Cameron’s Avatar (2009),1 the feature film industry has embraced 3-D feature film technology. With 3-D films now setting a new benchmark for contemporary cinemagoers, the primary focus is directed towards these new stunning visuals. Sound is often neglected until the final filmmaking process as the visuals are taking up much of the film budget. 3-D has changed the relationship between the imagery and the accompanying soundtrack, losing aspects of the cohesive union compared with 2-D film. Having designed sound effects on Australia’s first digital animated 3-D film, Legend of the Guardians: The Owls of Ga’Hoole (2010),2 and several internationally released 3-D films since, it became apparent to me that the visuals are evolving technologically and artistically at a rate far greater than the soundtrack. This is creating a dislocation between the image and the soundtrack. Although cinema sound technology companies are trialing and releasing new ‘immersive’ technologies, they are not necessarily addressing the spatial relationship between the images and soundtracks of 3-D digital films. Through first hand experience, I question many of the working methodologies currently employed within the production and creation of the soundtrack for 3-D films. There is limited documentation on sound design within the 3-D feature film context, and as such, there are no rules or standards associated with this new practice. Sound designers and film sound mixers are continuing to use previous 2-D work practices in cinema sound, with limited and cautious experimentation of spatial sound design for 3-D. Although emerging technologies are capable of providing a superior and ‘more immersive’ soundtrack than previous formats, this does not necessarily mean that they provide an ideal solution for 3-D film. Indeed the film industry and cinema managers are showing some resistance in adopting these technologies, despite the push from technology vendors. Through practice-led research, I propose to research and question the following:Does the contemporary soundtrack suit 3-D films? ; Has sound technology used in 2-D film changed with the introduction of 3-D film? If it has, is this technology an ideal solution, or are further technical developments needed to allow greater creativity and cohesiveness of 3-D film sound design? ; How might industry practices need to develop in order to accommodate the increased dimension and image depth of 3-D visuals? ; Does a language exist to describe spatial sound design in 3-D cinema? ; What is the audience awareness of emerging film technologies? And what does this mean for filmmakers and the cinema? ; Looking beyond contemporary cinema practices, is there an alternative approach to creating a soundtrack that better represents the accompanying 3-D imagery?