Thèses sur le sujet « Accessible digital musical instruments »

This thesis explores a particular research topic in the field of Sound and Music Computing, dedicated to the creation of Accessible Digital Musical Instruments (ADMIs) designed for users affected by quadriplegia or similar motor impairments. With such conditions an user is completely paralyzed from the neck down. The impossibility to control the upper and lower limbs, particularly fingers, makes it impossible for such users to play conventional musical instruments, both acoustic and digital. This makes it necessary to introduce specific and non-trivial design and development solutions. A first part of the work is dedicated to the analysis of the related context. After defining the pertinent jargon, an analysis of different physical interaction channels available to people with quadriplegic disabilities is provided, with a review of the sensors suitable for their detection. Some of these channels are then evaluated through an experimental methodology. Further chapters provide an analysis of the state of the art in ADMIs for quadriplegic users, as well as design tools dedicated to ADMIs in general. A chapter is dedicated to the design of musical interfaces controlled through gaze, one of the most employed channels in this context. The second part describes the design, development and testing of new ADMIs suitable for quadriplegic users. The implementation of a software library for developing or fast-prototyping software instruments is described, as well as two Open-Source Hardware sensor peripherals developed ad-hoc for the detection of breath and head rotation. The remaining chapters describe the design, implementation and evaluation of Netytar, Netychords and Resin, three ADMIs that are played through the detection of gaze, breath, head movement, and stimulated resonances in the upper vocal tract, therefore playable hands-free.

In digital music-making activities musicians with physical disabilities employ both accessible and generic control interfaces; accessible controllers capture broad input gestures and map them onto discrete output events, whereas consumer digital musical instruments (DMI's) offer extended control only through artefact multiplication (more buttons, sliders and dials). The interaction paradigm common to both consumer and specialised controllers reveals limited dimensions: click-and-drag or select-and-move. It is common practice in inclusive music activities for an able-bodied facilitator to expose access to low-level parameters via sequential and non-real-time processes, on behalf of the musician with a physical disability; these factors constrain real-time independent creative self-expression. The current research details the explicit needs and capabilities of a small group of digital musicians with quadriplegic cerebral palsy, garnered through participatory design and mixed methods. The project methodology draws on tools and models from the fields of assistive technologies and from mainstream DMI design. Project participants contribute data pertaining to preferences and capabilities, and evaluate key iterations of the evolving prototype. The practice-led and participatory design ethos relies on demonstrably repeatable and preferred gestural capabilities, without seeking to maximise physical ability in a rehabilitative context. The underlying mapping strategy exposes, in real time, a transparent hierarchy of dynamic sound parameters commonly accessed through facilitated, offline and sequential processes.

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 1999.
Includes bibliographical references (p. 87-92).
This thesis proposes to use technology to introduce children to musical expressivity and creativity. It describes a set of digital musical instruments that were developed in an effort to provide children with new tools for interaction, exploration and enjoyment of music. The thesis unfolds a multidisciplinary theoretical background, which reviews a number of philosophical, psychological, musical, and technological theories. The theoretical background focuses on enlightening a number of personal musical experiences and leads towards the formulation of three musical concepts that inform the design of the digital musical instruments. The musical concepts are: High and Low-level Musical Control, Immersive and Constructive Musical Experiences and Interdependent Group Playing. The thesis presents the embodiment of these concepts in digital musical instruments while emphasizing the importance of novel technology as a provider of creative and expressive musical experiences for children.
by Gil Weinberg.
S.M.

This thesis deals with the study of performer-instrument interaction during the performance of novel digital musical instruments (DMIs). Unlike acoustic instruments, digital musical instruments have no coupling between the sound generation system and the physical interface with which the performer interacts. As a result of this, such instruments also lack the direct physical feedback to the performer which is present in an acoustic instrument. In fact in contrast to acoustic musical instruments, haptic and vibrotactile feedback is generally not present in a DMI contributing to a poor feel for the instrument. The main goal of this thesis is to propose ways to improve the overall feel of digital musical instruments through the study and design of its physical interface: the instrument body, sensors and feedback actuators. It includes a detailed study of the existing theory and practice of the design on physical interfaces for digital musical instruments, including a survey of 266 existing DMIs presented since the inception of the NIME conference. From this, a number of differences become apparent between the existing theory and practice, particularly in the areas of sensors and feedback. The research in this thesis then addresses these differences. It includes a series of experiments on the optimal choice of sensors for a digital musical instrument. This is followed by research into the provision of vibrotactile feedback in a digital musical instrument, including the choice of actuator, modification of actuator frequency response, and the effects of response modification on human vibrotactile frequency discrimination. Following this, a number of new digital musical instruments are presented, which were created during the course of this work. This includes an instrument designed specifically to follow the results of research in this thesis and also instruments designed as part of larger collaborative projects involving engineers, composers and performers. Fro
Cette thèse porte sur l'étude de l'interaction ayant lieu, en situation de jeu,entre un(e) instrumentiste et un instrument musical numérique (IMN).A l'inverse des instruments acoustiques traditionnels, il n'existe aucun couplageentre le dispositif de production du son et l'interface sur laquelle agit l'instrumentistedans le cas des IMN. L'une des implications de cette observation est que cesinstruments ne procurent pas la rétroaction tactile normalement présente dans lesinstruments de musique traditionels. Par conséquent, les IMN sont souvent perçuspar leurs interprètes comme manquant d'âme, de personnalité.Le but de ce travail de thèse est d'avancer quelques solutions permettant d'insuer un peu plus âme à un instrument musical numérique. Le point focal de larecherche étant l'étude et la conception de l'interface physique (corps de l'instrument,capteurs et dispositifs de rétroaction utilisés) d'un tel instrument.Ce mémoire présente, en premier lieu, une étude détaillée de la théorie et de lapratique actuelles dans le domaine de la conception d'interfaces physiques pour lesIMN. L'inventaire des 266 instruments recensés depuis la création de la conférenceNIME constitue l'un des points majeurs de cette partie du travail. En effet, ce tour d'horizon permet de faire ressortir les incohérences entre théorie et pratique. Cesdifférences sont particulièrement frappantes en ce qui concerne les capteurs et lesdispositifs de rétroaction.Le travail de recherche de cette thèse a donc pour objectif de mieux comprendrecomment réduire ces incohérences. Des expériences portant sur le choix optimaldes capteurs à utiliser dans un IMN ont donc été menées. Différents dispositifs derétroaction vibrotactile ont aussi été étudiés en regardant d'abord quels actuateursutiliser, et en évaluant les effets de la modication de leur réponse en fréquencesur la discrimination fréquentielle de stimuli vibrotactiles chez des sujets humains.Des exemp

Despite the increasing sophistication of digital musical instruments, many performers, composer and listeners remain captivated by traditional acoustic instruments. Interest has grown in the past 2 decades in augmenting acoustic instruments with sensor and actuator technology and integrated digital signal processing, expanding the instrument's capabilities while retaining its essential acoustic character. In this thesis we present a technique, travelling wave control, which allows active control of the vibrations of musical strings and yet has been little explored in the musical instrument literature to date. The thesis seeks to demonstrate that travelling wave control is capable of active damping and of modifying the timbre of a musical string in ways that go beyond those available through the more conventional modal control paradigm. However, we show that travelling wave control is highly sensitive to nonlinearity, which in practical settings can lead to harmonic distortion and even instability in the string response. To avoid these problems, we design and build a highly linear optical string displacement sensor, and investigate the use of piezoelectric stacks to actuate the termination point of a string. With these components we design and build a functioning travelling wave control system which is capable of damping the vibrations of a plucked string without adversely affecting its timbre. We go on to show that by deliberately adding nonlinearity into the control system, we are able to modify the timbre of the string in a natural way by affecting the evolution of the modal amplitudes. The results demonstrate the feasibility of the concept and lay the groundwork for future integration of travelling wave control into future actuated musical instruments.

Playing a musical instrument has been shown to have a positive impact in the life of individuals in many different ways. Nevertheless, due to physical disabilities, some people are unable to play conventional musical instruments. In this dissertation, we consider different types of physical disabilities and implement specific digital musical instruments suitable for people with disabilities of each type. Firstly, we consider the case of people with limited sensorimotor upper limb functions, and we construct low-cost digital instruments for three different scenarios. Results indicate that the constructed prototypes allow musical expression and improve the quality of life of these users. Secondly, we consider disabilities such as tetraplegia or locked-in syndrome with unaffected eye-movements. For individuals with such conditions, we propose the EyeHarp, a gaze-controlled digital music instrument, and develop specific target selection algorithms which maximize the temporal and spatial accuracy required in music performance. We evaluate the instrument on subjects without physical disabilities, both from an audience and performer perspective. Results indicate that the EyeHarp has a steep learning curve and it allows expressive music performances. Finally, we examine the case of brain-controlled music interfaces. We mainly focus in auditory event related potential-based interfaces. In particular, we investigate and evaluate how timbre, pitch and spatialization auditory cues affect the performance of such interfaces.
Se ha demostrado que tocar instrumentos musicales tiene un impacto positivo en muchos aspectos de la vida de las personas. Sin embargo, debido a discapacidades físicas, a un gran número de personas les es imposible tocar instrumentos musicales tradicionales. En esta tesis doctoral consideramos diferentes tipos de discapacidades físicas e implementamos instrumentos musicales digitales adaptados a las capacidades de las personas que las padecen. En primer lugar, consideramos el caso de personas con discapacidad motora en los miembros superiores sensoriomotoras. Utilizando materiales de bajo coste implementamos prototipos en tres escenarios diferentes. Los resultados indican que los prototipos construidos permiten la expresión musical y mejoran la calidad de vida de los usuarios. En segundo lugar, consideramos discapacidades como la tetraplejia o el síndrome locked-in donde aun se conservan los movimientos oculares. En ese caso, se propone el EyeHarp, un instrumento que se controla con movimientos de los ojos. Hemos desarrollado algoritmos de selección que maximizan la precisión temporal y espacial requerida en la ejecución de instrumentos musicales y evaluamos el instrumento con gente sin discapacidades, desde la perspectiva de la audiencia y del músico. Los resultados indican que el EyeHarp tiene una curva de aprendizaje inclinada y permite interpretaciones musicales expresivas. Finalmente examinamos el caso de las interfaces musicales cerebro-ordenador. En particular, investigamos interfaces cerebro-ordenador basadas en potenciales relacionados con eventos auditivos. Investigamos cómo timbre, tono y espacialización afectan el rendimiento de dichas interfaces y proponemos y evaluamos interfaces musicales basadas en esta técnica.

Digital music technologies, and instruments in particular, are the result of specific systems of thought that define and enframe the user's creative options. Distinctive divisions between digital and acoustic instruments can be traced, contrasting the conceptually based design of software with the affordances and constraints of physical artefacts. Having lost the world's gift of physical properties, the digital instrument builder becomes more than a mere luthier. The process of designing and building the instrument is transformed into a process of composition, for it typically contains a greater degree of classification and music theory than its acoustic counterpart. Part I of this thesis begins by framing musical systems in the context of the philosophy of technology. Here technological conditions are questioned and theories introduced that will assist the investigation into the relationship between creativity and technology. After this general grounding, the ramifications digital technologies pose to the human body are explored in the context of human expression through tool use. The human-machine relationship is described from phenomenological perspectives and relevant theories of cognitive science. This analysis serves as a foundation for the concept of epistemic tools, defmed as the mechanism whereby techno-cultural models are inscribed into technological artefacts. The cultural element of tool use and tool origins is therefore emphasised, an aspect that is highly relevant in musical technologies. Part I thus frames the material properties of acoustic and digital instruments in relation to human culture, cognition, performance and epistemology. Part II contextualises these theories in practice. The ixiQuarks, the live improvisation musical environment that resulted from the current research, are presented as a system addressing some of the vital problems of musical performance with digital systems (such as the question of embodiment and theoretical inscriptions), proposing an innovative interaction model for screen-based musical instruments. The concept of virtual embodiment is introduced and framed in the context of the ixi interaction model. Two extensive user studies are described that support the report on ixiQuarks. Furthermore, comparative surveys on the relationship between expression and technology are presented: a) the phenomenology of musical instruments, where the divergence between the acoustic and the digital is investigated; b) the question of expressive freedom versus time constraints "in musical environments is explored with practitioners in the field; and c) the key players in the design of audio programming environments explain the rationale and philosophy behind their work. These are the first major surveys of this type conducted to date, and the results interweave smoothly with the observations and findings in the chapters on the nature and the design of digital instruments that make up the majority of Part II. This interdisciplinary research investigates the nature of making creative tools in the digital realm, through an active, phiIosophicaIIy framed and ethnographicaIIy inspired study, of both practical and theoretical engagement. It questions the nature of digital musical instruments, particularly in comparison with acoustic instruments. Through a survey of material epistemologies, the dichotomy between the acoustic and the digital is employed to illustrate the epistemic nature of digital artefacts, necessitating a theory of epistemic tools. Consequently virtual embodiment is presented as a definition of the specific interaction mode constituting human relations with digital technologies. It is demonstrated that such interactions are indeed embodied, contrasting common claims that interaction with software is a disembodied activity. The role of cultural context in such design is emphasised, through an analysis of how system design is always an intricate process of analyses, categorisations, normalisations, abstractions, and constructions; where the design paths taken are often defined by highly personal, culturaIIy conditioned and often arbitrary reasons. The dissertation therefore dissects the digital musical instrument from the perspectives of ontology, phenomenology and epistemology. Respective sections in Part I and Part II deal with these views. The practical outcome of this research - the ixiQuarks - embodies many of the theoretical points made on these pages. The software itself, together with the theoretical elucidation of its context, should therefore be viewed as equal contributions to the field of music technology. The thesis closes by considering what has been achieved through these investigations of the technological context, software development, user studies, surveys, and the phenomenological and epistemological enquiries into the realities of digital musical instruments, emphasising that technology can never be neutral.

This thesis describes the design and use of two novel digital musical instruments (DMIs) based on moving mechanical systems. The motivation behind using mechanical devices was threefold: to explore the effect of physical effort on DMIs, to make use of the device's inherent haptic and visual feedback, and to serve as a starting point for sound mappings. It was hoped that their mechanical nature would give the instruments a character that could emerge through each of the mappings. The first DMI built was the Gyrotyre, a hand-held DMI based around a small bicycle wheel outfitted with sensors that measure its speed of rotation and as well as its angle of orientation. The second DMI built was the Springwave, which consists of a loose metal spring stretched to one meter and fixed at both ends to a metal frame. The frame is in turn mounted horizontally on a hi-hat stand so that it can be raised and lowered with the pedal, thus inducing oscillation in the spring. Various mappings were designed to reflect and make use of the physical nature of both instruments. It was found that the nature of interaction with each instrument was very different depending on the mapping used. The use of mechanical devices was found to be a useful starting point for the development of mappings, and made playing the instruments engaging for the performer by the relationship between DMIs and musical contexts, a framework for characterizing DMIs that takes musical context into account is presented.

Digital musical instruments pose a number of unique challenges for designers and performers. These issues stem primarily from the lack of innate physical connection between the performance interface and means of sound generation, for the latter is usually dematerialised. Thus, this relationship must instead be explicitly determined by the designer, and can be essentially any desired. However, many design issues and constraints remain poorly understood, from the nature of control to the provision of performer-instrument feedback. This practice-based research contends that while the digital and acoustic domains are so different as to be fundamentally incompatible, useful antecedents for digital musical instruments can be found in the histories of electronic music. Specifically, it argues that the live electronics of David Tudor are of particular prescience. His home-made circuits offer an electronic music paradigm quite antithetical to both the familiar keyboard interface and the electronic music studios that grew up in the years after World War II, and are seen to embody a number of aspirational qualities. These include performer-instrument interaction more akin to steering rather than fine control, the potential for musical outcomes that are unknown and unknowable in advance, and distinct instrumental character. This leads to the central contribution of this research; the development of a Tudor-inspired conceptual framework that can inform how digital musical instruments are designed, played, and evaluated. To enable more detailed and nuanced discussion, the framework is broken down into a series of sub-themes. These include both design issues such as nuance, plasticity and emergence, and human issues such as experience, expressiveness, skill, learning, and mastery. The notion of sketching in hardware and software is also developed in relation to the rapid iteration of multiple designs. Informed by this framework, seven new digital musical instruments are presented. These instruments are tested from two different perspectives, with the personal experiences of the author supplemented with data from a series of smallscale user studies. Particular emphasis is placed on how the instruments are played, the music they can produce, and their capacity to convey the musical intentions of the performer (i.e. their expressiveness). After the evaluation of the instruments, the Tudorian framework is revisited to form the basis of the conclusions. A number of modifications to the original framework are proposed, from the addition of a dialogical model of performerinstrument interaction, to the situation of digital musical instruments within a wider musical ecology. The thesis then closes with a suggestion of possibilities for future research.

Human movement is central to instrumental musical performance. Beyond the apparent connection between sound-producing actions and the sounds themselves, movement can communicate emotion, musical intention and structure. In designing an acoustic instrument, the requirements to support the vibration and manipulation of strings or membranes constrain the possibilities for action that facilitate performance. In Digital Musical Instrument (OM!) design, however, no specific physical requirements for movements exist. Electronic sound production and sensing systems expand the possibilities for performance movement far beyond that typically associated with acoustic instruments. This is indicated by the OM! design community's focus on sound synthesis and sound-gesture mapping; little attention is given to movement qualities of the performance interaction. This thesis seeks to redress this imbalance, by developing and testing a coherent method for installing bodily movement in OM! designs. Upon considering existing frameworks for description of human movement, both generally and in musical performance specifically, Laban Movement Analysis (LMA) was identified as a suitable method for exploring and designing for movement. Laban 's theory of Effort was selected to observe and analyse existing Theremin performances, and, from this analysis, tested in a novel OM!, the Damper. Following this, further re-iterations of LMA observation and analysis were carried out to strengthen this descriptive method. From these initial studies, and existing design theory, a formal movement-based OM! design process was constructed, and implemented in the design of another novel OM! interface, the Twister. This interface was designed to the specific quality of movement, Carving, as defined by LMA. An observational analysis showed that naIve users did respond to the device with the intended movement qualities. This thesis therefore provides a procedural framework with which to design for movement in DMls, and initial testing indicates that it is indeed possible to design DM! interfaces that invite desired movement qualities

In acoustic instruments, physical properties of the instrument determine both the gestures that can be performed on it and the sound produced by it. This is not the case with digital musical instruments (DMIs) that consist of three distinct parts: the playable interface, the sound generating component, and the mapping between them. The implication is that mapping from sensors to sound is an integral part in the design of a DMI, and a process highly influential on how the instrument is played and sounds. Creating mappings in a DMI is a non-trivial task, and typically a lot of different mappings are explored throughout the stages of prototyping, composition, and production. Furthermore, production environments involving DMIs will typically bring together engineers, composers, and performers in valuable and often short meeting times, requiring configuration and visualization of the mapping layer to be quick and simple, catering to both technical and nontechnical participants. This thesis presents two software components developed at the Input Devices and Music Interaction Laboratory to aid in the mapping process: libmapper, a software library enabling connections to be made between data signals declared on a shared network, and Webmapper, a list-based graphical user interface to libmapper. An extension to Webmapper was created to address the following two concerns: to provide alternate interfaces for configuring mappings, and to provide an alternate visualizations of the mapping layer. Two new interfaces were created to investigate the tasks: a grid inspired view and a hive plot inspired view. The system was developed using HTML5 compliant technologies, and a framework architecture inspired by the model-view-controller paradigm was added to Webmapper for code modularity and maintainability. Advantages and disadvantages of the three different interfaces from Webmapper and its new extension are discussed in regards to their ability to act both as a software user interface and a data visualization tool. The contribution of the work is the demonstration of benefits to alternate methods for configuring and visualizing the mapping layer in DMIs, and the laying of a foundation for future investigations using the created HTML5 compliant software.
Dans le monde des instruments acoustiques, les propriétés physiques des instruments déterminent les gestes effectués ainsi que les sons produits; ce n'est pas le cas avec les Instruments de Musique Numériques (IMNs) constitués trois parties distinctes: une interface de contrôle, un système de production sonore et le système de correspondance entre les deux ("mapping"). Ceci implique que l'interaction entre les capteurs et le son fait partie intégrante de l'IMN et influence fortement comment l'instrument sonne et joue. La création de correspondance dans un IMN n'est pas une tâche triviale, et généralement l'exploration de différentes correspondances est nécessaire durant les phases de prototype, composition et production. De plus, les environnement de productions incluant l'utilisation d'IMN peuvent créer des opportunités de rencontre entre les ingénieurs, les compositeurs et les interprètes car la configuration et visualisation des correspondances, qui se doit d'être simple et rapide, requiert un mélange de savoir-faire technique et non-technique. Cette thèse est basée sur deux logiciels développés au sein du laboratoire IDMIL (Input Devices and Music Interaction Laboratory) aidant le processus de correspondance: Libmapper, une logithèque permettant de connecter des signaux numériques et Webmapper, une interface graphique pour Libmapper. Une extension de Webmapper a également été crée afin d'adresser les deux problèmes suivants: pour offrir une interface alternative de configuration de correspondance et pour offrir une visualisation alternative des couches de correspondance. Deux nouvelles interfaces ont également été crées afin d'ajouter deux nouvelles vues: une en forme de ruche et une en quadrillée. Le système a été développé en HTML5, utilisant une architecture inspiré par le patron modèle-vue-contrôleur. Les avantages et les désavantages des trois différentes interfaces de Webmapper ainsi que de sa nouvelle extension seront traités vis à vis de leur capacité d'agir en tant qu'interface et en tant qu'outil de visualisation. Les contributions de ce travail résident dans la démonstration des avantages des méthodes alternatives de configuration et de visualisation des couches de correspondances dans les IMN, ainsi que dans les fondations pour de futures recherches utilisant le logiciel HTML5.

Digital Music Instruments (DMIs) have become popular both in research laboratories and among performers of experimental music scenes. The concept of DMIs is very attractive, however few instruments make it beyond a laboratory environment. There is an apparent lack of historical and musical background enjoyed by acoustic instruments as well as a lack of pedagogy used to evaluate the effectiveness of DMIs and DMI players. While initial evaluation methods can be borrowed from the field of Human Computer Interaction (HCI), these methods need to be extended to evaluate the DMIs' integrated layers: the physical interface, the sound synthesizer and the abstract mapping block. This research was designed around the evaluation of a novel DMI called the "Ballagumi" with two main objectives: firstly determine the extent to which it is possible to obtain skills on the Ballagumi in a musical context (using musical compositions as tasks along with an improvisation setting) and secondly to determine how much the design of the mapping block affects the acquisition of skill on the instrument. For this purpose, two studies were conducted, a preliminary study that focused on discovering gestures on the Ballagumi and obtaining an initial perspective of the instrument features (e.g. latency), and a second study where professional musicians experienced in electronic music controllers were invited to play musical tasks and also improvise on the instrument with both mappings. The study results revealed interesting insights on the instrument itself, namely that the physical interface allowed for much of the learning in the interaction as it is built with material that have passive haptic feedback. The results also showed that of the two designed mappings, the simpler choice (a direct energy input mapping) was preferred for improvisation by participants. The choice of mapping resulted in very different impressions of the performance even though the instrument and the sound synthesis layer remained constant. Manipulating the instrument on an intuitive level was made easier by a direct energy input mapping since the physical interfaced required energy input and made use of both hands. The choice of mapping also affected the extent to which participants enjoyed playing the Ballagumi. The results from these two studies can be used to further improve the Ballagumi's sound and physical attributes to have it be used by more musicians. They also not only demonstrate the importance of the mapping block in acquiring skills to the point of improvising on the DMI (confirming in this regard past literature on the subject) but highlight the importance of accounting for the existing feedback before adding levels of complexity to the mapping.
Les instruments de musique numérique (DMIs) sont de plus en plus populaires dans les laboratoires de recherche ainsi qu'auprès des artistes de la scène musicale expérimentale. Le concept du DMI est très attrayant, cependant peu d'instruments vont au delà de la recherche en laboratoire. Au contraire des instruments acoustiques, les DMIs sont dépourvus de contexte historique ou musical et il n'existe pas de pédagogie permettant d'évaluer l'intéraction avec de tels instruments. Il est possible d'emprunter les méthodes d'évaluation au domaine de l'interaction homme-machine (IHM). Cependant, pour améliorer leur pertinence dans le contexte de 'interaction musicale, ces dernières doivent être enrichies pour intégrer les trois points suivants: l'interface physique, le synthétiseur de son et la partie intermédiaire de mapping. Cette recherche a été conçue autour de l'évaluation d'un DMI appelé "Ballagumi" avec deux objectifs principaux: tout d'abord déterminer dans quelle mesure il est possible d'acquérir des compétences musicales dans la pratique du Ballagumi, en concevant des études qui utilisent des compositions musicales et un cadre d'improvisation; puis déterminer comment la conception du mapping affecte l'acquisition de ces compétences. Pour cela, deux études ont été menées: une étude portée sur la découverte des gestes ainsi que sur l'influence d'aspects purement technologique (par exemple, la latence) sur la perception de l'instrument; et une seconde étude où des musiciens professionnels, familiers avec les instruments de musique électronique, étaient invités à interpréter des morceaux de musique composés pour l'instrument et aussi à improviser avec deux mappings. Les résultats de l'étude ont révélé un aspect intéressant de l'instrument, à savoir que l'interface elle-même donnait du retour sur l'interaction puisqu'elle possède des qualités haptiques propres. Par conséquent, des deux mappings conçus, le plus simple (un mapping à entrée d'énergie isotonique) s'est vu être privilégié par les participants, car perçu comme plus intuitif, pour l'improvisation plutôt qu'un mapping à entrée d'énergie continue. On a pu aussi noté que, bien que l'instrument et l'algorithme de synthèse sonore soient restés inchangés, le choix du mapping a donné lieu à des impressions clairement différentes pour les interprètes. Les résultats de ces deux études peuvent être utilisés pour améliorer le Ballagumi sur le plan technologique, ainsi qu'au niveau de l'esthétique de l'interface. En plus de démontrer l'importance du mapping pour acquérir des compétences permettant d'improviser avec l'instrument, ces expériences ont permis de montrer qu'il est pertinent de prendre en compte les retours d'informations propres à l'interface physique avant d'avoir recours à des configurations de mapping complexes.

Les ordinateurs et instruments numériques sont-ils des instruments de musique ? Selon des définitions traditionnelles, la réponse est négative. Pourtant, de nombreux outils numériques sont aujourd’hui utilisés pour jouer de la musique. L’objectif de cette thèse est d’approfondir, au moyen d’une enquête ontologique et d’une réflexion esthétique, le statut de ce genre d’instruments et des performances qu’ils mettent en œuvre. Nous étudions des systèmes musicaux caractérisés par l’élaboration numérique et algorithmique du son : des instruments à interfaces gestuelles, des contrôleurs MIDI, des logiciels de création musicale, des performances d’orchestres d’ordinateurs et de live coding. Nous cherchons à montrer que, sous certaines conditions, il est possible de les considérer comme de véritables instruments de musique. Nous explorons les dynamiques qu’ils font émerger ainsi que leurs productions : des œuvres musicales particulièrement orientées vers la performance, proposant une expérience sonore mais aussi fortement visuelle. Par leur rapport au temps et à l’espace de la scène, les instruments numériques sont intimement liés à leurs performances
Can we claim that computers and digital instruments are real musical instruments ? According to classical definitions, we cannot. However, nowadays many digital systems are being used to play music, calling for a reconsideration of the classical definitions of a musical instrument. This thesis investigates the ontological status of digital instruments and the aesthetics offered by them in music performances. We examine digital systems involving digital and algorithmic sound generation, including instruments with gestural interfaces, MIDI controllers, music software, laptop orchestra performances, and live coding. We seek to determine the conditions under which digital instruments can be considered real musical instruments. We explore the new dynamics emerging through the use of these new instruments. Digital instruments lead to the production of musical works particularly oriented towards performances, in which the visual component is an integral part of the experience. Digital instruments relate to temporality and space on the stage in a novel way, and there is a tight connection between digital instruments and the performances produced with them

The socio-musical practice of sampling is closely associated with the re-use of pre-existing sound recordings and the technological processes of looping. These practices, based on appropriation and repetition, have been particularly common within the genres of hip-hop and Electronic Dance Music (EDM). Yet early digital sampling instruments such as the Fairlight Computer Musical Instrument (CMI) were not designed for these purposes. The technologists at Fairlight Instruments in Australia were primarily interested in the use of digital synthesis to imitate the sounds of acoustic instruments; sampling was a secondary concern. In the first half of the thesis, I follow digital sampling instruments like the Fairlight CMI and the E-mu Emulator by drawing on interviews with their designers and users to trace how they were used to sample the sounds of everyday life, loop sequenced patterns of sampled sounds, and sample extracts from pre-existing sound recordings. The second half of the thesis consists of case studies that follow the users of digital sampling technologies across a range of socio-musical worlds to examine the diversity of contemporary sampling practices. Using concepts from the field of Science and Technology Studies (STS), this thesis focuses on the ‘user-technology nexus’ and continues a shift in the writing of histories of technologies from a focus on the designers of technologies towards the contexts of use and ‘the co-construction’ or ‘mutual shaping’ of technologies and their users. As an example of the ‘interpretative flexibility’ of music technologies, digital sampling technologies were used in ways unimagined by their designers and sampling became synonymous with re-appropriation. My argument is that a history of digital sampling technologies needs to be a history of both the designers and the users of digital sampling technologies.

In order to further investigate the efficacy of interactive musical toys developed by students in the course DM2799 at KTH Royal Institute of Technology, we conducted a user study. The intended users of the toys are primary school children with special needs. The trials were held at the Rullen Dibber special needs school in Solna, Sweden. The study was conducted in order to be able to further develop the instruments and to better understand the needs and wants of the intended users. In order to do this, we studied the emergence of positive behaviors linked to musical expression. The study concluded that although the instruments did not result in any significant change in togetherness and nonverbal communication in the children during playing with the instruments, they were observed to be an effective tool in music therapy.
För att närmare undersöka inverkan av interaktiva musikaliska leksaker utvecklade av studenter i kursen DM2799 på KTH så har vi utfört en utförlig användarstudie. Den avsedda användaren av leksakerna är primärt barn i skolålder med funktionsvariationer. Försöken utfördes på Rullen Dibber särskola i Solna. Studien utfördes för att vidareutveckla instrumenten och att bättre förstå behoven hos den avsedda användaren. För att göra detta undersökte vi framväxten av positiva beteende kopplade till musikaliskt uttryck. Studien visar att trots att instrumenten inte påvisar någon ökad gemenskap bland barnen är de effektiva verktyg för musikterapi.

Os avanços tecnológicos têm refletido não somente no modo de aprender e ensinar, mas também nas práticas pedagógicas e nas arquiteturas para o contexto educacional. Neste panorama, cresce a busca pela interdisciplinaridade, associada a uma visão holística do indivíduo, que leva em conta não somente a cognição, mas também os aspectos sociais e afetivos dos alunos. Diante desta realidade, percebe-se a necessidade de se construir e aplicar arquiteturas pedagógicas voltadas a esses aspectos, que englobem recursos tecnológicos tais como o computador e a construção de interfaces tangíveis pelos estudantes, a exemplo dos instrumentos musicais digitais (IMDs). Juntamente com esses, ampliaram-se as possibilidades da utilização da música no contexto educacional. Compreende-se que uma, dentre as características do público adolescente, é a busca pelo sentimento de pertença a um grupo. Esta pode ocorrer através da participação em diversas atividades voltadas para a expressão sonora e/ou musical através de tecnologias, tais como os IMDs. Perante essas premissas, esta pesquisa tem o objetivo de analisar como uma Arquitetura Pedagógica (AP) voltada para a construção e utilização de instrumentos musicais digitais (IMDs) pode contribuir para fomentar aspectos socioafetivos em estudantes adolescentes não musicistas. Para isso, o presente estudo fundamenta-se numa abordagem qualitativa. A coleta de dados foi realizada a partir de observações, questionários, vídeos, produções tecnológicas e registros sobre as interações sociais e os estados de ânimo dos estudantes. Entende-se que as contribuições desta investigação podem implicar em mudanças quanto ao design dos espaços de aprendizagem relacionados às arquiteturas pedagógicas para a construção e uso de protótipos físico-sonoros e/ou instrumentos musicais digitais com finalidade educacional.
Technological advances have reflected not only on the way of learning and teaching but also on pedagogical practices and architectures for the educational context. In this panorama, the search for interdisciplinarity, coupled with a holistic view of the individual, considers not only the cognition but also the social and affective aspects of the students. Faced with this reality, one can see the need to construct and apply pedagogical architectures focused on these aspects, which encompass technological resources such as the computer and the construction of tangible interfaces by students, such as digital musical instruments (DMIs). Along with these, the possibilities of the use of music in the educational context were extended. It is understood that one of the characteristics of the adolescent public is the search for the feeling of belonging to a group. This can occur through participation in various activities focused on sound and/or musical expression through technologies such as DMIs. Given these premises, this research has the goal of analyzing how a Pedagogical Architecture (PA) aimed at the construction and use of digital musical instruments (DMIs) can contribute to foster socio-affective aspects in non-musician adolescent students. For this, the present study is based on a qualitative approach. The data collection was made from observations, questionnaires, videos, technological productions and records about the social interactions and moods states of the students. It is understood that the contributions of this investigation may imply changes in the design of learning spaces related to pedagogical architectures for the construction and use of physical-sounding prototypes and/or digital musical instruments for educational purposes.

Thesis (D.M.A)--Ohio State University, 2004.
Title from first page of PDF file. Document formatted into pages; contains xiii, 57 p.; also includes graphics (some col.). Includes bibliographical references (p. 52-57). Available online via OhioLINK's ETD Center

Le travail de cette thèse porte sur l'étude du contrôle en temps réel de synthèse de voix chantée par une tablette graphique dans le cadre de l'instrument de musique numérique Cantor Digitalis.La pertinence de l'utilisation d'une telle interface pour le contrôle de l'intonation vocale a été traitée en premier lieu, démontrant que la tablette permet un contrôle de la hauteur mélodique plus précis que la voix réelle en situation expérimentale.Pour étendre la justesse du jeu à toutes situations, une méthode de correction dynamique de l'intonation a été développée, permettant de jouer en dessous du seuil de perception de justesse et préservant en même temps l'expressivité du musicien. Des évaluations objective et perceptive ont permis de valider l'efficacité de cette méthode.L'utilisation de nouvelles interfaces pour la musique pose la question des modalités impliquées dans le jeu de l'instrument. Une troisième étude révèle une prépondérance de la perception visuelle sur la perception auditive pour le contrôle de l'intonation, due à l'introduction d'indices visuels sur la surface de la tablette. Néanmoins, celle-ci est compensée par l'important pouvoir expressif de l'interface.En effet, la maîtrise de l'écriture ou du dessin dès l'enfance permet l'acquisition rapide d'un contrôle expert de l'instrument. Pour formaliser ce contrôle, nous proposons une suite de gestes adaptés à différents effets musicaux rencontrés dans la musique vocale. Enfin, une pratique intensive de l'instrument est réalisée au sein de l'ensemble Chorus Digitalis à des fins de test et de diffusion. Un travail de recherche artistique est conduit tant dans la mise en scène que dans le choix du répertoire musical à associer à l'instrument. De plus, un retour visuel dédié au public a été développé, afin d'aider à la compréhension du maniement de l'instrument
This thesis deals with the real-time control of singing voice synthesis by a graphic tablet, based on the digital musical instrument Cantor Digitalis.The relevance of the graphic tablet for the intonation control is first considered, showing that the tablet provides a more precise pitch control than real voice in experimental conditions.To extend the accuracy of control to any situation, a dynamic pitch warping method for intonation correction is developed. It enables to play under the pitch perception limens preserving at the same time the musician's expressivity. Objective and perceptive evaluations validate the method efficiency.The use of new interfaces for musical expression raises the question of the modalities implied in the playing of the instrument. A third study reveals a preponderance of the visual modality over the auditive perception for the intonation control, due to the introduction of visual clues on the tablet surface. Nevertheless, this is compensated by the expressivity allowed by the interface.The writing or drawing ability acquired since early childhood enables a quick acquisition of an expert control of the instrument. An ensemble of gestures dedicated to the control of different vocal effects is suggested.Finally, an intensive practice of the instrument is made through the Chorus Digitalis ensemble, to test and promote our work. An artistic research has been conducted for the choice of the Cantor Digitalis' musical repertoire. Moreover, a visual feedback dedicated to the audience has been developed, extending the perception of the players' pitch and articulation

Interface metaphors are often used in Human Computer Interaction (HCI) to exploit knowledge that users already have from other domains. A commonly used one in Digital Musical Instruments (DMIs) is the conductor metaphor. We theorize that part of the knowledge that users have from the domain that the interface metaphor replicates is user-specific. In this context, we argue that systems using an interface metaphor can see their usability improved by adapting to this user-specific knowledge. We propose strategies to design motion-sound mappings for DMIs that draw upon the conductor metaphor by adapting to personal nuances that can be automatically computed from spontaneous conducting movements. In addition to the above, we study the potential of the conductor metaphor in a gaming context as a mean to raise interest for classical music. These contributions are complemented by the data collected in all observational studies, which is made publicly available to the community
Las metáforas de interfaz se utilizan habitualmente en la interacción persona-ordenador para explotar el conocimiento que los usuarios tienen de otros dominios. Una comúnmente utilizada en instrumentos musicales digitales (DMIs) es la metáfora del director. Argumentamos que parte del conocimiento que los usuarios tienen del dominio que la metáfora replica es específico de cada usuario. En este contexto, proponemos que sistemas que hacen uso de metáforas de interfaz pueden mejorar su usabilidad adaptándose a este conocimiento específico. Implementamos estrategias para diseñar mapeos entre movimiento y sonido para DMIs basados en la metáfora del director, mediante la adaptación a matices personales que pueden ser automaticamente extraídos analizando movimientos de dirección espontáneos. Además, estudiamos el potencial de la metáfora del director en un contexto de juego, como medio para incrementar el interés por la música clásica. Estas contribuciones se complementan con los datos recopilados en los estudios observacionales, que se ponen a disposicion pública para la comunidad.

Les instruments de musique numériques se présentent comme des objets complexes, qui se situent à la fois dans une continuité historique avec l'histoire de la lutherie tout en étant marqués par une rupture forte provoquée par le numérique et ses conséquences en terme de possibilités sonores, de relations entre le geste et le son, de situations d'écoute, de re-configurabilité des instruments, etc. Ce travail de doctorat propose une analyse des caractéristiques émanant de l'intégration du numérique dans les instruments de musique, en s'appuyant notamment sur une réflexion musicologique, sur des développements logiciels et matériels et sur une pratique musicale, ainsi que sur des échanges avec d'autres musiciens, luthiers, compositeurs et chercheurs
Digital musical instruments appear as complex objects, being positioned in a continuum with the history of lutherie as well as marked with a strong disruption provoked by the digital technology and its consequences in terms of sonic possibilities, relations between gesture and sound, listening situations, reconfigurability of instruments and so on. This doctoral work tries to describe the characteristics originating from the integration of digital technology into musical instruments, drawing notably on a musicological reflection, on softwares and hardwares development, on musical practice, as well as a number of interactions with other musicians, instruments makers, composers and researchers

Interactive installations in public settings have increased in popularity over the past decade, as well as the construction of digital musical instruments. In this paper I present a study of the interaction with a large sized augmented string instrument intended for a large installation in a museum, with focus on encouraging creativity, learning, and providing engaging user experiences. In the study, 9 participants were video recorded while playing with the string on their own, followed by an interview focusing on their experiences, creativity, and the functionality of the string. I then used McCarthy and Wright’s framework for analysing technology as experience and Frank E Williams creativity taxonomy model to analyse the results. In line with previous research, results highlight the importance of designing for different levels of engagement (exploration, experimentation, challenge). However, results additionally show that these levels need to consider the users’ age and musical background as these profoundly affect the way the user plays with and experiences the string.
Interaktiva installationer i offentliga miljöer har ökat i popularitet under det senaste decenniet, liksom även skapandet av digitala musikinstrument. I denna uppsats presenterar jag en studie av interaktionen med ett stort augmenterat stränginstrument avsedd för en stor installation i ett museum, med fokus på att uppmuntra kreativitet, lärande, och att ge engagerande användarupplevelser. I studien blev 9 deltagare videoinspelade samtidigt som de spelade med strängen på egen hand, följt av en intervju med fokus på deras upplevelse, kreativitet, och strängens funktionalitet Jag använde sedan McCarthy och Wrights ramverk för att analysera teknik som upplevelse och Frank E Williams kreativitets taxonomi för att analysera resultaten. I linje med tidigare forskning så betonar resultaten vikten av att designa för olika nivåer av engagemang (undersökande, experimenterande, utmaning). Dock så visar resultaten dessutom på att dessa nivåer måste ta hänsyn till användarnas ålder och musikaliska bakgrund då dessa starkt påverkar hur användaren spelar med och upplever strängen.

Orientadores: Rafael Santos Mendes, Ricardo Goldemberg
Tese (doutorado) - Universidade Estadual de Campinas, Faculdade de Engenharia Elétrica e de Computação
Made available in DSpace on 2018-08-18T23:37:16Z (GMT). No. of bitstreams: 1 Oliveira_LuisCarlosde_D.pdf: 5521873 bytes, checksum: 20aeecff279d6a276f652e26512c7903 (MD5) Previous issue date: 2011
Resumo: Este trabalho aplica a técnica do guia de ondas digital ("Digital Waveguide") na síntese de instrumentos de sopro da família das madeiras, em particular, da clarineta. Esta técnica foi desenvolvida na década de 1980 e, desde então, tem sido utilizada em síntese de instrumentosmusicais por modelagem física. A técnica consiste, grosso modo, no processo de discretização da solução por ondas viajantes da Equação da Onda. Este procedimento permite obter uma estrutura de filtros digitais que simula a propagação da onda sonora para a direita e para a esquerda com velocidade de propagação equivalente à velocidade do som, c. No entanto, nos resultados de modelagem física publicados, constatou-se a ausência da velocidade de escoamento do ar, u. A contribuição deste trabalho consiste em reelaborar a estrutura de filtros digitais de modo a contemplar a velocidade de escoamento do fluido. A fundamentação teórica estabelece que para um meio de propagação sonora em movimento as velocidades de propagação para a direita é c + u e para a esquerda é c ? u. Propõe-se então, uma estratificação do tubo da clarineta em camadas divididas por mesma área ou mesma largura de modo a verificar a influência do perfil de velocidades na síntese sonora. Foi observado que a inclusão do escoamento implica uma "distorção" na curva da pressão interna do tubo, o que aproxima essa forma dos resultados experimentais
Abstract: This study aims to apply the technique of Digital Waveguide in synthesis of woodwind instruments, particularly the clarinet. This technique was developed in the mid-80 and since then, has application in the synthesis of musical instruments by physical modeling. It consists, roughly, in the process of discretization of the traveling wave solution of the Wave Equation. This leads to a digital filter structure that simulates the propagation of sound waves to the right and left with the speed of sound propagation, c. However, in the published results of physical modeling, the air flow velocity, u, was not included. The contribution of this work is to redesign the structure of digital filters in order to contemplate the velocity of the fluid. The theoretical framework provides that in a medium in motion the velocities of propagation to the right is c + u and to the left is c ? u. It is proposed then a stratification of the tube of the clarinet in layers divided by the same area or same width in order to verify the influence of velocity profile in the synthesized sound. It was observed that the inclusion of flow provides a "distortion" in the internal pressure of the tube. It sets resemblance to experimental results
Doutorado
Engenharia de Computação
Doutor em Engenharia Elétrica

Cette thèse en recherche-création est centrée autour de l’exploration des besoins des musiciens dans le cadre de la pratique de l’improvisation musicale électroacoustique. Cette réflexion a été menée à partir d’un travail d’observation participante au sein de différents ensembles musicaux. Elle vise aussi à présenter les stratégies adoptées dans le développement des outils numériques d’un instrument de notre création. Comment la pratique de l’improvisation libre dans la musique électroacoustique fonctionne-t-elle et quelles sont les connaissances nécessaires pour la réaliser ? Pour aborder cela, cette thèse explore plusieurs pistes : d’abord en considérant la situation de la performance elle-même, et en interrogeant les raisons musicales et extramusicales qui entourent une action improvisée. Ensuite, cette recherche s’articule autour de l’étude des caractéristiques d’un instrument numérique, dont l’aspiration consiste à détenir la même souplesse que les instruments acoustiques. Le volet « création » de cette thèse se décline à travers le développement d’un instrument numérique qui est notamment doté d’un système de recommandation des échantillons sonores. Il est utilisé dans des situations d’improvisation avec plusieurs ensembles, qui constituent le corpus des données de ce travail. Pour finir, cette thèse cherche à caractériser l’improvisation musicale à travers les nombreuses interactions qui se produisent lors d’une performance. La recherche a donc pour objectif d’offrir un cadre, pratique et théorique, pour l’utilisation et pour l’exploration des outils numériques contemporains à destination des musiciens qui pratiquent l’improvisation libre collective
This research/creation thesis is centred on the exploration of musicians’ needs for the practice of electroacoustic music improvisation. This investigation was conducted through participant observation with different musical ensembles. It also presents the strategies adopted for the development of digital tools in the case of a created musical instrument. How does the practice of free improvisation in electroacoustic music work and what knowledge is needed to achieve free improvisation? To answer these questions, several paths were taken: for example considering the context of the performance and by challenging the musical and extramusical aspects surrounding the improvised action. Furthermore, this research deployed a study of the characteristics of a digital instrument that endeavours to possess the same flexibility as acoustic instruments. The “creation” component of this thesis is through the development of a digital instrument (RJ). This instrument is implemented with a sound samples recommendation system. It was used in situation of improvisation with several ensembles, which form part of the corpus of data of this thesis. Lastly, this thesis seeks to characterize musical improvisation through the multiple interactions that occur during an improvised performance. The research therefore provides a practical and theoretical framework for the use and exploration of contemporary digital tools for musicians, which practice collective free improvisation

Aquesta tesi s'ha desenvolupat entorn les repercussions i oportunitats creades per la digitalització, accessibilitat en línia al patrimoni cultural i la preservació digital, doncs el llenguatge hipermèdia i els nous fluxos de comunicació han atorgat una nova dimensió als sistemes de comunicació i documentació i a les estratègies i programes de gestió, preservació, estudi, difusió i accés als béns culturals. L'estudi de cas són les col•leccions europees d'instruments musicals en línia. Hem pogut determinar les tendències de futur en relació a finalitats i objectius de la documentació i digitalització del patrimoni cultural, en l'àmbit de la Unió Europea, amb l'anàlisi de llocs web de les col•leccions europees d'instruments musicals en línia i amb l'anàlisi de les polítiques de digitalització. L'estudi de cas ens ha permès aportar una visió innovadora i multidisciplinària, de confluència entre els drets culturals, les Ciències de la Documentació i les metodologies i conceptes de la Museologia. Amb aquesta tesi defensem que l'accessibilitat d'estructures del coneixement, com les col•leccions en línia, és una de les garanties per a l'autonomia de les persones i el progrés de les societats.
This thesis has been developed around the impacts and opportunities created from the digitisation and online accessibility of cultural heritage and digital preservation, seeing how the language hypermedia and new communication flows have given a new dimension to communication systems and documentation strategies as well as programs management, preservation, study, dissemination and access to cultural heritage. The case study is the european online musical instruments collections. We have been able to determine future trends regarding aims and objectives of the documentation and digitisation of cultural heritage in the European Union, with the analysis of the wesites of the european online musical instruments collections and the analysis of the digitisation policies. The case study has allowed us to provided an innovative and multidisciplinary vision of convergence between the cultural rights, Information and Communication Sciences and Museology methodologies and concepts. This thesis is to defend the accessibility of knowledge structures, such as collective Collections Online, which is one of the guarantees for the independence of the people and the progress of societies.

L’évolution lexicale qui tend à remplacer les termes d’« écriture » ou de « composition » par celui de « création musicale » ou d’« art sonore », est un indice parmi d’autres des mutations profondes qui accompagnent l’activité du compositeur et de la difficulté à identifier nombre d’œuvres récentes comme émanant d’un processus d’écriture. Ces mutations sont évidemment corrélées à l’évolution des technologies au cours du dernier siècle, et surtout à celle de l’informatique et des outils numériques, évolution qui a fait éclater le lien de l’œuvre musicale aux catégories traditionnelles telles que la « partition », l’« instrument » ou l’« interprète », devenues ainsi parfois, à leur tour, l’indice d’une pensée musicale témoignant d’une époque révolue. Notre thèse soutient que l’existence de ces catégories demeure néanmoins fondamentale pour comprendre les enjeux de la composition, même dans les œuvres où elles semblent absentes. En effet, une des plus importantes conséquences des transformations induites par les nouvelles technologies est l’éclatement des catégories traditionnelles en une grande diversité d’objets techniques dont la mise en relation peut être assimilée à une écriture. Abordée du point de vue de l’évolution des technologies musicales, chaque œuvre apparaît alors comme un agencement, une nouvelle forme de « concrétisation », pour reprendre un terme du philosophe Gilbert Simondon, des fonctions musicales portées par des supports techniques, et c’est justement de la variété multiforme de ces agencements dont témoigne la création musicale contemporaine
The lexical evolution that tends to replace the terms "musical writing" or "composition" with that of “musical creation” or “sound art”, is an indication among others of the profound changes that the act of composition has faced, and of the difficulty of identifying many recent works as being created through a process of writing. These changes are obviously tied to evolutions in technology over the last century, especially pertaining to computing and digital tools – evolutions that have broken the link between musical work and traditional categories such as “score”, "instrument" or "performer", themselves sometimes seen to represent musical thought from a bygone era. Our thesis maintains however that the existence of these categories remains fundamental to understanding the concerns of composition, even in works where they seem to be absent. Indeed, one of the most important consequences of the transformations induced by new technologies is that traditional categories have been shattered into a multiplicity of technical objects whose interconnection can be considered as a form of writing. Discussed from the point of view of the evolution of musical technologies, each work appears as an “assemblage” (in the sense of “arrangement”), a new form of "concretization", to use Gilbert Simondon's term, of musical functions borne of technical means. Contemporary musical production bears witness to the immense variety of these assemblages

碩士
國立交通大學
工學院聲音與音樂創意科技碩士學位學程
103
Abstract According to Heidegger philosophy of technology presented, this thesis criticizes that Digital Musical Instruments (DMIs) cannot be merely restricted to explore the field of an instrumental perspective. Instead, it should reveal the phenomenon of being as ‘the creation of poetic meaning’ to be existed, so as to be tilted toward the practice and creation of DMIs. In the field of HCI, there is an influential essay ‘The three paradigm of HCI’ presented by Harrison et al. in 2007, which is also perceived as a research framework in this thesis. In addition, the third paradigm (phenomenology) is further traced back to the text of phenomenology to shed light on what the nature of DMIs is and to elaborate on the relationships between instruments, human beings, and phenomena via the music works 《4’33”》created by John Cage. Finally, the object-oriented concept of P. Dourish is applied to link the design thinking of DMIs; the phenomenon of being, intentionality, inter-subjectivity, coupling, and metaphor are explored by a preliminary methodology in this thesis. Through the perspective of phenomenology, DMIs open the diversified possibility in the structural sense of the universality of so-called ‘Being-in-the-world’ contended by Heidegger. Under this point of view, technology is not only the dimension of the technology, but also further closely associated with life to embody the reason and meaning of technology. The concept of body-subject in the phenomenology of perception proposed by Merleau-Ponty will be delineated in this thesis as well. Furthermore, John Cage’s 《4’33”》will be further elaborated on the state of Merleau-Ponty’s body-subject co-existence to understand performers who are a synthesis of music of art to be performed, instead of the manipulators separating the Cartesian instrumental rationality of the technology and music. In other words, the nature of ‘technology’ of DMIs is developed from the interaction between body-subject and instruments to present the meaning and culture. Keyword: Phenomenology, Being-in-the-world, Body, Instrumental rationality, Technology, Digital Musical Instruments

This thesis concerns the making and performing of music with new digital musical instruments (DMIs) designed for ensemble performance. While computer music has advanced to the point where a huge variety of digital instruments are common in educational, recreational, and professional music-making, these instruments rarely seek to enhance the ensemble context in which they are used. Interaction models that map individual gestures to sound have been previously studied, but the interactions of ensembles within these models are not well understood. In this research, new ensemble-focussed instruments have been designed and deployed in an ongoing artistic practice. These instruments have also been evaluated to find out whether, and if so how, they affect the ensembles and music that is made with them. Throughout this thesis, six ensemble-focussed DMIs are introduced for mobile touch-screen computers. A series of improvised rehearsals and performances leads to the identification of a vocabulary of continuous performative touch-gestures and a system for tracking these collaborative performances in real time using tools from machine learning. The tracking system is posed as an intelligent agent that can continually analyse the gestural states of performers, and trigger a response in the performers' user interfaces at appropriate moments. The hypothesis is that the agent interaction and UI response can enhance improvised performances, allowing performers to better explore creative interactions with each other, produce better music, and have a more enjoyable experience. Two formal studies are described where participants rate their perceptions of improvised performances with a variety of designs for agent-app interaction. The first, with three expert performers, informed refinements for a set of apps. The most successful interface was redesigned and investigated further in a second study with 16 non-expert participants. In the final interface, each performer freely improvised with a limited number of notes; at moments of peak gestural change, the agent presented users with the opportunity to try different notes. This interface is shown to produce performances that are longer, as well as demonstrate improved perceptions of musical structure, group interaction, enjoyment and overall quality. Overall, this research examined ensemble DMI performance in unprecedented scope and detail, with more than 150 interaction sessions recorded. Informed by the results of lab and field studies using quantitative and qualitative methods, four generations of ensemble-focussed interface have been developed and refined. The results of the most recent studies assure us that the intelligent agent interaction does enhance improvised performances.

Site-specific art understands that the place where the artwork is presented cannot be excluded from the artwork itself. The completion of the work is only achieved when the artwork and place intersect. Acoustically, sound presents a natural relation with place. The perception of sound is the result of place modulation on its spectral content, likewise perception of place is dependent on the sound content of that place. Even so, the number of sound artworks where place has a primary role is still very reduced. We thus purpose to create a tool to compose inherently place-specific sounds. Inherently because the sound is the result of the interaction between place and performer. Place because is the concept that is closer to human perception and of the idea of intimacy. Along this thesis we suggest that this interaction can be mediated by a digital musical instrument - Intonaspacio, that allows the performer to compose place-specific sounds and control it. In the rst part we describe the process of construction and design of Intonaspacio - how to access the sound present in the place, what gestures to measure, what sensors to use and where to place them, what mapping to design in order to compose place-speci c sound. We start by suggesting two di erent mappings to combine place and sound, where we look at di erent approaches on how to excite the structural sound of the place, i.e., the resonant frequencies. The rst one, uses a process where the performer can record a sample of sound ambiance and reproduce it, creating a feedback loop that excites at each iteration the resonances of the room. The second approach suggest a method where the input sound is analyzed and an ensemble of the frequencies of the place with the highest amplitudes is extracted. These are mapped to control several parameters of sound e ects. To evaluate Intonaspacio we conducted an experiment with participants who played the instrument during several trial sessions. The analysis of this experiment led us to propose a third mapping that combines the previous mappings. The second part of the thesis intends to create the conditions to give longevity to Intonaspacio. Starting from the premise that a musical instrument to be classi ed as such needs to have a dedicated instrumental technique and repertoire. These two conditions were achieved first, by suggesting a gestural vocabulary of the idiomatic gestures of Intonaspacio based on direct observation of the most repeated gestures of the participants of our experiment. Second, by collaborating with two composers whom wrote two pieces for Intonaspacio.
A arte situada é uma disciplina artística tradicionalmente ligada a Instalação que pretende criar obras que mantêm uma relação directa com o espaço onde são apresentadas. A obra de arte não pode assim ser separada desse mesmo espaço sem perder o significado inicial. O som pelas suas características físicas reflecte naturalmente o espaço onde foi emitido, isto é, a percepção que temos de um som resulta da combinação do som directo com as reflecções do mesmo no espaço (cujo tempo e amplitude est~ao directamente relacionados com a arquitectura do espaço). Nesta lógica a arte sonora seria aquela que mais directamente procuraria compôr som situado. No entanto, o espaço é raramente utilizado como fen omeno criativo intencional. Nesse sentido, o trabalho aqui apresentado propõem-se a investigar a possibiliade de criar sons situados. O termo Espaço está muitas vezes associado a algo de dimensões vastas e ilimitadas. Assim sendo e na óptica da arte situada, onde h a uma necessidade de criar uma relação, parece-nos que lugar é um termo mais adequado para enquadrar o nosso trabalho de investigação. O lugar, para além de representar um espaço onde se podem estabelecer relações de intimidade (proximidade), apresenta dimensões que são moldáveis consoante a percepcção e o corpo humano. Ou seja, o Homem ao deslocar-se no lugar vai ao mesmo tempo de nindo as fronteiras desse mesmo lugar. Esta visão do lugar aparece no final do século dezanove quando a filosofia começa a orientar o pensamento para uma visão mais direccionada para o Homem e para a percepcção humana. O lugar passa então a representar algo que e estabelecido na acção e pela percepcção humana, onde e possível estabelecer relações de intimidade, ao contrário dos não-lugares (sítios mais ou menos descaracterizados onde as pessoas estão só de passagem). Re-adaptámos por isso a nossa questão inicial não só para realçar esta ideia de lugar mas também para reflectir uma bi-direccionalidade perceptiva que é fulcral para a arte situada - como criar e controlar sons inerentemente localizados? Inerentemente porque para existir de facto uma interacção entre lugar e obra de arte sonora são necessárias duas condições: por um lado o som possa provocar uma resposta do lugar, e por outro, o lugar possa modificar a nossa percepcção dele mesmo. A existência de uma relação interactiva abre espaço a um novo ponto que não tinhámos considerado anteriormente e que acrescentámos a nossa nova questão, o controlo. Propomos como possível reposta a esta questão a construção de um instrumento musical digital, o Intonaspacio, que servir a de mediador desta interacção e que possibilitará ao performer a criação e o controlo de sons localizados. Primeiro poque o instrumento musical possibilita o aumento das capacidades humanas, através da extensão do corpo humano (tal como um garfo extende a nossa mão, por exemplo). Segundo, porque o instrumento musical digital pelas suas características, nomeadamente pela separação entre o sistema de controlo e o sistema de geração de som abre novas possibilidades sonoras antes excluídas por limitações mecâncias ou humanas. Podemos por isso visionar um acesso mais alargado a novas dimensões espaciais e temporais. Esta tese está dividida em duas partes, na primeira parte descrevemos a construção do Intonaspacio, e na segunda estabelecemos as bases para permitir a sua longevidade. A primeira parte começa por investigar formas de acesso ao som do lugar, composto pelo conjunto dos sons ambiente e dos sons estruturais do lugar (ressonâncias próprias resultantes da arquitectura). Pensamos que uma das possíveis formas de compôr sons localizados e precisamente através da possibilidade de poder ter os sons ambiente a gerar e a amplificar os sons estruturais. Surgem então duas novas questões de natureza técnica: Como integrar o som ambiente na obra sonora em tempo-real? Como permitir que estes excitem a resposta do espaço? Para as responder desenhámos dois mapeamentos diferentes. Um primeiro em que o performer pode gravar pequenos trechos de som ambiente que são emitidos e re-gravados criando um ciclo de feedback que excita as ressonâncias do lugar. Um segundo método onde se faz uma análise espectral ao som captado e se extrai um conjunto de frequências cujas amplitudes são as mais elevadas. Estas são posteriormente utilizadas para controlar parâmetros de vários efeitos sonoros. Colocámos ainda no instrumento um conjunto de sensores diferentes para captar o gesto do performer. Estes estão localizados em diferentes areas do esqueleto do instrumento de modo a permitir areas sensíveis maiores e consequentemente um maior n umero de graus de liberdade ao performer. Neste momento o Intonaspacio permite extrair cerca de 17 características diferentes, agrupadas em três secções - orientação, impacto e distância. Estas podem ser utilizadas para modelar o som gerado pelo instrumento através dos diferentes mapeamentos. Ambas as propostas de mapeamento foram avaliadas por um conjunto de pessoas durante um teste de utilização do Intonaspacio. Os resultados deste permitiram-nos chegar a uma terceira sugestão de mapeamento onde combinamos características de ambas as propostas anteriores. No terceiro mapeamento mantém-se a análise ao som captado pelo instrumento mas a informação recolhida e usada como material sonoro de um algoritmo de síntese aditiva. A segunda parte da tese parte de uma premissa estabelecida durante o trabalho realizado nesta tese. Um instrumento musical deve possuir uma técnica instrumental própria e um repertório dedicado para que seja considerado enquanto tal. Neste sentido e com base na observação directa dos gestos mais comuns entre participantes do nosso estudo, propusémos um vocabulário gestual dos gestos idiomáticos do Intonaspacio, ou seja, dos gestos que dependem exclusivamente da forma do próprio instrumento e da localização dos sensores na estrutura do instrumento (zonas sensíveis) e são independentes do mapeamento. Colaborámos ainda com dois compositores que escreveram duas peças musicais para o Intonaspacio. O Intonaspacio revelou ser um instrumento complexo e expressivo que possibilita aos performers incluir o lugar enquanto parâmetro criativo, no entanto apresenta ainda alguns problemas de controlo. No primeiro mapeamento, embora a integração do lugar seja sentida como mais directa e apresentando resultados sonoros mais interessantes (de acordo com os participantes do estudo), a sensação de controlo é muito baixa. Já no segundo mapeamento, embora tenha um controlo mais fácil, a presença do lugar é muito subtil e pouco perceptível. Esperamos que o terceiro mapeamento venha contribuir para solucionar este problema e aumentar o interesse no instrumento, principalmente por parte dos compositores com quem colaborámos e iremos colaborar no futuro.

Open-ended human-computer interactions, such as those in interactive digital art and music, are an increasingly popular area of study in HCI. They provide an opportunity to examine playfulness, creativity and expression and challenge conventional HCI notions of quality, evaluation and how to measure success. Jamming-improvisational group music making-is often held up as an example of open-ended creativity. This thesis describes the development of Viscotheque, an iPhone- based digital musical instrument (DMI) designed for jamming, over three major design-test cycles. Over these three iterations the interface evolved from a very simple 'process control' interface in v1 to a more expressive multi-touch sample manipulation tool in v3. At each stage of the design process, open-ended jam sessions held with local musicians suggested that the potential was there for the interface to support rich jamming experiences. Version 3 of the interface and the associated v3 jam session was the most in-depth of the three phases of the experiment, with the most expressive interface and also the most comprehensive field trial (using a multi-session longitudinal study of jamming musicians rather than the single jam sessions of v1 and v2). Situating the qualitative results of these experiments within the broader context of third wave HCI, this thesis discusses 'affect' in a guise perhaps unfamiliar to readers of mainstream HCI discourse. The jam sessions were characterised by intense sonic atmospheres, and the post-jam interviews reveal a complicated picture of agency in the interaction of the musician and their sound. The thesis also presents a detailed analysis of the quantitative log data, including the results of a Machine Learning (ML) approach to looking for patterns in this data. Finally, the thesis discusses the implications of the Viscotheque design process for HCI more broadly, including the powerful affective atmospheres which characterise musical interaction and an approach to data analysis which leverages the mathematical sophistication of modern ML techniques while remaining sensitive to the difficulties surrounding the measurement of experience. -- provided by Candidate.