Relevant bibliographies by topics / Gesture Synthesis / Journal articles
To see the other types of publications on this topic, follow the link: Gesture Synthesis.

Journal articles on the topic 'Gesture Synthesis'

Author: Grafiati
Published: 25 May 2024

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles

Select a source type:

Consult the top 50 journal articles for your research on the topic 'Gesture Synthesis.'

Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.

You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.

Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.

1

Pang, Kunkun, Dafei Qin, Yingruo Fan, Julian Habekost, Takaaki Shiratori, Junichi Yamagishi, and Taku Komura. "BodyFormer: Semantics-guided 3D Body Gesture Synthesis with Transformer." ACM Transactions on Graphics 42, no. 4 (July 26, 2023): 1–12. http://dx.doi.org/10.1145/3592456.

Full text
Abstract:
Automatic gesture synthesis from speech is a topic that has attracted researchers for applications in remote communication, video games and Metaverse. Learning the mapping between speech and 3D full-body gestures is difficult due to the stochastic nature of the problem and the lack of a rich cross-modal dataset that is needed for training. In this paper, we propose a novel transformer-based framework for automatic 3D body gesture synthesis from speech. To learn the stochastic nature of the body gesture during speech, we propose a variational transformer to effectively model a probabilistic distribution over gestures, which can produce diverse gestures during inference. Furthermore, we introduce a mode positional embedding layer to capture the different motion speeds in different speaking modes. To cope with the scarcity of data, we design an intra-modal pre-training scheme that can learn the complex mapping between the speech and the 3D gesture from a limited amount of data. Our system is trained with either the Trinity speech-gesture dataset or the Talking With Hands 16.2M dataset. The results show that our system can produce more realistic, appropriate, and diverse body gestures compared to existing state-of-the-art approaches.
APA, Harvard, Vancouver, ISO, and other styles
2

Deng, Linhai. "FPGA-based gesture recognition and voice interaction." Applied and Computational Engineering 40, no. 1 (February 21, 2024): 174–79. http://dx.doi.org/10.54254/2755-2721/40/20230646.

Full text
Abstract:
Human gestures, a fundamental trait, enable human-machine interactions and possibilities in interfaces. Amid technological advancements, gesture recognition research has gained prominence. Gesture recognition possesses merits in sample acquisition and intricate delineation. Delving into its nuances remains significant. Existing techniques leverage PC-based OpenCV and deep learnings computational prowess, showcasing complexity. This scholarly exposition outlines an experimental framework, centered on mobile FPGA for enhanced gesture recognition. The focus lies on DE2-115 as an image discernment base. A 51 microcontroller supports auditory synthesis. Specifically, this paper highlights the DE2-115 FPGA as the foundation for image discernment, while a 51 microcontroller assists in auditory synthesis. Our emphasis lies in recognizing basic gestures, particularly within the rock-paper-scissors taxonomy, to ensure precision and accuracy. This research underscores the potential of FPGA in enabling efficient gesture recognition on mobile platforms. As a result, the experiments conducted in this thesis successfully realize the recognition of simple gestures, such as the numbers 1, 2, 3, and 4, as well as rock-paper-scissors.
APA, Harvard, Vancouver, ISO, and other styles
3

Ao, Tenglong, Qingzhe Gao, Yuke Lou, Baoquan Chen, and Libin Liu. "Rhythmic Gesticulator." ACM Transactions on Graphics 41, no. 6 (November 30, 2022): 1–19. http://dx.doi.org/10.1145/3550454.3555435.

Full text
Abstract:
Automatic synthesis of realistic co-speech gestures is an increasingly important yet challenging task in artificial embodied agent creation. Previous systems mainly focus on generating gestures in an end-to-end manner, which leads to difficulties in mining the clear rhythm and semantics due to the complex yet subtle harmony between speech and gestures. We present a novel co-speech gesture synthesis method that achieves convincing results both on the rhythm and semantics. For the rhythm, our system contains a robust rhythm-based segmentation pipeline to ensure the temporal coherence between the vocalization and gestures explicitly. For the gesture semantics, we devise a mechanism to effectively disentangle both low- and high-level neural embeddings of speech and motion based on linguistic theory. The high-level embedding corresponds to semantics, while the low-level embedding relates to subtle variations. Lastly, we build correspondence between the hierarchical embeddings of the speech and the motion, resulting in rhythm- and semantics-aware gesture synthesis. Evaluations with existing objective metrics, a newly proposed rhythmic metric, and human feedback show that our method outperforms state-of-the-art systems by a clear margin.
APA, Harvard, Vancouver, ISO, and other styles
4

Yang, Qi, and Georg Essl. "Evaluating Gesture-Augmented Keyboard Performance." Computer Music Journal 38, no. 4 (December 2014): 68–79. http://dx.doi.org/10.1162/comj_a_00277.

Full text
Abstract:
The technology of depth cameras has made designing gesture-based augmentation for existing instruments inexpensive. We explored the use of this technology to augment keyboard performance with 3-D continuous gesture controls. In a user study, we compared the control of one or two continuous parameters using gestures versus the traditional control using pitch and modulation wheels. We found that the choice of mapping depends on the choice of synthesis parameter in use, and that the gesture control under suitable mappings can outperform pitch-wheel performance when two parameters are controlled simultaneously.
APA, Harvard, Vancouver, ISO, and other styles
5

Souza, Fernando, and Adolfo Maia Jr. "A Mathematical, Graphical and Visual Approach to Granular Synthesis Composition." Revista Vórtex 9, no. 2 (December 10, 2021): 1–27. http://dx.doi.org/10.33871/23179937.2021.9.2.4.

Full text
Abstract:
We show a method for Granular Synthesis Composition based on a mathematical modeling for the musical gesture. Each gesture is drawn as a curve generated from a particular mathematical model (or function) and coded as a MATLAB script. The gestures can be deterministic through defining mathematical time functions, hand free drawn, or even randomly generated. This parametric information of gestures is interpreted through OSC messages by a granular synthesizer (Granular Streamer). The musical composition is then realized with the models (scripts) written in MATLAB and exported to a graphical score (Granular Score). The method is amenable to allow statistical analysis of the granular sound streams and the final music composition. We also offer a way to create granular streams based on correlated pair of grains parameters.
APA, Harvard, Vancouver, ISO, and other styles
6

Bouënard, Alexandre, Marcelo M. M. Wanderley, and Sylvie Gibet. "Gesture Control of Sound Synthesis: Analysis and Classification of Percussion Gestures." Acta Acustica united with Acustica 96, no. 4 (July 1, 2010): 668–77. http://dx.doi.org/10.3813/aaa.918321.

Full text
APA, Harvard, Vancouver, ISO, and other styles
7

He, Zhiyuan. "Automatic Quality Assessment of Speech-Driven Synthesized Gestures." International Journal of Computer Games Technology 2022 (March 16, 2022): 1–11. http://dx.doi.org/10.1155/2022/1828293.

Full text
Abstract:
The automatic synthesis of realistic gestures has the ability to change the fields of animation, avatars, and communication agents. Although speech-driven synthetic gesture generation methods have been proposed and optimized, the evaluation system of synthetic gestures is still lacking. The current evaluation method still needs manual participation, but it is inefficient in the industry of synthetic gestures and has the interference of human factors. So we need a model that can construct an automatic and objective quantitative quality assessment of the synthesized gesture video. We noticed that recurrent neural networks (RNN) have advantages in modeling advanced spatiotemporal feature sequences, which are very suitable for use in the processing of synthetic gesture video data. Therefore, to build an automatic quality assessment system, we propose in our work a model based on Bi-LSTM and make a little adjustment on the attention mechanism in it. Also, the evaluation method is proposed and experiments are designed to prove that the improved model of the algorithm can complete the quantitative evaluation of synthetic gestures. At the same time, in terms of performance, the model has an improvement of about 20% compared to before the algorithm adjustment.
APA, Harvard, Vancouver, ISO, and other styles
8

Xu, Zunnan, Yachao Zhang, Sicheng Yang, Ronghui Li, and Xiu Li. "Chain of Generation: Multi-Modal Gesture Synthesis via Cascaded Conditional Control." Proceedings of the AAAI Conference on Artificial Intelligence 38, no. 6 (March 24, 2024): 6387–95. http://dx.doi.org/10.1609/aaai.v38i6.28458.

Full text
Abstract:
This study aims to improve the generation of 3D gestures by utilizing multimodal information from human speech. Previous studies have focused on incorporating additional modalities to enhance the quality of generated gestures. However, these methods perform poorly when certain modalities are missing during inference. To address this problem, we suggest using speech-derived multimodal priors to improve gesture generation. We introduce a novel method that separates priors from speech and employs multimodal priors as constraints for generating gestures. Our approach utilizes a chain-like modeling method to generate facial blendshapes, body movements, and hand gestures sequentially. Specifically, we incorporate rhythm cues derived from facial deformation and stylization prior based on speech emotions, into the process of generating gestures. By incorporating multimodal priors, our method improves the quality of generated gestures and eliminate the need for expensive setup preparation during inference. Extensive experiments and user studies confirm that our proposed approach achieves state-of-the-art performance.
APA, Harvard, Vancouver, ISO, and other styles
9

Fernández-Baena, Adso, Raúl Montaño, Marc Antonijoan, Arturo Roversi, David Miralles, and Francesc Alías. "Gesture synthesis adapted to speech emphasis." Speech Communication 57 (February 2014): 331–50. http://dx.doi.org/10.1016/j.specom.2013.06.005.

Full text
APA, Harvard, Vancouver, ISO, and other styles
10

Nakano, Atsushi, and Junichi Hoshino. "Composite conversation gesture synthesis using layered planning." Systems and Computers in Japan 38, no. 10 (2007): 58–68. http://dx.doi.org/10.1002/scj.20532.

Full text
APA, Harvard, Vancouver, ISO, and other styles
11

Arfib, D., J. M. Couturier, L. Kessous, and V. Verfaille. "Strategies of mapping between gesture data and synthesis model parameters using perceptual spaces." Organised Sound 7, no. 2 (August 2002): 127–44. http://dx.doi.org/10.1017/s1355771802002054.

Full text
Abstract:
This paper is about mapping strategies between gesture data and synthesis model parameters by means of perceptual spaces. We define three layers in the mapping chain: from gesture data to gesture perceptual space, from sound perceptual space to synthesis model parameters, and between the two perceptual spaces. This approach makes the implementation highly modular. Both perceptual spaces are developed and depicted with their features. To get a simple mapping between the gesture perceptual subspace and the sound perceptual subspace, we need to focus our attention on the two other mappings. We explain the mapping types: explicit/implicit, static/dynamic. We also present the technical and aesthetical limits introduced by mapping. Some practical examples are given of the use of perceptual spaces in experiments done at LMA in a musical context. Finally, we discuss several implications of the mapping strategies: the influence of chosen mapping limits onto performers' virtuosity, and the incidence of mapping on the learning process with virtual instruments and on improvisation possibilities.
APA, Harvard, Vancouver, ISO, and other styles
12

Dang, Xiaochao, Wenze Ke, Zhanjun Hao, Peng Jin, Han Deng, and Ying Sheng. "mm-TPG: Traffic Policemen Gesture Recognition Based on Millimeter Wave Radar Point Cloud." Sensors 23, no. 15 (July 31, 2023): 6816. http://dx.doi.org/10.3390/s23156816.

Full text
Abstract:
Automatic driving technology refers to equipment such as vehicle-mounted sensors and computers that are used to navigate and control vehicles autonomously by acquiring external environmental information. To achieve automatic driving, vehicles must be able to perceive the surrounding environment and recognize and understand traffic signs, traffic signals, pedestrians, and other traffic participants, as well as accurately plan and control their path. Recognition of traffic signs and signals is an essential part of automatic driving technology, and gesture recognition is a crucial aspect of traffic-signal recognition. This article introduces mm-TPG, a traffic-police gesture recognition system based on a millimeter-wave point cloud. The system uses a 60 GHz frequency-modulated continuous-wave (FMCW) millimeter-wave radar as a sensor to achieve high-precision recognition of traffic-police gestures. Initially, a double-threshold filtering algorithm is used to denoise the millimeter-wave raw data, followed by multi-frame synthesis processing of the generated point cloud data and feature extraction using a ResNet18 network. Finally, gated recurrent units are used for classification to enable the recognition of different traffic-police gestures. Experimental results demonstrate that the mm-TPG system has high accuracy and robustness and can effectively recognize traffic-police gestures in complex environments such as varying lighting and weather conditions, providing strong support for traffic safety.
APA, Harvard, Vancouver, ISO, and other styles
13

Valencia, C. Roncancio, J. Gomez Garcia-Bermejo, and E. Zalama Casanova. "Combined Gesture-Speech Recognition and Synthesis Using Neural Networks." IFAC Proceedings Volumes 41, no. 2 (2008): 2968–73. http://dx.doi.org/10.3182/20080706-5-kr-1001.00499.

Full text
APA, Harvard, Vancouver, ISO, and other styles
14

Montgermont, Nicolas, Benoit Fabre, and Patricio De La Cuadra. "Gesture synthesis: basic control of a flute physical model." Journal of the Acoustical Society of America 123, no. 5 (May 2008): 3797. http://dx.doi.org/10.1121/1.2935477.

Full text
APA, Harvard, Vancouver, ISO, and other styles
15

Alexanderson, Simon, Gustav Eje Henter, Taras Kucherenko, and Jonas Beskow. "Style‐Controllable Speech‐Driven Gesture Synthesis Using Normalising Flows." Computer Graphics Forum 39, no. 2 (May 2020): 487–96. http://dx.doi.org/10.1111/cgf.13946.

Full text
APA, Harvard, Vancouver, ISO, and other styles
16

Mo, Dong-Han, Chuen-Lin Tien, Yu-Ling Yeh, Yi-Ru Guo, Chern-Sheng Lin, Chih-Chin Chen, and Che-Ming Chang. "Design of Digital-Twin Human-Machine Interface Sensor with Intelligent Finger Gesture Recognition." Sensors 23, no. 7 (March 27, 2023): 3509. http://dx.doi.org/10.3390/s23073509.

Full text
Abstract:
In this study, the design of a Digital-twin human-machine interface sensor (DT-HMIS) is proposed. This is a digital-twin sensor (DT-Sensor) that can meet the demands of human-machine automation collaboration in Industry 5.0. The DT-HMIS allows users/patients to add, modify, delete, query, and restore their previously memorized DT finger gesture mapping model and programmable logic controller (PLC) logic program, enabling the operation or access of the programmable controller input-output (I/O) interface and achieving the extended limb collaboration capability of users/patients. The system has two main functions: the first is gesture-encoded virtual manipulation, which indirectly accesses the PLC through the DT mapping model to complete control of electronic peripherals for extension-limbs ability by executing logic control program instructions. The second is gesture-based virtual manipulation to help non-verbal individuals create special verbal sentences through gesture commands to improve their expression ability. The design method uses primitive image processing and eight-way dual-bit signal processing algorithms to capture the movement of human finger gestures and convert them into digital signals. The system service maps control instructions by observing the digital signals of the DT-HMIS and drives motion control through mechatronics integration or speech synthesis feedback to express the operation requirements of inconvenient work or complex handheld physical tools. Based on the human-machine interface sensor of DT computer vision, it can reflect the user’s command status without the need for additional wearable devices and promote interaction with the virtual world. When used for patients, the system ensures that the user’s virtual control is mapped to physical device control, providing the convenience of independent operation while reducing caregiver fatigue. This study shows that the recognition accuracy can reach 99%, demonstrating practicality and application prospects. In future applications, users/patients can interact virtually with other peripheral devices through the DT-HMIS to meet their own interaction needs and promote industry progress.
APA, Harvard, Vancouver, ISO, and other styles
17

Yu, Shi Cai, and Rong Lu. "Research of Sign Language Synthesis Based on VRML." Applied Mechanics and Materials 347-350 (August 2013): 2631–35. http://dx.doi.org/10.4028/www.scientific.net/amm.347-350.2631.

Full text
Abstract:
Sign language is to help the deaf and normal hearing people natural communication and computer assisted instruction. Through the analysis of language features, and proposed one kind based on the VRML human body modeling and virtual human based on context of gesture smoothing algorithm, thus the sign language synthesis research and implementation.
APA, Harvard, Vancouver, ISO, and other styles
18

LANZALONE, SILVIA. "Hidden grids: paths of expressive gesture between instruments, music and dance." Organised Sound 5, no. 1 (April 2000): 17–26. http://dx.doi.org/10.1017/s1355771800001047.

Full text
Abstract:
In his work Contropasso (1998–9) Michelangelo Lupone collaborates with Massimo Moricone in the dance showcase piegapiaga achieving direct interaction between dancers and live electronics performance. The choreography takes advantage of acoustic events as generated by three dancers and further elaborated on via computer by the composer through use of granular algorithms and digital filtering, allowing the construction of the musical events to occur in real time. The live electronics performer changes sound parameters in relation to the dancers' movements by use of the program SDP – Sonorous Drawing Plane (S. Lanzalone) – created specifically for the control of different synthesis algorithms allowing them to be processed on systems such as Fly30 (CRM) and Mars (IRIS). SDP reads and converts computer mouse data as the operator creates lines corresponding to performance gestures, thus creating both visible and audible output. This software allows a single gesture to control more than one parameter, thus creating complex changes in the audio programme output. The article deals with different compositions, performances and didactic situations the author has experienced using SDP.
APA, Harvard, Vancouver, ISO, and other styles
19

Ryumin, Dmitry, Ildar Kagirov, Alexandr Axyonov, Nikita Pavlyuk, Anton Saveliev, Irina Kipyatkova, Milos Zelezny, Iosif Mporas, and Alexey Karpov. "A Multimodal User Interface for an Assistive Robotic Shopping Cart." Electronics 9, no. 12 (December 8, 2020): 2093. http://dx.doi.org/10.3390/electronics9122093.

Full text
Abstract:
This paper presents the research and development of the prototype of the assistive mobile information robot (AMIR). The main features of the presented prototype are voice and gesture-based interfaces with Russian speech and sign language recognition and synthesis techniques and a high degree of robot autonomy. AMIR prototype’s aim is to be used as a robotic cart for shopping in grocery stores and/or supermarkets. Among the main topics covered in this paper are the presentation of the interface (three modalities), the single-handed gesture recognition system (based on a collected database of Russian sign language elements), as well as the technical description of the robotic platform (architecture, navigation algorithm). The use of multimodal interfaces, namely the speech and gesture modalities, make human-robot interaction natural and intuitive, as well as sign language recognition allows hearing-impaired people to use this robotic cart. AMIR prototype has promising perspectives for real usage in supermarkets, both due to its assistive capabilities and its multimodal user interface.
APA, Harvard, Vancouver, ISO, and other styles
20

K, Kavyasree. "Hand Glide: Gesture-Controlled Virtual Mouse with Voice Assistant." International Journal for Research in Applied Science and Engineering Technology 12, no. 4 (April 30, 2024): 5470–76. http://dx.doi.org/10.22214/ijraset.2024.61178.

Full text
Abstract:
Abstract: Hand Glide presents a unified system that uses the MediaPipe library for accurate hand landmark recognition and categorization, enabling smooth integration of gesture-based control (GBC) and voice-controlled assistant (VCA) features. The system allows users to effortlessly translate hand movements into actions such as mouse manipulation, clicking, scrolling, and system parameter adjustments by providing distinct classes for hand gesture recognition (HandRecog) and execution (Controller), as well as a main class orchestrating camera input and gesture control. In addition, Hand Glide also has VCA functionality, which uses speech recognition and synthesis libraries and natural language processing (NLP) techniques. With these tools, Hand Glide can reply to greetings, perform tasks like Wikipedia searches and website openings, and converse with users through the GPT-2 language model. By merging these features, Hand Glide providesusers with an interactive and intuitive means of computer control, with broad applications in user interfaces, accessibility solutions, and immersive experiences
APA, Harvard, Vancouver, ISO, and other styles
21

Rasamimanana, Nicolas, Florian Kaiser, and Frederic Bevilacqua. "Perspectives on Gesture–Sound Relationships Informed from Acoustic Instrument Studies." Organised Sound 14, no. 2 (June 29, 2009): 208–16. http://dx.doi.org/10.1017/s1355771809000314.

Full text
Abstract:
We present an experimental study on articulation in bowed strings that provides important elements for a discussion about sound synthesis control. The study focuses on bow acceleration profiles and transient noises, measured for different players for the bowing techniquesdetachéandmartelé. We found that maximum of these profiles are not synchronous, and temporal shifts are dependent on the bowing techniques. These results allow us to bring out important mechanisms in sound and gesture articulation. In particular, the results reveal a potential shortcoming of mapping strategies using simple frame-by-frame data-stream procedures. We propose instead to consider input control data as time functions, and consider gesture co-articulation processes.
APA, Harvard, Vancouver, ISO, and other styles
22

MARTIN, JEAN-CLAUDE, RADOSLAW NIEWIADOMSKI, LAURENCE DEVILLERS, STEPHANIE BUISINE, and CATHERINE PELACHAUD. "MULTIMODAL COMPLEX EMOTIONS: GESTURE EXPRESSIVITY AND BLENDED FACIAL EXPRESSIONS." International Journal of Humanoid Robotics 03, no. 03 (September 2006): 269–91. http://dx.doi.org/10.1142/s0219843606000825.

Full text
Abstract:
One of the challenges of designing virtual humans is the definition of appropriate models of the relation between realistic emotions and the coordination of behaviors in several modalities. In this paper, we present the annotation, representation and modeling of multimodal visual behaviors occurring during complex emotions. We illustrate our work using a corpus of TV interviews. This corpus has been annotated at several levels of information: communicative acts, emotion labels, and multimodal signs. We have defined a copy-synthesis approach to drive an Embodied Conversational Agent from these different levels of information. The second part of our paper focuses on a model of complex (superposition and masking of) emotions in facial expressions of the agent. We explain how the complementary aspects of our work on corpus and computational model is used to specify complex emotional behaviors.
APA, Harvard, Vancouver, ISO, and other styles
23

Zhou, Yuxuan, Huangxun Chen, Chenyu Huang, and Qian Zhang. "WiAdv." Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 6, no. 2 (July 4, 2022): 1–25. http://dx.doi.org/10.1145/3534618.

Full text
Abstract:
WiFi-based gesture recognition systems have attracted enormous interest owing to the non-intrusive of WiFi signals and the wide adoption of WiFi for communication. Despite boosted performance via integrating advanced deep neural network (DNN) classifiers, there lacks sufficient investigation on their security vulnerabilities, which are rooted in the open nature of the wireless medium and the inherent defects (e.g., adversarial attacks) of classifiers. To fill this gap, we aim to study adversarial attacks to DNN-powered WiFi-based gesture recognition to encourage proper countermeasures. We design WiAdv to construct physically realizable adversarial examples to fool these systems. WiAdv features a signal synthesis scheme to craft adversarial signals with desired motion features based on the fundamental principle of WiFi-based gesture recognition, and a black-box attack scheme to handle the inconsistency between the perturbation space and the input space of the classifier caused by the in-between non-differentiable processing modules. We realize and evaluate our attack strategies against a representative state-of-the-art system, Widar3.0 in realistic settings. The experimental results show that the adversarial wireless signals generated by WiAdv achieve over 70% attack success rate on average, and remain robust and effective across different physical settings. Our attack case study and analysis reveal the vulnerability of WiFi-based gesture recognition systems, and we hope WiAdv could help promote the improvement of the relevant systems.
APA, Harvard, Vancouver, ISO, and other styles
24

Ketabdar, Hamed, Amin Haji-Abolhassani, and Mehran Roshandel. "MagiThings." International Journal of Mobile Human Computer Interaction 5, no. 3 (July 2013): 23–41. http://dx.doi.org/10.4018/jmhci.2013070102.

Full text
Abstract:
The theory of around device interaction (ADI) has recently gained a lot of attention in the field of human computer interaction (HCI). As an alternative to the classic data entry methods, such as keypads and touch screens interaction, ADI proposes a touchless user interface that extends beyond the peripheral area of a device. In this paper, the authors propose a new approach for around mobile device interaction based on magnetic field. Our new approach, which we call it “MagiThings”, takes the advantage of digital compass (a magnetometer) embedded in new generation of mobile devices such as Apple’s iPhone 3GS/4G, and Google’s Nexus. The user movements of a properly shaped magnet around the device deform the original magnetic field. The magnet is taken or worn around the fingers. The changes made in the magnetic field pattern around the device constitute a new way of interacting with the device. Thus, the magnetic field encompassing the device plays the role of a communication channel and encodes the hand/finger movement patterns into temporal changes sensed by the compass sensor. The mobile device samples momentary status of the field. The field changes, caused by hand (finger) gesture, is used as a basis for sending interaction commands to the device. The pattern of change is matched against pre-recorded templates or trained models to recognize a gesture. The proposed methodology has been successfully tested for a variety of applications such as interaction with user interface of a mobile device, character (digit) entry, user authentication, gaming, and touchless mobile music synthesis. The experimental results show high accuracy in recognizing simple or complex gestures in a wide range of applications. The proposed method provides a practical and simple framework for touchless interaction with mobile devices relying only on an internally embedded sensor and a magnet.
APA, Harvard, Vancouver, ISO, and other styles
25

Thoret, Etienne, Mitsuko Aramaki, Charles Gondre, Sølvi Ystad, and Richard Kronland-Martinet. "Eluding the Physical Constraints in a Nonlinear Interaction Sound Synthesis Model for Gesture Guidance." Applied Sciences 6, no. 7 (June 30, 2016): 192. http://dx.doi.org/10.3390/app6070192.

Full text
APA, Harvard, Vancouver, ISO, and other styles
26

Camurri, Antonio, Giovanni De Poli, Anders Friberg, Marc Leman, and Gualtiero Volpe. "The MEGA Project: Analysis and Synthesis of Multisensory Expressive Gesture in Performing Art Applications." Journal of New Music Research 34, no. 1 (March 2005): 5–21. http://dx.doi.org/10.1080/09298210500123895.

Full text
APA, Harvard, Vancouver, ISO, and other styles
27

Bouënard, Alexandre, Marcelo M. Wanderley, Sylvie Gibet, and Fabrice Marandola. "Virtual Gesture Control and Synthesis of Music Performances: Qualitative Evaluation of Synthesized Timpani Exercises." Computer Music Journal 35, no. 3 (September 2011): 57–72. http://dx.doi.org/10.1162/comj_a_00069.

Full text
APA, Harvard, Vancouver, ISO, and other styles
28

Deinega, Volodymyr. "Influence of Timbral Sound Coloring on the Evolution of the Conductor's Gesture." Часопис Національної музичної академії України ім.П.І.Чайковського, no. 3(60) (September 27, 2023): 85–97. http://dx.doi.org/10.31318/2414-052x.3(60).2023.296801.

Full text
Abstract:
An overview of the professional aspects of conducting as a process of managing a performing team was carried out. The main directions of the development of the conductor's performance skills from the noisy type to the silent type are revealed. The dynamics of changes in the forms and methods of conducting have been traced while preserving the priority of gesturing as the main type of influence on the collective during the performance of a musical piece. It was found that the conductor controls not only the logic of the development of the musical material, but implements it by using a specific complex of real physical movements (gestures) organized in time. The essence of the conductor's orchestral thinking is analyzed, which is a synthesis of his logical and procedural thinking, which ensures the development of the timbre dramaturgy of the musical work. The dependence of the timbral coloring of sound on the involvement of various types of orchestral instruments in the reproduction of musical material is substantiated. It has been established that orchestration is the basis of the conductor's timbral thinking. Composers are divided (conditionally) into two main types of timbral thinking, where the first includes those for whom orchestration is a means of expressing musical thought (L. Beethoven, R. Wagner, P. Tchaikovsky, D. Shostakovich, and others), and the second includes those for whom orchestration is a means of expressing sophisticated imagery by creating an orchestral color and multi-colored sound (H. Berlioz, M. Rimskyi Korsakov, K. Debussy and others). It has been proven that: timbral coloring and the conductor's manual technique have a reciprocal relationship; The elements of the conductor's gesture reflect not only the metro-rhythmic structure, stroke realization, dynamic shades, emotional and figurative content of the musical work, but also reflect the quality of the new timbre, its modification and development; the timbre auditory imagination of the conductor develops in classes on orchestration (instrumentation) and in the process of conducting directly with the orchestra; the conductor's gesture, which is not supported by timbral feeling, leads to a formal gesticulation that does not reflect the emotional and figurative content of the musical work.
APA, Harvard, Vancouver, ISO, and other styles
29

Nichols, Charles. "The vBow: a virtual violin bow controller for mapping gesture to synthesis with haptic feedback." Organised Sound 7, no. 2 (August 2002): 215–20. http://dx.doi.org/10.1017/s135577180200211x.

Full text
Abstract:
The vBow, a virtual violin bow musical controller, has been designed to provide the computer musician with most of the gestural freedom of a bow on a violin string. Four cable and servomotor systems allow for four degrees of freedom, including the lateral motion of a bow stroke across a string, the rotational motion of a bow crossing strings, the vertical motion of a bow approaching and pushing into a string, and the longitudinal motion of a bow travelling along the length of a string. Encoders, attached to the shaft of the servomotors, sense the gesture of the performer, through the rotation of the servomotor shafts, turned by the motion of the cables. The data from each encoder is mapped to a parameter in synthesis software of a bowed-string physical model. The software also sends control voltages to the servomotors, engaging them and the cables attached to them with a haptic feedback simulation of friction, vibration, detents and elasticity.
APA, Harvard, Vancouver, ISO, and other styles
30

Phukon, Debasish. "A Deep Learning Approach for ASL Recognition and Text-to-Speech Synthesis using CNN." International Journal for Research in Applied Science and Engineering Technology 11, no. 8 (August 31, 2023): 2135–43. http://dx.doi.org/10.22214/ijraset.2023.55528.

Full text
Abstract:
Abstract: Sign language is a visual language that is used by the deaf and hard-of-hearing community to communicate. However, sign language is not universally understood by non-signers, which can create communication barriers for the deaf and hard-ofhearing individuals. In this paper, we present a novel application for American Sign Language (ASL) to text to speech conversion using deep learning techniques. Our app aims to bridge the communication gap between hearing-impaired individuals who use ASL as their primary mode of communication and individuals who do not understand ASL. The app comprises three main components: a hand gesture recognition module that detects and recognizes ASL signs, a text-tospeech synthesis module that converts recognized ASL signs to text, and a speech synthesis module that converts the text to speech
APA, Harvard, Vancouver, ISO, and other styles
31

Harrison, Reginald Langford, Stefan Bilbao, James Perry, and Trevor Wishart. "An Environment for Physical Modeling of Articulated Brass Instruments." Computer Music Journal 39, no. 4 (December 2015): 80–95. http://dx.doi.org/10.1162/comj_a_00332.

Full text
Abstract:
This article presents a synthesis environment for physical modeling of valved brass instrument sounds. Synthesis is performed using finite-difference time-domain methods that allow for flexible simulation of time-varying systems. Users have control over the instrument configuration as well as player parameters, such as mouth pressure, lip dynamics, and valve depressions, which can be varied over the duration of a gesture. This article introduces the model used in the environment, the development of code from prototyping in MATLAB and optimization in C, and the incorporation of the executable file in the Sound Loom interface of the Composers Desktop Project. Planned additions to the environment are then discussed. The environment binaries are available to download online along with example sounds and input files.
APA, Harvard, Vancouver, ISO, and other styles
32

Polykretis, Ioannis, Aditi Patil, Mridul Aanjaneya, and Konstantinos Michmizos. "An Interactive Framework for Visually Realistic 3D Motion Synthesis using Evolutionarily-trained Spiking Neural Networks." Proceedings of the ACM on Computer Graphics and Interactive Techniques 6, no. 1 (May 12, 2023): 1–19. http://dx.doi.org/10.1145/3585509.

Full text
Abstract:
We present an end-to-end method for capturing the dynamics of 3D human characters and translating them for synthesizing new, visually-realistic motion sequences. Conventional methods employ sophisticated, but generic, control approaches for driving the joints of articulated characters, paying little attention to the distinct dynamics of human joint movements. In contrast, our approach attempts to synthesize human-like joint movements by exploiting a biologically-plausible, compact network of spiking neurons that drive joint control in primates and rodents. We adapt the controller architecture by introducing learnable components and propose an evolutionary algorithm for training the spiking neural network architectures and capturing diverse joint dynamics. Our method requires only a few samples for capturing the dynamic properties of a joint's motion and exploits the biologically-inspired, trained controller for its reconstruction. More importantly, it can transfer the captured dynamics to new visually-plausible motion sequences. To enable user-dependent tailoring of the resulting motion sequences, we develop an interactive framework that allows for editing and real-time visualization of the controlled 3D character. We also demonstrate the applicability of our method to real human motion capture data by learning the hand joint dynamics from a gesture dataset and using our framework to reconstruct the gestures with our 3D animated character. The compact architecture of our joint controller emerging from its biologically-realistic design, and the inherent capacity of our evolutionary learning algorithm for parallelization, suggest that our approach could provide an efficient and scalable alternative for synthesizing 3D character animations with diverse and visually-realistic motion dynamics.
APA, Harvard, Vancouver, ISO, and other styles
33

Wołk, Krzysztof, Agnieszka Wołk, and Wojciech Glinkowski. "A Cross-Lingual Mobile Medical Communication System Prototype for Foreigners and Subjects with Speech, Hearing, and Mental Disabilities Based on Pictograms." Computational and Mathematical Methods in Medicine 2017 (2017): 1–9. http://dx.doi.org/10.1155/2017/4306416.

Full text
Abstract:
People with speech, hearing, or mental impairment require special communication assistance, especially for medical purposes. Automatic solutions for speech recognition and voice synthesis from text are poor fits for communication in the medical domain because they are dependent on error-prone statistical models. Systems dependent on manual text input are insufficient. Recently introduced systems for automatic sign language recognition are dependent on statistical models as well as on image and gesture quality. Such systems remain in early development and are based mostly on minimal hand gestures unsuitable for medical purposes. Furthermore, solutions that rely on the Internet cannot be used after disasters that require humanitarian aid. We propose a high-speed, intuitive, Internet-free, voice-free, and text-free tool suited for emergency medical communication. Our solution is a pictogram-based application that provides easy communication for individuals who have speech or hearing impairment or mental health issues that impair communication, as well as foreigners who do not speak the local language. It provides support and clarification in communication by using intuitive icons and interactive symbols that are easy to use on a mobile device. Such pictogram-based communication can be quite effective and ultimately make people’s lives happier, easier, and safer.
APA, Harvard, Vancouver, ISO, and other styles
34

Qureshi, Regula Burckhardt. "Musical Gesture and Extra-Musical Meaning: Words and Music in the Urdu Ghazal." Journal of the American Musicological Society 43, no. 3 (1990): 457–97. http://dx.doi.org/10.2307/831743.

Full text
Abstract:
This ethnomusicological inquiry into the text-music relationship assumes a broadly contextual perspective that encompasses, but goes beyond, traditional considerations of textual and musical structures with the aim of exploring the sources of meaning for what Edward Cone terms the "gestural character" of the musical utterance. The approach calls for making explicit the ideational framework as well as the performative function of a vocal genre, both of which inform the way its musical idiom serves to communicate a text in performance. The ghazal, subject of this case study, is the principal poetic form in Urdu; it is set to music in a number of related but distinct genres. Illustrated by a set of transcribed and translated examples, the analytical procedure first considers the text-music structure as a performance idiom that is subject to the cultural-historical background norms of Urdu poetry on the one hand and North Indian "light" music on the other. The second stage considers each particular genre in terms of its idiom's specific function in performance. The general conclusion is that music linked to ghazal poetry is structurally constrained by the text as performed, but that purely musical features articulate, and thus link, the text with the content of its performance. Hence ghazal music represents a synthesis of text and context in its embodiment of features drawn from both and expressed directly through a unique vocabulary of musical gestures.
APA, Harvard, Vancouver, ISO, and other styles
35

Moore, Carol-Lynne. "God geometricizes (and so does Laban)." Dance, Movement & Spiritualities 10, no. 1 (October 1, 2023): 73–90. http://dx.doi.org/10.1386/dmas_00047_1.

Full text
Abstract:
Although Rudolf Laban is primarily identified with the analytical frameworks he developed for studying and recording human movements, he remained deeply concerned to understand the complex synthesis of body, effort and space that occurs, seemingly spontaneously, in normal as well as skilled actions. Laban perceived a deeper meaning in this miraculous synthesis, noting that ‘every gesture and action of our body is a deeply rooted mystery’. This article traces how Laban used the metaphors of geometry and harmony to penetrate the physical reality of human movement as well as its metaphysical significance. Discussion centres on two Platonic Solids – the icosahedron and the cube – illustrating how Laban used each respectively to model spatial and dynamic patterns, to examine these elemental patterns in relation to each other and finally to place the human body at the centre of a universe of universal movements, linking microcosm and macrocosm.
APA, Harvard, Vancouver, ISO, and other styles
36

Huang, Chih-Fang, and Wei-Po Nien. "A Study of the Integrated Automated Emotion Music with the Motion Gesture Synthesis via ZigBee Wireless Communication." International Journal of Distributed Sensor Networks 9, no. 11 (January 2013): 645961. http://dx.doi.org/10.1155/2013/645961.

Full text
APA, Harvard, Vancouver, ISO, and other styles
37

Lee, Donghee, Dayoung You, Gyoungryul Cho, Hoirim Lee, Eunsoo Shin, Taehwan Choi, Sunghan Kim, Sangmin Lee, and Woochul Nam. "EMG-based hand gesture classifier robust to daily variation: Recursive domain adversarial neural network with data synthesis." Biomedical Signal Processing and Control 88 (February 2024): 105600. http://dx.doi.org/10.1016/j.bspc.2023.105600.

Full text
APA, Harvard, Vancouver, ISO, and other styles
38

Alexanderson, Simon, Rajmund Nagy, Jonas Beskow, and Gustav Eje Henter. "Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models." ACM Transactions on Graphics 42, no. 4 (July 26, 2023): 1–20. http://dx.doi.org/10.1145/3592458.

Full text
Abstract:
Diffusion models have experienced a surge of interest as highly expressive yet efficiently trainable probabilistic models. We show that these models are an excellent fit for synthesising human motion that co-occurs with audio, e.g., dancing and co-speech gesticulation, since motion is complex and highly ambiguous given audio, calling for a probabilistic description. Specifically, we adapt the DiffWave architecture to model 3D pose sequences, putting Conformers in place of dilated convolutions for improved modelling power. We also demonstrate control over motion style, using classifier-free guidance to adjust the strength of the stylistic expression. Experiments on gesture and dance generation confirm that the proposed method achieves top-of-the-line motion quality, with distinctive styles whose expression can be made more or less pronounced. We also synthesise path-driven locomotion using the same model architecture. Finally, we generalise the guidance procedure to obtain product-of-expert ensembles of diffusion models and demonstrate how these may be used for, e.g., style interpolation, a contribution we believe is of independent interest.
APA, Harvard, Vancouver, ISO, and other styles
39

Li, Yuerong, Xingce Wang, Zhongke Wu, Guoshuai Li, Shaolong Liu, and Mingquan Zhou. "Flexible indoor scene synthesis based on multi-object particle swarm intelligence optimization and user intentions with 3D gesture." Computers & Graphics 93 (December 2020): 1–12. http://dx.doi.org/10.1016/j.cag.2020.08.002.

Full text
APA, Harvard, Vancouver, ISO, and other styles
40

Van Nort, Doug, Marcelo M. Wanderley, and Philippe Depalle. "Mapping Control Structures for Sound Synthesis: Functional and Topological Perspectives." Computer Music Journal 38, no. 3 (September 2014): 6–22. http://dx.doi.org/10.1162/comj_a_00253.

Full text
Abstract:
This article contributes a holistic conceptual framework for the notion of “mapping” that extends the classical view of mapping as parameter association. In presenting this holistic approach to mapping techniques, we apply the framework to existing works from the literature as well as to new implementations that consider this approach in their construction. As any mapping control structure for a given digital instrument is determined by the musical context in which it is used, we present musical examples that relate the relatively abstract realm of mapping design to the physically and perceptually grounded notions of control and sonic gesture. Making this connection allows mapping to be more clearly seen as a linkage between a physical action and a sonic result. In this sense, the purpose of this work is to translate the discussion on mapping so that it links an abstract and formalized approach—intended for representation and conceptualization—with a viewpoint that considers mapping in its role as a perceived correspondence between physical materials (i.e., those that act on controllers and transducers) and sonic events. This correspondence is, at its heart, driven by our cognitive and embodied understanding of the acoustic world.
APA, Harvard, Vancouver, ISO, and other styles
41

Saafi, Houssem, Med Amine Laribi, and Said Zeghloul. "Design of a 4-DoF (degree of freedom) hybrid-haptic device for laparoscopic surgery." Mechanical Sciences 12, no. 1 (February 12, 2021): 155–64. http://dx.doi.org/10.5194/ms-12-155-2021.

Full text
Abstract:
Abstract. This paper presents a novel kinematics architecture with 4 DoFs (degrees of freedom) intended to be used as a haptic interface for laparoscopic surgery. The proposed architecture is a result of an association of serial and parallel kinematics chains, with each one handling a part of the whole device DoF. The serial chain allows one to handle the translation and self-rotation and the parallel chain handles the two tilt motions, and this in a disjoint way as the natural gesture of the surgeon. The proposed hybrid-haptic device (HH device) benefits from the split DoF to ensure a good kinematic performance, large workspace, as well as gravity compensation. The kinematics study of the HH device is presented and followed by the optimal dimensional synthesis and the gravity compensation model.
APA, Harvard, Vancouver, ISO, and other styles
42

Lin, Xu, and Gao Wen. "Human-Computer Chinese Sign Language Interaction System." International Journal of Virtual Reality 4, no. 3 (January 1, 2000): 82–92. http://dx.doi.org/10.20870/ijvr.2000.4.3.2651.

Full text
Abstract:
The generation and recognition of body language is a key technologies of VR. Sign Language is a visual-gestural language mainly used by hearing-impaired people. In this paper, gesture and facial expression models are created using computer graphics and used to synthesize Chinese Sign Language (CSL), and from it a human-computer CSL interaction system is implemented. Using a system combining CSL synthesis and CSL recognition subsystem, hearing-impaired people with data-gloves can pantomime CSL, which can then be displayed on the computer screen in real time and translated into Chinese text. Hearing people can also use the system by entering Chinese text, which is translated into CSL and displayed on the computer screen. In this way hearing-impaired people and hearing people can communicate with each other conveniently.
APA, Harvard, Vancouver, ISO, and other styles
43

Chopparapu, SaiTeja, and Joseph Beatrice Seventline. "An Efficient Multi-modal Facial Gesture-based Ensemble Classification and Reaction to Sound Framework for Large Video Sequences." Engineering, Technology & Applied Science Research 13, no. 4 (August 9, 2023): 11263–70. http://dx.doi.org/10.48084/etasr.6087.

Full text
Abstract:
Machine learning-based feature extraction and classification models play a vital role in evaluating and detecting patterns in multivariate facial expressions. Most conventional feature extraction and multi-modal pattern detection models are independent of filters for multi-class classification problems. In traditional multi-modal facial feature extraction models, it is difficult to detect the dependent correlated feature sets and use ensemble classification processes. This study used advanced feature filtering, feature extraction measures, and ensemble multi-class expression prediction to optimize the efficiency of feature classification. A filter-based multi-feature ranking-based voting framework was implemented on different multiple-based classifiers. Experimental results were evaluated on different multi-modal facial features for the automatic emotions listener using a speech synthesis library. The evaluation results showed that the proposed model had better feature classification, feature selection, prediction, and runtime than traditional approaches on heterogeneous facial databases.
APA, Harvard, Vancouver, ISO, and other styles
44

Du, Chuan, Lei Zhang, Xiping Sun, Junxu Wang, and Jialian Sheng. "Enhanced Multi-Channel Feature Synthesis for Hand Gesture Recognition Based on CNN With a Channel and Spatial Attention Mechanism." IEEE Access 8 (2020): 144610–20. http://dx.doi.org/10.1109/access.2020.3010063.

Full text
APA, Harvard, Vancouver, ISO, and other styles
45

Chiu, Jih-Ching, Guan-Yi Lee, Chih-Yang Hsieh, and Qing-You Lin. "Design and Implementation of Nursing-Secure-Care System with mmWave Radar by YOLO-v4 Computing Methods." Applied System Innovation 7, no. 1 (January 19, 2024): 10. http://dx.doi.org/10.3390/asi7010010.

Full text
Abstract:
In computer vision and image processing, the shift from traditional cameras to emerging sensing tools, such as gesture recognition and object detection, addresses privacy concerns. This study navigates the Integrated Sensing and Communication (ISAC) era, using millimeter-wave signals as radar via a Convolutional Neural Network (CNN) model for event sensing. Our focus is on leveraging deep learning to detect security-critical gestures, converting millimeter-wave parameters into point cloud images, and enhancing recognition accuracy. CNNs present complexity challenges in deep learning. To address this, we developed flexible quantization methods, simplifying You Only Look Once (YOLO)-v4 operations with an 8-bit fixed-point number representation. Cross-simulation validation showed that CPU-based quantization improves speed by 300% with minimal accuracy loss, even doubling the YOLO-tiny model’s speed in a GPU environment. We established a Raspberry Pi 4-based system, combining simplified deep learning with Message Queuing Telemetry Transport (MQTT) Internet of Things (IoT) technology for nursing care. Our quantification method significantly boosted identification speed by nearly 2.9 times, enabling millimeter-wave sensing in embedded systems. Additionally, we implemented hardware-based quantization, directly quantifying data from images or weight files, leading to circuit synthesis and chip design. This work integrates AI with mmWave sensors in the domain of nursing security and hardware implementation to enhance recognition accuracy and computational efficiency. Employing millimeter-wave radar in medical institutions or homes offers a strong solution to privacy concerns compared to conventional cameras that capture and analyze the appearance of patients or residents.
APA, Harvard, Vancouver, ISO, and other styles
46

Movchan, Larisa Anatol'evna. "The specifics of teaching the techniques of conducting gesture in the "Choral conducting" classroom." PHILHARMONICA. International Music Journal, no. 1 (January 2024): 18–32. http://dx.doi.org/10.7256/2453-613x.2024.1.69898.

Full text
Abstract:
The profession of conductor is one of the most important musical specialties in the world. The conductor plays the role of the head of the orchestra or choir, is responsible for the accuracy and expressiveness of the sound of the work, and serves as an intermediary between the author and the performer. The nature of the music and the performance interpretation depend on the correct work of the conductor. There is an educational side to the conductor's work. The choirmaster, engaged in teaching, solves a number of educational tasks. The article examines the specifics of teaching conducting gesture techniques in the classroom in the specialty "Choral conducting", highlights the main aspects and problems of conducting a choral group, and compares the work of a choral and symphonic conductor. The technique of managing a vocal group – an academic, folk or church choir - implies the ability to read scores in different keys, arrange them, and correctly distribute the parts depending on the characteristics of the voices. The choir director also works with the literary text of vocal compositions. It is impossible to do without general musical and theoretical knowledge in the field of solfeggio, harmony, and polyphony. The technique of staging the conducting apparatus and the schedule of the timing schemes practically do not differ among the heads of instrumental and vocal groups. This can be explained by the general nature, the psychology of collective music making. However, we must not forget about the specific features of choral conducting. This is the relative smoothness of gestures due to the different allocation of voices than in the orchestra, as well as the non-use of the conductor's baton. The second important difference between a choirmaster and a symphonic conductor is the need to analyze not only the musical, but also the verbal text of the score. The characteristic of the composer's work is added to the characteristic of the poet's work or the features of oral folk art, if the exact author of the text is unknown. In the work of a chorus conductor, vocal performance is of great importance, which is related to the possibilities of the voice. Choral conducting is more intimate than orchestral conducting. The most obvious synthesis of these arts, which almost completely eliminates all differences, can be noted in the management of both vocal and instrumental groups at once: in operas, cantatas, oratorios.
APA, Harvard, Vancouver, ISO, and other styles
47

Desvages, Charlotte, and Stefan Bilbao. "Two-Polarisation Physical Model of Bowed Strings with Nonlinear Contact and Friction Forces, and Application to Gesture-Based Sound Synthesis." Applied Sciences 6, no. 5 (May 10, 2016): 135. http://dx.doi.org/10.3390/app6050135.

Full text
APA, Harvard, Vancouver, ISO, and other styles
48

Ramos Flores, Cristohper. "The bodyless sound and the re-embodied sound: an expansion of the sonic body of the instrument." Ricercare, no. 15 (August 21, 2022): 30–58. http://dx.doi.org/10.17230/ricercare.2022.15.2.

Full text
Abstract:
The development of recording technologies, audio manipulation techniques, and sound synthesis opened new sonic horizons. At the same time, realising or reproducing these new sounds creates issues of disembodiment and/or a total lack of physical-gesture-to-audio relationship. Understanding the impact these issues have on our perception and comprehension of music becomes central in the light of new creative practices, in which developing hardware and software has become part of the creative process. These creative practices force us to re-think the role of performance and the medium (musical instruments) in the essence of the musical work. Building upon previous research, a set of possible configurations for hyperinstrument design is presented in this article with the aim to introduce novel ways of thinking about the relationship of the physical body of the instrument (resonant body), the sonic body (the acoustic phenomena unfolding in a physical space), and performance.
APA, Harvard, Vancouver, ISO, and other styles
49

Jokisch, Oliver, and Markus Huber. "Advances in the development of a cognitive user interface." MATEC Web of Conferences 161 (2018): 01003. http://dx.doi.org/10.1051/matecconf/201816101003.

Full text
Abstract:
In this contribution, we want to summarize recent development steps of the embedded cognitive user interface UCUI, which enables a user-adaptive scenario in human-machine or even human-robot interactions by considering sophisticated cognitive and semantic modelling. The interface prototype is developed by different German institutes and companies with their steering teams at Fraunhofer IKTS and Brandenburg University of Technology. The interface prototype is able to communicate with users via speech and gesture recognition, speech synthesis and a touch display. The device includes an autarkic semantic processing and beyond a cognitive behavior control, which supports an intuitive interaction to control different kinds of electronic devices, e. g. in a smart home environment or in interactive respectively collaborative robotics. Contrary to available speech assistance systems such as Amazon Echo or Google Home, the introduced cognitive user interface UCUI ensures the user privacy by processing all necessary information without any network access of the interface device.
APA, Harvard, Vancouver, ISO, and other styles
50

Magnenat-Thalmann, Nadia, and Arjan Egges. "Interactive Virtual Humans in Real-Time Virtual Environment." International Journal of Virtual Reality 5, no. 2 (January 1, 2006): 15–24. http://dx.doi.org/10.20870/ijvr.2006.5.2.2682.

Full text
Abstract:
In this paper, we will present an overview of existing research in the vast area of IVH systems. We will also present our ongoing work on improving the expressive capabilities of IVHs. Because of the complexity of interaction, a high level of control is required over the face and body motions of the virtual humans. In order to achieve this, current approaches try to generate face and body motions from a high-level description. Although this indeed allows for a precise control over the movement of the virtual human, it is difficult to generate a natural-looking motion from such a high-level description. Another problem that arises when animating IVHs is that motions are not generated all the time. Therefore a flexible animation scheme is required that ensures a natural posture even when no animation is playing. We will present MIRAnim, our animation engine, which uses a combination of motion synthesis from motion capture and a statistical analysis of prerecorded motion clips. As opposed to existing approaches that create new motions with limited flexibility, our model adapts existing motions, by automatically adding dependent joint motions. This renders the animation more natural, but since our model does not impose any conditions on the input motion, it can be linked easily with existing gesture synthesis techniques for IVHs. Because we use a linear representation for joint orientations, blending and interpolation is done very efficiently, resulting in an animation engine especially suitable for real-time applications
APA, Harvard, Vancouver, ISO, and other styles
You might also be interested in the bibliographies on the topic 'Gesture Synthesis' for other source types:
Dissertations / Theses
We offer discounts on all premium plans for authors whose works are included in thematic literature selections. Contact us to get a unique promo code!

To the bibliography