Journal articles: 'Multimodal user interface'

1

Reeves, Leah M., Jean-Claude Martin, Michael McTear, TV Raman, Kay M. Stanney, Hui Su, Qian Ying Wang, et al. "Guidelines for multimodal user interface design." Communications of the ACM 47, no. 1 (January 1, 2004): 57. http://dx.doi.org/10.1145/962081.962106.

Full text

APA, Harvard, Vancouver, ISO, and other styles

2

Karpov, A. A., and A. L. Ronzhin. "Information enquiry kiosk with multimodal user interface." Pattern Recognition and Image Analysis 19, no. 3 (September 2009): 546–58. http://dx.doi.org/10.1134/s1054661809030225.

Full text

APA, Harvard, Vancouver, ISO, and other styles

3

Baker, Kirk, Ashley Mckenzie, Alan Biermann, and Gert Webelhuth. "Constraining User Response via Multimodal Dialog Interface." International Journal of Speech Technology 7, no. 4 (October 2004): 251–58. http://dx.doi.org/10.1023/b:ijst.0000037069.82313.57.

Full text

APA, Harvard, Vancouver, ISO, and other styles

4

Ryumin, Dmitry, Ildar Kagirov, Alexandr Axyonov, Nikita Pavlyuk, Anton Saveliev, Irina Kipyatkova, Milos Zelezny, Iosif Mporas, and Alexey Karpov. "A Multimodal User Interface for an Assistive Robotic Shopping Cart." Electronics 9, no. 12 (December 8, 2020): 2093. http://dx.doi.org/10.3390/electronics9122093.

Full text

Abstract:

This paper presents the research and development of the prototype of the assistive mobile information robot (AMIR). The main features of the presented prototype are voice and gesture-based interfaces with Russian speech and sign language recognition and synthesis techniques and a high degree of robot autonomy. AMIR prototype’s aim is to be used as a robotic cart for shopping in grocery stores and/or supermarkets. Among the main topics covered in this paper are the presentation of the interface (three modalities), the single-handed gesture recognition system (based on a collected database of Russian sign language elements), as well as the technical description of the robotic platform (architecture, navigation algorithm). The use of multimodal interfaces, namely the speech and gesture modalities, make human-robot interaction natural and intuitive, as well as sign language recognition allows hearing-impaired people to use this robotic cart. AMIR prototype has promising perspectives for real usage in supermarkets, both due to its assistive capabilities and its multimodal user interface.

APA, Harvard, Vancouver, ISO, and other styles

5

Goyzueta, Denilson V., Joseph Guevara M., Andrés Montoya A., Erasmo Sulla E., Yuri Lester S., Pari L., and Elvis Supo C. "Analysis of a User Interface Based on Multimodal Interaction to Control a Robotic Arm for EOD Applications." Electronics 11, no. 11 (May 25, 2022): 1690. http://dx.doi.org/10.3390/electronics11111690.

Full text

Abstract:

A global human–robot interface that meets the needs of Technical Explosive Ordnance Disposal Specialists (TEDAX) for the manipulation of a robotic arm is of utmost importance to make the task of handling explosives safer, more intuitive and also provide high usability and efficiency. This paper aims to evaluate the performance of a multimodal system for a robotic arm that is based on Natural User Interface (NUI) and Graphical User Interface (GUI). The mentioned interfaces are compared to determine the best configuration for the control of the robotic arm in Explosive Ordnance Disposal (EOD) applications and to improve the user experience of TEDAX agents. Tests were conducted with the support of police agents Explosive Ordnance Disposal Unit-Arequipa (UDEX-AQP), who evaluated the developed interfaces to find a more intuitive system that generates the least stress load to the operator, resulting that our proposed multimodal interface presents better results compared to traditional interfaces. The evaluation of the laboratory experiences was based on measuring the workload and usability of each interface evaluated.

APA, Harvard, Vancouver, ISO, and other styles

6

Li Deng, Kuansan Wang, A. Acero, Hsiao-Wuen Hon, J. Droppo, C. Boulis, Ye-Yi Wang, et al. "Distributed speech processing in miPad's multimodal user interface." IEEE Transactions on Speech and Audio Processing 10, no. 8 (November 2002): 605–19. http://dx.doi.org/10.1109/tsa.2002.804538.

Full text

APA, Harvard, Vancouver, ISO, and other styles

7

Shi, Yu, Ronnie Taib, Natalie Ruiz, Eric Choi, and Fang Chen. "MULTIMODAL HUMAN-MACHINE INTERFACE AND USER COGNITIVE LOAD MEASUREMENT." IFAC Proceedings Volumes 40, no. 16 (2007): 200–205. http://dx.doi.org/10.3182/20070904-3-kr-2922.00035.

Full text

APA, Harvard, Vancouver, ISO, and other styles

8

La Tona, Giuseppe, Antonio Petitti, Adele Lorusso, Roberto Colella, Annalisa Milella, and Giovanni Attolico. "Modular multimodal user interface for distributed ambient intelligence architectures." Internet Technology Letters 1, no. 2 (February 9, 2018): e23. http://dx.doi.org/10.1002/itl2.23.

Full text

APA, Harvard, Vancouver, ISO, and other styles

9

Argyropoulos, Savvas, Konstantinos Moustakas, Alexey A. Karpov, Oya Aran, Dimitrios Tzovaras, Thanos Tsakiris, Giovanna Varni, and Byungjun Kwon. "Multimodal user interface for the communication of the disabled." Journal on Multimodal User Interfaces 2, no. 2 (July 15, 2008): 105–16. http://dx.doi.org/10.1007/s12193-008-0012-2.

Full text

APA, Harvard, Vancouver, ISO, and other styles

10

Gaouar, Lamia, Abdelkrim Benamar, Olivier Le Goaer, and Frédérique Biennier. "HCIDL: Human-computer interface description language for multi-target, multimodal, plastic user interfaces." Future Computing and Informatics Journal 3, no. 1 (June 2018): 110–30. http://dx.doi.org/10.1016/j.fcij.2018.02.001.

Full text

APA, Harvard, Vancouver, ISO, and other styles

11

Kim, Myeongseop, Eunjin Seong, Younkyung Jwa, Jieun Lee, and Seungjun Kim. "A Cascaded Multimodal Natural User Interface to Reduce Driver Distraction." IEEE Access 8 (2020): 112969–84. http://dx.doi.org/10.1109/access.2020.3002775.

Full text

APA, Harvard, Vancouver, ISO, and other styles

12

Choi, E. H. C., R. Taib, Y. Shi, and F. Chen. "Multimodal user interface for traffic incident management in control room." IET Intelligent Transport Systems 1, no. 1 (2007): 27. http://dx.doi.org/10.1049/iet-its:20060038.

Full text

APA, Harvard, Vancouver, ISO, and other styles

13

Hoffman, Donald D. "Sensory Experiences as Cryptic Symbols of a Multimodal User Interface." Activitas Nervosa Superior 52, no. 3-4 (September 2010): 95–104. http://dx.doi.org/10.1007/bf03379572.

Full text

APA, Harvard, Vancouver, ISO, and other styles

14

Diaz, Carlos, and Shahram Payandeh. "Multimodal Sensing Interface for Haptic Interaction." Journal of Sensors 2017 (2017): 1–24. http://dx.doi.org/10.1155/2017/2072951.

Full text

Abstract:

This paper investigates the integration of a multimodal sensing system for exploring limits of vibrato tactile haptic feedback when interacting with 3D representation of real objects. In this study, the spatial locations of the objects are mapped to the work volume of the user using a Kinect sensor. The position of the user’s hand is obtained using the marker-based visual processing. The depth information is used to build a vibrotactile map on a haptic glove enhanced with vibration motors. The users can perceive the location and dimension of remote objects by moving their hand inside a scanning region. A marker detection camera provides the location and orientation of the user’s hand (glove) to map the corresponding tactile message. A preliminary study was conducted to explore how different users can perceive such haptic experiences. Factors such as total number of objects detected, object separation resolution, and dimension-based and shape-based discrimination were evaluated. The preliminary results showed that the localization and counting of objects can be attained with a high degree of success. The users were able to classify groups of objects of different dimensions based on the perceived haptic feedback.

APA, Harvard, Vancouver, ISO, and other styles

15

Chandarana, Meghan, Erica L. Meszaros, Anna Trujillo, and B. Danette Allen. "Natural Language Based Multimodal Interface for UAV Mission Planning." Proceedings of the Human Factors and Ergonomics Society Annual Meeting 61, no. 1 (September 2017): 68–72. http://dx.doi.org/10.1177/1541931213601483.

Full text

Abstract:

As the number of viable applications for unmanned aerial vehicle (UAV) systems increases at an exponential rate, interfaces that reduce the reliance on highly skilled engineers and pilots must be developed. Recent work aims to make use of common human communication modalities such as speech and gesture. This paper explores a multimodal natural language interface that uses a combination of speech and gesture input modalities to build complex UAV flight paths by defining trajectory segment primitives. Gesture inputs are used to define the general shape of a segment while speech inputs provide additional geometric information needed to fully characterize a trajectory segment. A user study is conducted in order to evaluate the efficacy of the multimodal interface.

APA, Harvard, Vancouver, ISO, and other styles

16

KueBum Lee, SangHyeon Jin, and KwangSeok Hong. "An Implementation of Multimodal User Interface using Speech, Image and EOG." International Journal of Engineering and Industries 2, no. 2 (June 30, 2011): 76–87. http://dx.doi.org/10.4156/ijei.vol2.issue2.8.

Full text

APA, Harvard, Vancouver, ISO, and other styles

17

Takebayashi, Yoichi. "Spontaneous speech dialogue system TOSBURG IIâthe user-centered multimodal interface." Systems and Computers in Japan 26, no. 14 (1995): 77–91. http://dx.doi.org/10.1002/scj.4690261407.

Full text

APA, Harvard, Vancouver, ISO, and other styles

18

Rautiainen, Samu, Matteo Pantano, Konstantinos Traganos, Seyedamir Ahmadi, José Saenz, Wael M. Mohammed, and Jose L. Martinez Lastra. "Multimodal Interface for Human–Robot Collaboration." Machines 10, no. 10 (October 20, 2022): 957. http://dx.doi.org/10.3390/machines10100957.

Full text

Abstract:

Human–robot collaboration (HRC) is one of the key aspects of Industry 4.0 (I4.0) and requires intuitive modalities for humans to communicate seamlessly with robots, such as speech, touch, or bodily gestures. However, utilizing these modalities is usually not enough to ensure a good user experience and a consideration of the human factors. Therefore, this paper presents a software component, Multi-Modal Offline and Online Programming (M2O2P), which considers such characteristics and establishes a communication channel with a robot with predefined yet configurable hand gestures. The solution was evaluated within a smart factory use case in the Smart Human Oriented Platform for Connected Factories (SHOP4CF) EU project. The evaluation focused on the effects of the gesture personalization on the perceived workload of the users using NASA-TLX and the usability of the component. The results of the study showed that the personalization of the gestures reduced the physical and mental workload and was preferred by the participants, while overall the workload of the tasks did not significantly differ. Furthermore, the high system usability scale (SUS) score of the application, with a mean of 79.25, indicates the overall usability of the component. Additionally, the gesture recognition accuracy of M2O2P was measured as 99.05%, which is similar to the results of state-of-the-art applications.

APA, Harvard, Vancouver, ISO, and other styles

19

Obrenovic, Zeljko, and Dusan Starcevic. "Adapting the unified software development process for user interface development." Computer Science and Information Systems 3, no. 1 (2006): 33–52. http://dx.doi.org/10.2298/csis0601033o.

Full text

Abstract:

In this paper we describe how existing software developing processes, such as Rational Unified Process, can be adapted in order to allow disciplined and more efficient development of user interfaces. The main objective of this paper is to demonstrate that standard modeling environments, based on the UML, can be adapted and efficiently used for user interfaces development. We have integrated the HCI knowledge into developing processes by semantically enriching the models created in each of the process activities of the process. By using UML, we can make easier use of HCI knowledge for ordinary software engineers who, usually, are not familiar with results of HCI researches, so these results can have broader and more practical effects. By providing a standard means for representing human computer interaction, we can seamlessly transfer UML models of multimodal interfaces between design and specialized analysis tools. Standardization provides a significant driving force for further progress because it codifies best practices enables and encourages reuse, and facilitates inter working between complementary tools. Proposed solutions can be valuable for software developers, who can improve quality of user interfaces and their communication with user interface designers, as well as for human computer interaction researchers, who can use standard methods to include their results into software developing processes.

APA, Harvard, Vancouver, ISO, and other styles

20

Kazi, Zunaid, and Richard Foulds. "Knowledge Driven Planning and Multimodal Control of a Telerobot." Robotica 16, no. 5 (September 1998): 509–16. http://dx.doi.org/10.1017/s0263574798000666.

Full text

Abstract:

The Multimodal User Supervised Interface and Intelligent Control (MUSIIC) project focuses on a multimodal human-machine interface which addresses user need to manipulate familiar objects in an unstructured environment. The control of a robot by individuals with significant physical limitations presents a challenging problem of telemanipulation. This is addressed by a unique user-interface integrating the user's command (speech) and gestures (pointing) with autonomous planning techniques (knowledge-bases and 3-D vision). The resultant test-bed offers the opportunity to study telemanipulation by individuals with physical disabilities, and can be generalized to an effective technique for other, including remote and time-delayed, telemanipulation. This paper focuses on the knowledge-driven planning mechanism that is central to the MUSIIC system.

APA, Harvard, Vancouver, ISO, and other styles

21

Rigas, Dimitrios, and Badr Almutairi. "An Empirical Investigation into the Role of Avatars in Multimodal E-government Interfaces." International Journal of Sociotechnology and Knowledge Development 5, no. 1 (January 2013): 14–22. http://dx.doi.org/10.4018/jskd.2013010102.

Full text

Abstract:

Interfaces for e-government applications are becoming essential for the modern life. E-government uses web-based interfaces to deliver effective, efficient and convenient services to citizens, business and government. However, one of the main obstacles (or barriers) of using such applications is the lack of the user trust and usability. These issues are often neglected in the interfaces of e-government application. This paper describes an empirical comparative study that investigated the use of multimodal metaphors to enhance the usability and increase the user trust. Specific designs of multimodal metaphors were investigated to communicate information using a specially developed e-government interface platform. These designs involved facially animated expressive avatars and auditory stimuli in addition to the typical visual metaphors often used. An experiment was carried out with 30 users to explore and compare the usability and user performance. These results provided a set of empirically derived innovative guidelines for the design and use of these metaphors to generate more usable e-government interfaces. For example, when designing avatars as animated virtual messages or live mail in e-government interfaces, specific facial expression should be incorporated due to its positive influence in enhancing users‘ attitude towards the communication process.

APA, Harvard, Vancouver, ISO, and other styles

22

Bos, Edwin, Carla Huls, and Wim Claassen. "EDWARD: full integration of language and action in a multimodal user interface." International Journal of Human-Computer Studies 40, no. 3 (March 1994): 473–95. http://dx.doi.org/10.1006/ijhc.1994.1022.

Full text

APA, Harvard, Vancouver, ISO, and other styles

23

Crangle, Colleen. "Conversational interfaces to robots." Robotica 15, no. 1 (January 1997): 117–27. http://dx.doi.org/10.1017/s0263574797000143.

Full text

Abstract:

There is growing interest in robots that are designed specifically to interact with people and which respond to voice commands. Very little attention has been paid, however, to the kind of verbal interaction that is possible or desirable with robots. This paper presents recent work in multimodal interfaces that addresses this question. It proposes a new form of robot-user interface, namely a collaborative conversational interface. This article explains what collaborative conversational interfaces are, argues for their application in robots, and presents strategies for designing good conversational interfaces. It concludes with a discussion of the particular challenges faced in designing conversational interfaces for robots.

APA, Harvard, Vancouver, ISO, and other styles

24

Kasprzak, Włodzimierz, Wojciech Szynkiewicz, Maciej Stefańczyk, Wojciech Dudek, Maksym Figat, Maciej Węgierek, Dawid Seredyński, and Cezary Zieliński. "Agent Structure of Multimodal User Interface to the National Cybersecurity Platform – Part 1." Pomiary Automatyka Robotyka 23, no. 3 (September 30, 2019): 41–54. http://dx.doi.org/10.14313/par_233/41.

Full text

APA, Harvard, Vancouver, ISO, and other styles

25

Kasprzak, Włodzimierz, Wojciech Szynkiewicz, Maciej Stefańczyk, Wojciech Dudek, Maksym Figat, Maciej Węgierek, Dawid Seredyński, and Cezary Zieliński. "Agent Structure of Multimodal User Interface to the National Cybersecurity Platform – Part 2." Pomiary Automatyka Robotyka 23, no. 4 (December 30, 2019): 5–18. http://dx.doi.org/10.14313/par_234/5.

Full text

APA, Harvard, Vancouver, ISO, and other styles

26

Wang, Jian. "Integration model of eye-gaze, voice and manual response in multimodal user interface." Journal of Computer Science and Technology 11, no. 5 (September 1996): 512–18. http://dx.doi.org/10.1007/bf02947219.

Full text

APA, Harvard, Vancouver, ISO, and other styles

27

Faria, Brígida Mónica, Luís Paulo Reis, and Nuno Lau. "Knowledge Discovery and Multimodal Inputs for Driving an Intelligent Wheelchair." International Journal of Knowledge Discovery in Bioinformatics 2, no. 4 (October 2011): 18–34. http://dx.doi.org/10.4018/jkdb.2011100102.

Full text

Abstract:

Cerebral Palsy is defined as a group of permanent disorders in the development of movement and posture. The motor disorders in cerebral palsy are associated with deficits of perception, cognition, communication, and behaviour, which can affect autonomy and independence. The interface between the user and an intelligent wheelchair can be done with several input devices such as joysticks, microphones, and brain computer interfaces (BCI). BCI enables interaction between users and hardware systems through the recognition of brainwave activity. The current BCI systems have very low accuracy on the recognition of facial expressions and thoughts, making it difficult to use these devices to enable safe and robust commands of complex devices like an Intelligent Wheelchair. This paper presents an approach to expand the use of a brain computer interface for driving an intelligent wheelchair by patients suffering from cerebral palsy. The ability with the joystick, head movements, and voice inputs were tested, and the best possibility for driving the wheelchair is given to a specific user. Experiments were performed using 30 individuals suffering from IV and V degrees of cerebral palsy on the Gross Motor Function (GMF) measure. The results show that the pre-processing and variable selection methods are effective to improve the results of a commercial BCI product by 57%. With the developed system, it is also possible for users to perform a circuit in a simulated environment using just facial expressions and thoughts.

APA, Harvard, Vancouver, ISO, and other styles

28

de Ryckel, Xavier, Arthur Sluÿters, and Jean Vanderdonckt. "SnappView, a Software Development Kit for Supporting End-user Mobile Interface Review." Proceedings of the ACM on Human-Computer Interaction 6, EICS (June 14, 2022): 1–38. http://dx.doi.org/10.1145/3534527.

Full text

Abstract:

This paper presents SnappView, an open-source software development kit that facilitates end-user review of graphical user interfaces for mobile applications and streamlines their input into a continuous design life cycle. SnappView structures this user interface review process into four cumulative stages: (1) a developer creates a mobile application project with user interface code instrumented by only a few instructions governing SnappView and deploys the resulting application on an application store; (2) any tester, such as an end-user, a designer, a reviewer, while interacting with the instrumented user interface, shakes the mobile device to freeze and capture its screen and to provide insightful multimodal feedback such as textual comments, critics, suggestions, drawings by stroke gestures, voice or video records, with a level of importance; (3) the screenshot is captured with the application, browser, and status data and sent with the feedback to SnappView server; and (4) a designer then reviews collected and aggregated feedback data and passes them to the developer to address raised usability problems. Another cycle then initiates an iterative design. This paper presents the motivations and process for performing mobile application review based on SnappView. Based on this process, we deployed on the AppStore "WeTwo", a real-world mobile application to find various personal activities over a one-month period with 420 active users. This application served for a user experience evaluation conducted with N1=14 developers to reveal the advantages and shortcomings of the toolkit from a development point of view. The same application was also used in a usability evaluation conducted with N2=22 participants to reveal the advantages and shortcomings from an end-user viewpoint.

APA, Harvard, Vancouver, ISO, and other styles

29

Normand, Véronique, Didier Pernel, and Béatrice Bacconnet. "Speech-based Multimodal Interaction in Virtual Environments: Research at the Thomson-CSF Corporate Research Laboratories." Presence: Teleoperators and Virtual Environments 6, no. 6 (December 1997): 687–700. http://dx.doi.org/10.1162/pres.1997.6.6.687.

Full text

Abstract:

The Thomson-CSF Corporate Research Laboratories are investigating the issues of user-interface design, spoken and multimodal interaction design and realization in virtual environments. This paper describes our technical approach to speech-enabled multimodal virtual environments, based on our past achievements in the multimodal interaction domain, and presents our main supporting projects in this area. These projects involve augmented reality for maintenance, military situation building and assessment, and collaborative virtual environments.

APA, Harvard, Vancouver, ISO, and other styles

30

Ying, Fang Tian, Peng Cheng Zhu, Mi Lan Ye, Jing Chang Chen, Zhao He, and Yue Pan. "Bubble Journey: Multimodal Input Tools Design to Augment Sense Experience in Computer Game." Advanced Materials Research 102-104 (March 2010): 326–30. http://dx.doi.org/10.4028/www.scientific.net/amr.102-104.326.

Full text

Abstract:

In this paper, we discuss a novel approach to multimodal input design in Tangible User Interface (TUI). We present a prototype Bubble Journey, a game platform where users control the avatar in flash game by blowing a real handle. This computer game was combined multimodal input tool embedded sensor, which augment experience of user’s (children’s) five senses and body into game’s digital world with previous experience in daily life. Sensor embodied in multimodal input tool can convert data of sounds and movements produced by users (children) into digital signals to manipulate the virtual characters’ performance in the game.

APA, Harvard, Vancouver, ISO, and other styles

31

KETTEBEKOV, SANSHZAR, and RAJEEV SHARMA. "UNDERSTANDING GESTURES IN MULTIMODAL HUMAN COMPUTER INTERACTION." International Journal on Artificial Intelligence Tools 09, no. 02 (June 2000): 205–23. http://dx.doi.org/10.1142/s021821300000015x.

Full text

Abstract:

In recent years because of the advances in computer vision research, free hand gestures have been explored as a means of human-computer interaction (HCI). Gestures in combination with speech can be an important step toward natural, multimodal HCI. However, interpretation of gestures in a multimodal setting can be a particularly challenging problem. In this paper, we propose an approach for studying multimodal HCI in the context of a computerized map. An implemented testbed allows us to conduct user studies and address issues toward understanding of hand gestures in a multimodal computer interface. Absence of an adequate gesture classification in HCI makes gesture interpretation difficult. We formalize a method for bootstrapping the interpretation process by a semantic classification of gesture primitives in HCI context. We distinguish two main categories of gesture classes based on their spatio-temporal deixis. Results of user studies revealed that gesture primitives, originally extracted from weather map narration, form patterns of co-occurrence with speech parts in association with their meaning in a visual display control system. The results of these studies indicated two levels of gesture meaning: individual stroke and motion complex. These findings define a direction in approaching interpretation in natural gesture-speech interfaces.

APA, Harvard, Vancouver, ISO, and other styles

32

Seong, Ki Eun, Yu Jin Park, and Soon Ju Kang. "Design of Multimodal User Interface using Speech and Gesture Recognition for Wearable Watch Platform." KIISE Transactions on Computing Practices 21, no. 6 (June 15, 2015): 418–23. http://dx.doi.org/10.5626/ktcp.2015.21.6.418.

Full text

APA, Harvard, Vancouver, ISO, and other styles

33

Serefoglou, S., W. Lauer, A. Perneczky, T. Lutze, and K. Radermacher. "Multimodal user interface for a semi-robotic visual assistance system for image guided neurosurgery." International Congress Series 1281 (May 2005): 624–29. http://dx.doi.org/10.1016/j.ics.2005.03.292.

Full text

APA, Harvard, Vancouver, ISO, and other styles

34

Doumanis, Ioannis, and Serengul Smith. "An Empirical Investigation of the Impact of an Embodied Conversational Agent on the User's Perception and Performance with a Route-Finding Application." International Journal of Virtual and Augmented Reality 3, no. 2 (July 2019): 68–87. http://dx.doi.org/10.4018/ijvar.2019070106.

Full text

Abstract:

This article presents the design and evaluation of a gamified mobile tour guide prototype for route-finding in historical attractions. The mobile tour guide prototype uses photographs of landmarks carefully chosen to guide visitors on routes of cultural interest in the castle. It also features a fully multimodal interface using an Embodied Conversational Agent (ECA). The authors compared the impact of the multimodal interface to a non-multimodal control (narrated text) for the same navigational tasks. Participants experienced routes of variable difficulty (simple vs complex) in the lab under simulated mobile conditions. The experimental results substantiated aspects of the ECA-based interface related to visitors' subjective experiences, but not objective task performance. Finally, the authors discuss future work which will focus on running the second round of evaluations in the field with an improved version of the prototype, demonstrating features such as user planned routes and route re-calculation.

APA, Harvard, Vancouver, ISO, and other styles

35

YANAGIHARA, Yoshimasa, Sinyo MUTO, and Takao KAKIZAKI. "The Experimental Evaluation of User Interface of Multimodal Teaching Advisor using a Wearable Personal Computer." Journal of the Japan Society for Precision Engineering 67, no. 5 (2001): 739–43. http://dx.doi.org/10.2493/jjspe.67.739.

Full text

APA, Harvard, Vancouver, ISO, and other styles

36

Yu, Tianyou, Yuanqing Li, Jinyi Long, and Feng Li. "A Hybrid Brain-Computer Interface-Based Mail Client." Computational and Mathematical Methods in Medicine 2013 (2013): 1–9. http://dx.doi.org/10.1155/2013/750934.

Full text

Abstract:

Brain-computer interface-based communication plays an important role in brain-computer interface (BCI) applications; electronic mail is one of the most common communication tools. In this study, we propose a hybrid BCI-based mail client that implements electronic mail communication by means of real-time classification of multimodal features extracted from scalp electroencephalography (EEG). With this BCI mail client, users can receive, read, write, and attach files to their mail. Using a BCI mouse that utilizes hybrid brain signals, that is, motor imagery and P300 potential, the user can select and activate the function keys and links on the mail client graphical user interface (GUI). An adaptive P300 speller is employed for text input. The system has been tested with 6 subjects, and the experimental results validate the efficacy of the proposed method.

APA, Harvard, Vancouver, ISO, and other styles

37

Noto, Christopher T., Suleman Mazhar, James Gnadt, and Jagmeet S. Kanwal. "A flexible user-interface for audiovisual presentation and interactive control in neurobehavioral experiments." F1000Research 2 (June 6, 2013): 20. http://dx.doi.org/10.12688/f1000research.2-20.v2.

Full text

Abstract:

A major problem facing behavioral neuroscientists is a lack of unified, vendor-distributed data acquisition systems that allow stimulus presentation and behavioral monitoring while recording neural activity. Numerous systems perform one of these tasks well independently, but to our knowledge, a useful package with a straightforward user interface does not exist. Here we describe the development of a flexible, script-based user interface that enables customization for real-time stimulus presentation, behavioral monitoring and data acquisition. The experimental design can also incorporate neural microstimulation paradigms. We used this interface to deliver multimodal, auditory and visual (images or video) stimuli to a nonhuman primate and acquire single-unit data. Our design is cost-effective and works well with commercially available hardware and software. Our design incorporates a script, providing high-level control of data acquisition via a sequencer running on a digital signal processor to enable behaviorally triggered control of the presentation of visual and auditory stimuli. Our experiments were conducted in combination with eye-tracking hardware. The script, however, is designed to be broadly useful to neuroscientists who may want to deliver stimuli of different modalities using any animal model.

APA, Harvard, Vancouver, ISO, and other styles

38

Noto, Christopher T., Suleman Mazhar, James Gnadt, and Jagmeet S. Kanwal. "A flexible user-interface for audiovisual presentation and interactive control in neurobehavioral experiments." F1000Research 2 (June 10, 2014): 20. http://dx.doi.org/10.12688/f1000research.2-20.v3.

Full text

Abstract:

A major problem facing behavioral neuroscientists is a lack of unified, vendor-distributed data acquisition systems that allow stimulus presentation and behavioral monitoring while recording neural activity. Numerous systems perform one of these tasks well independently, but to our knowledge, a useful package with a straightforward user interface does not exist. Here we describe the development of a flexible, script-based user interface that enables customization for real-time stimulus presentation, behavioral monitoring and data acquisition. The experimental design can also incorporate neural microstimulation paradigms. We used this interface to deliver multimodal, auditory and visual (images or video) stimuli to a nonhuman primate and acquire single-unit data. Our design is cost-effective and works well with commercially available hardware and software. Our design incorporates a script, providing high-level control of data acquisition via a sequencer running on a digital signal processor to enable behaviorally triggered control of the presentation of visual and auditory stimuli. Our experiments were conducted in combination with eye-tracking hardware. The script, however, is designed to be broadly useful to neuroscientists who may want to deliver stimuli of different modalities using any animal model.

APA, Harvard, Vancouver, ISO, and other styles

39

Jones, Matt. "Classic and Alternative Mobile Search." International Journal of Mobile Human Computer Interaction 3, no. 1 (January 2011): 22–36. http://dx.doi.org/10.4018/jmhci.2011010102.

Full text

Abstract:

As mobile search turns into a mainstream activity, the author reflects on research that provides insights into the impact of current interfaces and pointers to yet unmet needs. Classic text dominated interface and interaction techniques are reviewed, showing how they can enhance the user experience. While today’s interfaces emphasise direct, query-result approaches, serving up discrete chunks of content, the author suggests an alternative set of features for future mobile search. With reference to example systems, the paper argues for indirect, continuous and multimodal approaches. Further, while almost all mobile search research has focused on the ‘developed’ world, the paper outlines challenges and impact of work targeted at ‘developing’ world contexts.

APA, Harvard, Vancouver, ISO, and other styles

40

Niu, Hongwei, Cees Van Leeuwen, Jia Hao, Guoxin Wang, and Thomas Lachmann. "Multimodal Natural Human–Computer Interfaces for Computer-Aided Design: A Review Paper." Applied Sciences 12, no. 13 (June 27, 2022): 6510. http://dx.doi.org/10.3390/app12136510.

Full text

Abstract:

Computer-aided design (CAD) systems have advanced to become a critical tool in product design. Nevertheless, they still primarily rely on the traditional mouse and keyboard interface. This limits the naturalness and intuitiveness of the 3D modeling process. Recently, a multimodal human–computer interface (HCI) has been proposed as the next-generation interaction paradigm. Widening the use of a multimodal HCI provides new opportunities for realizing natural interactions in 3D modeling. In this study, we conducted a literature review of a multimodal HCI for CAD to summarize the state-of-the-art research and establish a solid foundation for future research. We explore and categorize the requirements for natural HCIs and discuss paradigms for their implementation in CAD. Following this, factors to evaluate the system performance and user experience of a natural HCI are summarized and analyzed. We conclude by discussing challenges and key research directions for a natural HCI in product design to inspire future studies.

APA, Harvard, Vancouver, ISO, and other styles

41

Kaushik, Abhishek, Billy Jacob, and Pankaj Velavan. "An Exploratory Study on a Reinforcement Learning Prototype for Multimodal Image Retrieval Using a Conversational Search Interface." Knowledge 2, no. 1 (February 28, 2022): 116–38. http://dx.doi.org/10.3390/knowledge2010007.

Full text

Abstract:

In the realm of information, conversational search is a relatively new trend. In this study, we have developed, implemented, and evaluated a multiview conversational image search system to investigate user search behaviour. We have also explored the potential for reinforcement learning to learn from user search behaviour and support the user in the complex information seeking process. A conversational image search system may mimic a natural language discussion with a user via text or speech, and then assist the user in locating the required picture via a dialogue-based search. We modified and improved a dual-view search interface that displays discussions on one side and photos on the other. Based on the states, incentives, and dialogues in the initial run, we developed a reinforcement learning model and a customized search algorithm in the back end that predicts which reply and images would be provided to the user among a restricted set of fixed responses. Usability of the system was validated using methodologies such as Chatbot Usability Questionnaire, System Usability Scale, and User Experience Questionnaire, and the values were tabulated. The result of this usability experiment proved that most of the users found the system to be very usable and helpful for their image search.

APA, Harvard, Vancouver, ISO, and other styles

42

Muhammad Habib and Noor ul Qamar. "Multimodal Interaction Recognition Mechanism by Using Midas Featured By Data-Level and Decision-Level Fusion." Lahore Garrison University Research Journal of Computer Science and Information Technology 1, no. 2 (June 30, 2017): 41–51. http://dx.doi.org/10.54692/lgurjcsit.2017.010227.

Full text

Abstract:

Natural User Interfaces (NUI's) dealing with gestures is an alternative of traditional input devices on multi-touch panels. Rate of growth in the Sensor technology has increased the use of multiple sensors to deal with various monitoring and compatibility issues of machines. Research on data-level fusion models requires more focus on the fusion of multiple degradation-based sensor data. Midas, a novel declarative language to express multimodal interaction patterns has come up with the idea of developers required patterns description by employing multi-model interaction mechanism. The language as a base interface deals with minimum complexity issues like controlling inversion and intermediary states by means of data fusion, data processing and data selection provisioning high-level programming abstractions.

APA, Harvard, Vancouver, ISO, and other styles

43

Atzenbeck, Claus. "Interview with Beat Signer." ACM SIGWEB Newsletter, Winter (January 2021): 1–5. http://dx.doi.org/10.1145/3447879.3447881.

Full text

Abstract:

Beat Signer is Professor of Computer Science at the Vrije Universiteit Brussel (VUB) and co-director of the Web & Information Systems Engineering (WISE) research lab. He received a PhD in Computer Science from ETH Zurich where he has also been leading the Interactive Paper lab as a senior researcher for four years. He is an internationally distinguished expert in cross-media technologies and interactive paper solutions. His further research interests include human-information interaction, document engineering, data physicalisation, mixed reality as well as multimodal interaction. He has published more than 100 papers on these topics at international conferences and journals, and received multiple best paper awards. Beat has 20 years of experience in research on cross-media information management and multimodal user interfaces. As part of his PhD research, he investigated the use of paper as an interactive user interface and developed the resource-selector-link (RSL) hypermedia metamodel. With the interactive paper platform (iPaper), he strongly contributed to the interdisciplinary European Paper++ and PaperWorks research projects and the seminal research on paper-digital user interfaces led to innovative cross-media publishing solutions and novel forms of paper-based human-computer interaction. The RSL hypermedia metamodel is nowadays widely applied in his research lab and has, for example, been used for cross-media personal information management, an extensible cross-document link service, the MindXpres presentation platform as well as in a framework for cross-device and Internet of Things applications. For more details, please visit https://beatsigner.com.

APA, Harvard, Vancouver, ISO, and other styles

44

Fakhrurroja, Hanif, Carmadi Machbub, Ary Setijadi Prihatmanto, and Ayu Purwarianti. "Multimodal Fusion Algorithm and Reinforcement Learning-Based Dialog System in Human-Machine Interaction." International Journal on Electrical Engineering and Informatics 12, no. 4 (December 31, 2020): 1016–46. http://dx.doi.org/10.15676/ijeei.2020.12.4.19.

Full text

Abstract:

Studies on human-machine interaction system show positive results on system development accuracy. However, there are problems, especially using certain input modalities such as speech, gesture, face detection, and skeleton tracking. These problems include how to design an interface system for a machine to contextualize the existing conversations. Other problems include activating the system using various modalities, right multimodal fusion methods, machine understanding of human intentions, and methods for developing knowledge. This study developed a method of human-machine interaction system. It involved several stages, including a multimodal activation system, methods for recognizing speech modalities, gestures, face detection and skeleton tracking, multimodal fusion strategies, understanding human intent and Indonesian dialogue systems, as well as machine knowledge development methods and the right response. The research contributes to an easier and more natural humanmachine interaction system using multimodal fusion-based systems. The average accuracy rate of multimodal activation, testing dialogue system using Indonesian, gesture recognition interaction, and multimodal fusion is 87.42%, 92.11%, 93.54% and 93%, respectively. The level of user satisfaction towards the multimodal recognition-based human-machine interaction system developed was 95%. According to 76.2% of users, this interaction system was natural, while 79.4% agreed that the machine responded well to their wishes.

APA, Harvard, Vancouver, ISO, and other styles

45

Kobayashi, Toru. "An Application Framework for Trend Surfing System based on Multi-aspect, Multi-screen and Multimodal User Interface." Journal of Information Processing 23, no. 6 (2015): 795–803. http://dx.doi.org/10.2197/ipsjjip.23.795.

Full text

APA, Harvard, Vancouver, ISO, and other styles

46

Iliev, Yuliy, and Galina Ilieva. "A Framework for Smart Home System with Voice Control Using NLP Methods." Electronics 12, no. 1 (December 27, 2022): 116. http://dx.doi.org/10.3390/electronics12010116.

Full text

Abstract:

The proliferation of information technologies and the emergence of ubiquitous computing have quickly transformed electronic devices from isolated islands of data and control into interconnected parts of intelligent systems. These network-based systems have advanced features, including Internet of Things (IoT) sensors and actuators, multiple connectivity options and multimodal user interfaces, and they also enable remote monitoring and management. In order to develop a human machine interface of smart home systems with speech recognition, we propose a new IoT-fog-cloud framework using natural language processing (NLP) methods. The new methodology adds utterance to command transformation to the existing cloud-based speech-to-text and text-to-speech services. This approach is flexible and can be easily adapted for different types of automation systems and consumer electronics as well as to almost every non-tonal language not currently supported by online platforms for intent detection and classification. The proposed framework has been employed in the development of prototypes of voice user interface extension of existing smart security system via new service for speech intent recognition. Tests on the system were carried out and the obtained results show the effectiveness of the new voice communication option. The speech-based interface is reliable; it facilitates customers and improves their experience with smart home devices.

APA, Harvard, Vancouver, ISO, and other styles

47

Taghezout, Noria. "An Agent-Based Dialog System for Adaptive and Multimodal Interface: A Case Study." Advanced Materials Research 217-218 (March 2011): 578–83. http://dx.doi.org/10.4028/www.scientific.net/amr.217-218.578.

Full text

Abstract:

Graphical Interfaces using an agent-based dialog can handle errors and interruptions, and dynamically adapts to the current context and situation, the needs of the task performed, and the user model. This is especially true for the design of multimodal interfaces, where interaction designers need to physically explore and prototype new interaction modalities and therefore require development environments that especially support the interactivity and the dynamic of this creative development process. We argue that, in the domain of sophisticated human-machine interfaces, we can make use of the increasing tendency to design such interfaces as independent agents that themselves engage in an interactive dialogue (both graphical and linguistic) with their users. This paper focuses on the implementation of a flexible and robust dialogue system which integrates emotions and other influencing parameters in the dialogue flow. In order to achieve a higher degree of adaptability and multimodality, we present Spoken Language Dialogue System (SLDS) architecture. The manufacturing process of the oil plant (GLZ: Gas Liquefying Zone), is selected as an application domain in this study

APA, Harvard, Vancouver, ISO, and other styles

48

Kellmeyer, David, and Glenn A. Osga. "Usability Testing & Analysis of Advanced Multimodal Watchstation Functions." Proceedings of the Human Factors and Ergonomics Society Annual Meeting 44, no. 37 (July 2000): 654–57. http://dx.doi.org/10.1177/154193120004403729.

Full text

Abstract:

Usability testing is a well known method for obtaining user feedback during the design process, Dumas & Redish (1993), Gomoll (1996), Halgren & Smilowitz (1997), Treu (1994). Testing has been a continuous aspect of the Multimodal Watchstation (MMWS) conceptual design process with different usability testing techniques having been used through various portions of the design. Early cognitive walkthrough evaluation helped to focus design efforts toward complex or contusing portions of the interface. Lo-fidelity prototype design and testing then focused on improving these portions on an individual basis. Once satisfied, hi-fidelity prototype testing was used to determine how time and interaction workload affected performance. This paper presents the various usability techniques used throughout the design process, with some key results from each technique, including a discussion of project resource investment for various design prototypes. Results indicate that utilizing various usability testing techniques provides both time and cost effective answers throughout all stages of conceptual design.

APA, Harvard, Vancouver, ISO, and other styles

49

Zhao, Chen, Chuanqi Zheng, Leah Roldan, Thomas Shkurti, Ammar Nahari, Wyatt Newman, Dustin Tyler, Kiju Lee, and Michael Fu. "Adaptable Mixed-Reality Sensorimotor Interface for Human-Swarm Teaming: Person with Limb Loss Case Study and Field Experiments." Field Robotics 3, no. 1 (January 10, 2023): 243–65. http://dx.doi.org/10.55417/fr.2023007.

Full text

Abstract:

This paper presents the design, evaluation, and field experiment of the innovative Adapt- able Human-Swarm Teaming ( α -SWAT) interface developed to support military field operations. Human-swarm teaming requires collaboration between a team of humans and a team of robotic agents for strategic decision-making and task performance. α -SWAT allows multiple human team members with different roles, physical capabilities, or preferences to interact with the swarm via a configurable, multimodal user interface (UI). The system has an embedded task allocation algorithm for the rapid assignment of tasks created by the mission planner to the swarm. The multimodal UI supports swarm visualization via a mixed reality display or a conventional 2D display, human gesture inputs via a camera or an electromyography device, tactile feedback via a vibration motor or implanted peripheral nerve interface, and audio feedback. In particular, the UI system interfacing with the implanted electrodes through a neural interface enables gesture detection and tactile feedback for individuals with upper limb amputation to participate in human-swarm teaming. The multimodality of α -SWAT’s UI adapts to the needs of three different roles of the human team members: Swarm Planner, Swarm Tactician Rear, and Swarm Tactician Forward. A case study evaluated the functionality and usability of α -SWAT to enable a participant with limb loss and an implanted neural interface to assign three tasks to a simulated swarm of 150 robotic agents. α -SWAT was also used to visualize live telemetry from 40 veridical robotic agents for multiple simultaneous human participants at a field experiment.

APA, Harvard, Vancouver, ISO, and other styles

50

Khelifi, Adel, Gabriele Ciccone, Mark Altaweel, Tasnim Basmaji, and Mohammed Ghazal. "Autonomous Service Drones for Multimodal Detection and Monitoring of Archaeological Sites." Applied Sciences 11, no. 21 (November 5, 2021): 10424. http://dx.doi.org/10.3390/app112110424.

Full text

Abstract:

Constant detection and monitoring of archaeological sites and objects have always been an important national goal for many countries. The early identification of changes is crucial to preventive conservation. Archaeologists have always considered using service drones to automate collecting data on and below the ground surface of archaeological sites, with cost and technical barriers being the main hurdles against the wide-scale deployment. Advances in thermal imaging, depth imaging, drones, and artificial intelligence have driven the cost down and improved the quality and volume of data collected and processed. This paper proposes an end-to-end framework for archaeological sites detection and monitoring using autonomous service drones. We mount RGB, depth, and thermal cameras on an autonomous drone for low-altitude data acquisition. To align and aggregate collected images, we propose two-stage multimodal depth-to-RGB and thermal-to-RGB mosaicking algorithms. We then apply detection algorithms to the stitched images to identify change regions and design a user interface to monitor these regions over time. Our results show we can create overlays of aligned thermal and depth data on RGB mosaics of archaeological sites. We tested our change detection algorithm and found it has a root mean square error of 0.04. To validate the proposed framework, we tested our thermal image stitching pipeline against state-of-the-art commercial software. We cost-effectively replicated its functionality while adding a new depth-based modality and created a user interface for temporally monitoring changes in multimodal views of archaeological sites.

APA, Harvard, Vancouver, ISO, and other styles

Journal articles on the topic 'Multimodal user interface'

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles