Thèses sur le sujet « VIRTUAL REALITY 3D GRAPHIC INTERFACES »

Computers are developed continuously to satisfy the human demands, and typical tools used everywhere for ranging from daily life usage to all kinds of research. Virtual Reality (VR), a virtual environment simulated to present physical presence in the real word and imaginary worlds, has been widely applied to simulate the virtual environment. People’s feeling is limited to visual perception when only computers are applied for simulations, since computers are limited to display visualization of data, while human senses include sight, smell, hearing, taste, touch and so on. Other devices can be applied, such as haptics, a device for sense of touch, to enhance the human perception in virtual environment. A good way to apply VR applications is to place them in a virtual display system, a system with multiply tools displays a virtual environment with experiencing different human senses, to enhance the people’s feeling of being immersed in a virtual environment. Such virtual display systems include VR dome, recursive acronym CAVE, VR workbench, VR workstation and so on. Menus with lots of advantages in manipulating applications are common in conventional systems, operating systems or other systems in computers. Normally a system will not be usable without them. Although VR applications are more natural and intuitive, they are much less or not usable without menus. But very few studies have focused on user interfaces in VR. This situation motivates us working further in this area. We want to create two models on different purposes. One is inspired from menus in conventional system and the sense of touch. And the other one is designed based on the spatial presence of VR. The first model is a two-dimensional pie menu in pop-up style with spring force feedback. This model is in a pie shape with eight options on the root menu. And there is a pop-up style hierarchical menu belongs to each option on the root menu. When the haptics device is near an option on the root menu, the spring force will force the haptics device towards to the center of the option and that option will be selected, and then the sub menu with nine options will pop up. The pie shape together with the spring force effect is expected to both increase the speed of selection and decrease the error rate of selection. The other model is a semiautomatic three-dimensional cube menu. This cube menu is designed with a aim to provide a simple, elegant, efficient and accurate user interface approach. This model is designed with four faces, including the front, back, left and right faces of the cube. Each face represents a category and has nine widgets. Users can make selections in different categories. An efficient way to change between categories is to rotate the cube automatically. Thus, a navigable rotation animation system is built and is manipulating the cube rotate horizontally for ninety degrees each time, so one of the faces will always face users. These two models are built under H3DAPI, an open source haptics software development platform with UI toolkit, a user interface toolkit. After the implementation, we made a pilot study, which is a formative study, to evaluate the feasibility of both menus. This pilot study includes a list of tasks for each menu, a questionnaire regards to the menu performance for each subject and a discussion with each subject. Six students participated as test subjects. In the pie menu, most of the subjects feel the spring force guides them to the target option and they can control the haptics device comfortably under such force. In the cube menu, the navigation rotation system works well and the cube rotates accurately and efficiently. The results of the pilot study show the models work as we initially expected. The recorded task completion time for each menu shows that with the same amount of tasks and similar difficulties, subjects spent more time on the cube menu than on the pie menu. This may implicate that pie menu is a faster approach comparing to the cube menu. We further consider that both the pie shape and force feedback may help reducing the selection time. The result for the option selection error rate test on the cube menu may implicates that option selection without any force feedback may also achieve a considerable good effect. Through the answers from the questionnaire for each subject, both menus are comfortable to use and in good control.

The aim of this project is to design and create application for the visualization of interconnected video data. Visualization takes place in 3D space and seeks to exploit its advantages (such as depth perception). This document contains survey to different categories of spatial user interfaces. In addition, includes three possible designs of user interfaces and control. Implementation details and made usability tests are also described. Application is implemented in C++ using Open Inventor. The document includes evaluation of the results and made tests.

This thesis is framed within the field of 3D Character Animation. Virtual characters are used in many Human Computer Interaction applications such as video games and serious games. Within these virtual worlds they move and act in similar ways to humans controlled by users through some form of interface or by artificial intelligence. This work addresses the challenges of developing smoother movements and more natural behaviors driving motions in real-time, intuitively, and accurately. The interaction between virtual characters and intelligent objects will also be explored. With these subjects researched the work will contribute to creating more responsive, expressive, and tangible virtual characters. The navigation within virtual worlds uses locomotion such as walking, running, etc. To achieve maximum realism, actors' movements are captured and used to animate virtual characters. This is the philosophy of motion graphs: a structure that embeds movements where the continuous motion stream is generated from concatenating motion pieces. However, locomotion synthesis, using motion graphs, involves a tradeoff between the number of possible transitions between different kinds of locomotion, and the quality of these, meaning smooth transition between poses. To overcome this drawback, we propose the method of progressive transitions using Body Part Motion Graphs (BPMGs). This method deals with partial movements, and generates specific, synchronized transitions for each body part (group of joints) within a window of time. Therefore, the connectivity within the system is not linked to the similarity between global poses allowing us to find more and better quality transition points while increasing the speed of response and execution of these transitions in contrast to standard motion graphs method. Secondly, beyond getting faster transitions and smoother movements, virtual characters also interact with each other and with users by speaking. This interaction requires the creation of appropriate gestures according to the voice that they reproduced. Gestures are the nonverbal language that accompanies voiced language. The credibility of virtual characters when speaking is linked to the naturalness of their movements in sync with the voice in speech and intonation. Consequently, we analyzed the relationship between gestures, speech, and the performed gestures according to that speech. We defined intensity indicators for both gestures (GSI, Gesture Strength Indicator) and speech (PSI, Pitch Strength Indicator). We studied the relationship in time and intensity of these cues in order to establish synchronicity and intensity rules. Later we adapted the mentioned rules to select the appropriate gestures to the speech input (tagged text from speech signal) in the Gesture Motion Graph (GMG). The evaluation of resulting animations shows the importance of relating the intensity of speech and gestures to generate believable animations beyond time synchronization. Subsequently, we present a system that leads automatic generation of gestures and facial animation from a speech signal: BodySpeech. This system also includes animation improvements such as: increased use of data input, more flexible time synchronization, and new features like editing style of output animations. In addition, facial animation also takes into account speech intonation. Finally, we have moved virtual characters from virtual environments to the physical world in order to explore their interaction possibilities with real objects. To this end, we present AvatARs, virtual characters that have tangible representation and are integrated into reality through augmented reality apps on mobile devices. Users choose a physical object to manipulate in order to control the animation. They can select and configure the animation, which serves as a support for the virtual character represented. Then, we explored the interaction of AvatARs with intelligent physical objects like the Pleo social robot. Pleo is used to assist hospitalized children in therapy or simply for playing. Despite its benefits, there is a lack of emotional relationship and interaction between the children and Pleo which makes children lose interest eventually. This is why we have created a mixed reality scenario where Vleo (AvatAR as Pleo, virtual element) and Pleo (real element) interact naturally. This scenario has been tested and the results conclude that AvatARs enhances children's motivation to play with Pleo, opening a new horizon in the interaction between virtual characters and robots.
Aquesta tesi s'emmarca dins del món de l'animació de personatges virtuals tridimensionals. Els personatges virtuals s'utilitzen en moltes aplicacions d'interacció home màquina, com els videojocs o els serious games, on es mouen i actuen de forma similar als humans dins de mons virtuals, i on són controlats pels usuaris per mitjà d'alguna interfície, o d'altra manera per sistemes intel·ligents. Reptes com aconseguir moviments fluids i comportament natural, controlar en temps real el moviment de manera intuitiva i precisa, i inclús explorar la interacció dels personatges virtuals amb elements físics intel·ligents; són els que es treballen a continuació amb l'objectiu de contribuir en la generació de personatges virtuals responsius, expressius i tangibles. La navegació dins dels mons virtuals fa ús de locomocions com caminar, córrer, etc. Per tal d'aconseguir el màxim de realisme, es capturen i reutilitzen moviments d'actors per animar els personatges virtuals. Així funcionen els motion graphs, una estructura que encapsula moviments i per mitjà de cerques dins d'aquesta, els concatena creant un flux continu. La síntesi de locomocions usant els motion graphs comporta un compromís entre el número de transicions entre les diferents locomocions, i la qualitat d'aquestes (similitud entre les postures a connectar). Per superar aquest inconvenient, proposem el mètode transicions progressives usant Body Part Motion Graphs (BPMGs). Aquest mètode tracta els moviments de manera parcial, i genera transicions específiques i sincronitzades per cada part del cos (grup d'articulacions) dins d'una finestra temporal. Per tant, la conectivitat del sistema no està lligada a la similitud de postures globals, permetent trobar més punts de transició i de més qualitat, i sobretot incrementant la rapidesa en resposta i execució de les transicions respecte als motion graphs estàndards. En segon lloc, més enllà d'aconseguir transicions ràpides i moviments fluids, els personatges virtuals també interaccionen entre ells i amb els usuaris parlant, creant la necessitat de generar moviments apropiats a la veu que reprodueixen. Els gestos formen part del llenguatge no verbal que acostuma a acompanyar a la veu. La credibilitat dels personatges virtuals parlants està lligada a la naturalitat dels seus moviments i a la concordança que aquests tenen amb la veu, sobretot amb l'entonació d'aquesta. Així doncs, hem realitzat l'anàlisi de la relació entre els gestos i la veu, i la conseqüent generació de gestos d'acord a la veu. S'han definit indicadors d'intensitat tant per gestos (GSI, Gesture Strength Indicator) com per la veu (PSI, Pitch Strength Indicator), i s'ha estudiat la relació entre la temporalitat i la intensitat de les dues senyals per establir unes normes de sincronia temporal i d'intensitat. Més endavant es presenta el Gesture Motion Graph (GMG), que selecciona gestos adients a la veu d'entrada (text anotat a partir de la senyal de veu) i les regles esmentades. L'avaluació de les animaciones resultants demostra la importància de relacionar la intensitat per generar animacions cre\"{ibles, més enllà de la sincronització temporal. Posteriorment, presentem un sistema de generació automàtica de gestos i animació facial a partir d'una senyal de veu: BodySpeech. Aquest sistema també inclou millores en l'animació, major reaprofitament de les dades d'entrada i sincronització més flexible, i noves funcionalitats com l'edició de l'estil les animacions de sortida. A més, l'animació facial també té en compte l'entonació de la veu. Finalment, s'han traslladat els personatges virtuals dels entorns virtuals al món físic per tal d'explorar les possibilitats d'interacció amb objectes reals. Per aquest fi, presentem els AvatARs, personatges virtuals que tenen representació tangible i que es visualitzen integrats en la realitat a través d'un dispositiu mòbil gràcies a la realitat augmentada. El control de l'animació es duu a terme per mitjà d'un objecte físic que l'usuari manipula, seleccionant i parametritzant les animacions, i que al mateix temps serveix com a suport per a la representació del personatge virtual. Posteriorment, s'ha explorat la interacció dels AvatARs amb objectes físics intel·ligents com el robot social Pleo. El Pleo s'utilitza per a assistir a nens hospitalitzats en teràpia o simplement per jugar. Tot i els seus beneficis, hi ha una manca de relació emocional i interacció entre els nens i el Pleo que amb el temps fa que els nens perdin l'interès en ell. Així doncs, hem creat un escenari d'interacció mixt on el Vleo (un AvatAR en forma de Pleo; element virtual) i el Pleo (element real) interactuen de manera natural. Aquest escenari s'ha testejat i els resultats conclouen que els AvatARs milloren la motivació per jugar amb el Pleo, obrint un nou horitzó en la interacció dels personatges virtuals amb robots.
Esta tesis se enmarca dentro del mundo de la animación de personajes virtuales tridimensionales. Los personajes virtuales se utilizan en muchas aplicaciones de interacción hombre máquina, como los videojuegos y los serious games, donde dentro de mundo virtuales se mueven y actúan de manera similar a los humanos, y son controlados por usuarios por mediante de alguna interfaz, o de otro modo, por sistemas inteligentes. Retos como conseguir movimientos fluidos y comportamiento natural, controlar en tiempo real el movimiento de manera intuitiva y precisa, y incluso explorar la interacción de los personajes virtuales con elementos físicos inteligentes; son los que se trabajan a continuación con el objetivo de contribuir en la generación de personajes virtuales responsivos, expresivos y tangibles. La navegación dentro de los mundos virtuales hace uso de locomociones como andar, correr, etc. Para conseguir el máximo realismo, se capturan y reutilizan movimientos de actores para animar los personajes virtuales. Así funcionan los motion graphs, una estructura que encapsula movimientos y que por mediante búsquedas en ella, los concatena creando un flujo contínuo. La síntesi de locomociones usando los motion graphs comporta un compromiso entre el número de transiciones entre las distintas locomociones, y la calidad de estas (similitud entre las posturas a conectar). Para superar este inconveniente, proponemos el método transiciones progresivas usando Body Part Motion Graphs (BPMGs). Este método trata los movimientos de manera parcial, y genera transiciones específicas y sincronizadas para cada parte del cuerpo (grupo de articulaciones) dentro de una ventana temporal. Por lo tanto, la conectividad del sistema no está vinculada a la similitud de posturas globales, permitiendo encontrar más puntos de transición y de más calidad, incrementando la rapidez en respuesta y ejecución de las transiciones respeto a los motion graphs estándards. En segundo lugar, más allá de conseguir transiciones rápidas y movimientos fluídos, los personajes virtuales también interaccionan entre ellos y con los usuarios hablando, creando la necesidad de generar movimientos apropiados a la voz que reproducen. Los gestos forman parte del lenguaje no verbal que acostumbra a acompañar a la voz. La credibilidad de los personajes virtuales parlantes está vinculada a la naturalidad de sus movimientos y a la concordancia que estos tienen con la voz, sobretodo con la entonación de esta. Así pues, hemos realizado el análisis de la relación entre los gestos y la voz, y la consecuente generación de gestos de acuerdo a la voz. Se han definido indicadores de intensidad tanto para gestos (GSI, Gesture Strength Indicator) como para la voz (PSI, Pitch Strength Indicator), y se ha estudiado la relación temporal y de intensidad para establecer unas reglas de sincronía temporal y de intensidad. Más adelante se presenta el Gesture Motion Graph (GMG), que selecciona gestos adientes a la voz de entrada (texto etiquetado a partir de la señal de voz) y las normas mencionadas. La evaluación de las animaciones resultantes demuestra la importancia de relacionar la intensidad para generar animaciones creíbles, más allá de la sincronización temporal. Posteriormente, presentamos un sistema de generación automática de gestos y animación facial a partir de una señal de voz: BodySpeech. Este sistema también incluye mejoras en la animación, como un mayor aprovechamiento de los datos de entrada y una sincronización más flexible, y nuevas funcionalidades como la edición del estilo de las animaciones de salida. Además, la animación facial también tiene en cuenta la entonación de la voz. Finalmente, se han trasladado los personajes virtuales de los entornos virtuales al mundo físico para explorar las posibilidades de interacción con objetos reales. Para este fin, presentamos los AvatARs, personajes virtuales que tienen representación tangible y que se visualizan integrados en la realidad a través de un dispositivo móvil gracias a la realidad aumentada. El control de la animación se lleva a cabo mediante un objeto físico que el usuario manipula, seleccionando y configurando las animaciones, y que a su vez sirve como soporte para la representación del personaje. Posteriormente, se ha explorado la interacción de los AvatARs con objetos físicos inteligentes como el robot Pleo. Pleo se utiliza para asistir a niños en terapia o simplemente para jugar. Todo y sus beneficios, hay una falta de relación emocional y interacción entre los niños y Pleo que con el tiempo hace que los niños pierdan el interés. Así pues, hemos creado un escenario de interacción mixto donde Vleo (AvatAR en forma de Pleo; virtual) y Pleo (real) interactúan de manera natural. Este escenario se ha testeado y los resultados concluyen que los AvatARs mejoran la motivación para jugar con Pleo, abriendo un nuevo horizonte en la interacción de los personajes virtuales con robots.

La navigation à la première personne en Environnement Virtuel (EV) est essentielle à beaucoup d'applications de Réalité Virtuelle (RV), comme les simulations d'entraînement ou les visites virtuelles de musées ou de projets architecturaux. Les techniques de navigation doivent fournir un moyen efficace et écologique d'explorer les EV. De plus, comme dans toute autre application de RV, l'immersion des utilisateurs est aussi primordiale pour obtenir une bonne simulation. Dans cette thèse, nous avons proposé des nouvelles techniques d'interaction 3D et de retour sensoriel pour améliorer la navigation. Ainsi, nos contributions peuvent être découpées en deux parties : (1) une nouvelle technique d'interaction pour une navigation efficace et écologique en RV et (2) de nouvelles techniques de retour sensoriel conçues pour améliorer l'immersion et la sensation de marche des utilisateurs durant la navigation. Pour chacune des techniques proposées, nous avons conduit des évaluations poussées afin de valider que nos techniques atteignent leurs objectifs. Dans la première partie, nous avons proposé une nouvelle technique de navigation pour la RV, le Shake-Your-Head (SYH). Cette technique suit les mouvements de la tête de l'utilisateur lorsqu'il marche sûr place devant son écran afin de produire une navigation simulant la marche, ainsi que les sauts ou ramper. Nous avons trouvé que notre technique peut être utilisée efficacement sur des trajectoires complexes et est simple d'apprentissage. De plus, cette technique a été très appréciée par les utilisateurs. Dans la seconde partie, nous avons proposé une technique, les King Kong Effects (KKE), pour simuler les informations visuelles et vibrotactiles produites à chaque pas. Nous avons également proposé de nouveaux Mouvements de Caméra (MC) améliorés pour simuler les mouvements de la têtes lors de la marche, de la course et du sprint. De plus, nos MC sont adaptés à l'âge, au genre, au poids et à la condition physique de l'humain virtuel, ainsi qu'à la pente dans l'EV. Les KKE améliorent la sensation de marcher dans l'EV et ont également été très appréciés des utilisateurs. De plus, nous avons montrés que les différents paramètres des MC sont correctement perçus par les utilisateurs.

" Three dimensional travel in immersive virtual environments (IVE) has been a difficult problem since the beginning of virtual reality (VR), basically due to the difficulty of designing an intuitive, efficient, and precise three degrees of freedom (DOF) interface which can map the user's finite local movements in the real world to a potentially infinite virtual space. Inspired by the Silver Surfer Sci-Fi movie and the popularity of the Nintendo Wii Balance Board interface, a surfboard interface appears to be a good solution to this problem. Based on this idea, I designed and developed a VR Silver Surfer system which allows a user to surf in the sky of an infinite virtual environment, using either an isometric balance board or an elastic tilt board. Although the balance board is the industrial standard of board interface, the tilt board seems to provide the user more intuitive, realistic and enjoyable experiences, without any sacrifice of efficiency or precision. To validate this hypothesis we designed and conducted a user study that compared the two board interfaces in three independent experiments that break the travel procedure into separate DOFs. The results showed that in all experiments, the tilt board was not only as efficient and precise as the balance board, but also more intuitive, realistic and fun. In addition, despite the popularity of the balance board in the game industry, most subjects in the study preferred the tilt board in general, and in fact complained that the balance board could have been the cause of possible motion sickness. "

Walking is a natural part of our lives and is also becoming increasingly common in mixed reality. Wireless headsets and improved tracking systems allow us to easily navigate real and virtual environments by walking. In spite of the benefits, walking brings challenges to the design of new systems. In particular, designers must be aware of cognitive and motor requirements so that walking does not negatively impact the main task. Unfortunately, those demands are not yet fully understood. In this dissertation, we present new scientific evidence, interaction designs, and analysis of the role of walking in different mixed reality applications. We evaluated the difference in performance of users walking vs. manipulating a dataset during visual analysis. This is an important task, since virtual reality is increasingly being used as a way to make sense of progressively complex datasets. Our findings indicate that neither option is absolutely better: the optimal design choice should consider both user's experience with controllers and user's inherent spatial ability. Participants with reasonable game experience and low spatial ability performed better using the manipulation technique. However, we found that walking can still enable higher performance for participants with low spatial ability and without significant game experience. In augmented reality, specifying points in space is an essential step to create content that is registered with the world. However, this task can be challenging when information about the depth or geometry of the target is not available. We evaluated different augmented reality techniques for point marking that do not rely on any model of the environment. We found that triangulation by physically walking between points provides higher accuracy than purely perceptual methods. However, precision may be affected by head pointing tremors. To increase the precision, we designed a new technique that uses multiple samples to obtain a better estimate of the target position. This technique can also be used to mark points while walking. The effectiveness of this approach was demonstrated with a controlled augmented reality simulation and actual outdoor tests. Moving into the future, augmented reality will eventually replace our mobile devices as the main method of accessing information. Nonetheless, to achieve its full potential, augmented reality interfaces must support the fluid way we move in the world. We investigated the potential of adaptation in achieving this goal. We conceived and implemented an adaptive workspace system, based in the study of the design space and through user contextual studies. Our final design consists in a minimum set of techniques to support mobility and integration with the real world. We also identified a set of key interaction patterns and desirable properties of adaptation-based techniques, which can be used to guide the design of the next-generation walking-centered workspaces.
Ph. D.

The research on natural interaction has been growing significantly since, with the spread of personal computers, there is an increasing demand for interfaces that maximize productivity. Among these we can highlight the interfaces Virtual Reality and Augmented Reality (KIRNER, TORI e SISCOUTO, 2006), where users can perform simple tasks such as choosing a 3D object translation, rotation and scaling in 3D object. These tasks are usually performed through devices such as the keyboard and mouse, and can, thus, a loss of immersion happen by the user in the virtual environment. Thus, investigating methodologies for natural interaction in these environments, can help increase user immersion in the virtual environment. Another issue that have been the focus of many researches are Collaborative Virtual Environments (KIRNER e TORI, 2004), where it is possible for users to communicate and share information. These users can be physically close or not. The main focus of this work is precisely the communication among remotely dispersed users. Thus, it is intended with this work, proposing a system where it is possible to manipulate 3D objects using natural gestures and share data remotely dispersed users.
As pesquisas na área de interação natural vêm crescendo significativamente, pois, com a disseminação de computadores pessoais, existe uma demanda crescente por interfaces que maximizem a produtividade. Dentre essas interfaces podemos destacar a Realidade Virtual e a Realidade Aumentada (KIRNER, TORI e SISCOUTO, 2006), de forma que os usuários possam realizar tarefas simples como: escolha de um objeto 3D, translação, rotação e mudança de escala neste objeto 3D. Essas tarefas normalmente são realizadas através de dispositivos como o teclado e o mouse, podendo, dessa maneira, acontecer uma perda de imersão por parte do usuário no ambiente virtual. Sendo assim, investigar metodologias para interação natural nesses ambientes, pode ajudar a aumentar a imersão do usuário, no ambiente virtual. Outra questão que vem sendo o foco de muitas pesquisas são os Ambientes Virtuais Colaborativos (KIRNER e TORI, 2004), que permitem aos usuários se comunicarem e compartilhar informações. Esses usuários podem estar próximos fisicamente ou não. O foco principal desse trabalho é justamente a comunicação entre usuários dispersos remotamente. Dessa maneira, pretende-se, com esse trabalho, propor um sistema que possibilite a manipulação objetos 3D através de gestos naturais e compartilhar dados com usuários dispersos remotamente.
Mestre em Ciências

Améliorer l'interaction entre un utilisateur et un environnement 3D est un défi de recherche primordial pour le développement positif des technologies 3D interactives dans de nombreux domaines de nos sociétés, comme l'éducation. Dans ce document, je présente des interfaces utilisateur 3D que nous avons développées et qui contribuent à cette quête générale. Le premier chapitre se concentre sur l'interaction 3D pour des terminaux mobiles. En particulier, je présente des techniques dédiées à l'interaction à partir de touches, et à partir de gestes sur les écrans tactiles des terminaux mobiles. Puis, je présente deux prototypes à plusieurs degrés de liberté basés sur l'utilisation de flux vidéos. Dans le deuxième chapitre, je me concentre sur l'interaction 3D avec les écrans tactiles en général (tables, écrans interactifs). Je présente Navidget, un exemple de technique d'interaction dédié au controle de la caméra virtuelle à partir de gestes 2D, et je discute des défis de l'interaction 3D sur des écrans multi-points. Finalement, le troisième chapitre de ce document est dédié aux environnements virtuels immersifs, avec une coloration spéciale vers les interfaces musicales. Je présente les nouvelles directions que nous avons explorées pour améliorer l'interaction entre des musiciens, le public, le son, et les environements 3D interactifs. Je conclue en discutant du futur des interfaces utilisateur 3D.

Abstract The topic of this work is the transformation of real-world human activity into virtual environments. More specifically, the topic is the process of identifying various aspects of visible human activity with sensor networks and studying the different ways how the identified activity can be visualized in a virtual environment. The transformation of human activities into virtual environments is a rather new research area. While there is existing research on sensing and visualizing human activity in virtual environments, the focus of the research is carried out usually within a specific type of human activity, such as basic actions and locomotion. However, different types of sensors can provide very different human activity data, as well as lend itself to very different use-cases. This work is among the first to study the transformation of human activities on a larger scale, comparing various types of transformations from multiple theoretical viewpoints. This work utilizes constructs built for use-cases that require the transformation of human activity for various purposes. Each construct is a mixed reality application that utilizes a different type of source data and visualizes human activity in a different way. The constructs are evaluated from practical as well as theoretical viewpoints. The results imply that different types of activity transformations have significantly different characteristics. The most distinct theoretical finding is that there is a relationship between the level of detail of the transformed activity, specificity of the sensors involved and the extent of world knowledge required to transform the activity. The results also provide novel insights into using human activity transformations for various practical purposes. Transformations are evaluated as control devices for virtual environments, as well as in the context of visualization and simulation tools in elderly home care and urban studies
Tiivistelmä Tämän väitöskirjatyön aiheena on ihmistoiminnan muuntaminen todellisesta maailmasta virtuaalitodellisuuteen. Työssä käsitellään kuinka näkyvästä ihmistoiminnasta tunnistetaan sensoriverkkojen avulla erilaisia ominaisuuksia ja kuinka nämä ominaisuudet voidaan esittää eri tavoin virtuaaliympäristöissä. Ihmistoiminnan muuntaminen virtuaaliympäristöihin on kohtalaisen uusi tutkimusalue. Olemassa oleva tutkimus keskittyy yleensä kerrallaan vain tietyntyyppisen ihmistoiminnan, kuten perustoimintojen tai liikkumisen, tunnistamiseen ja visualisointiin. Erilaiset anturit ja muut datalähteet pystyvät kuitenkin tuottamaan hyvin erityyppistä dataa ja siten soveltuvat hyvin erilaisiin käyttötapauksiin. Tämä työ tutkii ensimmäisten joukossa ihmistoiminnan tunnistamista ja visualisointia virtuaaliympäristössä laajemmassa mittakaavassa ja useista teoreettisista näkökulmista tarkasteltuna. Työssä hyödynnetään konstrukteja jotka on kehitetty eri käyttötapauksia varten. Konstruktit ovat sekoitetun todellisuuden sovelluksia joissa hyödynnetään erityyppistä lähdedataa ja visualisoidaan ihmistoimintaa eri tavoin. Konstrukteja arvioidaan sekä niiden käytännön sovellusalueen, että erilaisten teoreettisten viitekehysten kannalta. Tulokset viittaavat siihen, että erilaisilla muunnoksilla on selkeästi erityyppiset ominaisuudet. Selkein teoreettinen löydös on, että mitä yksityiskohtaisemmasta toiminnasta on kyse, sitä vähemmän tunnistuksessa voidaan hyödyntää kontekstuaalista tietoa tai tavanomaisia datalähteitä. Tuloksissa tuodaan myös uusia näkökulmia ihmistoiminnan visualisoinnin hyödyntämisestä erilaisissa käytännön sovelluskohteissa. Sovelluskohteina toimivat ihmiskehon käyttäminen ohjauslaitteena sekä ihmistoiminnan visualisointi ja simulointi kotihoidon ja kaupunkisuunnittelun sovellusalueilla

En animation tridimensionnelle, on distingue principalement deux grandes approches basées soit sur des modèles descriptifs, soit sur des modèles générateurs. Dans la première, les possibilités de contrôle sur les trajectoires d'interpolation sont essentielles. Je décris une méthode offrant à l'utilisateur les mêmes possibilités d'interaction sur les trajectoires d'orientations que sur les positions. Le contrôle en temps réel des orientations est rendu possible par une paramétrisation à base de logarithme et d'exponentielle de quaternions et par la notion de tangente sphérique. Une interface 3d de haut niveau gère les interactions avec l'utilisateur. La vitesse de déplacement le long des trajectoires est déterminée par une paramétrisation automatique reprenant des principes de mécanique élémentaire. Enfin, ces techniques sont appliquées a la modélisation interactive de cylindres généralises. Les systèmes par paramètres-clés montrent vite leurs limitations dès qu'il s'agit d'animer de façon réaliste des objets complexes. Pour cela, les systèmes d'animation dits générateurs intégrant les lois de la mécanique ont été introduits. Dans la deuxième partie, je propose une approche alliant les possibilités de contrôle des systèmes descriptifs au réalisme des modèles générateurs. Le principe est de minimiser l'énergie au cours du mouvement. Les équations du mouvement pour des solides rigides articules sont construites sous forme symbolique. L'étape de minimisation est réalisée par un algorithme de contrôle optimal, traitant les équations du mouvement sous leur forme continue.

De nos jours, l’industrie aérospatiale est composée d’acteurs internationaux.Due à la complexité de leur production (taille, nombre de composants,variété des systèmes, ...), la conception d’un avion ou d’un lanceurnécessite un grand nombre d’ingénieurs avec des domaines d’expertises variés.De plus, les industries aérospatiales sont distribuées de par le monde. Dans cecontexte complexe, il est nécessaire d’utiliser la réalité virtuelle afin de proposerdes environnements virtuels accessibles depuis des sites distants afin departager une expérience collaborative. Des problèmes particuliers, alors, surviennent,particulièrement par rapport à la perception des autres utilisateursimmergés.Cette thèse, en partenariat avec Airbus Group, se concentre sur la conceptionde techniques d’interaction collaboratives efficaces. Ces sessions collaborativespermettent de connecter plusieurs sites distants via le même environnementvirtuel. Ainsi, des experts de domaines variés peuvent travailler ensembleen étant immergés simultanément. Par exemple, si un problème survient durantles dernières étapes d’assemblage d’un lanceur, il peut être nécessairede rassembler des experts qui étaient impliqués en amont du projet (bureaud’étude pour la conception, usine pour la fabrication des systèmes), afin deconcevoir des solutions pour résoudre le problème.Dans cette thèse, nous proposons différentes techniques d’interactions afinde faciliter la collaboration à différents niveaux d’un projet industriel. Nousnous sommes intéressés à la communication et la perception entre collaborateursimmergés, à la prise d’annotation et la circulation de ces annotations età la collaboration asymétrique entre une salle de réunion et un environnementvirtuel à l’aide d’outil de réalité mixte
The aerospace industry is no longer composed of local and individualbusinesses. Due to the complexity of the products (their size, the numberof components, the variety of systems, etc.), the design of an aircraft or alauncher involves a considerable number of engineers with various fields of expertise.Furthermore, aerospace companies often have industrial facilities allover the world. In such a complex setting, it is necessary to build virtual experimentsthat can be shared between different remote sites. Specific problemsthen arise, particularly in terms of the perception of other immersed users.We are working with Airbus Group in order to design efficient collaborativeinteraction methods. These collaborative sessions allow multiple sites to beconnected within the same virtual experiment and enable experts from differentfields to be immersed simultaneously. For instance, if a problem occurs duringthe final stages of a launcher assembly, it may be necessary to bring togetherexperts on different sites who were involved in previous steps ( initial design,manufacturing processes). In this thesis, we propose various interaction technique in order to ease thecollaboration at different moments of an industrial process. We contributedin the context of communication between immersed users, taking notes inthe virtual environment and sharing it outside virtual reality and asymmetriccollaboration between a physical meeting room and a virtual environment

This thesis deals with representation of large and complex 3D scenes which are usually used by modern computer games. Main aim is design and implementation of data driven rendering system. Proper rendering is directed (driven) by scene description. This description is also designed with respect to scene creators whose typically do not have deep knowledge of programming languages in contrast to game programming developers. First part is focused on design of efficient scene description and its possible applications at scene rendering. Second part is focused on proper system implementation. Finally, consequently important system optimizations are mentioned too.

Les récentes avancées technologiques dans les domaines de la Réalité Virtuelle (RV) et de la Réalité Augmentée (RA) ont donné naissance à un ensemble varié de supports permettant, de visualiser des médias 3D, et d'interagir avec eux. Cette grande diversité d'outils, permettant de manipuler le contenu 3D, apporte des perspectives nouvelles pour le support du travail collaboratif par la RA et la RV. En effet, le travail collaboratif implique couramment des acteurs ayant des métiers et des rôles différents. Leurs besoins en termes de visualisation, et surtout d'interaction avec le contenu 3D sont donc différents. S’il existe aujourd'hui des outils permettant la conception d’expériences collaboratives multi-supports, ceux-ci demandent aux concepteurs de médias 3D une expertise importante en développement d’applications de RV/RA. Ils doivent d'une part connaître les bonnes pratiques de conception propres à chaque support. Il faut d'autre part que les concepteurs aient une certaine expérience en développement d'interfaces plastiques. Si ce n'est pas le cas, une application différente est nécessaire pour chaque support.Le point de vue défendu dans cette thèse est qu'une nouvelle architecture des applications collaboratives en RV/RA est nécessaire de sorte à permettre leur conception sans connaissance par les développeurs des spécificités des supports. Après avoir présenté les outils existants pour créer des applications multi-supports en RV/RA, nous proposons une nouvelle architecture des applications 3D. Cette architecture sépare le média 3D du dispositif utilisateur. Pour permettre la communication entre le média et les différents supports, nous introduisons le protocole web UMI3D qui permet au dispositif de se connecter, grâce à un navigateur, à tout média 3D utilisant ce protocole. Le navigateur permet d'une part, le chargement du contenu 3D du média, et génère d'autre part dynamiquement une interface utilisateur ayant une ergonomie adaptée au support. Cette interface utilisateur permet la réalisation par l'utilisateur de contrats d'interaction établis par le média grâce au protocole. Le principal apport de la méthode de conception des médias 3D proposée est de ne pas demander au concepteur du média de compétences techniques spécifiques aux dispositifs de RV/RA. Par ailleurs, dans le cas où les navigateurs UMI3D sont existants pour les dispositifs utilisés, la conception d'un média 3D multi-support demande le même effort de développement que la conception d'une application 3D mono-support
Recent technological advances in the fields of Virtual Reality (VR) and Augmented Reality (RA) provide a new infrastructure for the visualization of 3D media, and to interact with them. This diversity of tools, allowing to manipulate 3D content, brings new perspectives for the support of cooperative work by AR and VR. Indeed, collaborative work involves actors with different roles and trades. Their needs in terms of visualization, and especially of interaction with 3D content are therefore different. If there are nowadays tools allowing the design of collaborative experiences for multiple devices, these tools require designers to have significant expertise in developing AR/VR applications. First, they must be aware of the design best-practices for each device. Second, the designers must have some experience in developing plastic interfaces. If it is not the case, designing a different application is needed for each device. The point of view we defend in this thesis is that a new architecture is needed for collaborative AR/VR applications to allow their conception without to enquire the developers to have a knowledge of each device specificities. After presenting the existing tools for creating AR/VR applications for multiple devices, we propose a new architecture for 3D applications. This architecture separates the 3D media from the user device. To allow communication between the media and the different devices, we introduce the UMI3D web protocol which allows the device to connect to any 3D media using a UMI3D browser. The browser allows the loading of 3D content from the media, and dynamically generates an adapted user interface for a given device. This user interface allows the user to draw up interaction contracts established by the media using the protocol. The main contribution of the proposed 3D media architecture is to avoid asking the media designer for technical skills about specific AR/VR devices. Furthermore, in the case where UMI3D browsers are existing for the devices used, the design of a cross-devices 3D media requires the same development effort as the design of a single-device 3D application

Lo scopo principale di questa tesi è la realizzazione di una faccia parlante facilmente integrabile in una pagina html e che fornisca il servizio di sintesi audio-visiva di un qualsiasi testo in lingua italiana con una produzio- ne del parlato visuale deve essere di elevata qualità. ` Il framework realizzato si chiama LUCIA-WebGL. E una versione completamente reingegnerizzata dell’applicazione LUCIA Talking Head (sviluppata presso lo ISTC-CNR di Padova negli anni 2003-2006) utilizzando la nuova tecnologia WebGL. LUCIA e LUCIA-WebGL sono sviluppate con tecnologie differenti, ma condividono gran parte delle logiche di progettazione e animazione. Sono entrambe totalmente basate sullo standard MPEG-4 SNHC (Synthetic/Natural Hybrid Coding) che fornisce le funzionalità per creare una animazione facciale in tempo reale guidata dai Parametri di Animazione Facciale (FAPs) con l’emulazione delle funzionalità dei muscoli mimici e la loro influenza sulla pelle della faccia. L’azione muscolare è resa esplicita per mezzo della deformazione del reticolo poligonale costruito attorno ai punti chiave FPs (feature points) che corrispondono alla giunzione dei muscoli mimici con la pelle. Tale deformazione dipende dal valore dei Parametri di Animazione Facciale FAPs (Facial Animation Parameters) che afferiscono ai movimenti dei singoli punti chiave. LUCIA-WebGL soddisfa le specifiche del Predictable Facial Animation Object Profile di MPEG-4 ovvero è in grado di importare modelli esterni per mezzo della ricezione dei Parametri di Definizione FDP (Facial Definition Parameters). Il sistema è stato integrato nel prototipo realizzato per il progetto ’Wikimemo.it: Il portale della lingua e della cultura italiana’ finanziato dal M.I.U.R. per promuovere la cultura italiana attraverso la lettura e l’ascolto. Il sistema permette all’utente di navigare nei contenuti, effettuare ricerche di frasi e parole e sentire come sono pronunciate in un contesto specifico. I risultati di una query di ricerca possono essere ascoltati con la voce di un italiano nativo o con la voce sintetica di LUCIA-WebGL che ne mostra il movimento labiale.

La plupart des applications qui intègrent le son 3D en environnement virtuel sont limitées à la simulation acoustique. L'objectif de cette thèse est d'étudier l'apport du son 3D dans l'interaction en environnement virtuel. Nous avons développé une technique de stimulation sonore appelée: Sensation Artificielle Spatialisée Auditive (SASA), basée sur la création d'effets sonores. Cette technique restitue des sensations de spatialisation de son permettant la localisation précise (en azimut et en élévation) de sources sonores. Afin d'améliorer le temps de localisation de sources sonores, nous avons développé un modèle qui intègre la fonction HRTF (Head-Related Transfer Function) avec la technique SASA. Pour simuler la profondeur de sources sonores, nous avons développé une technique basée sur la combinaison entre l'indice d'intensité et les différences interaurales de temps. Les techniques développées dans le cadre de cette thèse ont été utilisées pour le guidage auditif afin de fournir des informations spatiales en azimut, en élévation et en profondeur. Nous avons appliqué cette technique dans la simulation chirurgicale pour assister l'opérateur dans la phase de triangulation. Nous avons validé cette approche en termes de performance de manipulation et du retour d'informations de l'utilisateur. Les résultats obtenus des travaux de cette thèse sont prometteurs pour l'utilisation de son 3D dans l'interaction en environnement virtuel.

Thesis (M.Sc.)--York University, 2008. Graduate Programme in Computer Science.
Typescript. Includes bibliographical references (leaves 95-101). Also available on the Internet. MODE OF ACCESS via web browser by entering the following URL: http://gateway.proquest.com/openurl?url_ver=Z39.88-2004&res_dat=xri:pqdiss&rft_val_fmt=info:ofi/fmt:kev:mtx:dissertation&rft_dat=xri:pqdiss:MR38834

碩士
國立臺北教育大學
特殊教育學系碩士班
94
The purpose of this study was to investigate the effectiveness of the “3D Graphic-based Virtual Reality” on the improvement of walking skill to school for elementary school students with moderate and severe mental retardation. Single-subject experimental design with changing criterion design was used to evaluate the effectiveness.”3D Graphic-based Virtual Reality” is an independent variable and the “walking skills to school” are dependent variables. Two students with moderate and severe mental retardation were monitored, which their grade were ranged from four to six. During this experiment, each student received individual teaching. Every teaching class consisted of an indoor desktop learning and an outdoor practice at their commute routes. The teaching continued until students became skilled, which took about four to five weeks. The students received additional three other learning reservation tests after the experiment. These were to assure their learning reservation outcome. All data were videotaped, and were analyzed by using visual inspection techniques. The effects of other observation data as well as the social validity of the interview data were also analyzed. The results of this study included: 1. After teaching, two students did improve their walking skills to school. This implied that virtual experience could be successfully transferred, and applied to real environment, and the effectiveness of “3D Graphic-based Virtual Reality〞learning was positve. 2. The commute time to school of monitored students was gradually reduced even after this experiment. This confirms the walking skill has been improved. 3. The feedback from teachers and parents of monitored students on the “3D Graphic-based Virtual Reality〞learning were positive. Instead of outdoor learning, “3D Graphic-based Virtual Reality〞learning brought students more safety, and indoor learning made them more focused. As a result, monitored students improved their walking skills. In addition to the discussions of above-mentioned results, suggestions on the teaching contents and directions of future studies of students with mental retardation were proposed in this study.

碩士
國立屏東師範學院
教育科技研究所
93
This research was to look into that if teachers utilize different teaching methods on students could lead to different learning attitudes and effects. Besides, based on different gender and learning type, with different teaching methods, students would show variation on their leaning attitudes and mathematics performances. The samples of this study were fifty-nine sixth-grade students from two classes at one school, Kaohsiung. Each group will go through and seven hours, total two weeks experimental course. Before the course, both the 3D Virtual Reality Instruction and Traditional Instruction groups took a pre-test, called mathematics attitude scale. In one week after the course finished, the two groups will took a attitude post-test and mathematics achievement test, and the experimental group filled in a 3D Virtual Reality Learning Questionnaire. According to the analysis from the experiment , the study reached the following conclusions: 1.There was no significant interaction on students’ learning attitudes between the gender and teaching methods. 2.There was a significant interaction on students’ mathematics performance betw- een the gender and teaching methods. For the girls ,with “3D Virtual Reality Instruction”, students improves a lot compared to the Traditional Learning. For the boys, however, the difference was insignificant. 3.There was no significant interaction on students’ mathematics learning attitudes between the learning type and teaching methods. 4.There was no significant interaction on students’ mathematics performance between the learning type and teaching methods. 5.Most students in the experimental group are in favor of 3D Virtual Reality Instruction. This method can help them easily catch on mathematics courses and express their view of points. Moreover, it improves concentration . In the end, according to the develop CAI and research process and results, the study provided suggestions for 3D Virtual Reality Instruction develop and further researches.