Dissertations / Theses on the topic 'Perceptual interactions'

Proefschrift Universiteit van Amsterdam.
Op omslag: IFOTT, en :LOT, Netherlands Graduate School of Linguistics, Landelijke Onderzoekschool Taalwetenschap. Met index, lit. opg. - Met samenvatting in het Nederlands.

Amblyopia, a developmental disorder of the visual system, is widely known to cause a reduction in optotype acuity but it can also be associated with disrupted binocular vision, reduced contrast sensitivity and many other subtle high level visual processing deficits. The initial stages of the work presented in this thesis involved laboratory investigation of the functional visual deficit in global motion processing that has previously been reported abnormal in the presence of amblyopia. The key question is whether higher-levels of visual processing "inherit" abnormalities from lower levels, or whether additional developmental abnormalities arise in direct consequence of impoverished visual input. Overall, the results imply a far more complex perceptual change in amblyopia than would be predicted by the well -established losses in resolution and contrast sensitivity. The motivation behind Chapters 5 and 6 stems from the current observation that the recovery of visual function in amblyopia is contingent on even brief periods of correlated binocular vision, suggesting that amblyopia is intrinsically a binocular problem and that it is suppressive mechanisms that render the cortex, which is a structurally binocular system, functionally monocular. Research is now casting doubts on the idea that amblyopes do not possess cortical binocular connections, suggesting an active suppression rather than a deficit of cellular function. Interestingly, this is echoed in the clinical domain where, in cases of de-correlated visual input, strabismus clinical protocols have now established that the correction of refractive error alone can be sufficient to improve acuity, again implying incomplete inhibition mechanisms. The clinical investigations in this thesis have involved the validation of a series of psychophysical paradigms in cohorts of juvenile and adult amblyopes (as well as age-matched controls) to establish the degree of binocular interaction present and to explore the potential for treating amblyopia with prolonged viewing of a binocular stimulus adapted to correlate the visual imput from both eyes.

Thesis (Ph. D.)--Harvard-MIT Division of Health Sciences and Technology, February 2010.
"September 2009." Cataloged from PDF version of thesis.
Includes bibliographical references (p. 160-175).
This project is an experimental study of perceptual interactions between auditory and tactile stimuli. These experiments present vibrotactile stimuli to the fingertip and auditory tones diotically in broadband noise. Our hypothesis states that if the auditory and tactile systems integrate, the performance of the two sensory stimuli presented simultaneously will be different from the performance of the individual sensory stimuli. The research consists of work in two major areas: (1) Studies of the detection of auditory and tactile sinusoidal stimuli at levels near the threshold of perception (masked thresholds for auditory stimuli and absolute thresholds for tactile stimuli); and (2) Studies of loudness matching employing various combinations of auditory and tactile stimuli presented at supra-threshold levels. Results were compared to three models of auditory-tactile integration. The objective detection studies explore the effects of three major variables on perceptual integration: (a) the starting phase of the auditory relative to the tactile stimulus; (b) the temporal synchrony of stimulation within each of the two modalities; and (c) the frequency of stimulation within each modality. Detection performance for combined auditory-tactile (A+T) presentations was measured using stimulus levels that yielded 63%-77%-correct unimodal performance in a 2-Interval, 2-Alternative Forced- Choice procedure. Results for combined vibrotactile and auditory detection indicated: (1) For synchronous presentation of 500-msec, 250 Hz sinusoidal stimuli, percent-correct scores in the combined A+T conditions were significantly higher than scores within each single modality;
(cont.) (2) Scores in the A+T conditions were not affected by the relative phase of the 250 Hz auditory and tactile stimuli; (3) For asynchronous presentation of auditory and tactile 250 Hz stimuli, scores on the A+T conditions improved only when the tactile stimulus preceded the auditory stimulus (and not vice versa); and (4) The highest rates of detection in the combined-modality stimulus were obtained when stimulating frequencies in the two modalities were equal or closely spaced (and within the Pacinian range). The lack of phase effect suggests that integration operates on the envelopes rather than on temporal fine structure. The effects of asynchronous presentation imply a shorter time constant in the auditory compared to the tactile modality and are consistent with time constants deduced from single-modality masking experiments. The effects of frequency depend both on absolute frequency and on relative frequency of stimulation within each modality. In general, we found that an additive sensitivity model best explained detection performance when tones were presented synchronously and of the same frequency. In the second area of research, loudness matching was employed in a subjective study of the effects of frequency on auditory-tactile integration for stimuli presented at supra-threshold levels. These experiments, which were derived from previous auditory studies demonstrating the dependence of loudness on critical-band spacing of tonal signals, employed various combinations of auditory and tactile stimuli that were presented at equally loud levels in isolation.
(cont.) Loudness matches were obtained for auditory-only (A+A) and auditory-tactile (A+T) stimuli that were both close as well as farther apart in frequency. The results show that the matched loudness of an auditory pure tone is greater when the frequencies of combined stimuli (both A+A and A+T) are farther apart in frequency than when they are close in frequency. These results are consistent with the results found in the previous experiment exploring the frequency relationships at near-threshold levels, as well as with results in the psychoacoustic literature, and suggest that the auditory and tactile systems are interacting in a frequency-specific manner similar to the interactions of purely auditory stimuli. The research conducted here demonstrates objective and subjective perceptual effects that support the mounting anatomical and physiological evidence for interactions between the auditory and tactual sensory systems.
by E. Courtenay Wilson.
Ph.D.

Notre recherche se concentre sur la conception d’interfaces pour la communication émotionnelle à distance. Notre travail s’inscrit notamment dans le cadre de l’élaboration d’espaces numériques d’interactions tactiles, rendus possibles par la connexion en réseau de dispositifs de suppléance perceptive. Nous adoptons une approche énactive et interactionniste de la cognition sociale. Nous utilisons de plus la méthodologie minimaliste, qui offre une opportunité originale d’étudier la constitution des rencontres interpersonnelles au sein d’espaces mettant en place des couplages sensori-Moteurs inédits. Nous déroulons deux méthodologies, en articulant les domaines de la recherche fondamentale, de l’étude expérimentale et du design.Dans un premier temps, nous partons de la théorie autour du thème du toucher, avant de proposer une solution d’interface : l’application smartphone Touch Through. Elle permet une suppléance perceptive du toucher en se basant sur des propriétés fonctionnelles de ce sens. Touch Through sert ensuite à une expérimentation basée autour d’un test de Turing minimaliste, afin de tester la possibilité d’un sentiment de présence à distance avec notre application et de la mettre entre les mains d’utilisateurs.Dans un second temps, nous adressons la question de l’attention conjointe avant de mettre en place une étude expérimentale autour de ce thème (tâche de distinction entre des objets privés et des objets communs). Forts des résultats, nous proposons alors un prototype d’interface pour le réseau social facebook, en mettant en avant des possibilités d’attention conjointe au sein du site et en offrant une interaction inédite autour des posts
Our research is aimed at designing interfaces for the remote communication of emotions. In particular our work is part of the development of tactile interaction digital spaces. These are made possible by a network connection between perceptual supplementation devices. We consider an enactive and interactionist approach of social cognition. In addition, we use a minimalist methodology. This offers an original opportunity to study the very constitution of interpersonal encounters within spaces where unprecedented sensorimotor couplings happen. We work through two methodologies, each articulating the fields of fundamental research, experimental study and design.In a first phase, we start from theory around the theme of touch, before proposing an interface solution – the smartphone application ‘Touch Through’. This application allows a perceptual supplementation of touch using its functional properties. ‘Touch Through’ is then used in a experimental study based on a minimalist Turing test. This experiment is about testing the possibility of a presence in absence feeling through our application. It was also an opportunity to have users handling it.In the second phase we address the question of joint attention. After a study of this question, we design an experiment where we propose a task of discrimination between private and common objects. Based on the outcome of this experiment, we then propose an interface prototype for the social network ‘facebook’. With this prototype interface, the facebook website is enhanced with possibilities of joint attention. In addition, facebook users can take part in an original interaction around posts

The primary visual cortex (V1) has long been considered the main low level visual analysis area of the brain. The classical view is of a feedfoward system functioning as an edge detector, in which each cell has a receptive field (RF) and a preferred orientation. Whilst intuitive, this view is not the whole story. Although stimuli outside a neuron’s RF do not result in an increased response by themselves, they do modulate a neuron’s response to what’s inside its RF. We will refer to such extra-RF effects as contextual modulation. Contextual modulation is thought to underlie several perceptual phenomena, such as various orientation illusions and saliency of specific features (such as a contour or differing element). This gives a view of V1 as more than a collection of edge detectors, with neurons collectively extracting information beyond their RFs. However, many of the accounts linking psychophysics and physiology explain only a small subset of the illusions and saliency effects: we would like to find a common principle. So first, we assume the contextual modulations experienced by V1 neurons is determined by the elastica model, which describes the shape of the smoothest curve between two points. This single assumption gives rise to a wide range of known contextual modulation and psychophysical effects. Next, we consider the more general problem of encoding and decoding multi-variate stimuli (such as center surround gratings) in neurons, and how well the stimuli can be decoded under substantial noise levels with a maximum likelihood decoder. Although the maximum likelihood decoder is widely considered optimal and unbiased in the limit of no noise, under higher noise levels it is poorly understood. We show how higher noise levels lead to highly complex decoding distributions even for simple encoding models, which provides several psychophysical predictions. We next incorporate more updated experimental knowledge of contextual modulations. Perhaps the most common form of contextual modulations is center surround modulation. Here, the response to a center grating in the RF is modulated by the presence of a surrounding grating (the surround). Classically this modulation is considered strongest when the surround is aligned with the preferred orientation, but several studies have shown how many neurons instead experience strongest modulation whenever center and surround are aligned. We show how the latter type of modulation gives rise to stronger saliency effects and unbiased encoding of the center. Finally, we take an experimental perspective. Recently, both the presence and the underlying mechanisms of contextual modulations has been increasingly studied in mice using calcium imaging. However, cell signals extracted with calcium imaging are often highly contaminated by other sources. As contextual effects beyond center surround modulation can be subtle, a method is needed to remove the contamination. We present an analysis toolbox to de-contaminate calcium signals with blind source separation. This thesis thus expands our understanding of contextual modulation, predicts several new experimental results, and presents a toolbox to extract signals from calcium imaging data which should allow for more in depth studies of contextual modulation.

One way in which peripheral vision may acquire the functional role of the fovea, a part of the retina preferentially used for complex visual tasks (such as reading and face recognition) is by modulating the strength of intracortical connections in the humans’ visual areas with perceptual learning. Perceptual learning is a practice-dependent improvement in a visual task performance that can persist for several months, and is specific for stimulus, task, eye presentation and retinal locus of stimulation. These specificity effects have been explained on the basis of neural plasticity, consisting in long-term modifications of a number of mechanisms in the early visual cortices, that are selective for basic stimulus attributes (Karni & Sagi, 1991, 1993; Ahissar & Hochstein, 1993, 1996; Casco & Campana, 2001). Perceptual learning experiments with stimuli involving lateral masking (Polat & Sagi, 1994b, 1995; Polat, Ma-Naim, Belkin & Sagi, 2004) have suggested that practice is able to modulate short- and long-range lateral interactions between neurons responding to collinear elements. These studies showed that contrast thresholds for a target are modulated by the presence collinear flankers, and the type of modulation depends on the distance between the central target and the flankers: inhibitory for short target-to-flankers distance, and facilitatory for longer distances. With the training the suppression from the short target-flanker separation can be reduced and facilitation at relatively long target-flanker separation increases. These studies suggest that practice on lateral interactions increases the efficacy of the collinear interactions between neighbouring neurons, an effect that enhances connectivity with remote neurons via a cascade of local interactions. Most importantly, perceptual learning on lateral interactions has been showed to be useful for improving contrast sensitivity in people with normal low vision (Tan & Fong, 2008; Polat, 2009) or with impaired lateral interactions such as amblyopia (Polat et al., 2004). Notably, these studies showed that, differently from previous perceptual learning experiments where non transfer of learning to different stimulus attributes was observed, the effect of training on lateral interactions transferred to higher level visual tasks, like visual acuity (VA) (Tan & Fong, 2008) producing a long-lasting perceptual benefit in everyday visual tasks. However, in all these studies stimuli were presented in the fovea. This thesis aimed at investigating the possibility that the effect of training also improves lateral interaction in retinal regions eccentric with respect to the fovea. Lateral interactions strongly depend on eccentricity: in the periphery they are mostly inhibitory (Petrov, Carandini & McKee, 2005; Cavanaugh et al., 2002). These evidence leads to the hypothesis that peripheral vision may acquire some of the functional role of the fovea only if inhibition is reduced. Shani & Sagi (2005) showed that collinear facilitation in the near-periphery is weak and that perceptual learning seems not to be effective in modulating lateral interactions. However, in their study the training was very short, and only one target-to-flanker distance was tested. If lateral interactions could be modulated in the near periphery, and transfer to visual tasks such as VA and crowding was possible, this could be extremely important for rehabilitation of individuals with loss of central vision, such as in macular degeneration. These type of patients, after the loss of central vision, are forced to use the periphery of the visual field for the most demanding visual tasks like face recognition and reading. In Experiment 1, we were interested in verifying whether inhibitory lateral interactions in the near-periphery (4 degrees of eccentricity) may be reduced by training and if training transfers to other visual functions. Eight subjects were trained with different spatial frequencies (1, 2, 4, and 8 cycles per degree - cpd) and different target-to-flankers separations (2, 3, 4, and 8). Before the practice sessions subjects performed a series of pre-tests aimed at measuring their peripheral contrast sensitivity function (CSF), peripheral visual acuity (VA) and crowding effect. Consistently with previous studies (Petrov et al., 2005; Cavanagh et al., 2002), results of the Experiment 1 showed that, in the near periphery, lateral interactions are inhibitory even at target-to-flanker distances (4) where facilitatory interactions are found in central vision. Facilitation was reported for a target-to-flankers distance of 8, consistently with the most recent investigation on peripheral lateral interactions (Lev & Polat, 2011). Most importantly, Experiment 1 showed that lateral interactions in parafoveal vision can be modulated by training, reducing the inhibition, and that perceptual learning transfers to other visual abilities, leading to a reduction of crowding. Since learning specificity is viewed as the main indicator of the level of processing at which learning takes place, and since learning of lateral interaction has been shown to transfer to different visual functions, in Experiment 2 we tested the specificity of learning to basic stimulus features such as target-flankers local and global orientation and retinal position. We trained 4 new subjects in contrast detection of a collinearly flanked vertical target, and found a significant learning effect for the trained configuration but no transfer of learning to either the same stimulus presented in a symmetrical retinal location, nor to a 45 deg oriented collinear target-flankers configuration, presented in the same retinal position as the learning stimulus. The finding that these transfer stimuli are immune to perceptual learning of vertical orientations strongly suggests that the modulation of lateral interactions through perceptual learning is functionally specific, and that transfer to different visual functions can only occur when these are based on the specific early mechanism that is learned. In Experiment 3, we aimed at exploiting the effects of the perceptual learning of lateral interactions for improving peripheral vision in patients affected by macular degeneration. Training consisted in a contrast detection of a Gabor target with collinear high contrast Gabor flankers, at different target-to-flankers separations, located in their preferential retinal locus (PRL, the new fixation point that spontaneously emeges in this type of patients) and in a symmetrical location. The rationale behind the measuring or lateral interactions and the training in the PRL and in another retinal location was to point out possible differences in intracortical connectivity for this new fixation point respect to other retinal spots. Consistently with other studies (Dilks, Baker, Peli and Kanwisher, 2009), we did not find major differences in terms of lateral interactions and perceptual learning effects between PRL and the symmetrical locus. Training increased contrast sensitivity, and, despite not having any effect on crowding, improved visual acuity in the maculopathy subjects. The absence of crowding reduction could be due to a “roof effect”, since this type of patients naturally train their peripheral view, probably reaching their maximal performance even before the training.. Nevertheless, the improvement of visual acuity opens new perspectives for the rehabilitation of patients with macular degeneration, but also for improving peripheral vision in normal-sighted subjects, since recent studies showed the important role of the periphery of the visual field in tasks such as postural stability, locomotion and driving. In Experiment 4, we investigated the architecture of peripheral lateral interactions in a maculopathy patients, finding collinear facilitation at shorter target-to-flankers separation respect to normal-sighted subjects. Interestingly, collinear facilitation was reported for target presentation in the PRL but not in the No-PRL, where collinear interactions were only inhibitory. Moreover, perceptual learning training appeared to be effective in modulating lateral interactions only in the PRL, questioning the hypothesis of a “use-dependent” cortical reorganization, supported, among the others, by Dilks et al. (2009).
Un modo in cui la visione periferica può acquisire il ruolo funzionale della fovea, la parte della retina preferenzialmente utilizzata per compiti visivi complessi (come la lettura e il riconoscimento dei volti) è tramite la modulazione della forza delle connessioni intracorticali presenti nelle aree visive umane tramite apprendimento percettivo. L’apprendimento percettivo è un miglioramento della prestazione n un compito visivo in seguito alla pratica, può mantenersi per diversi mesi ed è specifico per lo stimolo, il computo e l’occhio utilizzato nel training e per il locus retinico in cui è avvenuta la stimolazione. Questi effetti di specificità sono stati spiegati sulla base della plasticità neurale, che consiste in una modifica a lungo termine di alcuni meccanismi presenti nelle prime aree visive corticali, selettive per caratteristiche basilari dello stimolo (Karni & Sagi, 1991, 1993; Ahissar & Hochstein, 1993, 1996; Casco & Campana, 2001). Esperimenti di apprendimento percettivo con stimoli di mascheramento laterale (Polat & Sagi, 1994b, 1995; Polat, Ma-Naim, Belkin & Sagi, 2004) hanno dimostrato che la pratica può modulare interazioni laterali a breve e lungo raggio tra neuroni che rispondono ad elementi collineari. Questi studi mostrano che le soglie di contrasto per un target centrale sono modulate dalla presenza di elementi vicini, orientati col linearmente, e che il tipo di modulazione dipende dalla distanza tra il target centrale e gli elementi vicini (flankers): inibitoria per brevi distanze tra target e flankers, facilitatoria per distanze maggiori. Con la pratica, l’inibizione per le brevi distanze target-flankers può essere ridotta e la facilitazione a maggiori distanze aumenta. Questi studi suggeriscono che la pratica sulle interazioni laterale può aumentare l’efficacia delle interazioni collineari tra neuroni vicini, un effetto che aumenta la connettività con neuroni più lontani in seguito ad una cascata di interazioni locali. Inoltre, è stato dimostrato che l’applicazione dell’apprendimento percettivo sulle interazioni laterali produce risultati effettivi nel miglioramento della sensibilità al contrasto in individui con miopia (Tan & Fong, 2008; Polat, 2009) o con interazioni laterali abnormali, come nel caso dell’ambliopia (Polat et al., 2004). Questi studi dimostrano che, a differenza dai precedenti esperimenti sull’apprendimento percettivo che non riportavano il trasferimento del miglioramento ad altri compiti, l’effetto della pratica sulle interazioni laterali si trasferisce ad abilità visive di più alto livello, come l’acuità visiva (Tan & Fong, 2008), dando luogo a benefici percettivi a lungo termine nella quotidianità di questi soggetti. Tuttavia, negli studi finora condotti, gli stimoli venivano presentati in fovea. In questa tesi, l’obiettivo è stato quello di studiare la possibilità che gli effetti del training possano migliorare le interazioni laterali in regioni retiniche periferiche rispetto alla fovea. Le interazioni laterali dipendono fortemente dall’eccentricità: nella periferia del campo visivo sono principalmente inibitorie (Petrov, Carandini & McKee, 2005; Cavanaugh et al., 2002). Questa evidenza porta ad ipotizzare che la visione periferica possa acquisire il ruolo funzionale della fovea solo se l’inibizione viene ridotta. Shani & Sagi (2005) hanno dimostrato che la facilitazione collineare nella periferia del campo visivo è debole e che l’apprendimento percettivo non sembra efficace nel modulare le interazioni laterali. Tuttavia, nel loro studio il periodo di pratica era molto breve, in più le distanze target-flankers testate erano ridotte. Se le interazioni laterali possono essere modulate nella periferia del campo visivo, e trasferire il miglioramento ad abilità visive come l’acuità visiva o il crowding (affollamento visivo), questo risultato sarebbe estremamente importante per la riabilitazione di individui con perdita della visione centrale, come nel caso della maculopatia. Questo tipo di pazienti, dopo la perdita della visione centrale, sono obbligati ad usare la periferia del campo visivo per i compiti visivi più complessi, come la lettura ed il riconoscimento dei volti. Nell’Esperimento 1, l’obiettivo era verificare se le interazioni laterali inibitorie nella periferia del campo visivo (4 gradi di eccentricità) potevano essere ridotte dall’apprendimento e se l’eventualmente miglioramento si trasferiva ad altre funzioni visive. I soggetti sono stati allenati con diverse frequenze spaziali (1, 2, 4, and 8 cicli per grado) e diverse distanze target-flankers (2, 3, 4 e 8). Prima delle sessioni di pratica, ai soggetti venivano misurate le prestazioni di base in una serie di compiti visivi, come la sensibilità al contrasto periferica (CSF), l’acuità visiva periferica (VA) e l’effetto di crowding (affollamento visivo).Coerentemente con studi precedenti (Petrov et al., 2005; Cavanagh et al., 2002), i risultati dell’Esperimento 1mostrano che, nella periferia del campo visivo, le interazioni laterali sono inibitorie anche a distanze target-flankers che producono facilitazione in visione centrale (4). La distanza alla quale è stata trovata la facilitazione in periferia (8), è coerente coi più recenti studi sulle interazioni laterali nella periferia del campo visivo (Lev & Polat, 2011). Soprattutto, l’Esperimento 1 mostra che le interazioni laterali in parafovea possono essere modulate dalla pratica, riducendo l’inibizione, e che l’apprendimento percettivo trasferisce il miglioramento ad altre abilità visive, portando alla riduzione del fenomeno di affollamento visivo (crowding). Dato che la specificità dell’apprendimento è considerata l’indicatore principale del livello di processamento dell’informazione in entrata a cui l’apprendimento si verifica, nell’Esperimento 2 abbiamo testato la specificità dell’apprendimento per caratteristiche base dello stimolo come l’orientamento locale e globale degli elementi utilizzati durante il training e la loro posizione spaziale. Un nuovo gruppo di soggetti è stato allenato in un compito di detezione del contrasto per uno stimolo centrale affiancato da elementi collineari, mostrano un effetto di apprendimento significativo per la configurazione allenata, ma non riportando alcun trasferimento per lo stesso stimolo presentato in una posizione retinica simmetrica, né per configurazioni con orientamento locale (tra gli elementi) o globale differente. L’evidenza che questi stimoli sono immuni all’apprendimento percettivo suggerisce in maniera decisa che la modulazione delle interazioni laterali tramite apprendimento percettivo sia specifica per la funzione, e che il trasferimento a diverse funzioni visive può avere luogo solo quando queste sono basate su specifici meccanismi precoci. Nell’Esperimento 3, abbiamo testato la possibilità di utilizzare gli effetti dell’apprendimento percettivo delle interazioni laterali per migliorare la vision periferica in pazienti affetti da degenerazione maculare. Il training consisteva in un compito di detezione del contrasto per uno stimolo centrale affiancato da elementi collineari, collocati a diverse distanze target-flankers, presentati nel locus retinico preferenziale (PRL, il nuovo punto di fissazione che spontaneamente questi pazienti sviluppano) ed in una posizione simmetrica. L’idea alla base della misurazione delle interazioni laterali e dell’apprendimento nel PRL ed in un’altra posizione retinica era di verificare se vi siano differenze nella connettività intracorticale tra il nuovo punto di fissazione periferico ed un altro locus retinico. Coerentemente con altri studi ((Dilks, Baker, Peli and Kanwisher, 2009), non abbiamo riscontrato evidenti differenze in termini di interazioni laterali ed effetti di apprendimento percettivo tra PRL e il locus retinico simmetrico.Il training ha migliorato la sensibilità al contrasto e, pur non avendo avuto effetto sul crowding, ha migliorato l’acuità visiva nei soggetti maculopatici. L’assenza di riduzione del crowding può essere dovuta ad un “effetto tetto”, dato che questo tipo di pazienti allenano “naturalmente”, nella quotidianità, la loro visione periferica, e probabilmente hanno raggiunto la massima prestazione possibile nella visione periferica prima di iniziare il training. Nondimeno, il miglioramento nell’acuità visiva apre nuove prospettive per la riabilitazione di pazienti con maculopatia, ma anche per migliorare la visione periferica in soggetti normovedenti, dato che studi recenti hanno mostrato l’importante ruolo della periferia del campo visivo in attività come la stabilità posturale, la locomozione e la guida. Nell’Esperimento 4, abbiamo studiato l’architettura delle interazioni laterali periferiche in un soggetto maculopatico, trovando facilitazione collineare a distanze target-flankers minori rispetto ai soggetti normovedenti. Il dato interessante è che la facilitazione collineare emerge solo per la presentazione del target nel PRL, mentre nel PRL le interazioni collineari sono solo inibitorie. Inoltre, l’apprendimento percettivo sembra efficace nel modulare le interazioni laterali solo nel PRL, mettendo in discussione l’ipotesi “uso-dipendente” per la riorganizzazione corticale, supportata, tra gli altri, da Dilks et al. (2009).

Cette recherche s’inscrit dans le cadre de l’élaboration d’espaces numériques d’interaction tactile, en tant qu’ils sont rendus possibles par la connexion en réseau d’un dispositif de suppléance perceptive. Dans ce contexte, nous avons articulé une recherche appliquée, centrée sur les analyses d’usage du dispositif dans des contextes écologiques, à une recherche fondamentale, dirigée par les questions théoriques soulevées par les usages eux-mêmes.Pour cela, la méthodologie minimaliste nous offre une occasion originale d’étudier la constitution même des rencontres interpersonnelles, à travers un espace de couplage sensorimoteur inédit. Nos résultats généraux suggèrent que le processus de l’interaction est une dynamique relationnelle autonome qui émerge de l’engagement mutuel des participants. Dans une approche énactive et interactionniste de la cognition sociale, il s’agit alors de considérer que le croisement perceptif entre deux sujets, même réduit à son expression la plus simple, consiste en la rencontre de deux activités perceptives construisant du sens. Dans le contexte de rencontres strictement dyadiques, nous tentons de caractériser cette dynamique, de façon à en isoler les éléments constitutifs. Nous montrons alors que la coordination interpersonnelle résulte d’un processus actif de co-ajustements dynamiques qui se joue autant à un niveau microscopique (qualité de l’accroche perceptive) qu’à un niveau macroscopique (organisation de séquences d’interaction). Dans le contexte de l’exploration mutuelle d’un contenu numérique, nous montrons que cette dynamique d’interaction permet aux participants de faire sens de leurs engagements respectifs en fonction des objets présents. La coordination des activités perceptives se présente alors comme le support de la co-constitution d’un monde partagé de significations, à travers une compréhension interpersonnelle ancrée dans un contexte pragmatique. En outre, l’articulation des expérimentations et des analyses d’usage écologiques nous a conduits à proposer des spécifications techniques et fonctionnelles pour les espaces numériques partagés, de façon à proposer un dispositif pertinent pour les utilisateurs déficients visuels
Our research is part of the design of tactile interaction digital spaces, as they are made possible by a network connection between perceptual supplementation devices. Within this framework, we articulated an applied research, focused on the analysis of use of the device in ecological contexts, with a fundamental one, led by theoretical issues raised by the usage itself. In this respect, minimalist methodology gives us an opportunity to study the very constitution of interpersonal encounters, through a new space of sensorimotor coupling for users.Our overall results suggest that interaction process is an autonomous relational dynamics that emerges from the mutual engagement of participants. Within an enactive and interactionist approach of social cognition, we consider that this perceptual crossing, even when reduced to its simplest expression, involves the meeting of two perceptual activities which inherently produce sense. In the context of strictly dyadic encounters, we try to characterize this dynamic, so as to isolate its fundamental features. We show that interpersonal coordination results from an active process of dynamic co-adjustments that unfolds both on a microscopic level (quality of the perceptual coupling) and on a macroscopic level (organization of interaction sequences). In the case of mutual exploration of digital content, we show that this dynamic interaction allows participants to make sense of their respective engagement relatively to the present objects. The coordination of perceptual activities is then presented as a support for the co-constitution of a shared world of meanings, from an interpersonal understanding rooted in a pragmatic context.In addition, these experiments, coupled to ecological usage analysis, have led us to propose technical and functional specifications for shared digital spaces, so as to provide a relevant device for visually impaired users

La directionnalité est une composante perceptivo-motrice et culturellement déterminée qui intervient de façon prééminente dans l’acquisition initiale de l’écriture. Cette thèse visait à apporter des données empiriques et des éclairages théoriques nouveaux sur 1) son évolution au cours du développement et, 2) sa contribution à la production et à la reconnaissance des formes de lettres. Ces deux axes de recherche ont été déclinés en cinq études. Une première étude investiguait le poids de l’influence culturelle sur la directionnalité graphique au long du développement. Elle permettait de comparer les performances de tracé chez des enfants âgés de 5 à 9 ans dans des conditions où des contraintes biomécaniques, syntaxiques et sémantiques variaient. Les résultats faisaient apparaître un renforcement de l’influence culturelle sur la directionnalité graphique à partir de 6 ans. Cependant, les enfants les plus âgés (9 ans) se montraient capables d’adopter des procédures plus flexibles et de se désengager de contraintes culturelles afin de satisfaire des contraintes contextuelles. La deuxième étude examinait l’évolution développementale de la directionnalité de l’écrit chez des enfants âgés de 4 à 11 ans en comparant la production de formes communes à l’écriture et au dessin. Les résultats obtenus mettaient en évidence une acquisition synchrone des invariants universels (i.e., l’écriture est uni-directionnelle) et des spécificités culturelles du système d’écriture (i.e., la langue est transcrite de gauche à droite). Dans le deuxième axe de recherche, deux études avaient pour objectif de mettre à l’épreuve des faits des hypothèses explicatives récentes de l’écriture en miroir dans l’acquisition initiale et typique de l’écrit. La comparaison de productions en temps réel et appariées d’écritures en miroir et d’écritures conventionnelles chez des enfants pré-lettrés révélait une implication majeure de la directionnalité de l’écrit et une invariance cinématique. Enfin, la dernière étude a permis de mettre en évidence, à l’aide d’un paradigme d’entrainement, une contribution de la directionnalité de l’écriture au rappel de l’orientation de lettres a été mise en évidence chez des enfants pré-lettrés. Les résultats obtenus sont discutés dans une approche incarnée de la cognition et des pistes pédagogiques sont amorcées
Directionality is a crucial perceptual-motor and culturally-based component of early writing acquisition. This research was aimed at providing empirical data and new theoretical insights on 1) its developmental evolution and, 2) its contribution to the production and the recognition of the shapes of letters. Five studies have been devised to fulfill the research objectives.The first study was aimed at investigating how print experience, as a cultural factor, influences directional tendencies in children’s drawing in the interplay with biomechanical, syntactic and semantic factors. Results showed a reinforcement of cultural influence on directional tendencies from 6 years onward. Older children were better able to disengage from a prevalent embodied behavior to meet contextual constraints. The second study examined the specific developmental evolution of writing directionality in children aged from 4 to 11 years while producing the same shapes in both writing and drawing tasks. The results suggested that universal and culture-specific features of writing appear concomitantly and early on in children’s productions. Another two studies were aimed to empirically test competing accounts of mirror writing in preliterate children. On-line productions of paired conventional and mirror writings revealed a predominant role of writing directionality and a kinematic invariance. Finally, a training study showed a contribution of writing directionality to the subsequent recall of the shapes of letters. Results are discussed in terms of embodiment and perspectives in educational settings

I investigated the interaction between perceptual grouping and attention, focusing specifically on distracter rejection. The novelty of the thesis lies in the study of different configural types and their effect on search across space and time. Grouping by configuration is likely to facilitate search by making distracter rejection easier. Grouping can be based on the regular locations of elements, the similarity of elements and whether the elements form a closed shape. The effects of grouping occurred under conditions in which the groups never contained the target, although detection was faster if the target fell internal to the group relative to when it fell outside the group. These results, together with those from neuropsychological studies reported here, are consistent with rapid suppression of irrelevant distractor groups. Primitive grouping, apparently based on clusters of similar proximal elements, took place even when attention was reduced in patients with chronic spatial biases in visual selection. However, neurological damage to attention-related brain regions did disrupt grouping effects dependent on element shape. Attention may, therefore, be more critical for some forms of grouping. Grouping interacts with attention to determine perceptual performance. This operates in a graded manner, determined by the type of grouping.

In the next years, robots are expected to massively move out from industrial environments and to slowly enter our everyday life. In the specific, interactive robots are supposed to become reliable assistants for humans, especially in hostile working conditions, e.g. rescue operations in hazardous environment. To this end, providing them with human-like interaction capabilities will become a key requirement. Natural language is probably the most flexible, intuitive and expressive interface used by humans to communicate. If robots could exhibit an analogous interaction capability, they would be easily accessible to every user, making a huge step forward towards their integration in everyday life. In this direction, Spoken Language Understanding is the field that stands at the crossroad of Natural Language Processing and Audio Signal Processing. It studies how to interpret the meaning conveyed by a speech signal. Its aim is to provide machines with the capability of understanding spoken language, and it has been deeply studied during the past years in the context of Spoken Dialogue Systems, and for the realization of Personal Digital Assistants. In this Thesis we want to study the problem of Spoken Language Understanding transposed to the Human-Robot Interaction field, or more in general when it is applied in the Robotics area. To this end, we consider a wide range of Machine Learning techniques that have been found successful in previous research on Natural Language Processing and Spoken Language Understanding, and that have not been applied yet for robotic platforms. Moreover, we foster here the idea of relying on linguistically sound theories for meaning representation inside the SLU computational frameworks, in order to develop approaches that are independent from the robotic system, the application and the domain. Such generalization would lead to devising solutions that are reusable, comparable and easily adaptable. To this regard, in this Thesis we also present HuRIC, a spoken resource that has been gathered to overcome the lack of training and 3 benchmarking data available for our research. Furthermore, the study of Spoken Language Understanding for robotic applications poses additional challenges to the research done so far. Robots, in fact, are physical entities, and they have to deal with a physical world. The language used to interact with them is in fact often grounded, as it refers to the operational environment the robot is acting into. On the other hand, the environment plays a crucial role in the understanding of grounded commands. Following psycholinguistic theories that studied how perception impacts on language understanding in human cognitive process, we explored the grounding problem of natural language and we propose a framework that make use of perceptual knowledge within the interpretation process of user commands. In such setting, grounding provides evidences that are injected inside the learning process together with linguistic aspects, in order to jointly find the interpretation that better grounds onto the current environment. To enable grounding for extracting perceptual knowledge, we exploited knowledge bases mapping features of entities in the environment over geometrical representations of the space, called semantic maps. They are knowledge bases acquired incrementally through the use of different sources of perception. We then devise a mechanism to ground language over semantic maps, that is exploited for both extracting grounded information and grounding the final interpretation in an executable command. The outcome of this study is then synthesized in a perception-enhanced Spoken Language Understanding processing pipeline for robotic applications.

The spatial representation of numbers is intimately related with action. Evidence for the influence of the numerical information on specific motor components has been found by means of various and original tasks. Automatic activation of the numerical magnitude code has been proposed to account for the numerical influence on motor tasks such as i) lateralized movements or unimanual responses, and ii) manual gestures such as grasping and different grip aperture. The present thesis contributes to this line of research exploring the MNL and ‘cognitive illusion’ hypotheses by means of visuo-spatial and motor tasks. The general question we address is to what extent and at which stage the spatial mapping of numbers in the representational space (MNL) and the interaction between numerical and physical magnitude (‘cognitive illusion’) interferes with the motor system.

As realistic virtual environments become more important, it is interesting to examine procedural generation. In this work, plausibility of virtual furniture layouts was evaluated in a perceptual study comparing layouts by a professional human interior designer with layout variants rearranged by a C# algorithm. It attached related furniture sides at specified distances in groups. Rasterized furniture footprints were tested, and layouts without overlaps between furniture or their clearance areas were accepted. The algorithm was inspired by Germer and Schwarz (Computer Graphics Forum, 28(8), pp. 2068–2078. 2009), and the evaluation methodology by Yu et al. (ACM Transactions on Graphics, 30(4), pp. 1–11. 2011). The algorithm was run by a Unity tool which placed furniture models in a virtual 3D room. Human-made layouts were also visualized and all layouts presented for 2 and 10 seconds with participants tasked to assess if they were human-made or not. The algorithm fulfilling side-based spatial relation constraints and clearance area constraints did not prove to produce as plausible virtual layouts as those by a human designer, but these constraints significantly improved the perceptual plausibility versus semi-arbitrary furniture placement with only wall-placement constraints for relevant furniture. Perceived orderliness of both human-made and generated layouts was significantly positively correlated with plausibility. The evaluation also provided knowledge of what characteristics are perceived as tells of procedural generation rather than human interior design.
Då realistiska virtuella miljöer blir viktigare är det intressant att undersöka processuell generering. I detta arbete utvärderades trovärdigheten av virtuella möbellayouter i en perceptuell studie som jämförde layouter gjorda av en inredningsdesigner med layoutvariationer omarrangerade av en C#-algoritm. Den satte ihop relaterade möbelsidor på specificerade avstånd i grupper. Rasteriserade möbelavtryck testades och layouter utan överlapp mellan möbler eller deras friytor accepterades. Algoritmen var inspirerad av Germer och Schwarz (Computer Graphics Forum, 28(8), s. 2068–2078. 2009) och utvärderingen av Yu et al. (ACM Transactions on Graphics, 30(4), s. 1–11. 2011). Ett Unity-verktyg placerade möbler i ett virtuellt 3D-rum. Designer-layouter visualiserades också och alla layouter visades i 2 & 10 sekunder för deltagare som bedömde om de var arrangerade av människa eller ej. Algoritmen som uppfyllde spatiala sidorelationsvillkor och villkor för fria ytor visade sig inte producera lika trovärdiga virtuella layouter som de gjorda av en människa, men dessa villkor förbättrade trovärdigheten signifikant jämfört med semi-godtycklig möbelplacering med enbart väggplaceringsvillkor för relevanta möbler. Uppfattad nivå av ordning var för både de mänskliga och genererade layouterna signifikant positivt korrelerad med deras trovärdighet. Utvärderingen gav också kunskap om vad som uppfattas som tecken för datorgenerering snarare än mänsklig inredning.

L’arôme fruité des vins rouges est décrit par une large gamme de descripteurs, allant des fruits frais aux fruits mûrs et confiturés, en passant par les fruits confits jusqu'aux notes de pruneaux. La qualité fruitée d’un vin rouge est décrite par des notes de fruits rouges et noirs frais et confiturés. Bien que de nombreux attributs sensoriels soient mentionnés dans la littérature, les connaissances concernant l'arôme fruité qualitatif des vins rouges par rapport à leur composition chimique nécessitent encore d'être approfondies. Cependant, au moins une composante de cette expression fruitée est le reflet d’interactions perceptives impliquant en majorité des esters, mais également d’autres familles de molécules aromatiques. Au cours de ces dernières années, des changements dans l’expression fruitée des vins rouges de Bordeaux, caractérisés par l’apparition des notes de fruits cuits et secs, ont été rapportés et corrélés au changement climatique. Ainsi, les techniciens du secteur vitivinicole et les scientifiques, soulignent la nécessité de prendre des précautions pour préserver l’arôme fruité frais des vins rouges de Bordeaux. Pour cela, une des stratégies d’adaptation consiste à introduire dans le vignoble bordelais des cépages tardifs sélectionnés pour leur cycle végétatif potentiellement adapté aux futures conditions climatiques. Cette thèse propose une meilleure compréhension de l’arôme fruité des vins rouges issus de cépages de Bordeaux, mais également de cépages cultivés autour du bassin méditerranéen, zone simulant les futures conditions climatiques de Bordeaux. Grâce à des approches sensorielles, il a été montré que des vins rouges de Bordeaux présentaient des similitudes avec des vins rouges issus de cépages implantés dans le pourtour méditerranéen au niveau de leur caractère fruité. Une caractérisation approfondie de la contribution des composés volatils d’intérêt dans la perception des notes fruitées a été réalisée. Des interactions perceptives entre des composés de la famille des monoterpènes et des C13-norisoprénoïdes en mélange avec des esters, ont révélé l’importance de ces composés dans la perception des notes fruitées. Enfin, l’addition de ces composés volatils dans un vin rouge montre une diminution des notes de fruits cuits et une augmentation des notes de fruits noirs frais. Ces travaux de recherche ouvrent de nouvelles perspectives en œnologie telles que la maîtrise des teneurs de ces composés d’intérêt par diverses pratiques, mais également la potentielle introduction des « nouveaux » cépages dans l’assemblage des vins de Bordeaux
The fruity aroma of red wines is described by a wide range of descriptors, ranging from fresh fruits to ripe and jammy fruits, to candied fruits and prunes notes. The fruity quality of a red wine is characterized by notes of fresh and jammy red- and black-berry fruits. Although many sensory attributes are mentioned in the literature, knowledge regarding the qualitative fruity aroma of red wines in relation to their chemical composition still requires further investigation. However, at least one component of this fruity expression reflects perceptual interactions primarily involving esters, as well as other families of aromatic molecules. In recent years, changes in the fruity expression of Bordeaux red wines, characterized by the emergence of notes of cooked and dried fruits, have been reported and correlated with climate change. Thus, technicians in the wine industry and scientists emphasize the need to take precautions to preserve the fresh fruity aroma of Bordeaux red wines. One adaptation strategy involves introducing late-ripening grape varieties selected for their potentially suitable vegetative cycle under future climatic conditions into Bordeaux vineyard. This thesis proposes a better understanding of the fruity aroma of red wines derived from Bordeaux grape varieties, as well as those cultivated around the Mediterranean basin, simulating the future climatic conditions in Bordeaux. Through sensory approaches, it has been demonstrated that Bordeaux red wines exhibit similarities with red wines derived from grape varieties grown around the Mediterranean in terms of their fruity character. A detailed characterization of the contribution of volatile compounds of interest in the perception of fruity notes has been conducted. Perceptual interactions between compounds from the monoterpene and C13 norisoprenoid families, mixed with esters, have revealed the importance of these compounds in the perception of fruity notes. Finally, the addition of these volatile compounds to a red wine shows a decrease in cooked fruit notes and an increase in fresh black-berry fruits notes. This research opens up new perspectives in oenology, such as controlling the levels of these compounds of interest through various practices, as well as the potential introduction of "new" grape varieties into Bordeaux wine blends

There is a large range of models of working memory, each with different scopes and emphases. Current interest focuses strongly on the interaction of working memory with long-term memory, as it has become clear that models of working memory alone are incapable of capturing some of our complex cognitive abilities. Most models have contrasting views on how this interaction is implemented. In this thesis, three classes of models are defined, each proposing a different type of interaction. The first model proposes that working memory acts as a gateway for perceptual input on its way to long-term memory. In the unitary model, working memory is seen as comprising the activated portion of long-term memory. The workspace model views working memory as a workspace that is separate from, and deals with the activated contents of long-term memory. The main aim of this thesis was to address the differences between these three models experimentally. Experiments 1 – 7 employed a dual-task paradigm to investigate the effects of irrelevant visual input on visuo-spatial working memory tasks. Two main findings emerged: (1) maintenance of images in working memory was largely insensitive to the effects of concurrent perceptual input; (2) mental imagery was susceptible to interference from irrelevant visual input. This interference effect was selective, as demonstrated by a lack of disruption of imagery by other secondary tasks. Experiment 8 further tested the three models by investigating implicit processing of visual information by neglect patients. It was found that implicit processing is mediated by the activation of long-term memory, in the absence of a conscious representation in working memory. These results together converge to support the workspace model, and suggest a view in which perceptual input activates the contents of long-term memory, prior to these activated representations being made available in a functionally separate working memory system for further processing. The gateway model and unitary model are unable to accommodate all findings. The implications of these results for existing theories about working memory are discussed.

Realistic visual and audio rendering still remains a technical challenge. Indeed, typical computers do not cope with the increasing complexity of today's virtual environments, both for audio and visuals, and the graphic design of such scenes require talented artists. In the first part of this thesis, we focus on audiovisual rendering algorithms for complex virtual environments which we improve using human perception of combined audio and visual cues. In particular, we developed a full perceptual audiovisual rendering engine integrating an efficient impact sounds rendering improved by using our perception of audiovisual simultaneity, a way to cluster sound sources using human's spatial tolerance between a sound and its visual representation, and a combined level of detail mechanism for both audio and visuals varying the impact sounds quality and the visually rendered material quality of the objects. All our crossmodal effects were supported by the prior work in neuroscience and demonstrated using our own experiments in virtual environments. In a second part, we use information present in photographs in order to guide a visual rendering. We thus provide two different tools to assist “casual artists” such as gamers, or engineers. The first extracts the visual hair appearance from a photograph thus allowing the rapid customization of avatars in virtual environments. The second allows for a fast previewing of 3D scenes reproducing the appearance of an input photograph following a user's 3D sketch. We thus propose a first step toward crossmodal audiovisual rendering algorithms and develop practical tools for non expert users to create virtual worlds using photograph's appearance.

Movement and perception interact continuously in daily activities. Motor output changes the outside world and affect perceptual representations. Similarly, perception has consequences on movement. Nevertheless, how movement and perception influence each other and share information is still an open question. Mappings from movement to perceptual outcome and vice versa change continuously throughout life. For example, a cerebrovascular accident (stroke) elicits in the nervous system a complex series of reorganization processes at various levels and with different temporal scales. Functional recovery after a stroke seems to be mediated by use-dependent reorganization of the preserved neural circuitry. The goal of this thesis is to discuss how interaction with the environment can influence the progress of both sensorimotor performance and neuromotor recovery. I investigate how individuals develop an implicit knowledge of the ways motor outputs regularly correlate with changes in sensory inputs, by interacting with the environment and experiencing the perceptual consequences of self-generated movements. Further, I applied this paradigm to model the exercise-based neurorehabilitation in stroke survivors, which aims at gradually improving both perceptual and motor performance through repeated exercise. The scientific findings of this thesis indicate that motor learning resolve visual perceptual uncertainty and contributes to persistent changes in visual and somatosensory perception. Moreover, computational neurorehabilitation may help to identify the underlying mechanisms of both motor and perceptual recovery, and may lead to more personalized therapies.

The perceptual load theory of selective attention proposed by Tsal and Lavie (1994) and Lavie (1995) argues that selective attention is predominantly necessitated by perceptual capacity limitations. In order to account for the experimental evidence where stimuli are attentionally selected either early or late, Lavie (1995) proposed that early selection occurs when perceptual capacity has been reached, while late selection occurs when perceptual capacity has not been reached. This effect has been demonstrated with the use of hybrid visual-search flanker search tasks on numerous occasions (Lavie, 2004). However, some researchers argue that the selection of stimuli is attributable to salience and not to perceptual load. Due to the increased salience of flankers in low perceptual load trials the distractor identity is much more readily processed, thereby leading to the distractor interference. Lavie (1995) attributes the increased distractor interference in low perceptual load trails to the automatic allocation of spare perceptual resources; a process that is mediated by perceptual load levels. This study investigates the potential interaction between perceptual load and distractor salience by presenting 20 participants with a hybrid visual-search flanker task, but placing salient colour singletons distractors in half the trials. The results indicate that the compatibility effect is largely nullified in low perceptual load trials containing salient distractors. The non-salient distractor trials, however, produced a significant compatibility effect as predicted by the perceptual load theory of selective attention. The lack of a significant compatibility effect in salient distractor trials might be an indication that top-down attentional control mechanisms can capitalise on the task-irrelevant colour feature to suppress the processing or perception of the distractor. This finding problematises the hypothesis that the automatic spill-over of perceptual capacity is responsible for the distractor interference in low perceptual load trials as necessitated by perceptual load theory.
Mini Dissertation (MA)--University of Pretoria, 2015.
Psychology
MA
Unrestricted

The research presented in this thesis represents a body of work which addresses issues in medical imaging, primarily as it applies to breast cancer screening and laparoscopic surgery. The concern here is how computer based methods can aid medical practitioners in these tasks. Thus, research is presented which develops both new techniques of analysing radiologists performance data and also new approaches of examining surgeons visual behaviour when they are undertaking laparoscopic training. Initially a new chest X-Ray self-assessment application is described which has been developed to assess and improve radiologists performance in detecting lung cancer. Then, in breast cancer screening, a method of identifying potential poor performance outliers at an early stage in a national self-assessment scheme is demonstrated. Additionally, a method is presented to optimize whether a radiologist, in using this scheme, has correctly localised and identified an abnormality or made an error. One issue in appropriately measuring radiological performance in breast screening is that both the size of clinical monitors used and the difficulty in linking the medical image to the observer s line of sight hinders suitable eye tracking. Consequently, a new method is presented which links these two items. Laparoscopic surgeons have similar issues to radiologists in interpreting a medical display but with the added complications of hand-eye co-ordination. Work is presented which examines whether visual search feedback of surgeons operations can be useful training aids.

The current study implements a speech perception experiment that interrogates local perceptions of Spanish varieties in Miami. Participants (N=292) listened to recordings of three Spanish varieties (Peninsular, Highland Colombian, and Post-Castro Cuban) and were given background information about the speakers, including the parents’ country of origin. In certain cases, the parents’ national-origin label matched the country of origin of the speaker, but otherwise the background information and voices were mismatched. The manipulation distinguishes perceptions determined by bottom-up cues (dialect) from top-down ones (social information). Participants then rated each voice for a range of personal characteristics and answered hypothetical questions about the speakers’ employment, family, and income. Results show clear top-down effects of the social information that often drive perceptions up or down depending on the traits themselves. Additionally, the data suggest differences in perceptions between Hispanic/non-Hispanic and Cuban/non-Cuban participants, although the Cuban participants do not drive the Hispanic participants’ perceptions.

This research details the development of a large-scale, computer vision-based touch screen capable of supporting a large number of simultaneous hand interactions. The system features a novel lightweight multi-point tracking algorithm to improve real-time responsiveness. This system was trialled for six months in an exhibition installation at World Expo 2005 in Aichi, Japan, providing a robust, fault-tolerant interface. A pilot study was then conducted to directly compare the system against other, more established input methods (a single-touch case, a two-mouse case and a physical prototype) to determine the effectiveness and affordances of the multi-touch technology for arranging information on a large-scale wall space in a paired collaborative task. To assist in this study, a separate visualisation and interaction classification tool was developed, allowing the replay of XML log data in real time to assist in the video analysis required for observation and hypothesis testing.

L'informatique ubiquiste se matérialise aujourd'hui par une croissance exponentielle du nombre d'objets communicants (OC) dans notre environnement. Dans cette perspective, un système complet de perception de l'environnement peut être constitué dynamiquement par une fédération d'OC répartis. Nous proposons un cadre conceptuel comme support d'un tel système perceptuel dans un environnement distribué. Ce cadre est utilisé pour la modélisation et l'intégration numérique des informations perçues, l'expression de la complémentarité des dispositifs de perception et leurs descriptions en vue d'une intégration dans une infrastructure de services. Nous présentons une méthode permettant au système perceptuel de se composer dynamiquement afin d'exploiter les capacités individuelles des OC et de répondre au mieux aux besoins des applications clientes. Cette approche a été validée par la réalisation d'applications adaptatives au contexte pour la communication ambiante
Entry into the era of Ubiquitous Computing is leading to an exponential growth in the number of communicating device (CD) that can be found in the human environment. A consequence is that it has become possible to dynamically compose systems for perception ofhuman activities from ad hoc assemblies ofavailable CD's. Ln this thesis we present a conceptual framework for dynamic composition of ad hoc perceptual systems. We examine the problem of modelling and integration for fusing perceptual information, the expression of the complementary nature of perceptual devices, and the description of devices required for dynamic composition. We present a method for the dynamic composition of perceptual services designed to exploit the complementary nature of the perceptual devices. This approach has been validated by realization of adaptive user services that are sensitive to the context of human activity

Many modern day work environments require some degree of dual tasking, particularly the simultaneous performance of cognitive jobs alongside physical activity. The nature of such tasks is often job dependent and may require cognitive functions such as perception, decision making, memory, or response selection/execution in conjunction to task specific physical requirements. Previous research has indicated a possible relationship between concurrent physical and cognitive demands and task performance, safety and efficiency. However, this research is limited and inconsistent. The current study aimed to identify the interaction, if any, between concurrent physical and mental demands, and determine the impact of sensory modality and stage of information processing on this interaction, with specific focus on the performance, perceptual and physical responses during different types of cognitive tasks performed concurrently with a lifting task. 20 (10 male, 10 female) Rhodes University students participated in this study. Each participant performed 9 test conditions - a physical lifting task, a visual and an auditory memory task, and a visual and an auditory decision making task in isolation, as well as the lifting task concurrently with each of these cognitive tasks. Performance was recorded via accuracy and work output of cognitive tasks and the number of lifts for each condition. Perceptual measures were obtained via the Borg RPE and Subjective Workload Assessment Technique. Spinal kinematics were measured using the Lumbar Motion Monitor, while muscle activity of the Erector Spinae, Rectus Abdominis, Rectus Femoris and Biceps Femoris muscles were recorded using the Biometrix Data Logger surface EMG equipment. Mean results were analysed using a dependent T-test to observe any general interaction, and a Two-way ANOVA for the impact of sensory modality and stage of processing. Individual responses were also considered to gain better understanding of both intra and inter-human variability under the various test conditions. Results showed a significant decrease in cognitive performance, increased perception of physical effort, time pressure, mental effort and psychological stress under simultaneous physical and mental demands, while no significant differences in physical responses were observed. Further observations included increased dual-task interference during visual and decision-making tasks when combined with physical demands compared to that of auditory and memory tasks respectively. Individual responses showed large variability between individuals indicating the presence of positive, negative and non-responders to concurrent physical and mental demands. Results therefore imply an individual specific interaction between concurrent physical and mental demands that may or may not be detrimental to worker productivity, job error, injury rates and worker well-being, and that the type of cognitive task performed may impact this interaction.

Istoriškai ir tradiciškai vaiko globa visada buvo jautri ir aktuali visuomenės problema, kuri neatsiejama nuo gailestingumo ir altruistinės pagalbos artimui. Statistika rodo, kad kasmet vis daugiau vaikų iš nedarnių šeimų suteikiama valstybės globa (Pabedinskienė, 2004). Valstybė, ratifikavusi Jungtinių Tautų Vaiko teisių konvenciją, įsipareigoja užtikrinti visapusišką paramą vaikui, sudaryti visas reikiamas sąlygas sėkmingai pastarojo socializacijai. Pastaruoju metu mūsų šalyje dėmesys yra sutelktas į globos namų auklėtinių socialinę integraciją visuomenėje. Tam turi reikšmės priimti Vaiko teisių apsaugos pagrindų (1996), Vaiko globos (1998) įstatymai, Vaiko globos organizavimo nuostatai (2002), akcentuojantys būtinybę suteikti visą reikiamą pagalbą sėkmingai šių vaikų adaptacijai ir integracijai visuomenėje (Raudeliūnaitė, Paigozina, 2009). Šiame kontekste didelis reikšmingumas suteikiamas asmens socialiniams įgūdžiams, nes pastarųjų ugdymas pagerina asmens socialinio funkcionavimo galimybes, sudaro sąlygas jam efektyviai dalyvauti visuomeninėje veikloje, tapti pilnaverčiu visuomenės nariu. Šio darbo tyrimo objektas - globos namų jaunesniųjų paauglių (12-14m.) socialiniai įgūdžiai. Tyrimo tikslas - ištirti vaikų globos namų specialistų nuomonę apie globos namų jaunesniųjų paauglių (12-14m.) socialinių įgūdžių raiškos ypatumus. Tyrime dalyvavo 82 respondentai, tyrimui panaudotas klausimynas, skirtas vaikų globos namuose dirbantiems specialistams. Tyrimas buvo atliktas... [toliau žr. visą tekstą]
Historically and traditionally, a child care has always been a sensitive and urgent public issue, which is inseparable from compassion and altruistic aid to a close person. Statistics show that every year more and more children from dysfunctional families, are given the state custody (Pabedinskienė, 2004). A Member which has ratified the United Nations Convention on the Rights of the Child, is committed to ensuring full support for the child, creating all necessary conditions for successful socialization. Recently, our country is focusing on social integration in society of children, who are living at care homes. The Child Rights Law on the basis (1996), the Child Care Law (1998), the Child care organization policies (2002), has a great significance for this, emphasizing the need to provide all necessary assistance to these children, taking care of successful adaptation and integration in society (Raudeliūnaitė, Paigozina, 2009). In this context, great significance is given to an individuals social skills, because of their education a child improves an individuals social functioning capabilities, he is enable to effectively participate in social activities as well as fully in our society. The object of the study is - young adolescents (12-14 years), who are staying at care home social skills. The aim of the study is - to explore child care home expert opinion about the young adolescents (12-14 years) peculiarities of social skills. For this study a questionnaire was used to... [to full text]

The present work investigates the mechanisms underpinning the integration of local signals (either local orientations, positions or directions) into whole configurations. The investigation is composed of three studies that try to disentangle the issue using a contour integration paradigm. Each of them focuses on a specific aspect of the problem. Study one compares two integration models: the first is the well known “association field model”, based on local lateral interactions between adjacent receptive fields tuned to similar orientation (in primary visual area V1). The second is a second-stage filter that follows rectification of first-order filters. Study two tests, instead, the idea that a local cooperative system is responsible for the integration of directional signals. In addition it investigates whether such a mechanism could explain the “motion facilitation effect” usually found with dynamic (compared to static contours). Finally, study three extends findings from study two recording Visual Evoked Potentials (VEPs) elicited by static and dynamic contours. My findings provide support to the idea that a mechanism based on local lateral interactions in V1 could account for the integration of static contours, whereas a local cooperative mechanism could account for the integration of static contours. Moreover, these two mechanisms interact, in a way that the cooperative motion system facilitates or impairs the input feeding the static association field.
Questa tesi investiga i meccanismi responsabili dell’integrazione di segnali locali (siano essi orientazioni, posizioni o direzioni di elementi locali) in configurazioni globali. Il lavoro si compone di tre studi, che provano a dare una risposta alla domanda attraverso l’utilizzo di un paradigma di integrazione di contorni. Ciascuno studio approfondisce uno specifico aspetto del problema. Il primo studio confronta due modelli di integrazione: il primo è il celebre “campo associativo”, basato su connessioni laterali (presenti nella corteccia visiva primaria) tra campi recettivi adiacenti e sensibili per orientazioni locali simili. Il secondo modello è un filtro di second’ordine che riceve come input il risultato di un processo di rettificazione dell’output filtri di primo ordine. Il secondo studio verifica, invece, se un sistema cooperativo locale spiega in maniera esaustiva l’integrazione di segnali locali di direzione. Inoltre, questo studio investiga anche la possibilità che il suddetto meccanismo cooperativo possa spiegare la “facilitazione data dal movimento” che si trova, di solito, quando si confronta la abilità di rilevare un la presenza di un contorno dinamico rispetto ad uno statico. In ultimo, lo studio tre amplia i risultati del secondo studio, avvalendosi di una tecnica di registrazione di potenziali evocati elicitati da contorni statici e dinamici. Nel complesso, i tre studi supportano l’idea che un sistema basato su connessioni laterali (presenti nella corteccia visiva primaria) possa determinare l’integrazione di contorni statici, mentre un sistema cooperativo spiega l’integrazione di segnali di movimento locali. In aggiunta, questi due sistemi interagiscono continuamente, con il sistema di movimento che determina la qualità dell’input che sarà utilizzato, successivamente, dal sistema associativo statico.

Intelligent devices are quickly becoming necessities to support our activities during both work and play. We are already bound in a symbiotic relationship with these devices. An unfortunate effect of the pervasiveness of intelligent devices is the substantial investment of our time and effort to communicate intent. Even though our increasing reliance on these intelligent devices is inevitable, the limits of conventional methods for devices to perceive human expression hinders communication efficiency. These constraints restrict the usefulness of intelligent devices to support our activities. Our communication time and effort must be minimized to leverage the benefits of intelligent devices and seamlessly integrate them into society. Minimizing the time and effort needed to communicate our intent will allow us to concentrate on tasks in which we excel, including creative thought and problem solving.

An intuitive method to minimize human communication effort with intelligent devices is to take advantage of our existing interpersonal communication experience. Recent advances in speech, hand gesture, and facial expression recognition provide alternate viable modes of communication that are more natural than conventional tactile interfaces. Use of natural human communication eliminates the need to adapt and invest time and effort using less intuitive techniques required for traditional keyboard and mouse based interfaces.

Although the state of the art in natural but isolated modes of communication achieves impressive results, significant hurdles must be conquered before communication with devices in our daily lives will feel natural and effortless. Research has shown that combining information between multiple noise-prone modalities improves accuracy. Leveraging this complementary and redundant content will improve communication robustness and relax current unimodal limitations.

This research presents and evaluates a novel multimodal framework to help reduce the total human effort and time required to communicate with intelligent devices. This reduction is realized by determining human intent using a knowledge-based architecture that combines and leverages conflicting information available across multiple natural communication modes and modalities. The effectiveness of this approach is demonstrated using dynamic hand gestures and simple facial expressions characterizing basic emotions. It is important to note that the framework is not restricted to these two forms of communication. The framework presented in this research provides the flexibility necessary to include additional or alternate modalities and channels of information in future research, including improving the robustness of speech understanding.

The primary contributions of this research include the leveraging of conflicts in a closed-loop multimodal framework, explicit use of uncertainty in knowledge representation and reasoning across multiple modalities, and a flexible approach for leveraging domain specific knowledge to help understand multimodal human expression. Experiments using a manually defined knowledge base demonstrate an improved average accuracy of individual concepts and an improved average accuracy of overall intents when leveraging conflicts as compared to an open-loop approach.

The purposes of this study were to (a) test the validity of a sensory reactivity measure adapted for parents of preschool-age children, (b) examine if different modalities of sensory reactivity (i.e. smell, touch, taste, etc.) emerge together or if differing thresholds of reactivity exist between sensory modalities, (c) see how parental ratings of preschoolers' sensory reactivity are related to children's behaviors in the classroom, and (d) see if sensory reactivity bears different relationships to children's social behaviors than do other aspects of temperament. A total of 260 parents (242 mothers, 18 fathers) and 10 teachers of 260 children (131 male, 129 female; M = 63 months; SD = 8.80; range = 39-81) participated. Parents completed the newly developed Children's Sensory Reactions Questionnaire and the Colorado Child Temperament Inventory. Teachers completed the Social Skills Questionnaire. Exploratory and confirmatory factor analyses extracted two factors from the CSRQ measure: sensory reactivity and perceptual awareness. Examination of the associations of sensory reactivity and perceptual awareness and children's behaviors with peers resulted in several significant relationships. Specifically, sensory reactive children appear to be less sociable (i. e. prosocial, friendly), more likely to engage in immature solitary pretend play, and more prone to utilize instrumental aggression in peer interactions. Perceptually aware children, on the other hand, tend to be more sociable (i.. e., prosocial, friendly, controls impulses), better able to appropriately and punctually comply with tasks given by teacher, less likely to engage in a number of solitary play behaviors (i. e., passive withdrawal and immature play), less likely to utilize instrumental or reactive aggressive strategies, and more likely to dodge negative peer interactions by avoiding bullies. Furthermore, the associations which sensory reactivity and perceptual awareness bear to children's sociable, non-social, and anti-social behaviors contrast those of other dimensions of temperament such as child activity level and emotionality. Therefore, the constructs extracted from the newly developed Children's Sensory Reactions Questionnaire appear to contribute to our overall understanding of child temperament as well as the associations between temperament and young children's social, nonsocial, and antisocial behaviors.

Ce mémoire porte sur l'étude et la conception de nouveaux outils d'aide à l'interprétation des images sismiques. L'objectif est de permettre au géologue d'extraire de manière interactive les réseaux de failles tridimensionnelles visibles dans les blocs sismiques. Dans la première partie du manuscrit, nous abordons l'étude des mécanismes perceptifs mis en jeu lors de l'inspection visuelle des données sismiques dans le but de détecter les failles. Cette analyse nous permet de proposer une nouvelle technique de segmentation supervisée, nommée pointé dynamique, qui est présentée dans la deuxième partie du manuscrit. Le mémoire se termine par la présentation de résultats de segmentation obtenus par pointé dynamique.

Despite two decades of empirical studies focusing on programmers and the problems with programming, usability of textual programming languages is still hard to achieve. Its younger relation, visual programming languages (VPLs) also share the same problem of poor usability. This research explores and investigates the usability issues relating to VPLs in order to suggest a set of design principles that emphasise usability. The approach adopted focuses on issues arising from the interaction and communication between the human (programmers), the computer (user interface), and the program. Being exploratory in nature, this PhD reviews the literature as a starting point for stimulating and developing research questions and hypotheses that experimental studies were conducted to investigate. However, the literature alone cannot provide a fully comprehensive list of possible usability problems in VPLs so that design principles can be confidently recommended. A commercial VPL was, therefore, holistically evaluated and a comprehensive list of usability problems was obtained from the research. Six empirical studies employing both quantitative and qualitative methodology were undertaken as dictated by the nature of the research. Five of these were controlled experiments and one was qualitative-naturalistic. The experiments studied the effect of a programming paradigm and of representation of program flow on novices' performances. The results indicated superiority of control-flow programs in relation to data-flow programs; a control-flow preference among novices; and in addition that directional representation does not affect performance while traversal direction does - due to cognitive demands imposed upon programmers. Results of the qualitative study included a list of 145 usability problems and these were further categorised into ten problem areas. These findings were integrated with other analytical work based upon the review of the literature in a structured fashion to form a checklist and a set of design principles for VPLs that are empirically grounded and evaluated against existing research in the literature. Furthermore, an extended framework for Cognitive Dimensions of Notations is also discussed and proposed as an evaluation method for diagrammatic VPLs on the basis of the qualitative study. The above consists of the major findings and deliverables of this research. Nevertheless, there are several other findings identified on the basis of the substantial amount of data obtained in the series of experiments carried out, which have made a novel contribution to knowledge in the fields of Human-Computer Interaction, Psychology of Programming, and Visual Programming Languages.

Esta dissertação investiga os efeitos da imersão física em jogos sérios com propósitos de aprendizado perceptivo. O principal objetivo é identificar um conjunto de orientações de design para ajudar desenvolvedores a escolher o nível apropriado de imersão a ser usado em jogos sérios a fim de assegurar que o jogo seja efetivo e confortável para os usuários. A hipótese principal é que altos níveis de imersão melhoram os resultados dos jogos sérios, mas níveis de imersão moderados também podem ser adequados com a vantagem de que causa menos mal-estar nos usuários. Nós organizamos uma revisão da literatura para melhor entender como a imersão física está sendo usada atualmente em jogos sérios, como a efetividade tem sido avaliada, e como a imersão afeta os resultados do jogo e a usabilidade. Então, nós conduzimos dois estudos de usuário empíricos para investigar os efeitos da fidelidade de display, interação e locomoção na percepção dos usuários e retenção de conhecimento. Para os experimentos, nós adaptamos um jogo sério previamente desenvolvido para análise de percepção de riscos, e desenvolvemos um novo jogo para educar trabalhadores sobre procedimentos de segurança que precisam ser seguidos durante instalações elétricas na linha de manutenção elétrica. Os resultados mostraram que, em geral, a fidelidade de display tem impacto na percepção de riscos quando procurando por riscos que não são óbvios. Alta fidelidade de display tem melhor desempenho na identificação de riscos complexos. A fidelidade de locomoção e interação não apresentou diferenças significantes no aprendizado perceptivo. Alta fidelidade também apresentou maior workload, mas nas mesmas condições a corretividade das tarefas foi maior e os participantes lembraram dos procedimentos tanto no pós-teste quanto no teste de retenção. Portanto, a retenção de conhecimento não foi afetada pelo workload introduzido pela técnica interativa. A partir destes experimentos, nós elaboramos e discutimos um conjunto de orientações de design que podem ser considerados para a escolha do nível apropriado de imersão física para ser usado no desenvolvimento de novos jogos sérios.
Nowadays, several application areas are adopting serious games as an ethical, safe and low-cost alternative to performing dangerous tasks, such as training of firefighters, and for medical surgeries. The employment of Virtual Reality in serious games helps to provide virtual environments much more realistic and better immersive experiences to the user. However, some VR systems can still cause sickness symptoms, such as nausea and headaches, and some of them can be expensive, such as CAVEs. This thesis investigates the effects of physical immersion on serious games with perceptual learning purposes. Its main goal is to identify a set of design guidelines to help developers on choosing the appropriate level of immersion to be used in serious games in order to assure it is effective and comfortable for users. The main hypothesis is that higher levels of immersion improve serious games outcomes, but moderate immersion may be also adequate with the advantage of causing less simulation sickness on users. We organized a survey of the literature to better understand how physical immersion is being currently used on serious games, how its effectiveness has been assessed, and how immersion impacts on the game outcomes and usability. Then, we conducted two empirical user studies looking for investigating the effects of the display, interaction and locomotion fidelity on users’ perception and knowledge retention. For the experiments, we adapted a serious game previously developed for risk assessment and developed a new one to educate workers in electricity-line maintenance on safety procedures that need to be followed during electric installations. Results showed that, in general, display fidelity has an effect on risk perception when searching for non-obvious risks. Higher display fidelity has better performance on complex risks identification. Interaction and locomotion fidelity did not show a significant difference in perceptual learning. Naturalness also presented higher workload, but in the same conditions the correctness of tasks was high and subjects recall the procedures in both post- and retention-test. Therefore, knowledge retention is not impacted by the workload imposed by the interactive technique. From these experiments, we elaborated and discussed a set of design guidelines that can be considered for the choice of the appropriate physical immersion to be used on the development of new serious games.

La perception de la flaveur des aliments dépend de la perception de plusieurs composés en mélange dans des proportions équilibrées. Interactions perceptives entre différents odorants ainsi que la libération dynamique des odorants par la matrice alimentaire sont les principaux facteurs qui influencent la perception globale. L’objectif de cette thèse de doctorat était d’étudier la perception du mélange d’odorants qui constituent l’arôme du fromage à pâte pressée, en prenant en compte les interactions perceptives et la dynamique de libération des composés par la matrice. Une stratégie, comprenant une caractérisation complète des odorants des fromages, suivie d’une recombinaison qualitative et d’une étude détaillée des rôles respectifs des composés clés, et finalement la prise en compte de la dynamique de libération par incorporation des composés dans une matrice fromagère, a été appliquée. Des caractérisations approfondies sensorielles et instrumentales nous ont permis de mettre en évidence des relations entre la perception sensorielle et la composition des composés volatils et proposer des hypothèses. La recombinaison des différents composés a été testée afin d’étudier le rôle de chacun en mélange. Aidés par des apprentis aromaticiens durant la phase des recombinaisons nous avons pu observer particulièrement que 12 composés étaient très importants pour l’odeur du fromage. Par la suite nous avons étudié des interactions entre odeurs en appliquant des techniques de stimulation odorante en phase gazeuse, puis la dynamique de libération des composés en incorporant l’arôme dans une matrice fromagère par des techniques permettant de mesurer la quantité libérée en fonction du temps
Perception of the aroma of food products depends both the chemical composition of food and human neurophysiology. The perception of food flavour, including cheese often relies on the perception of several aroma compounds in mixture in balanced proportions. Perceptual interactions among aroma compounds in mixtures and also the release of aroma compounds from the food product are the main factors that influence the global perceived aroma of food. Hence, the objective of this PhD study was to investigate the mixture of aroma compounds representing the aroma of semi-hard cheese by taking into account perceptual interactions among odorants and the dynamic release of the compounds by the cheese matrix. A strategy involving a complete characterisation of the cheeses followed by a recombination and investigation of the role of the key-aroma compounds and the dynamic release of the aroma compounds from the cheese matrix was taken into account. Comparison of the sensory and instrumental characteristics was made in order to highlight relationships between sensory perception of the aroma and the volatile composition of the cheeses and thus point out the molecular origins of the perceived cheese aroma. Recombination of selected aroma compounds was tested, in order to study the role of each aroma compound within the mixture. Especially 12 compounds seemed important for the semi-hard cheese aroma. To investigate the odour-odour interactions, different odour-stimulation tools were applied and dynamic release of aroma compounds was taken into account by incorporating the aroma compounds into a cheese matrix by different techniques allowing measuring the quantities released as function of time

Contexte et objectif:C’est presque une banalité que de dire qu’une des caractéristiques principales de la parole est sa variabilité : variabilité inter-sexe, inter-locuteur, mais aussi variabilité d’un contexte à un autre ou d’une répétition à une autre pour un même sujet. C’est cette variabilité qui fait à la fois la beauté de la parole mais aussi la complexité de son traitement par les technologies vocales, et la difficulté pour en comprendre les mécanismes. Dans cette thèse nous étudions certains aspects de cette variabilité, avec comme point de départ la variabilité observée chez un locuteur dans la répétition d’un même son dans les mêmes conditions, que nous appelons variabilité intrinsèque.Les modèles de contrôle moteur de la parole abordent principalement la variabilité contextuelle de la parole mais prennent rarement en compte sa variabilité intrinsèque, alors même que l’on sait que c’est cette variabilité qui donne à la parole tout son caractère naturel. Dans le contexte général du contrôle moteur, l’origine précise de la variabilité intrinsèque reste peu comprise et controversée. Cependant, une hypothèse courante est que la variabilité intrinsèque serait essentiellement due à du bruit neuronal dans la chaine d’exécution.L’objectif principal de cette thèse est d’aborder la variabilité intrinsèque et contextuelle de la production de la parole dans un cadre formel intégrateur. Pour cela nous faisons l’hypothèse que la variabilité intrinsèque n’est pas que le résultat d’un bruit d’exécution, mais qu’elle résulte aussi d’une stratégie de contrôle où la variabilité inter-répétition fait partie intégrante de la représentation de la tâche.Méthodologie:Nous formalisons cette idée dans un cadre computationnel probabiliste, la modélisation Bayésienne, où l’abondance de réalisations possibles d’un même item de parole est représentée naturellement sous la forme d’incertitudes, et où la variabilité est donc manipulée formellement. Nous illustrons la pertinence de cette approche à travers trois contributions.Résultats:Dans un premier temps, nous reformulons un modèle existant de contrôle optimal de la parole, le modèle GEPPETO, dans le formalisme probabiliste et démontrons que le modèle Bayésien contient GEPPETO comme un cas particulier. En particulier, nous illustrons comment l’approche Bayésienne permet de rendre compte de la variabilité intrinsèque tout en incluant les mêmes principes d’émergence et de structuration de la variabilité contextuelle proposés par GEPPETO.Dans un deuxième temps, le formalisme nous permet de dépasser le cadre de GEPPETO en y intégrant une composante somatosensorielle dans la représentation des buts. Cela permet d’introduire une variabilité interindividuelle sur la préférence sensorielle, c’est-à-dire la modulation des poids relatifs des cibles auditives et somatosensorielles, et permet d’expliquer la variabilité de compensation observée dans les études de perturbation sensorielle. Cette étape a nécessité l’élaboration d’hypothèses sur l’intégration des retours sensoriels dans la planification, dont nous avons cherché à évaluer la pertinence en concevant une expérience originale de production-perception de parole.Dans un troisième temps, nous exploitons le formalisme pour réinterpréter des données expérimentales récentes qui mettent en évidence un changement perceptif consécutif à un apprentissage moteur induit par une altération du retour auditif. Cela est rendu possible grâce à la représentation unifiée des connaissances dans le modèle, qui permet d’intégrer la production et la perception dans un cadre formel unique.L’ensemble de ces travaux illustre la capacité du formalisme Bayésien à proposer une démarche systématique et structurée pour la construction des modèles. Cette démarche facilite le développement des modèles et leur complexification progressive en précisant et explicitant les hypothèses formulées
Context and goal:It is almost a truism to affirm that one of the main features of speech is its variability: variability inter-gender, inter-speaker, but also variability from one context to another, or from one repetition to another for a given subject. Variability underlies at the same time the beauty of speech, the complexity of its treatment by speech technologies, and the difficulty for understanding its mechanism. In this thesis we study certain aspects of speech variability, our starting point being the variability characterizing the repetitions of a given utterance by a given subject, in a given condition, which we call intrinsic variability.Models of speech motor control have mainly focused on the contextual aspects of speech variability, and have rarely considered its intrinsic component, even though it is this fundamental component of variability that gives speech it naturalness. In the general context of motor control, the precise origin of the intrinsic variability of our movements remains controversial and poorly understood, however, a common assumption is that intrinsic variability would mainly originate from neural and muscular noise in the execution chain.The main goal of this thesis is to address the contextual and intrinsic component of speech variability in an integrative computational framework . To this aim, we postulate that the main component of the intrinsic variability of speech is not just execution noise, but that it results from a control strategy where intrinsic variability characterizes the abundance of possible productions of the intended speech item.Methodology:We formalize this idea in a probabilistic computational framework, Bayesian modeling, where the abundance of possible realizations of a given speech item is naturally represented as uncertainty, and where variability is thus formally manipulated. We illustrate the pertinence of this approach with three main contributions.Results:Firstly, we reformulate in Bayesian terms an existing model of speech motor control, the GEPPETO model, and demonstrate that this Bayesian reformulation, which we call B-GEPPETO, contains GEPPETO as a particular case . In particular, we illustrate how the Bayesian approach enables to account for the intrinsic component of speech variability while including the same principles proposed by GEPPETO for the emergence and structuration of its contextual component.Secondly, the Bayesian framework enable us to go beyond and extend B-GEPPETO in order to include a multisensory characterization of speech motor goals, with auditory and somatosensory components. We apply this extension to explore variability in the context of compensations to sensory-motor perturbation in speech production. We account for differences in compensation as sensory preferences implemented by modulating the relative contribution of each sensory modality in the model . The somatosensory characterization of speech motor goals involved a certain number of hypotheses that we intended to evaluate with two experimental studies.Finally, in our third contribution we exploit the formalism for the reinterpretation of recent experimental observations concerning perceptual changes following speech motor adaptation to auditory perturbations. This original analysis is made possible thanks to the unified representation of knowledge in the model, which enables to account for production and perception processes in a single computational framework.Taken together, these contributions illustrate how the Bayesian framework offers a structured and systematic approach for the construction of models in cognitive sciences . The framework facilitates the development of models and their progressive complexification by specifying and clarifying underlying assumptions

The meaning of spatial words can only be evaluated by establishing a reference to the properties of the environment in which the word is used. For example, in order to evaluate what is to the left of something or how fast is fast in a given context, we need to evaluate properties such as the position of objects in the scene, their typical function and behaviour, the size of the scene and the perspective from which the scene is viewed. Rather than encoding the semantic rules that define spatial expressions by hand, we developed a system where such rules are learned from descriptions produced by human commentators and information that a mobile robot has about itself and its environment. We concentrate on two scenarios and words that are used in them. In the first scenario, the robot is moving in an enclosed space and the descriptions refer to its motion ('You're going forward slowly' and 'Now you're turning right'). In the second scenario, the robot is static in an enclosed space which contains real-size objects such as desks, chairs and walls. Here we are primarily interested in prepositional phrases that describe relationships between objects ('The chair is to the left of you' and 'The table is further away than the chair'). The perspective can be varied by changing the location of the robot. Following the learning stage, which is performed offline, the system is able to use this domain specific knowledge to generate new descriptions in new environments or to 'understand' these expressions by providing feedback to the user, either linguistically or by performing motion actions. If a robot can be taught to 'understand' and use such expressions in a manner that would seem natural to a human observer, then we can be reasonably sure that we have captured at least something important about their semantics. Two kinds of evaluation were performed. First, the performance of machine learning classifiers was evaluated on independent test sets using 10-fold cross-validation. A comparison of classifier performance (in regard to their accuracy, the Kappa coefficient (κ), ROC and Precision-Recall graphs) is made between (a) the machine learning algorithms used to build them, (b) conditions under which the learning datasets were created and (c) the method by which data was structured into examples or instances for learning. Second, with some additional knowledge required to build a simple dialogue interface, the classifiers were tested live against human evaluators in a new environment. The results show that the system is able to learn semantics of spatial expressions from low level robotic data. For example, a group of human evaluators judged that the live system generated a correct description of motion in 93.47% of cases (the figure is averaged over four categories) and that it generated the correct description of object relation in 59.28% of cases.

Les émotions et leurs expressions par des agents virtuels sont deux enjeux importants pour les interfaces homme-machine affectives à venir. En effet, les évolutions récentes des travaux en psychologie des émotions, ainsi que la progression des techniques de l’informatique graphique, permettent aujourd’hui d’animer des personnages virtuels réalistes et capables d’exprimer leurs émotions via de nombreuses modalités. Si plusieurs systèmes d’agents virtuels existent, ils restent encore limités par la diversité des modèles d’émotions utilisés, par leur niveau de réalisme, et par leurs capacités d’interaction temps réel. Dans nos recherches, nous nous intéressons aux agents virtuels capables d’exprimer des émotions via leurs expressions faciales en situation d’interaction avec l’utilisateur. Nos travaux posent de nombreuses questions scientifiques et ouvrent sur les problématiques suivantes : Comment modéliser les émotions en informatique en se basant sur les différentes approches des émotions en psychologie ? Quel niveau de réalisme visuel de l’agent est nécessaire pour permettre une bonne expressivité émotionnelle ? Comment permettre l’interaction temps réel avec un agent virtuel ? Comment évaluer l’impact des émotions exprimées par l’agent virtuel sur l’utilisateur ? A partir de ces problématiques, nous avons axé nos travaux sur la modélisation informatique des émotions et sur leurs expressions faciales par un personnage virtuel réaliste. En effet, les expressions faciales sont une modalité privilégiée de la communication émotionnelle. Notre objectif principal est de contribuer l’amélioration de l’interaction entre l’utilisateur et un agent virtuel expressif. Nos études ont donc pour objectif de mettre en lumière les avantages et les inconvénients des différentes approches des émotions ainsi que des méthodes graphiques étudiées. Nous avons travaillé selon deux axes de recherches complémentaires. D’une part, nous avons exploré différentes approches des émotions (catégorielle, dimensionnelle, cognitive, et sociale). Pour chacune de ces approches, nous proposons un modèle informatique et une méthode d’animation faciale temps réel associée. Notre second axe de recherche porte sur l’apport du réalisme visuel et du niveau de détail graphique à l’expressivité de l’agent. Cet axe est complémentaire au premier, car un plus grand niveau de détail visuel pourrait permettre de mieux refléter la complexité du modèle émotionnel informatique utilisé. Les travaux que nous avons effectués selon ces deux axes ont été évalués par des études perceptives menées sur des utilisateurs.La combinaison de ces deux axes de recherche est rare dans les systèmes d’agents virtuels expressifs existants. Ainsi, nos travaux ouvrent des perspectives pour l’amélioration de la conception d’agents virtuels expressifs et de la qualité de l’interaction homme machine basée sur les agents virtuels expressifs interactifs. L’ensemble des logiciels que nous avons conçus forme notre plateforme d’agents virtuels MARC (Multimodal Affective and Reactive Characters). MARC a été utilisée dans des applications de natures diverses : jeu, intelligence ambiante, réalité virtuelle, applications thérapeutiques, performances artistiques, etc
Emotions and their expressions by virtual characters are two important issues for future affective human-machine interfaces. Recent advances in psychology of emotions as well as recent progress in computer graphics allow us to animate virtual characters that are capable of expressing emotions in a realistic way through various modalities. Existing virtual agent systems are often limited in terms of underlying emotional models, visual realism, and real-time interaction capabilities. In our research, we focus on virtual agents capable of expressing emotions through facial expressions while interacting with the user. Our work raises several issues: How can we design computational models of emotions inspired by the different approaches to emotion in Psychology? What is the level of visual realism required for the agent to express emotions? How can we enable real-time interaction with a virtual agent? How can we evaluate the impact on the user of the emotions expressed by the virtual agent? Our work focuses on computational modeling of emotions inspired by psychological theories of emotion and emotional facial expressions by a realistic virtual character. Facial expressions are known to be a privileged emotional communication modality. Our main goal is to contribute to the improvement of the interaction between a user and an expressive virtual agent. For this purpose, our research highlights the pros and cons of different approaches to emotions and different computer graphics techniques. We worked in two complementary directions. First, we explored different approaches to emotions (categorical, dimensional, cognitive, and social). For each of these approaches, a computational model has been designed together with a method for real-time facial animation. Our second line of research focuses on the contribution of visual realism and the level of graphic detail of the expressiveness of the agent. This axis is complementary to the first one, because a greater level of visual detail could contribute to a better expression of the complexity of the underlying computational model of emotion. Our work along these two lines was evaluated by several user-based perceptual studies. The combination of these two lines of research is seldom in existing expressive virtual agents systems. Our work opens future directions for improving human-computer interaction based on expressive and interactive virtual agents. The software modules that we have designed are integrated into our platform MARC (Multimodal Affective and Reactive Characters). MARC has been used in various kinds of applications: games, ubiquitous intelligence, virtual reality, therapeutic applications, performance art, etc

This study explores the concept of feedback from various perspectives in psychology and cognitive science. Specifically, the theories of ecological psychology, situated and Distributed Cognition, Cognitive Systems Engineering and Embodied cognition are investigated and compared. Cognitive Systems Engineering provides a model of feedback and related constructs, to understand human behavior in complex working environments. Earlier theories such as ecological psychology, considered feedback as direct perception. Situated cognition clearly inherits ideas from ecological psychology, whereas distributed cognition provides a deeper understanding of feedback through artifact use. Cognitive Systems Engineering provides a systematic view of feedback and control. This framework is a suitable perspective to understanding feedback in human-machine settings.

Thesis (Ph. D.)--Georgia Institute of Technology, 2006.
Title from PDF t.p. (viewed on Oct. 28, 2006). Marc Weissburg, Committee Chair ; Mark Hay, Committee Member ; Lin Jiang, Committee Member ; David Dusenbery, Committee Member ; Don Webster, Committee Member. Includes bibliographical references (p. 115-122).