Littérature scientifique sur le sujet « Perceptual interactions »

The verbal transformation effect (VTE) refers to perceptual switches while listening to a speech sound repeated rapidly and continuously. It is a specific case of perceptual multistability providing a rich paradigm for studying the processes underlying the perceptual organization of speech. While the VTE has been mainly considered as a purely auditory effect, this paper presents a review of recent behavioural and neuroimaging studies investigating the role of perceptuo-motor interactions in the effect. Behavioural data show that articulatory constraints and visual information from the speaker's articulatory gestures can influence verbal transformations. In line with these data, functional magnetic resonance imaging and intracranial electroencephalography studies demonstrate that articulatory-based representations play a key role in the emergence and the stabilization of speech percepts during a verbal transformation task. Overall, these results suggest that perceptuo (multisensory)-motor processes are involved in the perceptual organization of speech and the formation of speech perceptual objects.

The vestibular coriolis (or "cross-coupling") effect is traditionally explained by cross-coupled angular vectors, which, however, do not explain the differences in perceptual disturbance under different acceleration conditions. For example, during head roll tilt in a rotating chair, the magnitude of perceptual disturbance is affected by a number of factors, including acceleration or deceleration of the chair rotation or a zero-g environment. Therefore, it has been suggested that linear-angular interactions play a role. The present research investigated whether these perceptual differences and others involving linear coriolis accelerations could be explained under one common framework: the laws of motion in three dimensions, which include all linear-angular interactions among all six components of motion (three angular and three linear). The results show that the three-dimensional laws of motion predict the differences in perceptual disturbance. No special properties of the vestibular system or nervous system are required. In addition, simulations were performed with angular, linear, and tilt time constants inserted into the model, giving the same predictions. Three-dimensional graphics were used to highlight the manner in which linear-angular interaction causes perceptual disturbance, and a crucial component is the Stretch Factor, which measures the "unexpected" linear component.

Perceptual dependence—the existence of perceptual interactions between the component dimensions of the same stimulus—was investigated for shape and texture during haptic processing. The haptic system combines tactual and kinesthetic information. Previous research has demonstrated that haptic exploration influences the extent to which object properties are integrated. Conditions designed to promote and impede the integration of shape and texture were compared. Perceptual independence was assessed by the use of a speeded-classification paradigm and quantitative tests developed by Ashby and Maddox. Results indicate that shape and texture are perceptually dependent for both conditions. Hand-movement analyses show simultaneous exploration for both dimensions. The tendency to process dimensions dependently is discussed in terms of a limited-capacity model of haptic-information processing.

ABSTRACTThis study compared vocal features of deception that can be measured by acoustic equipment with vocal features of deception that can be measured perceptually by human coders. As deception researchers have traditionally measured vocal behavior with either acoustic or perceptual methods (but not both), it is uncertain what correspondence, if any, exists between these methods. This study attempted to determine the degree of this correspondence. Deceptive interactions from an earlier study (Burgoon, Buller, Ebesu, & Rockwell, 1994) were used to conduct a detailed analysis of the vocal features of deceptive speech. The vocal samples were analyzed perceptually and acoustically. Results indicated moderate correlations between some acoustic and perceptual variables; neither measurement type, however, proved conclusively superior to the other in discriminating between truth and deception.

Proefschrift Universiteit van Amsterdam.
Op omslag: IFOTT, en :LOT, Netherlands Graduate School of Linguistics, Landelijke Onderzoekschool Taalwetenschap. Met index, lit. opg. - Met samenvatting in het Nederlands.

Amblyopia, a developmental disorder of the visual system, is widely known to cause a reduction in optotype acuity but it can also be associated with disrupted binocular vision, reduced contrast sensitivity and many other subtle high level visual processing deficits. The initial stages of the work presented in this thesis involved laboratory investigation of the functional visual deficit in global motion processing that has previously been reported abnormal in the presence of amblyopia. The key question is whether higher-levels of visual processing "inherit" abnormalities from lower levels, or whether additional developmental abnormalities arise in direct consequence of impoverished visual input. Overall, the results imply a far more complex perceptual change in amblyopia than would be predicted by the well -established losses in resolution and contrast sensitivity. The motivation behind Chapters 5 and 6 stems from the current observation that the recovery of visual function in amblyopia is contingent on even brief periods of correlated binocular vision, suggesting that amblyopia is intrinsically a binocular problem and that it is suppressive mechanisms that render the cortex, which is a structurally binocular system, functionally monocular. Research is now casting doubts on the idea that amblyopes do not possess cortical binocular connections, suggesting an active suppression rather than a deficit of cellular function. Interestingly, this is echoed in the clinical domain where, in cases of de-correlated visual input, strabismus clinical protocols have now established that the correction of refractive error alone can be sufficient to improve acuity, again implying incomplete inhibition mechanisms. The clinical investigations in this thesis have involved the validation of a series of psychophysical paradigms in cohorts of juvenile and adult amblyopes (as well as age-matched controls) to establish the degree of binocular interaction present and to explore the potential for treating amblyopia with prolonged viewing of a binocular stimulus adapted to correlate the visual imput from both eyes.

Thesis (Ph. D.)--Harvard-MIT Division of Health Sciences and Technology, February 2010.
"September 2009." Cataloged from PDF version of thesis.
Includes bibliographical references (p. 160-175).
This project is an experimental study of perceptual interactions between auditory and tactile stimuli. These experiments present vibrotactile stimuli to the fingertip and auditory tones diotically in broadband noise. Our hypothesis states that if the auditory and tactile systems integrate, the performance of the two sensory stimuli presented simultaneously will be different from the performance of the individual sensory stimuli. The research consists of work in two major areas: (1) Studies of the detection of auditory and tactile sinusoidal stimuli at levels near the threshold of perception (masked thresholds for auditory stimuli and absolute thresholds for tactile stimuli); and (2) Studies of loudness matching employing various combinations of auditory and tactile stimuli presented at supra-threshold levels. Results were compared to three models of auditory-tactile integration. The objective detection studies explore the effects of three major variables on perceptual integration: (a) the starting phase of the auditory relative to the tactile stimulus; (b) the temporal synchrony of stimulation within each of the two modalities; and (c) the frequency of stimulation within each modality. Detection performance for combined auditory-tactile (A+T) presentations was measured using stimulus levels that yielded 63%-77%-correct unimodal performance in a 2-Interval, 2-Alternative Forced- Choice procedure. Results for combined vibrotactile and auditory detection indicated: (1) For synchronous presentation of 500-msec, 250 Hz sinusoidal stimuli, percent-correct scores in the combined A+T conditions were significantly higher than scores within each single modality;
(cont.) (2) Scores in the A+T conditions were not affected by the relative phase of the 250 Hz auditory and tactile stimuli; (3) For asynchronous presentation of auditory and tactile 250 Hz stimuli, scores on the A+T conditions improved only when the tactile stimulus preceded the auditory stimulus (and not vice versa); and (4) The highest rates of detection in the combined-modality stimulus were obtained when stimulating frequencies in the two modalities were equal or closely spaced (and within the Pacinian range). The lack of phase effect suggests that integration operates on the envelopes rather than on temporal fine structure. The effects of asynchronous presentation imply a shorter time constant in the auditory compared to the tactile modality and are consistent with time constants deduced from single-modality masking experiments. The effects of frequency depend both on absolute frequency and on relative frequency of stimulation within each modality. In general, we found that an additive sensitivity model best explained detection performance when tones were presented synchronously and of the same frequency. In the second area of research, loudness matching was employed in a subjective study of the effects of frequency on auditory-tactile integration for stimuli presented at supra-threshold levels. These experiments, which were derived from previous auditory studies demonstrating the dependence of loudness on critical-band spacing of tonal signals, employed various combinations of auditory and tactile stimuli that were presented at equally loud levels in isolation.
(cont.) Loudness matches were obtained for auditory-only (A+A) and auditory-tactile (A+T) stimuli that were both close as well as farther apart in frequency. The results show that the matched loudness of an auditory pure tone is greater when the frequencies of combined stimuli (both A+A and A+T) are farther apart in frequency than when they are close in frequency. These results are consistent with the results found in the previous experiment exploring the frequency relationships at near-threshold levels, as well as with results in the psychoacoustic literature, and suggest that the auditory and tactile systems are interacting in a frequency-specific manner similar to the interactions of purely auditory stimuli. The research conducted here demonstrates objective and subjective perceptual effects that support the mounting anatomical and physiological evidence for interactions between the auditory and tactual sensory systems.
by E. Courtenay Wilson.
Ph.D.

Notre recherche se concentre sur la conception d’interfaces pour la communication émotionnelle à distance. Notre travail s’inscrit notamment dans le cadre de l’élaboration d’espaces numériques d’interactions tactiles, rendus possibles par la connexion en réseau de dispositifs de suppléance perceptive. Nous adoptons une approche énactive et interactionniste de la cognition sociale. Nous utilisons de plus la méthodologie minimaliste, qui offre une opportunité originale d’étudier la constitution des rencontres interpersonnelles au sein d’espaces mettant en place des couplages sensori-Moteurs inédits. Nous déroulons deux méthodologies, en articulant les domaines de la recherche fondamentale, de l’étude expérimentale et du design.Dans un premier temps, nous partons de la théorie autour du thème du toucher, avant de proposer une solution d’interface : l’application smartphone Touch Through. Elle permet une suppléance perceptive du toucher en se basant sur des propriétés fonctionnelles de ce sens. Touch Through sert ensuite à une expérimentation basée autour d’un test de Turing minimaliste, afin de tester la possibilité d’un sentiment de présence à distance avec notre application et de la mettre entre les mains d’utilisateurs.Dans un second temps, nous adressons la question de l’attention conjointe avant de mettre en place une étude expérimentale autour de ce thème (tâche de distinction entre des objets privés et des objets communs). Forts des résultats, nous proposons alors un prototype d’interface pour le réseau social facebook, en mettant en avant des possibilités d’attention conjointe au sein du site et en offrant une interaction inédite autour des posts
Our research is aimed at designing interfaces for the remote communication of emotions. In particular our work is part of the development of tactile interaction digital spaces. These are made possible by a network connection between perceptual supplementation devices. We consider an enactive and interactionist approach of social cognition. In addition, we use a minimalist methodology. This offers an original opportunity to study the very constitution of interpersonal encounters within spaces where unprecedented sensorimotor couplings happen. We work through two methodologies, each articulating the fields of fundamental research, experimental study and design.In a first phase, we start from theory around the theme of touch, before proposing an interface solution – the smartphone application ‘Touch Through’. This application allows a perceptual supplementation of touch using its functional properties. ‘Touch Through’ is then used in a experimental study based on a minimalist Turing test. This experiment is about testing the possibility of a presence in absence feeling through our application. It was also an opportunity to have users handling it.In the second phase we address the question of joint attention. After a study of this question, we design an experiment where we propose a task of discrimination between private and common objects. Based on the outcome of this experiment, we then propose an interface prototype for the social network ‘facebook’. With this prototype interface, the facebook website is enhanced with possibilities of joint attention. In addition, facebook users can take part in an original interaction around posts

The primary visual cortex (V1) has long been considered the main low level visual analysis area of the brain. The classical view is of a feedfoward system functioning as an edge detector, in which each cell has a receptive field (RF) and a preferred orientation. Whilst intuitive, this view is not the whole story. Although stimuli outside a neuron’s RF do not result in an increased response by themselves, they do modulate a neuron’s response to what’s inside its RF. We will refer to such extra-RF effects as contextual modulation. Contextual modulation is thought to underlie several perceptual phenomena, such as various orientation illusions and saliency of specific features (such as a contour or differing element). This gives a view of V1 as more than a collection of edge detectors, with neurons collectively extracting information beyond their RFs. However, many of the accounts linking psychophysics and physiology explain only a small subset of the illusions and saliency effects: we would like to find a common principle. So first, we assume the contextual modulations experienced by V1 neurons is determined by the elastica model, which describes the shape of the smoothest curve between two points. This single assumption gives rise to a wide range of known contextual modulation and psychophysical effects. Next, we consider the more general problem of encoding and decoding multi-variate stimuli (such as center surround gratings) in neurons, and how well the stimuli can be decoded under substantial noise levels with a maximum likelihood decoder. Although the maximum likelihood decoder is widely considered optimal and unbiased in the limit of no noise, under higher noise levels it is poorly understood. We show how higher noise levels lead to highly complex decoding distributions even for simple encoding models, which provides several psychophysical predictions. We next incorporate more updated experimental knowledge of contextual modulations. Perhaps the most common form of contextual modulations is center surround modulation. Here, the response to a center grating in the RF is modulated by the presence of a surrounding grating (the surround). Classically this modulation is considered strongest when the surround is aligned with the preferred orientation, but several studies have shown how many neurons instead experience strongest modulation whenever center and surround are aligned. We show how the latter type of modulation gives rise to stronger saliency effects and unbiased encoding of the center. Finally, we take an experimental perspective. Recently, both the presence and the underlying mechanisms of contextual modulations has been increasingly studied in mice using calcium imaging. However, cell signals extracted with calcium imaging are often highly contaminated by other sources. As contextual effects beyond center surround modulation can be subtle, a method is needed to remove the contamination. We present an analysis toolbox to de-contaminate calcium signals with blind source separation. This thesis thus expands our understanding of contextual modulation, predicts several new experimental results, and presents a toolbox to extract signals from calcium imaging data which should allow for more in depth studies of contextual modulation.

One way in which peripheral vision may acquire the functional role of the fovea, a part of the retina preferentially used for complex visual tasks (such as reading and face recognition) is by modulating the strength of intracortical connections in the humans’ visual areas with perceptual learning. Perceptual learning is a practice-dependent improvement in a visual task performance that can persist for several months, and is specific for stimulus, task, eye presentation and retinal locus of stimulation. These specificity effects have been explained on the basis of neural plasticity, consisting in long-term modifications of a number of mechanisms in the early visual cortices, that are selective for basic stimulus attributes (Karni & Sagi, 1991, 1993; Ahissar & Hochstein, 1993, 1996; Casco & Campana, 2001). Perceptual learning experiments with stimuli involving lateral masking (Polat & Sagi, 1994b, 1995; Polat, Ma-Naim, Belkin & Sagi, 2004) have suggested that practice is able to modulate short- and long-range lateral interactions between neurons responding to collinear elements. These studies showed that contrast thresholds for a target are modulated by the presence collinear flankers, and the type of modulation depends on the distance between the central target and the flankers: inhibitory for short target-to-flankers distance, and facilitatory for longer distances. With the training the suppression from the short target-flanker separation can be reduced and facilitation at relatively long target-flanker separation increases. These studies suggest that practice on lateral interactions increases the efficacy of the collinear interactions between neighbouring neurons, an effect that enhances connectivity with remote neurons via a cascade of local interactions. Most importantly, perceptual learning on lateral interactions has been showed to be useful for improving contrast sensitivity in people with normal low vision (Tan & Fong, 2008; Polat, 2009) or with impaired lateral interactions such as amblyopia (Polat et al., 2004). Notably, these studies showed that, differently from previous perceptual learning experiments where non transfer of learning to different stimulus attributes was observed, the effect of training on lateral interactions transferred to higher level visual tasks, like visual acuity (VA) (Tan & Fong, 2008) producing a long-lasting perceptual benefit in everyday visual tasks. However, in all these studies stimuli were presented in the fovea. This thesis aimed at investigating the possibility that the effect of training also improves lateral interaction in retinal regions eccentric with respect to the fovea. Lateral interactions strongly depend on eccentricity: in the periphery they are mostly inhibitory (Petrov, Carandini & McKee, 2005; Cavanaugh et al., 2002). These evidence leads to the hypothesis that peripheral vision may acquire some of the functional role of the fovea only if inhibition is reduced. Shani & Sagi (2005) showed that collinear facilitation in the near-periphery is weak and that perceptual learning seems not to be effective in modulating lateral interactions. However, in their study the training was very short, and only one target-to-flanker distance was tested. If lateral interactions could be modulated in the near periphery, and transfer to visual tasks such as VA and crowding was possible, this could be extremely important for rehabilitation of individuals with loss of central vision, such as in macular degeneration. These type of patients, after the loss of central vision, are forced to use the periphery of the visual field for the most demanding visual tasks like face recognition and reading. In Experiment 1, we were interested in verifying whether inhibitory lateral interactions in the near-periphery (4 degrees of eccentricity) may be reduced by training and if training transfers to other visual functions. Eight subjects were trained with different spatial frequencies (1, 2, 4, and 8 cycles per degree - cpd) and different target-to-flankers separations (2, 3, 4, and 8). Before the practice sessions subjects performed a series of pre-tests aimed at measuring their peripheral contrast sensitivity function (CSF), peripheral visual acuity (VA) and crowding effect. Consistently with previous studies (Petrov et al., 2005; Cavanagh et al., 2002), results of the Experiment 1 showed that, in the near periphery, lateral interactions are inhibitory even at target-to-flanker distances (4) where facilitatory interactions are found in central vision. Facilitation was reported for a target-to-flankers distance of 8, consistently with the most recent investigation on peripheral lateral interactions (Lev & Polat, 2011). Most importantly, Experiment 1 showed that lateral interactions in parafoveal vision can be modulated by training, reducing the inhibition, and that perceptual learning transfers to other visual abilities, leading to a reduction of crowding. Since learning specificity is viewed as the main indicator of the level of processing at which learning takes place, and since learning of lateral interaction has been shown to transfer to different visual functions, in Experiment 2 we tested the specificity of learning to basic stimulus features such as target-flankers local and global orientation and retinal position. We trained 4 new subjects in contrast detection of a collinearly flanked vertical target, and found a significant learning effect for the trained configuration but no transfer of learning to either the same stimulus presented in a symmetrical retinal location, nor to a 45 deg oriented collinear target-flankers configuration, presented in the same retinal position as the learning stimulus. The finding that these transfer stimuli are immune to perceptual learning of vertical orientations strongly suggests that the modulation of lateral interactions through perceptual learning is functionally specific, and that transfer to different visual functions can only occur when these are based on the specific early mechanism that is learned. In Experiment 3, we aimed at exploiting the effects of the perceptual learning of lateral interactions for improving peripheral vision in patients affected by macular degeneration. Training consisted in a contrast detection of a Gabor target with collinear high contrast Gabor flankers, at different target-to-flankers separations, located in their preferential retinal locus (PRL, the new fixation point that spontaneously emeges in this type of patients) and in a symmetrical location. The rationale behind the measuring or lateral interactions and the training in the PRL and in another retinal location was to point out possible differences in intracortical connectivity for this new fixation point respect to other retinal spots. Consistently with other studies (Dilks, Baker, Peli and Kanwisher, 2009), we did not find major differences in terms of lateral interactions and perceptual learning effects between PRL and the symmetrical locus. Training increased contrast sensitivity, and, despite not having any effect on crowding, improved visual acuity in the maculopathy subjects. The absence of crowding reduction could be due to a “roof effect”, since this type of patients naturally train their peripheral view, probably reaching their maximal performance even before the training.. Nevertheless, the improvement of visual acuity opens new perspectives for the rehabilitation of patients with macular degeneration, but also for improving peripheral vision in normal-sighted subjects, since recent studies showed the important role of the periphery of the visual field in tasks such as postural stability, locomotion and driving. In Experiment 4, we investigated the architecture of peripheral lateral interactions in a maculopathy patients, finding collinear facilitation at shorter target-to-flankers separation respect to normal-sighted subjects. Interestingly, collinear facilitation was reported for target presentation in the PRL but not in the No-PRL, where collinear interactions were only inhibitory. Moreover, perceptual learning training appeared to be effective in modulating lateral interactions only in the PRL, questioning the hypothesis of a “use-dependent” cortical reorganization, supported, among the others, by Dilks et al. (2009).
Un modo in cui la visione periferica può acquisire il ruolo funzionale della fovea, la parte della retina preferenzialmente utilizzata per compiti visivi complessi (come la lettura e il riconoscimento dei volti) è tramite la modulazione della forza delle connessioni intracorticali presenti nelle aree visive umane tramite apprendimento percettivo. L’apprendimento percettivo è un miglioramento della prestazione n un compito visivo in seguito alla pratica, può mantenersi per diversi mesi ed è specifico per lo stimolo, il computo e l’occhio utilizzato nel training e per il locus retinico in cui è avvenuta la stimolazione. Questi effetti di specificità sono stati spiegati sulla base della plasticità neurale, che consiste in una modifica a lungo termine di alcuni meccanismi presenti nelle prime aree visive corticali, selettive per caratteristiche basilari dello stimolo (Karni & Sagi, 1991, 1993; Ahissar & Hochstein, 1993, 1996; Casco & Campana, 2001). Esperimenti di apprendimento percettivo con stimoli di mascheramento laterale (Polat & Sagi, 1994b, 1995; Polat, Ma-Naim, Belkin & Sagi, 2004) hanno dimostrato che la pratica può modulare interazioni laterali a breve e lungo raggio tra neuroni che rispondono ad elementi collineari. Questi studi mostrano che le soglie di contrasto per un target centrale sono modulate dalla presenza di elementi vicini, orientati col linearmente, e che il tipo di modulazione dipende dalla distanza tra il target centrale e gli elementi vicini (flankers): inibitoria per brevi distanze tra target e flankers, facilitatoria per distanze maggiori. Con la pratica, l’inibizione per le brevi distanze target-flankers può essere ridotta e la facilitazione a maggiori distanze aumenta. Questi studi suggeriscono che la pratica sulle interazioni laterale può aumentare l’efficacia delle interazioni collineari tra neuroni vicini, un effetto che aumenta la connettività con neuroni più lontani in seguito ad una cascata di interazioni locali. Inoltre, è stato dimostrato che l’applicazione dell’apprendimento percettivo sulle interazioni laterali produce risultati effettivi nel miglioramento della sensibilità al contrasto in individui con miopia (Tan & Fong, 2008; Polat, 2009) o con interazioni laterali abnormali, come nel caso dell’ambliopia (Polat et al., 2004). Questi studi dimostrano che, a differenza dai precedenti esperimenti sull’apprendimento percettivo che non riportavano il trasferimento del miglioramento ad altri compiti, l’effetto della pratica sulle interazioni laterali si trasferisce ad abilità visive di più alto livello, come l’acuità visiva (Tan & Fong, 2008), dando luogo a benefici percettivi a lungo termine nella quotidianità di questi soggetti. Tuttavia, negli studi finora condotti, gli stimoli venivano presentati in fovea. In questa tesi, l’obiettivo è stato quello di studiare la possibilità che gli effetti del training possano migliorare le interazioni laterali in regioni retiniche periferiche rispetto alla fovea. Le interazioni laterali dipendono fortemente dall’eccentricità: nella periferia del campo visivo sono principalmente inibitorie (Petrov, Carandini & McKee, 2005; Cavanaugh et al., 2002). Questa evidenza porta ad ipotizzare che la visione periferica possa acquisire il ruolo funzionale della fovea solo se l’inibizione viene ridotta. Shani & Sagi (2005) hanno dimostrato che la facilitazione collineare nella periferia del campo visivo è debole e che l’apprendimento percettivo non sembra efficace nel modulare le interazioni laterali. Tuttavia, nel loro studio il periodo di pratica era molto breve, in più le distanze target-flankers testate erano ridotte. Se le interazioni laterali possono essere modulate nella periferia del campo visivo, e trasferire il miglioramento ad abilità visive come l’acuità visiva o il crowding (affollamento visivo), questo risultato sarebbe estremamente importante per la riabilitazione di individui con perdita della visione centrale, come nel caso della maculopatia. Questo tipo di pazienti, dopo la perdita della visione centrale, sono obbligati ad usare la periferia del campo visivo per i compiti visivi più complessi, come la lettura ed il riconoscimento dei volti. Nell’Esperimento 1, l’obiettivo era verificare se le interazioni laterali inibitorie nella periferia del campo visivo (4 gradi di eccentricità) potevano essere ridotte dall’apprendimento e se l’eventualmente miglioramento si trasferiva ad altre funzioni visive. I soggetti sono stati allenati con diverse frequenze spaziali (1, 2, 4, and 8 cicli per grado) e diverse distanze target-flankers (2, 3, 4 e 8). Prima delle sessioni di pratica, ai soggetti venivano misurate le prestazioni di base in una serie di compiti visivi, come la sensibilità al contrasto periferica (CSF), l’acuità visiva periferica (VA) e l’effetto di crowding (affollamento visivo).Coerentemente con studi precedenti (Petrov et al., 2005; Cavanagh et al., 2002), i risultati dell’Esperimento 1mostrano che, nella periferia del campo visivo, le interazioni laterali sono inibitorie anche a distanze target-flankers che producono facilitazione in visione centrale (4). La distanza alla quale è stata trovata la facilitazione in periferia (8), è coerente coi più recenti studi sulle interazioni laterali nella periferia del campo visivo (Lev & Polat, 2011). Soprattutto, l’Esperimento 1 mostra che le interazioni laterali in parafovea possono essere modulate dalla pratica, riducendo l’inibizione, e che l’apprendimento percettivo trasferisce il miglioramento ad altre abilità visive, portando alla riduzione del fenomeno di affollamento visivo (crowding). Dato che la specificità dell’apprendimento è considerata l’indicatore principale del livello di processamento dell’informazione in entrata a cui l’apprendimento si verifica, nell’Esperimento 2 abbiamo testato la specificità dell’apprendimento per caratteristiche base dello stimolo come l’orientamento locale e globale degli elementi utilizzati durante il training e la loro posizione spaziale. Un nuovo gruppo di soggetti è stato allenato in un compito di detezione del contrasto per uno stimolo centrale affiancato da elementi collineari, mostrano un effetto di apprendimento significativo per la configurazione allenata, ma non riportando alcun trasferimento per lo stesso stimolo presentato in una posizione retinica simmetrica, né per configurazioni con orientamento locale (tra gli elementi) o globale differente. L’evidenza che questi stimoli sono immuni all’apprendimento percettivo suggerisce in maniera decisa che la modulazione delle interazioni laterali tramite apprendimento percettivo sia specifica per la funzione, e che il trasferimento a diverse funzioni visive può avere luogo solo quando queste sono basate su specifici meccanismi precoci. Nell’Esperimento 3, abbiamo testato la possibilità di utilizzare gli effetti dell’apprendimento percettivo delle interazioni laterali per migliorare la vision periferica in pazienti affetti da degenerazione maculare. Il training consisteva in un compito di detezione del contrasto per uno stimolo centrale affiancato da elementi collineari, collocati a diverse distanze target-flankers, presentati nel locus retinico preferenziale (PRL, il nuovo punto di fissazione che spontaneamente questi pazienti sviluppano) ed in una posizione simmetrica. L’idea alla base della misurazione delle interazioni laterali e dell’apprendimento nel PRL ed in un’altra posizione retinica era di verificare se vi siano differenze nella connettività intracorticale tra il nuovo punto di fissazione periferico ed un altro locus retinico. Coerentemente con altri studi ((Dilks, Baker, Peli and Kanwisher, 2009), non abbiamo riscontrato evidenti differenze in termini di interazioni laterali ed effetti di apprendimento percettivo tra PRL e il locus retinico simmetrico.Il training ha migliorato la sensibilità al contrasto e, pur non avendo avuto effetto sul crowding, ha migliorato l’acuità visiva nei soggetti maculopatici. L’assenza di riduzione del crowding può essere dovuta ad un “effetto tetto”, dato che questo tipo di pazienti allenano “naturalmente”, nella quotidianità, la loro visione periferica, e probabilmente hanno raggiunto la massima prestazione possibile nella visione periferica prima di iniziare il training. Nondimeno, il miglioramento nell’acuità visiva apre nuove prospettive per la riabilitazione di pazienti con maculopatia, ma anche per migliorare la visione periferica in soggetti normovedenti, dato che studi recenti hanno mostrato l’importante ruolo della periferia del campo visivo in attività come la stabilità posturale, la locomozione e la guida. Nell’Esperimento 4, abbiamo studiato l’architettura delle interazioni laterali periferiche in un soggetto maculopatico, trovando facilitazione collineare a distanze target-flankers minori rispetto ai soggetti normovedenti. Il dato interessante è che la facilitazione collineare emerge solo per la presentazione del target nel PRL, mentre nel PRL le interazioni collineari sono solo inibitorie. Inoltre, l’apprendimento percettivo sembra efficace nel modulare le interazioni laterali solo nel PRL, mettendo in discussione l’ipotesi “uso-dipendente” per la riorganizzazione corticale, supportata, tra gli altri, da Dilks et al. (2009).

This chapter shows how visual illusions arise from neural processes that play an adaptive role in achieving the remarkable perceptual capabilities of advanced brains. It clarifies that many visual percepts are visual illusions, in the sense that they arise from active processes that reorganize and complete perceptual representations from the noisy data received by retinas. Some of these representations look illusory, whereas others look real. The chapter heuristically summarizes explanations of illusions that arise due to completion of perceptual groupings, filling-in of surface lightnesses and colors, transformation of ambiguous motion signals into coherent percepts of object motion direction and speed, and interactions between the form and motion cortical processing streams. A central theme is that the brain is organized into parallel processing streams with computationally complementary properties, that interstream interactions overcome these complementary deficiencies to compute effective representations of the world, and how these representations generate visual illusions.

This chapter suggests that, when considering the philosophical problem of other minds, we distinguish between "thick" and "thin" versions of it. While traditional approaches take the problem to be a thick one, more recent work can be seen as addressing only a thin variant. Dretske, while acknowledging the thick problem, proposes a perceptual model of our knowledge of other minds which addresses only the thin version. The chapter proposes that, in the place of the thick problem, we consider the quality of our interactions with others. Following Wittgenstein, it suggests that where individuals share a nature their interactions exhibit a quality that it calls "depth." Where that nature is not, or is only partially, shared, there one might expect to find the quality of the interaction between persons disturbed. The chapter suggests that this disturbance might explain the impaired quality of interaction between autistic and non-autistic individuals.

This book argues that human perception and perceptual consciousness are richly multisensory. Its thesis is that the coordinated use of multiple senses enhances and extends human perceptual capacities and consciousness in three critical ways. First, crossmodal perceptual illusions reveal hidden multisensory interactions that typically make the senses more coherent and reliable sources of evidence about the environment. Second, the joint use of multiple senses discloses more of the world, including novel features and qualities, making possible new forms of perceptual experience. Third, through crossmodal dependence, plasticity, and perceptual learning, each sense is reshaped by the influence of others, at a time and over time. The implication is that no sense—not even vision itself—can be understood entirely in isolation from the others. This undermines the prevailing approach to perception, which proceeds sense by sense, and sets the stage for a revisionist multisensory approach that illuminates the nature, scope, and character of sense perception.

AbstractMetaverse can fully satisfy users’ design scenarios, meeting their emotional and functional imaginations without any physical constraints, and its superior functionality makes it particularly suitable for the design domain. However, because the interactivity that the Metaverse can provide has not been fully exploited, its application in the design domain has not been studied. In this paper, we propose an interactive design system for the metaverse to enhance human interaction with the metaverse and to realize the interconnection between digital and real spaces. The system is expected to integrate the physical content of the design (including product components) with the designer’s expertise. To this end, we modeled the product design process and its associated design knowledge, which is key to realizing the system. The designer’s expertise was extracted and modeled as multiple perceptual-cognitive models. A fuzzy transformer method was innovatively developed for the computational modeling of the perceptual cognitive models. Using these models, user-system interactions and interactions between virtual and real products were enhanced. This work provides a conceptual framework for a Metaverse interaction design system based on computational modeling of human perceptual cognition. The proposed system greatly extends the scope of Metaverse applications for the development of various product design systems.

AbstractReal-time auralization is essential in virtual reality (VR), gaming, and architecture to enable an immersive audio-visual experience. The audio rendering must be congruent with visual feedback and respond with minimal delay to interactive events and user motion. The wave nature of sound poses critical challenges for plausible and immersive rendering and leads to enormous computational costs. These costs have only increased as virtual scenes have progressed away from enclosures toward complex, city-scale scenes that mix indoor and outdoor areas. However, hard real-time constraints must be obeyed while supporting numerous dynamic sound sources, frequently within a tightly limited computational budget. In this chapter, we provide a general overview of VR auralization systems and approaches that allow them to meet such stringent requirements. We focus on the mathematical foundation, perceptual considerations, and application-specific design requirements of practical systems today, and the future challenges that remain.

AbstractSonic experiences are usually considered as the result of auditory feedback alone. From a psychological standpoint, however, this is true only when a listener is kept isolated from concurrent stimuli targeting the other senses. Such stimuli, in fact, may either interfere with the sonic experience if they distract the listener, or conversely enhance it if they convey sensations coherent with what is being heard. This chapter is concerned with haptic augmentations having effects on auditory perception, for example how different vibrotactile cues provided by an electronic musical instrument may affect its perceived sound quality or the playing experience. Results from different experiments are reviewed showing that the auditory and somatosensory channels together can produce constructive effects resulting in measurable perceptual enhancement. That may affect sonic dimensions ranging from basic auditory parameters, such as the perceived intensity of frequency components, up to more complex perceptions which contribute to forming our ecology of everyday or musical sounds.

AbstractThis chapter addresses the first building block of sonic interactions in virtual environments, i.e., the modeling and synthesis of sound sources. Our main focus is on procedural approaches, which strive to gain recognition in commercial applications and in the overall sound design workflow, firmly grounded in the use of samples and event-based logics. Special emphasis is placed on physics-based sound synthesis methods and their potential for improved interactivity. The chapter starts with a discussion of the categories, functions, and affordances of sounds that we listen to and interact with in real and virtual environments. We then address perceptual and cognitive aspects, with the aim of emphasizing the relevance of sound source modeling with respect to the senses of presence and embodiment of a user in a virtual environment. Next, procedural approaches are presented and compared to sample-based approaches, in terms of models, methods, and computational costs. Finally, we analyze the state of the art in current uses of these approaches for Virtual Reality applications.

Suppose we are good tennis players and want to learn to play ping-pong. Does the way we play tennis affect how we play ping-pong? Would we play ping-pong in the same way if we were not tennis experts? This was one of Albert’s recurring metaphors when drawing a line of thought toward language interactions in bilingual language processing. The argument behind the anecdote referred to what extent the sustained interaction between bilinguals’ two languages results in structural changes within the language network. This chapter aims to push the tennis metaphor one step further by asking whether playing tennis affects how we play football, a sport involving quite different skills. Bringing the sports metaphor into language, this chapter reviews interactions occurring between bilinguals’ two languages involving different articulatory and perceptual mechanisms, such as sign and oral languages. This chapter is then devoted to bimodal bilingualism, reviewing the most relevant results on cross-linguistic interactions across modalities.

AbstractThis chapter concerns concepts of adaption in a binaural audio context (i.e. headphone-based three-dimensional audio rendering and associated spatial hearing aspects), considering first the adaptation of the rendering system to the acoustic and perceptual properties of the user, and second the adaptation of the user to the rendering quality of the system. We start with an overview of the basic mechanisms of human sound source localisation, introducing expressions such as localisation cues and interaural differences, and the concept of the Head-Related Transfer Function (HRTF), which is the basis of most 3D spatialisation systems in VR. The chapter then moves to more complex concepts and processes, such as HRTF selection (system-to-user adaptation) and HRTF accommodation (user-to-system adaptation). State-of-the-art HRTF modelling and selection methods are presented, looking at various approaches and at how these have been evaluated. Similarly, the process of HRTF accommodation is detailed, with a case study employed as an example. Finally, the potential of these two approaches are discussed, considering their combined use in a practical context, as well as introducing a few open challenges for future research.

AbstractThe relationships between the listener, physical world, and virtual environment (VE) should not only inspire the design of natural multimodal interfaces but should be discovered to make sense of the mediating action of VR technologies. This chapter aims to transform an archipelago of studies related to sonic interactions in virtual environments (SIVE) into a research field equipped with a first theoretical framework with an inclusive vision of the challenges to come: the egocentric perspective of the auditory digital twin. In a VE with immersive audio technologies implemented, the role of VR simulations must be enacted by a participatory exploration of sense-making in a network of human and non-human agents, called actors. The guardian of such locus of agency is the auditory digital twin that fosters intra-actions between humans and technology, dynamically and fluidly redefining all those configurations that are crucial for an immersive and coherent experience. The idea of entanglement theory is here mainly declined in an egocentric spatial perspective related to emerging knowledge of the listener’s perceptual capabilities. This is an actively transformative relation with the digital twin potentials to create movement, transparency, and provocative activities in VEs. The chapter contains an original theoretical perspective complemented by several bibliographical references and links to the other book chapters that have contributed significantly to the proposal presented here.

Sonification is an increasingly popular mechanism for data exploration, promoting the need for a greater understanding of human auditory perception and of how sonified information is designed, presented, and interpreted. In this paper, perceptual modelling is used to explore and demonstrate how perceptual phenomena are accounted for in sonification design. The framework, extracted from a larger body of work, links perceptual phenomena such as stream segregation to sonification mappings to provide a systematic approach to identifying and addressing perceptually-driven problems in applied sonification. A web guide functions to situate and guide designers through the complex theoretical constituents of auditory perception incorporated in the Perceptually Congruent Sonification (PerCS) framework. This web guide (hosted on sonification.ie) highlights and summarises the perceptual phenomena most relevant to sonification design and uses simple audio-visual interactions to demonstrate their effect. Preliminary qualitative feedback from a brief survey elucidates a small number of end-user concerns and comments.

A number of researchers have reported that patterns have lower apparent contrast when surrounded by high-physical-contrast patterns with similar spatial frequency properties than when surrounded by lower-contrast patterns. This contrast contrast has been interpreted as revealing lateral interactions among neural gain signals. We report several cases where the phenomena seem to be closely connected to figural properties not captured by the neural interaction account. The test pattern’s apparent contrast can also be described in terms of the maximum and minimum brightnesses of it elements. We find that uniform bright patches are brighter and dim patches are dimmer on low contrast surrounds that on high contrasts; this is hard to reconcile with patternspecific neural interactions.

Abstract In this paper, we present a study to explore the symmetry of kinesthetic perception. Our goal is to add to the growing literature that investigates haptics technologies for therapeutic and rehabilitative applications. To this end, we study how selective activation/ deactivation of haptic (specifically force) feedback affects human perception during symmetric bi-manual (two-handed) spatial tasks. We conducted a simple experiment where healthy individuals are tasked with stretching a virtual spring using two symmetrically located haptics devices that provide an equal amount of resistive forces on each hand while pulling the spring. In this experiment, we implement four kinesthetic conditions, namely (1) feedback on both hands, (2) feedback only on dominant hand, (3) feedback only on non-dominant hand, and (4) no feedback as our control. Our first goal was to determine if there exists a range of spring stiffness in which the individual incorrectly perceives bi-manual forces when the feedback is deactivated on one hand. Subsequently, we also wanted to investigate what range of spring stiffness would lead to such perceptual illusions. Our studies show that not only does such a range exist, wide enough so as to be potentially utilized in future rehabilitative applications. Interestingly, we also observe that for few cases, symmetry can be independent of the kinesthetic perception.