Tesi sul tema "Stereoscopic"

It is well known that stereoscopic images and video can be used to simulate the natural process of stereopsis within the Human Visual System (HVS), by providing the two stereo images/video-streams separately to the two eyes. However as compared to presenting traditional two-dimensional images/video to the HVS, providing stereoscopic information requires double the resources, in the form of transmission bandwidth and/or storage space. Thus to handle this excess data effectively, data compression techniques are required, which is the main focus of the research presented in this thesis. The thesis proposes two novel stereoscopic video CODECs, based on the latest video coding standard, H.264.

In this day and age when 3D cinema is ever increasing in popularity and 3D television sets on the verge of becoming a reality in our homes, the media production companies are interested in looking at how they can start producing stereoscopic format media for this exciting medium. This thesis proposes a solution to parts of the stereoscopic production pipeline by examining and proposing the extent of the requirements of a plug-in to aid in the setting up and rendering of stereo pairs and ultimately proposing such a plug-in and modifying it to suit the proposed requirements. With a high emphasis on ensuring that the plug-in is capable of producing enjoyable stereoscopic content, the plug-in is tested by rendering a demanding scene previously used in factual production work. The findings of these tests ultimately lead to the evaluation of the plug-ins usefulness as a tool for not only the production studio involved but for anyone interested in generating material for this exciting medium.

The evaluation of stereoscopic displays for surface-based renderings is well established in terms of accurate depth perception and tasks that require an understanding of the spatial layout of the scene. In comparison direct volume rendering (DVR) that typically produces images with a high number of low opacity, overlapping features is only beginning to be critically studied on stereoscopic displays. The properties of the specific images and the choice of parameters for DVR algorithms make assessing the effectiveness of stereoscopic displays for DVR particularly challenging and as a result existing literature is sparse with inconclusive results. In this thesis stereoscopic volume rendering is analysed for tasks that require depth perception including: stereo-acuity tasks, spatial search tasks and observer preference ratings. The evaluations focus on aspects of the DVR rendering pipeline and assess how the parameters of volume resolution, reconstruction filter and transfer function may alter task performance and the perceived quality of the produced images. The results of the evaluations suggest that the transfer function and choice of recon- struction filter can have an effect on the performance on tasks with stereoscopic displays when all other parameters are kept consistent. Further, these were found to affect the sensitivity and bias response of the participants. The studies also show that properties of the reconstruction filters such as post-aliasing and smoothing do not correlate well with either task performance or quality ratings. Included in the contributions are guidelines and recommendations on the choice of pa- rameters for increased task performance and quality scores as well as image based methods of analysing stereoscopic DVR images.

The polarization of light was introduced last ten years ago in the field of imaging system is a physical phenomenon that can be controlled for the purposes of the vision system. As that found in the human eyes, in general the imaging sensors are not under construction which is sensitive to the polarization of light. These properties can be measured by adding optical components on a conventional camera. The purpose of this thesis is to develop an imaging system that is sensitive both to the stereoscopic and to the state of polarization. As well as the visual system on a various of insects in nature such as bees, that are have capability to move in space by extracted relevant information from the polarization. The developed prototype should be possible to reconstruct threedimensional of points of interest with the issues associated with a set of parameters of the state of polarization. The proposed system consists of two cameras, each camera equipped with liquid crystal components to obtain two images with different directions of polarization. For each acquisition, four images are acquired: two for each camera. Raised by the key of main capability to return polarization information from two different cameras. After an initial calibration step; geometric and photometric, the mapping of points of interest process is made difficult because of the optical components placed in front of different lenses. A detailed study of different methods of mapping was used to select sensitivity to the polarization effects. Once points are mapped, the polarization parameters of each point are calculated from the four values from four images acquired. The results on real scenes show the feasibility and desirability of this imaging system for robotic applications.

This thesis addresses the problem of calibrating a stereoscopic camera with a minimum of necessary post-processing. This is achieved through a two step procedure, the first step of which is a calibration of the sensors in rotation by means of laser diffraction, without attached lenses. The second step involves attaching the lenses and using a simplified conventional image-based calibration to determine the effects of motions of the optical centres due to lens focusing. Mounting considerations and long-term stability are also addressed. This method enables the construction of a stereoscopic camera which requires no interpolative rectification, with the calibration maintaining accuracy over a range of focal distances. Such a camera is built and calibrated, and tested to demonstrate the validity of the predicted error estimates. This approach is shown to be effective in producing stereoscopic images for display which meet the requirements of the human visual system. A comparison of this approach with previously published methods is presented. Some or all of the techniques described in this thesis may be incorporated into existing calibration schemes to improve the quality of the produced stereoscopic images. The improvements provided by a hardware calibration as described may be especially valuable in applications where maintaining full sensor resolution in the displayed image is desired.

Crosstalk is an important image quality attribute of stereoscopic 3D displays. The research presented in this thesis examines the presence, mechanisms, simulation, and reduction of crosstalk for a selection of stereoscopic display technologies. High levels of crosstalk degrade the perceived quality of stereoscopic displays hence it is important to minimise crosstalk. This thesis provides new insights which are critical to a detailed understanding of crosstalk and consequently to the development of effective crosstalk reduction techniques.

Past research on stereoscopic depth perception among the elderly has led to inconsistent findings. Some research on stereopsis and aging has found that younger and older adults are essentially the same in terms of their stereoscopic ability, while other research has found evidence of large differences. This past research has largely been limited to investigations of stereoacuity. The purpose of Experiment 1 was to extend this earlier research to compare how older and younger observers perceive the magnitude of stereoscopically defined depth intervals. Random-dot stereograms depicting sinusoidal surfaces were shown to seven younger (i.e., ages 30 and below) and six older (i.e., ages 60 and above) adults. These surfaces were defined by three levels of peak-trough image disparity, two spatial frequencies, and two densities of texture elements. The observers' task was to estimate the magnitude of the depth interval between the surfaces' peaks and troughs. It was found that the perceived depth intervals of the younger observers were closer to those predicted by the geometry of stereopsis: as disparity increased, so did the magnitudes of their perceived depth intervals. This finding was also true for the five out of the six older adults, but the magnitudes of their perceived depth intervals were less than their younger counterparts. The high frequency surfaces were more difficult to perceive for both groups, but were especially difficult for the elderly. In contrast, texture element density had essentially no effect upon the observers' performance for both groups. The results of this experiment showed that the elderly have a significant amount of stereoscopic functionality that is not qualitatively different from younger adults. Experiment 2 was designed to compare older and younger observers' ability to perceive the shape of stereoscopic surfaces. In this experiment, four different surfaces defined by disparity (i.e., bumps, saddles, vertical cylinders, and horizontal cylinders) were shown to five younger (i.e., 30 and below) and five older (i.e., 60 and above) observers. The random-dot stereograms varied in terms of their texture element density and amount of correspondence. The results showed that the older observers were less sensitive to stereoscopic depth and curvature. In all other respects, however, the results for the older observers were essentially identical to those of the younger observers. In particular, the reductions in density and correspondence led to nearly identical declines in performance for both age groups. In summary, the results of both experiments showed that, despite some reductions in perceptual sensitivity, older adults can effectively perceive and discriminate the shape and depth of stereoscopic surfaces.

Metrics for automatically predicting the compression settings for stereoscopic images, to minimize file size, while still maintaining an acceptable level of image quality are investigated. This research evaluates whether symmetric or asymmetric compression produces a better quality of stereoscopic image. Initially, how Peak Signal to Noise Ratio (PSNR) measures the quality of varyingly compressed stereoscopic image pairs was investigated. Two trials with human subjects, following the ITU-R BT.500-11 Double Stimulus Continuous Quality Scale (DSCQS) were undertaken to measure the quality of symmetric and asymmetric stereoscopic image compression. Computational models of the Human Visual System (HVS) were then investigated and a new stereoscopic image quality metric designed and implemented. The metric point matches regions of high spatial frequency between the left and right views of the stereo pair and accounts for HVS sensitivity to contrast and luminance changes in these regions. The PSNR results show that symmetric, as opposed to asymmetric stereo image compression, produces significantly better results. The human factors trial suggested that in general, symmetric compression of stereoscopic images should be used. The new metric, Stereo Band Limited Contrast, has been demonstrated as a better predictor of human image quality preference than PSNR and can be used to predict a perceptual threshold level for stereoscopic image compression. The threshold is the maximum compression that can be applied without the perceived image quality being altered. Overall, it is concluded that, symmetric, as opposed to asymmetric stereo image encoding, should be used for stereoscopic image compression. As PSNR measures of image quality are correctly criticized for correlating poorly with perceived visual quality, the new HVS based metric was developed. This metric produces a useful threshold to provide a practical starting point to decide the level of compression to use.

Many animals are able to perceive stereoscopic depth owing to the disparity information that arises from the left and right eyes' horizontal displacement on the head. The initial computation of disparity happens in primary visual cortex (V1) and is largely considered to be a correlation-based computation. In other words, the computational role of V1 as it pertains to stereoscopic vision can be seen to roughly perform a binocular cross-correlation between the images of the left and right eyes. This view is based on the unique success of a correlation-based model of disparity-selective cells { the binocular energy model (BEM). This thesis addresses two unresolved challenges to this narrative. First, recent evidence suggests that a correlation-based view of primary visual cortex is unable to account for human perception of depth in a stimulus where the binocular correlation is on average zero. Chapters 1 and 2 show how a simple extension of the BEM which better captures key properties of V1 neurons allows model cells to signal depth in such stimuli. We also build a psychophysical model which captures human performance closely, and recording from V1 in the macaque, we then show that these predicted properties are indeed observed in real V1 neurons. The second challenge relates to the long-standing inability of the BEM to capture responses to anticorrelated stimuli: stimuli where the contrast is reversed in the two eyes (e.g. black features in the left eye are matched with identical white features in the right eye). Real neurons respond less strongly to these stimuli than model cells. In Chapter 3 and 4, we make use of recent advances in optimisation routines and exhaustively test the ability of a generalised BEM to capture this property. We show that even the best- tting generalised BEM units only go some way towards describing neuronal responses. This is the rst exhaustive empirical test of this in uential modelling framework, and we speculate on what is needed to develop a more complete computational account of visual processing in primary visual cortex.

Two modern technologies show promise to dramatically increase immersion in virtual environments. Stereoscopic imaging captures two images representing the views of both eyes and allows for better depth perception. High dynamic range (HDR) imaging accurately represents real world lighting as opposed to traditional low dynamic range (LDR) imaging. HDR provides a better contrast and more natural looking scenes. The combination of the two technologies in order to gain advantages of both has been, until now, mostly unexplored due to the current limitations in the imaging pipeline. This thesis reviews both fields, proposes stereoscopic high dynamic range (SHDR) imaging pipeline outlining the challenges that need to be resolved to enable SHDR and focuses on capture and compression aspects of that pipeline. The problems of capturing SHDR images that would potentially require two HDR cameras and introduce ghosting, are mitigated by capturing an HDR and LDR pair and using it to generate SHDR images. A detailed user study compared four different methods of generating SHDR images. Results demonstrated that one of the methods may produce images perceptually indistinguishable from the ground truth. Insights obtained while developing static image operators guided the design of SHDR video techniques. Three methods for generating SHDR video from an HDR-LDR video pair are proposed and compared to the ground truth SHDR videos. Results showed little overall error and identified a method with the least error. Once captured, SHDR content needs to be efficiently compressed. Five SHDR compression methods that are backward compatible are presented. The proposed methods can encode SHDR content to little more than that of a traditional single LDR image (18% larger for one method) and the backward compatibility property encourages early adoption of the format. The work presented in this thesis has introduced and advanced capture and compression methods for the adoption of SHDR imaging. In general, this research paves the way for a novel field of SHDR imaging which should lead to improved and more realistic representation of captured scenes.

--Context-- This project looked at the development of a stereoscopic 3D environment in which a user is able to draw freely in all three dimensions. The main focus was on the storage and manipulation of the ‘digital ink’ with which the user draws. For a drawing and sketching package to be effective it must not only have an easy to use user interface, it must be able to handle all input data quickly and efficiently so that the user is able to focus fully on their drawing. --Background-- When it comes to sketching in three dimensions the majority of applications currently available rely on vector based drawing methods. This is primarily because the applications are designed to take a users two dimensional input and transform this into a three dimensional model. Having the sketch represented as vectors makes it simpler for the program to act upon its geometry and thus convert it to a model. There are a number of methods to achieve this aim including Gesture Based Modelling, Reconstruction and Blobby Inflation. Other vector based applications focus on the creation of curves allowing the user to draw within or on existing 3D models. They also allow the user to create wire frame type models. These stroke based applications bring the user closer to traditional sketching rather than the more structured modelling methods detailed. While at present the field is inundated with vector based applications mainly focused upon sketch-based modelling there are significantly less voxel based applications. The majority of these applications focus on the deformation and sculpting of voxmaps, almost the opposite of drawing and sketching, and the creation of three dimensional voxmaps from standard two dimensional pixmaps. How to actually sketch freely within a scene represented by a voxmap has rarely been explored. This comes as a surprise when so many of the standard 2D drawing programs in use today are pixel based. --Method-- As part of this project a simple three dimensional drawing program was designed and implemented using C and C++. This tool is known as Sketch3D and was created using a Model View Controller (MVC) architecture. Due to the modular nature of Sketch3Ds system architecture it is possible to plug a range of different data structures into the program to represent the ink in a variety of ways. A series of data structures have been implemented and were tested for efficiency. These structures were a simple list, a 3D array, and an octree. They have been tested for: the time it takes to insert or remove points from the structure; how easy it is to manipulate points once they are stored; and also how the number of points stored effects the draw and rendering times. One of the key issues brought up by this project was devising a means by which a user is able to draw in three dimensions while using only two dimensional input devices. The method settled upon and implemented involves using the mouse or a digital pen to sketch as one would in a standard 2D drawing package but also linking the up and down keyboard keys to the current depth. This allows the user to move in and out of the scene as they draw. A couple of user interface tools were also developed to assist the user. A 3D cursor was implemented and also a toggle, which when on, highlights all of the points intersecting the depth plane on which the cursor currently resides. These tools allow the user to see exactly where they are drawing in relation to previously drawn lines. --Results-- The tests conducted on the data structures clearly revealed that the octree was the most effective data structure. While not the most efficient in every area, it manages to avoid the major pitfalls of the other structures. The list was extremely quick to render and draw to the screen but suffered severely when it comes to finding and manipulating points already stored. In contrast the three dimensional array was able to erase or manipulate points effectively while the draw time rendered the structure effectively useless, taking huge amounts of time to draw each frame. The focus of this research was on how a 3D sketching package would go about storing and accessing the digital ink. This is just a basis for further research in this area and many issues touched upon in this paper will require a more in depth analysis. The primary area of this future research would be the creation of an effective user interface and the introduction of regular sketching package features such as the saving and loading of images.

We apply the notion that phase differences can be used to interpret disparity between a pair of stereoscopic images. Indeed, phase relationships can also be used to obtain orientation and probabilistic measures from both edges and comers, as well as the directional instantaneous frequency of an image field. The method of phase differences is shown to be equivalent to a Newton-Raphson root finding iteration through the resolutions of band-pass filtering. The method does, however, suffer from stability problems, and in particular stationary phase. The stability problems associated with this technique are implicitly derived from the mechanism used to interpet disparity, which in general requires an assumption of linear phase and the local instantaneous frequency. We present two techniques. Firstly, we use the centre frequency of the applied band-pass filter to interpret disparity. This interpretation, however, suffers heavily from phase error and requires considerable damping prior to convergence. Secondly, we use the derivative of phase to obtain the instantaneous frequency from an image, which is then used to improve the disparity estimate. The second measure is implicitly sensitive to regions that exhibit stationary phase. We prove that stationary phase is a form of aliasing. To maintain stability with this technique, it is essential to smooth the disparity signal at each resolution of filtering. These ideas are extended into 2-D where it is possible to extract both vertical and horizontal disparities. Unfortunately, extension into 2-D also introduces a similar form of the motion aperture problem. The best image regions to disambiguate both horizontal and vertical disparities lie in the presence of comers. Fortunately, we introduce a measure for identifying orthogonal image signals based upon the same filters that we use to interpret disparity. We find that in the presence of dominant edge energy, there is an error in horizontal disparity interpretation that varies as a cosine function. This error can be reduced by iteration or resolving the horizontal component of the disparity signal. These ideas are also applied towards the computation of deformation, which is related to the magnitude and direction of surface slant. This is a natural application to the ideas presented in this thesis.

This thesis explores and includes research on different impressions given in stereo rendering when creating a fire in three different manners.As technology progresses, stereoscopic rendering becomes increasingly more popular. Both movies and games are shown through VR to give users a more satisfying and lifelike interaction with a virtual environment.Three different versions of a fire were created in a game engine and then viewed with HTC Vive. Comparing mono and stereo rendering gave insights on how the different method looks when using stereoscopic rendering, and what impression the different methods gave the user.The different fires were tested with several users. Finally, this thesis concludes with an interview to one of the users and his different impressions of the fires.
Denna rapport undersöker och innehåller forskning om olika uppfattningar som uppkommer i stereo rendering när eld skapas på tre olika metoder.Allteftersom tekniken utvecklas blir stereo rendering allt mer populär. Både filmer och spel visas i VR för att ge användaren en mer tillfredsställande och verklighetstrogen upplevelse i en virtuell miljö.Tre olika versioner av eld skapades i en spelmotor och granskades sedan via en HTC Vive. Jämförandet av mono och stereo rendering gav en insikt i hur de olika metoderna uppvisas med användning av stereo rendering, och vilka uppfattningar de olika metoderna gav användaren.De olika eldarna testades av flera användare. Tillslut, avslutades denna tes med en intervju med en av användarna och hans uppfattningar av eldarna.

This thesis analyzes the problem of acquiring stereoscopic images in all gazing directions around a reference viewpoint in space with the purpose of creating stereoscopic panoramas of non-static scenes. The generation of immersive stereoscopic imagery suitable to stimulate human stereopsis requires images from two distinct viewpoints with horizontal parallax in all gazing directions, or to be able to simulate this situation in the generated imagery. The available techniques to produce omnistereoscopic imagery for human viewing are not suitable to capture dynamic scenes stereoscopically. This is a not trivial problem when considering acquiring the entire scene at once while avoiding self-occlusion between multiple cameras. In this thesis, the term omnidirectional refers to all possible gazing directions in azimuth and a limited set of directions in elevation. The acquisition of dynamic scenes restricts the problem to those techniques suitable for collecting in one simultaneous exposure all the necessary visual information to recreate stereoscopic imagery in arbitrary gazing directions. The analysis of the problem starts by defining an omnistereoscopic viewing model for the physical magnitude to be measured by a panoramic image sensor intended to produce stereoscopic imagery for human viewing. Based on this model, a novel acquisition model is proposed, which is suitable to describe the omnistereoscopic techniques based on horizontal stereo. From this acquisition model, an acquisition method based on multiple cameras combined with the rendering by mosaicking of partially overlapped stereoscopic images is identified as a good candidate to produce omnistereoscopic imagery of dynamic scenes. Experimental acquisition and rendering tests were performed for different multiple-camera configurations. Furthermore, a mosaicking criterion between partially overlapped stereoscopic images based on the continuity of the perceived depth and the prediction of the location and magnitude of unwanted vertical disparities in the final stereoscopic panorama are two main contributions of this thesis. In addition, two novel omnistereoscopic acquisition and rendering techniques were introduced. The main contributions to this field are to propose a general model for the acquisition of omnistereoscopic imagery, to devise novel methods to produce omnistereoscopic imagery, and more importantly, to contribute to the awareness of the problem of acquiring dynamic scenes within the scope of omnistereoscopic research.

After considering the growing demand for realistic video reproduction, in this thesis, we have proposed two stereoscopic video coding schemes that are evaluated for the suitability for 3D mobile applications. One is an H.264 based stereoscopic video codec and the other is a Distributed Video Coding (DVC) based stereoscopic codec. During the first half of the thesis, we propose an H.264 based stereoscopic codec that includes a modification to handle two video streams in order to exploit disparity and worldline correlations as well as a design of a novel motion/disparity vector prediction algorithm that operates more intelligently using the previously coded motion and disparity vector information of the surrounding macroblocks. Simulation results show th~t the proposed H.264 based stereoscopic video codec performs better than MPEG-2 based and ZTE based stereoscopic codecs. The results have also been compared with simulcast H.264 and later proposed MVC (Multiview Video Coding) standard. The former comparison demonstrates a PSNR gain of up to about 2dB, whereas the latter demonstrates comparable results. In the second half of the thesis, we have designed several algorithms to improve the coding efficiency of Wyner-Ziv frames and a novel algorithm for key frame coding in DVC. These algorithms include a spatial and temporal correlation exploitation algorithm for side information generation, a multiple side information handling algorithm and a bitplane by bitplane refinement technique in addition to the mentioned DVC based key frame coding scheme. In the proposed algorithms, we exploit the decoded information very efficiently by taking the advantage of decoded information together with the information available in the decoder. Simulation results clearly show that these algorithms outperform the existing DVC codecs by significant margins. With these algorithms as the backbone, we have then introduced a novel Stereoscopic Distributed Video Coding (SDVC) scheme. Simulation results of the proposed SDVC algorithm demonstrate that it performs better than MPEG-2 based and ZTE based benchmark stereoscopic codecs by up to 3dB and 2dB respectively in terms of objective quality gains. Enally this thesis, presents a W-CDMA channel simulation for both proposed stereoscopic codecs and the results suggest that, SDVC is a viable solution for stereoscopic video coding in mobile applications that comprises robust characteristics against channel noise together with the added advantage oflow encoding complexity.

Despite the recent explosion of interest in the stereoscopic 3D (S3D) technology, the ultimate prevailing of the S3D medium is still significantly hindered by adverse effects regarding the S3D viewing discomfort. This thesis attempts to improve the S3D viewing experience by investigating perceived depth control methods in stereoscopic cinematography on desktop 3D displays. The main contributions of this work are: (1) A new method was developed to carry out human factors studies on identifying the practical limits of the 3D Comfort Zone on a given 3D display. Our results suggest that it is necessary for cinematographers to identify the specific limits of 3D Comfort Zone on the target 3D display as different 3D systems have different ranges for the 3D Comfort Zone. (2) A new dynamic depth mapping approach was proposed to improve the depth perception in stereoscopic cinematography. The results of a human-based experiment confirmed its advantages in controlling the perceived depth in viewing 3D motion pictures over the existing depth mapping methods. (3) The practicability of employing the Depth of Field (DoF) blur technique in S3D was also investigated. Our results indicate that applying the DoF blur simulation on stereoscopic content may not improve the S3D viewing experience without the real time information about what the viewer is looking at. Finally, a basic guideline for stereoscopic cinematography was introduced to summarise the new findings of this thesis alongside several well-known key factors in 3D cinematography. It is our assumption that this guideline will be of particular interest not only to 3D filmmaking but also to 3D gaming, sports broadcasting, and TV production.

This thesis presents a novel investigation into the effect stereoscopic vision has upon the strength of perceived gloss on rough surfaces. We demonstrate that in certain cases disparity is necessary for accurate judgements of gloss strength. We first detail the process we used to create a two-level taxonomy of property terms, which helped to inform the early direction of this work, before presenting the eleven words which we found categorised the property space. This shaped careful examination of the relevant literature, leading us to conclude that most studies into roughness, gloss, and stereoscopic vision have been performed with unrealistic surfaces and physically inaccurate lighting models. To improve on the stimuli used in these earlier studies, advanced offline rendering techniques were employed to create images of complex, naturalistic, and realistically glossy 1/fβ noise surfaces. These images were rendered using multi-bounce path tracing to account for interreflections and soft shadows, with a reflectance model which observed all common light phenomena. Using these images in a series of psychophysical experiments, we first show that random phase spectra can alter the strength of perceived gloss. These results are presented alongside pairs of the surfaces tested which have similar levels of perceptual gloss. These surface pairs are then used to conclude that naïve observers consistently underestimate how glossy a surface is without the correct surface and highlight disparity, but only on the rougher surfaces presented.

Minimally invasive surgery has been a major advance in the practice of medicine as it reduces the morbidity associated with larger incisions required for open surgery. A videoscopic system is used to capture and transmit two-dimensional images of the patient during a procedure. In open surgery, the binocular configuration of the human visual system is used to generate key depth information. Minimally invasive surgery requires interpretation of monocular visual cues to perform visuospatial judgments and complex psychomotor skills. The absence of binocular depth cues extends the learning curve during which there is an increased risk of surgical error. Stereoendoscopes produce binocular visual cues by presenting horizontally disparate images of the operative field to each eye. Stereoscopic surgery is associated with improvements in surgical performance but historical projection mechanisms generated intolerable viewing conditions resulting in visual fatigue. Time-parallel passive polarising stereoscopic displays use polarising filters to simultaneously designate alternate pixel rows of horizontally disparate images. Circular polarising eyewear corresponding to the display surface filters allows disparate images to be viewed separately by each eye. The difference between these images is interpreted as a binocular depth cue. This thesis aims to identify the potential impact and tolerance of time-parallel passive polarising stereoscopic displays for minimally invasive surgery. Accommodative dynamic responses were used to objectively measure visual fatigue following stereoscopic viewing. Visual perception of stereoscopic stimuli was investigated by psychophysical performance during visual search and by quantifying attention deployment while viewing stereoscopic surgery. This work provides insight into the future role of stereoscopic visualisation for minimally invasive surgery. It indicates that time-parallel passive polarising displays improve performance of surgical skills and are well tolerated by experienced minimally invasive surgeons under stereoscopic conditions. Novice surgeons may experience increased visual fatigue while learning minimally invasive surgery due to disturbance of normal visual attention mechanisms. This thesis forms the basis for future clinical trials to evaluate the impact of this technology on the performance of minimally invasive surgery.

Aquesta tesi s'emmarca dins del projecte CICYT TAP 1999-0443-C05-01. L'objectiu d'aquest projecte és el disseny, implementació i avaluació de robots mòbils, amb un sistema de control distribuït, sistemes de sensorització i xarxa de comunicacions per realitzar tasques de vigilància. Els robots han de poder-se moure per un entorn reconeixent la posició i orientació dels diferents objectes que l'envolten. Aquesta informació ha de permetre al robot localitzar-se dins de l'entorn on es troba per poder-se moure evitant els possibles obstacles i dur a terme la tasca encomanada. El robot ha de generar un mapa dinàmic de l'entorn que serà utilitzat per localitzar la seva posició. L'objectiu principal d'aquest projecte és aconseguir que un robot explori i construeixi un mapa de l'entorn sense la necessitat de modificar el propi entorn.
Aquesta tesi està enfocada en l'estudi de la geometria dels sistemes de visió estereoscòpics formats per dues càmeres amb l'objectiu d'obtenir informació geomètrica 3D de l'entorn d'un vehicle. Aquest objectiu tracta de l'estudi del modelatge i la calibració de càmeres i en la comprensió de la geometria epipolar. Aquesta geometria està continguda en el que s'anomena emph{matriu fonamental}. Cal realitzar un estudi del càlcul de la matriu fonamental d'un sistema estereoscòpic amb la finalitat de reduir el problema de la correspondència entre dos plans imatge. Un altre objectiu és estudiar els mètodes d'estimació del moviment basats en la geometria epipolar diferencial per tal de percebre el moviment del robot i obtenir-ne la posició. Els estudis de la geometria que envolta els sistemes de visió estereoscòpics ens permeten presentar un sistema de visió per computador muntat en un robot mòbil que navega en un entorn desconegut. El sistema fa que el robot sigui capaç de generar un mapa dinàmic de l'entorn a mesura que es desplaça i determinar quin ha estat el moviment del robot per tal de emph{localitzar-se} dins del mapa.
La tesi presenta un estudi comparatiu dels mètodes de calibració de càmeres més utilitzats en les últimes dècades. Aquestes tècniques cobreixen un gran ventall dels mètodes de calibració clàssics. Aquest mètodes permeten estimar els paràmetres de la càmera a partir d'un conjunt de punts 3D i de les seves corresponents projeccions 2D en una imatge. Per tant, aquest estudi descriu un total de cinc tècniques de calibració diferents que inclouen la calibració implicita respecte l'explicita i calibració lineal respecte no lineal. Cal remarcar que s'ha fet un gran esforç en utilitzar la mateixa nomenclatura i s'ha estandaritzat la notació en totes les tècniques presentades. Aquesta és una de les dificultats principals a l'hora de poder comparar les tècniques de calibració ja què cada autor defineix diferents sistemes de coordenades i diferents conjunts de paràmetres. El lector és introduït a la calibració de càmeres amb la tècnica lineal i implícita proposada per Hall i amb la tècnica lineal i explicita proposada per Faugeras-Toscani. A continuació es passa a descriure el mètode a de Faugeras incloent el modelatge de la distorsió de les lents de forma radial. Seguidament es descriu el conegut mètode proposat per Tsai, i finalment es realitza una descripció detallada del mètode de calibració proposat per Weng. Tots els mètodes són comparats tant des del punt de vista de model de càmera utilitzat com de la precisió de la calibració. S'han implementat tots aquests mètodes i s'ha analitzat la precisió presentant resultats obtinguts tant utilitzant dades sintètiques com càmeres reals.
Calibrant cada una de les càmeres del sistema estereoscòpic es poden establir un conjunt de restriccions geomètri ques entre les dues imatges. Aquestes relacions són el que s'anomena geometria epipolar i estan contingudes en la matriu fonamental. Coneixent la geometria epipolar es pot: simplificar el problema de la correspondència reduint l'espai de cerca a llarg d'una línia epipolar; estimar el moviment d'una càmera quan aquesta està muntada sobre un robot mòbil per realitzar tasques de seguiment o de navegació; reconstruir una escena per aplicacions d'inspecció, propotipatge o generació de motlles. La matriu fonamental s'estima a partir d'un conjunt de punts en una imatges i les seves correspondències en una segona imatge. La tesi presenta un estat de l'art de les tècniques d'estimació de la matriu fonamental. Comença pels mètode lineals com el dels set punts o el mètode dels vuit punts, passa pels mètodes iteratius com el mètode basat en el gradient o el CFNS, fins arribar las mètodes robustos com el M-Estimators, el LMedS o el RANSAC. En aquest treball es descriuen fins a 15 mètodes amb 19 implementacions diferents. Aquestes tècniques són comparades tant des del punt de vista algorísmic com des del punt de vista de la precisió que obtenen. Es presenten el resultats obtinguts tant amb imatges reals com amb imatges sintètiques amb diferents nivells de soroll i amb diferent quantitat de falses correspondències.
Tradicionalment, l'estimació del moviment d'una càmera està basada en l'aplicació de la geometria epipolar entre cada dues imatges consecutives. No obstant el cas tradicional de la geometria epipolar té algunes limitacions en el cas d'una càmera situada en un robot mòbil. Les diferencies entre dues imatges consecutives són molt petites cosa que provoca inexactituds en el càlcul de matriu fonamental. A més cal resoldre el problema de la correspondència, aquest procés és molt costós en quant a temps de computació i no és gaire efectiu per aplicacions de temps real. En aquestes circumstàncies les tècniques d'estimació del moviment d'una càmera solen basar-se en el flux òptic i en la geometria epipolar diferencial. En la tesi es realitza un recull de totes aquestes tècniques degudament classificades. Aquests mètodes són descrits unificant la notació emprada i es remarquen les
semblances i les diferencies entre el cas discret i el cas diferencial de la geometria epipolar. Per tal de poder aplicar aquests mètodes a l'estimació de moviment d'un robot mòbil, aquest mètodes generals que estimen el moviment d'una càmera amb sis graus de llibertat, han estat adaptats al cas d'un robot mòbil que es desplaça en una superfície plana. Es presenten els resultats obtinguts tant amb el mètodes generals de sis graus de llibertat com amb els adaptats a un robot mòbil utilitzant dades sintètiques i seqüències d'imatges reals.
Aquest tesi finalitza amb una proposta de sistema de localització i de construcció d'un mapa fent servir un sistema estereoscòpic situat en un robot mòbil. Diverses aplicacions de robòtica mòbil requereixen d'un sistema de localització amb l'objectiu de facilitar la navegació del vehicle i l'execució del les trajectòries planificades. La localització es sempre relativa al mapa de l'entorn on el robot s'està movent. La construcció de mapes en un entorn desconegut és una tasca important a realitzar per les futures generacions de robots mòbils. El sistema que es presenta realitza la localització i construeix el mapa de l'entorn de forma simultània. A la tesi es descriu el robot mòbil GRILL, que ha estat la plataforma de treball emprada per aquesta aplicació, amb el sistema de visió estereoscòpic que s'ha dissenyat i s'ha muntat en el robot. També es descriu tots el processos que intervenen en el sistema de localització i construcció del mapa. La implementació d'aquest processos ha estat possible gràcies als estudis realitzats i presentats prèviament (calibració de càmeres, estimació de la matriu fonamental, i estimació del moviment) sense els quals no s'hauria pogut plantejar aquest sistema. Finalment es presenten els mapes en diverses trajectòries realitzades pel robot GRILL en el laboratori.
Les principals contribucions d'aquest treball són:
·Un estat de l'art sobre mètodes de calibració de càmeres. El mètodes són comparats tan des del punt de vista del model de càmera utilitzat com de la precisió dels mètodes.
·Un estudi dels mètodes d'estimació de la matriu fonamental. Totes les tècniques estudiades són classificades i descrites des d'un punt de vista algorísmic.
·Un recull de les tècniques d'estimació del moviment d'una càmera centrat en el mètodes basat en la geometria epipolar diferencial. Aquestes tècniques han estat adaptades per tal d'estimar el moviment d'un robot mòbil.
·Una aplicació de robòtica mòbil per tal de construir un mapa dinàmic de l'entorn i localitzar-se per mitja d'un sistema estereoscòpic. L'aplicació presentada es descriu tant des del punt de vista del maquinari com del programari que s'ha dissenyat i implementat.
Human eyes have been widely studied by the scientific community so that its operation principle is widely known. Computer vision tries to copy the way human beings perceive visual information by means of using cameras acting as eyeballs and computers aspiring to process this information in an --intelligent way". The complex task of being conscious of reality is obviously divided into a set of simpler problems which covers from image acquisition to scene description. One of the main applications is robot perception in which a mobile robot is equipped with a computer vision system. Robots may be able to navigate around an unknown structured environment acquiring visual information of their surroundings with the aim of estimating the position and orientation of every obstacle. Moreover, the pose of the vehicle has to be estimated as accurate as possible. Hence, the motion of the vehicle might be also computed allowing the localization of the vehicle with respect to the 3D map.
This thesis is focused on the study of the geometry involved in stereo vision systems composed by two cameras with the aim of obtaining 3D geometric information of the vehicle surroundings. This objective deals to the study of camera modelling and calibration and the comprehension of the epipolar geometry. Then, the computation of the fundamental matrix of a stereoscopic system is surveyed with the aim of reducing the correspondence problem between both image planes. An accurate estimation of the fundamental matrix allows us not only to compute 3D information of the vehicle environments, but to validate it. Nevertheless, the traditional case of the epipolar geometry has some limitations in the common case of a single camera attached to a mobile robot. Disparities between two consecutive images are rather small at common image rates leading to numerical inaccuracies on the computation of the fundamental matrix. Then, another objective is the study of general vision-based egomotion estimation methods based on the differential epipolar constraint with the aim of perceiving the robot movement instead of its position.
The study of the geometry involved in stereo vision systems leads us to present a computer vision system mounted on a vehicle which navigates in an unknown environment. Two main tasks are faced: a) the localization of the vehicle; and b) the building of an absolute 3D map.
El sistema de visión humano ha sido ampliamente estudiado por la comunidad científica de forma que su principio de funcionamiento es profundamente conocido. La Visión por Computador trata de copiar la forma que nosotros los humanos percibimos la información visual por medio del uso de cámaras actuando como ojos y un ordenador aspirando a procesar toda la información de "forma inteligente". La compleja tarea de ser consciente de la realidad es obviamente dividida en un conjunto de problemas mucho más simples, los cuales abarcan des de la adquisición de la imagen a la descripción de la escena. Una de las numerosas aplicaciones es la percepción por parte de un robot, donde un robot móvil es equipado con un sistema informático de visión por computador. Estos robots deben ser capaces de navegar a lo largo de un entorno estructurado desconocido mediante la adquisición de información visual de su alrededor, con el objetivo de estimar la posición y orientación de todos los obstáculos. Además, la posición del vehículo debe ser estimada de la forma más precisa posible. De esta forma, el movimiento del vehículo puede ser también calculado lo que permite la localización del vehículo con respeto al mapa 3D.
Esta tesis profundiza en el estudio de la geometría existente en los sistemas de visión estéreo compuestos por dos cámaras con la intención de obtener información geométrica 3D del entorno del vehículo. Este objetivo lleva consigo la necesidad inicial de realizar un estudio de modelado de la cámara y calibración, y la compensación de la geometría epipolar. A continuación, el cálculo de la matriz fundamental de un sistema esteresocópico es analizado para reducir el problema de la correspondencia entre ambos planos de la imagen. Una estimación precisa de la matriz fundamental nos permite no solamente obtener la información 3D del entorno, sino también validar la misma. No obstante, la geometría epipolar tradicional sufre algunas limitaciones en el caso de una cámara montada en un robot móvil. La disparidad entre dos imágenes consecutivas es realmente mínima trabajando a velocidad estándar lo que conlleva a errores numéricos en el cálculo de la matriz fundamental. Por esta razón, otro objetivo es el estudio de los métodos de estimación del movimiento basados en la geometría epipolar diferencial con el objetivo de pervivir el movimiento del robot y su posición.
El estudio de la geometría inmersa en los sistemas de visión estéreo nos lleva a presentar un sistema de visión por computador montado en un vehículo capaz de navegar en un entorno desconocido. Dos tareas básicas son consideradas: a) la localización del vehículo; y b) la construcción de un mapa 3D absoluto.

The aim of this study is to understand the effect of different depth levels on the overall 3D quality and develop an objective video quality metric for stereoscopic video sequences. Proposed method is designed to be used in video coding stages to improve overall 3D video quality. This study includes both objective and subjective evaluation. Test sequences with different coding schemes are used. Computer simulation results show that overall quality has a strong correlation with the quality of the background, where disparity is smaller relative to the foreground. This correlation indicates that background layer is more prone to coding errors. The results also showed that content type is an important factor in determining the visual quality.

3D stereoscopic technology intensifies and heightens the viewer s experience by adding an extra dimension to the viewing of visual content. However, with expansion of this technology to the commercial market concerns have been expressed about the potential negative effects on the visual system, producing viewer discomfort. The visual stimulus provided by a 3D stereoscopic display differs from that of the real world, and so it is important to understand whether these differences may pose a health hazard. The aim of this thesis is to investigate the effect of 3D stereoscopic stimulation on visual discomfort. To that end, four experimental studies were conducted. In the first study two hypotheses were tested. The first hypothesis was that the viewing of 3D stereoscopic stimuli, which are located geometrically beyond the screen on which the images are displayed, would induce adaptation changes in the resting position of the eyes (exophoric heterophoria changes). The second hypothesis was that participants whose heterophoria changed as a consequence of adaptation during the viewing of the stereoscopic stimuli would experience less visual discomfort than those people whose heterophoria did not adapt. In the experiment an increase of visual discomfort change in the 3D condition in comparison with the 2D condition was found. Also, there were statistically significant changes in heterophoria under 3D conditions as compared with 2D conditions. However, there was appreciable variability in the magnitude of this adaptation among individuals, and no correlation between the amount of heterophoria change and visual discomfort change was observed. In the second experiment the two hypotheses tested were based on the vergence-accommodation mismatch theory, and the visual-vestibular mismatch theory. The vergence-accommodation mismatch theory predicts that a greater mismatch between the stimuli to accommodation and to vergence would produce greater symptoms in visual discomfort when viewing in 3D conditions than when viewing in 2D conditions. An increase of visual discomfort change in the 3D condition in comparison with the 2D condition was indeed found; however the magnitude of visual discomfort reported did not correlate with the mismatch present during the watching of 3D stereoscopic stimuli. The visual-vestibular mismatch theory predicts that viewing a stimulus stereoscopically will produce a greater sense of vection than viewing it in 2D. This will increase the conflict between the signals from the visual and vestibular systems, producing greater VIMS (Visually- Induced Motion Sickness) symptoms. Participants did indeed report an increase in motion sickness symptoms in the 3D condition. Furthermore, participants with closer seating positions reported more VIMS than participants sitting farther away whilst viewing 3D stimuli. This suggests that the amount of visual field stimulated during 3D presentation affects VIMS, and is an important factor in terms of viewing comfort. In the study more younger viewers (21 to 39 years old) than older viewers (40 years old and older) reported a greater change in visual discomfort during the 3D condition than the 2D condition. This suggests that the visual system s response to a stimulus, rather than the stimulus itself, is a reason for discomfort. No influence of gender on viewing comfort was found. In the next experiment participants fusion capability, as measured by their fusional reserves, was examined to determine whether this component has an impact on reported discomfort during the watching of movies in the 3D condition versus the 2D condition. It was hypothesised that participants with limited fusional range would experience more visual discomfort than participants with a wide fusion range. The hypothesis was confirmed but only in the case of convergent and not divergent eye movement. This observation illustrates that participants capability to convergence has a significant impact on visual comfort. The aim of the last experiment was to examine responses of the accommodation system to changes in 3D stimulus position and to determine whether discrepancies in these responses (i.e. accommodation overshoot, accommodation undershoot) could account for visual discomfort experienced during 3D stereoscopic viewing. It was found that accommodation discrepancy was larger for perceived forwards movement than for perceived backwards movement. The discrepancy was slightly higher in the group susceptible to visual discomfort than in the group not susceptible to visual discomfort, but this difference was not statistically significant. When considering the research findings as a whole it was apparent that not all participants experienced more discomfort whilst watching 3D stereoscopic stimuli than whilst watching 2D stimuli. More visual discomfort in the 3D condition than in the 2D condition was reported by 35% of the participants, whilst 24% of the participants reported more headaches and 17% of the participants reported more VIMS. The research indicates that multiple causative factors have an impact on reported symptoms. The analysis of the data suggests that discomfort experienced by people during 3D stereoscopic stimulation may reveal binocular vision problems. This observation suggests that 3D technology could be used as a screening method to diagnose un-treated binocular vision disorder. Additionally, this work shows that 3D stereoscopic technology can be easily adopted to binocular vision measurement. The conclusion of this thesis is that many people do not suffer adverse symptoms when viewing 3D stereoscopic displays, but that if adverse symptoms are present they can be caused either by the conflict in the stimulus, or by the heightened experience of self-motion which leads to Visually-Induced Motion Sickness (VIMS).

We propose a method for real-time photorealistic stereo rendering of the natural phenomenon of fire. Applications include the use of virtual reality in fire fighting, military training, and entertainment. Rendering fire in real-time presents a challenge because of the transparency and non-static fluid-like behavior of fire. It is well known that, in general, methods that are effective for monoscopic rendering are not necessarily easily extended to stereo rendering because monoscopic methods often do not provide the depth information necessary to produce the parallax required for binocular disparity in stereoscopic rendering. We investigate the existing techniques used for monoscopic rendering of fire and discuss their suitability for extension to real-time stereo rendering. Methods include the use of precomputed textures, dynamic generation of textures, and rendering models resulting from the approximation of solutions of fluid dynamics equations through the use of ray-tracing algorithms. We have found that in order to attain real-time frame rates, our method based on billboarding is effective. Slicing is used to simulate depth. Texture mapping or 2D images are mapped onto polygons and alpha blending is used to treat transparency. We can use video recordings or pre-rendered high-quality images of fire as textures to attain photorealistic stereo.

The explosion in stereoscopic video distribution increases the concerns over its copyright protection. Watermarking can be considered as the most flexible property right protection technology. The watermarking applicative issue is to reach the trade-off between the properties of transparency, robustness, data payload and computational cost. While the capturing and displaying of the 3D content are solely based on the two left/right views, some alternative representations, like the disparity maps should also be considered during transmission/storage. A specific study on the optimal (with respect to the above-mentioned properties) insertion domain is also required. The present thesis tackles the above-mentioned challenges. First, a new disparity map (3D video-New Three Step Search - 3DV-NTSS) is designed. The performances of the 3DV-NTSS were evaluated in terms of visual quality of the reconstructed image and computational cost. When compared with state of the art methods (NTSS and FS-MPEG) average gains of 2dB in PSNR and 0.1 in SSIM are obtained. The computational cost is reduced by average factors between 1.3 and 13. Second, a comparative study on the main classes of 2D inherited watermarking methods and on their related optimal insertion domains is carried out. Four insertion methods are considered; they belong to the SS, SI and hybrid (Fast-IProtect) families. The experiments brought to light that the Fast-IProtect performed in the new disparity map domain (3DV-NTSS) would be generic enough so as to serve a large variety of applications. The statistical relevance of the results is given by the 95% confidence limits and their underlying relative errors lower than er<0.1

Chromo stereoscopy (CS) is a simple and cost effective 3D system that can easily deliver geospatial information. CS has been used in several scientific data presentations, including remote sensing, physical modelling and hydrographic applications. In some of these applications the 3D effect was solely CS-related, while others integrated CS with other methods of implementing 3D. CS was mainly used in static visualisation, but no dynamic applications were found. Also, the restricted use of colour was acknowledged as a limitation for CS suggesting its unsuitability for applications where colour conventions are significant. This research focuses on CS for the marine applications and aims to (i) investigate users’ perception to CS effect and its interaction with other depth cues, (ii) assess the acceptance of the potential users to the changes in conventional colouring systems, (iii) and evaluate the usability and practicality of CS as an additional visualisation system in dynamic marine applications. To address these, visual scenarios were developed and expert human participants were recruited and interview for the evaluation. CS was well perceived among the participants. The interaction between different depth cues has advantages of increasing the depth perception and comprehending the 3D nature of the surrounding environment. For instance, from a certain view angle where two objects block each other, CS enhances the interposition effect, that indicates which object is in the front and gives a qualitative estimation of the spatial separation between them. Shading increases the realism of surface objects, and provides information for their undulation. It also dilutes the colours used in CS and increases the range of colours perceived and enhances the effect perceived from CS. The advantage of using the colour coding system to indicate distance is a valuable and original outcome of this thesis. This coding improved the participants understanding of the behaviour of moving objects (whether vessels coming closer or drifting apart) and enabled users to locate them in reference to the surrounding topography. Such knowledge is important to attain safer operations in a 3D environment. Accepting changes in colours in a visual presentation is linked to experience gained during interaction with the system, and the changes would be tolerated by the users in favour of improvements in situation awareness. Blind navigation and underwater operations are examples of where CS can be beneficial.

As comunicações à distância estão crescendo consideravelmente pelo uso de aplicações através da Internet e de ambientes virtuais. Interaçõoes sociais e pessoais têm recebido especial enfoque, sobretudo a videoconferência, acarretando uma grande demanda de tecnologia apropriada para esses sistemas. Resultados de pesquisas em Presença, alguns deles obtidos nesta tese, permitiram inferir os quatro pilares da proposta: 1. visualização é muito importante nas interações sociais/pessoais; 2. tecnologia simples pode promover presença social; 3. imagens de câmera são mais satisfatórias do que avatares para algumas interações sociais/pessoais; 4. estereoscopia influência presença positivamente. Sob a restrição de usar tecnologia simples e de baixo custo, a tese propõe o uso de visualização estereoscópica para webcams, em tempo real. Porque webcams são equipamentos simples com baixa resolução de imagem, a obtenção de um estéreo com qualidade torna- se um desafio. Partindo do princípio que o olho humano não se comporta da mesma forma para ver de perto e de longe, a pesquisa analisou o processamento de estéreo natural do olho humano, e com base em resultados psicofísicos e fisiológicos da visão binocular, a tese propõe um modelo para o processamento de imagens estereoscópicas a serem visualizadas em curta distância, e usa este modelo na implementação apresentada. Os resultados obtidos foram satisfatórios para a visualização de imagens estéreo com webcams, em tempo real, e principalmente, eliminaram a necessidade de reajuste de paralaxe das imagens e do reposicionamento das câmeras a cada movimentação do observador, dando assim mais liberdade de visualização ao observador.
The improvement of communication among remotely located people constitutes a very comprehensive prominent subject nowadays. Among applications to attain this goal, some aim to support social and personal interactions to meet partners, friends and family in distance. Presence research results, some obtained as part of this thesis, allowed to infer the four pillars of the proposal: 1. visualization is very important in social/personal interactions, mainly when people have close social/personal relationship; 2. simple technology can afford social presence; 3. camera images are more satisfactory than avatars during some social/personal interactions; 4. stereoscopy influences presence positively. Under the constraint of using simple technology without adding any special hardware to a PC system, the thesis proposes the use of stereoscopy through webcams, in real- time, as one solution to add 3D features to camera images. Since webcams are simple equipments with low resolution, to obtain stereo images in good quality is a challenge. Considering the fact that human eyes treat short and long distance in a different way, this research analysed the natural stereo processing of human eye, and based on psychophysical and physiological features of binocular vision, the thesis proposes a model to process stereoscopic images to be visualized in short distance, and implemented it as a proof of concept. The obtained results are satisfactory for stereoscopic images visualization through webcams, in real-time, and mainly, eliminated the need of adjusting images parallax and cameras position for each viewer´s movement, allowing more freedom to the viewer.

Hand pose is emerging as an important interface for human-computer interaction. The problem of hand pose estimation from passive stereo inputs has received less attention in the literature compared to active depth sensors. This thesis seeks to address this gap by presenting a data-driven method to estimate a hand pose from a stereoscopic camera input, with experimental results comparable to more expensive active depth sensors. The frameworks presented in this thesis are based on a two camera stereo rig capture as it yields a simpler and cheaper set-up and calibration. Three frameworks are presented, describing the sequential steps taken to solve the problem of depth and pose estimation of hands. The first is a data-driven method to estimate a high quality depth map of a hand from a stereoscopic camera input by introducing a novel regression framework. The method first computes disparity using a robust stereo matching technique. Then, it applies a machine learning technique based on Random Forest to learn the mapping between the estimated disparity and depth given ground truth data. We introduce Eigen Leaf Node Features (ELNFs) that perform feature selection at the leaf nodes in each tree to identify features that are most discriminative for depth regression. The system provides a robust method for generating a depth image with an inexpensive stereo camera. The second framework improves on the task of hand depth estimation from stereo capture by introducing a novel superpixel-based regression framework that takes advantage of the smoothness of the depth surface of the hand. To this end, it introduces Conditional Regressive Random Forest (CRRF), a method that combines a Conditional Random Field (CRF) and a Regressive Random Forest (RRF) to model the mapping from a stereo RGB image pair to a depth image. The RRF provides a unary term that adaptively selects different stereo-matching measures as it implicitly determines matching pixels in a coarse-to-fine manner. While the RRF makes depth prediction for each super-pixel independently, the CRF unifies the prediction of depth by modeling pair-wise interactions between adjacent superpixels. The final framework introduces a stochastic approach to propose potential depth solutions to the observed stereo capture and evaluate these proposals using two convolutional neural networks (CNNs). The first CNN, configured in a Siamese network architecture, evaluates how consistent the proposed depth solution is to the observed stereo capture. The second CNN estimates a hand pose given the proposed depth. Unlike sequential approaches that reconstruct pose from a known depth, this method jointly optimizes the hand pose and depth estimation through Markov-chain Monte Carlo (MCMC) sampling. This way, pose estimation can correct for errors in depth estimation, and vice versa. Experimental results using an inexpensive stereo camera show that the proposed system measures pose more accurately than competing methods. More importantly, it presents the possibility of pose recovery from stereo capture that is on par with depth based pose recovery.

Stereoscopic 3-D has received considerable attention over the last few decades. Since a stereoscopic 3-D pair includes two 2-D images together, the amount of data for an uncompressed stereo image is double compared to that for an uncompressed 2-D image. Thus efficient compression techniques are of paramount importance. However, crosstalk effect is an inherent perceivable problem in current 3-D display technologies. It can lead not only to degradation in the perceived quality of 3-D images, but also to discomfort in some individuals. Correspondingly, when crosstalk occurs, the compression artifacts in a compressed stereo pair can be perceived, despite the fact that such artifacts are imperceptible in individual left and right images. This dissertation proposes a methodology for visually lossless compression of monochrome stereoscopic 3-D images in which crosstalk effect is carefully considered. In the proposed methodology for visually lossless compression of monochrome stereoscopic 3-D images, visibility thresholds are measured for quantization distortion in JPEG2000 to conceal perceivable compression artifacts. These thresholds are found to be functions of not only spatial frequency, but also of wavelet coefficient variance, as well as the gray level in both the left and right images. In order to avoid a daunting number of measurements of visibility thresholds during subjective experiments, a model for visibility thresholds is developed. The left image and right image of a stereo pair are then compressed jointly using the visibility thresholds obtained from the proposed model to ensure that quantization errors in each image are imperceptible to both eyes. This methodology is then demonstrated via a 3-D stereoscopic liquid crystal display (LCD) system with an associated viewing condition. The resulting images are visually lossless when displayed individually as 2-D images, and also when displayed in stereoscopic 3-D mode. In order to have better perceptual quality of stereoscopic 3-D images, hardware based techniques have been used to reduce crosstalk in 3-D stereoscopic display systems. However, crosstalk is still readily apparent in some 3-D viewing systems. To reduce crosstalk remains after hardware crosstalk compensation, a methodology for crosstalk compensation accomplished via image processing is provided in this dissertation. This methodology focuses on crosstalk compensation of 3-D stereoscopic LCD systems in which active shutter glasses are employed. Subjective experiments indicate that crosstalk is a function of not only the pixel intensity in both the left and right channels, but also of spatial location. Accordingly, look-up tables (LUTs) are developed for spatially-adaptive crosstalk compensation. For a given combination of gray levels in the left and right channels at a specific spatial location, the original pixel values are replaced by values contained in the LUTs. The crosstalk in the resulting stereo pair is significantly reduced, resulting in a significant increase in perceptual image quality.

This thesis focuses on the design of stereoscopic 3D user interfaces (UIs) for mobile devices. 3D output is no longer limited to large screens in cinemas or living rooms. Nowadays more and more mobile devices are equipped with autostereoscopic 3D (S3D) touchscreens, which do not require the use of special eyewear to perceive the depth effect. As a consequence, interaction with 3D content now also happens whilst users are on the move. This thesis presents an empirical study that was carried out to assess how users interact with mobile S3D devices, both whilst static and when walking. Following this, the design and implementation of a prototype S3D application for indoor navigation is presented. A user study (n=27) that investigated differences in touch accuracy between 2D touchscreens and mobile S3D touchscreens was carried out. The touch target sizes and aspect ratios required for reliable user interaction in each case were calculated, for example concluding that the increase in minimum touch target size caused by walking is larger for a S3D UI than for a 2D UI. This thesis also identifies differences in the angle at which users held the S3D device vs. the 2D device, and reports on depth perception, both in static and mobile contexts. Following on from the empirical study, a prototype S3D indoor navigation application was designed and implemented. Several findings related to visualisation issues in such interfaces are described, e.g. the correct way to use drop shadows as visual cues. This thesis provides valuable information to developers of the next generation of UIs and applications for mobile S3D displays and devices.

This research work describes of the developed software of endoscopic images processing. Calculating pairs of corresponding points, which in the future can be used for three-dimensional reconstruction, was conducted. The number of points for each frame is not large enough, so the 3D reconstruction should use the entire video stream. To increase the number of points should also conduct a study on setting the parameters of the detector. The study tested the stage of finding matches on stereo pairs of endoscopic images.

Thesis (M.S. in Astronautical Engineering and M.S. in Applied Physics) Naval Postgraduate School, December 1994.
Thesis advisor(s): David Cleary, Oscar Biblarz. "December 1994." Includes bibliographical references. Also available online.

Recent developments in 3D media technology have brought to life numerous applications of interactive entertainment such as 3D cinema, 3DTV and gaming. Due to the data intensive nature of 3D visual content, Quality of Experience (QoE) has become a major driving factor to optimise the end-to-end content delivery process. However, to ensure the QoE, there is a need to develop more robust and accurate objective metrics for stereoscopic image and video quality assessment. Existing stereoscopic QoE metrics tend to lack in accuracy and robustness compared to its 2D counterparts as they are either extensions of 2D metrics or are based on simple perceptual models. However, measuring stereoscopic QoE requires more perceptually inspired metrics. This research introduces full-reference stereoscopic image and video quality metrics based on a Human Visual System (HVS) model incorporating important physiological findings on binocular vision. Firstly, a novel HVS model extending existing models in the literature is proposed to include the phenomena of binocular suppression and recurrent excitation towards stereoscopic image quality assessment. Secondly the research is extended to the temporal domain using temporal pooling of the HVS model outputs for individual frames and using a spatio-temporal model in the HVS model towards two distinct temporally inspired stereoscopic video quality metrics. Finally, motion sensitivity is introduced to the HVS model towards a perception inspired stereoscopic video quality metric. The proposed QoE metrics are trained, verified and tested using four publicly available stereoscopic image databases and two stereoscopic video datasets. They indicate an increase of average correlation index from 0.66 (baseline method) to 0.86 for the stereoscopic images and a maximum increase of average correlation index from 0.57 (baseline method) to 0.93 for stereoscopic videos. These results demonstrate the benefits of using a perceptually inspired approach in this research.

碩士
國立成功大學
資訊工程學系
102
Both of image panorama and video synopsis are vital topics in the field of video post-processing. In this work, we aim at generating a long strip of stereo panoramic view from stereoscopic video to provide synopsis with depth perception. The problem comes from that how to preserve disparity well and rectify panorama without loss of boundary information. Previous approach focus on the generation of panorama from usual 2D video, in addition, only crop unwanted region after stitching step. Unlike previous work, after careful handling on key frame selection and disparity preservation in warping process, our system minimize distortion on the image plane and maintain sense of depth. With an eye to achieving appealing results, our algorithm consists of three steps. First, we select representative frame intelligently from stereoscopic video. Take image quality into consideration, blurry or shaking frame must be avoided and the regional coverage of key frame must be assured. Second, we stitch these key frames into a panorama, which have irregular boundary. Finally, we perform disparity-aware image warping step on left and right panoramic images to restore depth perception of original video and generate rectangular stereo panorama. Following series of experiments, we conclude our work is practicable for generating visually plausible stereoscopic panorama result and demonstrates the effectiveness of our method.

by Roland Siu-kwong Ip.
Thesis (M.Phil.)--Chinese University of Hong Kong, 1995.
Includes bibliographical references (leaves 101-[105]).
Chapter 1 --- Introduction --- p.1
Chapter 1.1 --- Motivation --- p.1
Chapter 1.2 --- Image Compression --- p.2
Chapter 1.2.1 --- Classification of Image Compression --- p.2
Chapter 1.2.2 --- Lossy Compression Approaches --- p.3
Chapter 1.3 --- Video Compression --- p.4
Chapter 1.3.1 --- Video Compression System --- p.5
Chapter 1.4 --- Stereoscopic Video Compression --- p.6
Chapter 1.5 --- Organization of the thesis --- p.6
Chapter 2 --- Motion Video Coding Theory --- p.8
Chapter 2.1 --- Introduction --- p.8
Chapter 2.2 --- Representations --- p.8
Chapter 2.2.1 --- Temporal Processing --- p.13
Chapter 2.2.2 --- Spatial Processing --- p.19
Chapter 2.3 --- Quantization --- p.25
Chapter 2.3.1 --- Scalar Quantization --- p.25
Chapter 2.3.2 --- Vector Quantization --- p.27
Chapter 2.4 --- Code Word Assignment --- p.29
Chapter 2.5 --- Selection of Video Coding Standard --- p.31
Chapter 3 --- MPEG Compatible Stereoscopic Coding --- p.34
Chapter 3.1 --- Introduction --- p.34
Chapter 3.2 --- MPEG Compatibility --- p.36
Chapter 3.3 --- Stereoscopic Video Coding --- p.37
Chapter 3.3.1 --- Coding by Stereoscopic Differences --- p.37
Chapter 3.3.2 --- I-pictures only Disparity Coding --- p.40
Chapter 3.4 --- Stereoscopic MPEG Encoder --- p.44
Chapter 3.4.1 --- Stereo Disparity Estimator --- p.45
Chapter 3.4.2 --- Improved Disparity Estimation --- p.47
Chapter 3.4.3 --- Stereo Bitstream Multiplexer --- p.49
Chapter 3.5 --- Generic Implementation --- p.50
Chapter 3.5.1 --- Macroblock Converter --- p.54
Chapter 3.5.2 --- DCT Functional Block --- p.55
Chapter 3.5.3 --- Rate Control --- p.57
Chapter 3.6 --- Stereoscopic MPEG Decoder --- p.58
Chapter 3.6.1 --- Mono Playback --- p.58
Chapter 3.6.2 --- Stereo Playback --- p.60
Chapter 4 --- Performance Evaluation --- p.63
Chapter 4.1 --- Introduction --- p.63
Chapter 4.2 --- Test Sequences Generation --- p.63
Chapter 4.3 --- Simulation Environment --- p.64
Chapter 4.4 --- Simulation Results --- p.65
Chapter 4.4.1 --- Objective Results --- p.65
Chapter 4.4.2 --- Subjective Results --- p.72
Chapter 5 --- Conclusions --- p.80
Chapter A --- MPEG ´ؤ An International Standard --- p.83
Chapter A.l --- Introduction --- p.83
Chapter A.2 --- Preprocessing --- p.84
Chapter A.3 --- Data Structure of Pictures --- p.85
Chapter A.4 --- Picture Coding --- p.86
Chapter A.4.1 --- Coding of Motion Vectors --- p.90
Chapter A.4.2 --- Coding of Quantized Coefficients --- p.94
References --- p.101

碩士
國立臺灣科技大學
資訊管理系
102
Stereoscopy technique has been widely used in various fields. Its purpose is to allow viewers to perceive three-dimensional effects, so that each viewer can personally experience a stereoscopic scene. In our study, we apply image stylization technique to stereoscopic images. In the first, users should input one pair of stereoscopic images and their corresponding disparity maps. In addition, users can adjust the stippling parameters: the size and amount of stippling, to change the stippling style. After setting up parameters, our system can automatically process and output images which can make people who wear red-cyan glasses experience the 3D effect. Our approach is to generate the left stippling image named L in the first, and generate the right stippling image named R through the left corresponding disparity map, and then we use the right corresponding disparity map to generate a new left stippling image L '.Finally we let L' and R become an anaglyph Image (Left: red, right: cyan), and users can perceive three-dimensional effect through the red-cyan glasses. We might encounter the problem of binocular rivalry when processing the stereoscopic images. The problem is due to the point parameters: the size and number of points; leading to regional inconsistencies, so when users watch the anaglyph Image results, the results make users’ eyes feel uncomfortable. To address this problem, we propose an interactive mutual check mechanism to improve the results.

碩士
國立中興大學
電機工程學系
88
Abstract Magnetic resonance angiography (MRA) is an imaging technique to show the blood vessels but suppress signals from all the other tissues. There are two approaches to acquire an image of MRA. One is done in two dimension by projecting the 3-D vessels onto 2-D plane. The other is to directly obtain the complete 3-D information. The advantage of 3-D MRA is that one can view the data from arbitrary direction. However, the scan time is usually very long for 3-D MRA. When the scan time is limited, we must use 2-D MRA and the depth of information is sacrificed. In this thesis, we research on the subject of recovering the depth information by reconstructing 3-D vessels from two projective MRA images. Stereoscopic angiography utilizes two images by projecting blood vessels onto 2-D plane in two angles. Two major modalities are digital subtraction angiography (DSA) and magnetic resonance angiography (MRA). For DSA, the pixel value is the integration of the attenuation value in the path of the X ray. We can use this property to derive the shape of the vessels by solving the integrals. On the other hands, there are many imaging parameters in MRA (such as T1 , T2 and proton density). Therefore, it is difficult to obtain the relation between the shape of the vessels and the pixel intensity. Therefore, we attempt to reconstruct the shape of the vessels by geometry. We assume that the shape of the vessel on every cross-section is an ellipse. Then, we develop an algorithm to estimate the parameters of the ellipse from the boundaries of the projective images. The reconstructed ellipses are then taken as the 3-D shape of the vessels. Eighty images of MRA were used to demonstrate the capability of our algorithm. From these images, we use fifty images to make two projective images. The two projections are 300 apart. We employ our algorithm to estimate all the ellipses and reconstruct the 3-D model of the vessels. Comparing the boundaries of the original projective images with the boundaries of the reconstructed 3-D model, the average error is 0.471 pixels.

碩士
國立臺灣大學
資訊網路與多媒體研究所
99
Image restoration has been a major subject of computer vision. Repairing damaged photos requires several computer vision techniques such as denoising, detection, relighting, color-transfer, and inpainting. In this thesis, we proposed an automatic system to repair noise and light inconsistency problems. Our approach combined modern image restoration techniques with additional information gathered from stereoscopic photographs to enhance the effect of image restoration and optimize stereoscopic experience of old stereoscopic photographs.