Academic literature on the topic 'Structured-light-stereo'

3D surface reconstruction has been widely used in reverse engineering, remanufacturing and automatic measurement. Binocular stereo imaging technology is the main method to obtain 3D information. Images are acquired from different positions of the object. The 3D coordinates of the point can be calculated according to the point’s coordinates in left and right cameras. The key to ensure the accuracy of 3D reconstruction is high precision and fast matching method of two images. As the image feature was not obvious, structured light is used. For its easy identification and extraction, the method has been more and more widely used. But the former research cannot ensure the matching exactness or matching algorithm takes too much time. In order to improve the precision and efficiency of 3D measurement, a fast and simple structured light matching method is designed based on epipolar geometry. This method can ensure 100% matching accuracy. It does not need other assistant tool except for structured light projector. On the beginning of the measurement, structured light is projected onto the object to be measured. Two images of the object with structured light are captured by two cameras in different positions. Structured lights extracted from the two images are matched by the new algorithm. The 3D coordinate of this structured light is calculated. The position of structured light is changed on the object till completed. From a lot of experiments, the proposed grating matching method is proved and it is a technique with high precision, low costs, easy operation, and an automatically matching method. Furthermore, it can be widely used in most of 3D reconstruction system.

The Monocular stereo vision sensor based on structured light is designed and invented for the detection of the weld seam. By incorporating the structured light, the corresponding points matching problem in the stereo vision has been solved effectively. Firstly, the built monocular stereo vision system is used to obtain the image information of seam surface. Then the image coordinate of the seam feature points in the two-dimensional plane can be identified and extracted, through the responding image processing and feature analysis of the seam image collected. After that, the three-dimensional coordinates are calculated by using the theory of projective geometry according to the positional relation between the CCD camera and structured light projected in the seam surface. By this way, the seam feature information such as the three-dimensional geometry sizes, the groove type, the angle, and the interval can be obtained. Finally, the algorithm performance is tested by the experiment of obtaining the three-dimensional coordinate for seams center points, and the results have proved the detection accuracy and reliability of the visual collection system.

It is a difficult task to get enough numbers of highly accurate control points for projector calibration in line structured light vision system. A new projector calibration method based on binocular stereo vision is proposed in this paper. Two cameras calibration can be done usingtraditional camera calibration method and they composed a binocular stereo vision. In projector calibration procedure, a planar template was located at several different positions in front of the stereo vision system. Two cameras captured the stripe images in each position simultaneously. Every center points of the laser stripe can be used as control points of the projector plane. The 3D coordinate of the stripe center points can be obtained through binocular stereo vision principle easily. So the light plane can be calculated quickly. Experiments were carried out and the result shows that the proposed method is flexible and stable.

Calibration is a critical step in structured light 3D imaging systems. However, in the traditional calibration process, since the calibration plate is based on a two-dimensional model, the flatness of the calibration plate and the angle of the photo will affect the subsequent stitching steps based on the feature points. The number of photos also affects the calibration results. To improve the calibration accuracy, multiple photos need to be taken. The primary objective of this study was to achieve the simple and fast calibration of system parameters, so a method obtaining a large number of calibration data by homography matrix is presented, and a corresponding stereo target is designed in symmetry. First, using the relationship between the corner coordinates of the left and right parts of the stereo calibration plate and the coordinates of the world coordinate system, the homography matrix of the left and right calibration plates from the image coordinates to the world coordinates is calculated. Second, all the pixels in the stereo calibration plate are matched to the world coordinate system by using the homography matrix. In addition, we also compared the results of this method with those of traditional calibration methods. The experimental results show that the 3D geometric surface of the reconstruction result is smooth, it avoids the missing parts and the visual effect is excellent. Furthermore, the error range of small and complex objects can be reduced to 0.03 mm~0.05 mm. This method simplifies the calibration steps, reduces the calibration costs and has practical application value.

Obstacle detection is a crucial issue for pilotless device guidance function and it has to be performed with high reliability to avoid any potential collision with the front object. The vision-based obstacle detection systems are regarded perfect for this purpose because they require little on all kind of condition. In this paper, an obstacle detection system using stereo vision sensors and structured light is developed. This system realizes rapid feature matching and high precision measurement distance with the help of structured light, avoiding the time-consuming of the initial corresponding pairs. After the initial detection, the system executes the tracking light strip algorithm for the obstacles. The proposed system can detect a front obstacle in vision field and obtain the size of obstacle. The proposed obstacle detection system is set up and its performance is verified experimentally

La percepció per visió es millorada quan es pot gaudir d'un camp de visió ampli. Aquesta tesi es concentra en la percepció visual de la profunditat amb l'ajuda de càmeres omnidireccionals. La percepció 3D s'obté generalment en la visió per computadora utilitzant configuracions estèreo amb el desavantatge del cost computacional elevat a l'hora de buscar els elements visuals comuns entre les imatges. La solució que ofereix aquesta tesi és l'ús de la llum estructurada per resoldre el problema de relacionar les correspondències.
S'ha realitzat un estudi sobre els sistemes de visió omnidireccional. S'han avaluat vàries configuracions estèreo i s'ha escollit la millor. Els paràmetres del model són difícils de mesurar directament i, en conseqüència, s'ha desenvolupat una sèrie de mètodes de calibració.
Els resultats obtinguts són prometedors i demostren que el sensor pot ésser utilitzat en aplicacions per a la percepció de la profunditat com serien el modelatge de l'escena, la inspecció de canonades, navegació de robots, etc.
Vision perception is enhanced when a large field of view is available. This thesis is focused on the visual perception of depth by means of omnidirectional cameras. The 3D sensing is obtained in computer vision by means of stereo configurations with the drawback of feature matching between images. The solution offered in this dissertation uses structured light projection for solving the matching problem.
First, a survey on omnidirectional vision systems was realized. Then, the sensor design was addressed and the particular stereo configuration of the proposed sensor was decided. An accurate model is obtained by a careful study of both components of the sensor. The model parameters are measured by a set of calibration methods.
The results obtained are encouraging and prove that the sensor can be used in depth perception applications such as scene modeling, pipe inspections, robot navigation, etc.

Nowadays, 3D modelling of objects from multiple images is a topic that has gained great recognition and is widely used in various fields. Recently, lots of progress has been made in identification of people using 3D face models, which are usually reconstructed from multiple face images. In this thesis, a system including stereo cameras and structured light is built for the purpose of 3D modelling. The system outputs are 3D shapes of the face and also the texture information registered to this shape. Although the system in this thesis is developed for face reconstruction, it is not specific to faces. Using the same methodology proposed in this study 3D reconstruction of any object can be achieved.

COORDENAÇÃO DE APERFEIÇOAMENTO DO PESSOAL DE ENSINO SUPERIOR
Há uma crescente demanda pela criação de modelos computacionais representativos de objetos reais para projetos de engenharia. Uma alternativa barata e eficaz consiste na utilização de técnicas de Visão Computacional baseada em câmeras e projetores disponíveis no mercado de computadores pessoais. Este trabalho avalia um sistema óptico estéreo ativo para capturar formas geométricas de objetos utilizando um par de câmeras e um projetor digital. O sistema se baseia em idéias de trabalhos anteriores, com duas contribuições nesta dissertação. A primeira é uma técnica mais robusta de detecção de pontos notáveis em padrões de calibração das câmeras. A segunda contribuição consiste num novo método de ajuste de cilindros que visa aplicar o sistema estudado na inspeção de instalações de dutos industriais. As conclusões apresentadas procuram avaliar a robustez e precisão do sistema proposto como um instrumento de medidas em Engenharia.
There has been a growing demand for creation of computer models based on real models for engineering projects. A cheap and effective alternative consists in using Computer Vision techniques based on cameras and projectors available at the personal computer market. This work evaluates a stereo optic system for capturing geometric shapes from objects using a pair of cameras and a single digital projector. The system is based on former works and a pair of contributions is obtained at this dissertation. The first contribution is a more robust technique for finding corners and points at cameras calibration patterns. And the second one consists on a new method for cylinder fit for inspecting industrial piping facilities with the studied system. The final conclusions evaluate the robustness and precision from the proposed system as a measurement tool for Engineering.

3D face modeling is currently a popular area in Computer Graphics and Computer Vision. Many techniques have been introduced for this purpose, such as using one or more cameras, 3D scanners, and many other systems of sophisticated hardware with related software. But the main goal is to find a good balance between visual reality and the cost of the system. In this thesis, reconstruction of a 3D human face from a pair of stereo cameras is studied. Unlike many other systems, facial feature points are obtained automatically from two photographs with the help of a dot pattern projected on the object&
#8217
s face. It is seen that using projection pattern also provided enough feature points to derive 3D face roughly. These points are then used to fit a generic face mesh for a more realistic model. To cover this 3D model, a single texture image is generated from the initial stereo photographs.

La mesura estructurada de la llum ens permet obtenir una forma tridimensional de superfície d’un objecte amb una gran precisió. Així, troba aplicacions àmplies en aplicacions industrials. A més, projectant patrons de franja digital a l’objecte mesurat, la llum estructurada garanteix una mesura de superfície flexible. D’ altra banda, la llum estructurada també ens permet mesurar tant objectes especulars com objectes difusors. Per a la mesura d’ objectes especulars, s’ introdueix la deflectometria. Aquí, els patrons de serrells són generats per una pantalla de cristall líquid (LCD). Per a la mesura d’objectes difusors, s’introdueix la profilometria. En aquesta tècnica, els patrons de franja són projectats per un videoprojector. En aquesta tesi, primer proposem un sistema de deflectometria de mesurament en fase estereoscòpica (SPMD), que conté dues càmeres i un LCD comercial, per complir la mesura d’ objectes especulars. En particular, mitjançant la introducció de la càmera estereoscòpica, s’elimina la ambigüitat normal d’altura-normal sense moure cap component del sistema. A continuació, proposem un algoritme de minimització d’ errors de fase, que es compleix cercant la diferència de fase mínima entre els píxels corresponents de la pantalla LCD i la càmera, per determinar simultàniament la superfície normal i l’ altura. A més, per aconseguir una minimització de fases eficient, utilitzem un mètode d’ encaix polinòmic. Finalment, s’utilitza la integració bidimensional de Fourier per reconstruir la forma de la superfície especular. A part del mesurament de superfície especular amb deflectometria, també proposem un sistema de profilometria de projecció de serrells estereoscòpics (SFPP) per realitzar el mesurament de la forma de la superfície de l’objecte difusor. En particular, un sistema SFPP utilitza la mateixa càmera estereoscòpica que en la configuració de la deflectometria, però adopta un projector de vídeo per substituir la pantalla LCD per projectar els patrons de franja. Amb la introducció de la càmera estereoscòpica, evitem la complexa calibració del videoprojector, aconseguint així una gran flexibilitat del sistema. Per a la reconstrucció superficial, la triangulació geomètrica s’ implementa realitzant una interpolació de subpíxels 2D, a partir de la qual millorem la precisió de reconstrucció superficial. Els esmentats dos sistemes només ens permeten mesurar la forma superficial d’ un objecte especular o d’ un difusor, però mostren insuficiències per mesurar un objecte híbrid especular-difusor. En aquest escenari, combinem tots dos sistemes per realitzar el mesurament d’objectes híbrids de difusió especular. Aquí, un sistema híbrid de deflectometria-perfilometria estereoscòpica (SDPH) conté la càmera estereoscòpica, una pantalla LCD i un projector de vídeo. En aquest cas, aquest sistema híbrid supera la inadequació de sistemes lluminosos estructurats de projecció única, ja que no només garanteix la mesura d’objectes especulars o difusors, sinó que també permet la mesura d’objectes híbrids especular-difusors. Per tant, s’ amplia encara més l’ aplicació de la mesura estructurada de la llum. En resum, demostrem en aquesta tesi tres sistemes de llum estructurats basats en càmeres estereoscòpics per realitzar el mesurament de la forma en superfície tridimensional. Aquests sistemes estereoscòpics revelen un gran potencial, ja que són capaços de mesurar les formes superficials d’objectes especulars, objectes difusors i fins i tot d’objectes híbrids de difusió especulativa amb alta resolució. Els sistemes proposats poden ser beneficiosos en diverses aplicacions industrials, on es requereix un sistema de mesura de forma precisa de superfície amb un esquema fàcil d’ implementar.
La medición de luz estructurada nos permite obtener la forma tridimensional de la superficie de un objeto con alta precisión. Por lo tanto, encuentra amplias aplicaciones en aplicaciones industriales. Además, al proyectar patrones de franjas digitales en el objeto medido, la luz estructurada garantiza una medición de superficie flexible. Por otro lado, la luz estructurada también nos permite medir objetos especulares y objetos difusores. Para la medición de objetos especulares, se introduce la deflectometría. Aquí, los patrones marginales son generados por una pantalla de cristal líquido (LCD). Para la medición de objetos difusores, se introduce la profilometría. En esta técnica, los patrones marginales son proyectados por un video proyector. En esta tesis, en primer lugar, proponemos un sistema de deflectometría de medición de fase estereoscópica (SPMD), que contiene dos cámaras y una pantalla LCD comercial, para cumplir con la medición del objeto especular. En particular, al introducir la cámara estereoscópica, se elimina la ambigüedad no deseada de altura normal sin mover ningún componente del sistema. Aquí, proponemos un algoritmo de minimización de error de fase, que se cumple buscando la diferencia de fase mínima entre los píxeles correspondientes de la pantalla LCD y la cámara, para determinar simultáneamente la superficie normal y la altura. Además, para lograr una minimización de fase eficiente, utilizamos un método de ajuste polinómico. Finalmente, la integración bidimensional de Fourier se utiliza para reconstruir la forma de la superficie especular. Además de la medición de superficie especular con deflectometría, también proponemos un sistema de perfilometría de proyección de franja estereoscópica (SFPP) para lograr la medición de la forma de la superficie del objeto difusor. En particular, un sistema SFPP utiliza la misma cámara estereoscópica que en la configuración de deflectometría, pero adopta un proyector de video para reemplazar la pantalla LCD y proyectar los patrones marginales. Al presentar la cámara estereoscópica, evitamos la compleja calibración del proyector de video y, por lo tanto, se logra una gran flexibilidad del sistema. Para la reconstrucción de la superficie, la triangulación geométrica se implementa mediante la realización de una interpolación de subpíxeles 2D, desde la cual mejoramos la precisión de la reconstrucción de la superficie. Los dos sistemas mencionados anteriormente solo nos permiten medir la forma de la superficie de un objeto especular o difusor, pero muestran insuficiencias para medir un objeto híbrido especular-difusor. Bajo este escenario, combinamos ambos sistemas para realizar la medición de objeto híbrido especular-difusor. Aquí, un sistema híbrido de deflectometría-profilometría estereoscópica (SDPH) contiene la cámara estereoscópica, un LCD y un proyector de video. En este caso, este sistema híbrido supera la insuficiencia de los sistemas de luz estructurada de proyección única, ya que no solo garantiza la medición de objetos especulares o difusores, sino que también permite la medición de objetos híbridos especulares-difusores. Por lo tanto, la aplicación de la medición de luz estructurada se amplía aún más. En resumen, demostramos en esta tesis tres sistemas de luz estructurada basados en cámaras estereoscópicas para realizar mediciones tridimensionales de la forma de la superficie. Estos sistemas estereoscópicos revelan un gran potencial, ya que son capaces de medir las formas superficiales de objetos especulares, objetos difusores e incluso objetos híbridos de difusor especular con alta resolución. Los sistemas propuestos podrían ser beneficiosos en diversas aplicaciones industriales, donde se requiere un sistema preciso de medición de la forma de la superficie con un esquema fácil de implementar.
Structured light measurement allows us to obtain three-dimensional surface shape of an object with high accuracy. Thus, it finds extensive applications in industrial applications. Moreover, by projecting digital fringe patterns to the measured object, structured light guarantees a flexible surface measurement. On the other hand, structured light also allows us to measure both specular objects and diffuser objects. For specular object measurement, deflectometry is introduced. Here, the fringe patterns are generated by a liquid crystal display (LCD). For diffuser object measurement, profilometry is introduced. In this technique, the fringe patterns are projected by a video projector. In this thesis, we firstly propose a stereoscopic phase measuring deflectometry (SPMD) system, which contains two cameras and a commercial LCD, to fulfill the specular object measurement. In particular, by introducing the stereoscopic camera, the undesired height-normal ambiguity is eliminated without moving any system component. Here, we propose a phase error minimization algorithm, which is fulfilled by searching the minimum phase difference between the corresponding pixels of the LCD and the camera, to simultaneously determine the surface normal and height. What is more, to accomplish an efficient phase minimization, we use a polynomial fitting method. Finally, two-dimensional Fourier integration is used to reconstruct the specular surface shape. Apart from the specular surface measurement with deflectometry, we also propose a stereoscopic fringe projection profilometry (SFPP) system to accomplish the diffuser object surface shape measurement. In particular, an SFPP system uses the same stereoscopic camera as in the deflectometry set-up, but it adopts a video projector to replace the LCD to project the fringe patterns. By introducing the stereoscopic camera, we avoid the complex video projector calibration, and thus, a great system flexibility is achieved. For surface reconstruction, geometric triangulation is implemented by performing a 2D sub-pixel interpolation, from which we enhance the surface reconstruction accuracy. The aforementioned two systems enable us only to measure the surface shape of either a specular or a diffuser object, but they show inadequacies to measure a specular-diffuser hybrid object. Under this scenario, we combine both systems to perform the specular-diffuser hybrid object measurement. Here, a stereoscopic deflectometry-profilometry hybrid (SDPH) system contains the stereoscopic camera, an LCD and a video projector. In this case, this hybrid system overcomes the inadequacy of single projection structured light systems, as it not only ensures to measure specular or diffuser object, but it also allows the measurement of specular-diffuser hybrid objects. Hence, the application of structured light measurement is further broadened. In summary, we demonstrate in this thesis three stereoscopic camera based structured light systems to perform three-dimensional surface shape measurement. These stereoscopic systems reveal great potential as they are able to measure the surface shapes of specular objects, diffuser objects and even specular-diffuser hybrid objects with high resolution. The proposed systems could be beneficial in various industrial applications, where an accurate surface shape measurement system with an easy implemented scheme is required.

The scene depth is an important information that can be used to retrieve the scene geometry, a missing element in standard color images. For this reason, the depth information is usually employed in many applications such as 3D reconstruction, autonomous driving and robotics. The last decade has seen the spread of different commercial devices able to sense the scene depth. Among these, Time-of-Flight (ToF) cameras are becoming popular because they are relatively cheap and they can be miniaturized and implemented on portable devices. Stereo vision systems are the most widespread 3D sensors and they are simply composed by two standard color cameras. However, they are not free from flaws, in particular they fail when the scene has no texture. Active stereo and structured light systems have been developed to overcome this issue by using external light projectors. This thesis collects the findings of my Ph.D. research, which are mainly devoted to the denoising of depth data. First, some of the most widespread commercial 3D sensors are introduced with their strengths and limitations. Then, some techniques for the quality enhancement of ToF depth acquisition are presented and compared with other state-of-the-art methods. A first proposed method is based on a hardware modification of the standard ToF projector. A second approach instead uses multi-frequency ToF recordings as input of a deep learning network to improve the depth estimation. A particular focus will be given to how the denoising performance degrades, when the network is trained on synthetic data and tested on real data. Thus, a method to reduce the gap in performance will be proposed. Since ToF and stereo vision systems have complementary characteristics, the possibility to fuse the information coming from these sensors is analysed and a method based on a locally consistent fusion, guided by a learning based reliability measure for the two sensors, is proposed. A part of this thesis is dedicated to the description of the data acquisition procedures and the related labeling required to collect the datasets we used for the training and evaluation of the proposed methods.
La profondità della scena è un importante informazione che può essere usata per recuperare la geometria della scena stessa, un elemento mancante nelle semplici immagini a colori. Per questo motivo, questi dati sono spesso usati in molte applicazioni come ricostruzione 3D, guida autonoma e robotica. L'ultima decade ha visto il diffondersi di diversi dispositivi capaci di stimare la profondità di una scena. Tra questi, le telecamere Time-of-Flight (ToF) stanno diventando sempre più popolari poiché sono relativamente poco costose e possono essere miniaturizzate e implementate su dispositivi portatili. I sistemi a visione stereoscopica sono i sensori 3D più diffusi e sono composti da due semplici telecamere a colori. Questi sensori non sono però privi di difetti, in particolare non riescono a stimare in maniera corretta la profondità di scene prive di texture. I sistemi stereoscopici attivi e i sistemi a luce strutturata sono stati sviluppati per risolvere questo problema usando un proiettore esterno. Questa tesi presenta i risultati che ho ottenuto durante il mio Dottorato di Ricerca presso l'Università degli Studi di Padova. Lo scopo principale del mio lavoro è stato quello di presentare metodi per il miglioramento dei dati 3D acquisiti con sensori commerciali. Nella prima parte della tesi i sensori 3D più diffusi verranno presentati introducendo i loro punti di forza e debolezza. In seguito verranno descritti dei metodi per il miglioramento della qualità dei dati di profondità acquisiti con telecamere ToF. Un primo metodo sfrutta una modifica hardware del proiettore ToF. Il secondo utilizza una rete neurale convoluzionale (CNN) che sfrutta dati acquisiti da una telecamera ToF per stimare un'accurata mappa di profondità della scena. Nel mio lavoro è stata data attenzione a come le prestazioni di questo metodo peggiorano quando la CNN è allenata su dati sintetici e testata su dati reali. Di conseguenza, un metodo per ridurre tale perdita di prestazioni verrà presentato. Poiché le mappe di profondità acquisite con sensori ToF e sistemi stereoscopici hanno proprietà complementari, la possibilità di fondere queste due sorgenti di informazioni è stata investigata. In particolare, è stato presentato un metodo di fusione che rinforza la consistenza locale dei dati e che sfrutta una stima dell'accuratezza dei due sensori, calcolata con una CNN, per guidare il processo di fusione. Una parte della tesi è dedita alla descrizione delle procedure di acquisizione dei dati utilizzati per l'allenamento e la valutazione dei metodi presentati.

碩士
中華大學
資訊工程學系(所)
97
This paper provides an approach to scanning a real object by Structured Light and reach 3D coordinate by digital camera and DLP projector. Camera shoots composes by lens and it brings the image distortion of capturing picture. Therefore, we must use non-linear correction to solve this issue. The image must be corrected before calculating the 3D coordinate of objects. Moreover, Color distortion is also happened on common projectors. Colors of projector are composed with base-colors which filtered by Color Wheel. Some colors are not perfect with the RGB ratio from projector. That is why we need to correct the colors used by projector. Thus, we need to adjust the RGB ratio of each color to ideal values. The coordinates of object surface are computed with corresponding of camera and projector. We use the colors Red、Green、Blue、Yellow、Cyan、Magenta and white. Assign the number 1~7 to each color and use permutation coding to generate the pattern. Beam the pattern on object surface by projector and capture the picture by digital camera. Then obtain the 3D coordinate information of object from the corresponding of camera and projector. The pattern is composed with Color Groups which contents some color stripes. We need to decode the color groups by color stripes of scan line. Obtain the sequence of each color stripe of current color group and reach the Index of color group. The color stripes index number can be obtained by offset value with the color group index. Then transfer the index numbers of stripes to angles. Index numbers of stripes are implied in each color group of pattern. This information can be obtained by pattern decoding. In order to get the information, we use the Hue value to determine the number of color. Hue detection is better than RGB approach because Hue value doesn’t include the Luminance information. That means we can reduce the effect of intensity that reflects from non-uniform object surface. Pattern coding by permutation also can reduce the Error Rate when color stripes lost in a single color group. We can say it provides the Fault Tolerance ability and reduces the overhead of computing. The result of resolution and performance of Permutation Coding is better than other approaches in the same strategies. We increase 46% resolution which is more than De Brujin 3D scanning of Spatial Neighborhood.

A number of methods have been demonstrated for 3-D coordinate measurements of an image. These include time of flight, stereo matching, and structured light. Most commercially available systems employ structured light, as this technique affords the resolution and speed required for many industrial tasks. In this paper we discuss a structured light 3-D inspection system with the potential for high speed and no moving parts.