Дисертації з теми "Spatio-temporal sequences"

Many applications use multiple cameras to simultaneously capture imagery of a scene from different vantage points on a rigid, moving camera system over time. Multiple cameras often provide unique viewing angles but also additional levels of detail of a scene at different spatio-temporal resolutions. However, in order to benefit from this added information the sources must be temporally aligned. As a result of cost and physical limitations it is often impractical to synchronize these sources via an external clock device. Most methods attempt synchronization through the recovery of a constant scale factor and offset with respect to time. This limits the generality of such alignment solutions. We present an unsupervised method that utilizes a content-based clustering mechanism in order to temporally align multiple non-synchronized image sequences of different and varying spatio-temporal resolutions. We show that the use of temporal constraints and dynamic programming adds robustness to changes in capture rates, field of view, and resolution.

Este trabalho introduz novos algoritmos de redes neurais para o processamento online de padrões espaço-temporais, estendendo o algoritmo Incremental Gaussian Mixture Network (IGMN). O algoritmo IGMN é uma rede neural online incremental que aprende a partir de uma única passada através de dados por meio de uma versão incremental do algoritmo Expectation-Maximization (EM) combinado com regressão localmente ponderada (Locally Weighted Regression, LWR). Quatro abordagens diferentes são usadas para dar capacidade de processamento temporal para o algoritmo IGMN: linhas de atraso (Time-Delay IGMN), uma camada de reservoir (Echo-State IGMN), média móvel exponencial do vetor de entrada reconstruído (Merge IGMN) e auto-referência (Recursive IGMN). Isso resulta em algoritmos que são online, incrementais, agressivos e têm capacidades temporais e, portanto, são adequados para tarefas com memória ou estados internos desconhecidos, caracterizados por fluxo contínuo ininterrupto de dados, e que exigem operação perpétua provendo previsões sem etapas separadas para aprendizado e execução. Os algoritmos propostos são comparados a outras redes neurais espaço-temporais em 8 tarefas de previsão de séries temporais. Dois deles mostram desempenhos satisfatórios, em geral, superando as abordagens existentes. Uma melhoria geral para o algoritmo IGMN também é descrita, eliminando um dos parâmetros ajustáveis manualmente e provendo melhores resultados.
This work introduces novel neural networks algorithms for online spatio-temporal pattern processing by extending the Incremental Gaussian Mixture Network (IGMN). The IGMN algorithm is an online incremental neural network that learns from a single scan through data by means of an incremental version of the Expectation-Maximization (EM) algorithm combined with locally weighted regression (LWR). Four different approaches are used to give temporal processing capabilities to the IGMN algorithm: time-delay lines (Time-Delay IGMN), a reservoir layer (Echo-State IGMN), exponential moving average of reconstructed input vector (Merge IGMN) and self-referencing (Recursive IGMN). This results in algorithms that are online, incremental, aggressive and have temporal capabilities, and therefore are suitable for tasks with memory or unknown internal states, characterized by continuous non-stopping data-flows, and that require life-long learning while operating and giving predictions without separated stages. The proposed algorithms are compared to other spatio-temporal neural networks in 8 time-series prediction tasks. Two of them show satisfactory performances, generally improving upon existing approaches. A general enhancement for the IGMN algorithm is also described, eliminating one of the algorithm’s manually tunable parameters and giving better results.

This PhD research has proposed new machine learning techniques to improve human action recognition based on local features. Several novel video representation and classification techniques have been proposed to increase the performance with lower computational complexity. The major contributions are the construction of new feature representation techniques, based on advanced machine learning techniques such as multiple instance dictionary learning, Latent Dirichlet Allocation (LDA) and Sparse coding. A Binary-tree based classification technique was also proposed to deal with large amounts of action categories. These techniques are not only improving the classification accuracy with constrained computational resources but are also robust to challenging environmental conditions. These developed techniques can be easily extended to a wide range of video applications to provide near real-time performance.

This thesis concerns the study of cell deformations and is interested in two complementary approaches, an experimental one and another theoretical one. The experimental approach is motivated by the demonstration from previous works of the existence of a certain auto-organization of the deformation patterns. This auto-organization consists of the appearance of recurring protrusive patterns in space and time. This has been shown, in particular, for round-shaped cells (leukocytes or keratinocytes) which present a relatively simple organization of theactin cytoskeleton. We have chosen to study murin fibroblasts (L929 line). The fibroblasts exhibit long membrane extensions such as filopods. This time, this type of protrusion is related to a more complex organization of the actin cytoskeleton, where the filaments tend to form bundles. Our aim has been to determine if there exists a similar self-organized componentof these fibroblasts membrane deformations. Experimental characterization has been performed from image sequences where the cells were observed by phase contrast videomicroscopy. The morphodynamical data of the cells have been extracted from the images with two different methods:(i) a classical segmentation of the cell boundaries for the individual study of each protrusive zone of the cell and (ii) an optical flow method for a global characterizationof the movement of the whole cell. The results obtained show that the cells exhibit mainly symmetrical morphologies with 2 to 4 protrusions. The 4-protrusion state (cross morphology), observed for the most isolated cells, is dynamically characterized by a synchronized pulsating movement between the two perpendicular protrusive directions, where the extension in one direction is accompanied by the simultaneous retraction in the other direction (. . . ) In conclusion, we defined how the work realized experimentally allows us to propose the possibility to use the morphodynamical parameters obtained from the characterization as criteria to identify the cell phenotypes. We also discuss how theoretical modelling can orientate the choice of new experimental protocols.

Movement is one of the most fundamental processes for living entities on earth at the core of scientific disciplines such as ecology and geography. In animal ecology, ongoing progress in tracking and remote sensing technologies has spurred an explosion of movement and environmental data collected at high spatial and temporal resolution, at a large scale, so that the interaction between animal movement and habitat features can now be investigated in much more detail. As a result, in recent years the field of animal ecology has produced a growing body of studies on movement-based patterns leading to habitat use and selection. In this regard, GIScience has contributed with several visual analytical approaches to study animals in relation to their environment and habitat. However, the pat - terns behind the sequential use of different habitat classes have remained largely unexplored. Sequential habitat use is defined as the consecutive use of habitat features along the trajectory of an animal, extracted from the context of its spatial movement. By account - ing for the sequence of use, it is possible to distinguish fundamentally different behavioural habitat use strategies that are important for the survival and fitness of an animal, such as habitat alternation versus random sequential use. Such distinctions would remain undetected by only considering the proportion of use. Sequential habitat use patterns occur in a spatial context, meaning sequential patterns are affected by what is actually available to the animal. In this dissertation we merge knowledge from different fields to present an innovative method to study the relation between animals and their environment by accounting for the sequential use of habitats, and animal movement rules. We developed a visually effective method to analyse and visualise sequential habitat use patterns of animals at multiple spatio- temporal scales by combining real and simulated sequences of habitat use. To study sequential habitat use patterns we use Sequence Analysis Methods (SAM), an approach widely applied in molecular biology, as well as many applications in different fields, to measure dissimilarity between sequences of characters. In brief, we use dissimilarity algorithms to measure the distance between all pairs of sequences, and then apply a cluster - ing algorithm to investigate how these sequences group together, which are visualised as dissimilarity trees. We propose a procedure consisting of three steps, including explo- ration, simulation and classification. In the exploration phase, we build exploratory trees, which visualise real sequential habitat use patterns. Second, by applying animal movement models we simulate expected sequential habitat use patterns, and assess how spatial context, and especially habitat availability, affects the clustering of sequential patterns. Third, we combine real and simulated sequences to identify which simulated pattern is most parsimonious with the real sequences. The research progress has been presented in three main chapters. In Chapter 3 we present seminal methodological development where SAM was applied to animal movement data. In Chapter 4 we introduce further methodological advancements to extend the applicability of SAM to animal ecology. In Chapter 5 we present a large-scale multi-population ecological application. All research was performed using GPS movement data of roe deer and environmental data provided by the Euroungulates database project. Chapter 3 presents the first application of SAM to identify ecologically relevant sequential patterns in animal habitat use. We exemplify the method using ecological data consisting of simulated and real trajectories from a roe deer population (Capreolus capreolus) in the Italian Alps, expressed as ordered sequences of four habitat use classes, i.e. high/open, high/closed, low/open, low/closed. In essence, the SAM framework identifies relevant sequential patterns in real trajectories by measuring their similarity to spatially-explicit simulated trajectories with known sequential patterns. Simulation trajectories were generated in arenas resembling the landscape structure of the roe deer population. Chapter 4 extends SAM to an individual-based approach (i.e. IM-SAM, Individual Movement – Sequence Analysis Methods), that is applicable over multiple populations. Specifically, instead of performing simulations in landscape-like arenas, we use real individual home ranges, thus accounting for individual spatial context, and landscape composition and structure. To assess usability of our advanced framework we investigate the sequential use of open and forest habitats for nine roe deer populations ranging in landscapes with different geographic contexts and anthropogenic disturbance. We also discuss implications for conservation and management. Chapter 5 addresses the functional role of landscapes throughout seasons by identifying both population level and individual level variability in the sequential habitat use patterns of roe deer, identified in the former nine roe deer populations. We show how identified sequential habitat use patterns can be treated as variables, and analysed with standard and well-accepted statistical methods. While the (IM-)SAM framework was developed for studying sequential habitat use in specific, we highlight that its methodological steps and study design can easily be gener- alised. Indeed, its dissimilarity and clustering algorithms, temporal resolution, sampling units, and number of classes for which sequential patterns are investigated can all be customised for the specific research questions in mind. (IM-)SAM is easily applicable to different types of sequential data that describe aspects of an animal's internal (e.g. heart rate) or external state (e.g. temperature). Through improvements in technology, including the growing number of information that can be collected through sensors (GPS trackers, biologgers and satellites), improving database infrastructures and the instant availability of advanced R packages dedicated to animal movement, (IM-)SAM could be easily integrated in a wide range of both local and broad-scaled behavioural spatio-temporal studies.

L’objectif principal de cette thèse est d’instruire l’organisation des successions culturales, à l’échelle de la France et sur un recul décennal, tant en terme de cinétiques localisées qu’en terme de dynamiques liées à des variables explicatives du milieu physique et socio-technico-économiques. Ce travail de « fouille de données » est appliqué en France métropolitaine, en utilisant les bases de données publiques disponibles. Notre étude couvre la période 1992-2003 où s’implémenta la réforme de 1992 de la Politique Agricole Commune européenne et l’agenda 2000 en France. A partir d’une fouille de données sur l’ensemble des points Terruti sur cette période, 2549 successions culturales de trois ans furent identifiées. Ensuite, 21 clusters de l’ensemble des 430 régions agricoles (RA) françaises, quatre systèmes de culture, 90 RA, parangons des 430 RA, et trois régions principales appartenant à cinq des 21 clusters, regroupant les cultures de céréales, oléagineux et protéagineux, ont été définis. Deux approches de cinétiques des successions ont été réalisées : Une étude envisageant les successions de culture qui suivent les retournements de prairies permanentes. Une recherche des dynamiques d’évolution de successions culturales en lien avec des conditions externes. Nous proposons une utilisation ultérieure des méthodes mobilisées qui ont montré leur capacité à cartographier les grandes tendances d’évolution en France et à identifier les principales variables explicatives de ces évolutions. Les apports de cette thèse contribuent à améliorer notre compréhension des processus qui organisent les successions culturales en France et construisent par ces pratiques agricoles très dynamiques des impacts forts sur le territoire agricole français
In the context of changing agricultural policy, the development of agricultural production systems, increasing concern for agricultural sustainability and shifts in agricultural land management practice-related land-use change, the main objective of this thesis was to mine crop sequence patterns (CSP) and the relationships between CSP and the biophysical and socio-technical-economic conditions in mainland France from historical census data (e.g. land-cover survey, agricultural censuses, population census). Our study period 1992-2003 covers the implementation period of the 1992 European Union Common Agricultural Policy reform and Agenda 2000 in France. Both the classical statistical and data mining technique were applied in alone or combined ways in this thesis. First, we proposed an innovative approach to representing CSP within a given area and period at a large regional scale in a stationary way. The 2549 3-year crop sequences (CSs) were first identified as major CSs within all 430 agricultural districts (ADs) in mainland France during this period. Next, 21 clusters of ADs , four types of cropping systems, 90 representative ADs and three principal planting zones of cereals, oilseeds, and protein crops belonging to five clusters identified previously were further defined. We then explored CSP in a dynamic way by investigating CSP after grassland-to-cropland conversion, the temporal variability of CSP, and the evolution of the relationships between CSP and the external conditions over the study period. We conclude that the approaches developed here permit the representation of CSP at the large regional scale in both stationary and dynamic ways using time series land-cover data denoting specific agricultural cover types. The findings of this thesis contribute to improving the understanding of the process and pattern of human land management practices by agriculture affecting the terrestrial biosphere

This diploma thesis deals with automatic soccer event detection . Its goal is to introduce reader to this issue , discuss possible ways of solution of this task and then implement event detection . This work aims at event recognition using spatio - temporal data of gaming objects . Introduced way of dealing with event detection lies in its converting to sequence labeling task . Then such task is solved using LSTM recurrent neural networks . Lastly , result of sequence labeling is interpreted as detected events . Library for event detection has been created as the output of this work . This library allow user to experiment with different variants how to formulate event detection as sequence labeling task .

碩士
國立交通大學
電子工程系
89
In this thesis, we propose a novel approach of motion segmentation. Conventional motion segmentation approaches usually utilize motion estimation to acquire the temporal information of the image sequence. However, this approach basically cannot handle the aperture problem, distortion problem, and occlusion problem. In this thesis, we propose a different approach to avoid these problems. We translate the transform of an image sequence into the spatio-temporal domain. With this translation, the motion segmentation issue becomes a 3-D object segmentation issue in the spatio-temporal domain. In our approach, we first transform the image sequence from RGB color space to CIE L* a* b* color space to achieve the color presentation similar to the perception of human visual system. Then, we perform four directional Balance operators on the image to search for the change points of the color trend. According to these change points, the low contrast regions are merged and then the 2-D image segmentation is achieved. About 3-D object segmentation, we apply the proposed 2-D segmentation approach to three perpendicular planes in the spatio-temporal domain. After combining these 2-D segmentation results together and applying some post-processing, the 3-D object segmentation is completed. The simulation result indicates that the objects in the image sequence can be segmented correctly.

博士
國立交通大學
資訊科學與工程研究所
96
In surveillance applications, search space reduction (SSR) is an essential element to efficient algorithms. In this study, spatial and temporal SSRs are integrated for license plate recognition (LPR) in video sequences. However, as more features are measured, the computational load may increase significantly. When regard to the fact that most input patterns are negatives, it is apparently efficient to reject a majority of negatives as soon as possible. Therefore, we propose a realtime LPR based on a cascaded rejection framework to reduce spatiotemporal search space rapidly, while ensuring that the performance is high. To extract plates accurately even in complicated situations, two representations, compact plate regions and repeated regions, are first presented. Compact plate regions, which bound the top and bottom of plate characters, could be extracted in the first stage to avoid the use of additional removal procedures. Our method started from spatial SSR by algorithms of one-pass compact plate extraction, bi-level plate character segmentation, and adaptive machine learning. Region candidates of compact plates or plate characters are extracted and verified by these algorithms performed on effectively calculated features, such as vertical gradients and extended Haar-like features. Moreover, we proposed to exclude repeated patterns with the similar appearances in the same location of consecutive frames, which usually include stopped vehicles or regular backgrounds and could be excluded from repeated classification. For efficiency, repeated patterns were detected only on the plate candidates, named spatiotemporal SSR, based on a block-based mechanism by estimating the tangent distance, which is invariant to the variations in positions, sizes, rotations, or brightness. In our experiments, the search space could be reduced up to 87.9% by the spatiotemporal SSR; the LPR system can recognize plates over 38 frames per second with a resolution of 640 x 480 pixels on a 3-GHz Intel P-IV PC.