Tesis sobre el tema "Maritime anomaly detection"

The surveillance of large sea areas typically involves  the analysis of huge quantities of heterogeneous data.  In order to support the operator while monitoring maritime traffic, the identification of anomalous behavior or situations that might need further investigation may reduce operators' cognitive load. While it is worth acknowledging that existing mining applications support the identification of anomalies, autonomous anomaly detection systems are rarely used for maritime surveillance. Anomaly detection is normally a complex task that can hardly be solved by using purely visual or purely computational methods. This thesis suggests and investigates the adoption of visual analytics principles to support the detection of anomalous vessel behavior in maritime traffic data. This adoption involves studying the analytical reasoning process that needs to be supported,  using combined automatic and visualization approaches to support such process, and evaluating such integration. The analysis of data gathered during interviews and participant observations at various maritime control centers and the inspection of video recordings of real anomalous incidents lead to a characterization of the analytical reasoning process that operators go through when monitoring traffic. These results are complemented with a literature review of anomaly detection techniques applied to sea traffic. A particular statistical-based technique is implemented, tested, and embedded in a proof-of-concept prototype that allows user involvement in the detection process. The quantitative evaluation carried out by employing the prototype reveals that participants who used the visualization of normal behavioral models outperformed the group without aid. The qualitative assessment shows that  domain experts are positive towards providing automatic support and the visualization of normal behavioral models, since these aids may reduce reaction time, as well as increase trust and comprehensibility in the system. Based on the lessons learned, this thesis provides recommendations for designers and developers of maritime control and anomaly detection systems, as well as guidelines for carrying out evaluations of visual analytics environments.
Maria Riveiro is also affiliated to Informatics Research Centre, Högskolan i Skövde
Information Fusion Research Program, Högskolan i Skövde

Context: Maritime Surveillance (MS) has received increased attention from a civilian perspective in recent years. Anomaly detection (AD) is one of the many techniques available for improving the safety and security in the MS domain. Maritime authorities utilize various confidential data sources for monitoring the maritime activities; however, a paradigm shift on the Internet has created new sources of data for MS. These newly identified data sources, which provide publicly accessible data, are the open data sources. Taking advantage of the open data sources in addition to the traditional sources of data in the AD process will increase the accuracy of the MS systems. Objectives: The goal is to investigate the potential open data as a complementary resource for AD in the MS domain. To achieve this goal, the first step is to identify the applicable open data sources for AD. Then, a framework for AD based on the integration of open and closed data sources is proposed. Finally, according to the proposed framework, an AD system with the ability of using open data sources is developed and the accuracy of the system and the validity of its results are evaluated. Methods: In order to measure the system accuracy, an experiment is performed by means of a two stage random sampling on the vessel traffic data and the number of true/false positive and negative alarms in the system is verified. To evaluate the validity of the system results, the system is used for a period of time by the subject matter experts from the Swedish Coastguard. The experts check the detected anomalies against the available data at the Coastguard in order to obtain the number of true and false alarms. Results: The experimental outcomes indicate that the accuracy of the system is 99%. In addition, the Coastguard validation results show that among the evaluated anomalies, 64.47% are true alarms, 26.32% are false and 9.21% belong to the vessels that remain unchecked due to the lack of corresponding data in the Coastguard data sources. Conclusions: This thesis concludes that using open data as a complementary resource for detecting anomalous behavior in the MS domain is not only feasible but also will improve the efficiency of the surveillance systems by increasing the accuracy and covering some unseen aspects of maritime activities.
This thesis investigated the potential open data as a complementary resource for Anomaly Detection (AD) in the Maritime Surveillance (MS) domain. A framework for AD was proposed based on the usage of open data sources along with other traditional sources of data. According to the proposed AD framework and the algorithms for implementing the expert rules, the Open Data Anomaly Detection System (ODADS) was developed. To evaluate the accuracy of the system, an experiment on the vessel traffic data was conducted and an accuracy of 99% was obtained for the system. There was a false negative case in the system results that decreased the accuracy. It was due to incorrect AIS data in a special situation that was not possible to be handled by the detection rules in the scope of this thesis. The validity of the results was investigated by the subject matter experts from the Swedish Coastguard. The validation results showed that the majority of the ODADS evaluated anomalies were true alarms. Moreover, a potential information gap in the closed data sources was observed during the validation process. Despite the high number of true alarms, the number of false alarms was also considerable that was mainly because of the inaccurate open data. This thesis provided insights into the open data as a complement to the common data sources in the MS domain and is concluded that using open data will improve the efficiency of the surveillance systems by increasing the accuracy and covering some unseen aspects of maritime activities.

In the post September 11 era, the demand for security has increased in virtually all parts of the society. The need for increased security originates from the emergence of new threats which differ from the traditional ones in such a way that they cannot be easily defined and are sometimes unknown or hidden in the “noise” of daily life. When the threats are known and definable, methods based on situation recognition can be used find them. However, when the threats are hard or impossible to define, other approaches must be used. One such approach is data-driven anomaly detection, where a model of normalcy is built and used to find anomalies, that is, things that do not fit the normal model. Anomaly detection has been identified as one of many enabling technologies for increasing security in the society. In this thesis, the problem of how to detect anomalies in the surveillance domain is studied. This is done by a characterisation of the surveillance domain and a literature review that identifies a number of weaknesses in previous anomaly detection methods used in the surveillance domain. Examples of identified weaknesses include: the handling of contextual information, the inclusion of expert knowledge and the handling of joint attributes. Based on the findings from this study, a new anomaly detection method is proposed. The proposed method is evaluated with respect to detection performance and computational cost on a number datasets, recorded from real-world sensors, in different application areas of the surveillance domain. Additionally, the method is also compared to two other commonly used anomaly detection methods. Finally, the method is evaluated on a dataset with anomalies developed together with maritime subject matter experts. The conclusion of the thesis is that the proposed method has a number of strengths compared to previous methods and is suitable foruse in operative maritime command and control systems.
Christoffer Brax forskar också vid högskolan i Skövde, Informatics Research Centre / Christoffer Brax also does research at the University of Skövde, Informatics Research Centre

De nos jours, beaucoup de données peuvent être facilement accessibles. Mais toutes ces données ne sont pas utiles si nous ne savons pas les traiter efficacement et si nous ne savons pas extraire facilement les informations pertinentes à partir d'une grande quantité de données. Les techniques de détection d'anomalies sont utilisées par de nombreux domaines afin de traiter automatiquement les données. Les techniques de détection d'anomalies dépendent du domaine d'application, des données utilisées ainsi que du type d'anomalie à détecter.Pour cette étude nous nous intéressons seulement aux données séquentielles. Une séquence est une liste ordonnée d'objets. Pour de nombreux domaines, il est important de pouvoir identifier les irrégularités contenues dans des données séquentielles comme par exemple les séquences ADN, les commandes d'utilisateur, les transactions bancaires etc.Cette thèse présente une nouvelle approche qui identifie et analyse les irrégularités de données séquentielles. Cette technique de détection d'anomalies peut détecter les anomalies de données séquentielles dont l'ordre des objets dans les séquences est important ainsi que la position des objets dans les séquences. Les séquences sont définies comme anormales si une séquence est presque identique à une séquence qui est fréquente (normale). Les séquences anormales sont donc les séquences qui diffèrent légèrement des séquences qui sont fréquentes dans la base de données.Dans cette thèse nous avons appliqué cette technique à la surveillance maritime, mais cette technique peut être utilisée pour tous les domaines utilisant des données séquentielles. Pour notre application, la surveillance maritime, nous avons utilisé cette technique afin d'identifier les conteneurs suspects. En effet, de nos jours 90% du commerce mondial est transporté par conteneurs maritimes mais seulement 1 à 2% des conteneurs peuvent être physiquement contrôlés. Ce faible pourcentage est dû à un coût financier très élevé et au besoin trop important de ressources humaines pour le contrôle physique des conteneurs. De plus, le nombre de conteneurs voyageant par jours dans le monde ne cesse d'augmenter, il est donc nécessaire de développer des outils automatiques afin d'orienter le contrôle fait par les douanes afin d'éviter les activités illégales comme les fraudes, les quotas, les produits illégaux, ainsi que les trafics d'armes et de drogues. Pour identifier les conteneurs suspects nous comparons les trajets des conteneurs de notre base de données avec les trajets des conteneurs dits normaux. Les trajets normaux sont les trajets qui sont fréquents dans notre base de données.Notre technique est divisée en deux parties. La première partie consiste à détecter les séquences qui sont fréquentes dans la base de données. La seconde partie identifie les séquences de la base de données qui diffèrent légèrement des séquences qui sont fréquentes. Afin de définir une séquence comme normale ou anormale, nous calculons une distance entre une séquence qui est fréquente et une séquence aléatoire de la base de données. La distance est calculée avec une méthode qui utilise les différences qualitative et quantitative entre deux séquences
Nowadays, huge quantities of data can be easily accessible, but all these data are not useful if we do not know how to process them efficiently and how to extract easily relevant information from a large quantity of data. The anomaly detection techniques are used in many domains in order to help to process the data in an automated way. The anomaly detection techniques depend on the application domain, on the type of data, and on the type of anomaly.For this study we are interested only in sequential data. A sequence is an ordered list of items, also called events. Identifying irregularities in sequential data is essential for many application domains like DNA sequences, system calls, user commands, banking transactions etc.This thesis presents a new approach for identifying and analyzing irregularities in sequential data. This anomaly detection technique can detect anomalies in sequential data where the order of the items in the sequences is important. Moreover, our technique does not consider only the order of the events, but also the position of the events within the sequences. The sequences are spotted as anomalous if a sequence is quasi-identical to a usual behavior which means if the sequence is slightly different from a frequent (common) sequence. The differences between two sequences are based on the order of the events and their position in the sequence.In this thesis we applied this technique to the maritime surveillance, but this technique can be used by any other domains that use sequential data. For the maritime surveillance, some automated tools are needed in order to facilitate the targeting of suspicious containers that is performed by the customs. Indeed, nowadays 90% of the world trade is transported by containers and only 1-2% of the containers can be physically checked because of the high financial cost and the high human resources needed to control a container. As the number of containers travelling every day all around the world is really important, it is necessary to control the containers in order to avoid illegal activities like fraud, quota-related, illegal products, hidden activities, drug smuggling or arm smuggling. For the maritime domain, we can use this technique to identify suspicious containers by comparing the container trips from the data set with itineraries that are known to be normal (common). A container trip, also called itinerary, is an ordered list of actions that are done on containers at specific geographical positions. The different actions are: loading, transshipment, and discharging. For each action that is done on a container, we know the container ID and its geographical position (port ID).This technique is divided into two parts. The first part is to detect the common (most frequent) sequences of the data set. The second part is to identify those sequences that are slightly different from the common sequences using a distance-based method in order to classify a given sequence as normal or suspicious. The distance is calculated using a method that combines quantitative and qualitative differences between two sequences

Human operators of modern surveillance systems are confronted with an increasing amount of trajectory data from moving objects, such as people, vehicles, vessels, and aircraft. A large majority of these trajectories reflect routine traffic and are uninteresting. Nevertheless, some objects are engaged in dangerous, illegal or otherwise interesting activities, which may manifest themselves as unusual and abnormal trajectories. These anomalous trajectories can be difficult to detect by human operators due to cognitive limitations. In this thesis, we study algorithms for the automated detection of anomalous trajectories in surveillance applications. The main results and contributions of the thesis are two-fold. Firstly, we propose and discuss a novel approach for anomaly detection, called conformal anomaly detection, which is based on conformal prediction (Vovk et al.). In particular, we propose two general algorithms for anomaly detection: the conformal anomaly detector (CAD) and the computationally more efficient inductive conformal anomaly detector (ICAD). A key property of conformal anomaly detection, in contrast to previous methods, is that it provides a well-founded approach for the tuning of the anomaly threshold that can be directly related to the expected or desired alarm rate. Secondly, we propose and analyse two parameter-light algorithms for unsupervised online learning and sequential detection of anomalous trajectories based on CAD and ICAD: the sequential Hausdorff nearest neighbours conformal anomaly detector (SHNN-CAD) and the sequential sub-trajectory local outlier inductive conformal anomaly detector (SSTLO-ICAD), which is more sensitive to local anomalous sub-trajectories. We implement the proposed algorithms and investigate their classification performance on a number of real and synthetic datasets from the video and maritime surveillance domains. The results show that SHNN-CAD achieves competitive classification performance with minimum parameter tuning on video trajectories. Moreover, we demonstrate that SSTLO-ICAD is able to accurately discriminate realistic anomalous vessel trajectories from normal background traffic.

Approved for public release; distribution is unlimited.
Techniques for anomaly detection in the maritime domain by extracting traffic patterns from ship position data to generate atlases of expected ocean travel are developed in this thesis. An archive of historical data is used to develop a traffic density grid. The Hough transformation is used to extract linear patterns of elevated density from the traffic density grid, which can be considered the highways of the oceans. These highways collectively create an atlas that is used to define geographical regions of expected ship locations. Ship position reports are compared to the atlas of highways to flag as anomalous any ship that is not operating on an expected highway. The atlas generation techniques are demonstrated using automated information system (AIS) ship position data to detect highways in both open-ocean and coastal areas. Additionally, the atlas generation techniques are used to explore variability in ship traffic as a result of extreme weather and seasonal variation. Finally, anomaly detection is demonstrated by comparing AIS data from 2013 to the highways detected in the archive of data from 2012. The development of an automatic atlas generation technique that can be used to develop a definition of normal maritime behavior is the significant result of this thesis.

Nowadays ships are usually equipped with a system of marine instruments, one of which is an Automatic Identification System (AIS) transponder. The availability of the global AIS ship tracking data opened the possibilities to develop maritime security far beyond the simple collision prevention. The research work summarized in this thesis explores this opportunity, with the aim of developing an intuitive and comprehensible method for traffic modeling and anomaly detection in the maritime domain. The novelty of the method lays in employing the technique of artificial potential fields. The general idea is for the potentials to represent typical patterns of vessels' behaviors. A conflict between potentials, which have been observed in the past, and the potential of a vessel currently in motion, indicates an anomaly. The developed potential field based method has been examined using a web-based anomaly detection system STRAND (for Seafaring TRansport ANomaly Detection). Its applicability has been demonstrated in several publications, examining its scalability, modeling capabilities and detection performance. The experimental investigations led to identifying optimal detection resolution for different traffic areas (open sea, harbor and river), and extracting traffic rules, e.g., with regard to speed limits and course, i.e., right-hand sailing rule. The map-based display of modeled traffic patterns and detection cases has been analyzed as well, using several demonstrative cases. The massive AIS database created for this study, together with a dataset of real traffic incidents, provides an abundance of challenges for future studies.

Ce travail de thèse se focalise sur une classe de méthodes d’apprentissage profond, probabilistes et non-supervisées qui utilisent l’inférence variationnelle pour créer des modèles évolutifs de grande capacité pour ce type de données. Nous présentons deux classes d’apprentissage variationnel profond, puis nous les appliquons à deux problèmes spécifiques liés au domaine maritime. La première application est l’identification de systèmes dynamiques à partir de données bruitées et partiellement observées. Nous introduisons un cadre qui fusionne l’assimilation de données classique et l’apprentissage profond moderne pour retrouver les équations différentielles qui contrôlent la dynamique du système. En utilisant une formulation d’espace d’états, le cadre proposé intègre des composantes stochastiques pour tenir compte des variabilités stochastiques, des erreurs de modèle et des incertitudes de reconstruction. La deuxième application est la surveillance du trafic maritime à l’aide des données AIS. Nous proposons une architecture d’apprentissage profond probabiliste multitâche pouvant atteindre des performances très prometteuses dans différentes tâches liées à la surveillance du trafic maritime, telles que la reconstruction de trajectoire, l’identification du type de navire et la détection d’anomalie, tout en réduisant considérablement la quantité de données à stocker et le temps de calcul. temps. Pour la tâche la plus importante - la détection d’anomalie, nous introduisons un détecteur géospatialisé qui utilise l’apprentissage profond variationnel pour construire une représentation probabiliste des trajectoires AIS, puis détecter les anomalies en jugeant la probabilité de cette trajectoire
This thesis work focuses on a class of unsupervised, probabilistic deep learning methods that use variational inference to create high capacity, scalable models for time series modelling and analysis. We present two classes of variational deep learning, then apply them to two specific problems related to the maritime domain. The first application is the identification of dynamical systems from noisy and partially observed data. We introduce a framework that merges classical data assimilation and modern deep learning to retrieve the differential equations that control the dynamics of the system. Using a state space formulation, the proposed framework embeds stochastic components to account for stochastic variabilities, model errors and reconstruction uncertainties. The second application is maritime traffic surveillance using AIS data. We propose a multitask probabilistic deep learning architecture can achieve state-of-the-art performance in different maritime traffic surveillance related tasks, such as trajectory reconstruction, vessel type identification and anomaly detection, while reducing significantly the amount data to be stored and the calculation time. For the most important task—anomaly detection, we introduce a geospatial detector that uses variational deep learning to builds a probabilistic representation of AIS trajectories, then detect anomalies by judging how likely this trajectory is

Context. A large portion of all the transportation in the world consists of voyages over the sea. Systems such as Automatic Identification Systems (AIS) have been developed to aid in the surveillance of the maritime traffic, in order to help keeping the amount accidents and illegal activities down. In recent years a lot of time and effort has gone into automated surveillance of maritime traffic, with the purpose of finding and reporting behaviour deviating from what is considered normal. An issue with many of the present approaches is inaccuracy and the amount of false positives that follow from it. Objectives. This study continues the work presented by Woxberg and Grahn in 2015. In their work they used quadtrees to improve upon the existing tool STRAND, created by Osekowska et al. STRAND utilizes potential fields to build a model of normal behaviour from received AIS data, which can then be used to detect anomalies in the traffic. The goal of this study is to further improve the system by adding statistical analysis to reduce the number of false positives detected by Grahn and Woxberg's implementation. Method. The method for reducing false positives proposed in this thesis uses the charge in overlapping potential fields to approximate a normal distribution of the charge in the area. If a charge is too similar to that of the overlapping potential fields the detection is dismissed as a false positive. A series of experiments were ran to find out which of the methods proposed by the thesis are most suited for this application.   Results. The tested methods for estimating the normal distribution of a cell in the potential field, i.e. the unbiased formula for estimating the standard deviation and a version using Kalman filtering, both find as many of the confirmed anomalies as the base implementation, i.e. 9/12. Furthermore, both suggested methods reduce the amount of false positives by 11.5% in comparison to the base implementation, bringing the amount of false positives down to 17.7%. However, there are indications that the unbiased method has more promise. Conclusion. The two proposed methods both work as intended and both proposed methods perform equally. There are however indications that the unbiased method may be better despite the test results, but a new extended set of training data is needed to confirm or deny this. The two methods can only work if the examined overlapping potential fields are independent from each other, which means that the methods can not be applied to anomalies of the positional variety. Constructing a filter for these anomalies is left for future study.

 

The international maritime industry is growing fast due to an increasing number of transportations over sea. In pace with this development, the maritime surveillance capacity must be expanded as well, in order to be able to handle the increasing numbers of hazardous cargo transports, attacks, piracy etc. In order to detect such events, anomaly detection methods and techniques can be used. Moreover, since surveillance systems process huge amounts of sensor data, anomaly detection techniques can be used to filter out or highlight interesting objects or situations to an operator. Making decisions upon large amounts of sensor data can be a challenging and demanding activity for the operator, not only due to the quantity of the data, but factors such as time pressure, high stress and uncertain information further aggravate the task. Bayesian networks can be used in order to detect anomalies in data and have, in contrast to many other opaque machine learning techniques, some important advantages. One of these advantages is the fact that it is possible for a user to understand and interpret the model, due to its graphical nature.

This thesis aims to investigate how the output from a Bayesian network can be explained to a user by first reviewing and presenting which methods exist and second, by making experiments. The experiments aim to investigate if two explanation methods can be used in order to give an explanation to the inferences made by a Bayesian network in order to support the operator’s situation awareness and decision making process when deployed in an anomaly detection problem in the maritime domain.

 

Detecting maritime anomalies is an extremely important task for maritime agencies around the globe. With the number of vessels at seas growing exponentially, the need for novel automated methods to support them with their routines and upgrade existing technologies is undeniable. MARISA, the Maritime Integrated Surveillance Awareness project, aims at fostering collaboration between 22 governmental organisations and enhance the reaction and decision-making capabilities of the maritime authorities. This work describes our contributions to the development of MARISA’s common toolkit for the detection of maritime anomalies. These efforts, as part of a Masters’ dissertation, lead to the development of the Modular Anomaly Detection Framework, MAD-F, a full data pipe-line which applies efficient and reliable routines to raw vessel navigational data in order to output potential maritime vessel anomalies. The anomalies considered for this work were defined by the experts from various maritime institutions, through MARISA, and allowed us to implement solutions given the real needs in the industry. The MADF functionalities will be validated through actual real maritime exercises. In its current state, we believe that the MAD-F is able to support maritime agencies and be integrated into their legacy systems.
Detetar anomalias marítimas é uma tarefa extremamente importante para agências marítimas á escala mundial. Com o número de embarcações em mar crescendo exponencial, a necessidade de desenvolver novas rotinas de suporte ás suas atividades e de atualizar as tecnologias existentes é inegável. MARISA, o projeto de Conscientização da Vigilância Integrada Marítima, visa fomentar a colaboração entre 22 organizações governamentais e melhorar as capacidades de reação e tomada de decisões das autoridades marítimas. Este trabalho descreve as nossas contribuições para o desenvolvimento do toolkit global MARISA, que tem como âmbito a deteção de anomalias marítimas. Estas contribuições servem como parte do desenvolvimento da Modular Anomaly Detection Framework (MAD-F), que serve como um data-pipeline completo que transforma dados de embarcações não estruturados em potenciais anomalias, através do uso de métodos eficientes para tal. As anomalias consideradas para este trabalho foram definidas através do projeto MARISA por especialistas marítimos, e permitiram-nos trabalhar em necessidades reais e atuais do sector. As funcionalidades desenvolvidas serão validadas através de exercícios marítimos reias. No estado atual do MAD-F acreditamos que este será capaz de apoiar agências marítimas, e de posteriormente ser integrado nos sistemas dos mesmos.

碩士
國立臺灣大學
地理環境資源學研究所
106
Currently, the Coast Guard Administration of the Executive Yuan carries out maritime surveillance tasks by analyzing and judging the abnormality of the ship''s movement behavior through the experience of the monitoring personnel. In the face of the continuous generation and increase of many ship movement trajectory data, the task of monitoring personnel performing marine traffic monitoring is even more onerous. On the other hand, for relatively few abnormal movements, the more difficult it is to detect and judge through human resources. However, whether the current navigation status of the ship is abnormal depends on the monitoring and analysis of the monitoring personnel throughout the entire process. If the personnel monitoring target has omissions or inexperienced situations, it may easily lead to the loss of an abnormal target, causing the decision-makers of the service unit to miss the precautions or check the deployment opportunities. Therefore, this study mainly focuses on marine ship navigation and monitoring data (large-scale mobile GIS data) and uses spatial data mining and spatial grouping techniques to establish a spatial data grouping model and analyze production. The ship''s trajectory was tracked, and the normal and abnormal tracks were judged based on the feature extraction and clustering of track data and the use of track feature space model and track simplification. During the period, we can use the algorithms developed in this research to establish methods and techniques related to Pattern Anomaly Detection. This study uses the AIS observation data around the Keelung Harbor of the National Taiwan Ocean University to conduct AIS data preprocessing in the Python programming language, importing AIS data from October to November 2017 into the PostgreSQL database, and outputting ships from the AIS position data. Using QGIS as a tool, with the help of Post-GIS function, it implements the function of extracting the feature points of the trajectory, analyzing the characteristic points of the track data, establishing the track feature space model and trajectory generalization, and establishing a vessel tracing model prototype architecture based on the AIS ship navigation data near the Keelung Harbor. In this study, AIS data in December 2017 were used to verify the model. After preliminary verification, it was confirmed that most of the ship''s track conditions are in line with the study to establish a ship track model. Therefore, this model can be used in the future to establish an abnormal trajectory model. Detection method.

For the past few years, oceans have become once again, an important means of communication and transport. In fact, traffic density throughout the globe has suffered a substantial growth, which has risen some concerns. With this expansion, the need to achieve a high Maritime Situational Awareness (MSA) is imperative. At the present time, this need may be more easily fulfilled thanks to the vast amount of data available regarding maritime traffic. However, this brings in another issue: data overload. Currently, there are so many data sources, so many data to obtain information from, that the operators cannot handle it. There is a pressing need for systems that help to sift through all the data, analysing and correlating, helping in this way the decision making process. In this dissertation, the main goal is to use different sources of data in order to detect anomalies and contribute to a clear Recognised Maritime Picture (RMP). In order to do so, it is necessary to know what types of data exist and which ones are available for further analysis. The data chosen for this dissertation was Automatic Identification System (AIS) and Monitorização Contínua das Atividades da Pesca (MONICAP) data, also known as Vessel Monitoring System (VMS) data. In order to store 1 year worth of AIS and MONICAP data, a PostgreSQL database was created. To analyse and draw conclusions from the data, a data mining tool was used, namely, Orange. Tests were conducted in order to assess the correlation between data sources and find anomalies. The importance of data correlation has never been so important and with this dissertation the aim is to show that there is a simple and effective way to get answers from great amounts of data.
Nos últimos anos, os oceanos tornaram-se, mais uma vez, um importante meio de comunicação e transporte. De facto, a densidade de tráfego global sofreu um crescimento substancial, o que levantou algumas preocupações. Com esta expansão, a necessidade de atingir um elevado Conhecimento Situacional Marítimo (CSM) é imperativa. Hoje em dia, esta necessidade pode ser satisfeita mais facilmente graças à vasta quantidade de dados disponíveis de tráfego marítimo. No entanto, isso leva a outra questão: sobrecarga de dados. Atualmente existem tantas fontes de dados, tantos dados dos quais extrair informação, que os operadores não conseguem acompanhar. Existe uma necessidade premente para sistemas que ajudem a escrutinar todos os dados, analisando e correlacionando, contribuindo desta maneira ao processo de tomada de decisão. Nesta dissertação, o principal objetivo é usar diferentes fontes de dados para detetar anomalias e contribuir para uma clara Recognised Maritime Picture (RMP). Para tal, é necessário saber que tipos de dados existem e quais é que se encontram disponíveis para análise posterior. Os dados escolhidos para esta dissertação foram dados Automatic Identification System (AIS) e dados de Monitorização Contínua das Atividades da Pesca (MONICAP), também conhecidos como dados de Vessel Monitoring System (VMS). De forma a armazenar dados correspondentes a um ano de AIS e MONICAP, foi criada uma base de dados em PostgreSQL. Para analisar e retirar conclusões, foi utilizada uma ferramenta de data mining, nomeadamente, o Orange. De modo a que pudesse ser avaliada a correlação entre fontes de dados e serem detetadas anomalias foram realizados vários testes. A correlação de dados nunca foi tão importante e pretende-se com esta dissertação mostrar que existe uma forma simples e eficaz de obter respostas de grandes quantidades de dados

The proposed research aims at contributing to advances in the anomaly detection methodologies within the framework of maritime domain, in order to improve the ability to reveal, understand, anticipate and prevent illegitimate activities at sea. This work has been developed based on three fundamental tools: a prior information from a maritime traffic graph that can be derived from a route atlas or from historical data, the Ornstein-Uhlenbeck mean reverting stochastic process to model the vessel's dynamics in deep waters, and the complete or incomplete observation of the available data from heterogeneous sensor systems. Relying on the statistical hypothesis testing framework, the work treats the problem of detecting a vessel's anomalous deviations from the expected conditions in the presence of different levels of data unavailability. The problem is further complicated by the possible falsification of dynamic data self-reported by the vessel. A worst-case scenario in terms of detection capability is finally tackled by proposing an optimization methodology to make the trajectory of a malicious vessel as stealth as possible. The effectiveness of the proposed strategies has been assessed through experimental analyses concerning both synthetic and real-world maritime operational scenarios.

Dissertação de Mestrado em Engenharia Informática apresentada à Faculdade de Ciências e Tecnologia
Há poucos anos atrás o Sistema de Identificação Automática (AIS) foi definido como o standard internacional para a comunicação entre navios com o objetivo de melhorar a segurança marítima, mas hoje em dia é utilizado para muitos mais fins porque os seus dados têm o potencial de conseguirem mapear todo o tráfego marítimo de uma determinada zona. Um desses fins é ajudar as autoridades a detetarem comportamentos anómalos através da análise dos movimentos dos navios. Desta forma, vários trabalhos científicos relacionados com dados do AIS têm sido publicados, apresentando abordagens de aprendizagem computacional e de visualização de informação, em áreas tão distintas como a extração de trajetórias, visualização de tráfego e deteção de anomalias. No entanto, considerando esta última área, apenas abordagens de aprendizagem computacional foram propostas, enquanto os trabalhos na área da visualização de informação tendem a propor representações do tráfego dos navios sem qualquer destaque aos comportamentos anómalos. Assim sendo, a presente tese tem como objetivo o desenvolvimento de estratégias de visualização capazes de identificar comportamentos anómalos, com a assistência de técnicas de análise de dados, e o teste dessas estratégias com dados AIS da zona marítima Portuguesa. Estas estratégias foram implementadas numa plataforma e incluem abordagens para uma análise geral dos dados e para a deteção de tipos específicos de comportamentos anómalos. A validação, feita através de casos de estudo, mostrou que as abordagens funcionam e que podem ser utilizadas como ferramenta de suporte aos peritos da área.
A few years ago the Automatic Identification System (AIS) was introduced as the international communication standard for vessels with the propose of improving maritime safety, but nowadays it is used for more proposes mainly because its data has the potential of mapping with detail the entire maritime traffic of an area. One of this new proposes is assisting law enforcement in detecting abnormal behaviors through movement analysis of the vessels. Because of that, several scientific works addressing AIS data have been published based on machine learning and data visualization approaches, in distinct areas such as trajectory mining, traffic visualization and anomaly detection. However, considering this last area, only machine learning approaches have been proposed, while the data visualization works tend to be focused on representing the vessel's traffic without any consideration for the anomalous behaviors. Therefore, this thesis is focused in developing visualization strategies that are able to identify these behaviors, with the assistance of data analysis, and in testing them with AIS data from the Portuguese maritime zone. These strategies were implemented on a platform and they include approaches for a general analysis of the data and for detecting specific types of anomalous behaviors. The validation, made through case studies, showed that the approaches are effective and can be used as a support tool for the domain experts.

Project Work presented as the partial requirement for obtaining a Master's degree in Information Management, specialization in Knowledge Management and Business Intelligence
Ship tracking systems allow Maritime Organizations that are concerned with the Safety at Sea to obtain information on the current location and route of merchant vessels. Thanks to Space technology in recent years the geographical coverage of the ship tracking platforms has increased significantly, from radar based near-shore traffic monitoring towards a worldwide picture of the maritime traffic situation. The long-range tracking systems currently in operations allow the storage of ship position data over many years: a valuable source of knowledge about the shipping routes between different ocean regions. The outcome of this Master project is a software prototype for the estimation of the most operated shipping route between any two geographical locations. The analysis is based on the historical ship positions acquired with long-range tracking systems. The proposed approach makes use of a Genetic Algorithm applied on a training set of relevant ship positions extracted from the long-term storage tracking database of the European Maritime Safety Agency (EMSA). The analysis of some representative shipping routes is presented and the quality of the results and their operational applications are assessed by a Maritime Safety expert.