Дисертації з теми "Multimodal information system"

The study of embodied communication requires access to multiple data sources such as multistream video and audio, various derived and meta-data such as gesture, head, posture, facial expression and gaze information. This thesis presents the data collection, annotation, and analysis for multiple participants engaged in planning meetings. In support of the analysis tasks, this thesis presents the multimedia Visualization for Situated Temporal Analysis for Macintosh (MacVisSTA) system. It supports the analysis of multimodal human communication through the use of video, audio, speech transcriptions, and gesture and head orientation data. The system uses a multiple linked representation strategy in which different representations are linked by the current time focus. MacVisSTA supports analysis of the synchronized data at varying timescales for coarse-to-fine observational studies. The hybrid architecture may be extended through plugins. Finally, this effort has resulted in encoding of behavioral and language data, enabling collaborative research and embodying it with the aid of, and interface to, a database management system.
Master of Science

Eye tracking and gestures are relatively new input methods, changing the way humans interact with computers. Gestures can be used for games or controlling a computer through an interface. Eye tracking is another way of interacting with computers, often by combining with other inputs such as a mouse or touch pad. Gestures and eye tracking have been used in commercially available products, but seldom combined to create a multimodal interaction. This thesis presents a prototype which combines eye tracking with gestures to interact with a computer. To accomplish this, the report investigates different methods of recognizing hand gestures. The aim is to combine the technologies in such a way that the gestures can be simple, and the location of a user’s gaze will decide what the gesture does. The report concludes by presenting a final prototype where the gestures are combined with eye tracking to interact with a computer. The final prototype uses an IR camera together with an eye tracker. The final prototype is evaluated with regards to learnability, usefulness, and intuitiveness. The evaluation of the prototype shows that usefulness is low, but learnability and intuitiveness are quite high.
Eye tracking och gester är relativt nya inmatningsmetoder, som förändra sättet människor interagerar med datorer. Gester kan användas för till exempel spel eller för att styra en dator via ett gränssnitt. Eye tracking är ett annat sätt att interagera med datorer, ofta med hjälp av genom att kombinera med andra styrenheter såsom en mus eller styrplatta. Gester och eye tracking har använts i kommersiellt tillgängliga produkter, men sällan kombinerats för att skapa en multimodal interaktion. Denna avhandling presenterar en prototyp som kombinerar eye tracking med gester för att interagera med en dator. För att åstadkomma detta undersöker rapporten olika metoder för att känna igen gester. Målet är att kombinera teknologierna på ett sådant sätt att gestern kan vara enkla, och platsen för användarens blick kommer bestämma vad gesten gör. Rapporten avslutas genom att presentera en slutlig prototyp där gester kombineras med eye tracking för att interagera med en dator. Den slutliga prototypen använder en IR kamera och en eye tracker. Den slutliga prototypen utvärderas med avseende på lärbarhet, användbarhet, och intuition. Utvärderingen av prototypen visar att användbarheten är låg, men både lärbarhet och intuition är ganska höga.

Thesis (MScEng)--Stellenbosch University, 2013.
ENGLISH ABSTRACT: The importance of transport should not be underestimated. Transport progresses a person’s quality of life: it connects people to one another and provides access to work, services and recreational opportunities. However, post-apartheid South Africa is unfortunately still faced with a legacy of segregation. While the less-privileged, who mostly live at the fringe of Central Business Districts (CBDs), are captive users of public transport, the private vehicle trend, under the privileged, is becoming more evident. This research project thus proposes that a balanced and integrated sustainable transport system be promoted. That is, one that will aid South Africa in growing and improving its general socio-economical status by providing all its citizens with (equal) access to a balanced transport network. It is believed that, if a multimodal system is promoted, the deficiencies of the current heterogeneous non-integrated systems may be overcome. Therefore, if South Africa’s transport network is augmented with a multimodal platform, the nation will be able to move its citizens effectively and efficiently, without jeopardising the economy, social matters and the environment, today and in the future. Moreover, South Africa will also have the necessary stimulus to utilise the already available resources at its disposal by working together as ‘one’. This research project thus stipulates a (generic) sequential approach needed in achieving an integrated (sustainable) public transport system. The goal of this research project is to create awareness of the benefits that may arise from, and the implementation steps required in obtaining, such a multimodal platform. The focus area for researching the proposition made herein is the City of Cape Town’s (CoCT’s) land transport networks. The City has a management facility, with resultant transport data repository, known as the Transport Management Centre (TMC). This TMC is regarded as one of the finest state of the art facilities in the world and the features thereof offer a sufficient base and point of departure for the promotion of a multimodal transport system. By conducting research in this field, the following portraying aspects, needed for the realisation of the proposition made herein, were found. Firstly, in order to obtain an integrated sustainable transport system, the appropriate Intelligent Transport Systems (ITS) need to be integrated. It is believed that, if an intelligent transport scheme, grounded on ITS applications, is advocated, the City will be in the position to more effectively monitor what is going on, to more accurately predict what might happen in the future, and to manage its transport system proactively on an area-wide basis. Secondly, in order to meet the integration requirements imposed by multimodal transport, a centralised database needs to be created. With such a database in place, information sharing across all modes of private- and public land transport, and thus also between the investors or the operators, will be possible. An example of such a database was developed in Microsoft Access and the modes considered therein are: MyCiti, Metrorail and Golden Arrow Bus Service (GABS). The data stored in this database is historic, but the incorporation of real time information was also catered for. Thirdly, it is believed that the success of the City’s transport system, and the development of a multimodal system, is dependent on the provision of an efficient Advanced Traveller Information System (ATIS). The idea is to promote multimodal transport as a convenient transit option by providing travellers with information on journey planning that aims to counteract their reluctance to change. In order to develop such a multimodal Journey Planner (JP), the unimodal networks considered herein were combined into a supernetwork on which Dijkstra’s Shortest Path Algorithm was applied. This algorithm was programmed in Microsoft Excel’s Visual Basic for Applications (VBA) and it incorporates the following user criteria: the origin, the destination, the user’s mode preference, and the user’s optimisation preference of either time or distance. In conclusion, it can be argued that, with information becoming such a vital commodity in everyday life, the catering for informed travellers are the key to successful future transport services. If travellers are informed about the transport networks’ performance, a positive attitude is fostered. Moreover, by providing travellers with information on journey planning, their feelings of uncertainty and fear of the unknown, that are present in (especially) public transport services, may be neutralised. This information will give the public carte blanche to make decisions that give them the perception of having more ‘control over their lives’. Therefore, if a multimodal JP that can be accessed from one portal is created, people’s inclination to acquire more information will be met. And as a result, traversing in an integrated manner may become the norm.
AFRIKAANSE OPSOMMING: Die belangrikheid van vervoer mag nie onderskat word nie. Vervoer speel ’n belangrike rol in die bevordering van ’n persoon se lewenskwaliteit: dit verbind mense met mekaar en verskaf toegang tot die werksplek, diensteverskaffers, en vryetydsbesteding. Post-apartheid Suid-Afrika is egter nog steeds vasgevang in ’n nalatenskap van rasseskeiding. Die minderbevoorregtes, wat meestal aan die buitewyke van die stad woon, is geforseerde gebruikers van openbare vervoer, terwyl die neiging (onder die bevoorregtes) om privaatvoertuie te gebruik, aan die toeneem is. Hierdie navorsingsprojek beveel dus aan dat ’n gebalanseerde, geïntegreerde en volhoubare vervoerstelsel bevorder moet word. ’n Sodanige stelsel sal help om die sosio-ekonomiese status van Suid-Afrika te bevorder. Dít kan net bereik word as alle landsburgers gelyke toegang tot ’n gebalanseeerde vervoernetwerk het. As ’n multimode-stelsel dus bevorder word, kan die tekortkominge van die huidige heterogene, nie-geïntegreerde stelsels oorkom word. Indien Suid-Afrika se vervoernetwerk ’n multimodale platform het, kan die landsburgers effektief en doeltreffend vervoer word sonder om die ekonomie, sosiale aangeleenthede of omgewing, tans en in die toekoms, in gedrang in te bring. Suid-Afrika sal boonop, met so ’n platform in plek, ook die nodige stimulus hê om die bestaande hulpbronne optimaal te gebruik. Hierdie navorsingsprojek verskaf ’n (generiese) in-volgorde-benadering om ’n volhoubare, geïntegreerde openbare vervoerstelsel daar te stel. Die doel is om bewustheid van die voordele wat uit ’n multimodale platform spruit, sowel as die nodige stappe vir die uitvoering daarvan, te kweek. Die fokusarea van die navorsing is die Stad van Kaapstad se land-vervoernetwerke. Die Stad het ’n beheerfasiliteit waar vervoerdata versamel word. Dit staan bekend as die vervoer-beheersentrum (TMC: Transport Management Centre). Hierdie fasiliteit word as toonaangewend in die wêreld beskou. Die kenmerkende eienskappe van hierdie fasiliteit bied verder ook ’n goeie vertrekpunt vir die bevordering van ’n multimodale stelsel. Die navorsing in hierdie veld het die volgende bydraende faktore, wat benodig word om die voorstelling te realiseer, geïdentifiseer. In die eerste plek moet die intelligente vervoerstelsels (ITS: Intelligent Transport Systems) geïntegreer word om ’n geïntegreerde volhoubare vervoerstelsel daar te stel. Indien ’n intelligente vervoerskema, gebaseer op tegnologiese inisiatiewe, aangemoedig word, sal die Stad van Kaapstad in die posisie wees om sy vervoerstelsel pro-aktief te bestuur deur meer effektief te monitor wat aangaan en meer akkuraat te voorspel wat in die toekoms mag gebeur. Tweedens moet daar ’n gesentraliseerde databasis geskep word. Met hierdie databasis sal die nodige integrasievereistes vir ’n multimodale vervoerstelsel, bereik word. Inligting kan dan gedeel word tussen privaat- en openbare landvervoer, asook tussen die beleggers en die operateurs van die verskillende vervoermodusse. ’n Voorbeeld van so ’n databasis is in Microsoft Access geskep en die modusse wat deel daarvan uitmaak, is: MyCiti, Metrorail en Golden Arrow Bus Services (GABS). Die data wat hierin vervat is, is histories, maar daar is vir intydse inligting voorsiening gemaak. In die derde plek is die sukses van die Stad van Kaapstad se vervoerstelsel en die ontwikkeling van ’n multimodale stelsel afhanklik van die daarstelling van ’n effektiewe, gevorderde inligtingsstelsel vir pendelaars (ATIS: Advanced Traveller Information Systems). Die idee is om ’n multimodale vervoerstelsel as ’n gerieflike opsie onder pendelaars te bevorder. Dit kan bereik word deur inligting rakende reisbeplanning aan pendelaars daar te stel. Met die verkryging van sodanige kennis sal die pendelaar se weerstand teen verandering ook afneem. Om so ’n multimodale reisbeplanner (JP: Journey Planner) te ontwikkel, is die eenmodaal-netwerke gekombineer om ’n supernetwerk te skep. Dijkstra se algoritme is op die supernetwerk toegepas. Die algoritme is in Microsoft Excel se VBA (Visual Basic for Applications) geprogrammeer en dit bevat die volgende gebruikerskriteria, nl. die begin- en eindpunt, die gebruiker se modes-voorkeur en die gebruiker se gekose optimeringsvoorkeur van tyd of afstand. Ten slotte kan gesê word dat inligting ’n groot rol in die mens se daaglikse lewe en aktiwiteite speel. Daar kan dus geredeneer word dat die sleutel tot suksesvolle vervoerdienste daarin lê om vir ingeligte pendelaars voorsiening te maak. As pendelaars ingelig is oor die stand van die vervoernetwerk maak dit hul houding meer positief. Verder, as pendelaars ook inligting oor reisbeplanning het, kan dit hul gevoel van onsekerheid en vrees jeens (veral) openbare vervoer teenwerk. Met al hierdie inligting tot sy beskikking kan ’n persoon dus sy eie keuses maak en dit lei daartoe dat die persoon meer in beheer voel. As ’n multimodale JP geskep word, voed dit die mens se begeerte vir meer inligting. En met ’n sodanige JP in plek, kan geïntegreerde pendel moontlik die norm word.

La politique des transports dans le monde et en Europe évolue vers une vision co-modale. Cette nouvelle politique n'oppose plus la voiture au transport public mais encourage une combinaison de tous les modes de transport en espérant ainsi assurer un développement rentable et durable.Nous focalisons notre étude sur le service transport de personnes qui s'inscrit au cœur des politiques co-modales en combinant tous les modes de transport en commun (métro, bus..) et promeut de nouveaux modes d'utilisation de la voiture particulière comme le covoiturage (partage d'un véhicule personnel) ou l'AutoPartage (voiture en libre-service).Toutefois, pour générer un itinéraire exploitant les services de plusieurs opérateurs de transport, il faut consulter plusieurs sites internet. Selon le déplacement à réaliser, cette tâche de planification complexe peut être très difficile à réaliser et ne garantit pas l'optimalité de l'itinéraire sélectionné.Nous nous sommes donc intéressés à la conception d'un système d'aide au déplacement capable de fournir une information voyageur (co-modale) en mettant en relation plusieurs opérateurs de transport (en commun et individuel). Le système en question doit être capable d'assister l'utilisateur dans la phase de planification par la constitution d'un carnet de voyage proposant plusieurs itinéraires multi-opérateurs. De plus, il assiste l'utilisateur en cas de perturbation en l'informant et en lui proposant des itinéraires de secours. Ce travail est basé sur des avancées technologiques qui facilitent l'optimisation dans un environnement distribué (Multi-agent - SOA) et rendent l'information accessible grâce à un grand nombre de médias (téléphone, PDA..)

Röntgenavdelningen har ett blandat flöde av patienter vilket medför att denna är en potentiell smittväg för olika sjukdomar. För att minska risken för smittspridning är det viktigt att de basala hygienrutinerna följs, dock visar studier på att följsamheten till de basala hygienrutinerna är låg. Syfte: Syftet med denna litteraturöversikt var att undersöka hur röntgensjuksköterskans följsamhet till handhygienrutiner kan förbättras. Metod: En integrativ litteraturöversikt där vetenskapliga artiklar har sökts i databaserna PubMed och CINAHL. 15 artiklar har kvalitetsgranskats, kategoriserats, analyserats och slutligen har resultatet sammanställts och en syntes skrivits. Resultat: Olika typer av interventioner, såsom utbildning, information, automatiserade observationssystem och multimodala interventioner kan vara hjälpsamma i arbetet med att förbättra följsamheten till handhygienrutinerna Slutsats: Praktisk och teoretisk utbildning, en engagerad ledning och ett klimat i kombination med god tillgång till handhygienprodukter och påminnelser om att utföra handhygien är viktiga delar i arbetet med att förbättra följsamheten till handhygienrutiner.
The radiology department has a mixed flow of patients which means that it has the potential to become a site for infection transmission. In order to reduce the risk of infection spreading it is important that the basic hand hygiene routines are followed. However, studies show that the compliance with the basic hand hygiene routines is low. Aim: The aim was to examine how the radiographer's adherence to hand hygiene practices can be improved. Method: An integrative literature review where scientific articles were searched in the PubMed and CINAHL databases. 15 articles have been quality reviewed, categorized, analyzed and finally the result has been compiled and a synthesis written. Result: Different types of interventions, such as education, information, automated observation systems and multimodal interventions, can be helpful in improving compliance with hand hygiene routines. Conclusion: Practical and theoretical education, dedicated management and a responsible climate combined with good access to hand hygiene products and reminders to perform hand hygiene are important parts of the work to improve compliance with hand hygiene routines.

Chronic pain is a complex health issue and a major cause of disability worldwide. Although multimodal rehabilitation (MMR) has been recognized as an effective form of treatment for chronic pain, some patients do not benefit from it. If treatment outcomes could be reliably predicted, then patients who would benefit more from MMR could be prioritized over others. Machine learning has been proven capable of accurately predicting outcomes in other healthcare related domains. Therefore, this study aims to investigate the use of it to predict outcomes of MMR, using data from the Swedish Quality Registry for Pain Rehabilitation (SQRP). XGBoost regression was used for this purpose, and its predictive performance was compared to Ridge regression. 12 models were trained on SQRP data for each algorithm, in order to predict pain and quality of life related outcomes. The results show similar performances for both algorithms, with mean cross-validated R² values of 0.323 and 0.321 for the XGBoost and Ridge models respectively. The average root mean squared errors of 6.744 for XGBoost and 6.743 for Ridge were similar as well. Since XGBoost performed similarly to a less computationally expensive method, the use of this method for MMR outcome prediction was not supported by the results of this study. However, machine learning has the potential to be more effective for this purpose, through the use of different hyperparameter values, correlation-based feature selection or other machine learning algorithms.

Orientador: Cláudia Maria Bauzer Medeiros
Dissertação (mestrado) - Universidade Estadual de Campinas, Instituto de Computação
Made available in DSpace on 2018-08-18T07:07:49Z (GMT). No. of bitstreams: 1 Fedel_GabrieldeSouza_M.pdf: 14390093 bytes, checksum: 63058da33a22121e927f1cdbaff297d3 (MD5) Previous issue date: 2011
Resumo: A pesquisa em computação aplicada à biodiversidade apresenta muitos desafios, que vão desde o grande volume de dados altamente heterogêneos até a variedade de tipos de usuários. Isto gera a necessidade de ferramentas versáteis de recuperação. As ferramentas disponíveis ainda são limitadas e normalmente só consideram dados textuais, deixando de explorar a potencialidade da busca por dados de outra natureza, como imagens ou sons. Esta dissertação analisa os problemas de realizar consultas multimodais a partir de predicados que envolvem texto e imagem para o domínio de biodiversidade, especificando e implementando um conjunto de ferramentas para processar tais consultas. As contribuições do trabalho, validado com dados reais, incluem a construção de uma ontologia taxonômica associada a nomes vulgares e a possibilidade de apoiar dois perfis de usuários (especialistas e leigos). Estas características estendem o escopo da consultas atualmente disponíveis em sistemas de biodiversidade. Este trabalho está inserido no projeto Bio-CORE, uma parceria entre pesquisadores de computação e biologia para criar ferramentas computacionais para dar apoio à pesquisa em biodiversidade
Abstract: Research on Computing applied to biodiversity present several challenges, ranging from the massive volumes of highly heterogeneous data to the variety in user profiles. This kind of scenario requires versatile data retrieval and management tools. Available tools are still limited. Most often, they only consider textual data and do not take advantage of the multiple data types available, such as images or sounds. This dissertation discusses issues concerning multimodal queries that involve both text and images as search parameters, for the domanin of biodiversity. It presents the specification and implementation of a set of tools to process such queries, which were validate with real data from Unicamp's Zoology Museum. The aim contributions also include the construction of a taxonomic ontology that includes species common names, and support to both researchers and non-experts in queries. Such features extend the scop of queries available in biodiversity information systems. This research is associated with the Biocore project, jointly conducted by researchers in computing and biology, to design and develop computational tools to support research in biodiversity
Mestrado
Banco de Dados
Mestre em Ciência da Computação

Bakgrund I slutet av 2019 började spridningen av SARS-CoV-2 i Kina. För a; förhindrasmi;spridning har myndigheterna i Sverige stå; inför en utmaning a; förmedlakrisinforma<on på e; effek<vt sä;. E; sä; de når många är genomsociala medier där bild och text oTa kombineras för a; förmedla e; budskap.Syfte SyTet är a; analysera hur tre regioner använt Instagram för a; förmedlainforma<on om pandemin <ll sina medborgare, samt vilken strategi somfungerat bäst.Metod Undersökningen använde intervjuer för a; se hur regionerna såg på si; arbeteoch mål med Instagram. DäreTer en enkätundersökning för a; kollamålgruppens aVtyd <ll regionernas inlägg. Inläggen jämfördes även mot enmul$modal analys för a; kunna klassificera och analysera dem.Resultat Undersökningen visade a; regionerna hade olika målgrupper och syTenmed Instagram, och a; de;a speglades i inläggen. Undersökningen visadea; vissa mul<modala u;ryck som kulturella symboler, analy$ska processeroch textuella metafunk$oner kan ha posi<v inverkan på medborgarnas upplevelseav Instagraminläggens informa<onsförmedling. Undersökningen visadeäven a; det är en utmaning för regionerna a; få medborgarna a; viljaläsa hela inläggen. I synnerhet om kulturella symbolerna används på e; felak<gt sä; eller om bildens budskap uppfa;as vara irrelevant. Undersökningenvisade a; regionerna ämnar följa myndigheternas skrivregler mena; det i prak<ken inte all<d uXörs.Slutsatser Undersökningen visade a; vissa mul<modala val kan leda <ll både posi<vaoch nega<va effekter på medborgarnas aVtyder <ll informa<onen som förmedlas.Regionerna behöver vara medvetna om a; inläggens budskap ochhur användandet av kulturella symboler kan ha en nega<v påverkan påmedborgarnas vilja a; ta <ll sig e; inläggs informa<on.
Background At the end of 2019 the spread of the SARS-CoV-2 virus began in China. Inorder to stall the spread of the virus, government agencies in Sweden havefaced a challenge with communica<ng informa<on in an effec<ve way. Oneway to reach a large number of people is through social media.Aim The study analyzed how three local government agencies had used Instagramto convey informa<on about the pandemic to their ci<zens. As well aswhat strategy has worked the best.Method Interviews were conducted in order to find out how the regions saw theirown work and goal with their use of Instagram. To accompany the interviews,an online form was used to pinpoint the target audience's aVtudetowards different types of Instagram posts. Together with a mul<modalanalysis of the posts, a triangula<on of the results was made.Results The strategies, target groups and goals with Instagram varied between theregional government agencies. Certain mul<modal strategies like culturalsymbols, analy<cal processes and textual metafunc<ons can have a posi<veimpact on the ci<zens aVtude towards the posts. It also showed the difficultywith designing informa<on for Instagram in a way that encourages citizensto read the post in its en<rety. This was especially prominent whenusing cultural symbols in a careless way or the message conveyed by theimage was perceived to be irrelevant. Finally, it showed that the local governmentsaim to follow the governmental wri<ng guidelines but don’t alwayssucceed.Conclusions The target audience and purpose of using Instagram varies between the regionalgovernment agencies and this is reflected in their mul<modal execu-<on of Instagram posts. Regional governments need to be aware that themessage of the image and the use of cultural symbols can have a nega<veimpact on the ci<zens' will to understand the content of the post.

Orientador: Ricardo da Silva Torres
Dissertação (mestrado) - Universidade Estadual de Campinas, Instituto de Computação
Made available in DSpace on 2018-08-16T05:18:58Z (GMT). No. of bitstreams: 1 Calumby_RodrigoTripodi_M.pdf: 15749586 bytes, checksum: 2493b0b703adc1973eeabf7eb70ad21c (MD5) Previous issue date: 2010
Resumo: Este trabalho apresenta uma abordagem para recuperação multimodal de imagens com realimentação de relevância baseada em programação genética. Supõe-se que cada imagem da coleção possui informação textual associada (metadado, descrição textual, etc.), além de ter suas propriedades visuais (por exemplo, cor e textura) codificadas em vetores de características. A partir da informação obtida ao longo das iterações de realimentação de relevância, programação genética é utilizada para a criação de funções de combinação de medidas de similaridades eficazes. Com essas novas funções, valores de similaridades diversos são combinados em uma única medida, que mais adequadamente reflete as necessidades do usuário. As principais contribuições deste trabalho consistem na proposta e implementação de dois arcabouços. O primeiro, RFCore, é um arcabouço genérico para atividades de realimentação de relevância para manipulação de objetos digitais. O segundo, MMRFGP, é um arcabouço para recuperação de objetos digitais com realimentação de relevância baseada em programação genética, construído sobre o RFCore. O método proposto de recuperação multimodal de imagens foi validado sobre duas coleções de imagens, uma desenvolvida pela Universidade de Washington e outra da ImageCLEF Photographic Retrieval Task. A abordagem proposta mostrou melhores resultados para recuperação multimodal frente a utilização das modalidades isoladas. Além disso, foram obtidos resultados para recuperação visual e multimodal melhores do que as melhores submissões para a ImageCLEF Photographic Retrieval Task 2008
Abstract: This work presents an approach for multimodal content-based image retrieval with relevance feedback based on genetic programming. We assume that there is textual information (e.g., metadata, textual descriptions) associated with collection images. Furthermore, image content properties (e.g., color and texture) are characterized by image descriptores. Given the information obtained over the relevance feedback iterations, genetic programming is used to create effective combination functions that combine similarities associated with different features. Hence using these new functions the different similarities are combined into a unique measure that more properly meets the user needs. The main contribution of this work is the proposal and implementation of two frameworks. The first one, RFCore, is a generic framework for relevance feedback tasks over digital objects. The second one, MMRF-GP, is a framework for digital object retrieval with relevance feedback based on genetic programming and it was built on top of RFCore. We have validated the proposed multimodal image retrieval approach over 2 datasets, one from the University of Washington and another from the ImageCLEF Photographic Retrieval Task. Our approach has yielded the best results for multimodal image retrieval when compared with one-modality approaches. Furthermore, it has achieved better results for visual and multimodal image retrieval than the best submissions for ImageCLEF Photographic Retrieval Task 2008
Mestrado
Sistemas de Recuperação da Informação
Mestre em Ciência da Computação

There has been an increasing demand for commercial organisations to foster real-time interaction with customers, because harnessing customer competencies has been shown to be a major contributor towards various benefits, such as growth, innovation and competition. This may drive organisations to embrace the development of multimodal interaction and complement Electronic Customer Knowledge Management Systems (E-CKMS) with metaphors of audio-visual nature. Although the implementation of E-CKMS encounters several challenges, such as lack of trust and information overload, few empirical studies were devoted to assess the role of audio-visual metaphors, and investigate whether these technologies can be put into practice. Therefore, this thesis describes a comparative evaluation study carried out to examine the implication of incorporating multimodal metaphors into E-CKMS interfaces on not only usability of E-CKMS, but also the user¿s trust, knowledge and acceptance. An experimental E-CKMS platform was implemented with three different modes of interaction: Visual-only E-CKMS (VCKMS) with text and graphics, Multimodal E-CKMS (MCKMS) with speech, earcons and auditory icons and Avatar-enhanced multimodal E-CKMS (ACKMS). The three platforms were evaluated by three independent groups of twenty participants each (total=60) who carried out eight common tasks of increasing complexity and design based on three different styles. Another dependent group of forty-eight participants (n=48) was instructed to interact with the systems under similar usability conditions by performing six common tasks of two styles, and fill a questionnaire devised to measure the aspects of user acceptance. The results therein revealed that ACKMS was more usable and acceptable than both MCKMS and VCKMS, whereas MCKMS was more usable than VCKMS, but less acceptable. Inferential Statistics indicated that these results were statistically significant.

There has been an increasing demand for commercial organisations to foster real-time interaction with customers, because harnessing customer competencies has been shown to be a major contributor towards various benefits, such as growth, innovation and competition. This may drive organisations to embrace the development of multimodal interaction and complement Electronic Customer Knowledge Management Systems (E-CKMS) with metaphors of audio-visual nature. Although the implementation of E-CKMS encounters several challenges, such as lack of trust and information overload, few empirical studies were devoted to assess the role of audio-visual metaphors, and investigate whether these technologies can be put into practice. Therefore, this thesis describes a comparative evaluation study carried out to examine the implication of incorporating multimodal metaphors into E-CKMS interfaces on not only usability of E-CKMS, but also the user's trust, knowledge and acceptance. An experimental E-CKMS platform was implemented with three different modes of interaction: Visual-only E-CKMS (VCKMS) with text and graphics, Multimodal E-CKMS (MCKMS) with speech, earcons and auditory icons and Avatar-enhanced multimodal E-CKMS (ACKMS). The three platforms were evaluated by three independent groups of twenty participants each (total=60) who carried out eight common tasks of increasing complexity and design based on three different styles. Another dependent group of forty-eight participants (n=48) was instructed to interact with the systems under similar usability conditions by performing six common tasks of two styles, and fill a questionnaire devised to measure the aspects of user acceptance. The results therein revealed that ACKMS was more usable and acceptable than both MCKMS and VCKMS, whereas MCKMS was more usable than VCKMS, but less acceptable. Inferential Statistics indicated that these results were statistically significant.

Le travail présenté dans ce mémoire contribue au domaine de l’entreprise étendue et le développement des systèmes d’informations distribués. et le développement des systèmes d’informations distribués. C’est bien évidemment un sujet d’étude important pour la communauté Logistique (chaîne logistique), mais aussi pour la communauté Génie logiciel. C’est dans cette perspective que se situent les objectifs de proposer une modélisation de la chaîne logistique globale dans un environnement de transport intermodal de porte à porte en vue de résoudre les problèmes : (i) d’interfaces entre les différents acteurs intervenants le long de la chaîne et (ii) de rupture des charges engendrés par les opérations de transfert de marchandises d’un mode de transport à l’autre. Ainsi, l’amélioration de performance de la chaine logistique dépend fortement du niveau de coopération et de coordination, et surtout du partage et de la validité des informations et des connaissances, entre ces différents acteurs de la chaîne (organisateur du transport, transitaire, fournisseur, etc.). L’applicatif se situe dans le corridor de la vallée de Seine. Le travail s’inscrit dans le projet APLOG (Amélioration et Performance de la LOgistique Globale) financé par la région Haute Normandie
The work presented in this thesis contributes to the field of the extended enterprise and the development of distributed information systems. This is obviously an important subject of study for the logistics community (supply chain), but also for the software engineering community. It is in this context that the objectives are to provide a model of global supply chain in an intermodal environment door to door service to solve the problems: (i) interfaces between the different actors involved along the chain and (ii) trans-loading operations generated by the goods transfer operations from one mode of transport to another. Thus, improved performance of the supply chain depends heavily on the level of cooperation and coordination, and especially sharing and validity of information and knowledge between the different actors in the chain (transport organizer, forwarding, Supplier, etc.). The application is located in the corridor of the Seine valley. The work is part of the project APLOG (Performance Improvement and global logistics) financed by the Haute Normandie region

The amount and variety of visual information presented on electronic displays is ever-increasing. Finding and acquiring relevant information in the most effective manner possible is of course desirable. While there are advantages to presenting a large number of information objects on a screen at the same time, it can also hinder fast detection of objects of interest. One way of addressing that problem is Voice Assisted Visual Search (VAVS). A user supported by VAVS calls out an object of interest and is immediately guided to the object by a highlighting cue. This thesis is an initial study of the VAVS user interface technique. The findings suggest that VAVS is a promising approach, supported by theory and practice. A working prototype shows that locating objects of interest can be sped up significantly, requiring only half the amount of time taken without the use of VAVS, on average.
Voice-Assisted Visual Search

Many countries have committed to goals of emission reductions outlined by the Paris Agreement. The actions and results seen thus far from national governments have been lackluster notwithstanding their good faith pledges. City governments have taken it upon themselves to pick up where their national governments have failed. A large portion of emissions are due to transportation. Individuals’ daily travel patterns are typically limited to within a relatively local area. As mobility patterns differ between cities, municipalities have a unique role in transitioning to a sustainable society that blanket policies by national governments cannot achieve. This results in a bottom-up approach in achieving national commitments to the Paris Agreement. In order to make effective policies and plans, municipalities should make decisions based on known information. Before reducing emissions, it must be known where and by whom the emissions are being produced. This report uses Uppsala as a case study to explore how GIS can be used to communicate and create an understanding between data scientists and politicians so that mitigation efforts can be evidencebased. The report results in several methods for visualizing personal automobile emissions based on registration data. The report continues in discussing some potential actions that can be taken to addressing the emissions from neighborhoods indicated as large contributors by the visualizations.

Building Information Modelling (BIM) as a Model-based design facilitates to investigate multiple solutions in the infrastructure planning process. The most important reason for implementing model-based design is to help designers and to increase communication between different design parties. It decentralizes and coordinates team collaboration and facilitates faster and lossless project data exchange and management across extended teams and external partners in project lifecycle. Infrastructure are fundamental facilities, services, and installations needed for the functioning of a community or society, such as transportation, roads, communication systems, water and power networks, as well as power plants. Geospatial Information Systems (GIS) as the digital representation of the world are systems for maintaining, managing, modelling, analyzing, and visualizing of the world data including infrastructure. High level infrastructure suits mostly facilitate to analyze the infrastructure design based on the international or user defined standards. Called regulation1-based design, this minimizes errors, reduces costly design conflicts, increases time savings and provides consistent project quality, yet mostly in standalone solutions. Tasks of infrastructure usually require both model based and regulation based design packages. Infrastructure tasks deal with cross-domain information. However, the corresponding data is split in several domain models. Besides infrastructure projects demand a lot of decision makings on governmental as well as on private level considering different data models. Therefore lossless flow of project data as well as documents like regulations across project team, stakeholders, governmental and private level is highly important. Yet infrastructure projects have largely been absent from product modelling discourses for a long time. Thus, as will be explained in chapter 2 interoperability is needed in infrastructure processes. Multimodel (MM) is one of the interoperability methods which enable heterogeneous data models from various domains get bundled together into a container keeping their original format. Existing interoperability methods including existing MM solutions can’t satisfactorily fulfill the typical demands of infrastructure information processes like dynamic data resources and a huge amount of inter model relations. Therefore chapter 3 concept of infrastructure information modelling investigates a method for loose and rule based coupling of exchangeable heterogeneous information spaces. This hypothesis is an extension for the existing MM to a rule-based Multimodel named extended Multimodel (eMM) with semantic rules – instead of static links. The semantic rules will be used to describe relations between data elements of various models dynamically in a link-database. Most of the confusion about geospatial data models arises from their diversity. In some of these data models spatial IDs are the basic identities of entities and in some other data models there are no IDs. That is why in the geospatial data, data structure is more important than data models. There are always spatial indexes that enable accessing to the geodata. The most important unification of data models involved in infrastructure projects is the spatiality. Explained in chapter 4 the method of infrastructure information modelling for interoperation in spatial domains generate interlinks through spatial identity of entities. Match finding through spatial links enables any kind of data models sharing spatial property get interlinked. Through such spatial links each entity receives the spatial information from other data models which is related to the target entity due to sharing equivalent spatial index. This information will be the virtual properties for the object. The thesis uses Nearest Neighborhood algorithm for spatial match finding and performs filtering and refining approaches. For the abstraction of the spatial matching results hierarchical filtering techniques are used for refining the virtual properties. These approaches focus on two main application areas which are product model and Level of Detail (LoD). For the eMM suggested in this thesis a rule based interoperability method between arbitrary data models of spatial domain has been developed. The implementation of this method enables transaction of data in spatial domains run loss less. The system architecture and the implementation which has been applied on the case study of this thesis namely infrastructure and geospatial data models are described in chapter 5. Achieving afore mentioned aims results in reducing the whole project lifecycle costs, increasing reliability of the comprehensive fundamental information, and consequently in independent, cost-effective, aesthetically pleasing, and environmentally sensitive infrastructure design.:ABSTRACT 4 KEYWORDS 7 TABLE OF CONTENT 8 LIST OF FIGURES 9 LIST OF TABLES 11 LIST OF ABBREVIATION 12 INTRODUCTION 13 1.1. A GENERAL VIEW 14 1.2. PROBLEM STATEMENT 15 1.3. OBJECTIVES 17 1.4. APPROACH 18 1.5. STRUCTURE OF THESIS 18 INTEROPERABILITY IN INFRASTRUCTURE ENGINEERING 20 2.1. STATE OF INTEROPERABILITY 21 2.1.1. Interoperability of GIS and BIM 23 2.1.2. Interoperability of GIS and Infrastructure 25 2.2. MAIN CHALLENGES AND RELATED WORK 27 2.3. INFRASTRUCTURE MODELING IN GEOSPATIAL CONTEXT 29 2.3.1. LamdXML: Infrastructure Data Standards 32 2.3.2. CityGML: Geospatial Data Standards 33 2.3.3. LandXML and CityGML 36 2.4. INTEROPERABILITY AND MULTIMODEL TECHNOLOGY 39 2.5. LIMITATIONS OF EXISTING APPROACHES 41 INFRASTRUCTURE INFORMATION MODELLING 44 3.1. MULTI MODEL FOR GEOSPATIAL AND INFRASTRUCTURE DATA MODELS 45 3.2. LINKING APPROACH, QUERYING AND FILTERING 48 3.2.1. Virtual Properties via Link Model 49 3.3. MULTI MODEL AS AN INTERDISCIPLINARY METHOD 52 3.4. USING LEVEL OF DETAIL (LOD) FOR FILTERING 53 SPATIAL MODELLING AND PROCESSING 58 4.1. SPATIAL IDENTIFIERS 59 4.1.1. Spatial Indexes 60 4.1.2. Tree-Based Spatial Indexes 61 4.2. NEAREST NEIGHBORHOOD AS A BASIC LINK METHOD 63 4.3. HIERARCHICAL FILTERING 70 4.4. OTHER FUNCTIONAL LINK METHODS 75 4.5. ADVANCES AND LIMITATIONS OF FUNCTIONAL LINK METHODS 76 IMPLEMENTATION OF THE PROPOSED IIM METHOD 77 5.1. IMPLEMENTATION 78 5.2. CASE STUDY 83 CONCLUSION 89 6.1. SUMMERY 90 6.2. DISCUSSION OF RESULTS 92 6.3. FUTURE WORK 93 BIBLIOGRAPHY 94 7.1. BOOKS AND PAPERS 95 7.2. WEBSITES 101

Dissertação de Mestrado em Tecnologias de Informação Geográfica apresentada à Faculdade de Ciências e Tecnologia
Os Sistemas de Informação Geográfica (SIG) são sistemas computacionais para capturar, armazenar, consultar, analisar e exibir dados geoespaciais e as suas características (Chang, 2010). A partir da massificação desta tecnologia e da forte componente espacial dos transportes, conciliou-se os SIG aos transportes, dando origem aos Sistemas de Informação Geográfica para os transportes (SIG-T), com metodologias e algoritmos específicos para a análise de redes.Parafraseando Silva (2006), “os SIG-T possibilitam análises integradas de todas as componentes de um sistema de transportes no seu contexto geográfico”.A aplicação dos SIG aos transportes, tem crescido nos últimos anos, principalmente como ferramenta para o auxílio a tomadas de decisão, análise, monitorização e gestão dos transportes de forma a facilitar a circulação de pessoas, bens e informação, garantindo o acesso à mobilidade.A grande premissa deste trabalho é o de automatizar o processo de planear uma viagem utilizando a rede dos transportes públicos (rede multimodal) a nível intermunicipal, disponibilizando a informação ao público com auxílio das funcionalidades dos SIG. Assim, pretende-se dar resposta à debilidade no planeamento de viagens intermunicipais - conciliando o transporte rodoviário (urbano e regional), ferroviário e rede pedonal - com o propósito de melhorar a acessibilidade.No que respeita ao caso de estudo, foi definida a Região de Coimbra que é constituída por dezanove concelhos, com uma rede multimodal consideravelmente extensa, uma vez que não existem plataformas que possibilitem o planeamento de viagens complexas (com mais do que um modo de transporte) de forma adequada por parte do utilizador, nem o planeamento de viagens intermunicipais na região.Para a simulação da rede multimodal e intermunicipal, foram definidos quatro cenários representativos do nosso caso de estudo que têm como origem quatro localizações diferentes, pertencentes à Região de Coimbra, e como destino o Hospital Central da Universidade de Coimbra. A informação resultante da criação de rotas conforme as preferências definidas pelo utilizador, é o tempo da viagem, a tarifa associada (€), a distância total, a distância pedestre, o modo de transporte, as paragens/estações, o percurso até aceder ao transporte, o percurso do transporte, os transbordos propostos, o tempo de espera e os horários, para uma viagem que corresponda às necessidades do utilizador.
Geographic Information Systems (GIS) are computer systems for capturing, storing, querying, analyzing and displaying geospatial data and its characteristics (Chang, 2010). From the massification of this technology and the strong spatial component of transport, GIS was reconciled with transport, rising the Geographic Information Systems for Transport (GIS-T), with methodologies and specific algorithms for the analysis of networks.Paraphrasing Silva (2006), "GIS-T enable integrated analysis of all components of a transport system in its geographic context."The application of GIS to transport has grown in recent years, mainly as a tool to aid decision-making, analysis, monitoring and transport management to facilitate the movement of people, goods and information, ensuring access to mobility.The main premise of this work is to automate the process of planning a trip using the public transport network (multimodal network) at intermunicipal level, making information available to the public with the help of the GIS functionalities. Aims to respond to the weakness of planning an intermunicipal travel - reconciling road transport (urban and regional), rail and pedestrian network - with the aim of improving accessibility.Regarding the case study, was chosen the Region of Coimbra that as nineteen municipalities, with a considerable multimodal network, since there are no platforms that allow the planning of complex trips (with more than one mode of transportation) in an appropriate way on the part of the user, neither the planning of intermunicipal trips in the region.For the simulation of the multimodal and intermunicipal network, four representative scenarios of our case study were defined that have as origin four different locations, belonging to the Region of Coimbra, and as destination the Hospital Central da Universidade de Coimbra. The information resulting from the creation of routes according to the preferences defined by the user, is the travel time, the associated fare (€), the total distance, the pedestrian distance, the mode of transportation, the stops/stations, the route until access transport, the transport route, the proposed trans-shipments, the waiting time and the timetable, for a journey that corresponds to the needs of the user.