Rozprawy doktorskie na temat „Multimodality”

PONTIFÍCIA UNIVERSIDADE CATÓLICA DO RIO DE JANEIRO
Os livros didáticos modernos constituem-se num objeto de estudo valioso para o lingüista por serem elaborados a partir de múltiplas modalidades semióticas e porque, em geral, são gêneros disciplina-específicos. Entretanto, no que tange à disciplina de Biologia, ainda há poucos estudos que exploram a interação entre as modalidades verbal e visual nos livros de Ensino Médio e, até o momento, nenhum trabalho de natureza contrastiva foi realizado no Brasil sobre esse aspecto. Na presente análise, investigam-se os tipos de representação visual e o modo de estabelecimento da relação entre figuras e o texto verbal associado em dois livros didáticos de Biologia do Ensino Médio, um brasileiro e outro norteamericano. Este estudo baseia-se principalmente nos pressupostos teóricos de Kress e van Leeuwen (1996, 2001) e inspira-se nos trabalhos de Myers (1997) e Nascimento (2002). Implícita nesses pressupostos está a abordagem sistêmicofuncional de Halliday e Hasan (1976) e de Halliday (1994). Os resultados desta pesquisa revelam que, dentre outros mecanismos, a coesão lexical exerce importante papel na interação entre o componente verbal das figuras, o texto principal e as legendas em ambos os livros analisados. Dentre as diferenças encontradas nos dois livros, destacam-se as representações visuais distintas referentes a um mesmo tópico, o modo de referenciação às figuras no texto, a função das legendas e a proporção de utilização de termos técnicos na coesão intermodal. Esses resultados mostram que existem variações no gênero livro didático de Biologia quanto ao modo de utilização dos mecanismos de interação entre figuras e texto verbal, que podem estar relacionadas com os propósitos dos autores. Isso pode ter importantes implicações para o ensino da Biologia e de Inglês para fins específicos, pois auxilia na conscientização de educadores e alunos sobre a necessidade de se levar em consideração os meios de interação entre as linguagens visual e verbal na interpretação e produção dos textos multimodais próprios de cada comunidade discursiva.
Modern textbooks are valuable as a research object in applied linguistics because they usually present their content through multiple semiotic modalities and because in general they are discipline-specific genres. However, concerning the discipline of Biology, there are still few studies that exploit the interaction between verbal and visual languages in highschool textbooks and, up to the moment, no contrastive works on this topic have been published in Brazil. For this reason, the goal of the present study is to investigate the types of visual representations and the strategies that establish the interaction between pictures and verbal text used in two highschool-level Biology textbooks, a Brazilian and a North-American one. This study is based mainly on Kress and van Leeuwen s (1996, 2001) theory of multimodality and is inspired in Myers (1997) and Nascimento s (2002) previous works. This theoretical basis presupposes Halliday and Hasan s (1976) and Halliday s (1997) systemic- functional approach to text. The results of this investigation reveal that, among other mechanisms, lexical cohesion has an important role in the interaction between the verbal element in images and the verbal language of the corresponding texts and captions in both books. Among the differences found between these two books, there are the different visual representations concerning the same topic, the way pictures are referred to in the text, the role of the captions and the proportion of technical terms used in intermodal cohesion. These results show that there are intrageneric variations in the use of such resources that are probably due to the authors purposes. The findings may have important implications for the teaching of Biology and of English for specific purposes. They may enhance the awareness of educators and students about the need of taking the interaction between visual and verbal languages into consideration for the appropriate interpretation of texts and for the production of multimodal texts specific to each discursive community.

PONTIFÍCIA UNIVERSIDADE CATÓLICA DO RIO DE JANEIRO
CONSELHO NACIONAL DE DESENVOLVIMENTO CIENTÍFICO E TECNOLÓGICO
Constatada uma lacuna no campo da tradução audiovisual (TAV) em relação ao tratamento teórico da multiplicidade de elementos semióticos que caracteriza os produtos audiovisuais, esta dissertação busca um caminho teórico para discutir a interpretação de imagens visuais (imagens que se apresentam de forma visível e concreta, por meio de algum suporte material), articulando os conceitos de discurso multimodal e comunidades interpretativas. Parte-se do pressuposto de que textos podem ser compostos por diferentes recursos semióticos além da linguagem verbal, como imagens, música, dança etc., os quais interagem entre si sem que haja uma relação hierárquica entre eles. O foco é lançado sobre os usos que se fazem das imagens nas práticas sociais e discursivas contemporâneas, os diferentes formatos em que elas se apresentam e as maneiras como circulam na sociedade, ressaltando o papel central da visão. A imagem visual é discutida a partir da distinção entre visão como operação física/fisiológica e visualidade como determinações sociais e discursivas que regulam essa operação. Tal distinção não implica uma relação dicotômica, mas uma diferenciação entre o mecanismo e os dados da visão e regras socialmente compartilhadas para a interpretação desses dados.
After acknowledging a gap in the field of audiovisual translation (AVT) regarding the theoretical approach of the multiplicity of semiotic elements that compose audiovisual products, this thesis searches for a theoretical path to discuss the interpretation of visual images (images that have a concrete visible form), articulating the concepts of multimodal discourse and interpretive communities. The premise is that texts are composed by different semiotic resources besides verbal language such as images, music, dance etc., which interact without having a hierarchical relationship. The focus is cast on the uses of images in social and discursive practices, the different formats in which they occur and the ways they circulate in society, highlighting the central role of vision. The discussion on visual images is based on the distinction between vision as a physical/physiological operation and visuality as social and discursive determinations that regulate that operation. Such distinction does not imply a dichotomic relationship, but a difference between the mechanism and data of vision and the socially shared rules for their interpretation.

O transporte é geralmente o elemento mais importante nos custos logísticos, para a maioria das empresas. A movimentação de fretes absorve, segundo Ballou (1993), entre um e dois terços do total de custos logísticos. Dentro desse panorama desenvolveu-se este estudo sobre multimodalidade. O foco está nas instalações e serviços que compõem o sistema de transporte multimodal, com especial destaque para a infra-estrutura portuária, nas taxas (custos) e no desempenho dos vários serviços envolvidos. O principal objetivo do trabalho é compreender como o processo de privatização das estruturas de transporte, portos, ferrovias e rodovias, viabilizou a evolução da Multimodalidade como alternativa ao transporte nacional. Um assunto relativamente novo, ainda pouco explorado academicamente e com escassas referencias bibliográficas. Para que se tenha um entendimento de como o processo de privatização transformou as estruturas de transporte, especialmente a atividade portuária brasileira, viabilizando o transporte multimodal e obviamente as vantagens dele decorrentes, fez- se necessária a abordagem de temas congruentes, como a logística e também os cenários dos modos de transporte rodoviário, ferroviário e o marítimo (cabotagem). Nesta fase também se analisam os modos de transporte e os respectivos processos de privatização e cenário atual, de suas estruturas e os operadores logísticos, assuntos que contribuem para um melhor entendimento das questões relativas ao Operador de Transporte Multimodal.
Inside of this panorama this study was developed on multimodality. The focus is in the installations and services that compose the system of multimodal transport, with special prominence for the port infrastructure, in the taxes (costs) and the performance of the some involved services. The main objective of the work is to understand as the process of privatization of the transport structures, ports, railroads and highways, it made possible the evolution of Multimodalidade as alternative to the national transport. A relatively new subject, still little explored academically and with scarce bibliographical references. For a well understanding as the privatization process transformed the transport structures, especially the Brazilian port activity, making possible the multimodal transport and obviously the decurrently advantages of the boarding of congruence´s subjects became necessary, as logistic and also the scenes in the ways of road transport, railroad worker and the marine (cabotage). In this phase also the ways of transport and the respective processes of privatization and current scene are analyzed, of its structures and the logistic operators, subjects that contribute for one better agreement of the relative questions the Operator of Multimodal Transport.

Segmentation is a hot issue in the domain of medical image analysis. It has a wide range of applications on medical research. A great many medical image segmentation algorithms have been proposed, and many good segmentation results were obtained. However, due to the noise, density inhomogenity, partial volume effects, and density overlap between normal and abnormal tissues in medical images, the segmentation accuracy and robustness of some state-of-the-art methods still have room for improvement. This thesis aims to deal with the above segmentation problems and improve the segmentation accuracy. This project investigated medical image segmentation methods across a range of modalities and clinical applications, covering magnetic resonance imaging (MRI) in brain tissue segmentation, MRI based multiple sclerosis (MS) lesions segmentation, histology based cell nuclei segmentation, and positron emission tomography (PET) based tumour detection. For the brain MRI tissue segmentation, a method based on mutual information was developed to estimate the number of brain tissue groups. Then a unsupervised segmentation method was proposed to segment the brain tissues. For the MS lesions segmentation, 2D/3D joint histogram modelling were proposed to model the grey level distribution of MS lesions in multimodality MRI. For the PET segmentation of the head and neck tumours, two hierarchical methods based on improved active contour/surface modelling were proposed to segment the tumours in PET volumes. For the histology based cell nuclei segmentation, a novel unsupervised segmentation based on adaptive active contour modelling driven by morphology initialization was proposed to segment the cell nuclei. Then the segmentation results were further processed for subtypes classification. Among these segmentation approaches, a number of techniques (such as modified bias field fuzzy c-means clustering, multiimage spatially joint histogram representation, and convex optimisation of deformable model, etc.) were developed to deal with the key problems in medical image segmentation. Experiments show that the novel methods in this thesis have great potential for various image segmentation scenarios and can obtain more accurate and robust segmentation results than some state-of-the-art methods.

Comme l’un des plus importants canaux de communication non verbale, le toucher est largement utilisé à des fins différentes. Dans le développement physique et psychologique humain le toucher participe à la construction des liens sociaux et à la communication des émotions. Cependant, même si les technologies de l'information et de la communication actuels permettent l’utilisation de divers langages non verbaux, le soutien à la communication par le sens du toucher est encore insuffisant. Inspiré par les interactions intermodales dans la perception humaine, l’approche que j'adopte dans cette thèse est d’utiliser la multimodalité pour améliorer l’interaction tactile médiée. Suivant cette approche, je présente trois dispositifs qui fournissent des contributions empiriques à l’interaction tactile multimodaux : VisualTouch, SansTouch, et In-Flat. Afin de comprendre si les stimuli multimodaux peuvent améliorer la perception émotionnelle du toucher, je développe VisualTouch, et j'évalue quantitativement l’interaction entre la modalité visuelle et tactile. Pour étudier l’utilisation de différentes modalités dans la communication tactile, je présente SansTouch et fournis des aperçus empiriques sur l’interaction multimodale et la génération de contact cutané dans le contexte de la communication en face à face. Enfin, pour aller plus loin dans l’utilisation de stimuli multimodaux en interaction tactile, je présente In-Flat, une superposition tactile entrée/sortie pour smartphones. In-Flat fournit non seulement des informations supplémentaires sur la génération du toucher de la peau, mais aussi une meilleure compréhension du rôle que joue le toucher médié dans des contextes plus généraux. En résumé, cette thèse vise à combler le fossé entre la communication tactile et l’IHM, en contribuant à la conception et la compréhension des stimuli multimodaux dans l’interaction tactile médiée
As one of the most important non-verbal communication channels, touch is widely used for different purposes. It is a powerful force in human physical and psychological development, shaping social structures as well as communicating emotions. However, even though current information and communication technology (ICT) systems enable the use of various non-verbal languages, the support of communicating through the sense of touch is still insufficient. Inspired by the cross-modal interaction of human perception, the approach I present in this dissertation is to use multimodality to improve mediated touch interaction. Following this approach, I present three devices that provide empirical contributions to multimodal touch interaction: VisualTouch, SansTouch, and In-Flat. To understand if multimodal stimuli can improve the emotional perception of touch, I present the VisualTouch device, and quantitatively evaluate the cross-modal interaction between the visual and tactile modality. To investigate the use of different modalities in real touch communication, I present the SansTouch device, which provides empirical insights on multimodal interaction and skin-like touch generation in the context of face-to-face communication. Going one step forward in the use of multimodal stimuli in touch interaction, I present the In-Flat device, an input/output touch overlay for smartphones. In-Flat not only provides further insights on the skin-like touch generation, but also a better understanding of the role that mediated touch plays in more general contexts. In summary, this dissertation strives to bridge the gap between touch communication and HCI, by contributing to the design and understanding of multimodal stimuli in mediated touch interaction

Age and hypertension lead to aortic remodeling and stiffness and are also major risk factors for abdominal aortic aneurysms (AAA). This study aimed to investigate: (i) the use of multimodal imaging to test the hypothesis that aneurysmal disease of the abdominal aorta is a remodeling response to aortic stiffness and systolic hypertension; (ii) the utility of a novel ultrasound-based device (AortaScan) for detection of AAA in the community setting. We used multimodality imaging tonometry and cardiovascular magnetic resonance imaging (CMR) to quantify pulse wave velocity (PWV, a measure of stiffness), in the aortic arch (Arch), thoracic aorta (TA) and the abdominal aorta (AA). Stiffness was also correlated with measures of calcification and metabolic activity measured on CT and PET/CT respectively. The thoracic aortae of patients with AAA were stiffer than those of sex matched controls. Although systolic hypertension was more common in AAA patients, multivariate analysis revealed that aortic stiffness and mean arterial pressure were associated with AAA disease. The likelihood of developing AAA disease increases >3-fold for 1m/sec increase in PWV. This data suggests that segmental stiffness is modified in the presence of AAA and provides further evidence that aneurysm formation may be an adaptive remodeling response to hypertension. The AortaScan can detect AAA without the need for a trained operator and has potential in a community-based screening programme. It would, however, need further technical improvement to increase sensitivity before it could be considered a replacement for trained screening personnel.

Dans cette thèse, nous discutons du stress, de l'incertitude et de la multimodalité des mesures de risque en accordant une attention particulière à deux parties. Les résultats ont une influence directe sur le calcul du capital économique et réglementaire des banques. Tout d'abord, nous fournissons une nouvelle mesure de risque - la VaR du stress du spectre (SSVaR) - pour quantifier et intégrer l'incertitude de la valeur à risque. C'est un modèle de mise en œuvre de la VaR stressée proposée par Bâle III. La SSVaR est basée sur l'intervalle de confiance de la VaR. Nous étudions la distribution asymptotique de la statistique de l'ordre, qui est un estimateur non paramétrique de la VaR, afin de construire l'intervalle de confiance. Deux intervalles de confiance sont obtenus soit par le résultat gaussien asymptotique, soit par l'approche saddlepoint. Nous les comparons avec l'intervalle de confiance en bootstrapping par des simulations, montrant que l'intervalle de confiance construit à partir de l'approche saddlepoint est robuste pour différentes tailles d'échantillons, distributions sous-jacentes et niveaux de confiance. Les applications de test de stress utilisant SSVaR sont effectuées avec des rendements historiques de l'indice boursier lors d'une crise financière, pour identifier les violations potentielles de la VaR pendant les périodes de turbulences sur les marchés financiers. Deuxièmement, nous étudions l'impact de la multimodalité des distributions sur les calculs de la VaR et de l'ES. Les distributions de probabilité unimodales ont été largement utilisées pour le calcul paramétrique de la VaR par les investisseurs, les gestionnaires de risques et les régulateurs. Cependant, les données financières peuvent être caractérisées par des distributions ayant plus d'un mode. Avec ces données nous montrons que les distributions multimodales peuvent surpasser la distribution unimodale au sens de la qualité de l'ajustement. Deux catégories de distributions multimodales sont considérées: la famille de Cobb et la famille Distortion. Nous développons un algorithme d'échantillonnage de rejet adapté, permettant de générer efficacement des échantillons aléatoires à partir de la fonction de densité de probabilité de la famille de Cobb. Pour une étude empirique, deux ensembles de données sont considérés: un ensemble de données quotidiennes concernant le risque opérationnel et un scénario de trois mois de rendement du portefeuille de marché construit avec cinq minutes de données intraday. Avec un éventail complet de niveaux de confiance, la VaR et l'ES à la fois des distributions unimodales et des distributions multimodales sont calculés. Nous analysons les résultats pour voir l'intérêt d'utiliser la distribution multimodale au lieu de la distribution unimodale en pratique
In this thesis, we focus on discussing the stress, uncertainty and multimodality of risk measures with special attention on two parts. The results have direct influence on the computation of bank economic and regulatory capital. First, we provide a novel risk measure - the Spectrum Stress VaR (SSVaR) - to quantify and integrate the uncertainty of the Value-at-Risk. It is an implementation model of stressed VaR proposed in Basel III. The SSVaR is based on the confidence interval of the VaR. We investigate the asymptotic distribution of the order statistic, which is a nonparametric estimator of the VaR, in order to build the confidence interval. Two confidence intervals are derived from either the asymptotic Gaussian result, or the saddlepoint approach. We compare them with the bootstrapping confidence interval by simulations, showing that the confidence interval built from the saddlepoint approach is robust for different sample sizes, underlying distributions and confidence levels. Stress testing applications using SSVaR are performed with historical stock index returns during financial crisis, for identifying potential violations of the VaR during turmoil periods on financial markets. Second, we investigate the impact of multimodality of distributions on VaR and ES calculations. Unimodal probability distributions have been widely used for parametric VaR computation by investors, risk managers and regulators. However, financial data may be characterized by distributions having more than one modes. For these data, we show that multimodal distributions may outperform unimodal distribution in the sense of goodness-of-fit. Two classes of multimodal distributions are considered: Cobb's family and Distortion family. We develop an adapted rejection sampling algorithm, permitting to generate random samples efficiently from the probability density function of Cobb's family. For empirical study, two data sets are considered: a daily data set concerning operational risk and a three month scenario of market portfolio return built with five minutes intraday data. With a complete spectrum of confidence levels, the VaR and the ES from both unimodal distributions and multimodal distributions are calculated. We analyze the results to see the interest of using multimodal distribution instead of unimodal distribution in practice

The purpose of this study was to explore the use of a blog in the classroom. This paper describes why the teacher created the blog and how the blog is used by teachers and pupils in and outside the classroom. The study also describes teachers’ thoughts regarding future use of the blog. Seven primary school teachers were interviewed and the material was analysed using a multimodal design theoretical perspective on learning (Selander & Kress 2010). The conclusion of the study was that the teachers created and designed blogs to give parents, pupils, relatives and others an opportunity to have an insight into the pupils’ school day. The blog is used mostly by teachers and pupils’ school work is published on the blog. The pupils in some classes use the blog to write weekly journals, make films and interviews that are published. Several teachers mention that publishing school work on the blog improves the quality of the school work. It seems that pupils try harder when the school work is published on the blog. Some teachers mention pupils’ age and their ability to write when discussing pupils posting work on the blog. The blog is though a multimodal way to communicate. It seems that the blog has the ability to support both pupils’ writing development and their competencies in multimodal communication. Regarding future use of the blog the teachers have many ideas and all teachers agree that pupils will be given the opportunity to use the blog more in the future. The ideas of future use can be related to the idea of the blog as a virtual third space, a link between home and school. In this third space pupils are able to use knowledge from home when carrying out school tasks.   Since there are few studies regarding blogs in the classroom and especially regarding younger pupils, this study has given a wider perspective regarding the use of blogs in the classroom.

This essay aims to compare two EFL (English as a Foreign Language) textbooks by using a multimodal discourse analysis in order to find out how EFL textbooks have changed in design and visually. In this essay the textual content is treated as one of several pillars making up design, this essay is interested in the visual changes. This analysis is done using two EFL textbooks with twenty years in between them, both are used in the same school by different teachers. A study like this is going to be published later this year but that study includes three subjects (English being one of them) and starts with textbooks from the 1930s up until now. In this essay, two chapters of each textbook will be looked at  in a closer analysis to represent each textbook; every page is analyzed without first reading the text. The conclusion of this essay is that the written communication still seem to be the most credible of the different communicative methods of making meaning, however, it is no longer the only credible way of making meaning.

L’hypothèse prédominante de cette thèse est que les changements métaboliques tumoraux mesurés par FDG-PET/CT sous l’influence des traitements anticancéreux, apparaissent plus précocement et parfois exclusivement par rapport aux modalités d’imagerie morphologique classique. L’imagerie multimodale, en combinant les avantages de chacune des techniques, dépasse leur limitations et pourrait permettre (i) une évaluation du bénéfice du traitement plus rapide et plus adéquate ;(ii) de modifier les algorithmes thérapeutiques à différents stades de cancer colorectal et (iii) d’améliorer la compréhension des mécanismes d’échappement aux traitements anticancéreux. Pour évaluer l’apport de l’imagerie multimodale dans l’évaluation de la réponse au traitement des cancers colorectaux (CCR), nous avons poursuivi 3 séries d’expérimentation cliniques.

1) Le premier projet explore l’imagerie multimodale comme un outil d’individualisation pour la radio-embolisation (microsphères chargées en 90Yttrium) chez des patients porteurs d’un CCR métastatique au niveau du foie, pour laquelle l’imagerie morphologique classique est incapable de mesurer l’effet thérapeutique. Nous montrons que l’usage non sélectif de la radio-embolisation améliore l’histoire clinique de ces patients, bien que certains d’entre eux ne semblent pas en bénéficier. Ensuite, par une analyse multimodale lésion par lésion intégrant angiographie-CT Scan, FDG-PET/CT et scintigraphie aux macro-agrégats d’albumine marqués au 99mTechnetium, nous démontrons que la distribution pré-thérapeutique des macro-agrégats d’albumine est hétérogène entre les différentes lésions des patients et prédictive de la réponse métabolique au sein de ces lésions, permettant le développement d’un outil de prédiction et de planification pour la radio-embolisation.

2) Le deuxième projet explore le domaine du CCR métastatique traité par chimiothérapie palliative. (i) Nous démontrons d’abord que la réponse métabolique (RM) tumorale après une cure de chimiothérapie cytolytique prédit plus vite et plus adéquatement que l’imagerie morphologique basée sur les critères RECIST les bénéfices cliniques du traitement. La RM précoce a une excellente valeur prédictive négative sur l’absence de réponse morphologique et met en évidence une variabilité de réponse inter-lésionnelle chez une proportion importante des patients. (ii) L’étude SoMore explore ensuite des patients présentant un CCR avancé et réfractaire, traités par capecitabine et sorafenib, et confirme l’importance pronostique des RM mixtes, suggérant une méthodologie de classification clinique basée sur la consistance de la RM. (iii) Cette classification cherche confirmation dans l’étude RegARd-C, encore en cours, évaluant les effets du regorafenib, et explorant également la signification génomique et épigénétique de la variabilité de RM.

3) Le troisième projet cherche à utiliser les propriétés de l’imagerie métabolique pour modifier l’algorithme de traitement adjuvant des patients porteurs d’un cancer du côlon de stade III. Ce projet, encore en cours, fait l’hypothèse que l’absence de RM de la lésion primitive après une cure de chimiothérapie prédit l’absence de bénéfice du traitement adjuvant complet. Une analyse intérimaire en démontre la faisabilité et confirme la présence de 40% de tumeurs présentant des caractéristiques métaboliques de chimio-résistance.

En conclusion, pour des patients porteurs d’un CCR, l’imagerie multimodale comprenant une évaluation du métabolisme tumoral permet une évaluation plus précoce et plus adéquate du bénéfice au traitement anticancéreux pour différentes modalités thérapeutiques comme la radio-embolisation, la chimiothérapie cytotoxique et les agents biologiques. L’imagerie multimodale permet de prédire et planifier les radio-embolisations et se révèle très prometteuse pour les traitements chimiothérapiques cytotoxiques ou combinés à des biologiques en situation adjuvante ou métastatique. Elle démontre par ailleurs une importante variabilité de réponse métabolique inter-lésionnelle qui représente un axe de recherche majeur sur les mécanismes moléculaires d’hétérogénéité génomique tumorale et de résistance aux traitements anti-cancéreux.

Doctorat en Sciences médicales
info:eu-repo/semantics/nonPublished

Studien syftar till att få en djupare förståelse för hur sångare använder och upplever olika verktyg för att forma och kommunicera musik, vilka verktyg de prioriterar samt vilken funktion dessa verktyg fyller. För att undersöka detta har jag utgått från olika teorier om musikalisk kommunikation samt ett multimodalt och socialsemiotiskt perspektiv med fokus på kommunikation som en social process av teckenskapande. Jag har genomfört åtta kvalitativa intervjuer med sångare som arbetar/arbetat professionellt med sång inom olika genrer. Resultaten visar att sångarnas musikaliska kommunikation formas och uttrycks genom ett användande av flera olika verktyg varav röstteknik, samspel, publikens förväntningar, personlighet och tradition är de viktigaste. Sångarnas användande och prioritering av dessa olika verktyg kan sammanfattas i olika övergripande förhållningssätt som sångarna har till sin musikaliska kommunikation, varav tradition och personlighet är de mest framträdande förhållningssätten. En slutsats är att en sångares musikaliska uttryck kan beskrivas som en väv av kommunikativa strukturer och att sångare strävar efter att gestalta helheter av dessa strukturer för att uppleva mening. En annan slutats är att sångare kan betona olika aspekter av den musikaliska kommunikationen vid olika tillfällen i musicerandet, att den sker på flera plan samtidigt samt att musikens sammanhang avgör hur den musikaliska gestaltningen formas. Jag vill slutligen tillägga att sångarens gestaltande av musikaliska helheter sker genom att skapa, tolka och imitera olika resurser i syfte att skapa en meningsfull kommunikation för både publik, medspelare samt med sångaren själv.
The study aims to demonstrate a deeper understanding in how singers can use different tools to shape and communicate music, and the order of priority and functions of these tools. To attain knowledge about this, I have applied different theories of musical communication, in addition to a multimodal and social semiotic perspective with focus on communication as a social process of sign making. This has been examined through qualitative interviews with eight active and former professional singers in various genres. The results show that the singers use many different tools to shape and communicate music, of which the most predominant are voice-technique, interplay, expectations from the audience, personality and tradition. The singers usage and priority of these tools can be summarised in different overall ways of musical communication, of which tradition and personality are the most conspicuous. One conclusion is that the musical expression of a singer can be described as a weave of communicative structures, and that the singers strive to achieve a wholeness of these structures to yield meaning. Another conclusion is that singers may emphasize different aspects of the musical communication at different stages in a performance, that this may occur at different levels simultaneously, and that the musical context decides how the musical communication is being shaped. Finally, I want to describe the singers’ function as achieving completeness in the musical expression by creating, interpreting and imitating different resources by using different tools, thereby creating a meaningful musical communication to the audience, in interplay with other musicians and the singer.

Sammanfattning Vi vill med denna kunskapsöversikt ta reda på hur matematikundervisning bortom boken kan bedrivas baserad på aktuell forskning. Denna kunskapsöversikt handlar om hur forskningen kring multimodalitet i matematikundervisning ser ut. Syftet är att beskriva vad som kännetecknar forskning inom ovanstående arena. Syfte och frågeställning Syftet med kunskapsöversikten är att kartlägga forskning om multimodalitet inom undervisning imatematik. Forskningsfrågan som denna kunskapsöversikt ämnar att besvara är följande: Vad kännetecknar forskning om multimodalitet inom matematikundervisningen? Metod Kunskapsöversikten baseras på artiklar inhämtade från riktade litteratursökningar från två referensdatabaser, ERIC och Proquest. Vi använde oss av tunktering som sökstrategi för att hitta variationer av multimodal och matematik. Urval Urvalet av studierna samlades in genom litteratursökningar i nämnda referensdatabaser. Urvalskriterierna som ligger till grund för kunskapsöversikten är att artiklarna ska ha multimodal* och math* i titeln, att artiklarna är Peer Reviewed och publicerade i en vetenskaplig tidskrift samt att det är tillgängliga och gratis att få tag på. I slutändan valdes sex artiklar ut för att användas och ligga till grund för denna kunskapsöversikt. Resultat Resultatet visar att det är fördelaktigt att använda sig av multimodalitet i matematikundervisningen. Vidare visar resultatet att forskningen kring fältet är relativt ungt och fortfarande outforskat. Studierna har stor geografisk spridning. Dock fann vi ingen forskning från Sverige. Avslutningsvis så visar resultatet en heltäckning av alla multimodala teckenvärldar.

When watching television, many people use social media to share real-time opinions and feelings. The mediated co-viewing is widely studied under the name of “second screen” or “social TV”. In Japan and China, a technology integrates social media—the comments section—into the video screen, creating an unprecedented form of participation called danmu or danmaku (“barrage”). This research aims to describe the characteristics of this emergent discursive genre. Using content and discourse analysis, we analyzed danmu from 1) a television series and 2) a thread of “funny danmu” from popular video sharing sites. The results revealed that users showed diverse interests (plot, language, culture), appropriated multimodal resources (color, position, symbols) to make fun, while constructing meaning using unconventional discursive strategies. The study indicates the potential of danmu as a space for informal learning, semiotic creativity, and (para)social interaction, and calls for future research into video sharing practices beyond YouTube.
Cuando ven la televisión, muchas personas usan las redes sociales para compartir opiniones y emociones en tiempo real. La covisualización mediada se estudia ampliamente bajo la denominación de “segunda pantalla” o “televisión social”. En Japón y China, una reciente tecnología permite incrustar las redes sociales (la sección de comentarios) dinámicamente en las secuencias del video, creando una forma de participación sin precedentes llamada danmu o danmaku (“barrera de fuego”). Este trabajo se propone describir las características de este género discurso emergente. Utilizando el análisis del contenido y del discurso, analizamos danmu de: 1) una serie de televisión, y 2) un hilo de “danmu graciosos” de sitios populares de repositorios de videos. Nuestros resultados revelan que los usuarios tienen diversos intereses (trama, lenguaje, cultura), se apropian de recursos multimodales (color, posición, símbolos) para hacer humor y construyen significados usando estrategias discursivas originales. Este estudio muestra el potencial del danmu como un espacio para el aprendizaje informal, la creatividad semiótica y la interacción (para)social, además de motivar futuras investigaciones sobre las prácticas de compartir videos más allá de YouTube.
Quan veuen la televisió, moltes persones fan servir les xarxes socials per compartir opinions i emocions en temps real. La covisualització mediada s’estudia àmpliament sota la denominació de “segona pantalla” o “televisió social”. Al Japó i a la Xina, una tecnologia recent permet incrustar les xarxes socials (la secció de comentaris) dinàmicament a les seqüències de vídeo, creant una forma de participació sense precedents anomenada danmu o danmaku (“barrera de foc”). Aquest treball es proposa descriure les característiques d’aquest gènere discursiu emergent. Utilitzant l’anàlisi del contingut i del discurs, estudiem danmu de: 1) una sèrie de televisió, i 2) un fil de “danmu graciosos” de llocs populars de repositoris de vídeos. Els nostres resultats revelen que els usuaris tenen diversos interessos (trama, llenguatge, cultura), s’apropien dels recursos multimodals (color, posició, símbols) per fer humor i construeixen significats utilitzant estratègies discursives originals. Aquest estudi mostra el potencial del danmu com a espai per a l’aprenentatge informal, la creativitat semiòtica i la interacció (para)social, a més de motivar futures investigacions sobre les pràctiques de compartir vídeos més enllà de YouTube.

This thesis describes the need for imaging the tumour pathophysiological microenvironment in order to understand response to treatment. Specifically looking at tumour vascularisation in in vivo murine xenograft models of disease, response to treatment with vascular disruption is assessed via photoacoustic tomography (PAT) and magnetic resonance imaging (MRI). Photoacoustic imaging is a novel imaging modality based on the detection of ultrasound waves created by the absorption of nano-second pulsed laser energy within tissue chromophores. It has the spectral specificity of optical techniques whilst also achieving the high resolution of ultrasound. Haemoglobin is the main chromophore found in biological tissue and this modality is therefore ideally suited to imaging tumour vascularisation. Using a Fabry-Perot interferometer this thesis demonstrates for the first time the feasibility of using PAT for re-clinical research and the characterisation of typical tumour vascular features in a non-invasive non-ionising manner. Response to different concentrations of a vascular disrupting drug is then demonstrated, with novel insights in to how tumours recover from vascular damage observed. MRI of response to vascular disruption is also presented. As MRI is widely used in the clinic it can serve as a translational tool of novel imaging biomarkers, and serves to further understand the differences in response of pathologically vascularised of tumours. This thesis looks at markers associated with disruption of haemodynamics, using apparent diffusion (ADC) to elucidate onset of necrosis, increase in haemoglobin concentration (R2*) as indication of impaired flow, and arterial spin labelling (ASL) as a marker of tumour blood perfusion. This is shown in both subcutaneous and clinically relevant liver metastasis models. Taken as whole, the results from this thesis indicate that whilst understanding the response of the tumour vasculature to pharmacological intervention is complex, novel imaging techniques can provide invaluable translational information on the pathophysiology of tumours.

[ES] El problema conocido como de secuencia a secuencia consiste en transformar una secuencia de entrada en una secuencia de salida. Bajo esta perspectiva se puede atacar una amplia cantidad de problemas, entre los cuales destacan la traducción automática o la descripción automática de objetos multimedia. La aplicación de redes neuronales profundas ha revolucionado esta disciplina, y se han logrado avances notables. Pero los sistemas automáticos todavía producen predicciones que distan mucho de ser perfectas. Para obtener predicciones de gran calidad, los sistemas automáticos se utilizan bajo la supervisión de un humano, quien corrige los errores. Esta tesis se centra principalmente en el problema de la traducción del lenguaje natural, usando modelos enteramente neuronales. Nuestro objetivo es desarrollar sistemas de traducción neuronal más eficientes. asentándonos sobre dos pilares fundamentales: cómo utilizar el sistema de una forma más eficiente y cómo aprovechar datos generados durante la fase de explotación del mismo. En el primer caso, aplicamos el marco teórico conocido como predicción interactiva a la traducción automática neuronal. Este proceso consiste en integrar usuario y sistema en un proceso de corrección cooperativo, con el objetivo de reducir el esfuerzo humano empleado en obtener traducciones de alta calidad. Desarrollamos distintos protocolos de interacción para dicha tecnología, aplicando interacción basada en prefijos y en segmentos, implementados modificando el proceso de búsqueda del sistema. Además, ideamos mecanismos para obtener una interacción con el sistema más precisa, manteniendo la velocidad de generación del mismo. Llevamos a cabo una extensa experimentación, que muestra el potencial de estas técnicas: superamos el estado del arte anterior por un gran margen y observamos que nuestros sistemas reaccionan mejor a las interacciones humanas. A continuación, estudiamos cómo mejorar un sistema neuronal mediante los datos generados como subproducto de este proceso de corrección. Para ello, nos basamos en dos paradigmas del aprendizaje automático: el aprendizaje muestra a muestra y el aprendizaje activo. En el primer caso, el sistema se actualiza inmediatamente después de que el usuario corrige una frase, aprendiendo de una manera continua a partir de correcciones, evitando cometer errores previos y especializándose en un usuario o dominio concretos. Evaluamos estos sistemas en una gran cantidad de situaciones y dominios diferentes, que demuestran el potencial que tienen los sistemas adaptativos. También llevamos a cabo una evaluación humana, con traductores profesionales. Éstos quedaron muy satisfechos con el sistema adaptativo. Además, fueron más eficientes cuando lo usaron, comparados con un sistema estático. El segundo paradigma lo aplicamos en un escenario en el que se deban traducir grandes cantidades de frases, siendo inviable la supervisión de todas. El sistema selecciona aquellas muestras que vale la pena supervisar, traduciendo el resto automáticamente. Aplicando este protocolo, redujimos de aproximadamente un cuarto el esfuerzo humano necesario para llegar a cierta calidad de traducción. Finalmente, atacamos el complejo problema de la descripción de objetos multimedia. Este problema consiste en describir en lenguaje natural un objeto visual, una imagen o un vídeo. Comenzamos con la tarea de descripción de vídeos pertenecientes a un dominio general. A continuación, nos movemos a un caso más específico: la descripción de eventos a partir de imágenes egocéntricas, capturadas a lo largo de un día. Buscamos extraer relaciones entre eventos para generar descripciones más informadas, desarrollando un sistema capaz de analizar un mayor contexto. El modelo con contexto extendido genera descripciones de mayor calidad que un modelo básico. Por último, aplicamos la predicción interactiva a estas tareas multimedia, disminuyendo el esfuerzo necesa
[CAT] El problema conegut com a de seqüència a seqüència consisteix en transformar una seqüència d'entrada en una seqüència d'eixida. Seguint aquesta perspectiva, es pot atacar una àmplia quantitat de problemes, entre els quals destaquen la traducció automàtica, el reconeixement automàtic de la parla o la descripció automàtica d'objectes multimèdia. L'aplicació de xarxes neuronals profundes ha revolucionat aquesta disciplina, i s'han aconseguit progressos notables. Però els sistemes automàtics encara produeixen prediccions que disten molt de ser perfectes. Per a obtindre prediccions de gran qualitat, els sistemes automàtics són utilitzats amb la supervisió d'un humà, qui corregeix els errors. Aquesta tesi se centra principalment en el problema de la traducció de llenguatge natural, el qual s'ataca emprant models enterament neuronals. El nostre objectiu principal és desenvolupar sistemes més eficients. Per a aquesta tasca, les nostres contribucions s'assenten sobre dos pilars fonamentals: com utilitzar el sistema d'una manera més eficient i com aprofitar dades generades durant la fase d'explotació d'aquest. En el primer cas, apliquem el marc teòric conegut com a predicció interactiva a la traducció automàtica neuronal. Aquest procés consisteix en integrar usuari i sistema en un procés de correcció cooperatiu, amb l'objectiu de reduir l'esforç humà emprat per obtindre traduccions d'alta qualitat. Desenvolupem diferents protocols d'interacció per a aquesta tecnologia, aplicant interacció basada en prefixos i en segments, implementats modificant el procés de cerca del sistema. A més a més, busquem mecanismes per a obtindre una interacció amb el sistema més precisa, mantenint la velocitat de generació. Duem a terme una extensa experimentació, que mostra el potencial d'aquestes tècniques: superem l'estat de l'art anterior per un gran marge i observem que els nostres sistemes reaccionen millor a les interacciones humanes. A continuació, estudiem com millorar un sistema neuronal mitjançant les dades generades com a subproducte d'aquest procés de correcció. Per a això, ens basem en dos paradigmes de l'aprenentatge automàtic: l'aprenentatge mostra a mostra i l'aprenentatge actiu. En el primer cas, el sistema s'actualitza immediatament després que l'usuari corregeix una frase. Per tant, el sistema aprén d'una manera contínua a partir de correccions, evitant cometre errors previs i especialitzant-se en un usuari o domini concrets. Avaluem aquests sistemes en una gran quantitat de situacions i per a dominis diferents, que demostren el potencial que tenen els sistemes adaptatius. També duem a terme una avaluació amb traductors professionals, qui varen quedar molt satisfets amb el sistema adaptatiu. A més, van ser més eficients quan ho van usar, si ho comparem amb el sistema estàtic. Pel que fa al segon paradigma, l'apliquem per a l'escenari en el qual han de traduir-se grans quantitats de frases, i la supervisió de totes elles és inviable. En aquest cas, el sistema selecciona les mostres que paga la pena supervisar, traduint la resta automàticament. Aplicant aquest protocol, reduírem en aproximadament un quart l'esforç necessari per a arribar a certa qualitat de traducció. Finalment, ataquem el complex problema de la descripció d'objectes multimèdia. Aquest problema consisteix en descriure, en llenguatge natural, un objecte visual, una imatge o un vídeo. Comencem amb la tasca de descripció de vídeos d'un domini general. A continuació, ens movem a un cas més específic: la descripció d''esdeveniments a partir d'imatges egocèntriques, capturades al llarg d'un dia. Busquem extraure relacions entre ells per a generar descripcions més informades, desenvolupant un sistema capaç d'analitzar un major context. El model amb context estés genera descripcions de major qualitat que el model bàsic. Finalment, apliquem la predicció interactiva a aquestes tasques multimèdia, di
[EN] The sequence-to-sequence problem consists in transforming an input sequence into an output sequence. A variety of problems can be posed in these terms, including machine translation, speech recognition or multimedia captioning. In the last years, the application of deep neural networks has revolutionized these fields, achieving impressive advances. However and despite the improvements, the output of the automatic systems is still far to be perfect. For achieving high-quality predictions, fully-automatic systems require to be supervised by a human agent, who corrects the errors. This is a common procedure in the translation industry. This thesis is mainly framed into the machine translation problem, tackled using fully neural systems. Our main objective is to develop more efficient neural machine translation systems, that allow for a more productive usage and deployment of the technology. To this end, we base our contributions on two main cornerstones: how to better use of the system and how to better leverage the data generated along its usage. First, we apply the so-called interactive-predictive framework to neural machine translation. This embeds the human agent and the system into a cooperative correction process, that seeks to reduce the human effort spent for obtaining high-quality translations. We develop different interactive protocols for the neural machine translation technology, namely, a prefix-based and a segment-based protocols. They are implemented by modifying the search space of the model. Moreover, we introduce mechanisms for achieving a fine-grained interaction while maintaining the decoding speed of the system. We carried out a wide experimentation that shows the potential of our contributions. The previous state of the art is overcame by a large margin and the current systems are able to react better to the human interactions. Next, we study how to improve a neural system using the data generated as a byproduct of this correction process. To this end, we rely on two main learning paradigms: online and active learning. Under the first one, the system is updated on the fly, as soon as a sentence is corrected. Hence, the system is continuously learning from the corrections, avoiding previous errors and specializing towards a given user or domain. A large experimentation stressed the adaptive systems under different conditions and domains, demonstrating the capabilities of adaptive systems. Moreover, we also carried out a human evaluation of the system, involving professional users. They were very pleased with the adaptive system, and worked more efficiently using it. The second paradigm, active learning, is devised for the translation of huge amounts of data, that are infeasible to being completely supervised. In this scenario, the system selects samples that are worth to be supervised, and leaves the rest automatically translated. Applying this framework, we obtained reductions of approximately a quarter of the effort required for reaching a desired translation quality. The neural approach also obtained large improvements compared with previous translation technologies. Finally, we address another challenging problem: visual captioning. It consists in generating a description in natural language from a visual object, namely an image or a video. We follow the sequence-to-sequence framework, under a a multimodal perspective. We start by tackling the task of generating captions of videos from a general domain. Next, we move on to a more specific case: describing events from egocentric images, acquired along the day. Since these events are consecutive, we aim to extract inter-eventual relationships, for generating more informed captions. The context-aware model improved the generation quality with respect to a regular one. As final point, we apply the intractive-predictive protocol to these multimodal captioning systems, reducing the effort required for correcting the outputs.
Section 5.4 describes an user evaluation of an adaptive translation system. This was done in collaboration with Miguel Domingo and the company Pangeanic, with funding from the Spanish Center for Technological and Industrial Development (Centro para el Desarrollo Tecnológico Industrial). [...] Most of Chapter 6 is the result of a collaboration with Marc Bolaños, supervised by Prof. Petia Radeva, from Universitat de Barcelona/CVC. This collaboration was supported by the R-MIPRCV network, under grant TIN2014-54728-REDC.
Peris Abril, Á. (2019). Interactivity, Adaptation and Multimodality in Neural Sequence-to-sequence Learning [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/134058
TESIS

Precision oncology plays an indispensable role in cancer prevention and the improvement of cancer survival rate and chances of cure. Accurate tumor and metabolic tumor volumes (MTV) delineations are essential in cancer diagnosis and effective treatment planning. T prolonged challenges in computational medical image segmentations are caused by the limited image resolutions and the variability in the location and appearance of pathologies, for instance the involvement of adjacent structures and the tumor metabolic heterogeneity. To tackle the above challenges and better interpret the different levels of requests from diagnosis and treatment aspects, this thesis provides graph-based models for multi-modality object boundary delineations as well as metabolic heterogeneity measurement. Our major contributions include: 1) abstract and compact extraction and representation of metabolic tumor activities which yields a new research area of topology in image segmentation; 2) a label propagation process with topology constraint for tumor separation; 3) an innovative topology polymorphism graph model to outline the entire heterogeneous lung tumor from adjacent structures with similar intensities; 4) a TopoGraphic model for indistinct target object boundary delineation from multimodality anatomical images; and finally 5) an insightful metabolic sub-volume measurement approach for lung cancer patients who need personalized dose painting and further metabolic feature tracking. These models have been validated by multiple evaluation criterions and compared to state-of-art well-known algorithms on comprehensive datasets include public datasets, clinical datasets from two hospitals and phantom images. The results demonstrated that the proposed models improved the target tumor / tissue segmentation accuracy where there are indistinct anatomical/functional boundaries and tumor metabolic inhomogeneity.

Syftet med denna studie var att undersöka vilken effekt tillämpandet av auditiva intryck hade för elevers läsupplevelse. Detta undersöktes genom en experimentell studie som genomfördes tillsammans med elever i årskurs 3. Eleverna fick då, under observation, läsa en text med och utan ljud. Resultatet från observationerna användes sedan för att urskilja eventuella likheter och skillnader i elevernas läsning utifrån de olika tillvägagångssätten. I samband med dessa observationer genomfördes även intervjuer med några av de elever som deltog i undersökningen, för att därigenom skapa en tydligare bild av elevernas individuella upplevelser av läsningen samt för att kunna stärka, bekräfta eller förkasta de tendenser som framgick under observationerna. Resultatet visade tydliga tendenser av ett möjligt samband mellan elevernas läsupplevelse och tillämpandet av auditiva intryck under läsning, där majoriteten av eleverna fick bättre inlevelse och kunde koncentrera sig mer under läsning.
The purpose of this study was to investigate which effect the application of auditory impressions had for students’ reading experience. This was investigated by an experimental study conducted together with pupils from the third grade. Students were then, under observation, reading texts with and without sound. The results of the observations were then used to identify similarities and differences in the students’ reading abilities in the two different scenarios. In conjunction with these observations, interviews were conducted with some of the students who participated in the study in order to create a clearer picture of their individual experiences of reading, and to strengthen, confirm or reject the trends that emerged during the observations. The results clearly showed tendencies of a possible connection between the students' reading experience and the application of auditory impression in reading, where the majority of students had a better involvement and could concentrate more during reading.

L'utilisation d'images de modalités différentes est très répandue dans la résolution de problèmes liés aux applications de la télédétection. La raison principale est que chaque image d'une certaine modalité contient des informations spécifiques qui peuvent être intégrées en un modèle unique, afin d'améliorer notre connaissance à propos d'une scène spécifique. A cause du grand volume de données disponibles, ces intégrations doivent être réalisées de manière automatique. Cependant, un problème apparaît dès les premiers stades du processus : la recherche, dans des images de modalités différentes, de régions en correspondance. Ce problème est difficile à résoudre car la décision de regrouper des régions doit nécessairement reposer sur la part d'information commune aux images, même si les modalités sont différentes. Dans cette thèse, nous nous proposons donc d'apporter une contribution à la résolution de ce problème
The use of several images of various modalities has been proved to be quite useful for solving problems arising in many different applications of remote sensing. The main reason is that each image of a given modality conveys its own part of specific information, which can be integrated into a single model in order to improve our knowledge on a given area. With the large amount of available data, any task of integration must be performed automatically. At the very first stage of an automated integration process, a rather direct problem arises : given a region of interest within a first image, the question is to find out its equivalent within a second image acquired over the same scene but with a different modality. This problem is difficult because the decision to match two regions must rely on the common part of information supported by the two images, even if their modalities are quite different. This is the problem that we wish to address in this thesis

El objetivo de esta tesis es contribuir a la investigación en pragmática del interlenguaje y multimodalidad. El objetivo principal es explorar como aprendices de lengua en distintos niveles de lengua realizan quejas desde la perspectiva del análisis de la conversación (Kasper, 2006). Un análisis multimodal de la conversación se ha realizado para examinar cómo diferentes modos interactúan en la construcción de la conversación. El marco teórico presentado en el estudio se centró en la naturaleza de la pragmática (Crystal, 1985; Leech, 1983; Thomas, 1983), pragmática del interlenguaje (Kasper & Blum-Kulka, 1993), análisis de la conversación (Sacks et al., 1974), nivel de lengua (e.g. Al-Gahtani & Roever, 2012), el acto de habla de las quejas (e.g. Trosborg, 1995; Laforest, 2002), y la multimodalidad (Jewitt, et al., 2016).
The aim of this thesis is to contribute to the research on interlanguage pragmatics and multimodality. The main purpose is to explore how learners at different proficiency levels perform complaints and responses to complaints following a conversation analysis approach (Kasper, 2006). Furthermore, a multimodal conversation analysis is conducted in order to examine how different modes interact in the construction of the conversation. To meet the objectives of the thesis, the theoretical framework presented in the study focused on the nature of pragmatics (Crystal, 1985; Leech, 1983; Thomas, 1983), interlanguage pragmatics (Kasper & Blum-Kulka, 1993), conversation analysis (Sacks et al., 1974), proficiency (e.g. Al-Gahtani & Roever, 2012), the speech act of complaints (e.g. Trosborg, 1995; Laforest, 2002), and multimodality (Jewitt, et al., 2016). This framework served to explore participants' performance of complaints sequences at different proficiency levels, specific conversational features such as backchannel and overlapping, paralanguage and kinesics.

Multimodal theory has developed out of social semiotics and can be seen as a response to the rise in the use of new technologies for the creation, distribution and consumption of media texts and the need to find new ways of describing and explaining their role in representation and communication. Its development is historical. It is a response to change over time. The incorporation of the visual into social semiotics marks a key moment in the development of multimodal theory. Visual literacy is discussed in relation to changes in modes of representation and a critique of this concept is provided. This is conducted in relation to how the visual modality has been integrated into social semiotics as a platform for research into multimodal communication more generally. Framing is developed along three main lines of enquiry (semiotic, cognitive and affective) as alternative ways of accounting for some of these shifts in communication and each are presented in the form of case studies. Framing and its close relationship with composition in media texts is discussed and this understanding, one that emphasise proximity as a multimodal principle, is applied to the visual design of content, the realisation of context through the provision visual cues, and later to embodiment and urban space. The three case studies, the application of framing to a range of media texts, the critical judgements made about the role visual in contemporary theory and the application of these concepts to multimodality are presented as part of an intellectual journey.

Despite that most vestibular schwannomas are successfully treated with radiotherapy, current follow-up protocols entail years of serial magnetic resonance imaging (MRI) scans to ensure cessation of growth. This pilot study sought to identify early predictors of radiation treatment response using a non-invasive multi-modality imaging approach. We hypothesized that by combining information acquired from dynamic contrast-enhanced MRI (DCE-MRI), diffusion tensor imaging (DTI), and L-¹¹C-methionine positron emission tomography (MET-PET) treatment response could be identified sooner than the current several year waiting period. This thesis presents the baseline MRI and MET-PET results of the pilot study acquired to-date with follow-up data to be acquired in the next six months. Baseline results suggest that DTI and DCE-MRI yield information that may be useful in identifying the response of vestibular schwannomas to radiotherapy. In particular, vestibular schwannomas display elevated mean diffusion coefficients relative to the contra-lateral cerebellum. Also, the novel use of arterial input functions derived from the anterior inferior cerebellar arteries has led to the successful implementation of DCE-MRI pharmaco-kinetic models which may be used to quantitatively monitor tumor response to radiotherapy. Furthermore, MET-PET has shown promise as a tool for evaluating response as all tumors exhibited enhancement under this modality as compared to the contra-lateral side of the brain. Single-voxel spectroscopy with 3T MRI has proven to be a poor technique with which to examine vestibular schwannomas since only two of eight spectra were acquired successfully. All of the techniques that have shown promise as investigatory tools of tumor response can potentially be implemented clinically in the near future.

PONTIFÍCIA UNIVERSIDADE CATÓLICA DO RIO DE JANEIRO
Neste estudo investigo como uma metodologia orientada para o letramento em gêneros, sugerido pelos Parâmetros Curriculares Nacionais, e ancorada na multimodalidade auxilia a aprendizagem de inglês como língua estrangeira por um grupo de alunos da 3ª série do Ensino Fundamental, na faixa etária de 9-10 anos, em uma escola particular no Rio de Janeiro. Baseio- me em Allwright & Bailey (1991) e Wenger (1998) que vêem a sala de aula como local propício à construção do conhecimento; em Kern (2000) e Johns (1997) quanto ao aspecto social e cognitivo do letramento e em Kress (2000), Stein (2000) e Royce (2002) no que concerne à complementaridade intersemiótica como facilitadora do processo de construção do conhecimento. Textos produzidos pelos alunos foram selecionados para compor o corpus deste estudo, exemplificando as etapas de escritura, reescritura, escritura ilustrada e transposição para histórias em quadrinhos desenvolvidas durante uma unidade do livro texto. Com suporte em Halliday & Hasan (1989) e Ventola (1987a), foram analisadas a configuração contextual e estrutura genérica dos textos. Quanto à multimodalidade e à importância da semiose visual foi observada a adequação imagem-texto (fala dos personagens e contorno de balão, expressão facial e corporal, cenário), segundo Kress & van Leeuwen (1996). A análise dos textos e a reflexão sobre o trabalho pedagógico desenvolvido sugerem a relevância da construção de atividades desafiadoras orientadas para o letramento em gêneros, como recursos para que os alunos apropriem-se da língua alvo em contextos sociais refletindo em seus textos suas experiências sócio-histórico-culturais.
This study investigates how a genre-based methodology, focused on literacy development, as suggested by the National Curricular Parameters, and anchored in multimodality facilitates knowledge- construction by a group of English as a Foreign Language students, aged 9-10, in the 3rd grade of a private elementary school in Rio de Janeiro. This research is based on Allwright & Bailey (1991), and Wenger (1998), for whom the classroom is a space favourable to knowledge-construction; on Kern (2000) and Johns (1997), whose theories focus on the social and cognitive aspects of literacy, and on Kress (2000), Stein (2000), and Royce (2002), concerning the intersemiotic complementarity which facilitates the knowledge-construction process. The corpus of this study is composed of student-produced texts selected to exemplify the stages of drafting, re-drafting, illustrated text-writing and text- transposition to comic strips, which were developed during the exploration of a text book unit. The analysis of this material was carried out according to Halliday and Hasan (1989) and Ventola (1987a), for whom language is an expression of social behaviour in situational contexts, and according to Kress & van Leeuwen (1996), who point out the importance of the visual semiotic system as a resource for meaning- making. The text analysis and the reflection on the pedagogical work developed suggest the importance of offering challenging activities that are focused on a genre-based approach, as resources for learners to appropriate themselves of the target language in social contexts, while showing in their texts their social-historicalcultural experiences.

PONTIFÍCIA UNIVERSIDADE CATÓLICA DO RIO DE JANEIRO
O objetivo principal deste trabalho de pesquisa é entender o uso de textos visuais na sala de aula e seu impacto na aprendizagem. A pesquisa enfoca o uso de textos visuais para facilitar a aplicação de uma perspectiva pedagógica que objetive a sócio-construção do conhecimento, o sócio-interacionismo, seguindo a concepção de Bakhtin (2000), Freire (1979), Mercer (1994) e Vygotsky (1994). Com essa base teórica, a pesquisa aplica a gramática do desenho visual (Kress e van Leeuwen, 1996) em um ambiente de sala de aula, ao examinar as interpretações feitas pelos próprios aprendizes em relação às imagens utilizadas pela professora, para que se observa de que forma as observações dos aprendizes evocam as categorias na gramática do desenho visual e, portanto, trazer luz sobre o conhecimento desses aprendizes de como ler imagens. Discute-se as implicações de uma abordagem pedagógica que objetiva construir o letramento visual no ambiente de ensino da língua inglesa.
The main purpose of this study is to understand the use of visual texts in the classroom and their impact on learning. The research focuses on how the use of visual texts can facilitate the application of a construction of meanings and the socio- interactional perspective on teaching/learning, according to Bakhtin (2000), Freire (1979), Mercer (1994) and Vygotsky (1994). With this theoretical framework, the study applies the grammar of visual design (Kress & Van Leeuwen, 1996) to a classroom setting by examining the learners’ interpretations of the images used by the teacher, to observe to what extent their observations evoke the categories in the grammar of visual design and thus indicate knowledge of how to read images. The study discusses the implications of an approach to teaching and learning that aims at building visual literacy in the setting of English language teaching.

Object recognition has been extensively investigated in computer vision area, since it is a fundamental and essential technique in many important applications, such as robotics, auto-driving, automated manufacturing, and security surveillance. According to the selection criteria, object recognition mechanisms can be broadly categorized into object proposal and classification, eye fixation prediction and saliency object detection. Object proposal tends to capture all potential objects from natural images, and then classify them into predefined groups for image description and interpretation. For a given natural image, human perception is normally attracted to the most visually important regions/objects. Therefore, eye fixation prediction attempts to localize some interesting points or small regions according to human visual system (HVS). Based on these interesting points and small regions, saliency object detection algorithms propagate the important extracted information to achieve a refined segmentation of the whole salient objects. In addition to natural images, object recognition also plays a critical role in clinical practice. The informative insights of anatomy and function of human body obtained from multimodality biomedical images such as magnetic resonance imaging (MRI), transrectal ultrasound (TRUS), computed tomography (CT) and positron emission tomography (PET) facilitate the precision medicine. Automated object recognition from biomedical images empowers the non-invasive diagnosis and treatments via automated tissue segmentation, tumor detection and cancer staging. The conventional recognition methods normally utilize handcrafted features (such as oriented gradients, curvature, Haar features, Haralick texture features, Laws energy features, etc.) depending on the image modalities and object characteristics. It is challenging to have a general model for object recognition. Superior to handcrafted features, deep neural networks (DNN) can extract self-adaptive features corresponding with specific task, hence can be employed for general object recognition models. These DNN-features are adjusted semantically and cognitively by over tens of millions parameters corresponding to the mechanism of human brain, therefore leads to more accurate and robust results. Motivated by it, in this thesis, we proposed DNN-based energy models to recognize object on multimodality images. For the aim of object recognition, the major contributions of this thesis can be summarized below: 1. We firstly proposed a new comprehensive autoencoder model to recognize the position and shape of prostate from magnetic resonance images. Different from the most autoencoder-based methods, we focused on positive samples to train the model in which the extracted features all come from prostate. After that, an image energy minimization scheme was applied to further improve the recognition accuracy. The proposed model was compared with three classic classifiers (i.e. support vector machine with radial basis function kernel, random forest, and naive Bayes), and demonstrated significant superiority for prostate recognition on magnetic resonance images. We further extended the proposed autoencoder model for saliency object detection on natural images, and the experimental validation proved the accurate and robust saliency object detection results of our model. 2. A general multi-contexts combined deep neural networks (MCDN) model was then proposed for object recognition from natural images and biomedical images. Under one uniform framework, our model was performed in multi-scale manner. Our model was applied for saliency object detection from natural images as well as prostate recognition from magnetic resonance images. Our experimental validation demonstrated that the proposed model was competitive to current state-of-the-art methods. 3. We designed a novel saliency image energy to finely segment salient objects on basis of our MCDN model. The region priors were taken into account in the energy function to avoid trivial errors. Our method outperformed state-of-the-art algorithms on five benchmarking datasets. In the experiments, we also demonstrated that our proposed saliency image energy can boost the results of other conventional saliency detection methods.

Multimodal theory has developed out of social semiotics and can be seen as a response to the rise in the use of new technologies for the creation, distribution and consumption of media texts and the need to find new ways of describing and explaining their role in representation and communication. Its development is historical. It is a response to change over time. The incorporation of the visual into social semiotics marks a key moment in the development of multimodal theory. Visual literacy is discussed in relation to changes in modes of representation and a critique of this concept is provided. This is conducted in relation to how the visual modality has been integrated into social semiotics as a platform for research into multimodal communication more generally. Framing is developed along three main lines of enquiry (semiotic, cognitive and affective) as alternative ways of accounting for some of these shifts in communication and each are presented in the form of case studies. Framing and its close relationship with composition in media texts is discussed and this understanding, one that emphasise proximity as a multimodal principle, is applied to the visual design of content, the realisation of context through the provision visual cues, and later to embodiment and urban space. The three case studies, the application of framing to a range of media texts, the critical judgements made about the role visual in contemporary theory and the application of these concepts to multimodality are presented as part of an intellectual journey.

In the present work, users’ interaction with advanced systems has been investigated in different application domains and with respect to different interfaces. The methods employed were carefully devised to respond to the peculiarities of the interfaces under examination. We could extract a set of recommendations for developers. The first application domain examined regards the home. In particular, we addressed the design of a gestural interface for controlling a lighting system embedded into a piece of furniture in the kitchen. A sample of end users was observed while interacting with the virtual simulation of the interface. Based on the videoanalysis of users’ spontaneous behaviors, we could derive a set of significant interaction trends The second application domain involved the exploration of an urban environment in mobility. In a comparative study, a haptic-audio interface and an audio-visual interface were employed for guiding users towards landmarks and for providing them with information. We showed that the two systems were equally efficient in supporting the users and they were both well- received by them. In a navigational task we compared two tactile displays each embedded in a different wearable device, i.e., a glove and a vest. Despite the differences in the shape and size, both systems successfully directed users to the target. The strengths and the flaws of the two devices were pointed out and commented by users. In a similar context, two devices supported Augmented Reality technology, i.e., a pair of smartglasses and a smartphone, were compared. The experiment allowed us to identify the circumstances favoring the use of smartglasses or the smartphone. Considered altogether, our findings suggest a set of recommendations for developers of advanced systems. First, we outline the importance of properly involving end users for unveiling intuitive interaction modalities with gestural interfaces. We also highlight the importance of providing the user the chance to choose the interaction mode better fitting the contextual characteristics and to adjust the features of every interaction mode. Finally, we outline the potential of wearable devices to support interactions on the move and the importance of finding a proper balance between the amount of information conveyed to the user and the size of the device.
I sistemi computazionali hanno ormai da tempo abbandonato lo scenario immobile della scrivania e tendono oggi a coinvolgere sempre di più ambiti della vita quotidiana, in altre parole pervadono le nostre vite. Nel contesto del pervasive o ubiquitous computing, l’interazione tra l’utente e la macchina dipende in misura sempre minore da specifici sistemi di input (per esempio mouse e tastiera) e sfrutta sempre di più modalità di controllo naturali per operare con i dispositivi (per esempio tramite i gesti o il riconoscimento vocale). Numerosi sono stati i tentativi di trasformare in modo sostanziale il design dei computer e delle modalità di interazione tra cui l’impiego di sistemi per il riconoscimento dei comandi gestuali, dispositivi indossabili e la realtà aumentata. In tali contesti, i metodi tradizionalmente impiegati per lo studio della relazione uomo-macchina si rivelano poco efficaci e si delinea la necessità di una adeguata revisione di tali metodi per poter indagare adeguatamente le caratteristiche dei nuovi sistemi. Nel presente lavoro, sono state analizzate le modalità di interazione dell’utente con diversi sistemi innovativi, ciascuno caratterizzato da un diverso tipo di interfaccia. Sono stati inoltre considerati contesti d’uso diversi. I metodi impiegati sono stati concepiti per rispondere alle diverse caratteristiche delle interfacce in esame e una serie di raccomandazioni per gli sviluppatori sono state derivate dai risultati degli esperimenti. Il primo dominio di applicazione investigato è quello domestico. In particolare, è stato esaminato il design di una interfaccia gesturale per il controllo di un sistema di illuminazione integrato in un mobile della cucina. Un gruppo rappresentativo di utenti è stato osservato mentre interagiva con una simulazione virtuale del prototipo. In base all’analisi dei comportamenti spontanei degli utenti, abbiamo potuto osservare una serie di regolarità nelle azioni dei partecipanti. Il secondo dominio di applicazione riguarda l’esplorazione di un ambiente urbano in mobilità. In un esperimento comparativo, sono state confrontate un’interfaccia audio-aptica e una interfaccia audio- visiva per guidare gli utenti verso dei punti di interesse e per fornire loro delle informazioni a riguardo. I risultati indicano che entrambi i sistemi sono ugualmente efficienti ed entrambi hanno ricevuto valutazioni positive da parte degli utenti. In un compito di navigazione sono stati confrontati due display tattili, ciascuno integrato in un diverso dispositivo indossabile, ovvero un guanto e un giubbotto. Nonostante le differenze nella forma e nella dimensione, entrambi i sistemi hanno condotto efficacemente l’utente verso il target. I punti di forza e le debolezze dei due sistemi sono state evidenziate dagli utenti. In un contesto simile, sono stati confrontati due dispositivi che supportano la Realtà Aumentata, ovvero un paio di smartglass e uno smartphone. L’esperimento ci ha permesso di identificare le circostanze che favoriscono l’impiego dell’uno o dell’altro dispositivo. Considerando i risultati degli esperimenti complessivamente, possiamo quindi delineare una serie di raccomandazione per gli sviluppatori di sistemi innovativi. Innanzitutto, si evidenzia l’importanza di coinvolgere in modo adeguato gli utenti per indentificare modalità di interazione intuitive con le interfacce gesturali. Inoltre emerge l’importanza di fornire all’utente la possibilità di scegliere la modalità di interazione che meglio risponde alle caratteristiche del contesto insieme alla possibilità di personalizzare le proprietà di ciascuna modalità di interazione alle proprie esigenze. Infine, viene messa in luce le potenzialità dei dispositivi indossabili nelle interazioni in mobilità insieme con l’importanza di trovare il giusto equilibrio tra la quantità di informazioni che il dispositivo è in grado di inviare e la dimensione dello stesso.

Cette thèse consacrée à la didactique de l’anglais à l’école élémentaire, s’inscrit dans une démarche à mi-chemin entre la recherche-action et la recherche-développement, pour laquelle a été privilégiée une dynamique collaborative avec les acteurs de terrain. Pour cette recherche, il s’agit principalement de se pencher sur le développement de la compétence phonologique des élèves de deux classes de CM1 à travers l’outil numérique Kinéphones inspiré du Silent Way de Gattegno. Le cœur de ce travail concerne particulièrement l’effet des couleurs lors du passage à la graphie en anglais L2 et sur le potentiel de facilitation d’un double encodage phonème-couleur en mémoire, qui viendrait ensuite soutenir, à la manière d’une béquille, la compréhension et l’application de règles de phonie-graphie ciblées.Grâce à une étude exploratoire menée en amont dans le cadre de notre Master en Didactique des Langues et Ingénierie Pédagogique Multimédia (Le Page, 2015), nous avons mis à profit notre expérience sur le terrain visé pour ancrer nos questionnements dans la réalité du quotidien des professeurs des écoles. Une enquête par questionnaire à destination des professeurs des écoles a permis de mieux cerner les caractéristiques de ce contexte particulier, tout en proposant en parallèle, une revue sélective des TICE pour l’apprentissage des langues et plus précisément celles qui proposent un travail plus ou moins approfondie de la compétence phonologique. Ces informations sur les enjeux liés à la formation des enseignants, les attentes institutionnelles et les pratiques réelles ainsi que ceux liés au développement de la compétence phonologique des apprenants et aux caractéristiques spécifiques de l’anglais et de son système phonie-graphie, ont servi de base pour la mise en place d’une expérimentation. Pour ce faire, un prototype d’extension de Kinéphones qui complète la présentation colorée des phonèmes déjà existante par de la graphie colorée pour un travail de la lecture en couleurs a été élaboré. Dans cette deuxième phase du travail, il a s’agit de construire une séquence pédagogique adaptée aux attentes des enseignants et au programme du cycle 2 en anglais tout en introduisant le travail de la phonétique/phonologie par le biais des couleurs et l’utilisation de Kinéphones. Dans une telle démarche, il a été nécessaire de revenir sur notre protocole initial pour nous adapter aux contraintes émanant du terrain.Les données collectées ne nous permettent pas d’établir avec certitude un lien entre l’utilisation des couleurs et la réussite éventuelle du passage à l’écrit puisque l’étape nécessaire de mémorisation des associations phonèmes-couleurs-graphies n’a pas été concluante. Toutefois un certain nombre de problèmes en lien avec l’utilisation même de la multimodalité par les couleurs et avec le support technologique et son introduction en classe ont pu être observés. Pour mieux analyser toutes les dimensions d’analyse de ces données complexes et complémentaires, nous nous sommes appuyés sur les critères d’évaluation des EIAH de Tricot et al. (2003). Les résultats de cette recherche ont aussi été complétés par des données inattendues qui apportent de nouvelles perspectives de recherche concernant notamment la modalité de la gestuelle pour la représentation des phonèmes et les évolutions nécessaires dans le développement futur de l’outil Kinéphones. De même, un certain nombre de données fournies par les enseignants participant offrent des pistes de réflexions concernant la formation des enseignants à l’exploitation de tels outils et les pistes didactiques en lien avec les pédagogies multimodales pour l’enseignement des langues
This thesis, dedicated to the didactics of English at elementary school, is part of an approach halfway between research-action and research-development, for which a collaborative dynamic with actors in the field was privileged. For this research, the main focus is on the development of the phonological competence of pupils in two 4th grade classes through the numerical tool Kinéphones inspired by Gattegno's Silent Way. The core of this work concerns in particular the effect of colours during the transition to L2 English writing and the potential for facilitating double phoneme-colour encoding in memory, which would then support, like a crutch, the understanding and application of targeted phoneme-graphy rules.Thanks to an exploratory study carried out beforehand as part of our Master's Degree in Language Teaching and Multimedia Pedagogical Engineering (Le Page, 2015), we have used our experience in the field to settle our questions in the reality of the daily lives of school teachers. A questionnaire survey addressed to school teachers allowed us to better define the characteristics of this particular context, while at the same time proposing a selective review of ICT for language learning and more precisely those that offer a more or less in-depth work on phonological competence. This information on the issues related to teacher training, institutional expectations and actual practices, as well as those related to the development of learners' phonological competence and the specific characteristics of English and its phone-graphy system, served as a basis for setting up an experiment. To this end, a prototype of a Kinéphones extension that complements the already existing coloured presentation of phonemes with coloured writing for colours reading work was developed. In this second phase of the work, it was a matter of constructing a pedagogical sequence adapted to the expectations of the teachers and to the curriculum of cycle 2 in English while introducing the work of phonetics/phonology through colours and the use of Kinéphones. In such an approach, it was necessary to go back on our initial protocol to adapt to the constraints emanating from the field.The data collected do not allow us to establish with certainty a link between the use of colours and the eventual success of the passage to writing since the necessary step of memorizing the phoneme-colour-graphic associations was not conclusive. However, a certain number of problems related to the use of colour multimodality itself and to the technological support and its introduction in the classroom could be observed. In order to better analyse all the dimensions of the analysis of these complex and complementary data, we relied on the evaluation criteria of Computer Environments for Human Learning of Tricot et al (2003). The results of this research were also supplemented by unexpected data that provide new research perspectives concerning in particular the gestural modality for the representation of phonemes and the necessary evolutions in the future development of the Kinéphones tool. In order to better analyse all the dimensions of the analysis of these complex and complimentary data, we relied on the evaluation criteria of Computer Environments for Human Learning of Tricot et al (2003). In the same way, a certain number of data provided by the participating teachers open up new perspectives concerning the training of teachers in the use of such tools and the didactic paths in relation to multimodal pedagogies for language teaching

The goal and purpose of this essay is to study if and how multimodality is used within a Swedish school in Kalmar. This is accomplished by observing the different modalities inside the classroom and how the use of different modalities affects the teaching. Unstructured observations combined with a material-based thematic presentation leads up to an analysis based on social semiotics and multimodality. The essay shows how the teaching uses lots of modalities: sound, images, movies, speeches and literary text in lots of different ways. These different types of modalities are used in a multimodal context, providing good conditions for learning.

This bachelor thesis consists of an analysis of museum environment. In the research, case studies have been done at the, Stockholm based museums, Swedish museum of Natural history’s exhibition Natur i Sverige (Nature in Sweden) and the Nordiska museet’s (Nordic history) exhibition Modemakt (Power of fashion) and learning environment Lekstugan (playhouse). The purpose of the research was to study how cultural heritage is used and developed within the museum pedagogy aimed at children. The research is based on a multimodal approach with focus on learning resources. The research examines the museums focus on children and youngsters, what the goals for this focus looks like on respective museum and how the focus is visible in the actual museums. The investigation has shown that both of the museums have a clear focus towards children and young people and follows the guidelines from the Swedish government. This thesis shows the differences between both museums in their separate ways of focus on children and youngsters. The different possibilities on focus on children and young can provide inspiration for either other museums or the specific museums in the thesis when developing or expanding their focus further. Both museums way to communicate with children and young people considered to have good potential for learning.
Denna kandidatuppsats består av en analys av museala miljöer. I undersökningen har fallstudier gjorts på Naturhistoriska riksmuseets utställning Natur i Sverige samt Nordiska museets utställning Modemakt samt lärmiljö Lekstugan. Syftet med undersökningen har varit att studera hur kulturarv används och utvecklas inom museipedagogiken riktad mot barn. Undersökningen utgår ifrån ett multimodalt perspektiv med fokus på lärresurser.   Frågeställningarna hanterar museernas riktning mot barn och unga, hur målen för detta ser ut samt hur det ser ut i verkligheten. Undersökningen har visat att båda berörda museer har en tydlig riktning mot barn och unga samt gott och väl följer regeringens riktlinjer för statliga museer.   I uppsatsen tydliggörs Naturhistoriska riksmuseets och Nordiska museets olika sätt att angripa frågan ”riktning mot barn och unga”. Detta kan utgöra inspiration för andra museer eller för de berörda museerna att utveckla sin riktning ytterligare. Båda museernas vis att kommunicera mot barn och unga bedöms ha goda förutsättningar för lärande.

Denna studie har undersökt hur en grupp elever använder ett virtuellt rum som klassrum. Effekten blev att de lärde sig fakta med hela kroppen. Det virtuella klassrummet kan bli ett nytt pedagogiskt redskap för skolungdomar där de kan vara med och bygga upp eller påverka innehållet. Studien är en etnografisk undersökning där jag har studerat multimodalitetsteorier av bl.a. Gunther Kress i vilken man tar hänsyn till alla teckenvärldar som ljud, bild, gester och kroppens rörelse i rummet. Undersökningen har behandlat frågorna; Vad produceras när barn och ungdomar använder virtuell verklighet i klassrummet? Vilka diskurser framträder när VR används som resurs för lärande? Hur kan designen i en virtuell klassrumsmiljö komma till uttryck? Hur kan virtuella läromedel distribueras? Jag har undersökt hur diskurser kan förändras efter platsens förutsättningar och hur designen av en VR-undervisning kan beskrivas. Studien innefattar en multimodal produktion under en bildlektion i ett virtuellt klassrum, undersökningen inkluderar även distribution av virtuell konst i lärmiljö. Jag har vidare funderat på vilka förutsättningar som krävs för att virtuell verklighet skall bli en standard i framtidens klassrum.

The purpose of this study is to investigate the preschool teachers’ understanding of digital competence in preschool and their use of it to create learning situations for children in the form of increased opportunities for literacy. Digitalization has been given a lot of space in the latest revised curriculum of the preschool (2018). This has made digital competence a prerequisite for the pedagogical task in preschool. I want to highlight the opportunities and risks that digitalisation in preschool activities could entail in the form of, on the one hand increased accessibility and participation, on the other hand increased stress and reduced equivalence. It is about democracy as all children should have the opportunity to develop their learning about handling digital tools in everyday life in order to be able to participate in societal issues. Digital competence is a necessity in today`s society, both adultse and children come into contact with the digital in society in our everyday lives. Developments in ICT (Information and communication technology) take place at a furious pace, this will affect children`s working lives in the future. The preschool teachers digital competence has a significant impact on an equivalent teaching in the preschool. The starting point for the essay is my experiences of professional life. I will present the wording of the governing document on digitization in general and the meaning of the teacher assignment. Iwill investigate various strands of research on digitization in preschool and investigate the connection between literacy and multimodality. Finally I will examine the teacher´s  competence development. We adults need new knowledge and skills to guide the children in their development. Finally I will discuss the duties of the academic leader of the preschool when it comes to the implementation of digital tools in the preschool.
Denna studie syftar till att undersöka förskollärarens uppfattning av digitala kompetens iförskolan, samt förskollärarens användning av dem för att skapa läs- och skriv situationer förbarnen. Digitalisering har fått stort utrymme i förskolans reviderade läroplan (2018).  Den snabba utvecklingen av det digitala i samhället avspeglar sig på förskolan i vårtpedagogiska uppdrag, att forma barnen till framtida medborgare som kan hantera teknik samtsamarbeta med andra människor i framtida yrkeslivet.  Utgångspunkten för essän är mina egna upplevelser av mitt yrkesliv, jag vill ge läsaren en ärligbild för att synliggöra den komplexitet som den innebär. Digitalisering medför både möjligheter och utmaningar i förskolans kontext. Pedagogerreagerar på olika sätt inför förändringar, ökade krav medför stress hos pedagoger idigitaliseringsarbetet. Jag kommer att börja med att presentera styrdokumentens, formuleringar om digitaliseringgenerellt och läraruppdragets innebörd. Hur ser läroplan för förskolan på digitalisering iförskolan? Jag kommer att undersöka vad forskningen säger om digitalisering i förskolan föratt belysa både risker och möjligheter som studier har visat. Jag kommer ta upp och diskuteravad begreppet litteracitet innebär, samt analysera samband med multimodalitet som är ettteoretiskt synsätt, och hur multimodala verktyg kan underlätta arbetet med litteracitet iundervisningssituationer i förskolan. Till slut kommer jag att undersöka förskollärarnaskompetensutveckling. Vi vuxna behöver nya kunskaper och färdigheter för att lära oss navigeraoch vägleda barnen i sin utveckling och lärande. Jag kommer även att diskutera rektorns roll idetta förändringsarbete.

The present paper focusses on the genre of recipes, its structure and multimodality. The aim of this thesis is to present the genre of recipes from theoretical perspective and to make a practical analysis about the structure of recipes and multimodality based on Oliver’s and Nicholson’s recipes. For the practical part Jamie Oliver’s recipe book Happy Days with the Naked Chef, Beata Nicholson’s recipe book Beatos Virtuvė: Kepinių Knyga and their websites are analysed. Recipes for the analysis are chosen randomly and qualitative method is used as all of them are described and compared in order to find differences and similarities. The paper presents the concept of genre and genre analysis that is discussed by many authors and explained in great details. Further, the genre of recipes and its historical change is looked at thus the influence on the structure of recipes made by new innovations can be noticed. In addition, the ideas suggested by various authors show that this genre is structured and has clearly distinguished parts that should be included in every recipe. Finally, the issue of multimodality is presented and it is of special importance today since not only textual but also visual information is found in books and web-sites. The structural analysis of Oliver’s and Nicholson’s recipes supports the ideas of various scholars that this genre is highly structured. The main difference is found between the recipes in printed and online sources. In recipe books the... [to full text]
Šiame darbe apžvelgiamas receptų žanras, jo struktūra ir multimodalumas. Darbo tikslas yra pristatyti receptų žanro teoriją ir analizuoti receptų struktūrą ir multimodalumą, remiantis Džeimio Oliverio ir Beatos Nicholson receptais. Praktinėje dalyje analizuojama Oliverio receptų knyga „Happy days with the naked chef“ ir Nicholson receptų knyga „Beatos virtuvė: Kepinių knyga“ bei šių autorių internetiniai puslapiai su receptais. Receptai yra pasirinkti atsitiktinai ir kokybinė analizė yra pritaikyta, nes jie visi yra apibūdinami ir palyginami tam, kad būtų galima atrasti skirtumus ir panašumus. Šiame darbe pristatomas ir išsamiai apibūdinamas daugelio autorių aptartas žanro terminas ir žanro analizė. Taip pat apžvelgiamas receptų žanras ir jo istorinė kaita ir čia galima pastebėti, kad patobulinimai technikos srityje padarė įtaką receptų struktūrai. Be to, įvairių autorių mintys, kaip turėtų atrodyti receptai, atskleidžia, kad šis žanras turi aiškią struktūrą ir jo dalys gali būti gana tiksliai apibrėžtos. Galiausiai yra pristatoma multimodalumo tema, kuri šiandien yra labai svarbi, nes vaizdinės informacijos yra vis dažniau pateikiama knygose ir internetiniuose puslapiuose. Oliverio ir Nicholson receptų, esančių knygose ir internetiniuose puslapiuose, struktūros analizė patvirtina įvairių autorių mintis, kad šis žanras turi aiškias dalis. Daugiausiai skirtumų randama tarp receptų, esančių knygose ir internetiniuose puslapiuose. Receptų knygose tekstas susideda iš šių dalių... [toliau žr. visą tekstą]

Positron emission tomography (PET) as a nuclear medicine technique offers very high sensitivity compared with other imaging modalities. PET is often combined with computed tomography (CT) to offer structural information. Incorporation of a positron emitting metal radioisotope such as 68Ga requires a bifunctional chelator (BFC) to form a stable complex in vivo and for covalent bond formation (conjugation) with a targeting moiety. In this work, the routinely used macrocyclic BFC DOTA has been modified to replace an acetate arm with a benzimidazole unit to give an alternative BFC structure. It does not affect coordination number, has a secondary amine for bioconjugation and is a chromophore, therefore increasing the number of potential applications beyond DOTA to include optical properties (fluorescence and lanthanide luminescence sensitisation). Four benzimidazole DO3A derivatives have been synthesised and radiolabelled with 68Ga in comparable radiochemical yields to the structurally similar DOTA. Reaction conditions of ca. 5 minutes at room temperature are shorter and milder than those required for DOTA, which can cause degradation of conjugated biomolecules. Magnetic resonance imaging (MRI) can replace CT as the structural partner in clinical PET imaging and has many advantages including the potential for use of MRI contrast agents. PET/MRI is an emerging field and there is scope for the development of multimodal imaging constructs combining a PET radioisotope with a MRI contrast agent. In this work, super-paramagnetic iron oxide nanoparticles (SPIONs) as T2 MRI contrast agents have been functionalised with a range of macrocyclic derivatives and radiolabelled with 68Ga in near quantitative yields to form PET/MRI multi-modal imaging agents which have been shown to be stable to EDTA competition and in serum over four hours. Modification of the surface of the SPIONs was shown to have no detrimental effect to their clinical applicability by size variation (aggregation). The use of the chelators synthesised in the work for other applications is also of interest, with preliminary studies carried out towards the development of 86Y and 90Y agents for PET and radioimmunotherapy respectively. Benzimidazole DO3A, along with the corresponding yttrium(III) and europium(III) complexes were studied by potentiometric titration, luminescent lifetime measurements and variable temperature NMR to fully characterise the coordination sphere and gain insight into their physicochemical characteristics.

Free radicals are defined as molecules with one or more unpaired electrons in their outer orbits. They have been implicated in a large number of disease states and consequently there is increasing interest in detecting them in vivo. Having an un-cancelled electron spin, free radicals are amenable to magnetic resonance experiments. For reasons of sensitivity commercially available electron paramagnetic resonance (EPR) spectrometers operate in the X-band (9 GHz). Such frequencies are unsuitable for large biological samples because of excessive electromagnetic losses. This thesis describes the development of a radio frequency continuous wave (RFCW) EPR spectrometer operating around 280 MHz suitable for in vivo studies. The instrument is based around an existing low field NMR imager. The spectrometer includes both automatic frequency control and automatic coupling systems to combat the problems of animal motion. The instrument has been able to detect free radicals in living animals. PEDRI is a technique that can provide high resolution images showing free radical distribution in living systems. The method is based on conventional pulsed NMR imaging combined with dynamic nuclear polarisation The disadvantage of PEDRI is that it is difficult to obtain spectral information such as EPR line-width and g-factor. These parameters are easy to obtain by CW-EPR, and can give useful information. A further development was the combining of the RF CW-EPR instrument with a PEDRI imager to produce a multimodality instrument capable of sequential PEDRI and CW-EPR on the same sample. Switch-over between the two modes of operation takes less than 5 seconds. This instrument combines the advantages of the two types of free radical detection in a single instrument providing an extremely useful and flexible tool.

L'objectif de cette thèse était de développer des outils multifonctionnels pour l’élaboration d’agents théranostiques en oncologie, notamment à base de nanotechnologies. Le projet est basé sur l’utilisation de plateformes multimodales monomoléculaires comme précurseurs pour la synthèse de nanoparticules, de bioconjugués ou encore de sondes imageantes à base de petites molécules.Les premiers travaux ont concerné la synthèse de précurseurs silanes comportant un agent chélatant. Ces nouveaux précurseurs ont été mis en oeuvre pour la fonctionnalisation de nanoparticules AGuIX et ont également été utilisés dans un nouveau procédé de synthèse « one-pot » d’AGuIX. Le radiomarquage de ces nanoparticules a été réalisé dans le but d'évaluer la stabilité de ces nanoparticules dans des conditions physiologiques. Par ailleurs, ces nanoparticules ont été étudiées dans un modèle de tumeur TSA afin de déterminer leur comportement pharmacocinétique, leur biodistribution et leur potentiel en imagerie bimodale TEP-IRM.Dans une deuxième partie, des blocs de construction / ligands ont été développés et utilisés avec succès pour la fonctionnalisation de nanoparticules d'or. Des ligands multimodaux contenant différentes combinaisons de chélateurs, de fonctions de ciblage, de groupes fonctionnels et de chromophores, ont été synthétisés. Les nanoparticules d'or pour l'imagerie TEP-IRM et SPECT (ciblant le PSMA) ont été ensuite radiomarquées (64Cu/111In) et ont montré une excellente stabilité. Le ligand à base de PSMA présente une bonne affinité vis-à-vis du PSMA, tandis que les études in vivo dans un modèle tumoral TSA des nanoparticules d'or pour l'IRM-TEP ont démontré les bonnes propriétés pharmacocinétiques de ces nanoparticules fonctionnalisées, et donc leur potentiel en imagerie et en radiothérapie.La dernière partie du travail de thèse concerne le développement de sondes TEP-Optique pour l'imagerie ciblée de tumeurs surexprimant le PSMA en utilisant de petites sondes moléculaires, ainsi que pour leur bioconjugaison sur des fragments d'anticorps et des nanoparticules. Une sonde ciblant le PSMA, à base de NODAGA et du fluorophore IR-783, a été développée et caractérisée afin de déterminer ses propriétés photophysiques, son affinité pour le PSMA, et les conditions de radiomarquage. Enfin, un analogue comportant une fonction maléimide a été utilisé avec succès pour le marquage site spécifique de fragments Fab' du trastuzumab, ainsi que pour la fonctionnalisation de nanoparticules AGuIX. Ces bioconjugués et ces nanoparticules ont été soigneusement caractérisés et évalués dans des modèles biologiques pertinents afin de mettre en évidence leur intérêt comme agents théranostiques
The objective of this thesis was to develop multifunctional tools for cancer theranosis, particularly based on nanotechnology. The work is based on the premise of monomolecular multimodal platform that aims to create the building blocks that can be used for the synthesis of functionalized nanoparticles, bioconjugates or targeted small molecular imaging probes.The initial work focused on the synthesis of the chelator based silane precursors. These novel precursors were employed for the functionalization of AGuIX nanoparticles and have also been implicated in a one pot AGuIX synthesis. Radiolabelling of these nanoparticles was performed with the aim to evaluate the stability of these nanoparticles under physiological conditions. Furthermore, these nanoparticles were investigated in TSA tumor model in order to evaluate their pharmacokinetics, biodistribution behaviour and value in PET-MRI imaging.In the second part of the work, building blocks/ligands have been developed and successfully deployed for the functionalization of gold nanoparticles. Multimodal ligands containing different combinations of chelators, targeting moieties, functional groups and dyes were synthesized. Gold nanoparticles for PET-MRI and PSMA targeting SPECT imaging were further radiolabeled (64Cu/111In) and showed excellent stability. PSMA targeting ligand exhibited good affinity towards PSMA, whereas the in vivo studies of the gold nanoparticles for PET-MRI in TSA tumor model demonstrated good pharmacokinetics properties, underscoring their potential in cancer imaging and radiotherapy.The latter half of the thesis work focusses on the development of PET-Optical probes for PSMA positive cancer targeting imaging using small molecular probes, and for bioconjugation to antibody fragments and nanoparticles. A PSMA targeting probe based on NODAGA and IR-783 dye has been developed and characterized for its affinity, photophysical and radiolabelling properties. Likewise, a similar probe bearing a maleimide handle has been successfully used for the site-specific labelling of trastuzumab Fab’ fragments and functionalization of AGuIX nanoparticles. These bioconjugates and the nanoparticles have been thoroughly characterized and evaluated in relevant biological models to highlight their value in cancer theranosis

In this thesis, the four types of math problems that occur in the digital educational math game Zcooly Affären 2 have been analyzed. Focus has been aimed at how the problems mediate instructions and mathematical concepts to players, and how the mathematical content is supported by the context within the game. The result shows both positive and negative aspects of the math problems, based on the theories for math education used in this thesis. Apart from the analysis of the game, this thesis also presents earlier research regarding what possibilities digital games possess as tools for learning. Researchers have partly different views on how digital games should be designed to enhance learning, and whether or not games are efficient to use for learning purposes.

Through the method Action Research this essay investigates the implementation of multimodality and images as a tool for teaching. The main aim of this essay is to evaluate the effectiveness of assignments that incorporate digital imagery as a means to broaden the students’ world view as well as their proficiency in the target language