To see the other types of publications on this topic, follow the link: Dati multimodali.
Journal articles on the topic 'Dati multimodali'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Dati multimodali.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Fatigante, Marilena, Cristina Zucchermaglio, Francesca Alby, and Mariacristina Nutricato. "La struttura della prima visita oncologica: uno studio conversazionale." PSICOLOGIA DELLA SALUTE, no. 1 (January 2021): 53–77. http://dx.doi.org/10.3280/pds2021-001005.
Full textAbstract:
La prima visita in oncologia è un evento istituzionale (Drew e Heritage, 1992) altamente complesso sia per medico che per paziente e accompagnatori coinvolti. Il lavoro presenta uno studio conversazionale su un corpus di prime visite oncologiche volto all'esame delle distinte fasi che compongono questo evento. Il corpus di dati è costituito da 36 video registrazioni di visite oncologiche, condotte da due oncologi senior in due differenti ospedali romani. Lo stu-dio aderisce alla prospettiva teorico-metodologica dell'Analisi Conversazionale (Schegloff, 2007), in particolare applicata al contesto medico (Heritage e Maynard, 2006). A partire da indicatori empirici discorsivi e multimodali, sono state identificate 8 fasi attraverso le quali, con diverse durate e complessità, si realizza la visita oncologica: Apertura, Anamnesi, Presentazio-ne della malattia, Stadiazione, Indicazione di trattamento, Prescrizioni e Chiusura. L'analisi qualitativa mostra come paziente e accompagnatore si orientino ai passaggi tra fasi distinte, cooperando con il medico ad assolvere la specifica agenda di attività della visita. Sono discusse le implicazioni dello studio per la ricerca e la comprensione delle forme di partecipazione e strategie di empowerment disponibili ai pazienti per fronteggiare la complessità comunicativa dell'incontro con l'oncologo e ridurre possibili stati d'ansia che si associano a questo evento.
2
Karatayli-Ozgursoy, S., J. A. Bishop, A. T. Hillel, L. M. Akst, and S. R. Best. "Tumori maligni delle ghiandole salivari della laringe: un'unica review istituzionale." Acta Otorhinolaryngologica Italica 36, no. 4 (August 2016): 289–94. http://dx.doi.org/10.14639/0392-100x-807.
Full textAbstract:
I tumori a istotipo salivare della laringe sono molto rari, con pochi report in letteratura in merito al loro andamento clinico. Nel presente manoscritto discutiamo un'esperienza di 10 anni presso una singola struttura. Abbiamo condotto una review retrospettiva della casistica di un centro di oncologia della testa e del collo di terzo livello. I pazienti sono stati individuati mediante analisi di un database e sono stati revisionati da un Anatomo Patologo testa collo. I dati inerenti la clinica, le modalità di trattamento e gli esiti sono stati prelevati da archivi elettronici. Sono stati inclusi sei pazienti nello studio, con un range di età dai 44 ai 69 anni. Tutti e sei erano affetti da neoplasie maligne a istotipo salivare della laringe. Gli istotipi includevano: tre carcinomi adenoido-cistici (2 sopraglottico, 1 sottoglottico), un carcinoma mucoepidermoidale (sopraglottico), un carcinoma epiteliale-mioepiteliale (sopraglottico), e un adenocarcinoma (transglottico). Tutti sono stati sottoposti a trattamento chirurgico (2 chirurgie laser, 4 open) e 5 dei 6 pazienti sono stati successivamente sottoposti a terapia adjuvante (4 a radioterapia, 1 a radio-chemioterapia concomitante). Un paziente era fumatore; nessun paziente aveva storia di abuso di alcolici. A un follow-up con mediana di 4,5 anni nessuno dei pazienti ha presentato recidiva o metastasi locali o a distanza. I tumori a istotipo salivare della laringe si presentano solitamente in pazienti della seconda/terza età, e possono essere trattati con successo mediante approcci multimodali, con un ottimo controllo locoregionale di malattia.
3
Moneglia, Massimo. "Le unità di informazione Parentetiche alla periferia destra del Comment nella Teoria della Lingua in Atto." DILEF. Rivista digitale del Dipartimento di Lettere e Filosofia, no. 1 (March 27, 2022): 88–123. http://dx.doi.org/10.35948/dilef/2022.3294.
Full textAbstract:
AbstractSecondo la teoria della Lingua in Atto, le unità di informazione parentetiche inseriscono nell’enunciato informazioni poste su un piano locutivo secondario rispetto alla relazione Topic/Comment. Possono apparire in tutte le posizioni, ma non in prima posizione e non sono mai composizionali con un’altra unità di informazione. Le Parentesi poste alla periferia destra del Comment possono essere confuse con le unità di Appendice, in quanto la realizzazione prosodica risulta simile, nonostante le Appendici esprimano informazioni ridondanti che invece integrano il Comment. Sulla base del Data Base dell’Articolazione dell’Informazione IPIC il lavoro esplora le ragioni che consentono l’individuazione dei Parentetici nei Corpora di parlato spontaneo Italiano. Dati i loro valori semantici, le valutazioni modali e i commenti metalinguistici (che sono la grande maggioranza delle parentesi brevi nel discorso) introducono un salto nella prospettiva dell’enunciato che li colloca automaticamente su un piano locutivo secondario e non possono mai essere Appendici. Al contrario, altri tipi di espressioni che anche riempiono l’unità Parentetica (congiunzioni, se-frasi, avverbi e argomenti esterni, parentesi lunghe) possono essere in linea di principio considerate integrazioni del Comment. Il lavoro sostiene che l'interpretazione tra parentesi di queste unità di informazione risulta sottodeterminata a meno che il parlante non segnali il valore parentetico attraverso segnali prosodici o multimodali. Secondo i risultati dell’analisi condotta, inoltre, le parentesi lunghe in posizione finale non vengono realizzate come unità informative dell’enunciato ma piuttosto come "enunciati parentetici", pienamente autonomi. According to the Language Into Act Theory, Parenthetical units insert in the Utterance information that is placed on a secondary locutive plan with respect to the Topic / Comment relation. They can appear in all but not in the first position of the utterance and are never compositional with another information unit. Because of prosodic similarities, however, Parenthesis placed at the right periphery of the Comment can be confused with Appendix units, which on the contrary express redundant information complementing the Comment. On the basis of the IPIC Information Structure Data-Base the paper explore the reasons which allow their detection in Italian Spontaneous Speech Corpora. Because of their semantic values, Modal evaluations and Metalinguistic commentaries (the large majority of Parenthesis in speech) introduce a jump in the perspective of the utterance, which places them on a secondary locutive plan and can never be Appendixes. On the contrary other types of expressions that can also fill the Parenthetical unit (conjunctions, if-sentences, adverbials and external arguments, long parenthetical) can be in principle also integrations of the Comment. The paper argues that the parenthetical interpretation of these information units will remain under-determined unless the speaker signals this value through prosodic or multimodal cues. Moreover, according to our finding, long parenthesis in the final position are not performed as information units of the utterance but rather as fully autonomous “parenthetical utterance”.
4
Spaliviero, Camilla. "Teaching Italian as a second language through digital storytelling: Students’ perceptions towards izi.TRAVEL." EuroAmerican Journal of Applied Linguistics and Languages 9, no. 1 (April 10, 2022): 91–121. http://dx.doi.org/10.21283/2376905x.15.1.265.
Full textAbstract:
EN The use of technology-enhanced language learning, representing an urgent issue due to the Covid-19 pandemic, has also been promoted by many studies in second language acquisition. Nevertheless, research in this field is only partially developed for the teaching of Italian as a second language (L2) within the university context and for investigating students’ perceptions. This article presents an action research project on the use of izi.TRAVEL, a website housing more than 15,000 audio guides for touring various sites in cities around the world. The aim of the study is to contribute to developing didactic practices relative to Italian as an L2 through digital storytelling, in order to raise and foster students’ linguistic and digital skills. Participants were a small group of students studying Italian as an L2 as part of a master’s program at an Italian university. Data were collected through a questionnaire, a focus group, and students’ multimodal artifacts. Results show the positive impact of project participation on students’ attitudes and perceived learning outcomes, as well as improvements in linguistic, cultural, environmental, and digital competences. Key words: TEACHING ITALIAN AS AN L2, DIGITAL STORYTELLING, IZI.TRAVEL, STUDENTS’ PERCEPTIONS, ACTION RESEARCH ES El uso de la tecnología para la adquisición lingüística es promovido por varios estudios, y constituye una cuestión urgente en el contexto de la pandemia del Covid-19. Sin embargo, este tema de investigación se ha desarrollado solo parcialmente respecto a la enseñanza del italiano como L2 en la universidad y a las percepciones del alumnado. En este artículo se presenta un proyecto de investigación-acción sobre el uso de izi.TRAVEL, un sitio web que tiene más de 15,000 audioguías para recorrer varios sitios de ciudades de todo el mundo. Izi.TRAVEL se implementó con un pequeño grupo de estudiantes de italiano como L2 matriculados en un Máster internacional de una universidad italiana. El objetivo del estudio es desarrollar prácticas didácticas del italiano como L2 a través de la narrativa digital para fomentar las habilidades lingüísticas y digitales del alumnado. Los datos se recogieron a través de un cuestionario, un grupo focal y los productos multimodales del alumnado. Los resultados muestran el impacto positivo de la participación en el proyecto en las actitudes del alumnado y en sus resultados de aprendizaje percibidos, así como mejoras en sus competencias lingüística, cultural, ambiental y digital. Palabras clave: ENSEÑAR ITALIANO L2, NARRATIVA DIGITAL, IZI.TRAVEL, PERCEPCIONES DE LOS ESTUDIANTES, INVESTIGACIÓN-ACCIÓN IT L’uso delle tecnologie per l’apprendimento linguistico è promosso da diversi studi e si è andato imponendo come questione urgente ne contesto della pandemia Covid-19. Tuttavia, in quest’ambito risultano ancora scarsi gli studi sulla didattica dell’italiano L2 nel contesto universitario e dalla prospettiva degli studenti. Su queste basi, nell’articolo si presenta un progetto di ricerca-azione riguardante l’uso di izi.TRAVEL, un sito web che contiene più di 15,000 audioguide per visitare le città di tutto il mondo. Il progetto è stato pilotato con un piccolo gruppo di apprendenti di italiano L2 iscritti in un Master internazionale di un’università italiana. L’obiettivo dello studio è quello di contribuire allo sviluppo di pratiche didattiche sull’italiano L2 attraverso il Digital Storytelling per promuovere le competenze linguistiche e digitali degli apprendenti. I dati sono stati raccolti attraverso un questionario, un focus group e i prodotti multimodali degli studenti. I risultati rivelano che l’impatto della partecipazione nel progetto sugli atteggiamenti degli/delle apprendenti e sui risultati dell’apprendimento percepiti è stato positivo poiché migliorano le competenze linguistiche, culturali, ambientali e digitali. Parole chiave: INSEGNARE ITALIANO L2, DIGITAL STORYTELLING, IZI.TRAVEL, PERCEZIONI DEGLI STUDENTI, RICERCA-AZIONE
5
Deng, Wan-Yu, Dan Liu, and Ying-Ying Dong. "Feature Selection and Classification for High-Dimensional Incomplete Multimodal Data." Mathematical Problems in Engineering 2018 (August 12, 2018): 1–9. http://dx.doi.org/10.1155/2018/1583969.
Full textAbstract:
Due to missing values, incomplete dataset is ubiquitous in multimodal scene. Complete data is a prerequisite of the most existing multimodality data fusion methods. For incomplete multimodal high-dimensional data, we propose a feature selection and classification method. Our method mainly focuses on extracting the most relevant features from the high-dimensional features and then improving the classification accuracy. The experimental results show that our method produces considerably better performance on incomplete multimodal data such as ADNI dataset and Office dataset, compared to the case of complete data.
6
Amundrud, Thomas. "Multimodal knowledge building in a Japanese secondary English as a foreign language class." Multimodality & Society 2, no. 1 (March 2022): 64–85. http://dx.doi.org/10.1177/26349795221081300.
Full textAbstract:
Multimodal analysis examines how different modes, such as space, gesture, and language, instantiate meaning together. In this paper, a Systemic Functional-Multimodal Discourse Analysis demonstrates how teachers enact their pedagogy with their students across modes through what is represented experientially, how relationships between people are construed interpersonally, and how coherent texts are realized textually. This paper is a preliminary study of classroom data from a larger project looking at the multimodal pedagogy of Japanese secondary school teachers of English through the paired lenses of Systemic Functional-Multimodal Discourse Analysis and Legitimation Code Theory. It demonstrates how methods from these perspectives may be productively combined. How this teacher builds cumulative knowledge multimodally can be uncovered through the analysis of pedagogic register (Rose, 2018) and exchange (Berry, 1981; Martin and Rose, 2007), as well as classroom space and representing and textual action (Amundrud, 2017; Martin and Zappavigna, 2019). How both gesture and dialogic exchange between the teacher and students modulate the contextual relation of the knowledge construed in class is also explored via semantic gravity, which looks at how closely connected knowledge practices are to their context (Maton, 2014). As a preliminary study, the paper closes with limitations and future directions for this pedagogic multimodality research.
7
Wan, Huan, Hui Wang, Bryan Scotney, Jun Liu, and Wing W. Y. Ng. "Within-class multimodal classification." Multimedia Tools and Applications 79, no. 39-40 (August 11, 2020): 29327–52. http://dx.doi.org/10.1007/s11042-020-09238-1.
Full textAbstract:
Abstract In many real-world classification problems there exist multiple subclasses (or clusters) within a class; in other words, the underlying data distribution is within-class multimodal. One example is face recognition where a face (i.e. a class) may be presented in frontal view or side view, corresponding to different modalities. This issue has been largely ignored in the literature or at least under studied. How to address the within-class multimodality issue is still an unsolved problem. In this paper, we present an extensive study of within-class multimodality classification. This study is guided by a number of research questions, and conducted through experimentation on artificial data and real data. In addition, we establish a case for within-class multimodal classification that is characterised by the concurrent maximisation of between-class separation, between-subclass separation and within-class compactness. Extensive experimental results show that within-class multimodal classification consistently leads to significant performance gains when within-class multimodality is present in data. Furthermore, it has been found that within-class multimodal classification offers a competitive solution to face recognition under different lighting and face pose conditions. It is our opinion that the case for within-class multimodal classification is established, therefore there is a milestone to be achieved in some machine learning algorithms (e.g. Gaussian mixture model) when within-class multimodal classification, or part of it, is pursued.
8
Silvestri, Katarina, Mary McVee, Christopher Jarmark, Lynn Shanahan, and Kenneth English. "Multimodal positioning of artifacts in interaction in a collaborative elementary engineering club." Multimodal Communication 10, no. 3 (December 1, 2021): 289–309. http://dx.doi.org/10.1515/mc-2020-0017.
Full textAbstract:
Abstract This exploratory case study uses multimodal positioning analysis to determine and describe how a purposefully crafted emergent artifact comes to influence and/or manipulate social dynamics, structure, and positionings of one design team comprised of five third-graders in an afterschool elementary engineering and literacy club. In addition to social semiotic theories of multimodality (e.g., Kress, G. (2010). Multimodality: a social semiotic approach to contemporary communication. New York, NY: Routledge) and multimodal interactional analysis (Norris, S. (2004). Analyzing multimodal interaction: a methodological framework. New York, NY: Routledge, Norris, S. (2019). Systematically working with multimodal data: research methods in multimodal discourse analysis. Hoboken, NJ: Wiley-Blackwell), Positioning Theory (Harré, R. and Van Langenhove, L. (1991). Varieties of positioning. J. Theor. Soc. Behav. 21: 393–407) is used to examine group interactions with the artifact, with observational data collected from audio, video, researcher field notes, analytic memos, photographs, student artifacts (e.g., drawn designs, built designs), and transcriptions of audio and video data. Analysis of interactions of the artifact as it unfolds demonstrates multiple types of role-based positioning with students (e.g., builder, helper, idea-sharer). Foregrounding analysis of the artifact, rather than the student participants, exposed students’ alignment or opposition with their groupmates during the project. This study contributes to multimodal and artifactual scholarship through a close examination of positions emergent across time through multimodal communicative actions and illustrates how perspectives on multimodality may be analytically combined with Positioning Theory.
9
Farías, Miguel, and Leonardo Véliz. "Multimodal Texts in Chilean English Teaching Education: Experiences From Educators and Pre-Service Teachers." Profile: Issues in Teachers´ Professional Development 21, no. 2 (July 1, 2019): 13–27. http://dx.doi.org/10.15446/profile.v21n2.75172.
Full textAbstract:
Drawing on 10 pedagogical standards issued by the Chilean Ministry of Education, three dealing with multimodality, we, in this research, examined English language pre-service teachers’ and educators’ approaches to the use of multimodal texts. Data were gathered through two online surveys that explored the use of multimodal texts by teacher educators and pre-service teachers. Results indicate that educators were familiar with the standards and multimodality when teaching reading and writing, but lack of resources, preparation, and time prevents them from working with multimodal texts. Candidates read printed and digital newspapers, novels, and magazines outside university, but rarely use them academically. They extensively use social media, even for academic purposes. There is a mismatch between the use of multimodal texts by teacher candidates and teacher educators.
10
Abdullah, Fuad, Arini Nurul Hidayati, Agis Andriani, Dea Silvani, Ruslan Ruslan, Soni T. Tandiana, and Nina Lisnawati. "Fostering students’ Multimodal Communicative Competence through genre-based multimodal text analysis." Studies in English Language and Education 9, no. 2 (May 23, 2022): 632–50. http://dx.doi.org/10.24815/siele.v9i2.23440.
Full textAbstract:
The multiplicity of semiotic resources employed in communication, the rapid advancement of information, communication, and technology (ICT), and burgeoning interdisciplinary research into multimodality have led to a paradigmatic shift from a mono-modal to the multimodal perspective of communication. Conversely, actualising multimodal concepts in teaching and learning practises remains underexplored, notably in developing the students’ multimodal communicative competence (MCC). For this reason, this study endeavoured to probe genre-based multimodal text analysis in fostering the students’ MCC. Grounded on Action Research (AR), the present study facilitated students to cultivate their MCC through the activities of Genre-based multimodal text analysis (hereafter, GBMTA). Practically speaking, students performed the analysing practises in the course at an English Education Department of a state university in Tasikmalaya, West Java, Indonesia, namely Grammar in Multimodal Discourse (GiMD. Four Indonesian EFL students were recruited as the participants. The data were collected through semi-structured interviews and analysed with thematic analysis. The findings showed that the students could: (1) build their knowledge on multimodality, (2) engage with theoretical and practical learning activities, (3) assign analytical and reflective task-based learning activities, and (4) provide constructive feedback about their learning performances, and (5) raise awareness of the contributions of multimodality to prospective English teachers’ competences. The main implication of this study is the promotion of increased awareness of deploying multimodal aspects to English language teaching, learning, and investigative practises to attain optimum MCC.
11
Basson, Abigail R., Fabio Cominelli, and Alexander Rodriguez-Palacios. "‘Statistical Irreproducibility’ Does Not Improve with Larger Sample Size: How to Quantify and Address Disease Data Multimodality in Human and Animal Research." Journal of Personalized Medicine 11, no. 3 (March 23, 2021): 234. http://dx.doi.org/10.3390/jpm11030234.
Full textAbstract:
Poor study reproducibility is a concern in translational research. As a solution, it is recommended to increase sample size (N), i.e., add more subjects to experiments. The goal of this study was to examine/visualize data multimodality (data with >1 data peak/mode) as cause of study irreproducibility. To emulate the repetition of studies and random sampling of study subjects, we first used various simulation methods of random number generation based on preclinical published disease outcome data from human gut microbiota-transplantation rodent studies (e.g., intestinal inflammation and univariate/continuous). We first used unimodal distributions (one-mode, Gaussian, and binomial) to generate random numbers. We showed that increasing N does not reproducibly identify statistical differences when group comparisons are repeatedly simulated. We then used multimodal distributions (>1-modes and Markov chain Monte Carlo methods of random sampling) to simulate similar multimodal datasets A and B (t-test-p = 0.95; N = 100,000), and confirmed that increasing N does not improve the ‘reproducibility of statistical results or direction of the effects’. Data visualization with violin plots of categorical random data simulations with five-integer categories/five-groups illustrated how multimodality leads to irreproducibility. Re-analysis of data from a human clinical trial that used maltodextrin as dietary placebo illustrated multimodal responses between human groups, and after placebo consumption. In conclusion, increasing N does not necessarily ensure reproducible statistical findings across repeated simulations due to randomness and multimodality. Herein, we clarify how to quantify, visualize and address disease data multimodality in research. Data visualization could facilitate study designs focused on disease subtypes/modes to help understand person–person differences and personalized medicine.
12
Rita, Rita. "Penyusunan Peooman Pemberian Izin Badan Usaha Angkutan Multimoda (BUAM)." Warta Penelitian Perhubungan 24, no. 6 (May 14, 2019): 567. http://dx.doi.org/10.25104/warlit.v24i6.1041.
Full textAbstract:
Freight grawth lately is remarkable progress, so we need measures to optimize the transport of goods. In order to support the national logistics system the government has issued Government Regulation No. 8 of 2011 on Transport Multimioda which is the mandate of the Traffic Act and Road Transport, Shipping, Aviation and Railways. Problems in multimodal transportation is not a business entity licensing guidelines multimodal transport. With the approach of qualitative descriptive analysis of the utilization of multimodal transport in the Port of Tanjung Priok Jakarta and Medan Belawan port indicated that the licensing guidelines Multimodal Transport Enterprises really need to be designed. The draft guidelines for licensing Multimodal Transport Entity shall have permanent offices, transport equipment, loading and unloading equipment, and human resources that have a certificate of competence.Keywords: Buam licensing guidelines. Pertumbuhan angkutan barang akhir-akhir ini sangat luar biasa perkembangannya, sehingga perlu dilakukan langkah-langkah uiltuk mengoptimasikan angkutan barang tersebut. Dalam rangka mendukung sistem logistik nasional pemerintah telah menerbitkan Peraturan Pemerintah Nomor 8 Tahun 2011 tentang Angkutan Multimioda yang merupakan amanat dari Undang-Undang Lalu Lintas dan Angkutan Jalan, Pelayaran, Penerbangan dan Perkeretaapian. Permasalahan dalam angkutan multimoda adalah belum adanya pedoman pemberian izin badan usaha angkutan multimoda. Dengan pendekatan analisis deskriptif kualitatif terhadap pengusahaan angkutan multimoda di Pelabuhan Tanjung Priok Jakarta, dan Pelabuhan Belawan Medan diperoleh gambaran bahwa pedoman pemberian izin Badan Usaha Angkutan Multimoda sangat perlu untuk dirancang. Rancangan pedoman pemberian izin Badan Usaha Angkutan Multimoda wajib memiliki kantor tetap, peralatan angkut, peralatan bongkar muat, dan sumber daya manusia yang memiliki sertifikat kompetensi.Kata Kunci: Pedoman Pemberian izin BUAM.
13
Mu, Su, Meng Cui, and Xiaodi Huang. "Multimodal Data Fusion in Learning Analytics: A Systematic Review." Sensors 20, no. 23 (November 30, 2020): 6856. http://dx.doi.org/10.3390/s20236856.
Full textAbstract:
Multimodal learning analytics (MMLA), which has become increasingly popular, can help provide an accurate understanding of learning processes. However, it is still unclear how multimodal data is integrated into MMLA. By following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, this paper systematically surveys 346 articles on MMLA published during the past three years. For this purpose, we first present a conceptual model for reviewing these articles from three dimensions: data types, learning indicators, and data fusion. Based on this model, we then answer the following questions: 1. What types of data and learning indicators are used in MMLA, together with their relationships; and 2. What are the classifications of the data fusion methods in MMLA. Finally, we point out the key stages in data fusion and the future research direction in MMLA. Our main findings from this review are (a) The data in MMLA are classified into digital data, physical data, physiological data, psychometric data, and environment data; (b) The learning indicators are behavior, cognition, emotion, collaboration, and engagement; (c) The relationships between multimodal data and learning indicators are one-to-one, one-to-any, and many-to-one. The complex relationships between multimodal data and learning indicators are the key for data fusion; (d) The main data fusion methods in MMLA are many-to-one, many-to-many and multiple validations among multimodal data; and (e) Multimodal data fusion can be characterized by the multimodality of data, multi-dimension of indicators, and diversity of methods.
14
Ilmi, Khalif Arfazul, and Dian Novita Dewi. "The Integration of Multimodality in EFL Classes: Students’ Perception." Jo-ELT (Journal of English Language Teaching) Fakultas Pendidikan Bahasa & Seni Prodi Pendidikan Bahasa Inggris IKIP 9, no. 2 (December 31, 2022): 150. http://dx.doi.org/10.33394/jo-elt.v9i2.6343.
Full textAbstract:
Technology has greatly influenced education in the twenty-first century. Multimodal teaching is one of many classroom methods brought by technology. However, only a few studies have been conducted on how to create a multimodal pedagogy to assist EFL students in enhancing their language skills. Therefore, the purpose of this study was to investigate the perceptions of high school English students regarding the integration of multimodality into English classroom practices. Participation in this study was determined using a random sample. Data was gathered from 100 high school students in SMA Panjura, Malang. These students have experienced using multimodal in their learning process for six weeks. In this research, a quantitative method with a survey research design was chosen. The data were analyzed using SPSS 24 and the data was presented descriptively. The study revealed that the majority of students had a positive attitude toward the implementation of multimodality in their learning process in response to the characteristics of current students.
15
Haniya, Samaa, Anastasia Olga Tzirides, Matthew Montebello, Keratso Georgiadou, Bill Cope, and Mary Kalantzis. "Maximizing Learning Potential with Multimodality: A Case Study." World Journal of Educational Research 6, no. 2 (May 4, 2019): 260. http://dx.doi.org/10.22158/wjer.v6n2p260.
Full textAbstract:
<p><em>In today’s increasingly fast-moving digital world, learners are immersed in multimodal online communication environments in their daily life, through Facebook, Twitter, Instagram, Snapchat and others. This requires educators to reflect the environment in which these learners live, and thus design instructional practices from a multimodal perspective. Multimodality offers new opportunities for digital learners to express themselves, analyze problems and make meaning in multimodal ways as they interpret knowledge differently according to their various educational needs (Kalantzis </em><em>&</em><em> Cope, 2015). In this paper we will discuss the significance of integrating multimodality in e-Learning contexts to make meaning and improve learning. The paper will also present a case study of an online course from the College of Education at the University of Illinois Urbana Champaign to show how multimodality works in practice to cater to learner differences by offering a range of activity options and modes of meaning. We will also examine learners’ perceptions of adopting such an approach in the online course. We used survey techniques for data collection and quantitative and qualitative methods for data analysis. Results revealed illuminating insights about the importance of multimodality approach to increase learning potential for digital learners and provided suggestions for future iterations.</em></p>
16
Li, Zhigang, Aimei Dong, and Jing Zhou. "Research of Low-Rank Representation and Discriminant Correlation Analysis for Alzheimer’s Disease Diagnosis." Computational and Mathematical Methods in Medicine 2020 (March 19, 2020): 1–8. http://dx.doi.org/10.1155/2020/5294840.
Full textAbstract:
As population aging is becoming more common worldwide, applying artificial intelligence into the diagnosis of Alzheimer’s disease (AD) is critical to improve the diagnostic level in recent years. In early diagnosis of AD, the fusion of complementary information contained in multimodality data (e.g., magnetic resonance imaging (MRI), positron emission tomography (PET), and cerebrospinal fluid (CSF)) has obtained enormous achievement. Detecting Alzheimer’s disease using multimodality data has two difficulties: (1) there exists noise information in multimodal data; (2) how to establish an effective mathematical model of the relationship between multimodal data? To this end, we proposed a method named LDF which is based on the combination of low-rank representation and discriminant correlation analysis (DCA) to fuse multimodal datasets. Specifically, the low-rank representation method is used to extract the latent features of the submodal data, so the noise information in the submodal data is removed. Then, discriminant correlation analysis is used to fuse the submodal data, so the complementary information can be fully utilized. The experimental results indicate the effectiveness of this method.
17
BASYSTIUK, Oleh, and Nataliia MELNYKOVA. "MULTIMODAL SPEECH RECOGNITION BASED ON AUDIO AND TEXT DATA." Herald of Khmelnytskyi National University. Technical sciences 313, no. 5 (October 27, 2022): 22–25. http://dx.doi.org/10.31891/2307-5732-2022-313-5-22-25.
Full textAbstract:
Systems of machine translation of texts from one language to another simulate the work of a human translator. Their performance depends on the ability to understand the grammar rules of the language. In translation, the basic units are not individual words, but word combinations or phraseological units that express different concepts. Only by using them, more complex ideas can be expressed through the translated text. The main feature of machine translation is different length for input and output. The ability to work with different lengths of input and output provides us with the approach of recurrent neural networks. A recurrent neural network (RNN) is a class of artificial neural network that has connections between nodes. In this case, a connection refers to a connection from a more distant node to a less distant node. The presence of connections allows the RNN to remember and reproduce the entire sequence of reactions to one stimulus. From the point of view of programming, such networks are analogous to cyclic execution, and from the point of view of the system, such networks are equivalent to a state machine. RNNs are commonly used to process word sequences in natural language processing. Usually, a hidden Markov model (HMM) and an N-program language model are used to process a sequence of words. Deep learning has completely changed the approach to machine translation. Researchers in the deep learning field has created simple solutions based on machine learning that outperform the best expert systems. In this paper was reviewed the main features of machine translation based on recurrent neural networks. The advantages of systems based on RNN using the sequence-to-sequence model against statistical translation systems are also highlighted in the article. Two machine translation systems based on the sequence-to-sequence model were constructed using Keras and PyTorch machine learning libraries. Based on the obtained results, libraries analysis was done, and their performance comparison.
18
Freschi, Ana, and Suzi Cavalari. "Corrective Feedback and Multimodality: Rethinking Categories in Telecollaborative Learning." TESL Canada Journal 37, no. 2 (December 2, 2020): 154–80. http://dx.doi.org/10.18806/tesl.v37i2.1335.
Full textAbstract:
Teletandem (Telles, 2009) is a model of telecollaboration in which pairs of foreign language students from different countries meet regularly and virtually to learn each other’s languages. Within this context, participants are expected to help their partners learn by providing feedback. The multimodal nature of this type of environment, however, may offer different learning opportunities (Guichon & Cohen, 2016) and have an impact on feedback provision. This research aims at investigating peer corrective feedback in Teletandem in relation to the different modes. Using a case study approach, we describe how three Brazilians offered feedback to learners of Portuguese as a foreign language. Data used came from 20 Teletandem oral sessions that took place over a period of three years and were stored in MulTeC (Aranha & Lopes, 2019). Data analysis revealed that CF provision is characterized by reformulations, with a blurred distinction between recasts and explicit corrections due to a combination of multimodal strategies. Results also indicate that error correction may be more (or less) emphasized depending on how interlocutors combine multimodal resources. Pedagogical implications are discussed.
Teletandem (Telles, 2009) est un modèle de télécollaboration par lequel des paires d’étudiants de langue étrangère originaires de différents pays se rencontrent régulièrement de façon virtuelle pour apprendre les langues des uns des autres. Dans ce contexte, on s’attend à ce que les participants aident leur partenaire à apprendre en leur fournissant de la rétroaction. La nature multimodale de ce type d’environnement peut, cependant, offrir diverses occasions d’apprentissage (Guichon & Cohen, 2016) et influencer la façon dont la rétroaction est fournie. Le but de cette recherche est d’étudier la rétroaction corrective par les pairs dans Teletandem par rapport aux différents modes. En se servant d’une approche par étude de cas, nous décrivons comment trois Brésiliens ont offert de la rétroaction à des apprenants de portugais langue étrangère. Les données utilisées venaient de 20 sessions orales de Teletandem qui s’étaient déroulées sur une période de trois ans et ont été stockées sur MulTeC (Aranha & Lopes, 2019). L’analyse des données a révélé que la fourniture de rétroaction corrective se caractérise par des reformulations, qui ne distinguent pas très bien entre des refontes et des corrections explicites, ce qui est dû à une combinaison de stratégies multimodales. Les résultats indiquent également que la correction des erreurs peut être plus ou moins soulignée selon la façon dont les interlocuteurs combinent les ressources multimodales. On discute des implications pédagogiques.
19
Shimek, Courtney. "Recursive readings and reckonings: kindergarteners’ multimodal transactions with a nonfiction picturebook." English Teaching: Practice & Critique 20, no. 2 (February 22, 2021): 149–62. http://dx.doi.org/10.1108/etpc-07-2020-0068.
Full textAbstract:
Purpose Our world had always been multimodal, but studying how young children enact and embody literacy practices, especially reading, has often been overlooked. The purpose of this study was to examine how young children respond to nonfiction picturebooks in multimodal ways. This paper aims to answer the question: What multimodal resources do readers use to respond to and construct meaning from nonfiction picturebooks? Design/methodology/approach Undergirded by Rosenblatt’s transactional theory of reading and social semiotic multimodality, a 9-min video clip of three boys making sense of one nonfiction picturebook during reading workshop was analyzed using Norris’ approach to multimodal data analysis. This research stemmed from a five-month-long case study of one kindergarten class’s multimodal and collective responses to nonfiction picturebooks. Findings Findings demonstrate how readers use gesture, gaze and proxemics in addition to language to signal agreement with one another, explain new ideas or concepts to one another and incorporate their background knowledge. In addition to reading images, the children learned to read each other. Originality/value This research indicates that reading is inherently multimodal, recursive and complex and provides implications for teachers to reconsider what kinds of responses they prioritize in their classrooms. Additionally, this research establishes the need to better understand how readers respond to nonfiction books and a broader examination of multimodality in the literacy curriculum.
20
Bilytska, Viktoriya M., Oksana R. Andriiashyk, Yaroslav V. Tsekhmister, Olha V. Pavlenko, and Iryna V. Savka. "Multimodal Interaction in a Foreign Language Class at Higher Education Institutions of Ukraine." Journal of Curriculum and Teaching 11, no. 1 (January 17, 2022): 218. http://dx.doi.org/10.5430/jct.v11n1p218.
Full textAbstract:
Multimodality is implemented to the modern learning environment in line with trends towards multidisciplinarity. In the current study, multimodal interaction is based on the mutual integration of understanding of multimodality in philological and pedagogical perspectives. The purpose of the article was to analyze and compare the results of learning a foreign language (German) for professional purposes (German for Economists) with an emphasis on multimodal interaction and without it (in a way of traditional language learning with a predominance of classical methods of classroom and extracurricular activities). There were universal scientific and specific methods used: a controlled-type educational experiment; Likert-scale type questionnaire; reliability test: Cronbach’s alpha using IBM SPSS Statistics 28.0.0.0; qualitative-quantitative interpretation and contrastive-comparative analysis of the obtained experimental data; statistical-mathematical interpretation of empirical data; comparative analysis; the functional analysis. Respondents of empirical intelligence were students of the Faculty of Management and Marketing, specialty 073 “Management”. Averagely in the experimental group, almost all the assessing criteria of the effectiveness of multimodal interaction outreached 4 points. These data were also confirmed by the results of self-reflection-questionnaire. The novelty of the research is in the principle of theoretical substantiation and practical application of the content of multimodal interaction as an umbrella term that integrates the most fundamental concepts of modern pedagogy in general and, in particular, methods of teaching a foreign language.
21
Palomeque, Cristina, and Joan-Tomàs Pujolà. "Managing multimodal data in virtual world research for language learning." ReCALL 30, no. 2 (January 24, 2018): 177–95. http://dx.doi.org/10.1017/s0958344017000374.
Full textAbstract:
AbstractThe study of multimodality in communication has attracted the attention of researchers studying online multimodal environments such as virtual worlds. Specifically, 3D virtual worlds have especially attracted the interest of educators and academics due to the multiplicity of verbal channels, which are often comprised of text and voice channels, as well as their 3D graphical interface, allowing for the study of non-verbal modes. This study offers a multilayered transcription method called the Multi-Modal MUVE Method or 3M Method (Palomeque, 2016; Pujolà & Palomeque, 2010) to account for the different modes present in the 3D virtual world of Second Life. This method works at two levels: the macro and the micro level. The macro level is a bird’s-eye view representation of the whole session as it fits into one page. This enables the researcher to grasp the essence of the class and to identify interesting sequences for analysis. The micro level consists of three transcripts to account for the different communication modes as well as the interface activity that occurs in the virtual world of Second Life. This paper will review the challenges when dealing with multimodal analysis in virtual worlds and how the multimodal data were analyzed and interpreted by using a multilayered multimodal method of analysis (3M transcription). Examples will be provided in the study to show how different modes of communication were used by participants in the virtual world of Second Life to create meaning or to avoid communication breakdowns.
22
Ladilova, Anna. "Multimodal Metaphors of Interculturereality / Metaforas multimodais da interculturealidade." REVISTA DE ESTUDOS DA LINGUAGEM 28, no. 2 (May 5, 2020): 917. http://dx.doi.org/10.17851/2237-2083.28.2.917-955.
Full textAbstract:
Abstract: The present paper looks at the interactive construction of multimodal metaphors of interculturereality – a term coined by the author from interculturality and intercorporeality, assuming that intercultural interaction is always an embodied phenomenon, shared among its participants. For this, two videotaped sequences of a group conversation are analyzed drawing upon interaction analysis (Couper-Kuhlen; Selting, 2018). The data was transcribed following the GAT2 (Selting et al., 2011) guidelines, including gesture form annotation, which relied on the system described by Bressem (2013). Gesture function was interpreted drawing on the interactional context and on the system proposed by Kendon (2004) and Bressem and Müller (2013). The results question the validity of the classical conduit metaphor of communication (Reddy, 1979) in the intercultural context and instead propose an embodied approach to the conceptualization of the understanding process among the participants. The analysis also shows that even though the metaphors are multimodal, the metaphoric content is not always evenly distributed among the different modalities (speech, gesture). Apart from that, the metaphorical content is constructed sequentially, referring to preceding metaphors used by the same or different interlocutors and associated with metaphorical blends.Keywords: metaphors; multimodality; interculturality; intercorporeality; migration.Resumo: O presente artigo analisa a construção interativa de metáforas multimodais da interculturealidade – um termo proveniente da interculturalidade e intercorporealidade, assumindo que a interação intercultural é sempre um fenômeno incorporado, compartilhado entre os seus participantes. Para tal, duas sequências gravadas em vídeo de uma conversa em grupo serão analisadas com base na análise da interação (COUPER-KUHLEN; SELTING, 2018). Os dados foram transcritos seguindo as orientações do sistema GAT2 (SELTING et al., 2011), incluindo a anotação da forma gestual, que se baseou no sistema descrito por Bressem (2013). A função dos gestos foi interpretada com base no contexto interacional e no sistema proposto por Kendon (2004) e Bressem e Müller (2013). Os resultados questionam a validade da metáfora clássica do conduto de comunicação (REDDY, 2012) no contexto intercultural e, além disso, propõem uma abordagem corporificada da conceituação do processo de entendimento entre os participantes. A análise também mostra que, embora as metáforas sejam multimodais, o conteúdo metafórico não é sempre uniformemente distribuído entre as diferentes modalidades (fala, gesto). Além disso, o conteúdo metafórico é construído sequencialmente, referindo-se a metáforas anteriores utilizadas pelos mesmos ou diferentes interlocutores e recorrendo a mesclagens metafóricas.Palavras-chave: metáforas; multimodalidade; interculturalidade; intercorporealidade; migração.
23
Kang, Young-Jin, and Yoojeong Noh. "Development of Hartigan’s Dip Statistic with Bimodality Coefficient to Assess Multimodality of Distributions." Mathematical Problems in Engineering 2019 (December 28, 2019): 1–17. http://dx.doi.org/10.1155/2019/4819475.
Full textAbstract:
In general, although some random variables such as wind speed, temperature, and load are known to have multimodal distributions, input or output random variables are considered to follow unimodal distributions without assessing the unimodality or multimodality of distributions from samples. In uncertainty analysis, estimating unimodal distribution as multimodal distribution or vice versa can lead to erroneous analysis results. Thus, whether a distribution is unimodal or multimodal must be assessed before the estimation of distributions. In this paper, the bimodality coefficient (BC) and Hartigan’s dip statistic (HDS), which are representative methods for assessing multimodality, are introduced and compared. Then, a combined HDS with BC method is proposed. The proposed method has the advantages of both BC and HDS by using the skewness and kurtosis of samples as well as the dip statistic through a link function between the BC values in BC and significance level in HDS. To verify the performance of the proposed method, statistical simulation tests were conducted to evaluate the multimodality for various unimodal, bimodal, and trimodal models. The implementation of the proposed method to real engineering data is shown through case studies. The results demonstrate that the proposed method is more accurate, robust, and reliable than the BC and original HDS alone.
24
Lobato-Delgado, Barbara, Blanca Priego-Torres, and Daniel Sanchez-Morillo. "Combining Molecular, Imaging, and Clinical Data Analysis for Predicting Cancer Prognosis." Cancers 14, no. 13 (June 30, 2022): 3215. http://dx.doi.org/10.3390/cancers14133215.
Full textAbstract:
Cancer is one of the most detrimental diseases globally. Accordingly, the prognosis prediction of cancer patients has become a field of interest. In this review, we have gathered 43 state-of-the-art scientific papers published in the last 6 years that built cancer prognosis predictive models using multimodal data. We have defined the multimodality of data as four main types: clinical, anatomopathological, molecular, and medical imaging; and we have expanded on the information that each modality provides. The 43 studies were divided into three categories based on the modelling approach taken, and their characteristics were further discussed together with current issues and future trends. Research in this area has evolved from survival analysis through statistical modelling using mainly clinical and anatomopathological data to the prediction of cancer prognosis through a multi-faceted data-driven approach by the integration of complex, multimodal, and high-dimensional data containing multi-omics and medical imaging information and by applying Machine Learning and, more recently, Deep Learning techniques. This review concludes that cancer prognosis predictive multimodal models are capable of better stratifying patients, which can improve clinical management and contribute to the implementation of personalised medicine as well as provide new and valuable knowledge on cancer biology and its progression.
25
Zou, Zhiqiang, Hongyu Gan, Qunying Huang, Tianhui Cai, and Kai Cao. "Disaster Image Classification by Fusing Multimodal Social Media Data." ISPRS International Journal of Geo-Information 10, no. 10 (September 24, 2021): 636. http://dx.doi.org/10.3390/ijgi10100636.
Full textAbstract:
Social media datasets have been widely used in disaster assessment and management. When a disaster occurs, many users post messages in a variety of formats, e.g., image and text, on social media platforms. Useful information could be mined from these multimodal data to enable situational awareness and to support decision making during disasters. However, the multimodal data collected from social media contain a lot of irrelevant and misleading content that needs to be filtered out. Existing work has mostly used unimodal methods to classify disaster messages. In other words, these methods treated the image and textual features separately. While a few methods adopted multimodality to deal with the data, their accuracy cannot be guaranteed. This research seamlessly integrates image and text information by developing a multimodal fusion approach to identify useful disaster images collected from social media platforms. In particular, a deep learning method is used to extract the visual features from social media, and a FastText framework is then used to extract the textual features. Next, a novel data fusion model is developed to combine both visual and textual features to classify relevant disaster images. Experiments on a real-world disaster dataset, CrisisMMD, are performed, and the validation results demonstrate that the method consistently and significantly outperforms the previously published state-of-the-art work by over 3%, with a performance improvement from 84.4% to 87.6%.
26
Zhang, Maqun, Feng Gao, Tiange Zhang, Yanhai Gan, Junyu Dong, and Hui Yu. "Attention Fusion of Transformer-Based and Scale-Based Method for Hyperspectral and LiDAR Joint Classification." Remote Sensing 15, no. 3 (January 21, 2023): 650. http://dx.doi.org/10.3390/rs15030650.
Full textAbstract:
In recent years, there have been many multimodal works in the field of remote sensing, and most of them have achieved good results in the task of land-cover classification. However, multi-scale information is seldom considered in the multi-modal fusion process. Secondly, the multimodal fusion task rarely considers the application of attention mechanism, resulting in a weak representation of the fused feature. In order to better use the multimodal data and reduce the losses caused by the fusion of different modalities, we proposed a TRMSF (Transformer and Multi-scale fusion) network for land-cover classification based on HSI (hyperspectral images) and LiDAR (Light Detection and Ranging) images joint classification. The network enhances multimodal information fusion ability by the method of attention mechanism from Transformer and enhancement using multi-scale information to fuse features from different modal structures. The network consists of three parts: multi-scale attention enhancement module (MSAE), multimodality fusion module (MMF) and multi-output module (MOM). MSAE enhances the ability of feature representation from extracting different multi-scale features of HSI, which are used to fuse with LiDAR feature, respectively. MMF integrates the data of different modalities through attention mechanism, thereby reducing the loss caused by the data fusion of different modal structures. MOM optimizes the network by controlling different outputs and enhances the stability of the results. The experimental results show that the proposed network is effective in multimodality joint classification.
27
Bateman, John A. "Growing theory for practice: empirical multimodality beyond the case study." Multimodal Communication 11, no. 1 (September 30, 2021): 63–74. http://dx.doi.org/10.1515/mc-2021-0006.
Full textAbstract:
Abstract Many studies investigating the use and effectiveness of multimodal communication are now confronting the need to engage with larger bodies of data in order to achieve more empirically robust accounts, moving beyond the earlier prevalence of small-scale ‘case studies’. In this article, I briefly characterise how recent developments in the theory of multimodality can be drawn upon to encourage and support this change in both scale and breadth. In particular, the contribution will show how refinements in the degree of formality of definitions of the core multimodal constructs of ‘semiotic mode’ and ‘materiality’ can help bridge the gap between exploratory investigations of complex multimodal practices and larger-scale corpus studies.
28
Guenier, Amily, and Ge Min. "Navigating the Digital World." International Journal of Computer-Assisted Language Learning and Teaching 12, no. 5 (October 10, 2022): 1–12. http://dx.doi.org/10.4018/ijcallt.313441.
Full textAbstract:
This study explores a multimodal approach to teaching contemporary Chinese culture to foster university students' intercultural awareness and intercultural communication competence via a third space. Two universities in the UK took part in the study where the course contents moved from static notions of culture-as-fact in terms of national traditions to digital presentation of and live discussion about contemporary China. The pedagogy includes discussing Chinese celebrities' digital videos and films on digital platforms, and students' digital presentations in multimodal modes. Findings from students' comments in the module evaluations, students' reflective essays, and lecturers' observations prove the viability of this approach, and the data analyzed via themes address the function of the third space, the application of multimodality, and approaches to intercultural awareness and intercultural communication competence. The paper suggests that multimodality can be an effective approach to advancing theory and practice in future contemporary culture teaching.
29
Zhe, Guo. "Observation and Reflection of English Intensive Reading Classroom from the Perspective of Multimodal Discourse Analysis." Journal of English Language and Literature 14, no. 1 (August 31, 2020): 1248–55. http://dx.doi.org/10.17722/jell.v14i1.1181.
Full textAbstract:
The application of Multimodal Discourse Analysis to language teaching classroom adapts to the requirements put forward by the multimodality environment in modern social life, since various multimodal teaching strategies are employed to enhance students' language skills. However, problems exist in the efficient collocation between various modes. Classroom observation is employed to collect the data, and the framework proposed by Zhang Delu (2016) is followed to conduct the Multimodal Discourse Analysis of the Intensive Reading class of the juniors of English majors in North China Electric Power University. Finally, the problems from the aspects of cultural context, contextual context, interactive context, modal and media are identified, and the suggestions to improve language teaching practice are raised accordingly in terms of teachers' employment of modality and the teaching design.
30
Dutschk, Beate, Philipp Ernst, and Michael Heizmann. "Registrierung von multimodalen Sensordaten für die Oberflächeninspektion / Registration of multimodal sensor data for surface inspection." tm - Technisches Messen 86, s1 (September 1, 2019): 72–76. http://dx.doi.org/10.1515/teme-2019-0036.
Full textAbstract:
ZusammenfassungDa vermehrt mehrere Sensorsysteme oder auch Multisensor-Systeme zur Charakterisierung von Oberflächen eingesetzt werden, kann durch eine geeignete Kombination der erhaltenen Ergebnisse ein holistisches Ergebnis erzielt werden. Die Grundlage dafür bildet eine passende Registrierung der unterschiedlichen Messungen der Oberflächentopografie. Dieser Beitrag beschreibt ein mögliches Verfahren für die Registrierung von multimodalen Sensordaten. Dabei wird für diesen Anwendungsfall zuerst eine Konturenextraktion zur Merkmalsgewinnung für eine grobe Registrierung eingesetzt, welche mit einem flächenbasierten Verfahren auf Basis der Transinformation zur feinen Registrierung ergänzt wird. Die Kombination kann die Zuverlässigkeit der Registrierung erhöhen, da die einzelnen Verfahren für sich alleine in manchen Fällen keine aussagekräftigen Ergebnisse liefern.
31
Fiftinova, Fiftinova, Nandang Heryana, and Ida Rosmalina. "Exploring Lecturers’ Standpoints in Composing Digital Fiction and Students’ Multimodal Literacy Level." Register Journal 15, no. 2 (September 7, 2022): 264–83. http://dx.doi.org/10.18326/rgt.v15i2.264-283.
Full textAbstract:
Multimodality which encourages the combination of text, image, sound, and videos could be varied from class to class. Multimodal literacy as a new dimension of literacy in the 21st century has emerged as a critical skill that EFL students must develop, given its role as a source of meaning in communication. The purpose of this study was to identify the level of students’ multimodal literacy and to identify lecturers’ standpoints on students’ multimodal literacy. This study was conducted both quantitatively and qualitatively and involved 71 EFL students who took creative writing subject in an English education program in one state university in South Sumatera. The data were collected by distributing a questionnaire from Bulut et al. (2015) and by interviewing 3 lecturers who taught the subject. Descriptive statistics and one-way ANOVA were used to determine mean, standard deviation and differences in terms of gender and classes with different lecturer; qualitative data were thematically analyzed to categorize the themes. The results indicated that students’ multimodal literacy level was categorized high as indicated by the mean of each aspect of questionnaire: 4.22, 4.11 and 3.6 respectively. There was no different level of multimodal literacy between male and female students, and different lecturers with different instructions did not influence the level.. Finally, the lecturers perceived positively to students’ multimodal literacy . Similarly, referring to the lecturers’ view, the students gave positive attitude towards multimodal writing and hence making their multimodal digital fiction successful. Keywords: Multimodal Literacy, Digital Fiction, Creative Writing, EFL
32
S. Gomathy, K. P. Deepa, T. Revathi, and L. Maria Michael Visuwasam. "Genre Specific Classification for Information Search and Multimodal Semantic Indexing for Data Retrieval." SIJ Transactions on Computer Science Engineering & its Applications (CSEA) 01, no. 01 (April 5, 2013): 10–15. http://dx.doi.org/10.9756/sijcsea/v1i1/01010159.
Full text
33
Brunner, Marie-Louise, and Stefan Diemer. "Multimodal meaning making: The annotation of nonverbal elements in multimodal corpus transcription." Research in Corpus Linguistics 10, no. 1 (2021): 63–88. http://dx.doi.org/10.32714/ricl.09.01.05.
Full textAbstract:
The article discusses how to integrate annotation for nonverbal elements (NVE) from multimodal raw data as part of a standardized corpus transcription. We argue that it is essential to include multimodal elements when investigating conversational data, and that in order to integrate these elements, a structured approach to complex multimodal data is needed. We discuss how to formulate a structured corpus-suitable standard syntax and taxonomy for nonverbal features such as gesture, facial expressions, and physical stance, and how to integrate it in a corpus. Using corpus examples, the article describes the development of a robust annotation system for spoken language in the corpus of Video-mediated English as a Lingua Franca Conversations (ViMELF 2018) and illustrates how the system can be used for the study of spoken discourse. The system takes into account previous research on multimodality, transcribes salient nonverbal features in a concise manner, and uses a standard syntax. While such an approach introduces a degree of subjectivity through the criteria of salience and conciseness, the system also offers considerable advantages: it is versatile and adaptable, flexible enough to work with a wide range of multimodal data, and it allows both quantitative and qualitative research on the pragmatics of interaction.
34
Syaifuddin S, M., Djatmika, Diah Kristina, and Tri Wiratno. "SPEECH ACT OF CRITICIZING, PERSUADING THE AUDIENTS BELIEVES ON “WAR MACHINE” FILM: A PRAGMATIC MUTIMODAL PERSPECTIVE." JEELL (Journal of English Education, Linguistics and Literature) 5, no. 1 (February 12, 2018): 90. http://dx.doi.org/10.32682/jeell.v5i1.952.
Full textAbstract:
This aims to reveal the ways of criticizing American government and wheedling the audients believe trough each speech act in movie. Pragmatic multimodal perspective is used to uncover both speech act and the cinematic aspect in order to know how the movie producer criticize and persuade audients to believe. This research is presented Qualitatively and the data taken from David Michôd (2016) work “War Machines”. It was found that the movie has three ways in criticizing the government and persuading the audien to believe. They are; 1. To contradict between speech and multimodality of cinematic to create an offensive criticism structure. 2. Use of mutually supportive between speech acts of and multimodality to create a common criticism structure (mutual affirmation of speech acts and multimodality). 3. The use of personification of institutions to direct criticism of policy holders.
35
Prihantoro, Prihantoro, Didi Suherdi, and Ahmad Bukhori Muslim. "Developing Students’ Multimodal Communicative Competence Through Multiliteracies Pedagogy." ENGLISH FRANCA : Academic Journal of English Language and Education 6, no. 2 (August 28, 2022): 191. http://dx.doi.org/10.29240/ef.v6i2.5242.
Full textAbstract:
With the rise of globalization, digitization, and the emergence of the fourth industrial revolution, meaning-making modes are now increasingly multimodal in which written linguistics modes of meaning interface with oral, visual, audio, gestural, tactile, and spatial patterns of meaning. In the EFL context, this situation unavoidably urges changes in the goal of English language teaching to include multimodal communicative competence. In response to such challenges, multiliteracies pedagogy offers plausible solutions. In a multiliteracies classroom, students are supported through a pedagogy that provides them with ample opportunities to develop their knowledge and skills to multimodally examine diverse information sources and content, critically explore the information, and become active meaning-makers and effective communicators. This study aims at investigating the implementation of multiliteracies pedagogy in English language teaching to foster students’ multimodal communicative competence. This is a qualitative case study involving twenty students of the English language department of IAIN Curup. The data of the study are gained through observation, interviews, and students’ artefacts. The data collected from these instruments were examined qualitatively using thematic and categorical coding. The result of this study demonstrates the enactment of multiliteracies pedagogy can develop five competences of multimodal communicative competence: linguistic competence, sociocultural competence, interactional competence, discourse competence, and multimodal competence. Meanwhile the other two competences, strategic competence and formulaic competence, are somewhat hindered due to lack of exposure to authentic English language.
36
Ouyang, Qianhua, Yi Yu, and Ai Fu. "Building disciplinary knowledge through multimodal presentation." APTIF 9 - Reality vs. Illusion 66, no. 4-5 (July 29, 2020): 655–73. http://dx.doi.org/10.1075/babel.00176.ouy.
Full textAbstract:
Abstract Digital innovations are revolutionizing education, bringing opportunities that are seized across disciplines including conference interpreting training. This research draws a transdisciplinary framework of Legitimation Code Theory and multimodality research to explore how to build and transfer the disciplinary knowledge of interpreting via an on-line course, a staple of today’s education. The paper first conceptualizes the disciplinary knowledge of interpreting as elite code that entails both specialist knowledge of high semantic density and tacit experience of professionals of the trade. Then, drawing on empirical data from the first interpreting MOOC in China, the paper describes how knowledge of different semantic features is built through distinctive patterns of multimodal presentation. Effectiveness of the multimodal presentation of knowledge is then triangulated with learning outcome research. Findings of this paper highlight how multimodal presentation in on-line lectures support the process of learning and hence elicit reflective perspectives on knowledge building of interpreting in the on-line space.
37
LEE, Vivian. "A platform for developing student competence: a look at the online Korean classroom." Bulletin of the Transilvania University of Brasov. Series IV: Philology and Cultural Studies 14 (63), no. 1 (November 2021): 37–46. http://dx.doi.org/10.31926/but.pcs.2021.63.14.1.3.
Full textAbstract:
This paper looks at the online undergraduate classroom in a South Korean context. Due to the prolonged COVID 19 pandemic, educational institutions around the world have adapted to online learning for course delivery. In addition, recent years have seen an increase in attention to multimodality, and according to Stein (2000), an array of resources including language and other modes, such as gesture, sound, images, textures, and silences should be used when making meaning. Multimodality can play an important part in developing L2 learners’ awareness and sensitivity to language and meaning, for increased effective communication. The current study looks at the undergraduate learner in an online, multimodal classroom context. Students met online through Zoom weekly for their classes and participated in online discussions with the lecturer and their fellow peers. Results indicate L2 learners’ heightened awareness to both linguistic and multimodal aspects developed through the online learning platform. The details of the class and overall design will be outlined in the paper, as will examples and data excerpts. While the current study was based in a South Korean undergraduate classroom, the pedagogical method can be easily applied across different contexts and adapted to suit various classrooms and needs.
38
Mondada, Lorenza. ""Te:mos o serra cura::do, temos o ni:sa, temos o serpa do:p." Suggesting products to buy, shaping materiality and multisensoriality in shop encounters." Calidoscópio 19, no. 2 (September 3, 2021): 278–301. http://dx.doi.org/10.4013/cld.2021.192.08.
Full textAbstract:
Adopting the perspective of multimodal conversation analysis, the paper shows the methodic organization of an action, making suggestions, achieved by sellers in response to customers’ requests for recommendations in shop encounters, and involving the showing and listing of available products. This focus on a specific sequential environment and institutional ecology, enables an exemplary discussion of how this action is multimodally formatted, embedded in its context, and shaped in relation to objects as discursive referents as well as materialities to be pointed at, looked at, touched and sensed in multiple ways. More generally, this focus enables to address two sets of issues: on the one hand, it elucidates the nexus between action, institutionality and materiality, including the role of multisensoriality in engaging with the qualities of buyable objects. On the other hand, it addresses the nexus between action and referential practices for introducing and presenting new referents, within an interactional perspective locating these grammatical practices and their systematic features within their praxeological context. On the basis of video data recorded in a gourmet shop in Lisbon, Portugal, this double focus targets issues of sensoriality and socialization in food culture, as well as issues of grammar in interaction, casting some light on situated uses of the verb ter for introducing new referents. Keywords: Social Interaction; Shop Encounters; Multimodality and Multisensoriality.
39
BOYKO, N., and O. PETROVSKYI. "METHODS OF CLASSIFICATION OF MACHINE LEARNING FOR CONSTRUCTION OF MATHEMATICAL MODELS ON MULTIMODAL DATA." Herald of Khmelnytskyi National University. Technical sciences 307, no. 2 (May 2, 2022): 25–32. http://dx.doi.org/10.31891/2307-5732-2022-307-2-25-32.
Full textAbstract:
This article is dedicated to topic modeling as an unsupervised machine learning technique. It is analyzed how it seems possible to determine the topics of documents in order to categorize them further with the help of topic modeling methods. Such methods as latent semantic analysis, probabilistic latent semantic analysis and latent Dirichlet allocation are considered. An approach that allows the construction of effective topic models of text document collections in Ukrainian and other synthetic languages based on peculiarities of this linguistic language type is proposed, and its main stages are described. The proposed approach consists of a custom input data preprocessing pipeline, which covers file loading, text extraction, removal of improper symbols, tokenization, removal of stop-words, stemming of each token and a newly introduced model pruning stage, which makes any of the modern topic modeling methods applicable for synthetic language topic modeling. The approach was implemented in Python programming language and used to obtain the topic model of the collection of Ukrainian-language scientific publications on civic identity and related topics. An expert in political psychology, who studies the phenomenon of civic identity, was involved in the research for the topic model quality evaluation. As a result of expert evaluation of the topics singled out during the modeling, it was proposed to clarify the formulation of cluster names based on the semantics of the sets of words that form them. In general, according to the expert, the topics singled out represent the concept of the civic identity of an individual and will allow researchers to simplify the work with literature sources on this issue when used to categorize documents. This demonstrates the efficiency of the proposed approach.
40
Yanto, Elih Sutisna, and Hikmah Pravitasari. "A NARRATIVE INQUIRY OF STUDENT TEACHERS MULTIMODAL PRACTICE EXPERIENCES IN THE INDONESIAN ESP CLASSROOM CONTEXT." Wiralodra English Journal 4, no. 1 (April 10, 2020): 12–20. http://dx.doi.org/10.31943/wej.v4i1.81.
Full textAbstract:
Drawing on narrative inquiry aiming to understand the experience of student teachers learning context texts through different modes of meaning. This paper reports on the use of narrative frames as a mean of investigating the intersemiotic practice experiences of student teachers majoring in English education program in Karawang, West Java, Indonesia. Students in depth interview data revealed that multimodal practice has the potential to promote the role of student teachers as designers and navigators in reading English disciplinary texts. In addition, multimodal reading encourages the student teachers to become autonomous language learners. Moreover, multimodality reading tasks can scaffold students in comprehending academic language and in increasing students’ academic vocabulary development as well. These findings have implications for policy makers and teacher educators.
41
Masson, Caroline, Sandrine Laverdure, and Catherine Calderaro-Viel. "Étayage de l'adulte et multimodalité : étude exploratoire des modalités d'interaction dans le cadre d'une prise en charge d'un enfant avec retard de langage." Travaux neuchâtelois de linguistique, no. 66 (January 1, 2017): 143–56. http://dx.doi.org/10.26034/tranel.2017.2903.
Full textAbstract:
The aim of this article is to explore adult/child interactions in a therapeutic situation with an augmentative communication that integrates vocal, gestural and visual modes. This study has an interactionist and multimodal approach of language acquisition. Based on ecological and video data of a child with language delay from 3;09 to 4;05, we analyse efficient conditions for the process of language acquisition. On the one hand, we showed how multimodality created by the situation is part of adults' strategies in order to facilitate linguistics forms and, on the other hand, how some modalities of interaction are more efficient than others (i.e. expansions and extensions). The purpose is to establish scaffolding and adjustment schemes that contribute to the emergence of language, in addition to the multimodal support.
42
Pasquandrea, Sergio. "Managing multiple actions through multimodality: Doctors' involvement in interpreter-mediated interactions." Language in Society 40, no. 4 (September 2011): 455–81. http://dx.doi.org/10.1017/s0047404511000479.
Full textAbstract:
AbstractMany studies (Heath 1984b, 1986; Robinson 1998; Robinson & Stivers 2001; Greatbach 2006; Frers 2009) have shown that, while interacting with their patients, doctors fulfill several tasks (speaking, reading, writing prescriptions, reporting data on the computer, etc.) through an interplay of diverse multimodal resources (speech, gesture, body posture, gaze, object manipulation, etc.). In comparison, multimodality in interpreter-mediated medical encounters has received scant attention. This contribution examines a corpus of interactions involving Italian doctors, Chinese patients, and an interpreter. Having to manage multiparty and multitask conversations with patients with whom no direct communication is possible, doctors are forced to rely heavily on multimodality. The analysis here shows how the simultaneous actions performed by the doctors are coordinated through the use of multiple modalities. The outcomes on the global organization of interpreter-mediated interaction are also discussed. (Conversation analysis, doctor-patient interaction, interpreting, multitasking, multimodality)*
43
Zhang, Hong, and Brian Hok-Shing Chan. "Translanguaging in multimodal Macao posters: Flexible versus separate multilingualism." International Journal of Bilingualism 21, no. 1 (July 27, 2016): 34–56. http://dx.doi.org/10.1177/1367006915594691.
Full textAbstract:
Aims: This paper suggests a framework of separate and flexible multilingualism to describe multilingual phenomena in Macao. The aims are to capture both conventional and creative language practice and to explore what exactly is the state of multilingualism in modern Macao under the context of globalization, and more specifically how we can capture variation in multilingual practice. Methodology: The objectives are achieved by analyzing the interplay and distance between languages in multilingual texts, focusing on the multimodality and intertextuality of the texts. Data and analysis: The database is a collection of 300 posters for cultural and entertainment events in Macao. The distance of languages is analyzed at the unit level in multimodal texts; separate and flexible multilingualism are exemplified and further elaborated. Conclusions: Multilingualism in Macao is mainly characterized by separate multilingualism, where different languages are demarcated clearly. However, Macao is undergoing a significant process of globalization, accompanied by a huge flow of people, and concomitantly flexible multilingualism is emergent and coexistent with separate multilingualism. Flexible multilingualism is often manifested in translanguaging. The various practices of translanguaging are performances of creativity and they show criticality by problematizing the widely accepted essentialist conceptions on boundaries between languages and modes. Originality: This paper extends the framework of separate and flexible multilingualism to explain multilingual practice in general. We analyze multimodal data using a combined method of multimodality and multilingualism while focusing on the linguistic elements. The paper treats the posters as a special and less studied type of linguistic landscape in Macao, and it provides an original and realistic interpretation of the written multilingual linguistic landscape in a unique Chinese city. Significance: This paper provides a new way of understanding multilingualism; translanguaging is broadened to account for written data. Multilingualism can be understood better by observing language-related practice in multimodal texts.
44
Nariendra, Pradhana Wahyu, and Mohamad Iman Taufiq. "KINERJA ANGKUTAN PETI KEMAS RUTE KAWASAN INDUSTRI KABUPATEN BANDUNG–PELABUHAN TANJUNG PRIOK." Jurnal Transportasi 20, no. 1 (April 23, 2020): 11–18. http://dx.doi.org/10.26593/jtrans.v20i1.3850.11-18.
Full textAbstract:
Abstract Fast and inexpensive logistic services are an absolute requirement not only to meet consumer needs but also to satisfy consumers. In addition to having an impact on economic growth, fast and inexpensive logistical services can cause national products to compete with foreign products. To meet logistical needs in industrial estates in Bandung Regency area, transporting logistic goods (containers) from and to the Port of Tanjung Priok has 2 alternative modes of choice with different amount of costs, namely by using single-modal, via land transportation (trucks), or using multimodal, using truck and train modes. From the analysis, the lowest combined cost for various alternatives can be obtained. For the single-modal alternative, a combined cost of Rp6,843,162.00/trip is obtained, for a 20 ft container, and Rp7,982,609.00/trip, for a 40 ft container. While the lowest combined cost for multimodal alternatives is Rp6,523,847.00/trip for a 20 ft container and Rp8,151,686.00/trip for a 40 ft container. Keywords: logistics, multimodal, single-modal, train, truck, container Abstrak Pelayanan logistik yang cepat dan murah menjadi syarat mutlak tidak hanya untuk memenuhi kebutuhan konsumen tetapi juga harus memuaskan konsumen. Selain berdampak pada pertumbuhan ekonomi, pelayanan logistik yang cepat dan murah dapat menyebabkan produk-produk nasional dapat bersaing dengan produk-produk luar negeri. Untuk memenuhi kebutuhan logistik di kawasan industri di daerah Kabupaten Bandung, pengiriman barang logistik (peti kemas) dari dan ke Pelabuhan Tanjung Priok memiliki 2 alternatif pilihan moda dengan besaran biaya yang berbeda, yaitu dengan menggunakan unimoda, melalui transportasi darat (truk), atau menggunakan multimoda, menggunakan moda truk dan kereta api. Dari hasil analisis diperoleh biaya gabungan terendah untuk berbagai alternatif. Untuk alternatif unimoda, diperoleh biaya gabungan sebesar Rp6.843.162,00/trip, untuk peti kemas berukuran 20 ft, dan Rp7.982.609,00/trip, untuk peti kemas berukuran 40 ft. Sedangkan biaya gabungan terendah untuk alternatif multimoda adalah Rp6.523.847,00/trip untuk peti kemas berukuran 20 ft dan Rp8.151.686,00/trip untuk peti kemas berukuran 40 ft. Kata-kata kunci: logistik, multimoda, unimoda, kereta api, truk, peti kemas
45
Carlson, Rolf, and Björn Granström. "Data-driven multimodal synthesis." Speech Communication 47, no. 1-2 (September 2005): 182–93. http://dx.doi.org/10.1016/j.specom.2005.02.015.
Full text
46
Dyrmo, Tomasz. "Gestural metaphorical scenarios and coming out narratives." Metaphor and the Social World 12, no. 1 (February 11, 2022): 23–45. http://dx.doi.org/10.1075/msw.20023.dyr.
Full textAbstract:
Abstract This article extends the framework of metaphorical scenarios proposed by Musolff (2006, 2016) by adding a gestural component. Coming out videos, serving as the source of data for the present analysis, help to uncover the conceptual mechanisms that shape the understanding and conceptualisation of this phenomenon. The extended framework of gestural metaphorical scenarios reveals that conceptual metaphors create cognitively and communicatively coherent wholes that are expressed multimodally, via speech and gesture. The article proposes that coming out, a highly individualised process, is conceptualised at various levels by both generic and specific metaphors. The analysis shows that metaphorical variation is present not only at the level of lexical scenarios, but also at the level of gesture, giving rise to multimodal discourse fragments. The extended framework, therefore, might be useful in analysing multimodal discourse.
47
Seely Flint, Amy, Rebecca Rohloff, and Sarah Williams. "“I like the first slide. I like how we put it like that [words and pictures on a diagonal]:” composing multimodal texts in a grade four classroom." English Teaching: Practice & Critique 20, no. 3 (June 18, 2021): 277–97. http://dx.doi.org/10.1108/etpc-12-2019-0173.
Full textAbstract:
Purpose Young children often enter formal schooling with a range of digital experiences, including using apps on tablets and engaging with interactive educational toys. The convergence and increased accessibility of digital resources has made it more convenient for young children to navigate multiple modes (e.g. words, images, sound and movement) as they construct meaning across many different texts. The purpose of the study is to examine affordances and choices when students compose multimodal texts. Design/methodology/approach Three lines of inquiry support this study: the social construction of writing practices, multiliteracies and multimodality and intertextuality. Data analysis used an iterative two-tiered process of reading, rereading and coding students’ multimodal compositions and supplemental field notes (Creswell, 1998; Strauss and Corbin, 1998). Findings Analysis of the 23 multimodal compositions revealed three significant findings related to choice and affordances of multimodal texts: the popularity of Minecraft as a topic choice based on the social interactions of students; semiotic concurrence and semiotic complementarity and sophisticated use of literary techniques (e.g. nonlinear structures, shifting point of view, asides and emojis) across the multimodal stories, particularly those that carried Minecraft themes. Originality/value Students’ intentionality with the modes in their compositions suggested they were fully aware of the “complexity, interrelatedness and interdependence between image [animation and sound] and language” (Shanahan, 2013, p. 213).
48
Pranata, Birgitta Eifelia Panggaraan. "A Multimodal Analysis of H&M Video Commercial “A Magical Holiday”." K@ta Kita 7, no. 3 (December 16, 2019): 337–45. http://dx.doi.org/10.9744/katakita.7.3.337-345.
Full textAbstract:
In indirect advertising, an advertisement does not boldly show the products on the video. The object of the study in this thesis is a video advertisement of a clothing line, H&M, consisting a story about a family in the holiday season. This thesis aims to show that commercials of H&M “A Magical Holiday” uses multimodality in order to attract the audience through six modes: linguistic, audio, spatial, oral, visual, and gestural modes. In this study, I analyzed the modes by Multimodal Discourse Analysis by Kress and Leeuwen onto the six modes to analyze the advertisement, using the study by Chan and Chia on modes in multimodality. This research is using qualitative analysis by Schreier (2012) since it deals with the connection between all the semiotic modes to bring the message of the advertisement. I analyzed the data by putting the scenes into 6 tables based on the modes. After that, I analyzed the interrelation between each modes. Based on the analysis, the modes helped the advertisers to convey the real message of the video. Beyond the moral message of the story, the advertisement is a marketing tool to promote their products. Key Words: indirect advertising, multimodal discourse analysis, semiotic modes, H&M
49
Bao, Xiaoli. "Application of Multimodality to Teaching Reading." English Language and Literature Studies 7, no. 3 (August 29, 2017): 78. http://dx.doi.org/10.5539/ells.v7n3p78.
Full textAbstract:
To improve students’ reading ability is one of the fundamental requirements for English teaching for English majors. However, some English majors are not interested in reading English and lack motivation to learn it. Even teachers may lose enthusiasm to teach them English. As a result, teaching English reading is inefficient.Application of multimodality in teaching English has attracted many researchers’ attention; the author applies multimodality to teaching English reading and attempts to answer the following question: Is the application of multimodality to teaching English reading effective?An experiment is carried out in two parallel classes for a whole term. In the experimental class, multimodality is applied in teaching reading and teaching procedures are all designed according to the theory of elements of designing multimodality while in the control class, ordinary multimedia teaching is applied. The data of pre-test, reading quizzes, a post-test are analyzed by SPSS 16.0. The research finds out that on the one hand, the application of multimodality in teaching reading is indeed effective. On the other hand, multimodal teaching is more popular among English majors for it can help activate classroom atmosphere, inspire students’ motivation to read after class and build up their confidence in learn English, especially English reading. The author also provides the suggestions for English teaching based on the research findings.
50
Firdaus, Mauajama, Nidhi Thakur, and Asif Ekbal. "Aspect-Aware Response Generation for Multimodal Dialogue System." ACM Transactions on Intelligent Systems and Technology 12, no. 2 (March 2021): 1–33. http://dx.doi.org/10.1145/3430752.
Full textAbstract:
Multimodality in dialogue systems has opened up new frontiers for the creation of robust conversational agents. Any multimodal system aims at bridging the gap between language and vision by leveraging diverse and often complementary information from image, audio, and video, as well as text. For every task-oriented dialog system, different aspects of the product or service are crucial for satisfying the user’s demands. Based upon the aspect, the user decides upon selecting the product or service. The ability to generate responses with the specified aspects in a goal-oriented dialogue setup facilitates user satisfaction by fulfilling the user’s goals. Therefore, in our current work, we propose the task of aspect controlled response generation in a multimodal task-oriented dialog system. We employ a multimodal hierarchical memory network for generating responses that utilize information from both text and images. As there was no readily available data for building such multimodal systems, we create a Multi-Domain Multi-Modal Dialog (MDMMD++) dataset. The dataset comprises the conversations having both text and images belonging to the four different domains, such as hotels, restaurants, electronics, and furniture. Quantitative and qualitative analysis on the newly created MDMMD++ dataset shows that the proposed methodology outperforms the baseline models for the proposed task of aspect controlled response generation.