Dissertations / Theses on the topic 'Computational linguistic models'
To see the other types of publications on this topic, follow the link: Computational linguistic models.
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 dissertations / theses for your research on the topic 'Computational linguistic models.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse dissertations / theses on a wide variety of disciplines and organise your bibliography correctly.
1
Penton, Dave. "Linguistic data models : presentation and representation /." Connect to thesis, 2006. http://eprints.unimelb.edu.au/archive/00002875.
Full text
2
Tonkes, Bradley. "On the origins of linguistic structure : computational models of the evolution of language /." St. Lucia, Qld, 2001. http://www.library.uq.edu.au/pdfserve.php?image=thesisabs/absthe16529.pdf.
Full text
3
vanCort, Tracy. "Computational Evolutionary Linguistics." Scholarship @ Claremont, 2001. https://scholarship.claremont.edu/hmc_theses/137.
Full textAbstract:
Languages and species both evolve by a process of repeated divergences, which can be described with the branching of a phylogenetic tree or phylogeny. Taking advantage of this fact, it is possible to study language change using computational tree building techniques developed for evolutionary biology. Mathematical approaches to the construction of phylogenies fall into two major categories: character based and distance based methods. Character based methods were used in prior work in the application of phylogenetic methods to the Indo-European family of languages by researchers at the University of Pennsylvania. Discussion of the limitations of character-based models leads to a similar presentation of distance based models. We present an adaptation of these methods to linguistic data, and the phylogenies generated by applying these methods to several modern Germanic languages and Spanish. We conclude that distance based for phylogenies are useful for historical linguistic reconstruction, and that it would be useful to extend existing tree drawing methods to better model the evolutionary effects of language contact.
4
Evans, Owain Rhys. "Bayesian computational models for inferring preferences." Thesis, Massachusetts Institute of Technology, 2015. http://hdl.handle.net/1721.1/101522.
Full textAbstract:
Thesis: Ph. D. in Linguistics, Massachusetts Institute of Technology, Department of Linguistics and Philosophy, 2015.Cataloged from PDF version of thesis.
Includes bibliographical references (pages 130-131).
This thesis is about learning the preferences of humans from observations of their choices. It builds on work in economics and decision theory (e.g. utility theory, revealed preference, utilities over bundles), Machine Learning (inverse reinforcement learning), and cognitive science (theory of mind and inverse planning). Chapter 1 lays the conceptual groundwork for the thesis and introduces key challenges for learning preferences that motivate chapters 2 and 3. I adopt a technical definition of 'preference' that is appropriate for inferring preferences from choices. I consider what class of objects preferences should be defined over. I discuss the distinction between actual preferences and informed preferences and the distinction between basic/intrinsic and derived/instrumental preferences. Chapter 2 focuses on the challenge of human 'suboptimality'. A person's choices are a function of their beliefs and plans, as well as their preferences. If they have inaccurate beliefs or make inefficient plans, then it will generally be more difficult to infer their preferences from choices. It is also more difficult if some of their beliefs might be inaccurate and some of their plans might be inefficient. I develop models for learning the preferences of agents subject to false beliefs and to time inconsistency. I use probabilistic programming to provide a concise, extendable implementation of preference inference for suboptimal agents. Agents performing suboptimal sequential planning are represented as functional programs. Chapter 3 considers how preferences vary under different combinations (or &compositions') of outcomes. I use simple mathematical functional forms to model composition. These forms are standard in microeconomics, where the outcomes in question are quantities of goods or services. These goods may provide the same purpose (and be substitutes for one another). Alternatively, they may combine together to perform some useful function (as with complements). I implement Bayesian inference for learning the preferences of agents making choices between different combinations of goods. I compare this procedure to empirical data for two different applications.
by Owain Rhys Evans.
Ph. D. in Linguistics
5
Heiberg, Andrea Jeanine. "Features in optimality theory: A computational model." Diss., The University of Arizona, 1999. http://hdl.handle.net/10150/288983.
Full textAbstract:
This dissertation presents a computational model of Optimality Theory (OT) (Prince and Smolensky 1993). The model provides an efficient solution to the problem of candidate generation and evaluation, and is demonstrated for the realm of phonological features. Explicit object-oriented implementations are proposed for autosegmental representations (Goldsmith 1976 and many others) and violable OT constraints and Gen operations on autosegmental representations. Previous computational models of OT (Ellison 1995, Tesar 1995, Eisner 1997, Hammond 1997, Karttunen 1998) have not dealt in depth with autosegmental representations. The proposed model provides a full treatment of autosegmental representations and constraints on autosegmental representations (Akinlabi 1996, Archangeli and Pulleyblank 1994, Ito, Mester, and Padgett 1995, Kirchner 1993, Padgett 1995, Pulleyblank 1993, 1996, 1998). Implementing Gen, the candidate generation component of OT, is a seemingly intractable problem. Gen in principle performs unlimited insertion; therefore, it may produce an infinite candidate set. For autosegmental representations, however, it is not necessary to think of Gen as infinite. The Obligatory Contour Principle (Leben 1973, McCarthy 1979, 1986) restricts the number of tokens of any one feature type in a single representation; hence, Gen for autosegmental features is finite. However, a finite Gen may produce a candidate set of exponential size. Consider an input representation with four anchors for each of five features: there are (2⁴ + 1)⁵, more than one million, candidates for such an input. The proposed model implements a method for significantly reducing the exponential size of the candidate set. Instead of first creating all candidates (Gen) and then evaluating them against the constraint hierarchy (Eval), candidate creation and evaluation are interleaved (cf. Eisner 1997, Hammond 1997) in a Gen-Eval loop. At each pass through the Gen-Eval loop, Gen operations apply to create the minimal number of candidates needed for constraint evaluation; this candidate set is evaluated and culled, and the set of Gen operations is reduced. The loop continues until the hierarchy is exhausted; the remaining candidate(s) are optimal. In providing explicit implementations of autosegmental representations, constraints, and Gen operations, the model provides a coherent view of autosegmental theory, Optimality Theory, and the interaction between the two.
6
Gwei, G. M. "New models of natural language for consultative computing." Thesis, University of Nottingham, 1987. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.378986.
Full text
7
Clark, Stephen. "Class-based statistical models for lexical knowledge acquisition." Thesis, University of Sussex, 2001. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.341541.
Full textAbstract:
This thesis is about the automatic acquisition of a particular kind of lexical knowledge, namely the knowledge of which noun senses can fill the argument slots of predicates. The knowledge is represented using probabilities, which agrees with the intuition that there are no absolute constraints on the arguments of predicates, but that the constraints are satisfied to a certain degree; thus the problem of knowledge acquisition becomes the problem of probability estimation from corpus data. The problem with defining a probability model in terms of senses is that this involves a huge number of parameters, which results in a sparse data problem. The proposal here is to define a probability model over senses in a semantic hierarchy, and exploit the fact that senses can be grouped into classes consisting of semantically similar senses. A novel class-based estimation technique is developed, together with a procedure that determines a suitable class for a sense (given a predicate and argument position). The problem of determining a suitable class can be thought of as finding a suitable level of generalisation in the hierarchy. The generalisation procedure uses a statistical test to locate areas consisting of semantically similar senses, and, as well as being used for probability estimation, is also employed as part of a re-estimation algorithm for estimating sense frequencies from incomplete data. The rest of the thesis considers how the lexical knowledge can be used to resolve structural ambiguities, and provides empirical evaluations. The estimation techniques are first integrated into a parse selection system, using a probabilistic dependency model to rank the alternative parses for a sentence. Then, a PP-attachment task is used to provide an evaluation which is more focussed on the class-based estimation technique, and, finally, a pseudo disambiguation task is used to compare the estimation technique with alternative approaches.
8
Belz, Anja. "Computational learning of finite-state models for natural language processing." Thesis, University of Sussex, 2000. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.311331.
Full text
9
Tang, Haijiang. "Building phrase based language model from large corpus /." View Abstract or Full-Text, 2002. http://library.ust.hk/cgi/db/thesis.pl?ELEC%202002%20TANG.
Full textAbstract:
Thesis (M. Phil.)--Hong Kong University of Science and Technology, 2002.Includes bibliographical references (leaves 74-79). Also available in electronic version. Access restricted to campus users.
10
Mitchell, Jeffrey John. "Composition in distributional models of semantics." Thesis, University of Edinburgh, 2011. http://hdl.handle.net/1842/4927.
Full textAbstract:
Distributional models of semantics have proven themselves invaluable both in cognitive modelling of semantic phenomena and also in practical applications. For example, they have been used to model judgments of semantic similarity (McDonald, 2000) and association (Denhire and Lemaire, 2004; Griffiths et al., 2007) and have been shown to achieve human level performance on synonymy tests (Landuaer and Dumais, 1997; Griffiths et al., 2007) such as those included in the Test of English as Foreign Language (TOEFL). This ability has been put to practical use in automatic thesaurus extraction (Grefenstette, 1994). However, while there has been a considerable amount of research directed at the most effective ways of constructing representations for individual words, the representation of larger constructions, e.g., phrases and sentences, has received relatively little attention. In this thesis we examine this issue of how to compose meanings within distributional models of semantics to form representations of multi-word structures. Natural language data typically consists of such complex structures, rather than just individual isolated words. Thus, a model of composition, in which individual word meanings are combined into phrases and phrases combine to form sentences, is of central importance in modelling this data. Commonly, however, distributional representations are combined in terms of addition (Landuaer and Dumais, 1997; Foltz et al., 1998), without any empirical evaluation of alternative choices. Constructing effective distributional representations of phrases and sentences requires that we have both a theoretical foundation to direct the development of models of composition and also a means of empirically evaluating those models. The approach we take is to first consider the general properties of semantic composition and from that basis define a comprehensive framework in which to consider the composition of distributional representations. The framework subsumes existing proposals, such as addition and tensor products, but also allows us to define novel composition functions. We then show that the effectiveness of these models can be evaluated on three empirical tasks. The first of these tasks involves modelling similarity judgements for short phrases gathered in human experiments. Distributional representations of individual words are commonly evaluated on tasks based on their ability to model semantic similarity relations, e.g., synonymy or priming. Thus, it seems appropriate to evaluate phrase representations in a similar manner. We then apply compositional models to language modelling, demonstrating that the issue of composition has practical consequences, and also providing an evaluation based on large amounts of natural data. In our third task, we use these language models in an analysis of reading times from an eye-movement study. This allows us to investigate the relationship between the composition of distributional representations and the processes involved in comprehending phrases and sentences. We find that these tasks do indeed allow us to evaluate and differentiate the proposed composition functions and that the results show a reasonable consistency across tasks. In particular, a simple multiplicative model is best for a semantic space based on word co-occurrence, whereas an additive model is better for the topic based model we consider. More generally, employing compositional models to construct representations of multi-word structures typically yields improvements in performance over non-compositonal models, which only represent individual words.
11
Batchkarov, Miroslav Manov. "Evaluating distributional models of compositional semantics." Thesis, University of Sussex, 2016. http://sro.sussex.ac.uk/id/eprint/61062/.
Full textAbstract:
Distributional models (DMs) are a family of unsupervised algorithms that represent the meaning of words as vectors. They have been shown to capture interesting aspects of semantics. Recent work has sought to compose word vectors in order to model phrases and sentences. The most commonly used measure of a compositional DM's performance to date has been the degree to which it agrees with human-provided phrase similarity scores. The contributions of this thesis are three-fold. First, I argue that existing intrinsic evaluations are unreliable as they make use of small and subjective gold-standard data sets and assume a notion of similarity that is independent of a particular application. Therefore, they do not necessarily measure how well a model performs in practice. I study four commonly used intrinsic datasets and demonstrate that all of them exhibit undesirable properties. Second, I propose a novel framework within which to compare word- or phrase-level DMs in terms of their ability to support document classification. My approach couples a classifier to a DM and provides a setting where classification performance is sensitive to the quality of the DM. Third, I present an empirical evaluation of several methods for building word representations and composing them within my framework. I find that the determining factor in building word representations is data quality rather than quantity; in some cases only a small amount of unlabelled data is required to reach peak performance. Neural algorithms for building single-word representations perform better than counting-based ones regardless of what composition is used, but simple composition algorithms can outperform more sophisticated competitors. Finally, I introduce a new algorithm for improving the quality of distributional thesauri using information from repeated runs of the same non deterministic algorithm.
12
Roberts, Philip J. "Towards a computer model of the historical phonology and morphology of Latin." Thesis, University of Oxford, 2012. http://ora.ox.ac.uk/objects/uuid:d3ef315c-3d5c-486b-8fbe-0fa6fdbb8219.
Full textAbstract:
Research projects in Optimality Theory tend to take a synchronic view of a particular generalisation, and set their standards for rigour in typological terms (see for example Suzuki 1998 on dissimilation, Crosswhite 2001 on vowel reduction). The goal of this thesis is to use Stratal OT to take a diachronic view of multiple generalisations within the morpho-phonology of one language, namely Latin, with the principal empirical aim of producing an analysis that is demonstrably true to all the attested facts of the generalisations in question. To that end, I have written PyOT, a computer program implementing the OT calculus and a theory of phonological representations, which I use in this work to model the histories of Lachmann’s Law, rhotacism and the phonologically conditioned allomorphy of the -alis/aris- suffix as active generalisations within the phonological component of the grammar. Appendix A gives the results of the computer model applied to a dataset consisting of 185 attested Latin forms, which suffice to illustrate the exact conditions of the generalisations in question. I show that producing a complete analysis of the three generalisations I have chosen to model entails analysis of other generalisations that interact with them, including the treatment of the Indo-European voiced aspirates in Latin (which interacts with rhotacism), and reduplication in forming perfect stems (which interacts with Lachmann’s Law). Constraint rankings sufficient to model these interactions, and consistent with the general conditions of the interacting generalisations have been included in the model. The intention is for this work to illustrate both the utility of formal phonological theory in advancing hypotheses within historical-comparative linguistics, and the potential of PyOT as a tool for producing Optimality-Theoretic models of (eventually) a language’s entire phonology.
13
Al-Raheb, Yafa. "Speaker/hearer representation in a discourse representation theory model of presupposition : a computational-linguistic approach." Thesis, University of East Anglia, 2005. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.426947.
Full text
14
Sandelius, Hugo. "Creating Knowledge Graphs using Distributional Semantic Models." Thesis, KTH, Skolan för datavetenskap och kommunikation (CSC), 2016. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-199702.
Full textAbstract:
This report researches a method for creating knowledge graphs, a specific way of structuring information, using distributional semantic models. Two different algorithms for selecting graph edges and two different algorithms for labelling edges are tried, and variations of those are evaluated. We perform experiments comparing our knowledge graphs with existing manually constructed knowledge graphs of high quality, with respect to graph structure and edge labels. We find that the algorithms usually produces graphs with a structure similar to that of manually constructed knowledge graphs, as long as the data set is sufficiently large and general, and that the similarity of edge labels to manually chosen edge labels vary widely depending on input.
15
Ritz, Julia. "Discourse-givenness of noun phrases : theoretical and computational models." Phd thesis, Universität Potsdam, 2013. http://opus.kobv.de/ubp/volltexte/2014/7081/.
Full textAbstract:
This thesis gives formal definitions of discourse-givenness, coreference and reference, and reports on experiments with computational models of discourse-givenness of noun phrases for English and German.
Definitions are based on Bach's (1987) work on reference, Kibble and van Deemter's (2000) work on coreference, and Kamp and Reyle's Discourse Representation Theory (1993).
For the experiments, the following corpora with coreference annotation were used: MUC-7, OntoNotes and ARRAU for Englisch, and TueBa-D/Z for German. As for classification algorithms, they cover J48 decision trees, the rule based learner Ripper, and linear support vector machines. New features are suggested, representing the noun phrase's specificity as well as its context, which lead to a significant improvement of classification quality.Die vorliegende Arbeit gibt formale Definitionen der Konzepte Diskursgegebenheit, Koreferenz und Referenz. Zudem wird über Experimente berichtet, Nominalphrasen im Deutschen und Englischen hinsichtlich ihrer Diskursgegebenheit zu klassifizieren. Die Definitionen basieren auf Arbeiten von Bach (1987) zu Referenz, Kibble und van Deemter (2000) zu Koreferenz und der Diskursrepräsentationstheorie (Kamp und Reyle, 1993). In den Experimenten wurden die koreferenzannotierten Korpora MUC-7, OntoNotes und ARRAU (Englisch) und TüBa-D/Z (Deutsch) verwendet. Sie umfassen die Klassifikationsalgorithmen J48 (Entscheidungsbäume), Ripper (regelbasiertes Lernen) und lineare Support Vector Machines. Mehrere neue Klassifikationsmerkmale werden vorgeschlagen, die die Spezifizität der Nominalphrase messen, sowie ihren Kontext abbilden. Mit Hilfe dieser Merkmale kann eine signifikante Verbesserung der Klassifikation erreicht werden.
16
Jonasson, Michael. "Fördomsfulla associationer i en svenskvektorbaserad semantisk modell." Thesis, Linköpings universitet, Institutionen för datavetenskap, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-159027.
Full textAbstract:
Semantiska vektormodeller är en kraftfull teknik där ords mening kan representeras av vektorervilka består av siffror. Vektorerna tillåter geometriska operationer vilka fångar semantiskt viktigaförhållanden mellan orden de representerar. I denna studie implementeras och appliceras WEAT-metoden för att undersöka om statistiska förhållanden mellan ord som kan uppfattas somfördomsfulla existerar i en svensk semantisk vektormodell av en svensk nyhetstidning. Resultatetpekar på att ordförhållanden i vektormodellen har förmågan att återspegla flera av de sedantidigare IAT-dokumenterade fördomar som undersöktes. I studien implementeras och applicerasockså WEFAT-metoden för att undersöka vektormodellens förmåga att representera två faktiskastatistiska samband i verkligheten, vilket görs framgångsrikt i båda undersökningarna. Resultatenav studien som helhet ger stöd till metoderna som används och belyser samtidigt problematik medatt använda semantiska vektormodeller i språkteknologiska applikationer.Word embeddings are a powerful technique where word meaning can be represented by vectors containing actual numbers. The vectors allow geometric operations that capture semantically important relationships between the words. In this study WEAT is applied in order to examine whether statistical properties of words pertaining to bias can be found in a swedish word embedding trained on a corpus from a swedish newspaper. The results shows that the word embedding can represent several of the IAT documented biases that where tested. A second method, WEFAT, is applied to the word embedding in order to explore the embeddings ability to represent actual statistical properties, which is also done successfully. The results from this study lends support to the validity of both methods aswell as illuminating the issue of problematic relationships between words in word embeddings.
17
Tengstrand, Lisa. "Abbreviation Expansion in Swedish Clinical Text : Using Distributional Semantic Models and Levenshtein Distance Normalization." Thesis, Uppsala universitet, Institutionen för lingvistik och filologi, 2014. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-226235.
Full textAbstract:
In the medical domain, especially in clinical texts, non-standard abbreviations are prevalent, which impairs readability for patients. To ease the understanding of the physicians' notes, abbreviations need to be identified and expanded into their original forms. This thesis presents a distributional semantic approach to find candidates of the original form of the abbreviation, which is combined with Levenshtein distance to choose the correct candidate among the semantically related words. The method is applied to radiology reports and medical journal texts, and a comparison is made to general Swedish. The results show that the correct expansion of the abbreviation can be found in 40% of the cases, an improvement by 24 percentage points compared to the baseline (0.16), and an increase by 22 percentage points compared to using word space models alone (0.18).
18
Ritz, Julia [Verfasser], and Stefan [Akademischer Betreuer] Evert. "Discourse-givenness of noun phrases : theoretical and computational models [[Elektronische Ressource]] / Julia Ritz. Betreuer: Stefan Evert." Potsdam : Universitätsbibliothek der Universität Potsdam, 2014. http://d-nb.info/1053125704/34.
Full text
19
Ritz, Julia Verfasser], and Stefan [Akademischer Betreuer] [Evert. "Discourse-givenness of noun phrases : theoretical and computational models [[Elektronische Ressource]] / Julia Ritz. Betreuer: Stefan Evert." Potsdam : Universitätsbibliothek der Universität Potsdam, 2014. http://d-nb.info/1053125704/34.
Full text
20
Konrad, Karsten. "Model generation for natural language interpretation and analysis /." Berlin [u.a.] : Springer, 2004. http://www.loc.gov/catdir/enhancements/fy0818/2004042936-d.html.
Full text
21
Prost, Jean-Philippe. "Modelling Syntactic Gradience with Loose Constraint-based Parsing." Phd thesis, Université de Provence - Aix-Marseille I, 2008. http://tel.archives-ouvertes.fr/tel-00352828.
Full textAbstract:
La grammaticalité d'une phrase est habituellement conçue comme une notion binaire : une phrase est soit grammaticale, soit agrammaticale. Cependant, bon nombre de travaux se penchent de plus en plus sur l'étude de degrés d'acceptabilité intermédiaires, auxquels le terme de gradience fait parfois référence. À ce jour, la majorité de ces travaux s'est concentrée sur l'étude de l'évaluation humaine de la gradience syntaxique. Cette étude explore la possibilité de construire un modèle robuste qui s'accorde avec ces jugements humains.Nous suggérons d'élargir au langage mal formé les concepts de Gradience Intersective et de Gradience Subsective, proposés par Aarts pour la modélisation de jugements graduels. Selon ce nouveau modèle, le problème que soulève la gradience concerne la classification d'un énoncé dans une catégorie particulière, selon des critères basés sur les caractéristiques syntaxiques de l'énoncé. Nous nous attachons à étendre la notion de Gradience Intersective (GI) afin qu'elle concerne le choix de la meilleure solution parmi un ensemble de candidats, et celle de Gradience Subsective (GS) pour qu'elle concerne le calcul du degré de typicité de cette structure au sein de sa catégorie. La GI est alors modélisée à l'aide d'un critère d'optimalité, tandis que la GS est modélisée par le calcul d'un degré d'acceptabilité grammaticale. Quant aux caractéristiques syntaxiques requises pour permettre de classer un énoncé, notre étude de différents cadres de représentation pour la syntaxe du langage naturel montre qu'elles peuvent aisément être représentées dans un cadre de syntaxe modèle-théorique (Model-Theoretic Syntax). Nous optons pour l'utilisation des Grammaires de Propriétés (GP), qui offrent, précisément, la possibilité de modéliser la caractérisation d'un énoncé. Nous présentons ici une solution entièrement automatisée pour la modélisation de la gradience syntaxique, qui procède de la caractérisation d'une phrase bien ou mal formée, de la génération d'un arbre syntaxique optimal, et du calcul d'un degré d'acceptabilité grammaticale pour l'énoncé.
À travers le développement de ce nouveau modèle, la contribution de ce travail comporte trois volets.
Premièrement, nous spécifions un système logique pour les GP qui permet la révision de sa formalisation sous l'angle de la théorie des modèles. Il s'attache notamment à formaliser les mécanismes de satisfaction et de relâche de contraintes mis en oeuvre dans les GP, ainsi que la façon dont ils permettent la projection d'une catégorie lors du processus d'analyse. Ce nouveau système introduit la notion de satisfaction relâchée, et une formulation en logique du premier ordre permettant de raisonner au sujet d'un énoncé.
Deuxièmement, nous présentons notre implantation du processus d'analyse syntaxique relâchée à base de contraintes (Loose Satisfaction Chart Parsing, ou LSCP), dont nous prouvons qu'elle génère toujours une analyse syntaxique complète et optimale. Cette approche est basée sur une technique de programmation dynamique (dynamic programming), ainsi que sur les mécanismes décrits ci-dessus. Bien que d'une complexité élevée, cette solution algorithmique présente des performances suffisantes pour nous permettre d'expérimenter notre modèle de gradience.
Et troisièmement, après avoir postulé que la prédiction de jugements humains d'acceptabilité peut se baser sur des facteurs dérivés de la LSCP, nous présentons un modèle numérique pour l'estimation du degré d'acceptabilité grammaticale d'un énoncé. Nous mesurons une bonne corrélation de ces scores avec des jugements humains d'acceptabilité grammaticale. Qui plus est, notre modèle s'avère obtenir de meilleures performances que celles obtenues par un modèle préexistant que nous utilisons comme référence, et qui, quant à lui, a été expérimenté à l'aide d'analyses syntaxiques générées manuellement.
22
Schillingmann, Lars [Verfasser]. "A computational model of acoustic packaging / Lars Schillingmann. Technische Fakultät. Research Institute for Cognition and Robotics." Bielefeld : Universitätsbibliothek Bielefeld, Hochschulschriften, 2012. http://d-nb.info/1028427573/34.
Full text
23
Wang, Zhen. "Human disease-behavior interactions on complex networks models: incorporating evolutionary game into epidemiology." HKBU Institutional Repository, 2014. https://repository.hkbu.edu.hk/etd_oa/22.
Full textAbstract:
In the past decade, the study of disease dynamics on complex networks has attracted great attention from both theoretical and empirical viewpoints. Under such a framework, people try to predict the outbreak of disease and propose immunization mechanisms. However, this framework possesses a limitation, which makes it inconsistent with realistic cases. That is, this framework does not consider the impact of human behavior or decision-making progress on disease dynamic characters and prevention measures. To further resolve this problem, we in this thesis propose behavioral epidemiology based on game theory, which involves the interactions between disease dynamics and human behavior in complex networks. Motivated by realistic cases, we proceed with the research from theoretical models and consider the following aspects. We .rst re-construct a scheme of risk perception incorporating local and global information and show that this new evaluation scenario not only promotes vaccination uptake, but also eliminates the disease spreading. This interesting .nding could be attributed to the positive feedback mechanism between vaccination uptake and disease spreading. Then, we introduce a self-protection measure, which, due to low cost, can only provide temporary protection. By simulations and analysis we show that this measure leads to multiple e.ects: contrary with cases of low (high) e.ciency and cost of the self-protection measure, middle values drive more infection and larger cost, which is related to the loss of positive feedback between prevention measures and disease propagation. Subsequently, another scheme of adaptive protection is proposed, where a healthy agent can cut the connection with infected ones. We .nd that adaptive protection can e.ectively eradicate the disease and result in an optimal level of pruning infected links. Di.erent from these proposals focusing on individual interest, we lastly study a subsidy policy from the viewpoint of population bene.t. We .nd that disease can be well controlled with an increase of the vaccination level, while the total expense reduces. Taken together, these .ndings of the thesis further demonstrate that the interplay between disease dynamics and human behavior plays an important role in the control of diseases. The models presented in this thesis, especially combining with empirical data, may serve as a foundation for further investigation of the subject in the future.
24
Yako, Mary. "Emotional Content in Novels for Literary Genre Prediction : And Impact of Feature Selection on Text Classification Models." Thesis, Uppsala universitet, Institutionen för lingvistik och filologi, 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-447148.
Full textAbstract:
Automatic literary genre classification presents a challenging task for Natural Language Processing (NLP) systems, mainly because literary texts have deeper levels of meanings, hold distinctive themes, and communicate certain messages and emotions. We conduct a study where we experiment with building literary genre classifiers based on emotions in novels, to investigate the effects that features pertinent to emotions have on models of genre prediction. We begin by performing an analysis of emotions describing emotional composition and density in the dataset. The experiments are carried out on a dataset consisting of novels categorized in eight different genres. Genre prediction models are built using three algorithms: Random Forest, Support Vector Machine, and k-Nearest Neighbor. We build models based on emotion-words counts and emotional words in a novel, and compare them to models of commonly used features, the bag-of-words and the TF-IDF features. Moreover, we use a feature selection dimensionality reduction procedure on the TF-IDF feature set and study its impact on classification performance. Finally, we train and test the classifiers on a combination of the two most optimal emotion-related feature sets, and compare them on classifiers trained and tested on a combination of bag-of-words and the reduced TF-IDF features. Our results confirm that: using features of emotional content in novels improves classification performance a 75% F1 compared to a bag-of-words baseline of 71% F1; TF-IDF feature filtering method positively impacts genre classification performance on literary texts.
25
Lin, Jing. "Using a rewriting system to model individual writing styles." Thesis, University of Aberdeen, 2012. http://digitool.abdn.ac.uk:80/webclient/DeliveryManager?pid=186641.
Full textAbstract:
Each individual has a distinguished writing style. But natural language generation systems pro- duce text with much less variety. Is it possible to produce more human-like text from natural language generation systems by mimicking the style of particular authors? We start by analysing the text of real authors. We collect a corpus of texts from a single genre (food recipes) with each text identified with its author, and summarise a variety of writing features in these texts. Each author's writing style is the combination of a set of features. Analysis of the writing features shows that not only does each individual author write differently but the differences are consistent over the whole of their corpus. Hence we conclude that authors do keep consistent style consisting of a variety of different features. When we discuss notions such as the style and meaning of texts, we are referring to the reac- tion that readers have to them. It is important, therefore, in the field of computational linguistics to experiment by showing texts to people and assessing their interpretation of the texts. In our research we move the thesis from simple discussion and statistical analysis of the properties of text and NLG systems, to perform experiments to verify the actual impact that lexical preference has on real readers. Through experiments that require participants to follow a recipe and prepare food, we conclude that it is possible to alter the lexicon of a recipe without altering the actions performed by the cook, hence that word choice is an aspect of style rather than semantics; and also that word choice is one of the writing features employed by readers in identifying the author of a text. Among all writing features, individual lexical preference is very important both for analysing and generating texts. So we choose individual lexical choice as our principal topic of research. Using a modified version of distributional similarity CDS) helps us to choose words used by in- dividual authors without the limitation of many other solutions such as a pre-built thesauri. We present an algorithm for analysis and rewriting, and assess the results. Based on the results we propose some further improvements.
26
Benitez-Quiroz, Carlos Fabian. "A Computational Study of American Sign Language Nonmanuals." The Ohio State University, 2015. http://rave.ohiolink.edu/etdc/view?acc_num=osu1436909704.
Full text
27
Luo, Ziyang. "Analyzing the Anisotropy Phenomenon in Transformer-based Masked Language Models." Thesis, Uppsala universitet, Institutionen för lingvistik och filologi, 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-445537.
Full textAbstract:
In this thesis, we examine the anisotropy phenomenon in popular masked language models, BERT and RoBERTa, in detail. We propose a possible explanation for this unreasonable phenomenon. First, we demonstrate that the contextualized word vectors derived from pretrained masked language model-based encoders share a common, perhaps undesirable pattern across layers. Namely, we find cases of persistent outlier neurons within BERT and RoBERTa's hidden state vectors that consistently bear the smallest or largest values in said vectors. In an attempt to investigate the source of this information, we introduce a neuron-level analysis method, which reveals that the outliers are closely related to information captured by positional embeddings. Second, we find that a simple normalization method, whitening can make the vector space isotropic. Lastly, we demonstrate that ''clipping'' the outliers or whitening can more accurately distinguish word senses, as well as lead to better sentence embeddings when mean pooling.
28
Faria, Pablo 1978. "Um modelo computacional de aquisição de primeira língua." [s.n.], 2013. http://repositorio.unicamp.br/jspui/handle/REPOSIP/268869.
Full textAbstract:
Orientadores: Ruth Elisabeth Vasconcellos Lopes, Charlotte Marie Chamberlland GalvesTese (doutorado) - Universidade Estadual de Campinas, Instituto de Estudos da Linguagem
Made available in DSpace on 2018-08-24T00:21:30Z (GMT). No. of bitstreams: 1 Faria_Pablo_D.pdf: 2488292 bytes, checksum: c75ee78601e9b73c790d12ce7e2d414e (MD5) Previous issue date: 2013
Resumo: Neste trabalho, o fenômeno de aquisição de uma língua natural é investigado através de uma modelagem computacional. O aprendiz modelado - apelidado de IASMIM - se caracteriza como um modelo computacional integrado de aquisição de primeira língua, visto que integra os processos de aquisição lexical e sintática. Além disso, o modelo foi concebido de modo a atender certos critérios de plausibilidade empírica e psicológica. A perspectiva teórica que norteia a investigação é a da Gramática Gerativa (cf. Chomsky, 1986) e este é um modelo voltado para a competência linguística, não um modelo de processamento ou de performance (i.e., de uso do conhecimento linguístico). O aprendiz modelado é capaz de adquirir um conhecimento gramatical relativamente abrangente e demonstra algum potencial translinguístico, particularmente no que diz respeito a variações de ordem. As simulações para avaliação do modelo permitem observar a emergência de padrões de adjunção e de recursividade na gramática, considerados aqui como as principais evidências de um conhecimento sintático mais elaborado. Finalmente, o modelo incorpora algumas noções caras à teoria sintática no âmbito do Programa Minimalista (cf. Chomsky, 1995b), tais como set- Merge, pair-Merge, "traço seletor" (cf. Chomsky, 1998), em conjunto com assunções sobre a binariedade das representações sintáticas e a hipótese de que a ordem linear não tem papel na sintaxe (cf. Uriagereka, 1999). O modelo incorpora, ainda, uma versão da representação semântico-conceitual proposta em Jackendoff (1990). Nesta modelagem, estas noções e assunções ganham uma interpretação concreta e integrada, interagindo na determinação das propriedades do conhecimento adquirido
Abstract: In the present work, the acquisition of natural languages is investigated through a computer simulation. The modelled learner - dubbed IASMIM - is characterized as an integrated computational model of first language acquisition, in the sense that it integrates the processes of lexical and syntactic acquisition. Furthermore, the model was conceived in order to be empirically and psychologically plausible. The theoretical perspective of this enterprise is that of Generative Grammar (cf. Chomsky, 1986) and this is a model concerned with linguistic competence, rather than language processing or performance (i.e., how the acquired knowledge is put to use). The modelled learner is capable of acquiring a relatively broad grammatical knowledge and shows some crosslinguistic abilities, in particular, the ability to handle languages with distinct word orders. In the simulations for evaluation of the model we can observe the emergence of adjunction and recursive patterns in the grammar, taken here as the main pieces of evidence of a more elaborated syntactic knowledge. Finally, the model embodies some central notions for syntactic theory under the Minimalist Program (cf. Chomsky, 1995b), such as set-Merge, pair-Merge and "selector feature" (cf. Chomsky, 1998), together with the assumptions that syntactic representations are strictly binary branching and that linear word order has no significant role in syntactic phenomena (cf. Uriagereka, 1999). The model also embodies a version of the semantic-conceptual representation proposed in Jackendoff (1990). They take a concrete and integrated existence in this model, interacting with one another to determine the properties of the acquired grammatical knowledge
Doutorado
Linguistica
Doutor em Linguística
29
Xie, Danke. "A computational biologically-plausible model of working memory for serial order, repetition and binding." Diss., [La Jolla, Calif.] : University of California, San Diego, 2009. http://wwwlib.umi.com/cr/ucsd/fullcit?p3344748.
Full textAbstract:
Thesis (Ph. D.)--University of California, San Diego, 2009.Title from first page of PDF file (viewed April 1, 2009). Available via ProQuest Digital Dissertations. Vita. Includes bibliographical references (p. 150-163).
30
Stevan, Ostrogonac. "Modeli srpskog jezika i njihova primena u govornim i jezičkim tehnologijama." Phd thesis, Univerzitet u Novom Sadu, Fakultet tehničkih nauka u Novom Sadu, 2018. https://www.cris.uns.ac.rs/record.jsf?recordId=107812&source=NDLTD&language=en.
Full textAbstract:
Statistički jezički model, u teoriji, predstavlja raspodelu verovatnoća nad skupom svih mogućih sekvenci reči nekog jezika. U praksi, to je mehanizam kojim se estimiraju verovatnoće sekvenci, koje su od interesa. Matematički aparat vezan za modele jezika je uglavnom nezavisan od jezika. Međutim, kvalitet obučenih modela ne zavisi samo od algoritama obuke, već prvenstveno od količine i kvaliteta podataka koji su na raspolaganju za obuku. Za jezike sa kompleksnom morfologijom, kao što je srpski, tekstualni korpus za obuku modela mora biti daleko obimniji od korpusa koji bi se koristio kod nekog od jezika sa relativno jednostavnom morfologijom, poput engleskog. Ovo istraživanje obuhvata razvoj jezičkih modela za srpski jezik, počevši od prikupljanja i inicijalne obrade tekstualnih sadržaja, preko adaptacije algoritama i razvoja metoda za rešavanje problema nedovoljne količine podataka za obuku, pa do prilagođavanja i primene modela u različitim tehnologijama, kao što su sinteza govora na osnovu teksta, automatsko prepoznavanje govora, automatska detekcija i korekcija gramatičkih i semantičkih grešaka u tekstovima, a postavljaju se i osnove za primenu jezičkih modela u automatskoj klasifikaciji dokumenata i drugim tehnologijama. Jezgro razvoja jezičkih modela za srpski predstavlja definisanje morfoloških klasa reči na osnovu informacija koje su sadržane u morfološkom rečniku, koji je nastao kao rezultat jednog od ranijih istraživanja.A statistical language model, in theory, represents a probability distribution over sequences of words of a language. In practice, it is a tool for estimating probabilities of word sequences of interest. Mathematical basis related to language models is mostly language independent. However, the quality of trained models depends not only on training algorithms, but on the amount and quality of available training data as well. For languages with complex morphology, such as Serbian, textual corpora for training language models need to be significantly larger than the corpora needed for training language models for languages with relatively simple morphology, such as English. This research represents the entire process of developing language models for Serbian, starting with collecting and preprocessing of textual contents, extending to adaptation of algorithms and development of methods for addressing the problem of insufficient training data, and finally to adaptation and application of the models in different technologies, such as text-to-speech synthesis, automatic speech recognition, automatic detection and correction of grammar and semantic errors in texts, and determining basics for the application of the models in automatic document classification and other tasks. The core of the development of language models for Serbian is defining morphologic classes of words, based on the information contained within the morphologic dictionary of Serbian, which was one of the results of a previous research.
31
Faria, Pablo 1978. "Propriedades das línguas naturais e o processo de aquisição = reflexões a partir da implementação do modelo em Berwick (1985)." [s.n.], 2009. http://repositorio.unicamp.br/jspui/handle/REPOSIP/271190.
Full textAbstract:
Orientador: Ruth Elisabeth Vasconcellos LopesDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de Estudos da Linguagem
Made available in DSpace on 2018-08-16T06:36:14Z (GMT). No. of bitstreams: 1 Faria_Pablo_M.pdf: 1248984 bytes, checksum: 9609dfcb9ddec33250dfdefdb367fb5a (MD5) Previous issue date: 2009
Resumo: Nesta dissertação de mestrado, o objetivo principal é refletir sobre algumas propriedades da linguagem e do processo de aquisição, tomando como ponto de partida questões que surgiram durante o processo de implementação do modelo proposto em Berwick (1985). O quadro teórico geral em que esta pesquisa se situa é o da Gramática Gerativa - na linha chomskiana - e, em particular, o modelo implementado aqui tem como principal base teórica a Gramática Transformacional (Cf. CHOMSKY, 1965). Entre as propriedades da linguagem que discutimos estão: os traços distintivos dos itens lexicais, a assimetria entre especificadores e complementos, categorias vazias e o papel da informação temática na sintaxe. A idéia subjacente que permeia as reflexões é a busca por um olhar mais abstrato sobre o conhecimento gramatical, procurando rever ou até eliminar dispositivos que, em primeiro lugar, aparecem como obstáculos significativos para o analisador e, em segundo lugar, resistem à identificação de evidências para sua aquisição, do ponto de vista do aprendiz da língua. Para atingir estes objetivos, a primeira metade do trabalho faz uma breve discussão teórica, para em seguida trazer uma apresentação razoavelmente detalhada do modelo de Berwick, incluindo exemplos de funcionamento. A segunda metade inclui a discussão dos resultados juntamente com as reflexões sobre a linguagem, procurando apontar caminhos que não apenas possam tornar o modelo de aquisição mais robusto, mas que levantem questões para investigação em teoria gramatical. Neste sentido, este trabalho - por se situar numa área multidisciplinar, a saber, a lingüística computacional - procura tornar explícitas as contribuições que esse tipo de investigação pode fazer à teoria lingüística.
Abstract: The main goal of this MA thesis is to discuss some properties of language and its acquisition process, taking as a starting point some issues that emerged during the implementation of the acquisition model proposed by Berwick (1985). Our general theoretical framework is the Generative Grammar - as proposed in Chomsky's works - and, in particular, the Transformational Grammar model (Cf. CHOMSKY, 1965). Some of the language properties discussed here involve: the set of distinctive features for lexical items, the asymmetry between specifiers and complements, empty categories and the role of thematic information in syntax. The subjacent idea surrounding the reections on language is the search for a more abstract view of the grammatical knowledge such that some of the theoretical devices can be revised or even abandoned. Two main goals drive this effort: first, the elimination of some significant obstacles in the parser's task to analyze sentences. Second, the elimination of those devices for which we cannot find proper evidence for their acquisition, considering the learner's perspective. In order to pursue these goals, in the first part of this work we set out the theoretical background for the whole discussion and give a somewhat detailed presentation of Berwick's acquisition model and examples of its functioning. In the second part we include a discussion of the results upon which the reection is built, whereby we point out some ways not only to increase the robustness of the acquisition model, but also to raise questions for further research in grammatical theory. In this sense, this work - which pertains to an interdisciplinary field, i.e, computational linguistics - tries to make explicit the contributions that this kind of investigation can offer to linguistic theory.
Mestrado
Linguistica
Mestre em Linguística
32
Roos, Daniel. "Evaluation of BERT-like models for small scale ad-hoc information retrieval." Thesis, Linköpings universitet, Artificiell intelligens och integrerade datorsystem, 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-177675.
Full textAbstract:
Measuring semantic similarity between two sentences is an ongoing research field with big leaps being taken every year. This thesis looks at using modern methods of semantic similarity measurement for an ad-hoc information retrieval (IR) system. The main challenge tackled was answering the question "What happens when you don’t have situation-specific data?". Using encoder-based transformer architectures pioneered by Devlin et al., which excel at fine-tuning to situationally specific domains, this thesis shows just how well the presented methodology can work and makes recommendations for future attempts at similar domain-specific tasks. It also shows an example of how a web application can be created to make use of these fast-learning architectures.
33
Tang, Hao. "Bidirectional LSTM-CNNs-CRF Models for POS Tagging." Thesis, Uppsala universitet, Institutionen för lingvistik och filologi, 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-362823.
Full textAbstract:
In order to achieve state-of-the-art performance for part-of-speech(POS) tagging, the traditional systems require a significant amount of hand-crafted features and data pre-processing. In this thesis, we present a discriminative word embedding, character embedding and byte pair encoding (BPE) hybrid neural network architecture to implement a true end-to-end system without feature engineering and data pre-processing. The neural network architecture is a combination of bidirectional LSTM, CNNs, and CRF, which can achieve a state-of-the-art performance for a wide range of sequence labeling tasks. We evaluate our model on Universal Dependencies (UD) dataset for English, Spanish, and German POS tagging. It outperforms other models with 95.1%, 98.15%, and 93.43% accuracy on testing datasets respectively. Moreover, the largest improvements of our model appear on out-of-vocabulary corpora for Spanish and German. According to statistical significance testing, the improvements of English on testing and out-of-vocabulary corpora are not statistically significant. However, the improvements of the other more morphological languages are statistically significant on their corresponding corpora.
34
Kunz, Jenny. "Neural Language Models with Explicit Coreference Decision." Thesis, Uppsala universitet, Institutionen för lingvistik och filologi, 2019. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-371827.
Full textAbstract:
Coreference is an important and frequent concept in any form of discourse, and Coreference Resolution (CR) a widely used task in Natural Language Understanding (NLU). In this thesis, we implement and explore two recent models that include the concept of coreference in Recurrent Neural Network (RNN)-based Language Models (LM). Entity and reference decisions are modeled explicitly in these models using attention mechanisms. Both models learn to save the previously observed entities in a set and to decide if the next token created by the LM is a mention of one of the entities in the set, an entity that has not been observed yet, or not an entity. After a theoretical analysis where we compare the two LMs to each other and to a state of the art Coreference Resolution system, we perform an extensive quantitative and qualitative analysis. For this purpose, we train the two models and a classical RNN-LM as the baseline model on the OntoNotes 5.0 corpus with coreference annotation. While we do not reach the baseline in the perplexity metric, we show that the models’ relative performance on entity tokens has the potential to improve when including the explicit entity modeling. We show that the most challenging point in the systems is the decision if the next token is an entity token, while the decision which entity the next token refers to performs comparatively well. Our analysis in the context of a text generation task shows that a wide-spread error source for the mention creation process is the confusion of tokens that refer to related but different entities in the real world, presumably a result of the context-based word representations in the models. Our re-implementation of the DeepMind model by Yang et al. 2016 performs notably better than the re-implementation of the EntityNLM model by Ji et al. 2017 with a perplexity of 107 compared to a perplexity of 131.
35
Lu, Xiaofei. "Hybrid models for Chinese unknown word resolution." Columbus, Ohio : Ohio State University, 2006. http://rave.ohiolink.edu/etdc/view?acc%5Fnum=osu1154631880.
Full text
36
Tao, Joakim, and David Thimrén. "Smoothening of Software documentation : comparing a self-made sequence to sequence model to a pre-trained model GPT-2." Thesis, Linköpings universitet, Institutionen för datavetenskap, 2021. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-178186.
Full textAbstract:
This thesis was done in collaboration with Ericsson AB with the goal of researching the possibility of creating a machine learning model that can transfer the style of a text into another arbitrary style depending on the data used. This had the purpose of making their technical documentation appear to have been written with one cohesive style for a better reading experience. Two approaches to solve this task were tested, the first one was to implement an encoder-decoder model from scratch, and the second was to use the pre-trained GPT-2 model created by a team from OpenAI and fine-tune the model on the specific task. Both of these models were trained on data provided by Ericsson, sentences were extracted from their documentation. To evaluate the models training loss, test sentences, and BLEU scores were used and these were compared to each other and with other state-of-the-art models. The models did not succeed in transforming text into a general technical documentation style but a good understanding of what would need to be improved and adjusted to improve the results were obtained.This thesis was presented on June 22, 2021, the presentation was done online on Microsoft teams.
37
Asghari, Parastoo. "Ambiguity in Peace Agreements : Cognitive and Computational Models for Processing Syntactic Ambiguity in Israeli-Palestinian Peace Agreements in English." Thesis, Stockholms universitet, Engelska institutionen, 2018. http://urn.kb.se/resolve?urn=urn:nbn:se:su:diva-152824.
Full textAbstract:
Systems that attempt to process texts and acquire information from texts in English need to be particularly alert to noun phrases since they carry so much information. Systems, whether comprehensional or computational, may face particular difficulties when dealing with complex noun phrases. One of the decomposition patterns for noun phrases is left or right branching, which determines the semantic relations between the constituents of the combination.This degree project seeks to describe a processing model that the comprehension system employs to process difficulties. Since the minicorpus studied in this research consists of four of the peace agreements that were produced in English for Israeli and Palestinian sides of their conflicts to sign and implement, the comprehension models that were used by a non-native speaker of English are described, and then a computational model to enhance performing this task is suggested which includes using the frequencies of the combinations of the constituents in two major contemporary corpora, the Corpus of Contemporary American English and the British National Corpus, to help decide how to nest the noun phrases as either left or right branching structures, to resolve the ambiguity problem. Hyphening is also suggested as a potential strategy to avoid unwanted structural ambiguity in adjective + noun + noun combinations.
38
Milajevs, Dmitrijs. "A study of model parameters for scaling up word to sentence similarity tasks in distributional semantics." Thesis, Queen Mary, University of London, 2018. http://qmro.qmul.ac.uk/xmlui/handle/123456789/36225.
Full textAbstract:
Representation of sentences that captures semantics is an essential part of natural language processing systems, such as information retrieval or machine translation. The representation of a sentence is commonly built by combining the representations of the words that the sentence consists of. Similarity between words is widely used as a proxy to evaluate semantic representations. Word similarity models are well-studied and are shown to positively correlate with human similarity judgements. Current evaluation of models of sentential similarity builds on the results obtained in lexical experiments. The main focus is how the lexical representations are used, rather than what they should be. It is often assumed that the optimal representations for word similarity are also optimal for sentence similarity. This work discards this assumption and systematically looks for lexical representations that are optimal for similarity measurement between sentences. We find that the best representation for word similarity is not always the best for sentence similarity and vice versa. The best models in word similarity tasks perform best with additive composition. However, the best result on compositional tasks is achieved with Kroneckerbased composition. There are representations that are equally good in both tasks when used with multiplicative composition. The systematic study of the parameters of similarity models reveals that the more information lexical representations contain, the more attention should be paid to noise. In particular, the word vectors in models with the feature size at the magnitude of the vocabulary size should be sparse, but if a small number of context features is used then the vectors should be dense. Given the right lexical representations, compositional operators achieve state-of-the-art performance, improving over models that use neural-word embeddings. To avoid overfitting, either several test datasets should be used or parameter selection should be based on parameters' average behaviours.
39
Kim, Seungyeon. "Modeling and visualization of version-controlled documents." Thesis, Georgia Institute of Technology, 2011. http://hdl.handle.net/1853/39603.
Full textAbstract:
Version-controlled documents, such as Wikipedia or program codes in Subversion, demands a novel methodology to be analyzed efficiently. The documents are continually edited by one or more authors in contrast of the case of static documents. These collaborative processses make traditional methodologies to be ineffective, yet needs for efficient methodologies are rapidly developing. This paper proposes two new models based on Local Space-time Smoothing (LSS) which captures important revision patterns while Cumulative Revision Map (CRM) tracks word insertions and deletions in particular positions of a document. These two methods enable us to understand and visualize the revision patterns intuitively and efficiently. Synthetic data and real-world data are used to demonstrate its applicability.
40
Grefenstette, Edward Thomas. "Category-theoretic quantitative compositional distributional models of natural language semantics." Thesis, University of Oxford, 2013. http://ora.ox.ac.uk/objects/uuid:d7f9433b-24c0-4fb5-925b-d8b3744b7012.
Full textAbstract:
This thesis is about the problem of compositionality in distributional semantics. Distributional semantics presupposes that the meanings of words are a function of their occurrences in textual contexts. It models words as distributions over these contexts and represents them as vectors in high dimensional spaces. The problem of compositionality for such models concerns itself with how to produce distributional representations for larger units of text (such as a verb and its arguments) by composing the distributional representations of smaller units of text (such as individual words). This thesis focuses on a particular approach to this compositionality problem, namely using the categorical framework developed by Coecke, Sadrzadeh, and Clark, which combines syntactic analysis formalisms with distributional semantic representations of meaning to produce syntactically motivated composition operations. This thesis shows how this approach can be theoretically extended and practically implemented to produce concrete compositional distributional models of natural language semantics. It furthermore demonstrates that such models can perform on par with, or better than, other competing approaches in the field of natural language processing. There are three principal contributions to computational linguistics in this thesis. The first is to extend the DisCoCat framework on the syntactic front and semantic front, incorporating a number of syntactic analysis formalisms and providing learning procedures allowing for the generation of concrete compositional distributional models. The second contribution is to evaluate the models developed from the procedures presented here, showing that they outperform other compositional distributional models present in the literature. The third contribution is to show how using category theory to solve linguistic problems forms a sound basis for research, illustrated by examples of work on this topic, that also suggest directions for future research.
41
Sundin, Albin. "Word Space Models for Web User Clustering and Page Prefetching." Thesis, Linköpings universitet, Institutionen för datavetenskap, 2012. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-82012.
Full textAbstract:
This study evaluates methods for clustering web users via vector space models, for the purpose of web page prefetching for possible applications of server optimization. An experiment using Latent Semantic Analysis (LSA) is deployed to investigate whether LSA can reproduce the encouraging results obtained from previous research with Random Indexing (RI) and a chaos based optimization algorithm (CAS-C). This is not only motivated by LSA being yet another vector space model, but also by a study indicating LSA to outperform RI in a task similar to the web user clustering and prefetching task. The prefetching task was used to verify the applicability of LSA, where both RI and CAS-C have shown promising results. The original data set from the RI web user clustering and prefetching task was modeled using weighted (tf-idf) LSA. Clusters were defined using a common clustering algorithm (k-means). The least scattered cluster configuration for the model was identified by combining an internal validity measure (SSE) and a relative criterion validity measure (SD index). The assumed optimal cluster configuration was used for the web page prefetching task. Precision and recall of the LSA based method is found to be on par with RI and CAS-C, in as much that it solves the web user clustering and web task with similar characteristics as unweighted RI. The hypothesized inherent gains to precision and recall by using LSA was neither confirmed nor conclusively disproved. The effects of different weighting functions for RI are discussed and a number of methodological factors are identified for further research concerning LSA based clustering and prefetching.
42
Fang, Yimai. "Proposition-based summarization with a coherence-driven incremental model." Thesis, University of Cambridge, 2019. https://www.repository.cam.ac.uk/handle/1810/287468.
Full textAbstract:
Summarization models which operate on meaning representations of documents have been neglected in the past, although they are a very promising and interesting class of methods for summarization and text understanding. In this thesis, I present one such summarizer, which uses the proposition as its meaning representation. My summarizer is an implementation of Kintsch and van Dijk's model of comprehension, which uses a tree of propositions to represent the working memory. The input document is processed incrementally in iterations. In each iteration, new propositions are connected to the tree under the principle of local coherence, and then a forgetting mechanism is applied so that only a few important propositions are retained in the tree for the next iteration. A summary can be generated using the propositions which are frequently retained. Originally, this model was only played through by hand by its inventors using human-created propositions. In this work, I turned it into a fully automatic model using current NLP technologies. First, I create propositions by obtaining and then transforming a syntactic parse. Second, I have devised algorithms to numerically evaluate alternative ways of adding a new proposition, as well as to predict necessary changes in the tree. Third, I compared different methods of modelling local coherence, including coreference resolution, distributional similarity, and lexical chains. In the first group of experiments, my summarizer realizes summary propositions by sentence extraction. These experiments show that my summarizer outperforms several state-of-the-art summarizers. The second group of experiments concerns abstractive generation from propositions, which is a collaborative project. I have investigated the option of compressing extracted sentences, but generation from propositions has been shown to provide better information packaging.
43
Buys, Jan Moolman. "Incremental generative models for syntactic and semantic natural language processing." Thesis, University of Oxford, 2017. https://ora.ox.ac.uk/objects/uuid:a9a7b5cf-3bb1-4e08-b109-de06bf387d1d.
Full textAbstract:
This thesis investigates the role of linguistically-motivated generative models of syntax and semantic structure in natural language processing (NLP). Syntactic well-formedness is crucial in language generation, but most statistical models do not account for the hierarchical structure of sentences. Many applications exhibiting natural language understanding rely on structured semantic representations to enable querying, inference and reasoning. Yet most semantic parsers produce domain-specific or inadequately expressive representations. We propose a series of generative transition-based models for dependency syntax which can be applied as both parsers and language models while being amenable to supervised or unsupervised learning. Two models are based on Markov assumptions commonly made in NLP: The first is a Bayesian model with hierarchical smoothing, the second is parameterised by feed-forward neural networks. The Bayesian model enables careful analysis of the structure of the conditioning contexts required for generative parsers, but the neural network is more accurate. As a language model the syntactic neural model outperforms both the Bayesian model and n-gram neural networks, pointing to the complementary nature of distributed and structured representations for syntactic prediction. We propose approximate inference methods based on particle filtering. The third model is parameterised by recurrent neural networks (RNNs), dropping the Markov assumptions. Exact inference with dynamic programming is made tractable here by simplifying the structure of the conditioning contexts. We then shift the focus to semantics and propose models for parsing sentences to labelled semantic graphs. We introduce a transition-based parser which incrementally predicts graph nodes (predicates) and edges (arguments). This approach is contrasted against predicting top-down graph traversals. RNNs and pointer networks are key components in approaching graph parsing as an incremental prediction problem. The RNN architecture is augmented to condition the model explicitly on the transition system configuration. We develop a robust parser for Minimal Recursion Semantics, a linguistically-expressive framework for compositional semantics which has previously been parsed only with grammar-based approaches. Our parser is much faster than the grammar-based model, while the same approach improves the accuracy of neural Abstract Meaning Representation parsing.
44
Erenmalm, Elsa. "Multilingual Dependency Parsing of Uralic Languages : Parsing with zero-shot transfer and cross-lingual models using geographically proximate, genealogically related, and syntactically similar transfer languages." Thesis, Uppsala universitet, Institutionen för lingvistik och filologi, 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-427278.
Full textAbstract:
One way to improve dependency parsing scores for low-resource languages is to make use of existing resources from other closely related or otherwise similar languages. In this paper, we look at eleven Uralic target languages (Estonian, Finnish, Hungarian, Karelian, Livvi, Komi Zyrian, Komi Permyak, Moksha, Erzya, North Sámi, and Skolt Sámi) with treebanks of varying sizes and select transfer languages based on geographical, genealogical, and syntactic distances. We focus primarily on the performance of parser models trained on various combinations of geographically proximate and genealogically related transfer languages, in target-trained, zero-shot, and cross-lingual configurations. We find that models trained on combinations of geographically proximate and genealogically related transfer languages reach the highest LAS in most zero-shot models, while our highest-performing cross-lingual models were trained on genealogically related languages. We also find that cross-lingual models outperform zero-shot transfer models. We then select syntactically similar transfer languages for three target languages, and find a slight improvement in the case of Hungarian. We discuss the results and conclude with suggestions for possible future work.
45
Andersson, Henrik. "Anchor-based Topic Modeling with Human Interpretable Results." Thesis, Linköpings universitet, Interaktiva och kognitiva system, 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-168134.
Full textAbstract:
Topic models are useful tools for exploring large data sets of textual content by exposing a generative process from which the text was produced. Anchor-based topic models utilize the anchor word assumption to define a set of algorithms with provable guarantees which recover the underlying topics with a run time practically independent of corpus size. A number of extensions to the initial anchor word-based algorithms, and enhancements made to tangential models, have been proposed which improve the intrinsic characteristics of the model making them more interpretable by humans. This thesis evaluates improvements to human interpretability due to: low-dimensional word embeddings in combination with a regularized objective function, automatic topic merging using tandem anchors, and utilizing word embeddings to synthetically increase corpus density. Results show that tandem anchors are viable vehicles for automatic topic merging, and that using word embeddings significantly improves the original anchor method across all measured metrics. Combining low-dimensional embeddings and a regularized objective results in computational downsides with small or no improvements to the metrics measured.
46
Hägglöf, Hillevi, and Lisa Tengstrand. "A Random Indexing Approach to Unsupervised Selectional Preference Induction." Thesis, Stockholms universitet, Avdelningen för datorlingvistik, 2011. http://urn.kb.se/resolve?urn=urn:nbn:se:su:diva-59493.
Full textAbstract:
A selectional preference is the relation between a head-word and plausible arguments of that head-word. Estimation of the association feature between these words is important to natural language processing applications such as Word Sense Disambiguation. This study presents a novel approach to selectional preference induction within a Random Indexing word space. This is a spatial representation of meaning where distributional patterns enable estimation of the similarity between words. Using only frequency statistics about words to estimate how strongly one word selects another, the aim of this study is to develop a flexible method that is not language dependent and does not require any annotated resourceswhich is in contrast to methods from previous research. In order to optimize the performance of the selectional preference model, experiments including parameter tuning and variation of corpus size were conducted. The selectional preference model was evaluated in a pseudo-word evaluation which lets the selectional preference model decide which of two arguments have a stronger correlation to a given verb. Results show that varying parameters and corpus size does not affect the performance of the selectional preference model in a notable way. The conclusion of the study is that the language modelused does not provide the adequate tools to model selectional preferences. This might be due to a noisy representation of head-words and their arguments.
47
Svensson, Karin, and Johan Blad. "Exploring NMF and LDA Topic Models of Swedish News Articles." Thesis, Uppsala universitet, Avdelningen för systemteknik, 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-429250.
Full textAbstract:
The ability to automatically analyze and segment news articles by their content is a growing research field. This thesis explores the unsupervised machine learning method topic modeling applied on Swedish news articles for generating topics to describe and segment articles. Specifically, the algorithms non-negative matrix factorization (NMF) and the latent Dirichlet allocation (LDA) are implemented and evaluated. Their usefulness in the news media industry is assessed by its ability to serve as a uniform categorization framework for news articles. This thesis fills a research gap by studying the application of topic modeling on Swedish news articles and contributes by showing that this can yield meaningful results. It is shown that Swedish text data requires extensive data preparation for successful topic models and that nouns exclusively and especially common nouns are the most suitable words to use. Furthermore, the results show that both NMF and LDA are valuable as content analysis tools and categorization frameworks, but they have different characteristics, hence optimal for different use cases. Lastly, the conclusion is that topic models have issues since they can generate unreliable topics that could be misleading for news consumers, but that they nonetheless can be powerful methods for analyzing and segmenting articles efficiently on a grand scale by organizations internally. The thesis project is a collaboration with one of Sweden’s largest media groups and its results have led to a topic modeling implementation for large-scale content analysis to gain insight into readers’ interests.
48
陸穎剛 and Wing-kong Luk. "Concept space approach for cross-lingual information retrieval." Thesis, The University of Hong Kong (Pokfulam, Hong Kong), 2000. http://hub.hku.hk/bib/B30147724.
Full text
49
Johansson, Oskar. "Parafrasidentifiering med maskinklassificerad data : utvärdering av olika metoder." Thesis, Linköpings universitet, Institutionen för datavetenskap, 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-167039.
Full textAbstract:
Detta arbete undersöker hur språkmodellen BERT och en MaLSTM-arkitektur fungerar att för att identifiera parafraser ur 'Microsoft Paraphrase Research Corpus' (MPRC) om dessa tränats på automatiskt identifierade parafraser ur 'Paraphrase Database' (PPDB). Metoderna ställs mot varandra för att undersöka vilken som presterar bäst och metoden att träna på maskinklassificerad data för att användas på mänskligt klassificerad data utvärderas i förhållande till annan klassificering av samma dataset. Meningsparen som används för att träna modellerna hämtas från de högst rankade parafraserna ur PPDB och genom en genereringsmetod som skapar icke-parafraser ur samma dataset. I resultatet visar sig BERT vara kapabel till att identifiera en del parafraser ur MPRC, medan MaLSTM-arkitekturen inte klarade av detta trots förmåga att särskilja på parafraser och icke-parafraser under träning. Både BERT och MaLSTM presterade sämre på att identifiera parafraser ur MPRC än modeller som till exempel StructBERT, som tränat och utvärderats på samma dataset, presterar. Anledningar till att MaLSTM inte klarar av uppgiften diskuteras och främst lyfts att meningarna från icke-parafraserna ur träningsdatan är för olika varandra i förhållande till hur de ser ut i MPRC. Slutligen diskuteras vikten av att forska vidare på hur man kan använda sig av maskinframtagna parafraser inom parafraseringsrelaterad forskning.
50
Zahra, Shorouq. "Targeted Topic Modeling for Levantine Arabic." Thesis, Uppsala universitet, Institutionen för lingvistik och filologi, 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:uu:diva-412975.
Full textAbstract:
Topic models for focused analysis aim to capture topics within the limiting scope of a targeted aspect (which could be thought of as some inner topic within a certain domain). To serve their analytic purposes, topics are expected to be semantically-coherent and closely aligned with human intuition – this in itself poses a major challenge for the more common topic modeling algorithms which, in a broader sense, perform a full analysis that covers all aspects and themes within a collection of texts. The paper attempts to construct a viable focused-analysis topic model which learns topics from Twitter data written in a closely related group of non-standardized varieties of Arabic widely spoken in the Levant region (i.e Levantine Arabic). Results are compared to a baseline model as well as another targeted topic model designed precisely to serve the purpose of focused analysis. The model is capable of adequately capturing topics containing terms which fall within the scope of the targeted aspect when judged overall. Nevertheless, it fails to produce human-friendly and semantically-coherent topics as several topics contained a number of intruding terms while others contained terms, while still relevant to the targeted aspect, thrown together seemingly at random.