Relevant bibliographies by topics / Task-specific representation learnining

Academic literature on the topic 'Task-specific representation learnining'

Author: Grafiati
Published: 6 September 2023

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles

Select a source type:

Contents

Consult the lists of relevant articles, books, theses, conference reports, and other scholarly sources on the topic 'Task-specific representation learnining.'

Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.

You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.

Journal articles on the topic "Task-specific representation learnining"

1

Wang, Chaoqun, Xuejin Chen, Shaobo Min, Xiaoyan Sun, and Houqiang Li. "Task-Independent Knowledge Makes for Transferable Representations for Generalized Zero-Shot Learning." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 3 (May 18, 2021): 2710–18. http://dx.doi.org/10.1609/aaai.v35i3.16375.

Full text
Abstract:
Generalized Zero-Shot Learning (GZSL) targets recognizing new categories by learning transferable image representations. Existing methods find that, by aligning image representations with corresponding semantic labels, the semantic-aligned representations can be transferred to unseen categories. However, supervised by only seen category labels, the learned semantic knowledge is highly task-specific, which makes image representations biased towards seen categories. In this paper, we propose a novel Dual-Contrastive Embedding Network (DCEN) that simultaneously learns task-specific and task-independent knowledge via semantic alignment and instance discrimination. First, DCEN leverages task labels to cluster representations of the same semantic category by cross-modal contrastive learning and exploring semantic-visual complementarity. Besides task-specific knowledge, DCEN then introduces task-independent knowledge by attracting representations of different views of the same image and repelling representations of different images. Compared to high-level seen category supervision, this instance discrimination supervision encourages DCEN to capture low-level visual knowledge, which is less biased toward seen categories and alleviates the representation bias. Consequently, the task-specific and task-independent knowledge jointly make for transferable representations of DCEN, which obtains averaged 4.1% improvement on four public benchmarks.
APA, Harvard, Vancouver, ISO, and other styles
2

Li, Yingcong, and Samet Oymak. "Provable Pathways: Learning Multiple Tasks over Multiple Paths." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 7 (June 26, 2023): 8701–10. http://dx.doi.org/10.1609/aaai.v37i7.26047.

Full text
Abstract:
Constructing useful representations across a large number of tasks is a key requirement for sample-efficient intelligent systems. A traditional idea in multitask learning (MTL) is building a shared representation across tasks which can then be adapted to new tasks by tuning last layers. A desirable refinement of using a shared one-fits-all representation is to construct task-specific representations. To this end, recent PathNet/muNet architectures represent individual tasks as pathways within a larger supernet. The subnetworks induced by pathways can be viewed as task-specific representations that are composition of modules within supernet's computation graph. This work explores the pathways proposal from the lens of statistical learning: We first develop novel generalization bounds for empirical risk minimization problems learning multiple tasks over multiple paths (Multipath MTL). In conjunction, we formalize the benefits of resulting multipath representation when adapting to new downstream tasks. Our bounds are expressed in terms of Gaussian complexity, lead to tangible guarantees for the class of linear representations, and provide novel insights into the quality and benefits of a multipath representation. When computation graph is a tree, Multipath MTL hierarchically clusters the tasks and builds cluster-specific representations. We provide further discussion and experiments for hierarchical MTL and rigorously identify the conditions under which Multipath MTL is provably superior to traditional MTL approaches with shallow supernets.
APA, Harvard, Vancouver, ISO, and other styles
3

Wang, Gerui, and Sheng Tang. "Generalized Zero-Shot Image Classification via Partially-Shared Multi-Task Representation Learning." Electronics 12, no. 9 (May 3, 2023): 2085. http://dx.doi.org/10.3390/electronics12092085.

Full text
Abstract:
Generalized Zero-Shot Learning (GZSL) holds significant research importance as it enables the classification of samples from both seen and unseen classes. A prevailing approach for GZSL is learning transferable representations that can generalize well to both seen and unseen classes during testing. This approach encompasses two key concepts: discriminative representations and semantic-relevant representations. “Semantic-relevant” facilitates the transfer of semantic knowledge using pre-defined semantic descriptors, while “discriminative” is crucial for accurate category discrimination. However, these two concepts are arguably inherently conflicting, as semantic descriptors are not specifically designed for image classification. Existing methods often struggle with balancing these two aspects and neglect the conflict between them, leading to suboptimal representation generalization and transferability to unseen classes. To address this issue, we propose a novel partially-shared multi-task representation learning method, termed PS-GZSL, which jointly preserves complementary and sharable knowledge between these two concepts. Specifically, we first propose a novel perspective that treats the learning of discriminative and semantic-relevant representations as optimizing a discrimination task and a visual-semantic alignment task, respectively. Then, to learn more complete and generalizable representations, PS-GZSL explicitly factorizes visual features into task-shared and task-specific representations and introduces two advanced tasks: an instance-level contrastive discrimination task and a relation-based visual-semantic alignment task. Furthermore, PS-GZSL employs Mixture-of-Experts (MoE) with a dropout mechanism to prevent representation degeneration and integrates a conditional GAN (cGAN) to synthesize unseen features for estimating unseen visual features. Extensive experiments and more competitive results on five widely-used GZSL benchmark datasets validate the effectiveness of our PS-GZSL.
APA, Harvard, Vancouver, ISO, and other styles
4

Basu Roy Chowdhury, Somnath, and Snigdha Chaturvedi. "Sustaining Fairness via Incremental Learning." Proceedings of the AAAI Conference on Artificial Intelligence 37, no. 6 (June 26, 2023): 6797–805. http://dx.doi.org/10.1609/aaai.v37i6.25833.

Full text
Abstract:
Machine learning systems are often deployed for making critical decisions like credit lending, hiring, etc. While making decisions, such systems often encode the user's demographic information (like gender, age) in their intermediate representations. This can lead to decisions that are biased towards specific demographics. Prior work has focused on debiasing intermediate representations to ensure fair decisions. However, these approaches fail to remain fair with changes in the task or demographic distribution. To ensure fairness in the wild, it is important for a system to adapt to such changes as it accesses new data in an incremental fashion. In this work, we propose to address this issue by introducing the problem of learning fair representations in an incremental learning setting. To this end, we present Fairness-aware Incremental Representation Learning (FaIRL), a representation learning system that can sustain fairness while incrementally learning new tasks. FaIRL is able to achieve fairness and learn new tasks by controlling the rate-distortion function of the learned representations. Our empirical evaluations show that FaIRL is able to make fair decisions while achieving high performance on the target task, outperforming several baselines.
APA, Harvard, Vancouver, ISO, and other styles
5

Davvetas, Athanasios, Iraklis A. Klampanos, Spiros Skiadopoulos, and Vangelis Karkaletsis. "Evidence Transfer: Learning Improved Representations According to External Heterogeneous Task Outcomes." ACM Transactions on Knowledge Discovery from Data 16, no. 5 (October 31, 2022): 1–22. http://dx.doi.org/10.1145/3502732.

Full text
Abstract:
Unsupervised representation learning tends to produce generic and reusable latent representations. However, these representations can often miss high-level features or semantic information, since they only observe the implicit properties of the dataset. On the other hand, supervised learning frameworks learn task-oriented latent representations that may not generalise in other tasks or domains. In this article, we introduce evidence transfer, a deep learning method that incorporates the outcomes of external tasks in the unsupervised learning process of an autoencoder. External task outcomes also referred to as categorical evidence, are represented by categorical variables, and are either directly or indirectly related to the primary dataset—in the most straightforward case they are the outcome of another task on the same dataset. Evidence transfer allows the manipulation of generic latent representations in order to include domain or task-specific knowledge that will aid their effectiveness in downstream tasks. Evidence transfer is robust against evidence of low quality and effective when introduced with related, corresponding, or meaningful evidence.
APA, Harvard, Vancouver, ISO, and other styles
6

Konidaris, George, Leslie Pack Kaelbling, and Tomas Lozano-Perez. "From Skills to Symbols: Learning Symbolic Representations for Abstract High-Level Planning." Journal of Artificial Intelligence Research 61 (January 31, 2018): 215–89. http://dx.doi.org/10.1613/jair.5575.

Full text
Abstract:
We consider the problem of constructing abstract representations for planning in high-dimensional, continuous environments. We assume an agent equipped with a collection of high-level actions, and construct representations provably capable of evaluating plans composed of sequences of those actions. We first consider the deterministic planning case, and show that the relevant computation involves set operations performed over sets of states. We define the specific collection of sets that is necessary and sufficient for planning, and use them to construct a grounded abstract symbolic representation that is provably suitable for deterministic planning. The resulting representation can be expressed in PDDL, a canonical high-level planning domain language; we construct such a representation for the Playroom domain and solve it in milliseconds using an off-the-shelf planner. We then consider probabilistic planning, which we show requires generalizing from sets of states to distributions over states. We identify the specific distributions required for planning, and use them to construct a grounded abstract symbolic representation that correctly estimates the expected reward and probability of success of any plan. In addition, we show that learning the relevant probability distributions corresponds to specific instances of probabilistic density estimation and probabilistic classification. We construct an agent that autonomously learns the correct abstract representation of a computer game domain, and rapidly solves it. Finally, we apply these techniques to create a physical robot system that autonomously learns its own symbolic representation of a mobile manipulation task directly from sensorimotor data---point clouds, map locations, and joint angles---and then plans using that representation. Together, these results establish a principled link between high-level actions and abstract representations, a concrete theoretical foundation for constructing abstract representations with provable properties, and a practical mechanism for autonomously learning abstract high-level representations.
APA, Harvard, Vancouver, ISO, and other styles
7

Gu, Jie, Feng Wang, Qinghui Sun, Zhiquan Ye, Xiaoxiao Xu, Jingmin Chen, and Jun Zhang. "Exploiting Behavioral Consistence for Universal User Representation." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 5 (May 18, 2021): 4063–71. http://dx.doi.org/10.1609/aaai.v35i5.16527.

Full text
Abstract:
User modeling is critical for developing personalized services in industry. A common way for user modeling is to learn user representations that can be distinguished by their interests or preferences. In this work, we focus on developing universal user representation model. The obtained universal representations are expected to contain rich information, and be applicable to various downstream applications without further modifications (e.g., user preference prediction and user profiling). Accordingly, we can be free from the heavy work of training task-specific models for every downstream task as in previous works. In specific, we propose Self-supervised User Modeling Network (SUMN) to encode behavior data into the universal representation. It includes two key components. The first one is a new learning objective, which guides the model to fully identify and preserve valuable user information under a self-supervised learning framework. The other one is a multi-hop aggregation layer, which benefits the model capacity in aggregating diverse behaviors. Extensive experiments on benchmark datasets show that our approach can outperform state-of-the-art unsupervised representation methods, and even compete with supervised ones.
APA, Harvard, Vancouver, ISO, and other styles
8

Yu, Wenmeng, Hua Xu, Ziqi Yuan, and Jiele Wu. "Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysis." Proceedings of the AAAI Conference on Artificial Intelligence 35, no. 12 (May 18, 2021): 10790–97. http://dx.doi.org/10.1609/aaai.v35i12.17289.

Full text
Abstract:
Representation Learning is a significant and challenging task in multimodal learning. Effective modality representations should contain two parts of characteristics: the consistency and the difference. Due to the unified multimodal annota- tion, existing methods are restricted in capturing differenti- ated information. However, additional unimodal annotations are high time- and labor-cost. In this paper, we design a la- bel generation module based on the self-supervised learning strategy to acquire independent unimodal supervisions. Then, joint training the multimodal and uni-modal tasks to learn the consistency and difference, respectively. Moreover, dur- ing the training stage, we design a weight-adjustment strat- egy to balance the learning progress among different sub- tasks. That is to guide the subtasks to focus on samples with the larger difference between modality supervisions. Last, we conduct extensive experiments on three public multimodal baseline datasets. The experimental results validate the re- liability and stability of auto-generated unimodal supervi- sions. On MOSI and MOSEI datasets, our method surpasses the current state-of-the-art methods. On the SIMS dataset, our method achieves comparable performance than human- annotated unimodal labels. The full codes are available at https://github.com/thuiar/Self-MM.
APA, Harvard, Vancouver, ISO, and other styles
9

Ahmed, Mahtab, and Robert E. Mercer. "Modelling Sentence Pairs via Reinforcement Learning: An Actor-Critic Approach to Learn the Irrelevant Words." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 05 (April 3, 2020): 7358–66. http://dx.doi.org/10.1609/aaai.v34i05.6230.

Full text
Abstract:
Learning sentence representation is a fundamental task in Natural Language Processing. Most of the existing sentence pair modelling architectures focus only on extracting and using the rich sentence pair features. The drawback of utilizing all of these features makes the learning process much harder. In this study, we propose a reinforcement learning (RL) method to learn a sentence pair representation when performing tasks like semantic similarity, paraphrase identification, and question-answer pair modelling. We formulate this learning problem as a sequential decision making task where the decision made in the current state will have a strong impact on the following decisions. We address this decision making with a policy gradient RL method which chooses the irrelevant words to delete by looking at the sub-optimal representation of the sentences being compared. With this policy, extensive experiments show that our model achieves on par performance when learning task-specific representations of sentence pairs without needing any further knowledge like parse trees. We suggest that the simplicity of each task inference provided by our RL model makes it easier to explain.
APA, Harvard, Vancouver, ISO, and other styles
10

Mehmood, Tahir, Ivan Serina, Alberto Lavelli, Luca Putelli, and Alfonso Gerevini. "On the Use of Knowledge Transfer Techniques for Biomedical Named Entity Recognition." Future Internet 15, no. 2 (February 17, 2023): 79. http://dx.doi.org/10.3390/fi15020079.

Full text
Abstract:
Biomedical named entity recognition (BioNER) is a preliminary task for many other tasks, e.g., relation extraction and semantic search. Extracting the text of interest from biomedical documents becomes more demanding as the availability of online data is increasing. Deep learning models have been adopted for biomedical named entity recognition (BioNER) as deep learning has been found very successful in many other tasks. Nevertheless, the complex structure of biomedical text data is still a challenging aspect for deep learning models. Limited annotated biomedical text data make it more difficult to train deep learning models with millions of trainable parameters. The single-task model, which focuses on learning a specific task, has issues in learning complex feature representations from a limited quantity of annotated data. Moreover, manually constructing annotated data is a time-consuming job. It is, therefore, vital to exploit other efficient ways to train deep learning models on the available annotated data. This work enhances the performance of the BioNER task by taking advantage of various knowledge transfer techniques: multitask learning and transfer learning. This work presents two multitask models (MTMs), which learn shared features and task-specific features by implementing the shared and task-specific layers. In addition, the presented trained MTM is also fine-tuned for each specific dataset to tailor it from a general features representation to a specialized features representation. The presented empirical results and statistical analysis from this work illustrate that the proposed techniques enhance significantly the performance of the corresponding single-task model (STM).
APA, Harvard, Vancouver, ISO, and other styles
More sources

Dissertations / Theses on the topic "Task-specific representation learnining"

1

Ekhtiari, Amiri Sorour. "Task-specific summarization of networks: Optimization and Learning." Diss., Virginia Tech, 2019. http://hdl.handle.net/10919/100993.

Full text
Abstract:
Networks (also known as graphs) are everywhere. People-contact networks, social networks, email communication networks, internet networks (among others) are examples of graphs in our daily life. The increasing size of these networks makes it harder to understand them. Instead, summarizing these graphs can reveal key patterns and also help in sensemaking as well as accelerating existing graph algorithms. Intuitively, different summarizes are desired for different purposes. For example, to stop viral infections, one may want to find an effective policy to immunize people in a people-contact network. In this case, a high-quality network summary should highlight roughly structurally important nodes. Others may want to detect communities in the same people-contact network, and hence, the summary should show cohesive groups of nodes. This implies that for each task, we should design a specific method to reveal related patterns. Despite the importance of task-specific summarization, there has not been much work in this area. Hence, in this thesis, we design task-specific summarization frameworks for univariate and multivariate networks. We start with optimization-based approaches to summarize graphs for a particular task and finally propose general frameworks which automatically learn how to summarize for a given task and generalize it to similar networks. 1. Optimization-based approaches: Given a large network and a task, we propose summarization algorithms to highlight specific characteristics of the graph (i.e., structure, attributes, labels, dynamics) with respect to the task. We develop effective and efficient algorithms for various tasks such as content-aware influence maximization and time segmentation. In addition, we study many real-world networks and their summary graphs such as people-contact, news-blogs, etc. and visualize them to make sense of their characteristics given the input task. 2. Learning-based approaches: As our next step, we propose a unified framework which learns the process of summarization itself for a given task. First, we design a generalizable algorithm to learn to summarize graphs for a set of graph optimization problems. Next, we go further and add sparse human feedback to the learning process for the given optimization task. To the best of our knowledge, we are the first to systematically bring the necessity of considering the given task to the forefront and emphasize the importance of learning-based approaches in network summarization. Our models and frameworks lead to meaningful discoveries. We also solve problems from various domains such as epidemiology, marketing, social media, cybersecurity, and interactive visualization.
Doctor of Philosophy
APA, Harvard, Vancouver, ISO, and other styles
2

Sen, Shiraj. "Bridging the gap between autonomous skill learning and task-specific planning." 2013. https://scholarworks.umass.edu/dissertations/AAI3556285.

Full text
Abstract:
Skill acquisition and task specific planning are essential components of any robot system, yet they have long been studied in isolation. This, I contend, is due to the lack of a common representational framework. I present a holistic approach to planning robot behavior, using previously acquired skills to represent control knowledge (and objects) directly, and to use this background knowledge to build plans in the space of control actions. Actions in this framework are closed-loop controllers constructed from combinations of sensors, effectors, and potential functions. I will show how robots can use reinforcement learning techniques to acquire sensorimotor programs. The agent then builds a functional model of its interactions with the world as distributions over the acquired skills. In addition, I present two planning algorithms that can reason about a task using the functional models. These algorithms are then applied to a variety of tasks such as object recognition and object manipulation to achieve its objective on two different robot platforms.
APA, Harvard, Vancouver, ISO, and other styles

Book chapters on the topic "Task-specific representation learnining"

1

Lin, Yankai, Ning Ding, Zhiyuan Liu, and Maosong Sun. "Pre-trained Models for Representation Learning." In Representation Learning for Natural Language Processing, 127–67. Singapore: Springer Nature Singapore, 2023. http://dx.doi.org/10.1007/978-981-99-1600-9_5.

Full text
Abstract:
AbstractPre-training-fine-tuning has recently become a new paradigm in natural language processing, learning better representations of words, sentences, and documents in a self-supervised manner. Pre-trained models not only unify semantic representations of multiple tasks, multiple languages, and multiple modalities but also emerge high-level capabilities approaching human beings. In this chapter, we introduce pre-trained models for representation learning, from pre-training tasks to adaptation approaches for specific tasks. After that, we discuss several advanced topics toward better pre-trained representations, including better model architecture, multilingual, multi-task, efficient representations, and chain-of-thought reasoning.
APA, Harvard, Vancouver, ISO, and other styles
2

Bragman, Felix J. S., Ryutaro Tanno, Sebastien Ourselin, Daniel C. Alexander, and M. Jorge Cardoso. "Learning Task-Specific and Shared Representations in Medical Imaging." In Lecture Notes in Computer Science, 374–83. Cham: Springer International Publishing, 2019. http://dx.doi.org/10.1007/978-3-030-32251-9_41.

Full text
APA, Harvard, Vancouver, ISO, and other styles
3

Chen, Tao, Ruifeng Xu, Yulan He, and Xuan Wang. "Learning Task Specific Distributed Paragraph Representations Using a 2-Tier Convolutional Neural Network." In Neural Information Processing, 467–75. Cham: Springer International Publishing, 2015. http://dx.doi.org/10.1007/978-3-319-26532-2_51.

Full text
APA, Harvard, Vancouver, ISO, and other styles
4

Kozłowski, Marek, Przemyslaw Buczkowski, and Piotr Brzezinski. "A Novel Process of Shoe Pairing Using Computer Vision and Deep Learning Methods." In Digital Interaction and Machine Intelligence, 35–44. Cham: Springer Nature Switzerland, 2023. http://dx.doi.org/10.1007/978-3-031-37649-8_4.

Full text
Abstract:
AbstractThe industrialisation of the footwear recycling processes is a major issue in the European Union—particularly in view of the fact that at least 90% of shoes consumed in western economies are ultimately sent to landfill. This requires new AI-empowered technologies that enable detection, classification, pairing, and quality assessment in a viable automatic process. This article discusses automatic shoe pairing, which comprises two sequential stages: a) deep multiview shoe embedding (compact representation of multiview data); and b) clustering of shoes’ embeddings with a fixed similarity threshold to return sets of possible pairs. Each shoe in our pipeline is represented by multiple images that are collected in industrial darkrooms. We present various approaches to shoe pairing—from fully unsupervised ones based on image descriptors to supervised ones that rely on deep neural networks—to identify the most effective one for this highly specific industrial task. The article also explains how the selected method can be improved by hyperparameter tuning, massive increases in training data, and data augmentation.
APA, Harvard, Vancouver, ISO, and other styles
5

Shi, Lei, Yahui Li, Boon Thau Loo, and Rajeev Alur. "Network Traffic Classification by Program Synthesis." In Tools and Algorithms for the Construction and Analysis of Systems, 430–48. Cham: Springer International Publishing, 2021. http://dx.doi.org/10.1007/978-3-030-72016-2_23.

Full text
Abstract:
AbstractWriting classification rules to identify interesting network traffic is a time-consuming and error-prone task. Learning-based classification systems automatically extract such rules from positive and negative traffic examples. However, due to limitations in the representation of network traffic and the learning strategy, these systems lack both expressiveness to cover a range of applications and interpretability in fully describing the traffic’s structure at the session layer. This paper presents Sharingan system, which uses program synthesis techniques to generate network classification programs at the session layer. Sharingan accepts raw network traces as inputs and reports potential patterns of the target traffic in NetQRE, a domain specific language designed for specifying session-layer quantitative properties. We develop a range of novel optimizations that reduce the synthesis time for large and complex tasks to a matter of minutes. Our experiments show that Sharingan is able to correctly identify patterns from a diverse set of network traces and generates explainable outputs, while achieving accuracy comparable to state-of-the-art learning-based systems.
APA, Harvard, Vancouver, ISO, and other styles
6

Chen, Tao, Ruifeng Xu, Yulan He, and Xuan Wang. "Learning Task Specific Distributed Paragraph Representations Using a 2-tier Convolutional Neural Network." In Social Media Content Analysis, 161–70. WORLD SCIENTIFIC, 2017. http://dx.doi.org/10.1142/9789813223615_0012.

Full text
APA, Harvard, Vancouver, ISO, and other styles
7

Kalyuga, Slava. "Managing Cognitive Load in Verbal and Pictorial Representations." In Managing Cognitive Load in Adaptive Multimedia Learning, 123–48. IGI Global, 2009. http://dx.doi.org/10.4018/978-1-60566-048-6.ch006.

Full text
Abstract:
Chapter VI describes specific evidence-based methods for managing cognitive load in verbal and pictorial information representations. According to the major forms of memory storage, there are verbal and pictorial representational modes, whereas according to major forms of sensory input, there are auditory and visual information modalities. The chapter will consider sources of cognitive load involving different modes and modalities of multimedia information presentations. When learners process text and visuals that could not be understood in isolation, the process of integrating verbal and pictorial representations is required for comprehension. When text and pictures are not appropriately located close to each other or not synchronized in time, integrating these referring sources of information may increase working memory load and inhibit learning. Instructional design techniques dealing with such split attention situations may enhance learning. Reducing split-attention in paper-based and on-screen text and graphics was one of the first and most commonly mentioned applications of cognitive load theory. Using dualmode presentations that involve different processing channels of human cognitive system is an alternative approach to dealing with split attention situations. This chapter discusses means for coordinating verbal and pictorial sources of information in space and time, eliminating redundant components of presentations, segmenting instructional presentations in units that could be processed with less cognitive load, and other techniques. The chapter also describes interactions between instructional efficiency of different formats of multimedia presentations and levels of learner expertise in specific task domains.
APA, Harvard, Vancouver, ISO, and other styles
8

Poon, Hoifung, Hai Wang, and Hunter Lang. "Chapter 14. Combining Probabilistic Logic and Deep Learning for Self-Supervised Learning." In Frontiers in Artificial Intelligence and Applications. IOS Press, 2021. http://dx.doi.org/10.3233/faia210361.

Full text
Abstract:
Deep learning has proven effective for various application tasks, but its applicability is limited by the reliance on annotated examples. Self-supervised learning has emerged as a promising direction to alleviate the supervision bottleneck, but existing work focuses on leveraging co-occurrences in unlabeled data for task-agnostic representation learning, as exemplified by masked language model pretraining. In this chapter, we explore task-specific self-supervision, which leverages domain knowledge to automatically annotate noisy training examples for end applications, either by introducing labeling functions for annotating individual instances, or by imposing constraints over interdependent label decisions. We first present deep probabilistic logic (DPL), which offers a unifying framework for task-specific self-supervision by composing probabilistic logic with deep learning. DPL represents unknown labels as latent variables and incorporates diverse self-supervision using probabilistic logic to train a deep neural network end-to-end using variational EM. Next, we present self-supervised self-supervision (S4), which adds to DPL the capability to learn new self-supervision automatically. Starting from an initial seed self-supervision, S4 iteratively uses the deep neural network to propose new self supervision. These are either added directly (a form of structured self-training) or verified by a human expert (as in feature-based active learning). Experiments on real-world applications such as biomedical machine reading and various text classification tasks show that task-specific self-supervision can effectively leverage domain expertise and often match the accuracy of supervised methods with a tiny fraction of human effort.
APA, Harvard, Vancouver, ISO, and other styles
9

"Space Representation and Gender Differences." In A Simplex Approach to Learning, Cognition, and Spatial Navigation, 23–28. IGI Global, 2018. http://dx.doi.org/10.4018/978-1-5225-2455-7.ch003.

Full text
Abstract:
Scientific literature highlighted gender differences in spatial orientation. In particular, men and women differ in terms of the navigational processes they use in daily life. Scientific literature highlighted that women use analytical strategies while men tend to use holistic strategies. According to classical studies, males show a net advantage at least in the two categories of mental rotation and spatial perception. Subsequently, brain-imaging studies have shown a difference between males and females in the activity of brain regions involved in spatial cognition tasks. What we can say with certainty is that, given the complex nature of the subprocesses involved in what we call spatial cognition, the gender differences recorded by numerous scientific studies conducted in this field are closely related to specific measured abilities. The evidence that emerges with certainty from diverse studies is, however, that of a huge variety of strategies that differ according to sex, context, purpose to reach, education, age, and profession. In the study presented here, the gender and age-related tests show a significant sex-based difference perspective-taking tasks, but there is no gender-based difference in the mental rotation task.
APA, Harvard, Vancouver, ISO, and other styles
10

De Luca, Stefano, and Walter Quattrociocchi. "Dynamic Contexts and Concepts as a Set of State Variations Under Emerging Functions." In Applications of Complex Adaptive Systems, 34–56. IGI Global, 2008. http://dx.doi.org/10.4018/978-1-59904-962-5.ch002.

Full text
Abstract:
In this paper we present a new agent-based model, CDYS – Complex DYnamic System, for intelligent, flexible, and context-aware multi-modal interaction on autonomous system. This model is focused on context models which facilitate the communication and the knowledge representation with an highly-customized and adaptable representation and distribution of the entities composing the environment. CDYS uses information from multiple perceptions and provides proactive real-time updates and context-specific guidance in the state representation and synthesis. Our work includes the design of interaction, evolution, context definition by states and ontologies; communication, context, task models based on these ontologies, a set of representations of perception to drive agent behavior, communication, and a compatible integration of rules and machine learning aimed to improve information retrieval and the Semantic web. Currently, we have completed the first stage of our research, producing first pass Ontologies, models, and the interaction to apply genetic algorithm to improve the global ontology, by local ontology representation, tested with an initial prototype of a small-scale test-bed on clustering for self improving of the agent’s knowledge base .Our approach is based on social systems in context-aware applications, informed by Autopoietic Systems, to use a system (an agent) that is able to describe and manage the evolution of its environment and of the knowledge base in the autopoietic based model.
APA, Harvard, Vancouver, ISO, and other styles

Conference papers on the topic "Task-specific representation learnining"

1

Peng, Minlong, Qi Zhang, Xiaoyu Xing, Tao Gui, Jinlan Fu, and Xuanjing Huang. "Learning Task-Specific Representation for Novel Words in Sequence Labeling." In Twenty-Eighth International Joint Conference on Artificial Intelligence {IJCAI-19}. California: International Joint Conferences on Artificial Intelligence Organization, 2019. http://dx.doi.org/10.24963/ijcai.2019/715.

Full text
Abstract:
Word representation is a key component in neural-network-based sequence labeling systems. However, representations of unseen or rare words trained on the end task are usually poor for appreciable performance. This is commonly referred to as the out-of-vocabulary (OOV) problem. In this work, we address the OOV problem in sequence labeling using only training data of the task. To this end, we propose a novel method to predict representations for OOV words from their surface-forms (e.g., character sequence) and contexts. The method is specifically designed to avoid the error propagation problem suffered by existing approaches in the same paradigm. To evaluate its effectiveness, we performed extensive empirical studies on four part-of-speech tagging (POS) tasks and four named entity recognition (NER) tasks. Experimental results show that the proposed method can achieve better or competitive performance on the OOV problem compared with existing state-of-the-art methods.
APA, Harvard, Vancouver, ISO, and other styles
2

Mannem, Renuka, Hima Jyothi R., Aravind Illa, and Prasanta Kumar Ghosh. "Speech Rate Task-Specific Representation Learning from Acoustic-Articulatory Data." In Interspeech 2020. ISCA: ISCA, 2020. http://dx.doi.org/10.21437/interspeech.2020-2259.

Full text
APA, Harvard, Vancouver, ISO, and other styles
3

Zhang, Chenrui, and Yuxin Peng. "Better and Faster: Knowledge Transfer from Multiple Self-supervised Learning Tasks via Graph Distillation for Video Classification." In Twenty-Seventh International Joint Conference on Artificial Intelligence {IJCAI-18}. California: International Joint Conferences on Artificial Intelligence Organization, 2018. http://dx.doi.org/10.24963/ijcai.2018/158.

Full text
Abstract:
Video representation learning is a vital problem for classification task. Recently, a promising unsupervised paradigm termed self-supervised learning has emerged, which explores inherent supervisory signals implied in massive data for feature learning via solving auxiliary tasks. However, existing methods in this regard suffer from two limitations when extended to video classification. First, they focus only on a single task, whereas ignoring complementarity among different task-specific features and thus resulting in suboptimal video representation. Second, high computational and memory cost hinders their application in real-world scenarios. In this paper, we propose a graph-based distillation framework to address these problems: (1) We propose logits graph and representation graph to transfer knowledge from multiple self-supervised tasks, where the former distills classifier-level knowledge by solving a multi-distribution joint matching problem, and the latter distills internal feature knowledge from pairwise ensembled representations with tackling the challenge of heterogeneity among different features; (2) The proposal that adopts a teacher-student framework can reduce the redundancy of knowledge learned from teachers dramatically, leading to a lighter student model that solves classification task more efficiently. Experimental results on 3 video datasets validate that our proposal not only helps learn better video representation but also compress model for faster inference.
APA, Harvard, Vancouver, ISO, and other styles
4

James, Steven. "Learning Portable Symbolic Representations." In Twenty-Seventh International Joint Conference on Artificial Intelligence {IJCAI-18}. California: International Joint Conferences on Artificial Intelligence Organization, 2018. http://dx.doi.org/10.24963/ijcai.2018/826.

Full text
Abstract:
An open question in artificial intelligence is how to learn useful representations of the real world. One approach is to learn symbols, which represent the world and its contents, as well as models describing the effects on these symbols when interacting with the world. To date, however, research has investigated learning such representations for a single specific task. Our research focuses on approaches to learning these models in a domain-independent manner. We intend to use these symbolic models to build even higher levels of abstraction, creating a hierarchical representation which could be used to solve complex tasks. This would allow an agent to gather knowledge over the course of its lifetime, which could then be leveraged when faced with a new task, obviating the need to relearn a model every time a new unseen problem is encountered.
APA, Harvard, Vancouver, ISO, and other styles
5

Tsvetkov, Yulia, Manaal Faruqui, Wang Ling, Brian MacWhinney, and Chris Dyer. "Learning the Curriculum with Bayesian Optimization for Task-Specific Word Representation Learning." In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA, USA: Association for Computational Linguistics, 2016. http://dx.doi.org/10.18653/v1/p16-1013.

Full text
APA, Harvard, Vancouver, ISO, and other styles
6

Chen, Shaoxiang, Ting Yao, and Yu-Gang Jiang. "Deep Learning for Video Captioning: A Review." In Twenty-Eighth International Joint Conference on Artificial Intelligence {IJCAI-19}. California: International Joint Conferences on Artificial Intelligence Organization, 2019. http://dx.doi.org/10.24963/ijcai.2019/877.

Full text
Abstract:
Deep learning has achieved great successes in solving specific artificial intelligence problems recently. Substantial progresses are made on Computer Vision (CV) and Natural Language Processing (NLP). As a connection between the two worlds of vision and language, video captioning is the task of producing a natural-language utterance (usually a sentence) that describes the visual content of a video. The task is naturally decomposed into two sub-tasks. One is to encode a video via a thorough understanding and learn visual representation. The other is caption generation, which decodes the learned representation into a sequential sentence, word by word. In this survey, we first formulate the problem of video captioning, then review state-of-the-art methods categorized by their emphasis on vision or language, and followed by a summary of standard datasets and representative approaches. Finally, we highlight the challenges which are not yet fully understood in this task and present future research directions.
APA, Harvard, Vancouver, ISO, and other styles
7

Liu, Yang, Jing Liu, Xiaoguang Zhu, Donglai Wei, Xiaohong Huang, and Liang Song. "Learning Task-Specific Representation for Video Anomaly Detection with Spatial-Temporal Attention." In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022. http://dx.doi.org/10.1109/icassp43922.2022.9746822.

Full text
APA, Harvard, Vancouver, ISO, and other styles
8

Sun, Jingyuan, and Marie-Francine Moens. "Fine-tuned vs. Prompt-tuned Supervised Representations: Which Better Account for Brain Language Representations?" In Thirty-Second International Joint Conference on Artificial Intelligence {IJCAI-23}. California: International Joint Conferences on Artificial Intelligence Organization, 2023. http://dx.doi.org/10.24963/ijcai.2023/577.

Full text
Abstract:
To decipher the algorithm underlying the human brain's language representation, previous work probed brain responses to language input with pre-trained artificial neural network (ANN) models fine-tuned on NLU tasks. However, full fine-tuning generally updates the entire parametric space and distorts pre-trained features, cognitively inconsistent with the brain's robust multi-task learning ability. Prompt-tuning, in contrast, protects pre-trained weights and learns task-specific embeddings to fit a task. Could prompt-tuning generate representations that better account for the brain's language representations than fine-tuning? If so, what kind of NLU task leads a pre-trained model to better decode the information represented in the human brain? We investigate these questions by comparing prompt-tuned and fine-tuned representations in neural decoding, that is predicting the linguistic stimulus from the brain activities evoked by the stimulus. We find that on none of the 10 NLU tasks, full fine-tuning significantly outperforms prompt-tuning in neural decoding, implicating that a more brain-consistent tuning method yields representations that better correlate with brain data. Moreover, we identify that tasks dealing with fine-grained concept meaning yield representations that better decode brain activation patterns than other tasks, especially the syntactic chunking task. This indicates that our brain encodes more fine-grained concept information than shallow syntactic information when representing languages.
APA, Harvard, Vancouver, ISO, and other styles
9

Zheng, Renjie, Junkun Chen, and Xipeng Qiu. "Same Representation, Different Attentions: Shareable Sentence Representation Learning from Multiple Tasks." In Twenty-Seventh International Joint Conference on Artificial Intelligence {IJCAI-18}. California: International Joint Conferences on Artificial Intelligence Organization, 2018. http://dx.doi.org/10.24963/ijcai.2018/642.

Full text
Abstract:
Distributed representation plays an important role in deep learning based natural language processing. However, the representation of a sentence often varies in different tasks, which is usually learned from scratch and suffers from the limited amounts of training data. In this paper, we claim that a good sentence representation should be invariant and can benefit the various subsequent tasks. To achieve this purpose, we propose a new scheme of information sharing for multi-task learning. More specifically, all tasks share the same sentence representation and each task can select the task-specific information from the shared sentence representation with attention mechanisms. The query vector of each task's attention could be either static parameters or generated dynamically. We conduct extensive experiments on 16 different text classification tasks, which demonstrate the benefits of our architecture. Source codes of this paper are available on Github.
APA, Harvard, Vancouver, ISO, and other styles
10

Shin, Kyuyong, Hanock Kwak, Wonjae Kim, Jisu Jeong, Seungjae Jung, Kyungmin Kim, Jung-Woo Ha, and Sang-Woo Lee. "Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning." In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA, USA: Association for Computational Linguistics, 2023. http://dx.doi.org/10.18653/v1/2023.acl-long.64.

Full text
APA, Harvard, Vancouver, ISO, and other styles
You might also be interested in the extended bibliographies on the topic 'Task-specific representation learnining' for particular source types:
Journal articles
We offer discounts on all premium plans for authors whose works are included in thematic literature selections. Contact us to get a unique promo code!

To the bibliography