Rozprawy doktorskie na temat „Limited training data”

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1995.
Includes bibliographical references (leaves 149-155).
by Eric I-Chao Chang.
Ph.D.

Scene understanding by a machine is a challenging task due to the profound variety of nature. Nevertheless, deep learning achieves impressive results in several scene understanding tasks such as semantic segmentation, depth estimation, or optical flow. However, these kinds of approaches need a large amount of labeled data, leading to massive manual annotations, which are incredibly tedious and expensive to collect. In this thesis, we will focus on understanding a scene through deep learning with limited data availability. First of all, we will tackle the problem of the lack of data for semantic segmentation. We will show that computer graphics come in handy to our purpose, both to create a new, efficient tool for annotation as well to render synthetic annotated datasets quickly. However, a network trained only on synthetic data suffers from the so-called domain-shift problem, i.e. unable to generalize to real data. Thus, we will show that we can mitigate this problem using a novel deep image to image translation technique. In the second part of the thesis, we will focus on the relationship between scene understanding tasks. We argue that building a model aware of the connections between tasks is the first building stone to create more robust, efficient, performant models that need less annotated training data. In particular, we demonstrate that we can decrease the need for labels by exploiting the relationship between visual tasks. Finally, in the last part, we propose a novel unified framework for comprehensive scene understanding, which exploits the synergies between tasks to be more robust, efficient, and performant.

Abstract This thesis presents a novel method of audio-visual fusion, known as multi- modal optimal feature fusion (MOFF), for person identification where both the speech and facial modalities may be corrupted, and there is a lack of prior knowl- edge about the corruption. Furthermore, it is assumed there is a limited amount of training data for each modality (e.g., a short training speech segment and a single training facial image for each person). A new multimodal feature rep- resentation and a modified cosine similarity are introduced for combining and comparing bimodal features with limited training data, as well as vastly differing data rates and feature sizes. Similarity-based optimal feature selection and multi- condition training are used to reduce the mismatch between training and testing, thereby making the system robust to unknown bimodal corruption. Low-level feature fusion is performed using optimal feature selection, which automatically changes the weighting given to each modality based on the level of corruption. The framework for robust person identification is also applied to noise robust speaker identification, given very limited training data. Experiments have been carried out on a bimodal data set created from the SPIDRE speaker recogni- tion database and AR face recognition database, with variable noise corruption of speech and occlusion in the face images. Combining both modalities using MOFF, leads to significantly improved identification accuracy compared to the component unimodal systems, even with simultaneous corruption of both modal- ities. A novel piecewise-constant illumination model (PCIlVI) is then introduced for illumination invariant facial recognition. This method can be used given a single training facial image for each person, and assuming no prior knowledge of the illumination conditions of both the training and testing images. Small areas of the face are represented using magnitude Fourier features, which takes advan- tage of the shift-invariance of the magnitude Fourier representation, to increase robustness to small misalignment errors and small facial expression changes. Fi- nally, cosine similarity is used as an illumination invariant similarity measure, to compare small facial areas. Experiments have been carried out on the YaleB, ex- tended YaleB and eMU-PIE facial illumination databases. Facial identification accuracy using PCIlVI is comparable to or exceeds that of the literature.

With the growing installation of surveillance video cameras in both private and public areas, it is an immediate requirement to develop intelligent video analysis system for the large-scale camera network. As a prerequisite step of person tracking and person retrieval in intelligent video analysis, person re-identification, which targets in matching person images across camera views is an important topic in computer vision community and has been received increasing attention in the recent years. In the supervised learning methods, the person re-identification task is formulated as a classification problem to extract matched person images/videos (positives) from unmatched person images/videos (negatives). Although the state-of-the-art supervised classification models could achieve encouraging re-identification performance, the assumption that label information is available for all the cameras, is impractical in large-scale camera network. That is because collecting the label information of every training subject from every camera in the large-scale network can be extremely time-consuming and expensive. While the unsupervised learning methods are flexible, their performance is typically weaker than the supervised ones. Though sufficient labels of the training subjects are not available from all the camera views, it is still reasonable to collect sufficient labels from a pair of camera views in the camera network or a few labeled data from each camera pair. Along this direction, we address two scenarios of person re-identification in large-scale camera network in this thesis, i.e. unsupervised domain adaptation and semi-supervised learning and proposed three methods to learn discriminative model using all available label information and domain knowledge in person re-identification. In the unsupervised domain adaptation scenario, we consider data with sufficient labels as the source domain, while data from the camera pair missing label information as the target domain. A novel domain adaptive approach is proposed to estimate the target label information and incorporate the labeled data from source domain with the estimated target label information for discriminative learning. Since the discriminative constraint of Support Vector Machines (SVM) can be relaxed into a necessary condition, which only relies on the mean of positive pairs (positive mean), a suboptimal classification model learning without target positive data can be those using target positive mean. A reliable positive mean estimation is given by using both the labeled data from the source domain and potential positive data selected from the unlabeled data in the target domain. An Adaptive Ranking Support Vector Machines (AdaRSVM) method is also proposed to improve the discriminability of the suboptimal mean based SVM model using source labeled data. Experimental results demonstrate the effectiveness of the proposed method. Different from the AdaRSVM method that using source labeled data, we can also improve the above mean based method by adapting it onto target unlabeled data. In more general situation, we improve a pre-learned classifier by adapting it onto target unlabeled data, where the pre-learned classifier can be domain adaptive or learned from only source labeled data. Since it is difficult to estimate positives from the imbalanced target unlabeled data, we propose to alternatively estimate positive neighbors which refer to data close to any true target positive. An optimization problem for positive neighbor estimation from unlabeled data is derived and solved by aligning the cross-person score distributions together with optimizing for multiple graphs based label propagation. To utilize the positive neighbors to learn discriminative classification model, a reliable multiple region metric learning method is proposed to learn a target adaptive metric using regularized affine hulls of positive neighbors as positive regions. Experimental results demonstrate the effectiveness of the proposed method. In the semi-supervised learning scenario, we propose a discriminative feature learning using all available information from the surveillance videos. To enrich the labeled data from target camera pair, image sequences (videos) of the tagged persons are collected from the surveillance videos by human tracking. To extract the discriminative and adaptable video feature representation, we propose to model the intra-view variations by a video variation dictionary and a video level adaptable feature by multiple sources domain adaptation and an adaptability-discriminability fusion. First, a novel video variation dictionary learning is proposed to model the large intra-view variations and solved as a constrained sparse dictionary learning problem. Second, a frame level adaptable feature is generated by multiple sources domain adaptation using the variation modeling. By mining the discriminative information of the frames from the reconstruction error of the variation dictionary, an adaptability-discriminability (AD) fusion is proposed to generate the video level adaptable feature. Experimental results demonstrate the effectiveness of the proposed method.

Big data is an increasingly attractive concept in many fields both in academia and in industry. The increasing amount of information actually builds an illusion that we are going to have enough data to solve all the data driven problems. Unfortunately it is not true, especially for areas where machine learning methods are heavily employed, since sufficient high-quality training data doesn't necessarily come with the big data, and it is not easy or sometimes impossible to collect sufficient training samples, which most computational algorithms depend on. This thesis mainly focuses on dealing situations with limited training data in visual object recognition, by developing novel machine learning algorithms to overcome the limited training data difficulty. We investigate three issues in object recognition involving limited training data: 1. one-shot object recognition, 2. cross-domain object recognition, and 3. object recognition for images with different picture styles. For Issue 1, we propose an unsupervised feature learning algorithm by constructing a deep structure of the stacked Hierarchical Dirichlet Process (HDP) auto-encoder, in order to extract "semantic" information from unlabeled source images. For Issue 2, we propose a Domain Adaptive Input-Output Kernel Learning algorithm to reduce the domain shifts in both input and output spaces. For Issue 3, we introduce a new problem involving images with different picture styles, successfully formulate the relationship between pixel mapping functions with gradient based image descriptors, and also propose a multiple kernel based algorithm to learn an optimal combination of basis pixel mapping functions to improve the recognition accuracy. For all the proposed algorithms, experimental results on publicly available data sets demonstrate the performance improvements over previous state-of-arts.

Today modern vehicles are expected to be safe, environmentally friendly, durable and economical. Monitoring the health of the vehicle is therefore more important than ever. As the complexity of vehicular systems increases the need for efficient monitoring methods has increased as well. Traditional methods of deriving models for the systems are today not as efficient as the complexity of the systems increases the time and skill needed to implement the models. An alternative is data driven methods where a collection of data associated with the behavior of the system is used to draw conclusions of the state of the system. Faults are however rare events and collecting sufficient data to cover all possible faults threatening a vehicle would be impossible. A method for drawing conclusions from limited historical data would therefore be desirable. In this thesis an algorithm using distiguishability as a method for fault classification and fault severity estimation is proposed. Historical data is interpolated over a fault severity vector using Gaussian process regression as a way to estimate fault modes for unknown fault sizes. The algorithm is then tested against validation data to evaluate the ability to detect and identify known fault classes and fault serveries, separate unknown fault classes from known fault classes, and estimate unknown fault sizes. The purpose of the study is to evaluate the possibility to use limited historical data to reduce the need for costly and time consuming data collection. The study shows promising results as fault class identification and fault size estimation using the proposed algorithm seem possible for fault sizes not included in the historical data.

This thesis deals with CT data segmentation using convolutional neural nets and describes the problem of training with limited training sets. User interaction is suggested as means of improving segmentation quality for the models trained on small training sets and the possibility of using transfer learning is also considered. All of the chosen methods help improve the segmentation quality in comparison with the baseline method, which is the use of automatic data specific segmentation model. The segmentation has improved by tens of percents in Dice score when trained with very small datasets. These methods can be used, for example, to simplify the creation of a new segmentation dataset.

碩士
國立臺灣大學
資訊網路與多媒體研究所
107
The quantity of the data today already surpassed the amount a man can handle. How to deal with and filter enormous text data with the help of machine is becoming more and more important. However, classification task on text data requires enough labeled data to back up the training procedure. If we somehow want to deal with this issue, it’s surely a problem we have to deal with. Before starting on this work, we observed multiple news example from Taiwanese online media and found some pattern that we can take advantage of: first, we found that some news about similar topics tend to appear over and over again. We assumed that news of these topics might be more tempting and thus can attract more readers and hype up emotions. This is a good place to start the work since news of these topics might appear again in the near future thus our work will have immediate impact. Second, we found that even if news of these topics tends to appear again and again, the objective and minor details are usually completely different. Using traditional word matching model on this task might not work very well. Based on above reasons, we propose an approach to use the few seed sentences we get from past news on these events to generate training data. Furthermore, for events of similar topic, we define a higher-level topic to include these events. We generate positive and negative training example based on seed sentences and propose a model that can take fully advantage of our generated dataset on classification task. A series of experiments are also conducted to measure the capability of our approach. To realize our approach, we crawled news articles published by public news media during the past 5 to 10 years to build a corpus from which we can sample negative data. Then for each higher-level topic we generate positive and negative datasets. In this work, our main approach can be divided into two part. The first part being the retrieval and generation of dataset and the second part being the training of classifier using the data generated. Our experiment results showed that generation and augmentation we applied can help boosting the performance on this task.

碩士
國立交通大學
資訊科學與工程研究所
100
In this thesis, we consider tracking an articulated hand without using markers. Our hand tracking method performs nearest-neighbor-based search in a 3D hand model large database. For robustly and efficiently, we choose to capture a small real hand images database for each user as an intermediate dataset. And use Hierarchical-searching and temporal consistency to efficiently search in the large database and disambiguate the result. Our prototype system can estimate hand pose including rigid and non-rigid out-of-image-plane rotation, slow and fast gesture charging when rotation, and recover after the hand left the camera in real time. We believe it can be a more intuitive way for advance human computer interaction.

abstract: In the current millennium, extensive use of computers and the internet caused an exponential increase in information. Few research areas are as important as information extraction, which primarily involves extracting concepts and the relations between them from free text. Limitations in the size of training data, lack of lexicons and lack of relationship patterns are major factors for poor performance in information extraction. This is because the training data cannot possibly contain all concepts and their synonyms; and it contains only limited examples of relationship patterns between concepts. Creating training data, lexicons and relationship patterns is expensive, especially in the biomedical domain (including clinical notes) because of the depth of domain knowledge required of the curators. Dictionary-based approaches for concept extraction in this domain are not sufficient to effectively overcome the complexities that arise because of the descriptive nature of human languages. For example, there is a relatively higher amount of abbreviations (not all of them present in lexicons) compared to everyday English text. Sometimes abbreviations are modifiers of an adjective (e.g. CD4-negative) rather than nouns (and hence, not usually considered named entities). There are many chemical names with numbers, commas, hyphens and parentheses (e.g. t(3;3)(q21;q26)), which will be separated by most tokenizers. In addition, partial words are used in place of full words (e.g. up- and downregulate); and some of the words used are highly specialized for the domain. Clinical notes contain peculiar drug names, anatomical nomenclature, other specialized names and phrases that are not standard in everyday English or in published articles (e.g. "l shoulder inj"). State of the art concept extraction systems use machine learning algorithms to overcome some of these challenges. However, they need a large annotated corpus for every concept class that needs to be extracted. A novel natural language processing approach to minimize this limitation in concept extraction is proposed here using distributional semantics. Distributional semantics is an emerging field arising from the notion that the meaning or semantics of a piece of text (discourse) depends on the distribution of the elements of that discourse in relation to its surroundings. Distributional information from large unlabeled data is used to automatically create lexicons for the concepts to be tagged, clusters of contextually similar words, and thesauri of distributionally similar words. These automatically generated lexical resources are shown here to be more useful than manually created lexicons for extracting concepts from both literature and narratives. Further, machine learning features based on distributional semantics are shown to improve the accuracy of BANNER, and could be used in other machine learning systems such as cTakes to improve their performance. In addition, in order to simplify the sentence patterns and facilitate association extraction, a new algorithm using a "shotgun" approach is proposed. The goal of sentence simplification has traditionally been to reduce the grammatical complexity of sentences while retaining the relevant information content and meaning to enable better readability for humans and enhanced processing by parsers. Sentence simplification is shown here to improve the performance of association extraction systems for both biomedical literature and clinical notes. It helps improve the accuracy of protein-protein interaction extraction from the literature and also improves relationship extraction from clinical notes (such as between medical problems, tests and treatments). Overall, the two main contributions of this work include the application of sentence simplification to association extraction as described above, and the use of distributional semantics for concept extraction. The proposed work on concept extraction amalgamates for the first time two diverse research areas -distributional semantics and information extraction. This approach renders all the advantages offered in other semi-supervised machine learning systems, and, unlike other proposed semi-supervised approaches, it can be used on top of different basic frameworks and algorithms.
Dissertation/Thesis
Ph.D. Biomedical Informatics 2011

In this study, to increase inference speed of convolutional neural networks, the optimization technique low-rank tensor decomposition has been implemented and applied to AlexNet which had been trained to classify dog breeds. Due to a small training set, transfer learning was used in order to be able to classify dog breeds. The purpose of the study is to investigate how effective low-rank tensor decomposition is when the training set is limited. The results obtained from this study, compared to a previous study, indicate that there is a strong relationship between the effects of the tensor decomposition and how much available training data exists. A significant speed up can be obtained in the different convolutional layers using tensor decomposition. However, since there is a need to retrain the network after the decomposition and due to the limited dataset there is a slight decrease in accuracy.
För att öka inferenshastigheten hos faltningssnätverk, har i denna studie optimeringstekniken low-rank tensor decomposition implementerats och applicerats på AlexNet, som har tränats för att klassificera hundraser. På grund av en begränsad mängd träningsdata användes transfer learning för uppgiften. Syftet med studien är att undersöka hur effektiv low-rank tensor decomposition är när träningsdatan är begränsad. Jämfört med resultaten från en tidigare studie visar resultaten från denna studie att det finns ett starkt samband mellan effekterna av low-rank tensor decomposition och hur mycket tillgänglig träningsdata som finns. En signifikant hastighetsökning kan uppnås i de olika faltningslagren med hjälp av low-rank tensor decomposition. Eftersom det finns ett behov av att träna om nätverket efter dekompositionen och på grund av den begränsade mängden data så uppnås hastighetsökningen dock på bekostnad av en viss minskning i precisionen för modellen.

The performance of any deep learning model depends heavily on the quantity and quality of the available training data. The generalization of the trained deep models improves with the availability of a large number of training samples and hence these models are often referred to as ‘data-hungry’. However, large scale datasets may not always be available in practice due to proprietary/privacy reasons or because of the high cost of generation, annotation, transmission and storage of data. Hence, efficient utilization of available data is of utmost importance, and this gives rise to a class of ML problems, which is often referred to as “data-efficient deep learning”. In this thesis we study the various types of such problems for diverse applications in computer vision, where the aim is to design deep neural network-based solutions that do not rely on the availability of large quantities of training data to attain the desired performance goals. Under the aforementioned thematic area, this thesis focuses on three different scenarios, namely - (1) learning in the absence of training data, (2) learning with limited training data and (3) learning using selected subset of training data. Absence of training data: Pre-trained deep models hold their learnt knowledge in the form of model parameters that act as ‘memory’ for the trained models and help them generalize well on unseen data. In the first part of this thesis, we present solutions to a diverse set of ‘zero-shot’ tasks, where in absence of any training data (or even their statistics) the trained models are leveraged to synthesize data-representative samples. We dub them Data Impressions (DIs), which act as proxy to the training data. As the DIs are not tied to any specific application, we show their utility in solving several CV/ML tasks under the challenging data-free setup, such as unsupervised domain adaptation, continual learning as well as knowledge distillation (KD). We also study the adversarial robustness of lightweight models trained via knowledge distillation using DIs. Further, we demonstrate the efficacy of DIs in generating data-free Universal Adversarial Perturbations (UAPs) with better fooling rates. However, one limiting factor of this solution is the relatively high computation (i.e., several rounds of backpropagation) to synthesize each sample. In fact, the other natural alternatives such as GAN based solutions also suffer from similar computational overhead and complicated training procedures. This motivated us to explore the utility of target class-balanced ‘arbitrary’ data as transfer set, which achieves competitive distillation performance and can yield strong baselines for data-free KD. We have also proposed data-free solutions beyond classification by extending zero-shot distillation to the object detection task, where we compose the pseudo transfer set by synthesizing multi-object impressions from a pretrained faster RCNN model. Another concern with the deployment of given trained models is their vulnerability against adversarial attacks. The popular adversarial training strategies rely on availability of original training data or explicit regularization-based techniques. On the contrary, we propose test-time adversarial defense (detection and correction framework), which can provide robustness in absence of training data and their statistics. We observe significant improvements in adversarial accuracy with minimal drop in clean accuracy against state-of-the-art ‘Auto Attack’ without having to retrain the model. Further, we explore an even more challenging problem setup and make the first attempt to provide adversarial robustness to ‘black box’ models (i.e., model architecture, weights, training details are inaccessible) under a complete data-free set up. Our method minimizes adversarial contamination on perturbed samples via proposed ‘wavelet noise remover’ (WNR) that remove coefficients corresponding to high frequency components which are most likely to be corrupted by adversarial attack, and recovers the lost image content by training a ‘regenerator’ network. This results in a high boost in adversarial accuracy when WNR combined with the trained regenerator network is prepended to black box network. Limited training data: In the second part, we assume the availability of a few training samples, where access to trained models may or may not be provided. In the few-shot setup, existing works obtain robustness using sophisticated meta-learning techniques which rely on the generation of adversarial samples in every episode of training - thereby making it computationally expensive. We propose the first computationally cheaper non-meta learning approach for robust few-shot learning that does not require any adversarial sample. We perform pretraining using self-distillation to make the feature representation of low-frequency samples close to original samples of base classes. Similarly, we also improve the discriminability of low-frequency query set features that further boost the robustness. Our method obtains massive improvement in adversarial performance while being ≈5x faster compared to state-of-the-art adversarial meta-learning methods. However, empirical robustness methods do not guarantee robustness of the trained models against all the adversarial perturbations possible within a given threat model. Thus, we also propose a novel problem of certified robustness of pretrained models in limited data settings. Our method provides a novel sample-generation strategy that synthesize ‘boundary’ and ‘interpolated’ samples to augment the limited training data and uses them in training the denoiser (prepended to pretrained classifier) via aligning the feature representations at multiple granularities (both instance and distribution levels). We achieve significant improvements across diverse sample budgets and noise levels in the white-box and observe similar performance under challenging black-box setup. Selected subset of training data: In the third part, we enforce efficient utilization via intelligently doing selective sampling on existing training datasets to obtain representative samples for the target task such as distillation, incremental learning and person-reid. Adversarial attacks recently have shown robustness bias, where certain subgroups in a dataset (e.g. based on class, gender, etc.) are less robust than others. Existing works characterize a subgroup’s robustness bias by only checking individual sample’s proximity to the decision boundary. We propose a holistic approach for quantifying adversarial vulnerability of a sample by combining different perspectives and further develop a trustworthy system to alert the humans about the incoming samples that are highly likely to be misclassified. Moreover, we demonstrate the utility of the proposed metric for data (and time)-efficient knowledge distillation which achieves better performance compared to competing baselines. Other applications such as incremental learning and video based person-reid can also be framed as a subset selection problem where representative samples need to be selected. We leverage DPP (Determinantal Point Process) for choosing the relevant and diverse samples. In Incremental learning, we propose a new variant of k-DPP that uses the RBF kernel (termed as “RBF k-DPP”) for challenging task of animal pose estimation and further tackle class imbalance by using image warping as an augmentation technique to generate varied poses for a given image, leading to further gains in performance. In video based re-id, we propose SLGDPP method which exploits the sequential nature of the frames in video while avoiding noisy and redundant (correlated) frames, resulting in outperforming the baseline sampling methods.