Academic literature on the topic 'Dataset shift'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the lists of relevant articles, books, theses, conference reports, and other scholarly sources on the topic 'Dataset shift.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Journal articles on the topic "Dataset shift"
1
Sharet, Nir, and Ilan Shimshoni. "Analyzing Data Changes using Mean Shift Clustering." International Journal of Pattern Recognition and Artificial Intelligence 30, no. 07 (May 25, 2016): 1650016. http://dx.doi.org/10.1142/s0218001416500166.
Full textAbstract:
A nonparametric unsupervised method for analyzing changes in complex datasets is proposed. It is based on the mean shift clustering algorithm. Mean shift is used to cluster the old and new datasets and compare the results in a nonparametric manner. Each point from the new dataset naturally belongs to a cluster of points from its dataset. The method is also able to find to which cluster the point belongs in the old dataset and use this information to report qualitative differences between that dataset and the new one. Changes in local cluster distribution are also reported. The report can then be used to try to understand the underlying reasons which caused the changes in the distributions. On the basis of this method, a transductive transfer learning method for automatically labeling data from the new dataset is also proposed. This labeled data is used, in addition to the old training set, to train a classifier better suited to the new dataset. The algorithm has been implemented and tested on simulated and real (a stereo image pair) datasets. Its performance was also compared with several state-of-the-art methods.
2
Adams, Niall. "Dataset Shift in Machine Learning." Journal of the Royal Statistical Society: Series A (Statistics in Society) 173, no. 1 (January 2010): 274. http://dx.doi.org/10.1111/j.1467-985x.2009.00624_10.x.
Full text
3
Guo, Lin Lawrence, Stephen R. Pfohl, Jason Fries, Jose Posada, Scott Lanyon Fleming, Catherine Aftandilian, Nigam Shah, and Lillian Sung. "Systematic Review of Approaches to Preserve Machine Learning Performance in the Presence of Temporal Dataset Shift in Clinical Medicine." Applied Clinical Informatics 12, no. 04 (August 2021): 808–15. http://dx.doi.org/10.1055/s-0041-1735184.
Full textAbstract:
Abstract Objective The change in performance of machine learning models over time as a result of temporal dataset shift is a barrier to machine learning-derived models facilitating decision-making in clinical practice. Our aim was to describe technical procedures used to preserve the performance of machine learning models in the presence of temporal dataset shifts. Methods Studies were included if they were fully published articles that used machine learning and implemented a procedure to mitigate the effects of temporal dataset shift in a clinical setting. We described how dataset shift was measured, the procedures used to preserve model performance, and their effects. Results Of 4,457 potentially relevant publications identified, 15 were included. The impact of temporal dataset shift was primarily quantified using changes, usually deterioration, in calibration or discrimination. Calibration deterioration was more common (n = 11) than discrimination deterioration (n = 3). Mitigation strategies were categorized as model level or feature level. Model-level approaches (n = 15) were more common than feature-level approaches (n = 2), with the most common approaches being model refitting (n = 12), probability calibration (n = 7), model updating (n = 6), and model selection (n = 6). In general, all mitigation strategies were successful at preserving calibration but not uniformly successful in preserving discrimination. Conclusion There was limited research in preserving the performance of machine learning models in the presence of temporal dataset shift in clinical medicine. Future research could focus on the impact of dataset shift on clinical decision making, benchmark the mitigation strategies on a wider range of datasets and tasks, and identify optimal strategies for specific settings.
4
He, Zhiqiang. "ECG Heartbeat Classification Under Dataset Shift." Journal of Intelligent Medicine and Healthcare 1, no. 2 (2022): 79–89. http://dx.doi.org/10.32604/jimh.2022.036624.
Full text
5
Kim, Doyoung, Inwoong Lee, Dohyung Kim, and Sanghoon Lee. "Action Recognition Using Close-Up of Maximum Activation and ETRI-Activity3D LivingLab Dataset." Sensors 21, no. 20 (October 12, 2021): 6774. http://dx.doi.org/10.3390/s21206774.
Full textAbstract:
The development of action recognition models has shown great performance on various video datasets. Nevertheless, because there is no rich data on target actions in existing datasets, it is insufficient to perform action recognition applications required by industries. To satisfy this requirement, datasets composed of target actions with high availability have been created, but it is difficult to capture various characteristics in actual environments because video data are generated in a specific environment. In this paper, we introduce a new ETRI-Activity3D-LivingLab dataset, which provides action sequences in actual environments and helps to handle a network generalization issue due to the dataset shift. When the action recognition model is trained on the ETRI-Activity3D and KIST SynADL datasets and evaluated on the ETRI-Activity3D-LivingLab dataset, the performance can be severely degraded because the datasets were captured in different environments domains. To reduce this dataset shift between training and testing datasets, we propose a close-up of maximum activation, which magnifies the most activated part of a video input in detail. In addition, we present various experimental results and analysis that show the dataset shift and demonstrate the effectiveness of the proposed method.
6
McGaughey, Georgia, W. Patrick Walters, and Brian Goldman. "Understanding covariate shift in model performance." F1000Research 5 (April 7, 2016): 597. http://dx.doi.org/10.12688/f1000research.8317.1.
Full textAbstract:
Three (3) different methods (logistic regression, covariate shift and k-NN) were applied to five (5) internal datasets and one (1) external, publically available dataset where covariate shift existed. In all cases, k-NN’s performance was inferior to either logistic regression or covariate shift. Surprisingly, there was no obvious advantage for using covariate shift to reweight the training data in the examined datasets.
7
McGaughey, Georgia, W. Patrick Walters, and Brian Goldman. "Understanding covariate shift in model performance." F1000Research 5 (June 17, 2016): 597. http://dx.doi.org/10.12688/f1000research.8317.2.
Full textAbstract:
Three (3) different methods (logistic regression, covariate shift and k-NN) were applied to five (5) internal datasets and one (1) external, publically available dataset where covariate shift existed. In all cases, k-NN’s performance was inferior to either logistic regression or covariate shift. Surprisingly, there was no obvious advantage for using covariate shift to reweight the training data in the examined datasets.
8
McGaughey, Georgia, W. Patrick Walters, and Brian Goldman. "Understanding covariate shift in model performance." F1000Research 5 (October 17, 2016): 597. http://dx.doi.org/10.12688/f1000research.8317.3.
Full textAbstract:
Three (3) different methods (logistic regression, covariate shift and k-NN) were applied to five (5) internal datasets and one (1) external, publically available dataset where covariate shift existed. In all cases, k-NN’s performance was inferior to either logistic regression or covariate shift. Surprisingly, there was no obvious advantage for using covariate shift to reweight the training data in the examined datasets.
9
Becker, Aneta, and Jarosław Becker. "Dataset shift assessment measures in monitoring predictive models." Procedia Computer Science 192 (2021): 3391–402. http://dx.doi.org/10.1016/j.procs.2021.09.112.
Full text
10
Finlayson, Samuel G., Adarsh Subbaswamy, Karandeep Singh, John Bowers, Annabel Kupke, Jonathan Zittrain, Isaac S. Kohane, and Suchi Saria. "The Clinician and Dataset Shift in Artificial Intelligence." New England Journal of Medicine 385, no. 3 (July 15, 2021): 283–86. http://dx.doi.org/10.1056/nejmc2104626.
Full textDissertations / Theses on the topic "Dataset shift"
1
Wang, Fulton. "Addressing two issues in machine learning : interpretability and dataset shift." Thesis, Massachusetts Institute of Technology, 2018. https://hdl.handle.net/1721.1/122870.
Full textAbstract:
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018Cataloged from PDF version of thesis.
Includes bibliographical references (pages 71-77).
In this thesis, I create solutions to two problems. In the first, I address the problem that many machine learning models are not interpretable, by creating a new form of classifier, called the Falling Rule List. This is a decision list classifier where the predicted probabilities are decreasing down the list. Experiments show that the gain in interpretability need not be accompanied by a large sacrifice in accuracy on real world datasets. I then briefly discuss possible extensions that allow one to directly optimize rank statistics over rule lists, and handle ordinal data. In the second, I address a shortcoming of a popular approach to handling covariate shift, in which the training distribution and that for which predictions need to be made have different covariate distributions. In particular, the existing importance weighting approach to handling covariate shift suffers from high variance if the two covariate distributions are very different. I develop a dimension reduction procedure that reduces this variance, at the expense of increased bias. Experiments show that this tradeoff can be worthwhile in some situations.
by Fulton Wang.
Ph. D.
Ph.D. Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science
2
Gogolashvili, Davit. "Global and local Kernel methods for dataset shift, scalable inference and optimization." Electronic Thesis or Diss., Sorbonne université, 2022. https://accesdistant.sorbonne-universite.fr/login?url=https://theses-intra.sorbonne-universite.fr/2022SORUS363v2.pdf.
Full textAbstract:
Dans de nombreux problèmes du monde réel, les données de formation et les données de test ont des distributions différentes. Cette situation est communément appelée " décalage de l'ensemble de données ". Les paramètres les plus courants pour le décalage des ensembles de données souvent considérés dans la littérature sont le décalage des covariables et le décalage des cibles. Dans cette thèse, nous étudions les modèles nonparamétriques appliqués au scénario de changement d'ensemble de données. Nous développons un nouveau cadre pour accélérer la régression par processus gaussien. En particulier, nous considérons des noyaux de localisation à chaque point de données pour réduire les contributions des autres points de données éloignés, et nous dérivons le modèle GPR découlant de l'application de cette opération de localisation. Grâce à une série d'expériences, nous démontrons la performance compétitive de l'approche proposée par rapport au GPR complet, à d'autres modèles localisés et aux processus gaussiens profonds. De manière cruciale, ces performances sont obtenues avec des accélérations considérables par rapport au GPR global standard en raison de l'effet de sparsification de la matrice de Gram induit par l'opération de localisation. Nous proposons une nouvelle méthode pour estimer le minimiseur et la valeur minimale d'une fonction de régression lisse et fortement convexe à partir d'observations contaminées par du bruit aléatoireIn many real world problems, the training data and test data have different distributions. The most common settings for dataset shift often considered in the literature are covariate shift and target shift. In this thesis, we investigate nonparametric models applied to the dataset shift scenario. We develop a novel framework to accelerate Gaussian process regression. In particular, we consider localization kernels at each data point to down-weigh the contributions from other data points that are far away, and we derive the GPR model stemming from the application of such localization operation. We propose a new method for estimating the minimizer and the minimum value of a smooth and strongly convex regression function from the observations contaminated by random noise
3
Spooner, Amy. "Developing a minimum dataset for nursing team leader handover in the intensive care unit: a prospective interventional study." Thesis, Griffith University, 2018. http://hdl.handle.net/10072/382227.
Full textAbstract:
Since the World Health Organization listed clinical handover as a top five priority area for patient safety, the evidence-base and resources generated to improve handover communication has increased. But literature specific to the intensive care unit (ICU) handover, particularly handover from shift-to-shift by the ICU nurse Team Leader (TL) remains limited.
The aims of this three-phase interventional study focused on understanding current TL handover practices and implementing a handover strategy to improve this practice. The aim of Phase 1 was to determine the content of ICU nursing TL handover. The aim of Phase 2, was to identify the key components for inclusion in a handover minimum dataset (MDS) and, the aim of Phase 3 was to implement and evaluate an electronic minimum dataset (eMDS) for nursing TL handover. A modified version of the Knowledge-to-Action framework guided each phase of this research. The study was conducted in a 21-bed ICU, at a tertiary referral hospital in Brisbane, Australia. Senior nurses working in TL roles were sampled for this study.
Phase 1 involved audiotaping TL handovers to identify the content discussed during handovers. Audio recordings were transcribed and content analysis was used to analyse the data. A quantitative approach was used to identify the frequency of a priori categories and subcategories. Phase 2 consisted of focus groups with TLs to determine the content to include in an MDS for handover. Descriptive statistics were used to analyse responses from focus groups. In Phase 3, TLs were given surveys to complete to determine the barriers and facilitators to eMDS use prior to implementation. Survey results were analysed using descriptive statistics and the frequency of recurring responses to dichotomous and open-ended questions were summarised. Three months post eMDS implementation, TLs’ use of the eMDS was assessed by auditing and evaluating nurses’ perceptions through the distribution of surveys to TLs. Descriptive statistics were used to summarise audit and survey data.
Phase 1 findings revealed that TL handovers contained variable content, and that aspects of handover did not meet the Australian National Standards (e.g. handovers were conducted at the desk rather than bedside).
In Phase 2, TLs identified the content to include in an MDS for handover. The content of the MDS was structured using the ISBAR (Identify-Situation-Background-Assessment-Recommendations) schema and included additional items specific to ICU nursing TL handover.
In Phase 3, the barriers and facilitators to eMDS use were identified prior to implementation. These focused on usability, content and efficiency of the eMDS, and informed implementation strategies adopted to implement the eMDS. Implementation strategies included education, champions, reminders and ad hoc audit and feedback. Three months post implementation, audit results revealed TLs had relocated handovers to the bedside, and TLs were using the eMDS. Some key content items were discussed frequently while others showed no improvement or were absent from handovers. Results also highlighted that additional documentation was required alongside the eMDS to conduct handovers. Surveys of TLs’ perceptions identified benefits and disadvantages to eMDS use. Benefits were: improved patient content and time saved updating the tool. Disadvantages were: irrelevant patient content included, with pertinent content missing from handovers, and difficulties navigating the tool. Shortcomings of the eMDS were a result of limitations within the clinical information system (CIS) to filter and draw relevant data required into the tool. Nurses suggested eMDS modifications were needed to increase usability.
This is the first study to examine nursing TL handover, and to implement and evaluate an evidence-based eMDS for nursing TL shift-to-shift handover in the ICU. While the eMDS requires further testing and modifications, it is the first evidence-based handover tool developed for the MetaVision CIS that can be utilised and adapted by other ICUs. Continual iterations of the eMDS should occur in collaboration with vendors, information technology teams, and in alignment with national guidelines, to increase patient safety. The use of simulation in education and training, is the next step to informing relevant changes to the eMDS and optimising ICU nursing TL handover practices. Organisations need to recognise the value of practice improvements by investing funds to successfully implement and sustain the use of evidence-based practices. Evidence-based practices that are embedded in healthcare settings will ensure patients receive quality care and will improve patient outcomes.Thesis (PhD Doctorate)
Doctor of Philosophy (PhD)
School of Nursing & Midwifery
Griffith Health
Full Text
4
Fonseca, Eduardo. "Training sound event classifiers using different types of supervision." Doctoral thesis, Universitat Pompeu Fabra, 2021. http://hdl.handle.net/10803/673067.
Full textAbstract:
The automatic recognition of sound events has gained attention in the past few years,
motivated by emerging applications in fields such as healthcare, smart homes, or urban
planning. When the work for this thesis started, research on sound event classification was
mainly focused on supervised learning using small datasets, often carefully annotated with
vocabularies limited to specific domains (e.g., urban or domestic). However, such small
datasets do not support training classifiers able to recognize hundreds of sound events
occurring in our everyday environment, such as kettle whistles, bird tweets, cars passing by,
or different types of alarms. At the same time, large amounts of environmental sound data
are hosted in websites such as Freesound or YouTube, which can be convenient for training
large-vocabulary classifiers, particularly using data-hungry deep learning approaches.
To advance the state-of-the-art in sound event classification, this thesis investigates several
strands of dataset creation as well as supervised and unsupervised learning to train
large-vocabulary sound event classifiers, using different types of supervision in novel and
alternative ways. Specifically, we focus on supervised learning using clean and noisy labels,
as well as self-supervised representation learning from unlabeled data.
The first part of this thesis focuses on the creation of FSD50K, a large-vocabulary dataset
with over 100h of audio manually labeled using 200 classes of sound events. We provide a
detailed description of the creation process and a comprehensive characterization of the
dataset. In addition, we explore architectural modifications to increase shift invariance in
CNNs, improving robustness to time/frequency shifts in input spectrograms. In the second
part, we focus on training sound event classifiers using noisy labels. First, we propose a
dataset that supports the investigation of real label noise. Then, we explore network-agnostic
approaches to mitigate the effect of label noise during training, including regularization
techniques, noise-robust loss functions, and strategies to reject noisy labeled examples.
Further, we develop a teacher-student framework to address the problem of missing labels in
sound event datasets. In the third part, we propose algorithms to learn audio representations
from unlabeled data. In particular, we develop self-supervised contrastive learning
frameworks, where representations are learned by comparing pairs of examples computed
via data augmentation and automatic sound separation methods.
Finally, we report on the organization of two DCASE Challenge Tasks on automatic audio
tagging with noisy labels. By providing data resources as well as state-of-the-art approaches
and audio representations, this thesis contributes to the advancement of open sound event
research, and to the transition from traditional supervised learning using clean labels to other
learning strategies less dependent on costly annotation efforts.El interés en el reconocimiento automático de eventos sonoros se ha incrementado en los últimos años, motivado por nuevas aplicaciones en campos como la asistencia médica, smart homes, o urbanismo. Al comienzo de esta tesis, la investigación en clasificación de eventos sonoros se centraba principalmente en aprendizaje supervisado usando datasets pequeños, a menudo anotados cuidadosamente con vocabularios limitados a dominios específicos (como el urbano o el doméstico). Sin embargo, tales datasets no permiten entrenar clasificadores capaces de reconocer los cientos de eventos sonoros que ocurren en nuestro entorno, como silbidos de kettle, sonidos de pájaros, coches pasando, o diferentes alarmas. Al mismo tiempo, websites como Freesound o YouTube albergan grandes cantidades de datos de sonido ambiental, que pueden ser útiles para entrenar clasificadores con un vocabulario más extenso, particularmente utilizando métodos de deep learning que requieren gran cantidad de datos. Para avanzar el estado del arte en la clasificación de eventos sonoros, esta tesis investiga varios aspectos de la creación de datasets, así como de aprendizaje supervisado y no supervisado para entrenar clasificadores de eventos sonoros con un vocabulario extenso, utilizando diferentes tipos de supervisión de manera novedosa y alternativa. En concreto, nos centramos en aprendizaje supervisado usando etiquetas sin ruido y con ruido, así como en aprendizaje de representaciones auto-supervisado a partir de datos no etiquetados. La primera parte de esta tesis se centra en la creación de FSD50K, un dataset con más de 100h de audio etiquetado manualmente usando 200 clases de eventos sonoros. Presentamos una descripción detallada del proceso de creación y una caracterización exhaustiva del dataset. Además, exploramos modificaciones arquitectónicas para aumentar la invariancia frente a desplazamientos en CNNs, mejorando la robustez frente a desplazamientos de tiempo/frecuencia en los espectrogramas de entrada. En la segunda parte, nos centramos en entrenar clasificadores de eventos sonoros usando etiquetas con ruido. Primero, proponemos un dataset que permite la investigación del ruido de etiquetas real. Después, exploramos métodos agnósticos a la arquitectura de red para mitigar el efecto del ruido en las etiquetas durante el entrenamiento, incluyendo técnicas de regularización, funciones de coste robustas al ruido, y estrategias para rechazar ejemplos etiquetados con ruido. Además, desarrollamos un método teacher-student para abordar el problema de las etiquetas ausentes en datasets de eventos sonoros. En la tercera parte, proponemos algoritmos para aprender representaciones de audio a partir de datos sin etiquetar. En particular, desarrollamos métodos de aprendizaje contrastivos auto-supervisados, donde las representaciones se aprenden comparando pares de ejemplos calculados a través de métodos de aumento de datos y separación automática de sonido. Finalmente, reportamos sobre la organización de dos DCASE Challenge Tasks para el tageado automático de audio a partir de etiquetas ruidosas. Mediante la propuesta de datasets, así como de métodos de vanguardia y representaciones de audio, esta tesis contribuye al avance de la investigación abierta sobre eventos sonoros y a la transición del aprendizaje supervisado tradicional utilizando etiquetas sin ruido a otras estrategias de aprendizaje menos dependientes de costosos esfuerzos de anotación.
5
Sarr, Jean Michel Amath. "Étude de l’augmentation de données pour la robustesse des réseaux de neurones profonds." Electronic Thesis or Diss., Sorbonne université, 2023. http://www.theses.fr/2023SORUS072.
Full textAbstract:
Dans cette thèse, nous avons considéré le problème de robustesse des réseaux de neurones. C’est-à-dire que nous avons considéré le cas où le jeu d’apprentissage et le jeu de déploiement ne sont pas indépendamment et identiquement distribués suivant la même source. On appelle cette hypothèse : l’hypothèse i.i.d. Notre principal outil de travail a été l’augmentation de données. En effet, une revue approfondie de la littérature et des expériences préliminaires nous ont montré le potentiel de régularisation de l’augmentation des données. Ainsi, dans un premier temps, nous avons cherché à utiliser l’augmentation de données pour rendre les réseaux de neurones plus robustes à divers glissements de données synthétiques et naturels. Un glissement de données étant simplement une violation de l’hypothèse i.i.d. Cependant, les résultats de cette approche se sont révélés mitigés. En effet, nous avons observé que dans certains cas l’augmentation de données pouvait donner lieu à des bonds de performance sur le jeu de déploiement. Mais ce phénomène ne se produisait pas à chaque fois. Dans certains cas, augmenter les données pouvait même réduire les performances sur le jeu de déploiement. Nous proposons une explication granulaire à ce phénomène dans nos conclusions. Une meilleure utilisation de l’augmentation des données pour la robustesse des réseaux de neurones consiste à générer des tests de résistance ou "stress test" pour observer le comportement d’un modèle lorsque divers glissements de données surviennent. Ensuite, ces informations sur le comportement du modèle sont utilisées pour estimer l’erreur sur l’ensemble de déploiement même sans étiquettes, nous appelons cela l’estimation de l’erreur de déploiement. Par ailleurs, nous montrons que l’utilisation d’augmentation de données indépendantes peut améliorer l’estimation de l’erreur de déploiement. Nous croyons que cet usage de l’augmentation de données permettra de mieux cerner quantitativement la fiabilité des réseaux de neurones lorsqu’ils seront déployés sur de nouveaux jeux de données inconnusIn this thesis, we considered the problem of the robustness of neural networks. That is, we have considered the case where the learning set and the deployment set are not independently and identically distributed from the same source. This hypothesis is called : the i.i.d hypothesis. Our main research axis has been data augmentation. Indeed, an extensive literature review and preliminary experiments showed us the regularization potential of data augmentation. Thus, as a first step, we sought to use data augmentation to make neural networks more robust to various synthetic and natural dataset shifts. A dataset shift being simply a violation of the i.i.d assumption. However, the results of this approach have been mixed. Indeed, we observed that in some cases the augmented data could lead to performance jumps on the deployment set. But this phenomenon did not occur every time. In some cases, the augmented data could even reduce performance on the deployment set. In our conclusion, we offer a granular explanation for this phenomenon. Better use of data augmentation toward neural network robustness is to generate stress tests to observe a model behavior when various shift occurs. Then, to use that information to estimate the error on the deployment set of interest even without labels, we call this deployment error estimation. Furthermore, we show that the use of independent data augmentation can improve deployment error estimation. We believe that this use of data augmentation will allow us to better quantify the reliability of neural networks when deployed on new unknown datasets
6
Vanck, Thomas [Verfasser], Jochen [Akademischer Betreuer] Garcke, Jochen [Gutachter] Garcke, and Reinhold [Gutachter] Schneider. "New importance sampling based algorithms for compensating dataset shifts / Thomas Vanck ; Gutachter: Jochen Garcke, Reinhold Schneider ; Betreuer: Jochen Garcke." Berlin : Technische Universität Berlin, 2016. http://d-nb.info/1156012562/34.
Full text
7
Luus, Francois Pierre Sarel. "Dataset shift in land-use classification for optical remote sensing." Thesis, 2016. http://hdl.handle.net/2263/56246.
Full textAbstract:
Multimodal dataset shifts consisting of both concept and covariate shifts are addressed in this study to improve texture-based land-use classification accuracy for optical panchromatic and multispectral remote sensing. Multitemporal and multisensor variances between train and test data are caused by atmospheric, phenological, sensor, illumination and viewing geometry differences, which cause supervised classification inaccuracies. The first dataset shift reduction strategy involves input modification through shadow removal before feature extraction with gray-level co-occurrence matrix and local binary pattern features.
Components of a Rayleigh quotient-based manifold alignment framework is investigated to reduce multimodal dataset shift at the input level of the classifier through unsupervised classification, followed by manifold matching to transfer classification labels by finding across-domain cluster correspondences. The ability of weighted hierarchical agglomerative clustering to partition poorly separated feature spaces is explored and weight-generalized internal validation is used for unsupervised cardinality determination. Manifold matching solves the Hungarian algorithm with a cost matrix featuring geometric similarity measurements that assume the preservation of intrinsic structure across the dataset shift. Local neighborhood geometric co-occurrence frequency information is recovered and a novel integration thereof is shown to improve matching accuracy.
A final strategy for addressing multimodal dataset shift is multiscale feature learning, which is used within a convolutional neural network to obtain optimal hierarchical feature representations instead of engineered texture features that may be sub-optimal. Feature learning is shown to produce features that are robust against multimodal acquisition differences in a benchmark land-use classification dataset. A novel multiscale input strategy is proposed for an optimized convolutional neural network that improves classification accuracy to a competitive level for the UC Merced benchmark dataset and outperforms single-scale input methods. All the proposed strategies for addressing multimodal dataset shift in land-use image classification have resulted in significant accuracy improvements for various multitemporal and multimodal datasets.Thesis (PhD)--University of Pretoria, 2016.
National Research Foundation (NRF)
University of Pretoria (UP)
Electrical, Electronic and Computer Engineering
PhD
Unrestricted
Books on the topic "Dataset shift"
1
Quiñonero-Candela, Joaquin. Dataset shift in machine learning. Cambridge, MA: MIT Press, 2009.
Find full text
2
Quiñonero-Candela, Joaquin, Masashi Sugiyama, Anton Schwaighofer, and Neil D. Lawrence, eds. Dataset Shift in Machine Learning. The MIT Press, 2008. http://dx.doi.org/10.7551/mitpress/9780262170055.001.0001.
Full text
3
Schwaighofer, Anton, Joaquin Quiñonero-Candela, Masashi Sugiyama, and Neil D. Lawrence. Dataset Shift in Machine Learning. MIT Press, 2018.
Find full text
4
Schwaighofer, Anton, Masashi Sugiyama, Neil D. Lawrence, and Joaquin Quinonero-Candela. Dataset Shift in Machine Learning. MIT Press, 2022.
Find full text
5
Ogorzalek, Thomas K. The Cities on the Hill. Oxford University Press, 2018. http://dx.doi.org/10.1093/oso/9780190668877.003.0006.
Full textAbstract:
This largely quantitative chapter zooms out to describe urbanicity in Congress and explore the book’s original dataset of congressional place character to show the downstream effects of the developments in the previous chapters. Several original analyses chronicle the birth of a distinct, national, urban political order and a shift from a “bimodal” Democratic coalition of urban and rural representatives to one in which the relationship between urbanicity and partisanship is monotonic: the more urban a constituency, the more likely it is to be represented by a Democrat. This shift has important implications for urban policymaking: when Democrats are in the majority, big-city representatives are more likely to occupy leadership positions in key policymaking positions. When Republicans hold the majority, however, city representatives are virtually excluded from important chamber positions. While the Long New Deal was a heyday for the city’s place in the national imagination, the urban political order is potentially more powerful, though also more fragile, today.
6
Loyle, Cyanne E. Transitional Justice During Armed Conflict. Oxford University Press, 2017. http://dx.doi.org/10.1093/acrefore/9780190228637.013.218.
Full textAbstract:
Armed conflict is ultimately about the violent confrontation between two or more groups; however, there is a range of behaviors, both violent and nonviolent, pursued by governments and rebel groups while conflict is ongoing that impacts the course and outcomes of that violence. The use of judicial or quasi-judicial institutions during armed conflict is one such behavior. While there is a well-developed body of literature that examines the conditions under which governments engage with the legacies of violence following armed conflict, we know comparatively little about these same institutions used while conflict is ongoing.Similar to the use of transitional justice following armed conflict or post-conflict justice, during-conflict transitional justice (DCJ) refers to “a judicial or quasi-judicial process initiated during an armed conflict that attempts to address wrongdoings that have taken or are taking place as part of that conflict” (according to Loyle and Binningsbø). DCJ includes a variety of institutional forms pursued by both governments and rebel groups such as human rights trials, truth commissions or commissions of inquiry, amnesty offers, reparations, purges, or exiles.As our current understanding of transitional justice has focused exclusively on these processes following a political transition or the termination of an armed conflict, we have a limited understanding of how and why these processes are used during conflict. Extant work has assumed, either implicitly or explicitly, that transitional justice is offered and put in place once violence has ended, but this is not the case. New data on this topic from the During-Conflict Justice dataset by Loyle and Binningsbø suggests that the use of transitional justice during conflict is a widespread and systematic policy across multiple actor groups. In 2017, Loyle and Binningsbø found that DCJ processes were used during over 60% of armed conflicts from 1946 through 2011; and of these processes 10% were put in place by rebel groups (i.e., the group challenging the government rather than the government in power).Three main questions arise from this new finding: Under what conditions are justice processes implemented during conflict, why are these processes put in place, and what is the likely effect of their implementation on the conflict itself? Answering these questions has important implications for understanding patterns of government and rebel behavior while conflict is ongoing and the impacts of those behaviors. Furthermore, this work helps us to broaden our understanding of the use of judicial and quasi-judicial processes to those periods where no power shift has taken place.
7
Poplack, Shana. Borrowing. Oxford University Press, 2017. http://dx.doi.org/10.1093/oso/9780190256388.001.0001.
Full textAbstract:
In virtually every bilingual situation empirically studied, borrowed items make up the overwhelming majority of other-language material, but short shrift has been given to this major manifestation of language contact. As a result, scholars have long been divided over whether borrowing is a process distinct from code-switching, leading to long-standing controversy over how best to theorize language mixing strategies. This volume focuses on lexical borrowing as it actually occurs in the discourse of bilingual speakers, building on more than three decades of original research. Based on vast quantities of spontaneous performance data and a highly ramified analytical apparatus, it characterizes the phenomenon in the speech community and in the grammar, both synchronically and diachronically. In contrast to most other treatments, which deal with the product of borrowing, this work examines the process: How speakers incorporate foreign items into their bilingual discourse, how they adapt them to recipient-language grammatical structure, how these forms diffuse across speakers and communities, how long they persist in real time, and whether they change over the duration. It proposes falsifiable hypotheses about established loanwords and nonce borrowings and tests them empirically on a wealth of unique datasets on a wide variety of typologically similar and distinct language pairs. A major focus is the detailed analysis of integration, the principal mechanism underlying the borrowing process. Though the shape the borrowed form assumes may be colored by community convention, we show that the act of transforming donor-language elements into native material is universal.
Book chapters on the topic "Dataset shift"
1
da Silva, Camilla, Jed Nisenson, and Jeff Boisvert. "Comparing and Detecting Stationarity and Dataset Shift." In Springer Proceedings in Earth and Environmental Sciences, 37–42. Cham: Springer International Publishing, 2023. http://dx.doi.org/10.1007/978-3-031-19845-8_3.
Full textAbstract:
AbstractMachine learning algorithms have been increasingly applied to spatial numerical modeling. However, it is important to understand when such methods will underperform. Machine learning algorithms are impacted by dataset shift; when modeling domains of interest present non-stationarities there is no guarantee that the trained models are effective in unsampled areas. This work aims to compare the stationarity requirement of geostatistical methods to the concept of dataset shift. Also, workflow is developed to detect dataset shift in spatial data prior to modeling, this involves applying a discriminative classifier and a two sample Kolmogorv-Smirnov test to model areas. And, when required a lazy learning modification of support vector regression is proposed to account for dataset shift. The benefits of the lazy learning algorithm are demonstrated on the well-known non-stationary Walker Lake dataset and improves root mean squared error up to 25% relative to standard SVR approach, in areas where dataset shift is present.
2
Qian, Hongyi, Baohui Wang, Ping Ma, Lei Peng, Songfeng Gao, and You Song. "Managing Dataset Shift by Adversarial Validation for Credit Scoring." In Lecture Notes in Computer Science, 477–88. Cham: Springer Nature Switzerland, 2022. http://dx.doi.org/10.1007/978-3-031-20862-1_35.
Full text
3
Esuli, Andrea, Alessandro Fabris, Alejandro Moreo, and Fabrizio Sebastiani. "The Case for Quantification." In The Information Retrieval Series, 1–17. Cham: Springer International Publishing, 2023. http://dx.doi.org/10.1007/978-3-031-20467-8_1.
Full textAbstract:
AbstractThis chapter sets the stage for the rest of the book by introducing notions fundamental to quantification, such as class proportions, class distributions and their estimation, dataset shift, and the various subtypes of dataset shift which are relevant to the quantification endeavour. In this chapter we also argue why using classification techniques for estimating class distributions is suboptimal, and we then discuss why learning to quantify has evolved as a task of its own, rather than remaining a by-product of classification.
4
Leyendecker, Lars, Shobhit Agarwal, Thorben Werner, Maximilian Motz, and Robert H. Schmitt. "A Study on Data Augmentation Techniques for Visual Defect Detection in Manufacturing." In Bildverarbeitung in der Automation, 73–94. Berlin, Heidelberg: Springer Berlin Heidelberg, 2023. http://dx.doi.org/10.1007/978-3-662-66769-9_6.
Full textAbstract:
AbstractDeep learning-based defect detection is rapidly gaining importance for automating visual quality control tasks in industrial applications. However, due to usually low rejection rates in manufacturing processes, industrial defect detection datasets are inherent to three severe data challenges: data sparsity, data imbalance, and data shift. Because the acquisition of defect data is highly cost″=intensive, and Deep Learning (DL) algorithms require a sufficiently large amount of data, we are investigating how to solve these challenges using data oversampling and data augmentation (DA) techniques. Given the problem of binary defect detection, we present a novel experimental procedure for analyzing the impact of different DA-techniques. Accordingly, pre-selected DA-techniques are used to generate experiments across multiple datasets and DL models. For each defect detection use-case, we configure a set of random DA-pipelines to generate datasets of different characteristics. To investigate the impact of DA-techniques on defect detection performance, we then train convolutional neural networks with two different but fixed architectures and hyperparameter sets. To quantify and evaluate the generalizability, we compute the distances between dataset derivatives to determine the degree of domain shift. The results show that we can precisely analyze the influences of individual DA-methods, thus laying the foundation for establishing a mapping between dataset properties and DA-induced performance enhancement aiming for enhancing DL development. We show that there is no one-fits all solution, but that within the categories of geometrical and color augmentations, certain DA-methods outperform others.
5
Xia, Tong, Jing Han, and Cecilia Mascolo. "Benchmarking Uncertainty Quantification on Biosignal Classification Tasks Under Dataset Shift." In Multimodal AI in Healthcare, 347–59. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-031-14771-5_25.
Full text
6
Raza, Haider, Girijesh Prasad, and Yuhua Li. "EWMA Based Two-Stage Dataset Shift-Detection in Non-stationary Environments." In IFIP Advances in Information and Communication Technology, 625–35. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013. http://dx.doi.org/10.1007/978-3-642-41142-7_63.
Full text
7
Jin, Qiao, Haoyang Ding, Linfeng Li, Haitao Huang, Lei Wang, and Jun Yan. "Tackling MeSH Indexing Dataset Shift with Time-Aware Concept Embedding Learning." In Database Systems for Advanced Applications, 474–88. Cham: Springer International Publishing, 2020. http://dx.doi.org/10.1007/978-3-030-59419-0_29.
Full text
8
Zhu, Calvin, Michael D. Noseworthy, and Thomas E. Doyle. "Addressing Dataset Shift for Trustworthy Deep Learning Diagnostic Ultrasound Decision Support." In Lecture Notes in Computer Science, 110–28. Berlin, Heidelberg: Springer Berlin Heidelberg, 2023. http://dx.doi.org/10.1007/978-3-662-67868-8_7.
Full text
9
Zhang, Jiaxin, Tomohiro Fukuda, and Nobuyoshi Yabuki. "A Large-Scale Measurement and Quantitative Analysis Method of Façade Color in the Urban Street Using Deep Learning." In Proceedings of the 2020 DigitalFUTURES, 93–102. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-33-4400-6_9.
Full textAbstract:
AbstractColor planning has become a significant issue in urban development, and an overall cognition of the urban color identities will help to design a better urban environment. However, the previous measurement and analysis methods for the facade color in the urban street are limited to manual collection, which is challenging to carry out on a city scale. Recent emerging dataset street view image and deep learning have revealed the possibility to overcome the previous limits, thus bringing forward a research paradigm shift. In the experimental part, we disassemble the goal into three steps: firstly, capturing the street view images with coordinate information through the API provided by the street view service; then extracting facade images and cleaning up invalid data by using the deep-learning segmentation method; finally, calculating the dominant color based on the data on the Munsell Color System. Results can show whether the color status satisfies the requirements of its urban plan for façade color in the street. This method can help to realize the refined measurement of façade color using open source data, and has good universality in practice.
10
Rizvi, Syed Zeeshan, Muhammad Umar Farooq, and Rana Hammad Raza. "Performance Comparison of Deep Residual Networks-Based Super Resolution Algorithms Using Thermal Images: Case Study of Crowd Counting." In Digital Interaction and Machine Intelligence, 75–87. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-031-11432-8_7.
Full textAbstract:
AbstractHumans are able to perceive objects only in the visible spectrum range which limits the perception abilities in poor weather or low illumination conditions. The limitations are usually handled through technological advancements in thermographic imaging. However, thermal cameras have poor spatial resolutions compared to RGB cameras. Super-resolution (SR) techniques are commonly used to improve the overall quality of low-resolution images. There has been a major shift of research among the Computer Vision researchers towards SR techniques particularly aimed for thermal images. This paper analyzes the performance of three deep learning-based state-of-the-art SR algorithms namely Enhanced Deep Super Resolution (EDSR), Residual Channel Attention Network (RCAN) and Residual Dense Network (RDN) on thermal images. The algorithms were trained from scratch for different upscaling factors of ×2 and ×4. The dataset was generated from two different thermal imaging sequences of BU-TIV benchmark. The sequences contain both sparse and highly dense type of crowds with a far field camera view. The trained models were then used to super-resolve unseen test images. The quantitative analysis of the test images was performed using common image quality metrics such as PSNR, SSIM and LPIPS, while qualitative analysis was provided by evaluating effectiveness of the algorithms for crowd counting application. After only 54 and 51 epochs of RCAN and RDN respectively, both approaches were able to output average scores of 37.878, 0.986, 0.0098 and 30.175, 0.945, 0.0636 for PSNR, SSIM and LPIPS respectively. The EDSR algorithm took the least computation time during both training and testing because of its simple architecture. This research proves that a reasonable accuracy can be achieved with fewer training epochs when an application-specific dataset is carefully selected.
Conference papers on the topic "Dataset shift"
1
Maggio, Simona, Victor Bouvier, and Leo Dreyfus-Schmidt. "Performance Prediction Under Dataset Shift." In 2022 26th International Conference on Pattern Recognition (ICPR). IEEE, 2022. http://dx.doi.org/10.1109/icpr56361.2022.9956676.
Full text
2
Tuia, Devis, Edoardo Pasolli, and William J. Emery. "Dataset shift adaptation with active queries." In 2011 Joint Urban Remote Sensing Event (JURSE). IEEE, 2011. http://dx.doi.org/10.1109/jurse.2011.5764734.
Full text
3
Spence, David, Christopher Inskip, Novi Quadrianto, and David Weir. "Quantification under class-conditional dataset shift." In ASONAM '19: International Conference on Advances in Social Networks Analysis and Mining. New York, NY, USA: ACM, 2019. http://dx.doi.org/10.1145/3341161.3342948.
Full text
4
Takahashi, Carla C., Luiz C. B. Torres, and Antonio P. Braga. "Gabriel Graph Transductive Approach to Dataset Shift." In 2019 6th International Conference on Control, Decision and Information Technologies (CoDIT). IEEE, 2019. http://dx.doi.org/10.1109/codit.2019.8820327.
Full text
5
Brugman, Simon, Tomas Sostak, Pradyot Patil, and Max Baak. "popmon: Analysis Package for Dataset Shift Detection." In Python in Science Conference. SciPy, 2022. http://dx.doi.org/10.25080/majora-212e5952-01d.
Full text
6
Lucas, Yvan, Pierre-Edouard Portier, Lea Laporte, Sylvie Calabretto, Liyun He-Guelton, Frederic Oble, and Michael Granitzer. "Dataset Shift Quantification for Credit Card Fraud Detection." In 2019 IEEE Second International Conference on Artificial Intelligence and Knowledge Engineering (AIKE). IEEE, 2019. http://dx.doi.org/10.1109/aike.2019.00024.
Full text
7
Wang, Ziming, Changwu Huang, and Xin Yao. "Feature Attribution Explanation to Detect Harmful Dataset Shift." In 2023 International Joint Conference on Neural Networks (IJCNN). IEEE, 2023. http://dx.doi.org/10.1109/ijcnn54540.2023.10191221.
Full text
8
Chen, Bo, Wai Lam, Ivor Tsang, and Tak-Lam Wong. "Location and Scatter Matching for Dataset Shift in Text Mining." In 2010 IEEE 10th International Conference on Data Mining (ICDM). IEEE, 2010. http://dx.doi.org/10.1109/icdm.2010.72.
Full text
9
Raza, Haider, Girijesh Prasad, and Yuhua Li. "Dataset Shift Detection in Non-stationary Environments Using EWMA Charts." In 2013 IEEE International Conference on Systems, Man and Cybernetics (SMC 2013). IEEE, 2013. http://dx.doi.org/10.1109/smc.2013.537.
Full text
10
Denham, Benjamin, Edmund M.-K. Lai, Roopak Sinha, and M. Asif Naeem. "Gain-Some-Lose-Some: Reliable Quantification Under General Dataset Shift." In 2021 IEEE International Conference on Data Mining (ICDM). IEEE, 2021. http://dx.doi.org/10.1109/icdm51629.2021.00121.
Full textReports on the topic "Dataset shift"
1
Mascagni, Giulia, and Fabrizio Santoro. The Tax Side of the Pandemic: Compliance Shifts and Funding for Recovery in Rwanda. Institute of Development Studies, October 2021. http://dx.doi.org/10.19088/ictd.2021.019.
Full textAbstract:
While much knowledge is being generated on the impact of the pandemic, we still know very little on its implications on taxation in low-income countries. Yet, tax is crucial to fund crisis response and recovery, in addition to broader development plans and expanded government expenditure. This paper starts addressing this gap using a unique dataset of survey and administrative data from Rwanda. We document two significant shifts in taxpayers’ views: perceptions about the fairness of the tax system improve by 40 per cent, and their attitudes to compliance become more conditional on the provision of public services of sufficiently good quality. Importantly, these shifts are accompanied by improvements in actual compliance behaviour: using data from tax returns, we show that firms that declare after the onset of the crisis are substantially more compliant than others. We then investigate public support for increasing various tax options to fund crisis response and recovery. Taxing large companies and the richest enjoy the greatest support, which, however, declines as income increases. These results allow us to make some recommendations and considerations on tax policy responses to the crisis.
2
Clark, Andrew E., Angela Greulich, and Hippolyte d’Albis. The age U-shape in Europe: the protective role of partnership. Verlag der Österreichischen Akademie der Wissenschaften, March 2021. http://dx.doi.org/10.1553/populationyearbook2021.res3.1.
Full textAbstract:
In this study, we ask whether the U-shaped relationship between life satisfactionand age is flatter for individuals who are partnered. An analysis of cross-sectionalEU-SILC data indicates that the decline in life satisfaction from the teens to thefifties is almost four times larger for non-partnered than for partnered individuals,whose life satisfaction essentially follows a slight downward trajectory with age.However, the same analysis applied to three panel datasets (BHPS, SOEP andHILDA) reveals a U-shape for both groups, albeit somewhat flatter for the partneredthan for the non-partnered individuals. We suggest that the difference between thecross-sectional and the panel results reflects compositional effects: i.e., there isa significant shift of the relatively dissatisfied out of marriage in mid-life. Thesecompositional effects tend to flatten the U-shape in age for the partnered individualsin the cross-sectional data.
3
Tait, Emma, Pia Ruisi-Besares, Matthias Sirch, Alyx Belisle, Jennifer Pontius, and Elissa Schuett. Technical Report: Monitoring and Communicating Changes in Disturbance Regimes (Version 1.0). Forest Ecosystem Monitoring Cooperative, October 2021. http://dx.doi.org/10.18125/cc0a0l.
Full textAbstract:
Shifts in disturbance patterns across the Northeast are of increasing concern as the climate continues to change. In particular, changes in patterns of frequency, severity and extent of disturbance event may have detrimental cascading impacts on forest ecosystems and human communities. To explore how changing disturbance regimes might impact future forest health and management it is necessary to understand the historical trends and impacts of disturbance in the region. Although individual types of disturbance have already been analyzed, there is a need for a consolidated overview of the current state of disturbance in northeastern forests. To address this need, the Forest Ecosystem Monitoring Cooperative (FEMC) developed the FEMC: Tracking Shifts in Disturbance Regimes web portal for users to explore changes over time of key disturbance drivers, identify important disturbance responses, and discover where monitoring is happening for both drivers and responses. In collaboration with our advisory committee, we identified key disturbance drivers—flood, high winds, fire, drought, pests—and responses—macroinvertebrates, cold-water fisheries, invasive plants—that are of particular concern in the region. For each of the drivers we identified a suitable regional dataset and analyzed changes over time in frequency, severity, and extent. We also created a structured framework to catalogue programs across the region that are monitoring for these disturbance drivers and responses. Version 1.0 of the FEMC: Tracking Shifts in Disturbance Regimes (https://uvm.edu/femc/disturbance) web portal, first released in October 2021, contains 272 data programs, 11 drivers and three responses. Through the web portal users can browse programs by state, driver type or response type, and explore where monitoring is happening across the region. Driver-specific analyses allow users to quickly see the trends in severity, frequency and extent of selected disturbances and compare the impacts in selected states to regional data. We hope that this collection of programs and the analysis of trends provide researchers and land managers with an easy way to understand the current state of disturbance in northeastern forests that enables them to analyze and plan for future impacts.
4
Calcagno, Juan Carlos, and Mariana Alfonso. Minority Enrollments at Public Universities of Diverse Selectivity Levels under Different Admission Regimes: The Case of Texas. Inter-American Development Bank, October 2007. http://dx.doi.org/10.18235/0010878.
Full textAbstract:
This study describes how minority enrollment probabilities respond to changes in admission policies from affirmative-action to merit-only programs and then to percentage plans when the demographic composition of the potential pool of applicants is also shifting. It takes advantage of admission policy changes that occurred in the state of Texas with the Hopwood and HB588 decisions and of a unique administrative dataset that includes applications, admissions, and enrollments for three public universities of different selectivity levels. The findings suggest that the elimination of affirmative action and the introduction of the Top 10% plan had differential effects on minority enrollment probabilities as well as on application behavior depending on the selectivity level of the postsecondary institution. In particular, Hopwood is related to shifts in minority enrollments from selective institutions to less selective ones as the cascading hypothesis predicts. And although the Top 10% plan seems to have helped increased minority enrollment probabilities at the selective college as the upgrading hypothesis predicts, once the increases in minority shares among high-school graduates are taken into account, we find that the Top 10% plan can no longer be related to improvements in minority representation at selective universities.
5
Linker, Taylor, and Timothy Jacobs. PR-457-18204-R02 Variable Fuel Effects on Legacy Compressor Engines Phase V Engine Control Enhancement. Chantilly, Virginia: Pipeline Research Council International, Inc. (PRCI), July 2020. http://dx.doi.org/10.55274/r0011729.
Full textAbstract:
Variations in natural gas composition not only change bulk properties like heating value and adiabatic flame temperature, but also affect the reactivity of the gas during combustion in legacy compressor engines. Gas blends with high amounts of non-methane hydrocarbons are more reactive and alter combustion phasing in ways which can negatively affect engine operation and NOx emissions. These issues have and will continue to become more prevalent as natural gas production continually shifts towards shale resources. This work investigates the impacts of changing fuel composition on engine operation and emissions, as well as on fundamental fuel properties. Several fuel sweep datasets from different legacy engines are used to help draw broad conclusions about the impact of fuel speciation depending on engine type and operating condition. Further, the relationship between this engine behavior and fundamental fuel properties is explored. The response of engine operation and emissions to changing fuel reactivity is also observed in the context of the trapped equivalence ratio control method. A correction to the method which accounts for fuel reactivity effects on NOx is proposed and assessed with available data.
6
Baxter, W., Amanda Barker, Samuel Beal, Lauren Bosche, Ryan Busby, Zoe Courville, Elias Deeb, et al. A comprehensive approach to data collection, management, and visualization for terrain characterization in cold regions. Engineer Research and Development Center (U.S.), February 2024. http://dx.doi.org/10.21079/11681/48212.
Full textAbstract:
As global focus shifts to northern latitudes for their enhanced access to newly viable resources, US Army operational readiness in these extreme environments is increasingly important. Rapid and accurate intelligence on the conditions influencing operations in these regions is essential to mission success and warfighter safety. Arctic and boreal environments are highly heterogeneous, including changing extents of frozen versus thawing ground, snow, and ice that affect ground trafficability and visibility, terrain physics, and physicochemical properties of water and soil. Furthermore, projected climatic warming in these regions makes the timing of seasonal transitions increasingly uncertain. Broad coverage of long-term datasets is critical for assessing spatial and temporal variability in these northern environments at the landscape-scale. However, decadal measurements are difficult to acquire, manage, and visualize in the field setting. Here, we present a synopsis of data collection, management, and visualization for long-term permafrost, snow, vegetation, geophysics, and biogeochemical data from Alaska and review related literature. We also synthesize short-term data from various permafrost affected sites in the US and northern Europe to further assess the state of northern landscapes. Altogether, this work provides a comprehensive approach for high-latitude field site management to accurately inform mission-related operations in extreme northern environments.
7
Lu, Tianjun, Jian-yu Ke, Fynnwin Prager, and Jose N. Martinez. “TELE-commuting” During the COVID-19 Pandemic and Beyond: Unveiling State-wide Patterns and Trends of Telecommuting in Relation to Transportation, Employment, Land Use, and Emissions in Calif. Mineta Transportation Institute, August 2022. http://dx.doi.org/10.31979/mti.2022.2147.
Full textAbstract:
Telecommuting, the practice of working remotely at home, increased significantly (25% to 35%) early in the COVID-19 pandemic. This shift represented a major societal change that reshaped the family, work, and social lives of many Californians. These changes also raise important questions about what factors influenced telecommuting before, during, and after COVID-19, and to what extent changes in telecommuting have influenced transportation patterns across commute modes, employment, land use, and environment. The research team conducted state-level telecommuting surveys using a crowd-sourced platform (i.e., Amazon Mechanical Turk) to obtain valid samples across California (n=1,985) and conducted state-level interviews among stakeholders (n=28) across ten major industries in California. The study leveraged secondary datasets and developed regression and time-series models. Our surveys found that, compared to pre-pandemic levels, more people had a dedicated workspace at home and had received adequate training and support for telecommuting, became more flexible to choose their own schedules, and had improved their working performance—but felt isolated and found it difficult to separate home and work life. Our interviews suggested that telecommuting policies were not commonly designed and implemented until COVID-19. Additionally, regression analyses showed that telecommuting practices have been influenced by COVID-19 related policies, public risk perception, home prices, broadband rates, and government employment. This study reveals advantages and disadvantages of telecommuting and unveils the complex relationships among the COVID-19 outbreak, transportation systems, employment, land use, and emissions as well as public risk perception and economic factors. The study informs statewide and regional policies to adapt to the new patterns of telecommuting.
8
Moreda, Fekadu, Benjamin Lord, Mauro Nalesso, Pedro Coli Valdes Daussa, and Juliana Corrales. Hydro-BID: New Functionalities (Reservoir, Sediment and Groundwater Simulation Modules). Inter-American Development Bank, November 2016. http://dx.doi.org/10.18235/0009312.
Full textAbstract:
The Inter-American Development Bank (IDB) provides financial and technicalsupport for infrastructure projects in water and sanitation, irrigation, flood control, transport, and energy, and for development projects in agriculture, urban systems, and natural resources. Many of these projects depend upon water resources and may be affected negatively by climate change and other developments that alter water availability, such as population growth and shifts in land use associated with urbanization, industrial growth, and agricultural practices. Assessing the potential for future changes in water availability is an important step toward ensuring that infrastructure and other development projects meet their operational, financial, and economic goals. It is also important to examine the implications of such projects for the future allocation of available water among competing users and uses to mitigate potential conflict and to ensure such projects are consistent with long-term regional development plans and preservation of essential ecosystem services. As part of its commitment to help member countries adapt to climate change, the IDB is sponsoring work to develop and apply the Regional Water Resources Simulation Model for Latin America and the Caribbean, an integrated suite of watershed modeling tools known as Hydro-BID. Hydro-BID is a highly scalable modeling system that includes hydrology and climate analysis modules to estimate the availability of surface water (stream flows) at the regional, basin, and sub-basin scales. The system includes modules for incorporating the effects of groundwater and reservoirs on surface water flows and for estimating sediment loading. Data produced by Hydro-BID are useful for water balance analysis, water allocation decisions, and economic analysis and decision support tools to help decision-makers make informed choices among alternative designs for infrastructure projects and alternative policies for water resources management. IDB sponsored the development of Hydro-BID and provides the software and basic training free of charge to authorized users; see hydrobidlac.org. The system was developed by RTI International as an adaptation of RTI's proprietary WaterFALL® modeling software, based on over 30 years of experience developing and using the U.S. National Hydrography Dataset (NHDPlus) in support to the U.S. Geological Survey and the U.S. Environmental Protection Agency. In Phase I of this effort, RTI prepared a working version of Hydro-BID that includes: (1) the Analytical Hydrography Dataset for Latin America and the Caribbean (LAC AHD), a digital representation of 229,300 catchments in Central America, South America, and the Caribbean with their corresponding topography, river, and stream segments; (2) a geographic information system (GIS)-based navigation tool to browse AHD catchments and streams with the capability of navigating upstream and downstream; (3) a user interface for specifying the area and period to be modeled and the period and location for which water availability will be simulated; (4) a climate data interface to obtain rainfall and temperature inputs for the area and period of interest; (5) a rainfall-runoff model based on the Generalized Watershed Loading Factor (GWLF) formulation; and (6) a routing scheme for quantifying time of travel and cumulative flow estimates across downstream catchments. Hydro-BID generates output in the form of daily time series of flow estimates for the selected location and period. The output can be summarized as a monthly time series at the user's discretion. In Phase II of this effort, RTI has prepared an updated version of Hydro-BID that includes (1) improvements to the user interface; (2) a module to simulate the effect of reservoirs on downstream flows; (3) a module to link Hydro-BID and groundwater models developed with MODFLOW and incorporate water exchanges between groundwater and surface water compartments into the simulation of sur
9
Heifetz, Yael, and Michael Bender. Success and failure in insect fertilization and reproduction - the role of the female accessory glands. United States Department of Agriculture, December 2006. http://dx.doi.org/10.32747/2006.7695586.bard.
Full textAbstract:
The research problem. Understanding of insect reproduction has been critical to the design of insect pest control strategies including disruptions of mate-finding, courtship and sperm transfer by male insects. It is well known that males transfer proteins to females during mating that profoundly affect female reproductive physiology, but little is known about the molecular basis of female mating response and no attempts have yet been made to interfere with female post-mating responses that directly bear on the efficacy of fertilization. The female reproductive tract provides a crucial environment for the events of fertilization yet thus far those events and the role of the female tract in influencing them are poorly understood. For this project, we have chosen to focus on the lower reproductive tract because it is the site of two processes critical to reproduction: sperm management (storage, maintenance, and release from storage) and fertilization. E,fforts during this project period centered on the elucidation of mating responses in the female lower reproductive tract The central goals of this project were: 1. To identify mating-responsive genes in the female lower reproductive tract using DNA microarray technology. 2. In parallel, to identify mating-responsive genes in these tissues using proteomic assays (2D gels and LC-MS/MS techniques). 3. To integrate proteomic and genomic analyses of reproductive tract gene expression to identify significant genes for functional analysis. Our main achievements were: 1. Identification of mating-responsive genes in the female lower reproductive tract. We identified 539 mating-responsive genes using genomic and proteomic approaches. This analysis revealed a shift from gene silencing to gene activation soon after mating and a peak in differential gene expression at 6 hours post-mating. In addition, comparison of the two datasets revealed an expression pattern consistent with the model that important reproductive proteins are pre-programmed for synthesis prior to mating. This work was published in Mack et al. (2006). Validation experiments using real-time PCR techniques suggest that microarray assays provide a conservativestimate of the true transcriptional activity in reproductive tissues. 2.lntegration of proteomics and genomics data sets. We compared the expression profiles from DNA microarray data with the proteins identified in our proteomic experiments. Although comparing the two data sets poses analyical challenges, it provides a more complete view of gene expression as well as insights into how specific genes may be regulated. This work was published in Mack et al. (2006). 3. Development of primary reproductive tract cell cultures. We developed primary cell cultures of dispersed reproductive tract cell types and determined conditions for organ culture of the entire reproductive tract. This work will allow us to rapidly screen mating-responsive genes for a variety of reproductive-tract specifi c functions. Scientific and agricultural significance. Together, these studies have defined the genetic response to mating in a part of the female reproductive tract that is critical for successful fertllization and have identified alarge set of mating-responsive genes. This work is the first to combine both genomic and proteomic approaches in determining female mating response in these tissues and has provided important insights into insect reproductive behavior.
10
Allen, Kathy, Andy Nadeau, and Andy Robertston. Natural resource condition assessment: Salinas Pueblo Missions National Monument. National Park Service, May 2022. http://dx.doi.org/10.36967/nrr-2293613.
Full textAbstract:
The Natural Resource Condition Assessment (NRCA) Program aims to provide documentation about the current conditions of important park natural resources through a spatially explicit, multi-disciplinary synthesis of existing scientific data and knowledge. Findings from the NRCA will help Salinas Pueblo Missions National Monument (SAPU) managers to develop near-term management priorities, engage in watershed or landscape scale partnership and education efforts, conduct park planning, and report program performance (e.g., Department of the Interior’s Strategic Plan “land health” goals, Government Performance and Results Act). The objectives of this assessment are to evaluate and report on current conditions of key park resources, to evaluate critical data and knowledge gaps, and to highlight selected existing stressors and emerging threats to resources or processes. For the purpose of this NRCA, staff from the National Park Service (NPS) and Saint Mary’s University of Minnesota – GeoSpatial Services (SMUMN GSS) identified key resources, referred to as “components” in the project. The selected components include natural resources and processes that are currently of the greatest concern to park management at SAPU. The final project framework contains nine resource components, each featuring discussions of measures, stressors, and reference conditions. This study involved reviewing existing literature and, where appropriate, analyzing data for each natural resource component in the framework to provide summaries of current condition and trends in selected resources. When possible, existing data for the established measures of each component were analyzed and compared to designated reference conditions. A weighted scoring system was applied to calculate the current condition of each component. Weighted Condition Scores, ranging from zero to one, were divided into three categories of condition: low concern, moderate concern, and significant concern. These scores help to determine the current overall condition of each resource. The discussions for each component, found in Chapter 4 of this report, represent a comprehensive summary of current available data and information for these resources, including unpublished park information and perspectives of park resource managers, and present a current condition designation when appropriate. Each component assessment was reviewed by SAPU resource managers, NPS Southern Colorado Plateau Network (SCPN) staff, or outside experts. Existing literature, short- and long-term datasets, and input from NPS and other outside agency scientists support condition designations for components in this assessment. However, in some cases, data were unavailable or insufficient for several of the measures of the featured components. In other instances, data establishing reference condition were limited or unavailable for components, making comparisons with current information inappropriate or invalid. In these cases, it was not possible to assign condition for the components. Current condition was not able to be determined for six of the ten components due to these data gaps. For those components with sufficient available data, the overall condition varied. Two components were determined to be in good condition: dark night skies and paleontological resources. However, both were at the edge of the good condition range, and any small decline in conditions could shift them into the moderate concern range. Of the components in good condition, a trend could not be assigned for paleontological resources and dark night skies is considered stable. Two components (wetland and riparian communities and viewshed) were of moderate concern, with no trend assigned for wetland and riparian communities and a stable trend for viewshed. Detailed discussion of these designations is presented in Chapters 4 and 5 of this report. Several park-wide threats and stressors influence the condition of priority resources in SAPU...