Relevant bibliographies by topics / Data Analysis Workflow

Academic literature on the topic 'Data Analysis Workflow'

Author: Grafiati
Published: 10 December 2022
Last updated: 28 January 2023

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles

Select a source type:

Contents

Consult the lists of relevant articles, books, theses, conference reports, and other scholarly sources on the topic 'Data Analysis Workflow.'

Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.

You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.

Journal articles on the topic "Data Analysis Workflow"

1

Pfaff, Claas-Thido, Karin Nadrowski, Sophia Ratcliffe, Christian Wirth, and Helge Bruelheide. "Readable workflows need simple data." F1000Research 3 (May 14, 2014): 110. http://dx.doi.org/10.12688/f1000research.3940.1.

Full text
Abstract:
Sharing scientific analyses via workflows has great potential to improve the reproducibility of science as well as communicating research results. This is particularly useful for trans-disciplinary research fields such as biodiversity - ecosystem functioning (BEF), where syntheses need to merge data ranging from genes to the biosphere. Here we argue that enabling simplicity in the very beginning of workflows, at the point of data description and merging, offers huge potentials in reducing workflow complexity and in fostering data and workflow reuse. We illustrate our points using a typical analysis in BEF research, the aggregation of carbon pools in a forest ecosystem. We introduce indicators for the complexity of workflow components including data sources. We show that workflow complexity decreases exponentially during the course of the analysis and that simple text-based measures help to identify bottlenecks in a workflow and group workflow components according to tasks. We thus suggest that focusing on simplifying steps of data aggregation and imputation will greatly improve workflow readability and thus reproducibility. Providing feedback to data providers about the complexity of their datasets may help to produce better focused data that can be used more easily in further studies. At the same time, providing feedback about the complexity of workflow components may help to exchange shorter and simpler workflows for easier reuse. Additionally, identifying repetitive tasks informs software development in providing automated solutions. We discuss current initiatives in software and script development that implement quality control for simplicity and social tools of script valuation. Taken together we argue that focusing on simplifying data sources and workflow components will improve and accelerate data and workflow reuse and simplify the reproducibility of data-driven science.
APA, Harvard, Vancouver, ISO, and other styles
2

Song, Tianhong, Sven Köhler, Bertram Ludäscher, James Hanken, Maureen Kelly, David Lowery, James A. Macklin, Paul J. Morris, and Robert A. Morris. "Towards Automated Design, Analysis and Optimization of Declarative Curation Workflows." International Journal of Digital Curation 9, no. 2 (October 29, 2014): 111–22. http://dx.doi.org/10.2218/ijdc.v9i2.337.

Full text
Abstract:
Data curation is increasingly important. Our previous work on a Kepler curation package has demonstrated advantages that come from automating data curation pipelines by using workflow systems. However, manually designed curation workflows can be error-prone and inefficient due to a lack of user understanding of the workflow system, misuse of actors, or human error. Correcting problematic workflows is often very time-consuming. A more proactive workflow system can help users avoid such pitfalls. For example, static analysis before execution can be used to detect the potential problems in a workflow and help the user to improve workflow design. In this paper, we propose a declarative workflow approach that supports semi-automated workflow design, analysis and optimization. We show how the workflow design engine helps users to construct data curation workflows, how the workflow analysis engine detects different design problems of workflows and how workflows can be optimized by exploiting parallelism.
APA, Harvard, Vancouver, ISO, and other styles
3

Hribar, Michelle R., Sarah Read-Brown, Isaac H. Goldstein, Leah G. Reznick, Lorinna Lombardi, Mansi Parikh, Winston Chamberlain, and Michael F. Chiang. "Secondary use of electronic health record data for clinical workflow analysis." Journal of the American Medical Informatics Association 25, no. 1 (September 26, 2017): 40–46. http://dx.doi.org/10.1093/jamia/ocx098.

Full text
Abstract:
Abstract Objective Outpatient clinics lack guidance for tackling modern efficiency and productivity demands. Workflow studies require large amounts of timing data that are prohibitively expensive to collect through observation or tracking devices. Electronic health records (EHRs) contain a vast amount of timing data – timestamps collected during regular use – that can be mapped to workflow steps. This study validates using EHR timestamp data to predict outpatient ophthalmology clinic workflow timings at Oregon Health and Science University and demonstrates their usefulness in 3 different studies. Materials and Methods Four outpatient ophthalmology clinics were observed to determine their workflows and to time each workflow step. EHR timestamps were mapped to the workflow steps and validated against the observed timings. Results The EHR timestamp analysis produced times that were within 3 min of the observed times for >80% of the appointments. EHR use patterns affected the accuracy of using EHR timestamps to predict workflow times. Discussion EHR timestamps provided a reasonable approximation of workflow and can be used for workflow studies. They can be used to create simulation models, analyze EHR use, and quantify the impact of trainees on workflow. Conclusion The secondary use of EHR timestamp data is a valuable resource for clinical workflow studies. Sample timestamp data files and algorithms for processing them are provided and can be used as a template for more studies in other clinical specialties and settings.
APA, Harvard, Vancouver, ISO, and other styles
4

Suetake, Hirotaka, Tomoya Tanjo, Manabu Ishii, Bruno P. Kinoshita, Takeshi Fujino, Tsuyoshi Hachiya, Yuichi Kodama, et al. "Sapporo: A workflow execution service that encourages the reuse of workflows in various languages in bioinformatics." F1000Research 11 (August 4, 2022): 889. http://dx.doi.org/10.12688/f1000research.122924.1.

Full text
Abstract:
The increased demand for efficient computation in data analysis encourages researchers in biomedical science to use workflow systems. Workflow systems, or so-called workflow languages, are used for the description and execution of a set of data analysis steps. Workflow systems increase the productivity of researchers, specifically in fields that use high-throughput DNA sequencing applications, where scalable computation is required. As systems have improved the portability of data analysis workflows, research communities are able to share workflows to reduce the cost of building ordinary analysis procedures. However, having multiple workflow systems in a research field has resulted in the distribution of efforts across different workflow system communities. As each workflow system has its unique characteristics, it is not feasible to learn every single system in order to use publicly shared workflows. Thus, we developed Sapporo, an application to provide a unified layer of workflow execution upon the differences of various workflow systems. Sapporo has two components: an application programming interface (API) that receives the request of a workflow run and a browser-based client for the API. The API follows the Workflow Execution Service API standard proposed by the Global Alliance for Genomics and Health. The current implementation supports the execution of workflows in four languages: Common Workflow Language, Workflow Description Language, Snakemake, and Nextflow. With its extensible and scalable design, Sapporo can support the research community in utilizing valuable resources for data analysis.
APA, Harvard, Vancouver, ISO, and other styles
5

Thang, Mike W. C., Xin-Yi Chua, Gareth Price, Dominique Gorse, and Matt A. Field. "MetaDEGalaxy: Galaxy workflow for differential abundance analysis of 16s metagenomic data." F1000Research 8 (May 23, 2019): 726. http://dx.doi.org/10.12688/f1000research.18866.1.

Full text
Abstract:
Metagenomic sequencing is an increasingly common tool in environmental and biomedical sciences yet analysis workflows remain immature relative to other field such as DNASeq and RNASeq analysis pipelines. While software for detailing the composition of microbial communities using 16S rRNA marker genes is constantly improving, increasingly researchers are interested in identifying changes exhibited within microbial communities under differing environmental conditions. In order to gain maximum value from metagenomic sequence data we must improve the existing analysis environment by providing accessible and scalable computational workflows able to generate reproducible results. Here we describe a complete end-to-end open-source metagenomics workflow running within Galaxy for 16S differential abundance analysis. The workflow accepts 454 or Illumina sequence data (either overlapping or non-overlapping paired end reads) and outputs lists of the operational taxonomic unit (OTUs) exhibiting the greatest change under differing conditions. A range of analysis steps and graphing options are available giving users a high-level of control over their data and analyses. Additionally, users are able to input complex sample-specific metadata information which can be incorporated into differential analysis and used for grouping / colouring within graphs. Detailed tutorials containing sample data and existing workflows are available for three different input types: overlapping and non-overlapping read pairs as well as for pre-generated Biological Observation Matrix (BIOM) files. Using the Galaxy platform we developed MetaDEGalaxy, a complete metagenomics differential abundance analysis workflow. MetaDEGalaxy is designed for bench scientists working with 16S data who are interested in comparative metagenomics. MetaDEGalaxy builds on momentum within the wider Galaxy metagenomics community with the hope that more tools will be added as existing methods mature.
APA, Harvard, Vancouver, ISO, and other styles
6

Šimko, Tibor, Lukas Heinrich, Harri Hirvonsalo, Dinos Kousidis, and Diego Rodríguez. "REANA: A System for Reusable Research Data Analyses." EPJ Web of Conferences 214 (2019): 06034. http://dx.doi.org/10.1051/epjconf/201921406034.

Full text
Abstract:
The revalidation, reinterpretation and reuse of research data analyses requires having access to the original computing environment, the experimental datasets, the analysis software, and the computational workflow steps which were used by researchers to produce the original scientific results in the first place. REANA (Reusable Analyses) is a nascent platform enabling researchers to structure their research data analyses in view of enabling future reuse. The analysis is described by means of a YAML file that captures sufficient information about the analysis assets, parameters and processes. The REANA platform consists of a set of micro-services allowing to launch and monitor container-based computational workflow jobs on the cloud. The REANA user interface and the command-line client enables researchers to easily rerun analysis workflows with new input parameters. The REANA platform aims at supporting several container technologies (Docker), workflow engines (CWL, Yadage), shared storage systems (Ceph, EOS) and compute cloud infrastructures (Ku-bernetes/OpenStack, HTCondor) used by the community. REANA was developed with the particle physics use case in mind and profits from synergies with general reusable research data analysis patterns in other scientific disciplines, such as bioinformatics and life sciences.
APA, Harvard, Vancouver, ISO, and other styles
7

Souza, Renan, Vitor Silva, Alexandre A. B. Lima, Daniel de Oliveira, Patrick Valduriez, and Marta Mattoso. "Distributed in-memory data management for workflow executions." PeerJ Computer Science 7 (May 7, 2021): e527. http://dx.doi.org/10.7717/peerj-cs.527.

Full text
Abstract:
Complex scientific experiments from various domains are typically modeled as workflows and executed on large-scale machines using a Parallel Workflow Management System (WMS). Since such executions usually last for hours or days, some WMSs provide user steering support, i.e., they allow users to run data analyses and, depending on the results, adapt the workflows at runtime. A challenge in the parallel execution control design is to manage workflow data for efficient executions while enabling user steering support. Data access for high scalability is typically transaction-oriented, while for data analysis, it is online analytical-oriented so that managing such hybrid workloads makes the challenge even harder. In this work, we present SchalaDB, an architecture with a set of design principles and techniques based on distributed in-memory data management for efficient workflow execution control and user steering. We propose a distributed data design for scalable workflow task scheduling and high availability driven by a parallel and distributed in-memory DBMS. To evaluate our proposal, we develop d-Chiron, a WMS designed according to SchalaDB’s principles. We carry out an extensive experimental evaluation on an HPC cluster with up to 960 computing cores. Among other analyses, we show that even when running data analyses for user steering, SchalaDB’s overhead is negligible for workloads composed of hundreds of concurrent tasks on shared data. Our results encourage workflow engine developers to follow a parallel and distributed data-oriented approach not only for scheduling and monitoring but also for user steering.
APA, Harvard, Vancouver, ISO, and other styles
8

Thang, Mike W. C., Xin-Yi Chua, Gareth Price, Dominique Gorse, and Matt A. Field. "MetaDEGalaxy: Galaxy workflow for differential abundance analysis of 16s metagenomic data." F1000Research 8 (October 18, 2019): 726. http://dx.doi.org/10.12688/f1000research.18866.2.

Full text
Abstract:
Metagenomic sequencing is an increasingly common tool in environmental and biomedical sciences. While software for detailing the composition of microbial communities using 16S rRNA marker genes is relatively mature, increasingly researchers are interested in identifying changes exhibited within microbial communities under differing environmental conditions. In order to gain maximum value from metagenomic sequence data we must improve the existing analysis environment by providing accessible and scalable computational workflows able to generate reproducible results. Here we describe a complete end-to-end open-source metagenomics workflow running within Galaxy for 16S differential abundance analysis. The workflow accepts 454 or Illumina sequence data (either overlapping or non-overlapping paired end reads) and outputs lists of the operational taxonomic unit (OTUs) exhibiting the greatest change under differing conditions. A range of analysis steps and graphing options are available giving users a high-level of control over their data and analyses. Additionally, users are able to input complex sample-specific metadata information which can be incorporated into differential analysis and used for grouping / colouring within graphs. Detailed tutorials containing sample data and existing workflows are available for three different input types: overlapping and non-overlapping read pairs as well as for pre-generated Biological Observation Matrix (BIOM) files. Using the Galaxy platform we developed MetaDEGalaxy, a complete metagenomics differential abundance analysis workflow. MetaDEGalaxy is designed for bench scientists working with 16S data who are interested in comparative metagenomics. MetaDEGalaxy builds on momentum within the wider Galaxy metagenomics community with the hope that more tools will be added as existing methods mature.
APA, Harvard, Vancouver, ISO, and other styles
9

Curcin, Vasa, Moustafa Ghanem, and Yike Guo. "The design and implementation of a workflow analysis tool." Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 368, no. 1926 (September 13, 2010): 4193–208. http://dx.doi.org/10.1098/rsta.2010.0157.

Full text
Abstract:
Motivated by the use of scientific workflows as a user-oriented mechanism for building executable scientific data integration and analysis applications, this article introduces a framework and a set of associated methods for analysing the execution properties of scientific workflows. Our framework uses a number of formal modelling techniques to characterize the process and data behaviour of workflows and workflow components and to reason about their functional and execution properties. We use the framework to design the architecture of a customizable tool that can be used to analyse the key execution properties of scientific workflows at authoring stage. Our design is generic and can be applied to a wide variety of scientific workflow languages and systems, and is evaluated by building a prototype of the tool for the Discovery Net system. We demonstrate and discuss the utility of the framework and tool using workflows from a real-world medical informatics study.
APA, Harvard, Vancouver, ISO, and other styles
10

Jackson, Michael, Kostas Kavoussanakis, and Edward W. J. Wallace. "Using prototyping to choose a bioinformatics workflow management system." PLOS Computational Biology 17, no. 2 (February 25, 2021): e1008622. http://dx.doi.org/10.1371/journal.pcbi.1008622.

Full text
Abstract:
Workflow management systems represent, manage, and execute multistep computational analyses and offer many benefits to bioinformaticians. They provide a common language for describing analysis workflows, contributing to reproducibility and to building libraries of reusable components. They can support both incremental build and re-entrancy—the ability to selectively re-execute parts of a workflow in the presence of additional inputs or changes in configuration and to resume execution from where a workflow previously stopped. Many workflow management systems enhance portability by supporting the use of containers, high-performance computing (HPC) systems, and clouds. Most importantly, workflow management systems allow bioinformaticians to delegate how their workflows are run to the workflow management system and its developers. This frees the bioinformaticians to focus on what these workflows should do, on their data analyses, and on their science. RiboViz is a package to extract biological insight from ribosome profiling data to help advance understanding of protein synthesis. At the heart of RiboViz is an analysis workflow, implemented in a Python script. To conform to best practices for scientific computing which recommend the use of build tools to automate workflows and to reuse code instead of rewriting it, the authors reimplemented this workflow within a workflow management system. To select a workflow management system, a rapid survey of available systems was undertaken, and candidates were shortlisted: Snakemake, cwltool, Toil, and Nextflow. Each candidate was evaluated by quickly prototyping a subset of the RiboViz workflow, and Nextflow was chosen. The selection process took 10 person-days, a small cost for the assurance that Nextflow satisfied the authors’ requirements. The use of prototyping can offer a low-cost way of making a more informed selection of software to use within projects, rather than relying solely upon reviews and recommendations by others.
APA, Harvard, Vancouver, ISO, and other styles
More sources

Dissertations / Theses on the topic "Data Analysis Workflow"

1

Rodrigues, Roberto Wagner da Silva. "Deviation analysis of inter-organisational workflow systems." Thesis, Imperial College London, 2002. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.271151.

Full text
APA, Harvard, Vancouver, ISO, and other styles
2

Marsolo, Keith Allen. "A workflow for the modeling and analysis of biomedical data." Columbus, Ohio : Ohio State University, 2007. http://rave.ohiolink.edu/etdc/view?acc%5Fnum=osu1180309265.

Full text
APA, Harvard, Vancouver, ISO, and other styles
3

Cutler, Darren W., and Tyler J. Rasmussen. "Usability Testing and Workflow Analysis of the TRADOC Data Visualization Tool." Thesis, Monterey, California. Naval Postgraduate School, 2012. http://hdl.handle.net/10945/17350.

Full text
Abstract:
Approved for public release; distribution is unlimited
The volume of data available to military decision makers is vast. Leaders need tools to sort, analyze, and present information in an effective manner. Software complexity is also increasing, with user interfaces becoming more intricate and interactive. The Data Visualization Tool (DaViTo) is an effort by TRAC Monterey to produce a tool for use by personnel with little statistical background to process and display this data. To meet the program goals and make analytical capabilities more widely available, the user interface and data representation techniques need refinement. This usability test is a task-oriented study using eye-tracking, data representation techniques, and surveys to generate recommendations for software improvement. Twenty-four subjects participated in three sessions using DaViTo over a three-week period. The first two sessions consisted of training followed by basic reinforcement tasks, evaluation of graphical methods, and a brief survey. The final session was a task-oriented session followed by graphical representations evaluation and an extensive survey. Results from the three sessions were analyzed and 37 recommendations generated for the improvement of DaViTo. Improving software latency, providing more graphing options and tools, and inclusion of an effective training product are examples of important recommendations that would greatly improve usability.
APA, Harvard, Vancouver, ISO, and other styles
4

Nagavaram, Ashish. "Cloud Based Dynamic Workflow with QOS For Mass Spectrometry Data Analysis." The Ohio State University, 2011. http://rave.ohiolink.edu/etdc/view?acc_num=osu1322681210.

Full text
APA, Harvard, Vancouver, ISO, and other styles
5

Kwak, Daniel (Daniel Joowon). "Investigation of intrinsic rotation dependencies in Alcator C-Mod using a new data analysis workflow." Thesis, Massachusetts Institute of Technology, 2015. http://hdl.handle.net/1721.1/103705.

Full text
Abstract:
Thesis: S.M., Massachusetts Institute of Technology, Department of Nuclear Science and Engineering, 2015.
Cataloged from PDF version of thesis.
Includes bibliographical references (pages 190-193).
Toroidal rotation, important for suppressing various turbulent modes, mitigating MHD instabilities, and preventing locked modes that cause disruptions, may not be sufficiently generated by external devices in larger devices i.e. ITER. One possible solution is intrinsic rotation, self-generated flow without external momentum input, which has been observed in multiple tokamaks. More specifically, rotation reversals, a sudden change in direction of intrinsic rotation without significant change in global plasma parameters, have also been observed and are not yet fully understood. Studying this phenomenon in ohmic L-mode plasmas presents a rich opportunity to gain better understanding of intrinsic rotation and of momentum transport as a whole. The literature presents many different hypotheses, and this thesis explores three in particular. The first two hypotheses each posits a unique parameter as the primary dependency of reversals - the dominant turbulent mode, or the fastest growing turbulent mode(TEM/ITG), and the local density and temperature profile gradients, especially the electron density gradient, respectively. Other studies state that neoclassical effects cause the reversals and one study in particular presents a 1-D analytical model. Utilizing a new data analysis workflow built around GYRO, a gyrokinetic-Maxwell solver, hundreds of intrinsic rotation shots at Alcator C-Mod can be processed and analyzed without constant user management, which is used to test the three hypotheses. By comparing the rotation gradient u', a proxy variable indicative of the core toroidal intrinsic rotation velocity, to the parameters identified by the hypotheses, little correlation has been found between u' and the dominant turbulence regime and the ion temperature, electron temperature, and electron density profile gradients. The plasma remains ITG-dominated based on linear stability analysis regardless of rotation direction and the local profile gradients are not statistically significant in predicting the u'. Additionally, the experimental results in C-Mod and ASDEX Upgrade have shown strong disagreement with the 1 -D neoclassical model. Strong correlation has been found between u' and the effective collisionality Veff. These findings are inconsistent with previous experimental studies and suggest that further work is required to identify other key dependencies and/or uncover the complex physics and mechanisms at play.
by Daniel (Joowon) Kwak
S.M.
APA, Harvard, Vancouver, ISO, and other styles
6

Ba, Mouhamadou. "Composition guidée de services : application aux workflows d’analyse de données en bio-informatique." Thesis, Rennes, INSA, 2015. http://www.theses.fr/2015ISAR0024/document.

Full text
Abstract:
Dans les domaines scientifiques, particulièrement en bioinformatique, des services élémentaires sont composés sous forme de workflows pour effectuer des expériences d’analyse de données complexes. À cause de l’hétérogénéité des ressources, la composition de services est une tâche difficile. Les utilisateurs, en composant des workflows, manquent d’assistance pour retrouver et interconnecter les services compatibles. Les solutions existantes utilisent des services spéciaux définis de manière manuelle pour gérer les conversions de formats de données entre les entrées et sorties des services dans les workflows. Cela est pénible pour un utilisateur final. Gérer les incompatibilités des services avec des convertisseurs manuels prend du temps et est lourd. Il existe des solutions automatisées pour faciliter la composition de workflows mais elles sont généralement limitées dans le guidage et l’adaptation des données entre services. La première contribution de cette thèse propose de détecter systématiquement la convertibilité des sorties vers les entrées des services. La détection de convertibilité repose sur un système de règles basé sur une abstraction des types d’entrée et sortie des services. L’abstraction de types permet de considérer la nature et la composition des données d’entrée et sortie. Les règles permettent la décomposition et la composition ainsi que la spécialisation et la généralisation de types. Elles permettent également de générer des convertisseurs de données à utiliser entre services dans les workflows. La deuxième contribution propose une approche interactive qui permet de guider des utilisateurs à composer des workflows en fournissant des suggestions de services et de liaisons compatibles basées sur la convertibilité de types d’entrée et sortie des services. L’approche est basée sur le modèle des Systèmes d’Information Logiques (LIS) qui permettent des requêtes et une navigation guidées et sûres sur des données représentées avec une logique uniforme. Avec notre approche, la composition de workflows est sûre et complète vis-à-vis de propriétés désirées. Les résultats et les expériences, effectués sur des services et des types de données en bioinformatique, montrent la pertinence de nos approches. Nos approches offrent des mécanismes adaptés pour gérer les incompatibilités de services dans les workflows, en prenant en compte la structure composite des données d’entrée et sortie. Elles permettent également de guider, étape par étape, des utilisateurs à définir des workflows bien formés à travers des suggestions pertinentes
In scientific domains, particularly in bioinformatics, elementary services are composed as workflows to perform complex data analysis experiments. Due to the heterogeneity of resources, the composition of services is a difficult task. Users, when composing workflows, lack assistance to find and interconnect compatible services. Existing solutions use special services manually defined to manage data format conversions between the inputs and outputs of services in workflows, it is difficult for an end user. Managing service incompatibilities with manual converters is time-consuming and heavy. There are automated solutions to facilitate composing workflows but they are generally limited in the guidance and the data adaptation between services they offer. The first contribution of this thesis proposes to systematically detect convertibility from outputs to inputs of services. Convertibility detection relies on a rule system based on an abstraction of input and output types of services. Type abstraction enables to consider the nature and the composition of input and output data. Rules enable decomposition and composition as well as specialization and generalization of types. They also enable to generate data converters to use between services in workflows. The second contribution proposes an interactive approach that enables to guide users to compose workflows by providing suggestions of compatible services and links based on convertibility of input and output types of services. The approach is based on the framework of Logical Information Systems (LIS) that enables safe and guided requests and navigation on data represented with a uniform logic. With our approach, composition of workflows is safe and complete w.r.t. desired properties. The results and experiences, conducted on bioinformatics services and datatypes, show the relevance of our approaches. Our approaches offer adapted mechanisms to manage service incompatibilities in workflows, by taking into account the composite structure of inputs and outputs data. They enable to guide, step by step, users to define well-formed workflows through relevant suggestions
APA, Harvard, Vancouver, ISO, and other styles
7

Kreiß, Lucas [Verfasser], Oliver [Akademischer Betreuer] Friedrich, and Maximilian [Gutachter] Waldner. "Advanced Optical Technologies for Label-free Tissue Diagnostics - A complete workflow from the optical bench, over experimental studies to data analysis / Lucas Kreiß ; Gutachter: Maximilian Waldner ; Betreuer: Oliver Friedrich." Erlangen : Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), 2021. http://d-nb.info/1228627568/34.

Full text
APA, Harvard, Vancouver, ISO, and other styles
8

Jaradat, Ward. "On the construction of decentralised service-oriented orchestration systems." Thesis, University of St Andrews, 2016. http://hdl.handle.net/10023/8036.

Full text
Abstract:
Modern science relies on workflow technology to capture, process, and analyse data obtained from scientific instruments. Scientific workflows are precise descriptions of experiments in which multiple computational tasks are coordinated based on the dataflows between them. Orchestrating scientific workflows presents a significant research challenge: they are typically executed in a manner such that all data pass through a centralised computer server known as the engine, which causes unnecessary network traffic that leads to a performance bottleneck. These workflows are commonly composed of services that perform computation over geographically distributed resources, and involve the management of dataflows between them. Centralised orchestration is clearly not a scalable approach for coordinating services dispersed across distant geographical locations. This thesis presents a scalable decentralised service-oriented orchestration system that relies on a high-level data coordination language for the specification and execution of workflows. This system's architecture consists of distributed engines, each of which is responsible for executing part of the overall workflow. It exploits parallelism in the workflow by decomposing it into smaller sub-workflows, and determines the most appropriate engines to execute them using computation placement analysis. This permits the workflow logic to be distributed closer to the services providing the data for execution, which reduces the overall data transfer in the workflow and improves its execution time. This thesis provides an evaluation of the presented system which concludes that decentralised orchestration provides scalability benefits over centralised orchestration, and improves the overall performance of executing a service-oriented workflow.
APA, Harvard, Vancouver, ISO, and other styles
9

Musaraj, Kreshnik. "Extraction automatique de protocoles de communication pour la composition de services Web." Thesis, Lyon 1, 2010. http://www.theses.fr/2010LYO10288/document.

Full text
Abstract:
La gestion des processus-métiers, des architectures orientées-services et leur rétro-ingénierie s’appuie fortement sur l’extraction des protocoles-métier des services Web et des modèles des processus-métiers à partir de fichiers de journaux. La fouille et l’extraction de ces modèles visent la (re)découverte du comportement d'un modèle mis en œuvre lors de son exécution en utilisant uniquement les traces d'activité, ne faisant usage d’aucune information a priori sur le modèle cible. Notre étude préliminaire montre que : (i) une minorité de données sur l'interaction sont enregistrées par le processus et les architectures de services, (ii) un nombre limité de méthodes d'extraction découvrent ce modèle sans connaître ni les instances positives du protocole, ni l'information pour les déduire, et (iii) les approches actuelles se basent sur des hypothèses restrictives que seule une fraction des services Web issus du monde réel satisfont. Rendre possible l'extraction de ces modèles d'interaction des journaux d'activité, en se basant sur des hypothèses réalistes nécessite: (i) des approches qui font abstraction du contexte de l'entreprise afin de permettre une utilisation élargie et générique, et (ii) des outils pour évaluer le résultat de la fouille à travers la mise en œuvre du cycle de vie des modèles découverts de services. En outre, puisque les journaux d'interaction sont souvent incomplets, comportent des erreurs et de l’information incertaine, alors les approches d'extraction proposées dans cette thèse doivent être capables de traiter ces imperfections correctement. Nous proposons un ensemble de modèles mathématiques qui englobent les différents aspects de la fouille des protocoles-métiers. Les approches d’extraction que nous présentons, issues de l'algèbre linéaire, nous permettent d'extraire le protocole-métier tout en fusionnant les étapes classiques de la fouille des processus-métiers. D'autre part, notre représentation du protocole basée sur des séries temporelles des variations de densité de flux permet de récupérer l'ordre temporel de l'exécution des événements et des messages dans un processus. En outre, nous proposons la définition des expirations propres pour identifier les transitions temporisées, et fournissons une méthode pour les extraire en dépit de leur propriété d'être invisible dans les journaux. Finalement, nous présentons un cadre multitâche visant à soutenir toutes les étapes du cycle de vie des workflow de processus et des protocoles, allant de la conception à l'optimisation. Les approches présentées dans ce manuscrit ont été implantées dans des outils de prototypage, et validées expérimentalement sur des ensembles de données et des modèles de processus et de services Web. Le protocole-métier découvert, peut ensuite être utilisé pour effectuer une multitude de tâches dans une organisation ou une entreprise
Business process management, service-oriented architectures and their reverse engineering heavily rely on the fundamental endeavor of mining business process models and Web service business protocols from log files. Model extraction and mining aim at the (re)discovery of the behavior of a running model implementation using solely its interaction and activity traces, and no a priori information on the target model. Our preliminary study shows that : (i) a minority of interaction data is recorded by process and service-aware architectures, (ii) a limited number of methods achieve model extraction without knowledge of either positive process and protocol instances or the information to infer them, and (iii) the existing approaches rely on restrictive assumptions that only a fraction of real-world Web services satisfy. Enabling the extraction of these interaction models from activity logs based on realistic hypothesis necessitates: (i) approaches that make abstraction of the business context in order to allow their extended and generic usage, and (ii) tools for assessing the mining result through implementation of the process and service life-cycle. Moreover, since interaction logs are often incomplete, uncertain and contain errors, then mining approaches proposed in this work need to be capable of handling these imperfections properly. We propose a set of mathematical models that encompass the different aspects of process and protocol mining. The extraction approaches that we present, issued from linear algebra, allow us to extract the business protocol while merging the classic process mining stages. On the other hand, our protocol representation based on time series of flow density variations makes it possible to recover the temporal order of execution of events and messages in the process. In addition, we propose the concept of proper timeouts to refer to timed transitions, and provide a method for extracting them despite their property of being invisible in logs. In the end, we present a multitask framework aimed at supporting all the steps of the process workflow and business protocol life-cycle from design to optimization.The approaches presented in this manuscript have been implemented in prototype tools, and experimentally validated on scalable datasets and real-world process and web service models.The discovered business protocols, can thus be used to perform a multitude of tasks in an organization or enterprise
APA, Harvard, Vancouver, ISO, and other styles
10

Khemiri, Wael. "Data-intensive interactive workflows for visual analytics." Phd thesis, Université Paris Sud - Paris XI, 2011. http://tel.archives-ouvertes.fr/tel-00659227.

Full text
Abstract:
The increasing amounts of electronic data of all forms, produced by humans (e.g. Web pages, structured content such as Wikipedia or the blogosphere etc.) and/or automatic tools (loggers, sensors, Web services, scientific programs or analysis tools etc.) leads to a situation of unprecedented potential for extracting new knowledge, finding new correlations, or simply making sense of the data.Visual analytics aims at combining interactive data visualization with data analysis tasks. Given the explosion in volume and complexity of scientific data, e.g., associated to biological or physical processes or social networks, visual analytics is called to play an important role in scientific data management.Most visual analytics platforms, however, are memory-based, and are therefore limited in the volume of data handled. Moreover, the integration of each new algorithm (e.g. for clustering) requires integrating it by hand into the platform. Finally, they lack the capability to define and deploy well-structured processes where users with different roles interact in a coordinated way sharing the same data and possibly the same visualizations.This work is at the convergence of three research areas: information visualization, database query processing and optimization, and workflow modeling. It provides two main contributions: (i) We propose a generic architecture for deploying a visual analytics platform on top of a database management system (DBMS) (ii) We show how to propagate data changes to the DBMS and visualizations, through the workflow process. Our approach has been implemented in a prototype called EdiFlow, and validated through several applications. It clearly demonstrates that visual analytics applications can benefit from robust storage and automatic process deployment provided by the DBMS while obtaining good performance and thus it provides scalability.Conversely, it could also be integrated into a data-intensive scientific workflow platform in order to increase its visualization features.
APA, Harvard, Vancouver, ISO, and other styles
More sources

Books on the topic "Data Analysis Workflow"

1

Long, J. Scott. The workflow of data analysis using Stata. College Station, Tex: Stata Press, 2009.

Find full text
APA, Harvard, Vancouver, ISO, and other styles
2

The workflow of data analysis using Stata. College Station, Tex: Stata Press, 2009.

Find full text
APA, Harvard, Vancouver, ISO, and other styles
3

Business process modeling: Software engineering, analysis and applications. Hauppauge, N.Y: Nova Science Publishers, 2010.

Find full text
APA, Harvard, Vancouver, ISO, and other styles
4

Ould, Martyn A. Business process management: A rigorous approach. Tampa, FL: Meghan-Kiffer Press, 2005.

Find full text
APA, Harvard, Vancouver, ISO, and other styles
5

Miura, Kota. Bioimage Data Analysis Workflows. Cham: Springer Nature, 2020.

Find full text
APA, Harvard, Vancouver, ISO, and other styles
6

Miura, Kota, and Nataša Sladoje, eds. Bioimage Data Analysis Workflows. Cham: Springer International Publishing, 2020. http://dx.doi.org/10.1007/978-3-030-22386-1.

Full text
APA, Harvard, Vancouver, ISO, and other styles
7

Tova, Milo, ed. Business processes: A database perspective. [San Rafael, Calif.]: Morgan & Claypool, 2012.

Find full text
APA, Harvard, Vancouver, ISO, and other styles
8

Miura, Kota, and Nataša Sladoje, eds. Bioimage Data Analysis Workflows ‒ Advanced Components and Methods. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-030-76394-7.

Full text
APA, Harvard, Vancouver, ISO, and other styles
9

Garg, Harish. Mastering Exploratory Analysis with pandas: Build an end-to-end data analysis workflow with Python. Packt Publishing, 2018.

Find full text
APA, Harvard, Vancouver, ISO, and other styles
10

(Editor), Ian J. Taylor, Ewa Deelman (Editor), Dennis B. Gannon (Editor), and Matthew Shields (Editor), eds. Workflows for e-Science: Scientific Workflows for Grids. Springer, 2006.

Find full text
APA, Harvard, Vancouver, ISO, and other styles
More sources

Book chapters on the topic "Data Analysis Workflow"

1

Louveaux, Marion, and Stéphane Verger. "How to Do the Deconstruction of Bioimage Analysis Workflows: A Case Study with SurfCut." In Bioimage Data Analysis Workflows ‒ Advanced Components and Methods, 115–46. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-030-76394-7_6.

Full text
Abstract:
AbstractPublished bioimage analysis workflows are designed for a specific biology use case and often hidden in the material and methods section of a biology paper. The art of the bioimage analyst is to find these workflows, deconstruct them and tune them to a new use case by replacing or modifying components of the workflow and/or linking them to other workflows.
APA, Harvard, Vancouver, ISO, and other styles
2

Belcastro, Loris, and Fabrizio Marozzo. "Workflow Systems for Big Data Analysis." In Encyclopedia of Big Data Technologies, 1–6. Cham: Springer International Publishing, 2018. http://dx.doi.org/10.1007/978-3-319-63962-8_137-1.

Full text
APA, Harvard, Vancouver, ISO, and other styles
3

Belcastro, Loris, and Fabrizio Marozzo. "Workflow Systems for Big Data Analysis." In Encyclopedia of Big Data Technologies, 1811–16. Cham: Springer International Publishing, 2019. http://dx.doi.org/10.1007/978-3-319-77525-8_137.

Full text
APA, Harvard, Vancouver, ISO, and other styles
4

Belcastro, Loris, and Fabrizio Marozzo. "Workflow Systems for Big Data Analysis." In Encyclopedia of Big Data Technologies, 1–7. Cham: Springer International Publishing, 2012. http://dx.doi.org/10.1007/978-3-319-63962-8_137-2.

Full text
APA, Harvard, Vancouver, ISO, and other styles
5

Aliano, Antonio, Giancarlo Cicero, Hossein Nili, Nicolas G. Green, Pablo García-Sánchez, Antonio Ramos, Andreas Lenshof, et al. "Automatic Data Analysis Workflow for RNAi." In Encyclopedia of Nanotechnology, 171. Dordrecht: Springer Netherlands, 2012. http://dx.doi.org/10.1007/978-90-481-9751-4_100047.

Full text
APA, Harvard, Vancouver, ISO, and other styles
6

Gaaloul, Walid, Karim Baïna, and Claude Godart. "A Bottom-Up Workflow Mining Approach for Workflow Applications Analysis." In Data Engineering Issues in E-Commerce and Services, 182–97. Berlin, Heidelberg: Springer Berlin Heidelberg, 2006. http://dx.doi.org/10.1007/11780397_15.

Full text
APA, Harvard, Vancouver, ISO, and other styles
7

Schätzle, R., and W. Stucky. "Modeling Concepts for Flexible Workflow Support." In Studies in Classification, Data Analysis, and Knowledge Organization, 376–83. Berlin, Heidelberg: Springer Berlin Heidelberg, 1999. http://dx.doi.org/10.1007/978-3-642-60187-3_39.

Full text
APA, Harvard, Vancouver, ISO, and other styles
8

Gómez-de-Mariscal, Estibaliz, Daniel Franco-Barranco, Arrate Muñoz-Barrutia, and Ignacio Arganda-Carreras. "Building a Bioimage Analysis Workflow Using Deep Learning." In Bioimage Data Analysis Workflows ‒ Advanced Components and Methods, 59–88. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-030-76394-7_4.

Full text
Abstract:
AbstractThe aim of this workflow is to quantify the morphology of pancreatic stem cells lying on a 2D polystyrene substrate from phase contrast microscopy images. For this purpose, the images are first processed with a Deep Learning model trained for semantic segmentation (cell/background); next, the result is refined and individual cell instances are segmented before characterizing their morphology. Through this workflow the readers will learn the nomenclature and understand the principles of Deep Learning applied to image processing.
APA, Harvard, Vancouver, ISO, and other styles
9

Kotouza, Maria Th, Fotis E. Psomopoulos, and Pericles A. Mitkas. "A Dockerized String Analysis Workflow for Big Data." In Communications in Computer and Information Science, 564–69. Cham: Springer International Publishing, 2019. http://dx.doi.org/10.1007/978-3-030-30278-8_55.

Full text
APA, Harvard, Vancouver, ISO, and other styles
10

Wendlinger, Lorenz, Emanuel Berndl, and Michael Granitzer. "Methods for Automatic Machine-Learning Workflow Analysis." In Machine Learning and Knowledge Discovery in Databases. Applied Data Science Track, 52–67. Cham: Springer International Publishing, 2021. http://dx.doi.org/10.1007/978-3-030-86517-7_4.

Full text
APA, Harvard, Vancouver, ISO, and other styles

Conference papers on the topic "Data Analysis Workflow"

1

Balis, Bartosz. "Hypermedia Workflow: A New Approach to Data-Driven Scientific Workflows." In 2012 SC Companion: High Performance Computing, Networking, Storage and Analysis (SCC). IEEE, 2012. http://dx.doi.org/10.1109/sc.companion.2012.25.

Full text
APA, Harvard, Vancouver, ISO, and other styles
2

Song, Tianhong. "Provenance-Driven Data Curation Workflow Analysis." In SIGMOD/PODS'15: International Conference on Management of Data. New York, NY, USA: ACM, 2015. http://dx.doi.org/10.1145/2744680.2744691.

Full text
APA, Harvard, Vancouver, ISO, and other styles
3

Keiswetter, Dean, and Tom Furuya. "DATA ANALYSIS WORKFLOW FOR UXO CLASSIFICATION." In Symposium on the Application of Geophysics to Engineering and Environmental Problems 2014. Society of Exploration Geophysicists and Environment and Engineering Geophysical Society, 2014. http://dx.doi.org/10.4133/sageep.27-164.

Full text
APA, Harvard, Vancouver, ISO, and other styles
4

Keiswetter, Dean, and Tom Furuya. "DATA ANALYSIS WORKFLOW FOR UXO CLASSIFICATION." In Symposium on the Application of Geophysics to Engineering and Environmental Problems 2014. Society of Exploration Geophysicists and Environment and Engineering Geophysical Society, 2014. http://dx.doi.org/10.1190/sageep.27-164.

Full text
APA, Harvard, Vancouver, ISO, and other styles
5

Song, Zhengyi, and Young Moon. "Data Modeling and Workflow Analysis of Cyber-Manufacturing Systems." In ASME 2020 International Mechanical Engineering Congress and Exposition. American Society of Mechanical Engineers, 2020. http://dx.doi.org/10.1115/imece2020-23149.

Full text
Abstract:
Abstract Cyber-Manufacturing System (CMS) is a vision for the factory of the future, where physical manufacturing resources and processes are integrated with computational workflows to provide on-demand, adaptive, and scalable manufacturing services. In CMS, functional manufacturing components in a factory floor are digitized and encapsulated in production services; and are accessible by users throughout the network. CMS utilizes data-centric technologies to program manufacturing activities in factory floors. Leveraging advanced technologies, CMS can provide robust solutions to achieve better manufacturing agility, flexibility, scalability, and sustainability than from traditional factories. While data is the main driver of the manufacturing activities in CMS, the lack of (i) a generic data model of explicit representation of the entities and stakeholders in CMS and (ii) workflow definition and analysis for service-orientated functionalities and manufacturing intelligence of CMS is still hindering the implementation of a fully executable CMS. To address such problems, this paper (i) formalizes a data modeling of CMS using Entity-Relationship (E-R) diagram, (ii) presents the definition and analysis of workflows along with data pipelines and Extract/Transform/Load (ETL) processes that automate the entire lifecycle activities in CMS and (iii) deploys the proposed data model and workflows in a Web-based application, and (iv) tests the functionality of this application with an industrial case and eventually validates the proposed data model and workflows.
APA, Harvard, Vancouver, ISO, and other styles
6

Neumuth, Thomas, Svetlana Mansmann, Marc H. Scholl, and Oliver Burgert. "Data Warehousing Technology for Surgical Workflow Analysis." In 2008 21st International Symposium on Computer-Based Medical Systems (CBMS). IEEE, 2008. http://dx.doi.org/10.1109/cbms.2008.41.

Full text
APA, Harvard, Vancouver, ISO, and other styles
7

Berlingerio, Michele, Fabio Pinelli, Mirco Nanni, and Fosca Giannotti. "Temporal mining for interactive workflow data analysis." In the 15th ACM SIGKDD international conference. New York, New York, USA: ACM Press, 2009. http://dx.doi.org/10.1145/1557019.1557038.

Full text
APA, Harvard, Vancouver, ISO, and other styles
8

Gancheva, Veska, and Violeta Todorova. "Workflow for Medical Data Classification and Analysis." In 2022 International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT). IEEE, 2022. http://dx.doi.org/10.1109/ismsit56059.2022.9932780.

Full text
APA, Harvard, Vancouver, ISO, and other styles
9

Lee, Carlyn-Ann, Mohsin A. Shaikh, Charles Lee, and Dominik Michels. "Lunar Terrain Coverage Analysis Data Delivery Workflow." In ASCEND 2021. Reston, Virginia: American Institute of Aeronautics and Astronautics, 2021. http://dx.doi.org/10.2514/6.2021-4039.

Full text
APA, Harvard, Vancouver, ISO, and other styles
10

Sethi, Ricky J., Hyunjoon Jo, and Yolanda Gil. "Re-Using Workflow Fragments across Multiple Data Domains." In 2012 SC Companion: High Performance Computing, Networking, Storage and Analysis (SCC). IEEE, 2012. http://dx.doi.org/10.1109/sc.companion.2012.24.

Full text
APA, Harvard, Vancouver, ISO, and other styles

Reports on the topic "Data Analysis Workflow"

1

Salter, R., Quyen Dong, Cody Coleman, Maria Seale, Alicia Ruvinsky, LaKenya Walker, and W. Bond. Data Lake Ecosystem Workflow. Engineer Research and Development Center (U.S.), April 2021. http://dx.doi.org/10.21079/11681/40203.

Full text
Abstract:
The Engineer Research and Development Center, Information Technology Laboratory’s (ERDC-ITL’s) Big Data Analytics team specializes in the analysis of large-scale datasets with capabilities across four research areas that require vast amounts of data to inform and drive analysis: large-scale data governance, deep learning and machine learning, natural language processing, and automated data labeling. Unfortunately, data transfer between government organizations is a complex and time-consuming process requiring coordination of multiple parties across multiple offices and organizations. Past successes in large-scale data analytics have placed a significant demand on ERDC-ITL researchers, highlighting that few individuals fully understand how to successfully transfer data between government organizations; future project success therefore depends on a small group of individuals to efficiently execute a complicated process. The Big Data Analytics team set out to develop a standardized workflow for the transfer of large-scale datasets to ERDC-ITL, in part to educate peers and future collaborators on the process required to transfer datasets between government organizations. Researchers also aim to increase workflow efficiency while protecting data integrity. This report provides an overview of the created Data Lake Ecosystem Workflow by focusing on the six phases required to efficiently transfer large datasets to supercomputing resources located at ERDC-ITL.
APA, Harvard, Vancouver, ISO, and other styles
2

Gerecke, William, Douglas Enas, and Susan Gottschlich. A Distributed Collaborative Workflow Based Approach to Data Collection and Analysis. Fort Belvoir, VA: Defense Technical Information Center, June 2004. http://dx.doi.org/10.21236/ada466000.

Full text
APA, Harvard, Vancouver, ISO, and other styles
3

Bauer, Andrew, James Forsythe, Jayanarayanan Sitaraman, Andrew Wissink, Buvana Jayaraman, and Robert Haehnel. In situ analysis and visualization to enable better workflows with CREATE-AV™ Helios. Engineer Research and Development Center (U.S.), June 2021. http://dx.doi.org/10.21079/11681/40846.

Full text
Abstract:
The CREATE-AV™ Helios CFD simulation code has been used to accurately predict rotorcraft performance under a variety of flight conditions. The Helios package contains a suite of tools that contain almost the entire set of functionality needed for a variety of workflows. These workflows include tools customized to properly specify many in situ analysis and visualization capabilities appropriate for rotorcraft analysis. In situ is the process of computing analysis and visualization information during a simulation run before data is saved to disk. In situ has been referred to with a variety of terms including co-processing, covisualization, coviz, etc. In this paper we describe the customization of the pre-processing GUI and corresponding development of the Helios solver code-base to effectively implement in situ analysis and visualization to reduce file IO and speed up workflows for CFD analysts. We showcase how the workflow enables the wide variety of Helios users to effectively work in post-processing tools they are already familiar with as opposed to forcing them to learn new tools in order post-process in situ data extracts being produced by Helios. These data extracts include various sources of information customized to Helios, such as knowledge about the near- and off-body grids, internal surface extracts with patch information, and volumetric extracts meant for fast post-processing of data. Additionally, we demonstrate how in situ can be used by workflow automation tools to help convey information to the user that would be much more difficult when using full data dumps.
APA, Harvard, Vancouver, ISO, and other styles
4

Tang, Pingo, Alper Yilmaz, and Nancy Cooke. Automatic Imagery Data Analysis for Proactive Computer-Based Workflow Management during Nuclear Power Plant Outages. Office of Scientific and Technical Information (OSTI), January 2019. http://dx.doi.org/10.2172/1491996.

Full text
APA, Harvard, Vancouver, ISO, and other styles
5

Fuentes, Anthony, Michelle Michaels, and Sally Shoop. Methodology for the analysis of geospatial and vehicle datasets in the R language. Cold Regions Research and Engineering Laboratory (U.S.), November 2021. http://dx.doi.org/10.21079/11681/42422.

Full text
Abstract:
The challenge of autonomous off-road operations necessitates a robust understanding of the relationships between remotely sensed terrain data and vehicle performance. The implementation of statistical analyses on large geospatial datasets often requires the transition between multiple software packages that may not be open-source. The lack of a single, modular, and open-source analysis environment can reduce the speed and reliability of an analysis due to an increased number of processing steps. Here we present the capabilities of a workflow, developed in R, to perform a series of spatial and statistical analyses on vehicle and terrain datasets to quantify the relationship between sensor data and vehicle performance in winter conditions. We implemented the R-based workflow on datasets from a large, coordinated field campaign aimed at quantifying the response of military vehicles on snow-covered terrains. This script greatly reduces processing times of these datasets by combining the GIS, data-assimilation and statistical analyses steps into one efficient and modular interface.
APA, Harvard, Vancouver, ISO, and other styles
6

Eisenhauer, Greg. PERFORMANCE UNDERSTANDING AND ANALYSIS FOR EXASCALE DATA MANAGEMENT WORKFLOWS. Office of Scientific and Technical Information (OSTI), January 2019. http://dx.doi.org/10.2172/1490730.

Full text
APA, Harvard, Vancouver, ISO, and other styles
7

Malony, Allen. Performance Understanding and Analysis for Exascale Data Management Workflows. Office of Scientific and Technical Information (OSTI), August 2019. http://dx.doi.org/10.2172/1546804.

Full text
APA, Harvard, Vancouver, ISO, and other styles
8

Lougheed, H. D., M. B. McClenaghan, D. Layton-Matthews, and M. I. Leybourne. Indicator minerals in fine-fraction till heavy-mineral concentrates determined by automated mineral analysis: examples from two Canadian polymetallic base-metal deposits. Natural Resources Canada/CMSS/Information Management, 2022. http://dx.doi.org/10.4095/328011.

Full text
Abstract:
Exploration under glacial sediment cover is a necessary part of modern mineral exploration in Canada. Traditional indicator methods use visual examination to identify mineral grains in the 250 to 2000 µm fraction of till heavy-mineral concentrates (HMC). This study tests automated mineralogical methods using scanning electron microscopy to identify indicator minerals in the fine (<250 µm) HMC fraction of till. Automated mineralogy of polished grains from the fine HMC enables rapid data collection (10 000-300 000 grains/sample). Samples collected near two deposits were used to test this method: four from the upper-amphibolite facies Izok Lake volcanogenic massive-sulfide deposit, Nunavut, and five from the Sisson granite-hosted W-Mo deposit, New Brunswick. The less than 250 µm HMC fraction of till samples collected down ice of each deposit contain ore and alteration minerals typical of their deposit type. Sulfide minerals occur mainly as inclusions in oxidation-resistant minerals, including minerals previously identified in each deposit's metamorphic alteration halo, and are found to occur farther down ice than the grains identified visually in the greater than 250 µm HMC fraction. This project's workflow expands the detectable footprint for certain indicator minerals and enhances the information that can be collected from till samples.
APA, Harvard, Vancouver, ISO, and other styles
9

Liu, X., Z. Chen, and S. E. Grasby. Using shallow temperature measurements to evaluate thermal flux anomalies in the southern Mount Meager volcanic area, British Columbia, Canada. Natural Resources Canada/CMSS/Information Management, 2022. http://dx.doi.org/10.4095/330009.

Full text
Abstract:
Geothermal is a clean and renewable energy resource. However, locating where elevated thermal gradient anomalies exist is a significant challenge when trying to assess potential resource volumes during early exploration of a prospective geothermal area. In this study, we deployed 22 temperature probes in the shallow subsurface along the south flank of the Mount Meager volcanic complex, to measure the transient temperature variation from September 2020 to August 2021. In our data analysis, a novel approach was developed to estimate the near-surface thermal distribution, and a workflow and code with python language have been completed for the thermal data pre-processing and analysis. The long-term temperature variation at different depths can be estimated by modelling, so that the relative difference of deducing deeper geothermal gradient anomalies can be assessed. Our proposed inversion and simulation methods were applied to calculating the temperature variation at 2.0 meters depth. The results identified a preferred high thermal flux anomalous zone in the south Mount Meager area. By combining with previous studies, the direct analysis and estimation of anomalous thermal fields based on the collected temperature data can provide a significant reference for interpretation of the regional thermal gradient variation.
APA, Harvard, Vancouver, ISO, and other styles
10

de Caritat, Patrice, Brent McInnes, and Stephen Rowins. Towards a heavy mineral map of the Australian continent: a feasibility study. Geoscience Australia, 2020. http://dx.doi.org/10.11636/record.2020.031.

Full text
Abstract:
Heavy minerals (HMs) are minerals with a specific gravity greater than 2.9 g/cm3. They are commonly highly resistant to physical and chemical weathering, and therefore persist in sediments as lasting indicators of the (former) presence of the rocks they formed in. The presence/absence of certain HMs, their associations with other HMs, their concentration levels, and the geochemical patterns they form in maps or 3D models can be indicative of geological processes that contributed to their formation. Furthermore trace element and isotopic analyses of HMs have been used to vector to mineralisation or constrain timing of geological processes. The positive role of HMs in mineral exploration is well established in other countries, but comparatively little understood in Australia. Here we present the results of a pilot project that was designed to establish, test and assess a workflow to produce a HM map (or atlas of maps) and dataset for Australia. This would represent a critical step in the ability to detect anomalous HM patterns as it would establish the background HM characteristics (i.e., unrelated to mineralisation). Further the extremely rich dataset produced would be a valuable input into any future machine learning/big data-based prospectivity analysis. The pilot project consisted in selecting ten sites from the National Geochemical Survey of Australia (NGSA) and separating and analysing the HM contents from the 75-430 µm grain-size fraction of the top (0-10 cm depth) sediment samples. A workflow was established and tested based on the density separation of the HM-rich phase by combining a shake table and the use of dense liquids. The automated mineralogy quantification was performed on a TESCAN® Integrated Mineral Analyser (TIMA) that identified and mapped thousands of grains in a matter of minutes for each sample. The results indicated that: (1) the NGSA samples are appropriate for HM analysis; (2) over 40 HMs were effectively identified and quantified using TIMA automated quantitative mineralogy; (3) the resultant HMs’ mineralogy is consistent with the samples’ bulk geochemistry and regional geological setting; and (4) the HM makeup of the NGSA samples varied across the country, as shown by the mineral mounts and preliminary maps. Based on these observations, HM mapping of the continent using NGSA samples will likely result in coherent and interpretable geological patterns relating to bedrock lithology, metamorphic grade, degree of alteration and mineralisation. It could assist in geological investigations especially where outcrop is minimal, challenging to correctly attribute due to extensive weathering, or simply difficult to access. It is believed that a continental-scale HM atlas for Australia could assist in derisking mineral exploration and lead to investment, e.g., via tenement uptake, exploration, discovery and ultimately exploitation. As some HMs are hosts for technology critical elements such as rare earth elements, their systematic and internally consistent quantification and mapping could lead to resource discovery essential for a more sustainable, lower-carbon economy.
APA, Harvard, Vancouver, ISO, and other styles
You might also be interested in the extended bibliographies on the topic 'Data Analysis Workflow' for particular source types:
Journal articles Dissertations / Theses Books
We offer discounts on all premium plans for authors whose works are included in thematic literature selections. Contact us to get a unique promo code!

To the bibliography