Dissertations / Theses on the topic 'Grid workflow'
To see the other types of publications on this topic, follow the link: Grid workflow.
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 dissertations / theses for your research on the topic 'Grid workflow.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse dissertations / theses on a wide variety of disciplines and organise your bibliography correctly.
1
Zheng, Wei. "Explorations in grid workflow scheduling." Thesis, University of Manchester, 2010. https://www.research.manchester.ac.uk/portal/en/theses/explorations-in-grid-workflow-scheduling(7766d1b8-8c6c-44da-8b5c-df45329fa3a5).html.
Full textAbstract:
Aiming at aggregating numerous distributed resources to provide immense computing power, Grid computing has emerged as a promising paradigm to run complex composite applications such as workflows. However, the inherent uncertainties of grid systems as well as the structural complexity of workflow applications make it extremely challenging to schedule workflows in an efficient way, regardless of whether the objective is to minimize execution time or meet specific user and/or system Quality of Service (QoS) requirements. For both these cases, this thesis considers scheduling problems motivated by grid uncertainties and advances the state-of-the-art by developing new techniques to address these problems.First, based on existing scheduling heuristics, a Monte-Carlo approach is developed to minimize the average makespan (i.e., the overall execution time) in the presence of task estimates exhibiting limited uncertainty in the form of (controlled) random behaviour. Next, a scenario where performance prediction is difficult to obtain and resource availability may vary over time, is considered. A low-cost efficient just-in-time heuristic is proposed to cope with grid uncertainties.After addressing these performance-driven scheduling problems, a QoS-driven problem, which considers not only the aforementioned uncertainties but also the uncertainty caused by queue-based scheduling, is examined. In order to tackle all these uncertainties, an integrated scheduling model consisting of three supportive techniques is developed. Extensive evaluation using simulation shows that the proposed techniques can achieve substantial improvements towards the ultimate goal of providing a good solution for QoS-driven workflow scheduling on the Grid.
2
Schneider, Jörg. "Grid workflow scheduling based on incomplete information /." kostenfrei, 2010. http://opus.kobv.de/tuberlin/volltexte/2010/2574/.
Full text
3
Wang, Yan. "Integration eines Workflow-Systems in das Grid." [S.l. : s.n.], 2003. http://www.bsz-bw.de/cgi-bin/xvms.cgi?SWB10605159.
Full text
4
Thandavan, Ashish. "Supporting multiple workflow systems in a grid access environment." Thesis, University of Reading, 2008. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.515799.
Full text
5
Azam, Nabeel Adeem. "A framework for grid-enabling scientific workflow systems : architecture and application case studies on interoperability and heterogeneity in support for grid workflow automation." Thesis, University of Bradford, 2010. http://hdl.handle.net/10454/5442.
Full textAbstract:
Since the early 2000s, Service Oriented Architectures (SOAs) have played a key role in the development of complex applications within a virtual organization (VO) context. Grids and workflows have emerged as vital technologies for addressing the (SOA) paradigm. Given the variety of Grid middleware, scientific workflow systems and Grid workflows available, bringing the two technologies together in a flexible, reusable and generalized way has been largely overlooked, particularly from a scientific end user perspective. The lack of domain focus in this area has led to a slow uptake of Grid technologies. This thesis aims to design a framework for Grid-enabling workflows, which identifies the essential technological components, how these components fit together in layered architecture and the interactions between them. To produce such a framework, this thesis first investigates the definition of a Grid-workflow architecture and mapping Grid functionality to workflow nodes, focusing on striking a balance between performance, usability and the Grid functionality supported. Next, it presents an examination of framework extensions for supporting various forms of Grid heterogeneity, essential for ii VO based collaboration. Given the complex nature of Grid technologies, the work presented here investigates abstracting Grid based workflows through high-level definitions and resolution using semantic technologies. Finally, this thesis presents a way to resolves abstract Grid workflows using semantic technologies and intelligent, autonomous agents. The frameworks presented in this thesis are tested and evaluated within the context of domain-based case studies defined in the SIMDAT, BRIDGE and ARGUGRID EU funded research projects.
6
Azam, Nabeel A. "A Framework for Grid-Enabling Scientific Workflow Systems. Architecture and application case studies on interoperability and heterogeneity in support for Grid workflow automation." Thesis, University of Bradford, 2010. http://hdl.handle.net/10454/5442.
Full textAbstract:
Since the early 2000s, Service Oriented Architectures (SOAs) have played a key role in the development of complex applications within a virtual organization (VO) context. Grids and workflows have emerged as vital technologies for addressing the (SOA) paradigm. Given the variety of Grid middleware, scientific workflow systems and Grid workflows available, bringing the two technologies together in a flexible, reusable and generalized way has been largely overlooked, particularly from a scientific end user perspective. The lack of domain focus in this area has led to a slow uptake of Grid technologies.
This thesis aims to design a framework for Grid-enabling workflows, which identifies the essential technological components, how these components fit together in layered architecture and the interactions between them. To produce such a framework, this thesis first investigates the definition of a Grid-workflow architecture and mapping Grid functionality to workflow nodes, focusing on striking a balance between performance, usability and the Grid functionality supported. Next, it presents an examination of framework extensions for supporting various forms of Grid heterogeneity, essential for
ii
VO based collaboration. Given the complex nature of Grid technologies, the work presented here investigates abstracting Grid based workflows through high-level definitions and resolution using semantic technologies. Finally, this thesis presents a way to resolves abstract Grid workflows using semantic technologies and intelligent, autonomous agents.
The frameworks presented in this thesis are tested and evaluated within the context of domain-based case studies defined in the SIMDAT, BRIDGE and ARGUGRID EU funded research projects.
7
Chen, Jinjun, and n/a. "Towards effective and efficient temporal verification in grid workflow systems." Swinburne University of Technology, 2007. http://adt.lib.swin.edu.au./public/adt-VSWT20070424.112326.
Full textAbstract:
In grid architecture, a grid workflow system is a type of high-level grid middleware
which aims to support large-scale sophisticated scientific or business processes in a
variety of complex e-science or e-business applications such as climate modelling,
disaster recovery, medical surgery, high energy physics, international stock market
modelling and so on. Such sophisticated processes often contain hundreds of
thousands of computation or data intensive activities and take a long time to
complete. In reality, they are normally time constrained. Correspondingly, temporal
constraints are enforced when they are modelled or redesigned as grid workflow
specifications at build-time. The main types of temporal constraints include upper
bound, lower bound and fixed-time. Then, temporal verification would be conducted
so that we can identify any temporal violations and handle them in time.
Conventional temporal verification research and practice have presented some
basic concepts and approaches. However, they have not paid sufficient attention to
overall temporal verification effectiveness and efficiency. In the context of grid
economy, any resources for executing grid workflows must be paid. Therefore, more
resources should be mainly used for execution of grid workflow itself rather than for
temporal verification. Poor temporal verification effectiveness or efficiency would
cause more resources diverted to temporal verification. Hence, temporal verification
effectiveness and efficiency become a prominent issue and deserve an in-depth
investigation.
This thesis systematically investigates the limitations of conventional temporal
verification in terms of temporal verification effectiveness and efficiency. The
detailed analysis of temporal verification effectiveness and efficiency is conducted
for each step of a temporal verification cycle. There are four steps in total: Step 1 -
defining temporal consistency; Step 2 - assigning temporal constraints; Step 3 -
selecting appropriate checkpoints; and Step 4 - verifying temporal constraints.
Based on the investigation and analysis, we propose some new concepts and develop
a set of innovative methods and algorithms towards more effective and efficient
temporal verification. Comparisons, quantitative evaluations and/or mathematical
proofs are also presented at each step of the temporal verification cycle. These
demonstrate that our new concepts, innovative methods and algorithms can
significantly improve overall temporal verification effectiveness and efficiency.
Specifically, in Step 1, we analyse the limitations of two temporal consistency
states which are defined by conventional verification work. After, we propose four
new states towards better temporal verification effectiveness. In Step 2, we analyse
the necessity of a number of temporal constraints in terms of temporal verification
effectiveness. Then we design a novel algorithm for assigning a series of finegrained
temporal constraints within a few user-set coarse-grained ones. In Step 3, we
discuss the problem of existing representative checkpoint selection strategies in
terms of temporal verification effectiveness and efficiency. The problem is that they
often ignore some necessary checkpoints and/or select some unnecessary ones. To
solve this problem, we develop an innovative strategy and corresponding algorithms
which only select sufficient and necessary checkpoints. In Step 4, we investigate a
phenomenon which is ignored by existing temporal verification work, i.e. temporal
dependency. Temporal dependency means temporal constraints are often dependent
on each other in terms of their verification. We analyse its impact on overall
temporal verification effectiveness and efficiency. Based on this, we develop some
novel temporal verification algorithms which can significantly improve overall
temporal verification effectiveness and efficiency. Finally, we present an extension
to our research about handling temporal verification results since these verification
results are based on our four new temporal consistency states.
The major contributions of this research are that we have provided a set of new
concepts, innovative methods and algorithms for temporal verification in grid
workflow systems. With these, we can significantly improve overall temporal
verification effectiveness and efficiency. This would eventually improve the overall
performance and usability of grid workflow systems because temporal verification
can be viewed as a service or function of grid workflow systems. Consequently, by
deploying the new concepts, innovative methods and algorithms, grid workflow
systems would be able to better support large-scale sophisticated scientific and
business processes in complex e-science and e-business applications in the context
of grid economy.
8
Chen, Jinjun. "Towards effective and efficient temporal verification in grid workflow systems." Australasian Digital Thesis Program, 2007. http://adt.lib.swin.edu.au/public/adt-VSWT20070424.112326/index.html.
Full textAbstract:
Thesis (Ph.D) - Swinburne University of Technology, Faculty of Information & Communication Technologies, Centre for Information Technology Research, 2007.A thesis to CITR - Centre for Information Technology Research, Faculty of Information and Communication Technologies, Swinburne University of Technology, for the degree of Doctor of Philosophy, 2007. Typescript. Bibliography p. 145-160.
9
Martínez, Gustavo Enrique. "Frame work for Integrating Scheduling Policies into Workflow Engines." Doctoral thesis, Universitat Autònoma de Barcelona, 2011. http://hdl.handle.net/10803/79127.
Full textAbstract:
En este trabajo se ha desarrollado una nueva solución para la integración de políticas de planificación de workflows en sistemas gestores de workflows (workflow engines). SchedFlow es la solución propuesta e implementada, la cual Permite que un usuario final pueda utilizar la política de planificación que desee, ya sea esta estática o dinámica, sin necesidad de modificar el sistema gestor de workflow. Además SchedFlow toma en cuenta cuando una tarea del workflow no se puede ejecutar por eventos externos, como que una máquina está siendo utilizada por el usuario principal de la misma. En este tipo de escenarios SchedFlow realiza la replanificación de tareas, siempre bajo la política de planificación definida o elegida por el usuario. Como resultado se ha obtenido una aportación novedosa ya que normalmente si el usuario desea utilizar una política de planificación específica, deberá modificar el gestor de workflow para que soporte dicha política de planificación. Un punto importante a destacar, es que esta tesis además de contener un estudio exhaustivo de la bibliografía existente en el área, y de realizar un análisis de problema y diseño de solución propuesta, implementa la solución en un entorno oportunístico real. Los resultados de esta tesis para la planificación de workflows en entornos Grid, abre una nueva vía para el estudio de políticas de planificación para este tipo de aplicaciones, que será aprovechado por futuras investigaciones en la UAB.
10
Sommerfeld, Dietmar [Verfasser]. "Gridification and workflow scheduling for the german D-grid / Dietmar Sommerfeld." Clausthal-Zellerfeld : Universitätsbibliothek Clausthal, 2013. http://d-nb.info/1031970649/34.
Full text
11
Aziz, Abdul. "Grid management architecture for efficient and scalable scheduling of workflow applications." Ann Arbor, Mich. : ProQuest, 2007. http://gateway.proquest.com/openurl?url_ver=Z39.88-2004&rft_val_fmt=info:ofi/fmt:kev:mtx:dissertation&res_dat=xri:pqdiss&rft_dat=xri:pqdiss:3288739.
Full textAbstract:
Thesis (Ph.D. in Computer Science)--S.M.U., 2007.Title from PDF title page (viewed Nov. 19, 2009). Source: Dissertation Abstracts International, Volume: 68-11, Section: B, page: 7420. Adviser: Hesham El-Rewini. Includes bibliographical references.
12
Gamber, Branka. "Die Verwendung von Workflow-Systemen und Web Services im Business Grid." [S.l. : s.n.], 2003. http://www.bsz-bw.de/cgi-bin/xvms.cgi?SWB10678404.
Full text
13
Nardi, Alexandre Ricardo. "Uma arquitetura de baixo acoplamento para execução de padrões de controle de fluxo em grades." Universidade de São Paulo, 2009. http://www.teses.usp.br/teses/disponiveis/45/45134/tde-29052009-143125/.
Full textAbstract:
O uso de padrões de workflow para controle de fluxo em aplicações de e-Science resulta em maior produtividade por parte do cientista, permitindo que se concentre em sua área de especialização. Todavia, o uso de padrões de workflow para paralelização em grades permanece uma questão em aberto. Este texto apresenta uma arquitetura de baixo acoplamento e extensível, para permitir a execução de padrões com ou sem a presença de grade, de modo transparente ao cientista. Descreve também o Padrão Junção Combinada, que atende a diversos cenários de paralelização comumente encontrados em aplicações de e-Science. Com isso, espera-se auxiliar o trabalho do cientista, oferecendo maior flexibilidade na utilização de grades e na representação de cenários de paralelização.The use of workflow control-flow patterns in e-Science applications results in productivity improvement, allowing the scientist to concentrate in his/her own research area. However, the use of workflow control-flow patterns for execution in grids remains an opened question. This work presents a loosely coupled and extensible architecture, allowing use of patterns with or without grids, transparently to the scientist. It also describes the Combined Join Pattern, compliant to parallelization scenarios, commonly found in e-Science applications. As a result, it is expected to help the scientist tasks, giving him or her greater flexibility in the grid usage and in representing parallelization scenarios.
14
Rehman, Attiqa [Verfasser]. "Workload Modeling and Prediction for Workflow Scheduling in Dynamic Grid Environments / Attiqa Rehman." Hagen : Fernuniversität Hagen, 2014. http://d-nb.info/104711464X/34.
Full text
15
Lee, Yih-Jiun. "Models of workflow in grid systems : with applications to security and mobile code." Thesis, University of Southampton, 2006. http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.427425.
Full text
16
Reichwald, Julian. "Modell-getriebene Unterstützung der Workflow-Abbildung in Service-orientierten Software-Umgebungen." Berlin wvb, Wiss. Verl, 2009. http://d-nb.info/994830408/04.
Full text
17
Freitag, Stefan [Verfasser], Uwe [Akademischer Betreuer] Schwiegelshohn, and Achim [Akademischer Betreuer] Streit. "Workflow-gestützte Bereitstellung von Grid Middleware-Diensten / Stefan Freitag. Betreuer: Uwe Schwiegelshohn. Gutachter: Achim Streit." Dortmund : Universitätsbibliothek Dortmund, 2013. http://d-nb.info/1099438489/34.
Full text
18
Ghorbani, Mohammadmersad. "Computational analysis of CpG site DNA methylation." Thesis, Brunel University, 2013. http://bura.brunel.ac.uk/handle/2438/8217.
Full textAbstract:
Epigenetics is the study of factors that can change DNA and passed to next generation without change to DNA sequence. DNA methylation is one of the categories of epigenetic change. DNA methylation is the attachment of methyl group (CH3) to DNA. Most of the time it occurs in the sequences that G is followed by C known as CpG sites and by addition of methyl to the cytosine residue. As science and technology progress new data are available about individual’s DNA methylation profile in different conditions. Also new features discovered that can have role in DNA methylation. The availability of new data on DNA methylation and other features of DNA provide challenge to bioinformatics and the opportunity to discover new knowledge from existing data. In this research multiple data series were used to identify classes of methylation DNA to CpG sites. These classes are a) Never methylated CpG sites,b) Always methylated CpG sites, c) Methylated CpG sites in cancer/disease samples and non-methylated in normal samples d) Methylated CpG sites in normal samples and non-methylated in cancer/disease samples. After identification of these sites and their classes, an analysis was carried out to find the features which can better classify these sites a matrix of features was generated using four applications in EMBOSS software suite. Features matrix was also generated using the gUse/WS-PGRADE portal workflow system. In order to do this each of the four applications were grid enabled and ported to BOINC platform. The gUse portal was connected to the BOINC project via 3G-bridge. Each node in the workflow created portion of matrix and then these portions were combined together to create final matrix. This final feature matrix used in a hill climbing workflow. Hill climbing node was a JAVA program ported to BOINC platform. A Hill climbing search workflow was used to search for a subset of features that are better at classifying the CpG sites using 5 different measurements and three different classification methods: support vector machine, naïve bayes and J48 decision tree. Using this approach the hill climbing search found the models which contain less than half the number of features and better classification results. It is also been demonstrated that using gUse/WS-PGRADE workflow system can provide a modular way of feature generation so adding new feature generator application can be done without changing other parts. It is also shown that using grid enabled applications can speedup both feature generation and feature subset selection. The approach used in this research for distributed workflow based feature generation is not restricted to this study and can be applied in other studies that involve feature generation. The approach also needs multiple binaries to generate portions of features. The grid enabled hill climbing search application can also be used in different context as it only requires to follow the same format of feature matrix.
19
Bangalore, Nagaraja Guruprasad. "A generic workflow-based model for the deployment problems in grid systems using MPIAB as a case study /." Available to subscribers only, 2006. http://proquest.umi.com/pqdweb?did=1240708331&sid=21&Fmt=2&clientId=1509&RQT=309&VName=PQD.
Full text
20
Ayoubi, Tarek. "Distributed Data Management Supporting Healthcare Workflow from Patients’ Point of View." Thesis, Blekinge Tekniska Högskola, Avdelningen för för interaktion och systemdesign, 2007. http://urn.kb.se/resolve?urn=urn:nbn:se:bth-6030.
Full textAbstract:
Patient’s mobility throughout his lifetime leaves a trial of information scattered in laboratories, clinical institutes, primary care units, and other hospitals. Hence, the medical history of a patient is valuable when subjected to special healthcare units or undergoes home-care/personal-care in elderly stage cases. Despite the rhetoric about patient-centred care, few attempts were made to measure and improve in this arena. In this thesis, we will describe and implement a high-level view of a Patient Centric information management, deploying at a preliminary stage, the use of Agent Technologies and Grid Computing. Thus, developing and proposing an infrastructure that allows us to monitor and survey the patient, from the doctor’s point of view, and investigate a Persona, from the patients’ side, that functions and collaborates among different medical information structures. The Persona will attempt to interconnect all the major agents (human and software), and realize a distributed grid info-structure that directly affect the patient, therefore, revealing an adequate and cost-effective solution for most critical information needs. The results comprehended in the literature survey, consolidating Healthcare Information Management with emerged intelligent Multi-Agent System Technologies (MAS) and Grid Computing; intends to provide a solid basis for further advancements and assessments in this field, by bridging and proposing a framework between the home-care sector and the flexible agent architecture throughout the healthcare domain.
21
Tordsson, Johan. "Portable Tools for Interoperable Grids : Modular Architectures and Software for Job and Workflow Management." Doctoral thesis, Umeå universitet, Institutionen för datavetenskap, 2009. http://urn.kb.se/resolve?urn=urn:nbn:se:umu:diva-19630.
Full textAbstract:
The emergence of Grid computing infrastructures enables researchers to shareresources and collaborate in more efficient ways than before, despite belongingto different organizations and being geographically distributed. While the Gridcomputing paradigm offers new opportunities, it also gives rise to newdifficulties. This thesis investigates methods, architectures, and algorithmsfor a range of topics in the area of Grid resource management. One studiedtopic is how to automate and improve resource selection, despite heterogeneityin Grid hardware, software, availability, ownership, and usage policies.Algorithmical difficulties for this are, e.g., characterization of jobs andresources, prediction of resource performance, and data placementconsiderations. Investigated Quality of Service aspects of resource selectioninclude how to guarantee job start and/or completion times as well as how tosynchronize multiple resources for coordinated use through coallocation.Another explored research topic is architectural considerations for frameworksthat simplify and automate submission, monitoring, and fault handling for largeamounts of jobs. This thesis also investigates suitable Grid interactionpatterns for scientific workflows, studies programming models that enable dataparallelism for such workflows, as well as analyzes how workflow compositiontools should be designed to increase flexibility and expressiveness. We today have the somewhat paradoxical situation where Grids, originally aimed tofederate resources and overcome interoperability problems between differentcomputing platforms, themselves struggle with interoperability problems causedby the wide range of interfaces, protocols, and data formats that are used indifferent environments. This thesis demonstrates how proof-of-concept softwaretools for Grid resource management can, by using (proposed) standard formatsand protocols as well as leveraging state-of-the-art principles fromservice-oriented architectures, be made independent of current Gridinfrastructures. Further interoperability contributions include an in-depthstudy that surveys issues related to the use of Grid resources in scientificworkflows. This study improves our understanding of interoperability amongscientific workflow systems by viewing this topic from three differentperspectives: model of computation, workflow language, and executionenvironment. A final contribution in this thesis is the investigation of how the design ofGrid middleware tools can adopt principles and concepts from softwareengineering in order to improve, e.g., adaptability and interoperability.
22
Abidi, Leila. "Revisiter les grilles de PCs avec des technologies du Web et le Cloud computing." Thesis, Sorbonne Paris Cité, 2015. http://www.theses.fr/2015USPCD006/document.
Full textAbstract:
Le contexte de cette thèse est à l’intersection des contextes des grilles de calculs, des nouvelles technologies du Web ainsi que des Clouds et des services à la demande. Depuis leur avènement au cours des années 90, les plates-formes distribuées, plus précisément les systèmes de grilles de calcul (Grid Computing), n’ont pas cessé d’évoluer permettant ainsi de susciter multiple efforts de recherche. Les grilles de PCs ont été proposées comme une alternative aux super-calculateurs par la fédération des milliers d’ordinateurs de bureau. Les détails de la mise en oeuvre d’une telle architecture de grille, en termes de mécanismes de mutualisation des ressources, restent très difficile à cerner. Parallèlement, le Web a complètement modifié notre façon d’accéder à l’information. Le Web est maintenant une composante essentielle de notre quotidien. Les équipements ont, à leur tour, évolué d’ordinateurs de bureau ou ordinateurs portables aux tablettes, lecteurs multimédias, consoles de jeux, smartphones, ou NetPCs. Cette évolution exige d’adapter et de repenser les applications/intergiciels de grille de PCs qui ont été développés ces dernières années. Notre contribution se résume dans la réalisation d’un intergiciel de grille de PCs que nous avons appelé RedisDG. Dans son fonctionnement, RedisDG reste similaire à la plupart des intergiciels de grilles de calcul, c’est-à-dire qu’il est capable d’exécuter des applications sous forme de «sacs de tâches» dans un environnement distribué, assurer le monitoring des noeuds, valider et certifier les résultats. L’innovation de RedisDG, réside dans l’intégration de la modélisation et la vérification formelles dans sa phase de conception, ce qui est non conventionnel mais très pertinent dans notre domaine. Notre approche consiste à repenser les grilles de PCs à partir d’une réflexion et d’un cadre formel permettant de les développer, de manière rigoureuse et de mieux maîtriser les évolutions technologiques à venirThe context of this work is at the intersection of grid computing, the new Web technologies and the Clouds and services on demand contexts. Desktop Grid have been proposed as an alternative to supercomputers by the federation of thousands of desktops. The details of the implementation of such an architecture, in terms of resource sharing mechanisms, remain very hard. Meanwhile, the Web has completely changed the way we access information. The equipment, in turn, have evolved from desktops or laptops to tablets, smartphones or NetPCs. Our approach is to rethink Desktop Grids from a reflexion and a formal framework to develop them rigorously and better control future technological developments. We have reconsidered the interactions between the traditional components of a Desktop Grid based on the Web technology, and given birth to RedisDG, a new Desktop Grid middelware capable to operate on small devices, ie smartphones, tablets like the more traditional devicves (PCs). Our system is entirely based on the publish-subscribe paradigm. RedisDG is developped with Python and uses Redis as advanced key-value cache and store
23
Doan, Trung-Tung. "Epidémiologie moléculaire et métagénomique à haut débit sur la grille." Phd thesis, Université Blaise Pascal - Clermont-Ferrand II, 2012. http://tel.archives-ouvertes.fr/tel-00778073.
Full text
24
Senna, Carlos Roberto 1956. "GPO : um middleware para orquestração de serviços em grades computacionais." [s.n.], 2007. http://repositorio.unicamp.br/jspui/handle/REPOSIP/276221.
Full textAbstract:
Orientador: Edmundo Roberto Mauro MadeiraDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de Computação
Made available in DSpace on 2018-08-09T11:43:09Z (GMT). No. of bitstreams: 1 Senna_CarlosRoberto_M.pdf: 1604896 bytes, checksum: 4d91ce46c46772043ce75490d16c3b98 (MD5) Previous issue date: 2007
Resumo: No ambiente colaborativo das grades computacionais são poucas as ferramentas para gerência de processos e serviços orientadas ao usuário. Esta dissertação apresenta o Grid Process Orchestration (GPO), uma infraestrutura que faz orquestração de serviços e processos em grades computacionais, permitindo ao usuário criar e gerenciar ?uxos complexos, com tarefas fortemente acopladas, sem suporte adicional. O GPO é baseado na OGSA (Open Grid Services Architecture) e descreve os ?uxos usando o conceito de orquestra¸ao de serviços Web aplicados aos serviços das Grades Computacionais. A dissertação descreve a arquitetura da infraestrutura proposta, detalha seus principais componentes, suas funcionalidades para gerência de ?uxos e alguns aspectos do protótipo implementado. Além disso, propõe uma linguagem compacta para descrever os work?ows. Uma aplicação exemplo é apresentada ilustrando as facilidades da infraestrutura proposta
Abstract: In the collaborative environment of the Computational Grids, there are few tools used for process management and user guide services. This work presents the Grid Process Orchestration (GPO), an infrastructure for service and process orchestration in computational grids, which allows to create and manage complex work?ows composed of strongcoupled jobs with no additional support. The GPO is based on the Open Grid Services Architecture (OGSA) and describes work?ows using Web Services orchestration concepts applied to computational grid services. This work describes the architecture of the proposed infrastructure, detailing its main components, functionalities for ?ow management, and shows an implemented prototype. In addition to the architecture, it proposes a compact language for describing work?ows. An application example is presented to illustrate the facilities of the proposed infrastructure
Mestrado
Redes de Computadores
Mestre em Ciência da Computação
25
Ashraf, Jawad. "Partner-based scheduling and routing for grid workflows." Thesis, University of Leicester, 2012. http://hdl.handle.net/2381/27683.
Full textAbstract:
The Grid has enabled the scientific community to make faster progress. Scientific experiments and data analyses once spanning several years can now be completed in a matter of hours. With the advancement of technology, the execution of scientific experiments, often represented as workflows, has become more demanding. Thus, there is a vital need for improvements in the scheduling of scientific workflows. Efficient execution of scientific workflows can be achieved by the timely allocation of the resources. Advance reservation can ensure the future availability of heterogeneous resources and help a scheduler to produce better schedules. We propose a novel resource mapping technique for jobs of a Grid workflow in an advance reservation environment. Using a dynamic critical path based job selection method, our proposed technique considers the conditional mapping of parent and child jobs to the same resource, trying to minimise the communication duration between jobs and thus optimising the workflow completion time. The proposed method is analysed in both static and dynamic environments, and the simulation results show encouraging performance especially for workflows where the communication costs are higher than the computation costs. We also propose a hybrid of multiple scheduling heuristics for the aforementioned problem, which chooses the best among multiple schedules computed by different algorithms. Simulation results show a significant improvement over well known scheduling heuristics in terms of workflow completion time. Considering the advance reservation environment, a better schedule for the earliest completion of a workflow can be achieved if better paths can be found for the transfer of data files between jobs executed on different resources. We propose a K-shortest path based routing algorithm for finding good paths in the advance reservation environment. The results show that our proposed algorithm performs very well in terms of the earliest arrival time of the data. Finally, we also study a modified partner based scheduling heuristic for non-advance reservation environments. The results demonstrate that our proposed algorithm is a promising candidate for adoption in such Grid environments.
26
Paventhan, Arumugam. "Grid approaches to data-driven scientific and engineering workflows." Thesis, University of Southampton, 2007. https://eprints.soton.ac.uk/49926/.
Full textAbstract:
Enabling the full life cycle of scientific and engineering workflows requires robust middleware and services that support near-realtime data movement, high-performance processing and effective data management. In this context, we consider two related technology areas: Grid computing which is fast emerging as an accepted way forward for the large-scale, distributed and multi-institutional resource sharing and Database systems whose capabilities are undergoing continuous change providing new possibilities for scientific data management in Grid. In this thesis, we look into the challenging requirements while integrating data-driven scientific and engineering experiment workflows onto Grid. We consider wind tunnels that house multiple experiments with differing characteristics, as an application exemplar. This thesis contributes two approaches while attempting to tackle some of the following questions: How to allow domain-specific workflow activity development by hiding the underlying complexity? Can new experiments be added to the system easily? How can the overall turnaround time be reduced by an end-to-end experimental workflow support? In the first approach, we show how experiment-specific workflows can help accelerate application development using Grid services. This has been realized with the development of MyCoG, the first Commodity Grid toolkit for .NET supporting multi-language programmability. In the second , we present an alternative approach based on federated database services to realize an end-to-end experimental workflow. We show with the help of a real-world example, how database services can be building blocks for scientific and engineering workflows.
27
Cicerre, Fábio Rodrigo de Lima. "Um middleware para execução de processos estruturados em grades computacionais." [s.n.], 2007. http://repositorio.unicamp.br/jspui/handle/REPOSIP/275797.
Full textAbstract:
Orientadores: Luiz Eduardo Buzato, Edmundo Roberto Mauro MadeiraTese (doutorado) - Universidade Estadual de Campinas, Instituto de Computação
Made available in DSpace on 2018-08-16T20:16:50Z (GMT). No. of bitstreams: 1 Cicerre_FabioRodrigodeLima_D.pdf: 1417286 bytes, checksum: 642d37f1cb522ec4a356bbb4e9b55b96 (MD5) Previous issue date: 2010
Resumo: O conceito de grade surgiu com a necessidade crescente de se aproveitar recursos computacionais disponíveis em uma ou mais organizações para resolver problemas que exigem compartilhamento de dados e um grande poder de processamento. Uma grade computacional tem como objetivo principal permitir a execução distribuída e paralela de tarefas em recursos compartilhados. Uma grade é constituída de infra-estrutura física, composta de uma ou mais redes autônomas de computadores, e de um sistema de suporte (middleware), que provê serviços de gerenciamento de informações sobre os recursos da grade, controle de acesso e execução de tarefas sobre esse recursos e mecanismos de comunicação. Atualmente existem diversos sistemas que suportam a execução de tarefas independentes em uma grade computacional, mas poucos consideram a execução de processos de workflow, que permitem a definição de dependência explícita de dados e controle entre tarefas, o que impede um melhor aproveitamento de recursos, escalabilidade, desempenho de execução e recuperação automática de processos com manutenção de consistência. O sistema Xavantes, proposto e descrito nesse trabalho, procura suprir essas deficiências, tendo como principal objetivo suportar a execução distribuída de processos de workflow em máquinas heterogêneas, em uma ou mais organizações autônomas e dinâmicas, provendo um middleware que forneça uma melhor escalabilidade, desempenho e confiabilidade para a execução de aplicações em grades computacionais
Abstract: The grid concept has emerged from the increasing necessity of using available computational resources in one or more organizations in order to solve problems that require data sharing and large processing power. The main goal of a computational grid is to allow the distributed and parallel execution of tasks in shared resources. A grid is composed of a physical infra-structure, with one or more autonomous networks of computers, and a middleware, which provides services of information management about the grid resources, access control and tasks execution in these resources and communication mechanisms. Nowadays, there are some systems that support the execution of independent tasks in a computational grid, but only ones consider the execution of workflow processes, which allow the explicit definition of data and control dependencies among tasks, and this restricts a better use of available resources, scalability, execution performance, and automatic recovery of processes with the correct consistency maintenance. The Xavantes system, proposed and described in this work, is designed to reduce these deficiencies, having as its main goal the supports to the distributed execution of workflows in heterogeneous resources of one or more autonomous and dynamic organizations, providing a middleware that delivers a better scalability, performance and reliability to the application execution in grid computing
Doutorado
Sistemas Distribuídos e Redes de Computadores
Doutor em Ciência da Computação
28
El-Gayyar, Mahmoud [Verfasser]. "Distributed Management of Grid-based Scientific Workflows / Mahmoud El-Gayyar." Bonn : Universitäts- und Landesbibliothek Bonn, 2012. http://d-nb.info/1043911162/34.
Full text
29
Sirvent, Pardell Raül. "GRID superscalar: a programming model for the Grid." Doctoral thesis, Universitat Politècnica de Catalunya, 2009. http://hdl.handle.net/10803/6015.
Full textAbstract:
Durant els darrers anys el Grid ha sorgit com una nova plataforma per la computació distribuïda. La tecnologia Gris permet unir diferents recursos de diferents dominis administratius i formar un superordinador virtual amb tots ells. Molts grups de recerca han dedicat els seus esforços a desenvolupar un conjunt de serveis bàsics per oferir un middleware de Grid: una capa que permet l'ús del Grid. De tota manera, utilitzar aquests serveis no és una tasca fácil per molts usuaris finals, cosa que empitjora si l'expertesa d'aquests usuaris no està relacionada amb la informàtica.Això té una influència negativa a l'hora de que la comunitat científica adopti la tecnologia Grid. Es veu com una tecnologia potent però molt difícil de fer servir. Per facilitar l'ús del Grid és necessària una capa extra que amagui la complexitat d'aquest i permeti als usuaris programar o portar les seves aplicacions de manera senzilla.
Existeixen moltes propostes d'eines de programació pel Grid. En aquesta tesi fem un resum d'algunes d'elles, i podem veure que existeixen eines conscients i no-conscients del Grid (es programen especificant o no els detalls del Grid, respectivament). A més, molt poques d'aquestes eines poden explotar el paral·lelisme implícit de l'aplicació, i en la majoria d'elles, l'usuari ha de definir aquest paral·lelisme de manera explícita. Una altra característica que considerem important és si es basen en llenguatges de programació molt populars (com C++ o Java), cosa que facilita l'adopció per part dels usuaris finals.
En aquesta tesi, el nostre objectiu principal ha estat crear un model de programació pel Grid basat en la programació seqüencial i els llenguatges més coneguts de la programació imperativa, capaç d'explotar el paral·lelisme implícit de les aplicacions i d'accelerar-les fent servir els recursos del Grid de manera concurrent. A més, com el Grid és de naturalesa distribuïda, heterogènia i dinàmica i degut també a que el nombre de recursos que pot formar un Grid pot ser molt gran, la probabilitat de que es produeixi una errada durant l'execució d'una aplicació és elevada. Per tant, un altre dels nostres objectius ha estat tractar qualsevol tipus d'error que pugui sorgir durant l'execució d'una aplicació de manera automàtica (ja siguin errors relacionats amb l'aplicació o amb el Grid). GRID superscalar (GRIDSs), la principal contribució d'aquesta tesi, és un model de programació que assoleix els
objectius mencionats proporcionant una interfície molt petita i simple i un entorn d'execució que és capaç d'executar en paral·lel el codi proporcionat fent servir el Grid. La nostra interfície de programació permet a un usuari programar una aplicació no-conscient del Grid, amb llenguatges imperatius coneguts i populars (com C/C++, Java, Perl o Shell script) i de manera seqüencial, per tant dóna un pas important per ajudar als usuaris a adoptar la tecnologia Grid.
Hem aplicat el nostre coneixement de l'arquitectura de computadors i el disseny de microprocessadors a l'entorn d'execució de GRIDSs. Tal com es fa a un processador superescalar, l'entorn d'execució de GRIDSs és capaç de realitzar un anàlisi de dependències entre les tasques que formen l'aplicació, i d'aplicar tècniques de renombrament per incrementar el seu paral·lelisme. GRIDSs genera automàticament a partir del codi principal de l'usuari un graf que descriu les dependències de dades en l'aplicació. També presentem casos d'ús reals del model de programació en els camps de la química computacional i la bioinformàtica, que demostren que els nostres objectius han estat assolits.
Finalment, hem estudiat l'aplicació de diferents tècniques per detectar i tractar fallades: checkpoint, reintent i replicació de tasques. La nostra proposta és proporcionar un entorn capaç de tractar qualsevol tipus d'errors, de manera transparent a l'usuari sempre que sigui possible. El principal avantatge d'implementar aquests mecanismos al nivell del model de programació és que el coneixement a nivell de l'aplicació pot ser explotat per crear dinàmicament una estratègia de tolerància a fallades per cada aplicació, i evitar introduir sobrecàrrega en entorns lliures d'errors.
During last years, the Grid has emerged as a new platform for distributed computing. The Grid technology allows joining different resources from different administrative domains and forming a virtual supercomputer with all of them.
Many research groups have dedicated their efforts to develop a set of basic services to offer a Grid middleware: a layer that enables the use of the Grid. Anyway, using these services is not an easy task for many end users, even more if their expertise is not related to computer science. This has a negative influence in the adoption of the Grid technology by the scientific community. They see it as a powerful technology but very difficult to exploit. In order to ease the way the Grid must be used, there is a need for an extra layer which hides all the complexity of the Grid, and allows users to program or port their applications in an easy way.
There has been many proposals of programming tools for the Grid. In this thesis we give an overview on some of them, and we can see that there exist both Grid-aware and Grid-unaware environments (programmed with or without specifying details of the Grid respectively). Besides, very few existing tools can exploit the implicit parallelism of the application and in the majority of them, the user must define the parallelism explicitly. Another important feature we consider is if they are based in widely used programming languages (as C++ or Java), so the adoption is easier for end users.
In this thesis, our main objective has been to create a programming model for the Grid based on sequential programming and well-known imperative programming languages, able to exploit the implicit parallelism of applications and to speed them up by using the Grid resources concurrently. Moreover, because the Grid has a distributed, heterogeneous and dynamic nature and also because the number of resources that form a Grid can be very big, the probability that an error arises during an application's execution is big. Thus, another of our objectives has been to automatically deal with any type of errors which may arise during the execution of the application (application related or Grid related).
GRID superscalar (GRIDSs), the main contribution of this thesis, is a programming model that achieves these mentioned objectives by providing a very small and simple interface and a runtime that is able to execute in parallel the code provided using the Grid. Our programming interface allows a user to program a Grid-unaware application with already known and popular imperative languages (such as C/C++, Java, Perl or Shell script) and in a sequential fashion, therefore giving an important step to assist end users in the adoption of the Grid technology.
We have applied our knowledge from computer architecture and microprocessor design to the GRIDSs runtime. As it is done in a superscalar processor, the GRIDSs runtime system is able to perform a data dependence analysis between the tasks that form an application, and to apply renaming techniques in order to increase its parallelism. GRIDSs generates automatically from user's main code a graph describing the data dependencies in the application.
We present real use cases of the programming model in the fields of computational chemistry and bioinformatics, which demonstrate that our objectives have been achieved.
Finally, we have studied the application of several fault detection and treatment techniques: checkpointing, task retry and task replication. Our proposal is to provide an environment able to deal with all types of failures, transparently for the user whenever possible. The main advantage in implementing these mechanisms at the programming model level is that application-level knowledge can be exploited in order to dynamically create a fault tolerance strategy for each application, and avoiding to introduce overhead in error-free environments.
30
Dang, Minh-Quan. "A framework for SLA-aware execution of Grid-based workflows theory and implementation." Saarbrücken VDM Verlag Dr. Müller, 2006. http://d-nb.info/989171663/04.
Full text
31
Da, Silva Rafael Ferreira. "A science-gateway for workflow executions : Online and non-clairvoyant self-healing of workflow executions on grids." Thesis, Lyon, INSA, 2013. http://www.theses.fr/2013ISAL0115/document.
Full textAbstract:
Les science-gateways, telles que la Plate-forme d’Imagerie Virtuelle (VIP), permettent l’accès à un grand nombre de ressources de calcul et de stockage de manière transparente. Cependant, la quantité d’informations et de couches intergicielles utilisées créent beaucoup d’échecs et d’erreurs de système. Dans la pratique, ce sont souvent les administrateurs du système qui contrôlent le déroulement des expériences en réalisant des manipulations simples mais cruciales, comme par exemple replanifier une tâche, redémarrer un service, supprimer une exécution défaillante, ou copier des données dans des unités de stockages fiables. De cette manière, la qualité de service fournie est correcte mais demande une intervention humaine importante. Automatiser ces opérations constitue un défi pour deux raisons. Premièrement, la charge de la plate-forme est en ligne, c’est-à-dire que de nouvelles exécutions peuvent se présenter à tout moment. Aucune prédiction sur l’activité des utilisateurs n’est donc possible. De fait, les modèles, décisions et actions considérés doivent rester simples et produire des résultats pendant l’exécution de l’application. Deuxièmement, la plate-forme est non-clairvoyante à cause du manque d’information concernant les applications et ressources en production. Les ressources de calcul sont d’ordinaire fournies dynamiquement par des grappes hétérogènes, des clouds ou des grilles de volontaires, sans estimation fiable de leur disponibilité ou de leur caractéristiques. Les temps d’exécution des applications sont difficilement estimables également, en particulier dans le cas de ressources de calculs hétérogènes. Dans ce manuscrit, nous proposons un mécanisme d’auto-guérison pour la détection autonome et traitement des incidents opérationnels dans les exécutions des chaînes de traitement. Les objets considérés sont modélisés comme des automates finis à états flous (FuSM) où le degré de pertinence d’un incident est déterminé par un processus externe de guérison. Les modèles utilisés pour déterminer le degré de pertinence reposent sur l’hypothèse que les erreurs, par exemple un site ou une invocation se comportant différemment des autres, sont rares. Le mécanisme d’auto-guérison détermine le seuil de gravité des erreurs à partir de l’historique de la plate-forme. Un ensemble d’actions spécifiques est alors sélectionné par règle d’association en fonction du niveau d’erreurScience gateways, such as the Virtual Imaging Platform (VIP), enable transparent access to distributed computing and storage resources for scientific computations. However, their large scale and the number of middleware systems involved in these gateways lead to many errors and faults. In practice, science gateways are often backed by substantial support staff who monitors running experiments by performing simple yet crucial actions such as rescheduling tasks, restarting services, killing misbehaving runs or replicating data files to reliable storage facilities. Fair quality of service (QoS) can then be delivered, yet with important human intervention. Automating such operations is challenging for two reasons. First, the problem is online by nature because no reliable user activity prediction can be assumed, and new workloads may arrive at any time. Therefore, the considered metrics, decisions and actions have to remain simple and to yield results while the application is still executing. Second, it is non-clairvoyant due to the lack of information about applications and resources in production conditions. Computing resources are usually dynamically provisioned from heterogeneous clusters, clouds or desktop grids without any reliable estimate of their availability and characteristics. Models of application execution times are hardly available either, in particular on heterogeneous computing resources. In this thesis, we propose a general self-healing process for autonomous detection and handling of operational incidents in workflow executions. Instances are modeled as Fuzzy Finite State Machines (FuSM) where state degrees of membership are determined by an external healing process. Degrees of membership are computed from metrics assuming that incidents have outlier performance, e.g. a site or a particular invocation behaves differently than the others. Based on incident degrees, the healing process identifies incident levels using thresholds determined from the platform history. A specific set of actions is then selected from association rules among incident levels
32
Chiu, David T. "Auspice: Automatic Service Planning in Cloud/Grid Environments." The Ohio State University, 2010. http://rave.ohiolink.edu/etdc/view?acc_num=osu1275012033.
Full text
33
Guimarães, Felipe Pontes. "Framework para execução adaptativa e tolerante a falhas de workflows em grid." reponame:Repositório Institucional da UnB, 2010. http://repositorio.unb.br/handle/10482/8528.
Full textAbstract:
Dissertação (mestrado)—Universidade de Brasília, Instituto de Ciências Exatas, Departamento de Ciência da Computação, 2010.Submitted by Luiza Moreira Camargo (luizaamc@gmail.com) on 2011-06-20T17:36:29Z No. of bitstreams: 1 2010_FelipePontesGuimarães.pdf: 3025286 bytes, checksum: 90ebea4efc3733e40b3ce19f7925beda (MD5)
Approved for entry into archive by Guilherme Lourenço Machado(gui.admin@gmail.com) on 2011-06-21T13:43:47Z (GMT) No. of bitstreams: 1 2010_FelipePontesGuimarães.pdf: 3025286 bytes, checksum: 90ebea4efc3733e40b3ce19f7925beda (MD5)
Made available in DSpace on 2011-06-21T13:43:47Z (GMT). No. of bitstreams: 1 2010_FelipePontesGuimarães.pdf: 3025286 bytes, checksum: 90ebea4efc3733e40b3ce19f7925beda (MD5)
A computação em Grid proporciona a seus usuários o compartilhamento de recursos autônomos e heterogêneos para solucionar problemas computacionais de grande complexidade. Em um Grid, os recursos possuem autonomia, logo podem entrar e sair do mesmo conforme suas necessidades. A computação em Grid é frequentemente usada para executar worflows científicos, que são uma rede de passos necessários à análise de grande volume de dados. Geralmente, a execução de workflows científicos é demorada, podendo levar vários minutos, várias horas ou mesmo dias. Ao se associar essas duas características - um ambiente dinâmico e workflows de longa duração - surge um problema: não há como se impedir que os recursos saiam do Grid durante a execução de tarefas de um workflow, causando assim um erro na execução. Não se pode, no entanto, permitir que tais erros inviabilizem a execução do workflow. Para contornar esse problema existem técnicas de tolerância a falhas, que procuram garantir que, mesmo em face de falhas na execução de algumas tarefas, o workflow como um todo será executado corretamente. Vários trabalhos lidam com técnicas de tolerância a falhas para workflows em Grid e várias técnicas diferentes já existem. No entanto, nenhuma das abordagens estudadas considera, em conjunto, as preferências do usuário e a situação atual do Grid. A presente dissertação de mestrado propõe e avalia um framework de execução adaptativa tolerante a falhas que permite ao usuário definir as regras pelas quais a seleção das técnicas de tolerância a falhas será realizada em tempo de execução e também permite a adição de novas técnicas de tolerância a falhas. Os resultados experimentais obtidos em um Grid com 5 máquinas mostram que o framework proposto de fato permite a definição de regras pelo usuário e a inclusão de novas técnicas de tolerância a falhas. Além disso, a sobrecarga no tempo de execução dos workflows foi baixo: cerca de 2%, na plataforma avaliada. ___________________________________________________________________________________ ABSTRACT
Grid computing allows its users to share autonomous and heterogeneous resources to solve highly complex computational problems. It creates an extremely dynamic environment, in which the resources may enter or leave at any given moment according to their needs. One of the major uses of Grid computing is the execution of scientific workflows, a set of necessary steps for analyzing great amounts of data. The execution time of these workflows may vary from several minutes to days. Once we combine these two characteristics - a dynamic environment and long execution times - a problem arises, since there is no mechanism to prevent resources from leaving the Grid during the execution of a task belonging to a workflow, thus introducing an error in the execution. However, the ocurrence of these errors must not make unfeasible the workflow execution. To deal with this issue fault tolerance techniques have been proposed. They allow for correct workflow execution even when facing errors during a number of faults in intermediary tasks. Many published papers deal with fault tolerance techniques for workflow execution in the Grid, but none of the studied approaches consider both the user preferences and the current status of the Grid. The current Master's Thesis proposes and evaluates a framework that provides adaptive fault tolerant execution for workflows in Grids allowing the user to define the rules by which the fault tolerance techniques will be chosen at runtime. Moreover, it allows the addition of new fault tolerance techniques. The experimental results obtained from a 5-machine Grid show that the framework is able to allow the user to set the rules and add new fault tolerance techniques at the cost of a very low overhead in the execution time: around 2% in the evalution platform.
34
Burkimsher, Andrew Marc. "Fair, responsive scheduling of engineering workflows on computing grids." Thesis, University of York, 2014. http://etheses.whiterose.ac.uk/8080/.
Full textAbstract:
This thesis considers scheduling in the context of a grid computing system used in engineering design. Users desire responsiveness and fairness in the treatment of the workflows they submit. Submissions outstrip the available computing capacity during the work day, and the queue is only caught up on overnight and at weekends. The execution times observed span a wide range of 10^0 to 10^7 core-minutes. The Projected Schedule Length Ratio (P-SLR) list scheduling policy is designed to use execution time estimates and the structure of the dependency graph to improve on the existing industrial FairShare policy. P-SLR aims to minimise the worst-case SLR of jobs and keep SLR fair across the space of job execution times. P-SLR is shown to equal or surpass all other evaluated policies in responsiveness and fairness across the spectra of load and networking delays. P-SLR is also dominant where execution time estimates are within an order of magnitude of the real value. Such estimates are considered achievable using user knowledge or automated profiling. Outside this range, the Shortest Remaining Time First (SRTF) policy achieved better responsiveness and fairness. The Projected Value Remaining (PVR) policy considers the case where a curve specifying the value of a job over time is given. PVR aims to maximise total workload value, even under overload, by maximising the worst-case job value in a workload. PVR is shown to be dominant across the load and networking spectra. Where execution time estimates are coarser than the nearest power of 2, SRTF delivers higher value than PVR. SRTF is also shown to have responsiveness, fairness and value close behind P-SLR and PVR throughout the range of load and network delays considered. However, the kinds of starvation under overload incurred by SRTF would almost certainly be undesirable if implemented in a production system.
35
Cargnelli, Matthieu. "OpenWP : étude et extension des technologies de Workflows pour le calcul haute performance sur grille." Paris 11, 2008. http://www.theses.fr/2008PA112265.
Full textAbstract:
Cette thèse s’inscrit dans un contexte industriel. Elle aborde la problématique du passage de code de calcul scientifique séquentiel à un code de calcul exécutable sur une grille. L’approche proposée dans cette thèse exploite les technologies de worflows pour leur adéquation avec les problèmes rencontrés sur une grille. Après avoir présenté les solutions existantes pour exécuter un workflow sur une grille et une solution pour paralléliser un code séquentiel : openMP, l’auteur présente sa proposition : OpenWP. OpenWP est un environnement proposant un langage d’annotation de code séquentiel à base de directives en vue de sa transformation en workflow, puis le contrôle de l’exécution de ce worflow à travers un moteur tierce partie sur la grille. Un mécanisme de mémoire distribuée virtuellement partagée est proposé. Le langage défini pour OpenWP est détaillé, son expressivité critiquée et comparée à celle d’OpenMP. La conception d’OpenWP est ensuite décrite, ainsi que les choix technologiques qui ont conduit à l’implémentation d’un prototype. La thèse se poursuit par une validation de concept et une série d’évaluations de performances d’OpenWP avec plusieurs applications, dont un mailleur industriel utilisé par EADS. Enfin un système hybride combinant OpenWP et OpenMP est décrit. Ce système doit permettre à OpenWP d’exploiter la hiérarchie de la grille en utilisant les machines multi-processeurs à mémoire partagée lorsque cela est possible, avec OpenMP. Une preuve de concept est apportée sur un cas test simpleThis thesis has been conduced in an industrial context. It studies the code refactoring from a sequential scientific code into a grid enabled program. The proposed approach is based on the workflow technologies which are well suited for grid. After a presentation of existing solutions for workflow execution on the grid as well as a solution for code parallelization (openMP), the author presents his proposition : OpenWP. OpenWP is a workflow definition language based on directives to turn a sequential code into a workflow. OpenWP allows the controlled execution of this workflow on the grip using a third party workflow enactment engine. A distributed virtually shared memory system is proposed. The defined language is presented in detail and its expressivity is criticized and compared to OpenMP’s. The conception of OpenWP is then described and the technology choices made are explained. A prototype is presented. The document then shows a proof of concept and a series of performance evaluation of OpenWP used on few programs, among which an industrial mesher used by EADS. An hybrid system based on OpenWP and OpenMP is also described. This system must give OpenWP the ability to exploit the resource hierarchy found in the grid, by using the shared memory multi-processors machines whenever possible through OpenMP. A proof of concept test case is provided and commented
36
Glatard, Tristan. "Description, deployment and optimization of medical image analysis workflows on production grids." Nice, 2007. http://www.theses.fr/2007NICE4049.
Full textAbstract:
Grids are interesting platforms for supporting the development of medical image analysis applications: they enable data and algorithms sharing and provide huge amounts of computing power and data storage. In this thesis, we investigate a medical image analysis problem that turns out to be a typical dimensioning application for grids, thus leading to develop new workflow description, implementation and optimization methods and tools. The basic application problem is the evaluation of medical image registration algorithms in absence of ground truth. Results obtained with a statistical method applied to a registration problem dealing with the follow-up of brain tumors in radiotherapy are presented. Those results allow to detect subtle flaws among the data. We extend this validation scheme in order to quantify the impact of lossy image compression on registration algorithms. This application is representative of typical grid problems so that we study its deployment and execution on such infrastructures. We adopt a generic workflow model to ease the application parallelization on a grid infrastructure. A novel taxonomy of workflow approaches is presented. Based on it, we select a suitable workflow language and we design and implement MOTEUR, an enactor exploiting all the parallelism levels of workflow applications. A new data composition operator is also defined, easing the description of medical image analysis applications on grids. Benchmarks on the EGEE production grid compared to controlled conditions on Grid'5000 reveal that the grid latency and its variability lead to strong performance drops. Therefore, we propose a probabilistic model of the execution time of a grid workflow. This model is user-centric: the whole grid is considered as a black-box introducing a random latency on the execution time of a job. Based on this model, we propose three optimization strategies aiming at reducing the impact of the grid latency and of its variability: (1) grouping sequentially linked jobs reduces the mean latency faced by a workflow, (2) optimizing the timeout value of jobs reduces the impact of outliers and (3) optimizing the jobs granularity reduces the risk to face high latencies. Significant speed-up are yielded by those strategiesEn permettant le partage à grande échelle de données et d'algorithmes et en fournissant une quantité importante de puissance de calcul et de stockage, les grilles de calcul sont des plates-formes intéressantes pour les applications d'analyse d'images médicales. Dans cette thèse, nous étudions un problème d'analyse d'images médicales qui s'avère être une application dimensionnante pour les grilles, conduisant au développement de nouvelles méthodes et outils pour la description, l'implémentation et l'optimisation de flots de traitements. Le problème applicatif étudié est l'évaluation de la précision d'algorithmes de recalage d'images médicales en l'absence de vérité terrain. Nous faisons passer à l'échelle une méthode statistique d'évaluation de ces algorithmes et nous montrons des résultats de précision sur une base de données liée au suivi de la radiothérapie du cerveau. Ces résultats permettent notamment de détecter des défauts très légers au sein des données. Nous étendons ce schéma pour quantifier l'impact de la compression des images sur la qualité du recalage. Cette application étant représentative de problèmes typiques survenant sur les grilles, nous nous attachons à son déploiement et à son exécution sur ce type d'infrastructures. Pour faciliter une parallélisation transparente, nous adoptons un modèle générique de flots de traitements, dont nous proposons une nouvelle taxonomie. Pour répondre aux limitations de performance des moteurs d'exécution de flots existants, nous présentons MOTEUR, qui permet d'exploiter les différents types de parallélisme inhérents à ces applications. La définition d'un nouvel opérateur de composition de données facilite la description des applications d'analyse d'images médicales sur les grilles. Par une comparaison entre la grille de production EGEE et des grappes dédiées de Grid'5000, nous mettons en évidence l'importance de la variabilité de la latence sur une grille de production. En conséquence, nous proposons un modèle probabiliste du temps d'exécution d'un flot de traitement sur une grille. Ce modèle est centré sur l'utilisateur : il considère la grille toute entière comme une boîte noire introduisant une latence aléatoire sur le temps d'exécution d'une tâche. A partir de ce modèle, nous proposons trois stratégies d'optimisation visant à réduire l'impact de la latence et de sa variabilité : (1) dans un flot de traitement, grouper les tâches séquentiellement liées permet de réduire la latence moyenne rencontrée, (2) optimiser la valeur du délai d'expiration des tâches prémunit contre les valeurs extrêmes de la latence et (3) optimiser la granularité des tâches permet de réduire le risque de rencontrer de fortes latences. Des accélérations significatives sont ainsi obtenues
37
Rossetto, Anubis Graciela de Moraes. "Uma abordagem para tratamento da desconexão de dispositivos móveis na utilização de recursos de Grif computacional." Florianópolis, SC, 2007. http://repositorio.ufsc.br/xmlui/handle/123456789/89586.
Full textAbstract:
Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Tecnológico. Programa de Pós-graduação em Ciência da ComputaçãoMade available in DSpace on 2012-10-22T23:46:37Z (GMT). No. of bitstreams: 1 242963.pdf: 1538614 bytes, checksum: 5b7becee02e088bd20c866cf550c555e (MD5)
A utilização de dispositivos móveis em ambientes de grid computacional tem sido explorada para aplicações técnico-científicas, que muitas vezes podem se beneficiar do poder computacional e mobilidade desses ambientes. Todavia, a integração dos dispositivos móveis requer um tratamento mais criterioso de algumas de suas limitações. Uma das vulnerabilidades existentes em uma configuração móvel é a sua imprevisibilidade quanto às suas conexões, assim, desconexões são usualmente freqüentes em ambientes móveis. Esta dissertação apresenta uma pesquisa sobre o tratamento das desconexões de dispositivos móveis utilizados pelos usuários para submissão e monitoração de aplicações em ambientes de grids computacionais. O conceito de workflow foi empregado para a submissão e monitoração de aplicações com várias tarefas para solução de um único problema. A abordagem proposta é constituída por um mecanismo de tratamento de falhas que possui três componentes: o observador, o analisador e o adaptador. Nos casos de desconexão, o mecanismo desenvolvido adapta o fluxo de execução das aplicações considerando a natureza da aplicação e as configurações prévias do usuário. Com a execução dos testes experimentais observou-se que o mecanismo proposto atua na verificação do status de conexão e, quando necessário, procede ao ajuste na execução da aplicação. Em adição, a abordagem possibilita o reinício de aplicações não finalizadas, sem perder o processamento já realizado. Neste contexto, o mecanismo garante a consistência das aplicações de maneira transparente para o usuário, bem como evita o desperdício dos recursos do ambiente grid. Desta forma, a abordagem atingiu com sucesso o objetivo de uma melhor utilização de um ambiente de grid computacional, através de um dispositivo móvel, considerando-se suas eventuais desconexões.
38
Aliaga, Alvaro Henry Mamani. "Estudo comparativo de técnicas de escalonamento de tarefas dependentes para grades computacionais." Universidade de São Paulo, 2011. http://www.teses.usp.br/teses/disponiveis/45/45134/tde-21102011-162130/.
Full textAbstract:
À medida que a ciência avança, muitas aplicações em diferentes áreas precisam de grande poder computacional. A computação em grade é uma importante alternativa para a obtenção de alto poder de processamento, no entanto, esse alto poder computacional deve ser bem aproveitado. Mediante o uso de técnicas de escalonamento especializadas, os recursos podem ser utilizados adequadamente. Atualmente existem vários algoritmos propostos para computação em grade, portanto, é necessário seguir uma boa metodologia para escolher o algoritmo que ofereça melhor desempenho, dadas determinadas características. No presente trabalho comparamos os algoritmos de escalonamento: Heterogeneous Earliest Finish Time (HEFT), (b) Critical Path on a Processor (CPOP) e (c) Path Clustering Heuristic (PCH); cada algoritmo é avaliado com diferentes aplicações e sobre diferentes arquiteturas usando técnicas de simulação, seguindo quatro critérios: (i) desempenho, (ii) escalabilidade, (iii) adaptabilidade e (iv) distribuição da carga do trabalho. Diferenciamos as aplicações para grade em dois tipos: (i) aplicações regulares e (ii) aplicações irregulares; dado que em aplicações irregulares não é facil comparar o critério de escalabilidade. Seguindo esse conjunto de critérios o algoritmo HEFT possui o melhor desempenho e escalabilidade; enquanto que os três algoritmos possuem o mesmo nível de adaptabilidade. Na distribuição de carga de trabalho o algoritmo HEFT aproveita melhor os recursos do que os outros. Por outro lado os algoritmos CPOP e PCH usam a técnica de escalonar o caminho crítico no processador que ofereça o melhor tempo de término, mas essa abordagem nem sempre é a mais adequada.As science advances, many applications in different areas need a big amount of computational power. Grid computing is an important alternative to obtain high processing power, but this high computational power must be well used. By using specialized scheduling techniques, resources can be properly used. Currently there are several algorithms for grid computing, therefore, is necessary to follow a good methodology to choose an algorithm that offers better performance given certain settings. In this work, we compare task dependent scheduling algorithms: (a) Heterogeneous Earliest Finish Time (HEFT), (b) Critical Path on a Processor (CPOP) e Path Clustering Heuristic (PCH); each algorithm is evaluated with different applications and on different architectures using simulation techniques, following four criterias: (i) performance, (ii) scalability, (iii) adaptability and (iv) workload distribution. We distinguish two kinds of grid applications: (i) regular applications and (ii) irregular applications, since in irregular applications is not easy to compare scalability criteria. Following this set of criteria the HEFT algorithm reaches the best performance and scalability, while the three algorithms have the same level of adaptability. In workload distribution HEFT algorithm makes better use of resources than others. On the other hand, CPOP and PCH algorithms use scheduling of tasks which belong to the critical path on the processor which minimizes the earliest finish time, but this approach is not always the most appropriate.
39
Enomoto, Cristina. "Uma linguagem para especificação de fluxo de execução em aplicações paralelas." [s.n.], 2005. http://repositorio.unicamp.br/jspui/handle/REPOSIP/261813.
Full textAbstract:
Orientador: Marco Aurelio Amaral HenriquesDissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Eletrica e de Computação
Made available in DSpace on 2018-08-05T12:56:47Z (GMT). No. of bitstreams: 1 Enomoto_Cristina_M.pdf: 856279 bytes, checksum: ce524a49db0f67734e28d8458d5deb0b (MD5) Previous issue date: 2005
Resumo: Vários sistemas de grid e computação distribuída existentes só permitem a execução de aplicações com um fluxo de execução de tarefas básico, no qual é feita a distribuição das tarefas executadas em paralelo e depois a coleta de seus resultados. Outros sistemas permitem definir uma relação de dependências entre as tarefas, formando um grafo direcionado acíclico. Porém, mesmo com este modelo de fluxo de execução não é possível executar vários tipos de aplicações que poderiam ser paralelizadas, como, por exemplo, algoritmos genéticos e de cálculo numérico que utilizam algum tipo de processamento iterativo. Nesta dissertação é proposta uma linguagem de especificação para fluxo de execução de aplicações paralelas que permite um controle de fluxo de tarefas mais flexível, viabilizando desvios condicionais e laços com iterações controladas. A linguagem é baseada na notação XML (eXtensible Markup Language), o que lhe confere características importantes tais como flexibilidade e simplicidade. Para avaliar estas e outras características da linguagem proposta, foi feita uma implementação sobre o sistema de processamento paralelo JoiN. Além de viabilizar a criação e execução de novas aplicações paralelas cujos fluxos de tarefas contêm laços e/ou desvios condicionais, a linguagem se mostrou simples de usar e não causou sobrecarga perceptível ao sistema paralelo
Abstract: Many distributed and parallel systems allow only a basic task flow, in which the parallel tasks are distributed and their results collected. In some systems the application execution flow gives support to a dependence relationship among tasks, represented by a directed acyclic graph. Even with this model it is not possible to execute in parallel some important applications as, for example, genetic algorithms. Therefore, there is a need for a new specification model with more sophisticated flow controls that allow some kind of iterative processing at the level of task management. The purpose of this work is to present a proposal for a specification language for parallel application execution workflow, which provides new types of control structures and allows the implementation of a broader range of applications. This language is based on XML (eXtensible Markup Language) notation, which provides characteristics like simplicity and flexibility to the proposed language. To evaluate these and other characteristics of the language, it was implemented on the JoiN parallel processing system. Besides allowing the creation and execution of new parallel applications containing task flows with loops and conditional branches, the proposedlanguage was easy to use and did not cause any significant overhead to the parallel system
Mestrado
Engenharia de Computação
Mestre em Engenharia Elétrica
40
Bittencourt, Luiz Fernando 1981. "Algoritmos para escalonamento de tarefas dependentes representadas por grafos acíclicos direcionados em grades computacionais." [s.n.], 2010. http://repositorio.unicamp.br/jspui/handle/REPOSIP/275815.
Full textAbstract:
Orientador: Edmundo Roberto Mauro MadeiraTese (doutorado) - Universidade Estadual de Campinas, Instituto de Computação
Made available in DSpace on 2018-08-16T05:33:50Z (GMT). No. of bitstreams: 1 Bittencourt_LuizFernando_D.pdf: 2691554 bytes, checksum: b936bb837e62d8c4b7bacaeaae71e167 (MD5) Previous issue date: 2010
Resumo: Grades computacionais são sistemas distribuídos compartilhados potencialmente grandes compostos por recursos heterogêneos que são ligados através de uma rede com enlaces heterogêneos. Esses sistemas tornaram-se ambientes largamente difundidos para execução de tarefas que demandam grande capacidade de processamento. Por serem sistemas compartilhados, a submissão de tarefas nas grades é oriunda de diversos usuários independentemente, o que gera uma demanda concorrente pelos recursos computacionais que deve ser gerenciada pelo middleware da grade. O escalonador é o componente responsável por decidir de que forma a distribuição dessas tarefas será realizada, devendo tratar das peculiaridades desse ambiente, tais como a heterogeneidade e o comportamento dinâmico dos recursos que o compõem, com variações tanto em quantidade quanto em qualidade. A função objetivo mais comum encontrada no escalonamento de tarefas é a minimização do makespan, ou seja, o tempo de término das tarefas que estão sendo escalonadas. Dentre os possíveis tipos de tarefas executadas em grades podemos destacar as tarefas independentes, que executam sem comunicação entre si, e as tarefas dependentes, que possuem dependências de dados que geram precedências de execução e são frequentemente modeladas como grafos acíclicos direcionados (DAGs - do inglês directed acyclic graphs). Dentre as aplicações compostas por tarefas dependentes, os DAGs de e-Ciência se sobressaem pela complexidade e necessidade crescente de recursos computacionais. Adicionalmente, o problema de escalonamento de tarefas, em sua forma geral, é NP-Completo. Dessa forma, o estudo do escalonamento de DAGs em grades computacionais é importante para o aprimoramento da execução de aplicações científicas utilizadas em diversas áreas do conhecimento. Nesta tese apresentamos algoritmos para quatro tipos de problema relacionados ao escalonamento de DAGs em grades: escalonamento estático de DAGs, escalonamento dinâmico de DAGs, escalonamento bi-critério e escalonamento de múltiplos DAGs. Apresentamos avaliações do makespan gerado pelos algoritmos após o escalonamento inicial e após a execução das tarefas com carga externa simulada nos recursos
Abstract: Computational grids are potentially large distributed systems composed of heterogeneous resources connected by a network with heterogeneous links. These systems became largely used in the execution of tasks which require large processing capacities. Because they are shared systems, task submission in grids independently originate from a number of users, leading to a concurrent demand over the computational resources, which must be managed by the grid middleware. The scheduler is the component responsible for deciding how the distribution of such tasks will occur, and it must deal with peculiarities of this environment, such as the heterogeneity and dynamic behavior of the resources, with variations in both quality and quantity. The objective function usually adopted in task scheduling is makespan minimization, which means that the scheduler tries to minimize the finish time of the tasks being scheduled. Among the tasks executed in grids we can find independent tasks, which execute without communication among them, and dependent tasks, which have data dependencies that yield in precedence constraints and are frequently modeled as directed acyclic graphs (DAGs). Among the applications composed of dependent tasks, e-Science DAGs are distinguished because of their complexity and increasing demand for computational resources. Additionally, the task scheduling problem, in its general form, is NP-Complete. Therefore, the study of scheduling of dependent tasks represented by directed acyclic graphs in computational grids is important to improve the execution of scientific applications in many areas of knowledge. In this thesis we present algorithms for four types of problems related to the DAG scheduling in grids: static scheduling of DAGs, dynamic scheduling of DAGs, bi-criteria scheduling, and scheduling of multiple DAGs. We present evaluations of the makespan generated by the algorithms after the initial scheduling and after the execution of the tasks with simulated external load in the resources
Doutorado
Sistemas de Computação
Doutor em Ciência da Computação
41
Bittencourt, Luiz Fernando 1981. "Uma heuristica de agrupamento de caminhos para escalonamento de tarefas em grades computacionais." [s.n.], 2006. http://repositorio.unicamp.br/jspui/handle/REPOSIP/276481.
Full textAbstract:
Orientador: Edmundo Roberto Mauro MadeiraDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de Computação
Made available in DSpace on 2018-08-06T12:20:00Z (GMT). No. of bitstreams: 1 Bittencourt_LuizFernando_M.pdf: 1217558 bytes, checksum: dcbdeb1eaf538ae17a83304451a73126 (MD5) Previous issue date: 2006
Resumo: Uma grade computacional é um sistema heterogêneo colaborativo, geograficamente distribuído, multi-institucional e dinâmico, onde qualquer recurso computacional ligado a uma rede, local ou não, é um potencial colaborador. Grades computacionais são atualmente um grande foco de estudos relacionados à execução de aplicações paralelas, tanto aquelas que demandam grande poder computacional quanto aquelas que se adaptam bem a ambientes distribuídos. Como os recursos de uma grade pertencem a vários domínios administrativos diferentes com políticas diferentes, cada recurso tem autonomia para participar ou deixar de participar da grade em qualquer momento. Essa característica dinâmica e a heterogeneidade tornam o escalonamento de aplicações, a gerência de recursos e a tolerância a falhas grandes desafios nesses sistemas. Particularmente, o escalonamento desempenha um papel de suma importância, pois é determinante no tempo de execução das aplicações. O escalonamento de tarefas é um problema NP-Completo [6], o que levou ao desenvolvimento de uma heurística para o problema de otimização associado. Neste trabalho apresentamos um escalonador de tarefas em grades computacionais baseado no Xavantes [3], um middleware que oferece suporte a execução de tarefas dependentes através de estruturas de controle hierárquicas chamadas controladores. O algoritmo desenvolvido, chamado de Path Clustering Heuristic (PCH), agrupa as tarefas com o objetivo de minimizar a comunicação entre os controladores e as tarefas, diminuindo o tempo de execução total do processo
Abstract: A computational grid is a collaborative heterogeneous, geographically distributed, multiinstitutional and dynamic system, where any computational resource with a network connection, local or remote, is a potential collaborator. In computational grids, problems related to the execution of parallel applications, those which need a lot of computational power, as well as those which fit well in distributed environments, are wide studied nowadays. As the grid resources belong to various different administrative domains with different policies, each resource has the autonomy to participate or leave the grid at any time. These dynamic and heterogeneous characteristics make the application scheduling, the resource management and the fault tolerance relevant issues on these systems. Particularly, the scheduler plays an important role, since it is determinative in the execution time of an application. The task scheduling problem is NP-Complete [6], what led to the development of a heuristic for the associated optimization problem. In this work we present a task scheduler for a computational grid based on Xavantes [3], a middleware that supports dependent task execution through control structures called controllers. The developed algorithm, called Path Clustering Heuristic (PCH), clusterizes tasks aiming to minimize the communication between controllers and tasks, reducing the process execution time
Mestrado
Sistemas de Computação
Mestre em Ciência da Computação
42
Dong, FANGPENG. "WORKFLOW SCHEDULING ALGORITHMS IN THE GRID." Thesis, 2009. http://hdl.handle.net/1974/1795.
Full textAbstract:
The development of wide-area networks and the availability of powerful computers as low-cost commodity components are changing the face of computation. These progresses in technology make it possible to utilize geographically distributed resources in multiple owner domains to solve large-scale problems in science, engineering and commerce. Research on this topic has led to the emergence of Grid computing. To achieve the promising potentials of tremendous distributed resources in the Grid, effective and efficient scheduling algorithms are fundamentally important.
However, scheduling problems are well known for their intractability, and many of instances are in fact NP-Complete. The situation becomes even more challenging in the Grid circumstances due to some unique characteristics of the Grid. Scheduling algorithms in traditional parallel and distributed systems, which usually run on homogeneous and dedicated resources, cannot work well in the new environments.
This work focuses on workflow scheduling algorithms in the Grid scenario. New challenges are discussed, previous research in this realm is surveyed, and novel heuristic algorithms addressing the challenges are proposed and tested.
The proposed algorithms contribute to the literature by taking the following factors into account when a schedule for a DAG-based workflow is produced: predictable performance fluctuation and non-deterministic performance model of Grid resources, the computation and data staging co-scheduling, the clustering characteristic of Grid resource distribution, and the ability to reschedule according to performance change after the initial schedule is made. The performance of proposed algorithms are tested and analyzed by simulation under different workflow and resource configurations.Thesis (Ph.D, Computing) -- Queen's University, 2009-04-23 22:30:09.646
43
"Grid-centric scheduling strategies for workflow applications." Thesis, 2010. http://hdl.handle.net/1911/62205.
Full textAbstract:
Grid computing faces a great challenge because the resources are not localized, but distributed, heterogeneous and dynamic. Thus, it is essential to provide a set of programming tools that execute an application on the Grid resources with as little input from the user as possible. The thesis of this work is that Grid-centric scheduling techniques of workflow applications can provide good usability of the Grid environment by reliably executing the application on a large scale distributed system with good performance. We support our thesis with new and effective approaches in the following five aspects.
First, we modeled the performance of the existing scheduling approaches in a multi-cluster Grid environment. We implemented several widely-used scheduling algorithms and identified the best candidate. The study further introduced a new measurement, based on our experiments, which can improve the schedule quality of some scheduling algorithms as much as 20 fold in a multi-cluster Grid environment.
Second, we studied the scalability of the existing Grid scheduling algorithms. To deal with Grid systems consisting of hundreds of thousands of resources, we designed and implemented a novel approach that performs explicit resource selection decoupled from scheduling Our experimental evaluation confirmed that our decoupled approach can be scalable in such an environment without sacrificing the quality of the schedule by more than 10%.
Third, we proposed solutions to address the dynamic nature of Grid computing with a new cluster-based hybrid scheduling mechanism. Our experimental results collected from real executions on production clusters demonstrated that this approach produces programs running 30% to 100% faster than the other scheduling approaches we implemented on both reserved and shared resources.
Fourth, we improved the reliability of Grid computing by incorporating fault- tolerance and recovery mechanisms into the workow application execution. Our experiments on a simulated multi-cluster Grid environment demonstrated the effectiveness of our approach and also characterized the three-way trade-off between reliability, performance and resource usage when executing a workflow application.
Finally, we improved the large batch-queue wait time often found in production Grid clusters. We developed a novel approach to partition the workow application and submit them judiciously to achieve less total batch-queue wait time. The experimental results derived from production site batch queue logs show that our approach can reduce total wait time by as much as 70%.
Our approaches combined can greatly improve the usability of Grid computing while increasing the performance of workow applications on a multi-cluster Grid environment.
44
Hsu, Chih-Chiang, and 許志強. "Online Scheduling of Workflow Applications in a Grid Environment." Thesis, 2009. http://ndltd.ncl.edu.tw/handle/91946719059405003207.
Full textAbstract:
碩士國立交通大學
資訊科學與工程研究所
98
Scheduling workflow applications in a Grid environment is a great challenge, because it is NP-complete problem. Many heuristic methods are presented, but most of them work in the domain of single workflow application. In recent years, there are several heuristic methods presented to deal with concurrent workflows or online workflows, but they do not work with workflows composed of data-parallel tasks. In the thesis, we present an approach for dealing with online workflows, which is named Online Workflow Management (OWM). For dealing with data-parallel problems, well-known approaches, e.g., first fit, conservative backfilling and easy backfilling are added into OWM. The experiments show that OWM outperforms other two methods in various workloads. For workflows composed of data-parallel tasks, the experiments show that OWM(FCFS) is almost equal OWM(conservative), and outperforms OWM(easy) and OWM(first fit).
45
Peng, Chin-Lung, and 彭金隆. "A High-Performance Grid Computing System Based on Workflow." Thesis, 2007. http://ndltd.ncl.edu.tw/handle/14087985573469837120.
Full textAbstract:
碩士南台科技大學
資訊管理系
95
Grid systems are able to consolidate resources between organizations to share what partners compute, store, and other resources through high-speed network to foster computation. Task dispatch and resource allocation are the most important topics among issues of grid computation. Therefore, this thesis proposes a workflow framework which takes advantage of automation of workflow to combine task dispatch and resource allocation. By the workflow framework this paper presented, an implementation of solving NP-complete problem is given to attack the Maximum Independent Set requiring massive computation resources.
46
Schneider, Jörg [Verfasser]. "Grid workflow scheduling based on incomplete information / vorgelegt von Jörg Schneider." 2010. http://d-nb.info/1002213339/34.
Full text
47
Chang, Ken-Jung, and 張根榮. "An enhanced particle swarm optimization for solving grid workflow schedule problem." Thesis, 2015. http://ndltd.ncl.edu.tw/handle/62638533556748273051.
Full textAbstract:
碩士國立勤益科技大學
資訊工程系
103
Grid is a technology of sharing computing ability that combines the scattered heterogeneous resources around the world, and the computing ability of the grid is comparable to supercomputers. There are lots of researchers devote to enhance the capacity of the grid by applying novel algorithms to facilitate distributing resources to the adequate jobs and decide the executing order of the jobs in the grid environment. In this thesis, an enhanced particle swarm optimization (PSO) algorithm with the additional forces exerted on particles is suggested. This proposed PSO is named “Multi-force Particle Swarm Optimization, MPSO”. In MPSO, two negative experiences of particle including the worst individual experience of particle and the worst group experience of the swarm are used to update the particle’s moving velocity. Meanwhile, this thesis utilizes a parameter based on the problem scale to control the exerting force for pushing particles to move away from the both worst experiences. The experimental results show that the proposed multi-force particle swarm optimization algorithm is able to solve the grid scheduling problems effectively and efficiently. Comparing to other studies, although the proposed MPSO is not outstanding for the small scale problems, nevertheless the MPSO outperform other schemes in large scale problems. Restated, this MPSO with additional forces included is suitable for solving large scale grid scheduling problems.
48
Colonnese, Daniel. "Grid service data needed for estimation of reliability in scientific workflow systems." 2004. http://www.lib.ncsu.edu/theses/available/etd-05122004-094219/unrestricted/etd.pdf.
Full text
49
Lin, Cheng-Fang, and 林正芳. "A Workflow-based Resource Broker Portal with Information Monitoring on Grid Computing Environments." Thesis, 2006. http://ndltd.ncl.edu.tw/handle/56298767263917164925.
Full textAbstract:
碩士東海大學
資訊工程與科學系
94
The computational Grid is the beacon to scientists for solving large-scale problems over the Internet as a tremendous virtual computer. As Grid Computing becomes a reality, a resource broker is needed to manage and monitor available resources. This thesis presents a workflow-based computational resource broker whose main function is to match available resources with user requests and consider network information status during matchmaking. The resource broker provides a uniform interface for accessing available and the appropriate resources via user credentials. We utilize NWS tool to monitor the network-related information and resources status. In order to identify and schedule jobs that are suitable for determined resources, an execution time estimation model is required. In this thesis, it is described a Chronological history-based execution time estimation model to predict current execution time, according to the previous execution results. The experimental results shown that our model can accurately predict the execution time of embarrassingly parallel applications. Also, we constructed a grid platform using Globus Toolkit that integrates the distributed resources of five universities in Taichung, Taiwan, under TIGER project, where the resource broker is developed. As a result, the proposed broker provides secure and updated information about available resources and serves as a link to the diverse systems available in the Grid.
50
Mandal, Anirban. "Toward a tool for scheduling application workflows onto distributed grid systems." Thesis, 2006. http://hdl.handle.net/1911/18944.
Full textAbstract:
In this dissertation, we present a design and implementation of a tool for automatic mapping and scheduling of large scientific application workflows onto distributed, heterogeneous Grid environments. The thesis of this work is that plan-ahead, application-independent scheduling of workflow applications based on performance models can reduce the turnaround time for Grid execution of the application, reducing burden of Grid application development. We applied the scheduling strategies successfully to Grid applications from the domains of bio-imaging and astronomy and demonstrated the effectiveness and efficiency of the scheduling approaches. We also proposed and evaluated a novel scheduling heuristic based on a middle-out traversal of the application workflow.
A study showed that jobs have to wait in batch queues for a considerable amount of time before they begin execution. Schedulers must consider batch queue waiting times when scheduling Grid applications onto resources with batch queue front ends. Hence, we developed a smart scheduler that considers estimates of batch queue wait times when it constructs schedules for Grid applications.
We compared the proposed scheduling techniques with existing dynamic scheduling strategies. An experimental evaluation of this scheduler on data-intensive workflows shows that its approach of planning schedules in advance improves over previous online scheduling approaches.
We studied the scalability of the proposed scheduling approaches. To deal with the scale of future Grids consisting of hundreds of thousands of resources, we designed and implemented a novel cluster-level scheduling algorithm, which scales linearly on the number of abstract resource classes. An experimental evaluation using workflows from two applications shows that the cluster-level scheduler achieves good scalability without sacrificing the quality of schedule.