Journal articles: 'WLCG'

1

Sharma, Mayank, Maarten Litmaath, Eraldo Silva Junior, and Renato Santana. "Lightweight WLCG Sites." EPJ Web of Conferences 214 (2019): 07019. http://dx.doi.org/10.1051/epjconf/201921407019.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

This article describes a new framework, called SIMPLE, for settingup and maintaining classic WLCG sites with minimal operational efforts and insights needed into the WLCG middleware. The framework provides a single common interface to install and configure any of its supported grid services, such as Compute Elements, Batch Systems, Worker Nodes and miscellaneous middleware packages. It leverages modern container orchestration tools like Kubernetes, Docker Swarm, and confiuration management tools like Puppet, Ansible, to automate deployment of the WLCG services on behalf of a site admin. The framework is modular and extensible by design. Therefore, it is easy to add support for more grid services as well as infrastructure automation tools to accommodate diverse scenarios at different sites. We provide insight into the design of the framework and our efforts towards development, release and deployment of its first implementation featuring CREAM E, TORQUE Batch System and TORQUE based Worker Nodes.

2

Andreeva, Julia, Alexey Anisenkov, Alessandro Di Girolamo, Panos Paparrigopoulos, and Natalia Diana Szczepanek. "CRIC as a core instrument for WLCG operations." EPJ Web of Conferences 295 (2024): 01014. http://dx.doi.org/10.1051/epjconf/202429501014.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

WLCG is a large heterogeneous computing infrastructure which provides resources for the LHC experiments. The Computing Resource Information Catalogue (CRIC) has been designed as the source for the WLCG topology information. CRIC aims to describe WLCG distributed sites, their services and how the provided resources are used by the LHC experiments. The CRIC instance dedicated to WLCG has become a key service for WLCG computing operations, playing an important role in monitoring, accounting and organisation of the large-scale data challenges and upgrade campaigns. The contribution describes CRIC functionality widely used for the WLCG central operations as well as development plans helping to decrease operational effort.

3

Babik, Marian, Martin Bly, Tim Chown, Jiři Chudoba, Catalin Condurache, Alastair Dewhurst, Xavier Espinal Curull, et al. "IPv6 in production: its deployment and usage in WLCG." EPJ Web of Conferences 214 (2019): 08010. http://dx.doi.org/10.1051/epjconf/201921408010.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

The fraction of general internet traffic carried over IPv6 continues to grow rapidly. The transition of WLCG central and storage services to dual-stack IPv4/IPv6 is progressing well, thus enabling the use of IPv6-only CPU resources as agreed by the WLCG Management Board and presented by us at CHEP2016. By April 2018, all WLCG Tier-1 data centres should have provided access to their services over IPv6. The LHC experiments have requested all WLCG Tier-2 centres to provide dual-stack access to their storage by the end of LHC Run 2. This paper reviews the status of IPv6 deployment in WLCG.

4

Dack, Thomas, Federica Agostini, Jim Basney, Linda Cornwall, John Steven De Stefano Jr, Dave Dykstra, Francesco Giacomini, et al. "WLCG Transition from X.509 to Tokens. Status, Plans, and Timeline." EPJ Web of Conferences 295 (2024): 04054. http://dx.doi.org/10.1051/epjconf/202429504054.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

Since 2017, the Worldwide LHC Computing Grid (WLCG) has been working towards enabling token-based authentication and authorization throughout its entire middleware stack. Following the initial publication of the WLCG Token Schema v1.0 in 2019, OAuth2.0 token workflows have been integrated across grid middleware. There are many complex challenges to be addressed before the WLCG can be end-to-end token-based, including not just technical hurdles but also interoperability with the wider authentication and authorization landscape. This paper presents the status of the WLCG coordination and deployment work, and how it relates to software providers and partner communities. The authors also detail how the WLCG token transition timeline has progressed, and how it has changed since its publication.

5

Dykstra, Dave. "Web Proxy Auto Discovery for Dynamically Created Web Proxies." EPJ Web of Conferences 245 (2020): 07026. http://dx.doi.org/10.1051/epjconf/202024507026.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

The WLCG Web Proxy Auto Discovery (WPAD) service provides a convenient mechanism for jobs running anywhere on the WLCG to dynamically discover web proxy cache servers that are nearby. The web proxy caches are general purpose for a number of different http applications, but different applications have different usage characteristics and not all proxy caches are engineered to work with the heaviest loads. For this reason, the initial sources of information for WLCG WPAD were the static configurations that ATLAS and CMS maintain for the Conditions data that they read through the Frontier Distributed Database system, which is the most demanding popular WLCG application for web proxy caches. That works well for use at traditional statically defined WLCG sites, but now that usage of commercial clouds is increasing, there is also a need for web proxy caches to dynamically register themselves as they are created. A package called Shoal had already been created to manage dynamically created web proxy caches. This paper describes the integration of the Shoal package into the WLCG WPAD system, such that both statically and dynamically created web proxy caches can be located from a single source. It also describes other improvements to the WLCG WPAD system since the last CHEP publication.

6

Garrido, Borja, Julia Andreeva, Derek Weitzel, Alessandra Forti, and Shawn McKee. "New XrootD Monitoring Implementation." EPJ Web of Conferences 295 (2024): 01040. http://dx.doi.org/10.1051/epjconf/202429501040.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

Complete and reliable monitoring of the WLCG data transfers is an important condition for effective computing operations of the LHC experiments. WLCG data challenges organized in 2021 and 2022 highlighted the need for improvements in WLCG data traffic monitoring. In particular, it concerns the implementation of remote data access monitoring via the root protocol. It includes data access to native XRootD storage, as well as to other storage solutions. We refer to it as XRootD monitoring. This contribution describes the new implementation of the XRootD monitoring flow, the overall architecture, the deployment scenario, and the integration with the WLCG global monitoring system.

7

Giordano, Domenico, Manfred Alef, and Michele Michelotto. "Next Generation of HEP CPU Benchmarks." EPJ Web of Conferences 214 (2019): 08011. http://dx.doi.org/10.1051/epjconf/201921408011.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

As of 2009, HEP-SPEC06 (HS06) is the benchmark adopted by the WLCG community to describe the computing requirements of the LHC experiments, to assess the computing capacity of the WLCG data centres and to procure new hardware. In the recent years, following the evolution of CPU architectures and the adoption of new programming paradigms, such as multi-threading and vectorization, it has turned out that HS06 is less representative of the relevant applications running on the WLCG infrastructure. Meanwhile, in 2017 a new SPEC generation of benchmarks for CPU intensive workloads has been released: SPEC CPU 2017. This report summarises the findings of the HEPiX Benchmarking Working Group in comparing SPEC CPU 2017 and other HEP benchmarks with the typical WLCG workloads mixes.

8

Fischer, Max, Eileen Kuehn, Manuel Giffels, Matthias Jochen Schnepf, Andreas Petzold, and Andreas Heiss. "Lightweight dynamic integration of opportunistic resources." EPJ Web of Conferences 245 (2020): 07040. http://dx.doi.org/10.1051/epjconf/202024507040.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

To satisfy future computing demands of the Worldwide LHC Computing Grid (WLCG), opportunistic usage of third-party resources is a promising approach. While the means to make such resources compatible with WLCG requirements are largely satisfied by virtual machines and containers technologies, strategies to acquire and disband many resources from many providers are still a focus of current research. Existing meta-schedulers that manage resources in the WLCG are hitting the limits of their design when tasked to manage heterogeneous resources from many diverse resource providers. To provide opportunistic resources to the WLCG as part of a regular WLCG site, we propose a new meta-scheduling approach suitable for opportunistic, heterogeneous resource provisioning. Instead of anticipating future resource requirements, our approach observes resource usage and promotes well-used resources. Following this approach, we have developed an inherently robust meta-scheduler, COBalD, for managing diverse, heterogeneous resources given unpredictable resource requirements. This paper explains the key concepts of our approach, and discusses the benefits and limitations of our new approach to dynamic resource provisioning compared to previous approaches.

9

Giordano, Domenico, Jean-Michel Barbet, Tommaso Boccali, Gonzalo Menéndez Borge, Christopher Hollowell, Vincenzo Innocente, Walter Lampl, et al. "HEPScore: A new CPU benchmark for the WLCG." EPJ Web of Conferences 295 (2024): 07024. http://dx.doi.org/10.1051/epjconf/202429507024.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

HEPScore is a new CPU benchmark created to replace the HEPSPEC06 benchmark that is currently used by the WLCG for procurement, computing resource pledges, usage accounting and performance studies. The development of the new benchmark, based on HEP applications or workloads, has involved many contributions from software developers, data analysts, experts of the experiments, representatives of several WLCG computing centres and WLCG site managers. In this contribution, we review the selection of workloads and the validation of the new HEPScore benchmark.

10

Babik, Marian, Martin Bly, Nick Buraglio, Tim Chown, Dimitrios Christidis, Jiri Chudoba, Phil DeMar, et al. "Overcoming obstacles to IPv6 on WLCG." EPJ Web of Conferences 295 (2024): 07036. http://dx.doi.org/10.1051/epjconf/202429507036.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

The transition of the Worldwide Large Hadron Collider Computing Grid (WLCG) storage services to dual-stack IPv6/IPv4 is almost complete; all Tier-1 and 94% of Tier-2 storage are IPv6 enabled. While most data transfers now use IPv6, a significant number of IPv4 transfers still occur even when both endpoints support IPv6. This paper presents the ongoing efforts of the HEPiX IPv6 working group to steer WLCG toward IPv6-only services by investigating and fixing the obstacles to the use of IPv6 and identifying cases where IPv4 is used when IPv6 is available. Removing IPv4 use is essential for the long-term agreed goal of IPv6-only access to resources within WLCG, thus eliminating the complexity and security concerns associated with dual-stack services. We present our achievements and ongoing challenges as we navigate the final stages of the transition from IPv4 to IPv6 within WLCG.

11

Fayer, Simon, and Dan Whitehouse. "Backfilling a cloud with grid jobs." EPJ Web of Conferences 295 (2024): 04009. http://dx.doi.org/10.1051/epjconf/202429504009.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

Imperial College London hosts a large Tier-2 WLCG grid site based around a HTCondor batch system; additionally it provides cloud computing facilities using OpenStack to non-WLCG activities. These cloud resources are open to opportunistic usage provided the impact on the primary cloud users remains low. In common with most Tier-2 sites we see constant job pressure from the WLCG VOs, while the usage of our cloud is much more intermittent. To allocate grid jobs to the available opportunistic cloud resources we implemented a lightweight backfill system based on the OpenStack Python API.

12

Bockelman, Brian, Andrea Ceccanti, Thomas Dack, Dave Dykstra, Maarten Litmaath, Mischa Sallé, and Hannah Short. "WLCG Token Usage and Discovery." EPJ Web of Conferences 251 (2021): 02028. http://dx.doi.org/10.1051/epjconf/202125102028.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

Since 2017, the Worldwide LHC Computing Grid (WLCG) has been working towards enabling token based authentication and authorisation throughout its entire middleware stack. Following the publication of the WLCG Common JSON Web Token (JWT) Schema v1.0 [1] in 2019, middleware developers have been able to enhance their services to consume and validate the JWT-based [2] OAuth2.0 [3] tokens and process the authorization information they convey. Complex scenarios, involving multiple delegation steps and command line flows, are a key challenge to be addressed in order for the system to be fully operational. This paper expands on the anticipated token based workflows, with a particular focus on local storage of tokens and their discovery by services. The authors include a walk-through of this token flow in the RUCIO managed data-transfer scenario, including delegation to FTS and authorised access to storage elements. Next steps are presented, including the current target of submitting production jobs authorised by Tokens within 2021.

13

Casey, James, Lionel Cons, Wojciech Lapka, Massimo Paladin, and Konstantin Skaburskas. "A Messaging Infrastructure for WLCG." Journal of Physics: Conference Series 331, no. 6 (December 23, 2011): 062015. http://dx.doi.org/10.1088/1742-6596/331/6/062015.

Full text

APA, Harvard, Vancouver, ISO, and other styles

14

Pradillo, María Alandes, Maria Dimou, Josep Flix, Alessandra Forti, and Andrea Sciabà. "Optimising costs in WLCG operations." Journal of Physics: Conference Series 664, no. 3 (December 23, 2015): 032025. http://dx.doi.org/10.1088/1742-6596/664/3/032025.

Full text

APA, Harvard, Vancouver, ISO, and other styles

15

Andreeva, J., D. Dieguez Arias, S. Campana, J. Flix, O. Keeble, N. Magini, Z. Molnar, et al. "Providing global WLCG transfer monitoring." Journal of Physics: Conference Series 396, no. 3 (December 13, 2012): 032005. http://dx.doi.org/10.1088/1742-6596/396/3/032005.

Full text

APA, Harvard, Vancouver, ISO, and other styles

16

Babik, Marian, Martin Bly, Tim Chown, Dimitrios Christidis, Jiři Chudoba, Catalin Condurache, Thomas Finnern, et al. "IPv6-only networking on WLCG." EPJ Web of Conferences 245 (2020): 07045. http://dx.doi.org/10.1051/epjconf/202024507045.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

The use of IPv6 on the general Internet continues to grow. The transition of the Worldwide Large Hadron Collider Computing Grid (WLCG) central and storage services to dual-stack IPv6/IPv4 is progressing well, thus enabling the use of IPv6-only CPU resources as agreed by the WLCG Management Board and presented by us at earlier CHEP conferences. During the last year, the HEPiX IPv6 Working Group has continued to chase and support the transition to dual-stack services. We present the status of the transition and some tests that have been made of IPv6-only CPU showing the successful use of IPv6 protocols in accessing WLCG services. The dual-stack deployment does however result in a networking environment which is more complex than when using just IPv6. The group is investigating the removal of the IPv4 protocol in places. We present the areas where this could be useful together with our future plans.

17

Andrade, Pedro, Alberto Aimar, Simone Brundu, Borja Garrido Bear, Gonzalo Menendez Borge, Luca Magnoni, Diogo Lima Nicolau, and Nikolay Tsvetkov. "WLCG Dashboards with Unified Monitoring." EPJ Web of Conferences 245 (2020): 07049. http://dx.doi.org/10.1051/epjconf/202024507049.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

Monitoring of the CERN Data Centres and the WLCG infrastructure is now largely based on the new monitoring infrastructure provided by CERN IT. This is the result of the migration from several old in-house developed monitoring tools into a common monitoring infrastructure based on open source technologies such as Collectd, Flume, Kafka, Spark, InfluxDB, Grafana and others. This new infrastructure relies on CERN IT services (OpenStack, Puppet, Gitlab, DBOD, etc) and covers the full range of monitoring tasks: metrics and logs collection, alarms generation, data validation and transport, data enrichment and aggregation (where applicable), dashboards visualisation, reports generation, etc. This contribution will present the different services offered by the infrastructure today, highlight the main monitoring use cases from the CERN Data Centres and WLCG, and analyse the last years experience of moving from legacy well-established custom monitoring tools into a common open source-based infrastructure.

18

Babik, Marian, Shawn McKee, Brian Paul Bockelman, Edgar Mauricio Fajardo Hernandez, Edoardo Martelli, Ilija Vukotic, Derek Weitzel, and Marian Zvada. "Improving WLCG Networks Through Monitoring and Analytics." EPJ Web of Conferences 214 (2019): 08006. http://dx.doi.org/10.1051/epjconf/201921408006.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

WLCG relies on the network as a critical part of its infrastructure and therefore needs to guarantee effective network usage and prompt detection and resolution of any network issues, including connection failures, congestion and traffic routing. OSG Networking Area in partnership with WLCG has focused on collecting, storing and making available all the network related metrics for further analysis and discovery of issues that might impact network performance and operations. In order to help sites and experiments better understand and fix the networking issues, WLCG Network Throughput working group was formed, which works on the analysis and integration of the network-related monitoring data collected by the OSG/WLCG infrastructure and operates a support unit to help find and fix the network performance issues. This paper describes the current state of the OSG network measurement platform and summarises the activities taken by the working group, including updates on the higher level services that were recently developed, network performance incidents investigated as well as past and present analytical activities related to networking and their results.

19

Andreeva, Julia, Alexey Anisenkov, Alessandro Di Girolamo, Alessandra Forti, Stephen Jones, Balazs Konya, Andrew McNab, and Panos Paparrigopoulos. "Evolution of the WLCG Information Infrastructure." EPJ Web of Conferences 245 (2020): 03029. http://dx.doi.org/10.1051/epjconf/202024503029.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

The WLCG project aimed to develop, build, and maintain a global computing facility for storage and analysis of the LHC data. While currently most of the LHC computing resources are being provided by the classical grid sites, over the last years the LHC experiments have been using more and more public clouds and HPCs, and this trend will certainly continue. The heterogeneity of the LHC computing resources is not limited to the procurement mode. It also implies variety of storage solutions and types of computer architecture which represent new challenges for the topology and configuration description of the LHC computing resources. The WLCG Information infrastructure has to evolve in order to meet these challenges and to be flexible enough to follow technology innovation. It should provide a complete and reliable description of all types of the storage and computing resources to ensure their effective use. This implies changes at all levels, starting from the primary information providers, through data publishing, transportation mechanism and central aggregators. This paper describes the proposed changes in the WLCG Information Infrastructure, their implementation and deployment.

20

Short, H., A. Manzi, V. De Notaris, O. Keeble, A. Kiryanov, H. Mikkonen, P. Tedesco, and R. Wartel. "x509-free access to WLCG resources." Journal of Physics: Conference Series 898 (October 2017): 102001. http://dx.doi.org/10.1088/1742-6596/898/10/102001.

Full text

APA, Harvard, Vancouver, ISO, and other styles

21

Coveney, Adrian, and Greg Corbett. "EGI Dataset Accounting and the WLCG." EPJ Web of Conferences 214 (2019): 03028. http://dx.doi.org/10.1051/epjconf/201921403028.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

This paper reviews the prototype dataset accounting developed during the EGI-Engage project and how it could be used to complement the view that the WLCG has of its datasets. This is a new feature of the EGI resource accounting system that will enable storing information on dataset usage such as who has accessed a dataset and how often. The new REST interface used for retrieving usage metrics from the EGI DataHub is described as well as further work that is required.

22

Saiz, P., A. Aimar, J. Andreeva, M. Babik, L. Cons, I. Dzhunov, A. Forti, et al. "WLCG Monitoring Consolidation and further evolution." Journal of Physics: Conference Series 664, no. 6 (December 23, 2015): 062054. http://dx.doi.org/10.1088/1742-6596/664/6/062054.

Full text

APA, Harvard, Vancouver, ISO, and other styles

23

Duarte, A., P. Nyczyk, A. Retico, and D. Vicinanza. "Monitoring the EGEE/WLCG grid services." Journal of Physics: Conference Series 119, no. 5 (July 1, 2008): 052014. http://dx.doi.org/10.1088/1742-6596/119/5/052014.

Full text

APA, Harvard, Vancouver, ISO, and other styles

24

Shiers, J. D. "Lessons learnt from WLCG service deployment." Journal of Physics: Conference Series 119, no. 5 (July 1, 2008): 052030. http://dx.doi.org/10.1088/1742-6596/119/5/052030.

Full text

APA, Harvard, Vancouver, ISO, and other styles

25

Gardner, Robert, Lincoln Bryant, Shawn McKee, Judith Stephen, Ilija Vukotic, Christopher Weaver, and Wenjing Wu. "Towards a NoOps Model for WLCG." EPJ Web of Conferences 245 (2020): 07024. http://dx.doi.org/10.1051/epjconf/202024507024.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

One of the most costly factors in providing a global computing infrastructure such as the WLCG is the human effort in deployment, integration, and operation of the distributed services supporting collaborative computing, data sharing and delivery, and analysis of extreme scale datasets. Furthermore, the time required to roll out global software updates, introduce new service components, or prototype novel systems requiring coordinated deployments across multiple facilities is often increased by communication latencies, staff availability, and in many cases expertise required for operations of bespoke services. While the WLCG (and distributed systems implemented throughout HEP) is a global service platform, it lacks the capability and flexibility of a modern platform-as-a-service including continuous integration/continuous delivery (CI/CD) methods, development-operations capabilities (DevOps, where developers assume a more direct role in the actual production infrastructure), and automation. Most importantly, tooling which reduces required training, bespoke service expertise, and the operational effort throughout the infrastructure, most notably at the resource endpoints (sites), is entirely absent in the current model. In this paper, we explore ideas and questions around potential NoOps models in this context: what is realistic given organizational policies and constraints? How should operational responsibility be organized across teams and facilities? What are the technical gaps? What are the social and cybersecurity challenges? Conversely what advantages does a NoOps model deliver for innovation and for accelerating the pace of delivery of new services needed for the HL-LHC era? We will describe initial work along these lines in the context of providing a data delivery network supporting IRIS-HEP DOMA R&D.

26

Xiaoyu Chen, Akram Khan, G. B. Wills, and L. H. Gilbert. "Developing Resource Usage Service in WLCG." IEEE Transactions on Nuclear Science 58, no. 2 (April 2011): 451–60. http://dx.doi.org/10.1109/tns.2011.2112775.

Full text

APA, Harvard, Vancouver, ISO, and other styles

27

Antoni, T., D. Bosio, and M. Dimou. "WLCG-specific special features in GGUS." Journal of Physics: Conference Series 219, no. 6 (April 1, 2010): 062032. http://dx.doi.org/10.1088/1742-6596/219/6/062032.

Full text

APA, Harvard, Vancouver, ISO, and other styles

28

Hanushevsky, Andrew, and Wei Yang. "XRootD S3 Gateway for WLCG Storage." EPJ Web of Conferences 295 (2024): 01057. http://dx.doi.org/10.1051/epjconf/202429501057.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

The S3 Gateway is a server based application that provides a bridge between protocols and security used in the HEP community to S3 protocols and its associated security model. This allows the use of common copy tools based on the HEP security models to store or download data from S3 based storage. This storage can reside in a public or private cloud. This paper details the motivation for implementing such a service and how it can address certain problem when dealing with storage only accessible via S3 protocol.

29

Ceccanti, Andrea, Enrico Vianello, and Diego Michelotto. "Token-based authorization in StoRM WebDAV." EPJ Web of Conferences 245 (2020): 04020. http://dx.doi.org/10.1051/epjconf/202024504020.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

At the end of May 2017 the Globus Alliance announced that the open-source Globus Toolkit (GT) would be no longer supported by the Globus team at the University of Chicago. This announcement had an obvious impact on WLCG, given the central role of the Globus Security Infrastructure (GSI) and GridFTP in the WLCG data management framework, so discussions started in the appropriate forums on the search for alternatives. At the same time, support for token-based authentication and authorization has emerged as a key requirement for storage elements powering WLCG data centers. In this contribution, we describe the work done to enable token-based authentication and authorization in the StoRM WebDAV service, describing and highlighting the differences between support for external OpenID connect providers, groupbased and capability-based authorization schemes, and locally-issued authorization tokens. We discuss how StoRM WebDAV token-based authorization is being exploited in several contexts, from WLCG DOMA activities to other scientific experiments hosted at the INFN Tier-1 data center. In this contribution, we also describe the methodology used to compare Globus GridFTP and StoRM WebDAV and we present initial results confirming how HTTP represent a viable alternative to GridFTP for data transfers also performance-wise.

30

Britton, David, Simone Campana, and Bernd Panzer-Stradel. "A holistic study of the WLCG energy needs for the LHC scientific program." EPJ Web of Conferences 295 (2024): 04001. http://dx.doi.org/10.1051/epjconf/202429504001.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

The WLCG infrastructure provides the compute power and storage capacity needed by the Large Hadron Collider (LHC) experiments at CERN. The infrastructure is distributed across over 170 data centres in more than 40 countries. The amount of energy consumed by the WLCG to support the scientific program of the LHC experiments, and its evolution, depends on different factors: the luminosity of the LHC and its operating conditions; the data volume and the data complexity; the evolving computing models and the offline software of the experiments; the ongoing R&D program in preparation for the next LHC phase (HL-LHC); the evolution of computing hardware technology towards better energy efficiency; and the modernization of the facilities hosting the data centres to improve Power Usage Effectiveness. This contribution presents a study of the WLCG energy needs and their potential evolution during the future LHC program based on the factors mentioned above. Some of the information is obtained from the CERN experience but then extrapolated to the whole of WLCG. The study provides, therefore, a holistic view for the infrastructure rather than a detailed prediction at the level of the individual facilities. It presents a clear view of the trends and offers a model for more refined studies.

31

Bockelman, Brian, Andrea Ceccanti, Fabrizio Furano, Paul Millar, Dmitry Litvintsev, and Alessandra Forti. "Third-party transfers in WLCG using HTTP." EPJ Web of Conferences 245 (2020): 04031. http://dx.doi.org/10.1051/epjconf/202024504031.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

Since its earliest days, the Worldwide LHC Computational Grid (WLCG) has relied on GridFTP to transfer data between sites. The announcement that Globus is dropping support of its open source Globus Toolkit (GT), which forms the basis for several FTP client and servers, has created an opportunity to reevaluate the use of FTP. HTTP-TPC, an extension to HTTP compatible with WebDAV, has arisen as a strong contender for an alternative approach. In this paper, we describe the HTTP-TPC protocol itself, along with the current status of its support in different implementations, and the interoperability testing done within the WLCG DOMA working group’s TPC activity. This protocol also provides the first real use-case for token-based authorisation for this community. We will demonstrate the benefits of such authorisation by showing how it allows HTTP-TPC to support new technologies (such as OAuth, OpenID Connect, Macaroons and SciTokens) without changing the protocol. We will also discuss the next steps for HTTP-TPC and the plans to use the protocol for WLCG transfers.

32

McKee, Shawn, Julia Andreeva, and Alessandra Forti. "Preparing for the next WLCG Network Data Challenge: Site Network Monitoring." EPJ Web of Conferences 295 (2024): 07039. http://dx.doi.org/10.1051/epjconf/202429507039.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

During the first WLCG Network Data Challenge in fall of 2021 (DC21) we identified shortcomings in the monitoring that impeded our ability to fully understand the results collected during the data challenge. One of the simplest missing components was site-specific network information, especially information about traffic entering and leaving any of the participating sites. Without this information, it is very difficult to understand which sites are experiencing bottlenecks or might be misconfigured or under-used based on their capacity. The WLCG Monitoring Task Force, formed at the end of 2021, was tasked with three main work areas, one of which was site network monitoring. We will describe the work carried out by the task force to enhance our knowledge of network use for WLCG by enabling site network documentation and use, the status of the deployment, and the implications for the next data challenge.

33

Andreeva, Julia, Dimitrios Christidis, Alessandro Di Girolamo, and Oliver Keeble. "WLCG space accounting in the SRM-less world." EPJ Web of Conferences 214 (2019): 04021. http://dx.doi.org/10.1051/epjconf/201921404021.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

The WLCG computing infrastructure provides distributed storage capacity hosted at the geographically dispersed computing sites. In order to effectively organize storage and processing of the LHC data, the LHC experiments require a reliable and complete overview of the storage capacity in terms of the occupied and free space, the storage shares allocated to different computing activities, and the possibility to detect “dark” data that occupies space while being unknown to the experiment's file catalogue. The task of the WLCG space accounting activity is to provide such an overview and to assist LHC experiments and WLCG operations to manage storage space and to understand future requirements. Several space accounting solutions which have been developed by the LHC experiments are currently based on Storage Resource Manager (SRM). In the coming years SRM becomes an optional service for sites which do not provide tape storage. Moreover, already now some of the storage implementations do not provide an SRM interface. Therefore, the next generation of the space accounting systems should not be based on SRM. In order to enable possibility for exposing storage topology and space accounting information the Storage Resource Reporting proposal has been agreed between LHC experiments, sites and storage providers. This contribution describes the WLCG storage resource accounting system which is being developed based on Storage Resource Reporting proposal.

34

Babik, Marian, Shawn McKee, Pedro Andrade, Brian Paul Bockelman, Robert Gardner, Edgar Mauricio Fajardo Hernandez, Edoardo Martelli, Ilija Vukotic, Derek Weitzel, and Marian Zvada. "WLCG Networks: Update on Monitoring and Analytics." EPJ Web of Conferences 245 (2020): 07053. http://dx.doi.org/10.1051/epjconf/202024507053.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

WLCG relies on the network as a critical part of its infrastructure and therefore needs to guarantee effective network usage and prompt detection and resolution of any network issues including connection failures, congestion and traffic routing. The OSG Networking Area, in partnership with WLCG, is focused on being the primary source of networking information for its partners and constituents. It was established to ensure sites and experiments can better understand and fix networking issues, while providing an analytics platform that aggregates network monitoring data with higher level workload and data transfer services. This has been facilitated by the global network of the perfSONAR instances that have been commissioned and are operated in collaboration with WLCG Network Throughput Working Group. An additional important update is the inclusion of the newly funded NSF project SAND (Service Analytics and Network Diagnosis) which is focusing on network analytics. This paper describes the current state of the network measurement and analytics platform and summarises the activities taken by the working group and our collaborators. This includes the progress being made in providing higher level analytics, alerting and alarming from the rich set of network metrics we are gathering.

35

Adye, T., B. Bockelman, K. Ellis, O. Freyermuth, F. Furano, G. Ganis, A. Hanushevsky, et al. "XRootD Third Party Copy for the WLCG and HLLHC." EPJ Web of Conferences 245 (2020): 04034. http://dx.doi.org/10.1051/epjconf/202024504034.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

A Third Party Copy (TPC) mechanism has existed in the pure XRootD storage environment for many years. However, using the XRootD TPC in the WLCG environment presents additional challenges due to the diversity of the storage systems involved such as EOS, dCache, DPM and ECHO, requiring that we carefully navigate the unique constraints imposed by these storage systems and their site-specific environments through customized configuration and software development. To support multitenant setups seen at many WLCG sites, X509 based authentication and authorization in XRootD was significantly improved to meet both security and functionality requirements. This paper presents architecture of the pull based TPC with optional X509 credential delegation, and how it is implemented in native XRootD and dCache. The paper discusses technical requirements, challenges, design choices and implementation details in the WLCG storage systems, as well as in FTS/gfal2. It also outlines XRootD’s plan to support newer TPC and security models such as token based authorization.

36

Ayllon, A. A., M. Salichos, M. K. Simon, and O. Keeble. "FTS3: New Data Movement Service For WLCG." Journal of Physics: Conference Series 513, no. 3 (June 11, 2014): 032081. http://dx.doi.org/10.1088/1742-6596/513/3/032081.

Full text

APA, Harvard, Vancouver, ISO, and other styles

37

Dykstra, D., J. Blomer, B. Blumenfeld, A. De Salvo, A. Dewhurst, and V. Verguilov. "Web Proxy Auto Discovery for the WLCG." Journal of Physics: Conference Series 898 (October 2017): 052043. http://dx.doi.org/10.1088/1742-6596/898/5/052043.

Full text

APA, Harvard, Vancouver, ISO, and other styles

38

Baranowski, Zbigniew, Lorena Lobato Pardavila, Marcin Blaszczyk, Gancho Dimitrov, and Luca Canali. "Evolution of Database Replication Technologies for WLCG." Journal of Physics: Conference Series 664, no. 4 (December 23, 2015): 042032. http://dx.doi.org/10.1088/1742-6596/664/4/042032.

Full text

APA, Harvard, Vancouver, ISO, and other styles

39

Bernier, J., S. Campana, K. Chadwick, J. Chudoba, A. Dewhurst, M. Eliáš, S. Fayer, et al. "The production deployment of IPv6 on WLCG." Journal of Physics: Conference Series 664, no. 5 (December 23, 2015): 052018. http://dx.doi.org/10.1088/1742-6596/664/5/052018.

Full text

APA, Harvard, Vancouver, ISO, and other styles

40

Gutsche, O., and C. Hajdu. "WLCG scale testing during CMS data challenges." Journal of Physics: Conference Series 119, no. 6 (July 1, 2008): 062033. http://dx.doi.org/10.1088/1742-6596/119/6/062033.

Full text

APA, Harvard, Vancouver, ISO, and other styles

41

Cons, Lionel, and Massimo Paladin. "The WLCG Messaging Service and its Future." Journal of Physics: Conference Series 396, no. 3 (December 13, 2012): 032084. http://dx.doi.org/10.1088/1742-6596/396/3/032084.

Full text

APA, Harvard, Vancouver, ISO, and other styles

42

Bockelman, Brian, Andrea Ceccanti, Ian Collier, Linda Cornwall, Thomas Dack, Jaroslav Guenther, Mario Lassnig, et al. "WLCG Authorisation from X.509 to Tokens." EPJ Web of Conferences 245 (2020): 03001. http://dx.doi.org/10.1051/epjconf/202024503001.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

The WLCG Authorisation Working Group was formed in July 2017 with the objective to understand and meet the needs of a future-looking Authentication and Authorisation Infrastructure (AAI) for WLCG experiments. Much has changed since the early 2000s when X.509 certificates presented the most suitable choice for authorisation within the grid; progress in token based authorisation and identity federation has provided an interesting alternative with notable advantages in usability and compatibility with external (commercial) partners. The need for interoperability in this new model is paramount as infrastructures and research communities become increasingly interdependent. Over the past two years, the working group has made significant steps towards identifying a system to meet the technical needs highlighted by the community during staged requirements gathering activities. Enhancement work has been possible thanks to externally funded projects, allowing existing AAI solutions to be adapted to our needs. A cornerstone of the infrastructure is the reliance on a common token schema in line with evolving standards and best practices, allowing for maximum compatibility and easy cooperation with peer infrastructures and services. We present the work of the group and an analysis of the anticipated changes in authorisation model by moving from X.509 to token based authorisation. A concrete example of token integration in Rucio is presented.

43

Simili, Emanuele, Gordon Stewart, Samuel Skipsey, Dwayne Spiteri, Albert Borbely, and David Britton. "ARMing HEP for the future Energy Efﬁciency of WLCG sites (ARM vs. x86)." EPJ Web of Conferences 295 (2024): 11007. http://dx.doi.org/10.1051/epjconf/202429511007.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

We present a case for ARM chips as an alternative to standard x86 at WLCG sites to help reduce power consumption. New measurements are presented on the performance and energy consumption of two machines (one ARM and one x86), that were otherwise similar in speciﬁcation and cost. The comparison was extended to a dual socket x86 node, representative of our site. These new results include the energy-eﬃciency and speed of singleand multithreaded jobs; the eﬀect of hyper-threading; and an initial look at clock throttling as a way of shaping power-load. We observe signiﬁcantly lower power consumption and often slightly better performance on the ARM machine and, noting the increased availability of ARM software builds from all LHC experiments and beyond, we plan to install a 2k-core ARM cluster at our WLCG Tier2 site at Glasgow in the summer of 2023. This will enable testing, physicsvalidation, and eventually an ARM production environment that will inform and inﬂuence other WLCG sites in the UK and worldwide.

44

Anisenkov, Alexey, Julia Andreeva, Alessandro Di Girolamo, Panos Paparrigopoulos, and Aresh Vedaee. "CRIC: a unified information system for WLCG and beyond." EPJ Web of Conferences 214 (2019): 03003. http://dx.doi.org/10.1051/epjconf/201921403003.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

The Worldwide LHC Computing Grid (WLCG) is an innovative distributed environment which is deployed through the use of grid computing technologiesin order to provide computing and storage resources to the LHC experimentsfor data processing and physics analysis. Following increasing demands of LHC computing needs toward high luminosity era, the experiments are engagdin an ambitious program to extend the capability of WLCG distributed environment, for instance including opportunistically used resources such as High-Performance Computers (HPCs), cloud platforms and volunteer computer. norder to be effectively used by the LHC experiments, all these diverse distributed resources should be described in detail. This implies easy service discovery of shared physical resources, detailed description of service configurations and experiment-specific data structures is needed. In this contribution, we present a high-level information component of a distributed computing environment, the Computing Resource Information Catalogue (CRIC) which aims to facilitate distributed computing operations for the LHC experiments and consolidate WLCG topology information. In addition, CRIC performs data validation and provides coherent view and topology descriptinto the LHC VOs for service discovery and configuration. CRIC represents teevolution of ATLAS Grid Information System (AGIS) into the common experiment independent high-level information framework. CRIC’s mission is to serve not just ATLAS Collaboration needs for the description of the distributed environment but any other virtual organization relying on large scale distributed infrastructure as well as the WLCG on the global scope. The contribution describes CRIC architecture, implementation of data model,collectors, user interfaces, advanced authentication and access control components of the system.

45

Currie, Robert, Wenlong Yuan, and Neofytos Themistokleous. "Exploring XRootD Optimisations Using Advanced Monitoring in the UK." EPJ Web of Conferences 295 (2024): 07019. http://dx.doi.org/10.1051/epjconf/202429507019.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

XRootD servers are commonplace to many parts of HEP data management and are a key component to data access and management strategies in both the WLCG and OSG. Deployments of XRootD instances across the UK have demonstrated the versatility and expandability of this data management software. As we become more reliant on these services, there is a requirement to collect low-level metrics to monitor service performance and behaviour. This presentation will share our recent experiences in the collection, monitoring and analysis of such metrics from servers at UK WLCG-Tier2 sites. Building on work presented at VCHEP-2021 we have been collecting XRootD service metrics from various UK sites. In addition to this, we have developed novel technologies for recording and verifying metrics from our grid storage systems at the Edinburgh Tier2. Our custom tooling is integrated with our recently deployed monitoring platform. This work complements ongoing WLCG efforts focussing on capturing and analysing XRootD based traffic flow. Through capturing and analysing additional metrics we’re able to better remotely assess service health and provide insights to assist in debugging. The goal of this is to offer a centralised resource that monitors the performance and health of remote storage services across the UK. Making use of these additional service metrics also allows us a greater insight into the performance of XRootD Proxy File Caches. By applying machine learning techniques to already collected metrics, we aim to determine if gains can potentially be made in optimising the behaviour of these caching systems in production at a UK WLCG-Tier2. The result of this potentially being the building and deployment of custom decision-making libraries for use with XRootDPFC services.

46

Tretyakov, Evgeniy, Alexey Artamonov, Maria Grigorieva, Alexei Klimentov, Shawn McKee, and Ilija Vukotic. "TRACER (TRACe route ExploRer): A tool to explore OSG/WLCG network route topologies." International Journal of Modern Physics A 36, no. 05 (February 20, 2021): 2130005. http://dx.doi.org/10.1142/s0217751x21300052.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

The experiments at the Large Hadron Collider (LHC) rely upon a complex distributed computing infrastructure (WLCG) consisting of hundreds of individual sites worldwide at universities and national laboratories, providing about half a billion computing job slots and an exabyte of storage interconnected through high speed networks. Wide Area Networking (WAN) is one of the three pillars (together with computational resources and storage) of LHC computing. More than 5 PB/day are transferred between WLCG sites. Monitoring is one of the crucial components of WAN and experiments operations. In the past years all experiments have invested significant effort to improve monitoring and integrate networking information with data management and workload management systems. All WLCG sites are equipped with perfSONAR servers to collect a wide range of network metrics. We will present the latest development to provide the 3D force directed graph visualization for data collected by perfSONAR. The visualization package allows site admins, network engineers, scientists and network researchers to better understand the topology of our Research and Education networks and it provides the ability to identify nonreliable or/and nonoptimal network paths, such as those with routing loops or rapidly changing routes.

47

Roy, Gareth, Emanuele Simili, Samuel Cadellin Skipsey, Gordon Stewart, and David Britton. "Using the Autopilot pattern to deploy container resources at a WLCG Tier-2." EPJ Web of Conferences 214 (2019): 07013. http://dx.doi.org/10.1051/epjconf/201921407013.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

Containers are becoming ubiquitous within the WLCG, with CMS announcing a requirement for its sites to provide Singularity during 2018. The ubiquity of containers means it is now possible to reify the combination of an application and its configuration into a single easy-to-deploy unit, avoiding the need to make use of a myriad of configuration management tools such as Puppet, Ansible or Salt. This allows use to be made of industry-standard devops techniques within the operations domain, such as Continuous Integration (CI) and Continuous Deployment (CD), which can lead to faster upgrades and greater system security. One interesting technique is the Autopilot pattern, which provides mechanisms for application life-cycle management which are accessible from within the container itself. Using modern service discovery techniques, each container manages its own configuration, monitors its own health, and adapts to changing requirements through the use of event triggers. In this paper, we expand on previous work to create and deploy resources to a WLCG Tier-2 via containers, and investigate the viability of using the Autopilot pattern at a WLCG site to deploy and manage computational resources.

48

Ceccanti, Andrea, Enrico Vianello, and Francesco Giacomini. "Beyond X.509: Token-based authentication and authorization in practice." EPJ Web of Conferences 245 (2020): 03021. http://dx.doi.org/10.1051/epjconf/202024503021.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

One of the key challenges identified by the HEP R&D roadmap for software and computing is the ability to integrate heterogeneous resources in support of the computing needs of HL-LHC. In order to meet this objective, a flexible Authentication and Authorization Infrastructure (AAI) has to be in place, to allow the secure composition of computing and storage resources provisioned across heterogeneous providers (e.g., Grid, private and commercial Clouds, HPC centers). At CHEP 2018, we presented how a flexible AAI based on modern, standard Web technologies (OpenID Connect, OAuth and JSON Web Tokens) and centered on the INDIGO Identity and Access Management (IAM) service could support the transition of the WLCG infrastructure to a token-based AAI. In the meanwhile, INDIGO IAM has been selected by the WLCG Management Board as the solution that will be adopted by LHC experiments, and is also at the core of the AAI envisioned to support the computing needs of the ESCAPE project. In this contribution, which represents a follow up to last-year plenary talk, we describe the work done recently on the IAM service to support WLCG requirements.

49

Sharma, Mayank, Eraldo Silva Junior, Boris Iliev Vasilev, Maarten Litmaath, and Renato Santana. "The SIMPLE Framework for deploying containerized grid services." EPJ Web of Conferences 245 (2020): 07017. http://dx.doi.org/10.1051/epjconf/202024507017.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

The Worldwide LHC Computing Grid (WLCG) currently has about 170 sites. In order to support WLCG workloads, each site has to deploy and maintain a number of possibly complex grid services. Quite often, site managers require assistance of WLCG experts, for example when new software versions need to be deployed. Modern configuration management (e.g. Puppet, Ansible), container orchestration (e.g. Docker Swarm, Kubernetes) and containerization technologies (e.g. Docker, Podman) can help make such activities more lightweight by means of packaging sensible configurations of grid services and providing simple mechanisms to distribute and deploy them across the infrastructure available at a site. This article describes the SIMPLE project: a Solution for Installation, Management and Provisioning of Lightweight Elements. The SIMPLE framework leverages modern infrastructure management tools to deploy containerized grid services, such as popular compute elements (e.g. HTCondor, ARC), batch systems (e.g. HTCondor, Slurm), worker nodes, etc. Its architecture follows principles of sustainability, scalability and extensibility. We describe how system administrators can use the framework, as well as the first results, featuring the migration of computing resources to HTCondor at 2 sites. We conclude with an outlook on further developments.

50

Geddes, Neil. "The Large Hadron Collider and Grid computing." Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 370, no. 1961 (February 28, 2012): 965–77. http://dx.doi.org/10.1098/rsta.2011.0465.

Full text

APA, Harvard, Vancouver, ISO, and other styles

Abstract:

We present a brief history of the beginnings, development and achievements of the worldwide Large Hadron Collider Computing Grid (wLCG). The wLCG is a huge international endeavour, which is itself embedded within, and directly influences, a much broader computing and information technology landscape. It is often impossible to identify true cause and effect, and they may appear very different from the different perspectives (e.g. information technology industry or academic researcher). This account is no different. It represents a personal view of the developments over the last two decades and is therefore inevitably biased towards those things in which the author has been personally involved.

Journal articles on the topic 'WLCG'

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles