Journal articles on the topic 'Remote Direct Memory Access (RDMA)'

To see the other types of publications on this topic, follow the link: Remote Direct Memory Access (RDMA).

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles

Select a source type:

Consult the top 44 journal articles for your research on the topic 'Remote Direct Memory Access (RDMA).'

Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.

You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.

Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.

1

Chen, Wei, Songping Yu, and Zhiying Wang. "Fast In-Memory Key–Value Cache System with RDMA." Journal of Circuits, Systems and Computers 28, no. 05 (May 2019): 1950074. http://dx.doi.org/10.1142/s0218126619500749.

Full text
Abstract:
The quick advances of Cloud and the advent of Fog computing impose more and more critical demand for computing and data transfer of low latency onto the underlying distributed computing infrastructure. Remote direct memory access (RDMA) technology has been widely applied for its low latency of remote data access. However, RDMA gives rise to a host of challenges in accelerating in-memory key–value stores, such as direct remote memory writes, making the remote system more vulnerable. This study presents an in-memory key–value system based on RDMA, named Craftscached, which enables: (1) buffering remote memory writes into a communication cache memory to eliminate direct remote memory writes to the data memory area; (2) dividing the communication cache memory into RDMA-writable and RDMA-readable memory zones to reduce the possibility of data corruption due to stray memory writes and caching data into an RDMA-readable memory zone to improve the remote memory read performance; and (3) adopting remote out-of-place direct memory write to achieve high performance of remote read and write. Experimental results in comparison with Memcached indicate that Craftscached provides a far better performance: (1) in the case of read-intensive workloads, the data access of Craftscached is about 7–43[Formula: see text] and 18–72.4% better than those of TCP/IP-based and RDMA-based Memcached, respectively; (2) the memory utilization of small objects is more efficient with only about 3.8% memory compaction overhead.
APA, Harvard, Vancouver, ISO, and other styles
2

Ziegler, Tobias, Viktor Leis, and Carsten Binnig. "RDMA Communciation Patterns." Datenbank-Spektrum 20, no. 3 (September 29, 2020): 199–210. http://dx.doi.org/10.1007/s13222-020-00355-7.

Full text
Abstract:
Abstract Remote Direct Memory Access (RDMA) is a networking protocol that provides high bandwidth and low latency accesses to a remote node’s main memory. Although there has been much work around RDMA, such as building libraries on top of RDMA or even applications leveraging RDMA, it remains a hard problem to identify the most suitable RDMA primitives and their combination for a given problem. While there have been some initial studies included in papers that aim to investigate selected performance characteristics of particular design choices, there has not been a systematic study to evaluate the communication patterns of scale-out systems. In this paper, we address this issue by systematically investigating how to efficiently use RDMA for building scale-out systems.
APA, Harvard, Vancouver, ISO, and other styles
3

Ziegler, Tobias, Jacob Nelson-Slivon, Viktor Leis, and Carsten Binnig. "Design Guidelines for Correct, Efficient, and Scalable Synchronization using One-Sided RDMA." Proceedings of the ACM on Management of Data 1, no. 2 (June 13, 2023): 1–26. http://dx.doi.org/10.1145/3589276.

Full text
Abstract:
Remote data structures built with one-sided Remote Direct Memory Access (RDMA) are at the heart of many disaggregated database management systems today. Concurrent access to these data structures by thousands of remote workers necessitates a highly efficient synchronization scheme. Remarkably, our investigation reveals that existing synchronization schemes display substantial variations in performance and scalability. Even worse, some schemes do not correctly synchronize, resulting in rare and hard-to-detect data corruption. Motivated by these observations, we conduct the first comprehensive analysis of one-sided synchronization techniques and provide general principles for correct synchronization using one-sided RDMA. Our research demonstrates that adherence to these principles not only guarantees correctness but also results in substantial performance enhancements.
APA, Harvard, Vancouver, ISO, and other styles
4

Gerstenberger, Robert, Maciej Besta, and Torsten Hoefler. "Enabling Highly-Scalable Remote Memory Access Programming with MPI-3 One Sided." Scientific Programming 22, no. 2 (2014): 75–91. http://dx.doi.org/10.1155/2014/571902.

Full text
Abstract:
Modern interconnects offer remote direct memory access (RDMA) features. Yet, most applications rely on explicit message passing for communications albeit their unwanted overheads. The MPI-3.0 standard defines a programming interface for exploiting RDMA networks directly, however, it's scalability and practicability has to be demonstrated in practice. In this work, we develop scalable bufferless protocols that implement the MPI-3.0 specification. Our protocols support scaling to millions of cores with negligible memory consumption while providing highest performance and minimal overheads. To arm programmers, we provide a spectrum of performance models for all critical functions and demonstrate the usability of our library and models with several application studies with up to half a million processes. We show that our design is comparable to, or better than UPC and Fortran Coarrays in terms of latency, bandwidth and message rate. We also demonstrate application performance improvements with comparable programming complexity.
APA, Harvard, Vancouver, ISO, and other styles
5

Zhu, Bohong, Youmin Chen, Qing Wang, Youyou Lu, and Jiwu Shu. "Octopus + : An RDMA-Enabled Distributed Persistent Memory File System." ACM Transactions on Storage 17, no. 3 (August 31, 2021): 1–25. http://dx.doi.org/10.1145/3448418.

Full text
Abstract:
Non-volatile memory and remote direct memory access (RDMA) provide extremely high performance in storage and network hardware. However, existing distributed file systems strictly isolate file system and network layers, and the heavy layered software designs leave high-speed hardware under-exploited. In this article, we propose an RDMA-enabled distributed persistent memory file system, Octopus + , to redesign file system internal mechanisms by closely coupling non-volatile memory and RDMA features. For data operations, Octopus + directly accesses a shared persistent memory pool to reduce memory copying overhead, and actively fetches and pushes data all in clients to rebalance the load between the server and network. For metadata operations, Octopus + introduces self-identified remote procedure calls for immediate notification between file systems and networking, and an efficient distributed transaction mechanism for consistency. Octopus + is enabled with replication feature to provide better availability. Evaluations on Intel Optane DC Persistent Memory Modules show that Octopus + achieves nearly the raw bandwidth for large I/Os and orders of magnitude better performance than existing distributed file systems.
APA, Harvard, Vancouver, ISO, and other styles
6

Koo, Bonmoo, Jaesang Hwang, Jonghyeok Park, and Wook-Hee Kim. "Converting Concurrent Range Index Structure to Range Index Structure for Disaggregated Memory." Applied Sciences 13, no. 20 (October 10, 2023): 11130. http://dx.doi.org/10.3390/app132011130.

Full text
Abstract:
In this work, we propose the Spread approach, which tailors a concurrent range index structure to a range index structure for disaggregated memory connected via RDMA (Remote Direct Memory Access). The Spread approach leverages the concept of tolerating transient inconsistencies in a concurrent range index structure to reduce the amount of expensive RDMA operations. Based on the Spread approach, we converted Blink-tree, a concurrent range index structure, to a range index structure for disaggregated memory called RF-tree. In our experimental study, RF-tree shows comparable performance to Sherman, a state-of-the-art and carefully crafted range index structure for disaggregated memory.
APA, Harvard, Vancouver, ISO, and other styles
7

Hemmatpour, Masoud, Bartolomeo Montrucchio, and Maurizio Rebaudengo. "Communicating Efficiently on Cluster-Based Remote Direct Memory Access (RDMA) over InfiniBand Protocol." Applied Sciences 8, no. 11 (October 24, 2018): 2034. http://dx.doi.org/10.3390/app8112034.

Full text
Abstract:
Distributed systems are commonly built under the assumption that the network is the primary bottleneck, however this assumption no longer holds by emerging high-performance RDMA enabled protocols in datacenters. Designing distributed applications over such protocols requires a fundamental rethinking in communication components in comparison with traditional protocols (i.e., TCP/IP). In this paper, communication paradigms in existing systems and new possible paradigms have been investigated. Advantages and drawbacks of each paradigm have been comprehensively analyzed and experimentally evaluated. The experimental results show that writing the requests to server and reading the response presents up to 10 times better performance comparing to other communication paradigms. To further expand the investigation, the proposed communication paradigm has been substituted in a real-world distributed application, and the performance has been enhanced up to seven times.
APA, Harvard, Vancouver, ISO, and other styles
8

Wang, Zhonghua, Yixing Guo, Kai Lu, Jiguang Wan, Daohui Wang, Ting Yao, and Huatao Wu. "Rcmp: Reconstructing RDMA-Based Memory Disaggregation via CXL." ACM Transactions on Architecture and Code Optimization 21, no. 1 (January 19, 2024): 1–26. http://dx.doi.org/10.1145/3634916.

Full text
Abstract:
Memory disaggregation is a promising architecture for modern datacenters that separates compute and memory resources into independent pools connected by ultra-fast networks, which can improve memory utilization, reduce cost, and enable elastic scaling of compute and memory resources. However, existing memory disaggregation solutions based on remote direct memory access (RDMA) suffer from high latency and additional overheads including page faults and code refactoring. Emerging cache-coherent interconnects such as CXL offer opportunities to reconstruct high-performance memory disaggregation. However, existing CXL-based approaches have physical distance limitation and cannot be deployed across racks. In this article, we propose Rcmp, a novel low-latency and highly scalable memory disaggregation system based on RDMA and CXL. The significant feature is that Rcmp improves the performance of RDMA-based systems via CXL, and leverages RDMA to overcome CXL’s distance limitation. To address the challenges of the mismatch between RDMA and CXL in terms of granularity, communication, and performance, Rcmp (1) provides a global page-based memory space management and enables fine-grained data access, (2) designs an efficient communication mechanism to avoid communication blocking issues, (3) proposes a hot-page identification and swapping strategy to reduce RDMA communications, and (4) designs an RDMA-optimized RPC framework to accelerate RDMA transfers. We implement a prototype of Rcmp and evaluate its performance by using micro-benchmarks and running a key-value store with YCSB benchmarks. The results show that Rcmp can achieve 5.2× lower latency and 3.8× higher throughput than RDMA-based systems. We also demonstrate that Rcmp can scale well with the increasing number of nodes without compromising performance.
APA, Harvard, Vancouver, ISO, and other styles
9

Chen, Hongzhi, Changji Li, Chenguang Zheng, Chenghuan Huang, Juncheng Fang, James Cheng, and Jian Zhang. "G-tran." Proceedings of the VLDB Endowment 15, no. 11 (July 2022): 2545–58. http://dx.doi.org/10.14778/3551793.3551813.

Full text
Abstract:
Graph transaction processing poses unique challenges such as random data access due to the irregularity of graph structures, low throughput and high abort rate due to the relatively large read/write sets in graph transactions. To address these challenges, we present G-Tran, a remote direct memory access (RDMA)-enabled distributed in-memory graph database with serializable and snapshot isolation support. First, we propose a graph-native data store to achieve good data locality and fast data access for transactional updates and queries. Second, G-Tran adopts a fully decentralized architecture that leverages RDMA to process distributed transactions with the massively parallel processing (MPP) model, which can achieve high performance by utilizing all computing resources. In addition, we propose a new multi-version optimistic concurrency control (MV-OCC) protocol with two optimizations to address the issue of large read/write sets in graph transactions. Extensive experiments show that G-Tran achieves competitive performance compared with other popular graph databases on benchmark workloads.
APA, Harvard, Vancouver, ISO, and other styles
10

Wei, Xingda, Rong Chen, Haibo Chen, and Binyu Zang. "XStore : Fast RDMA-Based Ordered Key-Value Store Using Remote Learned Cache." ACM Transactions on Storage 17, no. 3 (August 31, 2021): 1–32. http://dx.doi.org/10.1145/3468520.

Full text
Abstract:
RDMA ( Remote Direct Memory Access ) has gained considerable interests in network-attached in-memory key-value stores. However, traversing the remote tree-based index in ordered key-value stores with RDMA becomes a critical obstacle, causing an order-of-magnitude slowdown and limited scalability due to multiple round trips. Using index cache with conventional wisdom—caching partial data and traversing them locally—usually leads to limited effect because of unavoidable capacity misses, massive random accesses, and costly cache invalidations. We argue that the machine learning (ML) model is a perfect cache structure for the tree-based index, termed learned cache . Based on it, we design and implement XStore , an RDMA-based ordered key-value store with a new hybrid architecture that retains a tree-based index at the server to perform dynamic workloads (e.g., inserts) and leverages a learned cache at the client to perform static workloads (e.g., gets and scans). The key idea is to decouple ML model retraining from index updating by maintaining a layer of indirection from logical to actual positions of key-value pairs. It allows a stale learned cache to continue predicting a correct position for a lookup key. XStore ensures correctness using a validation mechanism with a fallback path and further uses speculative execution to minimize the cost of cache misses. Evaluations with YCSB benchmarks and production workloads show that a single XStore server can achieve over 80 million read-only requests per second. This number outperforms state-of-the-art RDMA-based ordered key-value stores (namely, DrTM-Tree, Cell, and eRPC+Masstree) by up to 5.9× (from 3.7×). For workloads with inserts, XStore still provides up to 3.5× (from 2.7×) throughput speedup, achieving 53M reqs/s. The learned cache can also reduce client-side memory usage and further provides an efficient memory-performance tradeoff, e.g., saving 99% memory at the cost of 20% peak throughput.
APA, Harvard, Vancouver, ISO, and other styles
11

Li, Shibao, Chengzhi Wang, Yunwu Zhang, Chenxu Ma, Longfei Li, Xuerong Cui, and Jianhang Liu. "FG-PFC: A Fine-Grained PFC Mechanism for Lossless RDMA." Journal of Physics: Conference Series 2575, no. 1 (August 1, 2023): 012008. http://dx.doi.org/10.1088/1742-6596/2575/1/012008.

Full text
Abstract:
Abstract Remote Direct Memory Access (RDMA) is widely deployed in data centers to improve the performance, efficiency, and reliability of data centers. Priority-based Flow Control (PFC) can ensure no packet loss as a necessary approach to achieve lossless RDMA. However, since PFC is a coarse-grained flow control mechanism based on port and priority, it can cause the head-of-line blocking problem and affect other flows, resulting in high latency. In this paper, we propose a fine-grained PFC mechanism (FG-PFC), which refines port and priority into flow granularity and further determines and pauses the flow most likely to cause congestion. The benefit is reducing the persistent high occupancy of the buffer and protecting the innocent flow. Simulations and experiments demonstrate that FG-PFC outperforms PFC in terms of flow completion time (FCT).
APA, Harvard, Vancouver, ISO, and other styles
12

Geng, Junjie, Jinyao Yan, and Yuan Zhang. "P4QCN: Congestion Control Using P4-Capable Device in Data Center Networks." Electronics 8, no. 3 (March 2, 2019): 280. http://dx.doi.org/10.3390/electronics8030280.

Full text
Abstract:
Modern data centers aim to offer very high throughput and ultra-low latency to meet the demands of applications such as online intensive services. Traditional TCP/IP stacks cannot meet these requirements due to their high CPU overhead and high-latency. Remote Direct Memory Access (RDMA) is an approach that can be designed to meet this demand. The mainstream transport protocol of RDMA over Ethernet is RoCE (RDMA over Converged Ethernet), which relies on Priority Flow Control (PFC) within the network to enable a lossless network. However, PFC is a coarse-grained protocol which can lead to problems such as congestion spreading, head-of-the-line blocking. A congestion control protocol that can alleviate these problems of PFC is needed. We propose a protocol, called P4QCN for this purpose. P4QCN is a congestion control scheme for RoCE and it is an improved Quantized Congestion Notification (QCN) design based on P4, which is a flow-level, rate-based congestion control mechanism. P4QCN extends the QCN protocol to make it compatible with IP-routed networks based on a framework of P4 and adopts a two-point algorithm architecture which is more effective than the three-point architecture used in QCN and Data Center QCN(DCQCN). Experiments show that our proposed P4QCN algorithm achieves the expected performance in terms of latency and throughput.
APA, Harvard, Vancouver, ISO, and other styles
13

Gil, Myeong-Seon, and Yang-Sae Moon. "SPinDP: A High-Speed Distributed Processing Platform for Sampling and Filtering Data Streams." Applied Sciences 13, no. 24 (December 5, 2023): 12998. http://dx.doi.org/10.3390/app132412998.

Full text
Abstract:
Recently, there has been an explosive generation of streaming data in various fields such as IoT and network attack detection, medical data monitoring, and financial trend analysis. These domains require precise and rapid analysis capabilities by minimizing noise from continuously generated raw data. In this paper, we propose SPinDP (Stream Purifier in Distributed Platform), an open source-based high-speed stream purification platform, to support real-time stream purification. SPinDP consists of four major components, Data Stream Processing Engine, Purification Library, Plan Manager, and Shared Storage, and operates based on open-source systems including Apache Storm and Apache Kafka. In these components, stream processing throughput and latency are critical performance metrics, and SPinDP significantly enhances distributed processing performance by utilizing the ultra-high-speed network RDMA (Remote Direct Memory Access). For the performance evaluation, we use a distributed cluster environment consisting of nine nodes, and we show that SPinDP’s stream processing throughput is more than 28 times higher than that of the existing Ethernet environment. SPinDP also significantly reduces the processing latency by more than 2473 times on average. These results indicate that the proposed SPinDP is an excellent integrated platform that can efficiently purify high-speed and large-scale streams through RDMA-based distributed processing.
APA, Harvard, Vancouver, ISO, and other styles
14

KEE, YANGSUK, and SOONHOI HA. "AN EFFICIENT IMPLEMENTATION OF THE BSP PROGRAMMING LIBRARY FOR VIA." Parallel Processing Letters 12, no. 01 (March 2002): 65–77. http://dx.doi.org/10.1142/s0129626402000835.

Full text
Abstract:
Virtual Interface Architecture(VIA) is a light-weight protocol for protected user-level zero-copy communication. In spite of the promised high performance of VIA, previous MPI implementations for GigaNet's cLAN revealed low communication performance. Two main sources of such low performance are the discrepancy in the communication model between MPI and VIA and the multi-threading overhead. In this paper, we propose a new implementation of the Bulk Synchronous Parallel(BSP) programming library for VIA called xBSP to overcome such problems. To the best of our knowledge, xBSP is the first implementation of the BSP library for VIA. xBSP demonstrates that the selection of a proper library is important to exploit the features of light-weight protocols. Intensive use of Remote Direct Memory Access(RDMA) operations leads to high performance close to the native VIA performance with respect to round trip delay and bandwidth. Considering the effects of multi-threading, memory registration, and completion policy on performance, we could obtain an efficient BSP implementation for cLAN, which was confirmed by experimental results.
APA, Harvard, Vancouver, ISO, and other styles
15

Krawczyk, Rafał Dominik, Tommaso Colombo, Niko Neufeld, Flavio Pisani, and Sébastien Valat. "Feasibility tests of RoCE v2 for LHCb event building." EPJ Web of Conferences 245 (2020): 01011. http://dx.doi.org/10.1051/epjconf/202024501011.

Full text
Abstract:
This paper evaluates the utilization of Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) for the Run 3 LHCb event building at CERN. The acquisition system of the detector will collect partial data from approximately 1000 separate detector streams. The total estimated throughput equals 32 Terabits per second. Full events will be assembled for subsequent processing and data selection in the filtering farm of the online trigger. High-throughput transmissions with up to 90% links utilization will be an essential feature of the system. The data exchange mechanism must support zero-copy transmissions. In this work, the RoCE high-throughput kernel bypass Ethernet protocol is benchmarked as a potential alternative to InfiniBand. A RoCE-based event building network is presented and two implementations are considered. The former variant combined shallow-buffered and deep-buffered switches with enabled flow control. In the latter setup, only deep-buffered devices are used, where operation relied on their memory throughput and capacity. Feasibility tests were conducted with selected Ethernet switches. Memory bandwidth utilization was investigated, in comparison with InfiniBand. Relevant utilization and interoperability issues of RoCE flow control are detailed with lessons learned along the road.
APA, Harvard, Vancouver, ISO, and other styles
16

Geetha J., Uday Bhaskar N, and Chenna Reddy P. "An Analytical Approach for Optimizing the Performance of Hadoop Map Reduce Over RoCE." International Journal of Information Communication Technologies and Human Development 10, no. 2 (April 2018): 1–14. http://dx.doi.org/10.4018/ijicthd.2018040101.

Full text
Abstract:
Data intensive systems aim to efficiently process “big” data. Several data processing engines have evolved over past decade. These data processing engines are modeled around the MapReduce paradigm. This article explores Hadoop's MapReduce engine and propose techniques to obtain a higher level of optimization by borrowing concepts from the world of High Performance Computing. Consequently, power consumed and heat generated is lowered. This article designs a system with a pipelined dataflow in contrast to the existing unregulated “bursty” flow of network traffic, the ability to carry out both Map and Reduce tasks in parallel, and a system which incorporates modern high-performance computing concepts using Remote Direct Memory Access (RDMA). To establish the claim of an increased performance measure of the proposed system, the authors provide an algorithm for RoCE enabled MapReduce and a mathematical derivation contrasting the runtime of vanilla Hadoop. This article proves mathematically, that the proposed system functions 1.67 times faster than the vanilla version of Hadoop.
APA, Harvard, Vancouver, ISO, and other styles
17

Barghash, Ahmad, Lina Hammad, and Ammar Gharaibeh. "Traditional vs. Modern Data Paths: A Comprehensive Survey." Computers 11, no. 9 (August 31, 2022): 132. http://dx.doi.org/10.3390/computers11090132.

Full text
Abstract:
Recently, many new network paths have been introduced while old paths are still in use. The trade-offs remain vague and should be further addressed. Since last decade, the Internet is playing a major role in people’s lives, and the demand on the Internet in all fields has increased rapidly. In order to get a fast and secure connection to the Internet, the networks providing the service should get faster and more reliable. Many network data paths have been proposed in order to achieve the previous objectives since the 1970s. It started with the Transmission Control Protocol (TCP) and the User Datagram Protocol (UDP) and later followed by several more modern paths including Quick UDP Internet Connections (QUIC), remote direct memory access (RDMA), and the Data Plane Development Kit (DPDK). This raised the question on which data path should be adopted and based on which features. In this work, we try to answer this question using different perspectives such as the protocol techniques, latency and congestion control, head of line blocking, the achieved throughput, middleboxes consideration, loss recovery mechanisms, developer productivity, host resources utilization and targeted application.
APA, Harvard, Vancouver, ISO, and other styles
18

Ponsard, Raphael, Nicolas Janvier, Jerome Kieffer, Dominique Houzet, and Vincent Fristot. "RDMA data transfer and GPU acceleration methods for high-throughput online processing of serial crystallography images." Journal of Synchrotron Radiation 27, no. 5 (July 31, 2020): 1297–306. http://dx.doi.org/10.1107/s1600577520008140.

Full text
Abstract:
The continual evolution of photon sources and high-performance detectors drives cutting-edge experiments that can produce very high throughput data streams and generate large data volumes that are challenging to manage and store. In these cases, efficient data transfer and processing architectures that allow online image correction, data reduction or compression become fundamental. This work investigates different technical options and methods for data placement from the detector head to the processing computing infrastructure, taking into account the particularities of modern modular high-performance detectors. In order to compare realistic figures, the future ESRF beamline dedicated to macromolecular X-ray crystallography, EBSL8, is taken as an example, which will use a PSI JUNGFRAU 4M detector generating up to 16 GB of data per second, operating continuously during several minutes. Although such an experiment seems possible at the target speed with the 100 Gb s−1 network cards that are currently available, the simulations generated highlight some potential bottlenecks when using a traditional software stack. An evaluation of solutions is presented that implements remote direct memory access (RDMA) over converged ethernet techniques. A synchronization mechanism is proposed between a RDMA network interface card (RNIC) and a graphics processing unit (GPU) accelerator in charge of the online data processing. The placement of the detector images onto the GPU is made to overlap with the computation carried out, potentially hiding the transfer latencies. As a proof of concept, a detector simulator and a backend GPU receiver with a rejection and compression algorithm suitable for a synchrotron serial crystallography (SSX) experiment are developed. It is concluded that the available transfer throughput from the RNIC to the GPU accelerator is at present the major bottleneck in online processing for SSX experiments.
APA, Harvard, Vancouver, ISO, and other styles
19

Zhuge, Qingfeng, Hao Zhang, Edwin Hsing-Mean Sha, Rui Xu, Jun Liu, and Shengyu Zhang. "Exploring Efficient Architectures on Remote In-Memory NVM over RDMA." ACM Transactions on Embedded Computing Systems 20, no. 5s (October 31, 2021): 1–20. http://dx.doi.org/10.1145/3477004.

Full text
Abstract:
Efficiently accessing remote file data remains a challenging problem for data processing systems. Development of technologies in non-volatile dual in-line memory modules (NVDIMMs), in-memory file systems, and RDMA networks provide new opportunities towards solving the problem of remote data access. A general understanding about NVDIMMs, such as Intel Optane DC Persistent Memory (DCPM), is that they expand main memory capacity with a cost of multiple times lower performance than DRAM. With an in-depth exploration presented in this paper, however, we show an interesting finding that the potential of NVDIMMs for high-performance, remote in-memory accesses can be revealed through careful design. We explore multiple architectural structures for accessing remote NVDIMMs in a real system using Optane DCPM, and compare the performance of various structures. Experiments are conducted to show significant performance gaps among different ways of using NVDIMMs as memory address space accessible through RDMA interface. Furthermore, we design and implement a prototype of user-level, in-memory file system, RIMFS, in the device DAX mode on Optane DCPM. By comparing against the DAX-supported Linux file system, Ext4-DAX, we show that the performance of remote reads on RIMFS over RDMA is 11.44 higher than that on a remote Ext4-DAX on average. The experimental results also show that the performance of remote accesses on RIMFS is maintained on a heavily loaded data server with CPU utilization as high as 90%, while the performance of remote reads on Ext4-DAX is significantly reduced by 49.3%, and the performance of local reads on Ext4-DAX is even more significantly reduced by 90.1%. The performance comparisons of writes exhibit the same trends.
APA, Harvard, Vancouver, ISO, and other styles
20

Zhang, Yingqiang, Chaoyi Ruan, Cheng Li, Xinjun Yang, Wei Cao, Feifei Li, Bo Wang, et al. "Towards cost-effective and elastic cloud database deployment via memory disaggregation." Proceedings of the VLDB Endowment 14, no. 10 (June 2021): 1900–1912. http://dx.doi.org/10.14778/3467861.3467877.

Full text
Abstract:
It is challenging for cloud-native relational databases to meet the ever-increasing needs of scaling compute and memory resources independently and elastically. The recent emergence of memory disaggregation architecture, relying on high-speed RDMA network, offers opportunities to build cost-effective and elastic cloud-native databases. There exist proposals to let unmodified applications run transparently on disaggregated systems. However, running relational database kernel atop such proposals experiences notable performance degradation and time-consuming failure recovery, offsetting the benefits of disaggregation. To address these challenges, in this paper, we propose a novel database architecture called LegoBase, which explores the co-design of database kernel and memory disaggregation. It pushes the memory management back to the database layer for bypassing the Linux I/O stack and re-using or designing (remote) memory access optimizations with an understanding of data access patterns. LegoBase further splits the conventional ARIES fault tolerance protocol to independently handle the local and remote memory failures for fast recovery of compute instances. We implemented LegoBase atop MySQL. We compare LegoBase against MySQL running on a standalone machine and the state-of-the-art disaggregation proposal Infiniswap. Our evaluation shows that even with a large fraction of data placed on the remote memory, LegoBase's system performance in terms of throughput (up to 9.41% drop) and P99 latency (up to 11.58% increase) is comparable to the monolithic MySQL setup, and significantly outperforms (1.99x-2.33x, respectively) the deployment of MySQL over Infiniswap. Meanwhile, LegoBase introduces an up to 3.87x and 5.48x speedup of the recovery and warm-up time, respectively, over the monolithic MySQL and MySQL over Infiniswap, when handling failures or planned re-configurations.
APA, Harvard, Vancouver, ISO, and other styles
21

Nyrkov, Anatoliy, Konstantin Ianiushkin, Andrey Nyrkov, Yulia Romanova, and Vagiz Gaskarov. "Data structures access model for remote shared memory." E3S Web of Conferences 244 (2021): 07001. http://dx.doi.org/10.1051/e3sconf/202124407001.

Full text
Abstract:
Recent achievements in high-performance computing significantly narrow the performance gap between single and multi-node computing, and open up opportunities for systems with remote shared memory. The combination of in-memory storage, remote direct memory access and remote calls requires rethinking how data organized, protected and queried in distributed systems. Reviewed models let us implement new interpretations of distributed algorithms allowing us to validate different approaches to avoid race conditions, decrease resource acquisition or synchronization time. In this paper, we describe the data model for mixed memory access with analysis of optimized data structures. We also provide the result of experiments, which contain a performance comparison of data structures, operating with different approaches, evaluate the limitations of these models, and show that the model does not always meet expectations. The purpose of this paper to assist developers in designing data structures that will help to achieve architectural benefits or improve the design of existing distributed system.
APA, Harvard, Vancouver, ISO, and other styles
22

Hoefler, Torsten, Duncan Roweth, Keith Underwood, Robert Alverson, Mark Griswold, Vahid Tabatabaee, Mohan Kalkunte, et al. "Data Center Ethernet and Remote Direct Memory Access: Issues at Hyperscale." Computer 56, no. 7 (July 2023): 67–77. http://dx.doi.org/10.1109/mc.2023.3261184.

Full text
APA, Harvard, Vancouver, ISO, and other styles
23

Stavrakakis, Dimitrios, Dimitra Giantsidi, Maurice Bailleu, Philip Sändig, Shady Issa, and Pramod Bhatotia. "Anchor: A Library for Building Secure Persistent Memory Systems." Proceedings of the ACM on Management of Data 1, no. 4 (December 8, 2023): 1–31. http://dx.doi.org/10.1145/3626718.

Full text
Abstract:
Cloud infrastructure is experiencing a shift towards disaggregated setups, especially with the introduction of the Compute Express Link (CXL) technology, where byte-addressable ersistent memory (PM) is becoming prominent. To fully utilize the potential of such devices, it is a necessity to access them through network stacks with equivalently high levels of performance (e.g., kernel-bypass, RDMA). While, these advancements are enabling the development of high-performance data management systems, their deployment on untrusted cloud environments also increases the security threats. To this end, we present Anchor, a library for building secure PM systems. Anchor provides strong hardware-assisted security properties, while ensuring crash consistency. Anchor exposes APIs for secure data management within the realms of the established PM programming model, targeting byte-addressable storage devices. Anchor leverages trusted execution environments (TEE) and extends their security properties on PM. While TEE's protected memory region provides a strong foundation for building secure systems, the key challenge is that: TEEs are fundamentally incompatible with PM and kernel-bypass networking approaches-in particular, TEEs are neither designed to protect untrusted non-volatile PM, nor the protected region can be accessed via an untrusted DMA connection. To overcome this challenge, we design a PM engine that ensures strong security properties for the PM data, using confidential and authenticated PM data structures, while preserving crash consistency through a secure logging protocol. We further extend the PM engine to provide remote PM data operations via a secure network stack and a formally verified remote attestation protocol to form an end-to-end system. Our evaluation shows that Anchor incurs reasonable overheads, while providing strong security properties.
APA, Harvard, Vancouver, ISO, and other styles
24

Rybintsev, Vladimir O. "Estimating the Performance of Computing Clusters without Accelerators Based on TOP500 Results." Mathematics 10, no. 19 (September 30, 2022): 3580. http://dx.doi.org/10.3390/math10193580.

Full text
Abstract:
Based on an analysis of TOP500 results, a functional dependence of the performance of clusters without accelerators according to the Linpack benchmark on their parameters was determined. The comparison of calculated and tested results showed that the estimation error does not exceed 2% for processors of different generations and manufacturers (Intel, AMD, Fujitsu) with different technologies of a system interconnect. The achieved accuracy of the calculation allows successful prediction of the performance of a cluster when its parameters (node performance, number of nodes, number of network interfaces, network technology, remote direct memory access, or remote direct memory access over converged Ethernet mode) are changed without resorting to a complex procedure of real testing.
APA, Harvard, Vancouver, ISO, and other styles
25

Cilardo, Alessandro. "Evaluation of HPC Acceleration and Interconnect Technologies for High-Throughput Data Acquisition." Sensors 21, no. 22 (November 22, 2021): 7759. http://dx.doi.org/10.3390/s21227759.

Full text
Abstract:
Efficient data movement in multi-node systems is a crucial issue at the crossroads of scientific computing, big data, and high-performance computing, impacting demanding data acquisition applications from high-energy physics to astronomy, where dedicated accelerators such as FPGA devices play a key role coupled with high-performance interconnect technologies. Building on the outcome of the RECIPE Horizon 2020 research project, this work evaluates the use of high-bandwidth interconnect standards, namely InfiniBand EDR and HDR, along with remote direct memory access functions for direct exposure of FPGA accelerator memory across a multi-node system. The prototype we present aims at avoiding dedicated network interfaces built in the FPGA accelerator itself, leaving most of the resources for user acceleration and supporting state-of-the-art interconnect technologies. We present the detail of the proposed system and a quantitative evaluation in terms of end-to-end bandwidth as concretely measured with a real-world FPGA-based multi-node HPC workload.
APA, Harvard, Vancouver, ISO, and other styles
26

Jang, Hankook, Sang-Hwa Chung, and Dae-Hyun Yoo. "Design and implementation of a protocol offload engine for TCP/IP and remote direct memory access based on hardware/software coprocessing." Microprocessors and Microsystems 33, no. 5-6 (August 2009): 333–42. http://dx.doi.org/10.1016/j.micpro.2009.03.001.

Full text
APA, Harvard, Vancouver, ISO, and other styles
27

Kalney, Marina S. "The role of humanities in the course of transmitting social memory." SHS Web of Conferences 103 (2021): 01019. http://dx.doi.org/10.1051/shsconf/202110301019.

Full text
Abstract:
The article examines the contradiction between the transformation of information technology into an integral part of contemporary educational activities and the threat of reducing the quality of the educational process and dehumanizing the individual, resulting from the implementation of digital educational technologies. The need for distance work and remote communication is particularly evident in the situation of the pandemic, which requires, on the one hand, restricting direct interpersonal contacts to prevent the spread of infection, and on the other hand – continuing labor activity to prevent an economic downturn. The disadvantages of the distance framework in the educational process concern a decrease in discipline among students, a deterioration in the quality of learning of educational material, economic inaccessibility for low-income families, insufficient coverage of territories with cellular communication, as well as housing conditions that do not allow organizing remote work and training. These shortcomings become a threat to increasing social stratification due to the impossibility of general access to even basic education. The essence of the protests against distance learning shows that the problems are seen not only in the lack of organization of the learning process but also in the very fact of abandoning the traditional learning model. One of the factors of such a threat is the absolutization of the role of information technology in the learning process, which leads to a deterioration in the level of education. To resolve this contradiction, the authors propose to consider the role of humanitarian knowledge as a meaningful aspect of the educational process.
APA, Harvard, Vancouver, ISO, and other styles
28

Ye, Yuejin, Zhenya Song, Shengchang Zhou, Yao Liu, Qi Shu, Bingzhuo Wang, Weiguo Liu, Fangli Qiao, and Lanning Wang. "swNEMO_v4.0: an ocean model based on NEMO4 for the new-generation Sunway supercomputer." Geoscientific Model Development 15, no. 14 (July 25, 2022): 5739–56. http://dx.doi.org/10.5194/gmd-15-5739-2022.

Full text
Abstract:
Abstract. The current large-scale parallel barrier of ocean general circulation models (OGCMs) makes it difficult to meet the computing demand of high resolution. Fully considering both the computational characteristics of OGCMs and the heterogeneous many-core architecture of the new Sunway supercomputer, swNEMO_v4.0, based on NEMO4 (Nucleus for European Modelling of the Ocean version 4), is developed with ultrahigh scalability. Three innovations and breakthroughs are shown in our work: (1) a highly adaptive, efficient four-level parallelization framework for OGCMs is proposed to release a new level of parallelism along the compute-dependency column dimension. (2) A many-core optimization method using blocking by remote memory access (RMA) and a dynamic cache scheduling strategy is applied, effectively utilizing the temporal and spatial locality of data. The test shows that the actual direct memory access (DMA) bandwidth is greater than 90 % of the ideal band-width after optimization, and the maximum is up to 95 %. (3) A mixed-precision optimization method with half, single and double precision is explored, which can effectively improve the computation performance while maintaining the simulated accuracy of OGCMs. The results demonstrate that swNEMO_v4.0 has ultrahigh scalability, achieving up to 99.29 % parallel efficiency with a resolution of 500 m using 27 988 480 cores, reaching the peak performance with 1.97 PFLOPS.
APA, Harvard, Vancouver, ISO, and other styles
29

He, Chun Lin. "The Design and Implementation of the Information Remote Monitoring and Security Management System Based on Internet." Advanced Materials Research 846-847 (November 2013): 1414–17. http://dx.doi.org/10.4028/www.scientific.net/amr.846-847.1414.

Full text
Abstract:
With the development of economy, the existence and application of the inflammable, explosive, toxic and harmful gases irresistible but dangerous at the same time in the industrial production application. for the sake of minimizing potential dangers, the remote security monitoring is of great significance. With the rapid development of network technology, the Internet network technology to implement the remote monitoring can effectively avoid some potential hazards in the industrial production and other dangerous things. We can draw a conclusion from the feedback that the effect of user experience is not ideal in the actual application. So based on the analysis of the traditional structure of remote monitoring system, a new remote monitoring system is created which can avoid these shortcomings through further research. This new system consists of three parts: 1. The client: using Flex rich client technology is more convenient and suitable for communication interface and let the customer feel familiar and easy to use. 2. data management layer: it contains three parts : the Web server layer, application server and database server layer . Application server promotes system real-time performance through Web Service technology, Shared memory and streaming Socket technology, which enables clients to have direct access to the site. 3. Data acquisition layer: data will be collected in a timely manner to send to data management with GPRS technology.
APA, Harvard, Vancouver, ISO, and other styles
30

Cai, Yan, Wenlong Xie, and Haihua Zhang. "Remote distributed monitoring system of switched reluctance motor." Measurement and Control 52, no. 3-4 (March 2019): 276–90. http://dx.doi.org/10.1177/0020294019836111.

Full text
Abstract:
The reliable operation, dynamic performance analysis and control strategy research of a switched reluctance motor (SRM) require an online monitoring system to display and record its operating status. However, due to the large amount of data, the nonlinear electromagnetic characteristics and the harsh working environment of SRM, it is very difficult to monitor a SRM’s operation status in real time. In order to solve these problems, a new structure of the SRM monitoring system, which uses Digital Signal Processor (DSP) and the hardwired Transmission Control Protocol/Internet Protocol embedded Ethernet controller W5500, is presented in this paper. The W5500 and DSP’s direct memory access modules are employed for data capture and transfer to reduce the digital signal processing workload. The digital signal processing program is implemented by the hybrid programming method, which shortens the filtering time. Consequently, the DSP has sufficient resources to acquire multiple signals with a high sampling frequency and can adopt a more complex filtering algorithm, which enhances the accuracy and real-time performance of the system. Moreover, the amplitude–frequency characteristics of signals are analyzed. Then, the detection circuits and finite impulse response filters are designed to achieve the targeted acquisition and filtering. Besides, the impact of harsh environment on the system is reduced by adjusting the data transmission modes according to different working conditions. As a result, the scope of application of the system has been extended. The proposed system has a novel structure and strong practicability, which exhibits great guiding significance for the development of a SRM.
APA, Harvard, Vancouver, ISO, and other styles
31

Saeed, Dr Abdul Razzaq Ahmed. "Geographical and modern technologies." ALUSTATH JOURNAL FOR HUMAN AND SOCIAL SCIENCES 216, no. 2 (March 1, 2016): 29–38. http://dx.doi.org/10.36473/ujhss.v216i2.589.

Full text
Abstract:
It is the modern techniques that are used in modern geographical science in scientific applications three Systems is a technology GIS Gis (Geograbhical InFormation System), sensor system technology remote R.S (Remoote Sensing), GPS system technology (Global Positioning System) These three systems contributed to the great scientific revolution in all geographic modern science and its applications, as the GIS GIS technology is a way to organize or style of geographical and non-geographical information by computer and linked to geographical their positions depending on the specific coordinates. Coordinates are therefore a way to link the geographical phenomena scattered on the surface of the ground coordinates of the system and stored in computer memory and link the metadata associated with these phenomena through a database and analyzed and reflected a specific scale, and then print them The sensor system technology remote RS and its use in modern applications in geographical science is represented a set of processes that allow access to information for some geographical characteristics of the phenomena on the surface of the earth without no direct contact between the geographical phenomenon and the sensor (capture device information). Can be arranged remote sensors on a wide variety of platforms air or space and at different heights, turning the initial information received by the sensor either to directly usable products such as photographs air or space visualizations or store this information in a private devices can refer to it when needed in the future
APA, Harvard, Vancouver, ISO, and other styles
32

Krawczyk, Rafał Dominik, Flavio Pisani, Tommaso Colombo, Markus Frank, and Niko Neufeld. "Ethernet evaluation in data distribution traffic for the LHCb filtering farm at CERN." EPJ Web of Conferences 251 (2021): 04001. http://dx.doi.org/10.1051/epjconf/202125104001.

Full text
Abstract:
This paper evaluates the real-time distribution of data over Ethernet for the upgraded LHCb data acquisition cluster at CERN. The system commissioning ends in 2021 and its total estimated input throughput is 32 Terabits per second. After the events are assembled, they must be distributed for further data selection to the filtering farm of the online trigger. High-throughput and very low overhead transmissions will be an essential feature of such a system. In this work RoCE (Remote Direct Memory Access over Converged Ethernet) high-throughput Ethernet protocol and Ethernet flow control algorithms have been used to implement lossless event distribution. To generate LHCb-like traffic, a custom benchmark has been implemented. It was used to stress-test the selected Ethernet networks and to check resilience to uneven workload distribution. Performance tests were made with selected evaluation clusters. 100 Gb/s and 25 Gb/s links were used. Performance results and overall evaluation of this Ethernet-based approach are discussed.
APA, Harvard, Vancouver, ISO, and other styles
33

Sheets, Payson. "PILGRIMAGES AND PERSISTENT SOCIAL MEMORY IN SPITE OF VOLCANIC DISASTERS IN THE ARENAL AREA, COSTA RICA." Ancient Mesoamerica 22, no. 2 (2011): 425–35. http://dx.doi.org/10.1017/s0956536111000265.

Full text
Abstract:
AbstractAncient Costa Ricans in the Arenal area exhibited extraordinary persistence in landscape use and social memory, in spite of repeated catastrophes caused by explosive volcanic eruptions. The Cañales village on the south shore of Lake Arenal was struck by two large explosive eruptions during the Arenal phase (500 b.c.–a.d. 600). Following ecological recovery, the village was reoccupied after each of these eruptions. I argue that the people who reoccupied the village were direct descendants of pre-disaster villagers due to the fact that they reinstated use of the same path to the village cemetery. While previous interpretations emphasized ecological reasons for village reoccupation, I suggest that a dominating reason for reoccupation was to re-establish contact with the spirits of deceased ancestors in the cemetery. The living and the spirits of the deceased constituted the functioning community. The refugees re-established processional access to their cemetery as soon as possible, perhaps even before the village was reoccupied. Archaeologists rarely discover evidence of ancient pilgrimages. However, the combination of remote sensing and detailed stratigraphic analyses allow them to be detected in the Arenal area. Villagers created and perpetuated social memory by regular linear ritual processions along precisely the same path, in spite of challenging topography and occasional regional disasters obscuring the path. This recognition has implications for the arguments of sedentism versus residential mobility during the Arenal phase.
APA, Harvard, Vancouver, ISO, and other styles
34

Wu, Shiyu, Zhichao Xu, Feng Wang, Dongkai Yang, and Gongjian Guo. "An Improved Back-Projection Algorithm for GNSS-R BSAR Imaging Based on CPU and GPU Platform." Remote Sensing 13, no. 11 (May 27, 2021): 2107. http://dx.doi.org/10.3390/rs13112107.

Full text
Abstract:
Global Navigation Satellite System Reflectometry Bistatic Synthetic Aperture Radar (GNSS-R BSAR) is becoming more and more important in remote sensing because of its low power, low mass, low cost, and real-time global coverage capability. The Back Projection Algorithm (BPA) was usually selected as the GNSS-R BSAR imaging algorithm because it can process echo signals of complex geometric configurations. However, the huge computational cost is a challenge for its application in GNSS-R BSAR. Graphics Processing Units (GPU) provides an efficient computing platform for GNSS-R BSAR processing. In this paper, a solution accelerating the BPA of GNSS-R BSAR using GPU is proposed to improve imaging efficiency, and a matching pre-processing program was proposed to synchronize direct and echo signals to improve imaging quality. To process hundreds of gigabytes of data collected by a long-time synthetic aperture in fixed station mode, a stream processing structure was used to process such a large amount of data to solve the problem of limited GPU memory. In the improvement of the imaging efficiency, the imaging task is divided into pre-processing and BPA, which are performed in the Central Processing Unit (CPU) and GPU, respectively, and a pixel-oriented parallel processing method in back projection is adopted to avoid memory access conflicts caused by excessive data volume. The improved BPA with the long synthetic aperture time is verified through the simulation of and experimenting on the GPS-L5 signal. The results show that the proposed accelerating solution is capable of taking approximately 128.04 s, which is 156 times lower than pure CPU framework for producing a size of 600 m × 600 m image with 1800 s synthetic aperture time; in addition, the same imaging quality with the existing processing solution can be retained.
APA, Harvard, Vancouver, ISO, and other styles
35

Naing, Kaung Myat, Siridech Boonsang, Santhad Chuwongin, Veerayuth Kittichai, Teerawat Tongloy, Samrerng Prommongkol, Paron Dekumyoy, and Dorn Watthanakulpanich. "Automatic recognition of parasitic products in stool examination using object detection approach." PeerJ Computer Science 8 (August 17, 2022): e1065. http://dx.doi.org/10.7717/peerj-cs.1065.

Full text
Abstract:
Background Object detection is a new artificial intelligence approach to morphological recognition and labeling parasitic pathogens. Due to the lack of equipment and trained personnel, artificial intelligence innovation for searching various parasitic products in stool examination will enable patients in remote areas of undeveloped countries to access diagnostic services. Because object detection is a developing approach that has been tested for its effectiveness in detecting intestinal parasitic objects such as protozoan cysts and helminthic eggs, it is suitable for use in rural areas where many factors supporting laboratory testing are still lacking. Based on the literatures, the YOLOv4-Tiny produces faster results and uses less memory with the support of low-end GPU devices. In comparison to the YOLOv3 and YOLOv3-Tiny models, this study aimed to propose an automated object detection approach, specifically the YOLOv4-Tiny model, for automatic recognition of intestinal parasitic products in stools. Methods To identify protozoan cysts and helminthic eggs in human feces, the three YOLO approaches; YOLOv4-Tiny, YOLOv3, and YOLOv3-Tiny, were trained to recognize 34 intestinal parasitic classes using training of image dataset. Feces were processed using a modified direct smear method adapted from the simple direct smear and the modified Kato-Katz methods. The image dataset was collected from intestinal parasitic objects discovered during stool examination and the three YOLO models were trained to recognize the image datasets. Results The non-maximum suppression technique and the threshold level were used to analyze the test dataset, yielding results of 96.25% precision and 95.08% sensitivity for YOLOv4-Tiny. Additionally, the YOLOv4-Tiny model had the best AUPRC performance of the three YOLO models, with a score of 0.963. Conclusion This study, to our knowledge, was the first to detect protozoan cysts and helminthic eggs in the 34 classes of intestinal parasitic objects in human stools.
APA, Harvard, Vancouver, ISO, and other styles
36

Ma, Han, Shunlin Liang, Changhao Xiong, Qian Wang, Aolin Jia, and Bing Li. "Global land surface 250 m 8 d fraction of absorbed photosynthetically active radiation (FAPAR) product from 2000 to 2021." Earth System Science Data 14, no. 12 (December 7, 2022): 5333–47. http://dx.doi.org/10.5194/essd-14-5333-2022.

Full text
Abstract:
Abstract. The fraction of absorbed photosynthetically active radiation (FAPAR) is a critical land surface variable for carbon cycle modeling and ecological monitoring. Several global FAPAR products have been released and have become widely used; however, spatiotemporal inconsistency remains a large issue for the current products, and their spatial resolutions and accuracies can hardly meet the user requirements. An effective solution to improve the spatiotemporal continuity and accuracy of FAPAR products is to take better advantage of the temporal information in the satellite data using deep learning approaches. In this study, the latest version (V6) of the FAPAR product with a 250 m resolution was generated from Moderate Resolution Imaging Spectroradiometer (MODIS) surface reflectance data and other information, as part of the Global LAnd Surface Satellite (GLASS) product suite. In addition, it was aggregated to multiple coarser resolutions (up to 0.25∘ and monthly). Three existing global FAPAR products (MODIS Collection 6; GLASS V5; and PRoject for On-Board Autonomy–Vegetation, PROBA-V, V1) were used to generate the time-series training samples, which were used to develop a bidirectional long short-term memory (Bi-LSTM) model. Direct validation using high-resolution FAPAR maps from the Validation of Land European Remote sensing Instrument (VALERI) and ImagineS networks revealed that the GLASS V6 FAPAR product has a higher accuracy than PROBA-V, MODIS, and GLASS V5, with an R2 value of 0.80 and root-mean-square errors (RMSEs) of 0.10–0.11 at the 250 m, 500 m, and 3 km scales, and a higher percentage (72 %) of retrievals for meeting the accuracy requirement of 0.1. Global spatial evaluation and temporal comparison at the AmeriFlux and National Ecological Observatory Network (NEON) sites revealed that the GLASS V6 FAPAR has a greater spatiotemporal continuity and reflects the variations in the vegetation better than the GLASS V5 FAPAR. The higher quality of the GLASS V6 FAPAR is attributed to the ability of the Bi-LSTM model, which involves high-quality training samples and combines the strengths of the existing FAPAR products, as well as the temporal and spectral information from the MODIS surface reflectance data and other information. The 250 m 8 d GLASS V6 FAPAR product for 2020 is freely available at https://doi.org/10.5281/zenodo.6405564 and https://doi.org/10.5281/zenodo.6430925 (Ma, 2022a, b) as well as at the University of Maryland for 2000–2021 (http://glass.umd.edu/FAPAR/MODIS/250m, last access 1 November 2022).
APA, Harvard, Vancouver, ISO, and other styles
37

Xing, Fei, Yi Ping Yao, Zhi Wen Jiang, and Bing Wang. "Fine-Grained Parallel and Distributed Spatial Stochastic Simulation of Biological Reactions." Advanced Materials Research 345 (September 2011): 104–12. http://dx.doi.org/10.4028/www.scientific.net/amr.345.104.

Full text
Abstract:
To date, discrete event stochastic simulations of large scale biological reaction systems are extremely compute-intensive and time-consuming. Besides, it has been widely accepted that spatial factor plays a critical role in the dynamics of most biological reaction systems. The NSM (the Next Sub-Volume Method), a spatial variation of the Gillespie’s stochastic simulation algorithm (SSA), has been proposed for spatially stochastic simulation of those systems. While being able to explore high degree of parallelism in systems, NSM is inherently sequential, which still suffers from the problem of low simulation speed. Fine-grained parallel execution is an elegant way to speed up sequential simulations. Thus, based on the discrete event simulation framework JAMES II, we design and implement a PDES (Parallel Discrete Event Simulation) TW (time warp) simulator to enable the fine-grained parallel execution of spatial stochastic simulations of biological reaction systems using the ANSM (the Abstract NSM), a parallel variation of the NSM. The simulation results of classical Lotka-Volterra biological reaction system show that our time warp simulator obtains remarkable parallel speed-up against sequential execution of the NSM.I.IntroductionThe goal of Systems biology is to obtain system-level investigations of the structure and behavior of biological reaction systems by integrating biology with system theory, mathematics and computer science [1][3], since the isolated knowledge of parts can not explain the dynamics of a whole system. As the complement of “wet-lab” experiments, stochastic simulation, being called the “dry-computational” experiment, plays a more and more important role in computing systems biology [2]. Among many methods explored in systems biology, discrete event stochastic simulation is of greatly importance [4][5][6], since a great number of researches have present that stochasticity or “noise” have a crucial effect on the dynamics of small population biological reaction systems [4][7]. Furthermore, recent research shows that the stochasticity is not only important in biological reaction systems with small population but also in some moderate/large population systems [7].To date, Gillespie’s SSA [8] is widely considered to be the most accurate way to capture the dynamics of biological reaction systems instead of traditional mathematical method [5][9]. However, SSA-based stochastic simulation is confronted with two main challenges: Firstly, this type of simulation is extremely time-consuming, since when the types of species and the number of reactions in the biological system are large, SSA requires a huge amount of steps to sample these reactions; Secondly, the assumption that the systems are spatially homogeneous or well-stirred is hardly met in most real biological systems and spatial factors play a key role in the behaviors of most real biological systems [19][20][21][22][23][24]. The next sub-volume method (NSM) [18], presents us an elegant way to access the special problem via domain partition. To our disappointment, sequential stochastic simulation with the NSM is still very time-consuming, and additionally introduced diffusion among neighbor sub-volumes makes things worse. Whereas, the NSM explores a very high degree of parallelism among sub-volumes, and parallelization has been widely accepted as the most meaningful way to tackle the performance bottleneck of sequential simulations [26][27]. Thus, adapting parallel discrete event simulation (PDES) techniques to discrete event stochastic simulation would be particularly promising. Although there are a few attempts have been conducted [29][30][31], research in this filed is still in its infancy and many issues are in need of further discussion. The next section of the paper presents the background and related work in this domain. In section III, we give the details of design and implementation of model interfaces of LP paradigm and the time warp simulator based on the discrete event simulation framework JAMES II; the benchmark model and experiment results are shown in Section IV; in the last section, we conclude the paper with some future work.II. Background and Related WorkA. Parallel Discrete Event Simulation (PDES)The notion Logical Process (LP) is introduced to PDES as the abstract of the physical process [26], where a system consisting of many physical processes is usually modeled by a set of LP. LP is regarded as the smallest unit that can be executed in PDES and each LP holds a sub-partition of the whole system’s state variables as its private ones. When a LP processes an event, it can only modify the state variables of its own. If one LP needs to modify one of its neighbors’ state variables, it has to schedule an event to the target neighbor. That is to say event message exchanging is the only way that LPs interact with each other. Because of the data dependences or interactions among LPs, synchronization protocols have to be introduced to PDES to guarantee the so-called local causality constraint (LCC) [26]. By now, there are a larger number of synchronization algorithms have been proposed, e.g. the null-message [26], the time warp (TW) [32], breath time warp (BTW) [33] and etc. According to whether can events of LPs be processed optimistically, they are generally divided into two types: conservative algorithms and optimistic algorithms. However, Dematté and Mazza have theoretically pointed out the disadvantages of pure conservative parallel simulation for biochemical reaction systems [31]. B. NSM and ANSM The NSM is a spatial variation of Gillespie’ SSA, which integrates the direct method (DM) [8] with the next reaction method (NRM) [25]. The NSM presents us a pretty good way to tackle the aspect of space in biological systems by partitioning a spatially inhomogeneous system into many much more smaller “homogeneous” ones, which can be simulated by SSA separately. However, the NSM is inherently combined with the sequential semantics, and all sub-volumes share one common data structure for events or messages. Thus, directly parallelization of the NSM may be confronted with the so-called boundary problem and high costs of synchronously accessing the common data structure [29]. In order to obtain higher efficiency of parallel simulation, parallelization of NSM has to firstly free the NSM from the sequential semantics and secondly partition the shared data structure into many “parallel” ones. One of these is the abstract next sub-volume method (ANSM) [30]. In the ANSM, each sub-volume is modeled by a logical process (LP) based on the LP paradigm of PDES, where each LP held its own event queue and state variables (see Fig. 1). In addition, the so-called retraction mechanism was introduced in the ANSM too (see algorithm 1). Besides, based on the ANSM, Wang etc. [30] have experimentally tested the performance of several PDES algorithms in the platform called YH-SUPE [27]. However, their platform is designed for general simulation applications, thus it would sacrifice some performance for being not able to take into account the characteristics of biological reaction systems. Using the similar ideas of the ANSM, Dematté and Mazza have designed and realized an optimistic simulator. However, they processed events in time-stepped manner, which would lose a specific degree of precisions compared with the discrete event manner, and it is very hard to transfer a time-stepped simulation to a discrete event one. In addition, Jeschke etc.[29] have designed and implemented a dynamic time-window simulator to execution the NSM in parallel on the grid computing environment, however, they paid main attention on the analysis of communication costs and determining a better size of the time-window.Fig. 1: the variations from SSA to NSM and from NSM to ANSMC. JAMES II JAMES II is an open source discrete event simulation experiment framework developed by the University of Rostock in Germany. It focuses on high flexibility and scalability [11][13]. Based on the plug-in scheme [12], each function of JAMES II is defined as a specific plug-in type, and all plug-in types and plug-ins are declared in XML-files [13]. Combined with the factory method pattern JAMES II innovatively split up the model and simulator, which makes JAMES II is very flexible to add and reuse both of models and simulators. In addition, JAMES II supports various types of modelling formalisms, e.g. cellular automata, discrete event system specification (DEVS), SpacePi, StochasticPi and etc.[14]. Besides, a well-defined simulator selection mechanism is designed and developed in JAMES II, which can not only automatically choose the proper simulators according to the modeling formalism but also pick out a specific simulator from a serious of simulators supporting the same modeling formalism according to the user settings [15].III. The Model Interface and SimulatorAs we have mentioned in section II (part C), model and simulator are split up into two separate parts. Thus, in this section, we introduce the designation and implementation of model interface of LP paradigm and more importantly the time warp simulator.A. The Mod Interface of LP ParadigmJAMES II provides abstract model interfaces for different modeling formalism, based on which Wang etc. have designed and implemented model interface of LP paradigm[16]. However, this interface is not scalable well for parallel and distributed simulation of larger scale systems. In our implementation, we accommodate the interface to the situation of parallel and distributed situations. Firstly, the neighbor LP’s reference is replaced by its name in LP’s neighbor queue, because it is improper even dangerous that a local LP hold the references of other LPs in remote memory space. In addition, (pseudo-)random number plays a crucial role to obtain valid and meaningful results in stochastic simulations. However, it is still a very challenge work to find a good random number generator (RNG) [34]. Thus, in order to focus on our problems, we introduce one of the uniform RNGs of JAMES II to this model interface, where each LP holds a private RNG so that random number streams of different LPs can be independent stochastically. B. The Time Warp SimulatorBased on the simulator interface provided by JAMES II, we design and implement the time warp simulator, which contains the (master-)simulator, (LP-)simulator. The simulator works strictly as master/worker(s) paradigm for fine-grained parallel and distributed stochastic simulations. Communication costs are crucial to the performance of a fine-grained parallel and distributed simulation. Based on the Java remote method invocation (RMI) mechanism, P2P (peer-to-peer) communication is implemented among all (master-and LP-)simulators, where a simulator holds all the proxies of targeted ones that work on remote workers. One of the advantages of this communication approach is that PDES codes can be transferred to various hardwire environment, such as Clusters, Grids and distributed computing environment, with only a little modification; The other is that RMI mechanism is easy to realized and independent to any other non-Java libraries. Since the straggler event problem, states have to be saved to rollback events that are pre-processed optimistically. Each time being modified, the state is cloned to a queue by Java clone mechanism. Problem of this copy state saving approach is that it would cause loads of memory space. However, the problem can be made up by a condign GVT calculating mechanism. GVT reduction scheme also has a significant impact on the performance of parallel simulators, since it marks the highest time boundary of events that can be committed so that memories of fossils (processed events and states) less than GVT can be reallocated. GVT calculating is a very knotty for the notorious simultaneous reporting problem and transient messages problem. According to our problem, another GVT algorithm, called Twice Notification (TN-GVT) (see algorithm 2), is contributed to this already rich repository instead of implementing one of GVT algorithms in reference [26] and [28].This algorithm looks like the synchronous algorithm described in reference [26] (pp. 114), however, they are essentially different from each other. This algorithm has never stopped the simulators from processing events when GVT reduction, while algorithm in reference [26] blocks all simulators for GVT calculating. As for the transient message problem, it can be neglect in our implementation, because RMI based remote communication approach is synchronized, that means a simulator will not go on its processing until the remote the massage get to its destination. And because of this, the high-costs message acknowledgement, prevalent over many classical asynchronous GVT algorithms, is not needed anymore too, which should be constructive to the whole performance of the time warp simulator.IV. Benchmark Model and Experiment ResultsA. The Lotka-Volterra Predator-prey SystemIn our experiment, the spatial version of Lotka-Volterra predator-prey system is introduced as the benchmark model (see Fig. 2). We choose the system for two considerations: 1) this system is a classical experimental model that has been used in many related researches [8][30][31], so it is credible and the simulation results are comparable; 2) it is simple but helpful enough to test the issues we are interested in. The space of predator-prey System is partitioned into a2D NXNgrid, whereNdenotes the edge size of the grid. Initially the population of the Grass, Preys and Predators are set to 1000 in each single sub-volume (LP). In Fig. 2,r1,r2,r3stand for the reaction constants of the reaction 1, 2 and 3 respectively. We usedGrass,dPreyanddPredatorto stand for the diffusion rate of Grass, Prey and Predator separately. Being similar to reference [8], we also take the assumption that the population of the grass remains stable, and thusdGrassis set to zero.R1:Grass + Prey ->2Prey(1)R2:Predator +Prey -> 2Predator(2)R3:Predator -> NULL(3)r1=0.01; r2=0.01; r3=10(4)dGrass=0.0;dPrey=2.5;dPredato=5.0(5)Fig. 2: predator-prey systemB. Experiment ResultsThe simulation runs have been executed on a Linux Cluster with 40 computing nodes. Each computing node is equipped with two 64bit 2.53 GHz Intel Xeon QuadCore Processors with 24GB RAM, and nodes are interconnected with Gigabit Ethernet connection. The operating system is Kylin Server 3.5, with kernel 2.6.18. Experiments have been conducted on the benchmark model of different size of mode to investigate the execution time and speedup of the time warp simulator. As shown in Fig. 3, the execution time of simulation on single processor with 8 cores is compared. The result shows that it will take more wall clock time to simulate much larger scale systems for the same simulation time. This testifies the fact that larger scale systems will leads to more events in the same time interval. More importantly, the blue line shows that the sequential simulation performance declines very fast when the mode scale becomes large. The bottleneck of sequential simulator is due to the costs of accessing a long event queue to choose the next events. Besides, from the comparison between group 1 and group 2 in this experiment, we could also conclude that high diffusion rate increased the simulation time greatly both in sequential and parallel simulations. This is because LP paradigm has to split diffusion into two processes (diffusion (in) and diffusion (out) event) for two interactive LPs involved in diffusion and high diffusion rate will lead to high proportional of diffusion to reaction. In the second step shown in Fig. 4, the relationship between the speedups from time warp of two different model sizes and the number of work cores involved are demonstrated. The speedup is calculated against the sequential execution of the spatial reaction-diffusion systems model with the same model size and parameters using NSM.Fig. 4 shows the comparison of speedup of time warp on a64X64grid and a100X100grid. In the case of a64X64grid, under the condition that only one node is used, the lowest speedup (a little bigger than 1) is achieved when two cores involved, and the highest speedup (about 6) is achieved when 8 cores involved. The influence of the number of cores used in parallel simulation is investigated. In most cases, large number of cores could bring in considerable improvements in the performance of parallel simulation. Also, compared with the two results in Fig. 4, the simulation of larger model achieves better speedup. Combined with time tests (Fig. 3), we find that sequential simulator’s performance declines sharply when the model scale becomes very large, which makes the time warp simulator get better speed-up correspondingly.Fig. 3: Execution time (wall clock time) of Seq. and time warp with respect to different model sizes (N=32, 64, 100, and 128) and model parameters based on single computing node with 8 cores. Results of the test are grouped by the diffusion rates (Group 1: Sequential 1 and Time Warp 1. dPrey=2.5, dPredator=5.0; Group 2: dPrey=0.25, dPredator=0.5, Sequential 2 and Time Warp 2).Fig. 4: Speedup of time warp with respect to the number of work cores and the model size (N=64 and 100). Work cores are chose from one computing node. Diffusion rates are dPrey=2.5, dPredator=5.0 and dGrass=0.0.V. Conclusion and Future WorkIn this paper, a time warp simulator based on the discrete event simulation framework JAMES II is designed and implemented for fine-grained parallel and distributed discrete event spatial stochastic simulation of biological reaction systems. Several challenges have been overcome, such as state saving, roll back and especially GVT reduction in parallel execution of simulations. The Lotka-Volterra Predator-Prey system is chosen as the benchmark model to test the performance of our time warp simulator and the best experiment results show that it can obtain about 6 times of speed-up against the sequential simulation. The domain this paper concerns with is in the infancy, many interesting issues are worthy of further investigated, e.g. there are many excellent PDES optimistic synchronization algorithms (e.g. the BTW) as well. Next step, we would like to fill some of them into JAMES II. In addition, Gillespie approximation methods (tau-leap[10] etc.) sacrifice some degree of precision for higher simulation speed, but still could not address the aspect of space of biological reaction systems. The combination of spatial element and approximation methods would be very interesting and promising; however, the parallel execution of tau-leap methods should have to overcome many obstacles on the road ahead.AcknowledgmentThis work is supported by the National Natural Science Foundation of China (NSF) Grant (No.60773019) and the Ph.D. Programs Foundation of Ministry of Education of China (No. 200899980004). The authors would like to show their great gratitude to Dr. Jan Himmelspach and Dr. Roland Ewald at the University of Rostock, Germany for their invaluable advice and kindly help with JAMES II.ReferencesH. Kitano, "Computational systems biology." Nature, vol. 420, no. 6912, pp. 206-210, November 2002.H. Kitano, "Systems biology: a brief overview." Science (New York, N.Y.), vol. 295, no. 5560, pp. 1662-1664, March 2002.A. Aderem, "Systems biology: Its practice and challenges," Cell, vol. 121, no. 4, pp. 511-513, May 2005. [Online]. Available: http://dx.doi.org/10.1016/j.cell.2005.04.020.H. de Jong, "Modeling and simulation of genetic regulatory systems: A literature review," Journal of Computational Biology, vol. 9, no. 1, pp. 67-103, January 2002.C. W. Gardiner, Handbook of Stochastic Methods: for Physics, Chemistry and the Natural Sciences (Springer Series in Synergetics), 3rd ed. Springer, April 2004.D. T. Gillespie, "Simulation methods in systems biology," in Formal Methods for Computational Systems Biology, ser. Lecture Notes in Computer Science, M. Bernardo, P. Degano, and G. Zavattaro, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, vol. 5016, ch. 5, pp. 125-167.Y. Tao, Y. Jia, and G. T. Dewey, "Stochastic fluctuations in gene expression far from equilibrium: Omega expansion and linear noise approximation," The Journal of Chemical Physics, vol. 122, no. 12, 2005.D. T. Gillespie, "Exact stochastic simulation of coupled chemical reactions," Journal of Physical Chemistry, vol. 81, no. 25, pp. 2340-2361, December 1977.D. T. Gillespie, "Stochastic simulation of chemical kinetics," Annual Review of Physical Chemistry, vol. 58, no. 1, pp. 35-55, 2007.D. T. Gillespie, "Approximate accelerated stochastic simulation of chemically reacting systems," The Journal of Chemical Physics, vol. 115, no. 4, pp. 1716-1733, 2001.J. Himmelspach, R. Ewald, and A. M. Uhrmacher, "A flexible and scalable experimentation layer," in WSC '08: Proceedings of the 40th Conference on Winter Simulation. Winter Simulation Conference, 2008, pp. 827-835.J. Himmelspach and A. M. Uhrmacher, "Plug'n simulate," in 40th Annual Simulation Symposium (ANSS'07). Washington, DC, USA: IEEE, March 2007, pp. 137-143.R. Ewald, J. Himmelspach, M. Jeschke, S. Leye, and A. M. Uhrmacher, "Flexible experimentation in the modeling and simulation framework james ii-implications for computational systems biology," Brief Bioinform, vol. 11, no. 3, pp. bbp067-300, January 2010.A. Uhrmacher, J. Himmelspach, M. Jeschke, M. John, S. Leye, C. Maus, M. Röhl, and R. Ewald, "One modelling formalism & simulator is not enough! a perspective for computational biology based on james ii," in Formal Methods in Systems Biology, ser. Lecture Notes in Computer Science, J. Fisher, Ed. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, vol. 5054, ch. 9, pp. 123-138. [Online]. Available: http://dx.doi.org/10.1007/978-3-540-68413-8_9.R. Ewald, J. Himmelspach, and A. M. Uhrmacher, "An algorithm selection approach for simulation systems," pads, vol. 0, pp. 91-98, 2008.Bing Wang, Jan Himmelspach, Roland Ewald, Yiping Yao, and Adelinde M Uhrmacher. Experimental analysis of logical process simulation algorithms in james ii[C]// In M. D. Rossetti, R. R. Hill, B. Johansson, A. Dunkin, and R. G. Ingalls, editors, Proceedings of the Winter Simulation Conference, IEEE Computer Science, 2009. 1167-1179.Ewald, J. Rössel, J. Himmelspach, and A. M. Uhrmacher, "A plug-in-based architecture for random number generation in simulation systems," in WSC '08: Proceedings of the 40th Conference on Winter Simulation. Winter Simulation Conference, 2008, pp. 836-844.J. Elf and M. Ehrenberg, "Spontaneous separation of bi-stable biochemical systems into spatial domains of opposite phases." Systems biology, vol. 1, no. 2, pp. 230-236, December 2004.K. Takahashi, S. Arjunan, and M. Tomita, "Space in systems biology of signaling pathways? Towards intracellular molecular crowding in silico," FEBS Letters, vol. 579, no. 8, pp. 1783-1788, March 2005.J. V. Rodriguez, J. A. Kaandorp, M. Dobrzynski, and J. G. Blom, "Spatial stochastic modelling of the phosphoenolpyruvate-dependent phosphotransferase (pts) pathway in escherichia coli," Bioinformatics, vol. 22, no. 15, pp. 1895-1901, August 2006.D. Ridgway, G. Broderick, and M. Ellison, "Accommodating space, time and randomness in network simulation," Current Opinion in Biotechnology, vol. 17, no. 5, pp. 493-498, October 2006.J. V. Rodriguez, J. A. Kaandorp, M. Dobrzynski, and J. G. Blom, "Spatial stochastic modelling of the phosphoenolpyruvate-dependent phosphotransferase (pts) pathway in escherichia coli," Bioinformatics, vol. 22, no. 15, pp. 1895-1901, August 2006.W. G. Wilson, A. M. Deroos, and E. Mccauley, "Spatial instabilities within the diffusive lotka-volterra system: Individual-based simulation results," Theoretical Population Biology, vol. 43, no. 1, pp. 91-127, February 1993.K. Kruse and J. Elf. Kinetics in spatially extended systems. In Z. Szallasi, J. Stelling, and V. Periwal, editors, System Modeling in Cellular Biology. From Concepts to Nuts and Bolts, pages 177–198. MIT Press, Cambridge, MA, 2006.M. A. Gibson and J. Bruck, "Efficient exact stochastic simulation of chemical systems with many species and many channels," The Journal of Physical Chemistry A, vol. 104, no. 9, pp. 1876-1889, March 2000.R. M. Fujimoto, Parallel and Distributed Simulation Systems (Wiley Series on Parallel and Distributed Computing). Wiley-Interscience, January 2000.Y. Yao and Y. Zhang, “Solution for analytic simulation based on parallel processing,” Journal of System Simulation, vol. 20, No.24, pp. 6617–6621, 2008.G. Chen and B. K. Szymanski, "Dsim: scaling time warp to 1,033 processors," in WSC '05: Proceedings of the 37th conference on Winter simulation. Winter Simulation Conference, 2005, pp. 346-355.M. Jeschke, A. Park, R. Ewald, R. Fujimoto, and A. M. Uhrmacher, "Parallel and distributed spatial simulation of chemical reactions," in 2008 22nd Workshop on Principles of Advanced and Distributed Simulation. Washington, DC, USA: IEEE, June 2008, pp. 51-59.B. Wang, Y. Yao, Y. Zhao, B. Hou, and S. Peng, "Experimental analysis of optimistic synchronization algorithms for parallel simulation of reaction-diffusion systems," High Performance Computational Systems Biology, International Workshop on, vol. 0, pp. 91-100, October 2009.L. Dematté and T. Mazza, "On parallel stochastic simulation of diffusive systems," in Computational Methods in Systems Biology, M. Heiner and A. M. Uhrmacher, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, vol. 5307, ch. 16, pp. 191-210.D. R. Jefferson, "Virtual time," ACM Trans. Program. Lang. Syst., vol. 7, no. 3, pp. 404-425, July 1985.J. S. Steinman, "Breathing time warp," SIGSIM Simul. Dig., vol. 23, no. 1, pp. 109-118, July 1993. [Online]. Available: http://dx.doi.org/10.1145/174134.158473 S. K. Park and K. W. Miller, "Random number generators: good ones are hard to find," Commun. ACM, vol. 31, no. 10, pp. 1192-1201, October 1988.
APA, Harvard, Vancouver, ISO, and other styles
38

Li, Pengfei, Yu Hua, Pengfei Zuo, Zhangyu Chen, and Jiajie Sheng. "A High-Performance RDMA-oriented Learned Key-Value Store for Disaggregated Memory Systems." ACM Transactions on Storage, September 5, 2023. http://dx.doi.org/10.1145/3620674.

Full text
Abstract:
Disaggregated memory systems separate monolithic servers into different components, including compute and memory nodes, to enjoy the benefits of high resource utilization, flexible hardware scalability, and efficient data sharing. By exploiting the high-performance RDMA (Remote Direct Memory Access), the compute nodes directly access the remote memory pool without involving remote CPUs. Hence, the ordered key-value (KV) stores (e.g., B-trees and learned indexes) keep all data sorted to provide rang query service via the high-performance network. However, existing ordered KVs fail to work well on the disaggregated memory systems, due to either consuming multiple network roundtrips to search the remote data or heavily relying on the memory nodes equipped with insufficient computing resources to process data modifications. In this paper, we propose a scalable RDMA-oriented KV store with learned indexes, called ROLEX, to coalesce the ordered KV store in the disaggregated systems for efficient data storage and retrieval. ROLEX leverages a retraining-decoupled learned index scheme to dissociate the model retraining from data modification operations via adding a bias and some data-movement constraints to learned models. Based on the operation decoupling, data modifications are directly executed in compute nodes via one-sided RDMA verbs with high scalability. The model retraining is hence removed from the critical path of data modification and asynchronously executed in memory nodes by using dedicated computing resources. ROLEX efficiently alleviates the fragmentation and garbage collection issues, due to allocating and reclaiming space via fixed-size leaves that are accessed via the atomic-size leaf numbers. Our experimental results on YCSB and real-world workloads demonstrate that ROLEX achieves competitive performance on the static workloads, as well as significantly improving the performance on dynamic workloads by up to 2.2 × than state-of-the-art schemes on the disaggregated memory systems. We have released the open-source codes for public use in GitHub.
APA, Harvard, Vancouver, ISO, and other styles
39

Leong, Kin-Wai, Zhilong Li, and Yunqu Leon Liu. "Reliable multicast using remote direct memory access (RDMA) over a passive optical cross-connect fabric enhanced with wavelength division multiplexing (WDM)." APSIPA Transactions on Signal and Information Processing 8 (2019). http://dx.doi.org/10.1017/atsip.2019.17.

Full text
Abstract:
It has been well studied that reliable multicast enables consistency protocols, including Byzantine Fault Tolerant protocols, for distributed systems. However, no transport-layer reliable multicast is used today due to limitations with existing switch fabrics and transport-layer protocols. In this paper, we introduce a layer-4 (L4) transport based on remote direct memory access (RDMA) datagram to achieve reliable multicast over a shared optical medium. By connecting a cluster of networking nodes using a passive optical cross-connect fabric enhanced with wavelength division multiplexing, all messages are broadcast to all nodes. This mechanism enables consistency in a distributed system to be maintained at a low latency cost. By further utilizing RDMA datagram as the L4 protocol, we have achieved a low-enough message loss-ratio (better than one in 68 billion) to make a simple Negative Acknowledge (NACK)-based L4 multicast practical to deploy. To our knowledge, it is the first multicast architecture able to demonstrate such low message loss-ratio. Furthermore, with this reliable multicast transport, end-to-end latencies of eight microseconds or less (< 8us) have been routinely achieved using an enhanced software RDMA implementation on a variety of commodity 10G Ethernet network adapters.
APA, Harvard, Vancouver, ISO, and other styles
40

Chen, Hanhua, Jie Yuan, Hai Jin, Yonghui Wang, Sijie Wu, and Zhihao Jiang. "RGraph: Asynchronous graph processing based on asymmetry of remote direct memory access." Software: Practice and Experience, April 26, 2021. http://dx.doi.org/10.1002/spe.2979.

Full text
APA, Harvard, Vancouver, ISO, and other styles
41

Sawant, Bhushan. "Latency and Throughput Optimization In Modern Networks." International Journal of Advanced Research in Science, Communication and Technology, May 30, 2021, 585–92. http://dx.doi.org/10.48175/ijarsct-1298.

Full text
Abstract:
The paper briefly describes about the subject which is Latency and Throughput Optimization in Modern Networks. Modern applications are highly sensitive to communication delays and throughput. This paper surveys major attempts on reducing latency and increasing the throughput. These methods are surveyed on different networks and surrounding’s such as wired networks, wireless networks, application layer transport control, Remote Direct Memory Access, and machine learning based transport control. On one hand every user likes to send and receive their data as quickly as possible. On the other hand the network infrastructure that connects users has limited capacities and these are usually shared among users. Thus Latency and Throughput Optimization is important in Modern networking.
APA, Harvard, Vancouver, ISO, and other styles
42

Teigen, Helene F., Cristina Paupini, and Henry Mainsah. "Intra-Acting With Technology During Remote Fieldwork in Homes: Curating Video Interviews and Drawing Methods." International Journal of Qualitative Methods 22 (January 2023). http://dx.doi.org/10.1177/16094069231185466.

Full text
Abstract:
The ubiquity of remote research practices and the increased adoption of videoconferencing tools are forcing researchers to pay attention to the features of these technologies and how they shape the research encounter. This article draws on experience from a Norwegian research project about digital vulnerabilities and living with smart home technologies. Participants were invited to draw floor plans of their homes, enact everyday routines, and perform a house tour on camera. By enacting collective memory work, reviewing, and discussing fieldnotes and other materials from the interviews, the authors have reflected upon experiences and identified lessons and implications for further research. Drawing on a socio-material approach that views digital technologies as active participants in enacting the research event, the article highlights how the presence of videoconferencing technology shapes the research situation and how data is generated. We show how videoconferencing tools can provide both access to and constraints on what can be done and who can participate and underline the importance of skills for both researchers and participants. We demonstrate how the handholding of the camera and the frame of the lens direct what is seen during a video interview, and how power dynamics between participants and researchers are shaped. We conclude that researchers need to be mindful of how agency is negotiated between technology and humans during remote fieldwork.
APA, Harvard, Vancouver, ISO, and other styles
43

Larsson, Chari. "Suspicious Images: Iconophobia and the Ethical Gaze." M/C Journal 15, no. 1 (November 4, 2011). http://dx.doi.org/10.5204/mcj.393.

Full text
Abstract:
If iconophobia is defined as the suspicion and anxiety towards the power exerted by images, its history is an ancient one in all of its Platonic, Christian, and Judaic forms. At its most radical, iconophobia results in an act of iconoclasm, or the total destruction of the image. At the other end of the spectrum, contemporary iconophobia may be more subtle. Images are simply withdrawn from circulation with the aim of eliminating their visibility. In his book Images in Spite of All, French art historian Georges Didi-Huberman questions the tradition of suspicion and denigration governing visual representations of the Holocaust, arguing we have abdicated our ethical obligation to try to imagine. This essay will argue that disruptions to traditional modes of spectatorship shift the terms of viewing from suspicion to ethical participation. By building on Didi-Huberman’s discussion of images and the spectatorial gaze, this essay will consider Laura Waddington’s 2002 documentary film Border. Waddington spent six months hiding with asylum seekers in the area surrounding the Red Cross refugee camp at Sangatte in northern France. I will argue that Waddington proposes a model of spectatorship that implicates the viewer into the ethical content of the film. By seeking to restore the dignity and humanity of the asylum seekers rather than viewing them with suspicion, Border is an acute reminder of our moral responsibility to bear witness to that which lies beyond the boundaries of conventional representations of asylum seekers.The economy managing the circulation of mainstream media images is a highly suspicious mechanism. After the initial process of image selection and distribution, what we are left with is an already homogenised collection of predictable and recyclable media images. The result is an increasingly iconophobic media gaze as the actual content of the image is depleted. In her essay “Precarious Life,” Judith Butler describes this economy in terms of the “normative processes” of control exercised by the mainstream media, arguing that the structurally unbalanced media representations of the ‘other’ result in creating a progressively dehumanised effect (Butler 146). This process of disidentification completes the iconophobic circle as the spectator, unable to develop empathy, views the dehumanised subject with increasing suspicion. Written in the aftermath of 9/11 and the ensuing War on Terror, Butler’s insights are important as they alert us to the possibility of a breach or rupture in the image economy. It is against Butler’s normative processes that Didi-Huberman’s critique of Holocaust iconoclasm and Waddington’s Border propose a slippage in representation and spectatorship capable of disrupting the homogeneity of the mass circulation of images.Most images that have come to represent the Holocaust in our collective memory were either recorded by the Nazis for propaganda or by the Allies on liberation in 1945. Virtually no photographs exist from inside the concentration camps. This is distinct from the endlessly recycled images of gaunt, emaciated survivors and bulldozers pushing aside corpses which have become critical in defining Holocaust iconography (Saxton 14). Familiar and recognisable, this visual record constitutes a “visual memory bank” that we readily draw upon when conjuring up images of the Holocaust. What occurs, however, when an image falls outside the familiar corpus of Holocaust representation? This was the question raised in a now infamous exhibition held in Paris in 2001 (Chéroux). The exhibition included four small photographs secretly taken by members of the Sonderkommando inside the Nazi extermination camp Auschwitz-Birkenau in August 1944. The Sonderkommando were the group of prisoners who were delegated the task of the day-to-day running of the crematoria. The photographs were smuggled out of the camps in a tube of toothpaste, and eventually reached the Polish Resistance.By evading the surveillance of the SS the photographs present a breach in the economy of Holocaust iconography. They exist as an exception to the rule, mere fragments stolen from beneath the all-seeing eye of the SS Guards and their watch towers. Despite operating in an impossible situation, the inmate maintained the belief that these images could provide visual proof of the existence of the gas chambers. The images are testimony produced inside the camp itself, a direct challenge to the discourse emphasising the prohibition of representation of the Holocaust and in particular the gas chambers. Figure 1 The Auschwitz crematorium in operation, photograph by Sonderkommando prisoners August 1944 © www.auschwitz.org.plDidi-Huberman’s essay marks a point of departure from the iconophobia which has stressed the unimaginable (Lanzmann), unknowable (Lyotard), and ultimately unrepresentable (Levinas) nature of the Holocaust since the 1980s. Denigrated and derided, images have been treated suspiciously by this philosophical line of thought, emphasising the irretrievable gap between representation and the Holocaust. In a direct assault on the tradition of framing the Holocaust as unrepresentable, Didi-Huberman’s essay becomes a plea to the moral and ethical responsibility to bear witness. He writes of the obligation to these images, arguing that “it is a response we must offer, as a debt to the words and images that certain prisoners snatched, for us, from the harrowing Real of their experience” (3). The photographs are not simply archival documents, but a testament to the humanity of the members of the Sonderkommando the Nazis sought to erase.Suspicion towards the potential power exerted by images has been neutralised by models of spectatorship privileging the viewer’s mastery and control. In traditional theories of film spectatorship, the spectator is rendered in terms of a general omnipotence described by Christian Metz as “an all-powerful position which is of God himself...” (49). It is a model of spectatorship that promotes mastery over the image by privileging the unilateral gaze of the spectator. Alternatively, Didi-Huberman evokes a long counter tradition within French literature and philosophy of the “seer seen,” where the object of the spectator’s gaze is endowed with the ability to return the gaze resulting in various degrees of anxiety and paranoia. The image of the “seer seen” recurs throughout the writing of Baudelaire, Sartre, Merleau-Ponty, Lacan, and Barthes, negating the unilateral gaze of an omnipotent spectator (Didi-Huberman, Ce que nous voyons).Didi-Huberman explicitly draws upon Jacques Lacan’s thinking about the gaze in light of this tradition of the image looking back. In his 1964 seminars on vision in the Four Fundamental Concepts of Psychoanalysis, Lacan dedicates several chapters to demonstrate how the visual field is structured by the symbolic order, the real, symbolic and the imaginary. Following Lacan, Didi-Huberman introduces two terms, the veil-image and the tear-image, which are analogous with Lacan’s imaginary and the real. The imaginary, with its connotations of illusion and fantasy, provides the sense of wholeness in both ourselves and what we perceive. For Didi-Huberman, the imaginary corresponds with the veil-image. Within the canon of Holocaust photography, the veil-image is the image “where nobody really looks,” the screen or veil maintaining the spectator’s illusion of mastery (81). We might say that in the circulation of Holocaust atrocity images, the veil serves to anaesthetise and normalise the content of the image.Lacan’s writing on the gaze, however, undermines the spectator’s mastery over the image by placing the spectator not at the all-seeing apex of the visual field, but located firmly within the visual field of the image. Lacan writes, “in the scopic field, the gaze is outside, I am looked at, that is to say, I am the picture...I am photo-graphed” (Lacan 106). The spectator is ensnared in the gaze of the image as the gaze is reciprocated. For Didi-Huberman, the veil-image seeks to disarm the threat to the spectator being caught in the image-gaze. Lacan describes this neutralisation in terms of “the pacifying, Apollonian effect of painting. Something is given not so much to the gaze as to the eye, something that involves the abandonment, the laying down, of the gaze” (101). Further on, Lacan expresses this in terms of the dompte-regarde, or a taming of the gaze (109). The veil-image maintains the fiction of the spectator’s ascendency by subduing the threat of the image-gaze. In opposition to the veil-image is the tear-image, in which for Didi-Huberman “a fragment of the real escapes” (81). This represents a rupture in the visual field. The real is presented here in terms of the tuché, or missed encounter, resulting in the spectator’s anxiety and trauma. As the real cannot be represented, it is the point where representation collapses, rupturing the illusion of coherency maintained by the veil-image. Operating as an exception or disruption to the rule, the tear-image disrupts the image economy. No longer neutralised, the image returns the gaze, shattering the illusion of the all-seeing mastery of the spectator. Didi-Huberman describes this tearing exception to the rule, “where everyone suddenly feels looked at” (81).To treat the Sonderkommando photographs as tear-images, not veil-images, we are offered a departure from classic models of spectatorship. We are forced to align ourselves and identify with the “inhuman” gaze of the Sonderkommando. The obvious response is to recoil. The gaze here is not the paranoid Sartrean gaze, evoking shame in the spectator-as-voyeur. Nor are these photographs reassuring narcissistic veil-images, but will always remain the inimical gaze of the Other—tearing, ripping images, which nonetheless demand that we do not turn away. It is an ethical response we must offer. If the power of the tear-image resides in its ability to disrupt traditional modes of representation and spectatorship, I would like to discuss this in relation to Laura Waddington’s 2004 film Border. Waddington is a Brussels based filmmaker with a particular interest in documenting the movement of displaced peoples. Just as the Sonderkommando photographs were taken clandestinely from beneath the gaze of the SS, Waddington evaded the surveillance of the French police and helicopter patrols as she bore witness to the plight of asylum seekers trying to reach England. Border presents her stolen testimony, operating outside the familiar iconography of mainstream media’s representation of asylum seekers. If we were to consider the portrayal of asylum seekers by the Australian media in terms of the veil-image, we are left with a predictable body of homogenised and neutralised stock media images. The myth of Australia being overrun by boat people is reinforced by the visual iconography of the news media. Much like the iconography of the Holocaust, these types of images have come to define the representations of asylum seekers. Traceable back to the 2001 Tampa affair images tend to be highly militarised, frequently with Australian Navy patrol boats in the background. The images reinforce the ‘stop the boats’ rhetoric exhibited on both sides of politics, paradoxically often working against the grain of the article’s editorial content. Figure 2 Thursday 16 Apr 2009 there was an explosion on board a suspected illegal entry vessel (SIEV) 36 in the vicinity of Ashmore Reef. © Commonwealth of Australia 2011Figure 3 The crew of HMAS Albany, Attack One, board suspected illegal entry vessel (SIEV) 38 © Commonwealth of Australia 2011 The media gaze is structurally unbalanced against the suffering of asylum seekers. In Australia asylum seekers are detained in mandatory detention, in remote sites such as Christmas Island and Woomera. Worryingly, the Department of Immigration maintains strict control over media representations of the conditions inside the camps, resulting in a further abstraction of representation. Geographical isolation coupled with a lack of transparent media access contributes to the ongoing process of dehumanisation of the asylum seekers. Judith Butler describes this as “The erasure of that suffering through the prohibition of images and representations” (146). In the endless recycling of images of leaky fishing boats and the perimeters of detention centres, our critical capacity to engage becomes progressively eroded. These images fulfil the function of the veil-image, where nobody really looks as there is nothing left to see. Figure 4 Asylum seekers arrive by boat on Christmas Island, Friday, July 8, 2011. AAP Image/JOSH JERGA Figure 5 Woomera Detention Centre. AAP Image/ROB HUTCHISON By reading Laura Waddington’s Border against an iconophobic media gaze, we are afforded the opportunity to reconsider this image economy and the suspicious gaze of the spectator it seeks to solicit. Border reminds us of the paradoxical function of the news image—it shows us everything, but nothing at all. In a subtle interrogation of our indifference to the existence of asylum seekers and their suffering, Border is a record of the six months Waddington spent hidden in the fields surrounding the French Red Cross camp at Sangatte in 2002. Sangatte is a small town in northern France, just south of Calais and only one and a half hours’ drive from Paris. The asylum seekers are predominantly Afghan and Iraqi. Border is a record of the last stop in their long desperate journey to reach England, which then had comparatively humane asylum seeking policies. The men are attempting to cross the channel tunnel, hidden in trucks and on freight trains. Many are killed or violently injured in their attempts to evade capture by the French police. Nevertheless they are sustained by the hope that England will offer them “a better life.” Figure 6 Still from Border showing asylum seekers in the fields of Sangatte ©Laura Waddington 2002Waddington dedicates the film, “for those I met.” It is an attempt to restore the humanity and dignity of the people who are denied individual identities. Waddington refuses to let “those who I met” remain nameless. She names them—Omar, Muhammad, Abdulla—and narrates their individual stories. Border is Waddington’s attempt to return a voice to those who have been systematically dehumanised, by-products of wars in Iraq and Afghanistan. In his classic account of documentary, Bill Nichols describes six modes of documentary representation (99–138). In Border, Waddington is working in the participatory mode, going into the field and participating in the lives of others (115). It is via this mode of representation that Waddington is able to heighten the ethical encounter with the asylum seekers. Waddington was afforded no special status as a filmmaker, but lived as a refugee among the asylum seekers during the six months of filming. At no point are we granted visible access to Waddington, yet we are acutely aware of her presence. She is physically participating in the drama unfolding before her. At times, we become alert to her immediate physical danger, as she too runs through the fields away from the police and their dogs.The suspicious gaze is predicated on maintaining a controlled distance between the spectator and the subject. Michele Aaron (82–123) has recently argued for a model of spectatorship as an intrinsically ethical encounter. Aaron demonstrates that spectatorship is not neutral but always complicit—it is a contract between the spectator and the film. Particularly relevant to the purposes of this essay is her argument concerning the “merging gaze,” where the gaze of the filmmaker and spectator are collapsed. This has the effect of folding the spectator into the film’s narrative (93). Waddington exploits the documentary medium to implicate the spectator into the structure of the film. It is in Waddington’s full participatory immersion into the documentary itself that undermines the conventional distance maintained by the spectator. The spectator can no longer remain neutral as the lines of demarcation between filmmaker and spectator collapse.Waddington was shooting alone with a small video camera at night in extremely low-light conditions. The opening scene is dark and grainy, refusing immediate entry into the film. As our eyes gradually adjust to the light, we realise we are looking at a young man, concealed in the bushes from the menacing glare of the lights of oncoming traffic. Waddington does not afford us the all-perceiving spectatorial mastery over the image. Rather, we are crouching with her as she records the furtive movements of the man. The background sound, a subtle and persistent hum, adds to a growing disquiet, a looming sense of apprehension concerning the fate of these asylum seekers. Figure 7 Grainy still showing the Red Cross camp in Border ©Laura Waddington 2002Waddington’s commentary has been deliberately pared back and her voice over is minimal with extended periods of silence. The camera alternates from meditative, lingering shots taken from the safety offered by the Red Cross camp, to the fields where the shots are truncated and chaotically framed. The actions of the asylum seekers jerk and shudder, producing an image akin to the flicker effect of early silent cinema because the film is not running at the full rate of 24 frames per second. Here the images become blurred to the point of unintelligibility. Like the Sonderkommando photographs, the asylum seekers exist as image-fragments, shards stolen by Waddington’s camera as she too works hard to evade capture. Tension gradually increases throughout the film, cumulating in a riot scene after a decision to close the camp down. The sweeping search lights of the police helicopter remind us of the increased surveillance undertaken by the border patrols. Without the safety of the Red Cross camp, the asylum seekers are offered no protection from the increasing police brutality. With nowhere else to go, the asylum seekers are forced into the town of Sangatte itself, to sleep in the streets. They are huddled together, and there is a faintly discernible chant repeating in the background, calling to the UN for help. At points during the riot scene, Waddington completely cuts the sound, enveloping the film in a haunting silence. We are left with a mute montage of distressing still images recording the clash between the asylum seekers and police. Again, we are reminded of Waddington’s lack of immunity to the violence, as the camera is deliberately knocked from her hand by a police officer. Figure 8 Clash between asylum seekers and police in Border ©Laura Waddington 2002It is via the merged gaze of the camera and the asylum seekers that Waddington exposes the fictional mastery of the spectator’s gaze. The fury of the tear-image is unleashed as the image-gaze absorbs the spectator into its visual field. No longer pacified by the veil, the spectator is unable to retreat to familiar modes of spectatorship to neutralise and disarm the image. With no possible recourse to desire and fantasy, the encounter becomes intrinsically ethical. Refusing to be neutralised by the Lacanian veil, the tear-image resists the anaesthetising effects of recycled and predictable images of asylum seekers.This essay has argued that a suspicious spectator is the product of an iconophobic media gaze. In the endless process of recycling, the critical capacity of the image to engage the viewer becomes progressively disarmed. Didi-Huberman’s reworking of the Lacanian gaze proposes a model of spectatorship designed to disrupt this iconophobic image economy. The veil-image asks little from us as spectators beyond our complicity. Protected by the gaze of the image, the fiction of the all—perceiving spectator is maintained. By abandoning this model of spectatorship as Didi-Huberman and Waddington are asking us to do, the unidirectional relationship between the viewer and the image is undermined. The terms of spectatorship may be relocated from suspicion to an ethical, participatory mode of engagement. We are laying down our weapons to receive the gaze of the Other. ReferencesAaron, Michele. Spectatorship: The Power of Looking On. London: Wallflower, 2007.Border. Waddington, Laura. Love Stream Productions, 2004.Butler, Judith. Precarious Life: The Powers of Mourning and Violence.London: Verso, 2004.Chéroux, Clément, ed. Mémoires des Camps. Photographies des Camps de Concentration et d'Extermination Nazis, 1933-1999. Paris: Marval, 2001.Didi-Huberman, Georges. Images in Spite of All: Four Photographs from Auschwitz. Trans. Lillis, Shane B. Chicago: U of Chicago P, 2008.Didi-Huberman, Georges. Ce Que Nous Voyons, Ce Qui Nous regarde.Critique. Paris: Editions de Minuit, 1992.Lacan, Jacques. The Four Fundamental Concepts of Psycho-Analysis.Trans. Sheridan, Alan. Harmondsworth: Penguin, 1986.Levinas, Emmanuel. "Reality and its Shadow." The Levinas Reader. Ed. Hand, Seán. Oxford: Blackwell, 1989. 130–43.Lyotard, Jean-François. The Differend: Phrases in Dispute. Minneapolis: University of Minnesota Press, 1988.Metz, Christian. The Imaginary Signifier: Psychoanalysis and the Cinema. Bloomington: Indiana U P, 1982.Nichols, Bill. Introduction to Documentary. Bloomington: Indiana U P, 2001.Saxton, Libby. Haunted Images: Film, Ethics, Testimony and the Holocaust. London: Wallflower, 2008.
APA, Harvard, Vancouver, ISO, and other styles
44

Ryan, Robin, and Uncle Ossie Cruse. "Welcome to the Peoples of the Mountains and the Sea: Evaluating an Inaugural Indigenous Cultural Festival." M/C Journal 22, no. 3 (June 19, 2019). http://dx.doi.org/10.5204/mcj.1535.

Full text
Abstract:
IntroductionFestivals, according to Chris Gibson and John Connell, are like “glue”, temporarily sticking together various stakeholders, economic transactions, and networks (9). Australia’s First Nations peoples see festivals as an opportunity to display cultural vitality (Henry 586), and to challenge a history which has rendered them absent (587). The 2017 Australia Council for the Arts Showcasing Creativity report indicates that performing arts by First Nations peoples are under-represented in Australia’s mainstream venues and festivals (1). Large Aboriginal cultural festivals have long thrived in Australia’s northern half, but have been under-developed in the south. Each regional happening develops a cultural landscape connected to a long and intimate relationship with the natural environment.The Far South East coast and mountainous hinterland of New South Wales is rich in pristine landscapes that ground the Yuin and Monaro Nations to Country as the Monaroo Bobberrer Gadu (Peoples of the Mountains and the Sea). This article highlights cross-sector interaction between Koori and mainstream organisations in producing the Giiyong (Guy-Yoong/Welcoming) Festival. This, the first large festival to be held within the Yuin Nation, took place on Aboriginal-owned land at Jigamy, via Eden, on 22 September 2018. Emerging regional artists joined national headline acts, most notably No Fixed Address (one of the earliest Aboriginal bands to break into the Australian mainstream music industry), and hip-hop artist Baker Boy (Danzal Baker, Young Australian of the Year 2019). The festival followed five years of sustained community preparation by South East Arts in association with Grow the Music, Twofold Aboriginal Corporation, the Eden Local Aboriginal Land Council, and its Elders. We offer dual understandings of the Giiyong Festival: the viewpoints of a male Yuin Elder wedded to an Australian woman of European descent. We acknowledge, and rely upon, key information, statistics, and photographs provided by the staff of South East Arts including Andrew Gray (General Manager), Jasmin Williams (Aboriginal Creative and Cultural Engagement Officer and Giiyong Festival Project Manager), and Kate Howarth (Screen Industry Development Officer). We are also grateful to Wiradjuri woman Alison Simpson (Program Manager at Twofold Aboriginal Corporation) for valuable feedback. As community leaders from First Nations and non-First Nations backgrounds, Simpson and Williams complement each other’s talents for empowering Indigenous communities. They plan a 2020 follow-up event on the basis of the huge success of the 2018 festival.The case study is informed by our personal involvement with community. Since the general population barely comprehends the number and diversity of Australia’s Indigenous ‘nations’, the burgeoning Indigenous festival movement encourages First Nations and non-First Nations peoples alike to openly and confidently refer to the places they live in according to Indigenous names, practices, histories, and knowledge. Consequently, in the mental image of a map of the island-continent, the straight lines and names of state borders fade as the colours of the Indigenous ‘Countries’ (represented by David Horton’s wall map of 1996) come to the foreground. We reason that, in terms of ‘regionality,’ the festival’s expressions of “the agency of country” (Slater 141) differ vastly from the centre-periphery structure and logic of the Australian colony. There is no fixed centre to the mutual exchange of knowledge, culture, and experience in Aboriginal Australia. The broader implication of this article is that Indigenous cultural festivals allow First Nations peoples cultures—in moments of time—to assume precedence, that is to ‘stitch’ back together the notion of a continent made up of hundreds of countries, as against the exploitative structure of ‘hub and region’ colonial Australia.Festival Concepts and ContextsHoward Becker observed that cultural production results from an interplay between the person of the artist and a multitude of support personnel whose work is not frequently studied: “It is through this network of cooperation that the art work we eventually see or hear comes to be and continues to be” (1). In assisting arts and culture throughout the Bega Valley, Eurobodalla, and Snowy Monaro, South East Arts delivers positive achievements in the Aboriginal arts and cultural sector. Their outcomes are significant in the light of the dispossession, segregation, and discrimination experienced by Aboriginal Australians. Michael Young, assisted by Indigenous authors Ellen Mundy and Debbie Mundy, recorded how Delegate Reserve residents relocating to the coast were faced with having their lives controlled by a Wallaga Lake Reserve manager or with life on the fringes of the towns in shacks (2–3). But as discovered in the records, “their retention of traditional beliefs, values and customs, reveal that the accommodation they were forced to make with the Europeans did not mean they had surrendered. The proof of this is the persistence of their belief in the value of their culture” (3–4). The goal of the Twofold Aboriginal Corporation is to create an inclusive place where Aboriginal people of the Twofold Bay Region can be proud of their heritage, connect with the local economy, and create a real future for their children. When Simpson told Williams of the Twofold Aboriginal Corporation’s and Eden Local Aboriginal Land Council’s dream of housing a large cultural festival at Jigamy, Williams rigorously consulted local Indigenous organisations to build a shared sense of community ownership of the event. She promoted the festival as “a rare opportunity in our region to learn about Aboriginal culture and have access to a huge program of Aboriginal musicians, dancers, visual artists, authors, academics, storytellers, cooks, poets, creative producers, and films” (McKnight).‘Uncle Ossie’ Cruse of Eden envisaged that the welcoming event would enliven the longstanding caring and sharing ethos of the Yuin-Monaro people. Uncle Ossie was instrumental in establishing Jigamy’s majestic Monaroo Bobberrer Gudu Keeping Place with the Eden Local Aboriginal Land Council in 1994. Built brick by brick by Indigenous workers, it is a centre for the teaching and celebration of Aboriginal culture, and for the preservation of artefacts. It represents the local community's determination to find their own solutions for “bridging the gap” by creating education and employment opportunities. The centre is also the gateway to the Bundian Way, the first Aboriginal pathway to be listed on the NSW State Heritage Register. Festival Lead-Up EventsEden’s Indigenous students learn a revived South Coast language at Primary and Secondary School. In 2015, Uncle Ossie vitally informed their input into The Black Ducks, a hip-hop song filmed in Eden by Desert Pea Media. A notable event boosting Koori musical socialisation was a Giiyong Grow the Music spectacle performed at Jigamy on 28 October 2017. Grow the Music—co-founded by Lizzy Rutten and Emily White—specialises in mentoring Indigenous artists in remote areas using digital recording equipment. Eden Marine High School students co-directed the film Scars as part of a programme of events with South East Arts and the Giiyong Festival 2018. The Eden Place Project and Campbell Page also create links between in- and out-of-school activities. Eden’s Indigenous students thus perform confidently at NAIDOC Week celebrations and at various festivals. Preparation and PersonnelAn early decision was made to allow free entry to the Giiyong Festival in order to attract a maximum number of Indigenous families. The prospect necessitated in-kind support from Twofold Aboriginal Corporation staff. They galvanised over 100 volunteers to enhance the unique features of Jigamy, while Uncle Ossie slashed fields of bushes to prepare copious parking space. The festival site was spatially focused around two large stages dedicated to the memory of two strong supporters of cultural creativity: Aunty Doris Kirby, and Aunty Liddy Stewart (Image 1). Image 1: Uncle Ossie Cruse Welcomes Festival-Goers to Country on the Aunty Liddy Stewart Stage. Image Credit: David Rogers for South East Arts, Reproduction Courtesy of South East Arts.Cultural festivals are peaceful weapons in a continuing ontological political contest (Slater 144). In a panel discussion, Uncle Ossie explained and defended the Makarrata: the call for a First Nations Voice to be enshrined in the Constitution.Williams also contracted artists with a view to capturing the past and present achievements of Aboriginal music. Apart from her brilliant centrepiece acts No Fixed Address and Baker Boy, she attracted Pitjantjatjara singer Frank Yamma (Image 2), Yorta Yorta singer/songwriter Benny Walker, the Central Desert Docker River Band, and Jessie Lloyd’s nostalgic Mission Songs Project. These stellar acts were joined by Wallaga Lake performers Robbie Bundle, Warren Foster, and Alison Walker as well as Nathan Lygon (Eden), Chelsy Atkins (Pambula), Gabadoo (Bermagui), and Drifting Doolgahls (Nowra). Stage presentations were technologically transformed by the live broadcast of acts on large screens surrounding the platforms. Image 2: Singer-Songwriter Frank Yamma Performs at Giiyong Festival 2018. Image Credit: David Rogers for South East Arts, Reproduction Courtesy of South East Arts.Giiyong Music and Dance Music and dance form the staple components of Indigenous festivals: a reflection on the cultural strength of ancient ceremony. Hundreds of Yuin-Monaro people once attended great corroborees on Mumbulla Mountain (Horton 1235), and oral history recorded by Janet Mathews evidences ceremonies at Fishy Flats, Eden, in the 1850s. Today’s highly regarded community musicians and dancers perform the social arrangements of direct communication, sometimes including their children on stage as apprentices. But artists are still negotiating the power structures through which they experience belonging and detachment in the representation of their musical identity.Youth gain positive identities from participating alongside national headline acts—a form of learning that propels talented individuals into performing careers. The One Mob Dreaming Choir of Koori students from three local schools were a popular feature (Image 3), as were Eden Marine student soloists Nikai Stewart, and Nikea Brooks. Grow the Music in particular has enabled these youngsters to exhibit the roots of their culture in a deep and touching way that contributes to their life-long learning and development. Image 3: The One Mob Dreaming Choir, Directed by Corinne Gibbons (L) and Chelsy Atkins (R). Image Credit: David Rogers for South East Arts, Reproduction Courtesy of South East Arts. Brydie-Leigh Bartleet describes how discourses of pride emerge when Indigenous Australian youth participate in hip-hop. At the Giiyong Festival the relationship between musical expression, cultural representation, and political positioning shone through the songs of Baker Boy and Gabadoo (Image 4). Channelling emotions into song, they led young audiences to engage with contemporary themes of Indigeneity. The drones launched above the carpark established a numerical figure close on 6,000 attendees, a third of whom were Indigenous. Extra teenagers arrived in time for Baker Boy’s evening performance (Williams), revealing the typical youthful audience composition associated with the hip-hop craze (Image 5).Image 4: Bermagui Resident Gabadoo Performs Hip-Hop at the Giiyong Festival. Image Credit: David Rogers for South East Arts, Reproduced Courtesy South East Arts.Image 5: A Youthful Audience Enjoys Baker Boy’s Giiyong Festival Performance. Image Credit: David Rogers for South East Arts, Reproduced Courtesy South East Arts.Wallaga Lake’s traditional Gulaga Dancers were joined by Bermagui’s Gadhu Dancers, Eden’s Duurunu Miru Dancers, and Narooma’s Djaadjawan Dancers. Sharon Mason founded Djaadjawan Dancers in 2015. Their cultural practice connects to the environment and Mingagia (Mother Earth). At their festival tent, dancers explained how they gather natural resources from Walbanja Country to hand-make traditional dance outfits, accessories, and craft. They collect nuts, seeds, and bark from the bush, body paint from ancient ochre pits, shells from beaches, and bird feathers from fresh roadkill. Duurunu Miru dancer/didjeriduist Nathan Lygon elaborates on the functions of the Far South East Coast dance performance tradition:Dance provides us with a platform, an opportunity to share our stories, our culture, and our way of being. It demonstrates a beautiful positivity—a feeling of connection, celebration, and inclusion. The community needs it. And our young people need a ‘space’ in which they can grow into the knowledge and practices of their culture. The festival also helped the wider community to learn more about these dimensions. (n.p.)While music and dance were at the heart of the festival, other traditional skills were included, for example the exhibitions mounted inside the Keeping Place featured a large number of visual artists. Traditional bush cooking took place near Lake Pambula, and yarn-ups, poetry, and readings were featured throughout the day. Cultural demonstrations in the Bunaan Ring (the Yuin name for a corroboree circle) included ‘Gum Leaf Playing.’ Robin Ryan explained how the Yuin’s use of cultural elements to entertain settlers (Cameron 79) led to the formation of the Wallaga Lake Gum Leaf Band. As the local custodian of this unique musical practice, Uncle Ossie performed items and conducted a workshop for numerous adults and children. Festival Feedback and Future PlanningThe Giiyong Festival gained huge Indigenous cultural capital. Feedback gleaned from artists, sponsors, supporters, volunteers, and audiences reflected on how—from the moment the day began—the spirit of so many performers and consumers gathered in one place took over. The festival’s success depended on its reception, for as Myers suggests: “It is the audience who create the response to performance and if the right chemistry is achieved the performers react and excel in their presentation” (59). The Bega District News, of 24 September 2018, described the “incredibly beautiful event” (n.p.), while Simpson enthused to the authors:I believe that the amount of people who came through the gates to attend the Giiyong Festival was a testament to the wider need and want for Aboriginal culture. Having almost double the population of Eden attend also highlights that this event was long overdue. (n.p.)Williams reported that the whole festival was “a giant exercise in the breaking down of walls. Some signed contracts for the first time, and all met their contracts professionally. National artists Baker Boy and No Fixed Address now keep in touch with us regularly” (Williams). Williams also expressed her delight that local artists are performing further afield this year, and that an awareness, recognition, and economic impact has been created for Jigamy, the Giiyong Festival, and Eden respectively:We believe that not only celebrating, but elevating these artists and Aboriginal culture, is one of the most important things South East Arts can do for the overall arts sector in the region. This work benefits artists, the economy and cultural tourism of the region. Most importantly it feeds our collective spirit, educates us, and creates a much richer place to live. (Giiyong Festival Report 1)Howarth received 150 responses to her post-event survey. All respondents felt welcome, included, and willing to attend another festival. One commented, “not even one piece of rubbish on the ground.” Vanessa Milton, ABC Open Producer for South East NSW, wrote: “Down to the tiniest detail it was so obvious that you understood the community, the audience, the performers and how to bring everyone together. What a coup to pull off this event, and what a gift to our region” (Giiyong Festival Report 4).The total running cost for the event was $257,533, including $209,606 in government grants from local, state, and federal agencies. Major donor Create NSW Regional Partnerships funded over $100,000, and State Aboriginal Affairs gave $6,000. Key corporate sponsors included Bendigo Bank, Snowy Hydro and Waterway Constructions, Local Land Services Bega, and the Eden Fisherman’s Club. Funding covered artists’ fees, staging, the hiring of toilets, and multiple generators, including delivery costs. South East Arts were satisfied with the funding amount: each time a new donation arrived they were able to invite more performers (Giiyong Festival Report 2; Gray; Williams). South East Arts now need to prove they have the leadership capacity, financial self-sufficiency, and material resources to produce another festival. They are planning 2020 will be similar to 2018, provided Twofold Aboriginal Corporation can provide extra support. Since South East Arts exists to service a wider area of NSW, they envisage that by 2024, they would hand over the festival to Twofold Aboriginal Corporation (Gray; Williams). Forthcoming festivals will not rotate around other venues because the Giiyong concept was developed Indigenously at Jigamy, and “Jigamy has the vibe” (Williams). Uncle Ossie insists that the Yuin-Monaro feel comfortable being connected to Country that once had a traditional campsite on the east side. Evaluation and ConclusionAlthough ostensibly intended for entertainment, large Aboriginal festivals significantly benefit the educational, political, and socio-economic landscape of contemporary Indigenous life. The cultural outpourings and dissemination of knowledges at the 2018 Giiyong Festival testified to the resilience of the Yuin-Monaro people. In contributing to the processes of Reconciliation and Recognition, the event privileged the performing arts as a peaceful—yet powerful truth-telling means—for dealing with the state. Performers representing the cultures of far-flung ancestral lands contributed to the reimagining of a First Nations people’s map representing hundreds of 'Countries.’It would be beneficial for the Far South East region to perpetuate the Giiyong Festival. It energised all those involved. But it took years of preparation and a vast network of cooperating people to create the feeling which made the 2018 festival unique. Uncle Ossie now sees aspects of the old sharing culture of his people springing back to life to mould the quality of life for families. Furthermore, the popular arts cultures are enhancing the quality of life for Eden youth. As the cross-sector efforts of stakeholders and volunteers so amply proved, a family-friendly, drug and alcohol-free event of the magnitude of the Giiyong Festival injects new growth into an Aboriginal arts industry designed for the future creative landscape of the whole South East region. AcknowledgementsMany thanks to Andrew Gray and Jasmin Williams for supplying a copy of the 2018 Giiyong Festival Report. We appreciated prompt responses to queries from Jasmin Williams, and from our editor Rachel Franks. We are humbly indebted to our two reviewers for their expert direction.ReferencesAustralian Government. Showcasing Creativity: Programming and Presenting First Nations Performing Arts. Australia Council for the Arts Report, 8 Mar. 2017. 20 May 2019 <https://tnn.org.au/2017/03/showcasing-creativity-programming-and-presenting-first-nations-performing-arts-australia-council/>.Bartleet, Brydie-Leigh. “‘Pride in Self, Pride in Community, Pride in Culture’: The Role of Stylin’ Up in Fostering Indigenous Community and Identity.” The Festivalization of Culture. Eds. Andy Bennett, Jodie Taylor, and Ian Woodward. New York: Routledge, 2014.Becker, Howard S. Art Worlds. 25th anniversary edition. Berkeley: U of California P, 2008.Brown, Bill. “The Monaroo Bubberer [Bobberer] Gudu Keeping Place: A Symbol of Aboriginal Self-determination.” ABC South East NSW, 9 Jul. 2015. 20 May 2019 <http://www.abc.net.au/local/photos/2015/07/09/4270480.htm>.Cameron, Stuart. "An Investigation of the History of the Aborigines of the Far South Coast of NSW in the 19th Century." PhD Thesis. Canberra: Australian National U, 1987. Desert Pea Media. The Black Ducks “People of the Mountains and the Sea.” <https://www.youtube.com/watch?v=8fbJNHAdbkg>.“Festival Fanfare.” Eden Magnet 28 June 2018. 1 Mar. 2019 <edenmagnet.com.au>.Gibson, Chris, and John Connell. Music Festivals and Regional Development in Australia. Aldershot: Ashgate, 2012.Gray, Andrew. Personal Communication, 28 Mar. 2019.Henry, Rosita. “Festivals.” The Oxford Companion to Aboriginal Art and Culture. Eds. Syvia Kleinert and Margot Neale. South Melbourne: Oxford UP, 586–87.Horton, David R. “Yuin.” Encyclopaedia of Aboriginal Australia. Ed. David R. Horton. Canberra: Aboriginal Studies Press, 1994.———. Aboriginal Australia Wall Map Compiled by David Horton. Aboriginal Studies Press, 1996.Lygon, Nathan. Personal Communication, 20 May 2019.Mathews, Janet. Albert Thomas Mentions the Leaf Bands That Used to Play in the Old Days. Cassette recorded at Wreck Bay, NSW on 9 July 1964 for the Australian Institute of Aboriginal and Torres Strait Islanders (AIATSIS). LAA1013. McKnight, Albert. “Giiyong Festival the First of Its Kind in Yuin Nation.” Bega District News 17 Sep. 2018. 1 Mar. 2019 <https://www.begadistrictnews.com.au/story/5649214/giiyong-festival-the-first-of-its-kind-in-yuin-nation/?cs=7523#slide=2>. ———. “Giiyong Festival Celebrates Diverse, Enduring Cultures.” Bega District News 24 Sep. 2018. 1 Mar. 2019 <https://www.begadistrictnews.com.au/story/5662590/giiyong-festival-celebrates-diverse-enduring-cultures-photos-videos/>.Myers, Doug. “The Fifth Festival of Pacific Arts.” Australian Aboriginal Studies 1 (1989): 59–62.Simpson, Alison. Personal Communication, 9 Apr. 2019.Slater, Lisa. “Sovereign Bodies: Australian Indigenous Cultural Festivals and Flourishing Lifeworlds.” The Festivalization of Culture. Eds. Andy Bennett, Jodie Taylor, and Ian Woodward. London: Ashgate, 2014. 131–46.South East Arts. "Giiyong Festival Report." Bega: South East Arts, 2018.———. Giiyong Grow the Music. Poster for Event Produced on Saturday, 28 Oct. 2017. Bega: South East Arts, 2017.Williams, Jasmin. Personal Communication, 28 Mar. 2019.Young, Michael, with Ellen, and Debbie Mundy. The Aboriginal People of the Monaro: A Documentary History. Sydney: NSW National Parks and Wildlife Service, 2000.
APA, Harvard, Vancouver, ISO, and other styles
We offer discounts on all premium plans for authors whose works are included in thematic literature selections. Contact us to get a unique promo code!

To the bibliography