To see the other types of publications on this topic, follow the link: Addresses (data processing).
Journal articles on the topic 'Addresses (data processing)'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the top 50 journal articles for your research on the topic 'Addresses (data processing).'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.
1
Romanchuk, Vitaliy. "Mathematical support and software for data processing in robotic neurocomputer systems." MATEC Web of Conferences 161 (2018): 03004. http://dx.doi.org/10.1051/matecconf/201816103004.
Full textAbstract:
The paper addresses classification and formal definition of neurocomputer systems for robotic complexes, based on the types of associations among their elements. We suggest analytical expressions for performance evaluation in neural computer information processing, aimed at development of methods, algorithms and software that optimize such systems.
2
Hahanov, V. I., V. H. Abdullayev, S. V. Chumachenko, E. I. Lytvynova, and I. V. Hahanova. "IN-MEMORY INTELLIGENT COMPUTING." Radio Electronics, Computer Science, Control, no. 1 (April 2, 2024): 161. http://dx.doi.org/10.15588/1607-3274-2024-1-15.
Full textAbstract:
Context. Processed big data has social significance for the development of society and industry. Intelligent processing of big data is a condition for creating a collective mind of a social group, company, state and the planet as a whole. At the same time, the economy of big data (Data Economy) takes first place in the evaluation of processing mechanisms, since two parameters are very important: speed of data processing and energy consumption. Therefore, mechanisms focused on parallel processing of large data within the data storage center will always be in demand on the IT market.
Objective. The goal of the investigation is to increase the economy of big data (Data Economy) thanks to the analysis of data as truth table addresses for the identification of patterns of production functionalities based on the similarity-difference metric.
Method. Intelligent computing architectures are proposed for managing cyber-social processes based on monitoring and analysis of big data. It is proposed to process big data as truth table addresses to solve the problems of identification, clustering, and classification of patterns of social and production processes. A family of automata is offered for the analysis of big data, such as addresses. The truth table is considered as a reasonable form of explicit data structures that have a useful constant – a standard address routing order. The goal of processing big data is to make it structured using a truth table for further identification before making actuator decisions. The truth table is considered as a mechanism for parallel structuring and packing of large data in its column to determine their similarity-difference and to equate data at the same addresses. Representation of data as addresses is associated with unitary encoding of patterns by binary vectors on the found universe of primitive data. The mechanism is focused on processorless data processing based on read-write transactions using in-memory computing technology with significant time and energy savings. The metric of truth table big data processing is parallelism, technological simplicity, and linear computational complexity. The price for such advantages is the exponential memory costs of storing explicit structured data.
Results. Parallel algorithms of in-memory computing are proposed for economic mechanisms of transformation of large unstructured data, such as addresses, into useful structured data. An in-memory computing architecture with global feedback and an algorithm for matrix parallel processing of large data such as addresses are proposed. It includes a framework for matrix analysis of big data to determine the similarity between vectors that are input to the matrix sequencer. Vector data analysis is transformed into matrix computing for big data processing. The speed of the parallel algorithm for the analysis of big data on the MDV matrix of deductive vectors is linearly dependent on the number of bits of the input vectors or the power of the universe of primitives. A method of identifying patterns using key words has been developed. It is characterized by the use of unitary coded data components for the synthesis of the truth table of the business process. This allows you to use read-write transactions for parallel processing of large data such as addresses.
Conclusions. The scientific novelty consists in the development of the following innovative solutions: 1) a new vector-matrix technology for parallel processing of large data, such as addresses, is proposed, characterized by the use of read-write transactions on matrix memory without the use of processor logic; 2) an in-memory computing architecture with global feedback and an algorithm for matrix parallel processing of large data such as addresses are proposed; 3) a method of identifying patterns using keywords is proposed, which is characterized by the use of unitary coded data components for the synthesis of the truth table of the business process, which makes it possible to use the read-write transaction for parallel processing of large data such as addresses. The practical significance of the study is that any task of artificial intelligence (similarity-difference, classification-clustering and recognition, pattern identification) can be solved technologically simply and efficiently with the help of a truth table (or its derivatives) and unitarily coded big data . Research prospects are related to the implementation of this digital modeling technology devices on the EDA market. KEYWORDS: Intelligent
3
Gururaj T. and Siddesh G. M. "Hybrid Approach for Enhancing Performance of Genomic Data for Stream Matching." International Journal of Cognitive Informatics and Natural Intelligence 15, no. 4 (October 2021): 1–18. http://dx.doi.org/10.4018/ijcini.20211001.oa38.
Full textAbstract:
In gene expression analysis, the expression levels of thousands of genes are analyzed, such as separate stages of treatments or diseases. Identifying particular gene sequence pattern is a challenging task with respect to performance issues. The proposed solution addresses the performance issues in genomic stream matching by involving assembly and sequencing. Counting the k-mer based on k-input value and while performing DNA sequencing tasks, the researches need to concentrate on sequence matching. The proposed solution addresses performance issue metrics such as processing time for k-mer counting, number of operations for matching similarity, memory utilization while performing similarity search, and processing time for stream matching. By suggesting an improved algorithm, Revised Rabin Karp(RRK) for basic operation and also to achieve more efficiency, the proposed solution suggests a novel framework based on Hadoop MapReduce blended with Pig & Apache Tez. The measure of memory utilization and processing time proposed model proves its efficiency when compared to existing approaches.
4
Sun, Xihuang, Peng Liu, Yan Ma, Dingsheng Liu, and Yechao Sun. "Streaming Remote Sensing Data Processing for the Future Smart Cities." International Journal of Distributed Systems and Technologies 7, no. 1 (January 2016): 1–14. http://dx.doi.org/10.4018/ijdst.2016010101.
Full textAbstract:
The explosion of data and the increase in processing complexity, together with the increasing needs of real-time processing and concurrent data access, make remote sensing data streaming processing a wide research area to study. This paper introduces current situation of remote sensing data processing and how timely remote sensing data processing can help build future smart cities. Current research on remote sensing data streaming is also introduced where the three typical and open-source stream processing frameworks are introduced. This paper also discusses some design concerns for remote sensing data streaming processing systems, such as data model and transmission, system model, programming interfaces, storage management, availability, etc. Finally, this research specifically addresses some of the challenges of remote sensing data streaming processing, such as scalability, fault tolerance, consistency, load balancing and throughput.
5
Krishnamurthi, Rajalakshmi, Adarsh Kumar, Dhanalekshmi Gopinathan, Anand Nayyar, and Basit Qureshi. "An Overview of IoT Sensor Data Processing, Fusion, and Analysis Techniques." Sensors 20, no. 21 (October 26, 2020): 6076. http://dx.doi.org/10.3390/s20216076.
Full textAbstract:
In the recent era of the Internet of Things, the dominant role of sensors and the Internet provides a solution to a wide variety of real-life problems. Such applications include smart city, smart healthcare systems, smart building, smart transport and smart environment. However, the real-time IoT sensor data include several challenges, such as a deluge of unclean sensor data and a high resource-consumption cost. As such, this paper addresses how to process IoT sensor data, fusion with other data sources, and analyses to produce knowledgeable insight into hidden data patterns for rapid decision-making. This paper addresses the data processing techniques such as data denoising, data outlier detection, missing data imputation and data aggregation. Further, it elaborates on the necessity of data fusion and various data fusion methods such as direct fusion, associated feature extraction, and identity declaration data fusion. This paper also aims to address data analysis integration with emerging technologies, such as cloud computing, fog computing and edge computing, towards various challenges in IoT sensor network and sensor data analysis. In summary, this paper is the first of its kind to present a complete overview of IoT sensor data processing, fusion and analysis techniques.
6
Nguyen, Minh Duc. "A Scientific Workflow System for Satellite Data Processing with Real-Time Monitoring." EPJ Web of Conferences 173 (2018): 05012. http://dx.doi.org/10.1051/epjconf/201817305012.
Full textAbstract:
This paper provides a case study on satellite data processing, storage, and distribution in the space weather domain by introducing the Satellite Data Downloading System (SDDS). The approach proposed in this paper was evaluated through real-world scenarios and addresses the challenges related to the specific field. Although SDDS is used for satellite data processing, it can potentially be adapted to a wide range of data processing scenarios in other fields of physics.
7
Prabagar, S., Vinay K. Nassa, Senthil V. M, Shilpa Abhang, Pravin P. Adivarekar, and Sridevi R. "Python-based social science applications’ profiling and optimization on HPC systems using task and data parallelism." Scientific Temper 14, no. 03 (September 26, 2023): 870–76. http://dx.doi.org/10.58414/scientifictemper.2023.14.3.48.
Full textAbstract:
This research addresses the pressing need to optimize Python-based social science applications for high-performance computing (HPC)systems, emphasizing the combined use of task and data parallelism techniques. The paper delves into a substantial body of research,recognizing Python’s interpreted nature as a challenge for efficient social science data processing. The paper introduces a Pythonprogram that exemplifies the proposed methodology. This program uses task parallelism with multi-processing and data parallelismwith dask to optimize data processing workflows. It showcases how researchers can effectively manage large datasets and intricatecomputations on HPC systems. The research offers a comprehensive framework for optimizing Python-based social science applicationson HPC systems. It addresses the challenges of Python’s performance limitations, data-intensive processing, and memory efficiency.Incorporating insights from a rich literature survey, it equips researchers with valuable tools and strategies for enhancing the efficiencyof their social science applications in HPC environments.
8
Šprem, Šimun, Nikola Tomažin, Jelena Matečić, and Marko Horvat. "Building Advanced Web Applications Using Data Ingestion and Data Processing Tools." Electronics 13, no. 4 (February 9, 2024): 709. http://dx.doi.org/10.3390/electronics13040709.
Full textAbstract:
Today, advanced websites serve as robust data repositories that constantly collect various user-centered information and prepare it for subsequent processing. The data collected can include a wide range of important information from email addresses, usernames, and passwords to demographic information such as age, gender, and geographic location. User behavior metrics are also collected, including browsing history, click patterns, and time spent on pages, as well as different preferences like product selection, language preferences, and individual settings. Interactions, device information, transaction history, authentication data, communication logs, and various analytics and metrics contribute to the comprehensive range of user-centric information collected by websites. A method to systematically ingest and transfer such differently structured information to a central message broker is thoroughly described. In this context, a novel tool—Dataphos Publisher—for the creation of ready-to-digest data packages is presented. Data acquired from the message broker are employed for data quality analysis, storage, conversion, and downstream processing. A brief overview of the commonly used and freely available tools for data ingestion and processing is also provided.
9
Chatzakis, Manos, Panagiota Fatourou, Eleftherios Kosmas, Themis Palpanas, and Botao Peng. "Odyssey: A Journey in the Land of Distributed Data Series Similarity Search." Proceedings of the VLDB Endowment 16, no. 5 (January 2023): 1140–53. http://dx.doi.org/10.14778/3579075.3579087.
Full textAbstract:
This paper presents Odyssey, a novel distributed data-series processing framework that efficiently addresses the critical challenges of exhibiting good speedup and ensuring high scalability in data series processing by taking advantage of the full computational capacity of modern distributed systems comprised of multi-core servers. Odyssey addresses a number of challenges in designing efficient and highly-scalable distributed data series index, including efficient scheduling, and load-balancing without paying the prohibitive cost of moving data around. It also supports a flexible partial replication scheme, which enables Odyssey to navigate through a fundamental trade-off between data scalability and good performance during query answering. Through a wide range of configurations and using several real and synthetic datasets, our experimental analysis demonstrates that Odyssey achieves its challenging goals.
10
Vijaya, Dr V. Krishna. "INVOICE DATA EXTRACTION USING OCR TECHNIQUE." INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT 08, no. 04 (April 9, 2024): 1–5. http://dx.doi.org/10.55041/ijsrem29981.
Full textAbstract:
Traditional invoice processing involves manual entry of data, leading to human errors, delays,and increased operational costs. The lack of automation results in inefficiencies, hindering organizations from promptly accessing critical financial information. This research addresses the pressing need for a reliable OCR-based solution to automate invoice data extraction, ultimately improving accuracy, reducing processing time, and enhancing overall business productivity. The project aims to automate invoice data extraction through Optical Character Recognition (OCR) techniques. Leveraging advanced image processing and machine learning, the system will analyze scanned or photographed invoices, extracting relevant information such as vendor details, itemized costs, and dates.This automation streamlines manual data entry processes, enhancing accuracy and efficiency in managing financial records. OCR invoicing is the process of training a template-based OCR model for specific invoice layouts, setting up input paths for these invoices, extracting data, and integrating the extracted data with a structured database. Keywords: Invoice, OCR, YOLO algorithm, Data Extraction, Image Processing, Database Integration.
11
Li, Wenqi, Pengyi Zhang, and Jun Wang. "Humanities Scholars' Understanding of Data and the Implications for Humanities Data Curation." Proceedings of the Association for Information Science and Technology 60, no. 1 (October 2023): 1034–36. http://dx.doi.org/10.1002/pra2.936.
Full textAbstract:
ABSTRACTThis study addresses the need for a shared understanding of humanities data to enhance data curation. Through interviews with 27 scholars, it identifies two ways scholars conceptualize data ‐ by format or role in research. It highlights three unique aspects: diverse requirements of materiality and processing levels, significance of authorship and perspective, and the dual role of tertiary sources. The study suggests prioritizing provenance, facilitating data documentation, curating tertiary sources for wider use, and establishing scholarly communication mechanisms for effective data curation.
12
Moustakidis, Serafeim, Athanasios Anagnostis, Apostolos Chondronasios, Patrik Karlsson, and Kostas Hrissagis. "Excitation-invariant pre-processing of thermographic data." Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability 232, no. 4 (April 23, 2018): 435–46. http://dx.doi.org/10.1177/1748006x18770888.
Full textAbstract:
There is a large number of industries that make extensive use of composite materials in their respective sectors. This rise in composites’ use has necessitated the development of new non-destructive inspection techniques that focus on manufacturing quality assurance, as well as in-service damage testing. Active infrared thermography is now a popular nondestructive testing method for detecting defects in composite structures. Non-uniform emissivity, uneven heating of the test surface, and variation in thermal properties of the test material are some of the crucial factors in experimental thermography. These unwanted thermal effects are typically coped with the application of a number of well-established thermographic techniques including pulse phase thermography and thermographic signal reconstruction. This article addresses this problem of the induced uneven heating at the pre-processing phase prior to the application of the thermographic processing techniques. To accomplish this, a number of excitation invariant pre-processing techniques were developed and tested in this article addressing the unwanted effect of non-uniform excitation in the collected thermographic data. Various fitting approaches were validated in light of modeling the non-uniform heating effect, and new normalization approaches were proposed following a time-dependent framework. The proposed pre-processing techniques were validated on a testing composite sample with pre-determined defects. The results demonstrated the effectiveness of the proposed processing algorithms in terms of removing the unwanted heat distribution effect along with the signal-to-noise ratio of the produced infrared images.
13
Crosetto, M., N. Devanthéry, M. Cuevas-González, O. Monserrat, and B. Crippa. "Exploitation of the full potential of Persistent Scatterer Interferometry data." ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XL-7 (September 19, 2014): 75–78. http://dx.doi.org/10.5194/isprsarchives-xl-7-75-2014.
Full textAbstract:
The potential of Persistent Scatterer Interferometry (PSI) for deformation monitoring has been increasing in the last years and it will continue to do so in the short future, especially with the advent of the Sentinel-1 mission. The full exploitation of this potential requires two important components. The first one is the improvement of the PSI processing tools, to achieve massive and systematic data processing capabilities. The second one is the need to increase the capabilities to correctly analyze and interpret the PSI results. The paper addresses both components. The key features of the PSI processing chain implemented by the authors, which is named PSIG chain, are described. This is followed by a brief discussion of the key elements needed to analyse and interpret the results of a given PSI processing. The paper concludes with a description of the results obtained by processing a full frame of very high resolution TerraSAR-X data that covers the metropolitan area of Barcelona (Spain).
14
Shkirdov, D. A., E. S. Sagatov, and P. S. Dmitrenko. "Trap method in ensuring data security." Information Technology and Nanotechnology, no. 2416 (2019): 189–98. http://dx.doi.org/10.18287/1613-0073-2019-2416-189-198.
Full textAbstract:
This paper presents the results of data analysis from a geographically distributed honeypot network. Such honeypot servers were deployed in Samara, Rostov on Don, Crimea and the USA two years ago. Methods for processing statistics are discussed in detail for secure remote access SSH. Lists of attacking addresses are highlighted, and their geographical affiliation is determined. Rank distributions were used as the basis for statistical analysis. The intensity of requests to each of the 10 installed services was then calculated.
15
Leja, Laura, Vitālijs Purlans, Rihards Novickis, Andrejs Cvetkovs, and Kaspars Ozols. "Mathematical Model and Synthetic Data Generation for Infra-Red Sensors." Sensors 22, no. 23 (December 3, 2022): 9458. http://dx.doi.org/10.3390/s22239458.
Full textAbstract:
A key challenge in further improving infrared (IR) sensor capabilities is the development of efficient data pre-processing algorithms. This paper addresses this challenge by providing a mathematical model and synthetic data generation framework for an uncooled IR sensor. The developed model is capable of generating synthetic data for the design of data pre-processing algorithms of uncooled IR sensors. The mathematical model accounts for the physical characteristics of the focal plane array, bolometer readout, optics and the environment. The framework permits the sensor simulation with a range of sensor configurations, pixel defectiveness, non-uniformity and noise parameters.
16
J.J.Jayakanth. "Dynamic Object Detection in Surveillance Videos using Temporal Convolutional Networks and Federated Learning in Edge Computing Environments." Journal of Electrical Systems 20, no. 5s (April 13, 2024): 2009–15. http://dx.doi.org/10.52783/jes.2537.
Full textAbstract:
This research addresses the importance of advancing dynamic object detection in surveillance videos by introducing a novel framework that integrates Temporal Convolutional Networks (TCNs) and Federated Learning (FL) within edge computing environments. This research is motivated by the critical need for real-time threat response, enhanced security measures, and privacy preservation in dynamic surveillance scenarios. Leveraging TCNs, the system captures temporal dependencies, providing a comprehensive understanding of object movements. FL ensures decentralized model training, mitigating privacy concerns associated with centralized approaches. Current challenges in real-time processing, privacy preservation, and adaptability to dynamic environments are addressed through innovative solutions. Model optimization techniques optimize TCN efficiency, ensuring real-time processing. Advanced privacy-preserving mechanisms secure FL, addressing privacy concerns. Transfer learning and data augmentation enhance adaptability to dynamic scenarios. The proposed system not only addresses existing challenges but also contributes to the evolution of surveillance technology. Comprehensive metrics, including accuracy, sensitivity, specificity, and real-time processing metrics, provide a thorough evaluation of the system's performance. This research introduces an approach to dynamic object detection, combining TCN and FL in edge computing environments. Results show accuracy exceeding 97%, sensitivity and specificity at 97% and 98%, and F1 score reaching 96%.
17
Moore, Philip T., and Hai V. Pham. "Personalization and rule strategies in data-intensive intelligent context-aware systems." Knowledge Engineering Review 30, no. 2 (March 2015): 140–56. http://dx.doi.org/10.1017/s0269888914000265.
Full textAbstract:
AbstractThe concept of personalization in its many forms has gained traction driven by the demands of computer-mediated interactions generally implemented in large-scale distributed systems and ad hoc wireless networks. Personalization requires the identification and selection of entities based on a defined profile (a context); an entity has been defined as a person, place, or physical or computational object. Context employs contextual information that combines to describe an entities current state. Historically, the range of contextual information utilized (in context-aware systems) has been limited to identity, location, and proximate data; there has, however, been advances in the range of data and information addressed. As such, context can be highly dynamic with inherent complexity. In addition, context-aware systems must accommodate constraint satisfaction and preference compliance.This article addresses personalization and context with consideration of the domains and systems to which context has been applied and the nature of the contextual data. The developments in computing and service provision are addressed with consideration of the relationship between the evolving computing landscape and context. There is a discussion around rule strategies and conditional relationships in decision support. Logic systems are addressed with an overview of the open world assumption versus the closed world assumption and the relationship with the Semantic Web. The event-driven rule-based approach, which forms the basis upon which intelligent context processing can be realized, is presented with an evaluation and proof-of-concept. The issues and challenges identified in the research are considered with potential solutions and research directions; alternative approaches to context processing are discussed. The article closes with conclusions and open research questions.
18
Comandè, Giovanni, and Giulia Schneider. "Regulatory Challenges of Data Mining Practices: The Case of the Never-ending Lifecycles of ‘Health Data’." European Journal of Health Law 25, no. 3 (April 18, 2018): 284–307. http://dx.doi.org/10.1163/15718093-12520368.
Full textAbstract:
Abstract Health data are the most special of the ‘special categories’ of data under Art. 9 of the General Data Protection Regulation (GDPR). The same Art. 9 GDPR prohibits, with broad exceptions, the processing of ‘data concerning health’. Our thesis is that, through data mining technologies, health data have progressively undergone a process of distancing from the healthcare sphere as far as the generation, the processing and the uses are concerned. The case study aims thus to test the endurance of the ‘special category’ of health data in the face of data mining technologies and the never-ending lifecycles of health data they feed. At a more general level of analysis, the case of health data shows that data mining techniques challenge core data protection notions, such as the distinction between sensitive and non-sensitive personal data, requiring a shift in terms of systemic perspectives that the GDPR only partly addresses.
19
Ganesan, Madhubala, Ah-Lian Kor, Colin Pattinson, and Eric Rondeau. "Green Cloud Software Engineering for Big Data Processing." Sustainability 12, no. 21 (November 7, 2020): 9255. http://dx.doi.org/10.3390/su12219255.
Full textAbstract:
Internet of Things (IoT) coupled with big data analytics is emerging as the core of smart and sustainable systems which bolsters economic, environmental and social sustainability. Cloud-based data centers provide high performance computing power to analyze voluminous IoT data to provide invaluable insights to support decision making. However, multifarious servers in data centers appear to be the black hole of superfluous energy consumption that contributes to 23% of the global carbon dioxide (CO2) emissions in ICT (Information and Communication Technology) industry. IoT-related energy research focuses on low-power sensors and enhanced machine-to-machine communication performance. To date, cloud-based data centers still face energy–related challenges which are detrimental to the environment. Virtual machine (VM) consolidation is a well-known approach to affect energy-efficient cloud infrastructures. Although several research works demonstrate positive results for VM consolidation in simulated environments, there is a gap for investigations on real, physical cloud infrastructure for big data workloads. This research work addresses the gap of conducting real physical cloud infrastructure-based experiments. The primary goal of setting up a real physical cloud infrastructure is for the evaluation of dynamic VM consolidation approaches which include integrated algorithms from existing relevant research. An open source VM consolidation framework, Openstack NEAT is adopted and experiments are conducted on a Multi-node Openstack Cloud with Apache Spark as the big data platform. Open sourced Openstack has been deployed because it enables rapid innovation, and boosts scalability as well as resource utilization. Additionally, this research work investigates the performance based on service level agreement (SLA) metrics and energy usage of compute hosts. Relevant results concerning the best performing combination of algorithms are presented and discussed.
20
Juneja, Poonam, Rachana Garg, and Parmod Kumar. "Uncertain data processing of PMU modules using fuzzy Petri net." Journal of Intelligent & Fuzzy Systems 41, no. 1 (August 11, 2021): 1855–67. http://dx.doi.org/10.3233/jifs-210602.
Full textAbstract:
The paper presents a novel method for processing uncertain data of Phasor measurement unit (PMU) modules first time in the literature using Fuzzy Reasoning Petri net (FPN). It addresses several key issues such as exploitation of Petri net representation from operating state of PMU to its failure state whereas Fuzzy logic is used to deal with the uncertain data of PMU modules. Sprouting tree, an information flow path, of PMU failure is drawn due to various components and estimation accuracy can be enhanced by integration of more truthiness input data. Fault tree diagram, Fuzzy Petri net model (FPN), production rule sets for PMU are developed and finally degree of truthiness of proposition is computed from sprouting tree. Fuzzy logic reasoning is used for routing the sprouting tree whereas Petri net is employed for dynamics of states due to failure of modules of PMU. The fusion of two technologies is made for the dynamic response, processing and reasoning to sprouting tree information flow from operating state to unavailability of PMU. The research work is useful to pinpoint the weakness in design of modules of PMU and to assess its reliability.
21
Sasmal, Shubhodip. "Real-time Data Processing with Machine Learning Algorithms." INTERNATIONAL RESEARCH JOURNAL OF ENGINEERING & APPLIED SCIENCES 11, no. 4 (2023): 91–96. http://dx.doi.org/10.55083/irjeas.2023.v11i04012.
Full textAbstract:
In the era of information abundance, organizations are faced with the challenge of harnessing real-time data streams to extract valuable insights swiftly. This research paper explores the intersection of real-time data processing and machine learning algorithms, aiming to develop a comprehensive understanding of their integration for efficient decision-making in dynamic environments. The paper begins by delineating the landscape of real-time data processing, emphasizing the significance of timely and accurate information in contemporary business scenarios. It delves into the challenges posed by the velocity and volume of data generated continuously, necessitating advanced processing mechanisms capable of handling data streams in real-time. As the focus shifts to machine learning algorithms, the research outlines the diverse range of algorithms suitable for real-time applications. From online learning methods to streaming algorithms, the exploration encompasses techniques tailored to adapt and evolve with incoming data. This section also addresses the trade-offs between accuracy and computational efficiency, crucial considerations in real-time processing environments. The core of the paper lies in the synthesis of real-time data processing and machine learning algorithms. It investigates how machine learning models can be seamlessly integrated into data processing pipelines to analyze and respond to streaming data instantaneously. Case studies and practical implementations exemplify instances where predictive analytics and anomaly detection algorithms contribute to real-time decision support. Ethical considerations and challenges related to the deployment of machine learning in real-time settings are also examined. The paper advocates for responsible and transparent use of algorithms, emphasizing the importance of mitigating biases and ensuring accountability in decision-making processes driven by machine learning insights. this research paper provides a roadmap for organizations seeking to harness the synergy between real-time data processing and machine learning. The insights gained from this exploration pave the way for advancements in adaptive decision-making systems, offering a competitive edge in industries where rapid response to evolving data is paramount.
22
da Silva, Erico Correia, Liria Matsumoto Sato, and Edson Toshimi Midorikawa. "Distributed File System to Leverage Data Locality for Large-File Processing." Electronics 13, no. 1 (December 26, 2023): 106. http://dx.doi.org/10.3390/electronics13010106.
Full textAbstract:
Over the past decade, significant technological advancements have led to a substantial increase in data proliferation. Both scientific computation and Big Data workloads play a central role, manipulating massive data and challenging conventional high-performance computing architectures. Efficiently processing voluminous files using cost-effective hardware remains a persistent challenge, limiting access to new technologies for individuals and organizations capable of higher investments. In response to this challenge, AwareFS, a novel distributed file system, addresses the efficient reading and updating of large files by consistently exploiting data locality on every copy. Its distributed metadata and lock management facilitate sequential and random I/O patterns with minimal data movement over the network. The evaluation of the AwareFS local-write protocol demonstrated efficiency across various update patterns, resulting in a performance improvement of approximately 13%, while benchmark assessments conducted across diverse cluster sizes and configurations underscored the flexibility and scalability of AwareFS. The innovative distributed mechanisms outlined herein are positioned to contribute to the evolution of emerging technologies related to the computation of data stored in large files.
23
Green, Andrew W., Elizabeth Mannering, Lloyd Harischandra, Minh Vuong, Simon O’Toole, Katrina Sealey, and Andrew M. Hopkins. "What will the future of cloud-based astronomical data processing look like?" Proceedings of the International Astronomical Union 12, S325 (October 2016): 27–31. http://dx.doi.org/10.1017/s1743921317001363.
Full textAbstract:
AbstractAstronomy is rapidly approaching an impasse: very large datasets require remote or cloud-based parallel processing, yet many astronomers still try to download the data and develop serial code locally. Astronomers understand the need for change, but the hurdles remain high. We are developing a data archive designed from the ground up to simplify and encourage cloud-based parallel processing. While the volume of data we host remains modest by some standards, it is still large enough that download and processing times are measured in days and even weeks. We plan to implement a python based, notebook-like interface that automatically parallelises execution. Our goal is to provide an interface sufficiently familiar and user-friendly that it encourages the astronomer to run their analysis on our system in the cloud—astroinformatics as a service. We describe how our system addresses the approaching impasse in astronomy using the SAMI Galaxy Survey as an example.
24
Ekanthaiah, Prathibha, Mesfin Megra, Aswathnarayana Manjunatha, and Likhitha Ramalingappa. "Design of FFNN architecture for power quality analysis and its complexity challenges on FPGA." Bulletin of Electrical Engineering and Informatics 11, no. 2 (April 1, 2022): 613–23. http://dx.doi.org/10.11591/eei.v11i2.3293.
Full textAbstract:
As we all know, power quality (PQ) issues are a major concern these days. Field programmable gate array (FPGA) are essential in PQ analysis, particularly in smart meters for data processing, storage, and transmission. One of the most significant advantages of FPGA is its reconfigurability, with vast hardware resources that can be used to implement complex as well as time-critical data processing units. Because the FPGA architecture supports fixed point arithmetic, data loss occurs in the data path unit, necessitating the realization of the PQ event detection module, and classification model to be more accurate than software implementation algorithms. The majority of the work reported, with feed forward neural network (FFNN) structure occupying large number of multipliers and adders for classification, most of the work reported has not addressed to minimize the data path resources for FFNN instead have addressed in improving classification accuracy. Based on these issues, this paper addresses the implementation challenges in FFNN architecture design by proposing improved and fast architectures. The proposed FFNN architecture design using optimum resources. The FFNN based classifier are designed to perform PQ event detection and classification with 99.5% accuracy. The FFNN processor operates at maximum frequency of 238 MHz.
25
Urblik, Lubomir, Erik Kajati, Peter Papcun, and Iveta Zolotova. "A Modular Framework for Data Processing at the Edge: Design and Implementation." Sensors 23, no. 17 (September 4, 2023): 7662. http://dx.doi.org/10.3390/s23177662.
Full textAbstract:
There is a rapid increase in the number of edge devices in IoT solutions, generating vast amounts of data that need to be processed and analyzed efficiently. Traditional cloud-based architectures can face latency, bandwidth, and privacy challenges when dealing with this data flood. There is currently no unified approach to the creation of edge computing solutions. This work addresses this problem by exploring containerization for data processing solutions at the network’s edge. The current approach involves creating a specialized application compatible with the device used. Another approach involves using containerization for deployment and monitoring. The heterogeneity of edge environments would greatly benefit from a universal modular platform. Our proposed edge computing-based framework implements a streaming extract, transform, and load pipeline for data processing and analysis using ZeroMQ as the communication backbone and containerization for scalable deployment. Results demonstrate the effectiveness of the proposed framework, making it suitable for time-sensitive IoT applications.
26
Wang, Xuesong, Zhenyang Xu, and Lianjun Guo. "A New Data Processing Approach for the SHPB Test Based on PSO-TWER." Applied Sciences 14, no. 9 (April 25, 2024): 3624. http://dx.doi.org/10.3390/app14093624.
Full textAbstract:
This study addresses the challenge of accurately determining the arrival time of stress wave signals in SHPB test data processing. To eliminate human error, we introduce the time-window energy ratio method and evaluate six filters for noise reduction using box fractal dimensions. A mathematical model is established to optimize the stress equilibrium and impact process, which is solved using particle swarm optimization, resulting in the PSO-TWER method. We explore the impact of inertia weight and calculation methods on optimization outcomes, defining a stress equilibrium evaluation index. The results indicate that time-window length significantly affects arrival-time outputs, and the dynamic inertia weight factor enhances optimization convergence. The method accurately determines arrival times and effectively screens test data, providing a robust approach for SHPB test data processing.
27
Václavová, Andrea, Pavol Tanuška, and Ján Jánošík. "Data Storing Proposal from Heterogeneous Systems into a Specialized Repository." Research Papers Faculty of Materials Science and Technology Slovak University of Technology 24, no. 39 (December 1, 2016): 123–28. http://dx.doi.org/10.1515/rput-2016-0026.
Full textAbstract:
Abstract The aim of this paper is to analyze and to propose an appropriate system for processing and simultaneously storing a vast volume of structured and unstructured data. The paper consists of three parts. The first part addresses the issue of structured and unstructured data. The second part provides the detailed analysis of data repositories and subsequent evaluation indicating which system would be for the given type and volume of data optimal. The third part focuses on the use of gathered information to transfer data to the proposed repository.
28
Foster, Jonathan. "Towards an understanding of data work in context." Library Hi Tech 34, no. 2 (June 20, 2016): 182–96. http://dx.doi.org/10.1108/lht-12-2015-0121.
Full textAbstract:
Purpose – It is a commonplace that innovation in the digital economy is now driven by data. Business organizations, media companies, and government, for example all create economic and societal value from the digital traces left by the user population. At the same time the data captured also contains information that personally identifies consumers, citizens and patients as individuals. The purpose of this paper is to place this new form of data work in the context of previous approaches to information work; to identify the differences between information and data work and the resulting challenges for data professionals. Design/methodology/approach – Informed by a review of previous approaches to information work, the paper argues that the shift in value from information to data as an economic asset and a societal good entails a new form of human-oriented data work. One that is more sensitive to the contextual conditions and consequences of the capture, processing and use of data than has been the case hitherto. The implications of this for a shift in emphasis from the data scientist to the data professional is addressed, as are emerging issues of governance and ethics. Findings – The main consequence for data professionals is to ensure that processes are in place not only to enable the creation of valued products and services from data, but also to mitigate the risks related to their development. The paper argues that ensuring this involves taking a contextual view that locates data processing within the user, governance, legal, and ethical conditions related to data work. The consequences for the governance of data, and the education of data professionals are addressed. Originality/value – The value of the paper rests in its development of an analytical and methodologically driven framework, that places new forms of data work in the context of their conditions and consequences. The framework builds on prior approaches to information work, current approaches to data work, and addresses the governance, and educational challenges arising from organizations’ emphasis on data-driven innovation in a digital economy.
29
Rafert, J. Bruce, Jaime Zabalza, Stephen Marshall, and Jinchang Ren. "Singular Spectrum Analysis: A Note on Data Processing for Fourier Transform Hyperspectral Imagers." Applied Spectroscopy 70, no. 9 (July 20, 2016): 1582–88. http://dx.doi.org/10.1177/0003702816641420.
Full textAbstract:
Hyperspectral remote sensing is experiencing a dazzling proliferation of new sensors, platforms, systems, and applications with the introduction of novel, low-cost, low-weight sensors. Curiously, relatively little development is now occurring in the use of Fourier transform (FT) systems, which have the potential to operate at extremely high throughput without use of a slit or reductions in both spatial and spectral resolution that thin film based mosaic sensors introduce. This study introduces a new physics-based analytical framework called singular spectrum analysis (SSA) to process raw hyperspectral imagery collected with FT imagers that addresses some of the data processing issues associated with the use of the inverse FT. Synthetic interferogram data are analyzed using SSA, which adaptively decomposes the original synthetic interferogram into several independent components associated with the signal, photon and system noise, and the field illumination pattern.
30
Olasz, A., D. Kristóf, M. Belényesi, K. Bakos, Z. Kovács, B. Balázs, and Sz Szabó. "IQPC 2015 TRACK: WATER DETECTION AND CLASSIFICATION ON MULTISOURCE REMOTE SENSING AND TERRAIN DATA." ISPRS - International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XL-3/W3 (August 20, 2015): 583–88. http://dx.doi.org/10.5194/isprsarchives-xl-3-w3-583-2015.
Full textAbstract:
Since 2013, the EU FP7 research project “IQmulus” encourages the participation of the whole scientific community as well as specific user groups in the IQmulus Processing Contest (IQPC). This year, IQPC 2015 consists of three processing tasks (tracks), from which “Water detection and classification on multi-source remote sensing and terrain data” is introduced in the present paper. This processing track addresses a particular problem in the field of big data processing and management with the objective of simulating a realistic remote sensing application scenario. The main focus is on the detection of water surfaces (natural waters, flood, inland excess water, other water-affected categories) using remotely sensed data. Multiple independent data sources are available and different tools could be used for data processing and evaluation. The main challenge is to identify the right combination of data and methods to solve the problem in the most efficient way. Although the first deadline for submitting track solutions has passed and the track has been successfully concluded, the track organizers decided to keep the possibility of result submission open to enable collecting a variety of approaches and solutions for this interesting problem.
31
Al-Mayali, Yahya Mahdi Hadi, and Zahraa Yahya Mahdi Al-Mayali. "Digitization, and Coding, with Optimizing, of Iraqi Personnel Home Addresses forward, minimizing Storage Space, and Processing time." BIO Web of Conferences 97 (2024): 00028. http://dx.doi.org/10.1051/bioconf/20249700028.
Full textAbstract:
Individual Home Address is considered as an important sub set of attributes, and never find personnel record, whether a person is student, patient, or employee, etc. without having a home address. It is required to be recorded, registered, and documented in all personnel information systems, whether the system is manual or computerized. This research concern on the Coding, and Digitization of home address of Iraqi personnel in order to enhancing storage efficiency and reduce processing time. The conventional method of maintaining paper-based records for personnel home addresses has proven to be costly, and timeconsuming, and in most cases inefficient. The proposed solution involves the conversion of physical addresses into digital formats, allowing for streamlined data storage and faster retrieval, and update processes. The work starting by indicate the current challenges related to the paper-based address systems, including the demands on physical storage space and the delays incurred during information retrieval. Subsequently, it explores the potential benefits of digitizing home addresses, such as reduced storage requirements, improved data accessibility, and enhanced overall organizational efficiency. To implement this digitization process, the research investigates various technological solutions, including geographic information systems (GIS), databases, and data management protocols. The study also addresses potential concerns related to data security and privacy, proposing measures to safeguard sensitive information. The proposed solution for Digitization, Coding, with Optimizing the Personnel Home Address provide at least 85% of Optimizing Factor for each individual personnel record required storage space and processing time, this solution subsequently leed to better daily decision-making process in most business organizations.
32
Pham-Van, Dung, and Thanh-Long Cung. "Image analysis and CNN-based crack depth estimation using eddy current data." Journal of Military Science and Technology 96 (June 25, 2024): 12–20. http://dx.doi.org/10.54939/1859-1043.j.mst.96.2024.12-20.
Full textAbstract:
This study presents a comprehensive approach for crack depth estimation utilizing advanced image analysis techniques and a Convolutional Neural Network (CNN) model. The aim is to enhance accuracy and reliability in predicting crack depths, particularly for sub-millimeter cracks. The research addresses challenges arising from noise in images by employing a pre-processing technique and augmentation methods. The proposed method's effectiveness is showcased through its application to experimental crack data from diverse specimens. The outcomes exhibit a Mean Relative Error (MRE) of around 6%, indicating a high level of precision. These results affirm the potential of the methodology for real-world industrial applications. Additionally, the study explores the integration of eddy current image processing with CNN for Non-Destructive Evaluation (NDE) problems, offering a new approach for tiny surface-crack detection and characterization.
33
Morkovin, S. V., and V. S. Panishchev. "AN OPTINIZED ALGORITHM OF PARALELL DATA PROCESSING BY SNORT CORE." Proceedings of the Southwest State University 21, no. 1 (February 28, 2017): 30–35. http://dx.doi.org/10.21869/2223-1560-2017-21-1-30-35.
Full textAbstract:
The paper addresses the problem of optimizing the firmware algorithm of detecting and preventing computer attacks on the Internet access workstations and networking equipment. The main objective was to boost the device capacity and save data processing resources. It has been proved that existing soft products that have been developed for single thread execution architectures need to be modified. In particular the paper discusses Snort network intrusion and prevention system that initially has been made to operate on the processor single core in single thread mode. Snort core paralleling principle is based on dividing the inbound traffic into lower-speed atomic channels that are distributed over several individually runnable Snort cores as individual processes that are interconnected and can exchange information. The authors suggest the algorithm optimization way that consists in utilizing the fast shared memory to facilitate information exchange between the processes. The paper focuses on a key element in the data processing paralleling algorithm which is the balance algorithm. The proposed algorithm has been used to optimize the performance of the inbound traffic balancing unit, which increased the operation speed of the total system. A test facility has been developed to simulate and refine the constructed intrusion detection distributed system. The paper presents the testing facility structure, testing method and test numerical results. The test item was a standard traffic routed to the system input from backbone link. The research results were used to determine the dependency of the traffic processing speed on the number of cores in the system.
34
Seo, Young Min, Paul F. Goldsmith, Volker Tolls, Russell Shipman, Craig Kulesa, William Peters, Christopher Walker, and Gary Melnick. "Applications of Machine Learning Algorithms in Processing Terahertz Spectroscopic Data." Journal of Astronomical Instrumentation 09, no. 03 (September 2020): 2050011. http://dx.doi.org/10.1142/s2251171720500117.
Full textAbstract:
We present the data reduction software and the distribution of Level 1 and Level 2 products of the Stratospheric Terahertz Observatory 2 (STO2). STO2, a balloon-borne Terahertz telescope, surveyed star-forming regions and the Galactic plane and produced approximately 300,000 spectra. The data are largely similar to spectra typically produced by single-dish radio telescopes. However, a fraction of the data contained rapidly varying fringe/baseline features and drift noise, which could not be adequately corrected using conventional data reduction software. To process the entire science data of the STO2 mission, we have adopted a new method to find proper off-source spectra to reduce large amplitude fringes and new algorithms including Asymmetric Least Square (ALS), Independent Component Analysis (ICA), and Density-based spatial clustering of applications with noise (DBSCAN). The STO2 data reduction software efficiently reduced the amplitude of fringes from a few hundred to 10[Formula: see text]K and resulted in baselines of amplitude down to a few K. The Level 1 products typically have noise of a few K in [CII] spectra and [Formula: see text]1[Formula: see text]K in [NII] spectra. Using a regridding algorithm, we made spectral maps of star-forming regions and the Galactic plane survey using an algorithm employing a Bessel–Gaussian kernel. The level 1 and level 2 products are available to the astronomical community through the STO2 data server and the DataVerse. The software is also accessible to the public through Github. The detailed addresses are given in Sec. 4 of this paper on data distribution.
35
Xiu, Weirong, Md Gapar Md Johar, Mohammed Hazim Alkawaz, and Chen Bian. "Efficient Edge Computing: A Survey of High-Throughput Concurrent Processing Strategies for Graph Data." Journal of Computing and Electronic Information Management 12, no. 3 (April 30, 2024): 101–6. http://dx.doi.org/10.54097/ceym1sxx.
Full textAbstract:
This paper reviews the strategies for high-throughput concurrent processing of graph data in edge computing environments. As information technology rapidly advances, particularly in areas such as the Internet of Things (IoT), smart cities, and autonomous driving, the need for real-time and efficient data processing continues to grow. Edge computing is a distributed computing paradigm that can process data at or near its origin, thereby reducing network latency, improving application performance, and reducing the load on central data centers. The discussion includes the application of edge computing in graph data processing, highlighting parallel computing models such as the Bulk Synchronous Parallel (BSP) model and other parallel optimization techniques like data partitioning and task scheduling. The paper also addresses specific challenges of parallel processing in edge computing, such as resource constraints, data communication delays, and strategies for security and privacy protection. Despite the significant potential edge computing has demonstrated in processing graph data, numerous challenges remain. Future research directions involve the development of new resource optimization algorithms, low-latency communication protocols, and technologies to enhance data security, aiming to achieve more efficient and secure graph data processing. This paper provides clear directions and a foundation for the efficient implementation of graph data processing in edge computing environments, positioning edge computing to play an increasingly significant role in real-time data analysis and intelligent applications.
36
Lu, Rui, and Zhichuan Guo. "An FPGA-Based High-Performance Stateful Packet Processing Method." Micromachines 14, no. 11 (November 8, 2023): 2074. http://dx.doi.org/10.3390/mi14112074.
Full textAbstract:
Compared to a stateless data plane, a stateful data plane offloads part of state information and control logic from a controller to a data plane to reduce communication overhead and improve packet processing efficiency. However, existing methods for implementing stateful data planes face challenges, particularly maintaining state consistency during packet processing and improving throughput performance. This paper presents a high-performance, FPGA (Field Programmable Gate Array)-based stateful packet processing approach, which addresses these challenges utilizing the PHV (Packet Header Vector) dynamic scheduling technique to ensure flow state consistency. Our experiments demonstrate that the proposed method could operate at 200 MHz while adding 3–12 microseconds latency. The method we proposed also provides a considerable degree of programmability.
37
Yang, Yixiang. "Parallel processing in data analysis of the JUNO experiment." Journal of Physics: Conference Series 2438, no. 1 (February 1, 2023): 012057. http://dx.doi.org/10.1088/1742-6596/2438/1/012057.
Full textAbstract:
Abstract The JUNO experiment is being built mainly to determine the neutrino mass hierarchy by detecting neutrinos generated in the Yangjiang and Taishan nuclear plants in southern China. The detector will record 5.6 TB raw data every day for offline analysis, but each day it can only collect about 60 neutrino events scattered among huge background events. Selection of extremely sparse neutrino events brings a big challenge to offline data analysis. A typical neutrino physics event normally spans across a number of consecutive readout events, flagged by a fast positron signal followed by a slow neutron signal within a varying-size time window. To facilitate this analysis, a two-step data processing scheme has been proposed. In the first step (called data preparation), the event index data is produced and skimmed, which only contains information of minimum physics quantities of events as well as their addresses in the original reconstructed data file. In the second step (called time correlation analysis), event index data is further selected with stricter criteria. And then, for each selected event, the time correlation analysis is performed by reading all associated events within a pre-defined time window from the original data file according to the selected event’s address and timestamp. This contribution will start to introduce the design of the above data processing scheme and then focus on the multi-threaded implementation of time correlation analysis based on the Intel Threading Building Block (TBB) in the SNiPER framework. Afterwards, this contribution will describe the implementation of distributed analysis using MPI in which the time correlation analysis task is divided into sub-tasks running on multiple computing nodes. At last, this contribution will present the detailed performance measurements made on a multiple-node test bed. By using both skimming and indexing techniques, the total amount of data finally used for neutrino signal time correlation analysis is significantly reduced, and the processing time could be reduced by two orders of magnitude.
38
van Hijfte, Levi, Marjolein Geurts, Wies R. Vallentgoed, Paul H. Eilers, Peter A. Sillevis Smitt, Reno Debets, and Pim J. French. "Abstract 1228: Spatial transcriptomics: Data processing revisited to address noise interference." Cancer Research 82, no. 12_Supplement (June 15, 2022): 1228. http://dx.doi.org/10.1158/1538-7445.am2022-1228.
Full textAbstract:
Abstract Background: Spatially resolved transcriptomics is a novel and already highly recognized method that allows RNA sequencing results to be annotated with local tissue phenotypes. The NanoString GeoMx Digital Spatial Profiling (DSP) Platform allows users to collect RNA expression data from manually selected Regions of Interest (ROIs) on FFPE tissue sections. Here, we extensively evaluated data from the DSP platform with its associated pipeline and identify significant background noise interference issues which compromise data interpretation. Alternative and more suitable workflows are presented for correct data analysis. Methods: In this study, 12 paired tumor samples were collected from six glioma patients who underwent two separate resections. For all patients, the first resection was a low grade astrocytoma (WHO grade II or III) and the second resection was a high grade astrocytoma (WHO grade IV). The DSP platform was used to collect expression data of 1,800 genes from 72 ROIs (i.e. 6 per sample). Biological replicates were made of eight tumors from four patients. Gene expression data was normalized with both standard NanoString methods and several alternative methods (e.g. DeSeq2, gamma fit correction and quantile normalization). Weighted Gene Co-expression Network analysis (WGCNA) was used for biological validation. In addition to our own study, six publicly available NanoString DSP datasets were evaluated. Results: Data distributions of all glioma samples, when exposed to standard data processing, were burdened with significant background noise interference. Notably, differences in noise interference were largest between biologically distinct tumor subgroups (i.e. between first and second glioma resections), which was confirmed in replicate experiments. The noise interference patterns were also present in all six publicly available NanoString DSP datasets which will invariably lead to incorrect interpretation of the underlying biology. To correct for noise interference, we tested several normalization methods. The relatively crude quantile normalization method provided the least biased result and showed the highest concordance with bulk RNA sequencing data. To evaluate the biological validity of our alternative approach, we used T cell counts from our tissue regions as an independent parameter, that were quantified using immune fluorescence. Unsupervised WGCNA identified gene clusters enriched for lymphocyte genes that highly correlated with T cell quantities in ROIs, confirming that alternative normalization can extract a biological signal from the DSP platform. Conclusion: The DSP Platform platform suffers from significant noise interference when using standard analysis tools that obscure its results. Here, we revised the workflow and provide an alternative normalization that adequately addresses noise interference and enables correct interpretation of gene expression data. Citation Format: Levi van Hijfte, Marjolein Geurts, Wies R. Vallentgoed, Paul H. Eilers, Peter A. Sillevis Smitt, Reno Debets, Pim J. French. Spatial transcriptomics: Data processing revisited to address noise interference [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr 1228.
39
Xu, Xin, Jun Fu, Zhijie Sun, and Xuemei Li. "The address matching method of malfunction service worksheet of power customers based on standard address base." MATEC Web of Conferences 173 (2018): 01017. http://dx.doi.org/10.1051/matecconf/201817301017.
Full textAbstract:
Processing methods stay at the manual processing level, which is inefficient. By analyzing the address of 95598 worksheet of power customers, a standard address base structure and matching rules for storing standard data sets are established, and a matching method of 95598 incident worksheet of power customers based on standard address base is proposed. Drawing on the segment of standard address base, the method also defines the maximum length of the forward matching algorithm and can match along the self-defined address matching rules. It reduces the number of matches between the address to be matched and the standard data set, decreases the target data set used in the next participle, and improves the matching efficiency. In addition, by defining ambiguous addresses and expanding the rule tree, the matching success rate of the addresses to be matched and the flexibility of system implementation are improved
40
Lguensat, Redouane, Phi Huynh Viet, Miao Sun, Ge Chen, Tian Fenglin, Bertrand Chapron, and Ronan Fablet. "Data-Driven Interpolation of Sea Level Anomalies Using Analog Data Assimilation." Remote Sensing 11, no. 7 (April 9, 2019): 858. http://dx.doi.org/10.3390/rs11070858.
Full textAbstract:
From the recent developments of data-driven methods as a means to better exploit large-scale observation, simulation and reanalysis datasets for solving inverse problems, this study addresses the improvement of the reconstruction of higher-resolution Sea Level Anomaly (SLA) fields using analog strategies. This reconstruction is stated as an analog data assimilation issue, where the analog models rely on patch-based and Empirical Orthogonal Functions (EOF)-based representations to circumvent the curse of dimensionality. We implement an Observation System Simulation Experiment (OSSE) in the South China Sea. The reported results show the relevance of the proposed framework with a significant gain in terms of Root Mean Square Error (RMSE) for scales below 100 km. We further discuss the usefulness of the proposed analog model as a means to exploit high-resolution model simulations for the processing and analysis of current and future satellite-derived altimetric data with regard to conventional interpolation schemes, especially optimal interpolation.
41
Shoenbill, Kimberly, Yiqiang Song, Lisa Gress, Heather Johnson, Maureen Smith, and Eneida A. Mendonca. "Natural language processing of lifestyle modification documentation." Health Informatics Journal 26, no. 1 (February 22, 2019): 388–405. http://dx.doi.org/10.1177/1460458218824742.
Full textAbstract:
Lifestyle modification, including diet, exercise, and tobacco cessation, is the first-line treatment of many disorders including hypertension, obesity, and diabetes. Lifestyle modification data are not easily retrieved or used in research due to their textual nature. This study addresses this knowledge gap using natural language processing to automatically identify lifestyle modification documentation from electronic health records. Electronic health record notes from hypertension patients were analyzed using an open-source natural language processing tool to retrieve assessment and advice regarding lifestyle modification. These data were classified as lifestyle modification assessment or advice and mapped to a coded standard ontology. Combined lifestyle modification (advice and assessment) recall was 99.27 percent, precision 94.44 percent, and correct classification 88.15 percent. Through extraction and transformation of narrative lifestyle modification data to coded data, this critical information can be used in research, metric development, and quality improvement efforts regarding care delivery for multiple medical conditions that benefit from lifestyle modification.
42
Sukumar, Sreenivas R., Ramachandran Natarajan, and Regina K. Ferrell. "Quality of Big Data in health care." International Journal of Health Care Quality Assurance 28, no. 6 (July 13, 2015): 621–34. http://dx.doi.org/10.1108/ijhcqa-07-2014-0080.
Full textAbstract:
Purpose – The current trend in Big Data analytics and in particular health information technology is toward building sophisticated models, methods and tools for business, operational and clinical intelligence. However, the critical issue of data quality required for these models is not getting the attention it deserves. The purpose of this paper is to highlight the issues of data quality in the context of Big Data health care analytics. Design/methodology/approach – The insights presented in this paper are the results of analytics work that was done in different organizations on a variety of health data sets. The data sets include Medicare and Medicaid claims, provider enrollment data sets from both public and private sources, electronic health records from regional health centers accessed through partnerships with health care claims processing entities under health privacy protected guidelines. Findings – Assessment of data quality in health care has to consider: first, the entire lifecycle of health data; second, problems arising from errors and inaccuracies in the data itself; third, the source(s) and the pedigree of the data; and fourth, how the underlying purpose of data collection impact the analytic processing and knowledge expected to be derived. Automation in the form of data handling, storage, entry and processing technologies is to be viewed as a double-edged sword. At one level, automation can be a good solution, while at another level it can create a different set of data quality issues. Implementation of health care analytics with Big Data is enabled by a road map that addresses the organizational and technological aspects of data quality assurance. Practical implications – The value derived from the use of analytics should be the primary determinant of data quality. Based on this premise, health care enterprises embracing Big Data should have a road map for a systematic approach to data quality. Health care data quality problems can be so very specific that organizations might have to build their own custom software or data quality rule engines. Originality/value – Today, data quality issues are diagnosed and addressed in a piece-meal fashion. The authors recommend a data lifecycle approach and provide a road map, that is more appropriate with the dimensions of Big Data and fits different stages in the analytical workflow.
43
Demirezen, Mustafa Umut, and Tuğba Selcen Navruz. "Performance Analysis of Lambda Architecture-Based Big-Data Systems on Air/Ground Surveillance Application with ADS-B Data." Sensors 23, no. 17 (August 31, 2023): 7580. http://dx.doi.org/10.3390/s23177580.
Full textAbstract:
This study introduces a novel methodology designed to assess the accuracy of data processing in the Lambda Architecture (LA), an advanced big-data framework qualified for processing streaming (data in motion) and batch (data at rest) data. Distinct from prior studies that have focused on hardware performance and scalability evaluations, our research uniquely targets the intricate aspects of data-processing accuracy within the various layers of LA. The salient contribution of this study lies in its empirical approach. For the first time, we provide empirical evidence that validates previously theoretical assertions about LA, which have remained largely unexamined due to LA’s intricate design. Our methodology encompasses the evaluation of prospective technologies across all levels of LA, the examination of layer-specific design limitations, and the implementation of a uniform software development framework across multiple layers. Specifically, our methodology employs a unique set of metrics, including data latency and processing accuracy under various conditions, which serve as critical indicators of LA’s accurate data-processing performance. Our findings compellingly illustrate LA’s “eventual consistency”. Despite potential transient inconsistencies during real-time processing in the Speed Layer (SL), the system ultimately converges to deliver precise and reliable results, as informed by the comprehensive computations of the Batch Layer (BL). This empirical validation not only confirms but also quantifies the claims posited by previous theoretical discourse, with our results indicating a 100% accuracy rate under various severe data-ingestion scenarios. We applied this methodology in a practical case study involving air/ground surveillance, a domain where data accuracy is paramount. This application demonstrates the effectiveness of the methodology using real-world data-intake scenarios, therefore distinguishing this study from hardware-centric evaluations. This study not only contributes to the existing body of knowledge on LA but also addresses a significant literature gap. By offering a novel, empirically supported methodology for testing LA, a methodology with potential applicability to other big-data architectures, this study sets a precedent for future research in this area, advancing beyond previous work that lacked empirical validation.
44
Ibragimov, Yusup M., Andrey A. Potapov, and Ibragim R. Avtorkhanov. "DATA MANAGEMENT AND ANALYTICS: THE USE OF INFORMATION IN THE DEVELOPMENT OF STRATEGIC DECISIONS." EKONOMIKA I UPRAVLENIE: PROBLEMY, RESHENIYA 12/1, no. 141 (2023): 20–26. http://dx.doi.org/10.36871/ek.up.p.r.2023.12.01.003.
Full textAbstract:
This scientific article addresses the pressing issue of data management and analytics in contemporary business. It outlines the significance of data collection, storage, processing, and analysis in the context of strategic decision-making for companies and organizations. The article presents methods and tools for data management and provides examples of successful data utilization in strategy development and gaining competitive advantages. This work emphasizes that data management and analytics are integral components of modern business and pivotal elements in successful strategic planning.
45
Schwartze, Christian, Franciska Zander, Sven Kralisch, and Wolfgang-Albert Flügel. "Virtual Appliances for geospatial data management and processing in the Integrated Land Management System (ILMS)." Acta Agraria Debreceniensis, no. 49 (November 13, 2012): 59–62. http://dx.doi.org/10.34101/actaagrar/49/2480.
Full textAbstract:
Virtualization is increasingly taking on a key role in various system architectures which follow new platform concepts like Software as a Service (SaaS). This trend addresses more instant and short-term environments and comes with new methods and strategies for the distribution of mainly complex application stacks not only in large IT infrastructures. The paper presents how a so called Virtual Appliance can be set up in order to operate in virtual server environments using hypervisor software like Oracle Virtual-Box. Using the example of two server-side components within the Integrated Land Management System (ILMS), it will be shown that the use of state-of-the-art methods, standardized tools and interfaces on servers enables different aspects of environmental system management, analysis and planning.
46
Habbouche, Houssem, Tarak Benkedjouh, Yassine Amirat, and Mohamed Benbouzid. "Gearbox Failure Diagnosis Using a Multisensor Data-Fusion Machine-Learning-Based Approach." Entropy 23, no. 6 (May 31, 2021): 697. http://dx.doi.org/10.3390/e23060697.
Full textAbstract:
Failure detection and diagnosis are of crucial importance for the reliable and safe operation of industrial equipment and systems, while gearbox failures are one of the main factors leading to long-term downtime. Condition-based maintenance addresses this issue using several expert systems for early failure diagnosis to avoid unplanned shutdowns. In this context, this paper provides a comparative study of two machine-learning-based approaches for gearbox failure diagnosis. The first uses linear predictive coefficients for signal processing and long short-term memory for learning, while the second is based on mel-frequency cepstral coefficients for signal processing, a convolutional neural network for feature extraction, and long short-term memory for classification. This comparative study proposes an improved predictive method using the early fusion technique of multisource sensing data. Using an experimental dataset, the proposals were tested, and their effectiveness was evaluated considering predictions based on statistical metrics.
47
Pontes, João Guilherme M., Antonio Jadson M. Brasil, Guilherme C. F. Cruz, Rafael N. de Souza, and Ljubica Tasic. "NMR-based metabolomics strategies: plants, animals and humans." Analytical Methods 9, no. 7 (2017): 1078–96. http://dx.doi.org/10.1039/c6ay03102a.
Full textAbstract:
This Tutorial Review addresses the principal steps from the sample preparation, acquisition and processing of spectra, data analysis and biomarker discovery and methodologies used in NMR-based metabolomics applied for pointing to key metabolites of diseases.
48
Babu, Shaikh, Ahankare Anand, Vatse Aditya, Waghmare Ajinkya, and Prof M. P. Shinde. "VISUAL CRYPTOGRAPHY: STRENGTHENING BANKING AUTHENTICATION WITH IMAGE PROCESSING." INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT 07, no. 10 (October 1, 2023): 1–11. http://dx.doi.org/10.55041/ijsrem26471.
Full textAbstract:
In today's digital era, we present a multi-factor authentication system that combines Visual Cryptography, Face Authentication, and OTP Verification to fortify banking security. Visual Cryptography splits images into secure shares, Face Authentication verifies unique facial features, and OTP Verification adds an extra layer. The synergy of these factors forms a robust, secure, and user- friendly system, reducing unauthorized access and fraud. This project contributes to cyber-security advancements and improves the banking user experience. In an ever-growing digital banking landscape, this innovative approach ensures data confidentiality and addresses evolving threats. Keywords: Visual Cryptography, Image Processing, Face Recognition, Encryption, Multi-factor Authentication
49
Nikolyukin, M. S., and A. D. Obukhov. "Adaptive Processing of Camera Video Stream with Limitations on the Network Data Transmission Bandwidth." Informacionnye Tehnologii 30, no. 5 (May 14, 2024): 252–60. http://dx.doi.org/10.17587/it.30.252-260.
Full textAbstract:
Video surveillance systems, cameras, and video stream processing are actively used in many enterprises as a monitoring and control tool for regular and emergency situations, as well as staff activities. The application of intelligent algorithms allows tracking and minimizing operator errors, but these approaches are highly sensitive to the quality of the original video, presence of noise, and low resolution. On the other hand, such video surveillance systems may be limited by network bandwidth. Therefore, this work considers an adaptive video stream processing algorithm that ensures efficient operation of computer vision and object recognition methods while minimizing the amount of transmitted information within network bandwidth constraints. The proposed algorithm addresses the task of determining boundary conditions that ensure the functionality of object recognition algorithms with the least amount of video stream. Corresponding experimental studies were conducted to determine the minimum values of frame resolution and video bitrate. The algorithm was tested in organizing video surveillance at warehouse complexes. The obtained results can be used in developing decision support systems for enterprises in various industries requiring intelligent processing of large volumes of data.
50
Simpson, James. "Three-Dimensional Gaze Projection Heat-Mapping of Outdoor Mobile Eye-Tracking Data." Interdisciplinary Journal of Signage and Wayfinding 5, no. 1 (March 31, 2021): 62–82. http://dx.doi.org/10.15763/issn.2470-9670.2021.v5.i1.a75.
Full textAbstract:
The mobilization of eye-tracking for use outside of the laboratory provides new opportunities for the assessment of pedestrian visual engagement with their surroundings. However, the development of data representation techniques that visualize the dynamics of pedestrian gaze distribution upon the environment they are situated within remains limited. The current study addresses this through highlighting how mobile eye-tracking data, which captures where pedestrian gaze is focused upon buildings along urban street edges, can be mapped as three-dimensional gaze projection heat-maps. This data processing and visualization technique is assessed during the current study along with future opportunities and associated challenges discussed.