Journal articles on the topic 'Multi objective RL'

Recent work on hybrid quantum-classical machine learning systems has demonstrated success in utilizing parameterized quantum circuits (PQCs) to solve the challenging reinforcement learning (RL) tasks, with provable learning advantages over classical systems, e.g., deep neural networks. While existing work demonstrates and exploits the strength of PQC-based models, the design choices of PQC architectures and the interactions between different quantum circuits on learning tasks are generally underexplored. In this work, we introduce a Multi-objective Evolutionary Architecture Search framework for parameterized quantum circuits (MEAS-PQC), which uses a multi-objective genetic algorithm with quantum-specific configurations to perform efficient searching of optimal PQC architectures. Experimental results show that our method can find architectures that have superior learning performance on three benchmark RL tasks, and are also optimized for additional objectives including reductions in quantum noise and model size. Further analysis of patterns and probability distributions of quantum operations helps identify performance-critical design choices of hybrid quantum-classical learning systems.

Multi-objective Markov decision processes (MOMDPs) provide an effective modeling framework for decision-making problems involving water systems. The traditional approach is to define many single-objective problems (resulting from different combinations of the objectives), each solvable by standard optimization. This paper presents an approach based on reinforcement learning (RL) that can learn the operating policies for all combinations of objectives in a single training process. The key idea is to enlarge the approximation of the action-value function, which is performed by single-objective RL over the state-action space, to the space of the objectives' weights. The batch-mode nature of the algorithm allows for enriching the training dataset without further interaction with the controlled system. The approach is demonstrated on a numerical test case study and evaluated on a real-world application, the Hoa Binh reservoir, Vietnam. Experimental results on the test case show that the proposed approach (multi-objective fitted Q-iteration; MOFQI) becomes computationally preferable over the repeated application of its single-objective version (fitted Q-iteration; FQI) when evaluating more than five weight combinations. In the Hoa Binh case study, the operating policies computed with MOFQI and FQI have comparable efficiency, while MOFQI provides a continuous approximation of the Pareto frontier with no additional computing costs.

Video streaming has become a major usage scenario for the Internet. The growing popularity of new applications, such as 4K and 360-degree videos, mandates that network resources must be carefully apportioned among different users in order to achieve the optimal Quality of Experience (QoE) and fairness objectives. This results in a challenging online optimization problem, as networks grow increasingly complex and the relevant QoE objectives are often nonlinear functions. Recently, data-driven approaches, deep Reinforcement Learning (RL) in particular, have been successfully applied to network optimization problems by modeling them as Markov decision processes. However, existing RL algorithms involving multiple agents fail to address nonlinear objective functions on different agents’ rewards. To this end, we leverage MAPG-finite, a policy gradient algorithm designed for multi-agent learning problems with nonlinear objectives. It allows us to optimize bandwidth distributions among multiple agents and to maximize QoE and fairness objectives on video streaming rewards. Implementing the proposed algorithm, we compare the MAPG-finite strategy with a number of baselines, including static, adaptive, and single-agent learning policies. The numerical results show that MAPG-finite significantly outperforms the baseline strategies with respect to different objective functions and in various settings, including both constant and adaptive bitrate videos. Specifically, our MAPG-finite algorithm maximizes QoE by 15.27% and maximizes fairness by 22.47% compared to the standard SARSA algorithm for a 2000 KB/s link.

Abstract Background: Portal vein embolization (PVE) is a common procedure to induce hypertrophy of the remnant liver (RL) before major hepatectomy. Objective: Evaluate increased RL volume after PVE based on CT volumetric measurement. Methods: Multi-detector computed tomography (MDCT) was used to measure hepatic volumetric measurement, including total liver volume and RL volumes of pre- and post-PVE. Complications were recorded from PVE and from three-month after post-extended hepatectomy liver dysfunction. Result and conclusion: There was a 10% increase in RL volume. Mean days between CT and PVE were 20 days. No major complications from PVE were observed.

Reinforcement learning (RL) is a state or action value based machine learning method which solves large-scale multi-stage decision problems such as Markov Decision Process (MDP) and Semi-Markov Decision Process (SMDP) problems. We minimize the makespan of flow shop scheduling problems with an RL algorithm. We convert flow shop scheduling problems into SMDPs by constructing elaborate state features, actions and the reward function. Minimizing the accumulated reward is equivalent to minimizing the schedule objective function. We apply on-line TD(λ) algorithm with linear gradient-descent function approximation to solve the SMDPs. To examine the performance of the proposed RL algorithm, computational experiments are conducted on benchmarking problems in comparison with other scheduling methods. The experimental results support the efficiency of the proposed algorithm and illustrate that the RL approach is a promising computational approach for flow shop scheduling problems worthy of further investigation.

Usually, real-world problems involve the optimization of multiple, possibly conflicting, objectives. These problems may be addressed by Multi-objective Reinforcement learning (MORL) techniques. MORL is a generalization of standard Reinforcement Learning (RL) where the single reward signal is extended to multiple signals, in particular, one for each objective. MORL is the process of learning policies that optimize multiple objectives simultaneously. In these problems, the use of directional/gradient information can be useful to guide the exploration to better and better behaviors. However, traditional policy-gradient approaches have two main drawbacks: they require the use of a batch of episodes to properly estimate the gradient information (reducing in this way the learning speed), and they use stochastic policies which could have a disastrous impact on the safety of the learning system. In this paper, we present a novel population-based MORL algorithm for problems in which the underlying objectives are reasonably smooth. It presents two main characteristics: fast computation of the gradient information for each objective through the use of neighboring solutions, and the use of this information to carry out a geometric partition of the search space and thus direct the exploration to promising areas. Finally, the algorithm is evaluated and compared to policy gradient MORL algorithms on different multi-objective problems: the water reservoir and the biped walking problem (the latter both on simulation and on a real robot).

Purpose – Selection of best product recovery alternative in reverse logistics (RL) has gained great attention in supply chain community. The purpose of this paper is to provide a robust group decision-making tool to select the best product recovery alternative. Design/methodology/approach – In this paper, fuzzy values, assigned to various criteria and alternatives by a number of decision makers, are converted into crisp values and then aggregated scores are evaluated. After obtaining experts’ scores, objective and subjective weights of the criteria have been calculated using variance method and analytic hierarchy process, respectively. Then integrated weights of criteria are evaluated using different proportions of the two weights. The superiority and inferiority ranking (SIR) method is then employed to achieve the final ranking of alternatives. An example is presented to demonstrate the methodology. Findings – The proposed methodology provides decision makers a systematic, flexible and realistic approach to effectively rank the product recovery alternatives in RL. The alternatives can easily be benchmarked and best recovery strategy can be obtained. The sensitivity analysis carried out by changing different proportion of objective and subjective weights reveals that best ranking alternative never changes and proves the robustness of the methodology. The present benchmarking framework can also be used by decision makers to simplify any problem which encounters multi-attribute decision making and multiple decision makers. Research limitations/implications – The proposed methodology should be tested in different situations having varied operational and environmental conditions dealing with different products. A real case study from an industrial set up can help to assess the behavior of the proposed methodology. The presented methodology however can deal with such multi-disciplinary and multi-criteria issues in a simple and structured manner and ease the managers to select the best alternative. Originality/value – A novel approach for decision making taking into account both objective and subjective weights for criteria has been proposed to rank the best recovery alternatives in RL. The proposed methodology uses SIR method to prioritize the alternatives. As RL alternative selection is an important issue and involves both technical and managerial criteria as well as multiple decision makers, the proposed robust methodology can provide guidelines for the practicing managers.

This paper solves the job-shop scheduling problem (JSP) considering job transport, with the aim to minimize the maximum makespan, tardiness, and energy consumption. In the first stage, the improved fast elitist nondominated sorting genetic algorithm II (INSGA-II) was combined with N5 neighborhood structure and the local search strategy of nondominant relationship to generate new neighborhood solutions by exchanging the operations on the key paths. In the second stage, the ant colony algorithm based on reinforcement learning (RL-ACA) was designed to optimize the job transport task, abstract the task into polar coordinates, and further optimizes the task. The proposed two-stage algorithm was tested on small, medium, and large-scale examples. The results show that our algorithm is superior to other algorithms in solving similar problems.

In this paper, we introduce a self-trained controller for autonomous navigation in static and dynamic (with moving walls and nets) challenging environments (including trees, nets, windows, and pipe) using deep reinforcement learning, simultaneously trained using multiple rewards. We train our RL algorithm in a multi-objective way. Our algorithm learns to generate continuous action for controlling the UAV. Our algorithm aims to generate waypoints for the UAV in such a way as to reach a goal area (shown by an RGB image) while avoiding static and dynamic obstacles. In this text, we use the RGB-D image as the input for the algorithm, and it learns to control the UAV in 3-DoF (x, y, and z). We train our robot in environments simulated by Gazebo sim. For communication between our algorithm and the simulated environments, we use the robot operating system. Finally, we visualize the trajectories generated by our trained algorithms using several methods and illustrate our results that clearly show our algorithm’s capability in learning to maximize the defined multi-objective reward.

Battery storage is emerging as a key component of intelligent green electricitiy systems. The battery is monetized through market participation, which usually involves bidding. Bidding is a multi-objective optimization problem, involving targets such as maximizing market compensation and minimizing penalties for failing to provide the service and costs for battery aging. In this article, battery participation is investigated on primary frequency reserve markets. Reinforcement learning is applied for the optimization. In previous research, only simplified formulations of battery aging have been used in the reinforcement learning formulation, so it is unclear how the optimizer would perform with a real battery. In this article, a physics-based battery aging model is used to assess the aging. The contribution of this article is a methodology involving a realistic battery simulation to assess the performance of the trained RL agent with respect to battery aging in order to inform the selection of the weighting of the aging term in the RL reward formula. The RL agent performs day-ahead bidding on the Finnish Frequency Containment Reserves for Normal Operation market, with the objective of maximizing market compensation, minimizing market penalties and minimizing aging costs.

The manipulation of complex robotics, which is in general high-dimensional continuous control without an accurate dynamic model, summons studies and applications of reinforcement learning (RL) algorithms. Typically, RL learns with the objective of maximizing the accumulated rewards from interactions with the environment. In reality, external rewards are not trivial, which depend on either expert knowledge or domain priors. Recent advances on hindsight experience replay (HER) instead enable a robot to learn from the automatically generated sparse and binary rewards, indicating whether it reaches the desired goals or pseudo goals. However, HER inevitably introduces hindsight bias that skews the optimal control since the replays against the achieved pseudo goals may often differ from the exploration of the desired goals. To tackle the problem, we analyze the skewed objective and induce the decayed hindsight (DH), which enables consistent multi-goal experience replay via countering the bias between exploration and hindsight replay. We implement DH for goal-conditioned RL both in online and offline settings. Experiments on online robotic control tasks demonstrate that DH achieves the best average performance and is competitive with state-of-the-art replay strategies. Experiments on offline robotic control tasks show that DH substantially improves the ability to extract near-optimal policies from offline datasets.

Maintaining a fair use of energy consumption in smart homes with many household appliances requires sophisticated algorithms working together in real time. Similarly, choosing a proper schedule for appliances operation can be used to reduce inappropriate energy consumption. However, scheduling appliances always depend on the behavior of a smart home user. Thus, modeling human interaction with appliances is needed to design an efficient scheduling algorithm with real-time support. In this regard, we propose a scheduling algorithm based on human appliances interaction in smart homes using reinforcement learning (RL). The proposed scheduling algorithm divides the entire day into various states. In each state, the agents attached to household appliances perform various actions to obtain the highest reward. To adjust the discomfort which arises due to performing inappropriate action, the household appliances are categorized into three groups i.e., (1) adoptable, (2) un-adoptable, (3) manageable. Finally, the proposed system is tested for the energy consumption and discomfort level of the home user against our previous scheduling algorithm based on least slack time phenomenon. The proposed scheme outperforms the Least Slack Time (LST) based scheduling in context of energy consumption and discomfort level of the home user.

Reinforcement learning (RL) has received widespread attention across multiple communities, but the experiments have focused primarily on large-scale game playing and robotics tasks. In this paper we introduce ORSuite, an open-source library containing environments, algorithms, and instrumentation for operational problems. Our package is designed to motivate researchers in the reinforcement learning community to develop and evaluate algorithms on operational tasks, and to consider the true multi-objective nature of these problems by considering metrics beyond cumulative reward.

Sequential Decision Making Problems (SDMPs) that can be modeled as Markov Decision Processes can be solved using methods that combine Dynamic Programming (DP) and Reinforcement Learning (RL). Depending on the problem scenarios and the available Decision Makers (DMs), such RL algorithms may be designed for single-agent systems or multi-agent systems that either consist of agents with individual goals and decision making capabilities, which are influenced by other agent’s decisions, or behave as a swarm of agents that collaboratively learn a single objective. Many studies have been conducted in this area; however, when concentrating on available swarm RL algorithms, one obtains a clear view of the areas that still require attention. Most of the studies in this area focus on homogeneous swarms and so far, systems introduced as Heterogeneous Swarms (HetSs) merely include very few, i.e., two or three sub-swarms of homogeneous agents, which either, according to their capabilities, deal with a specific sub-problem of the general problem or exhibit different behaviors in order to reduce the risk of bias. This study introduces a novel approach that allows agents, which are originally designed to solve different problems and hence have higher degrees of heterogeneity, to behave as a swarm when addressing identical sub-problems. In fact, the affinity between two agents, which measures the compatibility of agents to work together towards solving a specific sub-problem, is used in designing a Heterogeneous Swarm RL (HetSRL) algorithm that allows HetSs to solve the intended SDMPs.

In this paper, we study reinforcement learning (RL) algorithms to solve real-world decision problems with the objective of maximizing the long-term reward as well as satisfying cumulative constraints. We propose a novel first-order policy optimization method, Interior-point Policy Optimization (IPO), which augments the objective with logarithmic barrier functions, inspired by the interior-point method. Our proposed method is easy to implement with performance guarantees and can handle general types of cumulative multi-constraint settings. We conduct extensive evaluations to compare our approach with state-of-the-art baselines. Our algorithm outperforms the baseline algorithms, in terms of reward maximization and constraint satisfaction.

With the development of intelligent manufacturing, machine tools are considered the “mothership” of the equipment manufacturing industry, and the associated processing workshops are becoming more high-end, flexible, intelligent, and green. As the core of manufacturing management in a smart shop floor, research into the multi-objective dynamic flexible job shop scheduling problem (MODFJSP) focuses on optimizing scheduling decisions in real time according to changes in the production environment. In this paper, hierarchical reinforcement learning (HRL) is proposed to solve the MODFJSP considering random job arrival, with a focus on achieving the two practical goals of minimizing penalties for earliness and tardiness and reducing total machine load. A two-layer hierarchical architecture is proposed, namely the combination of a double deep Q-network (DDQN) and a dueling DDQN (DDDQN), and state features, actions, and external and internal rewards are designed. Meanwhile, a personal computer-based interaction feature is designed to integrate subjective decision information into the real-time optimization of HRL to obtain a satisfactory compromise. In addition, the proposed HRL framework is applied to multi-objective real-time flexible scheduling in a smart gear production workshop, and the experimental results show that the proposed HRL algorithm outperforms other reinforcement learning (RL) algorithms, metaheuristics, and heuristics in terms of solution quality and generalization and has the added benefit of real-time characteristics.

In this article we present two novel multipurpose reservoir optimization algorithms named nested stochastic dynamic programming (nSDP) and nested reinforcement learning (nRL). Both algorithms are built as a combination of two algorithms; in the nSDP case it is (1) stochastic dynamic programming (SDP) and (2) nested optimal allocation algorithm (nOAA) and in the nRL case it is (1) reinforcement learning (RL) and (2) nOAA. The nOAA is implemented with linear and non-linear optimization. The main novel idea is to include a nOAA at each SDP and RL state transition, that decreases starting problem dimension and alleviates curse of dimensionality. Both nSDP and nRL can solve multi-objective optimization problems without significant computational expenses and algorithm complexity and can handle dense and irregular variable discretization. The two algorithms were coded in Java as a prototype application and on the Knezevo reservoir, located in the Republic of Macedonia. The nSDP and nRL optimal reservoir policies were compared with nested dynamic programming policies, and overall conclusion is that nRL is more powerful, but significantly more complex than nSDP.

Nowadays, traditional power systems are being developed as an emergence for the use of smart grids that cover the integration of multi-renewable energy sources with power electronics converters. Efforts were made to design power quality controllers for multi-renewable energy systems (photovoltaic (PV), Fuel Cell and Battery) to meet huge energy demands. Though there have been several techniques employed so far, the power quality issue is a major concern. In this paper, a multi-objective optimal energy management system for electric vehicles (EVs) is proposed using a reinforcement learning mechanism. Furthermore, the maximum power point tracking (MPPT)-based Reinforcement Learning-Iterative cuckoo search optimization algorithm (RL-ICSO) along with the Proportional Integral Derivative (PID) controller is incorporated. For this, a renewable energy source is considered as input for eliminating voltage and current harmonics. Similarly, a DC to AC inverter using a Model Predictive Control (MPC) controller-based pulse generation process was carried out to incorporate the power quality compensation of multi-renewable energy microgrid harmonics in three-phase systems. The generated energy is checked for any liabilities by adding a fault in the transmission line and thereby rectifying the fault by means of the Unified Power Quality Controller (UPQC) device. Thus, the fault-rectified power is stored in the grid, and the transmitting power can be used for EV charging purposes. Thus, the energy storage system is effective in charging and storing the needed power for EVs. The performance estimation is carried out by estimating the simulation outcome on Total Harmonic Distortion (THD) values, parameters, load current and voltage. In addition, the performance estimation is employed, and the outcomes attained are represented. The analysis depicts the effectiveness of the power and energy management ability of the proposed approach.

Abstract At present, manufacturing models are characterized by multi-variety, small batch, and diversification. It is insufficient to use traditional scheduling methods for production management with high performance. A real-time production scheduling system based on reinforcement learning (RL) is suggested in an effort to address the aforementioned issues. A brand-new manufacturing neural network is created to learn the state-action values for production scheduling in real time using high-dimensional data as the input. The detailed setup of network inputs, neural network, action, and reward are also designed. Then, a policy-based reinforcement learning algorithm is proposed to achieve the optimum objective. Finally, By contrasting the proposed scheduling strategy with rule-based approaches in a smart manufacturing environment, its efficacy is demonstrated. according to experimental data, the suggested algorithm can successfully improve performance in the dynamic job-shop environment.

Background: Multi-drug resistant infections are a growing worldwide health concern. There is an urgent need to produce alternative antimicrobial agents. Objective : The study aimed to design a new hybrid antimicrobial peptide, and to evaluate its antimicrobial activity alone and in combination with traditional antibiotics. Methods: Herein, we designed a novel hybrid peptide (BMR-1) using the primary sequences of the parent peptides Frog Esculentin-1a and Monkey Rhesus cathelicidin (RL-37). The positive net charge was increased, and other physicochemical parameters were optimized. The antimicrobial activities of BMR-1 were tested against control and multi-drug resistant gram-negative bacteria. Results: BMR-1 adopted a bactericidal behavior with MIC values of 25-30 µM. These values reduced by over 75% upon combination with conventional antibiotics (levofloxacin, chloramphenicol, ampicillin, and rifampicin). The combination showed strong synergistic activities in most cases and particularly against multi-drug resistance P. aeruginosa and E. coli. BMR-1 showed similar potency against all tested strains regardless of their resistant mechanisms. BMR-1 exhibited no hemolytic effect on human red blood cells with the effective MIC values against the tested strains. Conclusion: BMR-1 hybrid peptide is a promising candidate to treat resistant infectious diseases caused by gramnegative bacteria.

Although unsupervised representation learning (RL) can tackle the performance deterioration caused by limited labeled data in synthetic aperture radar (SAR) object classification, the neglected discriminative detailed information and the ignored distinctive characteristics of SAR images can lead to performance degradation. In this paper, an unsupervised multi-scale convolution auto-encoder (MSCAE) was proposed which can simultaneously obtain the global features and local characteristics of targets with its U-shaped architecture and pyramid pooling modules (PPMs). The compact depth-wise separable convolution and the deconvolution counterpart were devised to decrease the trainable parameters. The PPM and the multi-scale feature learning scheme were designed to learn multi-scale features. Prior knowledge of SAR speckle was also embedded in the model. The reconstruction loss of the MSCAE was measured by the structural similarity index metric (SSIM) of the reconstructed data and the images filtered by the improved Lee sigma filter. A speckle suppression restriction was also added in the objective function to guarantee that the speckle suppression procedure would take place in the feature learning stage. Experimental results with the MSTAR dataset under the standard operating condition and several extended operating conditions demonstrated the effectiveness of the proposed model in SAR object classification tasks.

In this paper, a robust image encryption technique that utilizes Fourier–Mellin moments and intertwining logistic map is proposed. Fourier–Mellin moment-based intertwining logistic map has been designed to overcome the issue of low sensitivity of an input image. Multi-objective Non-Dominated Sorting Genetic Algorithm (NSGA-II) based on Reinforcement Learning (MNSGA-RL) has been used to optimize the required parameters of intertwining logistic map. Fourier–Mellin moments are used to make the secret keys more secure. Thereafter, permutation and diffusion operations are carried out on input image using secret keys. The performance of proposed image encryption technique has been evaluated on five well-known benchmark images and also compared with seven well-known existing encryption techniques. The experimental results reveal that the proposed technique outperforms others in terms of entropy, correlation analysis, a unified average changing intensity and the number of changing pixel rate. The simulation results reveal that the proposed technique provides high level of security and robustness against various types of attacks.

Cloud computing leads to efficient resource allocation for network users. In order to achieve efficient allocation, many research activities have been conducted so far. Some researchers focus on classical optimisation theory techniques (such as multi-objective optimisation, evolutionary optimisation, game theory, etc.) to satisfy network providers and network users’ service-level agreement (SLA) requirements. Normally, in a cloud data centre network (CDCN), it is difficult to jointly satisfy both the cloud provider and cloud customer’ utilities, and this leads to complex combinatorial problems, which are usually NP-hard. Recently, machine learning and artificial intelligence techniques have received much attention from the networking community because of their capability to solve complicated networking problems. In the current work, at first, the holistic utility satisfaction for the cloud data centre provider and customers is formulated as a reinforcement learning (RL) problem with a specific reward function, which is a convex summation of users’ utility functions and cloud provider’s utility. The user utility functions are modelled as a function of cloud virtualised resources (such as storage, CPU, RAM), connection bandwidth, and also, the network-based expected packet loss and round-trip time factors associated with the cloud users. The cloud provider utility function is modelled as a function of resource prices and energy dissipation costs. Afterwards, a Q-learning implementation of the mentioned RL algorithm is introduced, which is able to converge to the optimal solution in an online and fast manner. The simulation results exhibit the enhanced convergence speed and computational complexity properties of the proposed method in comparison with similar approaches from the joint cloud customer/provider utility satisfaction perspective. To evaluate the scalability property of the proposed method, the results are also repeated for different cloud user population scenarios (small, medium, and large).

Coverage is a fundamental issue in wireless sensor networks (WSNs). It plays a important role in network efficiency and performance. When sensor nodes are randomly scattered in the network environment, an ON/OFF scheduling mechanism can be designed for these nodes to ensure network coverage and increase the network lifetime. In this paper, we propose an appropriate and optimal area coverage method. The proposed area coverage scheme includes four phases: (1) Calculating the overlap between the sensing ranges of sensor nodes in the network. In this phase, we present a novel, distributed, and efficient method based on the digital matrix so that each sensor node can estimate the overlap between its sensing range and other neighboring nodes. (2) Designing a fuzzy scheduling mechanism. In this phase, an ON/OFF scheduling mechanism is designed using fuzzy logic. In this fuzzy system, if a sensor node has a high energy level, a low distance to the base station, and a low overlap between its sensing range and other neighboring nodes, then this node will be in the ON state for more time. (3) Predicting the node replacement time. In this phase, we seek to provide a suitable method to estimate the death time of sensor nodes and prevent possible holes in the network, and thus the data transmission process is not disturbed. (4) Reconstructing and covering the holes created in the network. In this phase, the goal is to find the best replacement strategy of mobile nodes to maximize the coverage rate and minimize the number of mobile sensor nodes used for covering the hole. For this purpose, we apply the shuffled frog-leaping algorithm (SFLA) and propose an appropriate multi-objective fitness function. To evaluate the performance of the proposed scheme, we simulate it using NS2 simulator and compare our scheme with three methods, including CCM-RL, CCA, and PCLA. The simulation results show that our proposed scheme outperformed the other methods in terms of the average number of active sensor nodes, coverage rate, energy consumption, and network lifetime.

Objective: The study evaluates quantitative ultrasound (QUS) texture features with machine learning (ML) to enhance the sensitivity of B-mode ultrasound (US) for the detection of fibrosis at an early stage and distinguish it from advanced fibrosis. Different ML methods were evaluated to determine the best diagnostic model. Methods: 233 B-mode images of liver lobes with early and advanced-stage fibrosis induced in a rat model were analyzed. Sixteen features describing liver texture were measured from regions of interest (ROIs) drawn on B-mode images. The texture features included a first-order statistics run length (RL) and gray-level co-occurrence matrix (GLCM). The features discriminating between early and advanced fibrosis were used to build diagnostic models with logistic regression (LR), naïve Bayes (nB), and multi-class perceptron (MLP). The diagnostic performances of the models were compared by ROC analysis using different train-test sampling approaches, including leave-one-out, 10-fold cross-validation, and varying percentage splits. METAVIR scoring was used for histological fibrosis staging of the liver. Results: 15 features showed a significant difference between the advanced and early liver fibrosis groups, p < 0.05. Among the individual features, first-order statics features led to the best classification with a sensitivity of 82.1–90.5% and a specificity of 87.1–89.8%. For the features combined, the diagnostic performances of nB and MLP were high, with the area under the ROC curve (AUC) approaching 0.95–0.96. LR also yielded high diagnostic performance (AUC = 0.91–0.92) but was lower than nB and MLP. The diagnostic variability between test-train trials, measured by the coefficient-of-variation (CV), was higher for LR (3–5%) than nB and MLP (1–2%). Conclusion: Quantitative ultrasound with machine learning differentiated early and advanced fibrosis. Ultrasound B-mode images contain a high level of information to enable accurate diagnosis with relatively straightforward machine learning methods like naïve Bayes and logistic regression. Implementing simple ML approaches with QUS features in clinical settings could reduce the user-dependent limitation of ultrasound in detecting early-stage liver fibrosis.

Currently, despite widespread use of the terms systemic inflammation (SI) and systemic inflammatory response (SIR), there are no generally accepted criteria for their verification. These processes are often identified (which is methodologically incorrect) and associated with an increase in pro-inflammatory mediators in the blood. However, SI is a complex process that requires integral criteria including assessment of SIR as reactivity level, and additional SI phenomena, such as microthrombosis, systemic alteration, and distress of the neuroendocrine system. At the same time, there is a need to assess individual CB indicators as a more affordable alternative for medical practice than the use of complex integral indicators. Our objective was to evaluate diagnostic efficacy of CRP and IL-6 levels as markers of acute and chronic systemic inflammation. The data of patients with acute critical conditions of infectious and non-infectious genesis were analyzed to study acute systemic inflammation (SI), data of patients with autoimmune diseases, chronic organ failure and other chronic destructive diseases were analyzed to study chronic systemic inflammation (ChrSI). SIR severity was evaluated by the calculation of an integral index reactivity level (RL). Differentiation of the inflammatory process to either classical inflammation (CI), or systemic inflammation was carried out using the previously proposed scale of SI, verification of chronic systemic inflammation was performed by means of ChrSI scale. SI (or ChrSI) was revealed in all groups of patients, and the frequency of SI registration in patients with acute conditions increased with development of multi-organ failure. The frequency of SIR was higher in all groups, thus confirming inability to equate these disorders. ROC analysis showed that CRP level had poor diagnostic efficacy on the development of SI/ChrSI (AUC 0.6), and IL-6 level had very good diagnostic value (AUC 0.8-0.9). The prognostic value of the markers for detecting the SIR was higher, with AUCIL-6 exceeding AUCCRP. Thus, IL-6 in many acute and chronic pathologies is sufficiently closer to integral indices than C-reactive protein with respect to diagnostic efficiency, and the dynamics of IL-6 in blood may be used to predict and evaluate complications associated with acute and chronic SI, as well as to prescribe and monitor the results of anticytokine therapy.

Objective: The present study aims to expose through a literature review the cleft lip and/or cleft palate (CL/CP) and its treatment in a multidisciplinary approach. Methodology: This literature review was conducted by the leading health databases: Pubmed (https://www.ncbi.nlm.nih.gov/pubmed). The keywords for the textual search were: Cleft Lip; Cleft Palate; Dental Staff; Classification; Embryology. The inclusion criteria were: literature on the subject under study, literature of the last years, english language, laboratory and clinical studies and systematic review. Literature Review: Fissures can be defined by a space at the junction between two bones, usually where there would be a suture. Orofacial clefts are part of the congenital facial anomalies resulted from the non-junction of the embryonic facial processes. These changes occur due to an alteration in the migratory velocity of the neural crest cells, in charge of the phenomenon of fusion of the facial prominences between the 6th and 9thweek of embryonic life. Conclusion: The treatment of patients with orofacial clefts requires the approach of a multidisciplinary team that involves physicians in the area of plastic surgery, otorhinolaryngology, pediatrics, geneticists, dentists, prosthetics, nurses and speech pathologists, focusing on patient prevention, recovery and rehabilitation. However, further studies are needed for a better understanding of the subject and the steps that should be applied for each particular case.Descriptors: Cleft Lip; Cleft Palate; Dental Staff; Classification; Embryology.ReferencesShaw WC, Brattström V, Mølsted K, Prahl-Andersen B, Roberts CT, Semb G. The Eurocleft study: intercenter study of treatment outcome in patients with complete cleft lip and palate. Part 5: discussion and conclusions. Cleft Palate Craniofac J. 2005;42(1):93-8. Friede H, Lilja J. The Eurocleft Study: Intercenter study of treatment outcome in patients with complete cleft lip and palate. Cleft Palate Craniofac J. 2005;42(4):453-54.Rosenstein SW, Grasseschi M, Dado D. The Eurocleft Study: Intercenter study of treatment outcome in patients with complete cleft lip and palate. Cleft Palate Craniofac J. 2005;42(4):453.Semb G, Brattström V, Mølsted K, Prahl-Andersen B, Zuurbier P, Rumsey N, Shaw WC. The Eurocleft study: intercenter study of treatment outcome in patients with complete cleft lip and palate. Part 4: relationship among treatment outcome, patient/parent satisfaction, and the burden of care. Cleft Palate Craniofac J. 2005;42(1):83-92. Watkins SE, Meyer RE, Strauss RP, Aylsworth AS. Classification, epidemiology, and genetics of orofacial clefts. Clin Plast Surg. 2014;41(2):149-63. Coleman JR Jr, Sykes JM. The embryology, classification, epidemiology, and genetics of facial clefting. Facial Plast Surg Clin North Am. 2001;9(1):1-13.Pengelly RJ, Arias L, Martínez J, Upstill-Goddard R, Seaby EG, Gibson J, Ennis S, Collins A, Briceño I. Deleterious coding variants in multi-case families with non-syndromic cleft lip and/or palate phenotypes. Sci Rep. 2016;6:30457.Ren Y, Steegman R, Dieters A, Jansma J, Stamatakis H. Bone-anchored maxillary protraction in patients with unilateral complete cleft lip and palate and Class III malocclusion. Clin Oral Investig. 2019;23(5):2429-2441.Alberconi TF, Siqueira GLC, Sathler R, Kelly KA, Garib DG. Assessment of Orthodontic Burden of Care in Patients With Unilateral Complete Cleft Lip and Palate. Cleft Palate Craniofac J. 2018;55(1):74-78.Eriguchi M, Watanabe A, Suga K, Nakano Y, Sakamoto T, Sueishi K, Uchiyama T. Growth of Palate in Unilateral Cleft Lip and Palate Patients Undergoing Two-stage Palatoplasty and Orthodontic Treatment. Bull Tokyo Dent Coll. 2018;59(3):183-91.Smane L, Pilmane M. Evaluation of the presence of MMP-2, TIMP-2, BMP2/4, and TGFβ3 in the facial tissue of children with cleft lip and palate. Acta Med Litu. 2018;25(2):86-94. AlHayyan WA, Pani SC, AlJohar AJ, AlQatami FM. The Effects of Presurgical Nasoalveolar Molding on the Midface Symmetry of Children with Unilateral Cleft Lip and Palate: A Long-term Follow-up Study. Plast Reconstr Surg Glob Open. 2018;6(7):e1764. Thakur S, Singh A, Thakur NS, Diwana VK. Achievement in Nasal Symmetry after Cheiloplasty in Unilateral Cleft Lip and Palate Infants Treated with Presurgical Nasoalveolar Molding. Contemp Clin Dent. 2018;9(3):357-60. Turri de Castro Ribeiro T, Petri Feitosa MC, Almeida Penhavel R, Zanda RS, Janson G, Mazzottini R, Garib DG. Extreme maxillomandibular discrepancy in unilateral cleft lip and palate: Longitudinal follow-up in a patient with mandibular prognathism. Am J Orthod Dentofacial Orthop. 2018;154(2):294-304. Perillo L, Vitale M, d'Apuzzo F, Isola G, Nucera R, Matarese G. Interdisciplinary approach for a patient with unilateral cleft lip and palate. Am J Orthod Dentofacial Orthop. 2018;153(6):883-94. Hoffmannova E, Moslerová V, Dupej J, Borský J, Bejdová Š, Velemínská J. Three-dimensional development of the upper dental arch in unilateral cleft lip and palate patients after early neonatal cheiloplasty. Int J Pediatr Otorhinolaryngol. 2018;109:1-6. Tan ELY, Kuek MC, Wong HC, Ong SAK, Yow M. Secondary Dentition Characteristics in Children With Nonsyndromic Unilateral Cleft Lip and Palate: A Retrospective Study. Cleft Palate Craniofac J. 2018;55(4):582-89. Rodrigues R, Fernandes MH, Monteiro AB, Furfuro R, Sequeira T, Silva CC, Manso MC. SPINA classification of cleft lip and palate: A suggestion for a complement. Arch Pediatr. 2018;25(7):439-41. Ortiz-Posadas MR, Vega-Alvarado L, Maya-Behar J. A new approach to classify cleft lip and palate. Cleft Palate Craniofac J. 2001;38(6):545-50.Spina V, Psillakis JM, Lapa FS, Ferreira MC. Classificação das fissuras lábio-palatinas. Sugestão de modificação [Classification of cleft lip and cleft palate. Suggested changes]. Rev Hosp Clin Fac Med Sao Paulo. 1972;27(1):5-6. Allori AC, Mulliken JB, Meara JG, Shusterman S, Marcus JR. Classification of Cleft Lip/Palate: Then and Now. Cleft Palate Craniofac J. 2017;54(2):175-88. Spina V. A proposed modification for the classification of cleft lip and cleft palate. Cleft Palate J. 1973;10:251-2. Yun-Chia Ku M, Lo LJ, Chen MC, Wen-Ching Ko E. Predicting need for orthognathic surgery in early permanent dentition patients with unilateral cleft lip and palate using receiver operating characteristic analysis. Am J Orthod Dentofacial Orthop. 2018;153(3):405-14. Garib D, Yatabe M, de Souza Faco RA, Gregório L, Cevidanes L, de Clerck H. Bone-anchored maxillary protraction in a patient with complete cleft lip and palate: A case report. Am J Orthod Dentofacial Orthop. 2018;153(2):290-97. De Stefani A, Bruno G, Balasso P, Mazzoleni S, Baciliero U, Gracco A. Teeth agenesis evaluation in an Italian sample of complete unilateral and bilateral cleft lip and palate patients. Minerva Stomatol. 2018;67(4):156-64. Chang SY, Lonic D, Pai BC, Lo LJ. Primary Repair in Patients With Unilateral Complete Cleft of Lip and Primary Palate: Assessment of Outcomes. Ann Plast Surg. 2018;80(2S Suppl 1):S2-6.Vura N, Gaddipati R, Palla Y, Kumar P. An Intraoral Appliance to Retract the Protrusive Premaxilla in Bilateral Cleft Lip Patients Presenting Late for Primary Lip Repair. Cleft Palate Craniofac J. 2018;55(4):622-25.Massie JP, Bruckman K, Rifkin WJ, Runyan CM, Shetye PR, Grayson B, Flores RL. The Effect of Nasoalveolar Molding on Nasal Airway Anatomy: A 9-Year Follow-up of Patients With Unilateral Cleft Lip and Palate. Cleft Palate Craniofac J. 2018;55(4):596-601. Jabbari F, Wiklander L, Reiser E, Thor A, Hakelius M, Nowinski D. Secondary Alveolar Bone Grafting in Patients Born With Unilateral Cleft Lip and Palate: A 20-Year Follow-up. Cleft Palate Craniofac J. 2018;55(2):173-79.Jones CM, Roth B, Mercado AM, Russell KA, Daskalogiannakis J, Samson TD, Hathaway RR, Smith A, Mackay DR, Long RE Jr. The Americleft Project: Comparison of Ratings Using Two-Dimensional Versus Three-Dimensional Images for Evaluation of Nasolabial Appearance in Patients With Unilateral Cleft Lip and Palate. J Craniofac Surg. 2018;29(1):105-8. Gatti GL, Freda N, Giacomina A, Montemagni M, Sisti A. Cleft Lip and Palate Repair. J Craniofac Surg. 2017;28(8):1918-24.

Background: Designing and implementation of reverse logistics (RL) network which meets the sustainability targets have been a matter of emerging concern for the electronics companies in India.Objectives: The present study developed a two-phase model for configuration of sustainable RL network design for an Indian manufacturing company to manage its end-of-life and endof-use electronic products. The notable feature of the model was the evaluation of facilities under financial, environmental and social considerations and integration of the facility selection decisions with the network design.Method: In the first phase, an integrated Analytical Hierarchical Process Complex Proportional Assessment methodology was used for the evaluation of the alternative locations in terms of their degree of utility, which in turn was based on the three dimensions of sustainability. In the second phase, the RL network was configured as a bi-objective programming problem, and fuzzy optimisation approach was utilised for obtaining a properly efficient solution to the problem.Results: The compromised solution attained by the proposed fuzzy model demonstrated that the cost differential for choosing recovery facilities with better environmental and social performance was not significant; therefore, Indian manufacturers must not compromise on the sustainability aspects for facility location decisions.Conclusion: The results reaffirmed that the bi-objective fuzzy decision-making model can serve as a decision tool for the Indian manufacturers in designing a sustainable RL network. The multi-objective optimisation model captured a reasonable trade-off between the fuzzy goals of minimising the cost of the RL network and maximising the sustainable performance of the facilities chosen.

AbstractReinforcement learning (RL) has emerged as an effective approach for building an intelligent system, which involves multiple self-operated agents to collectively accomplish a designated task. More importantly, there has been a renewed focus on RL since the introduction of deep learning that essentially makes RL feasible to operate in high-dimensional environments. However, there are many diversified research directions in the current literature, such as multi-agent and multi-objective learning, and human-machine interactions. Therefore, in this paper, we propose a comprehensive software architecture that not only plays a vital role in designing a connect-the-dots deep RL architecture but also provides a guideline to develop a realistic RL application in a short time span. By inheriting the proposed architecture, software managers can foresee any challenges when designing a deep RL-based system. As a result, they can expedite the design process and actively control every stage of software development, which is especially critical in agile development environments. For this reason, we design a deep RL-based framework that strictly ensures flexibility, robustness, and scalability. To enforce generalization, the proposed architecture also does not depend on a specific RL algorithm, a network configuration, the number of agents, or the type of agents.

PurposeTo maximize the network total profit by calculating the difference between costs and revenue (first objective function). To maximize the positive impact on the environment by integrating GSCM factors in RL (second objective function). To calculate the efficiency of disassembly centers by SDEA method, which are selected as suppliers and maximize the total efficiency (third objective function). To evaluate the resources and total efficiency of the proposed model to facilitate the allocation resource process, to increase resource efficiency and to improve the efficiency of disassembly centers by Inverse DEA.Design/methodology/approachThe design of a closed-loop logistics network for after-sales service for mobile phones and digital cameras has been developed by the mixed-integer linear programming method (MILP). Development of MILP method has been performed by simultaneously considering three main objectives including: total network profit, green supply chain factors (environmental sustainability) and maximizing the efficiency of disassembly centers. The proposed model of study is a six-level, multi-objective, single-period and multi-product that focuses on electrical waste. The efficiency of product return centers is calculated by SDEA method and the most efficient centers are selected.FindingsThe results of using the model in a case mining showed that, due to the use of green factors in network design, environmental pollution and undesirable disposal of some electronic waste were reduced. Also, with the reduction of waste disposal, valuable materials entered the market cycle and the network profit increased.Originality/value(1) Design a closed-loop reverse logistics network for after-sales services; (2) Introduce a multi-objective multi-echelon mixed integer linear programming model; (3) Sensitivity analysis use Inverse-DEA method to increase the efficiency of inefficient units; (4) Use the GSC factors and DEA method in reverse logistics network.

AbstractMulti-agent actor-critic algorithms are an important part of the Reinforcement Learning (RL) paradigm. We propose three fully decentralized multi-agent natural actor-critic (MAN) algorithms in this work. The objective is to collectively find a joint policy that maximizes the average long-term return of these agents. In the absence of a central controller and to preserve privacy, agents communicate some information to their neighbors via a time-varying communication network. We prove convergence of all the three MAN algorithms to a globally asymptotically stable set of the ODE corresponding to actor update; these use linear function approximations. We show that the Kullback–Leibler divergence between policies of successive iterates is proportional to the objective function’s gradient. We observe that the minimum singular value of the Fisher information matrix is well within the reciprocal of the policy parameter dimension. Using this, we theoretically show that the optimal value of the deterministic variant of the MAN algorithm at each iterate dominates that of the standard gradient-based multi-agent actor-critic (MAAC) algorithm. To our knowledge, it is the first such result in multi-agent reinforcement learning (MARL). To illustrate the usefulness of our proposed algorithms, we implement them on a bi-lane traffic network to reduce the average network congestion. We observe an almost 25% reduction in the average congestion in 2 MAN algorithms; the average congestion in another MAN algorithm is on par with the MAAC algorithm. We also consider a generic 15 agent MARL; the performance of the MAN algorithms is again as good as the MAAC algorithm.

AbstractAutonomous driving has attracted significant research interests in the past two decades as it offers many potential benefits, including releasing drivers from exhausting driving and mitigating traffic congestion, among others. Despite promising progress, lane-changing remains a great challenge for autonomous vehicles (AV), especially in mixed and dynamic traffic scenarios. Recently, reinforcement learning (RL) has been widely explored for lane-changing decision makings in AVs with encouraging results demonstrated. However, the majority of those studies are focused on a single-vehicle setting, and lane-changing in the context of multiple AVs coexisting with human-driven vehicles (HDVs) have received scarce attention. In this paper, we formulate the lane-changing decision-making of multiple AVs in a mixed-traffic highway environment as a multi-agent reinforcement learning (MARL) problem, where each AV makes lane-changing decisions based on the motions of both neighboring AVs and HDVs. Specifically, a multi-agent advantage actor-critic (MA2C) method is proposed with a novel local reward design and a parameter sharing scheme. In particular, a multi-objective reward function is designed to incorporate fuel efficiency, driving comfort, and the safety of autonomous driving. A comprehensive experimental study is made that our proposed MARL framework consistently outperforms several state-of-the-art benchmarks in terms of efficiency, safety, and driver comfort.

PurposeReverse logistics (RL) is a type of supply chain management that moves goods from the end customer to the original manufacturer for reuse, remanufacturing and disposal purposes. Owing to growing environmental legislations and the development of new technologies in marketing, RL has attracted more significance among experts and academicians. Outsourcing RL practices to third-party reverse logistics provider (3PRLP) has been identified as one of the most important management strategies due to complexity of RL operations and the lack of available resource. Current sustainability trends have made 3PRLP assessment and selection process more complex. In order to select the 3PRLP, the existence of several aspects of sustainability motivates the experts to establish a new multi-criteria decision analysis (MCDA) approach.Design/methodology/approachWith the growing complexity and high uncertainty of decision environments, the preference values of 3PRLPs are not always expressed with real numbers. As the generalized version of fuzzy set, intuitionistic fuzzy set and Fermatean fuzzy set, the theory of q-rung orthopair fuzzy set (q-ROFS) is used to permit decision experts (DEs) to their assessments in a larger space and to better cope with uncertain information. Given that the combined compromise solution (CoCoSo) is an innovative MCDA approach with higher degree of stability and reliability than several existing methods.FindingsTo exhibit the potentiality and applicability of the presented framework, a case study of S3PRLPs assessment is taken from q-rung orthopair fuzzy perspective. The assessment process consists of three sustainability aspects namely economic, environment and social dimensions related with a total of 14 criteria. Further, sensitivity and comparative analyses are made to display the solidity and strength of the presented approach. The results of this study approve that the presented methodology is more stable and efficient in comparison with other methods.Originality/valueThus, the objective of the study is to develop a hybrid decision-making methodology by combining CoCoSo method and discrimination measure with q-ROFS for selecting an appropriate sustainable 3PRLP (S3PRLP) candidate under uncertain environment. In the proposed method, a novel procedure is proposed to obtain the weights of DEs within q-ROFS context. To calculate the criteria weights, a new formula is presented based on discrimination measure, which provides more realistic weights. In this respect, a new discrimination measure is proposed for q-ROFSs.

Intelligent decision control is one key issue of automated vehicles. Complex dynamic traffic flow and multi-requirement of passengers including vehicle safety, comfort, vehicle efficiency bring about tremendous challenges to vehicle decision making. Overtaking maneuver is a demanding task due to its large potential of traffic collision. Therefore, this paper proposes a fuzzy-inference-based reinforcement learning (FIRL) approach of autonomous overtaking decision making. Firstly, the problem of overtaking is formulated as a multi-objective Markov decision process (MDP) considering vehicle safety, driving comfort, and vehicle efficiency. Secondly, a temporal difference learning based on dynamic fuzzy (DF-TDL) is presented to learn optimized policies for autonomous overtaking decision making. Fuzzy inference is introduced to deal with continuous states and boost learning process. The RL algorithm decides whether to overtake or not based on the learned policies. Then, the automated vehicle executes local path planning and tracking. Furthermore, a simulation platform based on simulation of urban mobility (SUMO) is established to generate the random training data, that is, various traffic flows for algorithm iterative learning and validate the proposed method, extensive test results demonstrate the effectiveness of the overtaking decision-making method.

Abstract Reinforcement learning algorithms can autonomously learn to search a design space for high-performance solutions. However, modern engineering often entails the use of computationally-intensive simulation, which can lead to slower design timelines with highly iterative approaches such as RL. This work provides a reinforcement learning framework that leverages models of varying fidelity to enable an effective solution search while reducing overall computational needs. Specifically, it utilizes models of varying fidelity while training the agent, iteratively progressing from low- to high-fidelity. To demonstrate the effectiveness of the proposed framework, we apply it to two multimodal multi-objective constrained mixed integer nonlinear design problems involving the components of a ground and aerial vehicle. Specifically, for each problem we utilize a high-fidelity and a low-fidelity deep neural network surrogate model, trained on performance data generated from underlying ground truth models. A tradeoff between solution quality and the proportion of low-fidelity surrogate model usage is observed. Specifically, high quality solutions are achieved with substantial reductions in computational expense, showcasing the effectiveness of the framework for design problems where the use of just a high-fidelity model is infeasible. This solution quality-computational efficiency tradeoff is contextualized by visualizing the exploration behavior of the design agents.

AbstractPredictive materials synthesis is the primary bottleneck in realizing functional and quantum materials. Strategies for synthesis of promising materials are currently identified by time-consuming trial and error and there are no known predictive schemes to design synthesis parameters for materials. We use offline reinforcement learning (RL) to predict optimal synthesis schedules, i.e., a time-sequence of reaction conditions like temperatures and concentrations, for the synthesis of semiconducting monolayer MoS2 using chemical vapor deposition. The RL agent, trained on 10,000 computational synthesis simulations, learned threshold temperatures and chemical potentials for onset of chemical reactions and predicted previously unknown synthesis schedules that produce well-sulfidized crystalline, phase-pure MoS2. The model can be extended to multi-task objectives such as predicting profiles for synthesis of complex structures including multi-phase heterostructures and can predict long-time behavior of reacting systems, far beyond the domain of molecular dynamics simulations, making these predictions directly relevant to experimental synthesis.

Social virtual spaces proliferate on the contemporary Internet and some 80% of Internet users may now be regularly visiting them (Daniel). In the following discussion, I shall discuss one such social space—a multi-user virtual environment (MUVE) called Second Life (SL as it is referred to by residents)—and argue that complex and dialogic links exist between the offline user and her/his online representative, the avatar.I shall begin by presenting a brief overview of relevant theoretical concepts drawn largely from symbolic interactionist theorists. I shall then discuss where we might situate the avatar within the wider context of persona studies and explore the complexity involved in developing a sense of self via an avatar (or persona). Finally, I shall draw on my own experience in SL to illustrate the two-way nature of the processes under discussion.Mirrors and SelvesWhen one looks into a mirror, the mirror’s silver backing (the tain) allows a view of self and surroundings reflected back. Yet even in this case, our reflection has subtle and fascinating differences, it is not quite exactly “us.” Explanations for the effect this has on us come from various theoretical perspectives. For example, psychoanalyst Jacques Lacan writes that very young children experience in play the relation “between the movements assumed in the image and the reflected environment, and between this virtual complex and the reality it reduplicates” (2). The word “virtual” suggests a gap between the child’s performance and the exhibition of that performance in the mirror. Lacan contends that the child itself does not perceive this gap, but rather mistakes the image for the thing itself—a méconnaissance that ensures we accept the mirror image as “us.” The tain is thus simultaneously restrictive, because it has a limited field of view, and enabling, because it provides one of our first experiences of our self as other.From a symbolic interaction perspective, C. H. Cooley develops the mirror analogy in his Looking Glass Self concept. For Cooley, the self develops in relation to how we imagine we look to others and we learn to see others as our “mirror.” Our reflection in their perceptions requires us to imagine how we appear to others and then how they judge our appearance. Finally, we develop a sense of ourselves based on that judgement. Thus, others with whom we interact reflect us back to ourselves and we view our self from the viewpoint of others whose signs we learn to interpret. Two consequences arise. First, we could receive different reflections back from different individuals. Second, we become habituated to a gap between performance of self, its reception, and the reflection of that performance back to us.Erving Goffman’s work on self-presentation and interaction rituals offers further dimensions to understanding this process, stressing the importance of impression management to monitor inconsistencies in performance of the self as it appears to others. For Goffman, this management occurs both before the performance as we prepare ourselves in the back stage, and on the front stage as we perform for our audience. Social interaction is thus a performance, governed by social rules and rituals understood by both audience and performer. The dialectic relationship between performance and its reception introduces an element of intentionality to the performance itself. When the performance is not well received, for example, its reflection back to the performer may lead to adjustments in the performance to correct flaws brought about by failure to appropriately follow the rules of interaction. Through interaction with others, therefore, we learn dramaturgically appropriate roles and performances and we come to understand the nuances involved in successful enactment. The “audience” for whom we perform may be either internalised or actual. In either case, the image that we see in the mirror, and the image we imagine reflected back is both “us” and something more. But it is unhelpful to think of the internalised and actual as dichotomous.All three models so far discussed intuit a space between performance and reflection, suggesting that we experience our self through “symbols, language, social structures, and situated variables of social interaction” (Waskul and Lust 338-39) rather than directly. Even our image in an actual mirror extends, then, beyond the tain by which the glass is backed because we overlay what we see in the mirror with social nuances arising outside of the image-reflection dyad. Rather than consider this image a reflection of “us,” it is helpful to think of it as a persona—the personality (or presence) that a person adopts and presents to other people. It may be “our” persona in that it is linked to a physical person, but, as noted above, it may also be contextually multiple and variously mediated through social interaction and symbols such as dress.To explore more fully the interplay between person and persona I shall now introduce online contexts as sites of reflection, beginning with a brief discussion of the avatar as persona.Online MirrorsMarshall argues that contemporary culture exhibits an expansive world of online persona creation with individuals increasingly engaging in self-branding (Personifying). Although Marshall does not discuss MUVEs, his observation is equally applicable to such environments. In MUVEs, as in the online contexts Barbour and Marshall discuss, persona creation is a process of strategic intentionality whereby we present a chosen aspect from among the many to be found in us all. In MUVES the vehicle for that creation is the avatar. The avatar is an individual’s embodiment in virtual space, an extension of self through which the user experiences the virtual world (Behm-Morawitz). Just as the persona permits us “to explore the masks of identity” (Marshall, Personifying 380), the avatar offers opportunities for exploration and experimentation. For Marshall the persona in the public on-line world is constructed by media and communication systems and enacted through individual intention and agency (Personifying). The avatar is similarly constructed and enacted. Both persona and avatar are mutable and, as Marshall suggests in relation to persona, part of a specular economy manifesting an increasing consciousness of self-presentation and others’ perceptions (Specular). I do not think it overstated to indicate these similarities with the composite term “avatar-persona.”The graphical object-body is the vehicle whereby MUVE users experience interacting with others and with their environment (Messinger et al.). They experience their avatar self as if it were their actual self (Behm-Morawitz). Our virtual experiences are grounded in, and inextricably linked to, our physicality. One’s “presence” with one’s avatar may facilitate and be uniquely linked to avatar influence on the offline self (Behm-Morawitz). In this sense “presence”—the sense of being actually present and being recognised as present by others there—may bridge both sides of the screen. This two-way transfer is analogous to the person’s capacity to move others into action noted by Marshall (Personifying). Further, as some research has shown, the representation of self through an avatar not only effects online behaviour but actually may also have continued effects on offline behaviour and avatars may come to change who we are in both online and offline environments (Yee and Bailenson).Marshall (Specular) argues that the online and mobile media screen as mirror produces persona and that the mirror as a surface reflects and allows one to be seen and to interrelate or communicate with others. The MUVE also acts as a virtual mirror screen within which the avatar-persona operates. The avatar-persona is the virtual analogue of the mirror persona discussed in relation to Lacan and symbolic interaction formulations. I turn now to how these processes and interconnections manifest in SL in order to explore the complexities inherent in the interplay of self, avatar-persona and other.SL is a three dimensional virtual world where “everyone you see is a real person and every place you visit is built by people just like you” (http://secondlife.com/whatis/?lang=en-US). SL “residents” (as they refer to themselves) engage in role-playing games in-world, co-create content with other residents, and indulge in a huge variety of social activities including sexual and/or affective relationships with other residents. SL is an immersive social environment offering sophisticated graphical building tools, avatar appearance modification potentials, and both synchronous (real time) and asynchronous (delayed) avatar-to-avatar communication for residents who are geographically located in all parts of the offline world.In MUVEs, one sees the avatar-persona as a third person. In SL this is due to the potential for 360-degree camera views of the avatar, ensuring that our avatar becomes the object of our view, placing us in a position both of an active I controlling an avatar and a distanced other watching that self move and speak (Zhao). These dynamics raise interesting questions about interaction, self-presentation, and self-construction (Gottschalk), the answers to which represent a continuum between two far from mutually exclusive poles. On the one hand, research based on SL (see for example work by Messinger et al., or Martey and Consalvo) has shown that, despite an almost limitless potential for modification, most avatars are idealised representations of their creator’s offline selves. Given this correlation between online and offline manifestations, I suggest that the avatar operates more like a mirror that is not wholly restricted by the tain.On the other hand, writers such as Sherry Turkle have argued (before the existence of MUVEs) that the Internet permits multiplicity and mutability in subjectivity. In a contemporary context social virtual worlds provide “a free ‘potential space’ where real individuals—qua avatars—can and do attempt to create an alternative reality. Here they simultaneously concretize their individualistic fantasies […] and enact aspects of their selves they did not know exist, were too embarrassed to admit, or always wanted to master” (Gottschalk 521-22). SL permits hybridity of identity, plasticity of form, and multiplicity of avatars, ensuring fluid and chimerical possibilities (Morie and Verhulsdonck) for single or multiple avatar-persona per offline self. Residents frequently switch between avatar-persona to suit particular needs or social contexts. In this respect, SL is less a mirror than a kaleidoscope, where changing patterns emerge with a turn of the lens.In neither case is the process one-way. “When people define the virtual as real, it becomes real in its consequences, and the reciprocal effects between the self and the avatar extend to more central aspects of one’s life as well” (Gottschalk 513). Avatars are distinct selves, not just conduits for offline identities. They socially manifest a projective identity or identities that are influential intersections of offline people and online representations situated within socially performed dramaturgical selves (Martey and Consalvo).Cunningham writes that “[a]fter virtual reality, ‘reality’ is not the same, but has been altered by the bleeding of both ‘worlds’ into each other, by their mutual inseparability” (16). In this mutual inseparability a dialogic interaction occurs between offline self and avatar-persona. Both engage in a continuous interaction and active negotiation between the parties. It is in this dialogic that we find the eventual outcome of a mirror without a tain. The mirror’s “glass” no longer requires its tain for reflection because the dialogic between offline self and avatar-persona is maintained by the process of creation, performance, reception and exhibition as all parties operate under the gaze of others outside their individual dialogic. Other SL residents also see the avatar-persona, just as the offline creator interacts with others in physical space outside SL.Symbolic interactionist perspectives assist in understanding the reflexive processes through which individuals come to see themselves as objects of their own and others’ gaze(s) (Aspling). As object to one’s self and to others, self and avatar have already rehearsed a performance in their own view before permitting others to see that performance and reflect it back. Watching others respond to our avatar-persona provides feedback for self-representation and communication patterns (Gottschalk). Just as a curator assembles and presents an exhibition, our avatar image is “returned” to us subtly changed, re-presented to us at one step removed from its creation, rearranged in the judgement of the other, and manifesting the imagined reception by its viewer. This expands the self’s repertoire beyond SL, continuing to inform us offline and online (Gottschalk).The avatar-persona stimulates objective and rational observation of oneself, generating the “observing ego” (Gottschalk 514-515). Crucially, however, the avatar is more than just a placeholder for the self. The avatar is a site for self-making in its own right because it informs our offline life. In this way, the avatar may force us to partake in our own self-construction by taking “the role of the other,” another who is in fact both a persona and a person (Waskul and Lust 349-350). My own experience in SL further illustrates these ideas.Self-Representation—One Avatar’s Experience“Choices about avatar appearance can be understood as social performances that communicate both social and individual identities” (Martey and Consalvo 166). Although SL purports to confer on its residents near total control over all aspects of appearance and in-world identity, as in the offline world personal appearance in SL involves situated, bodily practices that are both discursively practical and function as a collection of codes that communicate to other users.I first learned of SL in 2007. Then, as now, it was depicted as a world of limitless possibilities. My first experience of avatar life was, however, somewhat disappointing. Entering the first stage in the avatar creation process and hopeful of creating an androgynous avatar, I was given only two default choices—male or female. Reflecting my offline self, I chose to make my first avatar female. I chose a name that was not gender-specific but that had personal meaning to me. Once in-world, and realising that I could drastically modify my avatar’s body shape, I set about making the avatar as androgynous as I could. The body modification was challenging, but not impossible. The avatar appearance modifiers were not fine grained enough to make the face authentically androgynous however. To circumvent this drawback I decided to make my avatar a “Furry”. Furries are anthropomorphic animal avatars with human figures and animal heads, hands, feet and tails. An animal head for my avatar-persona allowed me to avoid gender specificity.Thinking that I had successfully met my goal, I next ventured into the social spaces of SL. The first thing another avatar said to me was “So are you a boy or a girl in real life?” I evaded the question then, and continued to evade similar questions for three months. During that time I was frequently derided for my reluctance to gender identify. Although the sociologist in me found this fascinating, the almost constant questioning began to impact upon my enjoyment of SL. Eventually after one particularly nasty attack (called “griefing” in SL) my avatar was left so badly distorted that I decided to “retire” it. Examining my reaction, I was surprised to find that the remorselessly unpleasant reaction to my avatar had generated the bleeding between worlds referred to by Cunningham—my performance was exhibited back to me in unfavourable terms. I was upset. I had chosen a name for the avatar, an animal identity, and a personality that all had RL significance for me.My experience underscored for me a point Martey and Consalvo make. Avatar identity is self-constructed, they argue, within the constraints of the offline user’s goals, (I wanted to create and live in an androgynous avatar) the interface used to create the online appearance (the SL viewer interface was not sophisticated enough for me to easily do this, nor did it give me a third choice for the gender of my avatar), and the social systems of the virtual space (there was clearly an expectation that I was not meeting by refusing to disclose my offline sex). The bleed back was enough to generate decisions by my offline self multiple times in the three-month life of that avatar. My avatar self had not met with favourable audience reaction because I had refused to comply with dramaturgical propriety by disclosing my offline sex and had failed to create a fictional offline identity to get around the issue. My fault lay in not sufficiently aligning how my avatar looked and acted with its offline correlate because I refused to disclose any actual or false offline correlate. In dramaturgical terms, I refused to interact with audience reception thus stepping outside the interaction order.ConclusionI have argued that one develops a sense of self in interaction with others through actual and conceptual mirroring. The process can occur in many contexts, even in the absence of a co-present other because we internalise the other’s view. We bring these dynamics with us to online settings. Using SL as an example, I suggested that online the mistaken singularity of self and reflection noted by Lacan becomes multiple in the way a kaleidoscope generates multiple patterns when turned by the user. SL’s visuality and its potential for three-dimensional viewing of one’s avatar encourages a looking-glass-self approach to identity—a re-presentation of the self-object as we imagine others see it. We achieve identity in the eyes of multiple others, including seeing our avatar self as object. Avatar-residents in SL may act as mirrors to the offline self, but interactions are multiply complex, blurring the boundaries between online and offline experience(s), between offline person and online avatar-persona and between avatars and other avatar-residents. Impression management, interactional dynamics, and strategic self-representation render the avatar-persona one facet of the offline self rather than its entirety. Audience reaction comes not only from other avatar residents, but also from the offline self responding to that reaction and their own understanding of dramaturgical propriety. SL is a place where avatar-personas are fashioned in liminal boundaries between interaction between the self and its avatar/other (Waskul and Lust) as well as clarifying the interactions between self and others. Like other online contexts, SL is a mirror without a tain. In its specular economy, reflectivity is unbounded, an endlessly recombinant field of self-other interactions, a kaleidoscope of potential.ReferencesAspling, Fredrik. “The Private and the Public in Online Presentations of the Self: A Critical Development of Goffman’s Dramaturgical Perspective.” Stockholm’s Universitet, 2011.Barbour, Kim, and David Marshall. “The Academic Online: Constructing Persona through the World Wide Web.” First Monday 17.9 (2012). 13 May 2014 ‹ http://journals.uic.edu/ojs/index.php/fm/article/vew/3969/3292 ›.Behm-Morawitz, Elizabeth. “Mirrored Selves: The Influence of Self-Presence in a Virtual World on Health, Appearance, and Well-Being.” Computers in Human Behavior 29 (2013): 119-128.Cooley, Charles Horton. Human Nature and the Social Order. New York: Scribner’s, 1964.Cunningham, Kim. “Virtually Transformed: Second Life’s Implications for the Status of the Body.” 102nd American Sociological Association Annual Meeting. 2006.Daniel, John. “The Self Set Free.” Therapy Today 19.8 (2008).Goffman, Erving. The Presentation of Self in Everyday Life. New York: Anchor, 1959.Gottschalk, Simon. “The Presentation of Avatars in Second Life: Self and Interaction in Social Virtual Spaces.” Symbolic Interaction 33.4 (2010).Lacan, Jacques. Ecrits: A Selection. London and New York: Routledge, 1977.Marshall, David. “Personifying Agency: The Public-Persona-Place-Issue Continuum.” Celebrity Studies 4.3 (2013): 369-371.Marshall, David. “The Specular Economy.” Society 47 (2010): 498-502.Martey, Rosa Mikael, and Mia Consalvo. “Performing the Looking-Glass Self: Avatar Appearance and Group Identity in Second Life.” Popular Communication 9.3 (2011): 165-80.Messinger, Paul R., et al. “On the Relationship between My Avatar and Myself.” Journal of Virtual Worlds Research 1.1 (2008): 1-17.Morie, Jacquelyn Ford, and Gustav Verhulsdonck. “Body/Persona/Action! Emerging Non-Anthropomorphic Communication and Interaction in Virtual Worlds.” Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology. ACM, 2008.Turkle, Sherry. Life on the Screen: Identity in the Age of the Internet. New York: Touchstone, 1995.Waskul, Dennis, and Matt Lust. “Role-Playing and Playing Roles: The Person, Player, and Persona in Fantasy Role-Playing.” Symbolic Interaction 27.3 (2004): 333-56.Yee, N., and J. Bailenson. “The Proteus Effect: The Effect of Transformed Self-Representation on Behavior.” Human Communication Research 33.3 (2007): 271-290.Zhao, Shanyang. “The Digital Self: Through the Looking Glass of Telecopresent Others.” Symbolic Interaction 28.3 (2005): 387-405.

Abstract Objective. To describe the quality of the relationship between nurses and patients under mechanical ventilation.Methods. This observational study, performed in a cardiac surgery intensive care unit in Iran, selected 10 nurses and 35 patients through simple random and convenience sampling, respectively. One of the researchers observed 175 communications between nurses and patients in different work shifts and recorded the results according to a checklist. Nurse and patient satisfaction with the communication was assessed by using a six-item Likert scale, 8 to 12 h after extubation.Results. Most of the patients were male (77.1%), while most of the nurses were female (60%). Patients started over 75% of the communications observed. The content of the communication was related mostly to physical needs and pain. Besides, the majority of patients used purposeful stares and hand gestures, and head nod for communication.Most of the communications between patients and nurses were satisfied ‘very low’ (45.7% in nurses, versus 54.3% in patients). However, ‘complete satisfaction’ was lower in nurses (0%), compared with patients (5.7%). No statistically significant correlation was found between patients’ and nurses’ satisfaction and demographic variables.Conclusion. The results showed that communication between nurses and mechanically ventilated patients was built through traditional methods and was based on the patients’ requests. This issue might be the cause of an undesirable level of their satisfaction with the communication, given that effective communication can lead to understanding and meeting the needs of the patients.Descriptors: non-verbal communication; ventilators, mechanical; cardiac care facilities; patient satisfaction; intensive care units.How to cite this article: Momennasab M, Ardakani MS, Rad FD, Dokoohaki R, Dakhesh R, Jaberi A. Quality of Nurses’ Communication with Mechanically Ventilated Patients in a Cardiac Surgery Intensive Care Unit. Invest. Educ. Enferm. 2019; 37(2):e02.ReferencesFelce D, Perry J. Quality of Life: Its Definition and Measurement. Res. Dev. Disabil. 1995; 16(1):51-74.Khalaila R, Zbidat W, Anwar K, Bayya A, Linton DM, Sviri S. Communication difficulties and psychoemotional distress in patients receiving mechanical ventilation. Am. J. Crit. Care. 2011; 20(6):470-9.Wang Y, Li H, Zou H, Li Y. Analysis of complaints from patients during mechanical ventilation after cardiac surgery: a retrospective study. J. Cardiothorac. Vasc. Anesth. 2015; 29(4):990-4.Myhren H, Ekeberg O, Stokland O. Satisfaction with communication in ICU patients and relatives: comparisons with medical staffs’ expectations and the relationship with psychological distress. Patient Educ. Couns. 2011; 85(2):237-44.Marasinghe M, Fonseka W, Wanishri P, Nissanka N, Silva B. An Exploration of Patients’ Experiences of Mechanical Ventilation. OUSL J. 2015; 9:83-96.Flinterud SI, Andershed B. Transitions in the communication experiences of tracheostomised patients in intensive care: a qualitative descriptive study. J. Clin. Nurs. 2015; 24(15-16):2295-304.Sutt A-L, Cornwell P, Mullany D, Kinneally T, Fraser JF. The use of tracheostomy speaking valves in mechanically ventilated patients results in improved communication and does not prolong ventilation time in cardiothoracic intensive care unit patients. J. Crit. Care. 2015; 30(3):491-4.Happ MB, Seaman JB, Nilsen ML, Sciulli A, Tate JA, Saul M, et al. The number of mechanically ventilated ICU patients meeting communication criteria. Heart Lung. 2015; 44(1):45-9.Happ MB, Sereika SM, Houze MP, Seaman JB, Tate JA, Nilsen ML, et al. Quality of care and resource use among mechanically ventilated patients before and after an intervention to assist nurse-nonvocal patient communication. Heart Lung.2015; 44(5):408-15.Alasad J, Ahmad M. Communication with critically ill patients. Journal of advanced nursing. 2005; 50(4):356-62.Sabet Sarvestani R, Moattari M, Nasrabadi AN, Momennasab M, Yektatalab S. Challenges of nursing handover: A qualitative study. Clin. Nurs. Res. 2015; 24(3):234-52.Anderson WG, Puntillo K, Boyle D, Barbour S, Turner K, Cimino J, et al. ICU Bedside Nurses’ Involvement in Palliative Care Communication: A Multicenter Survey. J. Pain Symptom Manage. 2016; 51(3):589-96.Otuzoğlu M, Karahan A. Determining the effectiveness of illustrated communication material for communication with intubated patients at an intensive care unit. Int. J. Nurs. Pract. 2014; 20(5):490-8.Jarvis C, Forbes H, Watt E. Jarnis’s physical examination & health assessment. Sydney: Saunders Elsevier Australia; 2012.Karlsson V, Forsberg A, Bergbom I. Communication when patients are conscious during respirator treatment—A hermeneutic observation study. Intensive Crit. Care Nurs. 2012; 28(4):197-207. Shafipour V, Mohammad E, Ahmadi F. Barriers to Nurse-Patient Communication in Cardiac Surgery Wards: A Qualitative Study. Glob. J. Health Science. 2014; 6(6):234-44.Arabi A, Tavakol K. Patient’s experiences of mechanical ventilation. Iran. J. Nurs. Midwifery Res. 2009; 14(2).Taylor RC, Lillis C, LeMone P. Fundamentals of Nursing: The Art and Science of Nursing Care. Philadelphia: Lippincott Williams & Wilkins; 2010.Chahraoui K, Laurent A, Bioy A, Quenot J-P. Psychological experience of patients 3 months after a stay in the intensive care unit: A descriptive and qualitative study. J. Crit. Care. 2015; 30(3):599-605.Happ MB, Garrett K, Thomas DD, et al. Nurse-patient communication interactions in the intensive care unit. American J. Crit. Care..2011; 20(2):e28-e40.Tadrisi S, Madani S, Farmand F, Ebadi A, KarimiZarchi AA, Mirhashemi S, et al. Richmond agitation–sedation scale validity and reliability in intensive care unit adult patients Persian version. J. Crit. Care. Nurs. 2009; 2(1):15-21.Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical scale. Lancet. 1974; 304(7872):81-4. Invest Educ Enferm. 2019; 37(2): e02 Quality of Nurses’ Communication with Mechanically Ventilated Patients in a Cardiac Surgery Intensive Care UnitNilsen ML, Sereika SM, Hoffman LA, Barnato A, Donovan H, Happ MB. Nurse and Patient Interaction Behaviors’ Effects on Nursing Care Quality for Mechanically Ventilated Older Adults in the ICU. Res. Gerontol. Nurs. 2014; 7(3):113-25.Gold RL. Roles in sociological field observations. Soc. Forces. 1958:217-23.Gashmard R, Bagherzadeh R, Pouladi Sh, Akaberuan S, Jahanor F. Evaluating the Factors Influencing Productivity of Medical Staff in Hospitals Affiliated Bushehr University of Medical Sciences 2012, Bushehr, Iran. World Appl. Sci. J. 2013; 28(12):2061-8.Magnus VS, Turkington L. Communication interaction in ICU—patient and staff experiences and perceptions. Intensive Crit. Care Nurs. 2006; 22(3):167-80.SabetSarvestani R, Moattari M, Nasrabadi AN, Momennasab M, Yektatalab S, Jafari A. Empowering nurses through action research for developing a new nursing handover program in a pediatric ward in Iran. Action Res. 2017; 15(2):214-35.Momennasab M, Ghahramani T, Yektatalab S, Zand F. Physical and Mental Health of Patients Immediately After Discharge From Intensive Care Unit and 24 Hours Later. Trauma Mon. 2016; 21(1):e29231.Hedayati E, Hazrati M, Momennasab M, et al. The effect of need-based spiritual/religious intervention on spiritual well-being and anxiety of elderly people. Holist. Nurs. Pract. 2015; 29(3):136-43. Balandin S, Hemsley B, Sigafoos J, Green V. Communicating with nurses: The experiences of 10 adults with cerebral palsy and complex communication needs. Appl. Nurs. Res. 2007; 20(2):56-62.Happ MB, Tuite P, Dobbin K, DiVirgilio-Thomas D, Kitutu J. Communication ability, method, and content among nonspeaking nonsurviving patients treated with mechanical ventilation in the intensive care unit. Am. J. Crit. Care. 2004; 13(3):210-8.Happ MB, Garrett KL, Tate JA, DiVirgilio D, Houze MP, Demirci JR, et al. Effect of a multi-level intervention on nurse–patient communication in the intensive care unit: results of the SPEACS trial. Heart Lung. 2014; 43(2):89- 98.