Journal articles: 'Sequential decision processes'

1

Alagoz, Oguzhan, Heather Hsu, Andrew J. Schaefer, and Mark S. Roberts. "Markov Decision Processes: A Tool for Sequential Decision Making under Uncertainty." Medical Decision Making 30, no. 4 (December 31, 2009): 474–83. http://dx.doi.org/10.1177/0272989x09353194.

Full text

Abstract:

We provide a tutorial on the construction and evaluation of Markov decision processes (MDPs), which are powerful analytical tools used for sequential decision making under uncertainty that have been widely used in many industrial and manufacturing applications but are underutilized in medical decision making (MDM). We demonstrate the use of an MDP to solve a sequential clinical treatment problem under uncertainty. Markov decision processes generalize standard Markov models in that a decision process is embedded in the model and multiple decisions are made over time. Furthermore, they have significant advantages over standard decision analysis. We compare MDPs to standard Markov-based simulation models by solving the problem of the optimal timing of living-donor liver transplantation using both methods. Both models result in the same optimal transplantation policy and the same total life expectancies for the same patient and living donor. The computation time for solving the MDP model is significantly smaller than that for solving the Markov model. We briefly describe the growing literature of MDPs applied to medical decisions.

APA, Harvard, Vancouver, ISO, and other styles

2

Sobel, Matthew J., and Wei Wei. "Myopic Solutions of Homogeneous Sequential Decision Processes." Operations Research 58, no. 4-part-2 (August 2010): 1235–46. http://dx.doi.org/10.1287/opre.1090.0767.

Full text

APA, Harvard, Vancouver, ISO, and other styles

3

El Chamie, Mahmoud, Dylan Janak, and Behçet Açıkmeşe. "Markov decision processes with sequential sensor measurements." Automatica 103 (May 2019): 450–60. http://dx.doi.org/10.1016/j.automatica.2019.02.026.

Full text

APA, Harvard, Vancouver, ISO, and other styles

4

Feinberg, Eugene A. "On essential information in sequential decision processes." Mathematical Methods of Operations Research 62, no. 3 (November 15, 2005): 399–410. http://dx.doi.org/10.1007/s00186-005-0035-3.

Full text

APA, Harvard, Vancouver, ISO, and other styles

5

Maruyama, Yukihiro. "Strong representation theorems for bitone sequential decision processes." Optimization Methods and Software 18, no. 4 (August 2003): 475–89. http://dx.doi.org/10.1080/1055678031000154707.

Full text

APA, Harvard, Vancouver, ISO, and other styles

6

Milani Fard, M., and J. Pineau. "Non-Deterministic Policies in Markovian Decision Processes." Journal of Artificial Intelligence Research 40 (January 5, 2011): 1–24. http://dx.doi.org/10.1613/jair.3175.

Full text

Abstract:

Markovian processes have long been used to model stochastic environments. Reinforcement learning has emerged as a framework to solve sequential planning and decision-making problems in such environments. In recent years, attempts were made to apply methods from reinforcement learning to construct decision support systems for action selection in Markovian environments. Although conventional methods in reinforcement learning have proved to be useful in problems concerning sequential decision-making, they cannot be applied in their current form to decision support systems, such as those in medical domains, as they suggest policies that are often highly prescriptive and leave little room for the user's input. Without the ability to provide flexible guidelines, it is unlikely that these methods can gain ground with users of such systems. This paper introduces the new concept of non-deterministic policies to allow more flexibility in the user's decision-making process, while constraining decisions to remain near optimal solutions. We provide two algorithms to compute non-deterministic policies in discrete domains. We study the output and running time of these method on a set of synthetic and real-world problems. In an experiment with human subjects, we show that humans assisted by hints based on non-deterministic policies outperform both human-only and computer-only agents in a web navigation task.

APA, Harvard, Vancouver, ISO, and other styles

7

Maruyama, Yukihiro. "SUPER-STRONG REPRESENTATION THEOREMS FOR NONDETERMINISTIC SEQUENTIAL DECISION PROCESSES." Journal of the Operations Research Society of Japan 60, no. 2 (2017): 136–55. http://dx.doi.org/10.15807/jorsj.60.136.

Full text

APA, Harvard, Vancouver, ISO, and other styles

8

Hantula, Donald A., and Charles R. Crowell. "Intermittent Reinforcement and Escalation Processes in Sequential Decision Making:." Journal of Organizational Behavior Management 14, no. 2 (June 30, 1994): 7–36. http://dx.doi.org/10.1300/j075v14n02_03.

Full text

APA, Harvard, Vancouver, ISO, and other styles

9

Canbolat, Pelin G., and Uriel G. Rothblum. "(Approximate) iterated successive approximations algorithm for sequential decision processes." Annals of Operations Research 208, no. 1 (February 8, 2012): 309–20. http://dx.doi.org/10.1007/s10479-012-1073-x.

Full text

APA, Harvard, Vancouver, ISO, and other styles

10

Ying, Ming-Sheng, Yuan Feng, and Sheng-Gang Ying. "Optimal Policies for Quantum Markov Decision Processes." International Journal of Automation and Computing 18, no. 3 (March 20, 2021): 410–21. http://dx.doi.org/10.1007/s11633-021-1278-z.

Full text

Abstract:

AbstractMarkov decision process (MDP) offers a general framework for modelling sequential decision making where outcomes are random. In particular, it serves as a mathematical framework for reinforcement learning. This paper introduces an extension of MDP, namely quantum MDP (qMDP), that can serve as a mathematical model of decision making about quantum systems. We develop dynamic programming algorithms for policy evaluation and finding optimal policies for qMDPs in the case of finite-horizon. The results obtained in this paper provide some useful mathematical tools for reinforcement learning techniques applied to the quantum world.

APA, Harvard, Vancouver, ISO, and other styles

11

Mármol, Amparo M., Justo Puerto, and Francisco R. Fernández. "Sequential incorporation of imprecise information in multiple criteria decision processes." European Journal of Operational Research 137, no. 1 (February 2002): 123–33. http://dx.doi.org/10.1016/s0377-2217(01)00082-0.

Full text

APA, Harvard, Vancouver, ISO, and other styles

12

PENG, Yanyan, and Xinwang LIU. "BIDDING DECISION IN LAND AUCTION USING PROSPECT THEORY." International Journal of Strategic Property Management 19, no. 2 (June 19, 2015): 186–205. http://dx.doi.org/10.3846/1648715x.2015.1047914.

Full text

Abstract:

Land auction is widely practiced in company and government decisions, especially in China. Bidders are always faced with two or more auctions in the period of a decision cycle. The outcome of the auction is under high risk. The bidder's risk attitude and preference will have a great influence on his/her bidding price. Prospect theory is currently the main descriptive theory of decision under risk. In this paper, we will consider the preferences of the decision-makers in land bidding decisions with the multi-attribute additive utility and reference point method in cumulative prospect theory. Three land auction models are proposed based on the appearance time of the land auctions. The simultaneous model uses cumulative prospect theory without considering the relationships between the auctions. The time sequential model involves the exchange auction decisions at different time with the third-generation prospect theory. The event sequential model further considers the reference point prediction in sequential land auction decisions. The three models can help the decision-makers make better bidding price decision when they are faced with several land auctions in the period of a decision cycle. A case study illustrates the processes and results of our approaches.

APA, Harvard, Vancouver, ISO, and other styles

13

Farina, Gabriele, Christian Kroer, and Tuomas Sandholm. "Online Convex Optimization for Sequential Decision Processes and Extensive-Form Games." Proceedings of the AAAI Conference on Artificial Intelligence 33 (July 17, 2019): 1917–25. http://dx.doi.org/10.1609/aaai.v33i01.33011917.

Full text

Abstract:

Regret minimization is a powerful tool for solving large-scale extensive-form games. State-of-the-art methods rely on minimizing regret locally at each decision point. In this work we derive a new framework for regret minimization on sequential decision problems and extensive-form games with general compact convex sets at each decision point and general convex losses, as opposed to prior work which has been for simplex decision points and linear losses. We call our framework laminar regret decomposition. It generalizes the CFR algorithm to this more general setting. Furthermore, our framework enables a new proof of CFR even in the known setting, which is derived from a perspective of decomposing polytope regret, thereby leading to an arguably simpler interpretation of the algorithm. Our generalization to convex compact sets and convex losses allows us to develop new algorithms for several problems: regularized sequential decision making, regularized Nash equilibria in zero-sum extensive-form games, and computing approximate extensive-form perfect equilibria. Our generalization also leads to the first regret-minimization algorithm for computing reduced-normal-form quantal response equilibria based on minimizing local regrets. Experiments show that our framework leads to algorithms that scale at a rate comparable to the fastest variants of counterfactual regret minimization for computing Nash equilibrium, and therefore our approach leads to the first algorithm for computing quantal response equilibria in extremely large games. Our algorithms for (quadratically) regularized equilibrium finding are orders of magnitude faster than the fastest algorithms for Nash equilibrium finding; this suggests regret-minimization algorithms based on decreasing regularization for Nash equilibrium finding as future work. Finally we show that our framework enables a new kind of scalable opponent exploitation approach.

APA, Harvard, Vancouver, ISO, and other styles

14

Oh-Hyun Jung. "Are Sequential Decision-Making Processes of Tourists and Consumers the Same?" Culinary Science & Hospitality Research 23, no. 6 (September 2017): 161–72. http://dx.doi.org/10.20878/cshr.2017.23.6.018.

Full text

APA, Harvard, Vancouver, ISO, and other styles

15

Chang, H. S. "A Model for Multi-timescaled Sequential Decision-making Processes with Adversary." Mathematical and Computer Modelling of Dynamical Systems 10, no. 3-4 (September 2004): 287–302. http://dx.doi.org/10.1080/13873950412331335261.

Full text

APA, Harvard, Vancouver, ISO, and other styles

16

Kneller, Wendy, Amina Memon, and Sarah Stevenage. "Simultaneous and sequential lineups: decision processes of accurate and inaccurate eyewitnesses." Applied Cognitive Psychology 15, no. 6 (2001): 659–71. http://dx.doi.org/10.1002/acp.739.

Full text

APA, Harvard, Vancouver, ISO, and other styles

17

Becker, R., S. Zilberstein, V. Lesser, and C. V. Goldman. "Solving Transition Independent Decentralized Markov Decision Processes." Journal of Artificial Intelligence Research 22 (December 1, 2004): 423–55. http://dx.doi.org/10.1613/jair.1497.

Full text

Abstract:

Formal treatment of collaborative multi-agent systems has been lagging behind the rapid progress in sequential decision making by individual agents. Recent work in the area of decentralized Markov Decision Processes (MDPs) has contributed to closing this gap, but the computational complexity of these models remains a serious obstacle. To overcome this complexity barrier, we identify a specific class of decentralized MDPs in which the agents' transitions are independent. The class consists of independent collaborating agents that are tied together through a structured global reward function that depends on all of their histories of states and actions. We present a novel algorithm for solving this class of problems and examine its properties, both as an optimal algorithm and as an anytime algorithm. To our best knowledge, this is the first algorithm to optimally solve a non-trivial subclass of decentralized MDPs. It lays the foundation for further work in this area on both exact and approximate algorithms.

APA, Harvard, Vancouver, ISO, and other styles

18

Khader, Patrick H., Thorsten Pachur, Lilian A. E. Weber, and Kerstin Jost. "Neural Signatures of Controlled and Automatic Retrieval Processes in Memory-based Decision-making." Journal of Cognitive Neuroscience 28, no. 1 (January 2016): 69–83. http://dx.doi.org/10.1162/jocn_a_00882.

Full text

Abstract:

Decision-making often requires retrieval from memory. Drawing on the neural ACT-R theory [Anderson, J. R., Fincham, J. M., Qin, Y., & Stocco, A. A central circuit of the mind. Trends in Cognitive Sciences, 12, 136–143, 2008] and other neural models of memory, we delineated the neural signatures of two fundamental retrieval aspects during decision-making: automatic and controlled activation of memory representations. To disentangle these processes, we combined a paradigm developed to examine neural correlates of selective and sequential memory retrieval in decision-making with a manipulation of associative fan (i.e., the decision options were associated with one, two, or three attributes). The results show that both the automatic activation of all attributes associated with a decision option and the controlled sequential retrieval of specific attributes can be traced in material-specific brain areas. Moreover, the two facets of memory retrieval were associated with distinct activation patterns within the frontoparietal network: The dorsolateral prefrontal cortex was found to reflect increasing retrieval effort during both automatic and controlled activation of attributes. In contrast, the superior parietal cortex only responded to controlled retrieval, arguably reflecting the sequential updating of attribute information in working memory. This dissociation in activation pattern is consistent with ACT-R and constitutes an important step toward a neural model of the retrieval dynamics involved in memory-based decision-making.

APA, Harvard, Vancouver, ISO, and other styles

19

Ortega-Gutiérrez, R. Israel, and H. Cruz-Suárez. "A Moreau-Yosida regularization for Markov decision processes." Proyecciones (Antofagasta) 40, no. 1 (February 1, 2020): 117–37. http://dx.doi.org/10.22199/issn.0717-6279-2021-01-0008.

Full text

Abstract:

This paper addresses a class of sequential optimization problems known as Markov decision processes. These kinds of processes are considered on Euclidean state and action spaces with the total expected discounted cost as the objective function. The main goal of the paper is to provide conditions to guarantee an adequate Moreau-Yosida regularization for Markov decision processes (named the original process). In this way, a new Markov decision process that conforms to the Markov control model of the original process except for the cost function induced via the Moreau-Yosida regularization is established. Compared to the original process, this new discounted Markov decision process has richer properties, such as the differentiability of its optimal value function, strictly convexity of the value function, uniqueness of optimal policy, and the optimal value function and the optimal policy of both processes, are the same. To complement the theory presented, an example is provided.

APA, Harvard, Vancouver, ISO, and other styles

20

Ortega-Gutiérrez, R. Israel, and H. Cruz-Suárez. "A Moreau-Yosida regularization for Markov decision processes." Proyecciones (Antofagasta) 40, no. 1 (February 1, 2020): 117–37. http://dx.doi.org/10.22199/issn.0717-6279-2021-01-0008.

Full text

Abstract:

This paper addresses a class of sequential optimization problems known as Markov decision processes. These kinds of processes are considered on Euclidean state and action spaces with the total expected discounted cost as the objective function. The main goal of the paper is to provide conditions to guarantee an adequate Moreau-Yosida regularization for Markov decision processes (named the original process). In this way, a new Markov decision process that conforms to the Markov control model of the original process except for the cost function induced via the Moreau-Yosida regularization is established. Compared to the original process, this new discounted Markov decision process has richer properties, such as the differentiability of its optimal value function, strictly convexity of the value function, uniqueness of optimal policy, and the optimal value function and the optimal policy of both processes, are the same. To complement the theory presented, an example is provided.

APA, Harvard, Vancouver, ISO, and other styles

21

Oscarsson, Henrik, and Maria Oskarson. "Sequential vote choice: Applying a consideration set model of heterogeneous decision processes." Electoral Studies 57 (February 2019): 275–83. http://dx.doi.org/10.1016/j.electstud.2018.08.005.

Full text

APA, Harvard, Vancouver, ISO, and other styles

22

Tump, Alan N., Timothy J. Pleskac, and Ralf H. J. M. Kurvers. "Wise or mad crowds? The cognitive mechanisms underlying information cascades." Science Advances 6, no. 29 (July 2020): eabb0266. http://dx.doi.org/10.1126/sciadv.abb0266.

Full text

Abstract:

Whether getting vaccinated, buying stocks, or crossing streets, people rarely make decisions alone. Rather, multiple people decide sequentially, setting the stage for information cascades whereby early-deciding individuals can influence others’ choices. To understand how information cascades through social systems, it is essential to capture the dynamics of the decision-making process. We introduce the social drift–diffusion model to capture these dynamics. We tested our model using a sequential choice task. The model was able to recover the dynamics of the social decision-making process, accurately capturing how individuals integrate personal and social information dynamically over time and when their decisions were timed. Our results show the importance of the interrelationships between accuracy, confidence, and response time in shaping the quality of information cascades. The model reveals the importance of capturing the dynamics of decision processes to understand how information cascades in social systems, paving the way for applications in other social systems.

APA, Harvard, Vancouver, ISO, and other styles

23

Tan, Chin Hon, and Joseph C. Hartman. "Sensitivity Analysis in Markov Decision Processes with Uncertain Reward Parameters." Journal of Applied Probability 48, no. 04 (December 2011): 954–67. http://dx.doi.org/10.1017/s002190020000855x.

Full text

Abstract:

Sequential decision problems can often be modeled as Markov decision processes. Classical solution approaches assume that the parameters of the model are known. However, model parameters are usually estimated and uncertain in practice. As a result, managers are often interested in how estimation errors affect the optimal solution. In this paper we illustrate how sensitivity analysis can be performed directly for a Markov decision process with uncertain reward parameters using the Bellman equations. In particular, we consider problems involving (i) a single stationary parameter, (ii) multiple stationary parameters, and (iii) multiple nonstationary parameters. We illustrate the applicability of this work through a capacitated stochastic lot-sizing problem.

APA, Harvard, Vancouver, ISO, and other styles

24

Tan, Chin Hon, and Joseph C. Hartman. "Sensitivity Analysis in Markov Decision Processes with Uncertain Reward Parameters." Journal of Applied Probability 48, no. 4 (December 2011): 954–67. http://dx.doi.org/10.1239/jap/1324046012.

Full text

Abstract:

Sequential decision problems can often be modeled as Markov decision processes. Classical solution approaches assume that the parameters of the model are known. However, model parameters are usually estimated and uncertain in practice. As a result, managers are often interested in how estimation errors affect the optimal solution. In this paper we illustrate how sensitivity analysis can be performed directly for a Markov decision process with uncertain reward parameters using the Bellman equations. In particular, we consider problems involving (i) a single stationary parameter, (ii) multiple stationary parameters, and (iii) multiple nonstationary parameters. We illustrate the applicability of this work through a capacitated stochastic lot-sizing problem.

APA, Harvard, Vancouver, ISO, and other styles

25

Varona, Pablo, and Mikhail I. Rabinovich. "Hierarchical dynamics of informational patterns and decision-making." Proceedings of the Royal Society B: Biological Sciences 283, no. 1832 (June 15, 2016): 20160475. http://dx.doi.org/10.1098/rspb.2016.0475.

Full text

Abstract:

Traditional studies on the interaction of cognitive functions in healthy and disordered brains have used the analyses of the connectivity of several specialized brain networks—the functional connectome. However, emerging evidence suggests that both brain networks and functional spontaneous brain-wide network communication are intrinsically dynamic. In the light of studies investigating the cooperation between different cognitive functions, we consider here the dynamics of hierarchical networks in cognitive space. We show, using an example of behavioural decision-making based on sequential episodic memory, how the description of metastable pattern dynamics underlying basic cognitive processes helps to understand and predict complex processes like sequential episodic memory recall and competition among decision strategies. The mathematical images of the discussed phenomena in the phase space of the corresponding cognitive model are hierarchical heteroclinic networks. One of the most important features of such networks is the robustness of their dynamics. Different kinds of instabilities of these dynamics can be related to ‘dynamical signatures’ of creativity and different psychiatric disorders. The suggested approach can also be useful for the understanding of the dynamical processes that are the basis of consciousness.

APA, Harvard, Vancouver, ISO, and other styles

26

Chen, Richard C., Kevin Wagner, and Gilmer L. Blankenship. "Constrained Partially Observed Markov Decision Processes With Probabilistic Criteria for Adaptive Sequential Detection." IEEE Transactions on Automatic Control 58, no. 2 (February 2013): 487–93. http://dx.doi.org/10.1109/tac.2012.2208312.

Full text

APA, Harvard, Vancouver, ISO, and other styles

27

Fejgin, Naomi, and Ronit Hanegby. "Physical Educators’ Participation in Decision-Making Processes in Dynamic Schools." Journal of Teaching in Physical Education 18, no. 2 (January 1999): 141–58. http://dx.doi.org/10.1123/jtpe.18.2.141.

Full text

Abstract:

Teacher participation in school decision-making processes is considered one of the major components of school dynamics. It is not known, however, whether all teachers participate in the process to the same extent. This study examines whether teacher participation is related to school dynamics and to subject matter taught. In a 3-step sequential model, the relative contribution of background variables, school measures, school dynamics, and subject matter taught to teacher participation was estimated. Findings showed that school dynamics had the strongest effect on teacher participation, but the effect was not the same for all teachers. Physical educators participated in school decision-making processes less than did other teachers. Physical educators in dynamic schools reported a higher degree of participation than physical educators in non-dynamic schools but a lower degree of participation compared to other teachers in dynamic schools.

APA, Harvard, Vancouver, ISO, and other styles

28

Schrift, Rom Y., Jeffrey R. Parker, Gal Zauberman, and Shalena Srna. "Multistage Decision Processes: The Impact of Attribute Order on How Consumers Mentally Represent Their Choice." Journal of Consumer Research 44, no. 6 (September 15, 2017): 1307–24. http://dx.doi.org/10.1093/jcr/ucx099.

Full text

Abstract:

Abstract With the ever-increasing number of options from which consumers can choose, many decisions are made in stages. Whether using decision tools to sort, screen, and eliminate options, or intuitively trying to reduce the complexity of a choice, consumers often reach a decision by making sequential, attribute-level choices. The current article explores how the order in which attribute-level choices are made in such multistage decisions affects how consumers mentally represent and categorize their chosen option. The authors find that attribute choices made in the initial stage play a dominant role in how the ultimately chosen option is mentally represented, while later attribute choices serve only to update and refine the representation of that option. Across 13 studies (six of which are reported in the supplemental online materials), the authors find that merely changing the order of attribute choices in multistage decision processes alters how consumers (1) describe the chosen option, (2) perceive its similarity to other available options, (3) categorize it, (4) intend to use it, and (5) replace it. Thus, while the extant decision-making literature has mainly explored how mental representations and categorization impact choice, the current article demonstrates the reverse: that the choice process itself can impact mental representations.

APA, Harvard, Vancouver, ISO, and other styles

29

Chen, Richard C., and Eugene A. Feinberg. "Compactness of the space of non-randomized policies in countable-state sequential decision processes." Mathematical Methods of Operations Research 71, no. 2 (January 10, 2010): 307–23. http://dx.doi.org/10.1007/s00186-009-0298-1.

Full text

APA, Harvard, Vancouver, ISO, and other styles

30

Tkatek, Said, Saadia Bahti, Otman Abdoun, and Jaafar Abouchabaka. "Intelligent system for recruitment decision making using an alternative parallel-sequential genetic algorithm." Indonesian Journal of Electrical Engineering and Computer Science 22, no. 1 (April 1, 2021): 385. http://dx.doi.org/10.11591/ijeecs.v22.i1.pp385-395.

Full text

Abstract:

<p>The human resources (HR) manager needs effective tools to be able to move away from traditional recruitment processes to make the good decision to select the good candidates for the good posts. To do this, we deliver an intelligent recruitment decision-making method for HR, incorporating a recruitment model based on the multipack model known as the NP-hard model. The system, which is a decision support tool, often integrates a genetic approach that operates alternately in parallel and sequentially. This approach will provide the best recruiting solution to allow HR managers to make the right decision to ensure the best possible compatibility with the desired objectives. Operationally, this system can also predict the altered choice of parallel genetic algorithm (PGA) or sequential genetic algorithm (SeqGA) depending on the size of the instance and constraints of the recruiting posts to produce the quality solution in a reduced CPU time for recruiting decision-making. The results obtained in various tests confirm the performance of this intelligent system which can be used as a decision support tool for intelligently optimized recruitment.</p>

APA, Harvard, Vancouver, ISO, and other styles

31

Brázdil, Tomáš, Krishnendu Chatterjee, Petr Novotný, and Jiří Vahala. "Reinforcement Learning of Risk-Constrained Policies in Markov Decision Processes." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 06 (April 3, 2020): 9794–801. http://dx.doi.org/10.1609/aaai.v34i06.6531.

Full text

Abstract:

Markov decision processes (MDPs) are the defacto framework for sequential decision making in the presence of stochastic uncertainty. A classical optimization criterion for MDPs is to maximize the expected discounted-sum payoff, which ignores low probability catastrophic events with highly negative impact on the system. On the other hand, risk-averse policies require the probability of undesirable events to be below a given threshold, but they do not account for optimization of the expected payoff. We consider MDPs with discounted-sum payoff with failure states which represent catastrophic outcomes. The objective of risk-constrained planning is to maximize the expected discounted-sum payoff among risk-averse policies that ensure the probability to encounter a failure state is below a desired threshold. Our main contribution is an efficient risk-constrained planning algorithm that combines UCT-like search with a predictor learned through interaction with the MDP (in the style of AlphaZero) and with a risk-constrained action selection via linear programming. We demonstrate the effectiveness of our approach with experiments on classical MDPs from the literature, including benchmarks with an order of 106 states.

APA, Harvard, Vancouver, ISO, and other styles

32

Smith, Terence R., W. A. V. Clark, and John W. Cotton. "Deriving and Testing Production System Models of Sequential Decision-Making Behavior." Geographical Analysis 16, no. 3 (September 3, 2010): 191–222. http://dx.doi.org/10.1111/j.1538-4632.1984.tb00810.x.

Full text

APA, Harvard, Vancouver, ISO, and other styles

33

Akbari, Zohreh, and Rainer Unland. "A Novel Heterogeneous Swarm Reinforcement Learning Method for Sequential Decision Making Problems." Machine Learning and Knowledge Extraction 1, no. 2 (April 16, 2019): 590–610. http://dx.doi.org/10.3390/make1020035.

Full text

Abstract:

Sequential Decision Making Problems (SDMPs) that can be modeled as Markov Decision Processes can be solved using methods that combine Dynamic Programming (DP) and Reinforcement Learning (RL). Depending on the problem scenarios and the available Decision Makers (DMs), such RL algorithms may be designed for single-agent systems or multi-agent systems that either consist of agents with individual goals and decision making capabilities, which are influenced by other agent’s decisions, or behave as a swarm of agents that collaboratively learn a single objective. Many studies have been conducted in this area; however, when concentrating on available swarm RL algorithms, one obtains a clear view of the areas that still require attention. Most of the studies in this area focus on homogeneous swarms and so far, systems introduced as Heterogeneous Swarms (HetSs) merely include very few, i.e., two or three sub-swarms of homogeneous agents, which either, according to their capabilities, deal with a specific sub-problem of the general problem or exhibit different behaviors in order to reduce the risk of bias. This study introduces a novel approach that allows agents, which are originally designed to solve different problems and hence have higher degrees of heterogeneity, to behave as a swarm when addressing identical sub-problems. In fact, the affinity between two agents, which measures the compatibility of agents to work together towards solving a specific sub-problem, is used in designing a Heterogeneous Swarm RL (HetSRL) algorithm that allows HetSs to solve the intended SDMPs.

APA, Harvard, Vancouver, ISO, and other styles

34

Ghaffari, Minou, and Susann Fiedler. "The Power of Attention: Using Eye Gaze to Predict Other-Regarding and Moral Choices." Psychological Science 29, no. 11 (October 8, 2018): 1878–89. http://dx.doi.org/10.1177/0956797618799301.

Full text

Abstract:

According to research studying the processes underlying decisions, a two-channel mechanism connects attention and choices: top-down and bottom-up processes. To identify the magnitude of each channel, we exogenously varied information intake by systematically interrupting participants’ decision processes in Study 1 ( N = 116). Results showed that participants were more likely to choose a predetermined target option. Because selection effects limited the interpretation of the results, we used a sequential-presentation paradigm in Study 2 (preregistered, N = 100). To partial out bottom-up effects of attention on choices, in particular, we presented alternatives by mirroring the gaze patterns of autonomous decision makers. Results revealed that final fixations successfully predicted choices when experimentally manipulated (bottom up). Specifically, up to 11.32% of the link between attention and choices is driven by exogenously guided attention (1.19% change in choices overall), while the remaining variance is explained by top-down preference formation.

APA, Harvard, Vancouver, ISO, and other styles

35

She, Chung, and Han. "Economic and Environmental Optimization of the Forest Supply Chain for Timber and Bioenergy Production from Beetle-Killed Forests in Northern Colorado." Forests 10, no. 8 (August 14, 2019): 689. http://dx.doi.org/10.3390/f10080689.

Full text

Abstract:

Harvesting mountain pine beetle-infested forest stands in the northern Colorado Rocky Mountains provides an opportunity to utilize otherwise wasted resources, generate net revenues, and minimize greenhouse gas (GHG) emissions. Timber and bioenergy production are commonly managed separately, and their integration is seldom considered. Yet, degraded wood and logging residues can provide a feedstock for bioenergy, while the sound wood from beetle-killed stands can still be used for traditional timber products. In addition, beneficial greenhouse gas emission (GHG) savings are often realized only by compromising net revenues during salvage harvest where beetle-killed wood has a relatively low market value and high harvesting cost. In this study we compared Sequential and Integrated decision-making scenarios for managing the supply chain from beetle-killed forest salvage operations. In the Sequential scenario, timber and bioenergy production was managed sequentially in two separate processes, where salvage harvest was conducted without considering influences on or from bioenergy production. Biomass availability was assessed next as an outcome from timber production managed to produce bioenergy products. In the Integrated scenario, timber and bioenergy production were managed jointly, where collective decisions were made regarding tree salvage harvest, residue treatment, and bioenergy product selection and production. We applied a multi-objective optimization approach to integrate the economic and environmental objectives of producing timber and bioenergy, and measured results by total net revenues and total net GHG emission savings, respectively. The optimization model results show that distinctively different decisions are made in selecting the harvesting system and residue treatment under the two scenarios. When the optimization is fully economic-oriented, 49.6% more forest areas are harvested under the Integrated scenario than the Sequential scenario, generating 12.3% more net revenues and 50.5% more net GHG emission savings. Comparison of modelled Pareto fronts also indicate the Integrated decision scenario provides more efficient trade-offs between the two objectives and performs better than the Sequential scenario in both objectives.

APA, Harvard, Vancouver, ISO, and other styles

36

Zhang, Jiaxiang, and Rafal Bogacz. "Optimal Decision Making on the Basis of Evidence Represented in Spike Trains." Neural Computation 22, no. 5 (May 2010): 1113–48. http://dx.doi.org/10.1162/neco.2009.05-09-1025.

Full text

Abstract:

Experimental data indicate that perceptual decision making involves integration of sensory evidence in certain cortical areas. Theoretical studies have proposed that the computation in neural decision circuits approximates statistically optimal decision procedures (e.g., sequential probability ratio test) that maximize the reward rate in sequential choice tasks. However, these previous studies assumed that the sensory evidence was represented by continuous values from gaussian distributions with the same variance across alternatives. In this article, we make a more realistic assumption that sensory evidence is represented in spike trains described by the Poisson processes, which naturally satisfy the mean-variance relationship observed in sensory neurons. We show that for such a representation, the neural circuits involving cortical integrators and basal ganglia can approximate the optimal decision procedures for two and multiple alternative choice tasks.

APA, Harvard, Vancouver, ISO, and other styles

37

Wang, C., S. Joshi, and R. Khardon. "First Order Decision Diagrams for Relational MDPs." Journal of Artificial Intelligence Research 31 (March 25, 2008): 431–72. http://dx.doi.org/10.1613/jair.2489.

Full text

Abstract:

Markov decision processes capture sequential decision making under uncertainty, where an agent must choose actions so as to optimize long term reward. The paper studies efficient reasoning mechanisms for Relational Markov Decision Processes (RMDP) where world states have an internal relational structure that can be naturally described in terms of objects and relations among them. Two contributions are presented. First, the paper develops First Order Decision Diagrams (FODD), a new compact representation for functions over relational structures, together with a set of operators to combine FODDs, and novel reduction techniques to keep the representation small. Second, the paper shows how FODDs can be used to develop solutions for RMDPs, where reasoning is performed at the abstract level and the resulting optimal policy is independent of domain size (number of objects) or instantiation. In particular, a variant of the value iteration algorithm is developed by using special operations over FODDs, and the algorithm is shown to converge to the optimal policy.

APA, Harvard, Vancouver, ISO, and other styles

38

Mentzas, Gregory N. "Coordination of Joint Tasks in Organizational Processes." Journal of Information Technology 8, no. 3 (September 1993): 139–50. http://dx.doi.org/10.1177/026839629300800303.

Full text

Abstract:

Organizational productivity can be maximized by creating, using and maintaining structural and dynamic configurations of multi-participant interaction. The paper highlights a number of areas for consideration that arise when studying coordination within an organizational setting. The focus of the analysis is on two types of tasks: decision-making tasks and routine office processes. The paper argues that a number of (conflicting) options exist when developing the coordination aspects of group systems; they are classified across the following axes: specification and implementation of coordination; use of synchronous and asynchronous working phases; information exchange and information sharing; support of sequential and concurrent processing; support of negotiation and conflict resolution; support of analytical modelling; and description of the organizational environment.

APA, Harvard, Vancouver, ISO, and other styles

39

Watkins, Fred, Thomas Jobe, and Cathy Helgason. "Measurable Differences between Sequential and Parallel Diagnostic Decision Processes for Determining Stroke Subtype: A Representation of Interacting Pathologies." Thrombosis and Haemostasis 88, no. 08 (2002): 210–12. http://dx.doi.org/10.1055/s-0037-1613189.

Full text

Abstract:

SummaryStroke diagnosis depends on causal subtype. The accepted classification procedure is a succession of diagnostic tests administered in an order based on prior reported frequencies of the subtypes. The first positive test result completely determines diagnosis. An alternative approach tests multiple concomitant diagnostic hypotheses in parallel. This method permits multiple simultaneous pathologies in the patient. These two diagnostic procedures can be compared by novel numeric criteria presented here.Thrombosis, a type of ischemic stroke, results from interaction between endothelium, blood flow and blood components. We tested for ischemic stroke on thirty patients using both methods. For each patient the procedure produced an assessment of severity as an ordered set of three numbers in the interval [0, 1]. We measured the difference in diagnosis between the sequential and parallel diagnostic algorithms. The computations reveal systematic differences: The sequential procedure tends to under-diagnose and excludes any measure of interaction between pathologic elements.

APA, Harvard, Vancouver, ISO, and other styles

40

Wang, Yuanbin, Robert Blache, and Xun Xu. "Selection of additive manufacturing processes." Rapid Prototyping Journal 23, no. 2 (March 20, 2017): 434–47. http://dx.doi.org/10.1108/rpj-09-2015-0123.

Full text

Abstract:

Purpose This study aims to review the existing methods for additive manufacturing (AM) process selection and evaluate their suitability for design for additive manufacturing (DfAM). AM has experienced a rapid development in recent years. New technologies, machines and service bureaus are being brought into the market at an exciting rate. While user’s choices are in abundance, finding the right choice can be a non-trivial task. Design/methodology/approach AM process selection methods are reviewed based on decision theory. The authors also examine how the user’s preferences and AM process performances are considered and approximated into mathematical models. The pros and cons and the limitations of these methods are discussed, and a new approach has been proposed to support the iterating process of DfAM. Findings All current studies follow a sequential decision process and focus on an “a priori” articulation of preferences approach. This kind of method has limitations for the user in the early design stage to implement the DfAM process. An “a posteriori” articulation of preferences approach is proposed to support DfAM and an iterative design process. Originality/value This paper reviews AM process selection methods in a new perspective. The users need to be aware of the underlying assumptions in these methods. The limitations of these methods for DfAM are discussed, and a new approach for AM process selection is proposed.

APA, Harvard, Vancouver, ISO, and other styles

41

Plesca, Cezar, Vincent Charvillat, and Romulus Grigoras. "Adapting Content Delivery to Limited Resources and Inferred User Interest." International Journal of Digital Multimedia Broadcasting 2008 (2008): 1–13. http://dx.doi.org/10.1155/2008/171385.

Full text

Abstract:

This paper discusses adaptation policies for information systems that are subject to dynamic and stochastic contexts such as mobile access to multimedia web sites. In our approach, adaptation agents apply sequential decisional policies under uncertainty. We focus on the modeling of such decisional processes depending on whether the context is fully or partially observable. Our case study is a movie browsing service in a mobile environment that we model by using Markov decision processes (MDPs) and partially observable MDP (POMDP). We derive adaptation policies for this service, that take into account the limited resources such as the network bandwidth. We further refine these policies according to the partially observable users' interest level estimated from implicit feedback. Our theoretical models are validated through numerous simulations.

APA, Harvard, Vancouver, ISO, and other styles

42

Kong, Zhenyu, Omer Beyca, Satish T. Bukkapatnam, and Ranga Komanduri. "Nonlinear Sequential Bayesian Analysis-Based Decision Making for End-Point Detection of Chemical Mechanical Planarization (CMP) Processes." IEEE Transactions on Semiconductor Manufacturing 24, no. 4 (November 2011): 523–32. http://dx.doi.org/10.1109/tsm.2011.2164100.

Full text

APA, Harvard, Vancouver, ISO, and other styles

43

Farrell, Henry, and Adrienne HÉRitier. "Interorganizational Negotiation and Intraorganizational Power in Shared Decision Making." Comparative Political Studies 37, no. 10 (December 2004): 1184–212. http://dx.doi.org/10.1177/0010414004269833.

Full text

Abstract:

The authors argue that closer attention should be paid to the interorganizational rules of decision making and their implications for intraorganizational processes. They claim that exogenous changes in macro-institutional rules, which result in a move from formal and sequential to informal and simultaneous interaction between collective actors, will lead to changes in individual actors’ respective influence over outcomes within organizations. Certain individuals controlling information flows between organizations will see an increase in their power over legislative outcomes. This begs the question of how organizations will respond to these shifts in the power balance among the individual actors that constitute them. The authors argue that collective actors that centralize coordination over dealings with external actors will respond effectively through internal rule change. In contrast, collective actors with multiple, ill-coordinated links to other organizations will find it difficult to change internal rules. The authors empirically explore the general argument by analyzing the relationship between the Council and the European Parliament in the process of codecision and its implications for intraorganizational processes.

APA, Harvard, Vancouver, ISO, and other styles

44

Wang, Yongheng, Xiaozan Zhang, and Zengwang Wang. "A Proactive Decision Support System for Online Event Streams." International Journal of Information Technology & Decision Making 17, no. 06 (November 2018): 1891–913. http://dx.doi.org/10.1142/s0219622018500463.

Full text

Abstract:

In-stream big data processing is an important part of big data processing. Proactive decision support systems can predict future system states and execute some actions to avoid unwanted states. In this paper, we propose a proactive decision support system for online event streams. Based on Complex Event Processing (CEP) technology, this method uses structure varying dynamic Bayesian network to predict future events and system states. Different Bayesian network structures are learned and used according to different event context. A networked distributed Markov decision processes model with predicting states is proposed as sequential decision making model. A Q-learning method is investigated for this model to find optimal joint policy. The experimental evaluations show that this method works well for congestion control in transportation system.

APA, Harvard, Vancouver, ISO, and other styles

45

Mochizuki, Kei, and Shintaro Funahashi. "Opposing history effect of preceding decision and action in the free choice of saccade direction." Journal of Neurophysiology 112, no. 4 (August 15, 2014): 923–32. http://dx.doi.org/10.1152/jn.00846.2013.

Full text

Abstract:

When we act voluntarily, we make a decision to do so prior to the actual execution. However, because of the strong tie between decision and action, it has been difficult to dissociate these two processes in an animal's free behavior. In the present study, we tried to characterize the differences in these processes on the basis of their unique history effect. Using simple eye movement tasks in which the direction of a saccade was either instructed by a computer or freely chosen by the subject, we found that the preceding decision and action had different effects on the animal's subsequent behavior. While choosing a direction (previous decision) produced a positive history effect that prompted the choice of the same saccade direction, making a saccadic response to a direction (previous action) produced a negative history effect that discouraged the monkey from choosing the same direction. This result suggests that the history effect in sequential behavior reported in previous studies was a mixture of these two different components. Future studies on decision-making need to consider the importance of the distinction between decision and action in animal behavior.

APA, Harvard, Vancouver, ISO, and other styles

46

Cheah, Jun-Hwa, Hiram Ting, Tat Huei Cham, and Mumtaz Ali Memon. "The effect of selfie promotion and celebrity endorsed advertisement on decision-making processes." Internet Research 29, no. 3 (June 3, 2019): 552–77. http://dx.doi.org/10.1108/intr-12-2017-0530.

Full text

Abstract:

Purpose The purpose of this paper is to assess the effect of two promotional methods, namely, celebrity endorsed advertisement and selfie promotion, on customers’ decision-making processes using the AISAS model. Design/methodology/approach A within-subject experimental design was used to observe how young adults in Malaysia would respond to two promotional methods about a new seafood restaurant. A total of 180 responses were collected using a structured questionnaire. Data were assessed and analysed using partial least squares structural equation modelling. Findings The results show that while celebrity endorsed advertisement remains relevant to customer’s decision-making processes, the effect of selfie promotion is comparable to celebrity endorsement. The sequential mediation for both models is found to be significant, but the AISAS model with selfie promotion produces better in-sample prediction (model selection criteria) and out-of-sample prediction (PLSpredict) compared to celebrity endorsed advertisement, thus suggesting its better representation to reality. Research limitations/implications Despite being limited to young adults in Malaysia and a particular product, the study is essential to understanding the effect of celebrity endorsed advertisement and selfie promotion on decision-making processes. Practical implications The study provides insights into how business organisations could exploit the advancement of communication technology to encourage selfie behaviour to promote their products in an innovative and competitive manner. Originality/value The assessment of the effect of celebrity endorsed advertisement and selfie promotion on decision-making processes using PLSpredict and model selection criteria articulates the relevance of selfie as a promotional tool. It also provides an alternative technique for conducting model comparison research.

APA, Harvard, Vancouver, ISO, and other styles

47

Coelho, Carlos, Pedro Narra, Bárbara Marinho, and Márcia Lima. "Coastal Management Software to Support the Decision-Makers to Mitigate Coastal Erosion." Journal of Marine Science and Engineering 8, no. 1 (January 11, 2020): 37. http://dx.doi.org/10.3390/jmse8010037.

Full text

Abstract:

There are no sequential and integrated approaches that include the steps needed to perform an adequate management and planning of the coastal zones to mitigate coastal erosion problems and climate change effects. Important numerical model packs are available for users, but often looking deeply to the physical processes, demanding big computational efforts and focusing on specific problems. Thus, it is important to provide adequate tools to the decision-makers, which can be easily interpreted by populations, promoting discussions of optimal intervention scenarios in medium to long-term horizons. COMASO (coastal management software) intends to fill this gap, presenting a group of tools that can be applied in standalone mode, or in a sequential order. The first tool should map the coastal erosion vulnerability and risk, also including the climate change effects, defining a hierarchy of priorities where coastal defense interventions should be performed, or limiting/constraining some land uses or activities. In the locations identified as priorities, a more detailed analysis should consider the application of shoreline and cross-shore evolution models (second tool), allowing discussing intervention scenarios, in medium to long-term horizons. After the defined scenarios, the design of the intervention should be discussed, both in case of being a hard coastal structure or an artificial nourishment (third type of tools). Finally, a cost-benefit assessment tool should optimize the decisions, forecasting costs and benefits for each different scenario, through definition of economic values to the interventions and to the land/services/ecosystems, weighting all the environmental, cultural, social and historical aspects. It is considered that COMASO tools can help giving answers to the major problems of the coastal planning and management entities, integrating transversal knowledge in risk assessment, physical processes, engineering and economic evaluations. The integrated coastal zone management needs these tools to ensure sustainable coastal zones, mitigating erosion and climate change effects.

APA, Harvard, Vancouver, ISO, and other styles

48

Gmytrasiewicz, P. J., and P. Doshi. "A Framework for Sequential Planning in Multi-Agent Settings." Journal of Artificial Intelligence Research 24 (July 1, 2005): 49–79. http://dx.doi.org/10.1613/jair.1579.

Full text

Abstract:

This paper extends the framework of partially observable Markov decision processes (POMDPs) to multi-agent settings by incorporating the notion of agent models into the state space. Agents maintain beliefs over physical states of the environment and over models of other agents, and they use Bayesian updates to maintain their beliefs over time. The solutions map belief states to actions. Models of other agents may include their belief states and are related to agent types considered in games of incomplete information. We express the agents' autonomy by postulating that their models are not directly manipulable or observable by other agents. We show that important properties of POMDPs, such as convergence of value iteration, the rate of convergence, and piece-wise linearity and convexity of the value functions carry over to our framework. Our approach complements a more traditional approach to interactive settings which uses Nash equilibria as a solution paradigm. We seek to avoid some of the drawbacks of equilibria which may be non-unique and do not capture off-equilibrium behaviors. We do so at the cost of having to represent, process and continuously revise models of other agents. Since the agent's beliefs may be arbitrarily nested, the optimal solutions to decision making problems are only asymptotically computable. However, approximate belief updates and approximately optimal plans are computable. We illustrate our framework using a simple application domain, and we show examples of belief updates and value functions.

APA, Harvard, Vancouver, ISO, and other styles

49

Kumar, Akshat, Shlomo Zilberstein, and Marc Toussaint. "Probabilistic Inference Techniques for Scalable Multiagent Decision Making." Journal of Artificial Intelligence Research 53 (June 29, 2015): 223–70. http://dx.doi.org/10.1613/jair.4649.

Full text

Abstract:

Decentralized POMDPs provide an expressive framework for multiagent sequential decision making. However, the complexity of these models---NEXP-Complete even for two agents---has limited their scalability. We present a promising new class of approximation algorithms by developing novel connections between multiagent planning and machine learning. We show how the multiagent planning problem can be reformulated as inference in a mixture of dynamic Bayesian networks (DBNs). This planning-as-inference approach paves the way for the application of efficient inference techniques in DBNs to multiagent decision making. To further improve scalability, we identify certain conditions that are sufficient to extend the approach to multiagent systems with dozens of agents. Specifically, we show that the necessary inference within the expectation-maximization framework can be decomposed into processes that often involve a small subset of agents, thereby facilitating scalability. We further show that a number of existing multiagent planning models satisfy these conditions. Experiments on large planning benchmarks confirm the benefits of our approach in terms of runtime and scalability with respect to existing techniques.

APA, Harvard, Vancouver, ISO, and other styles

50

Kool, Wouter, Samuel J. Gershman, and Fiery A. Cushman. "Planning Complexity Registers as a Cost in Metacontrol." Journal of Cognitive Neuroscience 30, no. 10 (October 2018): 1391–404. http://dx.doi.org/10.1162/jocn_a_01263.

Full text

Abstract:

Decision-making algorithms face a basic tradeoff between accuracy and effort (i.e., computational demands). It is widely agreed that humans can choose between multiple decision-making processes that embody different solutions to this tradeoff: Some are computationally cheap but inaccurate, whereas others are computationally expensive but accurate. Recent progress in understanding this tradeoff has been catalyzed by formalizing it in terms of model-free (i.e., habitual) versus model-based (i.e., planning) approaches to reinforcement learning. Intuitively, if two tasks offer the same rewards for accuracy but one of them is much more demanding, we might expect people to rely on habit more in the difficult task: Devoting significant computation to achieve slight marginal accuracy gains would not be “worth it.” We test and verify this prediction in a sequential reinforcement learning task. Because our paradigm is amenable to formal analysis, it contributes to the development of a computational model of how people balance the costs and benefits of different decision-making processes in a task-specific manner; in other words, how we decide when hard thinking is worth it.

APA, Harvard, Vancouver, ISO, and other styles

Journal articles on the topic 'Sequential decision processes'

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles