Log in

Relevant bibliographies by topics / Markov decision theory / Journal articles

Journal articles on the topic 'Markov decision theory'

To see the other types of publications on this topic, follow the link: Markov decision theory.

Author: Grafiati

Published: 10 December 2022

Last updated: 28 January 2023

Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles

Select a source type:

Consult the top 50 journal articles for your research on the topic 'Markov decision theory.'

Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.

You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.

Browse journal articles on a wide variety of disciplines and organise your bibliography correctly.

1

Weng, Paul, and Olivier Spanjaard. "Functional Reward Markov Decision Processes: Theory and Applications." International Journal on Artificial Intelligence Tools 26, no. 03 (June 2017): 1760014. http://dx.doi.org/10.1142/s0218213017600144.

Full text

Abstract:

Markov decision processes (MDP) have become one of the standard models for decisiontheoretic planning problems under uncertainty. In its standard form, rewards are assumed to be numerical additive scalars. In this paper, we propose a generalization of this model allowing rewards to be functional. The value of a history is recursively computed by composing the reward functions. We show that several variants of MDPs presented in the literature can be instantiated in this setting. We then identify sufficient conditions on these reward functions for dynamic programming to be valid. We also discuss the infinite horizon case and the case where a maximum operator does not exist. In order to show the potential of our framework, we conclude the paper by presenting several illustrative examples.

APA, Harvard, Vancouver, ISO, and other styles

2

Buchholz, Peter. "Bounding reward measures of Markov models using the Markov decision processes." Numerical Linear Algebra with Applications 18, no. 6 (October 18, 2011): 919–30. http://dx.doi.org/10.1002/nla.792.

Full text

APA, Harvard, Vancouver, ISO, and other styles

3

Ortega-Gutiérrez, R. Israel, and H. Cruz-Suárez. "A Moreau-Yosida regularization for Markov decision processes." Proyecciones (Antofagasta) 40, no. 1 (February 1, 2020): 117–37. http://dx.doi.org/10.22199/issn.0717-6279-2021-01-0008.

Full text

Abstract:

This paper addresses a class of sequential optimization problems known as Markov decision processes. These kinds of processes are considered on Euclidean state and action spaces with the total expected discounted cost as the objective function. The main goal of the paper is to provide conditions to guarantee an adequate Moreau-Yosida regularization for Markov decision processes (named the original process). In this way, a new Markov decision process that conforms to the Markov control model of the original process except for the cost function induced via the Moreau-Yosida regularization is established. Compared to the original process, this new discounted Markov decision process has richer properties, such as the differentiability of its optimal value function, strictly convexity of the value function, uniqueness of optimal policy, and the optimal value function and the optimal policy of both processes, are the same. To complement the theory presented, an example is provided.

APA, Harvard, Vancouver, ISO, and other styles

4

Ortega-Gutiérrez, R. Israel, and H. Cruz-Suárez. "A Moreau-Yosida regularization for Markov decision processes." Proyecciones (Antofagasta) 40, no. 1 (February 1, 2020): 117–37. http://dx.doi.org/10.22199/issn.0717-6279-2021-01-0008.

Full text

Abstract:

This paper addresses a class of sequential optimization problems known as Markov decision processes. These kinds of processes are considered on Euclidean state and action spaces with the total expected discounted cost as the objective function. The main goal of the paper is to provide conditions to guarantee an adequate Moreau-Yosida regularization for Markov decision processes (named the original process). In this way, a new Markov decision process that conforms to the Markov control model of the original process except for the cost function induced via the Moreau-Yosida regularization is established. Compared to the original process, this new discounted Markov decision process has richer properties, such as the differentiability of its optimal value function, strictly convexity of the value function, uniqueness of optimal policy, and the optimal value function and the optimal policy of both processes, are the same. To complement the theory presented, an example is provided.

APA, Harvard, Vancouver, ISO, and other styles

5

Koole, Ger. "Monotonicity in Markov Reward and Decision Chains: Theory and Applications." Foundations and Trends® in Stochastic Systems 1, no. 1 (2006): 1–76. http://dx.doi.org/10.1561/0900000002.

Full text

APA, Harvard, Vancouver, ISO, and other styles

6

Cai, Lin. "Research of Optimizing Computer Network Based on Dynamism Theory." Applied Mechanics and Materials 556-562 (May 2014): 5356–58. http://dx.doi.org/10.4028/www.scientific.net/amm.556-562.5356.

Full text

Abstract:

With the fast development of computer network services, the needs of every service is developing. To guarantee a effective computer network service we should improve the key quality of service (QoS) of each computer network service. Related QoS evaluating indexes mainly include throughput, delay time and packet loss of network. A optimizing method offers a theory support to a better computer network service quality and dynamic optimization is one of the most popular theoretical tools to study resource allocation and task scheduling problems in computer systems and computer networks. This text mainly introduces Markov decision process and Markov decision Petri nets.

APA, Harvard, Vancouver, ISO, and other styles

7

Brázdil, Tomáš, Václav Brožek, Vojtěch Forejt, and Antonín Kučera. "Reachability in recursive Markov decision processes." Information and Computation 206, no. 5 (May 2008): 520–37. http://dx.doi.org/10.1016/j.ic.2007.09.002.

Full text

APA, Harvard, Vancouver, ISO, and other styles

8

Barker, Richard J., and Matthew R. Schofield. "Putting Markov Chains Back into Markov Chain Monte Carlo." Journal of Applied Mathematics and Decision Sciences 2007 (October 30, 2007): 1–13. http://dx.doi.org/10.1155/2007/98086.

Full text

Abstract:

Markov chain theory plays an important role in statistical inference both in the formulation of models for data and in the construction of efficient algorithms for inference. The use of Markov chains in modeling data has a long history, however the use of Markov chain theory in developing algorithms for statistical inference has only become popular recently. Using mark-recapture models as an illustration, we show how Markov chains can be used for developing demographic models and also in developing efficient algorithms for inference. We anticipate that a major area of future research involving mark-recapture data will be the development of hierarchical models that lead to better demographic models that account for all uncertainties in the analysis. A key issue is determining when the chains produced by Markov chain Monte Carlo sampling have converged.

APA, Harvard, Vancouver, ISO, and other styles

9

Barkalov, S. A., A. V. Ananiev, K. S. Ivannikov, and S. I. Moiseev. "Algorithm and methods for management decision-making based on the theory of latent variables under time conditions." Bulletin of the South Ural State University. Ser. Computer Technologies, Automatic Control & Radioelectronics 22, no. 3 (2022): 106–16. http://dx.doi.org/10.14529/ctcr220310.

Full text

Abstract:

The paper describes a decision-making algorithm under conditions of dynamically changing external and internal factors that affect the result of decision-making. The main role in this algorithm is played by the stage of assessing the attractiveness of alternatives and, as a result, choosing the most attractive of them. The main problem for solving this problem is the choice of the best alternative under conditions of uncertainty, when there is no information about possible scenarios for the development of the situation. To solve this problem, the paper proposes to apply the Rasch model of estimating latent variables, which allows not only to obtain estimates of alternatives on a linear scale, but also to obtain estimates of the sensitivity of alternatives to possible scenarios that make sense of the risk of loss for each alternative in case of an unexpected change in the scenario of the development of the situation. The results of modeling the process of data refinement for decision-making based on non-stationary Markov random processes are also presented. Aim. The purpose of the study is to develop an algorithm and a dynamic decision-making model that takes into account situational risk management, which is based on the theory of latent variables and non-stationary Markov random processes. Materials and methods. To substantiate the algorithm of the decision-making process, there is a mathematical model for assessing the attractiveness of alternatives, based on the Rasch model for assessing latent variables. To substantiate the evaluation results, computational experiments were carried out, which substantiated the adequacy of the obtained estimates. In addition, to confirm the possibility of using a decision-making algorithm, which involves the use of iterations to collect information, mathematical modeling of the decision-making process was carried out, which showed a high probability of successful completion of the process within the specified time frame. Results. On the basis of the decision-making algorithm presented in the paper under conditions of limited time, the issue of calculating the weights of alternatives under conditions of uncertainty, taking into account dynamically changing external and internal conditions, is raised. An original method for obtaining estimates of the attractiveness of alternatives under uncertainty is described, which takes into account the sensitivity of alternatives to possible scenarios for the development of the situation. For a qualitative analysis of the results when making decisions in accordance with the developed algorithm, a model of dynamic control over the probability of making decisions within a given time frame was presented. Using non-stationary Markov random processes, it is possible to evaluate the success of decision-making in the allotted time and calculate the number of cycles required to refine the initial data. Conclusion. On the basis of the decision-making algorithm presented in the paper under conditions of limited time, the issue of calculating the weights of alternatives under conditions of uncertainty, taking into account dynamically changing external and internal conditions, is raised. An original method for obtaining estimates of the attractiveness of alternatives under uncertainty is described, which takes into account the sensitivity of alternatives to possible scenarios for the development of the situation. For a qualitative analysis of the results when making decisions in accordance with the developed algorithm, a model of dynamic control over the probability of making decisions within a given time frame was presented. Using non-stationary Markov random processes, it is possible to evaluate the success of decision-making in the allotted time and calculate the number of cycles required to refine the initial data.

APA, Harvard, Vancouver, ISO, and other styles

10

Kadota, Yoshinobu, Masami Kurano, and Masami Yasuda. "Discounted Markov decision processes with utility constraints." Computers & Mathematics with Applications 51, no. 2 (January 2006): 279–84. http://dx.doi.org/10.1016/j.camwa.2005.11.013.

Full text

APA, Harvard, Vancouver, ISO, and other styles

11

Hasteer, Nitasha, Abhay Bansal, and B. K. Murthy. "Crowdsourced Software Development Process: Investigation and Modeling through Markov Decision Theory." International Journal of Software Engineering and Its Applications 9, no. 9 (September 30, 2015): 41–54. http://dx.doi.org/10.14257/ijseia.2015.9.9.05.

Full text

APA, Harvard, Vancouver, ISO, and other styles

12

Usaha, W., and J. Barria. "Markov decision theory framework for resource allocation in LEO satellite constellations." IEE Proceedings - Communications 149, no. 5 (December 1, 2002): 270–76. http://dx.doi.org/10.1049/ip-com:20020510.

Full text

APA, Harvard, Vancouver, ISO, and other styles

13

Rummukainen, H., and J. Virtamo. "Polynomial cost approximations in Markov decision theory based call admission control." IEEE/ACM Transactions on Networking 9, no. 6 (2001): 769–79. http://dx.doi.org/10.1109/90.974530.

Full text

APA, Harvard, Vancouver, ISO, and other styles

14

Hinz, Juri. "An Algorithm for Making Regime-Changing Markov Decisions." Algorithms 14, no. 10 (October 4, 2021): 291. http://dx.doi.org/10.3390/a14100291.

Full text

Abstract:

In industrial applications, the processes of optimal sequential decision making are naturally formulated and optimized within a standard setting of Markov decision theory. In practice, however, decisions must be made under incomplete and uncertain information about parameters and transition probabilities. This situation occurs when a system may suffer a regime switch changing not only the transition probabilities but also the control costs. After such an event, the effect of the actions may turn to the opposite, meaning that all strategies must be revised. Due to practical importance of this problem, a variety of methods has been suggested, ranging from incorporating regime switches into Markov dynamics to numerous concepts addressing model uncertainty. In this work, we suggest a pragmatic and practical approach using a natural re-formulation of this problem as a so-called convex switching system, we make efficient numerical algorithms applicable.

APA, Harvard, Vancouver, ISO, and other styles

15

Ying, Shenggang, and Mingsheng Ying. "Reachability analysis of quantum Markov decision processes." Information and Computation 263 (December 2018): 31–51. http://dx.doi.org/10.1016/j.ic.2018.09.001.

Full text

APA, Harvard, Vancouver, ISO, and other styles

16

Doyen, Laurent, Thierry Massart, and Mahsa Shirmohammadi. "The complexity of synchronizing Markov decision processes." Journal of Computer and System Sciences 100 (March 2019): 96–129. http://dx.doi.org/10.1016/j.jcss.2018.09.004.

Full text

APA, Harvard, Vancouver, ISO, and other styles

17

Guerreiro, Sérgio. "Using Markov Theory to Deliver Informed Decisions in Partially Observable Business Processes Operation." International Journal of Operations Research and Information Systems 9, no. 2 (April 2018): 53–72. http://dx.doi.org/10.4018/ijoris.2018040103.

Full text

Abstract:

This article explores the stochastic capabilities offered by Markov theories combined with business transaction models, from the Enterprise Engineering field, to contribute to the decision-making body of knowledge. An agro-food case study shows the utility of this solution and the evaluation argues the management decisions value in situations where is not possible to fully observe the state of the reality, or to be fully aware about it. A full policy graph that forecasts the belief states from observations and enacted actions is delivered.

APA, Harvard, Vancouver, ISO, and other styles

18

Pazis, Jason, and Ronald Parr. "PAC Optimal Exploration in Continuous Space Markov Decision Processes." Proceedings of the AAAI Conference on Artificial Intelligence 27, no. 1 (June 30, 2013): 774–81. http://dx.doi.org/10.1609/aaai.v27i1.8678.

Full text

Abstract:

Current exploration algorithms can be classified in two broad categories: Heuristic, and PAC optimal. While numerous researchers have used heuristic approaches such as epsilon-greedy exploration successfully, such approaches lack formal, finite sample guarantees and may need a significant amount of fine-tuning to produce good results. PAC optimal exploration algorithms, on the other hand, offer strong theoretical guarantees but are inapplicable in domains of realistic size. The goal of this paper is to bridge the gap between theory and practice, by introducing C-PACE, an algorithm which offers strong theoretical guarantees and can be applied to interesting, continuous space problems.

APA, Harvard, Vancouver, ISO, and other styles

19

Van Dijk, Nico M., and Martin L. Puterman. "Perturbation theory for Markov reward processes with applications to queueing systems." Advances in Applied Probability 20, no. 1 (March 1988): 79–98. http://dx.doi.org/10.2307/1427271.

Full text

Abstract:

We study the effect of perturbations in the data of a discrete-time Markov reward process on the finite-horizon total expected reward, the infinite-horizon expected discounted and average reward and the total expected reward up to a first-passage time. Bounds for the absolute errors of these reward functions are obtained. The results are illustrated for a finite as well as infinite queueing systems (M/M/1/S and ). Extensions to Markov decision processes and other settings are discussed.

APA, Harvard, Vancouver, ISO, and other styles

20

Van Dijk, Nico M., and Martin L. Puterman. "Perturbation theory for Markov reward processes with applications to queueing systems." Advances in Applied Probability 20, no. 01 (March 1988): 79–98. http://dx.doi.org/10.1017/s000186780001795x.

Full text

Abstract:

We study the effect of perturbations in the data of a discrete-time Markov reward process on the finite-horizon total expected reward, the infinite-horizon expected discounted and average reward and the total expected reward up to a first-passage time. Bounds for the absolute errors of these reward functions are obtained. The results are illustrated for a finite as well as infinite queueing systems (M/M/1/S and ). Extensions to Markov decision processes and other settings are discussed.

APA, Harvard, Vancouver, ISO, and other styles

21

Jain, Rahul, and Pravin Varaiya. "Simulation-based optimization of Markov decision processes: An empirical process theory approach." Automatica 46, no. 8 (August 2010): 1297–304. http://dx.doi.org/10.1016/j.automatica.2010.05.021.

Full text

APA, Harvard, Vancouver, ISO, and other styles

22

Pothos, Emmanuel M., and Jerome R. Busemeyer. "A quantum probability explanation for violations of ‘rational’ decision theory." Proceedings of the Royal Society B: Biological Sciences 276, no. 1665 (March 25, 2009): 2171–78. http://dx.doi.org/10.1098/rspb.2009.0121.

Full text

Abstract:

Two experimental tasks in psychology, the two-stage gambling game and the Prisoner's Dilemma game, show that people violate the sure thing principle of decision theory. These paradoxical findings have resisted explanation by classical decision theory for over a decade. A quantum probability model, based on a Hilbert space representation and Schrödinger's equation, provides a simple and elegant explanation for this behaviour. The quantum model is compared with an equivalent Markov model and it is shown that the latter is unable to account for violations of the sure thing principle. Accordingly, it is argued that quantum probability provides a better framework for modelling human decision-making.

APA, Harvard, Vancouver, ISO, and other styles

23

Sennott, Linn I. "Average Cost Semi-Markov Decision Processes and the Control of Queueing Systems." Probability in the Engineering and Informational Sciences 3, no. 2 (April 1989): 247–72. http://dx.doi.org/10.1017/s0269964800001121.

Full text

Abstract:

Semi-Markov decision processes underlie the control of many queueing systems. In this paper, we deal with infinite state semi-Markov decision processes with nonnegative, unbounded costs and finite action sets. Axioms for the existence of an expected average cost optimal stationary policy are presented. These conditions generalize the work in Sennott [22] for Markov decision processes. Verifiable conditions for the axioms to hold are obtained. The theory is applied to control of the M/G/l queue with variable service parameter, with on-off server, and with batch processing, and to control of the G/M/m queue with variable arrival parameter and customer rejection. It is applied to a timesharing network of queues with a single server and finally to optimal routing of Poisson arrivals to parallel exponential servers. The final section extends the existence result to compact action spaces.

APA, Harvard, Vancouver, ISO, and other styles

24

Leder, Nicole, Bernd Heidergott, and Arie Hordijk. "An Approximation Approach for the Deviation Matrix of Continuous-Time Markov Processes with Application to Markov Decision Theory." Operations Research 58, no. 4-part-1 (August 2010): 918–32. http://dx.doi.org/10.1287/opre.1090.0786.

Full text

APA, Harvard, Vancouver, ISO, and other styles

25

Flesch, János, Arkadi Predtetchinski, and Eilon Solan. "Sporadic Overtaking Optimality in Markov Decision Problems." Dynamic Games and Applications 7, no. 2 (April 13, 2016): 212–28. http://dx.doi.org/10.1007/s13235-016-0186-2.

Full text

APA, Harvard, Vancouver, ISO, and other styles

26

Chen, Cai Ping, Qiao Jing Liu, and Pan Zheng. "Application of Grey-Markov Model in Predicting Container Throughput of Fujian Province." Advanced Materials Research 779-780 (September 2013): 720–23. http://dx.doi.org/10.4028/www.scientific.net/amr.779-780.720.

Full text

Abstract:

Port container throughput forecast is an important work for wharf project which has further influence on the development of ports. In this paper, we simply introduce the grey system theory and the grey markov model. Further more, we make a forecast for the container throughput by using grey markov model and the result can offer the decision-making bases for relevant department.

APA, Harvard, Vancouver, ISO, and other styles

27

Guerreiro, Sérgio. "(Re)Designing Business Processes Using Markov Theory and Constrained State, Transition and Actor Role Spaces." International Journal of Knowledge-Based Organizations 9, no. 2 (April 2019): 43–61. http://dx.doi.org/10.4018/ijkbo.2019040103.

Full text

Abstract:

Decision-making processes are the utmost important to steer the organizational change whenever business process workarounds are attempted during operational times. However, to decide the non-compliant situations, e.g., bypasses, social resistance, or collusion; the business manager demands contextualized and correct interpretations of the existing business process redesign options to cope with workarounds. This article explores the need to aid the decision-making process with a full constructional perspective to optimize the business processes redesign. So, the Markov decision process is combined with the body of knowledge of business processes, in specific, the concepts of designing enterprise-wide business transactions. This methodology supports the management initiatives with more knowledge about the value of business processes redesign. A classical chain of Order-to-Cash business processes (the order, the production, the distribution and the selling of goods) illustrate the benefits of this quantitative approach. Results obtained for business processes redesign in reaction to workarounds are reported. The analysis results show that this approach can anticipate the sub-optimal solutions before taking actions and highlights the impact of discount factors in the final obtained value. The contribution of this novel conceptual integration to the business processes community is the forecast of value function of business transaction redesign options when facing non-compliant workarounds. From related literature, business processes compliance usually comprises offline computation and the redesign is only considered in the forthcoming business processes instances. This article is innovative in the sense that it anticipates the value impact of a redesign, allowing more effective decisions to be taken.

APA, Harvard, Vancouver, ISO, and other styles

28

Salari, Nooshin, and Viliam Makis. "Application of Markov renewal theory and semi‐Markov decision processes in maintenance modeling and optimization of multi‐unit systems." Naval Research Logistics (NRL) 67, no. 7 (August 3, 2020): 548–58. http://dx.doi.org/10.1002/nav.21932.

Full text

APA, Harvard, Vancouver, ISO, and other styles

29

Morcous, George, and Zoubir Lounis. "Integration of stochastic deterioration models with multicriteria decision theory for optimizing maintenance of bridge decks." Canadian Journal of Civil Engineering 33, no. 6 (June 1, 2006): 756–65. http://dx.doi.org/10.1139/l06-011.

Full text

Abstract:

This paper presents a new approach to optimizing the maintenance of concrete bridge decks. This approach combines a stochastic deterioration model and a multiobjective optimization model. The stochastic deterioration model is based on the first-order Markov chain, which predicts the probabilistic time variation of bridge deck conditions. The multiobjective optimization model takes into account two important and conflicting criteria: the minimization of maintenance costs and the maximization of the network condition. This approach achieves the best compromise between these competing criteria while considering the uncertainty in bridge deck deterioration. The feasibility and capability of the proposed approach are demonstrated with field data for a sample network of bridge decks obtained from the Ministère des Transports du Québec database. This example illustrates the effectiveness of the proposed approach in determining the optimal set of maintenance alternatives for reinforced concrete bridge decks when two or more relevant optimization criteria are taken into consideration.Key words: concrete bridge deck, maintenance management, multicriteria optimization, Markov chain, deterioration model.

APA, Harvard, Vancouver, ISO, and other styles

30

Mikhalov, Oleksandr Illich, Oleksandr Afrykanovych Stenin, Viktor Petrovych Pasko, Oleksandr Serhiiovych Stenin, and Yurii Opanasovych Tymoshyn. "Situational planning and operational adjustment of the route of the Autonomous robotic underwater vehicle." System technologies 3, no. 122 (October 10, 2019): 3–11. http://dx.doi.org/10.34185/1562-9945-3-122-2019-01.

Full text

Abstract:

Currently, missions (tasks) for the underwater robot formed using imperative programming methods (both text and graphic), describing in detail the sequence of robot actions that need performed to achieve the desired goal. At the same time, only the operator of the underwater robot, which makes up the mission, for example, the delivery of cargo to the target point, has an idea of the goal itself. Such technology is effective if the robot's mission carried out within a priori scenario. In other cases, it can either not be executed at all, or it can be executed with large violations and a threat to the safety of the device.When assessing the effectiveness of an underwater robot, the degree of its information autonomy, i.e. the ability to act independently in an unknown or insufficiently defined environment, is of fundamental importance. Therefore, the "intellectualization" of the Autonomous control system of the underwater robot is extremely important for the mission in unforeseen circumstances. For this propose to use intelligent decision support system. Two ways to implement optimal decision-making strategies based on the mathematical apparatus of the theory of Markov and semi-Markov processes using the Bellman optimality principle propose. The considered ways of implementation of optimal strategies of decision - making process relate to the strategy for a short finite time of cargo delivery, which is the most common in practice, and for a long interval of cargo delivery relative to the entire task. In addition, the article discusses ways to find optimal strategies when the time of making single decisions is fixed or when the time of translation is implement randomly.Hence, the situational approach to decision-making in the planning of the route ARPA is very relevant and allows not only to assess the possible situation on the route, but also to determine the control solutions for the operational adjustment of the route using the intelligent decision support system (ISPR). The development of models of the routing process based on the representation of the situational model in the form of nodes of the graph, the transitions of which correspond to the control solutions.The paper proposes two ways to implement optimal strategies of decision - making based on the mathematical apparatus of the theory of Markov and semi-Markov processes using the Bellman principle of optimality.

APA, Harvard, Vancouver, ISO, and other styles

31

Liang, Jinglian, Chao Xu, Zhiyong Feng, and Xirong Ma. "Hidden Markov Model Decision Forest for Dynamic Facial Expression Recognition." International Journal of Pattern Recognition and Artificial Intelligence 29, no. 07 (September 28, 2015): 1556010. http://dx.doi.org/10.1142/s0218001415560108.

Full text

Abstract:

Facial expressions can be mainly conveyed by only a few discriminative facial regions of interest. In this paper, we study the discriminative regions for facial expression recognition from video sequences. The goal of our method is to explore and make use of the discriminative regions for different facial expressions. For this purpose, we propose a Hidden Markov Model (HMM) Decision Forest (HMMDF). In this framework, each tree node is a discriminative classifier, which is constructed by combining weighted HMMs. Motivated by a psychological theory of "elimination by aspects", several HMMs on each node are modeled respectively for facial regions, which have discriminative capabilities for classification. The weights for these HMMs can be further adjusted according to the contributions of facial regions. Extensive experiments validate the effectiveness of discriminative regions on facial expression, and the experimental results show that the proposed HMMDF framework yields dramatic improvements in facial expression recognition compared to existing methods.

APA, Harvard, Vancouver, ISO, and other styles

32

Azam, Md Ali, Hans D. Mittelmann, and Shankarachary Ragi. "UAV Formation Shape Control via Decentralized Markov Decision Processes." Algorithms 14, no. 3 (March 17, 2021): 91. http://dx.doi.org/10.3390/a14030091.

Full text

Abstract:

In this paper, we present a decentralized unmanned aerial vehicle (UAV) swarm formation control approach based on a decision theoretic approach. Specifically, we pose the UAV swarm motion control problem as a decentralized Markov decision process (Dec-MDP). Here, the goal is to drive the UAV swarm from an initial geographical region to another geographical region where the swarm must form a three-dimensional shape (e.g., surface of a sphere). As most decision-theoretic formulations suffer from the curse of dimensionality, we adapt an existing fast approximate dynamic programming method called nominal belief-state optimization (NBO) to approximately solve the formation control problem. We perform numerical studies in MATLAB to validate the performance of the above control algorithms.

APA, Harvard, Vancouver, ISO, and other styles

33

Cruz-Suárez, H., G. Zacarías-Espinoza, and V. Vázquez-Guevara. "A Version of the Euler Equation in Discounted Markov Decision Processes." Journal of Applied Mathematics 2012 (2012): 1–16. http://dx.doi.org/10.1155/2012/103698.

Full text

Abstract:

This paper deals with Markov decision processes (MDPs) on Euclidean spaces with an infinite horizon. An approach to study this kind of MDPs is using the dynamic programming technique (DP). Then the optimal value function is characterized through the value iteration functions. The paper provides conditions that guarantee the convergence of maximizers of the value iteration functions to the optimal policy. Then, using the Euler equation and an envelope formula, the optimal solution of the optimal control problem is obtained. Finally, this theory is applied to a linear-quadratic control problem in order to find its optimal policy.

APA, Harvard, Vancouver, ISO, and other styles

34

Huang, Chien-Cheng, Kwo-Jean Farn, Feng-Yu Lin, and Frank Yeong-Sung Lin. "Software Vulnerability Patch Management with Semi-Markov Decision Process." Applied Mathematics & Information Sciences 7, no. 6 (November 1, 2013): 2467–76. http://dx.doi.org/10.12785/amis/070640.

Full text

APA, Harvard, Vancouver, ISO, and other styles

35

Khalvati, Koosha, Seongmin A. Park, Saghar Mirbagheri, Remi Philippe, Mariateresa Sestito, Jean-Claude Dreher, and Rajesh P. N. Rao. "Modeling other minds: Bayesian inference explains human choices in group decision-making." Science Advances 5, no. 11 (November 2019): eaax8783. http://dx.doi.org/10.1126/sciadv.aax8783.

Full text

Abstract:

To make decisions in a social context, humans have to predict the behavior of others, an ability that is thought to rely on having a model of other minds known as “theory of mind.” Such a model becomes especially complex when the number of people one simultaneously interacts with is large and actions are anonymous. Here, we present results from a group decision-making task known as the volunteer’s dilemma and demonstrate that a Bayesian model based on partially observable Markov decision processes outperforms existing models in quantitatively predicting human behavior and outcomes of group interactions. Our results suggest that in decision-making tasks involving large groups with anonymous members, humans use Bayesian inference to model the “mind of the group,” making predictions of others’ decisions while also simulating the effects of their own actions on the group’s dynamics in the future.

APA, Harvard, Vancouver, ISO, and other styles

36

Junges, Sebastian, Joost-Pieter Katoen, Guillermo A. Pérez, and Tobias Winkler. "The complexity of reachability in parametric Markov decision processes." Journal of Computer and System Sciences 119 (August 2021): 183–210. http://dx.doi.org/10.1016/j.jcss.2021.02.006.

Full text

APA, Harvard, Vancouver, ISO, and other styles

37

Sennott, Linn I. "The Computation of Average Optimal Policies in Denumerable State Markov Decision Chains." Advances in Applied Probability 29, no. 1 (March 1997): 114–37. http://dx.doi.org/10.2307/1427863.

Full text

Abstract:

This paper studies the expected average cost control problem for discrete-time Markov decision processes with denumerably infinite state spaces. A sequence of finite state space truncations is defined such that the average costs and average optimal policies in the sequence converge to the optimal average cost and an optimal policy in the original process. The theory is illustrated with several examples from the control of discrete-time queueing systems. Numerical results are discussed.

APA, Harvard, Vancouver, ISO, and other styles

38

Sennott, Linn I. "The Computation of Average Optimal Policies in Denumerable State Markov Decision Chains." Advances in Applied Probability 29, no. 01 (March 1997): 114–37. http://dx.doi.org/10.1017/s0001867800027816.

Full text

Abstract:

This paper studies the expected average cost control problem for discrete-time Markov decision processes with denumerably infinite state spaces. A sequence of finite state space truncations is defined such that the average costs and average optimal policies in the sequence converge to the optimal average cost and an optimal policy in the original process. The theory is illustrated with several examples from the control of discrete-time queueing systems. Numerical results are discussed.

APA, Harvard, Vancouver, ISO, and other styles

39

Kolarov, Aleksandar, and Joseph Hui. "On computing Markov decision theory-based cost for routing in circuit-switched broadband networks." Journal of Network and Systems Management 3, no. 4 (December 1995): 405–26. http://dx.doi.org/10.1007/bf02139532.

Full text

APA, Harvard, Vancouver, ISO, and other styles

40

Wei, Qingda. "Finite approximation for finite-horizon continuous-time Markov decision processes." 4OR 15, no. 1 (June 11, 2016): 67–84. http://dx.doi.org/10.1007/s10288-016-0321-3.

Full text

APA, Harvard, Vancouver, ISO, and other styles

41

Melnik, Roderick V. Nicholas. "Dynamic system evolution and markov chain approximation." Discrete Dynamics in Nature and Society 2, no. 1 (1998): 7–39. http://dx.doi.org/10.1155/s1026022698000028.

Full text

Abstract:

In this paper computational aspects of the mathematical modelling of dynamic system evolution have been considered as a problem in information theory. The construction of mathematical models is treated as a decision making process with limited available information.The solution of the problem is associated with a computational model based on heuristics of a Markov Chain in a discrete space–time of events. A stable approximation of the chain has been derived and the limiting cases are discussed. An intrinsic interconnection of constructive, sequential, and evolutionary approaches in related optimization problems provides new challenges for future work.

APA, Harvard, Vancouver, ISO, and other styles

42

Sagum, Ria Ambrocio. "Filipino Native Language Identification using Markov Chain Model and Maximum Likelihood Decision Rule." Turkish Journal of Computer and Mathematics Education (TURCOMAT) 12, no. 3 (April 11, 2021): 5475–78. http://dx.doi.org/10.17762/turcomat.v12i3.2206.

Full text

Abstract:

The study developed a tool for identification of a Filipino Native Language given a textual data. The Filipino Language identified were Cebuano, Kapampangan and Pangasinan. It used Markov Chain Model for language modeling using bag of words (a total of 35,144 words for Cebuano, 14752 for Kapampangan, and 13969 of Pangasinan) from each language and maximum likelihood decision rule for the identification of the native language. The obtained model implementing Markov model, was applied in one hundred fifty text files with minimum length of ten words and maximum length of fifty words. The result of the evaluation shows the system’s accuracy of 86.25% and an F-Score of 90.55%.

APA, Harvard, Vancouver, ISO, and other styles

43

Vanneste, Stephan G. "A Generalized Age-Replacement Model." Probability in the Engineering and Informational Sciences 6, no. 4 (October 1992): 525–41. http://dx.doi.org/10.1017/s0269964800002710.

Full text

Abstract:

Four practically important extensions of the classical age-replacement problem are analyzed using Markov decision theory: (1) opportunity maintenance, (2) imperfect repair, (3) non-zero repair times, and (4) Markov degradation of the working unit. For this general model, we show that the optimal maintenance policy is of the control limit type and that the average costs are a unimodal function of the control limit. An efficient optimization procedure is provided to find the optimal policy and its average costs. The analysis extends and unifies existing results.

APA, Harvard, Vancouver, ISO, and other styles

44

Lan, Jian-yi, and Ying Zhou. "Application of Gray Markov SCGM1,1c Model to Prediction of Accidents Deaths in Coal Mining." International Scholarly Research Notices 2014 (November 4, 2014): 1–7. http://dx.doi.org/10.1155/2014/632804.

Full text

Abstract:

The prediction of mine accident is the basis of aviation safety assessment and decision making. Gray prediction is suitable for such kinds of system objects with few data, short time, and little fluctuation, and Markov chain theory is just suitable for forecasting stochastic fluctuating dynamic process. Analyzing the coal mine accident human error cause, combining the advantages of both Gray prediction and Markov theory, an amended Gray Markov SCGM1,1c model is proposed. The gray SCGM1,1c model is applied to imitate the development tendency of the mine safety accident, and adopt the amended model to improve prediction accuracy, while Markov prediction is used to predict the fluctuation along the tendency. Finally, the new model is applied to forecast the mine safety accident deaths from 1990 to 2010 in China, and, 2011–2014 coal accidents deaths were predicted. The results show that the new model not only discovers the trend of the mine human error accident death toll but also overcomes the random fluctuation of data affecting precision. It possesses stronger engineering application.

APA, Harvard, Vancouver, ISO, and other styles

45

GIRTLER, Jerzy. "Possibility of estimating the reliability of diesel engines by applying the theory of semi-Markov processes and making operational decisions by considering reliability of diagnosis on technical state of this sort of combustion engines." Combustion Engines 163, no. 4 (November 1, 2015): 57–66. http://dx.doi.org/10.19206/ce-116857.

Full text

Abstract:

The paper presents semi-Markov models of technical state transitions for diesel engines, useful for determining the reliability of engines. A possibility of application of a three-state model with a simplified matrix function, or even a two-state model, to determine reliability of the engines, has been described herein on examples of known from literature semi-Markov models, for the case when appropriate diagnosing systems (SDG) are used to identify the technical condition of the engines considered as diagnosed systems (SDN). A risk function and a renewal (restitution) function have been proposed for developing a two-state model of engine state transitions. An opportunity for considering the reliability of diagnosis for making operational decisions by applying the statistical decision theory, has also been presented. Conditional probability P(si/Ki) is recognized as a measure of reliability of diagnosis

APA, Harvard, Vancouver, ISO, and other styles

46

Guerreiro, Sérgio Luís Proença Duarte. "Decision-making in partially known business process environments using Markov theory and policy graph visualisation." International Journal of Business Information Systems 36, no. 3 (2021): 355. http://dx.doi.org/10.1504/ijbis.2021.113283.

Full text

APA, Harvard, Vancouver, ISO, and other styles

47

ALIDRISI, MUSTAFA M. "Optimal control of the service rate of an exponential queuing network using Markov decision theory." International Journal of Systems Science 21, no. 12 (December 1990): 2553–63. http://dx.doi.org/10.1080/00207729008910569.

Full text

APA, Harvard, Vancouver, ISO, and other styles

48

Guerreiro, Sérgio. "Decision-Making in Partially Known Business Process Environments using Markov Theory and Policy Graph Visualization." International Journal of Business Information Systems 1, no. 1 (2020): 1. http://dx.doi.org/10.1504/ijbis.2020.10024182.

Full text

APA, Harvard, Vancouver, ISO, and other styles

49

Li, Jing. "A Replica Selection Decision in Cloud Computing Environment." Advanced Materials Research 121-122 (June 2010): 801–6. http://dx.doi.org/10.4028/www.scientific.net/amr.121-122.801.

Full text

Abstract:

Data replication is adopted to improve data access performance in cloud computing environment. When different sites hold replicas, there are significant benefits while selecting the best replica. In this paper, we propose a new replica selection algorithm based on prediction. Analyzing the factors infecting replica selection, The grey system theory is utilized to help predicting the data response time on the basis the GM(1,1) grey dynamic model; Meanwhile the Markov chain is applied to achieve state transition probability matrix to predict the reliability of replicas in the form of probability by the system state classification. The simulation results show that in our replica selection model, the prediction is valid, helpful to selection decision, and is able to achieve load balance between nodes holding replicas.

APA, Harvard, Vancouver, ISO, and other styles

50

Linzner, Dominik, and Heinz Koeppl. "A Variational Perturbative Approach to Planning in Graph-Based Markov Decision Processes." Proceedings of the AAAI Conference on Artificial Intelligence 34, no. 05 (April 3, 2020): 7203–10. http://dx.doi.org/10.1609/aaai.v34i05.6210.

Full text

Abstract:

Coordinating multiple interacting agents to achieve a common goal is a difficult task with huge applicability. This problem remains hard to solve, even when limiting interactions to be mediated via a static interaction-graph. We present a novel approximate solution method for multi-agent Markov decision problems on graphs, based on variational perturbation theory. We adopt the strategy of planning via inference, which has been explored in various prior works. We employ a non-trivial extension of a novel high-order variational method that allows for approximate inference in large networks and has been shown to surpass the accuracy of existing variational methods. To compare our method to two state-of-the-art methods for multi-agent planning on graphs, we apply the method different standard GMDP problems. We show that in cases, where the goal is encoded as a non-local cost function, our method performs well, while state-of-the-art methods approach the performance of random guess. In a final experiment, we demonstrate that our method brings significant improvement for synchronization tasks.

APA, Harvard, Vancouver, ISO, and other styles

We offer discounts on all premium plans for authors whose works are included in thematic literature selections. Contact us to get a unique promo code!