Dissertations / Theses on the topic 'Game theory and optimal control'

In recent years, Unmanned Aerial Vehicles (UAVs) have been used extensively in military conflict situations to execute intelligence, surveillance and reconnaissance missions. However, most of the current UAV platforms have limited collaborative capabilities, and consequently they must be controlled individually by operators on the ground. The purpose of the research presented in this thesis is to derive algorithms that can enable multiple UAVs to reason about the movements of multiple ground targets and autonomously coordinate their efforts in real-time to ensure that the targets do not escape. By improving the autonomy of multivehicle systems, the workload placed on the command and control operators is reduced significantly. To derive effective adversarial control algorithms, the adversarial scenario is modeled as a multiplayer differential game. However, due to the inherent computational complexity of multiplayer differential games, three less computationally demanding differential pursuit-evasion game-based algorithms are presented. The purpose of the algorithms is to quickly derive interception strategies for a team of autonomous vehicles. The algorithms are applicable to scenarios with different base assumptions, that is, the three algorithms are meant to complement one another by addressing different types of adversarial problems.

Freight transport demand has escalated and will continue to do so as economiesgrow. As the traffic intensity increases, the drivers are faced with increasinglycomplex tasks and traffic safety is a growing issue. Simultaneously, fossil fuel usageis escalating. Heavy duty vehicle (HDV) platooning is a plausible solution to theseissues. Even though there has been a need for introducing automated HDV platooningsystems for several years, they have only recently become possible to implement.Advancements in on-board and external technology have ushered in new possibilitiesto aid the driver and enhance the system performance. Each vehicle is able to serveas an information node through wireless communication; enabling a cooperativenetworked transportation system. Thereby, vehicles can semi-autonomously travel atshort intermediate spacings, effectively reducing congestion, relieving driver tension,improving fuel consumption and emissions without compromising safety. This thesis presents contributions to a framework for the design and implementation of HDV platooning. The focus lies mainly on establishing and validating realconstraints for fuel optimal control for platooning vehicles. Nonlinear and linearvehicle models are presented together with a system architecture, which dividesthe complex problem into manageable subsystems. The fuel reduction potentialis investigated through simulation models and experimental results derived fromstandard vehicles traveling on a Swedish highway. It is shown through analyticaland experimental results that it is favorable with respect to the fuel consumption tooperate the vehicles at a much shorter intermediate spacing than what is currentlydone in commercially available systems. The results show that a maximum fuelreduction of 4.7–7.7 % depending on the inter-vehicle time gap, at a set speedof 70 km/h, can be obtained without compromising safety. A systematic designmethodology for inter-vehicle distance control is presented based on linear quadraticregulators (LQRs). The structure of the controller feedback matrix can be tailoredto the locally available state information. The results show that a decentralizedcontroller gives good tracking performance, a robust system and lowers the controleffort downstream in the platoon. It is also shown that the design methodologyproduces a string stable system for an arbitrary number of vehicles in the platoon,if the vehicle configurations and the LQR weighting parameters are identical for theconsidered subsystems. With the results obtained in this thesis, it is argued that a vast fuel reductionpotential exists for HDV platooning. Present commercial systems can be enhancedsignificantly through the introduction of wireless communication and decentralizedoptimal control.
QC 20111012

Electrical and Computer Engineering
Ph.D.
Game theory is the study of tactical interactions involving conflicts and cooperations among multiple decision makers called players with applications in diverse disciplines such as economics, biology, management, communication networks, electric power systems and control. This dissertation studies a statistical differential game problem where finite N players optimize their system performance by shaping the distribution of their cost function through cost cumulants. This research integrates game theory with statistical optimal control theory and considers a statistical Nash non-cooperative nonzero-sum game for a nonlinear dynamic system with nonquadratic cost functions. The objective of the statistical Nash game is to find the equilibrium solution where no player has the incentive to deviate once other players maintain their equilibrium strategy. The necessary condition for the existence of the Nash equilibrium solution is given for the m-th cumulant cost optimization using the Hamilton-Jacobi-Bellman (HJB) equations. In addition, the sufficient condition which is the verification theorem for the existence of Nash equilibrium solution is given for the m-th cumulant cost optimization using the Hamilton-Jacobi-Bellman (HJB) equations. However, solving the HJB equations even for relatively low dimensional game problem is not trivial, we propose to use neural network approximate method to find the solution of the HJB partial differential equations for the statistical game problem. Convergence proof of the neural network approximate method solution to exact solution is given. In addition, numerical examples are provided for the statistical game to demonstrate the applicability of the proposed theoretical developments.
Temple University--Theses

The present dissertation deals with how to use the precious wireless resources that are usually wasted by under-utilization of networks. We have been particularly interested by all resources that can be used in an opportunistic fashion using different technologies. We have designed new schemes for better and more efficient use of wireless systems by providing mathematical frameworks. In the first part, We have been interested in cognitive radio networks, where a cellular service provider can lease a part of its resources to secondary users or virtual providers. In the second part, we have chosen delay-tolerant networks as a solution to reduce the pressure on the cell traffic, where mobile users come to use available resources effectively and with a cheaper cost. We have focused on optimal strategy for smartphones in hybrid wireless networks. In the last part, an alternative to delay-tolerant networks, specially in regions that are not covered by the cellular network, is to use Ad-hoc networks. Indeed, they can be used as an extension of the coverage area. We have developed a new analytical modeling of the IEEE 802.11e DCF/EDCF. We have investigated the intricate interactions among layers by building a general cross-layered framework to represent multi-hop ad hoc networks with asymmetric topology and traffic

Cancer is one of the biggest challenges in healthcare. Fast diagnosis and personalized pharmacological therapies are essential for lowering the mortality rate. In this thesis, we propose a general-purpose model for cancer and an optimal control strategy to minimize its volume. Firstly, we analyze the literature about cancer in the System and Control community and produce a taxonomy of cancer typologies. We identify four main behaviors arising in these models: growth, mutation, migration, and drug response. After this preliminary analysis, we propose a cancer treatment model based on Ordinary Differential Equations (ODEs) and Evolutionary Game Theory, that captures these dynamics more generally. ODEs provide a framework for lumped-parameters representations, and Evolutionary Game Theory provides tools to describe competitive behaviors typical of these cell populations. Starting from this taxonomy, we chose a model representable with a 2-node graph that expressed all the dynamics of cancer processes. We studied the model, discretized it, and applied an optimal control method based on Differential Dynamic Programming (DDP). Bounded and unbounded DDP were ineffective. It was necessary to introduce regularized DDP via adaptive shift. With this algorithm, the results are promising: the system is successfully stabilized in the origin. It is also possible to control the system, driving it between two equilibria, tracking a demanded trajectory. Most of the testing was done in MATLAB. Then, the project was ported to Python. This was done to facilitate future expansion of the model and control strategies through scientific analysis toolboxes and frameworks.

Diese Arbeit behandelt zwei Gebiete: stochastische partielle Rückwerts-Differentialgleichungen (BSPDEs) und Mean-Field-Games (MFGs). Im ersten Teil wird über eine stochastische Variante der De Giorgischen Iteration ein Maximumprinzip für quasilineare reflektierte BSPDEs (RBSPDEs) auf allgemeinen Gebieten bewiesen. Als Folgerung erhalten wir ein Maximumprinzip für RBSPDEs auf beschränkten, sowie für BSPDEs auf allgemeinen Gebieten. Abschließend wird das lokale Verhalten schwacher Lösungen untersucht. Im zweiten Teil zeigen wir zunächst die Existenz von Gleichgewichten in MFGs mit singulärer Kontrolle. Wir beweisen, dass die Lösung eines MFG ohne Endkosten und ohne Kosten in der singulären Kontrolle durch die Lösungen eines MFGs mit strikt regulären Kontrollen approximiert werden kann. Die vorgelegten Existenz- und Approximationsresultat basieren entscheidend auf der Wahl der Storokhod M1 Topologie auf dem Raum der Càdlàg-Funktion. Anschließend betrachten wir ein MFG optimaler Portfolioliquidierung unter asymmetrischer Information. Die Lösung des MFG charakterisieren wir über eine stochastische Vorwärts-Rückwärts-Differentialgleichung (FBSDE) mit singulärer Endbedingung der Rückwärtsgleichung oder alternativ über eine FBSDE mit endlicher Endbedingung, jedoch singulärem Treiber. Wir geben ein Fixpunktargument, um die Existenz und Eindeutigkeit einer Kurzzeitlösung in einem gewichteten Funktionenraum zu zeigen. Dies ermöglicht es, das ursprüngliche MFG mit entsprechenden MFGs ohne Zustandsendbedinung zu approximieren. Der zweite Teil wird abgeschlossen mit einem Leader-Follower-MFG mit Zustandsendbedingung im Kontext optimaler Portfolioliquidierung bei hierarchischer Agentenstruktur. Wir zeigen, dass das Problem beider Spielertypen auf singuläre FBSDEs zurückgeführt werden kann, welche mit ähnlichen Methoden wie im vorangegangen Abschnitt behandelt werden können.
The thesis is concerned with two topics: backward stochastic partial differential equations and mean filed games. In the first part, we establish a maximum principle for quasi-linear reflected backward stochastic partial differential equations (RBSPDEs) on a general domain by using a stochastic version of De Giorgi’s iteration. The maximum principle for RBSPDEs on a bounded domain and the maximum principle for BSPDEs on a general domain are obtained as byproducts. Finally, the local behavior of the weak solutions is considered. In the second part, we first establish the existence of equilibria to mean field games (MFGs) with singular controls. We also prove that the solutions to MFGs with no terminal cost and no cost from singular controls can be approximated by the solutions, respectively control rules, for MFGs with purely regular controls. Our existence and approximation results strongly hinge on the use of the Skorokhod M1 topology on the space of càdlàg functions. Subsequently, we consider an MFG of optimal portfolio liquidation under asymmetric information. We prove that the solution to the MFG can be characterized in terms of a forward backward stochastic differential equation (FBSDE) with possibly singular terminal condition on the backward component or, equivalently, in terms of an FBSDE with finite terminal value, yet singular driver. We apply the fixed point argument to prove the existence and uniqueness on a short time horizon in a weighted space. Our existence and uniqueness result allows to prove that our MFG can be approximated by a sequence of MFGs without state constraint. The final result of the second part is a leader follower MFG with terminal constraint arising from optimal portfolio liquidation between hierarchical agents. We show the problems for both follower and leader reduce to the solvability of singular FBSDEs, which can be solved by a modified approach of the previous result.

This thesis is concerned with mathematical modelling and optimal control of constrained systems. Each of the systems under consideration is a system that can be controlled by one of the variables, and this control is subject to constraints. First, we consider middle-distance running where a runner's horizontal propulsive force is the control which is constrained to be within a given range. Middle-distance running is typically a strategy-intensive race as slipstreaming effects come into play since speeds are still relatively fast and runners can leave their starting lane. We formulate a two-runner coupled model and determine optimal strategies using optimal control theory. Second, we consider two applications of control systems with delay related to R&D expenditure. The first of these applications relates to the defence industry. The second relates to the pharmaceutical industry. Both applications are characterised by a long delay between initial investment in R&D and seeing the benefits of R&D realised. We formulate models tailored to each application and use optimal control theory to determine the optimal proportion of available funds to invest in R&D over a given time horizon. Third, we consider a mathematical model of urban burglary based on the Short model. We make some modifications to this model including the addition of deterrence due to police officer presence. Police officer density is the control variable, which is constrained due to a finite number of police officers. We look at different control strategies for the police and their effect on burglary hot-spot formation.

This dissertation presents efficient, on-line, convergent methods to find defense strategies against attacks in dynamic multi-stage attacker-defender games including adaptive learning. This effort culminated in four papers submitted to high quality journals and a book and they are partially published. The first paper presents a novel fictitious play approach to describe the interactions between the attackers and network administrator along a dynamic game. Multi-objective optimization methodology is used to predict the attacker's best actions at each decision node. The administrator also keeps track of the attacker's actions and updates his knowledge on the attacker's behavior and objectives after each detected attack, and uses this information to update the prediction of the attacker's future actions to find its best response strategies. The second paper proposes a Dynamic game tree based Fictitious Play (DFP) approach to describe the repeated interactive decision processes of the players. Each player considers all possibilities in future interactions with their uncertainties, which are based on learning the opponent's decision process (including risk attitude, objectives). Instead of searching the entire game tree, appropriate future time horizons are dynamically selected for both players. The administrator keeps tracking the opponent's actions, predicts the probabilities of future possible attacks, and then chooses its best moves. The third paper introduces an optimization model to maximize the deterministic equivalent of the random payoff function of a computer network administrator in defending the system against random attacks. By introducing new variables the transformed objective function becomes concave. A special optimization algorithm is developed which requires the computation of the unique solution of a single variable monotonic equation. The fourth paper, which is an invited book chapter, proposes a discrete-time stochastic control model to capture the process of finding the best current move of the defender. The defender's payoffs at each stage of the game depend on the attacker's and the defender's accumulative efforts and are considered random variables due to their uncertainty. Their certain equivalents can be approximated based on their first and second moments which is chosen as the cost functions of the dynamic system. An on-line, convergent, Scenarios based Proactive Defense (SPD) algorithm is developed based on Differential Dynamic Programming (DDP) to solve the associated optimal control problem.

In this thesis, we analyze allocation of two separate resources in wireless networks: transmit power and buffer space. Controlled allocation of power can provide good performance for both users and the network. Although centralized mechanisms are possible, distributed power control algorithms are preferable for efficient operation of the network. Viewing distributed power allocation as the collection of rational decisions of each user, we make game theoretic problem formulations, devise distributed algorithms and analyze them. First, equilibrium analysis of a vector power control game based on network energy efficiency in a multiple access point wireless network is presented. Then, a distributed mechanism is proposed that can smooth admission control type power control so that every user can stay in the system. Introducing a new externality into utility function, a game theoretic formulation that results in desired distributed actions is made. Next, the proposed externality is investigated in a control theoretic framework. Convergence of gradient based iterative power updates are investigated and stability of corresponding continuous time dynamical system is established. In the final part of the thesis, allocation of buffer space is addressed in a wireless downlink using a queueing theoretic framework. An efficient algorithm that finds optimal buffer partitioning is proposed and applications of the algorithm for different scenarios are illustrated. Implications of the results about cross layer design and multiuser diversity are discussed.

The remarkable evolution of wireless networks into the next generation to provide ubiquitous and seamless broadband applications has recently triggered the emergence of Wireless Mesh Networks (WMNs). The WMNs comprise stationary Wireless Mesh Routers (WMRs) forming Wireless Backbone Mesh Networks (WBMNs) and mobile Wireless Mesh Clients (WMCs) forming the WMN access. While WMCs are limited in function and radio resources, the WMRs are expected to support heavy duty applications : that is, WMRs have gateway and bridge functions to integrate WMNs with other networks such as the Internet, cellular, IEEE 802.11, IEEE 802.15, IEEE 802.16, sensor networks, et cetera. Consequently, WMRs are constructed from fast switching radios or multiple radio devices operating on multiple frequency channels. WMRs are expected to be self-organized, self-configured and constitute a reliable and robust WBMN which needs to sustain high traffic volumes and long "online" time. However, meeting such stringent service expectations requires the development of decentralized dynamic transmission power control (DTPC) approaches. This thesis addresses the DTPC problem for both single and multiple channel WBMNs. For single channel networks, the problem is formulated as the minimization of both the link-centric and network-centric convex cost function. In order to solve this issue, multiple access transmission aware (MATA) models and algorithms are proposed. For multi-radio multi-channel (MRMC) WBMNs, the network is modelled as sets of unified channel graphs (UCGs), each consisting of interconnected active network users communicating on the same frequency channel. For each UCG set, the minimization of stochastic quadratic cost functions are developed subject to the dynamic Link State Information (LSI) equations from all UCGs. An energy-efficient multi-radio unification protocol (PMMUP) is then suggested at the Link-Layer (LL). Predictive estimation algorithms based on this protocol are proposed to solve such objective functions. To address transmission energy and packet instabilities, and interference across multiple channels, singularly-perturbed weakly-coupled (SPWC) control problems are formulated. In order to solve the SPWC transmission power control problem, a generalized higher-order recursive algorithm (HORA) that obtains the Riccati Stabilizing Solutions to the control problem is developed. The performance behaviours of the proposed models and algorithms are evaluated both analytically and through computer simulations. Several simulations are performed on a large number of randomly generated topologies. Simulation and analytical results confirm the efficacy of the proposed algorithms compared to the most recently studied techniques

El mundo de las telecomunicaciones está cambiando de un escenario donde únicamente las personas estaban conectadas a un modelo donde prácticamente todos los dispositivos y sensores se encuentran conectados, también conocido como Internet de las cosas (IoT), donde miles de millones de dispositivos se conectarán a Internet a través de conexiones móviles y redes fijas. En este contexto, hay muchos retos que superar, desde el desarrollo de nuevos estándares de comunicación al estudio de la viabilidad económica de los posibles escenarios futuros. En esta tesis nos hemos centrado en el estudio de la viabilidad económica de diferentes escenarios mediante el uso de conceptos de microeconomía, teoría de juegos, optimización no lineal, economía de redes y redes inalámbricas. La tesis analiza la transición desde redes centradas en el servicio de tráfico HTC a redes centradas en tráfico MTC desde un punto de vista económico. El primer escenario ha sido diseñado para centrarse en las primeras etapas de la transición, en la que ambos tipos de tráfico son servidos bajo la misma infraestructura de red. En el segundo escenario analizamos la siguiente etapa, en la que el servicio a los usuarios MTC se realiza mediante una infraestructura dedicada. Finalmente, el tercer escenario analiza la provisión de servicios basados en MTC a usuarios finales, mediante la infraestructura analizada en el escenario anterior. Gracias al análisis de todos los escenarios, hemos observado que la transición de redes centradas en usuarios HTC a redes MTC es posible y que la provisión de servicios en tales escenarios es viable. Además, hemos observado que el comportamiento de los usuarios es esencial para determinar la viabilidad de los diferentes modelos de negocio, y por tanto, es necesario estudiar el comportamiento y las preferencias de los usuarios en profundidad en estudios futuros. Específicamente, los factores más relevantes son la sensibilidad de los usuarios al retardo en los datos recopilados por los sensores y la cantidad de los mismos. También hemos observado que la diferenciación del tráfico en categorías mejora el uso de las redes y permite crear nuevos servicios empleando datos que, de otro modo, no se aprovecharían, lo cual nos permite mejorar la monetización de la infraestructura. También hemos demostrado que la provisión de capacidad es un mecanismo válido, alternativo a la fijación de precios, para la optimización de los beneficios de los proveedores de servicio. Finalmente, se ha demostrado que es posible crear roles específicos para ofrecer servicios IoT en el mercado de las telecomunicaciones, específicamente, los IoT-SPs, que proporcionan servicios basados en sensores inalámbricos utilizando infraestructuras de acceso de terceros y sus propias redes de sensores. En resumen, en esta tesis hemos intentado demostrar la viabilidad económica de modelos de negocio basados en redes futuras IoT, así como la aparición de nuevas oportunidades y roles de negocio, lo cual nos permite justificar económicamente el desarrollo y la implementación de las tecnologías necesarias para ofrecer servicios de acceso inalámbrico masivo a dispositivos MTC.
The communications world is moving from a standalone devices scenario to a all-connected scenario known as Internet of Things (IoT), where billions of devices will be connected to the Internet through mobile and fixed networks. In this context, there are several challenges to face, from the development of new standards to the study of the economical viability of the different future scenarios. In this dissertation we have focused on the study of the economic viability of different scenarios using concepts of microeconomics, game theory, non-linear optimization, network economics and wireless networks. The dissertation analyzes the transition from a Human Type Communications (HTC) to a Machine Type Communications (MTC) centered network from an economic point of view. The first scenario is designed to focus on the first stages of the transition, where HTC and MTC traffic are served on a common network infrastructure. The second scenario analyzes the provision of connectivity service to MTC users using a dedicated network infrastructure, while the third stage is centered in the analysis of the provision of services based on the MTC data over the infrastructure studied in the previous scenario. Thanks to the analysis of all the scenarios we have observed that the transition from HTC users-centered networks to MTC networks is possible and that the provision of services in such scenarios is viable. In addition, we have observed that the behavior of the users is essential in order to determine the viability of a business model, and therefore, it is needed to study their behavior and preferences in depth in future studios. Specifically, the most relevant factors are the sensitivity of the users to the delay and to the amount of data gathered by the sensors. We also have observed that the differentiation of the traffic in categories improves the usage of the networks and allows to create new services thanks to the data that otherwise would not be used, improving the monetization of the infrastructure and the data. In addition, we have shown that the capacity provision is a valid mechanism for providers' profit optimization, as an alternative to the pricing mechanisms. Finally, it has been demonstrated that it is possible to create dedicated roles to offer IoT services in the telecommunications market, specifically, the IoT-SPs, which provide wireless-sensor-based services to the final users using a third party infrastructure. Summarizing, this dissertation tries to demonstrate the economic viability of the future IoT networks business models as well as the emergence of new business opportunities and roles in order to justify economically the development and implementation of the new technologies required to offer massive wireless access to machine devices.
El món de les telecomunicacions està canviant d'un escenari on únicament les persones estaven connectades a un model on pràcticament tots els dispositius i sensors es troben connectats, també conegut com a Internet de les Coses (IoT) , on milers de milions de dispositius es connectaran a Internet a través de connexions mòbils i xarxes fixes. En aquest context, hi ha molts reptes que superar, des del desenrotllament de nous estàndards de comunicació a l'estudi de la viabilitat econòmica dels possibles escenaris futurs. En aquesta tesi ens hem centrat en l'estudi de la viabilitat econòmica de diferents escenaris per mitjà de l'ús de conceptes de microeconomia, teoria de jocs, optimització no lineal, economia de xarxes i xarxes inalàmbriques. La tesi analitza la transició des de xarxes centrades en el servici de tràfic HTC a xarxes centrades en tràfic MTC des d'un punt de vista econòmic. El primer escenari ha sigut dissenyat per a centrar-se en les primeres etapes de la transició, en la que ambdós tipus de tràfic són servits davall la mateixa infraestructura de xarxa. En el segon escenari analitzem la següent etapa, en la que el servici als usuaris MTC es realitza per mitjà d'una infraestructura dedicada. Finalment, el tercer escenari analitza la provisió de servicis basats en MTC a usuaris finals, per mitjà de la infraestructura analitzada en l'escenari anterior. Als paràgrafs següents es descriu amb més detall cada escenari. Gràcies a l'anàlisi de tots els escenaris, hem observat que la transició de xarxes centrades en usuaris HTC a xarxes MTC és possible i que la provisió de servicis en tals escenaris és viable. A més a més, hem observat que el comportament dels usuaris és essencial per a determinar la viabilitat dels diferents models de negoci, i per tant, és necessari estudiar el comportament i les preferències dels usuaris en profunditat en estudis futurs. Específicament, els factors més rellevants són la sensibilitat dels usuaris al retard en les dades recopilats pels sensors i la quantitat dels mateixos. També hem observat que la diferenciació del tràfic en categories millora l'ús de les xarxes i permet crear nous servicis emprant dades que, d'una altra manera, no s'aprofitarien, la qual cosa ens permet millorar la monetització de la infraestructura. També hem demostrat que la provisió de capacitat és un mecanisme vàlid, alternatiu a la fixació de preus, per a l'optimització dels beneficis dels proveïdors de servici. Finalment, s'ha demostrat que és possible crear rols específics per a oferir servicis IoT en el mercat de les telecomunicacions, específicament, els IoT-SPs, que proporcionen servicis basats en sensors inalàmbrics utilitzant infraestructures d'accés de tercers i les seues pròpies xarxes de sensors. En resum, en aquesta tesi hem intentat demostrar la viabilitat econòmica de models de negoci basats en xarxes futures IoT, així com l'aparició de noves oportunitats i rols de negoci, la qual cosa ens permet justificar econòmicament el desenrotllament i la implementació de les tecnologies necessàries per a oferir servicis d'accés inalàmbric massiu a dispositius MTC.
Sanchis Cano, Á. (2018). Economic analysis of wireless sensor-based services in the framework of the Internet of Things. A game-theoretical approach [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/102642
TESIS

Thesis (M. S.)--Management, Georgia Institute of Technology, 2009.
Committee Chair: Cheryl Gaimon; Committee Co-Chair: Stylianos Kavadias; Committee Member: Marco Ceccagnoli; Committee Member: Pinar Keskinocak; Committee Member: Vinod Singhal.

Cette thèse porte sur l’étude du comportement en temps long des jeux à champ moyen (MFG) potentiels, indépendamment de la convexité du problème de minimisation associé. Pour le système hamiltonien de dimension finie, des problèmes de même nature ont été traités par la théorie KAM faible. Nous transposons de nombreux résultats de cette théorie dans le contexte des jeux à champ moyen potentiels. Tout d'abord, nous caractérisons par approximation ergodique la valeur limite associée aux systèmes MFG à horizon fini. Nous fournissons des exemples explicites dans lesquels cette valeur est strictement supérieure au niveau d’énergie des solutions stationnaires du système MFG ergodique. Cela implique que les trajectoires optimales des systèmes MFG à horizon fini ne peuvent pas converger vers des configurations stationnaires. Ensuite, nous prouvons la convergence du problème de minimisation associé à MFG à horizon fini vers une solution de l’équation Hamilton-Jacobi critique dans l’espace de mesures de probabilité. De plus, nous montrons une limite de champ moyen pour la constante ergodique associée à l’équation Hamilton-Jacobi de dimension finie correspondante. Dans la dernière partie, nous caractérisons la limite du problème de minimisation à horizon infini que nous avons utilisé pour l'approximation ergodique dans la première partie du manuscrit
The purpose of this thesis is to shed some light on the long time behavior of potential Mean Field Games (MFG), regardless of the convexity of the minimization problem associated. For finite dimensional Hamiltonian systems, problems of the same nature have been addressed through the so-called weak KAM theory. We transpose many results of this theory in the infinite dimensional context of potential MFG. First, we characterize through an ergodic approximation the limit value associated to time dependent MFG systems. We provide explicit examples where this value is strictly greater than the energy level of stationary solutions of the ergodic MFG system. This implies that optimal trajectories of time dependent MFG systems cannot converge to stationary configurations. Then, we prove the convergence of the minimization problem associated to time dependent MFGs to a solution of the critical Hamilton-Jacobi equation in the space of probability measures. In addition, we show a mean field limit for the ergodic constant associated with the corresponding finite dimensional Hamilton-Jacobi equation. In the last part we characterize the limit of the infinite horizon discounted minimization problem that we use for the ergodic approximation in the first part of the manuscript

This thesis presents several problems based on papers written jointly by the author and Dr. Christian-Oliver Ewald. Firstly, the author extends the model presented by Fershtman and Nitzan (1991), which studies a deterministic differential public good game. Two types of volatility are considered. In the first case the volatility of the diffusion term is dependent on the current level of public good, while in the second case the volatility is dependent on the current rate of public good provision by the agents. The result in the latter case is qualitatively different from the first one. These results are discussed in detail, along with numerical examples. Secondly, two existing lines of research in game theoretic studies of fisheries are combined and extended. The first line of research is the inclusion of the aspect of predation and the consideration of multi-species fisheries within classical game theoretic fishery models. The second line of research includes continuous time and uncertainty. This thesis considers a two species fishery game and compares the results of this with several cases. Thirdly, a model of a fishery is developed in which the dynamic of the unharvested fish population is given by the stochastic logistic growth equation and it is assumed that the fishery harvests the fish population following a constant effort strategy. Explicit formulas for optimal fishing effort are derived in problems considered and the effects of uncertainty, risk aversion and mean reversion speed on fishing efforts are investigated. Fourthly, a Dixit and Pindyck type irreversible investment problem in continuous time is solved, using the assumption that the project value follows a Cox-Ingersoll- Ross process. This solution differs from the two classical cases of geometric Brownian motion and geometric mean reversion and these differences are examined. The aim is to find the optimal stopping time, which can be applied to the problem of extracting resources.

This thesis applies game theory to study optimal toehold bidding strategies during takeover competition, the problem of optimal design of voting rules and the design of package bidding mechanism to implement the core allocations. It documents three different research questions that are all related to auction theory. Chapter 2 develops a two-stage takeover game to explain toehold puzzle in the context of takeover. Potential bidders are allowed to acquire target shares in the open market, subject to some limitations. This pre-bid ownership is known as a toehold. Purchasing a toehold prior to making any takeover offer looks like a profitable strategy given substantial takeover premiums. However actual toehold bidding has decreased since 1980s and now is not common. Its time-series patter is centred on either zero or a large value. Chapter 2 develops a two-stage takeover game. In the first stage of this two-stage game, each bidder simultaneously acquires a toehold. In the second stage, bidders observe acquired toehold sizes, and process this information to update their beliefs about rival's private valuation. Then each bidder competes to win the target under a sealed-bid second-price auction. Different from previous toehold puzzle literature focusing on toehold bidding costs in the form of target managerial entrenchment, this chapter develops a two-stage takeover game and points another possible toehold bidding cost - the opportunity loss of a profitable resale. Chapter 2 finds that, under some conditions, there exists a partial pooling Bayesian equilibrium, in which low-value bidders optimally avoid any toehold, while high-value bidders pool their decisions at one size. The equilibrium toehold acquisition strategies coincide with the bimodal distribution of the actual toehold purchasing behaviour. Chapter 3 studies the problem of optimal design of voting rules when each agent faces binary choice. The designer is allowed to use any type of non-transferable penalty on individuals in order to elicit agents' private valuations. And each agent's private valuation is assumed to be independently distributed. Early work showed that the simple majority rule has good normative properties in the situation of binary choice. However, their results relay on the assumption that agents' preferences have equal intensities. Chapter 3 shows that, under reasonable assumptions, the simple majority is the best voting mechanism in terms of utilitarian efficiency, even if voters' preferences are comparable and may have varying intensities. At equilibrium, the mechanism optimally assigns zero penalty to every voter. In other words, the designer does not extract private information from any agent in the society, because the expected penalty cost of eliciting private information to select the better alternative is too high. Chapter 4 presents a package bidding mechanism whose subgame perfect equilibrium outcomes coincide with the core of an underlying strictly convex transferable utility game. It adopts the concept of core as a competitive standard, which enables the mechanism to avoid the well-known weaknesses of VCG mechanism. In this mechanism, only core allocations generate subgame perfect equilibrium payoffs, because non-core allocations provide arbitrage opportunities for some players. By the strict convexity assumption, the implementation of the core is achieved in terms of expectation.

Dissertação de mestrado, M.A. (Econ), Department of Econometrics, Faculty of Economic and Social Studies,University of Manchester, 1995.
This dissertation analyses the optimal management of fisheries in the framework of dynamic game theory. The analysis rests upon a blend of the economic model of the fishery confined to the waters of a single state, with the theory of dynamic games. In order to accomplish this objective, this dissertation has been organised as follows. Chapter 1 presents some background material concerning noncooperative and cooperative game theory as well as a general formulation of continuous-time infinite dynamic games, also known as differential games. In Chapter 2 we provide some material concerning the dynamics of the fishery by summarising the Gordon-Schaefer bioeconomic model, which provides the foundation for our analysis. Finally, in Chapter 3, and by combining the subject presented in the preceding chapters, we examine the insights which dynamic game theory can provide in analysing fisheries management.

It is often desired to have control over a process or a physical system, to cause it to behave optimally. Optimal control theory deals with analyzing and finding solutions for optimal control for a system that can be represented by a set of differential equations. This thesis examines such a system in the form of a set of matrix differential equations known as a continuous linear time-invariant system. Conditions on the system, such as linearity, allow one to find an explicit closed form finite solution that can be more efficiently computed compared to other known types of solutions. This is done by optimizing a quadratic cost function. The optimization leads to solving a Riccati equation. Conditions are discussed for which solutions are possible. In particular, we will obtain a solution for a stable and controllable system. Numerical examples are given for a simple system with 2x2 matrix coefficients.

In this work we present a general duality-theory framework for revenue maximization in additive Bayesian auctions involving multiple items and many bidders whose values for the goods follow arbitrary continuous joint distributions over some multi-dimensional real interval. Although the single-item case has been resolved in a very elegant way by the seminal work of Myerson [1981], optimal solutions involving more items still remain elusive. The framework extends linear programming duality and complementarity to constraints with partial derivatives. The dual system reveals the natural geometric nature of the problem and highlights its connection with the theory of bipartite graph matchings. We demonstrate the power of the framework by applying it to various special monopoly settings where a seller of multiple heterogeneous goods faces a buyer with independent item values drawn from various distributions of interest, to design both exact and approximately optimal selling mechanisms. Previous optimal solutions were only known for up to two and three goods, and a very limited range of distributional priors. The duality framework is used not only for proving optimality, but perhaps more importantly, for deriving the optimal mechanisms themselves. Some of our main results include: the proposal of a simple deterministic mechanism, which we call Straight-Jacket Auction (SJA) and is defined in a greedy, recursive way through natural geometric constraints, for many uniformly distributed goods, where exact optimality is proven for up to six items and general optimality is conjectured; a scheme of sufficient conditions for exact optimality for two-good settings and general independent distributions; a technique for upper-bounding the optimal revenue for arbitrarily many goods, with an application to uniform and exponential priors; and the proof that offering deterministically all items in a single full bundle is the optimal way of selling multiple exponentially i.i.d. items.

Résumé français : Cette thèse est divisée en deux parties. Dans la première partie on s'intéresse aux problèmes de commande optimale déterministes et on étudie des approximations intérieures pour deux problèmes modèles avec des contraintes de non-négativité sur la commande. Le premier modèle est un problème de commande optimale dont la fonction de coût est quadratique et dont la dynamique est régie par une équation différentielle ordinaire. Pour une classe générale de fonctions de pénalité intérieure, on montre comment calculer le terme principal du développement ponctuel de l'état et de l'état adjoint. Notre argument principal se fonde sur le fait suivant: si la commande optimale pour le problème initial satisfait les conditions de complémentarité stricte pour le Hamiltonien sauf en un nombre fini d'instants, les estimations pour le problème de commande optimale pénalisé peuvent être obtenues à partir des estimations pour un problème stationnaire associé. Nos résultats fournissent plusieurs types de mesures de qualité de l'approximation pour la technique de pénalisation: estimations des erreurs de la commande , estimations des erreurs pour l'état et l'état adjoint et aussi estimations de erreurs pour la fonction valeur. Le second modèle est le problème de commande optimale d'une équation semi-linéaire elliptique avec conditions de Dirichlet homogène au bord, la commande étant distribuée sur le domaine et positive. L'approche est la même que pour le premier modèle, c'est-à-dire que l'on considère une famille de problèmes pénalisés, dont la solution définit une trajectoire centrale qui converge vers la solution du problème initial. De cette manière, on peut étendre les résultats, obtenus dans le cadre d'équations différentielles, au contrôle optimal d'équations elliptiques semi-linéaires. Dans la deuxième partie on s'intéresse aux problèmes de commande optimale stochastiques. Dans un premier temps, on considère un problème linéaire quadratique stochastique avec des contraintes de non-negativité sur la commande et on étend les estimations d'erreur pour l'approximation par pénalisation logarithmique. La preuve s'appuie sur le principe de Pontriaguine stochastique et un argument de dualité. Ensuite, on considère un problème de commande stochastique général avec des contraintes convexes sur la commande. L'approche dite variationnelle nous permet d'obtenir un développement au premier et au second ordre pour l'état et la fonction de coût, autour d'un minimum local. Avec ces développements on peut montrer des conditions générales d'optimalité de premier ordre et, sous une hypothèse géométrique sur l'ensemble des contraintes, des conditions nécessaires du second ordre sont aussi établies
Résumé anglais : This thesis is divided in two parts. In the first one we consider deterministic optimal control problems and we study interior approximations for two model problems with non-negativity constraints. The first model is a quadratic optimal control problem governed by a nonautonomous affine ordinary differential equation. We provide a first-order expansion for the penalized state an adjoint state (around the corresponding state and adjoint state of the original problem), for a general class of penalty functions. Our main argument relies on the following fact: if the optimal control satisfies strict complementarity conditions for its Hamiltonian, except for a set of times with null Lebesgue measure, the functional estimates of the penalized optimal control problem can be derived from the estimates of a related finite dimensional problem. Our results provide three types of measure to analyze the penalization technique: error estimates of the control, error estimates of the state and the adjoint state and also error estimates for the value function. The second model we study is the optimal control problem of a semilinear elliptic PDE with a Dirichlet boundary condition, where the control variable is distributed over the domain and is constrained to be non-negative. Following the same approach as in the first model, we consider an associated family of penalized problems, whose solutions define a central path converging to the solution of the original one. In this fashion, we are able to extend the results obtained in the ODE framework to the case of semilinear elliptic PDE constraints. In the second part of the thesis we consider stochastic optimal control problems. We begin withthe study of a stochastic linear quadratic problem with non-negativity control constraints and we extend the error estimates for the approximation by logarithmic penalization. The proof is based is the stochastic Pontryagin's principle and a duality argument. Next, we deal with a general stochastic optimal control problem with convex control constraints. Using the variational approach, we are able to obtain first and second-order expansions for the state and cost function, around a local minimum. This analysis allows us to prove general first order necessary condition and, under a geometrical assumption over the constraint set, second-order necessary conditions are also established

Many complex systems are hybrid in the sense that: (i) the state set possesses continuous and discrete components, and (ii) system evolution may occur in both continuous and discrete time. One important class of hybrid systems is that characterized by a feedback configuration of a set of continuous controlled low level systems and a high level discrete controller; such systems appear frequently in engineering and are particularly evident when a system is required to operate in a number of distinct modes. Other classes of hybrid systems are found in such diverse areas as (i) air traffic management systems, (ii) chemical process control, (iii) automotive engine-transmission systems, and (iv) intelligent vehicle-highway systems.
In this thesis we first formulate a class of hybrid optimal control problems (HOCPs) for systems with controlled and autonomous location transitions and then present necessary conditions for hybrid system trajectory optimality. These necessary conditions constitute generalizations of the standard Minimum Principle (MP) and are presented for the cases of open bounded control value sets and compact control value sets. These conditions give information about the behaviour of the Hamiltonian and the adjoint process at both autonomous and controlled switching times.
Such proofs of the necessary conditions for hybrid systems optimality which can be found in the literature are sufficiently complex that they are difficult to verify and use; in contrast, the formulation of the HOCP given in Chapter 2 of this thesis, together with the use of (i) classical variational methods and more recent needle variation techniques, and (ii) a local controllability condition, called the small time tubular fountain (STTF) condition, make the proofs in that chapter comparatively accessible. We note that the STTF condition is used to establish the adjoint and Hamiltonian jump conditions in the autonomous switchings case.
A hybrid Dynamic Programming Principle (HDPP) generalizing the standard dynamic programming principle to hybrid systems is also derived and this leads to hybrid Hamilton-Jacobi-Bellman (HJB) equation which is then used to establish a verification theorem within this framework. (Abstract shortened by UMI.)

A number of iterative algorithms for solving unconstrained continuous-time optimal control problems is developed. The method, named LORE is modelled locally on the linear quadratic problem and treats a whole family of algorithms in a unified manner, proposed approach is similar to the philosophy underlying Newton's, the Conjugate gradient and the quasi- Newton methods in the function minimization theory, The where the function being minimized is approximated locally by a quadratic function. In addition to the development of a unified theory of algorithms, the thesis contains several first-order implementable algorithms, the convergence speed of which is comparable to that of second-order methods. A proof of the reduction of the cost at each iterative step of the LORE algorithms, a convergence analysis in the Lm∞ space and a proof of the convergence in the space of relaxed controls are included. The power of the adopted approach lies in the use of the Riccati matrix differential equation which within the context of the LQRE method always has a bounded solution. Within the general framework of the analysis it is possible to obtain both first-order and second-order algorithms. The emphasis is however placed on the first-order LORE algorithms which are simpler and computationally less demanding per iterative step; their computational effectiveness is compared with the performance of known methods. There is a noticeable degree of similarity in the form of the differential equations used by the LORE algorithms and the differential equations in the well-known second order methods; in fact, it is possible to derive LORE variants of the second variation and differential dynamic programming methods. The LORE algorithms converge in one step on the linear quadratic problem and are well suited for solving nonlinear problems with linear constraints via the penalty Their application in the computation of the singular optimal control, by adding and subtracting a quadratic term to the cost, is suggested. function methods. The method has been extended to handle problems with terminal equality constraints, control constraints (LORE projection technique) and a class of state and control equality constraints (sequential LORE-restoration algorithm). The LORE method of discrete-time unconstrained systems has also been developed.

Decision making under uncertainty is a widely-studied area spanning a number of fields such as computational optimization, control theory, utility theory and game theory. A typical problem of decision making under uncertainty requires the design of an optimal decision rule, control policy, or behavioural function, that takes into account all available information regarding the uncertain parameters and returns the decision that is most suitable for the given objective. A popular requirement is to determine robust decisions that maintain certain desired characteristics despite the presence of uncertainty. In this thesis, we study three distinct problems that involve the design of robust decisions under different types of uncertainty. We investigate a dynamic multi-stage control problem with stochastic exogenous uncertainty, a dynamic two-stage robust optimization problem with epistemic exogenous uncertainty, and finally a game theoretic problem with both stochastic endogenous and epistemic exogenous uncertainty. Specifically, a) we develop an efficient algorithm that bounds the performance loss of affine policies operating in discrete-time, finite-horizon, stochastic systems with expected quadratic costs and mixed linear state and input constraints. Finding the optimal control policy for such problems is generally computationally intractable, but suboptimal policies can be computed by restricting the class of admissible policies to be affine on the observation. Our algorithm provides an estimate of the loss of optimality due to the use of such affine policies, and it is based on a novel dualization technique, where the dual variables are restricted to have an affine structure; b) we develop an efficient algorithm to bound the suboptimality of linear decision rules in two-stage dynamic linear robust optimization problems, where they have been shown to suffer a worst-case performance loss of the order $\Omega(\sqrt{m})$ for problems with $m$ linear constraints. Our algorithm is based on a scenario selection technique, where the original problem is evaluated only over a finite subset of the possible uncertain parameters. This set is constructed from the Lagrange multipliers associated with the computation of the linear adaptive decision rules. The resulting instance-wise bounds outperform known bounds, including the aforementioned worst-case bound, in the vast majority of problem instances; c) we develop an algorithm that enumerates all behavioural functions that are at equilibrium in a game where players face epistemic uncertainty regarding their opponent's utility functions. Traditionally, these games are solved as complete-information games where players are assumed to be risk-neutral, with a utility function that is positively affine in the monetary payoffs. We demonstrate that this assumption imposes severe limitations on the problem structure, and we propose that these games should be formulated as incomplete private information games where each player may have any increasing or increasing concave utility function. If the players are ambiguity-averse, then under these assumptions, they play either a pure strategy, a max-min strategy, or a convex combination of the two. By utilizing this result, we develop an algorithm that can enumerate all equilibria of the game.

In this dissertation we introduce a new class of pursuit-evasion games; the herding problem. Unlike regular pursuit evasion games where the pursuer aims to hunt the evader the objective of the pursuer in this game is to drive the evader to a certain location on the x-y grid. The dissertation deals with this problem using two different methodologies. In the first, the problem is introduced in the continuous-time, continuous-space domain. The continuous time model of the problem is proposed, analyzed and we came up with an optimal control law for the pursuer is obtained so that the evader is driven to the desired destination position in the x-y grid following the local shortest path in the Euler Lagrange sense. Then, a non-holonomic realization of the two agents is proposed. In this and we show that the optimal control policy is in the form of a feedback control law that enables the pursuer to achieve the same objective using the shortest path. The second methodology deals with the discrete model representation of the problem. In this formulation, the system is represented by a finite di-graph. In this di-graph, each state of the system is represented by a node in the graph. Applying dynamic programming technique and shortest path algorithms over the finite graph representing the system, we come up with the optimal control policy that the pursuer should follow to achieve the desired goal. To study the robustness, we formulate the problem in a stochastic setting also. We analyze the stochastic model and derive an optimal control law in this setting. Finally, the case with active evader is considered, the optimal control law for this case is obtained through the application of dynamic programming technique.
Ph. D.

Ad hoc networks are emerging as a cost-effective, yet, powerful tool for communication. These systems, where networks can emerge and converge on-the-fly, are guided by the forward-looking goals of providing ubiquitous connectivity and constant access to information. Due to power and bandwidth constraints, the vulnerability of the wireless medium, and the multi-hop nature of ad hoc networks, these networks are becoming increasingly complex dynamic systems. Besides, modern radios are empowered to be reconfigurable, which harbors the temptation to exploit the system. To understand the implications of these issues, some of which pose significant challenges to efficient network design, we study topology control using game theory. We develop a game-theoretic framework of topology control that broadly captures the radio parameters, one or more of which can be tuned under the purview of topology control. In this dissertation, we consider two parameters, viz. transmit power and channel, and study the impact of controlling these on the emergent topologies. We first examine the impact of node selfishness on the network connectivity and energy efficiency under two levels of selfishness: (a) nodes cooperate and forward packets for one another, but selfishly minimize transmit power levels and; (b) nodes selectively forward packets and selfishly control transmit powers. In the former case, we characterize all the Nash Equilibria of the game and evaluate the energy efficiency of the induced topologies. We develop a better-response-based dynamic that guarantees convergence to the minimal maximum power topology. We extend our analysis to dynamic networks where nodes have limited knowledge about network connectivity, and examine the tradeoff between network performance and the cost of obtaining knowledge. Due to the high cost of maintaining knowledge in networks that are dynamic, mobility actually helps in information-constrained networks. In the latter case, nodes selfishly adapt their transmit powers to minimize their energy consumption, taking into account partial packet forwarding in the network. This work quantifies the energy efficiency gains obtained by cooperation and corroborates the need for incentivizing nodes to forward packets in decentralized, energy-limited networks. We then examine the impact of selfish behavior on spectral efficiency and interference minimization in multi-channel systems. We develop a distributed channel assignment algorithm to minimize the spectral footprint of a network while establishing an interference-free connected network. In spite of selfish channel selections, the network spectrum utilization is shown to be within 12% of the minimum on average. We then extend the analysis to dynamic networks where nodes have incomplete network state knowledge, and quantify the price of ignorance. Under the limitations on the number of available channels and radio interfaces, we analyze the channel assignment game with respect to interference minimization and network connectivity goals. By quantifying the interference in multi-channel networks, we illuminate the interference reduction that can be achieved by utilizing orthogonal channels and by distributing interference over multiple channels. In spite of the non-cooperative behavior of nodes, we observe that the selfish channel selection algorithm achieves load balancing. Distributing the network control to autonomous agents leaves open the possibility that nodes can act selfishly and the overall system is compromised. We advance the need for considering selfish behavior from the outset, during protocol design. To overcome the effects of selfishness, we show that the performance of a non-cooperative network can be enhanced by appropriately incentivizing selfish nodes.
Ph. D.

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Ocean Engineering; and, (S.M.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2005.
Includes bibliographical references (leaves 70-71).
Ship drivers have long understood that powerful interaction forces exist when ships operate in close proximity to rigid boundaries or other vessels. Controlling the effects of these forces has been traditionally handled by experienced helmsmen. The purpose of this research is to apply modern optimal control theory to these maneuvering scenarios in order to show that helmsman may some day be replaced by modern controllers. The maneuvering equations of motion are cast in a linear state space framework, permitting the design of a linear quadratic (LQ) controller. In addition, the hydrodynamic effects are modeled using potential flow theory in order to simulate the interaction forces and test the efficacy of the controller. This research demonstrates that the linear quadratic regulator effectively controls ship motions due to the presence of a boundary or other vessel over a broad range of speeds and separation distances. Furthermore, the method proposed provides stable control in the presence of additional. stochastic disturbances.
by Brian S. Thomas.
S.M.

Mestrado em Matemática e Aplicações
O Cálculo das Variações e o Controlo Ótimo são dois ramos da Matemática que estão muito interligados entre si e também com outras áreas. Como exemplo, podemos citar a Geometria, a Física, a Mecânica, a Economia, a Biologia, bem como a Medicina. Nesta tese estudamos vários tipos de problemas variacionais e de Controlo Ótimo, estabelecendo a ligação entre alguns destes. Fazemos uma breve introdução sobre a Diabetes Mellitus, uma vez que estudamos um modelo matemático que traduz a interação entre a glicose e a insulina no sangue por forma a otimizar o estado de uma pessoa com diabetes tipo 1.
The Calculus of Variations and the Optimal Control are two branches of Mathematics that are very interconnected with each other and with other areas. As example, we can mention Geometry, Physics, Mechanics, Economics, Biology and Medicine. In this thesis we study various types of variational problems and of Optimal Control, establishing the connection between some of these. We make a brief introduction to the Diabetes Mellitus, because we study a mathematical model that reflects the interaction between glucose and insulin in the blood in order to optimize the state of a person with diabetes type 1.

The main contributions in this thesis are advances in parametric programming. The thesis is divided into three parts; theoretical advances, application areas and constrained control allocation. The first part deals with continuity properties and the structure of solutions to convex parametric quadratic and linear programs. The second part focuses on applications of parametric quadratic and linear programming in control theory. The third part deals with constrained control allocation and how parametric programming can be used to obtain explicit solutions to this problem.