Dissertations / Theses on the topic 'Adaptive critic designs'

Thesis (Ph.D)--Electrical and Computer Engineering, Georgia Institute of Technology, 2009.<br>Committee Chair: Ronald G. Harley; Committee Member: David G. Taylor; Committee Member: Deepakraj M. Divan; Committee Member: Ganesh Kumar Venayagamoorthy; Committee Member: Thomas G. Habetler. Part of the SMARTech Electronic Thesis and Dissertation Collection.

Modern power systems operate much closer to their stability limits than before. With the introduction of highly sensitive industrial and residential loads, the loss of system stability becomes increasingly costly. Reinforcing the power grid by installing additional transmission lines, creating more complicated meshed networks and increasing the voltage level are among the effective, yet expensive solutions. An alternative approach is to improve the performance of the existing power system components by incorporating more intelligent control techniques. This can be achieved in two ways: introducing intelligent local controllers for the existing components in the power network in order to employ their utmost capabilities, and implementing global intelligent schemes for optimizing the performance of multiple local controllers based on an objective function associated with the overall performance of the power system. Both these aspects are investigated in this thesis. In the first section, artificial neural networks are adopted for designing an optimal nonlinear controller for a static compensator (STATCOM) connected to a multimachine power system. The neurocontroller implementation is based on the adaptive critic designs (ACD) technique and provides an optimal control policy over the infinite horizon time of the problem. The ACD based neurocontroller outperforms a conventional controller both in terms of improving the power system dynamic stability and reducing the control effort required. The second section investigates the further improvement of the power system behavior by introducing an ACD based neurocontroller for hierarchical control of a multimachine power system. The proposed wide area controller improves the power system dynamic stability by generating optimal control signals as auxiliary reference signals for the synchronous generators automatic voltage regulators and the STATCOM line voltage controller. This multilevel hierarchical control scheme forces the different controllers throughout the power system to optimally respond to any fault or disturbance by reducing a predefined cost function associated with the power system performance.

[Truncated abstract. Please see the pdf version for the complete text. Also, formulae and special characters can only be approximated here. Please see the pdf version of this abstract for an accurate reproduction.] This thesis treats two aspects of intelligent control: The first part is about long-term optimization by approximating dynamic programming and in the second part a specific class of a fast neural network, related to support vector machines (SVMs), is considered. The first part relates to approximate dynamic programming, especially in the framework of adaptive critic designs (ACDs). Dynamic programming can be used to find an optimal decision or control policy over a long-term period. However, in practice it is difficult, and often impossible, to calculate a dynamic programming solution, due to the 'curse of dimensionality'. The adaptive critic design framework addresses this issue and tries to find a good solution by approximating the dynamic programming process for a stationary environment. In an adaptive critic design there are three modules, the plant or environment to be controlled, a critic to estimate the long-term cost and an action or controller module to produce the decision or control strategy. Even though there have been many publications on the subject over the past two decades, there are some points that have had less attention. While most of the publications address the training of the critic, one of the points that has not received systematic attention is training of the action module.¹ Normally, training starts with an arbitrary, hopefully stable, decision policy and its long-term cost is then estimated by the critic. Often the critic is a neural network that has to be trained, using a temporal difference and Bellman's principle of optimality. Once the critic network has converged, a policy improvement step is carried out by gradient descent to adjust the parameters of the controller network. Then the critic is retrained again to give the new long-term cost estimate. However, it would be preferable to focus more on extremal policies earlier in the training. Therefore, the Calculus of Variations is investigated to discard the idea of using the Euler equations to train the actor. However, an adaptive critic formulation for a continuous plant with a short-term cost as an integral cost density is made and the chain rule is applied to calculate the total derivative of the short-term cost with respect to the actor weights. This is different from the discrete systems, usually used in adaptive critics, which are used in conjunction with total ordered derivatives. This idea is then extended to second order derivatives such that Newton's method can be applied to speed up convergence. Based on this, an almost concurrent actor and critic training was proposed. The equations are developed for any non-linear system and short-term cost density function and these were tested on a linear quadratic regulator (LQR) setup. With this approach the solution to the actor and critic weights can be achieved in only a few actor-critic training cycles. Some other, more minor issues, in the adaptive critic framework are investigated, such as the influence of the discounting factor in the Bellman equation on total ordered derivatives, the target interpretation in backpropagation through time as moving and fixed targets, the relation between simultaneous recurrent networks and dynamic programming is stated and a reinterpretation of the recurrent generalized multilayer perceptron (GMLP) as a recurrent generalized finite impulse MLP (GFIR-MLP) is made. Another subject in this area that is investigated, is that of a hybrid dynamical system, characterized as a continuous plant and a set of basic feedback controllers, which are used to control the plant by finding a switching sequence to select one basic controller at a time. The special but important case is considered when the plant is linear but with some uncertainty in the state space and in the observation vector, and a quadratic cost function. This is a form of robust control, where a dynamic programming solution has to be calculated. &sup1Werbos comments that most treatment of action nets or policies either assume enumerative maximization, which is good only for small problems, except for the games of Backgammon or Go [1], or, gradient-based training. The latter is prone to difficulties with local minima due to the non-convex nature of the cost-to-go function. With incremental methods, such as backpropagation through time, calculus of variations and model-predictive control, the dangers of non-convexity of the cost-to-go function with respect to the control is much less than the with respect to the critic parameters, when the sampling times are small. Therefore, getting the critic right has priority. But with larger sampling times, when the control represents a more complex plan, non-convexity becomes more serious.

High penetration of wind energy requires innovations in different areas of power engineering. Methods for improving wind energy and power system interconnection, control, and operation are proposed in this dissertation. A feed-forward transient compensation control scheme is proposed to enhance the low-voltage ride-through capability of wind turbines equipped with doubly fed induction generators. Stator-voltage transient compensation terms are introduced to suppress rotor-current overshoots and torque ripples during grid faults. A dynamic stochastic optimal power flow control scheme is proposed to optimally reroute real-time active and reactive power flow in the presence of high variability and uncertainty. The performance of the proposed power flow control scheme is demonstrated in test power systems with large wind plants. A combined energy-and-reserve wind market scheme is proposed to reduce wind production uncertainty. Variable wind reserve products are created to absorb part of the wind production variation. These fast wind reserve products can then be used to regulate system frequency and improve system security.

ITC/USA 2011 Conference Proceedings / The Forty-Seventh Annual International Telemetering Conference and Technical Exhibition / October 24-27, 2011 / Bally's Las Vegas, Las Vegas, Nevada<br>In geometrically complex indoor industrial environments, such as factories, health care facilities, or offices, it can be challenging to determine where each telemetry receiver needs to be located to collect data from one or more mobile transmitters. Accurately estimating the areas that each transmitter frequently travels, rarely travels, and quickly travels through, helps to simplify the telemetry system planning problem and establishes which areas may be acceptable to provide marginal coverage. This paper discusses how using A* (A-Star) for transmitter path planning can assist in the telemetry system planning problem.

A variety of factors such as increasing electrical energy demand, slow expansion of transmission infrastructures, and electric energy market deregulation, are forcing utilities and system operators to operate power systems closer to their design limits. Operating under stressed regimes can have a detrimental effect on the rotor-angle stability of the system. This stability reduction is often reflected by the emergence or worsening of poorly damped low-frequency electromechanical oscillations. Without appropriate measures these can lead to costly blackouts. To guarantee system security, operators are sometimes forced to limit power transfers that are economically beneficial but that can result in poorly damped oscillations. Controllers that damp these oscillations can improve system reliability by preventing blackouts and provide long term economic gains by enabling more extensive utilization of the transmission infrastructure. Previous research in the use of artificial neural network-based intelligent controllers for power system damping control has shown promise when tested in small power system models. However, these controllers do not scale-up well enough to be deployed in realistically-sized power systems. The work in this dissertation focuses on improving the scalability of intelligent power system stabilizing controls so that they can significantly improve the rotor-angle stability of large-scale power systems. A framework for designing effective and robust intelligent controllers capable of scaling-up to large scale power systems is proposed. Extensive simulation results on a large-scale power system simulation model demonstrate the rotor-angle stability improvements attained by controllers designed using this framework.

Thesis (Master of Architecture)--University of Cincinnati, 2005.<br>Title from electronic thesis title page (viewed Jul. 11, 2006). Includes abstract. Keywords: Rural vernacular; Rural renewal; Rural details; Cultural Landscape; Details. Includes bibliographical references.

碩士<br>國立臺灣大學<br>電機工程學研究所<br>95<br>The goal of this research is to develop a methodology for the design of sequential learning dual heuristic programming (DHP) control systems. A direction-dependent radial basis function network (DDRBFN) with the sequential learning algorithm of generalized growing-pruning (GGP) mechanism is developed to implement the DHP control system. DDRBFN outperforms traditional RBFN in approximating asymmetrical functions. Ripple phenomenon in the DDRBFN approximation becomes insignificant so that the partial derivative quantities can be calculated correctly in polarities. Based on GGP-DDRBFN, the DHP motion control system and the associated updating rules of the critic and the actor are developed. By implementing the DHP control system to conduct the motion of an autonomous wheeled mobile robot, the performance of the proposed design is verified. Simulation results show that, in the sequential learning, the GGP-DDRBFN-based DHP design converges significantly faster than the multi-layered perceptron network based design. In addition, under the circumstances of system model with unmodeled dynamics and unknown disturbance, GGP-DDRBFN-based DHP design obtains better tracking property.

碩士<br>國立臺灣大學<br>電機工程學研究所<br>93<br>Previous researches on dynamic behavior about the wheeled mobile robot are mostly assuming a purely rolling case, i.e. no slipping. Instead, this research focuses on the wheeled mobile robot moving on muddy surface, on which the wheels may slip. The main contributions are deriving the mathematical model of the robot taking the road condition into consideration and developing the controller based on adaptive critic design. The controller is composed of a fuzzy posture controller and an adaptive critic velocity controller. The posture controller adopts posture error as inputs and produces desired velocities by fuzzy logic inference. The adaptive critic velocity controller conducts the linear and angular velocities with the capabilities of self-learning and optimization in response to unknown road condition and unmodeled nonlinearity. By computer simulation, the proposed design has been verified carefully by driving the wheeled mobile robot on variant muddy surface. The results show that the proposed adaptive critic velocity controller is a successful design to comply with unknown road conditions.

The Dynamic Voltage Scaling (DVS) technique has proven to be ideal in regard to balancing performance and energy consumption of a processor since it allows for almost cubic reduction in dynamic power consumption with only a nearly linear reduction in performance. Due to its virtue, the DVS technique has been used for the two main purposes: energy-saving and temperature reduction. However, recently, a Dynamic Voltage Scaled (DVS) processor has lost its appeal as process technology advances due to the increasing Process, Voltage and Temperature (PVT) variations. In order to make a processor tolerant to the increasing uncertainties caused by such variations, processor designers have used more timing margins. Therefore, in a modern-day DVS processor, reducing voltage requires comparatively more performance degradation when compared to its predecessors. For this reason, this technique has a lot of room for improvement for the following facts. (a) From an energy-saving viewpoint, excessive margins to account for the worst-case operating conditions in a DVS processor can be exploited because they are rarely used during run-time. (b) From a temperature reduction point of view, accurate prediction of the optimal performance point in a DVS processor can increase its performance. In this dissertation, we propose four performance improvement ideas from two different uses of the DVS technique. In regard to the DVS technique for energy-saving, in this dissertation, we introduce three different types of margin reduction (or margin decision) techniques. First, we introduce a new indirect Critical Path Monitor (CPM) to make a conventional DVS processor adaptive to its given environment. Our CPM is composed of several Slope Generators, each of which generates similar voltage scaling slopes to those of potential critical paths under a process corner. Each CPR in the Slope Generator tracks the delays of potential critical paths with minimum difference at any condition in a certain voltage range. The CPRs in the same Slope Generator are connected to a multiplexer and one of them is selected according to a current voltage level. Calibration steps are done by using conventional speed-binning process with clock duty-cycle modulation. Second, we propose a new direct CPM that is based on a non-speculative pre-sampling technique. A processor that is based on this technique predicts timing errors in the actual critical paths and undertakes preventive steps in order to avoid the timing errors in the event that the timing margins fall below a critical level. Unlike direct CPM that uses circuit-level speculative operation, although the shadow latch can have timing error, the main Flip-Flop (FF) of our direct CPM never fails, guaranteeing always-correct operation of the processor. Our non-speculative CPM is more suitable for high-performance processor designs than the speculative CPM in that it does not require original design modification and has lower power overhead. Third, we introduce a novel method that determines the most accurate margin that is based on the conventional binning process. By reusing the hold-scan FFs in a processor, we reduce design complexity, minimize hardware overhead and increase error detecting accuracy. Running workloads on the processor with Stop-Go clock gating allows us to find which paths have timing errors during the speed binning steps at various, fixed temperature levels. From this timing error information, we can determine the different maximum frequencies for diverse operating conditions. This method has high degree of accuracy without having a large overhead. In regard to the DVS technique for temperature reduction, we introduce a run-time temperature monitoring scheme that predicts the optimal performance point in a DVS processor with high accuracy. In order to increase the accuracy of the optimal performance point prediction, this technique monitors the thermal stress of a processor during run-time and uses several Look-Up Tables (LUTs) for different process corners. The monitoring is performed while applying Stop-Go clock gating, and the average EN value is calculated at the end of the monitoring time. Prediction of the optimal performance point is made using the average EN value and one of the LUTs that corresponds to the process corner under which the processor was manufactured. The simulation results show that we can achieve maximum processor performance while keeping the processor temperature within threshold temperature.<br>text

碩士<br>國立交通大學<br>電控工程系所<br>98<br>This thesis develops a mixed-signal critical mode power-factor-correction (PFC) AC-DC converter with load adaptive gain scheduling. The peak current mode control is applied and analyzed by using the analog circuit. The voltage control loop is implemented in digital approach by using a digital notch filter and load adaptive gain adjustment to optimize the dynamic responses and maintain high power-factor (PF) with low total-harmonic-distortion (THD) in line current. This thesis analyzes the effect of the hysteresis band effect of the analog current comparator and the effect of the reference voltage in zero current detecting comparator on current command. In digital voltage loop, the proper quantization resolutions of both analog-to-digital converter (ADC) and digital-to-analog converter (DAC) are analyzed and determined. The digital voltage controller uses a digital notch filter to achieve fast dynamic response and still maintain low THD with high PF. An adaptive gain sheduling is applied for achieving the optimal dynamic response of the output voltage at different load variation conditions. The proposed mixed-signal PFC AC-DC converter with load adaptive gain scheduling has been verified by using computer simulation software (PSIM). This thesis uses the DSP EVM board TMS320LF2407 from Texas Instrument and the CRM PFC IC L6561 from STMicroelectronics to implement the experimental verification. The simulation and experimental results can verify the viability and effectiveness of the proposed control architecture.

Can we design waste? This is a question I seek to answer through the research of design and systems. Waste is an ever evolving and growing issue in our world today. Buildings and the spaces we inhabit contribute to the vast destruction and increasing detriment to our natural world. There are many “remedies” in the construction industry that attempt to regulate building waste and inspire sustainability, but are merely ruses for a much deeper rooted problem than sustaining the way we live. Sustainability is not enough, it simply means we are doing less bad while still perpetuating the problem of waste. Design, architecture, and construction must go beyond this to eradicate the issue; producing “less” waste is not a solution, but a redefining of the essence in which we live is a mandate. This thesis seeks to explore the conundrum of waste through the lens of design. This thesis will study systems as a tool for waste remediation and regeneration. It will explore and scrutinize both building systems such as HVAC and energy efficiency as well as space making systems, scenario based, environmental, sociological, and economical systems, all which have an important and integral impact on design, our environment, and the human population. To answer the question, can we design waste, we must redefine our lives and the systems that propel us habitually in the ways we make, produce, work, eat, and live. Moving away from systems of simplicity to those of diversity and complexity. To do this we must re-examine new and existing systems from socioeconomic to the natural cycles of rain water and evaporation. We must re-define the way we live, on all levels, from how we live and what we use to what we actually need to survive happily and harmoniously with ourselves and our planet. The key – Design.