Stochastic Control in Discrete and Continuous Time

Stochastic Control in Discrete and Continuous Time
Author: Atle Seierstad
Publisher: Springer Science & Business Media
Total Pages: 299
Release: 2008-11-11
Genre: Mathematics
ISBN: 0387766162

This book contains an introduction to three topics in stochastic control: discrete time stochastic control, i. e. , stochastic dynamic programming (Chapter 1), piecewise - terministic control problems (Chapter 3), and control of Ito diffusions (Chapter 4). The chapters include treatments of optimal stopping problems. An Appendix - calls material from elementary probability theory and gives heuristic explanations of certain more advanced tools in probability theory. The book will hopefully be of interest to students in several ?elds: economics, engineering, operations research, ?nance, business, mathematics. In economics and business administration, graduate students should readily be able to read it, and the mathematical level can be suitable for advanced undergraduates in mathem- ics and science. The prerequisites for reading the book are only a calculus course and a course in elementary probability. (Certain technical comments may demand a slightly better background. ) As this book perhaps (and hopefully) will be read by readers with widely diff- ing backgrounds, some general advice may be useful: Don’t be put off if paragraphs, comments, or remarks contain material of a seemingly more technical nature that you don’t understand. Just skip such material and continue reading, it will surely not be needed in order to understand the main ideas and results. The presentation avoids the use of measure theory.

Linear Stochastic Control Systems

Linear Stochastic Control Systems
Author: Goong Chen
Publisher: CRC Press
Total Pages: 404
Release: 1995-07-12
Genre: Business & Economics
ISBN: 9780849380754

Linear Stochastic Control Systems presents a thorough description of the mathematical theory and fundamental principles of linear stochastic control systems. Both continuous-time and discrete-time systems are thoroughly covered. Reviews of the modern probability and random processes theories and the Itô stochastic differential equations are provided. Discrete-time stochastic systems theory, optimal estimation and Kalman filtering, and optimal stochastic control theory are studied in detail. A modern treatment of these same topics for continuous-time stochastic control systems is included. The text is written in an easy-to-understand style, and the reader needs only to have a background of elementary real analysis and linear deterministic systems theory to comprehend the subject matter. This graduate textbook is also suitable for self-study, professional training, and as a handy research reference. Linear Stochastic Control Systems is self-contained and provides a step-by-step development of the theory, with many illustrative examples, exercises, and engineering applications.

Stochastic Optimal Control

Stochastic Optimal Control
Author: Dimitri P. Bertsekas
Publisher:
Total Pages: 323
Release: 1961
Genre: Dynamic programming
ISBN: 9780120932603

Foundations of Deterministic and Stochastic Control

Foundations of Deterministic and Stochastic Control
Author: Jon H. Davis
Publisher: Springer Science & Business Media
Total Pages: 434
Release: 2012-12-06
Genre: Mathematics
ISBN: 1461200717

"This volume is a textbook on linear control systems with an emphasis on stochastic optimal control with solution methods using spectral factorization in line with the original approach of N. Wiener. Continuous-time and discrete-time versions are presented in parallel.... Two appendices introduce functional analytic concepts and probability theory, and there are 77 references and an index. The chapters (except for the last two) end with problems.... [T]he book presents in a clear way important concepts of control theory and can be used for teaching." —Zentralblatt Math "This is a textbook intended for use in courses on linear control and filtering and estimation on (advanced) levels. Its major purpose is an introduction to both deterministic and stochastic control and estimation. Topics are treated in both continuous time and discrete time versions.... Each chapter involves problems and exercises, and the book is supplemented by appendices, where fundamentals on Hilbert and Banach spaces, operator theory, and measure theoretic probability may be found. The book will be very useful for students, but also for a variety of specialists interested in deterministic and stochastic control and filtering." —Applications of Mathematics "The strength of the book under review lies in the choice of specialized topics it contains, which may not be found in this form elsewhere. Also, the first half would make a good standard course in linear control." —Journal of the Indian Institute of Science

Stochastic Differential Equations with Markovian Switching

Stochastic Differential Equations with Markovian Switching
Author: Xuerong Mao
Publisher: Imperial College Press
Total Pages: 430
Release: 2006
Genre: Mathematics
ISBN: 1860947018

This textbook provides the first systematic presentation of the theory of stochastic differential equations with Markovian switching. It presents the basic principles at an introductory level but emphasizes current advanced level research trends. The material takes into account all the features of Ito equations, Markovian switching, interval systems and time-lag. The theory developed is applicable in different and complicated situations in many branches of science and industry.

Discrete-Time Markov Jump Linear Systems

Discrete-Time Markov Jump Linear Systems
Author: O.L.V. Costa
Publisher: Springer Science & Business Media
Total Pages: 287
Release: 2006-03-30
Genre: Mathematics
ISBN: 1846280826

This will be the most up-to-date book in the area (the closest competition was published in 1990) This book takes a new slant and is in discrete rather than continuous time

Introduction to Stochastic Control Theory

Introduction to Stochastic Control Theory
Author: Karl J. Åström
Publisher: Courier Corporation
Total Pages: 322
Release: 2012-05-11
Genre: Technology & Engineering
ISBN: 0486138275

This text for upper-level undergraduates and graduate students explores stochastic control theory in terms of analysis, parametric optimization, and optimal stochastic control. Limited to linear systems with quadratic criteria, it covers discrete time as well as continuous time systems. The first three chapters provide motivation and background material on stochastic processes, followed by an analysis of dynamical systems with inputs of stochastic processes. A simple version of the problem of optimal control of stochastic systems is discussed, along with an example of an industrial application of this theory. Subsequent discussions cover filtering and prediction theory as well as the general stochastic control problem for linear systems with quadratic criteria. Each chapter begins with the discrete time version of a problem and progresses to a more challenging continuous time version of the same problem. Prerequisites include courses in analysis and probability theory in addition to a course in dynamical systems that covers frequency response and the state-space approach for continuous time and discrete time systems.

Controlled Diffusion Processes

Controlled Diffusion Processes
Author: N. V. Krylov
Publisher: Springer Science & Business Media
Total Pages: 314
Release: 2008-09-26
Genre: Science
ISBN: 3540709142

Stochastic control theory is a relatively young branch of mathematics. The beginning of its intensive development falls in the late 1950s and early 1960s. ~urin~ that period an extensive literature appeared on optimal stochastic control using the quadratic performance criterion (see references in Wonham [76]). At the same time, Girsanov [25] and Howard [26] made the first steps in constructing a general theory, based on Bellman's technique of dynamic programming, developed by him somewhat earlier [4]. Two types of engineering problems engendered two different parts of stochastic control theory. Problems of the first type are associated with multistep decision making in discrete time, and are treated in the theory of discrete stochastic dynamic programming. For more on this theory, we note in addition to the work of Howard and Bellman, mentioned above, the books by Derman [8], Mine and Osaki [55], and Dynkin and Yushkevich [12]. Another class of engineering problems which encouraged the development of the theory of stochastic control involves time continuous control of a dynamic system in the presence of random noise. The case where the system is described by a differential equation and the noise is modeled as a time continuous random process is the core of the optimal control theory of diffusion processes. This book deals with this latter theory.

Deterministic and Stochastic Optimal Control

Deterministic and Stochastic Optimal Control
Author: Wendell H. Fleming
Publisher: Springer Science & Business Media
Total Pages: 231
Release: 2012-12-06
Genre: Mathematics
ISBN: 1461263808

This book may be regarded as consisting of two parts. In Chapters I-IV we pre sent what we regard as essential topics in an introduction to deterministic optimal control theory. This material has been used by the authors for one semester graduate-level courses at Brown University and the University of Kentucky. The simplest problem in calculus of variations is taken as the point of departure, in Chapter I. Chapters II, III, and IV deal with necessary conditions for an opti mum, existence and regularity theorems for optimal controls, and the method of dynamic programming. The beginning reader may find it useful first to learn the main results, corollaries, and examples. These tend to be found in the earlier parts of each chapter. We have deliberately postponed some difficult technical proofs to later parts of these chapters. In the second part of the book we give an introduction to stochastic optimal control for Markov diffusion processes. Our treatment follows the dynamic pro gramming method, and depends on the intimate relationship between second order partial differential equations of parabolic type and stochastic differential equations. This relationship is reviewed in Chapter V, which may be read inde pendently of Chapters I-IV. Chapter VI is based to a considerable extent on the authors' work in stochastic control since 1961. It also includes two other topics important for applications, namely, the solution to the stochastic linear regulator and the separation principle.