Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems, 1990
Systems & Control: Foundations & Applications Series

The book deals with several closely related topics concerning approxima­ tions and perturbations of random processes and their applications to some important and fascinating classes of problems in the analysis and design of stochastic control systems and nonlinear filters. The basic mathematical methods which are used and developed are those of the theory of weak con­ vergence. The techniques are quite powerful for getting weak convergence or functional limit theorems for broad classes of problems and many of the techniques are new. The original need for some of the techniques which are developed here arose in connection with our study of the particular applica­ tions in this book, and related problems of approximation in control theory, but it will be clear that they have numerous applications elsewhere in weak convergence and process approximation theory. The book is a continuation of the author's long term interest in problems of the approximation of stochastic processes and its applications to problems arising in control and communication theory and related areas. In fact, the techniques used here can be fruitfully applied to many other areas. The basic random processes of interest can be described by solutions to either (multiple time scale) Ito differential equations driven by wide band or state dependent wide band noise or which are singularly perturbed. They might be controlled or not, and their state values might be fully observable or not (e. g. , as in the nonlinear filtering problem).

1 Weak Convergence.- 0. Outline of the Chapter.- 1. Basic Properties and Definitions.- 2. Examples.- 3. The Skorohod Representation.- 4. The Function Space Ck [0, T].- 5. The Function Space Dk [0, T].- 6. Measure Valued Random Variables and Processes.- 2 Stochastic Processes: Background.- 0. Outline of the Chapter.- 1. Martingales.- 2. Stochastic Integrals and Itô’s Lemma.- 3. Stochastic Differential Equations: Bounds.- 4. Controlled Stochastic Differential Equations: Existence of Solutions.- 5. Representing a Martingale as a Stochastic Integral.- 6. The Martingale Problem.- 7. Jump-Diffusion Processes.- 8. Jump-Diffusion Processes: The Martingale Problem Formulation.- 3 Controlled Stochastic Differential Equations.- 0. Outline of the Chapter.- 1. Controlled S.D.E.’s: Introduction.- 2. Relaxed Controls: Deterministic Case.- 3. Stochastic Relaxed Controls.- 4. The Martingale Problem Revisited.- 5. Approximations, Weak Convergence and Optimality.- 4 Controlled Singularly Perturbed Systems.- 0. Outline of the Chapter.- 1. Problem Formulation: Finite Time Interval.- 2. Approximation of the Optimal Controls and Value Functions.- 3. Discounted Cost and Optimal Stopping Problems.- 4. Average Cost Per Unit Time.- 5. Jump-Diffusion Processes.- 6. Other Approaches.- 5 Functional Occupation Measures and Average Cost Per Unit Time Problems.- 0. Outline of the Chapter.- 1. Measure Valued Random Variables.- 2. Limits of Functional Occupation Measures for Diffusions.- 3. The Control Problem.- 4. Singularly Perturbed Control Problems.- 5. Control of the Fast System.- 6. Reflected Diffusions.- 7. Discounted Cost Problem.- 6 The Nonlinear Filtering Problem.- 0. Outline of the Chapter.- 1. A Representation of the Nonlinear Filter.- 2. The Filtering Problem for the SingularlyPerturbed System.- 3. The Almost Optimality of the Averaged Filter.- 4. A Counterexample to the Averaged Filter.- 5. The Near Optimality of the Averaged Filter.- 6. A Repair and Maintainance Example.- 7. Robustness of the Averaged Filters.- 8. A Robust Computational Approximation to the Averaged Filter.- 9. The Averaged Filter on the Infinite Time Interval.- 7 Weak Convergence: The Perturbed Test Function Method.- 0. Outline of the Chapter.- 1. An Example.- 2. The Perturbed Test Function Method: Introduction.- 3. The Perturbed Test Function Method: Tightness and Weak Convergence.- 4. Characterization of the Limits.- 8 Singularly Perturbed Wide-Band Noise Driven Systems.- 0. Outline of the Chapter.- 1. The System and Noise Model.- 2. Weak Convergence of the Fast System.- 3. Convergence to the Averaged System.- 4. The Optimality Theorem.- 5. The Average Cost Per Unit Time Problem.- 9 Stability Theory.- 0. Outline of the Chapter.- 1. Stability Theory for Jump-Diffusion Processes of Itô Type.- 2. Singularly Perturbed Deterministic Systems: Bounds on Paths.- 3. Singularly Perturbed Itô Processes: Tightness.- 4. The Linear Case.- 5. Wide Bandwidth Noise.- 6. Singularly Perturbed Wide Bandwidth Noise Driven Systems.- 10 Parametric Singularities.- 0. Outline of the Chapter.- 1. Singularly Perturbed Itô Processes: Weak Convergence.- 2. Stability.- References.- List of Symbols.