Introduction
- Structure of Sequential Decision Problems
- Discrete-Time Optimal Control Problems - Measurability Questions
- The Present Work Related to the Literature
Monotone Mappings Underlying Dynamic Programming
Models
- Notation and Assumptions
- Main Results
- Application to Specific Models
- Deterministic Optimal Control
- Stochastic Optimal Control - Countable Disturbance Space
- Stochastic Optimal Control - Outer Integral Formulation
- Stochastic Optimal Control - Multiplicable Cost Functional
- Minimax Control
Finite Horizon Models
- General Results and Assumptions
- Main Results
- Application to Specific Models
Infinite Horizon Models under a Contraction Assumption
- General Results and Assumptions
- Convergence and Existence Results
- Computational Methods
- Successive Approximation
- Policy Iteration
- Mathematical Programming
- Application to Specific Models
Infinite Horizon Models under Monotonicity Assumptions
- General Results and Assumptions
- The Optimality Equation
- Characterization of Optimal Policies
- Convergence of the Dynamic Programming Algorithm - Existence of Stationary Policies
- Application to Specific Models
A Generalized Abstract Dynamic Programming Model
- General Results and Assumptions
- Analysis of Finite Horizon Models
- Analysis of Infinite Horizon Models under a Contraction Assumption
Borel Spaces and their Probability Measures
- Notation
- Metrizable Spaces
- Borel Spaces
- Probability Measures on Borel Spaces
- Characterization of Probability Measures
- The Weak Topology
- Stochastic Kernels
- Integration
- Semicontinuous Functions and Borel-Measurable Selection
- Analytic Sets
- Equivalent Definitions of Analytic Sets
- Measurability Properties of Analytic Sets
- An Analytic Set of Probability Measures
- Lower Semianalytic Functions and Universally Measurable Selection
The Finite Horizon Borel Model
- The Model
- The Dynamic Programming Algorithm - Existence of Optimal and epsilon-Optimal Policies
- The Semincontinuous Models
The Infinite Horizon Borel Models
- The Stochastic Model
- The Deterministic Model
- Relations Between the Models
- The Optimality Equation - Characterization of Optimal Policies
- Convergence of the Dynamic Programming Algorithm - Existence of Stationary Optimal Policies
- Existence of epsilon-Optimal Policies
The Imperfect State Information Model
- Reduction of the Nonstationary Model - State Augmentation
- Reduction of the Imperfect State Information Model - Sufficient Statistics
- Existence of Sufficient Statistics for Control
- Filtering and the Conditional Distribution of the States
- The Identity Mappings
Miscellaneous
- Limit-Measurable Policies
- Analytically Measurable Policies
- Models with Multiplicative Cost
Appendix A: The Outer Integral
Appendix B: Additional Measurability Properties of Borel Spaces
Appendix C: The Hausdorff Metric and the Exponential Topology