Intro

This book provides a broad introduction to algorithms for decision making under uncertainty. We cover a wide variety of topics related to decision making, introducing the underlying mathematical problem formulations and the algorithms for solving them.

Download

The full book is available as a PDF. You can also download individual chapters. The PDF is shared under a under a Creative Commons CC-BY-NC-ND license.

The copyright of this book has been licensed exclusively to The MIT Press. All inquiries regarding rights should be addressed to The MIT Press, Rights and Permissions Department.

Outline

  1. Introduction

Part I: Probabilistic Reasoning

  1. Representation
  2. Inference
  3. Parameter Learning
  4. Structure Learning
  5. Simple Decisions

Part II: Sequential Problems

  1. Exact Solution Methods
  2. Approximate Value Functions
  3. Online Planning
  4. Policy Search
  5. Policy Gradient Estimation
  6. Policy Gradient Optimization
  7. Actor-Critic Methods
  8. Policy Validation

Part III: Model Uncertainty

  1. Exploration and Exploitation
  2. Model-Based Methods
  3. Model-Free Methods
  4. Imitation Learning

Part IV: State Uncertainty

  1. Beliefs
  2. Exact Belief State Planning
  3. Offline Belief State Planning
  4. Online Belief State Planning
  5. Controller Abstractions

Part V: Multiagent Systems

  1. Multiagent Reasoning
  2. Sequential Problems
  3. State Uncertainty
  4. Collaborative Agents

Appendices

  1. A: Mathematical Concepts
  2. B: Probability Distributions
  3. C: Computational Complexity
  4. D: Neural Representations
  5. E: Search Algorithms
  6. F: Problems
  7. G: Julia

Errata

Please file issues on GitHub or email the address listed at the bottom of the pages of the PDF. The PDF is kept up to date with any corrections.