Intro

This book provides a broad introduction to algorithms for decision making under uncertainty. We cover a wide variety of topics related to decision making, introducing the underlying mathematical problem formulations and the algorithms for solving them.

Download

The full book is available as a PDF. You can also download individual chapters. We will be making improvements during January and February based on community feedback. It will be released in print at a future date, but this electronic version will remain pubicly available for free.

Outline

  1. Introduction

Part I: Probabilistic Reasoning

  1. Representation
  2. Inference
  3. Parameter Learning
  4. Structure Learning
  5. Simple Decisions

Part II: Sequential Problems

  1. Exact Solution Methods
  2. Approximate Value Functions
  3. Online Planning
  4. Policy Search
  5. Policy Gradient Estimation
  6. Policy Gradient Optimization
  7. Actor-Critic Methods
  8. Policy Validation

Part III: Model Uncertainty

  1. Exploration and Exploitation
  2. Model-Based Methods
  3. Model-Free Methods
  4. Imitation Learning

Part IV: State Uncertainty

  1. Beliefs
  2. Exact Belief State Planning
  3. Offline Belief State Planning
  4. Online Belief State Planning
  5. Controller Abstractions

Part V: Multiagent Systems

  1. Multiagent Reasoning
  2. Sequential Problems
  3. State Uncertainty
  4. Collaborative Agents

Appendices

  1. A: Mathematical Concepts
  2. B: Probability Distributions
  3. C: Computational Complexity
  4. D: Neural Representations
  5. E: Search Algorithms
  6. F: Problems
  7. G: Julia

Bugs

We are interested in all forms of feedback including, but not limited to: errors, improvements to code (especially improvements for clarity over speed), typos, areas that are confusing, critical topics that are missing, and ideas for examples or exercises.

Please file issues on GitHub or email the address listed at the bottom of the pages of the PDF.