MARKOV DECISION PROCESSES - ANALYTIC METHODS FOR SEQUENTIAL DECISIONS

Format Requirements: The course will begin with an overview of MDPs, including how they differ from other related types of decision models. Next we formally define the five components of an MDP and formulate multiple examples. We then discuss various solution techniques to MDPs, and conclude with an overview of the literature.

A Markov decision process (MDP) model is a mathematical optimization technique used to solve problems involving sequential decision making under uncertainty, e.g., dynamic treatment decisions over time as a function of patient health. 

The objectives of the course are:

• To provide an overview of Markov decision processes the ways in which they differ from other common MDM modeling techniques, e.g., “embedded decision nodes” in decision analysis structures. 
• To formally define the five components of an MDP and formulate multiple examples. 
• To demonstrate MDP solution techniques used to solve for an optimal policy and discuss potential difficulties in implementation.
• To present several successful applications of MDPs to problems in medical decision-making.