ELICITING PATIENTS' REVEALED PREFERENCES: AN INVERSE MARKOV DECISION PROCESS APPROACH

Purpose: We propose a new method for estimating patients' utilities based on observed decisions. We assume that patients make decisions that maximize their expected QALYs under health utilities. The goal is to find a set of health utilities such that the patient's observed behavior is optimal.

Method: We propose a new technique, an inverse Markov Decision Process (MDP). This inverse optimization technique, given observed patient decisions, infers the values of the rewards (QALYs) that make the observed decision optimal. That is, by considering constrained perturbations of non-quality adjusted expected life days, the inverse MDP produces a refined estimate of the patient's QALYs. The method is illustrated on the problem of determining when, as a function of health (MELD score), a patient should consent to a living-donor transplant. Clinical data from multiple sources is used to estimate the health progression parameters, non-quality adjusted expected life days and observed decisions.

Results: Numerical results are presented for a patient who opted for transplantation earlier than the optimal time under the non-quality adjusted expected life days. Surprisingly, the post-transplant QALYs that the inverse MDP yields are identical to the non-quality adjusted expected life days. However, the pre-transplant QALYs that the inverse MDP yields are strictly less than the non-quality adjusted expected life days and decrease in patient health (see figure).

Conclusion: The QALYs that the inverse MDP yields can be interpreted as a reflection of quality of life preferences across MELD scores, and/or a preference to end the optimal stopping problem sooner rather than later. Overall, the proposed inverse MDP method proves to be a promising method to estimate patients' QALYs.