Sequential Optimization of Dynamically Augmented Conditional Value-at-Risk

This talk describes methods for optimization of the Conditional Value-at-Risk (CVaR) of total discounted costs for Markov Decision Processes (MDPs) with finite state and action sets. It introduces the Dynamically augmented CVaR (DCVaR) risk measure and provides an algorithm for its optimization. This paper investigates a specially defined Robust MDP (RMDP), in which the state space is augmented with the tail risk factor. This RMDP, which we call the Dynamically augmented RMDP (DRMDP), was introduced to the literature for calculations of optimal CVaR values by value iteration, but, as was understood later, these value iterations compute lower bounds of minimal static CVaRs. DCVaR is defined as a time consistent version of the static CVaR, and it is a lower bound of the static CVaR. It also can be considered as a dynamic version of the nested CVaR. The correctness of the provided algorithm is proved by studying a special mass transfer problem.

BIO: Eugene A. Feinberg received MS in Applied Mathematics and Computer Engineering from Moscow University of Transportation, Russia, in 1976 and Ph.D. in Probability and Statistics from Vilnius University, Lithuania, in 1979. Currently he is Distinguished Professor at the Department of Applied Mathematics and Statistics of Stony Brook University.