Publication Date:
2020-03-20
Description:
Markov Decision Processes (MDP) or Partially Observable MDPs (POMDP) are
used for modelling situations in which the evolution of a process is partly random and
partly controllable. These MDP theories allow for computing the optimal control policy
for processes that can continuously or frequently be observed, even if only partially.
However, they cannot be applied if state observation is very costly and therefore rare
(in time). We present a novel MDP theory for rare, costly observations and derive the
corresponding Bellman equation. In the new theory, state information can be derived
for a particular cost after certain, rather long time intervals. The resulting information
costs enter into the total cost and thus into the optimization criterion. This approach
applies to many real world problems, particularly in the medical context, where the
medical condition is examined rather rarely because examination costs are high. At the
same time, the approach allows for efficient numerical realization. We demonstrate the
usefulness of the novel theory by determining, from the national economic perspective,
optimal therapeutic policies for the treatment of the human immunodefficiency virus
(HIV) in resource-rich and resource-poor settings. Based on the developed theory and
models, we discover that available drugs may not be utilized efficiently in resource-poor
settings due to exorbitant diagnostic costs.
Language:
English
Type:
reportzib
,
doc-type:preprint
Format:
application/pdf
Permalink