Note on the (non-)smoothness of discrete time value functions

We consider the discrete time stopping problem $$V(t,x)=\underset{\tau }{\sup }{E}_{(t,x)}[g(\tau ,X_{\tau })],$$ where $X$ is a random walk. It is well known that the value function $V$ is in general not smooth on the boundary of the continuation set $\mathrm{\partial }C$. We show that under some conditions $V$ is not smooth in the interior of $C$ either. More precisely we show that $V$ is not differentiable in the $x$ component on a dense subset of $C$. As an example we consider the Chow-Robbins game. We give evidence that as well $\mathrm{\partial }C$ is not smooth and that $C$ is not convex, even if $g(t,\cdot )$ is for every $t$.