Doob H-transforms

June 2, 2010 at 11:21 am (Brownian motion, Markov process, probability, Stochastic analysis)
Tags: brownian motion, conditioning, Doob, h-transform, Markov process, probability

I read today about Doob h-transforms in the Rogers-Williams … It is done quite quickly in the book so that I decided to practice on some simple examples to see how this works.

So we have an Markov process ${X_t}$ living in ${S}$ , and we want to see how this process looks like if we condition on the event ${X_T \in A}$ , say. To fix the notations we define ${p(t,t+s,x,y) = P(X_{t+s}=y|X_t=x)}$ and ${h(t,x)=P(X_T \in A \, | X_t=x)}$ . The conditioned semi-group ${\hat{p}(t,t+s,x,y)=P(X_{t+s}=y|X_t=x, X_T \in A)}$ is quite easily computed from ${p}$ and ${h}$ . Indeed, this is also equal to

$\displaystyle \hat{p}(t,t+s,x,y) = \frac{P(X_{t+s}=y; X_T \in A\;|X_t=x)}{P(X_T \in A \,|X_t=x)} = p(t,t+s,x,y) \frac{h(t+s,y)}{h(t,x)}.$

Notice also that ${\hat{p}(t,t+s,x,y) = p(t,t+s,x,y) \frac{h(t+s,y)}{h(t,x)}}$ is indeed a Markov kernel in the sense that ${\int_{y} \hat{p}(t,t+s,x,y) \, dy = 1}$ : the only property needed for that is

$\displaystyle h(t,x) = \int_{y} p(t,t+s,x,y)h(t+s,y)\,dy = E\left[ h(t+s,X_{t+s}) \, |X_t=x\right].$

In fact, we could take any function ${h}$ that satisfies this equality and define a new Markovian kernel ${\hat{p}}$ and study the associated Markov process. That’s what people usually do by the way.

Remark 1 we almost never know explicitly the quantity ${h(t,x)}$ , except in some extremely simple cases !

Before trying these ideas on some simple examples, let us see what this says on the generator of the process:

continuous time Markov chains, finite state space:let us suppose that the intensity matrix is ${Q}$ and that we want to know the dynamic on ${[0,T]}$ of this Markov chain conditioned on the event ${X_T=z}$ . Indeed ${p(t,t+s,i,j) = [\exp(sQ)]_{i,j}}$ so that ${\hat{p}(t,t+s,i,j) = [\exp(sQ)]_{i,j} \frac{p(t+s,T,j,z)}{p(t,T,i,z)}}$ so that in the limit we see that at time ${t}$ , the intensity of the jump from ${i}$ to ${j}$ of the conditioned Markov chain is
$\displaystyle Q(i,j) \frac{p(t+s,T,j,z)}{p(t,T,i,z)}.$

Notice how this behaves while ${t \rightarrow T}$ : if at ${t=T-\epsilon}$ the Markov chain is in state ${i \neq z}$ then the intensity of jump from ${i}$ to ${z}$ is equivalent to ${\approx \frac{1}{\epsilon}}$ .
diffusion processes:this time consider a ${1}$ -dimensional diffusion ${dX_t = \mu(X_t) \, dt + \sigma(X_t) \, dW_t}$ on ${[0,T]}$ conditioned on the event ${X_T \in A}$ and define as before ${h(t,x)=P(X_T \in A \,|X_t=x)}$ . The generator of the (non-homogeneous) conditioned diffusion is defined at time ${t}$ by
$\displaystyle \begin{array}{rcl} \mathcal{G}^{(t)} f(x) &=& \lim_{s \rightarrow 0} \frac{1}{s} \Big( E\left[f(X_{t+s}) \,| X_t=x, X_T \in A\right]-f(x) \Big)\\ &=& \lim_{s \rightarrow 0} \frac{E\left[ f(X_{t+s}) h(t+s, X_{t+s}) \,| X_t=x\right]-f(x) }{s\,h(t,x)} \end{array}$

so that if ${\mathcal{L} = \mu \partial_x + \frac{1}{2} \sigma^2 \partial^2_{xx}}$ is the generator of the original diffusion we get

$\displaystyle \mathcal{G}^{(t)} f = \frac{1}{h} \Big(\partial_t + \mathcal{L})(hf).$

Because ${(\partial_t + \mathcal{L})h=0}$ , this also reads

$\displaystyle \mathcal{G}^{(t)} f = \mathcal{L}f + \sigma^2 \frac{\partial_x h}{h} \partial_x f.$

This means that the conditioned diffusion ${Z}$ follows the SDE:

$\displaystyle dZ_t = \Big( \mu(Z_T) + \sigma(Z_t)^2 \frac{\partial_x h(t,Z_t)}{h(t,Z_t)} \Big) \, dt+ \sigma(Z_t) dW_t.$

The volatility function remains the same while an additional drift shows up.

We will try these ideas on some examples where the probability densities are extremely simple. Notice that in the case of diffusions, if we take ${A=\{x^+\}}$ , the function ${(t,x) \mapsto P(X_T = x^+ \, |X_t=x)}$ is identically equal to ${0}$ (except degenerate cases): to condition on the event ${X_T=x}$ we need instead to take ${h(t,x)}$ to be the transition probability ${p(t,T,x,x^+)}$ . This follows from the approximation ${P(X_T \in (x^+,x^+ + dx) \,|X_t=x ) \approx p(t,T,x,x^+) \, dx + o(dx)}$ . Let’s do it:

Brownian Bridge on ${[0,T]}$ :in this case ${p(t,T,x,0) \propto e^{-\frac{|x-y|^2}{2(T-t)}}}$ so that the additional drift reads ${\frac{-x}{T-t}}$ : a Brownian bridge follows the SDE
$\displaystyle dX_t = -\frac{X_t}{T-t} \, dt + dW_T.$

This might not be the best way to simulate a Brownian bridge though!
Poisson Bridge on ${[0,T]}$ :we condition a Poisson process of rate ${\lambda}$ on the event ${X_T=N}$ . The intensity matrix is simply ${Q(k,k+1)=\lambda=-Q(k,k)}$ and ${0}$ everywhere else while the transition probabilities are given by ${p(t,T,k,N) = e^{-\lambda (T-t)} \frac{(\lambda (T-t) )^{N-k}}{(N-k)!}}$ . This is why at time ${t}$ , the intensity from ${k}$ to ${k+1}$ is given by
$\displaystyle \lambda(t,k,k+1) = \frac{N-k}{T-t}.$

Again, that might not be the most efficient way to simulate a Poisson Bridge ! Notice how the intensity ${\lambda}$ has disappeared …
Ornstein-Uhlenbeck Bridge:Let’s consider the usual OU process given by the dynamic ${dX_t = -X_t + \sqrt{2}dW_t}$ : the invariant probability is the usual centred Gaussian distribution. Say that we want to know how does such an OU process behave if we condition on the event ${X_T = z}$ . Because ${p(t,T,x,z) \propto \exp(-\frac{|z-e^{-(T-t)}x|^2}{2(1-e^{-2(T-t)})^2})}$ we find that the conditioned O-U process follows the SDE
$\displaystyle dX_t = \Big(\frac{z-e^{-(T-t)x}}{e^{T-t}(1-e^{-2(T-t)})^2} - X_t \Big)\, dt+ \sqrt{2} \, dW_t.$

If we Taylor expand the additonal drift, it can be seen that this term behaves exactly as in the case of the Brownian bridge. Below is a plot of an O-U process conditioned on the event ${X_{10} = 10}$ , starting from ${X_0=0}$ .

conditioned O-U process

Like this:

Be the first to like this post.

3 Comments

TheBridge said,

August 6, 2010 at 8:45 am

Hi Alekk,

A brillant article indeed, thank’s alekk,

About the diffusion case I think it would be interesting to connect things with Girsanov Theorem (to check which kind of transform correspond to h-transform).

Another point is that you can check that for brownian bridge with the same start and end point on the time interval [0,1], we have some kind of symmetry for the law of the paths around point t=1/2, this is in some way some kind of reflection principle (in the time coordinate). And I was wondering how general this fact is (which might be use for simulation purposes).

Best Regards

Reply
Nick said,

September 25, 2010 at 7:18 am

Nice and intuitive notes. Thank you

Reply
sampling conditioned Markov chain, and diffusions « Journey into Randomness said,

November 7, 2010 at 3:45 pm

[...] Markov chain, which are usually not available: this has been discussed in this previous post on Doob h-transforms. Perhaps surprisingly, this article by Michael Sorensen and Mogens Bladt shows [...]

Reply

Journey into Randomness