Hitting and Occupation Times
Hitting and Occupation Times¶
In this chapter we will study some random times and random amounts of time associated with Markov Chains. We will start with hitting times. A hitting time is the first time a chain reaches a state after time 0. For any state $j$ that the chain is certain to reach, define the first hitting time of $j$ as
$$ W_j = \inf \{n \ge 1: X_n = j \} $$That's $W$ for "waiting time"; we are avoiding $H$ for "hitting" and $T$ for "time" because those tend to get confused with heads and tails.
We encountered such times in an earlier chapter where we were finding the expected number of tosses of a coin till a particular patter of heads and tails appeared. In the next section we will frame that calculation in terms of Markov Chains. This will allow us to set up a general method for finding waiting times for patters in i.i.d. sequences.
Then we will study hitting times for Markov Chains that have stationary distributions. We will follow that by an examination of occupation times, that is, the amounts of time spent in various states. For irreducible, aperiodic, finite state Markov Chains, we already know that the expected long run proportion of time spent at state $j$ is $\pi(j)$. In this chapter we will connect the stationary distribution $\pi$ to other occupation times as well.