Lagmental Vicfred

The Fundamental Matrix Sums Visits before Absorption by Vicfred

Last updated: Wed 28 August 2013

tags: mathematics, Markov chains, stationary distributions, random walks

For an absorbing chain, the inverse of I minus Q records expected transient-state visits. I want the notation, the mechanism, and the failure mode visible at the same time.

Definitions first

A discrete-time Markov chain has transition matrix \(P=(p_{ij})\) and forgets the past after conditioning on the present. Matrix powers \(P^n\) give multi-step transition probabilities.

$$ P=\begin{pmatrix}Q&R\\0&I\end{pmatrix} $$

The definition determines which expressions are legal; only then does the identity become meaningful. An equality in \(\mathcal A\) may change ambient meaning, so I keep \(\mathsf D\) separate from \(\mathsf C\).

$$ N=(I-Q)^{-1}=I+Q+Q^2+\cdots $$

A small case in full

This is the algebraic core of the note. Once this line is correct, the surrounding interpretation has something solid to refer to.

$$ t=N\mathbf1,\qquad B=NR,\qquad\begin{cases}t_i=\text{expected absorption time},\\B_{ij}=\text{absorption probability in }j.\end{cases} $$

The reusable statement

The compact alignment is a local map of the argument: assumptions on the first row, consequence on the second. Any generalisation must preserve that dependency.

$$ \begin{aligned} \mathsf{D}\;&:\quad P=\begin{pmatrix}Q&R\\0&I\end{pmatrix},\\[5pt] \mathsf{C}\;&:\quad N=(I-Q)^{-1}=I+Q+Q^2+\cdots. \end{aligned} $$

A nearby false statement

A stationary distribution need not be unique without irreducibility, and convergence to it can fail without aperiodicity.

$$ \boxed{\begin{gathered} \text{compact conclusion}\\[-2pt] N=(I-Q)^{-1}=I+Q+Q^2+\cdots \end{gathered}} $$

I would use the boxed line as a reference later, while returning to the full display whenever a hypothesis becomes uncertain. That division keeps compression from becoming ambiguity.

This article was posted on Sat 15 October 2011. Facts and circumstances may have changed since publication.
Please contact me before jumping to conclusions if something seems wrong or unclear.