Derive and apply Kolmogorov's forward equations for continuous time Markov multiple state models.

A disability income policy needs transition probabilities at every duration, not just annual snapshots. Kolmogorov's forward equations turn a diagram of transition intensities into a solvable system of ordinary differential equations.

A multi-state Markov model has a finite state space (labeled 0, 1, 2,...) and the Markov property: future evolution depends only on the current state. The model is fully specified by transition intensities.

$${\mu }_{x+t}^{ij}=\underset{h\to 0^{+}}{\lim }\frac{{}_h{p}_{x+t}^{ij}}{h},i\ne j.$$

Total outflow from state $i$ is ${\sum }_{j\ne i}{\mu }_{x+t}^{ij}$. Absorbing states (typically "dead") have no outgoing intensities at all.

Split the interval $[0,t+h]$ at $t$ using Chapman-Kolmogorov:

$${}_{t+h}{p}_x^{ij}=\sum _k{}_t{p}_x^{ik}\cdot {}_h{p}_{x+t}^{kj}.$$

For small $h$, ${}_h{p}_{x+t}^{kj}\approx h{\mu }_{x+t}^{kj}$ for $k\ne j$, and ${}_h{p}_{x+t}^{jj}\approx 1-h{\sum }_{m\ne j}{\mu }_{x+t}^{jm}$. Substitute, subtract ${}_t{p}_x^{ij}$, divide by $h$, and let $h\to 0$:

Common mistakes

• Flipping the sign on outflow. Outflow from $j$ reduces ${}_t{p}_x^{ij}$, so it enters with a minus sign. A positive outflow term is a frequent trap.
• Using ${\mu }_x^{ij}$ instead of ${\mu }_{x+t}^{ij}$. Intensities advance with age. Evaluating at the start age throughout silently forces time-homogeneity that the model rarely has.
• Confusing forward with backward. Forward conditions on the end state and last instant. Backward conditions on the start state and first instant. The ODE indices and inflow direction differ.

Bottom line

• A continuous-time Markov multi-state model is defined by transition intensities ${\mu }_{x+t}^{ij}$, the instantaneous rate of moving from state $i$ to state $j$
• The forward equation tracks how ${}_t{p}_x^{ij}$ changes as inflow to state $j$ minus outflow from $j$; derive it via Chapman-Kolmogorov split plus a first-order intensity expansion
• General form: $\frac{d}{dt}{}_t{p}_x^{ij}={\sum }_{k\ne j}\left[{}_t{p}_x^{ik}{\mu }_{x+t}^{kj}-{}_t{p}_x^{ij}{\mu }_{x+t}^{jk}\right]$
• Boundary values: ${}_0{p}_x^{ii}=1$, ${}_0{p}_x^{ij}=0$ for $i\ne j$, with row sum ${\sum }_j{}_t{p}_x^{ij}=1$ as a sanity check

Exam shortcut

Write each state's forward ODE as "inflow into $j$ minus outflow from $j$." Read the arrows on the state diagram and the equation writes itself. When intensities are constant and the model is acyclic (no return to a state), ${}_t{p}_x^{ii}=e^{-{\sigma }_{i}t}$ where ${\sigma }_i$ is the total outflow from $i$. This bypasses the ODE for diagonal entries. For Euler at step $h$, check ${\sum }_j{}_t{p}_x^{ij}$ every step.

The full lesson (about 6,198 words, 41 min read) adds 5 worked examples, all 15 common mistakes, a self-check, free in the app.

Learning objectives

• 1d

Browse all free Exam ALTAM lessons or jump into free Exam ALTAM practice questions.