1 Introduction
A stochastic process is a model for a time-dependent random phenomenon. So, just as a single random variable describes a static random phenomenon, a stochastic process is a collection of random variables Xt , one for each time t in some set J. The set of values that the random variables Xt are capable of taking is called the state space of the process, S.
The first choice that one faces when selecting a stochastic process to model a real life situation is that of the nature (discrete or continuous) of the time set J and of the state space S.
Example 1.1: Discrete state spaces with discrete time changes
A motor insurance company reviews the status of its customers yearly. Three levels of discount are possible (0, 25%, 40%) depending on the accident record of the driver. In this case the appropriate state space is S = {0, 25, 40} and the time set is J = {0, 1, 2, ...} where each interval represents a year. This problem is studied in Unit 3.
…
