Stats – Stochastic models/Markov chain.

1. Consider the continuous time Markov chain model for a single server queue with a

modification that allows for arrivals in pairs. The arrival rate is A and the service

rate is p, but each arrival, with probability 0, is of a pair of individuals rather than

a single individual. The corresponding continuous time Markov chain model for the

queue size has as its state space the set of all non-negative integers and the following

transition rates:

transition rate

i->i+1 A(1-0) (iZO)

2′ -> i + 2 Ao (i 2 0)

i -> i. – 1 p (i Z 1)

where/\>0,p>oando3031.

(i) \Nrite down an expression for the mean number of individuals who arrive per

unit time. [1]

(ii) Write down a formula for the tmfi‘ic intensity of the queue and explain on

intuitive grounds why the condition

A(1 + 0) < a

should be necessary and sufficient for the existence of an equilibrium distribu-

tion. [2]

(iii) Write down the balance equations for the equilibrium distribution (71} : i Z 0).

Show why the detailed balance equations cannot be valid for this model, except

in the special case 0 = O. [5]

(iv) Prove from the balance equations that the equilibrium distribution, if it exists,

satisfies the difference equation

71} Z /\7Ti-1 + Agfl’i-Q (‘1 Z 2).

W hat additional conditions would you use to determine the equilibrium distri-

bution? [7]

(v) Assuming the existence of an equilibrium distribution, sum the equations of

Part (iv) over i to find an expression for 7m. Deduce that the condition of

Part (ii) emerges again as the condition for the existence of an equilibrium

distribution. [5]

2. In considering the time to emission of a signal, a neuron is modelled as a continuous

time Markov chain {N (t) z t Z 0} with three states, “unblocked” (u) “blocked” (b)

and “emitted” (e). The neuron is assumed to be initially in state u, but with time

it reaches the absorbing state 6. The transition rates are given by:

transition rate

u -> e 1/

u -> b A

b -> e p

where 1/, A and p are strictly positive parameters. N 0 other instantaneous transitions

are possible.

(i) Write down the forward equations for pu(t), pb(t) and pe (t), the probabilities

that at time t the neuron is in state u, b and 8, respectively. [4]

(ii) Deduce expressions for the corresponding Laplace transforms, p; (s), p;(s) and

122(8) [7]

(iii) Let T denote the length of time for the neuron to first. reach state 6 from state

T = inf{t Z 0 : N(t) : 6}.

Show that the distribution function F of T is given by

F“) = Pe(t) (t Z 0)-

[2]

(iv) Deduce that the p.d.f. of T is given by

All 6-1.“ + (V+ A)(V _ M) e-(V+A)t (t > 0)

1/ + A – p 1/ + A – p