Stats – Stochastic models/Markov chain.
1. Consider the continuous time Markov chain model for a single server queue with a
modification that allows for arrivals in pairs. The arrival rate is A and the service
rate is p, but each arrival, with probability 0, is of a pair of individuals rather than
a single individual. The corresponding continuous time Markov chain model for the
queue size has as its state space the set of all non-negative integers and the following
transition rates:
transition rate
i->i+1 A(1-0) (iZO)
2′ -> i + 2 Ao (i 2 0)
i -> i. – 1 p (i Z 1)
where/\>0,p>oando3031.
(i) \Nrite down an expression for the mean number of individuals who arrive per
unit time. [1]
(ii) Write down a formula for the tmfi‘ic intensity of the queue and explain on
intuitive grounds why the condition
A(1 + 0) < a
should be necessary and sufficient for the existence of an equilibrium distribu-
tion. [2]
(iii) Write down the balance equations for the equilibrium distribution (71} : i Z 0).
Show why the detailed balance equations cannot be valid for this model, except
in the special case 0 = O. [5]
(iv) Prove from the balance equations that the equilibrium distribution, if it exists,
satisfies the difference equation
71} Z /\7Ti-1 + Agfl’i-Q (‘1 Z 2).
W hat additional conditions would you use to determine the equilibrium distri-
bution? [7]
(v) Assuming the existence of an equilibrium distribution, sum the equations of
Part (iv) over i to find an expression for 7m. Deduce that the condition of
Part (ii) emerges again as the condition for the existence of an equilibrium
distribution. [5]
2. In considering the time to emission of a signal, a neuron is modelled as a continuous
time Markov chain {N (t) z t Z 0} with three states, “unblocked” (u) “blocked” (b)
and “emitted” (e). The neuron is assumed to be initially in state u, but with time
it reaches the absorbing state 6. The transition rates are given by:
transition rate
u -> e 1/
u -> b A
b -> e p
where 1/, A and p are strictly positive parameters. N 0 other instantaneous transitions
are possible.
(i) Write down the forward equations for pu(t), pb(t) and pe (t), the probabilities
that at time t the neuron is in state u, b and 8, respectively. [4]
(ii) Deduce expressions for the corresponding Laplace transforms, p; (s), p;(s) and
122(8) [7]
(iii) Let T denote the length of time for the neuron to first. reach state 6 from state
T = inf{t Z 0 : N(t) : 6}.
Show that the distribution function F of T is given by
F“) = Pe(t) (t Z 0)-
[2]
(iv) Deduce that the p.d.f. of T is given by
All 6-1.“ + (V+ A)(V _ M) e-(V+A)t (t > 0)
1/ + A – p 1/ + A – p
