# Stochastics: Suggested Problems for Exam 1

Getting things kicked off on the new website. Here is a list of interesting problems to try from Durrett’s book. Please use the comments feature to ask questions and to start discussions about these problems. I’ll chime in whenever I can.

Stationary distributions and limit distributions

1.20, 1.25, 1.38, 1.41, 1.44

Random walks on graphs

1.49-1.51

Hitting times; hitting probabilities

1.59, 1.60

Infinite state spaces

1.70, 1.73

Hey guys! Can someone please explain what problem 1.38(a) is asking for? In general, what is the “long-run fraction of time spent at each state”? I thought it was the stationary distribution but the semantics in this class always confuse me!

Hi Alexa. Any time you read something that says something like “long-run fraction of time spent at each state” think in terms of

asymptotic frequencyand Theorems 1.22 and 1.23. Taken together they say that if a system is irreducible and all states are recurrent, then where is the number of visits to in the first steps and is the stationary distribution.Thank you very much Dr. McKinley, this makes sense!

Is the first part of 1.59 asking for the average hitting time of each these combinations? I got an answer for HHH and HHT, but for HTT and HTH, I’m getting my (I-Ptilda) matrix has a determinant of 0 and the inverse doesn’t exist. Is anyone else having this issue?

You must be computing something wrong? For {HHH,HHT,HTH,HTT,THH,THT,TTH,TTT} I got {14., 8., 10., 8., 8., 10., 8., 14.} . No zero determinants anywhere. But this isn’t actually how it’s supposed to be solved. http://plus.maths.org/content/os/issue55/features/nishiyama/index

Maybe you are doing right multiplication instead of left multiplication by accident.. That is you’re computing instead of .

That should be “=” in the second equation, not “-“

Alexa, for the case of HTH,

. Since the diagonal contains all positive entries, the matrix is invertible and gives a column vector with the entries 8,6,8,10,6,8,10. Thus,

E(tau)=3+1/8(8+6+8+10+6+8+10)=10

Thank you guys so much for all of the help!! I actually went to office hours yesterday and worked it all out. I had my P matrix all wrong and what I have now is what you’ve posted above. Thank you so much for all the help!! 😀

#1.73: Using the result from #1.70 state 1 is recurrent iff . But which converges. So state 1 isn’t recurrent and then by Thm. 1.29 the chain can’t have a stationary distribution. What do you guys think?

Whoops the index should start at 2.