In addition to proving the central limit theorem, Andrey Andreyevich Markov played a pivotal role in developing Markov chains. Markov chains are used to model discrete-time, discrete space random processes with applications across multiple domains including Finance, Advertising, NLP, SEO, Physics, etc. In this blog post, we shall discuss the in and outs of Markov Chains with a specific focus on computing the steady-state probabilities for a given Markov chain.
Recap:
Before we jump into the nitty-gritty of the Markov chain, let us take a moment to define the fundamental concepts of probability theory.