A stochastic process is a collection of random variables that are indexed by some mathematical sets. That is, each random variable of the stochastic process is uniquely associated with an element in the set. The set that is used to index the random variables is called the **index set **and the set of random variables forms the state space. A stochastic **process **can be classified in many ways based on state space, index set, etc.
When the stochastic **process **is interpreted as time, if the process has a finite number of elements such as integers, numbers, and natural numbers then it is Discrete Time.
Stochastic Model
It is a discrete-time process indexed at time 1,2,3,…that takes values called states which are observed.
For an example if the states **(S) ={hot , cold }**
_State series over time => _z∈ S_T
Weather for 4 days can be a sequence => **{z1=hot, z2 =cold, z3 =cold, z4 =hot}**
Markov and Hidden Markov models are engineered to handle data which can be represented as ‘sequence’ of observations over time. Hidden Markov models are probabilistic frameworks where the observed data are modeled as a series of outputs generated by one of several (hidden) internal states.
Markov Assumptions
Markov models are developed based on mainly two assumptions.
- Limited Horizon assumption: Probability of being in a state at a time t depend only on the state at the time (t-1).
Eq.1. Limited Horizon Assumption
That means state at time t represents _enough summary _of the past reasonably to predict the future. This assumption is an Order-1 Markov process. An order-k Markov process assumes conditional independence of state z_t from the states that are k + 1-time steps before it.
#hidden-markov-models #sequence-model #machine-learning #computer-science #time-series-model #deep learning
