Stochastic Processes, Markov Chains and Poissons Processes

---

Stochastic Processes

• All of statistics chapter 23

Stochastic Processes are used in sequences where the I.I.D assumption doesn’t hold. For Example: daily temperatures

A stochastic process is a collection of random variables. That can be also written as .

The variables takes values in a set called state space.

I.I.D and Stochastic Proccesses i.i.d random variables are an example of Stochastic Process. In fact, a sequence of i.i.d. random variables can be written as where

Examples of Stochastic Processes

The weather is a classical example also used in Markov Models.

Let {sunny,cloudy}. A typical sequence (depending on where you live) might be:

sunny,sunny,cloudy,sunny,cloudy,cloudy,...

This process has a discrete state space and a discrete index set.

Stock prices is another example of stochastic process, where the index set is continuos and the price is discrete but it can be trated as continuos for practical purposes

Recall that the join density can be written as: where

Markov Chains

• Markov Chain, Markov Models, Hidden Markov Models (HMM)

Poissons Processes

The Poisson process arises when we count occurrences of events over time, for example, traffic accidents, radioactive decay, arrival of email messages, etc.

Preliminaries for Poisson Process and counting process

Recall the Poisson Distribution

Also recall that

• the mean i.e and (as in the example) in a Poisson distribution the mean and the variance have the same value .
• If Poisson() and Poisson() and then Poisson()
• Finally if Poisson() and Binomial(), then the marginal distribution of is Poisson().

Imagine you record each time you get a new message. Let be the number of messages you have received up to and including time . Then, is a stochastic process with state space

A process of this form is called a counting process.

A poisson process is a counting process that satisfies certain conditions.

We need the following notation:

• if as this means that is smaller than when is close to zero. For example

Poisson Process Definition

A Poisson process is a stochastic process with state space such that:

2. For any the increments: are independent
3. There is a function such that:

We call the intensity function. The last condition means that the probability of an event in is approximately while the probability of more than one event is small.

Theorem If is a Poisson process with intensity function then:

• where

In particular, Poisson(). Hence and

Definition: homogenous Poisson process A poisson process with intensity function for some is called a homogenous Poisson process with rate . In this case, Poisson()

Waiting times and sojourn times Let be an homogenous Poisson process with rate . Let be the time at which the event occurs and set . The random variables are called waiting times. Let , then are called sojourn times or interarrival times.

Theorem The sojourn times are IID random variables. Their distribution is exponential with mean that is, they have density:

The waiting time Gamma() i.e. it has density:

Hence, and

Example: Consider requests to a WWW server in Calgary (see Calgary-HTTP).

Let’s assume that this is a homogenous Poisson process, Poisson()

• is the total observation time
• is the total number of events observed

The likelihood is:

maximizing this with respect to gives the standard estimator:

in events per minute.

Let’s test the assumption that the data follow a homogenous Poisson process using a goodness-of-fit test, i.e rate must be constant over time. (see Hypothesis Testing and p-values, chi2 , permutation, likelihood ratio test, multiple testing and goodness-of-fit testing)

We divide the interval into 4 equal lenght intervals . If the process is homogenous, then given the total number of events, the probability that an event fall into any of these intervals must be equal.

Let be the probability of a point being in .

This is the null hypothesis.

We can test this hypothesis using either a likelihood ratio test or a test.

We use the test. Let be the number of events in interval .

• Measures how far observed counts deviate from equal-probability assumptions.

is the expected number under the null.

This yields . The -value is near 0, meaning:

• the chance of seeing such uneven counts under a true homogenous Poisson process is practially zero.
• Therefore, we reject the null hypothesis.

The data aren’t from a homogenous Poisson process. This strongly suggests the arrival rate isn’t constant. Because web traffic is usually time-varying: less during the night, and more during the day.