---

Bivariate Distributions, Marginal Distributions, Independent Random Variables, Conditional Distributions, IID Samples §

• Source textbook: “All of Statistics”
• Recall content from: Random Variables, Distribution Functions and Probability Functions

Bivariate Distributions or Joint Mass Function §

Given a pair of discrete random variables and , the Joint mass function is defined as:

• this is notation

Example: Suppose we have the following bivariate distribution for two random variables and each taking values or :

	Y=0	Y=1
X=0	1/9	2/9	1/3
X=1	2/9	4/9	2/3
	1/3	2/3	1
Thus

Probability Density Function in two variables §

In the continuos case, a function is a PDF for the random variables if:

• for all
• for ay set ,

In the discrete or continuos case we define the joint CDF as .

Example Let be uniform on the unit square. Then:

We want to find . The event corresponds to a subset of the unit square.

Integrating over tuhis subset corresponds, in this case, to computing the area of the set which is 1/4. So .

Marginal Distributions §

If have joint distribution with mass function , then the marginal mass function for is defined by:

and the marginal mass function for is defined as:

For continuios random variables the marginal densities are:

and

denoted by and

Independent Random Variables §

Two random variables and are independent if for every and :

Otherwise we say that and are dependent.

Theorem: if and only if for all values and .

Example Let and have the following distribution:

	Y=0	Y=1
X=0	1/4	1/4	1/2
X=1	1/4	1/4	1/2
	1/2	1/2
Then and . and are independent because , , .

Suppose instead that and have the following distribution:

	Y=0	Y=1
X=0	1/2	0	1/2
X=1	0	1/2	1/2
	1/2	1/2	1

These are not independent because yet .

Conditional Distributions §

If and are discrete, then we can compute the conditional distribution of given that we are observed . Specifically, .

This leads us to define the conditional probability mass function as follows:

• if .

For continuos distributions we use the same definitions and we must integrate to get a probability as it is known.

Definition: for continuos random variables, the conditional probability density function is:

assuming that then:

Example: Suppose that Uniform(0,1). After obtaining a value we generate Uniform(x,1).

What is the marginal distribution of ?

Multivariate Distributions and IID samples §

• IID samples are used in ML i.e ML1 - Lecture 2 - iid samples

Let where are random variables. We call a random vector.

Let denote the PDF.

It is possible to define their marginals, conditionals etc much in the same ways as in the bivariate case.

We say that are independent if for every :

It suffices to check that

Definition: If are independent and each has the same marginal distribution with CDF , we say that are IID: independent and identically distributed and we write that:

If has density we also write . We also call a random sample of size from .

Much of statistical theory and practice beginds with IID observations.