Section 2: Summarizing random variables

R, R Markdown, LaTeX

• Questions, problems, or concerns?
• Please attempt to have R and R Markdown running by section next week (you still do not need to use R or Markdown for this week’s problem set).
• Feel free to stop by office hours, set up an individual appointment, or drop by the Stat Lab if you are having trouble.

Summarizing random variables

• Definition of a random variable: a variable that takes on a real value that is determined by a random generative process.
• A random variable is neither random nor a variable.
• Placeholder for a quantity that has yet to be determine by a random generative process (function).
• A random variable is a function.
• A random variable’s possible values might represent the possible outcomes of a yet-to-be-performed experiment, the results of a random process such as rolling a die, or the “subjective” randomness that results from incomplete knowledge of a quantity.
• How can we summarize random variables?

Probability Mass Functions

• Summarizes the probability of each outcome \(x\) occuring.
• A function that describes the probability that a discrete random variable is exactly equal to some value. The PMF maps possible outcomes of a random variable to the corresponding probabilities of that outcome occuring.
• How much “stuff” is there associated with a given event?
• All probabilities add up to 1.

Example formalization

Below is the probability mass function of an unfair die, where we observe a 1, 2, or 3 with probability \(\frac{1}{12}\), and a 4, 5, or 6 with probability \(\frac{1}{4}\). The PMF can therefore be defined as:

\[\begin{equation} f(x) = \begin{cases} \frac{1}{12} & : x = 1 \\ \frac{1}{12} & : x = 2 \\ \frac{1}{12} & : x = 3 \\ \frac{1}{4} & : x = 4 \\ \frac{1}{4} & : x = 5 \\ \frac{1}{4} & : x = 6 \\ 0 & : otherwise \\ \end{cases} \end{equation}\]

What is \(Pr[X \geq 2]\)?

\[Pr[X \geq 2] = \] \[\sum_{x = 2}^6 f(x) = \] \[\frac{1}{12} + \frac{1}{12} + \frac{1}{4} + \frac{1}{4} + \frac{1}{4} = \] \[\frac{11}{12}\]

Example visualization

Cumulative Distribution Functions

• Describes the distribution of all random variables, not just discrete random variables (i.e. can be discrete or continuous).
• Describes the probability that a random variable \(X\) will take a value less than or equal to \(x\).
• How much “stuff” is there to the left of a point?

Example visualizations

• Discrete CDF will have “jumps” or “steps.”
• Unfair die again:

• Continuous CDF is smooth. Example: standard normal distribution.

\[ \Phi(x) = \int_{-\infty}^{x} \frac{1}{\sqrt{2\pi}}e^\frac{-u^2}{2}du \]

Probability Density Functions

• The PDF specifies the probability of a random variable falling within a particular range of values (sand), as opposed to taking on any one value (sticks). The PDF therefore allows us to answer the question: how much of the distribution of a random variable is found in the filled area? In other words, how much probability mass is there between observations in a given range?
• The probability of a random variable falling within a particular range of values is therefore given by the integral of this variable’s PDF over that range.
• What is the absolute likelihood a continuous random variable takes on any particular value? Why?
• The probability density function is nonnegative everywhere, and its integral over the entire space is equal to one.
• The PDF is the derivative of the CDF. This is why the the slope of the CDF is greatest at the highest point of the PDF.

Example visualization

Practice Problems

1.) Suppose you are conducting a survey about access to education in a village with 100 families. 30 families have 1 child, 50 families have 2 children, and 20 families have 3 children. The birth rank of one of these children is 1 if the child is the firstborn, 2 if the child is the second born, and 3 if the child is the third born.

• A random family is chosen (with equal probabilities), and then a random child within that family is chosen (with equal probabilities) to be interviewed. Find the PMF of the child’s birth rank.

• A random child is chosen in the town (with equal probabilities). Find the PMF and draw the CDF of the childs birth rank.

Part 1 solution

\[f(1)=0.3∗1+0.5∗0.5+0.2∗(1/3)≈0.617\] \[f(2)=0.3∗0+0.5∗0.5+0.2∗(1/3)≈0.317\] \[f(3)=0.3∗0+0.5∗0+0.2∗(1/3)≈0.067\]

\[\begin{equation} f(x) = \begin{cases} 0.617 & : x = 1 \\ 0.317 & : x = 2 \\ 0.067 & : x = 3 \\ 0 & : otherwise \\ \end{cases} \end{equation}\]

Part 2 solution

PMF of Y, birth rank under individual level sampling:

\[\begin{equation} f(x) = \begin{cases} \frac{100}{190} & : x = 1 \\ \frac{70}{190} & : x = 2 \\ \frac{20}{190} & : x = 3 \\ 0 & : otherwise \\ \end{cases} \end{equation}\]

2.) Let \(X\) be a random variable with CDF \(F\), let \(Y = a + b X\), with \(a\) and \(b\) real, \(> 0\). The CDF \(X\) is \(F_x\) and the CDF of \(Y\) is \(F_y\). What is the CDF of \(F_y\) in terms of \(F_x\)?

Solution

The CDF of \(Y\) at a point \(y\) is the probability that \(Y\) takes on any value less than or equal to \(y\), or \(Pr[Y \leq y]\)

In terms of \(F_x\), the CDF of \(Y\) is:

\[F_y = Pr[Y \leq y]=\] \[Pr[a+bX \leq y] = \] \[Pr[X \leq \frac{y − a}{b}]\]

\[= F_x(\frac{y − a}{b})\]

CDFs are functions that can be manipulated.

3.) A marksman takes 10 shots at a target and has probability 0.2 of hitting the target with each shot, independently of all other shots. Let \(X\) be the number of hits. Calculate and sketch the PMF of X.

Hint: The binomial distribution is a discrete probability distribution of the successes in a sequence of \(n\) independent yes/no outcomes, and the probability of getting exactly \(k\) successes in \(n\) trials is given by the Probability Mass Function:

\[P(X = k) = {n \choose k}p^k(1-p)^{n-k} \]

In class we discussed the geometric distrubution \(Pr(X = k) = (1-p)^{k-1}p\), which describes the number of successes \(k\) until observing the first “failure.”

Solution

\(X\) is a binomial random variable with \(n = 10\), \(p = 0.2\). Therefore,

\[\begin{equation} P(X = k) = \begin{cases} {n \choose k}0.2^k 0.8^{10-k} & : x \in ℕ \\ 0 & : otherwise \\ \end{cases} \end{equation}\]

Bivariate distributions

Discrete marginal and conditional distributions

• The joint PMF shows us the probability that \(X=x\) and \(Y=y\) in a single realization of the random generative process.
• The joint CDF shows us the probability of observing \(X \leq x\) and \(Y \leq y\) in a single realization of the random generative process.

Example:

Assume some random vector \((X, Y)\) is characterized by the following joint PMF:

\[\begin{equation} f(x,y) = \begin{cases} \frac{1}{3} & : x = 0, y = 0 \\ \frac{1}{3} & : x = 0, y = 1 \\ \frac{1}{6} & : x = 1, y = 1 \\ \frac{1}{6} & : x = 1, y = 2 \\ 0 & : otherwise \\ \end{cases} \end{equation}\]

Compute the following:

a.) \(Pr[X = 1, Y = 1]\)
b.) \(Pr[X = 1 | Y = 1]\)
c.) Supp\([X]\)
d.) The marginal PMF of \(Y\)

Solutions

	\(X = 0\)	\(X = 1\)	Marginal
\(Y = 0\)	\(\frac{2}{6}\)	\(0\)	\(\frac{2}{6}\)
\(Y = 1\)	\(\frac{2}{6}\)	\(\frac{1}{6}\)	\(\frac{3}{6}\)
\(Y = 2\)	\(0\)	\(\frac{1}{6}\)	\(\frac{1}{6}\)
Marginal	\(\frac{4}{6}\)	\(\frac{2}{6}\)

a.) \(\frac{1}{6}\). This is given to us in the joint PMF.

b.) Conditional probability = joint / marginal or \(Pr[X = x | Y = y] = \frac{Pr[X = x, Y = y]}{Pr[Y = y]}\), so:

\[Pr[X = 1| Y = 1] =\] \[\frac{Pr[X = 1, Y = 1]}{Pr[Y = 1]} =\]

\[\frac{\frac{1}{6}}{\frac{3}{6}} =\]

\[\frac{1}{3}\]

c.) Supp[X] = \(\{0, 1\}\)

d.)

\[\begin{equation} f_Y(y) = \begin{cases} \frac{1}{3} & : y = 0 \\ \frac{1}{2} & : y = 1 \\ \frac{1}{6} & : y = 2 \\ 0 & : otherwise \\ \end{cases} \end{equation}\]

Jointly continuous random variables

Effectively the same as the discrete case, except now instead of summing we are integrating.

Example: Consider the bivariate distribution whose joint PDF is:

\[f(x,y) = \frac{3(x^2 + y)}{11}, \forall x \in [0,2], y \in [0,1]\] with \(f(x,y) = 0\) elsewhere. This joint PDF is plotted below and is sometimes referred to as the “curved roof distribution.”

a.) Show that the marginal PDF of X is \(f_X(x) = \frac{3(2x^2 + 1)}{22}, ∀x ∈ [0, 2]\) with \(f_X(x) = 0\) elsewhere.

b.) Derive \(f_{Y | X}(y|x)\), the conditional PDF of \(Y\) given \(X\), \(∀x ∈ [0,2]\).

c.) Find \(Pr[Y \leq 0.5|X = 1]\).

Solutions:

a.) Marginal PDF of Y of a jointly continuous random variable is: \(f_Y(y) = \int_{-\infty}^{\infty} f(x,y)dx, \forall y \in ℝ\), so:

\[f_X(x) = \int_{0}^1 \left(\frac{3}{11}x^2 + \frac{3}{11}y \right)dy = \] \[\frac{3}{11} \int_{0}^1 (x^2 + y)dy = \] \[\frac{3}{11} (x^2y + \frac{1}{2}y^2)|_{0}^1 = \]

\[\frac{3}{11} (x^2 + \frac{1}{2} - 0) = \] \[\frac{3x^2}{11} + \frac{3}{22} =\]

\[\frac{6x^2 + 3}{22} = \] \[\frac{3(2x^2 + 1)}{22} = \]

b.) Derive \(f_{Y | X}(y|x)\), the conditional PDF of \(Y\) given \(X\), \(∀x ∈ [0,2]\).

\[f_{Y | X}(y|x) = \frac{f(x, y)}{f_X(x)}, \forall x \in [0,2] = \]

\[ \frac{\frac{3(x^2 + y)}{11}}{\frac{3(2x^2 + 1)}{22}} = \] \[ 2 \left( \frac{x^2 + y}{2x^2 + 1} \right)= \] \[ \frac{2x^2 + 2y}{2x^2 + 1} \]

c.) Now that we know the conditional PDF of Y given X from above:

\[f_{Y|X=1} = \int_{0}^{0.5} \frac{2(1)^2 + 2y}{2(1)^2 + 1} \] \[f_{Y|X=1} = \int_{0}^{0.5} \frac{2 + 2y}{3} = \] \[f_{Y|X=1} = \frac{2}{3} \int_{0}^{0.5} (1 +y)dy = \]

\[\frac{2}{3}\left[ y + \frac{1}{2}y^2 \right] |_0^{0.5}\]

\[\frac{2}{3}\left[ \frac{1}{2} + \frac{1}{2} \times \frac{1}{4} \right] = \]

\[\frac{5}{12}\]