4.2 Discrete Random Variables and Probability Distributions

Discrete Random Variables & Probability Distributions

Introduction to Random Variables (RV)

A random variable (RV) is a function that assigns a real number to each outcome in a sample space, providing a practical way to describe and analyze random phenomena. In statistics, RVs are essential for quantifying outcomes and facilitating probability calculations.

• Definition: An RV is a function $X: S \rightarrow \mathbb{R}$, where $S$ is the sample space.

• Notation: Capital letters such as X, Y, or Z are commonly used to denote random variables.

• Examples:

  • Throwing a fair die: $X$ is the number obtained.

  • Flipping two coins: $Y$ is the number of heads.

  • Counting bike accidents in a month: $Z$ is the number of accidents.

  • Measuring unemployment rate: $W$ is the rate.

Discrete random variables take countable values (e.g., integers), while continuous random variables take any real value within an interval.

Probability Distributions

Probability Mass Function (PMF) and Probability Density Function (PDF)

The probability distribution of a random variable describes the likelihood of each possible outcome.

• Discrete RV: Use the probability mass function (PMF): $p_X(x) = P(X = x)$

• Continuous RV: Use the probability density function (PDF): $\int_a^b f_X(x)dx = P(a \leq X \leq b)$

• Cumulative Distribution Function (CDF): $F_X(x) := P(X \leq x)$

The CDF accumulates probabilities up to a given value and is defined for both discrete and continuous RVs.

Discrete Random Variable

Probability Mass Function (PMF)

The PMF assigns probabilities to each possible value of a discrete random variable.

• Example: Drawing a ball from a box with balls labeled 1, 2, and 2.

  x	$p_X(x)$
  1	1/3
  2	2/3

  Function form:

  $p_X(x) = \begin{cases} 1/3 & \text{if } x=1, \\ 2/3 & \text{if } x=2, \\ 0 & \text{otherwise} \end{cases}$

Properties of PMF

• $0 \leq p(x) \leq 1$ for every $x$

• $\sum_x p(x) = 1$

These properties ensure that the PMF is a valid probability distribution.

Examples of Valid PMFs

• Binomial Distribution: Number of heads in 2 coin flips

  x	$p_X(x)$
  0	1/4
  1	1/2
  2	1/4

• Geometric Distribution: Number of flips until first head

  y	$p_Y(y)$
  1	1/2
  2	1/4
  3	1/8
  ...	...

Verifying a PMF

To verify a PMF, check non-negativity and that the probabilities sum to 1.

• Example:

  x	$p(x)$
  0	0.2
  1	?
  2	0.4
  3	0.1

  Solution: $p(1) = 1 - (0.2 + 0.4 + 0.1) = 0.3$

• Normalizing constant: If $P(X = x) = c(x-1)$ for $x = 2,3,4$, then $c$ is found by solving $c + 2c + 3c = 1 \Rightarrow c = 1/6$.

Cumulative Distribution Function (CDF)

Definition and Properties

The cumulative distribution function (CDF) $F(x)$ gives the probability that the random variable is less than or equal to $x$.

• $F(x) = P(X \leq x)$

• For discrete RVs: $F(x) = \sum_{z \leq x} p(z)$

• The CDF is a step function for discrete RVs, increasing at each possible value.

Example: For $p(x)$ as below:

x	$p(x)$	$F(x)$
0	0.2	0.2
1	0.3	0.5
2	0.4	0.9
3	0.1	1.0

Expected Value (Mean)

Definition and Calculation

The expected value $E[X]$ of a discrete random variable is the weighted average of its possible values, using the PMF as weights.

• $E[X] = \sum_x x p_X(x)$

• Also called the population mean ($\mu_X$)

• For a function $f(X)$: $E[f(X)] = \sum_x f(x) p_X(x)$

Example: If $X$ has PMF:

x	$p_X(x)$
1	1/3
2	2/3

Then $E[X] = 1 \times 1/3 + 2 \times 2/3 = 5/3$

Properties of Expectation

• Linearity: $E[aX + b] = aE[X] + b$

• For independent RVs $X$ and $Y$: $E[aX + bY] = aE[X] + bE[Y]$

Variance and Standard Deviation

Definition and Calculation

Variance measures the spread of a random variable around its mean.

• $Var(X) = E[(X - \mu_X)^2]$

• Alternative formula: $Var(X) = E[X^2] - (E[X])^2$

• Standard deviation: $sd(X) = \sqrt{Var(X)}$

Example: If $E[X] = 5/3$ and $E[X^2] = 3$, then $Var(X) = 3 - (5/3)^2 = 3 - 25/9 = 2/9$

Properties of Variance

• $Var(X) \geq 0$

• $Var(aX + b) = a^2 Var(X)$

• If $X$ and $Y$ are independent: $Var(X + Y) = Var(X) + Var(Y)$

Conditional Distribution

Conditional PMF

The conditional probability mass function describes the probability of outcomes given a condition.

• $p(x | A) = \frac{p(x \cap A)}{P(A)}$

• For $p(x | X > 2)$, restrict $x$ to values greater than 2 and renormalize so probabilities sum to 1.

Example: If $p(x)$ for $x = 0,1,2,3$ is $0.2, 0.3, 0.4, 0.1$, then for $X > 1$:

x	$p(x | X > 1)$
2	$0.4 / (0.4 + 0.1) = 0.8$
3	$0.1 / (0.4 + 0.1) = 0.2$

Conditional Expectation and Variance

• $E[X | A] = \sum_x x p(x | A)$

• $Var(X | A) = E[X^2 | A] - (E[X | A])^2$

Example: For light bulbs lasting 4 or 5 years, with conditional PMF $p(x | X > 3)$, compute $E[X | X > 3]$ and $Var(X | X > 3)$ using the formulas above.

References

• Telhammer, R. C. (2013). Mathematical Statistics for Economics and Business. Springer.