L4.2 Discrete Random Variables and Probability Distributions: STA103 Lecture 4 Study Notes

Discrete Random Variables & Probability Distributions

Introduction to Random Variables (RV)

Random variables are fundamental concepts in probability and statistics, providing a practical way to describe outcomes of random experiments. Instead of working directly with sample spaces and events, random variables allow us to assign numerical values to outcomes, facilitating analysis and interpretation.

• Definition: A random variable (RV) is a function that maps outcomes from a sample space $S$ to real numbers: $X: S \rightarrow \mathbb{R}$.

• Notation: Capital letters such as $X$, $Y$, or $Z$ are typically used to denote random variables.

• Examples:

  • Throwing a fair die: $X$ is the number obtained.

  • Flipping two fair coins: $Y$ is the number of heads.

  • Counting bike accidents in a month: $Z$ is the number of accidents.

  • Measuring unemployment rate: $W$ is the rate.

Discrete random variables take values from a countable set (e.g., integers), while continuous random variables can take any value within an interval of real numbers.

Probability Distributions

Probability distributions describe the likelihood of different outcomes for a random variable. The type of function used depends on whether the variable is discrete or continuous.

• Probability Mass Function (PMF): For discrete RV $X$, the PMF is $p_X(x) = P(X = x)$.

• Probability Density Function (PDF): For continuous RV $X$, the PDF is $f_X(x)$, and probabilities are computed as $P(a \leq X \leq b) = \int_a^b f_X(x) dx$.

• Cumulative Distribution Function (CDF): For any RV $X$, the CDF is $F_X(x) = P(X \leq x)$.

Discrete Random Variable

Probability Mass Function (PMF)

The PMF assigns probabilities to each possible value of a discrete random variable.

• Example: Suppose a box contains three balls labeled 1, 2, and 2. Let $X$ be the number written on the ball drawn at random.

x	$p_X(x)$
1	$\frac{1}{3}$
2	$\frac{2}{3}$

Function form:

• $p_X(x) = \begin{cases} \frac{1}{3} & \text{if } x = 1, \\ \frac{2}{3} & \text{if } x = 2, \\ 0 & \text{otherwise} \end{cases}$

Properties of PMF

Not every function is a valid PMF. The following properties must be satisfied:

• $0 \leq p(x) \leq 1$ for every $x$

• $\sum_x p(x) = 1$

Examples of Valid PMFs

x	$p_X(x)$
0	$\frac{1}{4}$
1	$\frac{1}{2}$
2	$\frac{1}{4}$

This PMF describes the number of heads in two flips of a fair coin (Binomial distribution).

y	$p_Y(y)$
1	$\frac{1}{2}$
2	$\frac{1}{4}$
3	$\frac{1}{8}$

This PMF describes the number of flips until the first head (Geometric distribution).

Verifying a PMF

To verify if a function is a PMF, check the two properties above. For example:

x	$p(x)$
-1	0
0	$\frac{1}{2}$
1	$\frac{3}{4}$

Here, $\sum_x p(x) = \frac{1}{2} + \frac{3}{4} = \frac{5}{4} > 1$, so this is not a valid PMF.

For $p(x) = \frac{x}{15}$ for $x = 1,2,3,4,5$, $\sum_x p(x) = 1$, so this is a valid PMF.

To find a missing probability so that the PMF sums to 1:

x	$p(x)$
0	0.2
1	?
2	0.4
3	0.1

$p(1) = 1 - (0.2 + 0.4 + 0.1) = 0.3$

Normalizing Constant in PMF

Sometimes, a PMF is defined up to a constant $c$ that must be determined so the total probability is 1.

• Given $P(X = x) = c(x-1)$ for $x = 2,3,4$, find $c$:

$P(X=2)=c$, $P(X=3)=2c$, $P(X=4)=3c$

$c + 2c + 3c = 6c = 1 \implies c = \frac{1}{6}$

Expectation (Expected Value)

Definition and Calculation

The expected value $E[X]$ of a discrete random variable $X$ is the weighted average of its possible values, using the PMF as weights.

• $E[X] = \sum_x x p_X(x)$

• Also called the population mean $\mu_X$

• For a sample, the sample mean is $\bar{X} = \frac{1}{n} \sum_{i=1}^n X_i$

Interpretation: $E[X]$ is the long-run average value of $X$ over repeated experiments.

Examples

• Coin Flip Game: Win $10 if heads, lose $10 if tails. PMF:$x = -10, 10$,$p_X(x) = 0.5, 0.5$

• $E[X] = (-10) \times 0.5 + 10 \times 0.5 = 0$

• Insurance Example: Gain $14 if person survives, lose $986 if person dies.$p_X(x) = 0.994, 0.006$

• $E[X] = (-986) \times 0.006 + 14 \times 0.994 = 5$

Linearity of Expectation

• For constants $a, b$ and RVs $X, Y$:

$E[aX + b] = aE[X] + b$

$E[aX + bY] = aE[X] + bE[Y]$

Expectation of Functions of RVs

• For any function $f$ and discrete RV $X$:

$E[f(X)] = \sum_x f(x) p_X(x)$

Note: $E[f(X)] \neq f(E[X])$ in general.

Variance and Standard Deviation

Definition

Variance measures the spread of a random variable around its mean.

• $Var(X) = E[(X - \mu_X)^2]$

• Standard deviation: $sd(X) = \sqrt{Var(X)}$

• Variance is in squared units; standard deviation is in original units.

Calculation

• Alternative formula: $Var(X) = E[X^2] - (E[X])^2$

• Compute $E[X^2]$ using the PMF: $E[X^2] = \sum_x x^2 p_X(x)$

Properties of Variance

• $Var(X) \geq 0$

• If $Var(X) = 0$, then $X$ is constant.

• $Var(X + b) = Var(X)$

• $Var(aX + b) = a^2 Var(X)$

• $sd(aX + b) = |a| sd(X)$

• If $X$ and $Y$ are independent: $Var(X + Y) = Var(X) + Var(Y)$

• In general: $Var(X + Y) = Var(X) + Var(Y) + 2 Cov(X, Y)$

Conditional Distributions

Conditional PMF

The conditional probability mass function describes the probability of $X = x$ given some condition $A$.

• $p(x | A) = \frac{P(X = x \cap A)}{P(A)}$

• For example, $p(x | X > 2)$ is the probability that $X = x$ given $X > 2$.

Conditional Expectation and Variance

• Once the conditional PMF is known, compute expectation and variance as usual:

$E[X | A] = \sum_x x p(x | A)$

$Var(X | A) = E[X^2 | A] - (E[X | A])^2$

Interpretation: $E[X | A]$ is the updated best prediction of $X$ given $A$; $Var(X | A)$ is the variability of $X$ within $A$.

Example: Light Bulb Lifetimes

Suppose $X$ is the lifetime (in years) of a light bulb, with PMF:

x	$p(x)$
1	0.1
2	0.2
3	0.3
4	0.25
5	0.15

• Conditional PMF for $X > 3$:

x	$p(x | X > 3)$
4	0.625
5	0.375

• Conditional expectation: $E[X | X > 3] = 4 \times 0.625 + 5 \times 0.375 = 4.375$

• Conditional variance: $Var(X | X > 3) = E[X^2 | X > 3] - (E[X | X > 3])^2$

Summary Table: Key Concepts

Concept	Discrete RV	Continuous RV
Probability Function	PMF: $p_X(x)$	PDF: $f_X(x)$
Cumulative Function	CDF: $F_X(x) = P(X \leq x)$	CDF: $F_X(x) = P(X \leq x)$
Expectation	$E[X] = \sum_x x p_X(x)$	$E[X] = \int_{-\infty}^{\infty} x f_X(x) dx$
Variance	$Var(X) = E[X^2] - (E[X])^2$	$Var(X) = E[X^2] - (E[X])^2$

References: Mittelhammer, R. C. (2013). Mathematical Statistics for Economics and Business. Springer.