4.1 Discrete Random Variables and Probability Distributions

Random Variables & Probability Distributions

Introduction to Random Variables (RV)

A random variable (RV) is a function that assigns a real number to each outcome in a sample space. Random variables are fundamental in probability and statistics, providing a practical way to quantify outcomes of random experiments.

• Sample Space (S): The set of all possible outcomes of an experiment.

• Random Variable (X): A function mapping outcomes in S to real numbers, typically denoted by capital letters such as X, Y, or Z.

Example:

• Experiment: Observe the first car you see in the morning.

• Sample space S: All possible cars in Davis.

• Random variable X: A measurable attribute of the car (e.g., number of people inside, amount of gas, or color).

Why use random variables?

• The sample space S is often abstract or complex, while random variables provide a clearer, more practical description of outcomes.

• Random variables are usually the focus of statistical analysis.

Types of Random Variables

• Discrete Random Variable: Takes values from a countable set (e.g., integers).

• Continuous Random Variable: Takes values from an interval of real numbers.

Examples:

• Throwing a fair die: X = number shown (discrete).

• Flipping two coins: Y = number of heads (discrete).

• Number of bike accidents in a month: Z (discrete).

• Unemployment rate: W (continuous).

Probability Distributions

Probability Mass Function (PMF) and Probability Density Function (PDF)

The probability distribution of a random variable describes the likelihood of each possible value the variable can take.

• Discrete RV: Described by a probability mass function (PMF):

$p_X(x) = P(X = x)$

• Continuous RV: Described by a probability density function (PDF):

$P(a \leq X \leq b) = \int_a^b f_X(x) dx$

• Cumulative Distribution Function (CDF): For any RV X, the CDF is defined as:

$F_X(x) := P(X \leq x)$

Properties of the PMF

• For any function p(x) to be a valid PMF, it must satisfy:

  • $0 \leq p(x) \leq 1$ for every x

  • $\sum_x p(x) = 1$

Examples of PMFs

Example 1: Number of heads in 2 flips of a fair coin (Binomial distribution):

x	0	1	2
$p_X(x)$	1/4	1/2	1/4

Example 2: Number of flips until the first head (Geometric distribution):

y	1	2	3	...
$p_Y(y)$	1/2	1/4	1/8	...

Verifying a PMF

To check if a function is a valid PMF:

• All probabilities must be nonnegative and not exceed 1.

• The sum of all probabilities must be 1.

Example:

x	-1	0	1
p(x)	0	1/2	3/4

This is not a valid PMF because $\sum_x p(x) = 5/4 > 1$.

Example (Finding a missing probability):

x	0	1	2	3
p(x)	0.2	?	0.4	0.1

To find $p(1)$: $p(1) = 1 - (0.2 + 0.4 + 0.1) = 0.3$

Normalizing a PMF

Sometimes, a PMF is given up to a constant. To find the normalizing constant, set the sum of probabilities to 1 and solve for the constant.

Example: $P(X = x) = c(x-1)$ for $x = 2, 3, 4$

• $P(X=2) = c$, $P(X=3) = 2c$, $P(X=4) = 3c$

• Set $c + 2c + 3c = 1 \implies 6c = 1 \implies c = 1/6$

Identically Distributed Random Variables

Two random variables are identically distributed if they have the same PMF, even if they arise from different experiments.

Example: The number of heads in two coin flips and the number of tails in two coin flips have the same PMF.

Cumulative Distribution Function (CDF)

The cumulative distribution function (CDF) gives the probability that a random variable is less than or equal to a certain value.

$F_X(x) = P(X \leq x) = \sum_{z \leq x} p_X(z)$

• The CDF is non-decreasing and right-continuous.

• The CDF completely determines the distribution of a random variable.

Example: For a PMF $p_X(x)$ with $p_X(1) = 1/3$, $p_X(2) = 2/3$:

x	1	2
$p_X(x)$	1/3	2/3

x	$F_X(x)$
$x < 1$	0
$1 \leq x < 2$	1/3
$x \geq 2$	1

Expected Value

Definition and Properties

The expected value (or mean) of a discrete random variable X is the long-run average value of X over many repetitions of the experiment.

$E[X] = \sum_x x p_X(x)$

• Also called the population mean ($\mu_X$).

• The sample mean ($\bar{X}$) is an estimator of the population mean.

Example: If you flip a fair coin and win $10 for heads and lose $10 for tails, the expected gain is:

x	-10	10
$p_X(x)$	1/2	1/2

$E[X] = (-10) \times 1/2 + 10 \times 1/2 = 0$

Example (Insurance): An insurance company offers a $1,000 policy for a $14 premium. Probability of payout is 0.006.

x	-986	14
$p_X(x)$	0.006	0.994

$E[X] = (-986) \times 0.006 + 14 \times 0.994 = 8$

Properties of Expected Value

• For constants a and b:

• $E[aX + b] = aE[X] + b$

• $E[aX + bY] = aE[X] + bE[Y]$

• For any function f(X): $E[f(X)] = \sum_x f(x) p_X(x)$

• In general, $E[f(X)] \neq f(E[X])$ unless f is linear.

Variance and Standard Deviation

Definition and Calculation

The variance of a random variable X measures the spread of its distribution around the mean.

$Var(X) = E[(X - \mu_X)^2]$

The standard deviation is the square root of the variance:

$sd(X) = \sqrt{Var(X)}$

• Variance is in squared units; standard deviation is in the same units as X.

• Sample variance ($s^2$) is an estimator of the population variance.

Alternative formula for variance:

$Var(X) = E[X^2] - (E[X])^2$

Properties of Variance

• For constants a and b:

• $Var(aX + b) = a^2 Var(X)$

• $sd(aX + b) = |a| sd(X)$

• If $Var(X) = 0$, then X is constant with probability 1.

Summary Table: Key Concepts

Concept	Discrete RV	Continuous RV
Probability Function	PMF: $p_X(x) = P(X = x)$	PDF: $f_X(x)$
Cumulative Distribution	$F_X(x) = P(X \leq x)$	$F_X(x) = P(X \leq x)$
Expected Value	$E[X] = \sum_x x p_X(x)$	$E[X] = \int_{-\infty}^{\infty} x f_X(x) dx$
Variance	$Var(X) = E[(X - \mu_X)^2]$	$Var(X) = E[(X - \mu_X)^2]$

Additional info: Some content and examples were expanded for clarity and completeness, including the insurance example and the summary table.