Lecture 19: Binomial Distribution – Study Notes

Overview of Discrete Probability Distributions

Introduction

This lecture focuses on the Bernoulli and Binomial distributions, which are fundamental discrete probability distributions in statistics. These distributions model experiments with two possible outcomes and are widely used in data analysis, quality control, and inferential statistics.

Review: Random Variables and Probability Distributions

Key Concepts

• Random Variables (RV): Variables whose values are determined by the outcome of a random experiment.

• Probability Distributions: Functions that assign probabilities to each possible value of a random variable.

• Probability Mass Function (PMF): For discrete random variables, the PMF gives the probability that the variable takes a specific value.

• Discrete vs. Continuous Random Variables: Discrete variables take countable values; continuous variables take values in an interval.

• Mean and Variance: The mean (expected value) and variance measure the central tendency and spread of a random variable, respectively.

Rules for Variances

• The variance of the sum of two random variables is not always the sum of their variances, especially if the variables are dependent.

• Example: If X is the number of heads and Y the number of tails in 4 tosses of a fair coin, then X + Y = 4 always, so Var(X + Y) = 0 even though Var(X) = Var(Y) = 1.

• General Rule: $Var(X + Y) \neq Var(X) + Var(Y)$ in general.

Bernoulli Distribution

Definition and Properties

The Bernoulli distribution models a single trial experiment with two possible outcomes: success (S) and failure (F).

• Examples: Pass/Fail, Defective/Non-defective, Female/Male, Yes/No, Head/Tail.

• Bernoulli Experiment: An experiment with a single trial and two mutually exclusive outcomes.

• Parameter: $p = P(S)$, the probability of success.

• Notation: $X \sim Ber(p)$

Probability Mass Function (PMF)

• Let $X$ be the number of successes (0 or 1):

X	0	1
f(x) = P(X = x)	1 - p	p

General PMF:

$f(x) = p^x (1-p)^{1-x}, \quad x = 0, 1$

Mean and Variance of Bernoulli Distribution

• Mean (Expected Value): $\mu = p$

• Variance: $\sigma^2 = p(1-p)$

• Standard Deviation: $\sigma = \sqrt{p(1-p)}$

Binomial Distribution

Definition and Properties

The Binomial distribution models the number of successes in a fixed number of independent Bernoulli trials, each with the same probability of success.

• Discrete probability distribution for $X = 0, 1, 2, ..., n$

• Describes: Repeated Bernoulli trials (sequence of successes and failures)

• "Success" is a label for one of the two outcomes, not necessarily a positive result.

Conditions for a Binomial Experiment

1. The experiment is performed a fixed number of times ($n$ trials).

2. The trials are independent.

3. Each trial has two mutually exclusive outcomes (success or failure).

4. The probability of success ($p$) is constant for each trial.

Notation

• $n$ = number of trials

• $p$ = probability of success on each trial

• $q = 1 - p$ = probability of failure

• $X$ = number of successes in $n$ trials ($0 \leq X \leq n$)

• Notation: $X \sim Bin(n, p)$

Probability Mass Function (PMF) of Binomial Distribution

The probability of obtaining $x$ successes in $n$ independent trials:

$P(X = x) = {n \choose x} p^x (1-p)^{n-x}, \quad x = 0, 1, 2, ..., n$

where ${n \choose x} = \dfrac{n!}{x!(n-x)!}$ is the binomial coefficient.

Examples

• Example 1: In a random sample of 20 car-owning households, with $p = 0.35$ (probability of having three or more cars):

  • Probability exactly 5 have three or more cars: $P(5) = {20 \choose 5} (0.35)^5 (0.65)^{15} = 0.1272$

  • Probability less than 4 have three or more cars: $P(X < 4) = P(0) + P(1) + P(2) + P(3) = 0.0444$

• Example 2: Drawing 4 pets with 40% cats and 60% dogs, with replacement. $X$ = number of cats in 4 draws.

  • Binomial setting: $n = 4$, $p = 0.4$, $q = 0.6$

  • PMF: $f(x) = {4 \choose x} (0.4)^x (0.6)^{4-x}$ for $x = 0, 1, 2, 3, 4$

  • Table of probabilities:

  X	0	1	2	3	4
  f(x) = P(X = x)	0.13	0.35	0.35	0.15	0.03

Mean, Variance, and Standard Deviation of Binomial Distribution

• Mean (Expected Value): $\mu = E[X] = np$

• Variance: $\sigma^2 = Var[X] = np(1-p) = npq$

• Standard Deviation: $\sigma = \sqrt{np(1-p)} = \sqrt{npq}$

Reason: The binomial random variable is the sum of independent Bernoulli random variables.

Shape of the Binomial Distribution

• The shape depends on the value of $p$:

  • Right-skewed: $p < 0.5$

  • Symmetric: $p = 0.5$

  • Left-skewed: $p > 0.5$

• As the number of trials $n$ increases, the binomial distribution becomes more bell-shaped (approximately normal).

• Rule of Thumb: If $np(1-p) \geq 10$, the distribution is approximately bell-shaped.

Binomial Probability Histograms

• Histograms illustrate the probability distribution for different values of $n$ and $p$.

• For small $n$ or extreme $p$, the distribution is skewed; for large $n$ and $p$ near 0.5, it is symmetric and bell-shaped.

The Empirical Rule and Binomial Random Variables

Empirical Rule

• In a bell-shaped distribution, about 95% of all observations lie within two standard deviations of the mean.

• Interval: $\mu - 2\sigma$ to $\mu + 2\sigma$

• Observations outside this interval are considered unusual (occur less than 5% of the time).

Example: Using the Empirical Rule

• Suppose 35% of car-owning households have three or more cars. In a sample of 400, 162 have three or more cars. Is this unusual?

• Compute expected value and standard deviation:

  • $\mu_X = np = 400 \times 0.35 = 140$

  • $\sigma_X = \sqrt{np(1-p)} = \sqrt{400 \times 0.35 \times 0.65} = 9.54$

• Interval: $140 - 2(9.54) = 120.9$ to $140 + 2(9.54) = 159.1$

• Since 162 is outside [120.9, 159.1], the result is considered unusual.

Summary Table: Bernoulli vs. Binomial Distribution

Property	Bernoulli	Binomial
Number of Trials	1	n (fixed)
Possible Outcomes	2 (Success/Failure)	0, 1, ..., n (number of successes)
PMF	$f(x) = p^x (1-p)^{1-x}$	$f(x) = {n \choose x} p^x (1-p)^{n-x}$
Mean	$p$	$np$
Variance	$p(1-p)$	$np(1-p)$

Key Takeaways

• The Bernoulli and Binomial distributions are foundational for modeling binary outcomes and repeated trials.

• Understanding their properties, formulas, and applications is essential for statistical inference and hypothesis testing.

• The shape of the binomial distribution approaches normality as the number of trials increases, especially when $np(1-p) \geq 10$.