Binomial and Bernoulli Distributions: Definitions, Properties, and Applications

Binomial and Bernoulli Distributions

Introduction

This study guide covers the Bernoulli and Binomial distributions, their properties, and applications in probability and statistics. These topics are foundational for understanding discrete probability distributions and their use in statistical inference.

Random Variables and Probability Distributions

Key Concepts

• Random Variable (RV): A variable whose value is determined by the outcome of a random experiment.

• Probability Distribution: Describes how probabilities are distributed over the values of the random variable.

• Probability Mass Function (PMF): For discrete random variables, the PMF gives the probability that the variable takes a specific value.

• Discrete vs. Continuous Random Variables: Discrete variables take countable values; continuous variables take values in a continuum.

• Mean and Variance: The mean (expected value) and variance measure the central tendency and spread of a random variable.

Variance of the Sum of Random Variables

Rules for Variances

The variance of the sum of two random variables is not always the sum of their variances. If the variables are dependent, covariance must be considered.

• Example: Let $X$ be the number of heads and $Y$ the number of tails in 4 tosses of a fair coin. $X + Y = 4$ always, so $Var(X + Y) = 0$, even though $Var(X) = 1$ and $Var(Y) = 1$.

• General Rule: $Var(X + Y) = Var(X) + Var(Y)$ only if $X$ and $Y$ are independent.

Bernoulli Distribution

Definition and Properties

The Bernoulli distribution models experiments with two possible outcomes: success or failure.

• Bernoulli Experiment: A single trial with two mutually exclusive outcomes (Success $S$ or Failure $F$).

• Notation: $X \sim Ber(p)$, where $p$ is the probability of success.

• Outcomes: $X = 1$ (success), $X = 0$ (failure).

• PMF: $f(x) = p^x (1-p)^{1-x},\ x = 0,1$

Probability Table

X	0	1
f(x) = P(X = x)	1 - p	p

Mean and Variance

• Mean: $\mu = p$

• Variance: $\sigma^2 = p(1-p)$

• Standard Deviation: $\sigma = \sqrt{p(1-p)}$

Binomial Distribution

Definition and Properties

The Binomial distribution generalizes the Bernoulli distribution to $n$ independent trials, each with two possible outcomes.

• Binomial Experiment:

  1. Performed a fixed number $n$ of times (trials).

  2. Trials are independent.

  3. Each trial has two mutually exclusive outcomes (success or failure).

  4. Probability of success $p$ is constant for each trial.

• Random Variable: $X$ = number of successes in $n$ trials.

• Notation: $X \sim Bin(n, p)$

Probability Mass Function (PMF)

• Formula: $P(X = x) = {n \choose x} p^x (1-p)^{n-x},\ x = 0, 1, ..., n$

• Where: ${n \choose x} = \frac{n!}{x!(n-x)!}$

Example Table: Binomial PMF (n = 4, p = 0.4)

X	0	1	2	3	4
f(x) = P(X = x)	0.13	0.35	0.35	0.15	0.03

Mean, Variance, and Standard Deviation

• Mean: $\mu = E[X] = np$

• Variance: $\sigma^2 = Var[X] = np(1-p) = npq$

• Standard Deviation: $\sigma = \sqrt{np(1-p)} = \sqrt{npq}$

Examples

• Example 1: Probability that exactly 5 out of 20 households have three or more cars (with $p = 0.35$): $P(5) = {20 \choose 5} (0.35)^5 (0.65)^{15} = 0.1272$

• Example 2: Probability that fewer than 4 out of 20 households have three or more cars: $P(X < 4) = P(0) + P(1) + P(2) + P(3) = 0.0444$

Identifying Binomial Experiments

Criteria

• Fixed number of trials

• Independent trials

• Two mutually exclusive outcomes per trial

• Constant probability of success

Example: Drawing cards with replacement is binomial; without replacement is not, due to changing probabilities.

Shape of the Binomial Distribution

Skewness and Symmetry

• Right Skewed: $p < 0.5$

• Symmetric: $p = 0.5$

• Left Skewed: $p > 0.5$

As the number of trials $n$ increases, the binomial distribution approaches a bell-shaped (normal) curve.

Rule of Thumb for Bell-Shapedness

• If $np(1-p) \geq 10$, the binomial distribution is approximately bell-shaped.

Empirical Rule and Binomial Random Variables

Empirical Rule

• In a bell-shaped distribution, about 95% of observations lie within two standard deviations of the mean.

• Interval: $\mu - 2\sigma$ to $\mu + 2\sigma$

• Observations outside this interval are considered unusual (less than 5% probability).

Empirical Rule Example

• Given: $n = 400$, $p = 0.35$, observed $X = 162$

• Expected value: $\mu_X = np = 140$

• Standard deviation: $\sigma_X = \sqrt{np(1-p)} = \sqrt{400 \times 0.35 \times 0.65} = 9.54$

• Interval: $[140 - 2 \times 9.54, 140 + 2 \times 9.54] = [120.9, 159.1]$

• Conclusion: Since 162 is outside this interval, the result is considered unusual.

Summary Table: Bernoulli vs Binomial Distribution

Property	Bernoulli	Binomial
Number of Trials	1	n
Outcomes per Trial	2 (Success/Failure)	2 (Success/Failure)
PMF	$f(x) = p^x (1-p)^{1-x}$	$P(X = x) = {n \choose x} p^x (1-p)^{n-x}$
Mean	$p$	$np$
Variance	$p(1-p)$	$np(1-p)$

Conclusion

The Bernoulli and Binomial distributions are essential for modeling binary outcomes and repeated trials in probability. Understanding their properties, formulas, and applications is crucial for further study in statistics, including hypothesis testing and estimation.