Probability Rules and Applications: Study Notes for Statistics Students

5.1 Probability Rules

Introduction to Probability

Probability is a fundamental concept in statistics that quantifies the likelihood of a random phenomenon or chance event occurring. It is used to describe the long-term proportion with which a certain outcome will occur, even when short-term results are unpredictable.

• Probability: A measure of how likely an event is to occur, ranging from 0 (impossible) to 1 (certain).

• Outcome: The result of a single trial of a probability experiment.

• Law of Large Numbers: As the number of repetitions of a probability experiment increases, the observed proportion of a particular outcome approaches its theoretical probability.

Example: Flipping a coin 100 times and plotting the proportion of heads demonstrates how the observed proportion stabilizes near 0.5 as the number of flips increases.

Key Terms in Probability

• Experiment: Any process that can be repeated and has uncertain results.

• Sample Space (S): The set of all possible outcomes of an experiment.

• Event: Any collection of outcomes from a probability experiment. A simple event contains only one outcome.

Example: For the experiment of having two children:

• Outcomes: boy-boy, boy-girl, girl-boy, girl-girl

• Sample space: S = {(boy, boy), (boy, girl), (girl, boy), (girl, girl)}

• Event E = "have one boy": E = {(boy, girl), (girl, boy)}

Rules of Probabilities

• Rule 1: For any event E, $0 \leq P(E) \leq 1$

• Rule 2: The sum of the probabilities of all outcomes in the sample space is 1: $P(e_1) + P(e_2) + \ldots + P(e_n) = 1$

Probability Model

A probability model lists all possible outcomes and their probabilities, satisfying the above two rules.

Color	Probability
Brown	0.12
Yellow	0.15
Red	0.12
Blue	0.23
Orange	0.23
Green	0.15

All probabilities are between 0 and 1, and their sum is 1.

• Impossible event: Probability is 0.

• Certain event: Probability is 1.

• Unusual event: An event with a low probability of occurring.

Empirical (Experimental) Probability

Empirical probability is based on observations from experiments.

• Formula: $P(E) = \frac{\text{frequency of } E}{\text{number of trials}}$

Example: In a game where pigs are tossed as dice, the following frequencies were observed in 3,939 trials:

Outcome	Frequency
Side with no dot	1344
Side with dot	1294
Razorback	767
Trotter	365
Snouter	137
Leaning Jowler	32

Probabilities are calculated as frequency divided by total trials. For example, $P(\text{side with dot}) = \frac{1294}{3939} \approx 0.329$.

Classical Probability

The classical method applies when all outcomes are equally likely.

• Formula: $P(E) = \frac{\text{number of ways } E \text{ can occur}}{\text{number of possible outcomes}}$

• Or, $P(E) = \frac{N(E)}{N(S)}$ where $N(E)$ is the number of outcomes in E, and $N(S)$ is the number in the sample space.

Example: A bag contains 9 brown, 6 yellow, 7 red, 4 orange, 2 blue, and 2 green candies (total 30). Probability of yellow: $P(\text{yellow}) = \frac{6}{30} = 0.2$; blue: $P(\text{blue}) = \frac{2}{30} = 0.067$.

Simulation

Simulation uses random processes (like computer applets) to approximate probabilities by mimicking real experiments.

Example: Simulate rolling a die 100 or 1000 times to estimate the probability of rolling a 4, and compare to the theoretical value $\frac{1}{6}$.

Subjective Probability

Subjective probability is based on personal judgment or opinion, not on formal calculations or experiments.

• Example: An economist predicts a 20% chance of recession next year. This is subjective probability.

• Example: Betting odds for a horse race based on the amount of money wagered are subjective, not empirical or classical.

5.2 The Addition Rule and Complements

Addition Rule for Disjoint (Mutually Exclusive) Events

Two events are disjoint (mutually exclusive) if they have no outcomes in common.

• Addition Rule: If E and F are disjoint, $P(E \text{ or } F) = P(E) + P(F)$

• This rule extends to more than two disjoint events: $P(E_1 \text{ or } E_2 \text{ or } \ldots) = P(E_1) + P(E_2) + \ldots$

Example: Selecting a chip labeled 0-9. Let E = "number ≤ 2" (0,1,2), F = "number ≥ 8" (8,9). $P(E) = 0.3$, $P(F) = 0.2$, $P(E \text{ or } F) = 0.5$.

Venn Diagrams

Venn diagrams visually represent events and their relationships within the sample space.

General Addition Rule

For any two events E and F (not necessarily disjoint):

• $P(E \text{ or } F) = P(E) + P(F) - P(E \text{ and } F)$

This formula corrects for double-counting the overlap between E and F.

Complement Rule

The complement of an event E, denoted $E^c$, consists of all outcomes not in E.

• Complement Rule: $P(E^c) = 1 - P(E)$

Example: If 31.6% of households own a dog, the probability a household does not own a dog is $1 - 0.316 = 0.684$.

Tables and Data Interpretation

Probability Model Example (M&M Colors)

Color	Probability
Brown	0.12
Yellow	0.15
Red	0.12
Blue	0.23
Orange	0.23
Green	0.15

This table is used to verify the rules of probability models: all probabilities are between 0 and 1, and their sum is 1.

Empirical Probability Example (Pass the Pigs Game)

Outcome	Probability
Side with no dot	0.341
Side with dot	0.329
Razorback	0.195
Trotter	0.093
Snouter	0.035
Leaning Jowler	0.008

Probabilities are calculated by dividing the frequency of each outcome by the total number of trials.

Summary

• Probability quantifies uncertainty and is foundational to statistical inference.

• There are several approaches to probability: empirical (experimental), classical (theoretical), and subjective (personal judgment).

• Probability rules and models ensure consistency and logical reasoning in probability calculations.

• Tables and diagrams (such as Venn diagrams) are useful tools for visualizing and calculating probabilities.