Chapter 5: Probability – Foundations, Rules, and Applications

Probability

Basic Concepts of Probability

Probability is a measure of how likely an event is to occur. It is fundamental to statistics and is used to quantify uncertainty in various contexts.

• Event: Any collection of outcomes from a procedure (e.g., flipping a coin, rolling a die).

• Simple Event: An outcome or event that cannot be broken down into simpler components (e.g., rolling a 4 on a die).

• Compound Event: An event that combines two or more simple events (e.g., rolling an even number on a die).

• Sample Space (S): The set of all possible simple events for a procedure.

Notation:

• P denotes probability.

• P(A) denotes the probability of event A.

• Probabilities are always between 0 and 1: $0 \leq P(A) \leq 1$

Examples of Events and Sample Spaces

• Flipping a coin twice: Sample space S = {HH, HT, TH, TT}

• Flipping a coin three times: Sample space S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}

• Rolling a six-sided die: Sample space S = {1, 2, 3, 4, 5, 6}

• Event: Rolling an even number (2, 4, 6) is a subset of the sample space.

Order matters: For coin flips, HT and TH are different outcomes.

Approaches to Probability

There are two main approaches to calculating probability: the relative frequency (empirical) approach and the theoretical approach.

• Relative Frequency (Empirical) Approach: Probability is estimated by repeating a procedure many times and observing the proportion of times an event occurs.

$P(A) = \frac{\text{number of times A occurs}}{\text{number of times procedure was repeated}}$

• Theoretical Approach: Used when all outcomes are equally likely. Probability is calculated as:

$P(A) = \frac{\text{number of ways A occurs in the sample space}}{\text{number of different simple events in the sample space}}$

Examples

• Example 3: Rolling an 8-sided die. Probability of rolling a number less than 7: S = {1,2,3,4,5,6,7,8}, so numbers less than 7 are {1,2,3,4,5,6} (6 outcomes). $P(X < 7) = \frac{6}{8} = 0.75$

• Example 4: Probability a randomly selected high school driver texted while driving: 3785 texted, 4720 did not. $P(\text{texted}) = \frac{3785}{3785+4720} = 0.445$

• Example 5: Probability of dying in a skydiving jump: 21 deaths in 3,000,000 jumps. $P(\text{dying}) = \frac{21}{3,000,000} = 0.000007$

• Example 6: Probability of getting exactly two tails in three flips of a coin: There are 3 outcomes (HTT, THT, TTH) out of 8, so $P(\text{exactly 2 tails}) = \frac{3}{8} = 0.375$

Complements and the Addition Rule

The complement of an event A, denoted Ac or \( \overline{A} \), consists of all outcomes where A does not occur.

• $P(A) + P(A^c) = 1$

• Example 7: Probability of not dying in a skydiving jump: $P(\text{not dying}) = 1 - P(\text{dying}) = 1 - 0.000007 = 0.999993$

• Example 8: Probability of not drawing a heart from a standard deck: $P(\text{not heart}) = 1 - P(\text{heart}) = 1 - \frac{13}{52} = 0.75$

Addition Rule

The probability that event A or event B occurs is:

$P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$

• Disjoint (Mutually Exclusive) Events: Events that cannot occur at the same time. For disjoint events, $P(A \text{ and } B) = 0$.

• An event and its complement are always mutually exclusive.

• Example 9: Rolling a 5 or a 3 on a six-sided die: $P(5 \text{ or } 3) = P(5) + P(3) = \frac{1}{6} + \frac{1}{6} = \frac{2}{6} = 0.333$

• Example 10: Probability of selecting a king or a black card from a deck: $P(\text{king}) = \frac{4}{52}$, $P(\text{black}) = \frac{26}{52}$, $P(\text{king and black}) = \frac{2}{52}$, so $P(\text{king or black}) = \frac{4}{52} + \frac{26}{52} - \frac{2}{52} = \frac{28}{52} = 0.538$

• Example 11: Probability of selecting a red card or an even card: $P(\text{red}) = \frac{26}{52}$, $P(\text{even}) = \frac{20}{52}$, $P(\text{red and even}) = \frac{12}{52}$, so $P(\text{red or even}) = \frac{26}{52} + \frac{20}{52} - \frac{12}{52} = \frac{34}{52} = 0.654$ Additional info: Even cards are 2, 4, 6, 8, 10, Q in each suit.

Table Example: Blood Groups and Rh Types

The following table summarizes blood groups and Rh types for randomly selected subjects:

	O	A	B	AB
Rh+	39	35	8	4
Rh-	6	5	2	1

• Example 12a: Probability of selecting someone who is Group A or Rh+: $P(A \text{ or Rh}^+) = P(A) + P(Rh^+) - P(A \text{ and } Rh^+)$

• Example 12b: These events are not disjoint, since a person can be both Group A and Rh+.

Multiplication Rule and Conditional Probability

The multiplication rule is used to find the probability that two events both occur.

$P(A \text{ and } B) = P(A) \times P(B|A)$

• Conditional Probability: $P(B|A)$ is the probability that event B occurs given that event A has already occurred.

• If A and B are independent, $P(B|A) = P(B)$.

• Example 13a: Drawing two spades in a row from a deck without replacement: $P(\text{spade first}) = \frac{13}{52}$, $P(\text{spade second | spade first}) = \frac{12}{51}$, so $P(\text{both spades}) = \frac{13}{52} \times \frac{12}{51} = 0.059$

• Example 13b: Drawing a spade first, then a heart: $P(\text{spade first}) = \frac{13}{52}$, $P(\text{heart second | spade first}) = \frac{13}{51}$, so $P(\text{spade then heart}) = \frac{13}{52} \times \frac{13}{51} = 0.064$

• Example 14: Probability both selected airport baggage scales are defective (with replacement): $P(\text{defective}) = \frac{102}{810}$, so $P(\text{both defective}) = \left(\frac{102}{810}\right)^2 = 0.016$

Conditional Probability Example

• Example: Flip a coin twice. What is the probability both flips are heads given at least one is heads? S = {HH, HT, TH, TT}. The relevant outcomes are {HH, HT, TH}. Only HH is both heads, so $P(\text{both heads} | \text{at least one head}) = \frac{1}{3} = 0.33$

Independence and Dependence

• Independent Events: The occurrence of one event does not affect the probability of the other.

• Dependent Events: The occurrence of one event changes the probability of the other.

Contingency Tables and Joint Probability

Contingency tables summarize data for two categorical variables and allow calculation of joint, marginal, and conditional probabilities.

	Pay Higher Prices (Yes)	Pay Higher Prices (No)	Total
Environmental Group Member (Yes)	28	67	95
Environmental Group Member (No)	89	932	1021
Total	117	999	1116

• Example 15a: Probability of being both a member and willing to pay higher prices: $P(\text{member and willing}) = \frac{28}{1116} = 0.025$

• Example 15b: If independent: $P(\text{member}) = \frac{95}{1116}$, $P(\text{willing}) = \frac{117}{1116}$, so $P(\text{member and willing}) = P(\text{member}) \times P(\text{willing}) = 0.085 \times 0.105 = 0.009$

Bayes' Theorem

Bayes' Theorem allows calculation of conditional probabilities when the reverse conditional is known.

$P(A|B) = \frac{P(B|A) \times P(A)}{P(B)}$

• Example 16: Probability of drawing a king given the card is red: There are 2 red kings out of 26 red cards, so $P(\text{king} | \text{red}) = \frac{2}{26} = 0.077$

Blood Group and Rh Type Table (100 subjects)

	O	A	B	AB
Rh+	39	35	8	4
Rh-	6	5	2	1

• Example 18a: Probability of type O given Rh-: $P(O|Rh-) = \frac{6}{6+5+2+1} = \frac{6}{14} = 0.429$

• Example 18b: Probability of Rh- given type O: $P(Rh-|O) = \frac{6}{39+6} = \frac{6}{45} = 0.133$

Disease Test Table

	Has Disease	Doesn't have Disease	Total
Positive	100	156	256
Negative	35	709	744
Total	135	856	1000

• Example 19a: Sensitivity (true positive rate): $P(\text{positive} | \text{disease}) = \frac{100}{135} = 0.741$ (74.1%)

• Example 19b: Specificity (true negative rate): $P(\text{negative} | \text{no disease}) = \frac{709}{856} = 0.828$ (82.8%)

Summary Table: Key Probability Rules

Rule	Formula	When to Use
Complement Rule	$P(A^c) = 1 - P(A)$	Finding the probability that an event does not occur
Addition Rule	$P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$	Finding the probability that at least one of two events occurs
Multiplication Rule	$P(A \text{ and } B) = P(A) \times P(B|A)$	Finding the probability that two events both occur
Conditional Probability	$P(B|A) = \frac{P(A \text{ and } B)}{P(A)}$	Finding the probability of B given A has occurred
Bayes' Theorem	$P(A|B) = \frac{P(B|A) \times P(A)}{P(B)}$	Reversing conditional probabilities