Conditional Probability, Complements, and Bayes’ Theorem: Study Guide

Conditional Probability, Complements, and Bayes’ Theorem

The Big Picture

This chapter explores three interconnected concepts in probability: conditional probability, the complement rule (especially for "at least one" problems), and Bayes’ Theorem. Mastery of these ideas is essential for understanding how to update probabilities based on new information and how to interpret test results in real-world contexts.

Conditional Probability

What Is Conditional Probability?

Conditional probability is the probability of an event occurring, given that another event has already occurred. Instead of considering the entire sample space, we focus only on the subset where the given condition is true.

• Notation: $P(B|A)$ is read as "the probability of B given A." The event after the bar (|) is the condition that has already happened; the event before the bar is what we are finding the probability of.

• Formula:

$P(B|A) = \frac{P(A \text{ and } B)}{P(A)}$

• From a table: $P(B|A) = \frac{\text{Number of outcomes where A and B both occur}}{\text{Number of outcomes where A occurs}}$

Visualization: In a Venn diagram, once event A has occurred, only the overlap with B (A ∩ B) matters within A.

Example: Polygraph Test

• Let A = subject is telling the truth; B = polygraph indicates lying.

• $P(B|A)$ is the probability the polygraph indicates lying, given the subject is telling the truth (a false positive).

Conditional Probability from Tables

When using tables to find conditional probabilities:

• Identify the row or column for the given condition (the event after the bar).

• Use only that row/column total as the denominator.

• Divide the relevant cell count by that total.

Example 1: Students and Money

	Purchased Gum	Kept Money	Total
Given Four Quarters	25	17	42
Given $1 Bill	19	26	45
Total	44	43	87

• $P(\text{spent}|\text{4 quarters}) = \frac{25}{42} \approx 0.595$ (59.5%)

• $P(\text{kept}|\text{4 quarters}) = \frac{17}{42} \approx 0.405$ (40.5%)

Example 2: $1 Bill

	Purchased Gum	Kept Money	Total
Given Four Quarters	32	18	50
Given $1 Bill	11	26	37
Total	43	44	87

• $P(\text{spent}|\$1 \text{ bill}) = \frac{11}{37} \approx 0.297$ (29.7%)

• $P(\text{kept}|\$1 \text{ bill}) = \frac{26}{37} \approx 0.703$ (70.3%)

Interpretation: The form of money affects spending behavior.

Example 3: Disease Testing (False Positives)

	Had Disease	No Disease	Total
Positive Test	122	5	127
Negative Test	28	145	173
Total	150	150	300

• $P(\text{positive}|\text{no disease}) = \frac{5}{150} \approx 0.0333$ (3.33%)

This is the false positive rate.

Example 4: Disease Testing (False Negatives)

	Had Disease	No Disease	Total
Positive Test	329	3	332
Negative Test	10	1138	1148
Total	339	1141	1480

• $P(\text{negative}|\text{disease}) = \frac{10}{339} \approx 0.0295$ (2.95%)

False negatives can be more serious, as sick patients may not seek treatment.

Positive Predictive Value (PPV)

Definition: PPV is the probability that a subject actually has the condition, given a positive test result.

$\text{PPV} = P(\text{condition present} | \text{test positive})$

	No (Did Not Lie)	Yes (Lied)	Total
Positive Test	16	43	59
Negative Test	33	12	45
Total	49	55	104

• $P(\text{lied}|\text{positive}) = \frac{43}{59} \approx 0.729$ (72.9%)

About 27% of positive results were false alarms.

The Complement Rule

"At Least One" Problems

Calculating the probability of "at least one" occurrence is often easier by finding the probability of none and subtracting from 1.

$P(\text{at least one}) = 1 - P(\text{none})$

• Example 1: Six children, probability at least one girl: $1 - (\frac{1}{2})^6 = 1 - \frac{1}{64} = \frac{63}{64} \approx 0.984$ (98.4%)

• Example 2: Nine births, $P(\text{girl}) = 0.462$, probability at least one boy: $1 - (0.462)^9 \approx 0.999$ (99.9%)

• Example 3: Last four digits of a phone number, probability at least one 0: $1 - (\frac{9}{10})^4 = 1 - 0.6561 \approx 0.344$ (34.4%)

• Example 4: 68% of Wi-Fi users are careful, probability at least one of 4 is careful: $1 - (0.32)^4 \approx 0.990$ (99.0%)

Additional info: Survey data may be biased if self-reported.

Independence

Definition and Example

Independent events do not affect each other. The outcome of one event does not change the probability of another.

• Example: If the first two children are girls, the probability the third child is a girl is still 0.5.

$P(\text{all occur}) = P(A) \times P(B) \times P(C) \times \ldots$

Redundancy and Reliability

Using Redundancy to Increase Reliability

Adding independent backup systems (like hard drives) increases the probability that at least one will work.

• Example: Each drive has an 8% failure rate per year.

• Two drives: $P(\text{at least one works}) = 1 - (0.08)^2 = 0.9936$ (99.36%)

• Three drives: $1 - (0.08)^3 = 0.999488$ (99.95%)

Combined Sample Testing

Pooling Samples for Efficiency

Combining samples for a single test can save resources. If the combined test is negative, all are cleared; if positive, retest individually.

• Example 1: $P(\text{single positive}) = 0.1$, combined sample of 3: $1 - (0.9)^3 = 0.271$ (27.1%)

• Example 2: $P(\text{bacteria in one pool}) = 0.007$, two pools: $1 - (0.993)^2 \approx 0.0140$ (1.40%)

Bayes’ Theorem

What Is Bayes’ Theorem?

Bayes’ Theorem allows us to reverse conditional probabilities: given the result, what is the probability of a specific cause?

$P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B|A) \cdot P(A) + P(B|\overline{A}) \cdot P(\overline{A})}$

The denominator sums all ways the result B could occur; the numerator isolates the path through A.

Worked Example: Two Bags of Balls

• Bag A: 3 red, 7 blue; Bag B: 6 red, 4 blue; pick a bag at random, draw a red ball.

• Find $P(\text{Bag A}|\text{red})$.

Step-by-step:

• $P(A \cap \text{Red}) = \frac{1}{2} \times \frac{3}{10} = \frac{3}{20}$

• $P(B \cap \text{Red}) = \frac{1}{2} \times \frac{6}{10} = \frac{6}{20}$

• $P(\text{Red}) = \frac{3}{20} + \frac{6}{20} = \frac{9}{20}$

• $P(A|\text{Red}) = \frac{3/20}{9/20} = \frac{3}{9} = \frac{1}{3} \approx 0.333$ (33.3%)

Interpretation: Observing a red ball makes it less likely you picked Bag A, since Bag B has more red balls.

Practice Problem: Factory Phones

• Factory A: 60% of phones, 2% defective; Factory B: 40% of phones, 5% defective.

• Find $P(\text{Factory A}|\text{defective})$.

• $P(A \cap D) = 0.6 \times 0.02 = 0.012$

• $P(B \cap D) = 0.4 \times 0.05 = 0.020$

• $P(D) = 0.012 + 0.020 = 0.032$

• $P(A|D) = \frac{0.012}{0.032} = 0.375$ (37.5%)

Interpretation: Even though Factory A makes more phones, Factory B's higher defect rate means a defective phone is more likely from Factory B.

Summary & Quick Reference

Key Formulas

• Conditional Probability: $P(B|A) = \frac{P(A \text{ and } B)}{P(A)}$

• At Least One (Complement Rule): $P(\text{at least one}) = 1 - P(\text{none})$

• Independent Events (all occur): $P(\text{all occur}) = P(A) \times P(B) \times P(C) \times \ldots$

• Bayes’ Theorem: $P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B|A) \cdot P(A) + P(B|\overline{A}) \cdot P(\overline{A})}$

Concept Review: False Positives and False Negatives

• False Positive: Test indicates a condition is present when it is absent. $P(\text{positive test}|\text{condition absent})$

• False Negative: Test indicates a condition is absent when it is present. $P(\text{negative test}|\text{condition present})$

• Which is more serious? Usually false negatives, as they may prevent needed treatment.

Common Mistakes to Avoid

• Confusing $P(A|B)$ with $P(B|A)$

• Assuming previous outcomes affect future independent events

• Using the entire table instead of focusing on the given condition

• Forgetting to use the complement for "at least one" problems

• Assuming samples are representative (beware of volunteer bias)

• Confusing false positive with false negative

• Forgetting to add all paths in the denominator when using Bayes’ Theorem

• Plugging numbers into Bayes’ formula in the wrong order