Conditional Probability, Independence, and Bayes’ Theorem: Study Notes

Conditional Probability

Definition and Formula

Conditional probability quantifies the likelihood of an event occurring given that another event has already occurred. It is a fundamental concept in probability theory and statistics, especially when analyzing dependent events.

• Conditional Probability: The probability of event A given event B is denoted as P(A|B).

• Formula: $P(A|B) = \frac{P(A \cap B)}{P(B)}$

• Joint Probability: $P(A \cap B) = P(A|B) \cdot P(B)$

Example: Cards

Suppose two cards are dealt from a deck of 52. Let B be the event that the first card is an ace, and A be the event that the second card is an ace. To compute probabilities:

• P(B): Probability the first card is an ace = $\frac{4}{52}$

• P(A|B): Probability the second card is an ace given the first is an ace = $\frac{3}{51}$

• P(A \cap B): Joint probability = $P(A|B) \cdot P(B) = \frac{4}{52} \cdot \frac{3}{51}$

Probability Trees

Visual Representation

Probability trees are useful for visualizing conditional probabilities and sequences of events. Each branch represents a possible outcome and its associated probability.

• Branches: Represent different events and their conditional probabilities.

• Multiplication: Probabilities along a path are multiplied to find joint probabilities.

Example: Dice

Two dice are thrown (one red, one black). Let:

• A: Sum is at least 10

• B: Black die shows 6

• C: Exactly one die shows 6

To calculate:

• Pr(A): Count all pairs where sum ≥ 10

• Pr(A|B): Probability sum ≥ 10 given black die shows 6

• Pr(A|C): Probability sum ≥ 10 given exactly one die shows 6

Applications of Conditional Probability

Credit Default Analysis

Conditional probability is used in risk analysis, such as determining the likelihood of credit default given prior late payments.

• Overall probability of default: $\frac{1002}{14998}$

• Probability of default given late payment: $\frac{347}{833}$

• Probability of default given no late payment: $\frac{655}{14165}$

• Relative risk: $\frac{\frac{347}{833}}{\frac{655}{14165}}$

	Default: Yes	Default: No	Total
Previous late payment: Yes	347	486	833
Previous late payment: No	655	13510	14165
Total	1002	13996	14998

Advertising Effectiveness

Conditional probability helps evaluate the impact of advertising campaigns.

• P(B): Probability of buying product B = 0.20

• P(A): Probability of seeing advertisement = 0.40

• P(A|B): Probability of seeing ad given purchase = 0.60

• P(B|A): Probability of purchase given ad = $\frac{P(A|B) \cdot P(B)}{P(A)} = \frac{0.60 \cdot 0.20}{0.40} = 0.30$

• Lift: $\frac{P(A|B)}{P(A)}$

Manufacturing Defects

Conditional probability is used to determine defect rates and source identification in manufacturing.

• Machine A: 60% of products, 2% defective

• Machine B: 40% of products, 5% defective

• Overall defect rate: $0.6 \times 0.02 + 0.4 \times 0.05 = 0.012 + 0.02 = 0.032$ (3.2%)

• Probability defective product from B: $\frac{0.4 \times 0.05}{0.032}$

Bayes’ Theorem

Statement and Formula

Bayes’ Theorem allows the calculation of the probability of an event based on prior knowledge and new evidence. It is essential for updating probabilities as new information becomes available.

• Formula: $P(A_i|B) = \frac{P(B|A_i) \cdot P(A_i)}{P(B)}$

• Total Probability: $P(B) = P(B|A_1) \cdot P(A_1) + P(B|A_2) \cdot P(A_2) + \ldots$

Example: Advertising

Bayes’ Theorem is used to update the probability of a customer buying a product after seeing an advertisement.

• Prior: P(B) = probability of buying product

• Posterior: P(B|A) = probability of buying given ad

Independent Events

Definition and Properties

Events A and B are independent if the occurrence of one does not affect the probability of the other.

• Definition: $P(A|B) = P(A)$

• Joint Probability: $P(A \cap B) = P(A) \cdot P(B)$

Example: Credit Defaults

If a bank estimates 6% of customers will default, and it takes on 3 new customers:

• Probability all three default: $0.06^3$

• Probability at least one defaults: $1 - (1 - 0.06)^3$

Prosecutor's Fallacy and Abuse of Probability

Definition and Example

The prosecutor's fallacy is a misuse of probability in legal contexts, often confusing conditional probabilities and leading to incorrect conclusions.

• Example: Sally Clarke case, where the probability of two independent SIDS events was incorrectly multiplied.

• Correct Approach: Consider dependencies and context, not just multiply probabilities.

OJ Simpson Trial

Statistics can be misused in court to mislead about the likelihood of guilt or innocence. Proper interpretation requires understanding conditional probability and context.

Summary

• Conditional probability: $P(A|B) = \frac{P(A \cap B)}{P(B)}$

• Probability trees: Useful for visualizing conditional probabilities

• Independence: $P(A \cap B) = P(A) \cdot P(B)$ if A and B are independent

• Bayes’ Theorem: Updates probabilities based on new evidence

• Abuse of probability: Misinterpretation can lead to incorrect conclusions, especially in legal contexts