Bayes' Theorem and Conditional Probability: Study Notes

Bayes' Theorem

Conditional Probability

Conditional probability is the probability of an event A, given that another event B has occurred. It is denoted as P(A|B). Bayes' Theorem provides a way to update probabilities based on new information, especially when direct computation of conditional probabilities is difficult.

• Conditional Probability: The probability of event A given event B is written as P(A|B).

• Bayes' Theorem: Allows calculation of P(A|B) when P(B|A), P(A), and P(B) are known.

Formula:

$P(A|B) = \frac{P(B|A) \cdot P(A)}{P(B)}$

Where:

• P(A|B): Probability of A given B

• P(B|A): Probability of B given A

• P(A): Probability of A

• P(B): Probability of B

Example 1: Marble Bags

Suppose you play a game where you close your eyes and a marble is pulled from one of two bags. Each bag contains marbles of different colors. You want to find the probability that the marble came from the Left Bag, given that it is red.

• Given event (A): marble from Left Bag

• Observed event (B): marble is red

• P(A): Probability marble is from Left Bag

• P(B|A): Probability marble is red given it is from Left Bag

• P(B): Total probability marble is red

Bayes' Theorem applied:

$P(\text{Left Bag} | \text{Red}) = \frac{P(\text{Red} | \text{Left Bag}) \cdot P(\text{Left Bag})}{P(\text{Red})}$

Example: If Left Bag has 2 red, 4 blue; Right Bag has 5 red, 1 blue. If a red marble is drawn, what is the probability it came from the Left Bag?

• P(Left Bag) = 0.5 (equal chance to pick either bag)

• P(Red | Left Bag) = 2/6

• P(Red | Right Bag) = 5/6

• P(Red) = P(Red | Left Bag) * P(Left Bag) + P(Red | Right Bag) * P(Right Bag)

$P(\text{Red}) = \frac{2}{6} \times 0.5 + \frac{5}{6} \times 0.5 = \frac{7}{12}$

$P(\text{Left Bag} | \text{Red}) = \frac{\frac{2}{6} \times 0.5}{\frac{7}{12}} = \frac{1}{7}$

Example 2: Medical Testing

Suppose a test is used to detect a disease. The probability that a person has the disease is known, as well as the probability that the test is positive given the person has the disease, and the probability that the test is positive given the person does not have the disease. Bayes' Theorem can be used to find the probability that a person has the disease given a positive test result.

• P(Disease): Probability a person has the disease

• P(Positive | Disease): Probability test is positive given disease

• P(Positive | No Disease): Probability test is positive given no disease

• P(Positive): Total probability of a positive test

Bayes' Theorem applied:

$P(\text{Disease} | \text{Positive}) = \frac{P(\text{Positive} | \text{Disease}) \cdot P(\text{Disease})}{P(\text{Positive})}$

Example: If 1% of people have the disease, the test is positive 90% of the time for those with the disease, and 5% for those without. What is the probability a person has the disease given a positive test?

• P(Disease) = 0.01

• P(Positive | Disease) = 0.9

• P(Positive | No Disease) = 0.05

• P(No Disease) = 0.99

• P(Positive) = 0.9 * 0.01 + 0.05 * 0.99 = 0.0585

$P(\text{Disease} | \text{Positive}) = \frac{0.9 \times 0.01}{0.0585} \approx 0.154$

Example 3: Weather and Train Delays

Suppose 20% of trains are delayed. If it rains, 72% of trains are delayed. What is the probability it is raining given that a train is delayed?

• P(Rain): Probability it is raining

• P(Delayed | Rain): Probability train is delayed given rain

• P(Delayed): Probability train is delayed

Bayes' Theorem applied:

$P(\text{Rain} | \text{Delayed}) = \frac{P(\text{Delayed} | \text{Rain}) \cdot P(\text{Rain})}{P(\text{Delayed})}$

Summary Table: Bayes' Theorem Applications

Scenario	Given	Find	Bayes' Formula
Marble Bags	P(Red | Bag), P(Bag)	P(Bag | Red)	$P(Bag|Red) = \frac{P(Red|Bag) \cdot P(Bag)}{P(Red)}$
Medical Test	P(Pos|Disease), P(Disease), P(Pos|No Disease), P(No Disease)	P(Disease|Pos)	$P(Disease|Pos) = \frac{P(Pos|Disease) \cdot P(Disease)}{P(Pos)}$
Train Delay & Rain	P(Delayed|Rain), P(Rain), P(Delayed)	P(Rain|Delayed)	$P(Rain|Delayed) = \frac{P(Delayed|Rain) \cdot P(Rain)}{P(Delayed)}$

Key Points

• Bayes' Theorem is essential for updating probabilities based on new evidence.

• It is widely used in medical testing, decision making, and machine learning.

• Always identify the prior probability, likelihood, and marginal probability before applying the formula.