Lecture 13: Independence and Conditional Probability

Independence and Conditional Probability

Recap of Probability Concepts

This section reviews foundational probability concepts necessary for understanding independence and conditional probability.

• Probability models: Frameworks for assigning probabilities to outcomes in a sample space.

• Unions (or) and Intersections (and): Union refers to the event that at least one of several events occurs; intersection refers to all events occurring together.

• Mutually disjoint events: Events that cannot occur simultaneously (also called mutually exclusive).

• Addition law: For disjoint events, $P(E \text{ or } F) = P(E) + P(F)$.

• Complements: The complement of event E is the event that E does not occur.

• Complement law: $P(E^c) = 1 - P(E)$, where $E^c$ is the complement of E.

Independent and Dependent Events

Understanding the distinction between independent and dependent events is crucial for applying probability rules correctly.

• Independent events: Two events E and F are independent if the occurrence of one does not affect the probability of the other.

• Dependent events: Two events E and F are dependent if the occurrence of one affects the probability of the other.

Examples:

• Rolling two dice: The outcome of the first die does not affect the outcome of the second die (independent).

• Getting a bonus and buying a house: The event of receiving a bonus may affect the probability of buying a large house (dependent).

Independent vs Disjoint Events

It is important to distinguish between independent and disjoint (mutually exclusive) events.

• Disjoint events: If one event occurs, the other cannot occur. Thus, $P(E \text{ and } F) = 0$.

• Independent events: The occurrence of one event does not change the probability of the other.

• Disjoint events (except for the empty set) are not independent.

Multiplication Rule for Independent Events

The multiplication rule allows us to compute the probability of the intersection of independent events.

• If E and F are independent, then:

$P(E \text{ and } F) = P(E) \cdot P(F)$

Example: Roll a fair die twice. Let A = "6 on first roll", B = "6 on second roll". Since the rolls are independent:

• $P(A) = \frac{1}{6}$, $P(B) = \frac{1}{6}$

• $P(A \text{ and } B) = \frac{1}{6} \times \frac{1}{6} = \frac{1}{36}$

Independence: Example with Simple Random Sample (SRS)

Consider a set S = {1,2,3,4,5}. Take a simple random sample (SRS) of size 2. Let A = "1 is chosen", B = "2 is chosen". These events are not independent because if A occurs, the probability of B decreases (since sampling is without replacement).

• Cannot use $P(A \text{ and } B) = P(A)P(B)$ for dependent events.

Probability Distributions: Example with Replacement

When drawing two cards from a deck with replacement (reshuffling after each draw), the events are independent.

• Let A = "1st card is a heart", B = "2nd card is a heart".

• $P(A) = \frac{13}{52} = \frac{1}{4}$, $P(B) = \frac{1}{4}$

• $P(A \text{ and } B) = \frac{1}{4} \times \frac{1}{4} = \frac{1}{16}$

Multiplication Rule for n Independent Events

For multiple independent events, the multiplication rule generalizes as follows:

$P(E_1 \text{ and } E_2 \text{ and } \ldots \text{ and } E_n) = P(E_1) \cdot P(E_2) \cdot \ldots \cdot P(E_n)$

Computing "At Least" Probabilities

To find the probability that at least one event occurs, it is often easier to use the complement rule.

• Complement Rule: $P(\text{at least one}) = 1 - P(\text{none})$

Example: Probability that at least one male out of 1000 aged 24 dies in a year, given the probability of surviving is 0.9986:

• $P(\text{at least one dies}) = 1 - (0.9986)^{1000} \approx 0.7536$

Summary: Rules of Probability

The following table summarizes key probability rules:

Rule	Description
1	Probability of any event is between 0 and 1: $0 \leq P(E) \leq 1$
2	Sum of probabilities of all outcomes in the sample space is 1
3	If E and F are disjoint: $P(E \text{ or } F) = P(E) + P(F)$
4	Complement: $P(E^c) = 1 - P(E)$
5	If E and F are independent: $P(E \text{ and } F) = P(E)P(F)$

Note: Use the Addition Rule for "or" probabilities and the Multiplication Rule for "and" probabilities.

Conditional Probability

Definition and Notation

Conditional probability quantifies the probability of an event given that another event has occurred.

• Notation: $P(F|E)$ is "the probability of F given E".

Conditional Probability: Example

Suppose a die is rolled. $P(3) = \frac{1}{6}$. If told the outcome is odd, $P(3|\text{odd}) = \frac{1}{3}$, since the sample space is reduced to {1,3,5}.

Conditional Probability Rule

The conditional probability of F given E is:

$P(F|E) = \frac{P(E \text{ and } F)}{P(E)}$

Alternatively, if outcomes are equally likely:

$P(F|E) = \frac{N(E \text{ and } F)}{N(E)}$

where N(E and F) is the number of outcomes in both E and F, and N(E) is the number of outcomes in E.

Conditional Probability: Real-World Example

Given: 19.1% of murder victims are aged 20-24, and 16.6% are 20-24 year old males. What is the probability a randomly selected murder victim is male, given they are 20-24?

$P(\text{male} | \text{age 20-24}) = \frac{0.166}{0.191} \approx 0.8691$ (or 86.9%)

General Multiplicative Rule

For any two events E and F, the probability that both occur is:

$P(E \text{ and } F) = P(E) \cdot P(F|E)$

This generalizes the multiplication rule to dependent events.

Multiplicative Rule: Example 1

Probability a speeding driver is pulled over is 0.8. Probability of getting a ticket if pulled over is 0.9. Probability a speeding driver is pulled over and gets a ticket:

$P(\text{pulled over and ticket}) = 0.8 \times 0.9 = 0.72$

Multiplicative Rule: Example 2 (Sampling Without Replacement)

Suppose 100 circuits, 5 defective. Manager selects 2 at random. Probability at least 1 is defective (shipment rejected):

• Approach 1 (Tree Diagram): Calculate all possible outcomes and sum probabilities where at least one is defective.

• Approach 2 (Complement Rule): Compute probability both are not defective, then subtract from 1.

$P(\text{at least 1 defective}) = 1 - P(\text{none defective}) = 1 - \left(\frac{95}{100} \times \frac{94}{99}\right) = 1 - 0.902 = 0.098$

General Rule for Sampling and Independence

When sampling without replacement from a large population, events can be treated as independent if the sample size is less than 5% of the population.

• Two events E and F are independent if $P(E|F) = P(E)$ or $P(F|E) = P(F)$.

Probability, Exercise

Given $P(A) = 0.4$ and $P(B) = 0.7$, with no further assumptions, show that $P(A \text{ and } B)$ is greater than 0.1 and less than 0.4.

• Hint: Use the fact that $P(A \text{ and } B) \geq P(A) + P(B) - 1$ and $P(A \text{ and } B) \leq \min\{P(A), P(B)\}$.

Additional info: The notes provide a comprehensive overview of independence, conditional probability, and related probability rules, with examples and applications relevant to college-level statistics.