Conditional Probability, Bayes’s Formula, and Independence: Structured Study Notes

Sample Spaces and Equally Likely Outcomes

Runs of Wins and Losses

In probability, a run refers to a consecutive sequence of similar outcomes, such as wins or losses. When outcomes are equally likely, combinatorial methods can be used to compute probabilities related to runs.

• Sample Space: The set of all possible outcomes of an experiment.

• Equally Likely Outcomes: If all orderings of n wins and m losses are equally likely, the total number of orderings is $\frac{(n+m)!}{n!m!} = \binom{n+m}{n}$.

• Probability of r Runs of Wins: The probability that there are exactly r runs of wins is given by:

$P(\{r \text{ runs of wins}\}) = \frac{\binom{m+1}{r} \binom{n-1}{r-1}}{\binom{m+n}{n}}, \quad r \geq 1$

• Example: For n = 8, m = 6, the probability of 7 runs is $\frac{\binom{7}{7} \binom{7}{6}}{\binom{14}{8}} = \frac{1}{429}$.

Probability as a Continuous Set Function

Increasing and Decreasing Sequences of Events

Probability can be extended to sequences of events, allowing for limits and continuity.

• Increasing Sequence: $E_1 \subset E_2 \subset \cdots$

• Decreasing Sequence: $E_1 \supset E_2 \supset \cdots$

• Limit of Events: $\lim_{n \to \infty} E_n$ is defined as the union (for increasing) or intersection (for decreasing) of all $E_n$.

• Continuity of Probability: $\lim_{n \to \infty} P(E_n) = P(\lim_{n \to \infty} E_n)$ for both increasing and decreasing sequences.

Example: Probability and a Paradox

Consider an urn experiment with infinite balls and withdrawals. The outcome depends on the withdrawal method:

• If only balls numbered 10n are withdrawn, infinitely many balls remain.

• If each ball is eventually withdrawn, the urn is empty at the end.

• If withdrawals are random, the probability any specific ball remains is 0; thus, with probability 1, the urn is empty.

Probability as a Measure of Belief

Subjective Probability

Probability can represent personal belief, not just long-run frequency. This is called the subjective view of probability.

• Consistency: Subjective probabilities should satisfy the axioms of probability.

• Example: In a 7-horse race, if you assign probabilities to each horse, you can use these to make rational betting decisions.

Summary of Probability Axioms and Properties

• Sample Space (S): The set of all possible outcomes.

• Event: A subset of S.

• Union: $\bigcup_{i=1}^n A_i$ is the event that at least one $A_i$ occurs.

• Intersection: $\bigcap_{i=1}^n A_i$ is the event that all $A_i$ occur.

• Complement: $A^c$ is the event that A does not occur.

• Mutually Exclusive: $AB = \emptyset$ means A and B cannot both occur.

• Probability Function: $P(A)$ satisfies:

  • 0 ≤ $P(A)$ ≤ 1

  • $P(S) = 1$

  • If $A_i$ are mutually exclusive, $P(\bigcup_{i=1}^\infty A_i) = \sum_{i=1}^\infty P(A_i)$

• Complement Rule: $P(A^c) = 1 - P(A)$

• Inclusion-Exclusion Principle: $P(A \cup B) = P(A) + P(B) - P(AB)$

• General Inclusion-Exclusion:

$ P\left(\bigcup_{i=1}^n A_i\right) = \sum_{i=1}^n P(A_i) - \sum_{i

• Equally Likely Outcomes: If S is finite and all outcomes are equally likely, $P(A) = \frac{|A|}{|S|}$

Conditional Probability

Definition and Calculation

Conditional probability quantifies the likelihood of an event given that another event has occurred.

• Definition: If $P(F) > 0$, then $P(E|F) = \frac{P(EF)}{P(F)}$

• Interpretation: The probability of E given F is the probability of both E and F occurring, divided by the probability of F.

• Example: If two dice are tossed and the first die is a 3, the probability the sum is 8 is $\frac{1}{6}$.

Multiplication Rule

The multiplication rule allows calculation of the probability of intersection of multiple events.

$P(E_1E_2\cdots E_n) = P(E_1)P(E_2|E_1)P(E_3|E_1E_2)\cdots P(E_n|E_1\cdots E_{n-1})$

• Example: Probability that each pile has exactly one ace when dividing a deck into four piles:

• $P(E_1E_2E_3E_4) = P(E_1)P(E_2|E_1)P(E_3|E_1E_2)P(E_4|E_1E_2E_3)$

Bayes’s Formula

Law of Total Probability and Bayes’s Theorem

Bayes’s formula updates probabilities based on new evidence.

• Law of Total Probability: $P(E) = P(E|F)P(F) + P(E|F^c)P(F^c)$

• Bayes’s Formula: For mutually exclusive and exhaustive events $F_1, \ldots, F_n$:

$P(F_j|E) = \frac{P(E|F_j)P(F_j)}{\sum_{i=1}^n P(E|F_i)P(F_i)}$

• Odds: The odds of event H are $\frac{P(H)}{P(H^c)}$

• Odds Update: $\frac{P(H|E)}{P(H^c|E)} = \frac{P(H)}{P(H^c)} \cdot \frac{P(E|H)}{P(E|H^c)}$

• Example: Medical testing, DNA evidence, and subjective probability updates.

Independent Events

Definition and Properties

Events are independent if the occurrence of one does not affect the probability of the other.

• Definition: E and F are independent if $P(EF) = P(E)P(F)$

• Symmetry: Independence is symmetric: if E is independent of F, F is independent of E.

• Extension: For three events E, F, G, independence requires:

  • $P(EFG) = P(E)P(F)P(G)$

  • $P(EF) = P(E)P(F)$, $P(EG) = P(E)P(G)$, $P(FG) = P(F)P(G)$

• Generalization: A set of events $E_1, \ldots, E_n$ is independent if every subset satisfies $P(E_{i_1}\cdots E_{i_r}) = P(E_{i_1})\cdots P(E_{i_r})$

Examples and Applications

• Coin Tosses: Each toss is independent; probability of k successes in n trials is $\binom{n}{k}p^k(1-p)^{n-k}$

• Parallel Systems: Probability system functions is $1 - \prod_{i=1}^n (1 - p_i)$

• Gambler’s Ruin: Probability calculations for games with independent trials.

Conditional Probability as a Probability Function

Properties

Conditional probability satisfies the axioms of probability:

• 0 ≤ $P(E|F)$ ≤ 1

• $P(S|F) = 1$

• If $E_i$ are mutually exclusive, $P(\bigcup_{i=1}^\infty E_i|F) = \sum_{i=1}^\infty P(E_i|F)$

Table: Key Probability Formulas

Concept	Formula (LaTeX)	Description
Conditional Probability	$P(E|F) = \frac{P(EF)}{P(F)}$	Probability of E given F
Multiplication Rule	$P(E_1E_2\cdots E_n) = P(E_1)P(E_2|E_1)\cdots P(E_n|E_1\cdots E_{n-1})$	Probability of intersection of events
Law of Total Probability	$P(E) = P(E|F)P(F) + P(E|F^c)P(F^c)$	Probability by conditioning
Bayes’s Formula	$P(F_j|E) = \frac{P(E|F_j)P(F_j)}{\sum_{i=1}^n P(E|F_i)P(F_i)}$	Update probability with evidence
Independence	$P(EF) = P(E)P(F)$	Events E and F are independent
Inclusion-Exclusion	$P(A \cup B) = P(A) + P(B) - P(AB)$	Probability of union

Additional info:

• Some examples and applications were expanded for clarity and completeness.

• Table entries were inferred from context and standard probability theory.