Fundamental Concepts in Probability: Sample Spaces, Events, and Probability Rules

Sample Spaces and Events

Definitions and Basic Concepts

Probability theory begins with the definition of experiments, sample points, events, and sample spaces. These foundational concepts are essential for understanding how probabilities are assigned and calculated.

• Experiment: A process that generates observations or outcomes. Example: Tossing a die once.

• Sample Point: The most basic outcome of an experiment. Example: Tossing a coin once yields 'Heads' or 'Tails'.

• Event: A specific collection of sample points. Example: Tossing a coin three times and getting at least one 'Heads'.

• Sample Space (S): The set of all possible sample points for an experiment. Example: Tossing two dice: S contains all possible pairs of numbers (1,1), (1,2), ..., (6,6).

Probability of Sample Points and Events

The probability of a sample point $P(E)$ is a number between 0 and 1 that represents the relative frequency of occurrence of E.

• $0 \\leq P(E) \\leq 1$ for any sample point E.

• $\sum P(E_i) = 1$ for all sample points in S.

The probability of an event A is the sum of the probabilities of the sample points in A:

$A = \{E_1, E_2, ..., E_k\} \implies P(A) = P(E_1) + P(E_2) + ... + P(E_k)$

Equally Likely Outcomes

If all outcomes in the sample space S are equally likely, the probability of event A is:

$P(A) = \frac{n}{N}$

• n = number of elements in A

• N = total number of elements in S

Example: Tossing two fair dice. What is the probability that the sum is 7?

Operations on Events

Union, Intersection, and Complement

Events can be combined using set operations:

• Union (A ∪ B): The event that at least one of A or B occurs.

• Intersection (A ∩ B): The event that both A and B occur.

• Complement (Ac): The event that A does not occur.

Example: Tossing a balanced die once. Let A = {4, 5, 6} (outcome ≥ 4), B = {1, 3, 4, 6} (even or odd), C = {2, 4, 6} (even outcome). Find (i) P(A ∪ B), (ii) P(B ∩ C), (iii) P(Bc).

Probability Rules

Additive Rule of Probability

The probability that at least one of two events occurs is:

$P(A \\cup B) = P(A) + P(B) - P(A \\cap B)$

Example: Suppose 75% of households subscribe to a French newspaper, 50% to an English newspaper, and 32% to both. What is the probability that a household subscribes to at least one newspaper?

Mutually Exclusive Events

Events A and B are mutually exclusive or disjoint if they have no sample points in common. For such events:

$P(A \\cap B) = 0$

Conditional Probability

The probability of event A given that event B has occurred is:

$P(A|B) = \frac{P(A \\cap B)}{P(B)}$

Example: Toss a fair die. Let A = {1, 3, 4}, B = {2, 3, 5, 6}, C = {1, 2, 4, 5}. Find (i) P(A|B), (ii) P(B|C), (iii) P(C|A).

Multiplicative Rule of Probability

The probability that both A and B occur is:

$P(A \\cap B) = P(A)P(B|A)$

Also, $P(A \\cap B) = P(B)P(A|B)$

Independence of Events

Events A and B are independent if the occurrence of one does not affect the probability of the other:

$P(A \\cap B) = P(A)P(B)$

Events A, B, and C are independent if:

$P(A \\cap B \\cap C) = P(A)P(B)P(C)$

Random Sampling and Combinatorics

Factorials and Counting

The number of ways of selecting m elements from N elements is given by the binomial coefficient:

$N! = N(N-1)(N-2)...(3)(2)(1)$

$\binom{N}{m} = \frac{N!}{m!(N-m)!}$

• $\binom{N}{0} = 1$

• $\binom{N}{1} = N$

• $\binom{N}{N} = 1$

Example: A company has 10 Japanese cars and 12 American cars. What is the probability that 3 out of 5 randomly selected cars are Japanese?

The Law of Total Probability

Statement and Application

If $A_1, A_2, ..., A_n$ are mutually exclusive and exhaustive events, then for any event B:

$P(B) = P(A_1)P(B|A_1) + P(A_2)P(B|A_2) + ... + P(A_n)P(B|A_n)$

Or, in summation form:

$P(B) = \sum_{i=1}^{n} P(A_i)P(B|A_i)$

Example: A store sells 3 brands of cameras. If the percentage of cameras requiring warranty repair is different for each brand, what is the probability that a randomly selected camera needs repair?

Bayes' Theorem

Statement and Application

Bayes' Theorem allows us to update probabilities based on new information. If $A_1, A_2, ..., A_n$ are mutually exclusive and exhaustive events with $P(A_i) > 0$, then for any event B with $P(B) > 0$:

$P(A_k|B) = \frac{P(A_k)P(B|A_k)}{\sum_{i=1}^{n} P(A_i)P(B|A_i)}$

Example: If only 1 in 1000 patients has a rare disease, and a test for the disease is 99% accurate, what is the probability that a patient who tests positive actually has the disease?

Summary Table: Key Probability Rules

Rule	Formula (LaTeX)	Description
Additive Rule	$P(A \\cup B) = P(A) + P(B) - P(A \\cap B)$	Probability that at least one of A or B occurs
Multiplicative Rule	$P(A \\cap B) = P(A)P(B|A)$	Probability that both A and B occur
Conditional Probability	$P(A|B) = \frac{P(A \\cap B)}{P(B)}$	Probability of A given B has occurred
Independence	$P(A \\cap B) = P(A)P(B)$	Events A and B are independent
Law of Total Probability	$P(B) = \sum_{i=1}^{n} P(A_i)P(B|A_i)$	Probability of B over all partitions $A_i$
Bayes' Theorem	$P(A_k|B) = \frac{P(A_k)P(B|A_k)}{\sum_{i=1}^{n} P(A_i)P(B|A_i)}$	Posterior probability of $A_k$ given B

Additional info:

• Examples and applications are inferred from standard probability theory and textbook conventions.

• Some explanations are expanded for clarity and completeness.