Fundamental Concepts in Probability: Sample Spaces, Events, and Probability Rules

Sample Spaces and Events

Definitions and Basic Concepts

Probability theory begins with the definition of experiments, sample points, events, and sample spaces. These foundational concepts are essential for understanding how probabilities are assigned and calculated.

• Experiment: A process that generates observations or outcomes. Example: Toss a die once.

• Sample Point: The most basic outcome of an experiment. Example: Toss a coin once; possible sample points are 'Heads' or 'Tails'.

• Event: A specific collection of sample points. Example: Toss a coin three times; the event 'at least one head' includes all sample points with at least one 'Heads'.

• Sample Space (S): The set of all possible sample points for an experiment. Example: Toss two dice; the sample space consists of all possible pairs of numbers (1,1), (1,2), ..., (6,6).

Probability Assignment

The probability of a sample point $P(E)$ is a number between 0 and 1 that represents the relative frequency of occurrence of event E.

• For any sample point $E$, $0 \leq P(E) \leq 1$

• The sum of probabilities of all sample points in S is 1: $\sum_{i} P(E_i) = 1$

Probability of an Event

The probability of an event A is the sum of the probabilities of the sample points in A:

• If $A = \{E_1, E_2, ..., E_k\}$, then $P(A) = P(E_1) + P(E_2) + ... + P(E_k)$

Example: Toss a fair coin 3 times. Find the probability that (1) exactly one tail occurs, (2) at least one tail occurs, (3) the event that not all tails occur.

Equally Likely Outcomes

Definition and Calculation

When all outcomes in the sample space are equally likely, the probability of an event A is:

• $P(A) = \frac{n}{N}$ where n = number of elements in A, N = number of elements in S.

Example: Two fair dice are tossed. What is the probability that (i) a '4' appears on exactly one die? (ii) The sum of the two numbers is at least 10?

Set Operations and Events

Union, Intersection, and Complement

Events can be combined using set operations:

• Union (A ∪ B): Event that occurs if either A or B (or both) occur.

• Intersection (A ∩ B): Event that occurs if both A and B occur.

• Complement (Ac): Event that occurs if A does not occur.

Example: Toss a balanced die once. Let A = {4, 5, 6} (outcome ≥ 4), B = {1, 3, 4, 6} and C = {2, 4, 6} (outcome is even). Find (i) P(A ∪ B), (ii) P(B ∩ C), (iii) P(Bc).

Additive Rule of Probability

Rule and Application

The probability that either event A or event B occurs is given by:

• $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

Example: Suppose 75% of households subscribe to a French newspaper, 50% to an English newspaper, and 32% to both. What is the probability that a household subscribes to (a) at least one newspaper? (b) exactly one newspaper?

Mutually Exclusive Events

Events A and B are mutually exclusive or disjoint if they have no outcomes in common. For mutually exclusive events, $P(A \cap B) = 0$.

Conditional Probability

Definition and Formula

The conditional probability of A given B is:

• $P(A|B) = \frac{P(A \cap B)}{P(B)}$

Example: Toss a fair die once. Let A = {1, 3, 4}, B = {2, 3, 5, 6}, C = {1, 2, 4, 5}. Find (i) P(A|B), (ii) P(B|A), (iii) P(B|C), (iv) P(C|A).

Multiplicative Rule of Probability

Rule and Application

The probability of the intersection of two events is:

• $P(A \cap B) = P(A|B)P(B)$

• Also, $P(A \cap B) = P(B|A)P(A)$

Example: 12 marbles are randomly drawn one by one without replacement from a box containing 2 blue, 3 red, and 7 white marbles. Let A = {2 blue marbles are drawn}, B = {2 red marbles are drawn}. Find P(A) and P(B).

Independence of Events

Definition and Properties

Events A and B are independent if the occurrence of one does not affect the probability of the other. For independent events:

• $P(A \cap B) = P(A)P(B)$

Events A, B, and C are independent if $P(A \cap B \cap C) = P(A)P(B)P(C)$.

Example: A fair die is tossed once. Let A = {1, 2, 4}, B = {2, 5, 6}, C = {2, 3, 4, 6}. Are events A and B independent? Are events B and C independent?

Random Sampling

Counting and Combinatorics

Random sampling involves selecting elements from a set, often using combinatorial methods. The number of ways of selecting m elements from N elements is:

• $N! = N \times (N-1) \times (N-2) \times \cdots \times 2 \times 1$

• $\binom{N}{m} = \frac{N!}{m!(N-m)!}$

Example: A retail car company has 10 Japanese cars and 12 American cars. If 5 cars are rented, what is the probability that (i) 3 of them are Japanese cars? (ii) at most 2 of them are Japanese cars?

The Law of Total Probability

Statement and Application

If $A_1, A_2, ..., A_n$ are mutually exclusive and exhaustive events, then for any event B:

• $P(B) = P(A_1)P(B|A_1) + P(A_2)P(B|A_2) + \cdots + P(A_n)P(B|A_n)$

• Or, $P(B) = \sum_{i=1}^{n} P(A_i)P(B|A_i)$

Example: A store sells 3 brands of digital cameras. The camera sales for brands 1, 2, and 3 are 45%, 30%, and 25% respectively. Each manufacturer offers a one-year warranty. The percentages of cameras that require warranty repair work are 25%, 20%, and 10% for brands 1, 2, and 3. Find the probability that a randomly selected camera needs repair work.

Bayes' Theorem

Statement and Application

Bayes' Theorem allows us to update probabilities based on new information. If $A_1, A_2, ..., A_n$ are mutually exclusive and exhaustive events with $P(A_i) > 0$, then for any event B with $P(B) > 0$:

• $P(A_k|B) = \frac{P(A_k)P(B|A_k)}{\sum_{i=1}^{n} P(A_i)P(B|A_i)}$

Example: Only one in 1000 patients has a rare disease. If a patient actually has the disease, a positive test result will be diagnosed 99% of the time. If a patient without the disease is tested, the result will be positive 2% of the time. For a randomly selected patient, what is the probability that the patient has the disease given a positive test result?

Summary Table: Probability Rules

Rule	Formula	When to Use
Additive Rule	$P(A \cup B) = P(A) + P(B) - P(A \cap B)$	For union of two events
Multiplicative Rule	$P(A \cap B) = P(A|B)P(B)$	For intersection of two events
Conditional Probability	$P(A|B) = \frac{P(A \cap B)}{P(B)}$	Probability of A given B
Independence	$P(A \cap B) = P(A)P(B)$	When events are independent
Law of Total Probability	$P(B) = \sum_{i=1}^{n} P(A_i)P(B|A_i)$	For partitioned sample spaces
Bayes' Theorem	$P(A_k|B) = \frac{P(A_k)P(B|A_k)}{\sum_{i=1}^{n} P(A_i)P(B|A_i)}$	Updating probabilities with new evidence

Additional info: Some examples and explanations have been expanded for clarity and completeness. The notes cover foundational probability concepts suitable for college-level statistics courses.