Probability Models and Set Theory in Statistics

Probability Models

The Elements of a Probability Model

A probability model is a mathematical framework for representing random experiments and quantifying the likelihood of various outcomes. It consists of three essential elements:

• Description of the underlying probability experiment: A clear statement of the process or experiment being studied.

• Outcomes of the experiment: The possible results that can occur from the experiment.

• Rules to assign numbers between 0 and 1 to subsets of outcomes: These rules, called probability measures, assign probabilities to events (subsets of the sample space).

Key Terms in Probability

• Probability Experiment: A process that leads to well-defined results called outcomes.

• Outcome: A single possible result of a probability experiment.

• Sample Space (S): The set of all possible outcomes of a probability experiment.

• Event: Any subset of the sample space.

• Axioms of Probability Theory: Fundamental rules that probabilities must satisfy (see below).

Probability Experiment

A probability experiment is a test or process that allows us to observe possible outcomes. Since there are usually multiple possible results, probability helps us estimate how likely each outcome is to occur.

Outcome

An outcome is a single result from a probability experiment. For example, rolling a die and getting a 4 is one outcome.

Sample Space and Events

The sample space is the set of all possible outcomes. An event is any subset of the sample space. Each outcome is called a sample point.

Examples of Sample Spaces

• Example 2.1: Tossing a fair die once:

  • If interested in the number on top: S1 = {1, 2, 3, 4, 5, 6}

  • If interested in even/odd: S2 = {even, odd}

Tree Diagrams

Tree diagrams are visual tools used to list all possible outcomes, especially useful for multi-stage experiments.

Examples Using Tree Diagrams

• Rolling a die and tossing a coin: Each branch represents a possible outcome for each stage.

• Flipping a coin once, and again if heads occurs: The tree diagram shows all possible sequences of outcomes.

Continuous Sample Spaces

• Example 2.4: S = { t | t ≥ 0 }, where t is the lifetime (in years) of a component. Event A = { t | 0 ≤ t < 5 } means the component lasts less than 5 years. Event B = { x | x is an even factor of 7 } = {1} (since 2 and 7 are not both factors of 7).

Set Theory in Probability

Basic Set Concepts

• Set: A collection of objects (elements or members).

• Universal Set (U): The set containing all possible objects under consideration.

• Every set is a subset of the universal set.

Notation

• Belong to: a ∈ A means 'a is an element of A'.

• Not belong to: a ∉ A means 'a is not an element of A'.

• Subset: A ⊆ B means every element of A is also in B.

Ways to Write a Set

• Description in words

• Listing method (e.g., {1, 2, 3})

• Set-builder notation (e.g., {x | x is a natural number})

Examples of Sets

• Natural numbers, whole numbers, integers, rational numbers, irrational numbers, real numbers, complex numbers.

• First six prime numbers: A = {2, 3, 5, 7, 11, 13}

Empty Set

The empty set (or null set) contains no elements and is denoted by { } or ϕ.

Cardinality

The cardinality of a set A, denoted N(A) or |A|, is the number of elements in A. The cardinality of the empty set is 0.

Power Set

The power set of A, denoted P(A), is the set of all possible subsets of A, including the empty set and A itself.

• For A = {1, 2, 3, 4}, P(A) contains 24 = 16 subsets.

Power Set vs Universal Set

• The power set contains all subsets of a given set.

• The universal set contains all elements under consideration in a context.

Set Operations

• Union: $A \cup B = \{ x | x \in A \text{ or } x \in B \}$

• Intersection: $A \cap B = \{ x | x \in A \text{ and } x \in B \}$

• Complement: $A' = \{ x | x \in U, x \notin A \}$

• Difference: $A - B = \{ x | x \in A \text{ and } x \notin B \}$

Disjoint (Mutually Exclusive) Events

Events are disjoint if they cannot occur at the same time: $A \cap B = \varnothing$.

Venn Diagrams

Venn diagrams visually represent relationships between sets and events within the sample space. The sample space is shown as a rectangle, and events as circles within it.

Examples of Set Operations

• Given A = {a, b, c} and B = {b, c, d, e}, Venn diagrams can illustrate A, A ∪ B, A ∩ B, A − B, and Ac.

• Given M = {x | 3 < x < 9} and N = {x | 5 < x < 12}, calculate M ∪ N, M ∩ N, M − N, Mc.

Exercise Example

• Given U = {1,2,3,4,5,6,7,8,9,10}, A = {1,3,5,7,9}, B = {2,3,5,7}, find A', B', A ∪ B, A ∩ B, A' ∪ B', A' ∩ B', (A ∪ B)'.

Axioms of Probability Theory

Probability is governed by three fundamental axioms:

1. $0 \leq P(E) \leq 1$ for any event E

2. $P(U) = 1$, where U is the sample space

3. If $\{E_i\}$ are mutually exclusive events, then $P(\bigcup E_i) = \sum_i P(E_i)$

Discrete vs Continuous Probability Models

• Discrete Probability Models: Outcomes can be listed (finite or countably infinite).

• Continuous Probability Models: Outcomes form a continuum (e.g., all real numbers in [0,1]).

Equally Likely Outcomes

Outcomes are equally likely if each has the same probability. If there are n outcomes, each has probability 1/n.

Probability with Equally Likely Outcomes

If S is a sample space with n equally likely outcomes and E is an event with r outcomes:

$P(E) = \frac{N(E)}{N(S)} = \frac{r}{n}$

Examples

• Tossing 3 fair coins: How many ways does the sum equal 2? What is the probability the sum is exactly 1?

• Tossing a coin twice: Probability of at least one head?

• Randomly selecting a student from a class with different majors: Probability of selecting an industrial engineering major or a civil/electrical engineering major?

• Loaded die: Even number is twice as likely as odd. Find probability of number less than 4.

Addition and Complement Rules

• Complement Rule: $P(A^c) = 1 - P(A)$

• Addition Rule: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$

• Extension for Three Events: $P(A \cup B \cup C) = P(A) + P(B) + P(C) - P(A \cap B) - P(A \cap C) - P(B \cap C) + P(A \cap B \cap C)$

Examples

• Probability of getting at least one job offer from two companies, given probabilities for each and both.

• Probability of getting a total of 7 or 11 when two dice are tossed.

• Probability a car buyer chooses a car of a certain color, given probabilities for each color.

• Probability a randomly selected cable is too large or meets specifications, given probabilities for each scenario.

De Morgan’s Laws

• $(A \cap B)^c = A^c \cup B^c$

• $(A \cup B)^c = A^c \cap B^c$

Verification with Venn Diagrams

Venn diagrams can be used to visually verify De Morgan’s laws and other set relationships.

Probability Tables and Venn Diagram Boxes

Probability tables can be used to summarize the relationships between events and their complements, as well as intersections and unions.

A	A'	Total probability
Pr(A)	Pr(A')	1

B	Pr(B)
B'	Pr(B')
Total probability	1

	A	A'	Total probability
B	Pr(A∩B)	Pr(A'∩B)	Pr(B)
B'	Pr(A∩B')	Pr(A'∩B')	Pr(B')
Total probability	Pr(A)	Pr(A')	1

Additional info: The tables and Venn diagram boxes visually reinforce the partitioning of the sample space and the relationships between events, their complements, and intersections, which are foundational for understanding probability calculations and rules.