Chapter 5: Probability – Foundations, Models, and Rules

Probability: Fundamental Concepts

Definitions

Probability is a branch of mathematics that deals with quantifying uncertainty and predicting the likelihood of various outcomes in random experiments.

• Experiment: Any process or action with uncertain results. For example, flipping a coin or rolling a die.

• Sample Space (S): The set of all possible outcomes of an experiment. For a coin flip, $S = \{H, T\}$.

• Event: Any collection of outcomes from the sample space. Events can be simple (one outcome) or compound (multiple outcomes).

• Simple Event: An event consisting of a single outcome, denoted $e_i$.

Examples of Sample Spaces and Events

Understanding sample spaces and events is crucial for calculating probabilities.

Experiment	Sample Space	Simple Events	Example Events
Coin flip	S = {H, T}	e1 = {H}, e2 = {T}	E = {H}, F = {T}
Roll a die	S = {1,2,3,4,5,6}	e1 = {1}, e2 = {2}, ..., e6 = {6}	E = {even numbers}, F = {odd numbers}
Flip two coins	S = {HH, HT, TH, TT}	e1 = {HH}, e2 = {HT}, e3 = {TH}, e4 = {TT}	E = {at least one head}, F = {both tails}
Draw two balls (G=green, B=blue)	S = {GG, GB, BG, BB}	e1 = {GG}, e2 = {GB}, e3 = {BG}, e4 = {BB}	E = {at least one green}, F = {no green}

Random Processes & Simulation

Randomness and Simulation

A random process is one whose outcome cannot be predicted with certainty. Simulation is a technique used to mimic random events, either physically (e.g., flipping a coin) or virtually (e.g., using a computer).

• Simulations help estimate probabilities by repeating experiments and observing outcomes.

• Example: Flipping a coin 100 times and recording the proportion of heads.

Simulation Table Example

Number of flips	Outcome	Proportion of Heads
1	Tail	0
2	Tail, Head	0.5
3	Tail, Head, Head	0.66
...	...	...

Simulation Graphs

Graphs of simulation results show how the proportion of heads approaches the theoretical probability (0.5) as the number of flips increases.

Probability and the Law of Large Numbers

Probability of Getting a Head

As the number of coin flips increases, the observed proportion of heads approaches the theoretical probability:

• $P(\text{Head}) = 0.5$

• $P(\text{Tail}) = 0.5$

Law of Large Numbers (LLN)

The Law of Large Numbers states that as the number of repetitions of a probability experiment increases, the observed proportion of a particular outcome approaches its theoretical probability.

• Example: Flipping a coin many times, the proportion of heads approaches 0.5.

• Example: Rolling a die, the probability of each outcome approaches $\frac{1}{6}$.

Rules of Probability

Basic Rules

• Rule 1: Probabilities are between 0 and 1, inclusive: $0 \leq P(E) \leq 1$

• Rule 2: The sum of the probabilities of all simple events in the sample space is 1: $P(e_1) + P(e_2) + \ldots + P(e_n) = 1$

Probability Model

A probability model lists all possible outcomes and their probabilities, ensuring the rules above are satisfied.

Color	Probability
Brown	0.12
Yellow	0.15
Red	0.12
Blue	0.23
Orange	0.23
Green	0.15

All probabilities sum to 1: 0.12 + 0.15 + 0.12 + 0.23 + 0.23 + 0.15 = 1

Types of Events

• Impossible Event: Probability is 0.

• Certain Event: Probability is 1.

• Unusual Event: Probability is low (often less than 0.05).

Methods of Computing Probability

Three Basic Methods

• Empirical (Relative Frequency) Method: Probability is estimated by the ratio of the number of times an event occurs to the total number of trials. $P(E) \approx \frac{\text{frequency of } E}{\text{number of trials}}$

• Classical (Theoretical) Method: Used when all outcomes are equally likely. Probability is the ratio of the number of favorable outcomes to the total number of possible outcomes. $P(E) = \frac{\text{number of ways } E \text{ can occur}}{\text{number of possible outcomes}}$

• Subjective Method: Probability is based on personal judgment or expert opinion, not on actual data or equally likely outcomes.

Example: Empirical Probability

Distance (miles)	Number of Employees
10-19	309
20-29	257
30-39	78
40-49	2

Probability that a randomly selected employee travels 10-29 miles: $P = \frac{309 + 257}{309 + 257 + 78 + 2} = \frac{566}{646} \approx 0.876$ Additional info: The calculation above is inferred from the table and context.

Example: Classical Probability

• Probability of drawing a yellow candy from a bag with 6 yellow out of 30 total candies: $P(\text{yellow}) = \frac{6}{30} = 0.2$

• Probability of drawing a blue candy (2 blue out of 30): $P(\text{blue}) = \frac{2}{30} \approx 0.067$

Example: Subjective Probability

• An economist predicts a 20% chance of recession next year. This is a subjective probability.

Additional Probability Rules

Complement Rule

The complement of an event E, denoted $E^c$, consists of all outcomes in the sample space that are not in E.

• $P(E^c) = 1 - P(E)$

• Example: If $P(\text{own a dog}) = 0.316$, then $P(\text{do not own a dog}) = 1 - 0.316 = 0.684$

Addition Rule

For two events E and F:

• General Addition Rule: $P(E \text{ or } F) = P(E) + P(F) - P(E \text{ and } F)$

• Example: Probability of drawing a king or a club from a deck: $P(\text{king or club}) = P(\text{king}) + P(\text{club}) - P(\text{king of clubs})$

Multiplication Rule for Independent Events

If two events E and F are independent, the probability that both occur is:

• $P(E \text{ and } F) = P(E) \times P(F)$

• Example: Probability of getting a head on a coin and a 5 on a die: $P(\text{head and 5}) = P(\text{head}) \times P(5) = 0.5 \times \frac{1}{6} = 0.083$

Summary Table: Probability Methods

Method	Description	Formula	Example
Empirical	Based on observed data	$P(E) \approx \frac{\text{frequency of } E}{\text{number of trials}}$	Survey results
Classical	Assumes equally likely outcomes	$P(E) = \frac{m}{n}$	Rolling a die
Subjective	Based on judgment	N/A	Expert prediction

Key Terms

• Experiment

• Sample Space

• Event

• Simple Event

• Random Process

• Simulation

• Law of Large Numbers

• Probability Model

• Empirical, Classical, Subjective Probability

• Complement

• Addition Rule

• Multiplication Rule

Additional info: Some examples and table entries have been expanded for clarity and completeness.