Chapter 5: Probability – Foundations and Methods

Probability: Foundations and Methods

Introduction to Probability

Probability is a fundamental concept in statistics that quantifies the likelihood of events occurring in random processes. It provides a mathematical framework for analyzing experiments with uncertain outcomes and is essential for making informed decisions using data.

Random Processes and the Law of Large Numbers

Understanding Random Processes

• Random Process: A scenario where the outcome of any particular trial is unknown, but the proportion of a particular outcome approaches a specific value as the number of trials increases.

• Simulation: A technique to re-create random events, either physically (e.g., flipping a coin) or virtually (e.g., computer simulation), to observe outcome frequencies.

Example: Flipping a coin 100 times and plotting the proportion of heads after each toss demonstrates how the observed proportion stabilizes as the number of trials increases.

The Law of Large Numbers

• Law of Large Numbers: As the number of repetitions of a probability experiment increases, the observed proportion of a certain outcome approaches the theoretical probability of that outcome.

• This law is sometimes misinterpreted as the "Law of Averages," which is not a formal statistical law.

Basic Probability Concepts

Key Definitions

• Experiment: Any process with uncertain results that can be repeated.

• Sample Space (S): The set of all possible outcomes of an experiment.

• Event: Any collection of outcomes from a probability experiment. An event may consist of one or more outcomes.

• Simple Event: An event consisting of a single outcome.

Example: Rolling a Fair Die

• Outcomes: 1, 2, 3, 4, 5, 6

• Sample Space: S = {1, 2, 3, 4, 5, 6}

• Event E (even numbers): E = {2, 4, 6}

• A fair die has equally likely outcomes; a loaded die does not.

Rules of Probability

Probability Rules

• Rule 1: For any event E, $0 \leq P(E) \leq 1$

• Rule 2: The sum of the probabilities of all outcomes in the sample space is 1: $P(e_1) + P(e_2) + \cdots + P(e_n) = 1$

Probability Model

• A probability model lists all possible outcomes and their probabilities, satisfying the two rules above.

• If an event is impossible, its probability is 0; if certain, its probability is 1.

• An unusual event is one with a low probability of occurring.

Example: Probability Model for M&M Colors

Color	Probability
Brown	0.13
Yellow	0.14
Red	0.13
Blue	0.24
Orange	0.20
Green	0.16

Each probability is between 0 and 1, and their sum is 1, so this is a valid probability model.

Empirical (Experimental) Probability

Computing Probabilities Using the Empirical Method

• The probability of an event E is approximated by the relative frequency of E in a large number of trials:

$P(E) \approx \frac{\text{frequency of } E}{\text{number of trials}}$

Example: Insurance Claims

• Out of 182 teenage drivers, 24 filed a claim last year.

• Empirical probability: $P(\text{claim}) = \frac{24}{182} \approx 0.132$

• Interpretation: About 13 out of every 100 insured teenage drivers are expected to file a claim.

Example: Building a Probability Model from Survey Data

Means of Travel	Frequency	Probability
Drive alone	153	0.765
Carpool	22	0.11
Public transportation	10	0.05
Walk	5	0.025
Other means	3	0.015
Work at home	7	0.035

Interpretation: The probability that a randomly selected individual carpools to work is 0.11. It is unusual to select someone who walks to work (probability 0.025).

Classical (Theoretical) Probability

Computing Probabilities Using the Classical Method

• Requires equally likely outcomes.

• Formula: $P(E) = \frac{\text{number of ways } E \text{ can occur}}{\text{number of possible outcomes}}$

• Alternatively: $P(E) = \frac{N(E)}{N(S)}$ where $N(E)$ is the number of outcomes in event E, and $N(S)$ is the number of outcomes in the sample space.

Example: Rolling Two Fair Dice

• Sample space: 36 equally likely outcomes.

• Probability of rolling a seven: $P(7) = \frac{6}{36} = 0.167$

• Probability of rolling a two: $P(2) = \frac{1}{36} \approx 0.028$

• Rolling a seven is six times as likely as rolling a two.

Example: Selecting Random Samples

• Sophia randomly selects 2 out of 4 friends to attend a concert.

• Sample space (all possible pairs): Yolanda & Michael, Yolanda & Kevin, Yolanda & Marissa, Michael & Kevin, Michael & Marissa, Kevin & Marissa.

• Probability Michael and Kevin attend: $\frac{1}{6}$

• Probability Marissa attends: $\frac{3}{6} = 0.5$

Comparing Empirical and Classical Methods

• Empirical probability is based on observed data; classical probability is based on equally likely outcomes.

Example: In a survey of 500 families with three children, 180 had two boys and one girl. Empirical probability: $\frac{180}{500} = 0.36$. Classical probability (assuming equal likelihood): There are 8 possible outcomes, 3 of which have two boys and one girl, so $\frac{3}{8} = 0.375$.

Subjective Probability

Recognizing and Interpreting Subjective Probabilities

• Subjective Probability: A probability value based on personal judgment, intuition, or experience rather than on objective data or equally likely outcomes.

• Example: An economist predicting a 20% chance of recession next year.