Introduction to Probability: Concepts, Models, and Methods

Introduction to Probability

Overview

Probability is a foundational concept in statistics, providing a mathematical framework for quantifying uncertainty and randomness in experiments and real-world phenomena. This section introduces the basic principles, definitions, and methods used to calculate and interpret probabilities.

Randomness and Random Events

Definition of Randomness

• Random in statistics refers to more than just unpredictability or haphazardness. It describes situations where the outcome is uncertain, but a definite distribution of outcomes emerges if the situation is repeated many times under identical conditions.

• A random event is a situation in which:

  • The outcome is uncertain.

  • There is a definite distribution of outcomes after many repetitions under identical conditions.

Examples of Random Phenomena

• Tossing a fair coin and noting whether it lands heads (H) or tails (T).

• Selecting a simple random sample (SRS) from a population and recording opinions.

• Dealing two cards and checking if they are the same denomination.

• Predicting the outcome (win/loss) of a basketball game.

In each case, the distribution of outcomes becomes predictable after many repetitions.

Motivation for the Definition of Probability

Short-Term vs. Long-Term Behavior

• Random behavior is unpredictable in the short term but becomes predictable in the long term.

• For example, tossing a fair 6-sided die many times: the proportion of each outcome stabilizes as the number of tosses increases.

The Law of Large Numbers

• As the number of repetitions of a probability experiment increases, the observed proportion of a particular outcome approaches the theoretical probability of that outcome.

• This principle is known as the Law of Large Numbers.

Mathematical Statement: If $\hat{p}_n$ is the proportion of $n$ rolls that come up as a particular outcome, then as $n \to \infty$, $\hat{p}_n \to p$, where $p$ is the true probability.

What is Probability?

Definition

• Probability is a measure of the likelihood of a random phenomenon or chance behavior.

• It describes the long-term proportion with which a certain outcome will occur in situations with short-term uncertainty.

• The probability of an outcome is the long-term proportion in which that outcome is observed.

Key Definitions

• Simple Events (Sample Points): The individual outcomes of an experiment, denoted $e_1, e_2, ..., e_n$.

• Sample Space (S): The set of all possible outcomes of an experiment, $S = \{e_1, e_2, ..., e_n\}$.

• Event: Any collection (subset) of simple events from the sample space. Events are denoted by capital letters (e.g., $A$, $B$).

Examples of Sample Spaces and Events

Cat vs Dog Example

• Experiment: Having two pets, each can be a cat or a dog.

• Outcomes: $e_1 = (dog, dog)$, $e_2 = (dog, cat)$, $e_3 = (cat, dog)$, $e_4 = (cat, cat)$

• Sample Space: $S = \{(dog, dog), (dog, cat), (cat, dog), (cat, cat)\}$

• Event $E$ ("have one dog"): $E = \{(dog, cat), (cat, dog)\}$

Coin Toss Example

• Experiment: Toss a fair coin three times.

• Sample Space: $S = \{TTT, HTT, THT, TTH, HHT, THH, TTH, HTH\}$

• Event $E$ (exactly two heads): $E = \{HHT, HTH, THH\}$

Rules of Probability

• Rule 1: The probability of any event $E$, $P(E)$, must satisfy $0 \leq P(E) \leq 1$.

• Rule 2: The sum of the probabilities of all outcomes in the sample space must equal 1: $P(e_1) + P(e_2) + \cdots + P(e_n) = 1$

Probability Models

Definition

• A probability model lists all possible outcomes of a probability experiment and assigns a probability to each outcome.

• It must satisfy the two rules of probability above.

• Impossible event: Probability is 0.

• Certain event: Probability is 1.

• Unusual event: An event with a low probability of occurring.

Example: Probability Model for M&M Colors

Suppose a bag of peanut M&M candies contains candies of different colors. The probability of drawing each color is given below:

Color	Probability
Brown	0.12
Yellow	0.15
Red	0.12
Blue	0.23
Orange	0.23
Green	0.15

To check if this is a valid probability model, verify that all probabilities are between 0 and 1 and that their sum is 1.

Methods of Calculating Probability

Empirical (Experimental) Approach

• Probability is estimated by the relative frequency of an event in a large number of trials.

• Formula: $P(E) \approx \text{relative frequency of } E = \frac{\text{frequency of } E}{\text{number of trials of experiment}}$

Example: Pass the Pigs Game

In the game "Pass the Pigs," pigs are rolled like dice. The table below shows the frequency of each outcome in 3,939 rolls:

Outcome	Frequency
Side with no dot	1344
Side with dot	1294
Razorback	767
Trotter	365
Snouter	137
Leaning Jowler	32

Probabilities are calculated as frequency divided by total trials. For example, the probability of "side with dot" is $\frac{1294}{3939} \approx 0.329$.

Outcome	Probability
Side with no dot	0.341
Side with dot	0.329
Razorback	0.195
Trotter	0.093
Snouter	0.035
Leaning Jowler	0.008

Events with very low probability (e.g., "Leaning Jowler") are considered unusual.

Classical (Theoretical) Approach

• Assumes all outcomes are equally likely.

• Formula: $P(E) = \frac{\text{number of ways } E \text{ can occur}}{\text{number of possible outcomes}} = \frac{N(E)}{N(S)}$

Examples

• Tossing a fair coin: $S = \{H, T\}$, $P(H) = P(T) = \frac{1}{2}$

• Tossing a fair die: $S = \{1,2,3,4,5,6\}$, $P(e_i) = \frac{1}{6}$ for each $i$

• Tossing two coins: $S = \{HH, HT, TH, TT\}$, $P(HH) = P(HT) = P(TH) = P(TT) = \frac{1}{4}$

Classical Method Example: M&Ms

• Suppose a bag contains 9 brown, 6 yellow, 7 red, 4 orange, 2 blue, and 2 green candies (total 30).

• $P(\text{yellow}) = \frac{6}{30} = 0.2$

• $P(\text{blue}) = \frac{2}{30} = 0.067$

• Yellow is three times as likely as blue.

Applications and Additional Examples

Concert Ticket Example

• Sophia has three tickets and four friends (Juhl, Ashwin, Ramon, Destiny). She randomly selects two friends to join her.

• Sample space:

  Pair
  Juhl, Ashwin
  Juhl, Ramon
  Juhl, Destiny
  Ashwin, Ramon
  Ashwin, Destiny
  Ramon, Destiny

• Probability Ashwin and Ramon attend: $P = \frac{1}{6}$

• Probability Juhl attends: $P = \frac{3}{6} = 0.5$

• If repeated 1000 times, expect Juhl to attend about 500 times.

Family Gender Example

• Survey of 500 families with three children: 180 have two boys and one girl.

• Empirical probability: $P(E) = \frac{180}{500} = 0.36$

• Classical probability:

  • Sample space: $S = \{BBB, BBG, BGB, BGG, GBB, GBG, GGB, GGG\}$ ($N(S) = 8$)

  • Event $E$ (two boys and one girl): $\{BBG, BGB, GBB\}$ ($N(E) = 3$)

  • $P(E) = \frac{3}{8} = 0.375$

• If repeated 1000 times, expect about 375 families to have two boys and one girl (assuming equal likelihood).

Summary Table: Probability Calculation Methods

Method	Description	Formula	Example
Empirical	Based on observed data from experiments	$P(E) \approx \frac{\text{frequency of } E}{\text{number of trials}}$	Pass the Pigs game
Classical	Assumes equally likely outcomes	$P(E) = \frac{N(E)}{N(S)}$	Coin toss, dice roll

Key Takeaways

• Probability quantifies uncertainty and is foundational to statistical inference.

• Randomness leads to predictable patterns in the long run, as described by the Law of Large Numbers.

• Probability models must assign probabilities between 0 and 1, and the total probability across all outcomes must be 1.

• Empirical and classical methods are two primary approaches to calculating probabilities.