Probability Rules and Random Processes: Study Notes for Statistics

5.1 Probability Rules

Objectives

• Understand random processes and the Law of Large Numbers

• Apply the rules of probability

• Compute and interpret probabilities using the empirical method

• Compute and interpret probabilities using the classical method

• Recognize and interpret subjective probabilities

Random Processes and the Law of Large Numbers

Understanding Randomness

A random process is a scenario where the outcome of any particular trial of an experiment is unknown, but the proportion or relative frequency of a particular outcome is observed to approach a specific value as the number of trials increases.

• Simulation is a technique used to recreate a random event. The goal in simulations is to measure how often a goal is achieved.

• Example: Rolling a die multiple times and recording the outcome to estimate the probability of rolling a specific number.

The Law of Large Numbers

The Law of Large Numbers states that as the number of repetitions of a probability experiment increases, the proportion with which a certain outcome is observed gets closer to the theoretical probability of the outcome.

• In the short run, outcomes may vary significantly, but in the long run, the relative frequency stabilizes near the expected probability.

• Example: If you roll a fair six-sided die many times, the proportion of times you get a '4' will approach $\frac{1}{6}$ as the number of rolls increases.

Law of Large Numbers vs. Law of Averages

The Law of Large Numbers is often misunderstood as the non-existent Law of Averages, which incorrectly suggests that outcomes will "even out" in the short term. In reality, each trial is independent and does not "remember" past outcomes.

• Example: The probability of having a boy after four girls is still 0.5, not higher because of previous outcomes.

Sample Space and Events

Definitions

• The sample space ($S$) of a probability experiment is the collection of all possible outcomes.

• An event is any collection of outcomes from the sample space.

• Example: If you have two children, the sample space is {boy-boy, boy-girl, girl-boy, girl-girl}. The event "have one boy" includes {boy-girl, girl-boy}.

Rules of Probability

Basic Rules

• Rule 1: The probability of any event $E$, $P(E)$, must be greater than or equal to 0 and less than or equal to 1. That is, $0 \leq P(E) \leq 1$.

• Rule 2: The sum of the probabilities of all outcomes must equal 1. If the sample space $S = \{e_1, e_2, ..., e_n\}$, then $P(e_1) + P(e_2) + ... + P(e_n) = 1$.

Probability Model Example

A probability model lists the possible outcomes of a probability experiment and each outcome’s probability. The model must satisfy the rules above.

Color	Probability
Brown	0.12
Yellow	0.15
Red	0.23
Blue	0.23
Green	0.15

Additional info: The sum of probabilities is 0.88, which suggests either missing categories or a typo. In a valid probability model, the sum should be 1.

Unusual Events

• An unusual event is one that has a low probability of occurring. Typically, a probability less than 0.05 is considered unusual.

• If an event is impossible, its probability is 0. If it is a certainty, its probability is 1.

• The closer a probability is to 1, the more likely the event will occur.

Computing Probabilities

Empirical Method

The empirical method uses observed data from experiments to estimate probabilities.

• The probability of an event $E$ is approximated by the number of times event $E$ is observed divided by the number of repetitions of the experiment.

$P(E) = \frac{\text{frequency of } E}{\text{number of trials of experiment}}$

• Example: In a pig-throwing game, if "side with no dot" occurs 1344 times out of 3999 throws, $P(\text{side with no dot}) = \frac{1344}{3999}$.

Outcome	Frequency
Side with no dot	1344
Side with dot	1249
Razorback	767
Trotter	366
Snouter	137
Leaning Jowler	132

Classical Method

The classical method applies when outcomes are equally likely.

• If an experiment has $n$ equally likely outcomes and the number of ways that an event $E$ can occur is $N(E)$, then the probability of $E$ is:

$P(E) = \frac{N(E)}{N(S)}$

• Where $N(S)$ is the number of outcomes in the sample space.

• Example: If a bag contains 9 brown, 6 yellow, 7 red, 4 orange, 2 blue, and 2 green candies, and one is selected at random:

  • Total candies: $9 + 6 + 7 + 4 + 2 + 2 = 30$

  • Probability of yellow: $P(\text{yellow}) = \frac{6}{30} = 0.2$

  • Probability of blue: $P(\text{blue}) = \frac{2}{30} \approx 0.067$

Additional Key Terms and Concepts

• Outcome: The result of a single trial of a probability experiment.

• Event: A set of one or more outcomes.

• Relative frequency: The proportion of times an event occurs in a series of trials.

• Equally likely outcomes: Outcomes that have the same probability of occurring.

Summary Table: Probability Methods

Method	Description	Formula
Empirical	Based on observed data	$P(E) = \frac{\text{frequency of } E}{\text{number of trials}}$
Classical	Based on equally likely outcomes	$P(E) = \frac{N(E)}{N(S)}$
Subjective	Based on personal judgment or experience	N/A

Applications and Examples

• Estimating the probability of rolling a four on a die using simulation and empirical data.

• Calculating the probability of being stopped at a red light using recorded commute data.

• Modeling probabilities for outcomes in games and random experiments.

Additional info: These notes expand on brief points and activities in the original material, providing definitions, formulas, and examples for clarity and completeness.