Chapter 5: Probability – Rules, Methods, and Applications

Probability: Rules, Methods, and Applications

5.1 Probability Rules

Probability is a measure of the likelihood that a random phenomenon or chance behavior will occur. This section introduces the foundational rules and concepts of probability, including random processes, the Law of Large Numbers, and different methods for computing probabilities.

Random Processes and the Law of Large Numbers

• Random Process: A scenario where the outcome of any particular trial is unknown, but the proportion of a particular outcome approaches a specific value as the number of trials increases.

• Simulation: A technique used to recreate a random event to measure how often a goal is observed.

• Law of Large Numbers: As the number of repetitions of a probability experiment increases, the proportion with which a certain outcome is observed gets closer to the probability of that outcome.

Example: Rolling a 10-sided die many times and recording the proportion of times a "4" is rolled. As the number of rolls increases, the observed proportion converges to the theoretical probability.

Probability Concepts and Definitions

• Probability (P): The long-term proportion of times an outcome is observed in repeated trials.

• Experiment: Any process that can be repeated with uncertain results.

• Sample Space (S): The set of all possible outcomes of an experiment.

• Event (E): Any collection of outcomes from a probability experiment.

Empirical, Classical, and Subjective Probability

• Empirical Probability: Based on observations from experiments or simulations. $P(E) \approx \frac{\text{frequency of } E}{\text{number of trials}}$

• Classical Probability: Used when all outcomes are equally likely. $P(E) = \frac{\text{number of ways } E \text{ can occur}}{\text{number of possible outcomes}}$

• Subjective Probability: Based on personal judgment or experience, not on formal calculations.

Key Probability Rules

• $0 \leq P(E) \leq 1$ for any event E.

• The sum of probabilities for all outcomes in the sample space is 1.

• An event with probability close to 1 is very likely; close to 0 is unlikely.

• An unusual event is typically defined as one with probability less than 0.05.

Illustrating the Law of Large Numbers

Tracking the proportion of days a traffic light is red over many days demonstrates how the observed proportion stabilizes as more data is collected.

The Nonexistent Law of Averages

• The Law of Large Numbers does not imply that outcomes will "even out" in the short run. Each trial is independent and does not "remember" past outcomes.

• For example, the probability of having a boy or girl remains 0.5 for each child, regardless of previous outcomes.

5.2 The Addition Rule and Complements

This section covers how to calculate the probability of unions and complements of events, including the use of Venn diagrams and contingency tables.

Addition Rule for Disjoint (Mutually Exclusive) Events

• Events are disjoint if they have no outcomes in common.

• Addition Rule: If E and F are disjoint, $P(E \text{ or } F) = P(E) + P(F)$

General Addition Rule

• For any two events E and F (not necessarily disjoint): $P(E \text{ or } F) = P(E) + P(F) - P(E \text{ and } F)$

• This rule avoids double-counting outcomes that are in both E and F.

Complement Rule

• The complement of event E, denoted $E^c$, is the set of all outcomes not in E.

• Complement Rule: $P(E^c) = 1 - P(E)$

Using Frequency Tables in Probability

Frequency tables can be used to empirically estimate probabilities for categorical data, such as travel times.

5.3 Independence and the Multiplication Rule

This section explains how to identify independent events and use the multiplication rule to compute probabilities involving multiple events.

Independent vs. Dependent Events

• Independent Events: The occurrence of one event does not affect the probability of the other.

• Dependent Events: The occurrence of one event affects the probability of the other.

• Disjoint events are not independent because knowing one occurs means the other cannot.

Multiplication Rule for Independent Events

• If E and F are independent, $P(E \text{ and } F) = P(E) \times P(F)$

• For n independent events: $P(E_1 \text{ and } E_2 \text{ and } \ldots \text{ and } E_n) = P(E_1) \times P(E_2) \times \ldots \times P(E_n)$

At-Least Probabilities

• To find the probability that at least one event occurs, use the complement: $P(\text{at least one}) = 1 - P(\text{none})$

5.4 Conditional Probability and the General Multiplication Rule

Conditional probability measures the likelihood of an event given that another event has occurred. The general multiplication rule extends the multiplication rule to dependent events.

Conditional Probability

• Notation: $P(E|F)$ is the probability of E given F.

• Formula: $P(E|F) = \frac{P(E \text{ and } F)}{P(F)}$

General Multiplication Rule

• For any two events E and F: $P(E \text{ and } F) = P(F) \times P(E|F)$

5.5 Counting Techniques

Counting techniques are essential for determining the number of possible outcomes in probability problems, especially when dealing with arrangements and selections.

Multiplication Rule of Counting

• If a task consists of a sequence of choices, the total number of ways to complete the task is the product of the number of choices at each stage.

• For example, if there are p choices for the first step, q for the second, and r for the third, the total is $p \times q \times r$.

Permutations

• Permutation: An ordered arrangement of objects.

• Number of permutations of n objects taken r at a time: $P(n, r) = \frac{n!}{(n-r)!}$

Combinations

• Combination: A selection of objects where order does not matter.

• Number of combinations of n objects taken r at a time: $C(n, r) = \frac{n!}{r!(n-r)!}$

Permutations with Non-distinct Items

• Number of permutations of n objects, where there are groups of indistinguishable objects: $\frac{n!}{n_1! n_2! \ldots n_k!}$

5.6 Simulation

Simulation is used to model random processes and estimate probabilities when analytical solutions are difficult or impossible. It is especially useful in data collection and experimental design.

• Random selection and assignment are crucial for valid statistical inference.

• Simulations can estimate the probability of complex events by repeating random experiments many times.

5.7 Putting It Together: Choosing Methods

This section provides guidance on selecting the appropriate probability rule or counting technique based on the structure of the problem.

Summary Table: Probability and Counting Rules

Situation	Rule/Formula
Disjoint events (E and F)	$P(E \text{ or } F) = P(E) + P(F)$
Any two events (E and F)	$P(E \text{ or } F) = P(E) + P(F) - P(E \text{ and } F)$
Complement	$P(E^c) = 1 - P(E)$
Independent events	$P(E \text{ and } F) = P(E) \times P(F)$
Conditional probability	$P(E|F) = \frac{P(E \text{ and } F)}{P(F)}$
General multiplication rule	$P(E \text{ and } F) = P(F) \times P(E|F)$
Permutations	$P(n, r) = \frac{n!}{(n-r)!}$
Combinations	$C(n, r) = \frac{n!}{r!(n-r)!}$
Permutations with nondistinct items	$\frac{n!}{n_1! n_2! \ldots n_k!}$

Additional info: These notes cover the core probability concepts, rules, and methods essential for college-level statistics, including empirical, classical, and subjective probability, the Law of Large Numbers, addition and multiplication rules, conditional probability, and counting techniques. Flowcharts are provided to help students select the appropriate method for a given problem.