Randomness and Probability

Random Phenomena and Empirical Probability

Random phenomena are processes where individual outcomes cannot be predicted, but the long-run behavior can be understood. Each attempt (trial) produces an outcome, and outcomes can be grouped into events. The sample space (denoted S or Ω) is the set of all possible outcomes. Empirical probability is the long-run relative frequency of an event, estimated by repeated observation. The Law of Large Numbers (LLN) states that as the number of independent trials increases, the empirical probability approaches a single value.

• Trial: Each individual attempt of a random phenomenon.

• Outcome: The result of a single trial.

• Event: A set of outcomes or a combination of outcomes.

• Sample Space: The collection of all possible outcomes.

• Empirical Probability: Probability based on observed data.

• Independence: The outcome of one trial does not affect another.

Example: Tossing a coin repeatedly; each toss is a trial, the outcome is heads or tails, and the event could be getting heads.

The Nonexistent Law of Averages

Many confuse the Law of Large Numbers with the so-called "Law of Averages," which incorrectly suggests that outcomes must even out in the short run. The Law of Averages does not exist; only long-run behavior is predictable.

Types of Probability

Probability can be estimated in three ways:

• Empirical Probability: Based on observed frequencies.

• Theoretical (Model-Based) Probability: Calculated when outcomes are equally likely, using the formula:

$ P(A) = \frac{\text{Number of outcomes in } A}{\text{Number of possible outcomes}} $

• Subjective (Personal) Probability: Based on personal judgment or experience, not on frequencies or models. Subjective probability is prone to biases such as overconfidence, sunk cost, and recency bias.

Probability Rules

Several fundamental rules govern probability calculations:

• Rule 1: Probability is a number between 0 and 1. $ 0 \leq P(A) \leq 1 $

• Rule 2 (Assignment Rule): The probability of the sample space is 1. $ P(S) = 1 $

• Rule 3 (Complement Rule): $ P(A^C) = 1 - P(A) $ where $ A^C $ is the complement of event A.

• Rule 4 (Multiplication Rule for Independent Events): $ P(A \text{ and } B) = P(A) \times P(B) $ if A and B are independent.

• Rule 5 (Addition Rule for Disjoint Events): $ P(A \text{ or } B) = P(A) + P(B) $ if A and B are disjoint.

• Rule 6 (General Addition Rule): $ P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B) $

• Rule 7 (General Multiplication Rule): $ P(A \text{ and } B) = P(A) \times P(B|A) $

Joint Probability and Contingency Tables

Contingency tables organize data to show the frequency of combinations of events. They help calculate:

• Marginal Probability: Probability based on totals in the margins (e.g., probability of being a woman).

• Joint Probability: Probability of two events occurring together (e.g., being a woman and preferring a camera).

• Conditional Probability: Probability of one event given another (e.g., preferring a bike given the respondent is a woman).

Example Table: Prize preference for 478 customers.

Example Calculations:

• Marginal: $ P(\text{woman}) = \frac{251}{478} = 0.525 $

• Joint: $ P(\text{woman and camera}) = \frac{91}{478} = 0.190 $

• Conditional: $ P(\text{bike}|\text{woman}) = \frac{30}{251} = 0.120 $

Conditional Probability and Independence

Conditional probability is the probability of event B given event A, denoted $ P(B|A) $. Events A and B are independent if $ P(B|A) = P(B) $. Disjoint events cannot be independent.

$ P(B|A) = \frac{P(A \text{ and } B)}{P(A)} $

Constructing Contingency Tables

Given probabilities, you can construct contingency tables to find other probabilities. Marginal probabilities go in the margins, joint probabilities in the interior. The sum of disjoint probabilities equals the marginal totals.

Probability Trees

Probability trees visually represent the probabilities of multiple events, including conditional probabilities. Each branch shows the probability of an event, and subsequent branches show conditional probabilities.

Example: A company must fix a defect by adjusting the motherboard (10%), memory (30%), or case (60%). The probability tree shows these probabilities.

Conditional probabilities can be added to the tree, such as the probability that an adjustment fixes the problem.

Example: The probability that a case adjustment fixes the problem is 0.80, memory adjustment is 0.50, motherboard adjustment is 0.10. The tree shows all possible outcomes and their probabilities.

Joint probabilities are calculated by multiplying along the branches. For example:

$ P(\text{Case and Fixed}) = P(\text{Case}) \times P(\text{Fixed}|\text{Case}) = 0.60 \times 0.80 = 0.48 $

To find the probability that a minor adjustment will fix the problem, add the relevant joint probabilities:

$ P(\text{Fixed}) = P(\text{Case and Fixed}) + P(\text{Memory and Fixed}) + P(\text{Motherboard and Fixed}) $

$ P(\text{Fixed}) = 0.48 + 0.15 + 0.01 = 0.64 $

Bayes’s Rule

Bayes’s Rule allows us to reverse conditioning and update probability estimates given new information. For mutually exclusive and exhaustive events $A_i$:

$ P(A_i|B) = \frac{P(B|A_i) \times P(A_i)}{\sum_{j=1}^n P(B|A_j) \times P(A_j)} $

Example: If a problem is fixed, Bayes’s Rule can be used to find the probability that a specific adjustment (e.g., case) was responsible.

Common Pitfalls in Probability

• Probabilities must add up to 1.

• Only add probabilities for disjoint events.

• Only multiply probabilities for independent events.

• Do not confuse disjoint and independent events.

Summary of Key Concepts

• Probability is based on long-run frequencies.

• The Law of Large Numbers describes long-run behavior, not short-run balancing.

• Probability can be estimated empirically, theoretically, or subjectively.

• Basic rules help combine probabilities for complex events.

• Independence and disjointness are distinct concepts.

• Probability trees and Bayes’s Rule help update and reverse probability estimates.