From Randomness to Probability: Foundations and Rules

From Randomness to Probability

Dealing with Random Phenomena

Probability theory provides a framework for understanding and quantifying uncertainty in random phenomena. In statistics, a random phenomenon is a situation where the possible outcomes are known, but the specific outcome of any individual trial is unpredictable.

• Trial: Each observation or repetition of a random phenomenon.

• Outcome: The result of a single trial.

• Event: A combination of outcomes.

• Sample Space (S): The set of all possible outcomes.

Independence is a key concept: trials are independent if the outcome of one does not affect the outcome of another (e.g., coin flips).

The Law of Large Numbers (LLN)

The Law of Large Numbers states that as the number of independent repetitions of a random experiment increases, the long-run relative frequency of an event approaches a single value, called the probability of the event.

• This probability is often called empirical probability because it is based on observed frequencies.

• Example: Tossing a fair coin many times will result in heads about 50% of the time in the long run.

Important: The LLN does not guarantee short-run regularity. The so-called "Law of Averages" (that an outcome is "due" after not occurring for a while) is a misconception.

Modeling Probability

Probability models often start with situations where all outcomes are equally likely (e.g., rolling a fair die, tossing a fair coin). However, not all real-world events are equally likely (e.g., a skilled basketball player making a free throw).

• The probability of an event A is given by:

$P(A) = \frac{\text{Number of outcomes in } A}{\text{Total number of possible outcomes}}$

• Probabilities can also be subjective or personal, but formal probability uses consistent, long-run definitions.

Rules and Notation in Probability

Visualizing Probability: Venn Diagrams

Venn diagrams are commonly used to visualize relationships between events, such as overlap (intersection) and separation (disjointness).

Formal Probability Rules

• Probability Assignment Rule: For any event A, $0 \leq P(A) \leq 1$.

• Total Probability Rule: The probability of the entire sample space is 1: $P(S) = 1$.

• Addition Rule for Disjoint Events: If events A and B are disjoint (mutually exclusive), then $P(A \text{ or } B) = P(A) + P(B)$.

• Complement Rule: The probability that event A does not occur is $P(A^C) = 1 - P(A)$, where $A^C$ is the complement of A.

• Notation: $P(A \cup B)$ means "A or B"; $P(A \cap B)$ means "A and B".

General Addition Rule

When events are not disjoint, the addition rule must be adjusted to avoid double-counting the intersection:

$P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$

Applying Probability Rules: Example

Consider a sample of book pages where:

• 48% had a data display

• 27% had an equation

• 7% had both a data display and an equation

Let D = "data display", E = "equation".

• a) Venn Diagram: Visualizes the overlap and separation of D and E.

• b) Probability of neither event:

$P(D \text{ or } E) = P(D) + P(E) - P(D \text{ and } E) = 0.48 + 0.27 - 0.07 = 0.68$ $P((D \text{ or } E)^C) = 1 - 0.68 = 0.32$

• c) Probability of data display but no equation:

$P(D \text{ and not } E) = P(D) - P(D \text{ and } E) = 0.48 - 0.07 = 0.41$

Common Pitfalls in Probability

• Probabilities for all possible outcomes must sum to 1.

• Use the addition rule for disjoint events only when events are truly disjoint.

• For overlapping events, use the general addition rule to avoid double-counting.

Summary of Key Probability Rules

• Probability Assignment Rule

• Total Probability Rule

• Complement Rule

• Addition Rule for Disjoint Events

• General Addition Rule