Probability: Fundamental Concepts

Definition and Interpretation of Probability

Probability is a measure of the likelihood that a particular event will occur. It quantifies uncertainty and describes the long-term proportion with which a certain outcome is observed in repeated trials of an experiment. The value of probability always lies between 0 (impossible event) and 1 (certain event).

• Impossible Event: Probability is 0.

• Certain Event: Probability is 1.

• Unusual Event: Probability is typically less than 5% (0.05), but this threshold can vary depending on context.

Law of Large Numbers: As the number of repetitions of a probability experiment increases, the observed proportion of an outcome approaches its theoretical probability.

Key Terms in Probability

• Experiment: A repeatable process with uncertain results (e.g., flipping a coin).

• Sample Space (S): The set of all possible outcomes of an experiment.

• Event (E): Any collection of outcomes from a probability experiment. A simple event consists of a single outcome.

Probability Models and Rules

Rules of Probability

• Probability Notation: denotes the probability that event E occurs.

• Range Rule:

• Sum Rule: The sum of probabilities for all outcomes in the sample space is 1:

Probability Model: A table or list that shows all possible outcomes and their probabilities. All probabilities must be between 0 and 1, and their sum must be 1.

Methods for Calculating Probability

• Empirical Probability: Based on observed data from experiments.

• Classical Probability: Used when all outcomes are equally likely.

• Subjective Probability: Based on personal judgment or experience (e.g., weather forecasts).

Sample Spaces and Events

Examples of Sample Spaces

• Coin Toss: S = {Head, Tail}

• Roll a Die: S = {1, 2, 3, 4, 5, 6}

• Two Dice Sum: S = {2, 3, ..., 12}

Addition Rule and Complements

The General Addition Rule

The probability that event E or event F occurs is:

This rule accounts for any overlap between E and F.

Mutually Exclusive (Disjoint) Events

• Events are mutually exclusive if they cannot occur together (no outcomes in common).

• For disjoint events:

Complements

• The complement of event E (denoted ) consists of all outcomes not in E.

• Complement Rule:

Multiplication Rule and Independence

Independent and Dependent Events

• Independent Events: The occurrence of one event does not affect the probability of the other.

• Dependent Events: The occurrence of one event affects the probability of the other.

For independent events E and F:

Multiplication Rule for Multiple Events

For n independent events :

"At Least" Probabilities

To find the probability that at least one event occurs, use the complement rule:

Conditional Probability and the General Multiplication Rule

Conditional Probability

• Conditional Probability: The probability of event F given that event E has occurred is denoted .

• Formula:

General Multiplication Rule

For any two events E and F:

Testing for Independence

• Events E and F are independent if or .

Applications and Examples

Probability with Cards and Dice

• Probability of drawing a queen from a standard deck:

• Probability of rolling a sum greater than 10 with two dice: List all outcomes and count favorable cases.

Benford's Law

Benford's Law describes the frequency distribution of leading digits in many real-life sets of numerical data. The probability that the first digit is d (d = 1, ..., 9) is given by a specific probability model.

Income Distribution Example

Summary Table: Key Probability Rules

Additional info: These notes cover all foundational probability rules and applications as outlined in an introductory statistics curriculum, including empirical, classical, and subjective probability, the addition and multiplication rules, complements, independence, and conditional probability, with relevant examples and tables for practical understanding.