Chapter 4: Probability – Foundations and Rules in Statistics

Probability: Introduction and Motivation

Why Probability?

Probability is a fundamental concept in statistics, providing a framework for quantifying uncertainty and making informed predictions about future events. It is essential for inferential statistics, where we use sample data to make generalizations about a population.

• Rare Event Rule: If an observed event is highly unlikely under a given assumption, the assumption may be incorrect.

• Probability helps us assess the likelihood of events and make decisions under uncertainty.

Basic Probability Terminology

Key Terms

• Experiment: A process that leads to well-defined outcomes (e.g., rolling a die).

• Outcome: A possible result of an experiment (e.g., rolling a 4).

• Event: Any collection of outcomes (e.g., rolling an even number).

• Simple Event: An event that cannot be broken down further (e.g., rolling a 2).

• Sample Space (S): The set of all possible outcomes of an experiment.

Probability

• Probability (P): The likelihood that an event occurs, denoted as $P(E)$ for event E.

• Probabilities are expressed as fractions, decimals, or percentages between 0 and 1.

Key Probability Facts

• The probability of an impossible event is 0.

• The probability of a certain event is 1.

• For any event E: $0 \leq P(E) \leq 1$

Approaches to Probability

1. Relative Frequency Approach

Probability is estimated by the proportion of times an event occurs in a large number of trials.

• Formula: $P(E) = \frac{\text{Number of times E occurred}}{\text{Total number of trials}}$

• Assumes that as the number of trials increases, the estimate becomes more accurate.

2. Classical Approach

Probability is determined by logically analyzing all possible outcomes, assuming each is equally likely.

• Formula: $P(E) = \frac{\text{Number of ways E could occur}}{\text{Number of events in the sample space}}$

• Requires equally likely outcomes.

3. Subjective Approach

Probability is estimated based on personal judgment, experience, or intuition.

• Used when empirical or classical methods are not feasible.

Choosing an Approach

• Relative Frequency: Use when data from repeated trials is available.

• Classical: Use when all outcomes are equally likely and known.

• Subjective: Use when neither data nor equally likely outcomes are available.

Complements, Odds, and Probability Rules

Complements

• The complement of event E, denoted $E'$, consists of all outcomes in the sample space that are not in E.

• Rule of Complements: $P(E') = 1 - P(E)$

Odds

• Odds in favor of event A: $\frac{P(A)}{1 - P(A)}$

• Odds against event A: $\frac{1 - P(A)}{P(A)}$

• Odds are often expressed as ratios (e.g., 3:2).

Compound Events and the Addition Rule

Compound Events

• A compound event involves two or more simple events.

The Meaning of "OR"

• "A or B" means that at least one of the events A or B occurs (inclusive OR).

• Notation: $P(A \text{ or } B)$

The Meaning of "AND"

• "A and B" means both events A and B occur.

• Notation: $P(A \text{ and } B)$

Addition Rule ("OR" Formula)

• For any two events A and B:

$P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$

• If A and B are disjoint (mutually exclusive): $P(A \text{ and } B) = 0$, so $P(A \text{ or } B) = P(A) + P(B)$

Visualizing with Venn Diagrams and Tables

• Venn diagrams and tables can help illustrate the relationships between events and their probabilities.

Multiplication Rule ("AND" Formula)

Independent and Dependent Events

• Events A and B are independent if the occurrence of one does not affect the probability of the other.

• If not, they are dependent.

Multiplication Rule

• For independent events: $P(A \text{ and } B) = P(A) \times P(B)$

• For dependent events: $P(A \text{ and } B) = P(A) \times P(B|A)$, where $P(B|A)$ is the probability of B given A has occurred.

Tree Diagrams and Sample Spaces

• Tree diagrams can be used to visualize sequences of events and calculate probabilities for compound events.

Probability of "At Least One"

• The probability that at least one event occurs is often easier to find by calculating the complement (none occur) and subtracting from 1.

$P(\text{at least one}) = 1 - P(\text{none})$

Conditional Probability

Definition and Formula

• Conditional probability is the probability that event B occurs given that event A has already occurred.

$P(B|A) = \frac{P(A \text{ and } B)}{P(A)}$

Example Table: Conditional Probability

The following table (as referenced in the slides) is used to illustrate conditional probability calculations, such as in medical testing or polygraph results.

	Polygraph test: Lie	Polygraph test: Not Lie	Total
Subject lied	a	b	a + b
Subject did not lie	c	d	c + d
Total	a + c	b + d	a + b + c + d

Additional info: In practice, the values a, b, c, d would be filled in with actual data to compute probabilities such as false positives, false negatives, sensitivity, and specificity.

Summary Table: Probability Rules and Concepts

Concept	Formula	When to Use
Relative Frequency	$P(E) = \frac{\text{Number of times E occurred}}{\text{Total number of trials}}$	Empirical data available
Classical	$P(E) = \frac{\text{Number of ways E could occur}}{\text{Number of events in sample space}}$	Equally likely outcomes
Complement	$P(E') = 1 - P(E)$	Finding probability of 'not E'
Addition Rule	$P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$	Probability of A or B
Multiplication Rule (Independent)	$P(A \text{ and } B) = P(A) \times P(B)$	Independent events
Multiplication Rule (Dependent)	$P(A \text{ and } B) = P(A) \times P(B|A)$	Dependent events
Conditional Probability	$P(B|A) = \frac{P(A \text{ and } B)}{P(A)}$	Probability of B given A
At Least One	$P(\text{at least one}) = 1 - P(\text{none})$	Probability at least one event occurs

Applications and Examples

• Calculating the probability of drawing certain cards from a deck.

• Finding the probability that at least one event occurs in repeated trials.

• Using conditional probability to interpret test results (e.g., medical tests, polygraph tests).

• Applying the addition and multiplication rules to real-world problems.

Additional info: These notes provide a comprehensive overview of foundational probability concepts, rules, and applications, suitable for college-level statistics students preparing for exams or assignments.