Probability and Counting Techniques in Statistics

Probability

Random Processes and the Law of Large Numbers

Probability is the study of random phenomena and quantifies the likelihood of various outcomes. A random process is one where the outcome of any particular trial is unpredictable, but the relative frequency of outcomes stabilizes as the number of trials increases. This is formalized by the Law of Large Numbers, which states that as the number of repetitions of a probability experiment increases, the observed proportion of a particular outcome approaches its theoretical probability.

• Experiment: Any process with uncertain results that can be repeated.

• Sample Space (S): The set of all possible outcomes.

• Event: Any collection of outcomes from a probability experiment.

Example: Rolling a fair die has a sample space S = {1, 2, 3, 4, 5, 6}. The event E = "roll an even number" = {2, 4, 6}.

Probability Models and Rules

A probability model lists all possible outcomes and their probabilities. The following rules must be satisfied:

• For any event E, $0 \leq P(E) \leq 1$

• The sum of the probabilities of all outcomes in the sample space is 1: $\sum P(E_i) = 1$

If an event is impossible, its probability is 0; if certain, its probability is 1. An unusual event has a low probability of occurring.

Empirical and Classical Probability

• Empirical Probability: Based on observed data. $P(E) \approx \frac{\text{number of times E occurs}}{\text{number of trials}}$

• Classical Probability: Used when outcomes are equally likely. $P(E) = \frac{\text{number of outcomes in E}}{\text{number of outcomes in S}}$

Example: Rolling two dice, the probability of rolling a seven is $P(7) = \frac{6}{36} = 0.167$.

Subjective Probability

Subjective probability is based on personal judgment rather than formal calculations, such as predicting the outcome of a sports event.

Addition Rule and Complements

Addition Rule for Disjoint Events

Two events are disjoint (mutually exclusive) if they have no outcomes in common. For disjoint events E and F:

• $P(E \text{ or } F) = P(E) + P(F)$

This extends to more than two disjoint events.

General Addition Rule

For any two events E and F (not necessarily disjoint):

• $P(E \text{ or } F) = P(E) + P(F) - P(E \text{ and } F)$

Example: Drawing a king or a diamond from a deck of cards requires the general addition rule because the king of diamonds is in both events.

Complement Rule

The complement of event E, denoted $E^c$, consists of all outcomes not in E. The probability of the complement is:

• $P(E^c) = 1 - P(E)$

Example: If 52% of Americans have played the lottery, the probability that a randomly selected American has not played is $1 - 0.52 = 0.48$.

Independence and the Multiplication Rule

Independent Events

Two events E and F are independent if the occurrence of one does not affect the probability of the other. If not, they are dependent.

• Disjoint events are not independent (if one occurs, the other cannot).

Multiplication Rule for Independent Events

• If E and F are independent, $P(E \text{ and } F) = P(E) \times P(F)$

This extends to n independent events: $P(E_1 \text{ and } E_2 \text{ and } \ldots \text{ and } E_n) = P(E_1) \times P(E_2) \times \ldots \times P(E_n)$

At-Least Probabilities

To find the probability that at least one event occurs, use the complement rule:

• $P(\text{at least one}) = 1 - P(\text{none})$

Conditional Probability and the General Multiplication Rule

Conditional Probability

The conditional probability of F given E is:

• $P(F|E) = \frac{P(E \text{ and } F)}{P(E)}$

Conditional probability reduces the sample space to only those outcomes where E has occurred.

General Multiplication Rule

• $P(E \text{ and } F) = P(E) \times P(F|E)$

This rule applies whether or not the events are independent.

Counting Techniques

Multiplication Rule of Counting

If a task consists of a sequence of choices, the total number of ways to complete the task is the product of the number of choices at each stage.

• For choices p, q, r: Total ways = $p \times q \times r$

Permutations

A permutation is an ordered arrangement of objects. The number of permutations of n objects taken r at a time is:

• $P(n, r) = \frac{n!}{(n - r)!}$

Where $n!$ (n factorial) is the product of all positive integers up to n.

Combinations

A combination is a selection of objects where order does not matter. The number of combinations of n objects taken r at a time is:

• $C(n, r) = \frac{n!}{r!(n - r)!}$

Permutations with Nondistinct Items

If there are n objects, with $n_1$ of one kind, $n_2$ of another, ..., $n_k$ of a kth kind, the number of distinguishable permutations is:

• $\frac{n!}{n_1! n_2! \ldots n_k!}$

Applications to Probability

Permutations and combinations are used to compute probabilities in experiments where order matters (permutations) or does not matter (combinations).

Example: The probability of winning a lottery where 6 numbers are chosen from 44 (order does not matter) is $P = \frac{2}{C(44, 6)}$ (since each ticket has two chances).

Simulation in Probability

Using Simulation to Approximate Probabilities

Simulations use random processes (physical or computer-generated) to approximate probabilities when theoretical calculation is complex or infeasible. By repeating the simulated experiment many times, the relative frequency of an event approximates its probability.

Choosing Probability and Counting Methods

Which Probability Rule to Use?

• Use the empirical method when you have observed data.

• Use the classical method when outcomes are equally likely.

• Use subjective probability for personal estimates.

• Use the addition rule for 'or' events, and the multiplication rule for 'and' events.

• Use the complement rule for 'at least' or 'at most' probabilities.

Which Counting Technique to Use?

• Use the multiplication rule for sequences of choices.

• Use permutations when order matters.

• Use combinations when order does not matter.

• Use permutations with nondistinct items when some objects are identical.