Probability Rules and Counting Techniques – Study Notes for Statistics

Probability Rules and Counting Techniques

Introduction to Probability

Probability is a fundamental concept in statistics that quantifies the likelihood of random events. It is essential for understanding uncertainty and making informed decisions based on data. This chapter covers the basic rules of probability, methods for determining probabilities, and essential counting techniques used in probability calculations.

Probability Rules

Random Processes and Probability

A random process is a situation where the outcome cannot be predicted with certainty for a single trial, but the long-term proportion of outcomes can be determined through repeated trials. For example, flipping a coin multiple times allows us to estimate the probability of getting heads.

• Probability: A measure of the likelihood of a random event, representing the long-term proportion of times a particular outcome is observed.

• Experiment: Any process with uncertain results that can be repeated (e.g., rolling a die).

• Sample Space (S): The set of all possible outcomes of an experiment.

• Event: Any subset of outcomes from the sample space that we are interested in. A simple event contains only one outcome.

Example: The graph above shows how the proportion of heads approaches 0.5 as the number of coin flips increases, illustrating the Law of Large Numbers.

Basic Rules of Probability

• The probability of any event is between 0 and 1: $0 \leq P(E) \leq 1$

• The sum of probabilities of all possible outcomes in the sample space is 1: $\sum P(E_i) = 1$

• An unusual event is one with a low probability, typically less than 0.05.

Example: Rolling a fair six-sided die, each outcome has probability $1/6$.

Probability Models

A probability model lists all possible outcomes and the probability assigned to each outcome. Valid models must satisfy the basic rules above.

Methods to Determine Probabilities

Empirical Method

The empirical method estimates probability based on observed data (relative frequency):

• $P(E) \approx \frac{\text{frequency of } E}{\text{number of trials}}$

Example: The table above shows survey data on means of travel to work. Probabilities can be estimated by dividing each frequency by the total number of responses (200).

Means of Travel	Frequency	Estimated Probability
Drive alone	83	0.415
Carpool	22	0.110
Public transportation	10	0.050
Walk	5	0.025
Other means	3	0.015
Work at home	77	0.385

Interpretation: The probability that a randomly selected individual carpools to work is 0.11.

Classical Method

The classical method is used when all outcomes are equally likely. Probability is calculated as:

• $P(E) = \frac{\text{number of ways } E \text{ can occur}}{\text{number of possible outcomes}} = \frac{N(E)}{N(S)}$

Example: Flipping a fair coin twice, the sample space is {HH, HT, TH, TT}. The probability of two heads is $1/4$.

Subjective Method

Subjective probability is based on personal judgment or experience when empirical or classical methods are not applicable. For example, estimating the probability of a recession next year.

Addition Rule and Complements

Addition Rule for Disjoint Events

Two events are disjoint (mutually exclusive) if they cannot occur at the same time. For disjoint events E and F:

• $P(E \text{ or } F) = P(E) + P(F)$

Example: Drawing a king or a queen from a standard deck of cards. Since these events are disjoint, add their probabilities.

General Addition Rule

For any two events E and F (not necessarily disjoint):

• $P(E \text{ or } F) = P(E) + P(F) - P(E \text{ and } F)$

Using Contingency Tables

Contingency tables display the frequency distribution of variables and are useful for calculating probabilities involving two categories.

	Males (millions)	Females (millions)
Never married	49.5	43.2
Married	66.3	64.2
Widowed	3.5	11.5
Divorced	12.3	16.6
Separated	2.0	2.6

Example: The probability that a randomly selected U.S. resident is widowed is $\frac{3.5 + 11.5}{133.6 + 138.1} \approx 0.055$.

Complement of an Event

The complement of an event E, denoted $E'$, is the event that E does not occur. The probability of the complement is:

• $P(E') = 1 - P(E)$

Example: If 52% of Americans have played state lotteries, the probability that a randomly selected American has not played is $1 - 0.52 = 0.48$.

Independence and the Multiplication Rule

Independent Events

Two events E and F are independent if the occurrence of one does not affect the probability of the other. For independent events:

• $P(E \text{ and } F) = P(E) \times P(F)$

Example: Flipping a coin and rolling a die are independent events.

Multiplication Rule for Independent Events

• For n independent events $E_1, E_2, ..., E_n$:

$P(E_1 \text{ and } E_2 \text{ and } ... \text{ and } E_n) = P(E_1) \times P(E_2) \times ... \times P(E_n)$

Computing At-Least Probabilities

To find the probability of "at least one" event occurring, use the complement rule:

• $P(\text{at least one}) = 1 - P(\text{none})$

Conditional Probability and the General Multiplication Rule

Conditional Probability

The probability of event F given that event E has occurred is called conditional probability:

• $P(F|E) = \frac{P(E \text{ and } F)}{P(E)}$

General Multiplication Rule

For any two events E and F:

• $P(E \text{ and } F) = P(E) \times P(F|E)$

Determining Independence

Events E and F are independent if $P(F|E) = P(F)$. For small samples from large populations (less than 5%), independence can be assumed.

Counting Techniques

General Counting Principle (Multiplication Rule of Counting)

If a task consists of a sequence of choices, the total number of outcomes is the product of the number of choices at each stage:

• Total outcomes = $p \times q \times r \times ...$

Factorial Notation

The factorial of n, denoted $n!$, is the product of all positive integers up to n:

• $n! = n \times (n-1) \times (n-2) \times ... \times 1$

Permutations

A permutation is an ordered arrangement of r objects chosen from n distinct objects (no repetition):

• $nP_r = \frac{n!}{(n-r)!}$

Combinations

A combination is a selection of r objects from n distinct objects where order does not matter:

• $nC_r = \frac{n!}{r!(n-r)!}$

Using Counting Methods in Probability Calculations

Counting techniques are used to determine the number of possible outcomes for probability calculations. For example, the probability of winning a lottery or drawing certain cards from a deck can be found using combinations or permutations.

Determining Which Method to Use

Choosing the correct probability or counting technique depends on the structure of the problem. Flowcharts can help guide the selection of the appropriate rule or formula.

Summary of Key Formulas

Additional info: These notes provide a comprehensive overview of probability rules and counting techniques, including definitions, examples, and key formulas. The included tables and flowcharts help clarify the application of these concepts in practical problems.