Probability Rules and Counting Techniques: Comprehensive Study Notes

Probability: Fundamental Concepts and Rules

Introduction to Probability

Probability is a mathematical measure of the likelihood that a random phenomenon or chance behavior will occur. It quantifies uncertainty in the short term but reveals predictable patterns in the long term. The Law of Large Numbers states that as the number of repetitions of a probability experiment increases, the observed proportion of a particular outcome approaches its theoretical probability.

• Experiment: Any process that can be repeated and has uncertain results.

• Sample Space (S): The set of all possible outcomes of an experiment.

• Event: Any collection of outcomes from a probability experiment. A simple event contains only one outcome.

Example: For the experiment of having two children, the sample space is {(boy, boy), (boy, girl), (girl, boy), (girl, girl)}. The event E = "have one boy" is {(boy, girl), (girl, boy)}.

Rules of Probability

• Rule 1: For any event E, $0 \leq P(E) \leq 1$.

• Rule 2: The sum of the probabilities of all outcomes in the sample space is 1: $P(e_1) + P(e_2) + \ldots + P(e_n) = 1$.

A probability model lists all possible outcomes and their probabilities, satisfying the above rules.

Empirical and Classical Probability

• Empirical Method: Probability is approximated by the relative frequency of an event in a large number of trials: $P(E) \approx \frac{\text{frequency of E}}{\text{number of trials}}$.

• Classical Method: If all outcomes are equally likely, $P(E) = \frac{\text{number of ways E can occur}}{\text{number of possible outcomes}}$.

Example: If a bag contains 6 yellow and 2 blue candies out of 30, $P(\text{yellow}) = \frac{6}{30} = 0.2$, $P(\text{blue}) = \frac{2}{30} = 0.067$.

Subjective Probability

Subjective probability is based on personal judgment or experience rather than precise calculation. For example, an economist estimating a 20% chance of recession is using subjective probability.

Addition Rule and Complements

Addition Rule for Disjoint (Mutually Exclusive) Events

Two events are disjoint if they have no outcomes in common. For disjoint events E and F:

• $P(E \text{ or } F) = P(E) + P(F)$

This extends to more than two disjoint events.

Example: If the probability a housing unit has two rooms is 0.032 and three rooms is 0.093, then $P(\text{two or three}) = 0.032 + 0.093 = 0.125$.

General Addition Rule

For any two events E and F (not necessarily disjoint):

• $P(E \text{ or } F) = P(E) + P(F) - P(E \text{ and } F)$

Example: For dice, let E = "first die is a two" and F = "sum ≤ 5". The probabilities are calculated using the sample space of 36 outcomes.

Complement Rule

The complement of event E, denoted $E^C$, consists of all outcomes not in E. The probability of the complement is:

• $P(E^C) = 1 - P(E)$

Example: If 31.6% of households own a dog, the probability a household does not own a dog is $1 - 0.316 = 0.684$.

Independence and the Multiplication Rule

Independent and Dependent Events

Two events E and F are independent if the occurrence of one does not affect the probability of the other. Otherwise, they are dependent.

• Example: Drawing a card and rolling a die are independent events.

Multiplication Rule for Independent Events

If E and F are independent:

• $P(E \text{ and } F) = P(E) \cdot P(F)$

This extends to n independent events:

• $P(E_1 \text{ and } E_2 \text{ and } \ldots \text{ and } E_n) = P(E_1) \cdot P(E_2) \cdot \ldots \cdot P(E_n)$

Example: If the probability a 60-year-old female survives the year is 0.99186, the probability that four such females all survive is $(0.99186)^4 = 0.9678$.

At-Least Probabilities

The probability that at least one event occurs is:

• $P(\text{at least one}) = 1 - P(\text{none})$

Example: The probability that at least one of 500 randomly selected 60-year-old females dies in a year is $1 - (0.99186)^{500} = 0.9832$.

Conditional Probability and the General Multiplication Rule

Conditional Probability

The probability of event F given that event E has occurred is denoted $P(F|E)$ and is calculated as:

• $P(F|E) = \frac{P(E \text{ and } F)}{P(E)}$

Example: If 19.1% of murder victims are aged 20–24 and 16.6% are 20–24-year-old males, then the probability a murder victim is male given age 20–24 is $0.166 / 0.191 = 0.869$.

General Multiplication Rule

The probability that both E and F occur is:

• $P(E \text{ and } F) = P(E) \cdot P(F|E)$

Example: If $P(20\text{–}24) = 0.191$ and $P(\text{male}|20\text{–}24) = 0.869$, then $P(\text{male and } 20\text{–}24) = 0.191 \times 0.869 = 0.166$.

Counting Techniques

Multiplication Rule of Counting

If a task consists of a sequence of choices, the total number of ways to complete the task is the product of the number of choices at each stage:

• Total ways = $p \cdot q \cdot r \cdot \ldots$

Permutations

A permutation is an ordered arrangement of objects. The number of permutations of n distinct objects taken r at a time is:

• $nP_r = \frac{n!}{(n-r)!}$

Where $n! = n \times (n-1) \times \ldots \times 1$ and $0! = 1$.

Combinations

A combination is a selection of objects where order does not matter. The number of combinations of n objects taken r at a time is:

• $nC_r = \frac{n!}{r!(n-r)!}$

Permutations with Nondistinct Items

If there are n objects, with $n_1$ of one kind, $n_2$ of another, ..., $n_k$ of a k-th kind, the number of distinct permutations is:

• $\frac{n!}{n_1! n_2! \ldots n_k!}$

Probabilities Involving Permutations and Combinations

To compute probabilities in complex scenarios (e.g., lotteries), use the ratio of favorable outcomes to total possible outcomes, often calculated using combinations.

Choosing Probability and Counting Methods

Choosing the Appropriate Probability Rule

Use a decision process to select the correct probability rule based on the nature of the events (e.g., equally likely, empirical, subjective, disjoint, independent, etc.).

Choosing the Appropriate Counting Technique

Use a decision process to select the correct counting technique (multiplication rule, permutations, combinations, etc.) based on whether order matters, objects are distinct, and choices are independent.

Bayes’s Rule and the Rule of Total Probability

Rule of Total Probability

If the sample space is partitioned into disjoint events $A_1, A_2, ..., A_n$, then for any event E:

• $P(E) = P(A_1)P(E|A_1) + P(A_2)P(E|A_2) + \ldots + P(A_n)P(E|A_n)$

Example: If 55% of students are female (15% business majors) and 45% are male (20% business majors), then the probability a student is a business major is $0.55 \times 0.15 + 0.45 \times 0.20 = 0.1725$.

Bayes’s Rule

Bayes’s Rule allows us to reverse conditional probabilities. For two disjoint sets $U_1$ and $U_2$:

• $P(U_1|B) = \frac{P(U_1)P(B|U_1)}{P(U_1)P(B|U_1) + P(U_2)P(B|U_2)}$

Example: Given the business major example above, the probability that a randomly chosen business major is female is $\frac{0.55 \times 0.15}{0.55 \times 0.15 + 0.45 \times 0.20} = 0.4783$.

Additional info: These notes cover all major probability rules, empirical and classical methods, conditional probability, counting techniques, and the application of Bayes’s Rule, as outlined in a typical college statistics curriculum.