Probability, Counting, and Normal Distributions: Structured Study Notes for Statistics Students

Probability and Its Applications

Basic Concepts of Probability

Probability is a fundamental concept in statistics, quantifying the likelihood of events. Values range from 0 (impossible event) to 1 (certain event). Understanding probability is essential for hypothesis testing and inferential statistics.

• Probability Value: Expressed as a decimal between 0 and 1.

• Rare Event Rule: Events with very low probability (≤0.05) are considered rare and may indicate significant results.

• Notation: P(A) denotes the probability of event A; P(A—) denotes the probability that event A does not occur.

Three Approaches to Probability:

• Relative Frequency Approximation: $P(A) = \frac{\text{number of times A occurred}}{\text{number of times the procedure was repeated}}$

• Classical Approach (Equally Likely Outcomes): $P(A) = \frac{\text{number of ways A occurs}}{\text{number of different simple events}}$

• Subjective Probability: Estimated using knowledge and judgment.

Law of Large Numbers: As the number of trials increases, the relative frequency probability approaches the true probability.

Complementary Events

The complement of event A is the event that A does not occur. The probability of the complement is:

• $P(A—) = 1 - P(A)$

• Example: If $P(I) = 0.89$ (adult uses Internet), then $P(I—) = 0.11$ (adult does not use Internet).

Odds

Odds are another way to express likelihood, commonly used in gambling.

• Actual Odds Against: $\text{Odds against} = \frac{P(A—)}{P(A)}$

• Payoff Odds: Set by gambling operators, often differ from actual odds to ensure profit.

• Advantages: Probabilities are preferred for calculations; odds are used for money transfers.

Probability Examples

• Probability of rolling a 5 on a six-sided die: $\frac{1}{6}$

• Probability of getting heads on a coin: $\frac{1}{2}$

• Probability of winning California Daily 4 lottery: $\frac{1}{10,000}$

• Probability of a car rollover in a crash: $\frac{83,600}{5,200,000} \approx 0.016$

Addition and Multiplication Rules

Addition Rule

The addition rule is used to find the probability that either event A or event B occurs (or both).

• General Addition Rule: $P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$

• Disjoint Events: If A and B cannot occur together, $P(A \text{ or } B) = P(A) + P(B)$

• Complementary Events: $P(A \text{ or } A—) = 1$

Multiplication Rule

The multiplication rule is used to find the probability that both event A and event B occur.

• General Multiplication Rule: $P(A \text{ and } B) = P(A) \times P(B|A)$

• Independent Events: $P(A \text{ and } B) = P(A) \times P(B)$

• Dependent Events: Adjust $P(B)$ based on the outcome of A.

• Sampling: With replacement = independent; without replacement = dependent.

• 5% Guideline: For small samples from large populations, treat events as independent.

Redundancy

Redundancy increases reliability by using multiple components. The probability that all components fail is the product of their individual failure probabilities.

• Example: Probability both hard drives fail: $0.0366^2$

• Probability at least one works: $1 - (0.0366^2)$

Complements, Conditional Probability, and Bayes' Theorem

Probability of "At Least One"

To find the probability of at least one occurrence of an event, use the complement rule:

• $P(\text{at least one occurrence}) = 1 - P(\text{no occurrences})$

• Example: Probability of at least one malfunctioning iPhone in 3 selected: $1 - (0.925)^3 = 0.209$

Conditional Probability

Conditional probability is the probability of event B occurring given that event A has already occurred.

• Notation: $P(B|A)$

• Example: Probability of a professional golfer making a hole in one: $\frac{1}{2,500}$ (given professional status)

• Confusion of the Inverse: $P(B|A) \neq P(A|B)$ in general.

Bayes' Theorem

Bayes' theorem updates the probability of an event based on new information.

• Formula: $P(A|B) = \frac{P(A) \cdot P(B|A)}{P(A) \cdot P(B|A) + P(\neg A) \cdot P(B|\neg A)}$

• Example: Probability of cancer given a positive test result:

• $P(C|\text{positive}) = \frac{0.01 \times 0.80}{0.01 \times 0.80 + 0.99 \times 0.10} = 0.0748$

Counting Principles

Multiplication Counting Rule

Used to find the total number of possibilities from a sequence of events.

• Example: Number of possible 3-digit codes: $10 \times 10 \times 10 = 1,000$

Factorial Rule

The number of ways n different items can be arranged (order matters):

• $n! = n \times (n-1) \times \ldots \times 1$

• Example: $5! = 120$

Permutations and Combinations

• Permutations (Order Matters): $P(n, r) = \frac{n!}{(n-r)!}$

• Permutations with Identical Items: $\frac{n!}{n_1! \cdot n_2! \cdot \ldots \cdot n_k!}$

• Combinations (Order Does Not Matter): $C(n, r) = \frac{n!}{r! (n-r)!}$

Examples:

• Number of ways to select 5 players from 11: $C(11, 5)$

• Number of ways to arrange the word "statistics": Use permutation formula for multiset.

• Probability of winning Powerball: Use combinations for main numbers and separate probability for Powerball number.

Binomial Probability Distributions

Binomial Distribution

A binomial distribution describes the probability of a fixed number of successes in a fixed number of independent trials, each with the same probability of success.

• Criteria: Fixed number of trials, independent trials, only two possible outcomes, constant probability of success.

• Binomial Probability Formula: $P(x) = C(n, x) p^x q^{n-x}$, where $q = 1 - p$

• Mean: $\mu = n p$

• Standard Deviation: $\sigma = \sqrt{n p q}$

Range Rule of Thumb

• Significantly Low: $\leq \mu - 2\sigma$

• Significantly High: $\geq \mu + 2\sigma$

• Values between these are not significant.

Other Discrete Distributions

• Geometric Distribution: Probability of first success on the x-th trial: $P(x) = p (1-p)^{x-1}$

• Multinomial Distribution: For more than two categories: $P(x_1, x_2, x_3) = \frac{n!}{x_1! x_2! x_3!} p_1^{x_1} p_2^{x_2} p_3^{x_3}$

• Hypergeometric Distribution: Sampling without replacement: $P(x) = \frac{\binom{A}{x} \binom{B}{n-x}}{\binom{A+B}{n}}$

Normal Probability Distributions

Standard Normal Distribution

The standard normal distribution is a bell-shaped curve with mean 0 and standard deviation 1.

• Properties: Bell-shaped, mean = 0, standard deviation = 1.

• z Score: Distance from the mean in standard deviation units.

• Empirical Rule:

  • 68% within 1 standard deviation

  • 95% within 2 standard deviations

  • 99.7% within 3 standard deviations

Finding Probabilities and z Scores

• Use technology or Table A-2 to find areas under the curve.

• For a given z score, find the cumulative area from the left.

• For a given area, find the corresponding z score.

• Critical values: $z_{\alpha}$ denotes the z score with area $\alpha$ to its right.

Nonstandard Normal Distributions

Normal distributions with mean $\mu$ and standard deviation $\sigma$ can be converted to the standard normal distribution using:

• $z = \frac{x - \mu}{\sigma}$

• To find x from z: $x = \mu + z \cdot \sigma$

Applications

• IQ scores: $\mu = 100$, $\sigma = 15$

• Pulse rates, heights, weights, and other measurements are often normally distributed.

• Percentiles and quartiles can be calculated using z scores.

The Central Limit Theorem

Statement and Importance

The central limit theorem states that the distribution of sample means approaches a normal distribution as the sample size increases (n > 30), regardless of the population's distribution.

• For sample means: $z = \frac{\bar{x} - \mu}{\sigma / \sqrt{n}}$

• Allows estimation of population parameters and hypothesis testing.

Finite Population Correction

When sampling without replacement from a finite population, adjust the standard deviation:

• Correction factor: $\sqrt{\frac{N - n}{N - 1}}$

• Corrected standard deviation: $\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}} \times \sqrt{\frac{N - n}{N - 1}}$

Simulations for Hypothesis Tests

Purpose and Method

Simulations are used to test claims about population parameters and to find probabilities for complex scenarios.

• Generate many samples with assumed parameters.

• Compare observed sample statistics to simulated results.

• If observed statistics are significantly low or high, the claim is likely incorrect.

Summary Table: Probability Approaches

Approach	Definition	Formula
Relative Frequency	Based on observed outcomes	$P(A) = \frac{\text{number of times A occurred}}{\text{number of trials}}$
Classical	Equally likely outcomes	$P(A) = \frac{\text{number of ways A occurs}}{\text{number of possible outcomes}}$
Subjective	Based on judgment	N/A

Summary Table: Counting Rules

Rule	Formula	When to Use
Multiplication	$n_1 \times n_2 \times \ldots \times n_k$	Sequence of events
Factorial	$n!$	Arranging n items
Permutation	$P(n, r) = \frac{n!}{(n-r)!}$	Order matters
Combination	$C(n, r) = \frac{n!}{r! (n-r)!}$	Order does not matter

Summary Table: Binomial Distribution

Parameter	Formula
Mean	$\mu = n p$
Standard Deviation	$\sigma = \sqrt{n p q}$
Probability of x successes	$P(x) = C(n, x) p^x q^{n-x}$

Summary Table: Normal Distribution

Parameter	Formula
z Score	$z = \frac{x - \mu}{\sigma}$
Area under curve	Use Table A-2 or technology
Sample mean z	$z = \frac{\bar{x} - \mu}{\sigma / \sqrt{n}}$
Finite population correction	$\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}} \times \sqrt{\frac{N - n}{N - 1}}$

Key Definitions

• Probability: Measure of likelihood of an event.

• Complement: Probability that an event does not occur.

• Conditional Probability: Probability of one event given another.

• Permutation: Arrangement of items where order matters.

• Combination: Selection of items where order does not matter.

• Binomial Distribution: Probability distribution for number of successes in fixed trials.

• Normal Distribution: Bell-shaped probability distribution.

• Central Limit Theorem: Distribution of sample means approaches normal as sample size increases.

Additional info: These notes expand on brief points with academic context, examples, and formulas to ensure completeness and clarity for exam preparation.