Probability, Random Variables, and Statistical Inference: Study Notes for Elementary Statistics

Probability and Its Rules

Types of Probability

Probability quantifies the likelihood of events occurring. There are several approaches to defining probability:

• Empirical Probability: Based on observed data from experiments or historical records.

• Theoretical Probability: Calculated using known possible outcomes, assuming all outcomes are equally likely.

• Subjective Probability: Based on personal judgment or experience, not strictly on data or theory.

Probability Rules

• Complement Rule: The probability that an event does not occur is $P(A^c) = 1 - P(A)$.

• Addition Rule: For disjoint (mutually exclusive) events, $P(A ext{ or } B) = P(A) + P(B)$.

• Multiplication Rule: For independent events, $P(A ext{ and } B) = P(A) \times P(B)$.

Disjoint Events (Mutually Exclusive)

• Events that cannot occur simultaneously. If $A$ and $B$ are disjoint, $P(A \cap B) = 0$.

Legitimate Assignment of Probabilities

• Probabilities must be between 0 and 1.

• The sum of probabilities for all possible outcomes must equal 1.

General Probability Rules

General Addition Rule

• For any two events: $P(A \text{ or } B) = P(A) + P(B) - P(A \cap B)$

Conditional Probability

• The probability of event $A$ given event $B$ has occurred: $P(A|B) = \frac{P(A \cap B)}{P(B)}$

General Multiplication Rule

• For any two events: $P(A \cap B) = P(A|B) \times P(B)$

Independence Criteria

• Events $A$ and $B$ are independent if $P(A|B) = P(A)$ and $P(B|A) = P(B)$.

Tree Diagram

• A graphical tool to visualize sequences of events and their probabilities.

• Useful for calculating joint and conditional probabilities.

Random Variables

Types of Random Variables

• Discrete Random Variable: Takes on countable values (e.g., number of heads in coin tosses).

• Continuous Random Variable: Takes on any value within an interval (e.g., height, weight).

Probability Model

• A mathematical description of the probabilities associated with all possible values of a random variable.

Expected Value

• The mean of a random variable, representing its long-run average value: $E(X) = \sum x_i P(x_i)$ for discrete variables.

Standard Deviation and Variance

• Variance: $Var(X) = \sum (x_i - E(X))^2 P(x_i)$

• Standard Deviation: $\sigma_X = \sqrt{Var(X)}$

Transforming Random Variables

• Shifting: Adding a constant $c$ to $X$ increases the mean by $c$ but does not affect the standard deviation.

• Scaling: Multiplying $X$ by a constant $a$ multiplies both the mean and standard deviation by $a$.

Combining Random Variables

• For independent random variables $X$ and $Y$:

• Mean: $E(X \pm Y) = E(X) \pm E(Y)$

• Variance: $Var(X \pm Y) = Var(X) + Var(Y)$

Probability Models

Geometric Probability Model

• Models the number of trials until the first success in repeated, independent Bernoulli trials.

• Probability: $P(X = k) = (1-p)^{k-1}p$

Binomial Probability Model

• Models the number of successes in a fixed number of independent Bernoulli trials.

• Probability: $P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}$

• Success/Failure Condition: Both $np \geq 10$ and $n(1-p) \geq 10$ for normal approximation.

Poisson Probability Model

• Models the number of events occurring in a fixed interval of time or space, given a constant mean rate.

• Probability: $P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!}$

Uniform Probability Model

• All outcomes are equally likely.

• Probability: $P(A) = \frac{\text{Number of outcomes in } A}{\text{Total number of outcomes}}$

Sampling Distributions and Confidence Intervals for Proportions

Sampling Distribution Model

• Describes the distribution of a statistic (e.g., sample mean, sample proportion) over repeated samples.

Sampling Variability (Sampling Error)

• The natural variation in sample statistics from sample to sample.

Confidence Interval

• A range of values, derived from sample data, that is likely to contain the population parameter.

One-Proportion z-Interval

• Used to estimate a population proportion.

• Formula: $\hat{p} \pm z^* \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$

Margin of Error

• The maximum expected difference between the true population parameter and a sample estimate.

Confidence Intervals for Means

Central Limit Theorem (CLT)

• States that the sampling distribution of the sample mean approaches a normal distribution as sample size increases, regardless of the population's distribution.

Student's t-Distribution and Degrees of Freedom (DF)

• Used when the population standard deviation is unknown and sample size is small.

• Degrees of freedom: $df = n - 1$

One-Sample t-Interval for the Mean

• Formula: $\bar{x} \pm t^* \frac{s}{\sqrt{n}}$

Testing Hypotheses

Hypothesis Testing Concepts

• Null Hypothesis ($H_0$): The default assumption (e.g., no effect, no difference).

• Alternative Hypothesis ($H_A$): The claim being tested (e.g., there is an effect).

P-Value

• The probability of observing data as extreme as, or more extreme than, the observed data, assuming $H_0$ is true.

Types of Alternatives

• Two-Sided Alternative: Tests for difference in either direction.

• One-Sided Alternative: Tests for difference in a specific direction.

One-Proportion z-Test

• Tests a hypothesis about a population proportion.

• Test statistic: $z = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}$

Effect Size

• Measures the magnitude of a difference, independent of sample size.

One-Sample t-Test for the Mean

• Test statistic: $t = \frac{\bar{x} - \mu_0}{s/\sqrt{n}}$

More About Tests and Intervals

Statistical Significance and Significance Level

• Statistically Significant: When the p-value is less than the chosen significance level ($\alpha$), the result is considered statistically significant.

• Alpha Level ($\alpha$): The threshold for statistical significance, commonly set at 0.05.

• Critical Value: The value that the test statistic must exceed to reject $H_0$.

Errors in Hypothesis Testing

• Type I Error: Rejecting $H_0$ when it is true (false positive). Probability = $\alpha$.

• Type II Error: Failing to reject $H_0$ when $H_A$ is true (false negative). Probability = $\beta$.

• Power: Probability of correctly rejecting $H_0$ when $H_A$ is true. $\text{Power} = 1 - \beta$.

Error Type	Description	Probability
Type I Error	Reject $H_0$ when $H_0$ is true	$\alpha$
Type II Error	Fail to reject $H_0$ when $H_A$ is true	$\beta$
Power	Correctly reject $H_0$ when $H_A$ is true	$1 - \beta$