Statistics Final Exam Study Notes: Probability, Random Variables, and Regression

Probability Rules and Concepts

Basic Probability Rules

Probability is the mathematical study of randomness and uncertainty. The following rules are fundamental for calculating probabilities in various scenarios:

• Total Probability Rule: The probability of the sample space S is always 1. $P(S) = 1$

• Complement Rule: The probability of an event A is 1 minus the probability of its complement. $P(A) = 1 - P(A^C)$

• Addition Rule for Disjoint Events: If A and B are mutually exclusive, $P(A \text{ or } B) = P(A) + P(B)$

• General Addition Rule: For any two events, $P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$

• Conditional Probability: The probability of B given A is $P(B|A) = \frac{P(A \text{ and } B)}{P(A)}$

• General Multiplication Rule: $P(A \text{ and } B) = P(A) \times P(B|A)$

• Multiplication Rule for Independent Events: If A and B are independent, $P(A \text{ and } B) = P(A) \times P(B)$

Random Variables and Their Properties

Expected Value and Variance

A random variable is a numerical outcome of a random process. The expected value (mean), variance, and standard deviation are key characteristics:

• Expected Value (Mean): $E(X) = \mu = \sum x \cdot P(x)$

• Variance: $Var(X) = \sigma^2 = \sum (x - \mu)^2 \cdot P(x)$

• Standard Deviation: $SD(X) = \sigma = \sqrt{Var(X)}$

Combining Random Variables

• Adding/Subtracting Constants: Adding a constant to a random variable shifts the mean but does not affect the variance or standard deviation.

• Multiplying by Constants: Multiplying a random variable by a constant a multiplies the mean by a and the standard deviation by |a|: $SD(aX) = |a| SD(X)$

• If X and Y are independent: $Var(X \pm Y) = Var(X) + Var(Y)$

Binomial Model

Binomial Probability

The binomial model describes the probability of k successes in n independent trials, each with probability p of success:

• Binomial Coefficient: $C(n, k) = \frac{n!}{k!(n-k)!}$

• Probability of k successes: $P(X = k) = C(n, k) p^k q^{n-k}$, where $q = 1 - p$

• Mean (Expected Number of Successes): $\mu = np$

• Standard Deviation: $\sigma = \sqrt{npq}$

n = number of trials, k = number of successes, p = probability of success, q = probability of failure.

Correlation and Regression

Correlation Coefficient

The correlation coefficient (r) measures the strength and direction of a linear relationship between two quantitative variables:

• Formula: $r = \frac{S_{xy}}{S_x S_y}$

• Where $S_{xy}$ is the covariance, $S_x$ and $S_y$ are the standard deviations of X and Y, respectively.

Regression Analysis

Regression is used to model the relationship between a dependent variable Y and an independent variable X.

• Residual: $e = Y - \hat{Y}$

• Line of Best Fit: $\hat{Y} = b_0 + b_1 x$

• Slope: $b_1 = r \frac{S_y}{S_x}$

• Intercept: $b_0 = \bar{Y} - b_1 \bar{X}$

• Regression to the Mean: The predicted value $\hat{Y}$ is closer to the mean of Y than X is to the mean of X, unless r = ±1.

• Residual Standard Deviation: Measures the typical size of the residuals (errors) from the regression line.

The Normal Model and Z-Scores

Standard Deviation as a Ruler and the Normal Model

The normal model is a symmetric, bell-shaped distribution characterized by its mean (μ) and standard deviation (σ). Z-scores standardize values for comparison:

• Z-Score: $z = \frac{x - \mu}{\sigma}$

• Z-scores indicate how many standard deviations a value is from the mean.

• Normal tables (Z-tables) are used to find probabilities and percentiles for standard normal distributions.

Using the Standard Normal Table

The standard normal table provides the area (probability) to the left of a given z-score in the standard normal distribution. To use the table:

• Find the row for the first two digits of the z-score.

• Find the column for the second decimal place.

• The intersection gives the cumulative probability.

• For negative z-scores, use the left table; for positive z-scores, use the right table.

Summary Table: Key Formulas and Concepts

Concept	Formula	Description
Expected Value	$E(X) = \sum x \cdot P(x)$	Mean of a random variable
Variance	$Var(X) = \sum (x - \mu)^2 \cdot P(x)$	Spread of a random variable
Standard Deviation	$SD(X) = \sqrt{Var(X)}$	Typical deviation from the mean
Binomial Probability	$P(X = k) = C(n, k) p^k q^{n-k}$	Probability of k successes in n trials
Correlation Coefficient	$r = \frac{S_{xy}}{S_x S_y}$	Strength of linear relationship
Regression Line	$\hat{Y} = b_0 + b_1 x$	Best fit line for prediction
Z-Score	$z = \frac{x - \mu}{\sigma}$	Standardized value