Probability Distributions, Sampling, and Statistical Inference: Study Notes

Probability Distributions and Their Properties

True/False Concepts in Probability and Statistics

This section reviews foundational concepts in probability distributions, sampling, and statistical inference, clarifying common misconceptions and key definitions.

• Area Under the Curve (AUC) for Continuous Distributions: The total area under the probability density function (PDF) of a continuous probability distribution is always 1. This represents the certainty that some value in the distribution will occur.

• Binomial Distribution: The binomial distribution is a specific type of discrete probability distribution that models the number of successes in a fixed number of independent Bernoulli trials, each with the same probability of success.

• Standard Error: The standard error quantifies the variability of a sampling distribution, measuring how much sample statistics (like the mean or proportion) are expected to vary from sample to sample.

• Central Limit Theorem (CLT): The CLT states that, for sufficiently large sample sizes, the sampling distribution of the sample mean (or proportion) will be approximately normal, regardless of the population's distribution. It does not require the population itself to be normal.

• Z-score of 0: A z-score of 0 indicates that the observation is exactly at the mean of the distribution.

Discrete Probability Distributions

Probability Mass Functions (PMFs) and Validity

Discrete random variables have probability mass functions (PMFs) that assign probabilities to each possible value. For a PMF to be valid:

• All probabilities must be between 0 and 1.

• The sum of all probabilities must equal 1: $\sum P(X = x_i) = 1$

Example Table: Probability distribution for number of heads in 3 coin flips:

X	P(X=x)
0	0.1
1	0.2
2	0.3
3	0.4

If $\sum P(X=x) \neq 1$, the distribution is not valid. Adjust probabilities as needed to ensure validity.

Expected Value and Probability Calculations

• Expected Value (Mean): $E(X) = \sum x_i P(X = x_i)$

• Example: For X = {0,1,2,3} with P(X) = {0.1, 0.2, 0.3, 0.4}, $E(X) = 0(0.1) + 1(0.2) + 2(0.3) + 3(0.4)$

• Cumulative Probability: $P(X < k) = \sum_{x < k} P(X = x)$

Binomial and Geometric Distributions

Binomial Distribution

The binomial distribution models the number of successes in a fixed number of independent trials, each with the same probability of success.

• Parameters: n = number of trials, p = probability of success

• Probability Formula: $P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}$

• Mean: $\mu = np$

• Standard Deviation: $\sigma = \sqrt{np(1-p)}$

• Example: Probability of exactly 5 successes in 8 trials with $p=0.6$: $P(X=5) = \binom{8}{5} (0.6)^5 (0.4)^3$

Geometric Distribution

The geometric distribution models the number of trials needed to get the first success in a sequence of independent Bernoulli trials.

• Probability Formula: $P(X = k) = (1-p)^{k-1} p$

• Example: Probability the first success is on the 10th trial with $p=0.05$: $P(X=10) = (0.95)^9 (0.05)$

Sampling Distributions and the Central Limit Theorem

Sampling Distributions

A sampling distribution is the probability distribution of a statistic (like the mean or proportion) based on a random sample.

• Standard Error (SE): Measures the variability of a statistic across samples. For sample mean: $SE = \frac{\sigma}{\sqrt{n}}$ For sample proportion: $SE = \sqrt{\frac{p(1-p)}{n}}$

• Expected Value of Sample Proportion: $E(\hat{p}) = p$

Central Limit Theorem (CLT)

• For large sample sizes, the sampling distribution of the sample mean (or proportion) is approximately normal, regardless of the population's distribution.

• Z-score Formula: $z = \frac{\text{statistic} - \text{mean}}{SE}$

• Example: For $p=0.7$, $n=60$, $SE = \sqrt{\frac{0.7 \times 0.3}{60}} = 0.059$

Normal Distribution and Z-scores

Normal Distribution

The normal distribution is a continuous, symmetric, bell-shaped distribution defined by its mean ($\mu$) and standard deviation ($\sigma$).

• Z-score: Measures how many standard deviations an observation is from the mean. $z = \frac{x - \mu}{\sigma}$

• Percentiles: To find the value corresponding to a given percentile, use the z-score for that percentile and solve for $x$: $x = z \cdot \sigma + \mu$

• Example: If $\mu=120$, $\sigma=4$, and $z=1.29$ (for the 90th percentile): $x = 1.29 \times 4 + 120 = 125.16$

Summary Table: Key Probability Distributions

Distribution	Type	Parameters	Mean	Variance
Binomial	Discrete	n, p	$np$	$np(1-p)$
Geometric	Discrete	p	$1/p$	$(1-p)/p^2$
Normal	Continuous	$\mu, \sigma$	$\mu$	$\sigma^2$

Applications and Problem-Solving Steps

• Check if the distribution is valid (probabilities sum to 1).

• Identify the type of distribution (binomial, geometric, normal, etc.).

• Use appropriate formulas for mean, standard deviation, and probabilities.

• For sampling distributions, use standard error and the CLT for inference.

• Translate real-world problems into statistical models before calculation.