Statistical Hypothesis Testing and Confidence Intervals: Key Concepts and Applications

Chapter 16: Hypothesis Testing Fundamentals

Null and Alternative Hypotheses

In statistical hypothesis testing, we begin by stating two competing hypotheses:

• Null Hypothesis (H0): The default assumption that there is no effect or no difference.

• Alternative Hypothesis (Ha): The hypothesis that there is an effect or a difference.

For example, in testing whether a coin is fair:

• H0: The coin is fair (p = 0.5).

• Ha: The coin is not fair (p ≠ 0.5).

P-Value and Significance Level

The p-value measures the probability of observing data as extreme as the sample, assuming the null hypothesis is true. The significance level ($\alpha$) is the threshold for rejecting H0.

• If p-value < $\alpha$, reject H0.

• If p-value > $\alpha$, fail to reject H0.

One-Sided vs. Two-Sided Tests

A one-sided test checks for deviation in one direction (e.g., greater than), while a two-sided test checks for deviation in both directions.

• One-sided p-value: Probability of observing a value as extreme in one direction.

• Two-sided p-value: Probability of observing a value as extreme in either direction (often double the one-sided p-value).

Calculating P-Values for Z-Scores

For a z-score, the p-value is found using the standard normal distribution:

• One-sided: $p = P(Z > z)$ or $p = P(Z < z)$

• Two-sided: $p = 2 \times P(Z > |z|)$

Assumptions for One-Proportion Z-Test

• Random sample

• Sample size large enough for normal approximation: $np \geq 10$ and $n(1-p) \geq 10$

Performing a One-Proportion Z-Test

• Calculate test statistic: $z = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}$

• Find p-value and compare to $\alpha$

Stating Conclusions

• Clearly state whether H0 is rejected or not, and interpret in context.

Chapter 18: Confidence Intervals and One-Sample T-Test

Constructing a Confidence Interval for a Mean

A confidence interval estimates the range in which a population mean likely falls.

• Formula: $\bar{x} \pm t^* \frac{s}{\sqrt{n}}$

• $\bar{x}$ = sample mean, $s$ = sample standard deviation, $n$ = sample size, $t^*$ = critical value from t-distribution

Interpreting a Confidence Interval

• A 95% confidence interval means that, in repeated sampling, 95% of intervals will contain the true mean.

Assumptions for T-Test and Confidence Interval

• Random sample

• Approximately normal population or large sample size ($n \geq 30$)

Performing a One-Sample T-Test

• Test statistic: $t = \frac{\bar{x} - \mu_0}{s / \sqrt{n}}$

• Compare p-value to $\alpha$

Interpreting T-Test Output

• State whether the sample mean is significantly different from the hypothesized mean.

Chapter 19: Two-Sample T-Test and Confidence Intervals for Differences

Assumptions for Two-Sample T-Test

• Independent random samples

• Approximately normal populations or large sample sizes

Confidence Interval for Difference Between Means

• Formula: $(\bar{x}_1 - \bar{x}_2) \pm t^* \sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}$

Using Confidence Interval to Approximate Hypothesis Test

• If the interval does not contain 0, there is evidence of a difference.

Performing a Two-Sample T-Test

• Test statistic: $t = \frac{(\bar{x}_1 - \bar{x}_2) - \Delta_0}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}$

Interpreting Output

• State whether there is a significant difference between the two means.

Chapter 20: Paired T-Test, Sign Test, and Choosing Tests

Graphs of Distributions

• Visualize data using histograms, boxplots, or density plots to assess normality and identify outliers.

Paired T-Test

• Used when samples are paired (e.g., before-and-after measurements).

• Test statistic: $t = \frac{\bar{d}}{s_d / \sqrt{n}}$ where $\bar{d}$ is mean difference, $s_d$ is standard deviation of differences.

Interpreting Paired T-Test Output

• Determine if the mean difference is significant.

Choosing Appropriate Test

• Assess data type (mean or proportion), sample structure (paired or independent), and assumptions.

Confidence Interval for a Difference

• Construct as for two-sample or paired data, depending on design.

Sign Test for Paired Samples

• Non-parametric test for paired data; counts number of positive and negative differences.

Z-Test for Proportion in Sign Test

• Used to find p-value for sign test when sample size is large.

Interpreting Sign Test Output

• State whether there is a significant difference in paired samples.

Chapter 28 (Sections 28.1 and 28.3): Errors and Reporting

Type I and Type II Errors

• Type I Error: Rejecting H0 when it is true (false positive).

• Type II Error: Failing to reject H0 when it is false (false negative).

Reporting Results

• Clearly state test used, assumptions checked, p-value, and conclusion.

• Provide biological or practical interpretation if relevant.

Study Design for Testing Means

• Ensure randomization, appropriate sample size, and correct test selection.

Chapter 29 (Second Edition, Sections 29.1 and 29.3): Non-Parametric Tests

Wilcoxon Rank Sum Test

• Non-parametric test for comparing two independent samples.

• Assumptions: independent samples, ordinal or continuous data.

Interpreting Wilcoxon Rank Sum Test Output

• State whether there is a significant difference in distributions.

Wilcoxon Signed Rank Test

• Non-parametric test for paired samples.

• Assumptions: paired samples, differences are symmetric.

Interpreting Wilcoxon Signed Rank Test Output

• State whether there is a significant difference in paired data.

When to Use Non-Parametric Tests

• When data do not meet parametric test assumptions (e.g., non-normality, ordinal data).

Summary Table: Choosing the Right Statistical Test

Test	Data Type	Sample Structure	Assumptions
One-Proportion Z-Test	Proportion	Single sample	Large sample, random
One-Sample T-Test	Mean	Single sample	Normality or large n
Two-Sample T-Test	Mean	Two independent samples	Normality or large n
Paired T-Test	Mean	Paired samples	Normality of differences
Sign Test	Paired differences	Paired samples	None (non-parametric)
Wilcoxon Rank Sum	Ordinal/continuous	Two independent samples	Independent samples
Wilcoxon Signed Rank	Ordinal/continuous	Paired samples	Symmetric differences

Additional info: Some details on formulas and assumptions were expanded for clarity and completeness.