Statistical Power and Sample Size Determination

Statistical Power and Sample Size Determination

Introduction

Statistical power and sample size calculations are essential components of experimental design in statistics. They ensure that studies are adequately equipped to detect meaningful effects and that resources are used efficiently. This guide covers the concepts of statistical errors, power, effect size, and the practical use of power analysis in research.

Errors in Hypothesis Testing

Types of Errors

• Type I Error (α): Rejecting the null hypothesis when it is actually true. The probability of a Type I error is denoted by α (significance level).

• Type II Error (β): Failing to reject the null hypothesis when it is actually false. The probability of a Type II error is denoted by β.

Decision Table for Hypothesis Testing

Decision	H0 is really true (no effect)	H0 is really false (effect exists)
Retain H0	Correct decision: prob = 1 - α	Type II error: prob = β
Reject H0	Type I error: prob = α	Correct decision: prob = 1 - β

Statistical Power

Definition and Importance

• Statistical Power (1 - β): The probability of correctly rejecting a false null hypothesis (i.e., detecting a true effect).

• Power is typically set at 80% as a standard in research.

• Power depends on:

  • Significance level (α)

  • Sample size (n)

  • Effect size

Power Analysis

Purpose and Benefits

• Determines the sample size needed for an experiment to achieve a desired power.

• Ensures results are conclusive and interpretable.

• Minimizes unnecessary use of resources and participant burden.

• Addresses ethical considerations by avoiding underpowered or overpowered studies.

Types of Power Analysis

• A priori power analysis: Determines the required sample size before data collection, given a desired effect size and significance level.

• Post hoc power analysis: Calculates the power of a test after data collection, useful for interpreting non-significant results.

• Compromised power analysis: Balances error risks when sample size is restricted, adjusting α and β accordingly.

Examples of Power Analysis

A Priori Power Analysis Example

• Clinical trial comparing Drug A and control for reducing cancer cell proliferation.

• Hypotheses:

  • H0: No difference in proliferation rate between Drug A and control.

  • H1: Drug A reduces proliferation rate by 15% compared to control.

• Power analysis determines the sample size needed to detect a 15% difference.

Post Hoc Power Analysis Example

• Given a fixed sample (e.g., 40 patients per group), power analysis can assess whether a non-significant result is due to insufficient power (high β).

Numerical Example: Fasting Blood Glucose

• Population mean: 85 mg/dL, standard deviation: 7.2, sample size: 30, α = 0.05.

• Critical values for hypothesis testing are calculated using:

$z=\frac{\overline{X} - \mu}{\sigma/ \sqrt{n}}Criticalvaluesforhypothesistestingarecalculatedusing:Criticalvaluesforhypothesistestingarecalculatedusing:Criticalvaluesforhypothesistestingarecalculatedusing:Criticalvaluesforhypothesistestingarecalculatedusing:Criticalvaluesforhypothesistestingarecalculatedusing:Criticalvaluesforhypothesistestingarecalculatedusing:Criticalvaluesforhypothesistestingarecalculatedusing:Criticalvaluesforhypothesistestingarecalculatedusing:Criticalvaluesforhypothesistestingarecalculatedusing:$

• If the true mean is 90 mg/dL, power analysis estimates the probability of detecting this difference.

How to Increase Statistical Power

1. Increase Alpha (α)

• Higher α increases power but also increases the risk of Type I error.

• Common values: α = 0.05 (5%), α = 0.01 (1%).

2. Increase Sample Size (n)

• Larger sample sizes reduce β, increasing power.

• Resource and ethical considerations must be balanced.

3. Increase Effect Size

• Effect size is the magnitude of the difference between groups, relative to the standard deviation.

• Larger effect sizes are easier to detect, requiring smaller sample sizes for the same power.

• Effect size can be quantified using Cohen's d:

$\text{Cohen's } d = \frac{\text{Mean difference between } \overline{X}_1 \text{ and } \overline{X}_2}{\text{Pooled standard deviation}}$

Effect Size Classification

Effect size	Difference between means
Small effect	d = 0.2 standard deviations
Medium effect	d = 0.5 standard deviations
Large effect	d = 0.8 standard deviations

Example: Calculating Cohen's d

Subject	Blood Glucose (mmol/L) after Chinese Tea	Blood Glucose (mmol/L) after Milk Tea	Difference
Peter	7	10	3
Simon	8	11	3
David	9	11	2
John	11	13	2
Mike	11	13	2
Mean	8.6	10.8	2.2
SD	1.51	1.48	1.49

Cohen's d = 1.472 (Large effect)

Using G*Power Software

Overview

• G*Power is a statistical software tool for conducting power analyses.

• Allows users to specify test type, effect size, α, β, and sample size to calculate the required parameters for adequate power.

Example Output from G*Power

• For a given dataset, G*Power may suggest that 7 samples per group are needed to achieve power > 80%.

• If only 5 subjects are available, the program may indicate that 2 more are needed.

Summary Table: Factors Affecting Power

Factor	Effect on Power
Increase α	Increases power, but increases Type I error risk
Increase sample size	Increases power, reduces Type II error
Increase effect size	Increases power, easier to detect differences

Conclusion

Understanding statistical power and sample size is crucial for designing robust experiments and interpreting results. Proper power analysis ensures that studies are neither underpowered (risking false negatives) nor wastefully overpowered, balancing scientific rigor with ethical and practical considerations.