The Power of a Hypothesis Test and the Power Function

The Power of a Hypothesis Test

Introduction to Hypothesis Testing Power

In hypothesis testing, it is important to evaluate how effective a test is at detecting false null hypotheses. The power of a test quantifies this effectiveness and helps compare different statistical tests.

• Criteria for a Good Test: A good test should have a high probability of correctly rejecting a false null hypothesis.

• Test Comparison: Tests can be compared based on their power functions.

• Power Function: The power function describes the probability of rejecting the null hypothesis for different parameter values.

Type I Error (α): The probability of rejecting the null hypothesis when it is actually true. Type II Error (β): The probability of failing to reject the null hypothesis when it is false. Power: The probability of correctly rejecting the null hypothesis when it is false (i.e., 1 - β).

The Power Function

Definition and Properties

The power function of a hypothesis test is defined as the probability of rejecting the null hypothesis for each possible value of the parameter being tested. It is denoted as:

$\gamma(\theta) = P_\theta(\text{reject } H_0)$

• The power function depends on the true value of the parameter $\theta$.

• For values of $\theta$ consistent with $H_0$, the power function should be low (ideally equal to the significance level $\alpha$).

• For values of $\theta$ consistent with $H_1$, the power function should be high.

Ideal Power Function

The ideal power function is:

$\gamma(\theta) = \begin{cases} \alpha, & \theta \in \Theta_0 \\ 1, & \theta \in \Theta_1 \end{cases}$

where $\Theta_0$ is the set of parameter values under $H_0$ and $\Theta_1$ is the set under $H_1$.

Example: Binomial Test

Suppose $X$ is the number of successes in $n$ Bernoulli trials with probability $\theta$. We test $H_0: \theta = \theta_0$ vs. $H_1: \theta = \theta_1 > \theta_0$. The test rejects $H_0$ if $X \geq c$ for some critical value $c$.

The power function is:

$\gamma(\theta) = P_\theta(X \geq c) = \sum_{k=c}^n {n \choose k} \theta^k (1-\theta)^{n-k}$

Graphical Comparison

Power functions can be plotted for different tests to visually compare their effectiveness. The test with the higher power function for values of $\theta$ in $\Theta_1$ is generally preferred.

Normal Approximation

For large $n$, the binomial distribution can be approximated by the normal distribution. The power function becomes:

$\gamma(\theta) = 1 - \Phi\left( \frac{c - n\theta}{\sqrt{n\theta(1-\theta)}} \right)$

where $\Phi$ is the cumulative distribution function of the standard normal distribution.

Tabular Comparison of Power Values

The following table shows example power values for different parameter values (inferred from context):

θ	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9
Power	0.05	0.10	0.20	0.35	0.50	0.65	0.80	0.90	0.95

Additional info: Table values are illustrative and inferred for demonstration.

Remarks

• Fixing a test to have a small significance level $\alpha$ may reduce power, especially for small sample sizes.

• The operating characteristic curve is another name for the power function, especially in quality control contexts.

Exercises

1. Given a coin with unknown probability $p$, test $H_0: p = 0.5$ vs. $H_1: p = 0.7$ using $n = 10$ tosses. The test rejects $H_0$ if at least 8 heads are observed. Compute the significance level and power of the test.

2. Use the Central Limit Theorem to approximate the power function for a test of $H_0: \mu = 0$ vs. $H_1: \mu = 1$ for a normal population with known variance, using a sample mean and a critical value $c$.

3. For a test statistic $Z = \frac{\bar{X} - \mu_0}{\sigma/\sqrt{n}}$, calculate the power function for different values of $n$ and $\Delta = \mu_1 - \mu_0$.

Additional info: These exercises reinforce the calculation and interpretation of power functions in hypothesis testing.