Exam Review: Regression Analysis and Chi-Square/Nonparametric Tests

Chapter 12: Chi-Square and Nonparametric Tests

Overview of Chi-Square and Nonparametric Tests

Chi-square and nonparametric tests are statistical methods used when data do not meet the assumptions required for parametric tests, such as normality. These tests are especially useful for categorical data and for analyzing relationships between variables without assuming a specific distribution.

• Chi-Square Test: Used to test the independence of two categorical variables or the goodness-of-fit of observed data to an expected distribution.

• Nonparametric Tests: Include tests such as the Mann-Whitney U test, Wilcoxon signed-rank test, and Kruskal-Wallis test, which are alternatives to t-tests and ANOVA when data are not normally distributed.

Example: Testing whether customer satisfaction is independent of gender using a chi-square test of independence.

Chapter 13: Simple Linear Regression

Building and Interpreting a Linear Regression Model

Simple linear regression is a statistical method used to model the relationship between a dependent variable and one independent variable. The model is typically built using software such as Excel.

• Regression Equation: The general form is $Y = \beta_0 + \beta_1 X + \varepsilon$, where $Y$ is the dependent variable, $X$ is the independent variable, $\beta_0$ is the intercept, $\beta_1$ is the slope, and $\varepsilon$ is the error term.

• Running Regression in Excel: Input raw data, use the Data Analysis Toolpak, and select 'Regression' to generate output including coefficients, R-square, and residuals.

Interpreting Regression Coefficients and Confidence Intervals

• Regression Coefficient ($\beta_1$): Represents the expected change in $Y$ for a one-unit increase in $X$.

• Intercept ($\beta_0$): The expected value of $Y$ when $X = 0$.

• 95% Confidence Interval: Provides a range of plausible values for the regression coefficients. If the interval does not include zero, the coefficient is statistically significant.

Example: If the 95% confidence interval for $\beta_1$ is [1.2, 2.5], we are 95% confident that the true effect of $X$ on $Y$ lies within this range.

Coefficient of Determination (R-Square)

• Definition: $R^2$ measures the proportion of variance in the dependent variable explained by the independent variable(s).

• Interpretation: An $R^2$ of 0.80 means 80% of the variability in $Y$ is explained by $X$.

Residual Analysis and Model Assumptions

• Residuals: The differences between observed and predicted values ($e_i = y_i - \hat{y}_i$).

• Assumptions: Linearity, independence, homoscedasticity (constant variance), and normality of residuals.

• Residual Plots: Used to check for patterns that violate assumptions.

Hypothesis Testing in Regression

• Overall Model Test (F-test): Tests whether the regression model explains a significant amount of variance in $Y$.

• Null Hypothesis ($H_0$): All regression coefficients are zero (no relationship).

• Alternative Hypothesis ($H_a$): At least one coefficient is not zero.

• Test Statistic: $F = \frac{\text{MSR}}{\text{MSE}}$

Testing Individual Predictors (t-test)

• Purpose: To determine if an independent variable is a significant predictor of the dependent variable.

• Null Hypothesis ($H_0$): $\beta_1 = 0$

• Test Statistic: $t = \frac{\hat{\beta}_1}{SE(\hat{\beta}_1)}$

Prediction and Confidence Intervals

• Prediction: Use the regression equation to estimate $Y$ for a given $X$.

• 95% Confidence Interval for Prediction: Provides a range in which a future observation is expected to fall with 95% confidence.

Example: Predicting sales for a given advertising budget and constructing a 95% confidence interval for the prediction.

Dummy Coding for Categorical Variables

• Dummy Variables: Used to include categorical variables in regression models by coding categories as 0 or 1.

• Building the Model: Each category (except one reference group) is represented by a dummy variable.

• Interpretation: The coefficient for a dummy variable represents the expected change in $Y$ compared to the reference group.

Example: For a variable 'Region' with categories 'North' and 'South', create a dummy variable: North = 1 if region is North, 0 otherwise.

Summary Table: Key Regression Concepts

Concept	Definition	Formula
Regression Equation	Predicts dependent variable from independent variable	$Y = \beta_0 + \beta_1 X + \varepsilon$
R-Square	Proportion of variance explained	$R^2 = 1 - \frac{SS_{res}}{SS_{tot}}$
t-test for Coefficient	Tests if predictor is significant	$t = \frac{\hat{\beta}_1}{SE(\hat{\beta}_1)}$
F-test for Model	Tests if model is significant	$F = \frac{\text{MSR}}{\text{MSE}}$
Dummy Variable	Represents categorical variable in regression	0 or 1 coding

Additional info: Students should review all class examples, practice problems, and homework for comprehensive understanding and application of these concepts.