Statistical Tests, Regression, and Correlation: Study Guide

Regression Analysis

Simple Linear Regression

Simple linear regression is a statistical method used to model the relationship between a dependent variable and a single independent variable by fitting a linear equation to observed data.

• Equation: $y = a + b x$

• y: Dependent variable (outcome)

• x: Independent variable (predictor)

• a: Intercept (value of y when x = 0)

• b: Slope (change in y for a one-unit change in x)

• Application: Predicting a student's test score (y) based on hours studied (x).

Multiple Regression

Multiple regression extends simple linear regression by modeling the relationship between a dependent variable and two or more independent variables.

• Equation: $y = a + b_1 x_1 + b_2 x_2 + \ldots + b_r x_r$

• Application: Predicting blood pressure (y) based on age (x1), weight (x2), and exercise frequency (x3).

Regression Models & Standard Error

• Standard Error: Measures the accuracy of predictions made by the regression model. The model with the lowest standard error is considered the best fit.

Logistic Regression

Logistic regression is used when the dependent variable is binary (e.g., yes/no, success/failure).

• Equation: No explicit equation provided in the notes, but generally: $\log\left(\frac{p}{1-p}\right) = a + b_1 x_1 + b_2 x_2 + \ldots$

• Application: Predicting whether a patient has diabetes (yes/no) based on age, BMI, and blood sugar level.

Statistical Tests

t-Tests

• Independent t-test: Compares the means of two independent groups (e.g., males vs. females).

• Dependent t-test (Paired t-test): Compares means from the same group at different times (e.g., pretest vs. posttest).

• Key Words: 'difference', 'mean difference', 'pretest', 'posttest'

• Level of Measurement: Interval or ratio

ANOVA (Analysis of Variance)

• Purpose: Compares means across three or more independent groups.

• Types:

  • One-way ANOVA: One independent variable with multiple groups.

  • Repeated Measures ANOVA: Same subjects measured multiple times.

• Level of Measurement: Dependent variable must be interval or ratio.

• Example: Comparing math scores among students from three different schools.

Chi-Square Tests

• Purpose: Tests association between categorical variables.

• Types:

  • Chi-Square Test of Independence: Tests if two categorical variables are independent.

  • McNemar Test: Used for paired nominal data (e.g., before/after in the same subjects).

• Level of Measurement: Nominal or ordinal

• Example: Testing if gender is associated with preference for a new product.

Correlation

Pearson Correlation Coefficient (r)

The Pearson correlation coefficient measures the strength and direction of the linear relationship between two continuous variables.

• Range: -1 (perfect negative) to +1 (perfect positive)

• Interpretation:

  • r = 0: No linear relationship

  • r = +1: Perfect positive linear relationship

  • r = -1: Perfect negative linear relationship

• Level of Measurement: Interval or ratio

• Example: Correlation between height and weight.

Spearman's Rho

Spearman's rho is a non-parametric measure of rank correlation, used when data are ordinal or not normally distributed.

• Application: Correlation between class rank and test scores.

Levels of Measurement

Understanding the level of measurement is crucial for selecting the appropriate statistical test.

Level	Description	Examples
Nominal	Categories with no order	Gender, blood type
Ordinal	Ordered categories	Class rank, satisfaction rating
Interval	Ordered, equal intervals, no true zero	Temperature (Celsius), IQ score
Ratio	Ordered, equal intervals, true zero	Height, weight, age

Choosing the Right Statistical Test

The choice of statistical test depends on the research question, the number of groups, the type of variables, and the level of measurement.

Test	Purpose	Variables	Level of Measurement	Groups
Independ t-test	Compare means	1 dep, 1 indep	Dep: Interval/Ratio	2 independent
Depend t-test	Compare means	1 dep, 1 indep	Dep: Interval/Ratio	2 related
ANOVA	Compare means	1 dep, 1 indep	Dep: Interval/Ratio	3+ independent
Chi-Square	Test association	2 categorical	Nominal/Ordinal	2+ groups
Pearson Correlat	Relationship	2 continuous	Interval/Ratio	--
Spearman's Rho	Relationship	2 variables	Ordinal	--
Logistic Regression	Predict binary outcome	1 binary dep, 1+ indep	Dep: Nominal (binary)	--

Key Terms and Concepts

• Dependent Variable (DV): The outcome variable being measured.

• Independent Variable (IV): The variable that is manipulated or categorized to observe its effect on the DV.

• Binary Variable: A variable with only two categories (e.g., yes/no).

• Continuous Variable: A variable that can take any value within a range (e.g., height, weight).

• Association: A relationship between two variables.

• Difference: A comparison of means or proportions between groups.