Statistical Tests, Regression, and Correlation: Study Guide

Regression Analysis

Simple Linear Regression

Simple linear regression is a statistical method used to model the relationship between a dependent variable and a single independent variable by fitting a linear equation to observed data.

• Equation: $y = a + b_1x$

• y: Dependent variable (outcome)

• x: Independent variable (predictor)

• a: Intercept (value of y when x = 0)

• b_1: Slope (change in y for a one-unit change in x)

• Application: Predicting a student's test score based on hours studied.

Multiple Regression

Multiple regression extends simple linear regression by modeling the relationship between a dependent variable and two or more independent variables.

• Equation: $y = a + b_1x_1 + b_2x_2 + ... + b_rx_r$

• Application: Predicting blood pressure based on age, weight, and exercise frequency.

Regression Models & Standard Error

• Lowest Standard Error: The best regression model is often the one with the lowest standard error, indicating the best fit to the data.

Logistic Regression

Logistic regression is used when the dependent variable is binary (e.g., yes/no, success/failure).

• Equation: No explicit equation provided in the notes, but generally: $\text{logit}(p) = a + b_1x_1 + ... + b_rx_r$

• Application: Predicting the probability of disease presence (yes/no) based on risk factors.

Correlation

Pearson's Correlation Coefficient (r)

Pearson's r measures the strength and direction of the linear relationship between two continuous variables.

• Range: -1 (perfect negative) to +1 (perfect positive)

• Equation: $r = \frac{\sum (x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum (x_i - \bar{x})^2 \sum (y_i - \bar{y})^2}}$

• Interpretation: r = 0 (no linear relationship), r = 1 (perfect positive), r = -1 (perfect negative)

• Example: Correlation between height and weight in adults.

Spearman's Rho

Spearman's rho is a non-parametric measure of rank correlation, used when data are ordinal or not normally distributed.

• Application: Correlation between class rank and test scores.

Statistical Tests

t-Tests

• Independent t-test: Compares means between two independent groups (e.g., males vs. females).

• Dependent t-test (Paired t-test): Compares means within the same group at two time points (e.g., pretest vs. posttest).

• Equation for t-test: $t = \frac{\bar{x}_1 - \bar{x}_2}{\sqrt{\frac{s_1^2}{n_1} + \frac{s_2^2}{n_2}}}$

ANOVA (Analysis of Variance)

ANOVA is used to compare means across three or more independent groups.

• One-way ANOVA: Tests for differences among group means for one independent variable.

• Repeated Measures ANOVA: Used when the same subjects are measured multiple times.

• Equation (F-statistic): $F = \frac{\text{Between-group variance}}{\text{Within-group variance}}$

• Example: Comparing test scores among students from three different classes.

Chi-Square Tests

• Chi-Square Test of Independence: Tests association between two categorical variables (e.g., gender and preference

• McNemar's Test: Used for paired nominal data (e.g., pretest/posttest in the same subjects).

• Example: Testing if the proportion of smokers differs by gender.

Levels of Measurement

Understanding the level of measurement is crucial for selecting the appropriate statistical test.

Level	Description	Examples
Nominal	Categories with no order	Gender, blood type
Ordinal	Ordered categories	Class rank, satisfaction rating
Interval	Ordered, equal intervals, no true zero	Temperature (Celsius)
Ratio	Ordered, equal intervals, true zero	Height, weight, age

Choosing the Right Test

The choice of statistical test depends on the type of variables (categorical or continuous), the number of groups, and whether the groups are independent or dependent.

Test	Type of Data	Groups	Example
Independent t-test	Continuous (Interval/Ratio)	2, independent	Male vs. female test scores
Dependent t-test	Continuous (Interval/Ratio)	2, paired	Pretest vs. posttest
One-way ANOVA	Continuous (Interval/Ratio)	3+, independent	Scores across 3 classes
Repeated Measures ANOVA	Continuous (Interval/Ratio)	3+, paired	Scores at 3 time points
Chi-Square	Categorical (Nominal/Ordinal)	2+, independent	Gender vs. preference
McNemar's Test	Categorical (Nominal)	2, paired	Pre/post intervention
Pearson's r	Continuous (Interval/Ratio)	2 variables	Height and weight
Spearman's rho	Ordinal or non-normal	2 variables	Rank and score

Key Terms and Concepts

• Dependent Variable (DV): The outcome being measured.

• Independent Variable (IV): The variable being manipulated or categorized.

• Binary Variable: A variable with two categories (e.g., yes/no).

• Association/Relationship: Indicates a connection between variables (correlation or causation).

• Difference: Indicates a comparison between groups or conditions.

Additional info: Some content and examples were inferred and expanded for clarity and completeness based on standard statistics curriculum.