Research Methods in Psychology: Reliability and Validity

Chapter 2: Research Methods pt. 1

Introduction to Measurement in Psychology

Psychological research relies on accurate measurement of concepts such as personality traits, behaviors, and attitudes. Researchers must ensure that their measures are both reliable and valid to draw meaningful conclusions.

• Key Question: Are these measures actively doing a good job?

• Example: Administering surveys to capture concepts like extroversion.

Reliability of Measurement

Definition and Types of Reliability

Reliability refers to how consistent a measure is. A reliable measure yields similar results under consistent conditions.

• Test-Retest Reliability: The degree to which people get the same score on several occasions or at different time points. Example: If a person takes a personality test twice over a month and gets similar scores, the test has high test-retest reliability.

• Inter-Rater Reliability: The extent to which different raters or judges provide the same score or rating. Example: Two psychologists independently rate a child's behavior and assign similar scores.

Formula for Reliability:

$\text{Reliability} = \frac{\text{True Score Variance}}{\text{Observed Score Variance}}$

Application of Inter-Rater Reliability

Inter-rater reliability is crucial in observational studies, where multiple observers rate behaviors or outcomes. High agreement among raters increases confidence in the measurement.

• Example: Judges scoring a performance and assigning similar ratings.

Validity of Measurement

Definition and Types of Validity

Validity refers to how well a measure captures what it is supposed to measure. A valid measure accurately reflects the concept of interest.

• Convergent Validity: The extent to which a measure is related to other measures of the same construct. Example: A new test for conscientiousness should correlate highly with established conscientiousness measures.

• Discriminant Validity: The extent to which a measure is unrelated to measures of different constructs. Example: A conscientiousness test should not correlate strongly with measures of unrelated traits, such as extraversion.

Application: Convergent and Discriminant Validity

Researchers often publish studies examining the validity of psychological measures. For example, a study in the Journal of Research in Personality might compare different measures of conscientiousness to ensure they are not simply measuring something else (avoiding the "jangle fallacy").

• Jangle Fallacy: Mistaking two measures as different when they actually assess the same construct.

Summary Table: Reliability and Validity

Type	Definition	Example
Test-Retest Reliability	Consistency of scores over time	Personality test taken twice yields similar scores
Inter-Rater Reliability	Agreement between different raters	Two judges rate a behavior similarly
Convergent Validity	Measure correlates with similar constructs	New anxiety scale correlates with established anxiety scales
Discriminant Validity	Measure does not correlate with different constructs	Anxiety scale does not correlate with extraversion scale

Additional info:

• Reliability is necessary but not sufficient for validity; a measure can be reliable but not valid.

• Validity ensures that the measure is meaningful and useful for research purposes.

• Researchers use statistical methods (e.g., correlation coefficients) to assess reliability and validity.