Statistical Foundations and Data Analysis: Mini-Textbook Study Notes

Statistical Foundations and Data Analysis

Introduction to Statistics

Statistics is the science of collecting, organizing, summarizing, and analyzing data to draw conclusions and quantify the confidence in those conclusions. Proper statistical analysis is essential for supporting or refuting claims, and it is important to recognize that statistical results always carry some uncertainty.

• Key Considerations: Proper analysis can support or refute claims; poorly collected data are useless; lurking variables can create spurious relationships; statistical results are never certain, but they provide a measure of uncertainty.

The Statistical Process

The statistical process consists of four main steps, each crucial for valid inference and analysis.

1. Identify the research objective: Clearly state the question and the population of interest.

2. Collect the data: Use appropriate sampling or measurement methods.

3. Describe the data: Summarize with numerical measures, tables, and graphs.

4. Infer to the population: Extend sample results to the whole population and assess reliability (confidence intervals, hypothesis tests).

Data Types and Levels of Measurement

Types of Data

Data can be classified as quantitative or categorical, each with distinct properties and uses.

• Quantitative (Numerical): Measures that can be added, subtracted, multiplied, or divided.

  • Discrete: Finite or countable set of values (e.g., number of children).

  • Continuous: Infinite set of values; any value within an interval is possible (e.g., height, weight).

• Categorical (Qualitative): Classifications based on attributes (e.g., gender, color).

Levels of Measurement

Levels of measurement determine the mathematical operations that can be performed on data.

• Nominal: Names or labels without intrinsic order.

• Ordinal: Ordered categories but without meaningful distance (e.g., ranking).

• Interval: Ordered with meaningful differences; zero does not indicate absence (e.g., temperature in Celsius).

• Ratio: Ordered with meaningful ratios; true zero exists (e.g., weight, age).

Populations, Samples, Parameters, and Statistics

Definitions

• Population: The entire set of individuals or objects of interest.

• Individual: A single member of the population.

• Sample: A subset of the population from which data are collected.

• Census: Data collection from every member of the population.

• Parameter: A numerical summary that describes a population (e.g., population mean $\mu$, population standard deviation $\sigma$).

• Statistic: A numerical summary that describes a sample (e.g., sample mean $\bar{x}$, sample standard deviation $s$).

Sampling Methods

Types of Sampling

• Simple Random Sampling (SRS): Every possible sample of size $n$ from a population of size $N$ has an equal chance of selection.

• Stratified Sampling: The population is divided into non-overlapping subgroups (strata); an SRS is taken from each stratum.

• Systematic Sampling: Choose a random start between 1 and $k$ and then select every $k$th element.

• Cluster Sampling: The population is divided into clusters; a random sample of clusters is selected and all members of chosen clusters are surveyed.

• Convenience Sampling: Samples are drawn from readily available subjects; rarely represents the population.

Bias in Sampling

• Sampling Bias: The selection procedure favours certain parts of the population.

• Nonresponse Bias: Individuals who do not respond differ systematically from respondents.

• Response Bias: Answers do not reflect true feelings due to interviewer error, misrepresented answers, question wording, question order, or question type.

• Data-Entry Error: Mistakes when transferring data to a computer.

• Nonsampling Errors: All errors arising from data collection (sampling bias, nonresponse, response bias, data-entry error).

• Sampling Errors: Random variation due to using a sample instead of the whole population.

Observational Studies vs. Experiments

Types of Studies

• Observational Study: Researchers record variables without manipulating any factor.

  • Cross-sectional: Data collected at a single point in time.

  • Case-control: Retrospective comparison of subjects with a condition to those without.

  • Cohort: Prospective follow-up of a group over time.

• Experiment: Researchers deliberately manipulate one or more explanatory variables (factors) and observe the effect on a response variable.

Variables

• Confounding Variable: An explanatory variable whose effect cannot be separated from another factor.

• Lurking Variable: An unmeasured factor that influences both the explanatory and response variables.

Design of Experiments

• Treatment: Any specific combination of factor levels.

• Level: A particular value of a factor (e.g., 0 mg, 50 mg dosage).

• Experimental Unit: The individual or object receiving a treatment.

• Control Group: Baseline group receiving no treatment or a standard treatment.

• Placebo: An inert treatment that mimics the active treatment’s appearance, taste, or procedure.

• Blinding: Concealing treatment allocation (single-blind or double-blind).

Experimental Designs

• Completely Randomized Design: Each experimental unit is assigned randomly to a treatment.

• Matched-Pairs Design: Units are paired on similarity; each pair receives different treatments.

• Randomized Block Design: Units are grouped into homogeneous blocks; treatments are assigned randomly within each block.

Steps in Conducting an Experiment

1. Identify the problem.

2. Determine factors affecting the response.

3. Choose the number of experimental units.

4. Set the levels (treatments) for each factor.

5. Conduct the experiment, ensuring replication and randomization.

6. Analyze the data and test the claim.

Organizing Data

Organizing Qualitative Data

• Frequency Distribution: Table listing each category and its count.

• Relative Frequency: Proportion of observations in a category: $\text{Relative Frequency} = \frac{\text{Frequency}}{\sum \text{Frequencies}}$

• Bar Graph: Rectangles of equal width; height represents frequency or relative frequency.

• Pareto Chart: Bar graph ordered from highest to lowest frequency.

• Pie Chart: Circular chart where sector area is proportional to category frequency.

Organizing Quantitative Data

• Frequency Table (Grouped Data): Data are divided into classes (intervals).

• Class Width: Computed as: $\text{Class Width} = \frac{\text{Maximum} - \text{Minimum}}{\text{Number of Classes}}$

• Lower and Upper Class Limits: Smallest and largest values that can belong to a class.

• Class Midpoint: Average of the lower and upper limits: $\text{Midpoint} = \frac{\text{Lower Limit} + \text{Upper Limit}}{2}$

• Histogram: Bars with width equal to class width; bars touch, representing frequencies or relative frequencies.

• Stem-and-Leaf Plot: Stems are leading digits; leaves are trailing digits, sorted within each stem.

• Dot Plot: Dots placed above a number line; identical values stack vertically.

• Time Plot: Variable plotted against time to reveal trends.

Shapes of Distributions

• Uniform: Approximately equal frequency across classes (flat histogram).

• Normal (Bell-shaped): Symmetric peak with tapering tails.

• Skewed Right: Long tail to the right; mean > median.

• Skewed Left: Long tail to the left; mean < median.

Numerically Summarizing Data

Measures of Center

• Mean: Sum of all values divided by the number of observations. Sample mean: $\bar{x} = \frac{\sum_{i=1}^n x_i}{n}$ Population mean: $\mu = \frac{\sum_{i=1}^N x_i}{N}$

• Median: The middle observation after ordering the data. Odd $n$: position $\frac{n+1}{2}$. Even $n$: average of positions $\frac{n}{2}$ and $\frac{n}{2}+1$.

• Mode: Value(s) occurring most frequently; data may have no mode, one mode, or multiple modes.

• Midrange: Average of the minimum and maximum: $\text{Midrange} = \frac{\text{Maximum} + \text{Minimum}}{2}$

Comparing Mean and Median: Outliers pull the mean toward the tail; the median remains stable. In symmetric distributions mean ≈ median; in right-skewed mean > median; in left-skewed mean < median.

Measures of Dispersion (Spread)

• Range: Difference between maximum and minimum: $\text{Range} = \text{Maximum} - \text{Minimum}$

• Variance: Average squared deviation from the mean. Sample variance: $s^2 = \frac{\sum (x_i - \bar{x})^2}{n-1}$ Population variance: $\sigma^2 = \frac{\sum (x_i - \mu)^2}{N}$

• Standard Deviation: Square root of variance; same units as the original data. Sample standard deviation: $s = \sqrt{s^2}$ Population standard deviation: $\sigma = \sqrt{\sigma^2}$

Empirical Rule (Normal Approximation)

• About 68% within $\mu \pm \sigma$

• About 95% within $\mu \pm 2\sigma$

• About 99.7% within $\mu \pm 3\sigma$

Range Rule of Thumb

• Usual values lie within $\bar{x} \pm 2s$; values outside are considered unusual.

Chebyshev’s Inequality

• For any distribution and any $k > 1$: at least $1 - \frac{1}{k^2}$ of observations lie within $k$ standard deviations of the mean.

Approximate Measures from Grouped Data

• Approximate Mean (Grouped): Use class midpoints $x_j$ and frequencies $f_j$: $\bar{x}_{\text{approx}} = \frac{\sum f_j x_j}{\sum f_j}$

• Weighted Mean: When each observation has a weight $w_i$: $\bar{x}_w = \frac{\sum w_i x_i}{\sum w_i}$

• Approximate Standard Deviation (Grouped): Compute using midpoints and frequencies: $s_{\text{approx}} = \sqrt{\frac{\sum f_j (x_j - \bar{x}_{\text{approx}})^2}{\sum f_j - 1}}$

Measures of Position and Outliers

Z-Score

• Number of standard deviations a value $x$ is from the mean: Population: $z = \frac{x - \mu}{\sigma}$ Sample: $z = \frac{x - \bar{x}}{s}$

Percentiles and Quartiles

• Percentiles: The $k$th percentile $P_k$ is the value below which $k$% of observations fall.

• Quartiles: Divide data into four equal parts:

  • $Q_1$ (25th percentile)

  • $Q_2$ (median, 50th percentile)

  • $Q_3$ (75th percentile)

• Interquartile Range (IQR): Spread of the middle 50%: $\text{IQR} = Q_3 - Q_1$

Outlier Detection (Fence Method)

• Lower fence: $Q_1 - 1.5 \times \text{IQR}$

• Upper fence: $Q_3 + 1.5 \times \text{IQR}$

• Observations outside these fences are flagged as outliers.

Five-Number Summary and Boxplots

• Five-Number Summary: Minimum, $Q_1$, median ($Q_2$), $Q_3$, and maximum.

• Boxplot Construction:

  1. Draw a number line covering the data range.

  2. Mark the five-number summary; draw a box from $Q_1$ to $Q_3$ with a line at the median.

  3. Add “whiskers” from the box to the smallest and largest observations that lie within the fences.

  4. Plot outliers individually (often as asterisks).

• Interpreting Boxplots: Box width indicates IQR; whisker length indicates overall spread; outliers are highlighted; the relative positions of median and box reveal skewness.

Graphical Misrepresentation

Common Pitfalls and Avoiding Deception

• Inconsistent scales, truncated axes, omitted zero baseline, three-dimensional effects, excessive gridlines or decorations, and omission of data labels can mislead viewers.

• Use uniform intervals, label axes with units and source, start axes at zero unless a justified reason is provided, and keep design simple to let the data speak.

Summary of Core Concepts

• Statistics transforms raw data into meaningful information through careful definition of population, sampling, measurement, and analysis.

• Proper sampling and awareness of bias are essential for valid inference.

• Experimental design, including randomization, control, and blinding, isolates causal effects.

• Descriptive tools (frequency tables, graphs, measures of center and spread) reveal patterns; inferential tools extend findings to broader contexts.

Example: Suppose a researcher wants to estimate the average height of college students. They use stratified sampling to ensure representation from each year (freshman, sophomore, etc.), calculate the sample mean and standard deviation, and use a confidence interval to infer the population mean. Outliers are checked using the fence method, and results are visualized with a boxplot.