Least-Squares Regression and The Normal Probability Distribution

Least-Squares Regression

Finding the Least-Squares Regression Line

The least-squares regression line is a statistical method used to model the relationship between two quantitative variables. It minimizes the sum of the squared differences between observed values and the values predicted by the line.

• Definition: The least-squares regression line is the line that best fits the data points on a scatterplot, minimizing the sum of the squared vertical distances (residuals) from each point to the line.

• Equation: The general form is $\hat{y} = b_0 + b_1x$, where $b_0$ is the y-intercept and $b_1$ is the slope.

• Example: For the data in Table 2:

  x	y
  1	(value not shown)
  3	(value not shown)
  3	(value not shown)
  6	(value not shown)
  7	(value not shown)

  The least-squares regression line is: $\hat{y} = -2.1676x + 18.6704$

• Interpretation: The slope ($-2.1676$) indicates the average change in $y$ for each unit increase in $x$. The intercept ($18.6704$) is the predicted value of $y$ when $x = 0$.

Additional info: The actual $y$ values are not shown in the table, but the regression equation is provided. The process involves calculating means, variances, and covariances of $x$ and $y$.

The Normal Probability Distribution

Introduction to the Normal Distribution

The normal distribution is a fundamental probability distribution in statistics, often used to model real-world phenomena that cluster around a mean.

• Definition: A normal distribution is a continuous probability distribution that is symmetric about its mean, showing that data near the mean are more frequent in occurrence than data far from the mean.

• Normal Curve: The graphical representation of a normal distribution is called the normal curve, which is bell-shaped and symmetric.

• Continuous Random Variable: A variable that can take any value within a given range and is said to be normally distributed if its histogram approximates the shape of a normal curve.

Properties of the Normal Curve

The normal curve has several important properties that make it useful for statistical analysis.

• Symmetry: The curve is symmetric about its mean $\mu$.

• Mean, Median, Mode: All are equal and located at the center of the distribution.

• Inflection Points: The points at $x = \mu - \sigma$ and $x = \mu + \sigma$ are where the curvature changes direction.

• Total Area: The area under the curve is 1, representing the total probability.

• Asymptotic: The curve approaches, but never touches, the horizontal axis as $x$ moves away from the mean in either direction.

Effect of Mean and Standard Deviation

Changing the mean ($\mu$) shifts the curve left or right, while changing the standard deviation ($\sigma$) affects the spread and height of the curve.

• Increasing $\mu$: Shifts the curve horizontally without changing its shape.

• Increasing $\sigma$: Makes the curve flatter and more spread out.

The Empirical Rule (68-95-99.7 Rule)

The empirical rule describes the proportion of data within certain standard deviations of the mean in a normal distribution.

• Approximately 68% of the data falls within $\mu \pm \sigma$.

• Approximately 95% falls within $\mu \pm 2\sigma$.

• Approximately 99.7% falls within $\mu \pm 3\sigma$.

Role of Area in the Normal Density Function

The area under the normal curve for a given interval represents the probability or proportion of observations within that interval.

• Probability Density Function (pdf): The normal pdf is given by: $f(x) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^2}$ where $\mu$ is the mean and $\sigma$ is the standard deviation.

• Interpretation: The area under the curve between two values $a$ and $b$ gives $P(a < X < b)$, the probability that $X$ falls in that interval.

• Example: If the area to the right of $x = 200$ is 0.2903, then 29.03% of the population has values greater than 200, and the probability that a randomly selected individual exceeds 200 is 0.2903.

Standardizing and the Standard Normal Distribution

Standard Normal Distribution

The standard normal distribution is a special case of the normal distribution with mean 0 and standard deviation 1.

• Z-score: The standardized value (z-score) is calculated as: $z = \frac{x - \mu}{\sigma}$ It represents the number of standard deviations $x$ is from the mean.

• Use: Z-scores allow comparison across different normal distributions and facilitate finding probabilities using standard normal tables.

• Example: An IQ score of 120 with $\mu = 100$ and $\sigma = 15$ has a z-score: $z = \frac{120 - 100}{15} = 1.33$ This means the score is 1.33 standard deviations above the mean.

Finding Areas Under the Normal Curve

Areas under the normal curve correspond to probabilities and can be found using z-tables or technology.

• To the Left of z: Use the standard normal table to find $P(Z < z)$.

• To the Right of z: Use the complement rule: $P(Z > z) = 1 - P(Z < z)$.

• Between Two Values: $P(a < Z < b) = P(Z < b) - P(Z < a)$.

• Example: For $z = 1.33$, $P(Z < 1.33) = 0.9082$, so $P(Z > 1.33) = 1 - 0.9082 = 0.0918$.

Applications: Heights Example

Suppose the heights of three-year-old females are normally distributed with mean 38.72 inches and standard deviation 3.17 inches.

• Proportion less than 35 inches:

  • Calculate z-score: $z = \frac{35 - 38.72}{3.17} = -1.17$

  • Look up $z = -1.17$ in the table: $P(Z < -1.17) = 0.1210$

  • Interpretation: 12.1% of three-year-old females are less than 35 inches tall.

• Proportion between 35 and 40 inches:

  • Find z-scores: $z_1 = -1.17$ for 35 inches, $z_2 = 0.40$ for 40 inches

  • Find areas: $P(Z < 0.40) = 0.6554$, $P(Z < -1.17) = 0.1210$

  • Area between: $0.6554 - 0.1210 = 0.5344$

  • Interpretation: About 53.4% of three-year-old females are between 35 and 40 inches tall.

Finding Values Corresponding to Given Areas (Percentiles)

To find the value of a normal random variable corresponding to a given percentile:

1. Find the z-score for the desired area (percentile) using the standard normal table.

2. Convert the z-score to the original scale: $x = \mu + z\sigma$

3. Example: 20th percentile for heights ($\mu = 38.72$, $\sigma = 3.17$):

   • z-score for 0.20 area: $z = -0.84$

   • $x = 38.72 + (-0.84)(3.17) = 36.1$ inches

Percentiles for SAT Scores Example

• Mean SAT math score: $\mu = 516$, $\sigma = 116$

• 5th percentile: $z = -1.645$, $x = 516 + (-1.645)(116) = 325$

• 95th percentile: $z = 1.645$, $x = 516 + (1.645)(116) = 707$

• Interpretation: Scores of 325 and 707 separate the bottom and top 5% from the middle 90% of test takers.

Probability of a Single Value

For any continuous random variable, the probability of observing any single exact value is zero: $P(X = a) = 0$.

• Probabilities are only meaningful for intervals, not for exact values.