In the context of regression analysis, what is a residual, and what does it indicate when a residual is positive? (A residual is typically denoted as $e$, where $e=y-\hat{y}$, with $y$ as the observed value and $\hat{y}$ as the predicted value.)

• A. A residual is the predicted value minus the observed value; a positive residual means the predicted value is greater than the observed value. ($e=\hat{y}-y$)
• B. A residual is the average of observed values; a positive residual means the data has no outliers.
• C. A residual is the sum of all prediction errors; a positive residual means the model fits the data perfectly.
• D. A residual is the difference between the observed value and the predicted value; a positive residual means the observed value is greater than the predicted value. ($e=y-\hat{y}$)

Answer: (D)

Verified step by step guidance

1. Understand that in regression analysis, a residual represents the difference between the observed value (actual data point) and the predicted value (value estimated by the regression model). Mathematically, it is expressed as: \(\text{Residual} = \text{Observed value} - \text{Predicted value}\).
2. Recognize that the residual measures the error or deviation of the prediction from the actual data point, helping to assess how well the regression model fits the data.
3. Interpret the sign of the residual: if the residual is positive, it means the observed value is greater than the predicted value, indicating the model underestimated the actual data point.
4. Conversely, if the residual is negative, the observed value is less than the predicted value, meaning the model overestimated the data point.
5. Use residuals to diagnose the model's performance by analyzing their pattern and magnitude, which can reveal issues like non-linearity, heteroscedasticity, or outliers.