Multiple Linear Regression (MLR) Assumptions and Equations

Effect of smaller variance

• A smaller variance can sometimes overcompensate for omitted variable bias in a misspecified model.

• A smaller variance always leads to more accurate predictions.

• A smaller variance indicates a higher level of uncertainty in the data.

• A smaller variance has no effect on model specification.

Importance of MLR Assumptions

• MLR assumptions ensure the efficiency and unbiasedness of OLS estimators.

• MLR assumptions are only relevant for large sample sizes.

• MLR assumptions guarantee the accuracy of all statistical models.

• MLR assumptions are not necessary for regression analysis.

Multicollinearity

• It occurs when independent variables in a regression model are highly correlated, making it difficult to isolate the individual effect of each variable.

• It refers to the situation where dependent variables are correlated with each other.

• It is a method used to increase the sample size in regression analysis.

• It describes the relationship between the dependent variable and the independent variables in a linear regression model.

Heteroskedasticity Impact

• If heteroskedasticity is present, OLS is not the best estimator.

• Heteroskedasticity improves the efficiency of OLS estimators.

• Heteroskedasticity has no effect on OLS estimators.

• OLS is always the best estimator regardless of heteroskedasticity.

Error variance

• A measure of the variability of the error term in a regression model.

• A measure of the total variance in a dataset.

• A technique used to minimize errors in data collection.

• A statistical method for predicting future values.

Trade-off between bias and variance

• Bias will vanish as the sample size increases.

• Variance increases with the complexity of the model.

• Trade-off between bias and variance; bias will not vanish even in large samples.

• High bias leads to overfitting the model.

Estimated sampling variation of the estimated βj

• se(β̂j) = √Var(β̂j) = √σ̂² / [SSTj(1 - R²j)]

• se(β̂j) = σ̂² / [SSTj(1 + R²j)]

• se(β̂j) = √(σ̂² / SSTj)

• se(β̂j) = √(σ̂² * SSTj)

Omitted variable bias

• Occurs when a relevant variable is left out of a model, potentially leading to biased estimates.

• Occurs when all relevant variables are included in a model, ensuring accurate estimates.

• Occurs when a variable is included in a model but has no effect on the outcome.

• Occurs when the model is overfitted to the data, leading to poor generalization.

Variance Inflation Factor (VIF)

• A statistical measure that quantifies the severity of multicollinearity in an ordinary least squares regression analysis.

• A method to calculate the mean of a dataset.

• A technique used to determine the correlation between two variables.

• A tool for assessing the normality of residuals in regression analysis.

Gauss-Markov Theorem

• The Gauss-Markov Theorem states that the OLS estimator is the Best Linear Unbiased Estimator (BLUE) under certain conditions.

• The Gauss-Markov Theorem states that the OLS estimator is always biased.

• The Gauss-Markov Theorem states that the OLS estimator is the worst linear unbiased estimator.

• The Gauss-Markov Theorem states that the OLS estimator is only applicable in non-linear models.