Endogeneity and Autocorrelation

Large N & Leeuwenhoek (70700173)

Yue Hu

Overview

Endogeneity
- Issue
- Diagnosis
- Adjustment
Autocorrelation
- Issue
- Diagnosis
- Adjustment

1 Endogeneity

1.1 Egg vs. Chicken

Substantively

Institutional Confidence ~ Evaluation of Redistribution Performance

Formally

Given Y_i = β₀ + β₁X_i + u_i,

Exogeneity ⇔ cov(X_i, u_i) = 0.
Endogeneity ⇔ cov(X_i, u_i) ≠ 0.

Explicitly, in \(X = \beta'_0 + \beta'_1Y + \nu,\)

\[\beta_1' = 0.\]

1.2 When does Exogeneity imply Causality

Compliance with Gauss-Markov Assumptions
- Exogeneity of X
Temporal Precedence
Significant Correlation

1.3 Cause of endogeneity: Yule-Simpson’s Paradox

1.4 What does the data look like under Simpson’s Paradox

1.5 Cause of endogeneity: Confounding factor

Selections on an observable confounder

Diagnosis

Hausman Procedure:
- Full (true) model is \(Y = \beta_0 + \beta_1X_1 + \beta_2X_2 + \epsilon\), and if one estimates \(Y = \beta_0 + \beta_1X_1 + \epsilon'\), X₂ is missed.
- To test if this leads to endogeneity, we regress: \(X_1 = \gamma X_2 + \nu\), and test: H₀: cov(ε’, ν) = 0.

Unobservable Confounders

Diagnosis

1.6 Correction for Observable confounders

Fully ~: Matching, weighting, even better controls would be helpful.
Partially ~: e.g., Wage ~ X_labor, but only X_employed is observable.
- If the omission has a fixed, known limit determining what observations get in to the sample → a Tobit model.
- If not → a Heckman model (two/three-stage model)

Stage I: E¹ ~ Z²γ + ε_e;

Stage II:
Wage|(E, X), ~ X_employed + ε_w

E[W|X,E]=Xβ +ρσ_{ε_w}λ (Zγ),

λ: Inverse Mills Ratio from stage I;
- \[\begin{align} m(x) :=& \frac{\bar{F}(x)}{f(x)},\\ \bar{F}(x) :=& \Pr[X>x] = \int_x^{+\infty} f(u)\, du \end{align}\]
ρ = corr(ε_e, ε_w)^*;
σ_{ε_w}: Standard deviation of ε_w.

1.7 Correction for Unobservable confounders

The famous (notorious, maybe) instrumental variables (IV) method

Statistically, we need a variable Z_i that simultaneously satisfies:
1. cov(X_i, Z_i) ≠ 0
2. cov(ε_i, Z_i) = 0

1.8 Examples

The effect of education on earnings

OV: Income
EV: Likelihood of going to school

IV candidates:
1. Proximity to college (Card 1995)
2. Month of birth (Angrist and Krueger 1991)

How does economic growth affect within-country conflicts?

OV: Incidence of civil war
EV: Economic growth per capita

(What’s the problem?)

IV : Rainfall growth (% change in rainfall from the previous year)

Any concern?

1.9 Secret to find a proper IV

Seeking something seemingly irrelevant, like geographic stuff
Seeking back to the time, such as old distribution, parental features
Seeking for diverse laws or policies
Making some assumptions, like weather is random

Ok, …IV is a myth, unless it is by design…

Expedient strategy: Multiple IVs

Dreaming that Z_weak1 + Z_weak2 + Z_weak3 +… = Z_super
- Picking out the one
- Different sources of endogeneity

1.10 Weak Instruments

An ideal Z_i simultaneously satisfies cov(X_i, Z_i) ≠ 0, and cov(u_i, Z_i) = 0. When cov(Y, Z) is small but not equal 0, then “weak” ~.

\[\beta_1 = \frac{cov(Y, Z) - cov(\epsilon, Z)}{cov(X, Z)}\]

Check: Sargan–Hansen test (Sargan’s J test, Statistics: \(\chi^2\))

Computing from residuals from instrumental variables regression by constructing a quadratic form based on the cross-product of the residuals and exogenous variables
\(H_0\): The over-identifying restrictions are valid

When cov(X, Z) is salient, not fatal.
When it’s small, the estimate will be biased

1.11 Assuming You’ve Got One

Two Stage Least Squares (2SLS)

Regress X on Z to get \(\boldsymbol{\hat X}\);
Regress Y on \(\boldsymbol{\hat X}\).
Solvable on one condition: Being identifiable, \[\hat{\boldsymbol{\beta}}_{IV} = (\boldsymbol{Z'X})^{-1}\boldsymbol{Z'Y}\]

Over-identified model

\[\begin{align} \hat{\boldsymbol{\beta}}_{2SLS} =& (\boldsymbol{X'P_ZX})^{-1}(\boldsymbol{X'P_ZY})\\ =& (\boldsymbol{X'Z(Z'Z)^{-1}Z' X})^{-1}\boldsymbol{X'Z(Z'Z)^{-1}Z'Y} \end{align}\]

where \[\boldsymbol{P_z} = Z(Z'Z)^{-1}Z'\] (a.k.a., the projection matrix), and \[\boldsymbol{\hat X} = \boldsymbol{P_ZX}\]

1.12 Consequences

\[\begin{align} cov(Y_i, Z_i) =& cov(\beta_0 + \beta_1X_i + \epsilon_i, Z_i)\\ =& cov(\beta_0, Z_i) + cov(\beta_1X_i, Z_i) + cov(\epsilon_i, Z_i)\\ =& \beta_1cov(X_i, Z_i)\\ \Rightarrow\beta_1 =& \frac{cov(Y_i, Z_i)}{\beta_1cov(X_i, Z_i)} = \frac{\sum(Y_i - \bar Y)(Z_i - \bar Z)}{\sum(X_i - \bar X)(Z_i - \bar Z)},\\ var(\hat\beta_1) =& \frac{\sigma^2}{n\sigma_X^2\rho_{XZ}^2} \geq \frac{\sigma^2}{n\sigma_X^2\rho_{XX}^2} = \frac{\sigma^2}{\sum(X_i - \bar X)^2} \end{align}\]

Trade-off of IV estimates

Unbiased estimates from IV, yet a bigger variance (ρ²_XX = 1 > ρ²_XZ)

2 Autocorrelation

2.1 Typology of Autocorrelation

E(ε₁, ε₂) = 0

Spatial autocorrelation

Heteroscedasticity (Multilevel)
Spillover (Spatial models)

Temporal autocorrelation

A quick taste of time series analysis 💀

2.2 Temporal autocorrelation: Shock

\[\begin{align} Y_t =& \beta_0 + \color{maroon}{\beta_1(Z_t + 1)} + \beta_2Z_{t-1} + \cdots + \epsilon\\ Y_{t + 1} =& \beta_0 + \beta_1Z_{t + 1} + \color{maroon}{\beta_2(Z_t + 1)} + \beta_3Z_{t-1} + \cdots + \epsilon\\ Y_{t + 2} =& \beta_0 + \beta_1Z_{t + 2} + \beta_2Z_{t + 1} + \color{maroon}{\beta_3(Z_t + 1)} + \cdots + \epsilon \end{align}\]

2.3 Temporal autocorrelation: Path

\[\begin{align} Y_t =& \beta_0 + \color{maroon}{\beta_1(Z_{t + 1})} + \cdots + \epsilon\\ Y_{t + 1} =& \beta_0 + \color{maroon}{\beta_1(Z_{t + 1} + 1)} \\ &+ \color{maroon}{\beta_2(Z_t + 1)} + \cdots + \epsilon. \end{align}\]

阿西莫夫的科幻小说《永恒的终结》讲述了一个围绕时间旅行和伦理抉择的扣人心弦的故事。小说的背景设定在一个庞大的超越时间的组织——“永恒时空”（The Eternals），它负责通过微调时间线来消除社会灾难和历史性错误，以达到最优的“终结状态”。故事的主人公是安德鲁·哈伦，一位才华横溢的时间技师，他深刻理解时空规律，却因一次任务结识了一位女子诺雅，从而陷入了爱情的漩涡。随着情节的展开，哈伦逐渐发现“永恒”的调控不仅限制了个人选择，也阻碍了人类文明的真正发展。他面临一个艰难的决定——是否破坏永恒，放手让人类面对未知的未来。小说通过复杂的时间逻辑和伦理议题探讨了自由意志、进步与牺牲的主题，被誉为科幻文学的经典之作。

2.4 Time Dependency and the adjustment

\[E(\epsilon_{t_1}, \epsilon_{t_2}|X_{t_1}, X_{t_2}) \neq 0.\]

When two variables both have, e.g., a positive trend, then they appear correlated, although they are actually not.

Adjustment

Fixed Effect: \(Y_t\sim X_t + Time_t\)
De-trend: \(Y_t\sim X_t\)
- \[\begin{align} \text{Run } Y_t =& \beta_0 + \beta_1Time + \epsilon_t;\\ X_t =& \gamma_0 + \gamma_1Time + \nu_t;\\ \text{Run } Y'_t =& Y_t - (\hat\beta_0 + \hat\beta_1Time);\\ X'_t =& X_t - (\hat\gamma_0 + \hat\gamma_1Time).\\ \Rightarrow\ Y'_t =& \delta X'_t + e_t. \end{align}\]

Spline
Smooth function
- t + t² + t³

2.5 When Time is More than Time

Affecting the relations between OV and EVs and among IVs…

(Real) Time series analysis

Random Walk & Unit Root: Let ρ∈[-1, 1] identify the direction the path will go (↓-1, ↑1, →0).
- \[\begin{align} Y_t =& \rho Y_{t - 1} + e_t\\ \epsilon_t =& \rho \epsilon_{t - 1} + e_t, e_t\sim iid(0, \sigma^2)\\ \Rightarrow\ \epsilon_{t + 1} =& \rho \epsilon_t + e_{t+1}\\ =& \rho(\rho \epsilon_{t - 1} + e_t) + e_{t + 1}\\ =& \rho(\rho (\rho \epsilon_{t - 2} + e_{t-1}) + e_t) + e_{t + 1} \end{align}\]
- |ρ| < 1 implies the effects of previous errors goes away eventually;
- |ρ| = 1, the pass never goes away, and the trend can end anywhere, a.k.a. one Unit Root.

2.6 Diagnosis: Durbin-Watson test

Testing only AR(1)

\[\begin{align} DW =& \frac{\sum^T_{t = 2}(\hat \epsilon_t - \hat \epsilon_{t - 1})^2}{\sum^T_{t = 2}\hat \epsilon_t^2}= \frac{\sum^T_{t = 2}(\rho\hat \epsilon_{t-1} + e_t - \hat \epsilon_{t - 1})^2}{\sum^T_{t = 2}\hat \epsilon_t^2}\\ =& \frac{\sum^T_{t = 2}[(\rho - 1)\hat \epsilon_{t - 1} + e_t]^2}{\frac{\sigma_e^2}{1 - \rho^2}}\\ =& \frac{\sum^T_{t = 2}\{[(1 - \rho)^2\hat \epsilon^2_{t - 1}] + 2(\rho - 1)\hat \epsilon_{t - 1}e_t + e_t^2\}}{\frac{\sigma_e^2}{1 - \rho^2}}\\ \approx& \frac{\sum^T_{t = 2}[(1 - \rho)^2\sigma_u^2 + \sigma_e^2]}{\frac{\sigma_e^2}{1 - \rho^2}} = (1 - \rho)^2 + (1 - \rho^2) = 2(1 - \rho) \end{align}\]

Since the range of ρ is [-1, 1], DW ∈ [0, 4].

In practice, DW test specifies a range of values, [d_l, d_u], and set the hypotheses:

Positive AR: H₀: ρ = 0; H₁: ρ > 0

Positive autocorrelation

DW < d_l, reject H₀;
d_l < DW < d_u, inconclusive;
DW > d_u, fail to reject H₀

Negative autocorrelation

(4 - DW) < d_l, reject H₀;
d_l < (4 - DW) < d_u, inconclusive;
(4 - DW) > d_u, fail to reject H₀

2.7 Regression accounting for temporal autocorrelation

Most common model: Autoregressive distributed lag model (ADL)

Mitchell, Sara B. 2017. “Time Series Analysis for the Social Sciences.” Contemporary Sociology 46(5).
Box-Steffensmeier, Janet M. 2014. Time Series Analysis for the Social Sciences. New York: Cambridge University Press.

2.8 Take-home point

Appendix

2.9 Special Case: Panel

Different from time series: T small, N large

General model: \(Y_{it} = \beta_0 + \beta_1X_{it} + \epsilon_{it}\)

Solution 1: Fixed effect

\[Y_{it} = \beta_0 + \beta_1X_{it} + a_i + \epsilon_{it}\]

Solution 2: Fixed differentiating

\[\begin{align} Y_{i2} - Y_{i1} =& (\beta_0 - \beta_0) + \beta_1(X_{i2} - X_{i1}),\\ &+ (\epsilon_{i2} - \epsilon_{i1}),\\ \Delta Y_i =& \beta_1\Delta X_i + \Delta \epsilon_i. \end{align}\]

Both are unbiased and consistent, nevertheless:

When N = 2, FE and FD are identical;

When N > 2, not necessarily;

When autocorrelation is weak in the errors, FE might be better with smaller variance.

Hint: Neither methods allows time invariant X.

2.10 Special Case: TSCS

General model:

\[Y_{it} = \beta_0 + \beta_1X_{it} + \beta_2X_t + \beta_3X_i + \epsilon_{it}\]

X_t: Only varying by time.

X_i: Only varying by unit.

2.11 TSCS Dealing with Time

Solution 1: Detrend

Detrending functions, e.g., the time smooth function/spline
Adding lag Y [AR(1)] (.magenta[Hint]: May cause bias).

Solution 2: Regime switching

\[\begin{align} Y_{it} =& \beta_0 + \beta_1X_{it} + \beta_2X_{it}dZ_t + \beta_3dZ_t + \epsilon_{it}\\ dZ_t =& 1, \text{if } t \leq t^*. \end{align}\]

Structural break at t^* in the X-Y relationship.

2.12 Treating time dependency as unit heterogeneity

Solution 1: LSDV, \(Y_{it} = \delta_td_{t} + \beta_1X_{it} + a_i + \epsilon_{it}\)

\(\delta_td_{t}\): Fixed effect for time

\(a_i\): Fixed effect for unit (unit-specific mean differences)

Cons:

Using up the d.f.
Aggravating multicollinearity
.magenta[Losing time invariant variables].

Solution 2: Within-between model or random effect model

Modeling time as a level

\[Y_{it} = \delta_td_{t} + \beta_1X_{it} + a_i + \epsilon_{it}\]

Instead of using binaries, modeling \(a_i\sim N(0, \tau^2)\)

[ASSUMPTION].red:

cov(a_i, X_it)= 0 (strong)
cov(a_i, u_it)= 0

2.13 Stationary

For every collection of time periods (X_t₁, X_t₂, X_t₃,… where t₁ < t₂ < t₃…), their distributions are the same as X_t₁ + h, X_t₂ + h, X_t₃ + … i.e., the .navy[same joint distribution], or saying the .navy[error terms are the same] for every period.

2.14 Covariance Stationary

Given:

t(X_t) is constant;
var(X_t) is constant;
cov(X_t,X_{t + n})does not rely on t, i.e., cov(X_t,X_{t + n}) = cov(X_{t + 1},X_{t + 2}).

Then, a moving average process that is covariance stationary is weakly dependent if cov(X_t,X_{t + n})→0 as n→∞.

When covariance stationary plus weakly dependent (e.g., \(Y_t = \rho Y_{t - 1} + X_t + e_t\)), the OLS is consistent.

2.15 Influence on the Coefficient

Let’s assume the mean of X is zero to simplify the maths

\[\begin{align} \hat\beta_1 =& \beta_1 + \frac{\sum(X_t - \bar X)}{\sum(X_t - \bar X)^2}\epsilon_t\\ =& \beta_1 + \frac{\sum X_t}{\sum X_t^2}\epsilon_t, \epsilon_t = \rho \epsilon_{t - 1} + e_t \end{align}\]

Only when ρ = 0 (or cov(X_t, X_{t + n}) = 0), the estimate is identical with the OLS estimate.

In other cases, β is biased.

2.16 Influence on the Variance

\[\begin{align} var(\epsilon_t|X) =& var(\rho \epsilon_{t - 1} + e_t|X),\\ =& var(\rho(\rho \epsilon_{t - 2} + e_{t - 1}) + e_t|X),\\ =& var(\rho(\rho (\rho \epsilon_{t - 3} + e_{t-2}) + e_{t-1}) + e_t|X),\\ =& var(\rho^3\epsilon_{t - 3} + \rho^2e_{t-2} + \rho e_{t - 1} + e_t |X),\\ =& \rho^6var(\epsilon_{t - 3}) + \rho^4\sigma_{e_{t-2}} + \rho^2\sigma_{e_{t-2}} + \sigma_e^2. \end{align}\]

Following this manner, when t increases,

\[\begin{align} var(\epsilon_t|X) =& \sigma_e^2 + \rho^2\sigma_e^2 + \rho^4\sigma^2 +\cdots,\\ =& \sigma_e^2\sum^T_{t = 1}\rho^{2(t - 1)} = \frac{\sigma_e^2}{1 - \rho^2}. \end{align}\]

When ρ > 0, the estimation of variance through OLS will be overestimated.

Stretch

Meditation