wtx的博客

7 Unconfounded Treatment Assignment & Estimating Propensity Score

2026-11-04T15:20:24.000Z

Randomized Experiments and Observational Studies

Randomized Experiments

A randomized experiment is an assignment mechanism that

is probabilistic and
has a known functional form that is controlled by the researcher

And there are a special case of randomized experiment called classical randomized experiment whose assignment mechanism is

individualistic, and
unconfounded

Four important special cases:

Bernoulli trials
Complete randomized experiments
Stratified randomized experiments
Paired randomized experiments

Observational Studies

An observational study is an assignment mechanism if the functional form of the assignment mechanism is unknown.

Two categories: Regular Assignment Mechanism and Irregular Assignment Mechanism.

An assignment mechanism is regular if it is

individualistic
probabilistic
unconfounded

Introduce the Irregular Assignment Mechanism briefly here.

Assignment to treatment may differ for some units from the receipt of treatment, i.e., noncompliance.
We often assume that assignment to treatment itself is unconfounded, but allow receipt of treatment to be confounded.

Caution

Unconfoundedness Is Not Testable. There is no information in the data that can tell us that unconfoundedness does not hold, even in settings with a large number of units.

RAMs vs SREs

Common features:

Base inference on strata defined by covariates $X_{i}=x$ .
Stratum-level causal effects are identifiable & estimable.

Key differences

$e(j)=\frac{N_{t}(j)}{N}$ are known in SREs.
But unknown in a RAM.

RAM can be treated as SREs with unknown propensity score in each stratum. The textbook says:

The fact that we do not know a priori the probability of assignment to the treatment is irrelevant here: we can use the results for the analysis of completely randomized experiments by conditioning on the number of treated women and treated men. (Textbook p259)

▶

A look at the validity

Just see the finite sample expectation of $\bar{Y}^{\mathrm{obs}}_{t}(f)=\frac{1}{N_{t}(f)}\sum _{i:X_{i}=f}W_{i}Y_{i}(1)$ . Let

\mathcal{C}= \left( \boldsymbol{X},\boldsymbol{Y}(1),\boldsymbol{Y}(0),N_{t}(f),N_{t}(m) \right)

then the finite sample expectation

\begin{align} \mathbb{E}[\bar{Y}^{\mathrm{obs}}_{t}(f)\mid \mathcal{C}] & = \frac{1}{N_{t}(f)}\sum _{i:X_{i}=f}Y_{i}(1)\mathbb{E}[W_{i}\mid \mathcal{C}] \\ & = \frac{1}{N_{t}(f)}\sum _{i:X_{i}=f}Y_{i}(1) \frac{N_{t}(f)}{N(f)} \\ & = \frac{1}{N(f)}\sum _{i:X_{i}=f}Y_{i}(1) \end{align}

The fact that $P(W_{i}=1\mid \mathcal{C})=N_{t}(f)/N(f)$ is not obvious. The second part of the individualistic assignment is the core (refer here).

Since we don't know the propensity score, we want to estimate them. Note that in a RAM

e(x)=P(W_{i}\mid X_{i}=x)=\mathbb{E}[W_{i}\mid X_{i}=x]

so we can use the sample mean of $W_{i}\mid X_{i}=x$ , which is $\frac{N_{t}(x)}{N(x)}$ , to estimate it. This is guaranteed by LLN (the version that allows dependency).

Then here comes the problem. If $X_{i}$ can take many values, then in each stratum, the sample size would be rather small. So balancing score is introduced.

Balancing Score and Propensity Score

Concepts

Definition (Balancing Scores)
A balancing score $b(x)$ is a function of the covariates such that $W_{i}\perp X_{i}\mid b(X_{i})$ .

Lemma (Unconfoundedness Given a Balancing Score)
Suppose assignment to treatment is unconfounded. Then assignment is unconfounded given any balancing score:

W_{i}\perp (Y_{i}(1),Y_{i}(0))\mid b(X_{i})

The proof is as follows:

Let

\mathcal{C}=(Y_{i}(1),Y_{i}(0))

Then

\begin{align}P(W_{i}=1\mid \mathcal{C},b(X_{i})) & = \mathbb{E}[W_{i}\mid \mathcal{C},b(X_{i})] \\ & = \mathbb{E}\left[ {\color{blue}\mathbb{E}[W_{i}\mid \mathcal{C},b(X_{i}),X_{i}]} \mid \mathcal{C},b(X_{i})\right] \\ & = \mathbb{E}\left[ {\color{blue}\mathbb{E}[W_{i}\mid b(X_{i}),X_{i}]} \mid \mathcal{C},b(X_{i})\right]\quad \text{(Unconfounded)} \\ & = \mathbb{E}\left[ {\color{blue}\mathbb{E}[W_{i}\mid b(X_{i})]} \mid \mathcal{C},b(X_{i})\right]\quad \text{(Definition of PS)} \\ & = \mathbb{E}[W_{i}\mid b(X_{i})] \\ & = P(W_{i}=1\mid b(X_{i}))\end{align}

Lemma (Balancing Property of Propensity Score)
The propensity score is a balancing score.

On one hand,

P(W_{i}=1\mid X_{i},e(X_{i}))=P(W_{i}=1\mid X_{i})=e(X_{i})

On the other hand

\begin{align}P(W_{i}=1\mid e(X_{i})) & =\mathbb{E}[W_{i}\mid e(X_{i})] \\ & = \mathbb{E}\big[\mathbb{E}[W_{i}\mid e(X_{i}),X_{i}]\mid e(X_{i})\big] \\ & = \mathbb{E}\big[\mathbb{E}[W_{i}\mid X_{i}]\mid e(X_{i})\big] \\ & = \mathbb{E}[e(X_{i})\mid e(X_{i})] \\ & = e(X_{i})\end{align}

P(W_{i}=1\mid X_{i},e(X_{i}))=P(W_{i}=1\mid e(X_{i}))

thus

W_{i}\perp X_{i}\mid e(X_{i})

Back to the main topic. We want to reduce the number of strata, so $b(X_{i})$ should take less possible values, i.e. the coarser the better.

How to define coarseness? We know a mapping usually loses information. So if $b'=f \circ b$ , where $f$ is not injection (单射), then we say $b'$ is coarser.

What is the coarsest balancing score? The propensity score has this good property.

Lemma (Coarseness of Balancing Scores)
The propensity score is the coarsest balancing score.

Suppose $e(X_{i})$ is not the coarsest. Then there exist $x$ and $x'$ , such that

e(x)\ne e(x')\ \text{but}\ b(x)=b(x')

But we have

\begin{align}e(x) & = P(W_{i}=1\mid X_{i}=x) \\ & = P(W_{i}=1\mid X_{i}=x,b(X_{i})=b(x)) \\ & = P(W_{i}=1\mid b(X_{i})=b(x)) \\ & = P(W_{i}=1\mid b(X_{i})=b(x')) \\ & = P(W_{i}=1\mid X_{i}=x',b(X_{i})=b(x')) \\ & = P(W_{i}=1\mid X_{i}=x') \\ & = e(x')\end{align}

It is contradictory!

Methods for Estimation in RAMs

Imputation

The core of the model-based method and the regression method is to predict the counter-factual potential outcomes and use the predicted results to estimate the ATE.

Model-Based Method

Regression Method

Summary of Imputation Methods

The two methods are not recommended in RAMs because of the systematical difference of the distribution of the covariates between the treated and the controlled groups. The predictions are not very trustable. Consider the setting: a model is trained on the data of males but is used to predict females.

Balance the Covariates

Weighting Estimators

Can show that

\mathbb{E}\left[ \frac{Y_{i}^{\mathrm{obs}}W_{i}}{e(X_{i})} \right]=\mathbb{E}_{sp}[Y_{i}(1)]

▶

The proof

\begin{align} \mathbb{E}\left[ \frac{Y_{i}^{\mathrm{obs}}W_{i}}{e(X_{i})} \right] & = \mathbb{E}\left[ \mathbb{E}\left[ \frac{Y_{i}^{\mathrm{obs}}W_{i}}{e(X_{i})} \mid X_{i}\right] \right] \\ \\ & = \mathbb{E}\left[ \mathbb{E}\left[ \frac{Y_{i}(1)W_{i}}{e(X_{i})} \mid X_{i}\right] \right]\\ & = \mathbb{E}\left[ \frac{1}{e(X_{i})}\mathbb{E}[Y_{i}(1)\mid X_{i}]\cdot\mathbb{E}[W_{i}\mid X_{i}] \right] \\ & = \mathbb{E}\big[\mathbb{E}[Y_{i}(1)\mid X_{i}]\big] \\ & = \mathbb{E}[Y_{i}(1)] \end{align}

and

\mathbb{E}\left[ \frac{Y_{i}^{\mathrm{obs}}(1-W_{i})}{1-e(X_{i})} \right]=\mathbb{E}_{sp}[Y_{i}(0)]

So we can use

\hat{\tau}^{\mathrm{ht}}=\frac{1}{N}\sum_{i=1}^{N} \frac{W_{i}Y_{i}^{\mathrm{obs}}}{e(X_{i})}-\frac{1}{N}\sum_{i=1}^{N} \frac{(1-W_{i})Y_{i}^{\mathrm{obs}}}{1-e(X_{i})}

to estimate the ATE.

But in practice we don't know the propensity score, so we have to estimate it. Later we will talk about how to estimate it.

Blocking Estimators

A more robust approach involving the propensity score is to coarsen it through blocking (i.e., subclassification). In this third approach, the sample is partitioned into subclasses, based on the value of the estimated propensity score. Within each subclass, the data can be analyzed as if they arose from a completely randomized experiment. (Textbook p274)

The estimator comes with issues of bias.

In particular, we recommend reducing the bias and increasing the precision further by using covariance adjustment within the subclasses. (Textbook p275)

Matching Estimators

... it relies on finding direct comparisons, that is, matches, for each unit. For a given treated unit with a particular set of values for the covariates, one looks for a control unit with as similar a set of covariates as possible. (Textbook p275)

Estimating Propensity Score

Does the estimation accuracy matter?

Be clear about the goal.

用估计的 PS，估计量的统计性质不会被破坏吗？比如无偏性啥的。

Three comments:

▶

Notes on class

只看满足分配假设的观察性研究
观察性研究中， $N(f)$ 、 $N_{t}(f)$ 等有随机性，或者说未知，propensity score 无法计算，需要估计
$X$ 取值很多，或者是个向量，分层会分得很细，样本量少，propensity score 估计不准确，需要 bs
balancing score
- given bs still unconfounded
- why named like this?
- the propensity score is a bs
- ps is the coarsest. 映射会丢失信息，故若能表示为某 bs 的映射，即为更粗
观察性研究
- 贝叶斯方法，不推荐，对模型假设敏感，若 $X$ 取值多，往往不准
- 回归法，不推荐
- Weighting estimators, unbiased
- Blocking estimators
- matching estimators
估计 PS

线性回归分析-HW7

2026-11-04T11:38:02.000Z

HW.

Prob1 KNNL5.4 KNNL5.12

5.4

We have

Y=\begin{bmatrix}7.8 & 9.0 & 10.2 & 11.0 & 11.7\end{bmatrix}^T\\X=\begin{bmatrix}1 & 1 & 1 & 1 & 1 \\8 & 4 & 0 & -4 & -8\end{bmatrix}^T

therefore

Y^TY=503.77\\X^TX=\begin{bmatrix}5 & 0 \\0 & 160\end{bmatrix}\\X^TY=\begin{bmatrix}49.7 \\-39.2\end{bmatrix}

5.12

(X^TX)^{-1}=\begin{bmatrix}\frac{1}{5} & 0 \\ 0 & \frac{1}{160} \end{bmatrix}

Prob2 KNNL 5.23

2-a

Estimated regression coefficients

b=(X^TX)^{-1}X^TY=\begin{bmatrix}9.94 \\-0.245\end{bmatrix}

Vector of residuals

e=Y-Xb=\begin{bmatrix}-0.18 & 0.04 & 0.26 & 0.08 & -0.2\end{bmatrix}^T

SSR

SSR=\|Xb-\bar{Y}\|^{2}=9.604

SSE

SSE=\|e||^{2}=0.148

Estimated variance-covariance matrix of $b$ :

\widehat{\mathrm{Cov}}(b)=MSE\cdot(X^TX)^{-1}=\begin{bmatrix}0.00986667 & 0 \\0 &0.00030833\end{bmatrix}

Point estimate of $\mathbb{E}[Y_{h}]$ when $X_{h}=-6$ :

\hat{Y}_{h}=\begin{bmatrix}1 & -6\end{bmatrix}b=11.41

Estimated variance of $\hat{Y}_{h}$ when $X_{h}=-6$ :

\mathbb{V}[\hat{Y}_{h}]=\sigma^{2}\cdot x^T(X^TX)^{-1}X^T\cdot X(X^TX)^{-1}x=\sigma^{2}\cdot x^T(X^TX)^{-1}x\\\hat{\mathbb{V}}[\hat{Y}_{h}]=MSE\cdot x^T(X^TX)^{-1}x=0.02096

2-b

$X^TX$ is diagonal.

2-c

H=X(X^TX)^{-1}X^T=\begin{bmatrix}0.6 & 0.4 & 0.2 & 0.0 & -0.2 \\0.4 & 0.3 & 0.2 & 0.1 & 0.0 \\0.2 & 0.2 & 0.2 & 0.2 & 0.2 \\0.0 & 0.1 & 0.2 & 0.3 & 0.4 \\-0.2 & 0.0 & 0.2 & 0.4 & 0.6\end{bmatrix}

and $\mathrm{rank}(H)=\mathrm{rank}(X)=2$ .

Verify $H$ is idempotent:

2-d

\mathrm{Cov}(e)=\sigma^{2}(I-H)

therefore

s^{2}(e)=MSE\cdot(I-H)=\begin{bmatrix}0.019733 & -0.019733 & -0.009867 & 0.000000 & 0.009867 \\ -0.019733 & 0.034533 & -0.009867 & -0.004933 & 0.000000 \\ -0.009867 & -0.009867 & 0.039467 & -0.009867 & -0.009867 \\ 0.000000 & -0.004933 & -0.009867 & 0.034533 & -0.019733 \\ 0.009867 & 0.000000 & -0.009867 & -0.019733 & 0.019733 \\\end{bmatrix}

Bonus Question

Let

X=\begin{bmatrix}X_{1} & \dots & X_{n}\end{bmatrix}^T,\, a=\begin{bmatrix}a_{1} & \dots & a_{n}\end{bmatrix}^T

Then

Y=a^TX,\ \mathbb{E}[X]=0,\ \mathrm{Var}(X)=\begin{bmatrix}1 & p & \dots & p \\p & 1 & \dots & p \\\vdots & \vdots & \ddots & \vdots\\p & p & \dots & 1\end{bmatrix}

\mathbb{E}[Y]=a^T\mathbb{E}[X]=0\\\mathbb{V}[Y]=a^T\mathrm{Var}[X]a=a^T((1-p)I+p\boldsymbol{1}\boldsymbol{1}^T)a=(1-p)\sum_{i=1}^{n} a_{i}^{2}+p\left( \sum_{i=1}^{n} a_{i} \right)^{2}

Let $S_{1}=\sum a_{i}^{2},S_{2}=\left( \sum a_{i} \right)^{2}$ . With Cauchy-Schwarz Inequality we have $nS_{1}\geq S_{2}$ . $p$ have to ensure that $\mathbb{V}[Y]\geq 0,\ \forall a\in \mathbb{R}^{n}$ .

When $S_{1}\leq S_{2}$ , $p$ have to satisfy

p\geq \frac{S_{1}}{S_{1}-S_{2}}=\frac{1}{1-S_{2}/S_{1}}\geq \frac{1}{1-n}

When $S_{1}>S_{2}$ , $p$ have to satisfy

p\leq \frac{S_{1}}{S_{1}-S_{2}}

which is guaranteed because $p\leq 1$ and $\frac{S_{1}}{S_{1}-S_{2}}> 1$ .

So in all, the minimum of $p$ is $\frac{1}{1-n}$ .

Think a bit deeper about this conclusion: if we want to impose a shared correlation on $n$ standardized r.v.s, then the imposition cannot be too hard. Take $n=3$ as an example. If we impose a shared correlation of $-1$ . Intuitively thinking, $X_{1}$ goes inversely against $X_{2}$ , and $X_{2}$ goes inversely against $X_{3}$ , then $X_{3}$ should goes consistently with $X_{1}$ . So $X_{1}$ and $X_{3}$ cannot be negatively correlated.

7 Matrix Approach to Linear Regression & Multiple Linear Regression

2026-11-03T12:18:11.000Z

Slides.

Multiple Linear Regression

Set Up

The model

Y=X\beta+\varepsilon

where $\varepsilon\sim N(\boldsymbol{0},\sigma^{2}I_{n})\implies Y\sim N(X\beta,\sigma^{2}I_{n})$ . Here we see $X$ as a fixed matrix or we condition on $X$ (the whole matrix).

Suppose $X_{n\times p}$ has full column rank.

OLS estimator for $\beta$ is

b=(X^TX)^{-1}X^TY

Projection matrix

H=X(X^TX)^{-1}X^T

Fitted value

\hat{Y}=HY=Xb

Residual

e=Y-\hat{Y}=(I-H)Y

Here are some properties:

$e^T\hat{Y}=0$
$\mathrm{Tr}(H)=p$ .

Estimation of $\sigma^{2}$

MSE=\frac{SSE}{n-p}=\frac{e^Te}{n-p}

$MSE$ is valid for estimating $\sigma^{2}$ because

\mathbb{E}[MSE]=\frac{1}{n-p}\mathbb{E}[Y^T(I-H)Y]=\sigma^{2}

Let $X$ be a random vector with $\mathbb{E}X=\mu,\mathrm{Cov}(X)=\Sigma$ , then

\mathbb{E}[X^TAX]=\mathrm{Tr}(A\Sigma)+\mu^TA\mu

The one-dimension version:

\mathbb{E}[aX^{2}]=a\mathbb{V}[X]+a\mathbb{E}^{2}[X]

▶

The proof (the trace trick)

Let $\mathrm{Cov}(X)=\Sigma, \mathbb{E}[X]=\mu$

\begin{align} \mathbb{E}[X^TAX] & = \mathrm{Tr}[(\mathbb{E}[X^TAX])] \\ & = \mathbb{E}[\mathrm{Tr}(X^TAX)] \\ & = \mathbb{E}[\mathrm{Tr}(AXX^T)] \\ & = \mathrm{Tr}[A\mathbb{E}(XX^T)] \\ & = \mathrm{Tr}[A(\Sigma+\mu \mu^T)] \\ & = \mathrm{Tr}(A\Sigma)+\mu^TA\mu \end{align}

Sampling Distribution of $b$

Recall that

Y\sim N(X\beta,\sigma^{2}I),\ b=(X^TX)^{-1}X^TY

b\sim N\left(\beta,\sigma^{2}(X^TX)^{-1}\right)

Replace $\sigma^{2}$ with $s^{2}=MSE$ to get the estimated covariance matrix.

Gauss-Markov Theorem

Under Gauss-Markov assumptions:

Linearity: the parameters are themselves linear
Randomness: the data must have been randomly sampled from the population
Non-Collinearity: the regressors aren’t perfectly correlated with each other
Exogeneity: the regressors aren't correlated with the error term, $\mathbb{E}[\varepsilon\mid X]=0$ . （给定向量还是给定整个设计矩阵？）
Homoscedasticity: no matter what the values of the regressors might be, the error of the variance is constant.
the OLS estimate for regression coefficients gives you the best linear unbiased estimate (BLUE) possible.

The Leverage

On one hand

H^T=H=H^{2}\implies h_{ii}=e_{i}^THe_{i}=e_{i}^TH^THe_{i}\geq 0

On the other hand,

e_{i}^THe_{i}=h_{ii}=e_{i}^TH^{2}e_{i}=h_{ii}^{2}+\sum _{i\ne j}h_{ij}^{2}\implies h_{ii}\geq h_{ii}^{2}\implies h_{ii}\leq 1

In conclusion we have

h_{ii}\in [0,1]

有时间研究一下取等条件

Don't forget that

h_{ii}=\frac{ \partial \hat{Y}_{i} }{ \partial Y_{i} } = \frac{ \partial \sum_{j=1}^{n} h_{ij}Y_{j} }{ \partial Y_{i} }

Recall that here we say the leverage describes the distance of $X_{i}$ from the center $\bar{X}$ . Now we extend this concept to higher dimension.

The overall principle is to adjust the spread of each direction.

Let $(1, x_{i}^T)$ be the $i$ -th row of $X$ and let $\bar{x}=\frac{1}{n}\sum x_{i}$ . Here we define the distance of $x_{i}$ from the center $\bar{x}$ :

D(x_{i},\bar{x})=\sqrt{(x_{i}-\bar{x})^T\Sigma^{-1}(x-\bar{x})}

where $\Sigma$ is the sample covariance matrix. Can show that

h_{ii}=\frac1n+\frac{D^2(x_{i},\bar{x})}{n-1}

▶

The proof

This distance measure is known as Mahalanobis Distance.

Let $X=[\boldsymbol{1}\ Z]$ , and $Z_{c}=Z-\frac{1}{n}\boldsymbol{1}\boldsymbol{1}^TZ=(I-J)Z$ (centralize it). Then

\Sigma=\frac{1}{n-1}Z_{c}^TZ_{c}=\frac{1}{n-1}Z^T(I-J)Z

We hope to find the relationship between the diagonal elements of

Z\Sigma^{-1}Z^T\ \text{and}\ H

Expand $H$ as

[1\ Z]\left( [1\ Z]^T[1\ Z] \right) ^{-1}[1\ Z]^T= [1\ Z] \begin{bmatrix} n & \boldsymbol{1}^TZ \\ Z^T\boldsymbol{1} & Z^TZ \end{bmatrix}^{-1} [1\ Z]^T

Recall the knowledge of Shur complement. Shur complement of $n$ is

Z^TZ-(Z^T\boldsymbol{1})n^{-1}(\boldsymbol{1}^TZ)=Z^T(I-J)Z=:S

it is reversible iff $\boldsymbol{1}\not\in \mathcal{R}(Z)$ , which is guaranteed. So

\begin{bmatrix} n & \boldsymbol{1}^TZ \\ Z^T\boldsymbol{1} & Z^TZ \end{bmatrix}^{-1} =\begin{bmatrix} \frac1n+\bar z^TS^{-1}\bar z&-\bar z^TS^{-1}\\ -S^{-1}\bar z&S^{-1} \end{bmatrix}

therefore

h_{ii}= \begin{bmatrix} 1 & z_{i}^T \end{bmatrix} \begin{bmatrix} \frac1n+\bar z^TS^{-1}\bar z&-\bar z^TS^{-1}\\ -S^{-1}\bar z&S^{-1} \end{bmatrix} \begin{bmatrix} 1 \\ z_{i} \end{bmatrix} = \frac{1}{n}+(z_{i}-\bar{z})^TS^{-1}(z_{i}-\bar{z})

Note that $S^{-1}=\Sigma^{-1}/(n-1)$ , so

h_{ii}=\frac{1}{n}+\frac{D^{2}(z_{i},\bar{z})}{n-1}

Covariance Matrix of $e$

\mathrm{Cov}(e)=\mathrm{Cov}((I-H)Y)=\sigma^{2}(I-H)

\begin{align}\mathrm{Var}(e_{i}) & =\sigma^{2}(1-h_{ii}) \\\mathrm{Cov}(e_{i},e_{j}) & =-\sigma^{2}h_{ij}\end{align}

Think a bit deeper: $h_{ii}$ large $\implies$ $\mathrm{Var}(e_{i})$ closer to 0 $\implies$ fitting is more stable.

Expected Mean Squares and ANOVA F-Test

Already showed that

\mathbb{E}[MSE]=\sigma^{2}

Let's see $MSR=\frac{\|(H-J)Y\|^{2}}{p-1}$ .

\begin{align}\mathbb{E}[MSR] & = \frac{1}{p-1}\mathbb{E}[Y^T(H-J)Y] \\ & =\frac{1}{p-1}(\mathrm{Tr}((H-J)\sigma^{2})+(X\beta)^T(H-J)(X\beta)) \\ & = \sigma^{2}+\frac{1}{p-1}(X\beta)^T(H-J)(X\beta)\end{align}

Easy to see that

(X\beta)^T(H-J)(X\beta)=0 \iff \beta_{1}=\beta_{2}=\dots=\beta_{p-1}=0

▶

The proof

Note that $H-J$ is a projection matrix. So

(X\beta)^T(H-J)(X\beta)=0 \iff (H-J)X\beta=0

Because $X\beta\in \mathcal{R}(H)$ and $\mathcal{R}(H)=\mathcal{R}(H-J)\oplus \mathcal{R}(J)$ , we have

(H-J)X\beta=0 \iff X\beta\in \mathcal{R}(J)=\mathrm{span}(\boldsymbol{1}) \iff \beta_{2}=\dots=\beta_{p-1}=0

This can be used to conduct the F-test:

$H_{0}:$ $\beta_{1}=\beta_{2}=\dots=\beta_{p-1}=0$ .
$H_{1}:$ $\beta_{k}\ne 0$ for at least one $k$ in $1,2,\dots,p-1$ .

The F-statistic is

F=\frac{MSR}{MSE}\sim F_{p-1,n-p}

Reject $H_{0}$ if $F$ is large enough.

$R^{2}$ and Adjusted $R^{2}$

R^{2}=1-\frac{SSE}{SST}

But adding regressors can always decrease $SSE$ . Penalty on adding regressors needed.

R_{a}^{2}=1-\frac{MSE}{MST}=1-\frac{n-1}{\color{red}n-p}\cdot\frac{SSE}{SST}

线性回归分析-HW6

2026-11-02T16:32:58.000Z

Prob-1

残差幅度随 $X$ 增大而减小，提示异方差；残差符号呈“正、负、正”的变化，也提示均值关系可能弯曲。适当的变量变换可能改善这些问题，具体变换需结合原始数据选择。

Prob-2

2-a

使用完整的 CH03PR15 数据，最小二乘回归结果为

\boxed{\widehat Y=2.575333-0.324000X.}

SSE=2.924653333,\qquad R^2=0.811577.

2-b

设五个时间点的总体均值为 $\mu_j$ 。

H_0:\mu_j=\beta_0+\beta_1x_j\quad(j=1,\ldots,5);

$H_1$ ：五个总体均值不全在同一直线上，即线性模型存在失拟。

来源	SS	df	MS
失拟	2.767253333	3	0.922417778
纯误差	0.157400000	10	0.015740000
残差合计	2.924653333	13	0.224973333

F^*=\frac{MS_{LOF}}{MS_{PE}}=\frac{0.922417778}{0.015740000}=58.603417.

在 $\alpha=0.025$ 下，拒绝规则为

F^*>F_{0.975}(3,10)=4.825621.

由于 $58.603417>4.825621$ ，且 $p=1.1945\times10^{-6}<0.025$ ，拒绝 $H_0$ ，线性模型存在显著失拟。

2-c

不能。该检验只能判断直线模型是否失拟，不能确定应采用哪一种回归函数；具体形式需结合散点图、科学背景和残差诊断选择。

Prob-3

3-a

浓度随时间呈指数衰减趋势，且浓度越高，组内波动越大。可尝试 $Y'=\log_{10}Y$ ，以改善线性关系并稳定方差。

3-b

几何均值为 $g=0.476297409$ 。按公式 (3.36) 标准化：

W_i(\lambda)=\begin{cases}\displaystyle\frac{Y_i^\lambda-1}{\lambda g^{\lambda-1}},&\lambda\ne0,\\[6pt]g\ln Y_i,&\lambda=0.\end{cases}

分别拟合 $W_i(\lambda)$ 对 $X_i$ 的直线，得到：

$\lambda$	SSE
-0.2	0.123530471
-0.1	0.065050668
0.0	0.038973035
0.1	0.043960618
0.2	0.081317930

$\lambda=0$ 的 SSE 最小，建议采用对数变换。

3-c

\boxed{\widehat{\log_{10}Y}=0.654879760-0.195400308X.}

SSE=0.032402343,\qquad MSE=0.002492488,\qquad R^2=0.992978.

3-d

各观测点接近回归直线，没有明显弯曲趋势，拟合良好。

3-e

令 $Z_i=\log_{10}Y_i$ ，残差为 $e_i=Z_i-\widehat Z_i$ 。

$i$	$X_i$	$Y_i$	$\widehat Z_i$	$e_i$
1	9	0.07	-1.103723	-0.051179
2	9	0.09	-1.103723	0.057966
3	9	0.08	-1.103723	0.006813
4	7	0.16	-0.712922	-0.082958
5	7	0.17	-0.712922	-0.056629
6	7	0.21	-0.712922	0.035142
7	5	0.49	-0.322122	0.012318
8	5	0.58	-0.322122	0.085550
9	5	0.53	-0.322122	0.046398
10	3	1.22	0.068679	0.017681
11	3	1.15	0.068679	-0.007981
12	3	1.07	0.068679	-0.039295
13	1	2.84	0.459479	-0.006161
14	1	2.57	0.459479	-0.049546
15	1	3.10	0.459479	0.031882

残差在零附近散布，没有明显漏斗形或曲线趋势；正态概率图中的点大致沿直线排列。线性、等方差及近似正态假设基本合理。

3-f

原单位下的反变换拟合曲线为

\boxed{\widetilde Y(X)=10^{0.654879760-0.195400308X}=4.517309(0.637675)^X.}

该曲线估计条件中位数；若对数误差服从正态分布，条件均值的修正估计为

\widehat{E(Y\mid X)}=1.006629\times10^{0.654879760-0.195400308X}.

Prob-4

4-a

速度随浓度增加而上升，增速逐渐减小并趋于平台，符合 Michaelis-Menten 饱和曲线的形状。

4-b

令 $U=1/C$ 、 $W=1/V$ 。

两者大致呈正线性关系。但 $C=0.02$ 的两点波动明显较大，提示异方差；这两点也远离其余解释变量取值，具有高杠杆。

4-c

分布不同。倒数变换颠倒取值顺序，并拉大低浓度值之间的间距。

$C$	$1/C$
0.02	50.000000
0.06	16.666667
0.11	9.090909
0.22	4.545455
0.56	1.785714
1.10	0.909091

$C=0.02$ 的两个观测变为孤立的 $1/C=50$ ，其杠杆值均为 $0.459862$ ，Cook 距离分别为 $4.089$ 和 $3.192$ ，对拟合有很强的影响。

4-d

最小二乘回归直线为

\boxed{\widehat{\frac1V}=0.005107182+0.000247221\frac1C.}

SSE=3.580648427\times10^{-5},\qquad R^2=0.855695.

令 $\widehat W_i=\widehat{1/V_i}$ 、 $e_i=1/V_i-\widehat W_i$ 。

$i$	$C_i$	$V_i$	$\widehat W_i$	$e_i$
1	0.02	76	0.017468230	-0.004310335
2	0.02	47	0.017468230	0.003808366
3	0.06	97	0.009227531	0.001081747
4	0.06	107	0.009227531	0.000118263
5	0.11	123	0.007354645	0.000775436
6	0.11	139	0.007354645	-0.000160400
7	0.22	159	0.006230913	0.000058395
8	0.22	152	0.006230913	0.000348034
9	0.56	191	0.005548648	-0.000313046
10	0.56	201	0.005548648	-0.000573523
11	1.10	207	0.005331928	-0.000501010
12	1.10	200	0.005331928	-0.000331928

残差波动随拟合值增大而明显增大，提示异方差；残差均值也有一定系统性趋势，模型假设存在问题。

4-e

反变换后的拟合曲线为

\boxed{\widehat V(C)=\frac1{0.005107181642+0.000247220961/C}=\frac{195.802709C}{0.048406534+C}.}

故 $\widehat\theta_1=195.802709$ 、 $\widehat\theta_2=0.048406534$ 。

$C$	观测速度均值	$\widehat V$
0.02	61.5	57.246786
0.06	102.0	108.371351
0.11	131.0	135.968495
0.22	155.5	160.490117
0.56	196.0	180.224095
1.10	203.5	187.549420

曲线捕捉了饱和趋势，但明显低估高浓度下的速度。结合异方差和低浓度点的高影响性，该倒数回归拟合不够理想。

CI-HW5

2026-11-01T20:05:10.000Z

Problem 1 (SRE vs CRE based on Neyman’s estimator)

The estimator for SRE is

\hat{\tau}^{\mathrm{strat}}=q\hat{\tau}(f)+(1-q)\hat{\tau}(m)

Its variance is

\begin{align}\mathbb{V}_{sp}[\hat{\tau}^{\mathrm{strat}}] & = \mathbb{V}_{sp}[q\cdot+(1-q)\cdot] \\ & = q^{2}\mathbb{V}_{sp}[\cdot]+(1-q)^{2}\mathbb{V}_{sp}[\cdot] \\ & = \frac{q}{N}\left( \frac{\sigma_{t}^{2}(f)}{p}+\frac{\sigma_{c}^{2}(f)}{1-p} \right) + \frac{1-q}{N}\left( \frac{\sigma^{2}_{t}(m)}{p}+ \frac{\sigma^{2}_{c}(m)}{1-p}\right)\end{align}

The estimator for CRE is

\hat{\tau}^{\mathrm{dif}}=\bar{Y}_{t}^{\mathrm{obs}}-\bar{Y}_{c}^{\mathrm{obs}}

Its variance is

\begin{align}\mathbb{V}_{sp}[\hat{\tau}^{\mathrm{dif}}] & = \frac{\sigma^{2}_{t}}{N_{t}}+\frac{\sigma^{2}_{c}}{N_{c}} \\ & = \frac{q\sigma^{2}_{t}(f)+(1-q)\sigma_{t}^{2}(m)+q(1-q)[\mu_{t}(f)-\mu_{t}(m)]^{2}}{N_{t}}+\frac{\dots}{N_{c}} \\\end{align}

\mathbb{V}_{sp}[\hat{\tau}^{\mathrm{dif}}]-\mathbb{V}[\hat{\tau}^{\mathrm{strat}}]=\frac{q(1-q)[\mu_{t}(f)-\mu_{t}(m)]^{2}}{Np}+\frac{q(1-q)[\mu_{c}(f)-\mu_{c}(m)]^{2}}{N(1-p)}\geq 0

Problem 2 (PRE vs SRE based on Neyman’s estimator)

2-1

For SRE, the estimator is

\hat{\tau}^{\mathrm{S}}=\frac{6}{16}\hat{\tau}_{X>16}+\frac{10}{16}\hat{\tau}_{X\leq 16}=7.8875

For PRE, the estimator is

\hat{\tau}^{\mathrm{P}}=\frac{2}{16}\sum_{j=1}^{8}\hat{\tau}^{\mathrm{pair}}(j)=13.425

They are not the same.

Both estimators are unbiased because treatment is randomly assigned within each group, and both are weighted averages of the within-group differences in means, with weights based on group sizes.

2-2

I think the estimator for PRE has smaller variance. Because the gain in precision comes from the prediction capability for potential outcomes of the covariates. Dividing samples into more groups can tap the prediction capability of the covariates to the full.

Problem 3

3-1

Valid. Because $N_{t}(f)/N(f)=N_{t}(m)/N(m)$ , $\hat{\tau}^{\mathrm{dif}}=\hat{\tau}^{\mathrm{strat}}$ .
Valid. Adding covariates doesn't break consistency.
Valid. It's equivalent to 1.
Valid. The reason is the same as 1.
Valid. It's equivalent to 4.
Valid. Adding covariates doesn't break consistency.
Valid. Adding covariates doesn't break consistency.

3-2

First we have

\mathbb{V}_{1}=\mathbb{V}_{3},\ \mathbb{V}_{4}=\mathbb{V}_{5}

and

\mathbb{V}_{2}= \mathbb{V}_{3},\ \mathbb{V}_{4}\leq \mathbb{V}_{1}, \ \mathbb{V}_{6}\leq \mathbb{V}_{4}, \ \mathbb{V}_{7}= \mathbb{V}_{4}

\mathbb{V}_{6}\leq\mathbb{V}_{4}=\mathbb{V}_{5}=\mathbb{V}_{7}\leq\mathbb{V}_{1}=\mathbb{V}_{2}=\mathbb{V}_{3}

6 SRE and PRE

2026-10-28T13:34:42.000Z

Slides.

The two experiments introduced below are aimed to get unbiased estimators with less variance.

Stratified Randomized Experiment

Notations

Group the data into $J$ strata according to the covariate.
Implement a CRE within each data strata
- In each stratum $j$ , there are $N(j),N_{c}(j),N_{t}(j)$ .
- Proportion of stratum $j$ : $q(j)=N(j)/N$ .
- Propensity score of stratum $j$ : $e(j)=N_{t}(j)/N(j)$ .
Let $G_{i}$ denote the strata of unit $i$ , and let $B_{i}(j)=\mathbb{1}(G_{i}=j)$ .
The assignment mechanism:

P(\boldsymbol{W}\mid \boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))=\prod_{j=1}^{J} \binom{N(j)}{N_{t}(j)}^{-1} \text{for reasonable}\ \boldsymbol{W}, 0\ \text{for others}

Also known as Randomized Blocks Experiments (RBEs).

Causal Estimands in SREs

Take a two-stratum example, male and female.

ACE in females

\tau_{fs}(f)=\frac{1}{N(f)}\sum _{i:G_{i}=f}\left( Y_{i}(1)-Y_{i}(0) \right)

ACE in males

\tau_{fs}(m)=\frac{1}{N(m)}\sum _{i:G_{i}=m}\left( Y_{i}(1)-Y_{i}(0) \right)

ACE in all units (weighted average)

\tau_{fs}=\frac{1}{N}\sum_{i=1}^{N} \left( Y_{i}(1)-Y_{i}(0) \right) = q(f)\cdot\tau_{fs}(f)+q(m)\cdot\tau_{fs}(m)

Inference Approaches for SREs

Still use the example above.

Fisher's Exact P-Values

Keep in mind the purpose of hypothesis testing: to check if the observed data is extreme under $H_{0}$ .

Fisher's sharp null hypothesis:

H_{0}: Y_{i}(0)=Y_{i}(1)\ \text{for all}\ i

Choices of test statistics:

$T^{\mathrm{dif}}=\left| \bar{Y}_{t}^{\mathrm{obs}}-\bar{Y}_{c}^{\mathrm{obs}} \right|$ $T^{dif} = \overset{ˉ}{Y}_{t}^{obs} - \overset{ˉ}{Y}_{c}^{obs}$
- valid
- Not good because under $H_{0}$ , it doesn't have zero expectation.
$T^{\mathrm{dif},\lambda}=\left| \lambda\cdot(\bar{Y}_{t}^{\mathrm{obs}}(f)-\bar{Y}_{c}^{\mathrm{obs}}(f))+(1-\lambda)\cdot(\bar{Y}_{t}^{\mathrm{obs}}(m)-\bar{Y}_{c}^{\mathrm{obs}}(m)) \right|$ $T^{dif, λ} = λ \cdot (\overset{ˉ}{Y}_{t}^{obs} (f) - \overset{ˉ}{Y}_{c}^{obs} (f)) + (1 - λ) \cdot (\overset{ˉ}{Y}_{t}^{obs} (m) - \overset{ˉ}{Y}_{c}^{obs} (m))$
- valid
- Choosing a good $\lambda$ (i.e. $q(f)$ ) can produce a statistic behaving as we want.

And two rank statistics

But the rank statistics here have lower power compared with CRE due to few outliers in the space of class.

Neyman's Approach

The estimator:

\hat{\tau}^{\mathrm{strat}}= q(f)\cdot\hat{\tau}^{\mathrm{dif}}(f)+q(m)\cdot\hat{\tau}^{\mathrm{dif}}(m)

The estimator's expectation and variance can be deduced:

\begin{align} \mathbb{E}_{W}[\hat{\tau}^{\mathrm{strat}}] & =q(f)\cdot\tau_{fs}(f)+q(m)\cdot\tau_{fs}(m)=\tau_{fs} \\ \mathbb{V}_{W}(\hat{\tau}^{\mathrm{strat}}) & =q(f)^{2}\mathbb{V}_{W}(\hat{\tau}^{\mathrm{dif}}_{f})+q(m)^{2}\mathbb{V}_{W}(\hat{\tau}^{\mathrm{dif}}_{m})+q(f)q(m)\mathrm{Cov}(\cdot,\cdot) \\ & = q(f)^{2}\mathbb{V}_{W}(\hat{\tau}^{\mathrm{dif}}_{f})+q(m)^{2}\mathbb{V}_{W}(\hat{\tau}^{\mathrm{dif}}_{m})\quad (\hat{\tau}^{\mathrm{dif}}_{f}\ \text{and}\ \hat{\tau}^{\mathrm{dif}}_{m}\ \text{are independent}) \\ \end{align}

where the expression of $\mathbb{V}_{W}(\hat{\tau}^{\mathrm{dif}}_{f}),\mathbb{V}_{W}(\hat{\tau}^{\mathrm{dif}}_{m})$ can be found here.

The estimator $\bar{Y}^{\mathrm{obs}}_{t}-\bar{Y}^{\mathrm{obs}}_{c}$ is biased and has larger variance.

SREs vs CREs

We compare the super-population variances of CRE and SRE under two sampling designs.

For CRE, individuals are independently sampled from the super-population, where $P(G_{i}=f)=q$ , followed by complete randomization with treatment probability $p$ in the sense that $N_{t}/N=p$ . The estimator is $\hat{\tau}^{\mathrm{dif}}$ .
For SRE, we use stratified sampling with fixed stratum proportion $q=N(f)/N$ and, followed by independent complete randomization within each stratum with the same treatment fraction $p$ . The estimator is $\hat{\tau}^{\mathrm{strat}}$ .
(See textbook p211.)

Recall Neyman estimator for CRE:

\hat{\tau}^{\mathrm{dif}}=\bar{Y}^{\mathrm{obs}}_{t}-\bar{Y}^{\mathrm{obs}}_{c}

with super population variance

\mathbb{V}_{sp}(\hat{\tau}^{\mathrm{dif}})=\frac{\sigma_{c}^{2}}{N_{c}}+\frac{\sigma^{2}_{t}}{N_{t}}

In order to compare it with SRE, we can divide $\sigma_{c}^{2}$ into

q\sigma_{c}^{2}(f)+(1-q)\sigma_{c}^{2}(m)+q(1-q)[\mu_{c}(f)-\mu_{c}(m)]^{2}

by the total variance formula. Similarly $\sigma_{t}^{2}$ can be divided.

Within each stratum, there is a CRE, so

\mathbb{V}_{sp}[\hat{\tau}^{\mathrm{dif}}(f)]=\dots,\ \mathbb{V}_{sp}[\hat{\tau}^{\mathrm{dif}}(m)]=\dots

and $\hat{\tau}^{\mathrm{dif}}(f)\perp \hat{\tau}^{\mathrm{dif}}(m)$ .

Therefore

\begin{align}\mathbb{V}_{sp}[\hat{\tau}^{\mathrm{strat}}] & = \mathbb{V}_{sp}[q\cdot+(1-q)\cdot] \\ & = q^{2}\mathbb{V}_{sp}[\cdot]+(1-q)^{2}\mathbb{V}_{sp}[\cdot] \\ & = \frac{q}{N}\left( \frac{\sigma_{t}^{2}(f)}{p}+\frac{\sigma_{c}^{2}(f)}{1-p} \right) + \frac{1-q}{N}\left( \frac{\sigma^{2}_{t}(m)}{p}+ \frac{\sigma^{2}_{c}(m)}{1-p}\right)\end{align}

where $\sigma_{t}^{2}(f)=\mathbb{V}_{sp}[Y_{i}(1)\mid G_{i}=f],\dots$

Can verify that

\mathbb{V}_{sp}[\hat{\tau}^{\mathrm{dif}}]-\mathbb{V}_{sp}[\hat{\tau}^{\mathrm{strat}}]=q(1-q)\cdot\left[ \frac{(\mu_{c}(f)-\mu_{c}(m))^{2}}{N(1-p)}+\frac{(\mu_{t}(f)-\mu_{t}(m))^{2}}{Np} \right]\geq 0

One is never worse off stratifying on a covariate. The gain in precision comes from the prediction capability of the stratification.

That is saying that if the stratification does have the ability to distinguish the potential outcomes, than the gain in precision is good. We fully utilized the information.

Regression Analysis of SREs

Keep in mind that the estimand is

\begin{align}\tau_{sp} & =\mathbb{E}[Y_{i}(1)-Y_{i}(0)] \\ & = \sum_{j=1}^{J} q(j)\tau_{sp}(j)\end{align}

Where $q(j)=P(B_{i}(j)=1)$ instead of $N(j)/N$ . It's the perspective of super population now.

The following model is the real one, we should directly estimate this one. I think the section "Without Interaction" is nonsense. Why do we have to discuss it?

Without Interaction

Regression model:

Y_i^{\mathrm{obs}}=\tau W_i+\sum_{j=1}^{J}\beta(j)B_i(j)+\varepsilon_i

Least square estimate:

(\hat{\tau}^{ol s},\hat{\beta}^{ol s})=\operatorname*{argmin}_{\tau,\beta}\sum_{i=1}^{N} (\dots)^{2}

Population counterparts to these OLS:

(\tau^{*},\beta^{*})=\operatorname*{argmin}_{\tau,\beta}\mathbb{E}[ (\dots)^{2}]

Suppose we conduct a stratified randomized experiment in a sample drawn at random from an infinite population. Then

$\tau^{*}=\tau_{\omega}$ , where

\tau_{\omega}=\sum_{j=1}^{J} \omega(j)\tau_{sp}(j)\big/ \left( \sum_{j=1}^{J} \omega(j) \right),\ \omega(j)=q(j)e(j)(1-e(j))

$\sqrt{N} \cdot \left( \hat{\tau}^{\text{ols}} - \tau_{\omega} \right) \xrightarrow{d} \mathcal{N} \left( 0, \frac{\mathbb{E}\left[ \left( W_i - \sum_{j=1}^{J} q(j) \cdot B_i(j) \right)^2 \cdot \left( Y_i^{\text{obs}} - \tau^* \cdot W_i - \sum_{j=1}^{J} \beta_j^* \cdot B_i(j) \right)^2 \right]}{\left( \sum_{j=1}^{J} q(j) \cdot e(j) \cdot (1 - e(j)) \right)^2} \right)$

Note that $\tau_{\omega}$ doesn't necessarily equals to $\tau_{sp}$ unless

$e(j)$ are all the same (i.e. the prob. of being treated is the same across strata), or
$\tau_{sp}(j)$ are all the same

Adding Interactions

The model is

Y_i^{obs} =\sum_{j=1}^J\beta_j B_i(j) +\sum_{j=1}^J\tau_j W_i B_i(j) +\varepsilon_i

which allows each stratum has its own intersection and treatment effect. Having gotten $\hat{\tau}_{j}$ , we can take a weighted average of them to get $\hat{\tau}^{\mathrm{strat}}$ . We can also directly obtain it by rewriting the model as

Y_i^{obs} =\sum_{j=1}^J\beta(j)B_i(j) +\tau W_i\frac{B_i(J)}{q(J)}+\sum_{j=1}^{J-1}\gamma(j)W_i \left(B_i(j)-\frac{q(j)}{q(J)}B_i(J)\right) +\varepsilon_i

▶

The essence

Use a simple example to catch the essence of the rewriting: suppose we have the model

Y=\beta_{0}+\beta_{1}X_{1}+\beta_{2}X_{2}+\varepsilon

And we want to estimate $a\beta_{1}+b\beta_{2}$ directly, we can rewrite it as

Y=\beta_{0}+(a\beta_{1}+b\beta_{2}) \frac{X_{1}}{a}+\beta_{2}\left( X_{2}-\frac{b}{a}X_{1} \right)+\varepsilon

More generally,

Y=\beta^TX+\varepsilon=(A\beta)^T(A^{-1}X)+\varepsilon

Least square estimate: $\hat{\tau}^{ol s,inter}$ .
Population counterpart: $\tau^{*}$ .

Suppose we conduct a stratified randomized experiment in a sample drawn at random from an infinite population. Then

$\tau^* = \tau_{\text{sp}}$
$\sqrt{N} \cdot \left( \hat{\tau}^{\text{ols,inter}} - \tau_{\text{sp}} \right) \xrightarrow{d} \mathcal{N}\left(0, \sum_{j=1}^{J} q(j)^2 \cdot \left( \frac{\sigma_c^2(j)}{(1 - e(j)) \cdot q(j)} + \frac{\sigma_t^2(j)}{e(j) \cdot q(j)} \right) \right)$

Model-Based Analysis

Not required.

Pairwise Randomized Experiments

A special case of a SRE with $J=N/2$
One treated and one control unit (a pair) within each stratum
Assignment probability:

P(\boldsymbol{W}\mid \boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))=\prod_{j=1}^{N/2} \frac{1}{2}=2^{-N/2}\ \text{for reasonable}\ \boldsymbol{W}, 0\ \text{for others}

Intuition behind PREs
- Within-pair variation in potential outcomes is less than between-pair variation -- gain in precision
- If the population is relatively large, it may be possible to get very close matches of units in each pair

Notations

Causal Estimands in PREs

ACE in strata $j$ :

\tau_{\mathrm{pair}}(j)=\frac{1}{2}\sum_{i:G_{i}=j} (Y_{i}(1)-Y_{i}(0))=\frac{1}{2}\left[ (Y_{j,A}(1)-Y_{j,A}(0))+ (Y_{j,B}(1)-Y_{j,B}(0)) \right]

ACE in all units:

\tau_{fs}=\frac{1}{N}\sum_{i=1}^{N} (Y_{i}(1)-Y_{i}(0))=\frac{2}{N}\sum_{j=1}^{N/2} \tau_{\mathrm{pair}}(j)

Inference Approaches for PREs

Fisher's Exact P-Values

Same as SREs.

Neyman’s Repeated Sampling Approach

The Challenges

One treated and one control unit in each strata lead to specified challenges.

Let's see

\hat{\tau}^{\mathrm{dif}}=\frac{2}{N}\sum_{j=1}^{N/2} \hat{\tau}_{\mathrm{pair}}(j)

where $\hat{\tau}^{\mathrm{pair}}(j)=Y^{\mathrm{obs}}_{j,t}-Y^{\mathrm{obs}}_{j,c}=\sum _{i:G_{i}=j}(2W_{i}-1)Y^{\mathrm{obs}}_{i}$ .

Can verify that

\begin{align}\mathbb{E}_{W}[\hat{\tau}^{\mathrm{pair}}(j)] & =\tau_{fs}(j) \\\mathbb{V}_{W}[\hat{\tau}^{\mathrm{pair}}(j)] & = \frac{S_{c}(j)^{2}}{N_{c}(j)}+\frac{S_{t}(j)^{2}}{N_{t}(j)}-\frac{S_{tc}(j)^{2}}{N(j)} \\ & = S_{c}(j)^{2}+S_{t}(j)^{2}-\frac{S_{tc}(j)^{2}}{2}\end{align}

\begin{align}\mathbb{E}_{W}[\hat{\tau}^{\mathrm{dif}}] & =\tau_{fs} \\\mathbb{V}_{W}[\hat{\tau}^{\mathrm{dif}}] & = \left( \frac{2}{N} \right)^{2}\cdot\sum_{j=1}^{N/2} \left( S_{c}(j)^{2}+S_{t}(j)^{2}-\frac{S_{tc}(j)^{2}}{2} \right)\end{align}

Till now it seem good. Seems like we can use this to estimate and conduct inference. But we cannot estimate the finite sample variances $S_{c}(j)^{2}, S_{t}(j)^{2}$ because of the sample size in each stratum. (Not to mention $S_{tc}(j)^{2}$ , which cannot be estimated even in CREs)

Variance Estimation via Extra Assumptions

constant treatment effect within pairs：

Y_{j,A}(1)-Y_{j,A}(0)=Y_{j,B}(1)-Y_{j,B}(0)=:\tau_{\mathrm{pair}}(j)\quad \forall j

- This leads to $S_c^2(j)=S_t^2(j)=:S(j)$ and $S_{tc}^2(j)=0$. - So $\mathbb{V}_{W}[\hat{\tau}^{\mathrm{pair}}(j)]=2S^2(j)$.

treatment effect is constant across pairs:

\tau_{\mathrm{pair}}(j)=\tau_{fs}\quad \forall j

within-pair variances are constant:

S^{2}(j)=S^{2}\quad \forall j

With this assumptions holding, we have

\mathbb{V}_{W}[\hat{\tau}^{\mathrm{dif}}]=\dots=\frac{4}{N}S^{2}

A variance estimator can be proposed:

\hat{\mathbb{V}}^{\mathrm{pair}}(\hat{\tau}^{\mathrm{dif}})=\frac{4}{N(N-2)}\sum_{j=1}^{N/2} \left( \hat{\tau}^{\mathrm{pair}}(j)-\hat{\tau}^{\mathrm{dif}} \right)^{2}

(Note that $\hat{\tau}^{\mathrm{dif}}$ is the average of all $\hat{\tau}^{\mathrm{pair}}(j)$ 's)

Assumption 1 and 2 are enough to keep the estimator unbiased.

Can show that

\mathbb{E}[\hat{\mathbb{V}}^{\mathrm{pair}}(\hat{\tau}^{\mathrm{dif}})]=\mathbb{V}_{W}[\hat{\tau}^{\mathrm{dif}}]+\frac{4}{N(N-2)}\sum_{j=1}^{N/2}(\tau_{\mathrm{pair}}(j)-\tau)^{2}

Under the assumptions above, the estimator is unbiased. If there is heterogeneity in the treatment effects (i.e. $\tau_{\mathrm{pair}}(j)$ are not constant) , then this sampling variance estimator is upwardly biased, and the corresponding confidence intervals will be conservative but still valid.

▶

The math behind the variance estimator

We have $N$ mutually independent r.v.s $X_{1},\dots X_{N}$ . They have equal expectations $\mu$ but different variances. Let $\mathbb{V}[X_{i}]=\sigma_{i}^{2}$ . Then

\mathbb{V}[\bar{X}]=\frac{1}{N^{2}}\sum_{i=1}^{N} \sigma_{i}^{2}

and it can be estimated by

\frac{s^{2}}{N}=\frac{1}{N(N-1)}\sum_{i=1}^{N} (X_{i}-\bar{X})^{2}

because one can verify that

\mathbb{E}\left[ \frac{s^{2}}{N} \right]=\frac{1}{N^{2}} \sum_{i=1}^{N} \sigma^{2}_{i}=\mathbb{V}[\bar{X}]

If the expectations are different, then

\mathbb{E}\left[ \frac{s^{2}}{N} \right]=\mathbb{V}[\bar{X}]+\frac{1}{N(N-1)}\sum_{i=1}^{N} (\mu_{i}-\bar{\mu})^{2}

where $\mu_{i}=\mathbb{E}[X_{i}],\bar{\mu}=\frac{1}{N}\sum_{i=1}^{N}\mu_{i}$ .

\sum_{i=1}^N(X_i-\mu)^2 = \sum_{i=1}^{N} (X_{i}-\bar{X})^{2}+\sum_{i=1}^{N} (\bar{X}-\mu)^{2}

This is an algebraic identity (恒等式). The deviation of many numbers to a certain constant can be divided into two parts: the deviation from the mean and the deviation of the mean from the constant.

Regression Method

Four models are proposed:

$\hat{\tau}^{\mathrm{pair}}(j)=\tau+\varepsilon_{j}$ . This one is equivalent to Neyman's approach above.
$\hat{\tau}^{\mathrm{pair}}(j)=\tau+\beta \Delta_{X,j}+\varepsilon_{j}$ , where

\Delta_{X,j}=W_{j,A}(X_{j,A}-X_{j,B})+(1-W_{j,A})(X_{j,B}-X_{j,A})

alway ensure that it's the diff between treated one and controlled one.

$\hat{\tau}^{\mathrm{pair}}(j)=\tau+\gamma(\bar{X_{j}}-\bar{X})+\varepsilon_{j}$ , where $\bar{X}_{j}=(X_{j,A}+X_{j,B})/2$ .
$\hat{\tau}^{\mathrm{pair}}(j)=\tau+\beta \Delta_{X,j}+\gamma(\bar{X_{j}}-\bar{X})+\varepsilon_{j}$ .

For model 4, we have

▶

class notes

大班教学还是小班教学

SUTVA，以班级为 unit
分层随机化，为了避免 Simpson's 悖论，相对 CRE 锦上添花，功效更高

FEP

每一层 $\hat{\tau}^{\mathrm{dif}}$ 都是无偏估计
加权平均
SRE 中， $\bar{Y}^{obs}_{t}-\bar{Y}^{obs}_{c}$ 不是无偏的，但 valid，因为能得知 $H_{0}$ 下的分布，用于衡量观测的极端程度，拒绝或接受原假设
为啥秩统计量在 SRE 中不好？

Neyman

$\hat{\tau}^{SRE}$ 的方差，交叉项的协方差为 0
SRE 满足无混杂假设，避免 Simpson's paradox，也就是 confoundedness bias
分层使得估计量方差更小
- 小多少，取决于 $X$ 对潜在结果的预测能力，预测越准方差越小
$P_{CRE}(\overrightarrow{W}=\overrightarrow{w}\mid \overrightarrow{N})=P_{SRE}(\overrightarrow{W}=\overrightarrow{w})$ $P_{C R E} (W = w ∣ N) = P_{S R E} (W = w)$
- 明明是 CRE 拿来的数据，可以当作 SRE 来分析
- Post-stratification in a CRE
不要求每一组 ATE 一样
- 等价于回归法中加交叉项的回归

Reg

相当于每层分别做回归，然后组合起来

PRE

特殊的 SRE
改进精度，小方差
每层样本量只有 2
四种方法
- FEP 不受影响
- Model based 不受影响
- Neyman 和 Reg 受样本量影响
每一对当成一个样本，作为 $\tau$ 的一个代表。认为是 $\tau$ 的多次观测
回归
- 系数跟样本量一样多，过拟合，方差大，精度差
- 四种方法

6 Remedies and Miscellaneous Topics

2026-10-27T13:35:05.000Z

Slides.

More Diagnostics for Residuals

Review Residual Plots

Should do EDA first. Use

1	`scatter.smooth(X,Y)`

to get a smoothed curve fit to see the relation ship between $X$ and $Y$ .

Let see some examples:

No obvious violation
Nonlinearity. Because at some $X$ the residuals are all negative or positive.
Non-constant variance. Note that the residual plot below uses absolute residual as the y-label, aiming to see the scale of residuals.

Built-in diagnostic plots in R

1 2	`reg <- lm(hours ~ lotsize, data=toluca) plot(reg)`

R will show four diagnostic plots one by one:

Residuals vs Fitted
Normal Q-Q
Scale-Location
Residuals vs Leverage

Points with numbers next to them in each plot are extreme values. The numbers are the index of them, letting us to check the original data.

Residuals vs Fitted

Why $\hat{Y}$ ? To combine all explanatory variables' influences, easy to move to multiple linear regression.
Used to check if the residuals have non-linear patterns.
It’s good if residuals equally spread around a horizontal line without distinct patterns.

Normal Q-Q

Used to check if the residuals are normally distributed.
It’s good if residuals are lined well on the straight dashed line.

Standardized residual is defined as

\frac{e_{i}}{s\sqrt{1-h_{ii}}}

where $h_{ii}= \frac{\partial \hat{Y}_{i}}{\partial Y_{i}}=\frac{1}{n}+ \frac{(X_{i}-\bar{X})^{2}}{S_{x x}}$ , the leverage. It will be discussed in the next lecture.

Note that

\mathbb{V}[e_{i}\mid \boldsymbol{X}] = (1-h_{ii})\sigma^{2}

The reason why standardized residual is used is that $e_{i}$ are not i.i.d. so they cannot be directly used to delegate the real $\varepsilon_{i}$ . Pre-process needed.

Scale-Location

It shows if residuals are spread equally along the ranges of predictors
Check the assumption of equal variance (homoscedasticity)
It’s good if you see a horizontal line with equally (randomly) spread points

Residuals vs Leverage

Leverage, also known as the observation self-sensitivity or self-influence

h_{ii}=\frac{\partial \hat{Y}_{i}}{\partial Y_{i}}=\frac{1}{n}+ \frac{(X_{i}-\bar{X})^{2}}{S_{x x}}\in [0,1]

For design matrix $X_{n\times p}$ , $s^{2}=\frac{e^Te}{n-p}$ . Cook's distance is defined as

D_{i}=\frac{\sum_{j=1}^{N} (\hat{Y}_{j}-\hat{Y}_{j(-i)})^{2}}{ps^{2}}=\frac{e_{i}^{2}}{ps^{2}}\cdot \frac{h_{ii}}{(1-h_{ii})^{2}}

where $\hat{Y}_{j(-i)}$ is the fitted response value obtained when excluding the $i$ -th observation. It measures the influence of $i$ -th case on all predicted values.

给定 $h_{ii}$ 和 $D_{i}$ ，可以解出标准化残差的平方，故可以画出两条对称的等值线。

F-Test for Lack of Fit

When we have repeat observations at various values of $X$ , the error term will be partitioned into pure error (error within replicates) and a lack of fit error.

\begin{align}H_0 & : \text{There is no lack of fit in the regression model} \\H_{1} & : \text{o.w.}\end{align}

Use GLT, the basic idea is to compare two models -- the full one and the reduced one. Then here comes the question, what are the two models?

Suppose there are $c$ different observed values of $X_{i}$ : $x_{1},\dots,x_{c}$ . So there are $c$ groups, and within each group $i$ , there are $n_{i}$ observations. Let $Y_{ij}$ be the $j$ -th response in the $i$ -th group.

If $H_{0}$ holds, then the mean response $\mu_{i}=\beta_{0}+\beta_{1}X_{i}$ . The reduced model is

Y_{ij}=\beta_{0}+\beta_{1}X_{i}+\varepsilon_{ij}

Run regression and get $SSE(R)$ . Its df is $n-2$ .

Suppose that $X_{i}$ can take $m$ values $x_{1},\dots,x_{m}$ . Then the full model is

Y_{ij}=\mu_{i}+\varepsilon_{ij}

Run regression and get $SSE(F)=SSPE$ (Pure Error). Note that this model has as many parameters as the number of the groups. So its df is $n-c$ .

And we can get

F=\frac{[SSE(R)-SSE(F)]/(df_{ER}-df_{EF})}{SSE(F)/df_{EF}}

There is a relationship

SSE=SSPE+SSLF

Let's Have a Deeper Look at DF (Cont'd)

Look at this before going on.

Remember that $SSE=\|\hat{Y}-Y\|^{2}=\|(I-P)Y\|^{2}$ . Below we show $(I-P)Y$ can continue to be divided into two orthogonal vectors.

Let $G_{ij}=I(X_{i}=j),\ j=1,..,c$ . Then there is another design matrix

Z=\begin{bmatrix}G_{ij}\end{bmatrix}_{n\times c}

The full model can be written as

Y=Z\mu+\varepsilon

note that there is not an all-one column in $Z$ , and $\mathcal{R}(P)\subseteq \mathcal{R}(P_{Z})$ (there exists a matrix $A$ satisfying $AZ=X$ ) where $P_{Z}$ is the projection matrix of $Z$ .

Rewrite $(I-P)Y$ as

(I-P)Y=(I-P_{Z})Y+(P_{Z}-P)Y

We can verify that $\mathcal{R}(I-P)=\mathcal{R}(I-P_{Z})\oplus \mathcal{R}(P_{Z}-P)$ . The core is $\mathcal{R}(P)\subseteq \mathcal{R}(P_{Z})$ .

Note that

\|(I-P_{Z})Y\|^{2}=SSPE,\ \|(P_{Z}-P)Y\|^{2}=SSLF

Remedies

Nonlinear Relationships

Do transformations like $\log,\exp,$ etc. Then we may get linear relationship. But many of the times, such transformations may not exist. Then, nonlinear regression analysis is required.

Non-Constant Variance

Use a weighted analysis.

Box-Cox Transformations

Also called power transformations.
Designed for strictly positive responses.
Adjust for non-Normality and non-constant variance
$Y^{*}=(Y^{\lambda}-1)/\lambda$ .

Putting it simple, in order to find the best $\lambda$ , we can fix other parameters and optimize $SSE(\lambda)$ only.

Why we concern the confidence interval:

Interpretability
Check if 1 lies in the CI -- that means a linear relationship cannot be rejected.

CI-HW4

2026-10-24T14:44:30.000Z

HW4.

Problem 1

1-1

$Y_{i}(0)$ is the potential outcome of unit $i$ if it doesn't receive treatment.

$Y_{i}^{obs}$ is the observed outcome of unit $i$ . Whether it is $Y_{i}(1)$ or $Y_{i}(0)$ depends on the assignment.

1-2

It represents the expected outcome for individuals who received treatment, had they not received the treatment.

1-3

$\mathbb{E}[Y_{i}(0)]$ is the expected potential outcome of unit $i$ under no treatment.

While $\mathbb{E}[Y_{i}^{obs}\mid W_{i}=0]=\mathbb{E}[Y_{i}(0)\mid W_{i}=0]$ is the expected potential outcome of unit $i$ under no treatment, conditional on the unit actually being assigned to the control group.

1-4

$\mathbb{E}[Y_{i}(0)\mid W_{i}=1]$ : the expected potential outcome of $i$ under no treatment, conditioning on it being assigned to the treatment group.

$\mathbb{E}[Y_{i}(0)\mid W_{i}=0]$ : the expected potential outcome of $i$ under no treatment, conditioning on it being assigned to the control group.

1-5

$\mathbb{E}[Y_{i}^{obs}\mid W_{i}=0]$ and $\mathbb{E}[Y_{i}(0)\mid W_{i}=0]$ .

1-6

This assignment guarantees that

\overrightarrow{W}\perp (\boldsymbol{Y}(0),\boldsymbol{Y}(0))

Therefore

p_{i}(\boldsymbol{Y}(0),\boldsymbol{Y}(1))=P(W_{i}=1)=.5

and

P(\overrightarrow{W}\mid \boldsymbol{Y}(0),\boldsymbol{Y}(1))=P(\overrightarrow{W})=\begin{cases}1/C_{n}^{n/2} & \text{if}\ \overrightarrow{1}\cdot\overrightarrow{W}=\frac{n}{2} \\0 & \text{o.w.}\end{cases}

So unconfounded assumption is satisfied.

And because $1/C_{n}^{n/2}$ can be expressed as

c\cdot \prod_{i=1}^{n}0.5 = c\cdot \prod_{i=1}^{n} (.5)^{W_{i}}(1-.5)^{1-W_{i}}

So $\overrightarrow{W}$ satisfies the individualistic assumption.

All quantities in (5) can be identified.

Problem 2

2-1

I prefer a small puncture procedure. Because the successful rate is higher.

2-2

I would choose the open surgical procedure, because it has a higher success rate in both stone-size groups

The paradox arises because the proportion of small-stone cases differs between the two procedures. Small stones are easier to treat and have higher success rates, and they make up a much larger share of the puncture group than of the open-surgery group. As a result, small-stone cases dominate the overall success rate of the puncture procedure and inflate it, even though open surgery performs better within each stone-size group.

2-3

It is possible. Because $W_{i}$ can still be influenced by $X_{i}$ . Patients with larger stone are more likely to be assigned to open surgical procedure.

$W_{i}\perp (Y_{i}(0),Y_{i}(1))$ is enough to justify

\hat{\tau}^{\mathrm{dif}}=\bar{Y}^{obs}(1)-\bar{Y}^{obs}(0)

is valid for estimating $\tau_{fs}$ and $\tau_{sp}$ .

The result is $\hat{\tau}^{\mathrm{dif}}=-0.0457$ .

It doesn't conflict with my results in (2), because (2) is saying that

\mathbb{E}[Y_{i}^{obs}\mid W_{i}=1,X=x]-\mathbb{E}[Y_{i}^{obs}\mid W_{i}=0,X=x]>0

for $x=1,0$ .

Problem 3

It is from a random experiment.
It is reasonable
Because it is a complete randomization. $\boldsymbol{W}$ is independent of all of the covariates and potential outcomes.

线性回归分析-HW5

2026-10-23T19:04:06.000Z

HW5.
Data.

Prob-1

1-a

Consistent, as the age groups are similar in size.

1-b

The residuals present good normality.

1-c

The two plots provide the same information as $\hat{Y}_{i}$ is a linear transformation of $X_{i}$ .

The two plots ascertain $\mathbb{E}(\varepsilon\mid X)=0$ and $\mathbb{V}[\varepsilon\mid X]=\sigma^{2}$ .

1-d

1 2	`q <- qnorm(res) cor(q$x,q$y)`

The coefficient of correlation: 0.9896158

Use the correlation test

t=\frac{r}{\sqrt{1-r^{2}}}\sqrt{n-2}\sim t_{n-2}

Here $t=52.434,n=60$ .

The results are

We have enough evidence to say that they have linear correlation.

1-e

Suppose

\sigma_{i}^{2}=\exp(\alpha_{0}+\alpha_{1}X_{i})

The alternatives:

\begin{align}H_{0} & :\alpha_{1}=0 \\H_{1} & :\text{o.w.}\end{align}

The test statistic:

LM=\frac{SSR^{*}}{2}\big/\left( \frac{SSE}{n} \right)^{2}=\frac{31833}{2}\big/\left( \frac{3874.4}{60} \right)^{2}=3.817167

The decision rule:

LM>\chi^{2}_{0.99,1}=6.634897

The conclusion: cannot reject.

Prob-2

It is saying that the median is smaller than the mean, which usually indicates that the distribution is right-skewed.

5 Assignment Design

2026-10-21T13:39:56.000Z

Slides.

Intro -- Simpson's Paradox

The overall order is reversed after looking at each group. If the question is that, if a child is to be sent to one of the states to receive education, which one would be a better choice, then the overall index cannot be trusted.

The reason why this paradox happens is that the distribution of the covariate $X$ , which is the unit's race, is different between the two groups. This is equivalent to the difference in assignment mechanism $W$ across each race group. The following example explains:

	group1	group2
Treat	$a$	$b$
Control	$c$	$d$

\frac{a}{b}=\frac{c}{d}\iff \frac{a}{c}=\frac{b}{d}

This is math from elementary school.

So we need to know the assignment mechanism before conducting correct inferences.

Note that even in CREs, Simpson's paradox can still happen. If it happens that the distribution of the covariate differs between the treated group and the control group, though it doesn't affect the validity of the estimator.

Definitions for Assignment Mechanism

Three Concepts

Definition (Assignment Mechanism)
Given a population of $N$ units, the assignment mechanism is a row-exchangeable function $P(\boldsymbol{W}\mid \boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))$ , taking on values in $[0,1]$ , satisfying

\sum _{W\in\{0,1\}^{N}} P(\boldsymbol{W}\mid \boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))=1,\ \forall \boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1)

Definition (Unit Assignment Probability)

p_{i}(\boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))=P(W_{i}=1\mid \boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))

Definition (Finite Population Propensity Score)
The propensity score at $x$ is the average unit assignment probability for units with $X_{i}=x$ ,

e(x)=\frac{1}{N(x)}\sum _{i:X_{i}=x}p_{i}(\boldsymbol{X},\boldsymbol{Y}(1),\boldsymbol{Y}(0))

where $N(x)=\#\{i\in\{1,2,\dots, N\}:X_{i}=x\}$ is the number of units with $X_{i}=x$ . If $N(x)=0, e(x)$ is defined to be $0$ .

Examples

Some notes on example III:

Because there is no covariate, the propensity score degrades to

e(x)=\frac{1}{N}\sum_{i=1}^{N} p_{i}(\boldsymbol{Y}(1),\boldsymbol{Y}(0))

Whatever $\boldsymbol{Y}(1),\boldsymbol{Y}(0)$ are, $e(x)=\frac{1}{2}$ . It's easy to verify.

Below is a more complex example.

A sequential tree should be used to analyze this example.

Let $A=Y_{1}(1)-Y_{2}(0), B=Y_{1}(0)-Y_{2}(1)$ . Then the value space of $\boldsymbol{Y}(1),\boldsymbol{Y}(0)$ can be divided into four mutually exclusive parts:

\begin{align}(1)\ & A>0,B>0 \\(2)\ & A>0,B<0 \\(3)\ & A<0,B>0 \\(4)\ & A<0,B<0\end{align}

For the first part, $\boldsymbol{W}$ can only be $(1,0,1)$ and $(0,1,0)$ with equal probs $0.5$ . Similarly we can get the full assignment mechanism.

As for the unit assignment prob., we can easily verify that

p_{1}(\boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))=p_{2}(\boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))=\frac{1}{2}

(Note that it doesn't matter if we add $\boldsymbol{X}$ or not because the assignment mechanism, in this setting, has nothing to do with $\boldsymbol{X}$ .)

For the 3rd unit, we have

p_{3}(\boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1)) =\begin{cases}1/2, & A>0,\ B>0 \\1, & A>0,\ B<0 \\0, & A<0,\ B>0 \\1/2, & A<0,\ B<0\end{cases}

Let's talk about the (finite sample) propensity score now:

\begin{align}e(1) & =p_{1}(\boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1)) \\e(2) & =p_{2}(\boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1)) \\e(3) & = p_{3}(\boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))\end{align}

世界运行规律（ $\boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1)$ ）就那一套，但我们不知道是哪一套，所以分情况讨论

Assumptions for Assignment Mechanism

The four examples above are somewhat complex already though the number of the units is rather small. Cannot imagine how hard it would be if the number of units becomes large. So we need some assumptions to simplify our analysis.

Assumptions

Definition (Individualistic Assignment)
An assignment mechanism $P(\boldsymbol{W}\mid \boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))$ is individualistic if, for some function $q(\cdot)\in [0,1]$ ,

p_{i}(\boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))=q(X_{i},Y_{i}(0),Y_{i}(1))\quad\forall i=1,\dots,N

and

P(\boldsymbol{W}\mid \boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))=c\cdot \prod_{i=1}^{N} q(X_{i},Y_{i}(0),Y_{i}(1))^{W_{i}}(1-q(X_{i},Y_{i}(0),Y_{i}(1)))^{1-W_{i}}

for $(\boldsymbol{W}, \boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))\in \mathbb{A}$ , for some set $\mathbb{A}$ , and 0 elsewhere ( $c$ is the constant that ensures that the probabilities sum to unity).

The line $\dots\prod\dots$ is weaker than saying that $W_{i}$ 's are independent given $\boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1)$ . For example, complete randomization satisfies this.

Example IV above doesn't satisfy this.

Given individualistic assignment, the propensity score simplifies to

e(x)=\frac{1}{N_{x}}\sum _{i:X_{i}=x}q(X_{i},Y_{i}(0),Y_{i}(1))

$3N$ -variate function $\to$ 3-variate function.

Definition (Probabilistic Assignment)
An assignment mechanism $P(\boldsymbol{W}\mid \boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))$ is probabilistic if the probability of assignment to treatment for unit $i$ is strictly between zero and one:

p_{i}(\boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))\in (0,1),\ \text{for each possible}\ \boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1)

for all $i=1,\dots,N$ .

Don't know the purpose of this assumption.

Definition (Unconfounded Assignment)
An assignment mechanism is unconfounded if it does not depend on the potential outcomes:

P(\boldsymbol{W}\mid \boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1))=P(\boldsymbol{W}\mid \boldsymbol{X},\boldsymbol{Y}'(0),\boldsymbol{Y}'(1))

for all $\boldsymbol{W},\boldsymbol{X},\boldsymbol{Y}(0),\boldsymbol{Y}(1),\boldsymbol{Y}'(0),\boldsymbol{Y}'(1)$ .

If an assignment mechanism is unconfounded, we can drop the two potential outcomes and write the assignment mechanism as $P(\boldsymbol{W}\mid \boldsymbol{X})$ .

This kind of assignment can avoid Simpson's paradox. Why?

Application

Here is an example of application, which will be used here. Suppose $X_{i}\in \{f,m\}$ . Then

P(\mathbf W\mid\mathbf X,\mathbf Y(0),\mathbf Y(1)) =c\cdot e(f)^{N_t(f)}[1-e(f)]^{N_c(f)}\cdot e(m)^{N_t(m)}[1-e(m)]^{N_c(m)}

for $\boldsymbol{W}\in \mathbb{A}$ , where $N_{t}(f)=\sum _{i:X_{i}=f}W_{i},N_{c}(f)=\sum _{i:X_{i}=f}(1-W_{i})$ ...

So if we further condition on $N_{t}(f)$ and $N_{t}(m)$ , then for $\boldsymbol{W}\in \mathbb{A},$ the assignment probabilities are all the same. Therefore

P(\mathbf W\mid\mathbf X,\mathbf Y(0),\mathbf Y(1),N_{t}(f),N_{t}(m))= \frac{1}{\binom{N(f)}{N_{t}(f)}\binom{N(m)}{N_t(m)}}

and one step forward, we have

P(W_{i}=1\mid \dots)=\begin{cases}\frac{1}{\binom{N(f)}{N_{t}(f)}\binom{N(m)}{N_t(m)}}\cdot \binom{N(f)-1}{N_{t}(f)-1}\cdot \binom{N(m)}{N_{t}(m)} = \frac{N_{t}(f)}{N(f)} & X_{i}=f\\\frac{1}{\binom{N(f)}{N_{t}(f)}\binom{N(m)}{N_t(m)}}\cdot \binom{N(m)-1}{N_{t}(m)-1}\cdot \binom{N(f)}{N_{t}(f)}=\frac{N_{t}(m)}{N(m)} & X_{i}=m\end{cases}

▶

note in class

Simpson's Paradox，权重，统计学常见现象
- $Y$ 跟 $X$ 有关
- $W$ 跟 $X,Y$ 都有关，这种协变量称为混杂变量。confounding, confounding bias，会得出错误因果作用推断
- 协变量分布平衡，即无矛盾
分配机制 $P(\boldsymbol{W}\mid \boldsymbol{Y}(1),\boldsymbol{Y}(0),\boldsymbol{X})$ $P (W ∣ Y (1), Y (0), X)$ . 随机性来自有限总体那一层
- 向量概率
- 个体概率
- 倾向得分（最重要）
倾向得分，横着比例一样 $\iff$ $⟺$ 竖着比例一样
- 为啥非得竖着看？倾向得分一样的可以打包起来一起算
序贯树
三个重要假设
1. 个体化假设：降维，同一个函数
2. 无混杂假设，条件独立（最根本，克服缺失数据问题，但没办法检验）
3. 概率假设

wtx的博客

7 Unconfounded Treatment Assignment & Estimating Propensity Score

Randomized Experiments and Observational Studies

Randomized Experiments

Observational Studies

Caution

RAMs vs SREs

Balancing Score and Propensity Score

Concepts

Methods for Estimation in RAMs

Imputation

Model-Based Method

Regression Method

Summary of Imputation Methods

Balance the Covariates

Weighting Estimators

Blocking Estimators

Matching Estimators

Estimating Propensity Score

线性回归分析-HW7

Prob1 KNNL5.4 KNNL5.12

5.4

5.12

Prob2 KNNL 5.23

2-a

2-b

2-c

2-d

Bonus Question

7 Matrix Approach to Linear Regression & Multiple Linear Regression

Multiple Linear Regression

Set Up

Estimation of σ2\sigma^{2}σ2

Sampling Distribution of bbb

Gauss-Markov Theorem

The Leverage

Covariance Matrix of eee

Expected Mean Squares and ANOVA F-Test

R2R^{2}R2 and Adjusted R2R^{2}R2

线性回归分析-HW6

Prob-1

Prob-2

2-a

2-b

2-c

Prob-3

3-a

3-b

3-c

3-d

3-e

3-f

Prob-4

4-a

4-b

4-c

4-d

4-e

CI-HW5

Problem 1 (SRE vs CRE based on Neyman’s estimator)

Problem 2 (PRE vs SRE based on Neyman’s estimator)

2-1

2-2

Problem 3

3-1

3-2

6 SRE and PRE

Stratified Randomized Experiment

Notations

Causal Estimands in SREs

Inference Approaches for SREs

Fisher's Exact P-Values

Neyman's Approach

SREs vs CREs

Regression Analysis of SREs

Without Interaction

Adding Interactions

Model-Based Analysis

Pairwise Randomized Experiments

Notations

Estimation of $\sigma^{2}$

Sampling Distribution of $b$

Covariance Matrix of $e$

$R^{2}$ and Adjusted $R^{2}$