CI-HW1

HW1.
Official Solutions.

Problem 1 (Fisher's test)

a.

To ensure that there was no other factor that might influence the result.

b.

The R code follows:

seeded <- c(2745.6, 274.7, 115.3, 1697.1, 274.7, 92.4, 1656.4, 255.0,
            40.6, 978.0, 242.5, 32.7, 703.4, 200.7, 31.4, 489.1, 198.6,
            17.5, 430.0, 129.6, 7.7, 334.1, 119.0, 4.1, 302.8, 118.3)

unseeded <- c(1202.6, 87.0, 26.0, 830.1, 81.2, 24.4, 372.4, 68.5, 21.4,
              345.5, 47.3, 17.3, 321.2, 41.1, 11.5, 244.3, 36.6, 4.9,
              163.0, 29.0, 4.9, 147.8, 28.6, 1.0, 95.0, 26.3)
              
y <- c(seeded, unseeded)              # 52 observed daily rainfalls
n <- length(y)                        # 52 days
n_treat <- length(seeded)             # 26 seeded days

t_obs <- mean(seeded) - mean(unseeded)

B <- 1000
t_sim <- numeric(B)

for (b in seq_len(B)) {
  treated <- sample.int(n, n_treat)   # random assignment of 26 of 52 days
  t_sim[b] <- mean(y[treated]) - mean(y[-treated])
}

p <- mean(abs(t_sim) >= abs(t_obs))

cat("Observed difference in means (seeded - unseeded):", t_obs, "\n")
cat("Mean of simulated statistics under the null:     ", mean(t_sim), "\n")
cat("Simulated draws (B):                             ", B, "\n")
cat("p-value  P(|T_sim| >= |T_obs|):                  ", p, "\n")

Run results follow:

c.

The variance of our estimated p-value is $\frac{p(1-p)}{1000}$ where $p$ is the real p-value. One upper bound for the large-sample standard error is $\frac{1}{2\sqrt{1000}}\approx 0.0158$, which is on the same order of magnitude as the p-value we just estimated. So I would say the simulation is not very accurate.

d.

The R code follows:

seeded <- c(2745.6, 274.7, 115.3, 1697.1, 274.7, 92.4, 1656.4, 255.0,
            40.6, 978.0, 242.5, 32.7, 703.4, 200.7, 31.4, 489.1, 198.6,
            17.5, 430.0, 129.6, 7.7, 334.1, 119.0, 4.1, 302.8, 118.3)

unseeded <- c(1202.6, 87.0, 26.0, 830.1, 81.2, 24.4, 372.4, 68.5, 21.4,
              345.5, 47.3, 17.3, 321.2, 41.1, 11.5, 244.3, 36.6, 4.9,
              163.0, 29.0, 4.9, 147.8, 28.6, 1.0, 95.0, 26.3)

welch <- t.test(seeded, unseeded)                      # unequal variances (R default)
pooled <- t.test(seeded, unseeded, var.equal = TRUE)   # pooled / Student t-test

cat("Welch t-test:  t =", welch$statistic,
    ", df =", welch$parameter,
    ", p =", welch$p.value, "\n")
cat("Pooled t-test: t =", pooled$statistic,
    ", df =", pooled$parameter,
    ", p =", pooled$p.value, "\n\n")

The results are

It is close to the p-value in (b). After all they are both approximation to the real p-value, so they should be close to each other.

e.

Let

$$\begin{align} s_t^{2} & = \frac{1}{N_t-1}\sum_{i:W_i=1}\left( Y_i^{obs}-\bar{Y}_t^{obs} \right) ^{2} \\ s_c^{2} & = \frac{1}{N_c-1}\sum_{i:W_i=0}\left( Y_i^{obs}-\bar{Y}_c^{obs} \right) ^{2} \end{align}$$

I would propose $s_t^{2}-s_c^{2}$ as the test statistic.

Replace the code in part (b) t_obs <- mean(seeded) - mean(unseeded) with t_obs <- var(seeded) - var(unseeded). The results are

f.

Replace the code in part (b) t_obs <- mean(seeded) - mean(unseeded) with t_obs <- mean(log(seeded)) - mean(log(unseeded)). The results are

g.

Test statistic corresponding to the true model has more power.

Problem 2 (Another test statistic for Fisher’s test)

a.

First, it can be computed with the data at hand. Second, the statistic is sensitive to the null hypothesis because when it doesn't hold, the statistic is more likely to be larger or smaller compared to when it holds. Units that receive the treatment are more likely to rank higher or lower when there is treatment effect.

b.

$$\begin{align} \mathbb{E}(T_W) & = \sum_{i=1}^{n} R_i\mathbb{E}W_i \\ & = \frac{n_1}{n}\sum_{i=1}^{n} R_i \\ & = \frac{n_1}{n} \frac{n(n+1)}{2} \\ & = \frac{n_1(n+1)}{2} \end{align}$$

(Do we need to assume $Y_i\ne Y_j$ for different $i,j$ ?)

$$\begin{align} \mathbb{V}(T_W) & = \sum R_i^{2}\mathbb{V}(W_i)+\sum R_iR_j\mathrm{Cov}(W_i,W_j) \\ & = \frac{n_0n_1}{n^{2}}\sum R_i^{2}-\frac{n_0n_1}{n^{2}(n-1)}\sum_{i\ne j}R_iR_j \\ & = \frac{n_0n_1}{n}\sum R_i^{2}-\frac{n_0n_1}{n^{2}(n-1)}\left[ \left( \sum R_i \right)^{2}-\sum R_i^{2} \right] \\ & = \frac{n_0n_1}{n(n-1)}\sum R_i^{2} - \frac{n_0n_1}{n^{2}(n-1)}\left( \sum R_i \right) ^{2} \\ & = \cdots \\ & = \frac{n_0n_1(n+1)}{12} \end{align}$$

c.

seeded <- c(2745.6, 274.7, 115.3, 1697.1, 274.7, 92.4, 1656.4, 255.0,
            40.6, 978.0, 242.5, 32.7, 703.4, 200.7, 31.4, 489.1, 198.6,
            17.5, 430.0, 129.6, 7.7, 334.1, 119.0, 4.1, 302.8, 118.3)

unseeded <- c(1202.6, 87.0, 26.0, 830.1, 81.2, 24.4, 372.4, 68.5, 21.4,
              345.5, 47.3, 17.3, 321.2, 41.1, 11.5, 244.3, 36.6, 4.9,
              163.0, 29.0, 4.9, 147.8, 28.6, 1.0, 95.0, 26.3)
              
y <- c(seeded, unseeded)              # 52 observed daily rainfalls
r <- rank(y)
n <- length(y)                        # 52 days
n_treat <- length(seeded)             # 26 seeded days

t_obs <- sum(r[1:26]) - n_treat*(n+1)/2

B <- 1000
t_sim <- numeric(B)

for (b in seq_len(B)) {
  treated <- sample.int(n, n_treat)   # random assignment of 26 of 52 days
  t_sim[b] <- sum(r[treated]) - n_treat*(n+1)/2
}

p <- mean(abs(t_sim) >= abs(t_obs))

cat("Observed:                                        ", t_obs, "\n")
cat("Mean of simulated statistics under the null:     ", mean(t_sim), "\n")
cat("Simulated draws (B):                             ", B, "\n")
cat("p-value:                                         ", p, "\n")

The results are

d.

$z=\frac{T_W-\mathbb{E}(T_W)}{\sqrt{\mathbb{V}(T_W)}}=\frac{135}{54.64}=2.47$, the p-value is $\Phi(-2.47)+(1-\Phi(2.47))=0.0135$

e.

Wilcoxon statistic is more robust to extreme values.

Problem 3

1.

Unit	$D_i$	$D_{i-1}$	$Y_i(1,1)$	$Y_i(1,0)$	$Y_i(0,1)$	$Y_i(0,0)$
1	1	1	4%
2	1	1	2%
3	0	1			8%
4	1	0		9%
5	0	1			12%
6	0	0				13%
7	0	0				4%
8	1	0		1%

2.

(i)

The estimand: $\bar{Y}(1,1)-\bar{Y}(0,1)$.

The estimator: $\frac{\sum D_iD_{i-1}Y_i}{\sum D_iD_{i-1}}-\frac{\sum (1-D_i)D_{i-1}Y_i}{\sum (1-D_i)D_{i-1}}= -7\%$

(ii)

The estimand: $\bar{Y}(1,0)-\bar{Y}(0,0)$.

The estimator: $\frac{\sum D_i(1-D_{i-1})Y_i}{\sum D_i(1-D_{i-1})}-\frac{\sum (1-D_i)(1-D_{i-1})Y_i}{\sum (1-D_i)(1-D_{i-1})}= -3.5\%$

(iii)

The estimand: $[\bar{Y}(1,1)-\bar{Y}(0,1)]-[\bar{Y}(1,0)-\bar{Y}(0,0)]$.

The estimator: $\frac{\sum D_iD_{i-1}Y_i}{\sum D_iD_{i-1}}-\frac{\sum (1-D_i)D_{i-1}Y_i}{\sum (1-D_i)D_{i-1}}-\left[ \frac{\sum D_i(1-D_{i-1})Y_i}{\sum D_i(1-D_{i-1})}-\frac{\sum (1-D_i)(1-D_{i-1})Y_i}{\sum (1-D_i)(1-D_{i-1})} \right]= -3.5\%$

Problem 4

1.

$\delta_1=\delta_2$:

$Y(1)=[1,1,1]$, $Y(0)=[1,1,1]$

$\delta_1>\delta_2$:

$Y(1)=[1,100,101]$, $Y(0)=[4,5,11]$

$\delta_1<\delta_2$:

$Y(1)=[1,100,110]$, $Y(0)=[-100,5,6]$

2.

I think $\delta_2$ makes more sense because it represents the effect on a single unit while $\delta_1$ represents the difference between two populations.

Problem 5

1.

Question: Does taking a structured online course (vs. self-studying with free resources) improve students' exam scores in an introductory statistics class?

• Unit: Individual students enrolled in the course.
• Actions: Treatment = enroll in the structured online course; Control = self-study using free online materials.
• Potential Outcomes: $Y_i(1)$ = exam score if student i takes the course; $Y_i(0)$ = exam score if student i self-studies.
• Covariates: Prior GPA, math background, weekly study hours, whether the student works part-time.

2.

Partially reasonable. No-interference is plausible if students study independently, but could be violated if students discuss course content or study together, letting one student's treatment affect another's outcome. The "no hidden variations of treatment" part is also questionable, since course quality and self-study resource quality can vary across students.

3.

Recruit a pool of students planning to take intro statistics. Randomly assign half to the online course and half to self-study, controlling access to materials. Collect baseline covariates via a pre-survey, then record final exam scores for all participants.

4.

Since assignment is done by the researcher, this would be data from a randomized experiment, not an observational study — assuming compliance is enforced and randomization is properly implemented.