Two-Sample Welch's T-Test
Problem statement
Implement Welch's two-sample t-test, which asks whether two independent samples have different means without assuming equal variances.
Write two_sample_t_test(sample1, sample2, alpha=0.05) that returns a dictionary with:
t_statistic: $t = \dfrac{\bar X_1 - \bar X_2}{\sqrt{s_1^2/n_1 + s_2^2/n_2}}$, with $s_i^2$ the sample variances (divisor $n_i - 1$)degrees_of_freedom: the Welch-Satterthwaite value
$$\nu = \frac{\left(s_1^2/n_1 + s_2^2/n_2\right)^2}{\dfrac{(s_1^2/n_1)^2}{n_1 - 1} + \dfrac{(s_2^2/n_2)^2}{n_2 - 1}}$$ (a float, generally not an integer)
p_value: the two-sided p-value $P(|T_\nu| > |t|)$ from the Student t distribution with $\nu$ degrees of freedomreject_null:Trueifp_value < alphacohens_d: $(\bar X_1 - \bar X_2) / s_p$ with the pooled standard deviation $s_p = \sqrt{\dfrac{(n_1 - 1)s_1^2 + (n_2 - 1)s_2^2}{n_1 + n_2 - 2}}$
scipy is not available. The starter code provides _betainc(a, b, x), the regularized incomplete beta function $I_x(a, b)$, which gives the t-distribution tail through
$$P(|T_\nu| > |t|) = I_{\nu/(\nu + t^2)}\left(\tfrac{\nu}{2}, \tfrac{1}{2}\right)$$
Each sample has at least 2 values and nonzero variance.
Example
Input:
sample1=[12, 14, 13, 15, 14], sample2=[8, 9, 10, 9, 11], alpha=0.05
Output:
{'t_statistic': 5.8244, 'p_value': 0.000394, 'degrees_of_freedom': 8.0, 'reject_null': True, 'cohens_d': 3.6836}
Reasoning: Means are 13.6 and 9.4 and both sample variances are 1.3, so SE = sqrt(1.3/5 + 1.3/5) = 0.7211 and t = 4.2 / 0.7211 = 5.8244 with 8 degrees of freedom. The two-sided p-value 0.000394 is below 0.05, so we reject the null, and Cohen's d = 4.2 / sqrt(1.3) = 3.6836.
Show hint
Which test statistic compares two means while letting each group keep its own variance?
Relevant roles: Data Scientist, Applied Scientist, ML Engineer and Research Scientist.
Related free problems
- Factors That Do Not Affect Test PowerStatistics & InferenceEasy
- Multi-Head AttentionMedium
- Masked Self-AttentionMedium