Resampling

Resampling refers to the act of repeatedly drawing samples from the original observed data sample for the statistical inference of population parameters. The two commonly used methods of resampling are bootstrap and jackknife.

Bootstrap

Using a computer, the bootstrap resampling method simulates drawing multiple random samples from the original sample. Each resample is the same size as the original sample. These resamples are used to create a sampling distribution.

In the bootstrap method, the number of repeated samples to be drawn is at the the researcher’s discretion. Note that bootstrap resampling is done with replacement.

Furthermore, we can calculate the standard error of the sample mean. This is done by resampling and calculating the mean of each sample. The following formula is used to estimate the standard error.

$$ s_{\bar{X}}=\sqrt{\frac{1}{B-1}\sum_{b=1}^{{B}}\left({\hat{\theta}}_b-\bar{\theta}\right)^2} $$

Where:

\(s_{\bar{X}}\) = Estimate of the standard error of the sample mean.

\(B\) = Number of resamples drawn from the original sample.

\({\hat{\theta}}_b\) = Mean of a resample.

\(\bar{\theta}\)= Mean across all the resample means.

The bootstrap resampling method can also be applied in estimating the confidence intervals for the statistic of other population parameters such as median.

Advantages of Bootstrap Resampling

1. No Reliance on Analytical Formulas: Bootstrap differs from traditional statistics because it doesn’t rely on an analytical formula for estimating distributions. This makes it versatile for complex estimators and especially useful when analytical formulas are unavailable.
2. Applicability to Complicated Estimators: Bootstrap is a simple yet powerful method that can handle complicated estimators effectively. It can handle a wide range of statistical models, making it suitable for various applications in finance where complex estimations are common.
3. Increased Accuracy:  Bootstrap can enhance accuracy by creating multiple resampled datasets and estimating population parameters on each. This helps understand estimator variability and robustness, ultimately improving result accuracy.

Jackknife

Jackknife is a resampling method in which samples are drawn by omitting one observation at a time from the original data sample. This process involves drawing samples without replacement. For a sample size of \(n\), we need \(n\) repeated samples. This method can be used to reduce the bias of an estimator or to estimate the standard error and the confidence interval of an estimator.

Question

Assume that you are studying the median height of 100 students in a university. You draw a sample of 1000 students and obtain 1000 median heights. The mean across all resample means is 5.8. The sum of squares of the differences between each sample mean, and the mean across all resample means \(\sum_{b=1}^{{B}}\left({\hat{\theta}}_b-\bar{\theta}\right)^2\) is 2.3.

The Estimate of the standard error of the sample mean is closest to:

1. 0.05.
2. 0.08.
3. 0.10.

Solution

The correct answer is A.

$$ \begin{align*}
s_{\bar{X}}&=\sqrt{\frac{1}{B-1}\sum_{b=1}^{{B}}\left({\hat{\theta}}_b-\bar{\theta}\right)^2} \\
&=\sqrt{\frac{1}{1000-1}\times 2.3}=0.04798\approx 0.05
\end{align*} $$