Lobachevskii Journal of Mathematics

http://ljm.ksu.ru

Vol. 18, 2005, 3–20 \begin{matrix}c \\ ◯ \end{matrix} S. E. Ahmed, D. Li, A. Rosalsky, and A. Volodin

S. Ejaz Ahmed, Deli Li, Andrew Rosalsky, and Andrei Volodin

ON THE ASYMPTOTIC PROBABILITY FOR THE DEVIATIONS OF DEPENDENT BOOTSTRAP MEANS

FROM THE SAMPLE MEAN

(submitted by F. G. Avkhadiev)

Abstract. In this paper, the asymptotic probability for the deviations of dependent bootstrap means from the sample mean is obtained without imposing any conditions on the joint distributions associated with the original sequence of random variables from which the dependent bootstrap sample is selected. The mild condition of stochastic domination by a random variable is imposed on the marginal distributions of the random variables in this sequence.

Dedicated to Professor Mushtari on the occasion of his 60th birthday

1. Introduction.

It is a great pleasure for us to contribute this paper in honour of Professor Daniar Khamidovich Mushtari on the occasion of his 60th birthday.

2000 Mathematical Subject Classification. 60F05, 60F25, 60G42. Key words and phrases. Dependent bootstrap; Bootstrap means; Asymp-

totic probability for deviations; Exponential inequalities; Strong law of large numbers; Stochastic domination.

The work of A. Volodin is supported by a grant from the Natural Sciences and Engineering Research Council of Canada.

The main focus of the present investigation is to obtain asymptotic results for the probability of the deviations of dependent bootstrap means from the sample mean.

The work on the validity of bootstrap estimators has received much attention in recent years due to a growing demand for the procedure, both theoretically and practically. As is mentioned in Mikosch (1994), the sample mean is fundamental for parameter estimation in statistics. Therefore, most of the recent literature on the bootstrap is devoted to statistics of this type. This literature is mainly concerned with bootstrap validity; that is, with showing that a statistic and its bootstrap version have the similar asymptotic distributional behaviour.

However, the limiting behaviour of bootstrap statistics is also of interest since it is by no means clear whether the bootstrap version of a consistent estimator is itself consistent. From our point of view, this explains the usefulness and impact on statistical inference of deviations from the sample means for “exogenously generated” bootstrap samples. Furthermore, asymptotic probabilities for the deviations of bootstrap means are a quite useful tool for the study of bootstrap moments. It is important to note that exponential inequalities are of practical use in establishing the strong asymptotic validity of bootstrap means.

We call the reader’s attention to the special issue of the journal Statistical Science (2003) Volume 18, Number 2 devoted to the Silver Anniversary of the Bootstrap, where the wide applications of the bootstrap procedure to diverse areas of statistics are discussed.

We begin with a brief discussion of results in the literature pertaining to a sequence of independent and identically distributed (i.i.d.) random variables and to the classical (independent) bootstrap of the mean. Let {X, X_{n}, n ≥ 1} be a sequence of i.i.d. random variables defined on a probability space (Ω, F, P). For ω ∈ Ω and n ≥ 1, let P_{n}(ω) = n^{−1} ∑_{i=1}^{n} δ_{X_{i}(ω)} denote the empirical measure and let {X̂_{n,j}^{(ω)}, 1 ≤ j ≤ m(n)} be i.i.d. random variables with law P_{n}(ω) where {m(n), n ≥ 1} is a sequence of positive integers. In other words, the random variables {X̂_{n,j}^{(ω)}, 1 ≤ j ≤ m(n)} result by sampling m(n) times with replacement from the n observations X_{1}(ω), ⋯, X_{n}(ω) such that for each of the m(n) selections, each X_{j}(ω) has probability n^{−1} of being chosen.

For each n ≥ 1, {X̂_{n,j}^{(ω)}, 1 ≤ j ≤ m(n)} is the so-called Efron (1979) independent bootstrap sample from X_{1}, ⋯, X_{n} with bootstrap sample size m(n). Let \overline{X}_{n}(ω) = \frac{1}{n} ∑_{j=1}^{n} X_{j}(ω) denote the sample mean of {X_{j}(ω), 1 ≤ j ≤ n}, n ≥ 1.

When X is nondegenerate and EX^{2} < ∞, Bickel and Freedman (1981) showed that for almost every ω ∈ Ω the central limit theorem (CLT)

n^{1/2} (\frac{1}{n} ∑_{j=1}^{n} X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω)) →^{d} N(0, σ^{2})

obtains. Here and below and σ^{2} = Var X. Note that by the Glivenko-Cantelli theorem P_{n}(ω) is close to L(X) for almost every ω ∈ Ω and all large n, and by the classical Lévy CLT

n^{1/2} (\frac{1}{n} ∑_{j=1}^{n} X_{j} − EX) →^{d} N(0, σ^{2}).

It follows that for almost every ω ∈ Ω, the bootstrap statistic

n^{1/2} (\frac{1}{n} ∑_{j=1}^{n} X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))

is close in distribution to that of

n^{1/2} (\frac{1}{n} ∑_{j=1}^{n} X_{j} − EX)

for all large n. This is the basic idea behind the bootstrap. See the pioneering work of Efron (1979) where this nice idea is made explicit and where it is substantiated with several important examples.

A strong law of large numbers (SLLN) was first proved by Athreya (1983) for bootstrap means from the classical bootstrap. Arenal-Gutiérrez, Matrán, and Cuesta-Albertos (1996) analyzed the work of Athreya (1983) and, by taking into account different growth rates for the bootstrap sample size m(n), they gave new and simple proofs of even more general results. They also provided examples that show that the sizes of resampling required by their results to ensure almost sure (a.s.) convergence are not far from optimal.

An article which is important for this paper is that of Mikosch (1994). He established a series of exponential inequalities (cf. Lemma 6 below) that are an important tool for deriving results on the consistency of the bootstrap mean. Based on these exponential inequalities, he proved an a.s. convergence result for bootstrap means (Theorem 1 below). Next, using the same exponential inequalities, the Baum - Katz / Erdös / Hsu - Robbins / Spitzer type complete convergence result for bootstrap means (Theorem 2 below) and a moment result for the supremum of normed bootstrap sums were established in Li, Rosalsky, and Ahmed (1999).

The following bootstrap counterpart to the Marcinkiewicz-Zygmund SLLN was obtained by Mikosch (1994), Proposition 3.3.

Theorem 1. Let {X_{n}, n ≥ 1} be a sequence of i.i.d. random variables and let 0 < α < 2. If m(n) ≡ n and

E|X_{1}|^{α} |log |X_{1}||^{α} < ∞,

then for almost every ω ∈ Ω

\frac{1}{n^{1/α}} ∑_{j=1}^{n} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω)) → 0 a.s.,

where {X̂_{n,j}^{(ω)}, 1 ≤ j ≤ m(n)} is Efron’s independent bootstrap sample from X_{1}, ⋯, X_{n}.

We note that the classical Efron independent bootstrap sample {X̂_{n,j}^{(ω)}, 1 ≤ j ≤ m(n), n ≥ 1} can of course be defined in the same manner even if the original sequence {X_{n}, n ≥ 1} is not comprised of independent or identically distributed random variables. The following result was proved by Li, Rosalsky, and Ahmed (1999), Theorem 2.1 and Remark 2.4.

Theorem 2. Let {X_{n}, n ≥ 1} be a sequence of pairwise i.i.d. random variables and let 0 < α < 2. If m(n) ≡ n and

E|X_{1}|^{α}| log |X_{1}||^{α} < ∞,

then for every real number q, every ϵ > 0, and almost every ω ∈ Ω

∑_{n=1}^{∞} n^{q}P \{\frac{1}{n^{1/α}} |∑_{j=1}^{n} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))| ≥ ϵ\} < ∞,

where {X̂_{n,j}^{(ω)}, 1 ≤ j ≤ m(n)} is Efron’s independent bootstrap sample from X_{1}, ⋯, X_{n}.

Remark 1. Taking q = 0, it follows from the Borel-Cantelli lemma and Theorem 2 that for almost every ω ∈ Ω,

\frac{1}{n^{1/α}} ∑_{j=1}^{n} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω)) → 0 a.s.

We also refer the reader to the recent expository paper by Csörgő and Rosalsky (2003) where a detailed and comprehensive survey of limit laws for bootstrap sums is given.

The notion of the dependent bootstrap procedure was introduced by Smith and Taylor (2001a and 2001b) for a sequence of i.i.d. random variables where some important properties were also established. However, the dependent bootstrap procedure can be defined as follows for an arbitrary sequence of random variables. Let {X_{n}, n ≥ 1} be a sequence of random variables (which are not necessarily independent or identically distributed) defined on a probability space (Ω, F, P). Let {m(n), n ≥ 1} and {k(n), n ≥ 1} be two sequences of positive integers such that m(n) ≤ nk(n) for all n ≥ 1. For ω ∈ Ω and n ≥ 1, the dependent bootstrap is defined to be the sample of size m(n), denoted {X̂_{n,j}^{(ω)}, 1 ≤ j ≤ m(n)}, drawn without replacement from the collection of nk(n) items made up of k(n) copies each of the sample observations X_{1}(ω), ⋯, X_{n}(ω).

This dependent bootstrap procedure is proposed as a procedure to reduce variation of estimators and to obtain better confidence intervals. We refer to Smith and Taylor (2001b) for details and where simulated confidence intervals are obtained to examine possible gains in coverage probabilities and interval lengths.

We may consider the dependent bootstrap procedure as a more general procedure than the classical Efron independent bootstrap. If we take k(n) = ∞ for all n ≥ 1, then the dependent bootstrap reduces to the classical Efron independent bootstrap. The main results presented in this paper do not require any assumptions on k(n); they are certainly true for the independent bootstrap as well.

Henceforth we let {X̂_{n,j}^{(ω)}, 1 ≤ j ≤ m(n)} denote the dependent bootstrap sample from X_{1}, ⋯, X_{n}.

The following result from Volodin, Ordóñez Cabrera, and Hu (2005) extends the above cited result of Li, Rosalsky, and Ahmed (1999) to the case of the dependent bootstrap. The content of Remark 1 also pertains to Theorem 3.

Theorem 3. Let {X_{n}, n ≥ 1} be a sequence of identically distributed (not necessary independent) random variables and let 0 < α < 2. If

E|X_{1}|^{α} |log |X_{1}||^{α} < ∞,

then for every real number q, every ϵ > 0, and almost every ω ∈ Ω

∑_{n=1}^{∞} n^{q}P \{\frac{1}{n^{1/α}} |∑_{j=1}^{n} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))| ≥ ϵ\} < ∞.

The initial objective of the investigation resulting in the present paper was only to extend the results of Hu, Ordóñez Cabrera, and Volodin (2005) on the SLLN to the dependent bootstrap of the mean. But we are even able to establish in Theorem 4 a more general result than Theorem 3. Notice that Theorem 3 has the moment assumption E|X_{1}|^{α}| log |X_{1}||^{α} < ∞ whereas our result has much more general moment assumption. The no independence condition in Theorem 3 is noteworthy and it also prevails in Theorem 4; the identical distributions assumption is relaxed in Theorem 4 to stochastic domination by a random variable.

The following notion is well known. We recall that a sequence of random variables {X_{n}, n ≥ 1} is stochastically dominated by a random variable X if there exists a constant C > 0 such that

P {|X_{n}| > t} ≤ CP {|X| > t}

for all t ≥ 0 and all n ≥ 1.

The main focus of this paper is to obtain in Theorem 4 asymptotic results for

P \{\frac{1}{n^{1/α}} |∑_{j=1}^{m(n)} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))| ≥ ϵ\}

as n → ∞ where ϵ > 0, 0 < α < 2, and {X̂_{n,j}^{(ω)}, 1 ≤ j ≤ m(n)} is the dependent bootstrap sample from X_{1}, ⋯, X_{n}. The sequence {X_{n}, n ≥ 1} is not necessary a sequence of independent random variables but it is assumed to be stochastically dominated.

2. Some general results on the dependent bootstrap

The results from this section are modifications, generalizations, or extensions of the results of Smith and Taylor (2001a and 2001b) for the dependent bootstrap from a sequence of random variables {X_{n}, n ≥ 1} which are not necessarily i.i.d. We note again that Smith and Taylor (2001a and 2001b) consider only the i.i.d. case. The results in this section are of general interest and play a role in establishing the asymptotic results discussed above.

The first proposition gives the joint distribution of the random variables in the dependent bootstrap sample. We need the following notation.

For ω ∈ Ω, n ≥ 1, and a real number x, denote

τ(x) = ∑_{j=1}^{n} I{X_{j}(ω) ≤ x} and μ(x) = ∑_{j=1}^{n} I{X_{j}(ω) > x}

where I(·) is the indicator function. Hence, τ(x) is the random variable that counts the number of observations X_{j}(ω), 1 ≤ j ≤ n, that are less than or equal to x, while μ(x) is the random variable that counts the number of observations X_{j}(ω), 1 ≤ j ≤ n, that are strictly greater than x. Certainly, τ(x) + μ(x) = n for every x.

For a finite sequence {x_{1}, x_{2}, ⋯, x_{m}} of real numbers, denote

{x_{(1)}, x_{(2)}, ⋯, x_{(m)}}

its nondecreasing rearrangement, that is x_{(1)} ≤ x_{(2)} ≤ ⋯ ≤ x_{(m)} and for any 1 ≤ j ≤ m there exists 1 ≤ i ≤ m such that x_{i} = x_{(j)}.

Proposition 1. For ω ∈ Ω, n ≥ 1, and a sequence {x_{1}, x_{2}, ⋯, x_{m(n)}} of real numbers:

1) If k(n)τ(x_{(j)}) ≥ j for all 1 ≤ j ≤ m(n), then

P \{X̂_{n,1}^{(ω)} ≤ x_{1}, ⋯, X̂_{n,m(n)}^{(ω)} ≤ x_{m(n)}\} = ∏_{j=1}^{m(n)} \frac{k(n)τ(x_{(j)}) − (j − 1)}{k(n)n − (j − 1)}.

If k(n)τ(x_{(j)}) < j for at least one 1 ≤ j ≤ m(n), then the above probability is 0.

2) If k(n)μ(x_{(i)}) ≥ m(n) − i for all 1 ≤ i ≤ m(n), then

P \{X̂_{n,1}^{(ω)} > x_{1}, ⋯, X̂_{n,m(n)}^{(ω)} > x_{m(n)}\} = ∏_{i=1}^{m(n)} \frac{k(n)μ(x_{(i)}) − (m(n) − i + 1)}{k(n)n − (m(n) − i + 1)}.

If k(n)μ(x_{(i)}) < m(n) − i for at least one 1 ≤ i ≤ m(n), then the above probability is 0.

Proof. Let π be the reordering of {1, 2, ⋯, m(n)} such that π(j) = i for x_{i} = x_{(j)}.

For the proof of the first statement of Proposition 1, note that

P {X̂_{n,1}^{(ω)} ≤ x_{1}, ⋯, X̂_{n,m(n)}^{(ω)} ≤ x_{m(n)}}
= P {X̂_{n,π(1)}^{(ω)} ≤ x_{(1)}, ⋯, X̂_{n,π(m(n))}^{(ω)} ≤ x_{(m(n))}}
= P {X̂_{n,π(1)}^{(ω)} ≤ x_{(1)}} × P {X̂_{n,π(2)}^{(ω)} ≤ x_{(2)}|X̂_{n,π(1)}^{(ω)} ≤ x_{((1))}} × ⋯

×P {X̂_{n,π(m(n))}^{(ω)} ≤ x_{(m(n))}|X̂_{n,π(1)}^{(ω)} ≤ x_{(1)}, ⋯, X̂_{n,π(m(n)−1)}^{(ω)} ≤ x_{(m(n)−1)}}

= ∏_{j=1}^{m(n)} \frac{k(n)τ(x_{(j)}) − (j − 1)}{k(n)n − (j − 1)}

if k(n)τ(x_{(j)}) ≥ j for all 1 ≤ j ≤ m(n). The second part of the first statement is obvious.

For the proof of the second statement of Proposition 1, note that

P {X̂_{n,1}^{(ω)} > x_{1}, ⋯, X̂_{n,m(n)}^{(ω)} > x_{m(n)}}

= P {X̂_{n,π(m(n))}^{(ω)} > x_{(m(n))}, ⋯, X̂_{n,π(1)}^{(ω)} > x_{(1)}}
= P {X̂_{n,π(m(n))}^{(ω)} > x_{(m(n))}}
×P {X̂_{n,π(m(n)−1)}^{(ω)} > x_{(m(n)−1)}|X̂_{n,π(m(n))}^{(ω)} > x_{((m(n)))}} × ⋯
×P {X̂_{n,π(1)}^{(ω)} > x_{(1)}|X̂_{n,π(m(n))}^{(ω)} > x_{(m(n))}, ⋯, X̂_{n,π(2)}^{(ω)} > x_{(2)}}

= ∏_{j=1}^{m(n)} \frac{k(n)μ(x_{(m(n)−j+1)}) − (j − 1)}{k(n)n − (j − 1)}

= ∏_{i=1}^{m(n)} \frac{k(n)μ(x_{(i)}) − (m(n) − i + 1)}{k(n)n − (m(n) − i + 1)}

if k(n)μ(x_{(i)}) ≥ m(n) − i for all 1 ≤ i ≤ m(n). The second part of the second statement is obvious.

Of course, the dependent bootstrap random variables {X̂_{n,j}^{(ω)}, 1 ≤ j ≤ m(n)} are indeed dependent. They obey the socalled negatively dependent property; this property will be established in Proposition 2. The concept of negatively dependent random variables was introduced by Lehmann (1966) as follows.

Random variables Y_{1}, Y_{2}, ⋯ are said to be negatively dependent if for each n ≥ 2, the following two inequalities hold:

P {Y_{1} ≤ y_{1}, ⋯, Y_{n} ≤ y_{n}} ≤ ∏_{i=1}^{n} P {Y_{i} ≤ y_{i}}

and

P {Y_{1} > y_{1}, ⋯, Y_{n} > y_{n}} ≤ ∏_{i=1}^{n} P {Y_{i} > y_{i}}

for every sequence {y_{1}, ⋯, y_{n}} of real numbers.

Proposition 2. For ω ∈ Ω and n ≥ 1, the dependent bootstrap random variables {X̂_{n,j}^{(ω)}, 1 ≤ j ≤ m(n)} are negatively dependent and exchangeable.

Proof. Let {x_{1}, x_{2}, ⋯, x_{m(n)}} be a sequence of real numbers. For the first inequality of the negative dependence property, we note that we only need to consider the case k(n)τ(x_{(j)}) ≥ j for all 1 ≤ j ≤ m(n). By Proposition 1(1)

P {X̂_{n,1}^{(ω)} ≤ x_{1}, ⋯, X̂_{n,m(n)}^{(ω)} ≤ x_{m(n)}}
= ∏_{j=1}^{m(n)} \frac{k(n)τ(x_{(j)}) − (j − 1)}{k(n)n − (j − 1)}

≤ ∏_{j=1}^{m(n)} \frac{k(n)τ(x_{(j)})}{k(n)n} = ∏_{j=1}^{m(n)} P \{X̂_{n,j}^{(ω)} ≤ x_{j}\}.

For the second inequality of the negative dependence property, we note that we only need to consider the case k(n)μ(x_{(i)}) ≥ m(n) − i for all 1 ≤ i ≤ m(n). By Proposition 1(2)

P {X̂_{n,1}^{(ω)} > x_{1}, ⋯, X̂_{n,m(n)}^{(ω)} > x_{m(n)}}
= ∏_{i=1}^{m(n)} \frac{k(n)μ(x_{(i)}) − (m(n) − i + 1)}{k(n)n − (m(n) − i + 1)}

≤ ∏_{i=1}^{m(n)} \frac{k(n)μ(x_{(i)})}{k(n)n} = ∏_{i=1}^{m(n)} P \{X̂_{n,i}^{(ω)} > x_{i}\}.

The exchangeability is obvious by Proposition 1.

3. Some technical lemmas

In this section we present six technical results that we will use in establishing the main result of this paper including its corollaries. Some of the lemmas are only generalizations and extensions of well-known results. For expository purposes we outline many of their proofs.

For the simplicity, by the log-function in this section we mean the natural logarithm function. The results can be easily generalized to any other logarithm function with base greater than one.

The first lemma is well known and trivial.

Lemma 1. Let {Y_{n}, n ≥ 1} be a sequence of negatively dependent random variables.

1) If {f_{n}, n ≥ 1} is a sequence of real functions all of which are monotone increasing (or all monotone decreasing), then {f_{n}(Y_{n}), n ≥ 1} is a sequence of negatively dependent random variables.

2) For every n ≥ 1, E( ∏_{j=1}^{n} Y_{j}) ≤ ∏_{j=1}^{n} E(Y_{j}) provided the expectations are

finite.

Proof. 1) Let f_{n}^{−1} denote the inverse function of f_{n}, n ≥ 1 and assume that all f_{n} are increasing. Then for any n ≥ 1 and all real y_{1}, ⋯, y_{n} we have by the definition of negative dependence that

P {f_{1}(Y_{1}) ≤ y_{1}, ⋯, f_{n}(Y_{n}) ≤ y_{n}}
= P {Y_{1} ≤ f_{1}^{−1}(y_{1}), ⋯, Y_{n} ≤ f_{n}^{−1}(y_{n})}
≤ ∏_{j=1}^{n} P {Y_{j} ≤ f_{j}^{−1}(y_{j})} = ∏_{j=1}^{n} P {f_{j}(Y_{j}) ≤ y_{j}}.

The second inequality also follows from the definition of negative dependence. The case of decreasing functions can be proved in the same manner.

2) Consider the expectation

E(∏_{j=1}^{n} Y_{j}) = ∫_{−∞}^{+∞} ⋯ ∫_{−∞}^{+∞} P {Y_{1} > y_{1}, ⋯, Y_{n} > y_{n}} ∏_{j=1}^{n} dy_{j}

≤ ∫_{−∞}^{+∞} ⋯ ∫_{−∞}^{+∞} ∏_{j=1}^{n} P {Y_{j} > y_{j}}dy_{j} = ∏_{j=1}^{n} E(Y_{j}).

The next lemma is in effect the special case a_{n} ≡ 1 of Theorem 2 of Adler and Rosalsky (1987) and we omit the proof. The random variables {Y_{n}, n ≥ 1} are not assumed to be independent.

Lemma 2. Let ϕ(t), t > 0, be a continuous function that is positive, strictly increasing and satisfying the condition ϕ(t) → ∞ as t → ∞. Put b_{n} = ϕ^{−1}(n), n ≥ 1, where ϕ^{−1}(t) is the inverse function of ϕ(t). Let {Y_{n}, n ≥ 1} be a sequence of random variables which is stochastically dominated by a random variable Y. If

∑_{n=k}^{∞} b_{n}^{−1} = O(kb_{k}^{−1}) and Eϕ(|Y |) < ∞,

then

\frac{1}{b_{n}} ∑_{j=1}^{n} Y_{j} → 0 a.s.

The third lemma deals with convergence of maxima of random variables and is a generalization of the Corollary to Theorem 3 of Barnes and Tucker (1977). Again, no assumption of independence is made.

Lemma 3. Let ψ(t), t ≥ 0 be an increasing function such that ψ(t) → ∞ as t → ∞ and let {b_{n}, n ≥ 1} be a sequence of positive numbers such that b_{n} = ψ^{−1}(n), n ≥ 1, , where ψ^{−1}(t) is the inverse function of ψ(t). Let {X_{n}, n ≥ 1} be a sequence of positive random variables which is stochastically dominated by a random variable X such that Eψ(CX) < ∞ for all C > 0. Then

\frac{1}{b_{n}} max_{1≤j≤n} X_{j} → 0 a.s.

Proof. We recall at the outset that for any random variable Y, the conditions E|Y | < ∞ and ∑_{n=1}^{∞} P {|Y | ≥ n} < ∞ are equivalent. Hence the assumption

Eψ (\frac{X}{ϵ} ) < ∞ for all ϵ > 0

is equivalent to

∑_{n=1}^{∞} P {X > ϵb_{n}} < ∞ for all ϵ > 0.

Then by the stochastic domination hypothesis and the Borel-Cantelli lemma X_{n}/b_{n} → 0 a.s. For arbitrary n ≥ k ≥ 2,

\frac{1}{b_{n}} max_{1≤j≤n} X_{j} ≤ \frac{1}{b_{n}} max_{1≤j≤k−1} X_{j} + \frac{1}{b_{n}} max_{k≤j≤n} X_{j}

≤ \frac{1}{b_{n}} max_{1≤j≤k−1} X_{j} + max_{k≤j≤n} X_{j}/b_{j} (since \{b_{n}, n ≥ 1\} is nondecreasing)
≤ \frac{1}{b_{n}} max_{1≤j≤k−1} X_{j} + sup_{j≥k} X_{j}/b_{j} → 0

as first n → ∞ and then k → ∞.

Unfortunately, it is not possible to find the inverse to the function ϕ(t) = t^{1/β}/ log^{γ} t, t > 0, β > 0, and γ > 0 in closed form. But the following lemma gives a good “approximation” to the inverse function.

Lemma 4. Let ϕ(t) = t^{1/β} log^{−γ/β} t and ψ(t) = t^{β} log^{γ} t, t ≥ e, β > 0, and 0 < γ < e. Then for any ϵ > 0 and for all sufficiently large t

β^{−γ}(1 − ϵ)t ≤ ψ(ϕ(t)) ≤ β^{−γ}t

and, consequently, for all sufficiently large t

β^{−γ}(1 − ϵ)ϕ^{−1}(t) ≤ ψ(t) ≤ β^{−γ}ϕ^{−1}(t).

Proof. Note that

ψ(ϕ(t)) = \frac{t}{β^{γ}} (1 − γ \frac{log log t}{log t})^{γ}

and

\frac{log log t}{log t} ↓ 0

for t ≥ e^{e} which can be established by the differentiation.

It follows from Lemma 4 that for a positive random variable Y, the conditions Eϕ^{−1}(Y) < ∞ and Eψ(Y) < ∞ are equivalent.

Lemma 5. Let β > 1 and b_{n} = n^{β} log^{−2} n, n ≥ 2, then

∑_{j=n}^{∞} b_{j}^{−1} = O(nb_{n}^{−1}).

Proof. Note that for all large n, letting C_{i} (1 ≤ i ≤ 4) denote positive constants,

∑_{j=n}^{∞} \frac{1}{b_{j}} = ∑_{j=n}^{∞} \frac{log^{2} j}{j^{β}} = ∑_{m=1}^{∞} ∑_{j=mn}^{(m+1)n−1} \frac{log^{2} j}{j^{β}}

≤ ∑_{m=1}^{∞} \frac{n log^{2}(mn)}{(mn)^{β}}

(since the sequence \{\frac{log^{2} j}{j^{β}} , j ≥ e^{2/β}\} is strictly decreasing)

= \frac{n}{n^{β}} ∑_{m=1}^{∞} (\frac{log^{2} m}{m^{β}} + \frac{2(log m)(log n)}{m^{β}} + \frac{log^{2} n}{m^{β}})
= \frac{n}{n^{β}} (C_{1} + C_{2} log n + C_{3} log^{2} n) (since β > 1)
≤ C_{4} \frac{n log^{2} n}{n^{β}} = C_{4} \frac{n}{b_{n}} .

The exponential inequality presented in the last lemma is the key tool used in establishing in Theorem 4 the asymptotic probability for the deviations of dependent bootstrap means from the sample mean. It is a dependent bootstrap analog of the Mikosch exponential inequality (Mikosch (1994), Lemma 5.1). We mention that this result was proved by Mikosch (1994) under the assumption that {X_{n}, n ≥ 1} is a sequence of i.i.d. random variables, for supremum (not partial sums) of bootstrap random variables, and for the independent bootstrap procedure.

Lemma 6. Let {a_{n}, n ≥ 1} and {h_{n}, n ≥ 1} be two sequences of positive real numbers and let {X_{n}, n ≥ 1} be a sequence of (not necessary independent or identically distributed) random variables. Then for ω ∈ Ω and n ≥ 1 such that h_{n}M_{n}(ω) < 1 the following inequality holds for all ϵ > 0:

P \{|∑_{j=1}^{m(n)} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))| ≥ ϵa_{n}\} ≤ 2 exp \{−ϵ\frac{h_{n}a_{n}}{m(n)} + \frac{h_{n}^{2}B_{n}(ω)}{2 (1 − h_{n}M_{n}(ω))}\} ,

where

M_{n}(ω) = \frac{1}{m(n)} max_{1≤j≤n} |X_{j}(ω) − \overline{X}_{n}(ω)|

and

B_{n}(ω) = \frac{1}{nm(n)} ∑_{j=1}^{n} (X_{j}(ω) − \overline{X}_{n}(ω))^{2} .

Proof. By the Markov inequality,

P \{|∑_{j=1}^{m(n)} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))| ≥ ϵa_{n}\}

= P \{|\frac{∑_{j=1}^{m(n)} X̂_{n,j}^{(ω)}}{m(n)} − \overline{X}_{n}(ω)| ≥ \frac{ϵa_{n}}{m(n)}\}

≤ exp\{−\frac{ϵh_{n}a_{n}}{m(n)} \}E exp \{h_{n} |\frac{∑_{j=1}^{m(n)} X̂_{n,j}^{(ω)}}{m(n)} − \overline{X}_{n}(ω)|\}

≤ exp\{−\frac{ϵh_{n}a_{n}}{m(n)} \}E exp \{h_{n} (\frac{∑_{j=1}^{m(n)} X̂_{n,j}^{(ω)}}{m(n)} − \overline{X}_{n}(ω))\}

+ exp\{−\frac{ϵh_{n}a_{n}}{m(n)} \}E exp \{−h_{n} (\frac{∑_{j=1}^{m(n)} X̂_{n,j}^{(ω)}}{m(n)} − \overline{X}_{n}(ω))\} .

We will estimate only the expectation in the first term of the last expression; the same bound is valid for the second expectation.

Now by Proposition 2 the dependent bootstrap random variables {X̂_{n,j}^{(ω)}, 1 ≤ j ≤ m(n)}, n ≥ 1, are negatively dependent and exchangeable. Hence, by Lemma 1(1) the random variables

\{exp \{\frac{h_{n}}{m(n)} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))\} , 1 ≤ j ≤ m(n)\}

are negatively dependent and identically distributed.

Therefore

E exp \{h_{n} (\frac{∑_{j=1}^{m(n)} X̂_{n,j}^{(ω)}}{m(n)} − \overline{X}_{n}(ω))\}

= E [ ∏_{j=1}^{m(n)} exp \{\frac{h_{n}}{m(n)} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))\}]

≤ ∏_{j=1}^{m(n)} E exp \{\frac{h_{n}}{m(n)} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))\} (by Lemma 1(2))
= [E exp \{\frac{h_{n}}{m(n)} (X̂_{n,1}^{(ω)} − \overline{X}_{n}(ω))\}]^{m(n)} (by identical distribution)
= [\frac{1}{n} ∑_{i=1}^{n} exp \{\frac{h_{n}}{m(n)} (X_{i}(ω) − \overline{X}_{n}(ω))\}]^{m(n)}
= [1 + \frac{1}{n} ∑_{i=1}^{n} (\frac{h_{n}^{2}}{2!m(n)^{2}} (X_{i}(ω) − \overline{X}_{n}(ω))^{2} + \frac{h_{n}^{3}}{3!m(n)^{3}} (X_{i}(ω) − \overline{X}_{n}(ω))^{3}

+ \frac{h_{n}^{4}}{4!m(n)^{4}} (X_{i}(ω) − \overline{X}_{n}(ω))^{4} + ⋯)]^{m(n)}

= [1 + \frac{h_{n}^{2}}{m(n)} ∑_{i=1}^{n} \frac{(X_{i}(ω) − \overline{X}_{n}(ω))^{2}}{nm(n)} [\frac{1}{2!} + \frac{h_{n}}{3!} (\frac{X_{i}(ω) − \overline{X}_{n}(ω)}{m(n)})

+ \frac{h_{n}^{2}}{4!} (\frac{X_{i}(ω) − \overline{X}_{n}(ω)}{m(n)})^{2} + ⋯]]^{m(n)}

≤ [1 + \frac{h_{n}^{2}}{m(n)} \frac{B_{n}(ω)}{2} (1 + h_{n}M_{n}(ω) + (h_{n}M_{n}(ω))^{2} + ⋯)]^{m(n)}
= [1 + \frac{h_{n}^{2}}{2m(n)} \frac{B_{n}(ω)}{(1 − h_{n}M_{n}(ω))}]^{m(n)} (since h_{n}M_{n}(ω) < 1)
≤ [exp \{\frac{h_{n}^{2}}{2m(n)} \frac{B_{n}(ω)}{(1 − h_{n}M_{n}(ω))}\}]^{m(n)}
= exp \{\frac{h_{n}^{2}B_{n}(ω)}{2 (1 − h_{n}M_{n}(ω))}\} .

Hence,

P \{|∑_{j=1}^{m(n)} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))| ≥ ϵa_{n}\} ≤ 2 exp \{−\frac{ϵh_{n}a_{n}}{m(n)} + \frac{h_{n}^{2}B_{n}(ω)}{2(1 − h_{n}M_{n}(ω))}\} .

4. The main result

With the preliminaries accounted for, we can formulate and prove the main result of this paper, that is the asymptotic probability for the deviations of dependent bootstrap means from the sample mean. We emphasize that there are no independence or identical distribution assumptions on the original sequence of random variables {X_{n}, n ≥ 1}.

Theorem 4. Let ψ(t), t ≥ 0 be an increasing function such that

∑_{j=n}^{∞} \frac{1}{(ψ^{−1}(j))^{2}} = O (\frac{n}{(ψ^{−1}(n))^{2}} ) , n ≥ 1,  (∗)

where ψ^{−1}(t) is the inverse function of ψ(t). Let {X_{n}, n ≥ 1} be a sequence of random variables which is stochastically dominated by a random variables X such that Eψ(CX) < ∞ for all C > 0 and let {a_{n}, n ≥ 1} be a sequence of positive constants. Then for almost every ω ∈ Ω, for every ϵ > 0, and for every real number r,

P \{|∑_{j=1}^{m(n)} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))| ≥ ϵa_{n}\} = O (exp \{−r\frac{a_{n}}{ψ^{−1}(n)} + \frac{m(n)}{n}o(1)\}) .

Proof. Fix the arbitrary constants r and ϵ > 0 and let h_{n} = \frac{rm(n)}{ϵψ^{−1}(n)}, n ≥ 1. We may assume that r > 0. The fact that

h_{n}M_{n} ≤ \frac{r}{ϵ} \frac{2}{ψ^{−1}(n)} max_{1≤j≤n} |X_{j}| → 0 a.s.

follows directly from Lemma 3.

Next, in Lemma 2 consider Y_{j} = X_{j}^{2}, Y = X^{2}, and ϕ(t) = ψ(\sqrt{t}). Then ϕ^{−1}(n) = (ψ^{−1}(n))^{2} and Eϕ(Y) = Eψ(X) < ∞. By Lemma 2

h_{n}^{2}B_{n} = \frac{r^{2}}{ϵ^{2}} \frac{m(n)}{n} \frac{1}{(ψ^{−1}(n))^{2}} ∑_{j=1}^{n} X_{j}^{2} = \frac{m(n)}{n}o(1) a.s.

Hence,

\frac{h_{n}^{2}B_{n}}{2(1 − h_{n}M_{n})} = \frac{m(n)}{n}o(1) a.s.

We also note that

ϵ\frac{h_{n}a_{n}}{m(n)} = r\frac{a_{n}}{ψ^{−1}(n)}, n ≥ 1.

The result then follows directly from Lemma 6.

Remark 2. The conclusion of Theorem 4 is of course stronger the larger r is taken. The constant r does not play a role in any assumptions and it can be taken to be arbitrary large.

Using different moment assumptions, we can now derive different results on the asymptotic probability for the deviations of dependent bootstrap means from the sample mean.

Corollary 1. Let {X_{n}, n ≥ 1} be a sequence of random variables which is stochastically dominated by a random variable X and let 0 < α < 2. If

E|X|^{α} < ∞,

then for almost every ω ∈ Ω and every ϵ > 0

P \{\frac{1}{n^{1/α}} |∑_{j=1}^{n} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))| ≥ ϵ\} = o(1);

that is, for almost every ω ∈ Ω the weak law of large numbers

\frac{1}{n^{1/α}} ∑_{j=1}^{n} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω)) → 0 in probability

obtains.

Proof. Let ψ(t) = t^{α}, t > 0. Then ψ^{−1}(n) = n^{1/α}, n ≥ 1. The relation (*) holds trivially since 2/α > 1. If we take a_{n} = n^{1/α} and m(n) = n, n ≥ 1, then according to Theorem 4 for almost every ω ∈ Ω, for every ϵ > 0 and every r > 0, and for all sufficiently large n and some constant C < ∞,

P \{\frac{1}{n^{1/α}} |∑_{j=1}^{n} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))| ≥ ϵ\} ≤ C exp\{−r\};

that is, since r > 0 is arbitrary

\frac{1}{n^{1/α}} ∑_{j=1}^{n} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω)) → 0 in probability.

Corollary 2. Let {X_{n}, n ≥ 1} be a sequence of random variables which is stochastically dominated by a random variable X and let 0 < α < 2.

If

E|X|^{α}| log |X||^{α} < ∞,

then for every ϵ > 0, every real number r, and almost every ω ∈ Ω

P \{\frac{1}{n^{1/α}} |∑_{j=1}^{n} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))| ≥ ϵ\} = O(n^{−r}).

Proof. Let ψ(t) = t^{α} log^{α} t, t ≥ 1. Then according to Lemma 4 the sequence ψ^{−1}(n) is equivalent to \frac{n^{1/α}}{log n}, n ≥ 2. The relation (*) holds by Lemma 5 since 2/α > 1. For fixed r, ϵ > 0, m(n) = n, and a_{n} = n^{1/α}, n ≥ 1, applying Theorem 4 we obtain the result.

Remark 3. Theorem 3 easily follows from Corollary 2. To see this, for any constant q from Theorem 3, let r = q + 2 and apply Corollary 2.

Corollary 3. Let {X_{n}, n ≥ 1} be a sequence of random variables which is stochastically dominated by a random variable X and let 0 < α < 2. If

E|X|^{δ} < ∞

for some α < δ < 2, then for every ϵ > 0, every r, and almost all ω ∈ Ω

P \{\frac{1}{n^{1/α}} |∑_{j=1}^{n} (X̂_{n,j}^{(ω)} − \overline{X}_{n}(ω))| ≥ ϵ\} = O (exp\{−rn^{\frac{1}{α} − \frac{1}{δ}} \}) .

Proof. Let ψ(t) = t^{δ}, t > 0, then ψ^{−1}(n) = n^{1/δ}. The relation (*) holds trivially since 2/δ > 1. For fixed r, ϵ > 0, m(n) = n, and a_{n} = n^{1/α}, n ≥ 1, applying Theorem 4 we obtain the result.

References

[1] Adler, A., Rosalsky, A. Some general strong laws for weighted sums of stochastically dominated random variables. Stochastic Anal. Appl. 5, (1987), 1-16.

[2] Arenal-Gutiérrez, E., Matrán, C., Cuesta-Albertos, J.A. On the unconditional strong law of large numbers for the bootstrap mean. Statist. Probab. Lett. 27, (1996), 49-60.

[3] Athreya, K.B. Strong law for the bootstrap. Statist. Probab. Lett. 1, (1983), 147-150.

[4] Barnes, G.R., Tucker, H.G. On almost sure convergence of normed maxima of independent random variables. J. London Math. Soc. (2), 16, (1977), 377-383.

[5] Bickel, P.J., Freedman, D.A. Some asymptotic theory for the bootstrap. Ann. Statist. 9, (1981), 1196-1217.

[6] Csörgő, S., Rosalsky, A. A survey of limit laws for bootstrapped sums. Internat. J. Math. & Math. Sci. 2003, (2003), 2835–2861.

20 S. E. AHMED, D. LI, A. ROSALSKY, AND A. VOLODIN

[7] Efron, B. Bootstrap methods: another look at the jackknife. Ann Statist. 7, (1979), 1-26.

[8] Hu, T.C. Strong consistencies of the bootstrap moments. Internat. J. Math. & Math. Sci. 14, (1991), 797-802.

[9] Hu, T.-C., Ordóñez Cabrera, M., Volodin, A. Asymptotic probability for the bootstrapped means deviations from the sample mean. To appear in Statist. Probab. Lett. (2005).

[10] Lehmann, E. Some concepts of dependence. Ann. Math. Statist., 37, (1966), 1137-1153.

[11] Li, D., Rosalsky, A., Ahmed, S. E. Complete convergence of bootstrapped means and moments of the supremum of normed bootstrapped sums. Stochastic Anal. Appl. 17, (1999), 799-814.

[12] Mikosch, T. Almost sure convergence of bootstrapped means and U-statistics. J. Statist. Plann. Inference. 41, (1994), 1-19.

[13] Smith, W.D., Taylor, R.L. Consistency of dependent bootstrap estimators. Statistics, combinatorics and related areas (Mobile, AL, 1999). Amer. J. Math. Management Sci., 21 no. 3-4, (2001a), 359-382.

[14] Smith, W.D., Taylor, R.L. Dependent bootstrap confidence intervals. Selected Proceedings of the Symposium on Inference for Stochastic Processes (Athens, GA, 2000). IMS Lecture Notes - Monograph Series, 37, (2001b), 91-107.

[15] Volodin, A., Ordóñez Cabrera M., Hu T.-C. Convergence rate of the dependent bootstrapped means. To appear in Theory Probab. Appl. 50, (2005).

Department of Mathematics and Statistics, University of Windsor, Windsor, Ontario, Canada N9B 3P4 E-mail address: seahmed@uwindsor.ca

Department of Mathematics and Statistics, Lakehead University, Thunder Bay, Ontario, Canada P7B 5E1 E-mail address: dli@sleet.lakeheadu.ca

Department of Statistics, University of Florida, Gainesville, Florida 32611 USA E-mail address: rosalsky@stat.ufl.edu

Department of Mathematics and Statistics, University of Regina, Regina, Saskatchewan, Canada S4S 0A2 E-mail address: andrei@math.uregina.ca

Received May 5, 2005