GEORGIAN MATHEMATICAL JOURNAL: Vol. 2, No. 4, 1995, 335-346

ON OPTIMAL STOPPING OF INHOMOGENEOUS STANDARD MARKOV PROCESSES

B. DOCHVIRI

Abstract. The connection between the optimal stopping problems for inhomogeneous standard Markov process and the corresponding homogeneous Markov process constructed in the extended state space is established. An excessive characterization of the value-function and the limit procedure for its construction in the problem of optimal stopping of an inhomogeneous standard Markov process is given. The form of ε-optimal (optimal) stopping times is also found.

1. Introduction and Statement of the Problem

General questions of the theory of optimal stopping of homogeneous standard Markov processes are set forth in the monograph [1]. In various restrictions on the payoff function there are given an excessive characterization of the value, the methods of its construction, and the form of ε-optimal and optimal stopping times.

In the present work the questions of optimal stopping theory for inhomogeneous (with infinite lifetime) standard Markov processes are studied. By means of extension of the state space and the space of elementary events it is possible to reduce the problems of optimal stopping for the inhomogeneous case to the corresponding problems for homogeneous standard Markov processes from which the excessive characterization of the value-function, the method of its construction, and the form of ε-optimal (optimal) stopping times for the initial problem are found.

Note that the idea of reducing the inhomogeneous Markov process to the homogeneous one is wellknown. However, it is nontrivial to construct the appropriate homogeneous Markov process in the extended state space, to show the measurability of transition probabilities, and to prove the coincidence of the corresponding value-functions.

335

1991 Mathematics Subject Classification. 62L15, 93E20, 60J25.

Key words and phrases. ε-optimal stopping time, excessive function, inhomogeneous case, extension of state space, universal completion.

1072-947X/95/0700-0335$07.50/0 \begin{matrix}c \\ ◯ \end{matrix} 1995 Plenum Publishing Corporation

It should be noted that using the method of extension of state space in the paper [2], the form of ε-optimal stopping times has been established for the case of optimal stopping of homogeneous Markov processes on bounded time interval.

Consider now the inhomogeneous (with infinite lifetime) standard Markov process

X = (Ω, M^{s}, M_{t}^{s}, X_{t}, P_{s,x}), 0 ≤ s ≤ t < ∞,

in the state space (S, B), i.e., it is assumed that:

(1) S is a locally compact Hausdorff space with a countable base, B is the σ-algebra of Borel sets of the space;

(2) for every s ≥ 0, x ∈ S, P_{s,x} is a probability measure on the σ-algebra M^{s}; M_{t}^{s}, t ≥ s, is the increasing family of sub-σ-algebras of the σ-algebra M^{s}, where

M^{s_{1}} ⊇ M^{s_{2}}, M_{t}^{s} ⊆ M_{v}^{u}, for s_{1} ≤ s_{2}, u ≤ s ≤ t ≤ v,

it is assumed as well that

\overline{M}^{s} = M^{s}, \overline{M}_{t}^{s} = M_{t}^{s} = M_{t+}^{s}, 0 ≤ s ≤ t < ∞,

where \overline{M}^{s} is a completion of M^{s} with respect to the family of measures \{P_{u,x}, u ≤ s, x ∈ S\}, \overline{M}_{t}^{s} is the completion of M_{t}^{s} in \overline{M}^{s} with respect to the same family of measures ([3], Ch.I, Sect. 5);

(3) the paths of the process X = (X_{t}(ω)), t ≥ 0, are right continuous on the time interval [0, ∞);

(4) for each t ≥ 0 the random variables X_{t}(ω) (with values in (S, B)) are M_{t}^{s}-measurable, t ≥ s, where it is supposed that

P_{s,x}(ω : X_{s}(ω) = x) = 1

and the function P_{s,x}(X_{s+h} ∈ B) is measurable in (s, x) for the fixed h ≥ 0, B ∈ B (with respect to B[0, ∞) ⊗ B);

(5) the process X is strong Markov: for every (M_{t}^{s}, t ≥ s)-stopping time τ (i.e., {ω : τ(ω) ≤ t} ∈ M_{t}^{s}, t ≥ s) we should have

P_{s,x}(X_{τ+h} ∈ B|M_{τ}^{s}) = P (τ, X_{τ} , τ + h, B) ({τ < ∞}, P_{s,x}-a.s.),

where

P (s, x, s + h, B) ≡ P_{s,x}(X_{s+h} ∈ B);

(6) the process X is quasi-left-continuous: for every nondecreasing sequence of (M_{t}^{s}, t ≥ s)-stopping times τ_{n} ↑ τ should be

X_{τ_{n}} → X_{τ} ({τ < ∞}, P_{s,x}-a.s.).

Let g(t, x) be the Borel measurable function (i.e., measurable with respect to the product σ-algebra B′ = B[0, ∞) ⊗ B) which is defined on [0, ∞) × S and takes its values in (−∞, +∞].

Assume now the following integrability condition of the random process g(t, X_{t}(ω)), t ≥ 0:

E_{s,x} sup_{t≥s} g^{−}(t, X_{t}) < ∞, s ≥ 0, x ∈ S.  (1)

The problem of optimal stopping of the process

X = (Ω, M^{s}, M_{t}^{s}, X_{t}, P_{s,x}), 0 ≤ s ≤ t < ∞,

with the payoff g(t, x) is stated as follows: the value-function v(s, x) is introduced as

v(s, x) = sup_{τ∈𝔐_{s}} E_{s,x}g(τ, X_{τ} ),  (2)

where 𝔐_{s} is the class of all finite (P_{s,x}-a.s.) (M_{t}^{s}, t ≥ s)-stopping times and it is required to find the stopping time τ_{ε} (for each ε ≥ 0) for which

E_{s,x}g(τ_{ε}, X_{τ_{ε}}) ≥ v(s, x) − ε

for any x ∈ S.

Such a stopping time is called ε-optimal, and in the case of ε = 0 it is called simply optimal stopping time.

To construct ε-optimal (optimal) stopping times it is needed to characterize the value v(s, x) and for this purpose the following notion of excessive function turns out to be fundamental.

The function f(t, x) given on [0, ∞)×S and taking its values in (−∞, +∞] such that it is measurable with respect to the universal completion B^{′∗} of the σ-algebra B′ = B[0, ∞) ⊗ B, is called excessive (relative to X) if

1) E_{s,x}f^{−}(t, X_{t}) < ∞, 0 ≤ s ≤ t < ∞, x ∈ S,

2) E_{s,x}f(t, X_{t}) ≤ f(s, x), t ≥ s, x ∈ S,  (3)

3) E_{s,x}f(t, X_{t}) → f(s, x), if t ↓ s, x ∈ S.

2. Construction of the Homogeneous Standard Markov Process in the Extended State Space

Let us introduce now the new space of elementary events Ω′ = [0, ∞)×Ω with elements ω′ = (s, ω), the new state space (the extended state space) S′ = [0, ∞) × S with the σ-algebra B′ = B[0, ∞) ⊗ B, the new random process X′ with values in (S′, B′)

X′_{t}(ω′) = X′_{t}(s, ω) = (s + t, X_{s+t}(ω)), s ≥ 0, t ≥ 0,

and the translation operators Θ′_{t}:

Θ′_{t}(s, ω) = (s + t, ω), s ≥ 0, t ≥ 0,

where it is obvious that

X′_{u}(Θ′_{t}(ω′)) = X′_{u+t}(ω′), u ≥ 0, t ≥ 0.

Introduce in the space Ω′ the σ-algebra:

N^{0} = σ(X′_{u}, u ≥ 0), N_{t}^{0} = σ(X′_{u}, 0 ≤ u ≤ t)

and on the σ-algebra N^{0} introduce the probability measures

P′_{x′}(A) = P′_{(s,x)}(A) ≡ P_{s,x}(A_{s}),

where A ∈ N^{0}, and A_{s} is the section of A at the point s:

A_{s} = {ω : (s, x) ∈ A},

where it is easy to see that A_{s} ∈ F^{s} ≡ σ(X_{u}, u = s) and if a ∈ N_{t}^{0}, then A_{s} ∈ F_{s+t}^{s} ≡ σ(X_{u}, s ≤ u ≤ s + t).

Consider the function

P′(h, x′, B′) ≡ P′_{x′}(X′_{h} ∈ B′).

We have to verify that this function is measurable in x′ for fixed h ≥ 0. For the rectangles B′ = Γ × B which generate the σ-algebra B′ we have

P′(h, x′, B′) = P_{s,x}(ω : (s + h, X_{s+h}(ω)) ∈ Γ × B) =
= I_{(s+h∈Γ)}P_{s,x}(X_{s+h} ∈ B);

therefore for the rectangles the function P′(h, x′, B′) is measurable in x′. Consider now the class of all sets B′, B′ ∈ B′ for which the function P′_{x′}(X′_{h} ∈ B′) is B′-measurable in x′. It is easy to verify that this class of sets satisfies all the requirements of the monotone class theorem; therefore it coincides with the σ-algebra B′.

Thus the function P′(h, x′, B′) is measurable in x′, and hence we can introduce the measures P′_{μ′} on the σ-algebra N^{0} for every finite measure μ′ on (S′, B′) by averaging P′_{x′} with respect to μ′ [3]. Let us perform the completion of σ-algebra N^{0} with respect to the family of all measures P′_{μ′}, denote this completion by N′ and then perform the completion of each σalgebra N_{t}^{0} in N′ with respect to the same family of measures denoting them by N′_{t}.

The following key result (in a somewhat different form) was proved in the paper [2].

Theorem 1. The random process

X = (Ω′, N′, N′_{t}, X′_{t}, Θ′_{t}, P′_{x′}), t ≥ 0,

is a homogeneous standard Markov process in the space (S′, B′).

Proof. The main step in the proof is to verify that the process (Ω′, N^{0}, N_{t+}^{0}, X′_{t}, Θ′_{t}, P′_{x′}), t ≥ 0, is strong Markov, i.e., we have to show that

E′_{x′}[f′(X′_{τ′+h}) · I_{(τ′<∞)}] = E′_{x′}[M′_{X′_{τ′}} f′(X′_{h})I_{(τ′<∞)}],  (4)

where f′(x′) is an arbitrary bounded B′-measurable function and τ′ is an arbitrary N_{t+}^{0}-stopping time. Using again the monotone class theorem, it is clear that this relation suffices to be proved for the indicator functions

f′(x′) = I_{(s∈Γ)} · I_{(x∈B)}.

Thus it is needed to check that

E_{s,x}[I_{(s+τ′(s,ω)+h∈Γ)} · I_{(X_{s+τ′(s,ω)+h∈B})} ⋯ I_{(τ′(s,ω)<∞)}] =

= E_{s,x}[I_{(s+τ′(s,ω)+h∈Γ)}P_{u,y}(X_{u+h} ∈ B)|\begin{matrix}u=s+τ′(s,ω) \\ y=X_{s+τ′(s,ω)} \end{matrix} · I_{(τ′(s,x)<∞)}].

For this purpose we shall use the following

Lemma 1. If τ′(ω′) is an N_{t+}^{0}-stopping time, then τ(ω) = s + τ′(s, ω) is a (F_{t+}^{s}, t ≥ s)-stopping time, where F_{t}^{s} = σ(X_{u}, s ≤ u ≤ t), t ≥ s.

Proof. Indeed, we have

(ω : τ(ω) < t) = (ω : τ′(s, ω) < t − s) =
= (ω′ : τ′(ω′) < t − s)_{s},

but (ω′ : τ′(ω′) < t − s) ∈ N_{t−s}^{0}; therefore the section (ω′ : τ′(ω′) < t − s)_{s} belongs to F_{t}^{s}. Thus τ(ω) is a (F_{t+}^{s}, t ≥ s)-stopping time.

It follows from this Lemma that since F_{t+}^{s} ⊆ M_{t+}^{s} = M_{t}^{s}, the variable τ(ω) = s + τ′(s, ω) is a (M_{t}^{s}, t ≥ s)-stopping time. Hence the desired relation to be proved admits the following form:

E_{s,x}[I_{(τ+h∈Γ)} · I_{(X_{τ+h}∈B)} × I_{(τ<∞)}] =

= E_{s,x}[I_{(τ+h∈Γ)}P_{u,y}(X_{u+h} ∈ B)|\begin{matrix}u=τ \\ y=X_{τ} \end{matrix} · I_{(τ<∞)}],

which obviously is a consequence of the strong Markov property of the process X.

We know from Proposition 7.3, Ch. I in [3] that the strong Markov property (4) of the process X′ remains true for arbitrary N′_{t}, t ≥ 0-stopping times τ′ and from Proposition 8.12, Ch. I in [3] we get that N′_{t} = N′_{t+}. Quasi-left-continuity of the process X′ now easily follows from the same property of X with the help of Lemma 1. Theorem 1 is proved.

3. The Optimal Stopping Problem for Processes X and X′ and the Connection Between Them

Let f(x′) = f(s, x) be an arbitrary Borel measurable function (i.e., B′- measurable) which is given on S′ and takes its values in (−∞, +∞]. Consider the following sets:

A = {ω′ : lim_{t↓0} f(X′_{t}) = f(X′_{0})},

B = {ω′ : the path f(X′_{t}(ω′)) is right continuous on [0, ∞)}.

Obviously, sections A_{s} and B_{s} are of the following form:

A_{s} = {ω : lim_{t↓s} f(t, X_{t}(ω)) = f(s, X_{s}(ω))},

B_{s} = {ω : the path f(t, X_{t}(ω)) is right continuous on [s, ∞)}.

Lemma 2. The sets A and B belong to N^{0∗} (N^{0∗} is the universal completion of N^{0}) and sections A_{s} and B_{s} belong to F^{s∗} (F^{s∗} is the universal completion of F^{s} = σ(X_{u}, u ≥ s)).

Further we have

P′_{s,x}(A) = P_{s,x}(A_{s}), P′_{s,x}(B) = P_{s,x}(B_{s}).  (5)

Proof. The set A can be written as follows:

A = \{ω′ : lim_{k→∞} sup_{0<t< \frac{1}{k}} f(X′_{t}(ω′)) = lim_{k→∞} inf_{0<t< \frac{1}{k}} f(X′_{t}(ω′)) = f(X′_{0}(ω′))\}.

Since

\{ω′ : sup_{0<t< \frac{1}{k}} f(X′_{t}(ω′)) > a\} = pr_{Ω′} \{(t, ω′) : 0 < t < \frac{1}{k} , f(X′_{t}(ω′)) > a\},

\{ω′ : inf_{0<t< \frac{1}{k}} f(X′_{t}(ω′)) < a\} = pr_{Ω′} \{(t, ω′) : 0 < t < \frac{1}{k} , f(X′_{t}(ω′)) < a\},

we get from Theorem 13, Ch. III in [4] that the latter sets are N^{0}-analytic and hence they belong to the universal completion of N^{0}. Thus the set A itself belongs to N^{0∗}. As for the set B, we get from Theorem 34, Ch. IV in [4] that this set is the completion of the N^{0}-analytic set, hence B ∈ N^{0∗}. The same reasoning shows that A_{s} and B_{s} belong to the universal completion F^{s∗} of the σ-algebra F^{s}. For the measure P′_{s,x} and for the sets A and B belonging to the universal completion of N^{0} there obviously exist the sets A^{1}, A^{2}, B^{1}, B^{2} belonging to N^{0} such that

A^{1} ⊆ A ⊆ A^{2}, B^{1} ⊆ B ⊆ B^{2},
P′_{s,x}(A^{1}) = P′_{s,x}(A) = P′_{s,x}(A^{2}),
P′_{s,x}(B^{1}) = P′_{s,x}(B) = P′_{s,x}(B^{2}).

But by the definition of the measure P′_{s,x} we have

P′_{s,x}(A^{1}) = P′_{s,x}(A_{s}^{1}), P′_{s,x}(A^{2}) = P_{s,x}(A_{s}^{2}),
P′_{s,x}(B^{1}) = P′_{s,x}(B_{s}^{1}), P′_{s,x}(B^{2}) = P_{s,x}(B_{s}^{2}).

From these relations and the inclusions A_{s}^{1} ⊆ A_{s} ⊆ A_{s}^{2} and B^{1} ⊆ B ⊆ B^{2} it easily follows that (5) is true.

Now we return to the optimal stopping problem of the process X with the payoff function g(t, x) satisfying the integrability condition (1)

M_{s,x} sup_{t≥s} g^{−}(t, X_{t}) < ∞, s ≥ 0, x ∈ S.

We get from Lemma 2 that the set

A_{s} = {ω : lim_{t↓s} g(t, X_{t}(ω)) = g(s, X_{s}(ω))}

belongs to the universal completion F^{s∗} of the σ-algebra F^{s}, hence it has the probability measure

P_{s,x}{ω : lim_{t↓s} g(t, X_{t}) = g(s, X_{s})} = P_{s,x}{ω : lim_{t↓s} g(t, X_{t}) = g(s, x)}.

The second condition we need from the random process g(t, X_{t}), t ≥ 0, consists in the requirement that these probabilities should be equal to 1

P_{s,x}{ω : lim_{t↓s} g(t, X_{t}) = g(s, x)} = 1, s ≥ 0, x ∈ S.  (6)

Simultaneously with the problem of optimal stopping of the process X let us consider the problem of optimal stopping of the process

X′ = (Ω′, N′, N′_{t}, X′_{t}, Θ′_{t}, P′_{x′}), t ≥ 0

with the same payoff g(x′) = g(s, x) (x′ = (s, x)) satisfying the conditions

E′_{x′} sup_{t≥0} g^{−}(X′_{t}) < ∞, x′ ∈ S′,  (7)
P′_{x′}{ω′ : lim_{t↓0} g(X′_{t}) = g(x′)} = 1, x′ ∈ S′,  (8)

and with the value v′(x′) defined by

v′(x′) = sup_{τ′∈𝔐′} M′_{x′}g(X′_{τ′}),  (9)

where 𝔐′ is the class of all finite (P′_{x′}-a.s.) (N′_{t}, t ≥ 0)-stopping times.

Notice that condition (7) is exactly that of (1), and the same is true for conditions (8) and (6) by virtue of Lemma 2.

Observe that the condition (8) shows that the function g(x′) is finely continuous (relative to X′) but, as it is well known, the paths g(X′_{t}(ω′)) are then (P′_{x′}-a.s.) right continuous (Theorem 4.8, Ch. II in [3]), that is,

P′_{s,x}(B) = 1, x′ ∈ S′,

where

B = {ω′ : the path g(X′_{t}(ω′)) is right continuous on [0, ∞)}.

Now again by Lemma 2 we obtain

P_{s,x}{ω : the path g(t, X_{t}(ω)) is right
continuous on [s, ∞)} = 1, s ≥ 0, x ∈ S.  (10)

Thus we have obtained an interesting fact that condition (6) and condition (10) are equivalent.

Our next step consists in establishing the connection between the valuefunctions v(s, x) and v′(s, x).

Lemma 3. The values of the initial optimal stopping problem (9) coincide

v(s, x) = v′(s, x), s ≥ 0, x ∈ S.  (11)

Proof. Consider first the (N′_{t}, t ≥ 0)-stopping time τ′. By Proposition 7.3, Ch. I in [3] for τ′ and fixed x′ = (s, x) there exists (N_{t+}^{0}, t ≥ 0)-stopping time τ′ such that P′_{x′}(τ′ = τ′) = 1. We have

E′_{x′}g(X′_{τ′}) = E_{s,x}g(s + τ′(s, ω), X_{s+τ′(s,ω)}) =
= E_{s,x}g(τ(ω), X_{τ(ω)}),

where s + τ′(s, ω) ≡ τ(ω) is the (M_{t}^{s}, t ≥ s)-stopping time by Lemma 1. Whence it is obvious that

v′(s, x) ≤ v(s, x).

It remains to establish the opposite inequality. Denote by 𝔐_{s}^{n} the class of all (M_{t}^{s}, t ≥ s)-stopping times taking its values from the finite set

s, s + 2^{−n}, …, s + k · 2^{−n}, …, s + n.

Obviously,

𝔐_{s}^{n} ⊆ 𝔐_{s}^{n+1}, n = 1, 2, ….

For every τ ∈ 𝔐_{s} define the sequence τ_{n} of stopping times

τ_{n} = \begin{cases}s + k2^{−n}, & if s + (k − 1)2^{−n} ≤ τ < s + k2^{−n}, \\ s + n, & if τ ≥ s + n. \end{cases}

It is clear that τ_{n} ∈ 𝔐_{s}^{n} and starting from some n(ω) the sequence τ_{n}(ω) decreases to τ(ω). Using condition (10) and the right continuity of paths g(t, X_{t}(ω)), t ≥ s (P_{s,x}-a.s.), we can write

g(τ, X_{τ} ) = lim_{n→∞} g(τ_{n}, X_{τ_{n}}) (P_{s,x}-a.s.).

Hence by Fatou’s lemma we get

E_{s,x}g(τ, X_{τ} ) ≤ \underline{lim}_{n}E_{s,x}g(τ_{n}, X_{τ_{n}}).

Consequently

v(s, x) = sup_{τ∈∪_{n}𝔐_{s}^{n}} E_{s,x}g(τ, X_{τ} ) = lim_{n→∞} sup_{τ∈𝔐_{s}^{n}} E_{s,x}g(τ, X_{τ} ).

Consider separately the expression

sup_{τ∈𝔐_{s}^{n}} E_{s,x}g(τ, X_{τ} )

which represents the value in the problem of optimal stopping of the sequence

{g(s + k2^{−n}, X_{s+k2^{−n}}), M_{s+k2^{−n}}^{s}}, k = 0, 1, …, n2^{−n}.

It is wellknown that for this problem there always exists an optimal stopping time which has the following form:

σ_{n} = min {s + k2^{−n} : γ_{k}^{n} = g(s + k2^{−n}, X_{s+k2^{−n}}},

where the sequence γ_{k}^{n} is constructed recursively:

γ_{k}^{n} = max {g(s + k2^{−n}, X_{s+k2^{−n}}), M_{s,x}(γ_{k+1}^{n}|M_{s+k2^{−n}})}.

It easily follows from these recursion relations that γ_{k}^{n} is a Borel function of X_{s+k2^{−n}}. Therefore σ_{n} has the following form:

σ_{n} = min {s + k2^{−n} : X_{s+k2^{−n}} ∈ B_{k}^{n}},

where the sets B_{k}^{n} belong to the σ-algebra B (of course, B_{n2^{−n}}^{n} = S).

Thus we get

v(s, x) = lim_{n→∞} ↑ E_{s,x}g(σ_{n}, X_{σ_{n}}).

Define now the corresponding (N_{t}^{0}, t ≥ 0)-stopping times

σ′_{n} = min {k2^{−n} : X′_{k2^{−n}} ∈ [0, ∞) × B_{k}^{n}}.

We have

E′_{s,x}g(X′_{σ′_{n}}) = E_{s,x}g(X′_{σ′_{n}(s,ω)}(s, ω)) =
= E_{s,x}g(s + σ′_{n}(s, ω), X_{s+σ′_{n}(s,ω)}(ω)) = E_{s,x}g(σ_{n}, X_{σ_{n}})

as s + σ′_{n}(s, ω) = σ_{n}(ω). Therefore

E_{s,x}g(σ_{n}, X_{σ_{n}}) = E′_{s,x}g(X′_{σ′_{n}}) ≤ v′(s, x).

Thus v(s, x) ≤ v′(s, x), and finally v(s, x) = v′(s, x).

Our next purpose is the excessive characterization of the cost v(s, x). Let us note (as can be easily seen) that our definition of the excessive function (relative to X) coincides exactly with the usual definition of the excessive function (relative to X′). Therefore we can directly use Theorem 1, Ch. III in [1] and get the following result.

Theorem 2. Suppose that conditions (1) and (6) are satisfied. Then the value v(s, x) is the least excessive majorant of the function g(s, x). The value v(s, x) is the Borel measurable function, which can be found by the following limit procedure:

v(s, x) = lim_{n→∞} lim_{N→∞} Q_{n}^{N}g(s, x),  (12)

where

Q_{n}g(s, x) = max {g(s, x), E_{s,x}g(s + 2^{−n}, X_{s+2^{−n}})}

and Q_{n}^{N} is the Nth power of the operator Q_{n}.

Proof. The assertion is the consequence of the coincidence of the values v(s, x) and v′(s, x) and of Lemma 3, Ch. III in [3] which states that

v′(x′) = lim_{n→∞} lim_{N→∞} Q_{n}^{N}g(x′),

where

Q_{n}g(x′) = max {g(x′), E′_{x′}g(X′_{2^{−n}})}.

Note also that E′_{x′}g(X′_{2^{−n}}) is B′-measurable in x′, hence the functions Q_{n}g(x′), Q_{n}^{N}g(x′) and the function v′(x′), being the limit of these functions, are also B′-measurable.

Thus the value v′(x′) is the Borel measurable excessive function (relative to X′) which obviously satisfies the condition

E′_{x′} sup_{t≥0} v^{−}(X′_{t}) < ∞, x′ ∈ S′.

Then it is well-known (Theorem 2.12, Ch. II in [3]) that the paths v(X′_{t}(ω′)) are right continuous with the left-hand limits on [0, ∞) (P′_{x′}-a.s.). Using again Lemma 2 we obtain

P_{s,x}{ω : the path v(t, X_{t}(ω)) is right
continuous on [s, ∞)} = 1, s ≥ 0, x ∈ S.  (13)

To prove the main result of the present work we can now apply Theorem 3, Ch. III in [1].

Theorem 3. Let the payoff g(t, x) satisfy (relative to X) the following conditions:

(1) M_{s,x} sup_{t≥s} |g(t, X_{t})| < ∞, s ≥ 0, x ∈ S;

(2) P_{s,x}{ω : lim_{t↓s} g(t, X_{t}(ω)) = g(s, x) = 1}, s ≥ 0, x ∈ S.

Then

(i) for every ε > 0 the stopping times

τ_{ε} = inf {t ≥ s : v(t, X_{t}) ≤ g(t, X_{t}) + ε}  (14)

are ε-optimal; (ii) if the function g(t, x) is upper semi-continuous, that is,

g(s, x) ≥ \overline{lim}_{\begin{matrix}t→s \\ y→x \end{matrix}}g(t, y)

and the stopping time

τ_{0}(ω) = inf {t ≥ s : v(t, X_{t}) = g(t, X_{t})}  (15)

is finite (P_{s,x}-a.s.), then τ_{0}(ω) is the optimal stopping time.

Proof. From Theorem 3, Ch. III in [1] we know that for every ε > 0 the stopping time

τ′_{ε} = inf {t : v(X′_{t}) ≤ g(X′_{t}) + ε}

is ε-optimal:

E′_{x′}g(X′_{τ′_{ε}}) ≥ v(x′) − ε, x′ ∈ S′,

that is,

E_{s,x}g(s + τ′_{ve}(s, ω), X_{s+τ′_{ε}(s,ω)}(ω)) ≥ v(s, x) − ε.

But it is obvious that s + τ′_{ε}(s, ω) = τ_{ε}(ω), hence

E_{s,x}g(τ_{ε}, X_{τε}) ≥ v(s, x) − ε.

Assume now the upper semi-continuity of the function g(x′). Then from the same theorem we get again that the stopping time

τ′_{0} = inf{t ≥ 0 : v(X′_{t}) = g(x′_{t})}

is optimal:

E′_{x′}g(X′_{τ′_{0}}) = v(x′).

From this, similarly to the previous reasoning, we get the optimality of the stopping time τ_{0}(ω).

References

1. A. Shiryayev, Optimal stopping rules. Springer-Verlag, Berlin, 1978.

2. V. M. Dochviri and M. A. Shashiashvili, On optimal stopping of homogeneous Markov process on finite time interval. (Russian) Math. Nachr. 156(1992), 269–281.

3. R. M. Blumental and R. K. Getoor, Markov processes and potential theory. Academic Press, New York, 1968.

4. C. Dellacherie and P. Meyer, Probabilities and potential. North-Holland Math. Studies, 29, Amsterdam, 1978.

(Received 12.11.1993)

Author’s address: Laboratory of Stochastic Analysis and Statistical Decisions I. Javakhishvili Tbilisi State University 2, University St., Tbilisi 380043 Republic of Georgia