Lobachevskii Journal of Mathematics

http://ljm.ksu.ru

Vol. 17, 2005, 11 – 23 \begin{matrix}c \\ ◯ \end{matrix} Md. A. Baten

Md. Azizul Baten

ON THE SMOOTHNESS OF SOLUTIONS OF LINEAR-QUADRATIC REGULATOR FOR DEGENERATE

DIFFUSIONS

(submitted by A. Lapin)

Abstract. The paper studies the smoothness of solutions of the degenerate Hamilton-Jacobi-Bellman (HJB) equation associated with a linear-quadratic regulator control problem. We establish the existence of a classical solution of the degenerate HJB equation associated with this problem by the technique of viscosity solutions, and hence derive an optimal control from the optimality conditions in the HJB equation.

1. Introduction

We are concerned with the quadratic control problem to minimize the expected cost with discount factor β > 0:

J(c) = E[∫_{0}^{∞} e^{−βt}{h(x_{t}) + |c_{t}|^{2}}dt]  (1)

over c ∈ A subject to the degenerate stochastic differential equation

dx_{t} = [Ax_{t} + c_{t}]dt + σx_{t}dw_{t}, x_{0} = x ∈ R, t ≥ 0,  (2)

Key words and phrases. Stochastic differential equation, Hamilton-Jacobi-Bellman equation, Linear-Quadratic problem, Viscosity solutions, Applications to control theory. 2000 Mathematical Subject Classification. 60H10, 49N10, 49J15, 49L25,

58E25.

for non-zero constants A, σ ≠ 0, and a continuous function h on R, where w_{t} is a one-dimensional standard Brownian motion on a complete probability space (Ω, F, P) endowed with the natural filtration F_{t} generated by σ(w_{s}, s ≤ t), and A denotes the class of all F_{t}−progressively measurable processes c = (c_{t}) with J(c) < ∞.

This kind of stochastic control problem has been studied by many authors [3, 6] for non-degenerate diffusions to (1) and (2). We also assume that h satisfies the following properties:

h(x) ≥ 0 : convex;  (3)
There exists C > 0 such that h(x) ≤ C(1 + |x|^{n}), xϵR, (4)

for some constant C > 0, n ≥ 2. We refer to [5] for the quadratic case of degenerate diffusions related to Riccati equations in case of h(x) = Cx^{2} and n = 2 with infinite horizon.

The purpose of this paper is to show the existence of a smooth solution u of the associated Hamilton-Jacobi-Bellman (in short, HJB) equation of the form:

−βu + \frac{1}{2}σ^{2}x^{2}u′′ + Axu′ + min_{r∈R}(r^{2} + ru′) + h(x) = 0 in R, (5)

and to give a synthesis of optimal control. Our method consists in finding the viscosity solution u of (5) [4, 6], by the limit of the solution v = v_{L}, L > 0, to the HJB equation

−βv_{L} + \frac{1}{2}σ^{2}x^{2}v′′_{L} + Axv′_{L} + min_{|r|≤L}(r^{2} + rv′_{L}) + h(x) = 0 in R, (6)

as L → ∞, and then in considering the smoothness of u by it’s convexity. To show the existence of the viscosity solution v_{L}, we assume that h has the following property: there exists C_{ρ} > 0, for any ρ > 0, such that

|h(x) − h(y)| ≤ C_{ρ}|x − y|^{n} + ρ(1 + |x|^{n} + |y|^{n}), ∀x, y ∈ R, (7)

for a fixed integer n ≥ 2.

This condition acts as the uniform continuity of h with order n, and plays an important role for the existence of viscosity solutions [7, 8]. We notice that (7) holds for h(x) = |x|n̄, n̄ ∈ [2, n],

In §2 we show that u(x) := lim_{L→∞} v_{L}(x) is a viscosity solution of (5), as L → ∞. §3 is devoted to the study of smoothness of u. Finally in §4 we present an optimal control to the optimization problem (1) and (2).

2. Convergences of the value function

In this subsection we show that v_{L}(x) is a viscosity solution of the Bellman equation (5) for any fixed L > 0, and then converges to a viscosity solution u(x) of the Bellman equation (5). In order to introduce solutions in the viscosity sense, given a continuous and degenerate elliptic map H : R × R × R × R → R, we recall by [4] the definition of viscosity solutions of

H(x, w, w′, w′′) = 0 in R.  (8)

Definition 2.1. w ∈ C(R) is called a viscosity subsolution (resp., supersolution) of (8) if, whenever for φ ∈ C^{2}(R), w − φ attains its local maximum (resp., minimum) at x ∈ R, then

H(x, w(x), φ′(x), φ′′(x)) ≤ 0  (9)
resp., H(x, w(x), φ′(x), φ′′(x)) ≥ 0.  (10)

We also call w ∈ C(R) a viscosity solution of (8) if it is both viscosity sub- and supersolution of (8). According to Crandall, Ishii and Lions [4] and Fleming and Soner [6] this definition is equivalent to the following: for any x ∈ R,

H(x, w(x), p, q) ≤ 0 for (p, q) ∈ J^{2,+}w(x)
H(x, w(x), p, q) ≥ 0 for (p, q) ∈ J^{2,−}w(x),

where J^{2,+} and J^{2,−} are the second-order superjets and subjets defined by J^{2,+}w(x)

= \{(p, q) ∈ R^{2} : lim sup_{y→x} \frac{w(y)−w(x)−p(y−x)− \frac{1}{2} q|y−x|^{2}}{|y−x|^{2}} ≤ 0\},
J^{2,−}w(x)
= \{(p, q) ∈ R^{2} : lim inf_{y→x} \frac{w(y)−w(x)−p(y−x)− \frac{1}{2} q|y−x|^{2}}{|y−x|^{2}} ≥ 0\}.

Let us define the value function v_{L}(x) = inf_{c∈A_{L}} J(c), where A_{L} = {c = (c_{t}) ∈ A : |c_{t}| ≤ L for all t ≥ 0}. We assume that there exists β_{0} ∈ (0, β) satisfying

−β_{0} + σ^{2}n(2n − 1) + 2n|A| < 0,  (11)

and we set f_{k}(x) = γ + |x|^{k} for any 2 ≤ k ≤ 2n and a constant γ ≥ 1 chosen later.

Lemma 2.2. Assume (11). Then there exist γ ≥ 1 and η > 0, depending on L, k, such that

−β_{0}f_{k} + \frac{1}{2}σ^{2}x^{2}f′′_{k} + Axf′_{k} + max_{|r|≤L}(r^{2} + rf′_{k}) + ηf_{k} ≤ 0  (12)

Further

E[∫_{0}^{τ} e^{−β_{0}s}ηf_{k}(x_{s})ds + e^{−β_{0}τ}f_{k}(x_{τ})] ≤ f_{k}(x) for 2 ≤ k ≤ 2n, (13)
E[sup_{t} e^{−β_{0}t}f_{k}(x_{t})] < ∞ for 2 ≤ k ≤ n,  (14)

where τ is any stopping time and x_{t} is the response to (c_{t}) ∈ A_{L}.

Proof. By (11), we choose η ∈ (0, β_{0}) such that

−β_{0} + \frac{1}{2}σ^{2}k(k − 1) + k|A| + η < 0,  (15)

and then γ ≥ 1 such that

(−β_{0} + \frac{1}{2}σ^{2}k(k − 1) + k|A| + η)|x|^{k} + Lk|x|^{k−1} + (L^{2} + ηγ − β_{0}γ) ≤ 0.

Then (12) is immediate. By (12) and Ito’s formula, we deduce (13). Moreover, by moment inequalities for martingales we get

E[sup_{t} e^{−β_{0}t}f_{k}(x_{t})] ≤ f_{k}(x) + E[sup_{t} | ∫_{0}^{t} e^{−β_{0}s}f′_{k}(x_{s})σx_{s}dw_{s}|]
≤ f_{k}(x) + KE[(∫_{0}^{∞} e^{−2β_{0}s}σ^{2}|x_{s}|^{2k}ds)^{1/2}],

for some constant K > 0. Therefore (14) follows from this relation together with (13).

Theorem 2.3. We assume (3), (4), (7) and (11). Then

v_{L} satisfies (3), (4), (7),  (16)

and the dynamic programming principle holds, i.e.,

v_{L}(x) = inf_{c∈A_{L}} E[∫_{0}^{τ} e^{−βt}{h(x_{t}) + |c_{t}|^{2}}dt + e^{−βτ}v_{L}(x_{τ})]  (17)

for any stopping time τ.

Proof. We suppress L of v_{L} for simplicity. The convexity of v follows from the same line as [5,Chap. 4, Lemma 10.6]. Let x_{t}^{0} be the unique solution of

dx_{t}^{0} = Ax_{t}^{0}dt + σx_{t}^{0}dw_{t},  x_{0}^{0} = x.  (18)

Then, by (13) and (4)

v(x) ≤ E[∫_{0}^{∞} e^{−βt}h(x_{t}^{0})dt] ≤ CE[∫_{0}^{∞} e^{−β_{0}t}f_{n}(x_{t}^{0})dt] ≤ Cf_{n}(x)/η. (19)

For the solution y_{t} of (2) with y_{0} = y, it is clear that x_{t} − y_{t} satisfies (18) with initial condition x − y. We note by (15) with k = n and Ito’s formula that

E[e^{−β_{0}t}|x_{t}^{0}|^{n}] ≤ |x|^{n}.

Thus by (7) and (13)

|v(x) − v(y)| ≤ sup_{c∈A_{L}} E[∫_{0}^{∞} e^{−βt}|h(x_{t}) − h(y_{t})|dt]

≤ sup_{c∈A_{L}} E[∫_{0}^{∞} e^{−βt}{C_{ρ}|x_{t} − y_{t}|^{n} + ρ(1 + |x_{t}|^{n} + |y_{t}|^{n})}dt]

≤ sup_{c∈A_{L}} ∫_{0}^{∞} e^{−βt}{C_{ρ}|x − y|^{n}e^{β_{0}t} + ρ(h_{n}(x) + h_{n}(y))e^{β_{0}t}}dt

≤ \frac{1}{β − β_{0}} [C_{ρ}|x − y|^{n} + 2ργ(1 + |x|^{n} + |y|^{n})] (20)

Therefore we get (16).

To prove (17), we denote by v^{r}(x) the right hand side of (17). By the formal Markov property

E[∫_{τ}^{∞} e^{−βt}{h(x_{t}) + |c_{t}|^{2}}dt|F_{τ}] = E[∫_{0}^{∞} e^{−β(t+τ)}{h(x_{τ+t}) + |c_{τ+t}|^{2}}dt|F_{τ}]
= e^{−βτ}Jc̃(x_{τ}),

with c̃ equal to c shifted by τ. Thus

J_{c}(x) = E[∫_{0}^{τ} + ∫_{τ}^{∞} e^{−βt}{h(x_{t}) + |c_{t}|^{2}}dt]
= E[∫_{0}^{τ} e^{−βt}{h(x_{t}) + |c_{t}|^{2}}dt]+E[ ∫_{τ}^{∞} e^{−βt}{h(x_{t}) + |c_{t}|^{2}}dt/F_{τ}]
≥ E[∫_{0}^{τ} e^{−βt}{h(x_{t}) + |c_{t}|^{2}}dt + e^{−βτ}v_{L}(x_{τ})].

It is known in [6, 9] that this formal argument can be verified, and we deduce v_{L}(x) ≥ v^{r}(x).

To prove the reverse inequality, let ρ > 0 be arbitrary. We set

V_{c}(x) = E[∫_{0}^{∞} e^{−βt}{h(x_{t}) + |c_{t}|^{2}}dt]  (21)

By the same calculation as (20), there exists C_{ρ} > 0 such that

|V_{c}(x) − V_{c}(y)| ≤ C_{ρ}|x − y|^{n} + ρ(1 + |x|^{n} + |y|^{n}).

16 MD. A. BATEN Take 0 < δ < 1 with C_{ρ}δ^{n} < ρ. Then, we have for |x − y| < δ,

|v(x) − v(y)| ≤ sup_{c∈A_{L}} |V_{c}(x) − V_{c}(y)|
≤ ρ(2 + |x|^{n} + |y|^{n})
≤ ρ[2 + |x|^{n} + 2^{n}(1 + |x|^{n})]
= ρ[(2 + 2^{n}) + (1 + 2^{n})|x|^{n}]
≤ Ξ_{ρ}(x) := ρ(2^{n} + 2)(1 + |x|^{n}).

Let {S_{i}} be a sequence of disjoint subsets of R such that

diam(S_{i}) < δ and ∪_{i} S_{i} = R.

For any i, we take x^{(i)} ∈ S_{i} and c^{(i)} ∈ A_{L} such that

V_{c^{(i)}}(x^{(i)}) ≤ inf_{c∈A_{L}} V_{c}(x^{(i)}) + ρ.

Define c^{τ} ∈ A_{L} by

c_{t}^{τ} = c_{t}1_{{t<τ}} + c_{t−τ}^{(i)}1_{{x_{τ} ∈S_{i}}}1_{{t≥τ}}, for x_{τ} ∈ S_{i}.

Hence,

V_{c^{(i)}}(x_{τ}) = V_{c^{(i)}}(x_{τ}) − V_{c^{(i)}}(x^{(i)}) + V_{c^{(i)}}(x^{(i)})
≤ Ξ_{ρ}(x_{τ}) + V_{c^{(i)}}(x^{(i)})
≤ Ξ_{ρ}(x_{τ}) + inf_{c∈A_{L}} V_{c}(x^{(i)}) + ρ
= Ξ_{ρ}(x_{τ}) + v(x^{(i)}) + ρ
≤ 2Ξ_{ρ}(x_{τ}) + v(x_{τ}) + ρ

Now, by the definition of v^{r}(x), we can find c ∈ A_{L} such that

v^{r}(x) + ρ ≥ E[∫_{0}^{τ} e^{−βt}{h(x_{t}) + |c_{t}|^{2}}dt] + e^{−βτ}v(x_{τ})]. Thus, using the formal Markov property [6], we have

v^{r}(x) + ρ

≥ ∑_{i} E[∫_{0}^{τ} e^{−βt}{h(x_{t}) + |c_{t}|^{2}}dt + e^{−βτ}(V_{c^{(i)}}(x_{τ}) − 2Ξ_{ρ}(x_{τ}) − ρ) : x_{τ} ∈ S_{i}]
= E[∫_{0}^{τ} e^{−βt}{h(x_{t}^{τ}) + |c_{t}^{τ}|^{2}}dt + ∫_{τ}^{∞} e^{−βt}{h(x_{t}^{τ}) + |c_{t}^{τ}|^{2}}dt|F_{τ}]
− 2E[e^{−βτ}Ξ_{ρ}(x_{τ})] − ρ
≥ v(x) − 2Ξ_{ρ}(x) − ρ,

where x_{t}^{τ} is the response to c_{t}^{τ} with x_{0}^{τ} = x_{τ}. Letting ρ → 0, we deduce v^{r}(x) ≥ v(x), which completes the proof.

Theorem 2.4. We assume (3), (4), (7) and (11). Then v_{L} is a viscosity solution of (5). Furthermore, v_{L} converges locally uniformly to a viscosity solution u ∈ C(R) of (6) satisfying (4), (7) as L → ∞.

Proof. We note that (13) gives E[∫_{0}^{h} |x_{t}|^{2}dt] ≤ e^{β_{0}h}hf_{2}(x) for h > 0, and

E[ sup_{0≤s≤h} |x_{s} − x|^{2}]

≤ 3^{2}(E[(∫_{0}^{h} |Ax_{t}|dt)^{2} + (∫_{0}^{h} |c_{t}|dt)^{2} + ( sup_{0≤s≤h} | ∫_{0}^{s} σx_{t}dw_{t}|)^{2}])
≤ 3^{2}(|A|^{2}hE[(∫_{0}^{h} |x_{t}|^{2}dt)] + h^{2}L^{2} + CE[∫_{0}^{h} |x_{t}|^{2}dt]).

for constant C > 0. Hence we have

lim_{h→0} sup_{c∈A_{L}} E[ sup_{0≤s≤h} |x_{s} − x|^{2}] = 0.

Thus we can apply a standard result of viscosity solutions [[4], Thm. 3.1, p. 220] to obtain the viscosity property of v_{L}, taking into account the uniform continuity of h on each compact interval. Since v_{L}(x) is nonincreasing, we can define u(x) by u(x) = lim_{L→∞} v_{L}(x). By Theorem 2.3, it is clear that u satisfies (4), (7). Thus by Dini’s theorem, we can observe the locally uniform convergence and the viscosity property of u [4]. The proof is complete.

3. Classical solutions

We here study the smoothness of the viscosity solution u of (5).

Proposition 3.1. We assume (3), (4), (7) and (11). Further we assume that h(x) : convex, then v_{L}(x) and u(x) are convex.

Proof. For any ϵ > 0, there exist c, ĉ ∈ A_{L} such that

E[∫_{0}^{∞} e^{−βt}{h(x_{t}) + |c_{t}|^{2}}dt] < v_{L}(x) + ϵ,
E[∫_{0}^{∞} e^{−βt}{h(x̂_{t}) + |ĉ_{t}|^{2}}dt] < v_{L}(x̂) + ϵ,

where

dx_{t} = [Ax_{t} + c_{t}]dt + σx_{t}dw_{t}, x_{0} = x ∈ R,
dx̂_{t} = [Ax̂_{t} + ĉ_{t}]dt + σx̂_{t}dw_{t}, x̂_{0} = x̂ ∈ R.

We set

c̃_{t} = ξc_{t} + (1 − ξ)ĉ_{t},
x̃_{t} = ξx_{t} + (1 − ξ)x̂_{t},
x̃_{0} = ξx + (1 − ξ)x̂ ≡ x̃,

for 0 < ξ < 1. Clearly,

dx̃_{t} = [Ax̃_{t} + c̃_{t}]dt + σx̃_{t}dw_{t}.

Hence, by convexity

v_{L}(x̃) ≤ E[∫_{0}^{∞} e^{−βt}{h(x̃_{t}) + |c̃_{t}|^{2}}dt]
≤ ξE[∫_{0}^{∞} e^{−βt}{h(x_{t}) + |c_{t}|^{2}}dt]
+(1 − ξ)E[∫_{0}^{∞} e^{−βt}{h(x̂_{t}) + |ĉ_{t}|^{2}}dt]
≤ ξ(v_{L}(x) + ϵ) + (1 − ξ)(v_{L}(x̂) + ϵ).

Letting ϵ ⟶ 0, we get

v_{L}(x̃) = v_{L}(ξx + (1 − ξ)x̂) ≤ ξv_{L}(x) + (1 − ξ)v_{L}(x̂),

which completes the convexity of v_{L}(x). From the definition of v_{L}(x), for each positive integer L, we have 0 ≤ v_{L+1}(x) ≤ v_{L}(x), x ∈ R. Since v_{L}(x) is non-increasing, we can define u(x) by u(x) = lim_{L→∞} v_{L}(x). Hence we see that u(x) is also convex.

Theorem 3.2. We assume (3), (4), (7) and (11). Then we have

u ∈ C^{2}(R \ {0}).  (22)

Proof. Step 1: By the convexity of u we recall a classical result of Alexandrov [6] to see that Lebesgue measure of R \ D ∪ {0} = 0, where

D = {x ∈ R : u is twice differentiable at x}. By the definition of

twice-differentiability, we have (u′(x), u′′(x)) ∈ J^{+2}u(x)∩J^{−2}u(x) for all x ∈ D, and hence

−βu + \frac{1}{2}σ^{2}x^{2}u′′ + Axu′ − \frac{(u′)^{2}}{4} + h(x) = 0, ∀x ∈ D.

Let d^{+}u(x) and d^{−}u(x) denote the right- and left-hand derivatives respectively. For all x ∈ (R \ {0}), define r^{±}(x) by

−βu(x) + \frac{1}{2}σ^{2}x^{2}r^{±}(x) + Axd^{±}u(x) − \frac{(d^{±}u(x))^{2}}{4} + h(x) = 0. (23)

Since d^{+}u = d^{−}u = u′ on D, we have r^{+} = r^{−} = u′′ a.e. By definition, d^{+}u(x) is right continuous, and so is r^{+}(x). Hence it is easy to see that

u(y) − u(x) = ∫_{x}^{y} d^{+}u(s)ds

d^{+}u(s) − d^{+}u(x) = ∫_{x}^{s} r^{+}(t)dt, s > x.

Thus we get

R(u; y) : = \{u(y) − u(x) − d^{+}u(x)(y − x) − \frac{1}{2}r^{+}(x)|y − x|^{2}\}/|y − x|^{2}
= ∫_{x}^{y} (d^{+}u(s) − d^{+}u(x) − r^{+}(x)(s − x))ds/|y − x|^{2} (24)
= ∫_{x}^{y} { ∫_{x}^{s} (r^{+}(t) − r^{+}(x))dt}ds/|y − x|^{2} ⟶ 0 as y ↓ x.

Step 2: We claim that u(x) is differentiable at x ∈ R \ D ∪ {0} = 0.

It is well known in [2] that δu(x) = [d^{+}u(x), d^{−}u(x)], for all x ∈

(R \ {0}), where δu(x) is the generalized gradient of u at x. Suppose d^{+}u(x) > d^{−}u(x). We set

p̂ = ξd^{+}u(x) + (1 − ξ)d^{−}u(x)
r̂ = ξr^{+}(x) + (1 − ξ)r^{−}(x), 0 < ξ < 1.

If lim inf_{y→x} R(u; y) < 0, then we can find a sequence y_{m} ⟶ x such that lim_{m→∞} R(u; y_{m}) < 0. By (24), we may consider that y_{m} ≤ y_{m+1} < x for every m, taking a subsequence if necessary. Hence

lim_{m→∞} \frac{u(y_{m}) − u(x) − d^{+}u(x)(y_{m} − x)}{|y_{m} − x|} ≤ 0,

this leads to d^{+}u(x) ≤ d^{−}u(x), which is a contradiction. Thus we have (d^{+}u(x), r^{+}(x)) ∈ J^{2,−}u(x) and similarly, (d^{−}u(x), r^{−}(x)) ∈ J^{2,−}u(x). By the convexity of J^{2,−}u(x), we get (p̂, r̂) ∈ J^{2,−}u(x). Now we note that

(p̂)^{2} < ξ(d^{+}u(x))^{2} + (1 − ξ)(d^{−}u(x))^{2},

and hence by (23)

−βu(x) + \frac{1}{2}σ^{2}x^{2}r̂ + Axp̂ − \frac{(p̂)^{2}}{4} + h(x) > 0.

On the other hand, by the definition of viscosity solution

−βu(x) + \frac{1}{2}σ^{2}x^{2}q + Axp − \frac{p^{2}}{4} + h(x) ≤ 0 ∀(p, q) ∈ J^{2,−}u(x),

which is a contradiction. Therefore we deduce that δu(x) is a singleton, and so u is differentiable at x [2]. Step 3: We claim that u′ is continuous on (R \ {0}). Let x_{m} ⟶ x and p_{m} = u′(x_{m}) ⟶ p. Then we have by convexity u(y) ≥ u(x) + p(y − x), for all y. Hence we see that p ∈ D^{−}u(x), where

D^{−}u(x) = {p ∈ R : lim inf_{y→x}{u(y) − u(x) − p(y − x)}/|y − x| ≥ 0}.

Since δu(x) = D^{−}u(x) and δu(x) is a singleton, we deduce p = u′(x) [[2], prop.4.7,p.66]. Step 4: We set w = u′. Since

−βw(x_{m}) + \frac{1}{2}σ^{2}x_{m}^{2}w′(x_{m}) + Ax_{m}w(x_{m}) − \frac{(w(x_{m}))^{2}}{4} + h(x_{m}) = 0,

where x_{m} ∈ D, the sequence {w′(x_{m})} converges uniquely as x_{m} ⟶ x ∈ R \ D ∪ {0}, and w is Lipschitz near x by monotonicity. Hence, we have a well-known result in nonsmooth analysis that δw(x) coincides with the convex hull of the set

D^{∗}w(x) = {q ∈ R : q = lim_{m→∞} w′(x_{m}), x_{m} ∈ D → x}.

Then

−βu(x) + \frac{1}{2}σ^{2}x^{2}q + Axw(x) − \frac{(w′(x))^{2}}{4} + h(x) = 0 ∀q ∈ δw(x).

Hence we observe that δw(x) is a singleton, and then w(x) is differentiable at x. The continuity of w′(x) follows immediately. Thus we conclude that w ∈ C^{1}(R \ {0}) and (R \ D ∪ {0}) is empty. The proof is complete.

Theorem 3.3. We assume (3), (4), (7) and (11). Further we assume that

h(x)/x^{2} → ĥ ∈ R_{+} as x → 0.  (25)

Then we have

u ∈ C^{1}(R) ∩ C^{2}(R \ {0}).  (26)

In addition, if ĥ = 0, then

u ∈ C^{2}(R).  (27)

Proof. We first observe that v_{L} is a viscosity solution of the boundary value problem:

V′′ + G(x, V, V′) = 0 in (a, b)  (28)
V (a) = v_{L}(a), V (b) = v_{L}(b),

for any interval [a, b] ⊂ R \ {0} where

G(x, V, V′) = 2{−αV + AxV′ + min_{|r|≤L}(|r|^{2} + rV′) + h(x)}/σ^{2}x^{2} = 0.

Standard elliptic regularity theory Fleming and Soner [[6], Thm. 4.1] and the uniqueness of viscosity solutions Crandall, Ishii and Lions [4] yield that v_{L} is smooth in (a, b). Thus

| min_{|r|≤L}(|r|^{2} + rv_{L}′)| ≤ | min_{r∈R}(|r|^{2} + rv_{L}′)| = (|v_{L}′|/2)^{2}
≤ {(|v_{L}′|/2)^{2} + 1}.

By the Theorem 3.2, we have u ∈ C^{2}(R \ {0}).

To prove (26), it suffices to show that u has the following property:

u′(x) = o(1) as x → 0.  (29)

By (25), there exists λ > 0, for any ε > 0 such that h(x) ≤ (ĥ + ε)x^{2} for |x| < λ, and hence, by (4)

h(x) ≤ (ĥ + ε)x^{2} + C(1/λ^{n} + 1)|x|^{n}, ∀x ∈ R.  (30)

Note that u(x) ≤ E[∫_{0}^{∞} e^{−βt}h(x_{t}^{0})dt]. Then we have by (13)

u′(x) = 0(x^{2}) as x → 0.  (31)

Now, by convexity

u(y) ≥ u(x) + u′(x)(y − x), x ≠ 0.

Substituting y = 2x, and y = 0 we get u(2x) ≥ u(x) + u′(x)x and u(x) − u′(x)x ≤ u(0) = 0 by (31). Hence

\frac{u(2x)}{x^{2}} ≥ \frac{u′(x)}{x} ≥ \frac{u(x)}{x^{2}},  (32)

which implies (29).

Finally, suppose ĥ = 0. Then, by virtue of (30), we have u(x) = o(x^{2}) as x → 0. Moreover, by (32), u′(x) = o(x) as x → 0. Dividing (5) by x^{2} and passing to the limit, we get u′′(0) = 0, which implies (27).

4. An application to quadratic control theory

We shall study the quadratic control problem (1) over the class A_{ad} of admissible controls, subject to (2), where A_{ad} = {c = (c_{t}) ∈ A : lim_{T →∞} E[e^{−βT} |x_{T} |^{n}] = 0 for the response x_{t} to c_{t}}. We consider the stochastic differential equation

dx_{t}^{∗} = [Ax_{t}^{∗} − u′(x_{t}^{∗})/2]dt + σx_{t}^{∗}dw_{t}, x_{0}^{∗} = x.  (33)

Theorem 4.1. We assume (3), (4), (7), (11) and (25). Then the optimal control c_{t}^{∗} is given by

c_{t}^{∗} = −u′(x_{t}^{∗})/2.  (34)

Proof. Since u′ is continuous, (33) admits a weak solution x_{t}^{∗} up to explosion time σ = inf{t : |x_{t}^{∗}| = ∞}. Taking into account xu′(x) ≥ 0, we can show (x_{t}^{∗})^{2} ≤ (x_{t}^{0})^{2} by the comparison theorem. Hence σ = ∞. By the monotonicity of u′(x), the uniqueness of (33) holds. Thus we conclude that (33) has a unique strong solution (x_{t}^{∗}).

It follows from (14) that

E[e^{−βT} (1 + |x_{T}^{∗} |^{n})] ≤ e^{−(β−β_{0})T} E[e^{−β_{0}T} f_{n}(x_{T}^{0} )] ⟶ 0 as T → ∞,

where x_{t}^{0} is a unique solution of (18). So (c_{t}^{∗}) ∈ A_{ad}. Since u satisfies (4), we see by (32) and (13) that

E[∫_{0}^{T} e^{−2βt}(x_{t}^{∗}u′(x_{t}^{∗}))^{2}dt] ≤ E[∫_{0}^{T} e^{−2βt}u(2x_{t}^{∗})^{2}dt]
≤ CE[∫_{0}^{T} e^{−2βt}(1 + |x_{t}^{∗}|^{2n})dt]
≤ CE[∫_{0}^{T} e^{−2βt}f_{2n}(x_{t}^{0})dt] < ∞,

and hence ∫_{0}^{t} e^{−βs}σx_{s}^{∗}u′(x_{s}^{∗})dw_{s} is a martingale. Then we apply Ito’s formula for convex functions [7,p.219] to obtain

E[e^{−βT} u(x_{T}^{∗})]
= u(x) + E[ ∫_{0}^{T} e^{−βt}( − βu + Axu′ + c_{t}^{∗}u′ + \frac{1}{2}σ^{2}x^{2}u′′)|_{x=x_{t}^{∗}}dt]
= u(x) − E[∫_{0}^{T} e^{−βt}{h(x_{t}^{∗}) + |c_{t}^{∗}|^{2}}dt].

Passing to the limit, we have J(c^{∗}) = u(x). By the same calculation as above , we can see that

E[e^{−βT ∧τ_{n}}u(x_{T ∧τ_{n}})] ≥ u(x) − E[∫_{0}^{T ∧τ_{n}} e^{−βt}{h(x_{t}) + |c_{t}|^{2}}dt],

where {τ_{n}} is a sequence of localizing stopping times for the local martingale. Letting τ_{n} ⟶ ∞ and then T ⟶ ∞, we obtain u(x) ≤ J(c) for all c ∈ A_{ad}. The proof is complete. General stochastic control problem: we can further study a stochastic control problem for linear degenerate systems to minimize the

LINEAR-QUADRATIC REGULATOR 23

discounted expected cost:

J(c) = E[∫_{0}^{∞} e^{−βt}{h(x_{t}) + |c_{t}|^{n}}dt]

over c ∈ A subject to the degenerate stochastic differential equation (2) and a continuous function f on R such that (4) and (7), in addition

k_{0}|x|^{n} − k_{1} ≤ h(x)

for some constants k_{0}, k_{1} > 0 and for a fixed integer n ≥ 2. Acknowledgements: I would like to express my sincere gratitude to Professor A.B.M. Abdus Sobhan Miah for his advice, support and encouragement.

References

[1] T. M. Apostol. Mathematical Analysis, Addison-Wesley, (1974).

[2] Bardi, M. and Capuzzo-Dolcetta, I. Optimal Control and Viscosity Solutions of Hamilton-Jacobi- Bellman Equations, Birkhäuser, Boston, (1997).

[3] Bensoussan, A. Stochastic Control by Functional Analysis Methods, North-Holland, Amsterdam, (1982).

[4] Crandall, M.G., Ishii, H. and Lions, P.L. User’s guide to viscosity solutions of second order partial differential equations. Bull. Amer. Math. Soc. 27, (1982),1-67.

[5] Da Prato, G. Direct solution of a Riccati equation arising in stochastic control theory. Appl. Math. Optim. 11, (1984),191-208.

[6] Fleming, W.H. and Soner, H.M. Controlled Markov Processes and Viscosity Solutions, Springer-Verlag, New York, (1993).

[7] S. Koike and M. Morimoto. On variational inequalities for leavable boundedvelocity control, Appl. Math. Optim., 18, (2003), pp.1-20.

[8] J.L. Menaldi and M. Robin. On some cheap control problems for diffusions process, Trans. Amer. Math. Soc. vol. 278, (1983), pp. 771-802.

[9] Nisio, M. Stochastic Control Theory, ISI Lecture Notes 9, MacMillan, (1981).

Department of Statistics, ShahJalal University of Science and Technology, Sylhet-3114, Bangladesh. E-mail address: baten math@yahoo.com

Received February 18, 2005