Journal of Lie Theory Volume 5 (1995) 225–240 C ◯1995 Heldermann Verlag

Laplace transform and unitary highest weight modules

Jean-Louis Clerc

Communicated by K.-H. Neeb

Abstract. The unitarizable modules in the analytic continuation of the holomorphic discrete series for tube type domains are realized as Hilbert spaces obtained through the Laplace transform.

0. Introduction

Let G be a connected real semi-simple Lie group with finite center, U a maximal compact subgroup, and assume G/U is a Hermitian symmetric space. Harish Chandra constructed a family of irreducible unitary representations of G, called the holomorphic discrete series, realized on holomorphic sections of some vector bundles over G/U, square-integrable with respect to the invariant measure on G/U.

The vector bundles are associated to irreducible (finite-dimensional) representations of U, with some restriction on the dominant weight of the representation in order to have non-trivial L^{2} -sections. However the formulae for the action of G make sense for all values of the dominant weight, and Harish Chandra indicated the possibility that some of these modules (or some submodules) might be unitarizable ([10]).

This problem was completely solved ([5],[11]), but the proofs are of algebraic nature and use case-by-case arguments. Moreover there is no concrete realization of the corresponding Hilbert spaces. For more recent work in this direction, see [2], [6], [12]. A few years earlier the special case of line bundles (associated to characters of U) had been studied, both from the algebraic point of view [19] and the analytic counterpart [17]. We follow here the second approach. To avoid complications, we restrict our attention to tube type domains. We obtain a characterization and a realization of the unitarizable modules in terms of some (operator-valued) measure on a cone \overline{Ω}. In some cases, we are able to completely determine the corresponding measures.

ISSN 0940–2268 / $2.50 ^{_{C} ◯} Heldermann Verlag

1. Geometric preliminaries

Let V be a Euclidean Jordan algebra (for this notion and further properties, see [7]). For sake of simplicity, V is assumed to be simple. Let < , > be the inner product, e the neutral element and let n = dim V .

Let Ω be the associated cone. Denote by L the connected component of the automorphism group of the cone Ω, i.e.

L = {l ∈ GL(V) | lΩ = Ω}^{0}

The groupe L is closed under conjugation, hence is reductive. The subgroup K = {l ∈ L | le = e} is a maximal compact subgroup of L, and coincides with the connected component of the automorphism group of V .

If x ∈ V , we denote by L(x) the mapping y ↦ xy, and by P (x) = 2L(x)^{2} − L(x^{2}) the so called quadratic representation of V .

If x ∈ Ω, then P (x) is symmetric with respect to the inner product and belongs to L. Every element of L can be written as g = kP (x), for some k ∈ K and some x ∈ Ω. This is in fact the Cartan decomposition of L. For further use, notice the formula exp 2L(x) = P (exp x), for x ∈ V , where on the right handside exp stands for the exponential map in the Jordan algebra V .

Let Δ be the Koecher norm function (also called determinant). Δ is a polynomial of degree r, where r is the rank of the Jordan algebra V (r is also the rank of the symmetric space Ω ≃ L/K). Up to a positive constant, Ω has a unique L-invariant measure given by

d^{∗}x = Δ(x)^{−m}dx ,

where m = \frac{n}{r} (m turns out to be an integer or half an integer). For further use, notice the formula Det(P (x)) = Δ(x)^{2n/r}, for x ∈ V .

The Iwasawa decomposition also has a specific realization. Fix a Jordan frame , i.e. e = c_{1} +c_{2} +…+c_{r}, where the (c_{i})_{1≤i≤r} form a complete orthogonal system of primitive idempotents. Let R = {a = ∑_{i=1}^{r} a_{i}c_{i} , a_{i} ∈ ℝ}. The space 𝔞 = {L(a)|a ∈ R} is a Cartan subspace in 𝔭 = {L(x)|x ∈ V }. Let A = {exp L(a), a ∈ R} be the corresponding Lie subgroup, which can also be viewed as A = {P (a), a = ∑_{i=1}^{r} a_{i}c_{i} , a_{i} > 0}. Now for 1 ≤ i < j ≤ r, let

V_{ij} = \{x ∈ V | c_{i}x = c_{j}x = \frac{1}{2}x\} .

Then V = ⊕_{j=1}^{r}ℝc_{j} ⨁_{1≤i<j≤r} V_{ij} (Peirce decomposition). For x, y ∈ V , write x y = L(xy) + [L(x), L(y)], and for 1 ≤ i < j ≤ r let 𝔫_{ij} = V_{ij} c_{i} . Then 𝔫 = ⨁_{1≤i<j≤r} 𝔫_{ij} is the Iwasawa nilpotent subalgebra associated to the Weyl chamber

𝔞^{+} = {L(a), a = ∑_{i=1}^{r} a_{i}c_{i} , a_{1} < a_{2} < … < a_{r}}. Let N be the analytic subgroup of L with Lie(N) = 𝔫.

Closely associated to this Iwasawa decomposition is a parametrization of Ω. Let

V^{+} = {u = ∑_{i=1}^{r} u_{j}c_{j} + ∑_{j<k} u_{jk} | u_{j} > 0, u_{jk} ∈ V_{jk}} .

For u^{(j)} ∈ ⨁_{k=j+1}^{r} V_{jk} let τ(u^{(j)}) = exp(2u c_{j}) , and for u = ∑_{i=1}^{r} u_{i}c_{i} +

∑_{j<k} u_{jk} in V^{+} , let b_{j} = c_{1} + … + c_{j−1} + u_{j}c_{j} + c_{j+1} + … + c_{r} , 1 ≤ j ≤ r, u^{(j)} = ∑_{k=j+1}^{r} u_{jk} , 1 ≤ j ≤ r − 1, and define

t(u) = P (b_{1})τ(u^{(1)})P (b_{2})τ(u^{(2)}) ⋯ τ(u^{(r−1)})P (b_{r}).

Proposition 1.1. The map u ↦ t(u)e is a bijection from V_{+} onto Ω. If

x = ∑_{j=1}^{r} x_{j}c_{j} + ∑_{j<k} x_{jk}

is the Peirce decomposition of x = t(u)e, then

x_{j} = u_{j}^{2} + \frac{1}{2} ∑_{k=1}^{j−1} ∥ u_{kj} ∥^{2} ,

x_{jk} = u_{j}u_{jk} + 2 ∑_{l=1}^{j−1} u_{lj}u_{lk} .

The invariant measure on Ω is given by :

∫_{Ω} f(x)d^{∗}x = 2^{r} ∫_{V_{+}} f(t(u)e) ∏_{j=1}^{r} u_{j}^{−d(j−1)−1}du_{j} ∏_{1≤j<k≤r} du_{jk} .

For later use, we need a careful analysis of the L orbits in \overline{Ω}. For p, 0 ≤ p ≤ r, let e_{p} = ∑_{i=1}^{p} c_{i}, and let O_{p} be the orbit under L of e_{p}. Observe that O_{0} = {0}, and O_{r} = Ω.

Proposition 1.2. \overline{Ω} = ⨆_{0 ≤ p ≤r} O_{p}.

Each orbit can be parametrized in a way similar to the parametrization of Ω. We need some more notations. For 0 ≤ p ≤ r − 1, let L_{p} be the stabilizer of e_{p}, and 𝔩_{p} its Lie algebra. Now let

𝔞_{p} = {L( ∑_{1≤i≤p} a_{i}c_{i}), a_{i} ∈ ℝ}

𝔫_{p} = ⨁_{1≤i≤p} ( ⨁_{i+1≤j≤r} 𝔫_{ij})

It is easily checked that 𝔩 = 𝔩_{p} ⊕ 𝔞_{p} ⊕ 𝔫_{p}. On the group level, we get the density of L_{p}A_{p}N_{p} in L, where A_{p} and N_{p} are the analytic subgroups of L corresponding to respectively 𝔞_{p} and 𝔫_{p} (see e.g. [17]). To state this result in a way similar to Proposition 1.1, let

V_{p}^{+} = ∑_{i=1}^{p} ℝ^{+}c_{i} ⊕ 𝔫_{p} .

For u = ∑_{i=1}^{p} u_{i}c_{i} + ∑ u_{ij} ∈ V_{p}^{+} , let as above

t(u) = P (b_{1})τ(u^{(1)})P (b_{2})τ(u^{(2)}) ⋯ τ(u^{(p−1)})P (b_{p}),

where u^{(i)} = ∑_{j=i+1}^{r} u_{ij}, 1 ≤ i ≤ p.

Proposition 1.3. Let 1 ≤ p ≤ r − 1. The mapping u ↦ t(u)e_{p} from V_{p}^{+} into V is a one-to-one map into a dense subset O′_{p} of O_{p} . There exists a unique (up to a positive scalar) relatively invariant measure ν_{p} on O_{p}.The complementary set O_{p} \ O′_{p} has ν_{p} measure 0. In the corresponding coordinates ν_{p} is given by

∫_{O_{p}} f(x)dν_{p}(x) = ∫_{V_{p}^{+}} f(t(u)e_{p}) ∏_{j=1}^{p} u_{j}^{(p−j+1)d−1}du_{j} ∏_{{1≤i≤p \atop i<j≤r}} du_{ij} .

The relative invariance is expressed by the following formula :

∫_{O_{p}} f(lx)dν_{p}(x) = ( det l)^{−s_{p}} ∫_{O_{p}} f(x)dν_{p}(x) , ∀l ∈ L,

where s_{p} = \frac{r}{2n}dp (d is the common dimension of all 𝔫_{ij}, for 1 ≤ i < j ≤ r). For a proof, see [17] or [7], p. 134.

Let V_{ℂ} = V + iV be the complexification of V and let T_{Ω} be the associated tube domain. When equipped with the Bergman metric, T_{Ω} is a hermitian symmetric space. Let G = G(T_{Ω}) be the connected component of the identity in the group of bi-holomorphic transformations of T_{Ω} , and let U be the stabilizer in G of the base point ie ∈ T_{Ω} . An important fact is that L and U are two real forms of the same complex Lie group, namely the connected component of the identity in Str(V_{ℂ}), where

Str(V_{ℂ}) = {g ∈ GL(V_{ℂ}) | ∀x ∈ V_{ℂ} , P (gx) = gP (x)g^{t} },

with the obvious extension of P to V_{ℂ}.

If g ∈ L, still denote by g its complex linear extension to V_{ℂ}. It clearly preserves T_{Ω} , giving a natural map from L into G. The Cartan subspace 𝔞 (for the pair (𝔩, 𝔨)) turns out to be also a Cartan subspace for the pair (𝔤, 𝔲), where 𝔤 = Lie(G) and 𝔲 = Lie(U). Another important subgroup of G is the subgroup of translations N^{+} . In fact to each v ∈ V , is associated the translation t_{v} given by z ↦ z + v. N^{+} is clearly isomorphic to V (as Abelian group). Moreover, the semi-direct product LN^{+} (where L acts on N^{+} by its natural action on V ) is the group of all affine transformations of T_{Ω} . N^{+} is a subgroup of the Iwasawa subgroup associated to the positive Weyl chamber 𝔞^{++} = {∑_{i=1}^{r} a_{i}c_{i} | 0 < a_{1} < a_{2} < … < a_{r}}. In fact the full Iwasawa subgroup is the semi-direct product NN^{+} . Finally, it is worth mentioning that the group G is generated by L, N^{+} , and the inversion z ↦ −z^{−1} .

The action of G on T_{Ω} can be (locally) extended to an action of G^{ℂ} on T_{Ω} . For g ∈ G^{ℂ}, z ∈ T_{Ω} and g.z ∈ T_{Ω} , define the automorphy factor J(g, z) = \frac{∂(g.z)}{∂z}. When defined, it turns out that J(g, z) is always in L^{ℂ}, and satisfies the cocycle identity J(g_{1}g_{2}, z) = J(g_{1}, g_{2}.z)J(g_{2}, z) . Obviously, if l is in L, then J(l, z) = l.

2. Invariant cones and Ol’shanskiĭ semigroups

An important property of the hermitian pairs is the existence of Ad(G) invariant cones in 𝔤. Cones are assumed to be convex, closed with a nonvoid interior and proper. One of the main facts is the existence of a minimal invariant cone C_{min} and a maximal invariant cone C_{max} , in the sense that any invariant cone C contains either C_{min} or −C_{min} and similarly is contained in C_{max} or −C_{max} .

Theorem 2.1. The cone C_{min} is generated (up to ±1) by t_{e_{r}} viewed as an element in 𝔫^{+} = Lie(N^{+}), i.e. C_{min} is the smallest closed convex cone containing the Ad(G)-orbit of t_{e_{r}} in 𝔤.

Proof. By Vinberg’s theorem ([18]), C_{min} contains a (unique) ray which is invariant by a minimal parabolic subgroup. Thanks to the structure of the nilpotent factor NN^{+} , it is clear that this ray can only be ±ℝ^{+}t_{e_{r}}.

From now on, denote by C_{min} (resp., C_{max} ) the minimal (resp., maximal) cone that contains t_{e_{r}}.

As discovered by Ol’shanskiĭ (see [16]), to any invariant cone C, it is possible to associate a semi-group Γ_{C} = G exp iC in G^{ℂ}. The semi-group Γ_{max} = G exp iC_{max} is exactly the semi-group of compressions of T_{Ω} , namely

Γ_{max} = {g ∈ G^{ℂ} | g(T_{Ω}) ⊂ T_{Ω}} .

Theorem 2.2. Γ_{min} ⊃ \{t_{iv}\}_{v∈\overline{Ω}} .

Proof. As t_{e_{r}} ∈ C_{min} , it is easily seen, using the action of L and the convexity of \overline{Ω}, that t_{v} is contained in C_{min} for any v ∈ \overline{Ω}. Hence the result follows.

The importance of these cones and semi-groups for highest weight representations has been noticed by Ol’shanskiĭ and in fact if (π, H) is any unitary representation, let H^{∞} be the space of C^{∞} vectors, and let

C_{π} = {X ∈ 𝔤 | < idπ(X)ξ, ξ > ≤ 0, ∀ ξ ∈ H^{∞} } .

C_{π} is a cone, which is non trivial if and only if π has a highest weight, and then the representation π can be extended as a (holomorphic) representation of Γ_{π} = G exp iC_{π} by contractions.

3. Reproducing kernels and unitarity

Let (μ, V_{μ}) be a finite dimensional irreducible unitary representation of the maximal compact subgroup U of G. As explained before, it is convenient to consider μ as a finite dimensional (holomorphic) representation of L^{ℂ}. Moreover it satisfies the relation μ(l^{∗}) = μ(l)^{∗}, where l^{∗} = \overline{l}^{t}, for l ∈ L^{ℂ} (extension of the unitarity property of μ). For g ∈ G, and z ∈ T_{Ω} , set J_{μ}(g, z) = μ(J(g, z))

Now let V_{μ} be the space of holomorphic functions on T_{Ω} with values in V_{μ}. Define the following action of G on V_{μ} :

T_{μ}(g)f(z) = (J_{μ}(g^{−1}, z))^{−1}f(g^{−1}z) ,

where f ∈ V_{μ}, z ∈ T_{Ω} and g ∈ G.

We want to discuss the existence of an invariant inner product on V_{μ}. There is in fact a natural inner product given by

(f, g)_{μ} = ∫_{T_{Ω}}(μ(P (y)^{−1})f(z)|g(z))_{V_{μ}}d_{∗}z ,

where z ∈ T_{Ω} , y = ℑ(z), and d_{∗}z is the G invariant measure on T_{Ω} . The invariance of the inner product by N^{+} is obvious, its invariance by L is easy. It remains to check invariance by the inversion z ↦ −z^{−1} . But this is a consequence of the following formula : P (\overline{z})P (ℑ(z))^{−1}P (z) = P (ℑ(−z^{−1}))^{−1} (see [4] p. 163). Now let H_{μ} = {f ∈ V_{μ}|(f, f)_{μ} < +∞}. Then if H_{μ} ≠ {0}, (T_{μ}, H_{μ}) defines a unitary representation, and in fact this is the celebrated holomorphic discrete series.

Now let H_{μ} be an irreducible unitary representation of G, and assume there exists a continuous non trivial intertwining operator from H_{μ} into V_{μ}, where the latter space is equipped with the compact-open topology. Then the evaluation map at any point z ∈ T_{Ω} is a continuous linear map on H_{μ}, so H_{μ} admits a reproducing kernel. In fact, let E_{z} : H_{μ} → V_{μ} be the evaluation map at z ∈ T_{Ω} and define

ℚ_{μ}(z, w) = E_{z}E_{w}^{∗}.

Then ℚ_{μ} : T_{Ω} × T_{Ω} → End (V_{μ}) satisfies

(3.1i)  ℚ_{μ} is holomorphic in z and antiholomorphic in w

(3.1ii)  ℚ_{μ}(w, z) = ℚ_{μ}(z, w)^{∗}

(3.1iii)  ∀q ∈ ℕ, ∀ (w_{j})_{1≤j≤q} ∈ T_{Ω} , ∀ (ξ_{j})_{1≤j≤q} ∈ V_{μ}

∑_{i} ∑_{j} (ℚ_{μ}(w_{j}, w_{i})ξ_{i}|ξ_{j})_{V_{μ}} ≥ 0

(3.1iv)  ℚ_{μ}(g.z, g.w) = J_{μ}(g, z)ℚ_{μ}(z, w)J_{μ}(g, w)^{∗}

A mapping ℚ : T_{Ω} × T_{Ω} → End (V_{μ}) which satisfies (3.1 i, ii, and iii) is said to be a positive definite (operator-valued) kernel (see [14]). If it moreover satisfies (3.1iv), the kernel ℚ is said to be invariant (with respect to μ).

Proposition 3.1. Let μ be a finite dimensional holomorphic irreducible representation of L^{ℂ}, and let ℚ be an invariant positive definite kernel (with respect to μ). Let L_{μ} be the span of the functions z ↦ ℚ(z, w)ξ, where w is arbitrary in T_{Ω} , and ξ arbitrary in V_{μ}. Introduce the (well defined) Hermitian form on L_{μ} given by

( ∑_{i} ℚ(., w_{i})ξ_{i}| ∑_{j} ℚ(., w′_{j})ξ′_{j}) = ∑_{i} ∑_{j} (ℚ(w′_{j}, w_{i})ξ_{i}|ξ′_{j})_{V_{μ}} .

Let H_{μ} be the usual (separate) completion of L_{μ} with respect to this (welldefined) form. Then H_{μ} is invariant under T_{μ} and the restriction of T_{μ} to H_{μ} is unitary and irreducible.

For the proof, see [14]. Let us observe moreover, that L_{μ} always contains the “highest weight vector”, namely ℚ(z, ie)ξ_{μ}, where ξ_{μ} is the highest weight vector in V_{μ} (cf. [17]). So H_{μ} is a highest weight representation.

However, the kernel ℚ_{μ}(z, w) satisfies another important condition which is related to the remark due to Ol’shanskiĭ we mentioned above.

Proposition 3.2. Let μ be a finite dimensional holomorphic representation of L^{ℂ}, and assume that ℚ_{μ} is positive definite. Then ℚ_{μ} satisfies

(3.1v)  ∀ q ∈ ℕ, ∀ (w_{j})_{1≤j≤q} ∈ T_{Ω} , ∀ (ξ_{j})_{1≤j≤q} ∈ V_{μ} , ∀ y ∈ \overline{Ω}

( ∑_{i} ∑_{j} ℚ_{μ}(w_{j} + iy, w_{i} + iy)ξ_{i}|ξ_{j})_{V_{μ}} ≤ ( ∑_{i} ∑_{j} ℚ_{μ}(w_{j}, w_{i})ξ_{i}|ξ_{j})_{V_{μ}} .

Proof. In fact, the cone C_{T_{μ}} = C_{μ} contains −t_{e_{r}} (see the original argument in [10]), hence C_{μ} ⊃ −C_{min} by Theorem 2.2. (cf [16]), and from the holomorphic extension of T_{μ} to the Olshanskiĭ semigroup G exp iC_{π} by contractions

yields ∥T_{μ}(t_{−iy})Φ∥^{2} ≤ ∥Φ∥^{2} , where y ∈ \overline{Ω}, and Φ(.) = ∑_{i} ℚ_{μ}(., w_{i})ξ_{i} ∈ H_{μ}.

But this is exactly the inequality we were looking for, once observed that T_{μ}(t_{−iy})ℚ_{μ}(., w) = ℚ_{μ}(., w + iy).

The conditions (3.1i-iv) completely determine (up to a positive scalar) the possible kernels (cf [4]). In fact by using the action of the translations \{t_{y}\}_{y∈V} , it is easily seen that ℚ must be of the form ℚ(z, w) = Q(\frac{z−\overline{w}}{2} ), where Q is a holomorphic map from T_{Ω} into End (V_{μ}). Moreover, if one considers the origin ie ∈ T_{Ω} , then from (3.1iv) we immediately see that Q(ie) must commute with the operators μ(J(k, ie)) for any k in the stabilizer U of ie in G. An application of Schur’s lemma forces Q(ie) to be a multiple of the identity. The invariance property applied to P (y^{1/2}), where y ∈ Ω shows that Q(iy) = Q(P (y^{1/2}).ie) = μ(P (y)), up to a positive constant. As Q is holomorphic, the only possibility for ℚ is (up to a positive constant)

ℚ(z, w) = μ(P (\frac{z − \overline{w}}{2i})).

Conversely, properties (3.1i) and (3.1ii) are immediate. The invariance property can easily be established for the translations and the elements of L. For the inversion z ↦ −z^{−1} , one uses the identity

P (\overline{w} ^{−1} − z^{−1}) = P (\overline{w} ^{−1})P (z − \overline{w})P (z^{−1}), for z, w ∈ T_{Ω}

(cf. [7] page 200), and takes images of both sides under μ to get the desired invariance property.

Henceforth we concentrate our effort towards property (3.1v), which is crucial for discussing unitarity.

If W is a finite-dimensional Hilbert space, denote by Herm(W) the the space of Hermitian operators on W and by Herm^{+}(W) the cone of positive semidefinite Hermitian operators on W. In what follows, by a measure on \overline{Ω} with values in Herm^{+}W, we mean, following Bourbaki (see [1]), a linear map R from the space C_{c}(\overline{Ω}) of continuous real valued functions with compact support on \overline{Ω} into Herm(W), which is continuous for the usual topology on C_{c}(\overline{Ω}), and such that for any nonnegative function φ in C_{c}(\overline{Ω}), R(φ) ∈ Herm^{+}(W).

Theorem 3.3. Let W be a finite dimensional Hilbert space and let q : Ω → Herm^{+}(W) be a continuous map with the property (3.1v). Then there exists a unique measure R on \overline{Ω}, with values in Herm^{+}(W), such that :

q(y) = ∫\overline{Ω} e^{−(y|v)}dR(v) ,

for all y ∈ Ω. Proof. First fix ξ ∈ W. Define q_{ξ}(y) = (q(y)ξ|ξ). Clearly q_{ξ} is a continuous function on Ω, which satisfies

0 ≤ ∑_{i} ∑_{j} λ_{i}\overline{λ}_{j}q_{ξ}(y_{i} + y_{j} + y) ≤ ∑_{i} ∑_{j} λ_{i}\overline{λ}_{j}q_{ξ}(y_{i} + y_{j}),

for all (y_{i})_{1≤i≤n}, y ∈ Ω, (λ_{i})_{1≤i≤n} ∈ ℂ. By Nussbaum’s theorem (see [15],[17]), there exists a unique positive measure R_{ξ} on \overline{Ω}, such that

q_{ξ}(y) = ∫\overline{Ω} e^{−(y|w)}dR_{ξ}(w) .

Now define for ξ, η ∈ W

R_{ξ,η} = \frac{1}{4}[R_{ξ+η} − R_{ξ−η} + iR_{ξ+iη} − iR_{ξ−iη}] .

The way it depends on ξ, η is clearly of Hermitian nature. So there exists a measure R on \overline{Ω}, mith values in Herm(W), such that R_{ξ,η}(.) = (R(.)ξ|η). As R_{ξ} = R_{ξ,ξ} , R has values in Herm^{+}(W), and the result follows. The uniqueness is clear from properties of the Laplace transform.

It is now possible to apply this result to the reproducing kernels ℚ_{μ}.

Theorem 3.4. Let μ be a finite dimensional representation of L on a vector space V_{μ}. Then the associated kernel ℚ_{μ} is positive definite if and only if there exists a measure R_{μ} on \overline{Ω}, with values in Herm^{+}(V_{μ}), such that

(3.4i)  dR_{μ}(l.) = μ(l)^{∗^{−1}}dR_{μ}(.)μ(l)^{−1} , ∀l ∈ L
(3.4ii)  ∫\overline{Ω} e^{−trv} dR_{μ}(v) = Id

Proof. The existence of such a measure, when ℚ_{μ} is positive definite is clear from the preceding results. Conversely, use a change of variable to get from properties (3.4 i) and (3.4 ii) the equality μ(P (x)) = ∫\overline{Ω} e^{−(x|w)}dR_{μ}(w) which gives immediately ℚ_{μ}(z, w) = ∫_{Ω} e^{−( \frac{z−\overline{w}}{2i} |v)}dR_{μ}(v) proving the positivedefiniteness of ℚ_{μ}.

It is possible to give a more concrete realization of the Hilbert space H_{μ} corresponding to the kernel ℚ_{μ} (according to Proposition (3.1)). In fact define G_{μ} as the space of all measurable functions Φ : \overline{Ω} → V_{μ}, which satisfy

||Φ||_{μ}^{2} = ∫\overline{Ω} (dR_{μ}(2v)Φ(v)|Φ(v)) < +∞.

Then, after identifying two functions which are equal R_{μ} -almost everywhere, G_{μ} has a Hilbert space structure for the inner product

(Φ, Ψ)_{G_{μ}} = ∫\overline{Ω} (dR_{μ}(2v)Φ(v)|Ψ(v)).

If Φ ∈ G_{μ} define, for z ∈ T_{Ω} , FΦ : T_{Ω} → V_{μ}

FΦ(z) = ∫\overline{Ω} e^{i<z|v>}dR_{μ}(2v)Φ(v).

Let ξ ∈ V_{μ} ; then

|(dR_{μ}(2v)Φ(v)|ξ)_{V_{μ}} | ≤ (dR_{μ}(2v)Φ(v)|Φ(v))^{1/2}(dR_{μ}(2v)ξ|ξ)^{1/2},

and by applying Schwarz inequality, we get

| < FΦ(z)|ξ > |^{2} ≤ ( ∫\overline{Ω}(dR_{μ}(2v)Φ(v)|Φ(v))( ∫\overline{Ω} e^{−2<y|v>}(dR_{μ}(2v)ξ|ξ)) ,

where z = x + iy. This shows that the integral in the definition of FΦ is (absolutely) convergent and it is then easy to verify that FΦ is holomorphic. Now let F_{μ} be the space of all (holomorphic) V_{μ} -valued functions of the form FΦ with Φ ∈ G_{μ}, and define ||FΦ||_{F_{μ}} = ||Φ||_{G_{μ}}. Thanks to the injectivity of the Laplace transform,||FΦ||_{F_{μ}} = 0 if and only if Φ = 0 dR_{μ} − a.e., so if and only if FΦ(z) = 0 everywhere. Hence F_{μ} is a Hilbert space. Moreover, the evaluation map at any point z ∈ T_{Ω} is a continuous linear (V_{μ} -valued) map. So F_{μ} has a reproducing kernel 𝕂(z, w). By definition, there exists a measurable function k : \overline{Ω} × T_{Ω} → End (V_{μ}), such that, for every ξ ∈ V_{μ}

𝕂(z, w)ξ = (Fk(., w)ξ)(z), z, w ∈ T_{Ω}.

For every Φ ∈ G_{μ}, and w ∈ T_{Ω} ,

\begin{aligned}((FΦ)(w)|ξ)_{V_{μ}} & = (FΦ|𝕂(., w)ξ)_{F_{μ}} \\ & = (Φ|k(., w)ξ)_{G_{μ}}. \end{aligned}

The first term is

∫\overline{Ω} e^{i<w|v>}(dR_{μ}(2v)Φ(v)|ξ)_{V_{μ}},

whereas the last is

∫\overline{Ω} (dR_{μ}(2v)Φ(v)|k(v, w)ξ)_{V_{μ}}.

We easily conclude that k(v, w)ξ = e^{−i<\overline{w}|v>}ξ , for R_{μ} -almost every v in \overline{Ω}. Hence

𝕂(z, w)ξ = ∫\overline{Ω} e^{i<z|v>}e^{−i<\overline{w}|v>}dR_{μ}(2v)ξ = ℚ_{μ}(z, w)ξ.

Hence the following conclusion :

Theorem 3.5. Let μ be a representation of L^{ℂ} such that ℚ_{μ} is positive definite. Let F_{μ} be as above. Then F_{μ} is a Hilbert space with reproducing kernel ℚ_{μ}(z, w). The space F_{μ} is stable under T_{μ} and the restriction of T_{μ} to F_{μ} is unitary and irreducible.

4. Some necessary conditions for the existence of the measure R_{μ}

Let μ be a holomorphic finite dimensional representation of L^{ℂ}. Still denote by μ the restricted highest weight of the representation μ with respect to the Iwasawa decomposition considered in section 1 and by ξ_{μ} a non-zero highest weight vector. To be more explicit, one has

μ( exp 2 ∑_{k=1}^{r} a_{i}L(c_{i})) ξ_{μ} = ∏_{k=1}^{r} e^{a_{k}m_{k}} ξ_{μ} ,

for all (a_{k})_{1≤k≤r} ∈ ℝ, and μ(n)ξ_{μ} = ξ_{μ}, for all n ∈ N. The restricted highest weights are characterized by the conditions

∀ 1 ≤ k ≤ r, m_{k} ∈ ℤ and m_{1} ≤ m_{2} ≤ … ≤ m_{r} .

(cf [4], p. 167). For further use, notice the formula μ(P (a))ξ_{μ} = ∏_{k=1}^{r} a_{k}^{m_{k}} ξ_{μ}, where a = ∑_{k=1}^{r} a_{k}c_{k}, a_{k} > 0, ∀k, 1 ≤ k ≤ r .

The property (3.4i) clearly shows the fact that the support of R_{μ} is a union of L orbits. Because of the structure of these orbits, there is an integer p, with 0 ≤ p ≤ r, such that Supp(R_{μ}) ⊂ \overline{O}_{p} and Supp(R_{μ}) ⊄ \overline{O}_{p−1} .

Theorem 4.1. Let μ = (m_{1}, m_{2}, …, m_{r}) as above. A necessary condition for the existence of a measure R_{μ} satisfying the conditions (3.4i) and (3.4ii) and such that Supp(R_{μ}) = \overline{Ω} is :

(4.1i)  m_{r} < −\frac{d(r − 1)}{2} .

A necessary condition for the existence of a measure R_{μ} satisfying the conditions (3.4i) and (3.4ii) and such that Supp(R_{μ}) = \overline{O}_{p}, for some p, 0 ≤ p ≤ r − 1 is

(4.1ii)  m_{p+1} = m_{p+2} = … = m_{r} = −\frac{dp}{2} .

Proof. Assume first that Supp(R_{μ}) = \overline{O}_{p} , for some p, 0 ≤ p ≤ r − 1. Consider the restriction of R_{μ} to O_{p} as a distribution . It must coincide with a C^{∞} function. In fact, let X ∈ 𝔩 = Lie(L) ⊂ 𝔤l(V). It induces a vector field X̃ on O_{p} . The invariance property (3.4i) implies the differential relation :

X̃R_{μ} = −μ(X^{t}) ∘ R_{μ} − R_{μ} ∘ μ(X) .

Choose vectors X_{1}, X_{2}, …, X_{k} ∈ 𝔩, such that X̃_{1}, X̃_{2}, …, X̃_{k} form a basis of the tangent plane in a neigbourhood of some point of the the orbit O_{p} (say, e_{p} for example). Compute ∑_{j=1}^{k} X̃_{j}^{2}R_{μ} near e_{p} using the last relation. It shows that R_{μ} is (near e_{p}) solution of a partial differential system, which is clearly elliptic. Hence, by the classical regularity results, R_{μ} has locally near e_{p} a C^{∞} density w.r.t. the relatively invariant measure ν_{p}. From the invariance property (3.4i), this property is true everywhere on O_{p}. In other words, there exists an analytic function ρ_{μ} : O_{p} → Herm^{+}(V_{μ}), such that R_{μ} coincides with ρ_{μ}dν_{p} on O_{p}. The invariance condition now reads :

μ(l)^{∗^{−1}}ρ_{μ}(w)μ(l)^{−1} = ( det l)^{s_{p}}ρ_{μ}(lw) ,

for l ∈ L and w ∈ O_{p}.

Let E_{μ} = ρ_{μ}(e_{p}). As Supp(R_{μ}) = \overline{O}_{μ}, E_{μ} ≠ 0. For l ∈ L_{p}, the invariance condition (3.4i) implies

E_{μ} ∘ μ(l) = ( det l)^{−s_{p}}μ(l)^{∗^{−1}} ∘ E_{μ} .

Now let ξ_{μ} be a non-zero vector in V_{μ} of highest restricted weight μ, and consider the function Φ : L → ℂ defined by Φ(l) = (E_{μ}μ(l)ξ_{μ}|μ(l)ξ_{μ}). Recall that L_{p}A_{p}N_{p} is dense in L, and take l = l_{p}a_{p}n_{p}, where l_{p} ∈ L_{p}, a_{p} ∈ A_{p} and n ∈ N_{p}. Then

Φ(l) = a_{p}^{2μ}(E_{μ} ∘ μ(l_{p})ξ_{μ}|μ(l_{p})ξ_{μ}) = a_{p}^{2μ}( det l_{p})^{−s_{p}}(μ(l_{p})^{∗^{−1}} ∘ E_{μ}ξ_{μ}|μ(l_{p})ξ_{μ})

= a_{p}^{2μ}( det l_{p})^{−s_{p}}(E_{μ}ξ_{μ}|ξ_{μ}) .

As E_{μ} ≠ 0, Φ(l) cannot be 0 for all l ∈ L, hence (E_{μ}ξ_{μ}|ξ_{μ}) ≠ 0. Now, for (a_{k})_{p+1≤k≤r} ∈ ℝ^{+} , consider the element

a = P (c_{1} + c_{2} + ⋯ + c_{p} + a_{p+1}c_{p+1} + a_{p+2}c_{p+2} + ⋯ + a_{r}c_{r}).

Now μ(a)ξ_{μ} = a_{p+1}^{m_{p+1}}a_{p+2}^{m_{p+2}} … a_{r}^{m_{r}} ξ_{μ}, whereas det (a) = (a_{p+1}a_{p+2} … a_{r})^{\frac{2n}{r}} . So Φ(a) = a^{2μ}(E_{μ}ξ_{μ}|ξ_{μ}) = ( det a)^{−s_{p}}(E_{μ}ξ_{μ}|ξ_{μ}). Hence the relation

m_{p+1} = m_{p+2} = … = m_{r} = −\frac{dp}{2} .

Let now consider the case where Supp(R_{μ}) = \overline{Ω}. The first part of the preceding argument is still valid. In particular, the restriction of R_{μ} to Ω has an analytic density, say ρ_{μ}(x) with respect to the invariant measure d^{∗}x. Let E_{μ} = ρ_{μ}(e). It is still true that (E_{μ}ξ_{μ}|ξ_{μ}) ≠ 0, and the invariance condition now implies :

ρ(t(u)e) = μ(t(u))^{∗^{−1}}ρ_{μ}(e) μ(t(u))^{−1} ,

for u ∈ V^{+} . Now the first condition (3.4ii) implies in particular

∫_{Ω} e^{−trv}(ρ_{μ}(v)ξ_{μ}|ξ_{μ})d^{∗}v < ∞ .

Use the parametrization described in section 2 (cf [7] p. 123). As μ(t(u))^{−1}ξ_{μ} = ∏_{j=1}^{r} u_{j}^{−m_{j}}ξ_{μ}, the integral converges if (and only if)

∫_{o}^{+∞} … ∫_{o}^{+∞} ∏_{j=1}^{r} u_{j}^{−2m_{j}}u_{j}^{−d(j−1)−1}e^{−u_{j}^{2}}du_{j} < ∞ .

But this happens if and only if m_{r} < −\frac{d(r − 1)}{2} .

To finish the proof, observe that the conditions already obtained are mutually incompatible. So that, if Supp(R_{μ}) = \overline{O}_{p} and if ρ_{μ} is its density on O_{p}, then the difference dR_{μ} − ρ_{μ}(.)dν_{p} has its support contained in \overline{O}_{p−1} and still satisfies the condition (3.4ii). If it were non zero on O_{p−1} , the first part of the proof would imply m_{r} = −\frac{(p−1)d}{2} , whereas the condition m_{r} = −\frac{pd}{2} (or m_{r} < −\frac{d(r−1)}{2} in case p = r) has been shown to be necessary. By induction we eventually get dR_{μ} − ρ_{μ}(.)dν_{p} = 0, completing the proof of theorem (4.1).

5. An example

It seems in general quite hard to find explicit expressions for the measure R_{μ}. These measures are known when μ has dimension 1 (see [17]). Here we want to discuss a vector-valued case, where however, computations are easy because of the fact that the representation μ stays irreducible when restricted to the maximal compact subgroup K of L (see also [9]).

Let H = H_{r} be the real vector space of r × r Hermitian matrices, and define the Jordan product to be x.y = \frac{1}{2}(xy + yx), which turns H into a Euclidean Jordan algebra for the standard inner product tr xy. The cone Ω is the cone of positive-definite matrices, the group L may be identified with ℝ^{+} × SL(r, ℂ), where SL(r, ℂ) acts by l.x = lxl^{∗} (l ∈ SL(r, ℂ), x ∈ H), its maximal compact subgroup K is SU(r) and for x, y ∈ H, P (x)y = xyx. As for a Jordan frame, the natural choice is

c_{i} = ( \begin{matrix}0 \\ & . & . & . \\ & & & & 1 \\ & & & & & . & . & . \\ & & & & & & & & 0 \end{matrix} ) , 1 ≤ i ≤ r ,

where 1 stands in the i th row and column. The corresponding Cartan subspace is

𝔞 = \{( \begin{matrix}a_{1} & 0 & ⋯ & 0 \\ 0 & a_{2} & ⋯ & 0 \\ ⋮ & ⋮ & ... & ⋮ \\ 0 & 0 & ⋯ & a_{r} \end{matrix} ) , a_{1}, a_{2}, …, a_{r} ∈ ℝ\} .

The dimension of H is n = r^{2} , and d = 2 .

Let μ be a (finite-dimensional) representation of L of the form l ↦ det (l)^{m}ν(l), where m is an integer and ν is a holomorphic representation of SL(r, ℂ), (for short we say μ is holomorphic) and still denote by μ its dominant weight μ = (m_{1}, m_{2}, …, m_{r}), where for 1 ≤ i ≤ r, m_{i} ∈ ℤ, and m_{1} ≤ m_{2} ≤ … ≤ m_{r}. Notice that the weights of the representation are complex linear forms on 𝔞^{ℂ}, and so are determined by their restrictions to 𝔞.

Theorem 5.1. Let μ as above. The kernel ℚ_{μ} is of positive definite type if and only if either :

(5.i)  m_{r} < −(r − 1)

(5.ii)  m_{p+1} = m_{p+2} = … = m_{r} = −p .

Proof. In the first case, the corresponding measure R_{μ} is supported in \overline{Ω}, whereas it is supported in \overline{O}_{p} in the second case. These measures are made explicit in due course of the proof.

Sticking to notations used in section 4, first consider the functional equation for the regular orbit Ω. Observe that E_{μ} must commute with μ(l), when l ∈ SU(r). But by assumption μ is a holomorphic representation and so is still irreducible when restricted to SU(r). By Schur’s lemma this implies the fact that E_{μ} must be a multiple of the identity. But now this forces the equality ρ_{μ}(x) = μ(P (x))^{−1}, for all x ∈ Ω, up to a positive scalar. As the positivity condition is clearly satisfied, it remains to check the integrability condition. To this end, define

W_{μ} = {ξ ∈ V_{μ}| ∫_{Ω} e^{− tr v}(ρ_{μ}(v)ξ, ξ)d^{∗}v < +∞}

Clearly by Schwarz inquality, W_{μ} is a vector subspace, and it is invariant under K. As the restriction of μ to K is irreducible, W_{μ} is 0 or V_{μ}, but W_{μ} = {0} would imply E_{μ} = 0. So W_{μ} = V_{μ}. So it suffices to check the integrability condition for, say, a highest weight vector. As the integrability condition for a highest weight vector was already tested in the general case, this finishes this case.

Now assume Supp(R_{μ}) = \overline{O}_{p}, for some p, 0 ≤ p ≤ r − 1. This forces m_{p+1} = m_{p+2} = … = m_{r} = −p. As before, let ρ_{μ} be the density with respect to the relatively invariant measure ν_{p}, and let E_{μ} = ρ_{μ}(e_{p}). Let

l = ( \begin{matrix}e^{iθ_{1}} \\ & . & . & . \\ & & & & e^{iθ_{r}} \end{matrix} ) ,

where θ_{1} + … + θ_{r} ≡ 0 mod 2π. Observe that l.e_{p} = e_{p}, l^{∗} = l^{−1} and | det (l)| = 1. The condition (3.4i) clearly implies that E_{μ} commutes with all matrices μ(l), when l is diagonal in SU(r). So E_{μ} preserves the weight spaces of V_{μ}. Now let ξ_{λ} be a weight vector corresponding to the weight λ = (l_{1}, l_{2}, …, l_{r}). Notice from the preceding remark that E_{μ}ξ_{λ} is also of weight λ. Let

l = ( \begin{matrix}1 \\ & . & . & . \\ & & & & 1 \\ & & & & & a_{p+1} \\ & & & & & & . & . & . \\ & & & & & & & & & a_{r} \end{matrix} ) ,

where a_{p+1}, a_{p+2}, …, a_{r} ∈ ℂ^{∗}, a_{p+1}a_{p+2} … a_{r} ∈ ℝ. Then l ∈ L_{p},

E_{μ}ξ_{λ} = |a_{p+1}a_{p+2} … a_{r}|^{−2p}|a_{p+1}|^{−2l_{p+1}}|a_{p+2}|^{−2l_{p+2}} … |a_{r}|^{−2l_{r}}E_{μ}ξ_{λ} .

Hence, if E_{μ}ξ_{λ} ≠ 0, l_{p+1} = l_{p+2} +… = l_{r} = −p. Let W be the sum of all weight spaces with a weight satisfying this condition. W coincides with the submodule of V_{μ} generated by the highest weight vector ξ_{λ} under the action of the subgroup

H_{p} = \{ ( \begin{matrix}h & 0 \\ 0 & 1_{q} \end{matrix} ) , h ∈ SL(p, ℂ) and q = r − p\}.

Clearly, W as H_{p} module is isomorphic with the highest weight module of SL(p, ℂ) with highest weight (m_{1}, m_{2}, …, m_{p}) and in particular is irreducible. Since μ is holomorphic, W is also irreducible under the action of the maximal compact subgroup K_{p} of H_{p} (isomorphic to p)). But E_{μ} commutes with μ(l) when l belongs to K_{p}, so is the identity (up to a scalar) on W. In other terms, E_{μ} is (up to a positive scalar) the orthogonal projection on W.

Consider now the representation l ↦ det (l)^{−p}μ(l^{t})^{−1} . Its lowest weight is (−p − m_{1}, −p − m_{2}, …, −p − m_{p}, 0, 0, …, 0), so this representation can be extended polynomially to the full algebra M_{r}(ℂ). By checking on each weight vector, one verifies μ̃(e_{p}) = E_{μ}. By a simple computation using the condition (3.3i), this implies that ρ_{μ}(y) = μ̃(y), for all y ∈ O_{p}. For the integrability condition, one has (with obvious notations)

∫_{O_{p}} e^{− tr w}(μ̃(w)ξ_{μ}|ξ_{μ})dν_{p}(w)

= ∫_{0}^{+∞} … ∫_{0}^{+∞} … ∫_{ℂ} … ∫_{ℂ^{q×p}} e^{−(a_{1}^{2}+...+a_{p}^{2})}e^{−∥u∥^{2}}e^{−∥v∥^{2}} …

… a_{1}^{2(−m_{1}−p)} … a_{p}^{2(−m_{p}−p)}da_{1} … da_{p} … du_{ij} … d\overline{u}_{ij} … dv d\overline{v} ,

and the last integral converges, as m_{1} ≤ m_{2} ≤ … ≤ m_{p} ≤ −p.

References

[1] Bourbaki, N., “Intégration,” 6, Eléments de mathématiques, Hermann, 1963.

[2] Davidson M., T. Enright, and R. Stanke, Differential operators and highest weight representations, Memoirs Amer. Math. Soc. 455, 1991.

[3] Ding H., Hilbert spaces of Vector-Valued Holomorphic Functions and Irreducibility of Multiplier Representations, Contemp. Math. 138 (1992), 119–122.

[4] Ding H., and K. I. Gross, Operator-valued Bessel functions on Jordan algebras., J. reine angew. Math. 435 (1993), 157–196.

[5] Enright T., R. Howe, and N. Wallach, A classification of unitary highest weight modules, in: Representation theory of reductive groups, 97–143, Trombi ed. Progress in Math. 40, Birkhäuser, 1983.

[6] Enright T. J., and A. Joseph, An intrinsic analysis of unitarizable highest weight modules, Math. Ann. 288 (1990), 571–594.

[7] Faraut J., and A. Korányi, “Analysis on Symmetric Cones,” Oxford Mathematical Monographs, Clarendon Press, Oxford 1994.

[8] Gross K. I., and R. A. Kunze, Bessel functions and representation theory, I, J. Funct. Anal. 22 (1976), 73–105.

[9] —, Bessel functions and representation theory, II : Holomorphic discrete series and metaplectic representations, J. Funct. Anal. 25 (1977), 1–49.

[10] Harish Chandra, Representations of semisimple Lie groups IV, Amer. J. Math. 77 (1955), 743–777.

[11] Jakobsen H. P., Hermitian Symmetric spaces and their unitary highest weight modules, J. Funct. Anal. 52 (1983), 385–412.

[12] —, An intrinsic classification of the unitarizable highest weight modules as well as their associated varieties, Preprint.

[13] Jakobsen, H. P., and M. Vergne, Wave and Dirac operators, and representations of the conformal group. J. Funct. Anal. 24 (1977), 52–106.

[14] Kunze R., Positive definite operator valued kernels and unitary representations, in: B. R. Gelbaum Ed., Proc. Conf. Functional Anal., Irvine, 1966.

[15] Nussbaum E. A., The Hausdorff-Bernstein-Widder theorem for semigroups in locally compact Abelian groups Duke Math. J. 22 (1955), 573– 582.

[16] Olshanskiĭ, G.I., Invariant cones in Lie algebras, Lie semi-groups and the holomorphic discrete series, Funct. Anal. and Appl.15 (1981), 275–285.

[17] Rossi H., and M. Vergne, Analytic continuation of the holomorphic discrete series of a semi-simple Lie group, Acta Math. 136 (1976), 1–5.

[18] Vinberg, E. B., Homogeneous cones, Soviet Math. Dokl. 1 (1960), 787– 240 Clerc

[19] Wallach N., The analytic continuation of the discrete series, I, Trans. Amer. Math. Soc. 251 (1979), 1–17, II, Trans. Amer. Math. Soc. 251 (1979), 18–37.

Institut Elie Cartan Université Henri Poincaré B.P. 239 54506 Vandoeuvre-lès-Nancy Cedex France clerc@iecn.u-nancy.fr

Received October 9, 1995 and in final form January 18