Journal of Lie Theory Volume 7 (1997) 245–267 C ◯1997 Heldermann Verlag

Hardy spaces on two–sheeted covering semigroups

K. Koufany and B. Ørsted

Communicated by J. Hilgert

Abstract. In this paper we study the minimal complex Lie semigroups associated with three classical series of groups by using a holomorphic continuation of a certain Cayley transform for the group. In particular we show, that for the symplectic group the odd part of the Hardy space on the double cover is isomorphic to the classical Hardy space on the Siegel upper half space corresponding to the symplectic group of twice the rank of the given group.

0. Introduction

In this paper we shall give a detailed description of the minimal complex Lie semigroups associated with three of the four classical series of groups with an Hermitian symmetric space. These were found by Ol’shanskiĭ ([21]) as were the associated Hardy spaces on these semigroups ([22]), and recently there has been much interest in analysis of this type of function space (see [3], [4], [7], [8], [14], [15], [18], [19], [20] and [23]). In particular one would like to calculate the Cauchy–Szegö kernel explicitly, and to compare these new Hardy spaces with those for classical bounded domains (see [3], [18], [19] and [23]). By using a natural Cayley transform, which might be thought of as a holomorphic continuation of a causal compactification of the Lie group, we show that for the symplectic group the odd part of the Hardy space on the double cover is indeed isomorphic to the classical Hardy space on the Siegel upper half space corresponding to the symplectic group of twice the rank of the given group. One of the main technical points in this work is the actual construction of the double cover semigroup, isomorphic to Howe’s oscillator semigroup (see [12] and [16]), via a choice of square root of a certain Jacobian; this is close in spirit to the construction of the Riemann surface for \sqrt{z}. Thus for the metaplectic group we obtain an explicit formula for the Cauchy–Szegö kernel as well as the branching

ISSN 0949–5932 / $2.50 ^{_{C} ◯} Heldermann Verlag law for the classical Hardy space with respect to the product of two metaplectic groups. For the two remaining series of groups the Cayley transform goes into the tube domains for the Jordan algebras of complex resp. quaternion Hermitian matrices; in the first case the Ol’shanskiĭ Hardy space is never isomorphic to the classical one, and in the second case we show that even after a necessary passage to the double cover the classical Hardy space is strictly contained in the odd part of the Hardy space on the double cover. This is due to a decay condition at infinity in the complex semigroup which holds on the classical space but not on all of the semigroup Hardy space.

1980 Mathematics Subject Classification: 22E46, 22E30, 43A17, 43A85.

Key words & phrases: Cauchy-Szegö kernel, Cayley transform, Hardy space, Holomorphic discrete series, Jordan algebra, Ol’shanskiĭ semigroup.

It should be interesting to pursue this line of investigation for other groups and symmetric spaces which admit holomorphic discrete series representations and Hardy spaces. First, one would like to find the analogue of the Cayley transform studied here; second one should use this to calculate Cauchy– Szegö kernels and to compare to classical Hardy spaces. For the first problem there has been recent interesting progress in a quite general setting by Wolfgang Bertram ([1]) and Frank Betten ([2]) (independently). For the second the Cayley type spaces were treated by Mohamed Chadli ([3]) and Ólafsson–Ørsted ([20]) (independently).

In the first section we recall the Ol’shanskiĭ Hardy spaces and their decomposition as highest weight modules and in section 2 we describe the contraction semigroup for the classical series we consider: U(p, q), Sp(r, ℝ) or SO^{∗}(2l). Also here we construct the relevant Cayley transform. In section 3 we parametrize the scalar-valued holomorphic discrete series and its analytic continuation for the conformal group G^{♭}, where G^{♭} is SU(n, n), Sp(2r, ℝ) or SO^{∗}(4l) respectively in our three cases. Here n = p + q, and the large group G^{♭} acts by local orderpreserving transformations on the smaller one G. One way to look at our study is that we consider the spectrum of G in certain unitary highest weight representations of G^{♭}, in particular the Hardy space (the classical one) on the tube domain for G^{♭}; this we begin in section 4, and in section 5 we give, first for the metaplectic group, the explicit construction of the double cover semigroup. This is only abstractly isomorphic to Howe’s oscillator semigroup, and we need the present construction in terms of geometrically defined cocycles. This section in particular contains the formula for the Cauchy-Szegö kernel for the odd part of the Ol’shanskiĭ Hardy space on Howe’s oscillator semigroup, namely:

K_{odd}(γ_{1}, γ_{2}) = Det(J − γ_{2}^{∗}Jγ_{1})^{−(r+1/2)}

where the right-hand side is the usual matrix determinant. The square root is exactly well-defined because of our double covering. Finally sections 6 and 7 contain the last two series; and in none of these cases does the spectrum of G coincide for the classical and for the Ol’shanskiĭ Hardy space. In the SO^{∗}(2l) case a double cover similar to the metaplectic case is constructed, perhaps of independent interest. The results of this paper were announced in [19].

Acknowledgement. The first author thanks the Institute of Mathematics, Odense University, for support and hospitality when this work was being started and the second author thanks the Department of Mathematics, Nancy University, for hospitality when this work was being completed.

It is a pleasure to thank the referee for several good suggestions to improve this paper.

1. The Ol’shanskiĭ Hardy spaces

Let 𝔤 be a simple Lie algebra over the reals ℝ, and 𝔤 = 𝔨 ⊕ 𝔭 a Cartan decomposition of 𝔤. Let 𝔱 ⊂ 𝔨 be a Cartan subalgebra of 𝔨. We shall suppose that 𝔨 has a non–zero center 𝔷 ; then 𝔷 is one dimensional and 𝔱 is also a Cartan subalgebra of 𝔤.

Let G_{ℂ} be the simply connected complex Lie group corresponding to 𝔤_{ℂ} := 𝔤 + i𝔤, and let G, K and T be the connected subgroups in G_{ℂ} corresponding to 𝔤, 𝔨 and 𝔱, respectively. By the Kostant–Paneitz–Vinberg Theorem ([25]), there are non–trivial regular cones C in i𝔤 which are Ad(G)−invariant, where regular means, convex, closed, pointed (C ∩ −C = {0}) and generating (C − C = i𝔤). Let Cone_{G} (i𝔤) be the set of all regular Ad(G)−invariant cones in i𝔤.

For such a cone C in Cone_{G} (i𝔤), Ol’shanskiĭ associates a semigroup Γ(C) := G exp(C) in G_{ℂ}, and for this semigroup he associates a “non-commut-ative” Hardy space H^{2}(Γ(C)) which is the set of holomorphic functions f on the complex manifold Γ(C)^{∘} = G exp(C^{∘}), the interior of Γ(C), such that

sup_{γ∈Γ(C)^{∘}} ∫_{G} |f(γg)|^{2} dg < ∞.

Note that sometimes the order of g and γ is interchanged in this integral; we use this convention here - they are of course equivalent. For any γ ∈ Γ(C)^{∘} the linear functional f ⟼ f(γ) is continuous on H^{2}(Γ(C)), which is a Hilbert space, see for example [15]. Therefore by the Riesz representation theorem, there exists a vector K_{γ} ∈ H^{2}(Γ(C)) such that (f, K_{γ}) = f(γ). The reproducing kernel K which is called the Cauchy–Szegö kernel is defined by

K(γ_{1}, γ_{2}) = K_{γ_{2}}(γ_{1}).

It is Hermitian, holomorphic in γ_{1} and anti–holomorphic in γ_{2} .

Let Δ = Δ(𝔤_{ℂ}, 𝔱_{ℂ}) be the set of roots of 𝔤_{ℂ} relative to 𝔱_{ℂ}. Let Δ^{+} ⊂ Δ be the set of positive roots relative to some order (namely the one where the center of 𝔨 comes first), Δ_{𝔨}^{+} and Δ_{𝔭}^{+} the set of positive compact and noncompact roots, respectively. Put 𝔱_{ℝ} := i𝔱 ⊂ 𝔱_{ℂ}. We identify 𝔱_{ℝ} with its own dual via the Cartan–Killing form. Then we can consider Δ ⊂ 𝔱_{ℝ}. Let P ⊂ 𝔱_{ℝ}^{∗} ≃ 𝔱_{ℝ} be the set of weights relative to T and let R be the set of all highest weights relative to Δ_{𝔨}^{+} ,

(1.1)  R = {λ ∈ P | (∀α ∈ Δ_{𝔨}^{+}) ⟨λ, α⟩ ≥ 0}.

Let ρ be the half sum of all positive roots. Then by Harish–Chandra ([9], [10], [11]) the holomorphic discrete series representations for the group G are those irreducible unitary representations of G that are square–integrable with a highest weight λ belonging to

(1.2)  R′ = {λ ∈ R | (∀β ∈ Δ_{𝔭}^{+}) ⟨λ + ρ, β⟩ < 0}.

We will say that λ ∈ R satisfies the Harish–Chandra condition if

(1.3)  ⟨λ + ρ, β⟩ < 0, ∀β ∈ Δ_{𝔭}^{+}.

By Vinberg ([25]), there exists in Cone_{G} (i𝔤) a unique (up to multiplication by −1) maximal cone C_{max}, such that

C_{max} ∩ 𝔱_{ℝ} = c_{max} := {X ∈ 𝔱_{ℝ} | (∀α ∈ Δ_{𝔭}^{+}) ⟨X, α⟩ ≥ 0},

and a unique minimal cone C_{min} = C_{max}^{∗}, such that C_{min} ∩ 𝔱_{ℝ} = c_{min} is the convex cone spanned by all α in Δ_{𝔭}^{+} .

A unitary representation π of G in a Hilbert space H is said to be C−dissipative if for all X ∈ C and all ξ ∈ H^{∞}, the space of C^{∞} vectors in H,

(π(X)ξ|ξ) ≤ 0.

We can now state the Theorem B of Olshanskiĭ ([22]) on the non–commutative Hardy spaces

Theorem 1.1. The Hardy space H^{2}(Γ(C)) is a non–trivial Hilbert space for any C ∈ Cone_{G} (i𝔤). The representation of G in H^{2}(Γ(C)) can be decomposed into a direct sum of irreducible unitary representations of G. The components of this decomposition are precisely all the holomorphic discrete series representations of G which are C−dissipative.

The group G × G acts on H^{2}(Γ(C)) via left and right regular representations. Therefore

(1.4)  H^{2}(Γ(C)) = ⨁_{λ∈(C^{∗}∩𝔱_{ℝ})∩R′} π_{λ} ⊗ π_{λ}^{∗},

where π_{λ} is the contraction representation of Γ(C) corresponding to a unitary highest weight representation of G with highest weight λ. Recall here the correspondence as in [21] between holomorphic contraction representations of the semigroup and admissible unitary highest weight representations of the group. In particular, Γ(C) acts by contractions on the Hardy space, and this action is holomorphic on the interior Γ(C)^{∘}. Morever, the corresponding function of the Cauchy–Szegö kernel K of H^{2}(Γ(C)) can be written on Γ(C)^{∘} as follows

(1.5)  K(γ) := K(γ, e) = ∑_{λ∈(C^{∗}∩𝔱_{ℝ})∩R′} d_{λ}tr(π_{λ}(γ)),

where d_{λ} denotes the formal dimension of the representation π_{λ}. The series for K converges uniformly on compact subsets in Γ(C)^{∘}.

Remark 1.2. Whenever C is the minimal cone the decompositions (1.4) and (1.5) are over all the holomorphic discrete series, namely over λ ∈ R′.

One of the most important problems in this area is to give an explicit formula for the function K(γ).

2. The contraction semigroup in G_{ℂ} and the Cayley transform

From now on we assume that G is one of the classical groups U(p, q), Sp(r, ℝ) or SO^{∗}(2l). Let σ be an involution on 𝔤_{ℂ} such that

𝔤 = {X ∈ 𝔤_{ℂ} | σ(X) = −X}.

Then

σ(X) = JX^{∗}J,

where X^{∗} is the adjoint matrix and

for 𝔤 = 𝔲(p, q) J = (\begin{matrix}−I_{p} & 0 \\ 0 & I_{q} \end{matrix})
for 𝔤 = 𝔰𝔭(r, ℝ) J = (\begin{matrix}−I_{r} & 0 \\ 0 & I_{r} \end{matrix})
for 𝔤 = 𝔬^{∗}(2l) J = (\begin{matrix}−I_{l} & 0 \\ 0 & I_{l} \end{matrix}) .

Remark 2.1. U(p, q) is not a simple Hermitian Lie group. Since

U(p, q) ≃ (U(1) × SU(p, q))/ℤ_{p+q},

the holomorphic discrete series representations of U(p, q) are the holomorphic discrete series representations of the circle times the Hermitian group SU(p, q) which are trivial on (ζ, ζ^{−1}I_{n}) where n = p + q and ζ^{n} = 1. Therefore one can easily generalize the results of section 1 to the reductive group U(p, q); details will be given in section 7.

Let C be the regular cone in i𝔤 defined by

(2.1),  C := {X ∈ i𝔤 | JX ≤ 0}

and let Γ(C) := G exp(C) be the corresponding Ol’shanskiĭ semigroup. An element γ of G_{ℂ} is said to be a J−contraction (resp. a strict J−contraction) if J − γ^{∗}Jγ ≥ 0 (resp. J − γ^{∗}Jγ ≫ 0).

Proposition 2.2. The semigroup Γ(C) is the J−contraction semigroup,

Γ(C) = {γ ∈ G_{ℂ} | J − γ^{∗}Jγ ≥ 0},

and Γ(C)^{∘} is the semigroup of strict J−contractions,

Γ(C)^{∘} = {γ ∈ G_{ℂ} | J − γ^{∗}Jγ ≫ 0}.

Proof. The case G = U(p, q) is done by Hilgert and Neeb in [13]. For G = Sp(r, ℝ), Γ_{Sp(r,ℝ)} = G_{ℂ} ∩Γ_{U(r,r)} and for G = SO^{∗}(2l), Γ_{SO^{∗}(2l)} = G_{ℂ} ∩Γ_{U(l,l)} .

Let V be one of the Jordan algebras Herm(n, ℂ), Sym(2r, ℝ) or Herm(l, ℍ) and let Ω be the corresponding symmetric cone. Then Ω = V^{+} is the set of positive definite matrices in V . The tube domain T_{Ω} := V + iΩ is a Hermitian symmetric space isomorphic to G^{♭}/K^{♭}, where G^{♭} is SU(n, n), Sp(2r, ℝ) or SO^{∗}(4l) respectively and K^{♭} the corresponding maximal compact subgroup, i.e. S(U(n) × U(n)), U(2r) or U(2l) respectively.

Let C be the Cayley transform defined by

(2.2)  C(Z) := (Z − iJ)(Z + iJ)^{−1}

whenever the matrix (Z + iJ) is invertible.

Proposition 2.3. The Cayley transform C is a biholomorphic bijection from an open subset of the tube domain T_{Ω} onto the complex manifold Γ(C)^{∘}. More precisely, if Σ denotes the hypersurface Σ = {Z ∈ T_{Ω} | det(Z + iJ) = 0}, then

(2.3)  C(T_{Ω} \ Σ) = Γ(C)^{∘}.

Proof. We use the similar arguments as in the proof of Lemma 1.1 in [18]. Here “det” denotes the Koecher norm in the Jordan algebra V_{ℂ} (cf. [5]).

A crucial point in this paper is to compare holomorphic functions on the tube domain with their pull-backs on the semigroup via the Cayley transform, and vice versa. In particular, it will be important to know the rate of growth of the functions near the singularity Σ above. Assuming γ = C(Z) we have that

Z + iJ = 2(I − γ)^{−1}iJ

so that to approach the singularity in the Z variable, means that det(I−γ) tends to infinity in the γ variable. Clearly this condition is invariant under conjugation with G, so we may reduce the question of the growth near the singularity to a question on the compact Cartan subspace. Suppose the holomorphic functions f and F are related by

f(Z) = det(I − γ)^{p}F (γ)

so that F is holomorphic on Γ(C)^{∘} and f therefore holomorphic on T_{Ω} \ Σ. Then for f to admit a holomorphic continuation to all of T_{Ω} it is necessary and sufficient that it stays bounded as the determinant factor tends to infinity, i.e. that F satisfies a decay condition related to p. This is what we shall make precise in the following.

Lemma 2.4. Given the correspondence as above between f and F and suppose F is a matrix coefficient of a holomorphic representation of the semigroup relative to an orthonormal basis v_{0}, v_{1}, .., of weight vectors for the compact Cartan subspace 𝔱_{ℝ} such that γv_{i} = γ^{b_{i}}v_{i} and on exp(C^{∘} ∩ 𝔱_{ℝ}) we have that γ^{b_{0}} ≥ γ^{b_{i}} for all i. Then the singularity of f can be removed provided det(I −γ)^{p}γ^{b_{0}} stays bounded when the determinant factor tends to infinity along exp(C^{∘} ∩ 𝔱_{ℝ}). Proof. By Vinberg [25] each element of C^{∘} is conjugate to an element of 𝔱_{ℝ}, so to look at an arbitrary matrix coefficient along the compact Cartan subspace is sufficient to decide the removability of the singularity. Namely, we have

(γ ∑_{i} a_{i}v_{i}, ∑_{i} a_{i}v_{i}) ≤ γ^{b_{o}}(∑_{i} a_{i}v_{i}, ∑_{i} a_{i}v_{i})

which shows the assertion, namely that the rate of growth in the direction of the singularity is controlled by b_{0} .

3. The holomorphic discrete series for G^{♭}

All definitions and results in this section are taken from [5] and [24].

Let N and R be the dimension and the rank of the Jordan algebra V . For a complex manifold M we denote by O(M) the space of holomorphic functions on M.

The group G^{♭} acts on T_{Ω} via

g · Z = (AZ + B)(CZ + D)^{−1}, g = (\begin{matrix}A & B \\ C & D \end{matrix}) ,

and the scalar–valued holomorphic discrete series representations of G^{♭} are

(U_{λ}(g)f)(Z) = det(CZ + D)^{−λ}f(g^{−1} · Z), g^{−1} = (\begin{matrix}A & B \\ C & D \end{matrix})

for λ ≥ 2\frac{N}{R} , which all are unitary and irreducible in the Hilbert spaces

H_{λ}(T_{Ω}) := \{f ∈ O(T_{Ω}) | ∫_{T_{Ω}} |f(X + iY)|^{2}det(Y)^{λ−2 \frac{N}{R}} dXdY < ∞\}.

Morever the reproducing kernel of H_{λ}(T_{Ω}) is given by

K_{λ}^{T_{Ω}}(Z, W) = det(\frac{Z − W^{∗}}{2i} )^{−λ}.

The classical Hardy space H^{2}(T_{Ω}) on T_{Ω} is defined as the space of holomorphic functions f on T_{Ω} such that

sup_{Y ∈Ω} ∫_{V} |f(X + iY)|^{2} dX < ∞.

Proposition 3.1. The Hardy space H^{2}(T_{Ω}) may be thought of as the space H_{λ}(T_{Ω}) for λ = \frac{N}{R} , and the Cauchy–Szegö kernel of T_{Ω} is given by

(3.1)  K(Z, W) = det(\frac{Z − W^{∗}}{2i} )^{−N/R}.

We list here the groups G and the corresponding group G^{♭}, Jordan algebra V , its rank R, its dimension N, and the Koecher norm det :

G  G^{♭}  V  N  R  det
Sp(r, ℝ) Sp(2r, ℝ) Sym(2r, ℝ) r(2r + 1) 2r Det
SO^{∗}(2l) SO^{∗}(4l) Herm(l, ℍ) l(2l − 1) l Det^{1/2}
U(p, q) SU(n, n) Herm(n, ℂ)  n^{2}  n  Det

4. The conformal image of the holomorphic discrete series for G^{♭}

As in [18], we wish to make a correspondence between holomorphic functions on T_{Ω} and holomorphic functions on Γ(C)^{∘}. First, we can easily extend Proposition 1.5 in [18] to any Euclidean Jordan algebra.

Proposition 4.1. Let f ∈ O(T_{Ω} \ Σ) such that

∫_{T_{Ω}} |f(Z)|^{2}det(Y)^{λ−2 \frac{N}{R}} dXdY < ∞,

for λ ≥ 2\frac{N}{R} . Then f is actually holomorphic on all of T_{Ω} .

Let λ ≥ 2 \frac{N}{R} and introduce the Hilbert space

H_{λ}(Γ(C)) = {F ∈ O(Γ(C)^{∘}) | ∫_{Γ(C)^{∘}} |F (γ)|^{2}dν_{λ}(γ) < ∞},

where dν_{λ} is the Lebesgue measure of G_{ℂ} restricted to Γ(C)^{∘} times the density

det(J − γ^{∗}Jγ)^{λ−2 \frac{N}{R}} .

The Cayley transform γ = C(Z) gives as in Lemma 2.4 a correspondence between O(T_{Ω} \ Σ) and O(Γ(C)^{∘}), namely,

(4.1)  f = C_{λ}(F) : f(Z) = det(Z + iJ)^{−λ}F (γ).

Theorem 4.2. Let λ ≥ 2 \frac{N}{R} . If F belongs to H_{λ}(Γ(C)), then the function f = C_{λ}(F) belongs to H_{λ}(T_{Ω}) and the map C_{λ} is a unitary linear isomorphism from H_{λ}(Γ(C)) onto H_{λ}(T_{Ω}). Furthermore, H_{λ}(Γ(C)) is a reproducing kernel Hilbert space with the kernel

K_{λ}(γ_{1}, γ_{2}) = det(J − γ_{2}^{∗}Jγ_{1})^{−λ}.

Proof. This is just like Theorem 1.7 in [19], using the above Proposition.

5. The Hardy space of the metaplectic semigroup

In this section we assume that G = Sp(r, ℝ). Then the Hardy parameter is \frac{N}{R} = r + \frac{1}{2} ∈ ℤ + \frac{1}{2} and the Koecher norm “det” coincides with the usual matrix determinant “Det” . This suggests that the operator C_{\frac{N}{R}} = C_{r+ \frac{1}{2}} ,

(5.1)  f = C_{r+ \frac{1}{2}} (F) : f(Z) = Det(Z + iJ)^{−(r+ \frac{1}{2} )}F (γ)

may be an intertwining operator between H^{2}(T_{Ω}) and the odd part of the Hardy space H^{2}(Γ(C)_{2}) on the double covering Γ(C)_{2} of Γ(C).

The appearance of a square root in (5.1) means that we have to consider double coverings defined in terms of the determinant function on the right-hand side of (5.1). This is another main point of this paper. We shall use the following two general principles:

Principle 1. Suppose G × M → M is a holomorphic action of a Lie group G on a simply connected complex manifold M, and that there is given a non-vanishing cocycle λ : G × M → ℂ^{∗} which is holomorphic in the second variable and satisfies

λ(g_{1}g_{2}, z) = λ(g_{1}, g_{2} · z)λ(g_{2}, z)

for all g_{1}, g_{2} ∈ G, z ∈ M. Then we can define

G_{2} := {(g, ω(g, ·)) | g ∈ G, ω(g, ·)^{2} = λ(g, ·)}

where z → ω(g, z) is holomorphic on M. Since we are on a simply connected space, this holomorphic square root is well-defined up to a sign change. G_{2} is a double covering of G, and it is a Lie group endowed with the product

(g_{1}, ω(g_{1}, z))(g_{2}, ω(g_{2}, z)) = (g_{1}g_{2}, ω(g_{1}, g_{2} · z)ω(g_{2}, z))

with z on both sides viewed as a holomorphic variable. Note that this product is well-defined, since on the right-hand side we indeed have a holomorphic square root of λ(g_{1}g_{2}, z). The same construction applies to Lie semigroups. This product is associative, since for either position of the parentheses the ω part of the element becomes

ω(g_{1}, g_{2}g_{3} · z)ω(g_{2}, g_{3} · z)ω(g_{3}, z)

so (h_{1}h_{2})h_{3} = h_{1}(h_{2}h_{3}) where h_{1} = (g_{1}, ω(g_{1}, z)) etc. This principle is also to be applied in (5.3). Note finally that ω defines a cocycle for G_{2} , namely ω(g̃, z) = ω(g, z) where g̃ = (g, ω(g, z)). By definition of the product in G_{2} this satisfies the cocycle relation, similar to the one for λ above.

Principle 2. Let N be a complex manifold (not necessarily simply connected) and ϕ a nowhere vanishing holomorphic function on N. Then we can define

N_{2} := {(z, w) | z ∈ N, w ∈ ℂ, w^{2} = ϕ(z)}

which will be a holomorphic double cover of N. By construction, ϕ(z)^{1/2} is a well-defined holomorphic function on N_{2} .

An abstract construction of double (and universal) covering semigroups can be found in much detail in [13]. For the open subset Γ(C)_{2}^{∘} we have a new and explicit construction; but first we recall (in a way suitable to our choices) how to obtain the double cover G_{2} of G, based on a slight modification via the Cayley transform of Principle 1 above:

Let J^{♭} = (\begin{matrix}J & 0 \\ 0 & J \end{matrix} ) and let G^{♭} := Sp(2r, ℝ) be the group of all matrices in Sp(2r, ℂ) satisfying

g^{∗}J^{♭}g = J^{♭}.

We imbed G in a natural way in G^{♭} as follows :

g ⟼ ( \begin{matrix}g & 0 \\ 0 & I_{2r} \end{matrix} ) .

We also view the Cayley transform C as the element of G_{ℂ}^{♭} given by the matrix

C = \frac{1}{\sqrt{2}} ( \begin{matrix}I_{2r} & −iJ \\ I_{2r} & iJ \end{matrix} ) .

Our precise definition of G_{2} is to be the set of all pairs (g, ω(gc, ·))

with g ∈ Sp(r, ℝ), gc = C^{−1} g C = (\begin{matrix}A & B \\ C & D \end{matrix}), ω(gc, ·)^{2} = Det(C · +D)^{−1} and

Z ⟶ ω(gc, Z) is holomorphic on T_{Ω} . Note that this is analogous to the definition of the double cover of SU(1, 1), where we take all pairs (g, \sqrt{cz + d})

with g = ( \begin{matrix}a & b \\ c & d \end{matrix} ) ∈ SU(1, 1) and \sqrt{cz + d} a holomorphic choice of square

root of the non-zero function cz + d on the unit disc. Indeed, it sometimes is convenient to think in terms of such multivalued functions when doing practical calculations, but of course, the precise definition is behind this. We also recall the more informal definition of G_{2} as follows:

Take again Z ∈ T_{Ω} and g ∈ Sp(r, ℝ) such that C^{−1} g C = (\begin{matrix}A & B \\ C & D \end{matrix}). A

determination on T_{Ω} of the square root Det(CZ+D)^{− \frac{1}{2}} is completely determined by its value on Z = iI. For each g ∈ Sp(n, ℝ) we choose a determination of Det(CZ +D)^{− \frac{1}{2}} as in Principle 1, noting that this is a global determination. We consider here Z as a variable, since the group (and indeed all contractions) acts on the tube domain, and we consider the function

(5.2)  Z ⟼ ω(gc, Z) := Det(CZ + D)^{− \frac{1}{2}}

from T_{Ω} into ℂ \ {0}, where gc = C^{−1} g C. We read ω as “a holomorphic choice of square root of the determinant”. It follows from Principle 1 that ω may be viewed as a cocycle for G_{2} , and it gives a choice of square root at the product of two elements as follows:

ω(gc_{1}gc_{2}, Z) = ω(gc_{1}, gc_{2} · Z)ω(gc_{2}, Z).

This equation is to be understood as an equation for the two-valued function ω; it does not hold for any single-valued function. More generally, assuming the determinant to be non-zero, we let

ω_{2}((\begin{matrix}A & B \\ C & D \end{matrix}) , Z) := Det(CZ + D)^{−1}

and correspondingly ω((\begin{matrix}A & B \\ C & D \end{matrix}) , Z) a choice of one of the two square roots of this, either (as here) global and holomorphic, or (as below) local, i.e. at the fixed point Z. Then we may consider our double covering group to be

G_{2} := {g̃:= (g, ω(gc, ·)) | g ∈ Sp(r, ℝ)},

endowed with the group law

(5.3)  (g_{1}, ω(gc_{1}, Z)) (g_{2}, ω(gc_{2}, Z)) = (g_{1}g_{2}, ω(gc_{1}, gc_{2} · Z)ω(gc_{2}, Z)) .

G_{2} is a two–sheeted covering group of G, since we are considering both choices of square root. G_{2} is called the metaplectic group.

Now we wish to apply Principle 2 to give another (and by Lemma 5.2 isomorphic) version of the double covering construction. Here N will be the open semigroup, realized as a subset of the tube domain as in Proposition 2.3. For Z ∈ T_{Ω} \ Σ and for a choice of a local determination of Det(Z + iJ)^{− \frac{1}{2}} we note that up to a constant

Det(Z + iJ)^{− \frac{1}{2}} = ω(C, Z).

This is again an identity between two-valued functions. Hence at each fixed point Z we make a choice between the two possible values of the square root, so here the notation does not consider Z as a variable. Note that we may extend our cocycle to the complexified group in the natural way. Therefore, the complex manifold

Γ(C)_{2}^{∘} := {γ̃ = (γ, ω(C, Z)) | γ ∈ Γ(C)^{∘}, γ = C(Z), Z ∈ T_{Ω} \ Σ},

is a two–sheeted covering of the semigroup Γ(C)^{∘}. As before, we consider both choices of square root here, and corresponding to the modern point of view, the more precise definition is as follows:

Γ(C)_{2}^{∘} = {γ̃ = (γ, w) ∈ Γ(C)^{∘} × ℂ | γ = C(Z), Z ∈ T_{Ω} \ Σ, w^{2} = Det(Z + iJ)^{−1}}.

In particular, w is just a complex number.

Lemma 5.1. The group G_{2} acts on the right on the manifold Γ(C)_{2}^{∘} Proof. We define the action at the same time: Indeed, letting Z′ satisfy g^{−1}γ = C · Z′, which implies that Z = gc · Z′,

(γ, ω(C, Z)) · (g, ω(gc, ·)) = (g^{−1}γ, ω(C, Z)ω(gc, Z′)) by definition
= (g^{−1}γ, ω(C gc, Z′))
= (g^{−1}γ, ω(g C, Z′))
= (g^{−1}γ, ω(g, C · Z′)ω(C, Z′))
= (g^{−1}γ, ω(C, Z′))

because ω_{2}(g, C · Z′) = 1.

In this calculation, Z and Z′ are fixed points, and the cocycles indicate choices of square roots at the relevant points. Indeed, the same calculation holds for ω_{2} instead, and the ω then is a complex number with the relevant square. To see that we have indeed defined a group action, we note that the two sides of the equation checking that we have an action both have as ω part

ω(C, Z)ω(gc_{1}, gc_{2} · Z′′)ω(gc_{2}, Z′′)

where gc_{2} · Z′′ = Z′ and Z = gc_{1} · Z′. To show that Γ(C)_{2}^{∘} is a semigroup we consider just like for G_{2} the following manifold

Γ(C)_{2}^{∘}′ := {(γ, ω(γc, ·)) | γ ∈ Γ(C)^{∘}}.

It is clear that Γ(C)_{2}^{∘}′ is a double covering of Γ(C)^{∘} and has a semigroup structure with respect to the law (5.3). See Principle 1 and the remark following (5.3).

Consider the map φ from Γ(C)_{2}^{∘}′ to Γ(C)_{2}^{∘} defined by

(γ, ω(γc, ·)) ⟼ (γ, ω(C, Z)), where γ = C(Z) ∈ Γ(C)^{∘}.

This is to be understood as using the relation (up to a constant)

ω_{2}((−γ)c, Z_{o}) = Det(I − γ)^{−1}

where Z_{o} = 0 and γ ∈ Γ(C)^{∘}. Hence a choice of global square root corresponds to a choice of local square root in a one-to-one way, since a global choice is determined by its value at a single point. Thus we have

Lemma 5.2. φ is a homeomorphism from Γ(C)_{2}^{∘}′ onto Γ(C)_{2}^{∘}.

Remark 5.3. The semigroup Γ(C)_{2}^{∘} is isomorphic to the interior of the metaplectic semigroup or the Howe oscillator semigroup. We call it the open metaplectic semigroup.

The Hardy space H^{2}(Γ(C)_{2}) on the metaplectic semigroup Γ(C)_{2} is the space of holomorphic functions F ∈ O(Γ(C)_{2}^{∘}) such that

sup_{_{γ}∈Γ(C)_{2}^{∘}} ∫_{G_{2}} |F (γ̃ g̃)|^{2} d g̃ < ∞.

The maximal compact subgroup K of G = Sp(r, ℝ) is isomorphic to U(r) and the maximal split abelian subalgebra

𝔱_{ℝ} = \{ (\begin{matrix}X & 0 \\ 0 & −X \end{matrix}) ∈ M(r × r, ℝ) | X = ( ..._{x_{r}}^{x_{1}} )\},

can be identified with ℝ^{r}. Let ϵ_{1}, …, ϵ_{r} be the canonical basis of 𝔱_{ℝ}^{∗} = 𝔱_{ℝ} = ℝ^{r}. Then the root system Δ = Δ(𝔤_{ℂ}, 𝔱_{ℂ}) is of type C_{r} :

Δ = {±(ϵ_{i} ± ϵ_{j}) (1 ≤ i < j ≤ r) , ±2ϵ_{i} (1 ≤ i ≤ r)}
Δ^{+} = {ϵ_{i} ± ϵ_{j} (1 ≤ i < j ≤ r) , 2ϵ_{i} (1 ≤ i ≤ r)}
Δ_{𝔨}^{+} = {ϵ_{i} − ϵ_{j} (1 ≤ i < j ≤ r)}
Δ_{𝔭}^{+} = {ϵ_{i} + ϵ_{j} (1 ≤ i ≤ j ≤ r)}
ρ = rϵ_{1} + (r − 1)ϵ_{2} + … + ϵ_{r}
≃ (r, r − 1, …, 1).

Furthermore P is the lattice ℤ^{r}, the set of highest weights relative to Δ_{𝔨}^{+} is given by

R = {λ = (λ_{1}, …, λ_{r}) ∈ ℤ^{r} | λ_{1} ≥ … ≥ λ_{r}},

and λ ∈ R satisfies the Harish–Chandra condition if

−r > λ_{1} ≥ … ≥ λ_{r},

which gives the set R′.

Let K_{2} ⊂ G_{2} , resp. T_{2} ⊂ K_{2} be the corresponding covering of K and T. Then the corresponding P_{2} , R_{∈} and R′_{∈} are given by

P_{2} = ℤ^{r} ∪ (ℤ^{r} + \frac{1}{2}) = P ∪ (P + \frac{1}{2}) = P_{2,even} ∪ P_{2,odd},
R_{2} = {λ ∈ P_{2} | (∀α ∈ Δ_{𝔨}^{+}) ⟨λ, α⟩ ≥ 0}

= \{λ = (λ_{1}, …, λ_{r}) ∈ ℤ^{r} ∪ (ℤ^{r} + \frac{1}{2}) | λ_{1} ≥ … ≥ λ_{r}\},

= R_{2,even} ∪ R_{2,odd}

R′_{∈} = {λ ∈ R_{2} | ⟨λ + ρ, β⟩ < 0, ∀β ∈ Δ_{𝔭}^{+}},

= \{λ = (λ_{1}, …, λ_{r}) ∈ ℤ^{r} ∪ (ℤ^{r} + \frac{1}{2}) | −r > λ_{1} ≥ … ≥ λ_{r}\},

= R′_{2,even} ∪ R′_{2,odd},

where \frac{1}{2} stands for the tuple (\frac{1}{2}, …, \frac{1}{2}). The holomorphic discrete series representations for the metaplectic group G_{2} are those irreducible unitary representations π_{λ} of G_{2} that are square–integrable with a highest weight λ ∈ R′_{2} = R′_{2,even} ∪ R′_{2,odd} . Therefore

(5.4)  H^{2}(Γ(C)_{2}) = ⨁_{λ∈(C^{∗}∩𝔱_{ℝ})∩R′_{2}} π_{λ} ⊗ π_{λ}^{∗}.

The cone C is the minimal one in i𝔤, so the above summation is over R′_{2} and the Hardy space H^{2}(Γ(C)_{2}) splits into two parts, namely, even and odd part,

H^{2}(Γ(C)_{2}) = H_{even}^{2}(Γ(C)_{2}) ⊕ H_{odd}^{2}(Γ(C)_{2})

= ( ⨁_{λ∈R′_{2,even}} π_{λ} ⊗ π_{λ}^{∗}) ⊕ ( ⨁_{λ∈R′_{2,odd}} π_{λ} ⊗ π_{λ}^{∗}).

Remark 5.4. The even part

H^{2}(Γ(C)_{2})_{even} = ⨁_{λ∈R′_{2,even}} π_{λ} ⊗ π_{λ}^{∗}

= ⨁_{{(λ_{1},...,λ_{r})∈ℤ^{r} \atop −r>λ_{1}≥...≥λ_{r}}} π_{λ} ⊗ π_{λ}^{∗}

coincides with the Hardy space H^{2}(Γ(C)) on the semigroup Γ(C). For this space Olshanskiĭ (cf. [23]) gives an explicit formula for the Cauchy–Szegö kernel using a Littlewood-type combinatorial formula : for γ ∈ Γ(C)^{∘} with eigenvalues x_{1}, …, x_{r}, x_{1}^{−1}, …, x_{r}^{−1} where |x_{1}|, …, |x_{r}| < 1,

L(γ) = K(γ, e) = ∏_{i=1}^{r} \frac{x_{i}^{r+1}}{(1 + x_{i})(1 − x_{i})^{2r+1}} .

Our goal in this section is to identify the odd part

\begin{aligned}H^{2}(Γ(C)_{2})_{odd} & = ⨁_{λ∈R′_{2,odd}} π_{λ} ⊗ π_{λ}^{∗} \\ & = ⨁_{{(λ_{1},...,λ_{r})∈ℤ^{r}+ \frac{1}{2} \atop −(r+ \frac{1}{2} )≥λ_{1}≥...≥λ_{r}}} π_{λ} ⊗ π_{λ}^{∗} \end{aligned}

with the classical Hardy space H^{2}(Sp(2r, ℝ)/U(2r)).

Theorem 5.5. The operator C_{r+ \frac{1}{2}} given by (5.1) induces a unitary isomorphism

H^{2}(Γ(C)_{2})_{odd} ≃ H^{2}(Sp(2r, ℝ)/U(2r)).

Proof. From (5.1), C_{r+ \frac{1}{2}}^{−1} imbeds the classical Hardy space into H^{2}(Γ(C)_{2})_{odd} . This is because the multiplier factor in the transformation is chosen as the square root of the Radon-Nikodym derivative, i.e. the L^{2} norms on the respective Shilov boundaries agree. The image is inside the odd functions because Det(Z +iJ)^{−1/2} is a well-defined odd function (with respect to the group ℤ_{2} of deck transformations) and F (γ) is even – hence the product is an odd function. It suffices then to prove that C_{r+ \frac{1}{2}}^{−1} is onto. For this we will show that for every representation of the odd part of the holomorphic discrete series for G_{2} the corresponding matrix coefficients extend to the tube domain T_{Ω} .

We shall use Lemma 2.4 to find the necessary and sufficient condition for the removal of singularities i.e. to estimate the growth of the function f pulled back from a matrix coefficient. This turns out below as exactly Harish-Chandra’s condition for the holomorphic discrete series.

Let π_{λ} be among these representations with the highest weight λ = (λ_{1}, …, λ_{r}) ∈ ℤ^{r} + \frac{1}{2} , λ_{1} ≥ … ≥ λ_{r} and let v_{λ} be a corresponding normalized highest weight vector. Take γ̃ ∈ Γ(C)_{2} , with γ ∈ exp(C_{min} ∩ 𝔱_{ℝ}). Then γ is of the form

γ = ( ..._{e^{sr}}^{e^{s_{1}}} ) , s_{i} ≥ 0

and

F (γ̃) := (π_{λ}(γ̃)v_{λ}, v_{λ}) = γ^{λ} = e^{s_{1}λ_{1}} ⋯ e^{s_{r}λ_{r}}.

Then the corresponding function on T_{Ω} is given (up to a constant factor) by

f(Z) = Det(I − γ)^{r+ \frac{1}{2}} F (γ̃)

= ((1 − e^{s_{1}})^{r+ \frac{1}{2}} ⋯ (1 − e^{s_{r}})^{r+ \frac{1}{2}} )(e^{s_{1}λ_{1}} ⋯ e^{s_{r}λ_{r}})

= e^{s_{1}(λ_{1}+r+ \frac{1}{2} )} ⋯ e^{s_{r}(λ_{r}+r+ \frac{1}{2} )} + ⋯

∼_{s_{1},...,s_{r}→+∞}e^{s_{1}(λ_{1}+r+ \frac{1}{2} )}.

Thus, to remove the singularity of the holomorphic function on T_{Ω} we need to take

−(r + \frac{1}{2}) ≥ λ_{1}

and this is exactly the Harish-Chandra condition for G_{2} we gave above. The sufficiency of this condition comes from Lemma 2.4.

Corollary 5.6. Under the action of Mp(r, ℝ) × Mp(r, ℝ) the Hardy space H^{2}(T_{Ω}) can be decomposed into a direct sum of the ‘odd’ holomorphic discrete series representations of Mp(r, ℝ), i.e.

H^{2}(Sp(2r, ℝ)/U(2r))_{|_{Mp(r,ℝ)×Mp(r,ℝ)}} = ⨁_{λ∈R′_{2,odd}} π_{λ} ⊗ π_{λ}^{∗}.

Corollary 5.7. Let K_{odd} be the kernel corresponding to H^{2}(Γ(C)_{2})_{odd} . Then for every γ̃_{1}, γ̃_{2} ∈ Γ(C)_{2}

K_{odd}(γ̃_{1}, γ̃_{2}) = Det(J − γ_{2}^{∗}Jγ_{1})^{−(r+1/2)}.

Proof. This is a simple application of (3.1) and the isomorphism in Theorem 5.5.

Corollary 5.8. On the interior of the metaplectic semigroup the holomorphic function Det(I − γ)^{−(r+1/2)} has the following expansion

Det(I − γ)^{−(r+1/2)} = ∑_{λ∈R′_{2,odd}} d_{λ}tr(π_{λ}(γ)),

where d_{λ} is the formal dimension of π_{λ}.

The Bergman space on Γ(C) is H_{2r+1}(Γ(C)) and its reproducing kernel is given by

K_{B}(γ_{1}, γ_{2}) = Det(J − γ_{2}^{∗}Jγ_{1})^{−(2r+1)}.

Corollary 5.9. The Bergman kernel K_{B} on the semigroup Γ(C) is the square of the odd part K_{odd} of the Cauchy-Szegö kernel for Γ(C)_{2} .

6. The case of G = SO^{∗}(2l)

Let G = SO^{∗}(2l) realized as a subgroup of U(l, l),

G = \{g ∈ SO^{∗}(2l, ℂ) | g^{∗}Jg = J\}, J = ( \begin{matrix}−I_{l} & 0 \\ 0 & I_{l} \end{matrix} ) .

The Hardy parameter in this case is N/R = l(2l − 1)/l = 2l − 1 and the Koecher norm “det” is the square root of the usual determinant “Det” (det = Det^{1/2} ). Thus the operator C_{\frac{N}{R}} = C_{2l−1} ,

(6.1)  f = C_{2l−1}(F) : f(Z) = Det(Z + iJ)^{−(l−1/2)}F (γ)

provides an equivariant embedding of the classical Hardy space H^{2}(SO^{∗}(4l)/U(2l)) into the odd part of the Hardy space H^{2}(Γ(C)_{2})_{odd} on the double covering semigroup Γ(C)_{2} of the minimal semigroup

Γ(C) = {γ ∈ SO^{∗}(2l, ℂ) | J − γ^{∗}Jγ ≥ 0}

(because of the square root in Det(Z + iJ)^{(l−1/2)} ). This is just like the symplectic case; again the power of the determinant is chosen so that L^{2} norms are preserved. We will identify the maximal compact subgroup with U(l) as in the above section. The determinanat factor is again exactly the Jacobian to a power such that we have preservation of L^{2}−norms on the respective boundaries. Then 𝔱_{ℝ} is given by the same formula as in Sp(r, ℝ) case. Let ϵ_{1}, ϵ_{2}, …, ϵ_{l} be the canonical basis of 𝔱_{ℝ}^{∗} = 𝔱_{ℝ} = ℝ^{l}. The root system Δ = Δ(𝔤_{ℂ}, 𝔱_{ℂ}) is of type D_{l} :

Δ = {±ϵ_{i} ± ϵ_{j} | 1 ≤ i < j ≤ l},
Δ^{+} = {ϵ_{i} ± ϵ_{j} | 1 ≤ i < j ≤ l},
Δ_{𝔨}^{+} = {ϵ_{i} − ϵ_{j} | 1 ≤ i < j ≤ l},
Δ_{𝔭}^{+} = {ϵ_{i} + ϵ_{j} | 1 ≤ i < j ≤ l},
ρ = (l − 1)ϵ_{1} + (l − 2)ϵ_{2} + … + ϵ_{l−1}
≃ (l − 1, l − 2, …, 1, 0).

The set of highest weights relative to the positive roots of SO^{∗}(2l) is

R = {λ = (λ_{1} …, λ_{l}) ∈ ℤ^{l} | λ_{1} ≥ … ≥ λ_{l}},

and λ ∈ R satisfies to the Harish-Chandra condition if and only if

(6.2)  −2l + 3 > λ_{1} + λ_{2}.

Therefore, the odd holomorphic discrete serie representations of the double covering group G_{2} of SO^{∗}(2l) are those irreducible unitary representations π_{λ}, square-integrable with a highest weight λ = (λ_{1}, …, λ_{l}) ∈ ℤ^{l} − \frac{1}{2} such that 0 ≥ λ_{1} ≥ … ≥ λ_{l} and satisfying

(6.3)  −2l + 2 ≥ λ_{1} + λ_{2}.

Let R′_{2,odd} denotes the set of these λ’s. The Hardy space H^{2}(Γ(C)_{2}) on the minimal cone Γ(C)_{2} has then the following decomposition

(6.4)  H^{2}(Γ(C)_{2})_{odd} = ⨁_{λ∈R′_{2,odd}} π_{λ} ⊗ π_{λ}^{∗}.

Theorem 6.1. The classical Hardy space H^{2}(SO^{∗}(4l)/U(2l)) is a proper invariant subspace of the “non–classical” Hardy space H^{2}(Γ(C)_{2})_{odd} ⊂ H^{2}(Γ(C)_{2}). Proof. With the same arguments as in the proof of Theorem 5.5, we prove that a unitary irreducible representation π_{λ} of G_{2} continue to the tube domain SO^{∗}(4l)/U(2l) if and only if −(l − 1/2) ≥ λ_{1} , which is different from (6.3).

Corollary 6.2. The representation of SO^{∗}(2l) × SO^{∗}(2l) in the Hardy space H^{2}(Γ(C)) cannot be obtained by a restriction of a representation of the holomorphic discrete series of SO^{∗}(4l) nor any continuation of this, such as the Hardy space.

Let H^{2}(Γ(C)) the conformal image of H^{2}(SO^{∗}(4l)/U(2l)) via the operator C_{2l−1} .

Corollary 6.3. H^{2}(Γ(C)) is a reproducing kernel Hilbert space and its reproducing kernel K is the pre-image of the Cauchy–Szegö kernel of

H^{2}(SO^{∗}(4l)/U(2l)), i.e.,K(γ_{1}, γ_{2}) = Det(J − γ_{2}^{∗}Jγ_{1})^{−(l−1/2)}.

Corollary 6.4. On Γ(C)^{∘} the holomorphic function Det(I − γ)^{−(l−1/2)} has the following expansion

Det(I − γ)^{−(l−1/2)} = ∑_{−l+1/2≥λ_{1}≥...≥λ_{l}} d_{λ}tr(π_{λ}(γ)),

where d_{λ} is the formal dimension of π_{λ}.

7. The case of G = U(p, q)

In this section we fix G = U(p, q) realized by

G = U(p, q) = \{g ∈ GL(n, ℂ) | g^{∗}Jg = J\}, J = ( \begin{matrix}−I_{p} & 0 \\ 0 & I_{q} \end{matrix} )

where n = p + q. In this case the Hardy parameter is N/R = n^{2}/n = n and the Koecher norm “det” is the usual determinant “Det”. Therefore the operator C_{\frac{N}{R}} given by

(7.1)  f = C_{n}(F) : f(Z) = Det(Z + iJ)^{−n}F (γ)

may be an intertwining operator between the Hardy space

H^{2}(SU(n, n)/S(U(n) × U(n)) and the Hardy space H^{2}(Γ(C))

over the semigroup

Γ(C) = {γ ∈ GL(n, ℂ) | J − γ^{∗}Jγ ≥ 0}.

To study unitary representations of G = U(p, q) we identify it with (U(1) × SU(p, q))/ℤ_{p+q}. Thus the unitary irreducible representations of G are those of U(1) × SU(p, q) that are trivial on (ζ, ζ^{−1}I_{n}) as in Remark 2.1, where I_{n} is the identity matrix and ζ^{n} = 1. Therefore, the holomorphic discrete series representations of G that we are interested in are

(7.2)  π_{λ,k}(e^{iθ}g) = e^{ikθ}π_{λ}(g) , g ∈ SU(p, q), θ ∈ ℝ

where k ∈ ℤ and π_{λ} are the holomorphic discrete series representations of SU(p, q), realized on D = SU(p, q)/S(U(p) × U(q)), for example in the scalar case:

(π_{λ}(g)f)(Z) = Det(CZ + D)^{−λ}f((AZ + B)(CZ + D)^{−1}), g^{−1} = ( \begin{matrix}A & B \\ C & D \end{matrix} ) ,

with λ an integer, and in general λ = (λ_{1}, …, λ_{n}) ∈ ℤ^{n}. There will an underlying parity condition to make the representations trivial on ℤ_{n} as above; for example in the scalar case we must have that k − qλ is divisible by n.

Let 𝔱 ⊂ 𝔨 be a Cartan subalgebra consisting of diagonal matrices with purely imaginary values and 𝔱_{ℝ} = i𝔱. Then the root system Δ = Δ(𝔤_{ℂ}, 𝔱_{ℂ}) is of type A_{n−1} :

Δ = {ϵ_{i} − ϵ_{j} | 1 ≤ i ≠ j ≤ n},
Δ^{+} = {ϵ_{i} − ϵ_{j} | 1 ≤ i < j ≤ n},
Δ_{𝔨}^{+} = {ϵ_{i} − ϵ_{j} | 1 ≤ i < j ≤ p or p + 1 ≤ i < j ≤ n},
Δ_{𝔭}^{+} = {ϵ_{i} − ϵ_{j} | 1 ≤ i ≤ p and p + 1 ≤ j ≤ n},
2ρ = (n − 1)ϵ_{1} + (n − 3)ϵ_{2} + … − (n − 3)ϵ_{n−1} − (n − 1)ϵ_{n}
≃ (n − 1, n − 3, n − 5, …, −n + 3, −n + 1),

where ϵ_{1}, …, ϵ_{n} is the canonical basis of 𝔱_{ℝ}^{∗} ≃ 𝔱_{ℝ} ≃ ℝ^{n}. Then the holomorphic discrete series representations of SU(p, q) are the above representations π_{λ} with λ = (λ_{1}, …, λ_{n}) ∈ ℤ^{n} satisfying

λ_{i} − λ_{i+1} ≥ 0, i ≠ p, 1 ≤ i ≤ n − 1,

and the Harish-Chandra condition

λ_{n} − λ_{1} > n − 1.

Let π_{λ,k} be an irreducible unitary representation of G with highest weight (λ, k), λ = (λ_{1}, …, λ_{n}). Then according to (7.2), its highest weight character is given by

χ_{λ,k}(e^{iθ}g) = e^{ikθ}χ_{λ}(g)

where χ_{λ} is the character of the representation π_{λ} of SU(p, q). The cone C ∩ 𝔱_{ℝ} consists of all matrices of rank (n × n),

( \begin{matrix}−T & 0 \\ 0 & S \end{matrix} )

where T is the diagonal matrix with diagonal entries t_{1} ≤ 0, …, t_{p} ≤ 0 and S the diagonal matrix with diagonal entries s_{p+1} ≤ 0, …, s_{n} ≤ 0. Take γ ∈

exp(C ∩ 𝔱_{ℝ}). Then γ is of the form γ = ( \begin{matrix}e^{−T} & 0 \\ 0 & e^{S} \end{matrix} ) with ( \begin{matrix}−T & 0 \\ 0 & S \end{matrix} ) ∈ C ∩ 𝔱_{ℝ}.

Therefore, if Det(γ) = 1,

χ_{λ,k}(γ) = χ_{λ}(γ) = e^{−t_{1}λ_{1}} ⋯ e^{−t_{p}λ_{p}}e^{s_{p+1}λ_{p+1}} ⋯ e^{s_{n−1}λ_{n−1}}.

Thus in this case the C−dissipativity condition is

0 ≥ λ_{1} ≥ … ≥ λ_{p}, λ_{p+1} ≥ … ≥ λ_{n} ≥ 0.

On the other hand if Det(γ) ≠ 1 we put

c = (Det(γ))^{1/n} = e^{−(t_{1}+...+t_{p})/n}e^{(s_{p+1}+...+s_{n})/n},

hence

χ_{λ,k}(γ) = χ_{λ,k}(c ( \begin{matrix}e^{−T} /c & 0 \\ 0 & e^{S}/c \end{matrix} )),

= c^{k}χ_{λ}(( \begin{matrix}e^{−T} /c & 0 \\ 0 & e^{S}/c \end{matrix} )),

= e^{−k(t_{1}+...+t_{p})/n}e^{k(s_{p+1}+...+s_{n})/n}

· e^{λ_{1}(−t_{1}+(t_{1}+...+t_{p})/n−(s_{p+1}+...+s_{n})/n)}
⋮
· e^{λ_{p}(−t_{p}+(t_{1}+...+t_{p})/n−(s_{p+1}+...+s_{n})/n)}
· e^{λ_{p+1}(s_{p+1}+(t_{1}+...+t_{p})/n−(s_{p+1}+...+s_{n})/n)}
⋮
· e^{λ_{n}(s_{n}+(t_{1}+...+t_{p})/n−(s_{p+1}+...+s_{n})/n)},

= e^{t_{1}(\frac{[λ]−k}{n} −λ_{1})} ⋯ e^{t_{p}(\frac{[λ]−k}{n} −λ_{p})}e^{s_{p+1}(\frac{k−[λ]}{n} +λ_{p+1})} ⋯ e^{s_{n}(\frac{k−[λ]}{n} +λ_{n})},

where [λ] = λ_{1} + … + λ_{n}. So, in this case, the C−dissipativity condition is

(7.3)  [λ] − nλ_{n} ≤ k ≤ [λ] − nλ_{1}.

Therefore, π_{λ,k} is a C−dissipative representation of the holomorphic discrete series if and only if (λ, k) belongs to the set R_{diss}

R_{diss} = \{(λ_{1}, …, λ_{n}, k) ∈ ℤ^{n+1} | \{ \begin{matrix}λ_{n} − λ_{1} > n − 1 \\ 0 ≥ λ_{1} ≥ … ≥ λ_{p}, λ_{p+1} ≥ … ≥ λ_{n} ≥ 0 \\ [λ] − nλ_{n} ≤ k ≤ [λ] − nλ_{1} \end{matrix} \}.

Hence the Hardy space on the semigroup Γ(C) has the following decomposition

(7.4)  H^{2}(Γ(C)) = ⨁_{(λ,k)∈R_{diss}} π_{λ,k} ⊗ π_{λ,k}^{∗}.

Theorem 7.1. The classical Hardy space H^{2}(SU(n, n)/S(U(n) × U(n))) is a proper invariant subspace of the “non–classical” Hardy space H^{2}(Γ(C)). Proof. We just compare those two Hardy spaces according to the transformation C_{n} given in (7.1). Let π_{λ,k} be a representation of the holomorphic discrete series of G = U(p, q). The matrix coefficient χ_{λ,k}(γ) = (π_{λ,k}(γ)v_{λ,k}, v_{λ,k}) is a holomorphic function on the semigroup Γ(C)^{∘} (v_{λ,k} is the highest weight vector of highest weight (λ, k)). Put F := Det(I − γ)^{n}χ_{λ,k}(γ). Again, we use Lemma 2.4 - recall the argument: F is holomorphic on Γ(C)^{∘}, so it is determined by its restriction to exp(C^{∘}). Since each X ∈ C^{∘} is conjugate to an element of 𝔱_{ℝ} (see Vinberg [25]), to find the decay condition at infinity in the semigroup to ensure a removal of the singularities on Σ of the holomorphic function F it suffices to check it on exp(C^{∘} ∩ 𝔱_{ℝ}). This is because any other matrix coefficient will have a better decay at the singularity than the highest one above, and by conjugation it is enough to check the removal of singularity on the compact Cartan subspace.

Let γ ( \begin{matrix}e^{−T} & 0 \\ 0 & e^{S} \end{matrix} ) ∈ exp(C^{∘} ∩ 𝔱_{ℝ}), then

Det(I − γ)^{n}χ_{λ,k}(γ) = (1 − e^{−t_{1}})^{n} … (1 − e^{−t_{p}})^{n}(1 − e^{s_{p+1}})^{n} … (1 − e^{s_{n}})^{n}

· e^{t_{1}(\frac{[λ]−k}{n} −λ_{1})} ⋯ e^{t_{p}(\frac{[λ]−k}{n} −λ_{p})}e^{s_{p+1}(\frac{k−[λ]}{n} +λ_{p+1})} ⋯ e^{s_{n}(\frac{k−[λ]}{n} +λ_{n})}

∼_{t_{1},...,s_{n}→−∞} e^{t_{1}(−n−λ_{1}+ \frac{[λ]−k}{n})} ⋯ e^{t_{p}(−n−λ_{p}+ \frac{[λ]−k}{n})}

· e^{s_{p+1}(n+λ_{p+1}+ \frac{k−[λ]}{n})} ⋯ e^{s_{n}(n+λ_{n}+ \frac{k−[λ]}{n})}.

Therefore the decay condition is

[λ] − n(λ_{n} + n) ≤ k ≤ [λ] − n(λ_{1} + n),

which is different from the dissipativity condition in (7.3).

Corollary 7.2. The representation of S(U(p, q) × U(p, q)) in the Hardy space H^{2}(Γ(C)) cannot be obtained by a restriction of a representation of the holomorphic discrete series of SU(n, n) nor any analytic continuation of this, such as the Hardy space.

Note that this should be taken in the strong sense, that for no choice of cone C do the spaces coincide. This is seen by noting that a larger cone will impose dissipativity conditions which only make the spectral overlap smaller. Let H^{2}(Γ(C)) be the conformal image of H^{2}(SU(n, n)/S(U(n) × U(n))) via the operator C_{n}.

Corollary 7.3. H^{2}(Γ(C)) is a reproducing Hilbert space and its reproducing kernel K is the pre-image of the Cauchy-Szegö kernel of H^{2}(SU(n, n)/S(U(n)× U(n))), i.e.

K(γ_{1}, γ_{2}) = Det(J − γ_{2}^{∗}Jγ_{1})^{−n}.

Corollary 7.4. On Γ(C)^{∘} the holomorphic function Det(I − γ)^{−n} has the following expansion

Det(I − γ)^{−n} = ∑_{(λ,k)∈R_{decay}} d_{λ,k}tr(π_{λ,k}(γ)),

where d_{λ,k} is the formal dimension of π_{λ,k} and

R_{decay} = \{(λ_{1}, …, λ_{n}, k) ∈ ℤ^{n+1} | \{ \begin{matrix}λ_{n} − λ_{1} > n − 1 \\ 0 ≥ λ_{1} ≥ … ≥ λ_{p}, λ_{p+1} ≥ … ≥ λ_{n} ≥ 0 \\ [λ] − n(λ_{n} + n) ≤ k ≤ [λ] − n(λ_{1} + n) \end{matrix} \}

8. Remarks and conjectures

Our geometric construction of the double cover semigroup could naturally be extended to the n−fold cover by choosing n’th roots of the Jacobian ; similarly a choice of logarithm could yield the universal cover. Hence we could compare, again via the Cayley transform, modules in the analytic continuation of the scalar holomorphic discrete series for G^{♭} with function spaces on the covering semigroup. For example we saw in [18] how the wave equation representation of SU(2, 2) naturally lives on U(1, 1) (and the semigroup), solving the wave equation here.

Another aspect for the further work is to extend to the remaining groups and symmetric spaces of Hermitian type. We conjecture that in many cases will the non–commutative Hardy spaces be different from their natural classical counterpart, and that in some cases, the double cover will be necessary and indeed natural. We also conjecture that in the bad case the difference between the two Hardy spaces may be a (sum of) non–commutative Hardy space(s) of lower rank. For example in [18], we prove, using the Cauchy–Szegö kernel, that the difference between the classical Hardy space of SU(2, 2) and the non–commutative Hardy space of U(1, 1) is the non–commutative Hardy space of SU(1, 1) studied by Gel’fand and Gindikin in [6]. Hence several of the goals aimed in this paper, and especially that of calculating Cauchy–Szegö kernels, remain open problems in general.

References

[1] Bertram, W., “On some causal and conformal groups,” Preprint, Mittag-Leffler Institute, 1996.

[2] Betten, F., “Kausale Kompaktifizierung kompakt–kausaler Räume,” Dissertation, Göttingen, 1996.

[3] Chadli, M., “Espace de Hardy associé à un espace symétrique de type Cayley,” Preprint, 1996.

[4] Faraut, J., “Hardy spaces in non-commutative harmonic analysis,” Lectures, Summer School, Tuczno, 1994.

[5] Faraut, J., and A. Korànyi, “Analysis on Symmetric Cones,” Oxford University Press, 1994.

[6] Gel’fand, I. M., and S. G. Gindikin, Complex manifolds whose skeletons are semi-simple real Lie groups, and analytic discrete series of representations, Funct. Anal. Appl. 11 (1977), 258–265.

[7] Gindikin, S. G., Generalized conformal structures on classical real Lie groups and related problems on the theory of representations, C. R. Acad. Sci. Paris 315 (1992), 675–679.

[8] —, Fourier transform and Hardy spaces of ¯∂−cohomology in tube domains, C. R. Acad. Sci. Paris 315 (1992), 1139–1143.

[9] Harish–Chandra, Representations of semisimple Lie groups, IV, Amer. J. Math. 77 (1955), 743–777.

[10] —, Representations of semisimple Lie groups, V, Amer. J. Math. 78 (1956), 1–41.

[11] —, Representations of semisimple Lie groups, VI, Amer. J. Math. 78 (1956), 564–628.

[12] Hilgert, J., A note on Howe’s oscillator semigroup, Annal. Inst. Fourier 39, (1989), 663–688.

[13] Hilgert, J., and K.-H. Neeb, “Lie Semigroups and their Applications,” Lectures Notes in Math. (Springer-Verlag) 1552, 1993.

[14] Hilgert, J., and G. Ólafsson, Analytic continuation of representations, the solvable case, Japanese J. Math. 18 (1992), 213–290.

[15] Hilgert, J., G. Ólafsson, and B. Ørsted, Hardy spaces on affine symmetric spaces, J. reine angew. Math 415 (1991), 189–218.

[16] Howe, R., The Oscillator Semigroup, Proc. Symp. Pure Math. 48 (1988), 61–132.

[17] Kashiwara, M., and M. Vergne, “K−types and singular spectrum,” Lecture Notes in Math. (Springer-Verlag) 728, 1978.

[18] Koufany, K., and B. Ørsted, Function Spaces on the Ol’shanskiĭ Semigroup and the Gel’fand-Gindikin Program, Ann. Inst. Fourier 46 (1996), 689–722.

[19] —, Espace de Hardy sur le semi-groupe métaplectique, C. R. Acad. Sci. Paris 322 (1996), 113–116.

[20] Ólafsson, G., and B. Ørsted, “Causal compactification and Hardy spaces,” Preprint No 19, Institut Mittag–Leffler, 1996.

[21] Ol’shanskiĭ, G. I., Invariant cones in Lie algebras, Lie semigroups, and the holomorphic discrete series, Funct. Anal. Appl.15 (1982), 275–285.

[22] —, Complex Lie semigroups, Hardy spaces and the Gel’fand Gindikin program, in: Topics in Group Theory and Homological Algebra, Varoslavl University Press, 1982, 85–98 (Russian); English translation in: Differential Geometry and its Applications 1 (1991), 235–246.

[23] —, Cauchy-Szegö kernels for Hardy spaces on simple Lie groups, J. Lie Theory 5 (1996), 241–273.

[24] Vergne, M., and G. Rossi, Analytic continuation of the holomorphic discrete series of a semi–simple Lie group, Acta Math. 136 (1976), 1–59.

[25] Vinberg, E. B., Invariant cones and orderings in Lie groups, Funct. Anal. App. 14 (1980), 1–10.

Bent Ørsted Matematisk Institut Odense universitet Campusvej 55 DK-5230 Odense M Danmark orsted@imada.ou.dk Khalid Koufany Institut E. Cartan Université H. Poincaré B.P. 239 F-54506 Vandœuvre-Lès-Nancy France koufany@iecn.u-nancy.fr

Received June 27, 1969 and in final form April 16, 1997