Journal of Lie Theory Volume 6 (1996) 69–146 C ◯1996 Heldermann Verlag

On Hyperbolic Cones and Mixed Symmetric Spaces

Bernhard Krötz and Karl-Hermann Neeb

Communicated by J. Hilgert

Abstract. A real semisimple Lie algebra 𝔤 admits a Cartan involution, θ, for which the corresponding eigenspace decomposition 𝔤=𝔨+𝔭 has the property that all operators ad X, X∈𝔭 are diagonalizable over ℝ. We call such elements hyperbolic, and the elements X∈𝔨 are elliptic in the sense that ad X is semisimple with purely imaginary eigenvalues. The pairs (𝔤,θ) are examples of symmetric Lie algebras, i.e., Lie algebras endowed with an involutive automorphism, such that the −1-eigenspace of θ contains only hyperbolic elements. Let (𝔤,τ) be a symmetric Lie algebra and 𝔤= 𝔥+𝔮 the corresponding eigenspace decomposition for τ. The existence of “enough” hyperbolic elements in 𝔮 is important for the structural analysis of symmetric Lie algebras in terms of root decompositions with respect to abelian subspaces of 𝔮 consisting of hyperbolic elements. We study the convexity properties of the action of Inn_{𝔤}(𝔥) on the space 𝔮. The key role will be played by those invariant convex subsets of 𝔮 whose interior points are hyperbolic.

Introduction

It was a fundamental observation of Cartan’s that each real semisimple Lie algebra 𝔤 admits an involutive automorphism, nowadays called Cartan involution, θ for which the corresponding eigenspace decomposition 𝔤 = 𝔨+𝔭 has the property that all operators ad X, X ∈ 𝔭 are diagonalizable over ℝ, we call such elements hyperbolic, and the elements X ∈ 𝔨 are elliptic in the sense that ad X is semisimple with purely imaginary eigenvalues. In this sense the Cartan involutions are the basic tool to separate hyperbolic from elliptic elements and this is why they play such a crucial role in the structure and representation theory of semisimple real groups. The pairs (𝔤, θ) are examples of symmetric Lie algebras, i.e., Lie algebras endowed with an involutive automorphism, such that the −1-eigenspace of θ contains only hyperbolic elements. It even can be shown that, up to adding central factors, this property characterizes the Cartan involutions.

Let (𝔤, τ) be a symmetric Lie algebra and 𝔤 = 𝔥 + 𝔮 the corresponding eigenspace decomposition for τ. The existence of “enough” hyperbolic elements

ISSN 0949–5932 / $2.50 ^{_{C} ◯} Heldermann Verlag in 𝔮 is important in many contexts. For Cartan decompositions it is crucial for the restricted root decomposition of semisimple real Lie algebras, and hence for the whole structure theory of these algebras. There are other important classes of symmetric Lie algebras where the set 𝔮_{hyp} of the hyperbolic elements in 𝔮 still has interior points but is different from all of 𝔮. If (𝔤, τ) is a non-compactly causal symmetric (NCC) Lie algebra in the sense of [8], then 𝔮 contains open convex cones which are invariant under the group Inn_{𝔤}(𝔥) of inner automorphisms of 𝔤 generated by e^{ad 𝔥} and which consist entirely of hyperbolic elements. In the last years this class of reductive symmetric Lie algebras and the associated symmetric spaces has become a topic of very active research spreading in more and more areas. For a survey of the state of the art we refer to [8] and the literature cited there.

On the other hand there have been attempts to push this theory further to symmetric Lie algebras which are not necessarily semisimple or reductive. The simplest type (called the complex type) is where 𝔤 = 𝔥_{ℂ} is a complexification and τ is complex conjugation. Among these symmetric Lie algebras those for which 𝔥 contains an open invariant convex cone W consisting of elliptic elements play a crucuial role (cf. [14], [17], [18]). Then iW ⊆ 𝔮 = i𝔥 is an open cone consisting of hyperbolic elements so that, in the special case of reductive Lie algebras, we obtains on the one hand the non-compactly causal spaces of complex type and, if we allow W = 𝔥, also the Riemannian symmetric spaces coming from Cartan involutions of complex symmetric Lie algebras. For the associated symmetric spaces of complex type and the reductive spaces mentioned above one nowadays has a quite well developed picture of the harmonic analysis (holomorphic representations: [15], [16]; spherical functions [2], [6]; Hardy spaces [9], [12]) and the invariant complex analysis (invariant Stein domains and plurisubharmonic functions [18]).

The next step in this program is to pass from Lie algebras of complex type (𝔥_{ℂ}, τ) to the general case. The main problem one has to face here is to find the appropriate class of symmetric Lie algebras which is general enough to encorporate all the cases mentioned above such as the mixed complex type case, the non-compactly causal spaces, and also the Riemannian symmetric spaces. Our main objective in this paper is to describe and develop the structure theory and convex geometry of such a class of symmetric Lie algebras. We have tried to keep the exposition as self-contained as possible. As far as the structure theory of symmetric spaces is concerned we use not much more than [3] for well known facts on the structure of Riemannian symmetric spaces.

We now give a short overview over the contents of this paper:

Section I starts with a collection of structural results concerning Levi decompositions and Cartan decompositions which are invariant under some compact group of automorphisms. Then we introduce the basic notions concerning symmetric Lie algebras (𝔤, τ).

The key notions in our structural analysis of symmetric Lie algebras are those of hyperbolic and elliptic elements. Looking for large subspaces of hyperbolic elements, we are led to the notion of a maximal abelian subspace of 𝔮 consisting of hyperbolic elements (always denoted 𝔞) and a maximal hyperbolic Lie triple system 𝔭 of 𝔮, i.e., [𝔭, [𝔭, 𝔭]] ⊆ 𝔭. One has similar notions for elliptic replaced by hyperbolic. The main results of Section II are Theorem II.8 and Corollary II.9 stating that maximal elliptic and hyperbolic Lie triple systems are always conjugate under the group Inn_{𝔤}(𝔥) = ⟨e^{ad 𝔥}⟩ acting naturally on 𝔮. From that Lie algebraic result one easily deduces that all maximal compact subspaces of a symmetric space which contain a given point are conjugate under the isotropy group of this point. This generalizes the well known theorem that all maximal compact subgroups of a connected Lie group are conjugate under inner automorphisms.

In Section III we turn to a closer study of hyperbolic elements and their orbits under the group Inn_{𝔤}(𝔥). We show that all maximal hyperbolic abelian subspaces 𝔞 of 𝔮 are conjugate under Inn_{𝔤}(𝔥) and deduce that the set 𝔮_{hyp} of hyperbolic elements in 𝔮 has non-empty interior if and only if 𝔞 is maximal abelian in 𝔮 (Theorem III.3). These observations imply in particular that 𝔮_{hyp} = Inn_{𝔤}(𝔥).𝔞, i.e., that each hyperbolic element is conjugate to an element in 𝔞. Now the question arises how the orbit O_{X} of such an element X intersects 𝔞. The surprisingly simple answer to this question is given in Theorem III.10 saying that for X ∈ 𝔞 the orbit O_{X} intersects 𝔞 in the orbit of X under the Weyl group of a maximal hyperbolic Lie triple system 𝔭 containing 𝔞. This result is obtained by showing that the Weyl group of 𝔞 is not bigger than the Weyl group of the Riemannian symmetric Lie algebra generated by 𝔭.

We have seen in Section III that the size of the set 𝔮_{hyp} is related to the subspaces 𝔞 in the sense that it has interior points if and only if 𝔞 is maximal abelian in 𝔮. From now on we always assume this. In Section IV we consider the root decomposition of the Lie algebra 𝔤 with respect to a subspace 𝔞. It turns out that each root vector Z generates an at most three dimensional τ -invariant subalgebra:

(R) The Riemannian type, where 𝔤(Z) ≅ 𝔰𝔩(2, ℝ) endowed with the Cartan involution. This is the type which exclusively occurs for Riemannian symmetric Lie algebras.

(SR) The semi-Riemannian type, where 𝔤(Z) ≅ 𝔰𝔩(2, ℝ) endowed with the

involution corresponding to 𝔥 = 𝔰𝔬(1, 1). (A) The abelian type, where 𝔤(Z) ≅ ℝ^{2} . (N) The nilpotent type, where 𝔤(Z) is isomorphic to the three dimensional

Heisenberg algebra.

In semisimple symmetric Lie algebras only the first three types occur and the occurence of (N) is a “solvable” phenomenon. It is quite illuminating that the type of the test algebra 𝔤(Z) is, up to the distinction between type (A) and (N), characterized by the sign of the quadratic form κ_{τ} (Z) := tr ( ad Z ad τ(Z)) (cf. Proposition IV.7). We conclude Section IV with the discussion of some examples which display the different types of behaviour that can occur.

Section V is dedicated to a discussion of a class of symmetric Lie algebras that we call quasihermitian and which are characterized by the property that for a maximal hyperbolic Lie triple system 𝔭 the centralizer of its center in 𝔮 is not bigger than 𝔭.

In Section VI we come to the subject proper of this paper, the convexity properties of the action of Inn_{𝔤}(𝔥) on the space 𝔮. The key role will be played by those invariant convex subsets of 𝔮 whose interior points are hyperbolic. We call such invariant convex sets hyperbolic. The main point of this section is that the existence of hyperbolic invariant convex sets has significant consequences for the structure of the symmetric Lie algebra (𝔤, τ). In particular we show that this implies that (𝔤, τ) is quasihermitian, and, whenever pointed cones with these properties exist, also that it has strong cone potential (cf. Definition V.1(g)).

Having already dealt with the quasihermitian Lie algebras in Section V, we turn to the Lie algebras with (strong) cone potential in Section VII. We show that it imposes quite restrictive conditions on the structure of the root decomposition of 𝔤. The most crucial results are the Short String Theorem (Theorem VII.18) and its consequences. We also describe a method to construct interesting examples of mixed symmetric Lie algebras having all of the properties mentioned above. A Lie algebra which displays many features of the theory is the symmetric Jacobi algebra 𝔤 = 𝔥_{n} ⋊ 𝔰𝔭(n, ℝ), where 𝔥_{n} is a (2n + 1)-dimensional Heisenberg algebra, 𝔰𝔭(n, ℝ) acts naturally on it, and both are endowed with compatible involutions turning 𝔤 into a quasihermitian symmetric Lie algebra with strong cone potential (cf. Example VII.17).

For many applications concerning analysis on symmetric spaces and in particular phenomena related to analytic continuation aspects it is important to understand quite well the embedding of the symmetric Lie algebra 𝔤 into its complexification 𝔤_{ℂ} which is endowed with the antilinear extension τ̂ of τ. This corresponds to an embedding 𝔮 → 𝔮̂ := 𝔮 + i𝔥 = i𝔤^{c}, where 𝔤^{c} = 𝔥 + i𝔮 is the dual symmetric Lie algebra. We call 𝔤_{ℂ}, resp. 𝔮̂, the canonical extension of 𝔤, resp. 𝔮. The Inheritage Theorem (Theorem VIII.1) states that whenever a symmetric Lie algebra (𝔤, τ) is quasihermitian and the centralizer 𝔷_{𝔥}(𝔞) of 𝔞 in 𝔥 is compactly embedded, then (𝔤_{ℂ}, τ̂) is also quasihermitian. This result is a rather important tool because it makes many results that have been proved for the special case of symmetric Lie algebras where τ is complex conjugation available in the general context. In the remainder of Section VIII we apply this method to derive convexity theorems which describe the (convex hull of) projections of hyperbolic orbits in 𝔮 onto 𝔞 along the complementary subspace [𝔞, 𝔥].

Section IX is almost entirely devoted to the characterization of those symmetric Lie algebras for which there exists an open convex invariant cone in 𝔮_{hyp} . In this case there always exists finitely many maximal cones W_{max}^{0} having this property. The main difficulty is to show that the interior of the cone W_{max} consists of hyperbolic elements which is not at all evident from the definition.

In Section X we finally use the aforementioned convexity theorems to obtain a characterization of invariant subsets of 𝔮 whose interior consists of hyperbolic elements by their intersections with 𝔞. These results can in particular be used to obtain a classification of the invariant hyperbolic cones in 𝔮 by their intersections with 𝔞. From this characterization we derive a particularly interesting result saying that in the natural setup all invariant hyperbolic convex subsets C ⊆ 𝔮 can be extended to invariant convex hyperbolic subsets Ĉ ⊆ 𝔮̂ satisfying Ĉ ∩ 𝔮 = C. We expect that these results have many applications in the further investigation of related Hardy spaces, spherical functions, invariant plurisubharmonic functions, Stein domains, and representations on spaces of holomorphic functions.

We would like express our gratitude to the Mittag-Leffler Institute in Djursholm, Sweden for the hospitality and pleasant working atmosphere during our stay in spring 1996.

List of symbols

𝔞 maximal abelian hyperbolic subspace in 𝔮 (Definition I.7(b)) 𝔞̂ maximal abelian hyperbolic subspace in 𝔮̂ extending 𝔞 α̌ coroot of α ∈ Δ (Definition V.1(e)) A_{α} representing element of α ∈ Δ in 𝔞 (Proposition IV.7(iv),(v))

B(C) lower bounded linear functionals on C (Definition VI.2(a)) C_{max} maximal cone in 𝔞 (Definition V.1(d)) C_{min} minimal cone in 𝔞 (Definition V.1(d)) C_{min,p} minimal semisimple cone in 𝔞 (Definition V.1(d)) C_{min,z} minimal central cone in 𝔞 (Definition V.1(d))

Δ root system of 𝔤 with respect to 𝔞 (Theorem IV.1) Δ̂ root system of 𝔤_{ℂ} with respect to 𝔞̂ Δ^{+} system of positive roots in Δ Δ̌ coroots of Δ (Definition V.1(e)) Δ_{k} compact roots in Δ (Definition V.1(a)) Δ_{n} non-compact roots in Δ (Definition V.1(a)) Δ_{p} non-compact semisimple roots in Δ(Definition V.1(a)) Δ_{r} solvable roots in Δ (Definition IV.4) Δ_{s} semisimple roots in Δ (Definition IV.4)

(𝔤, τ) finite dimensional real symmetric Lie algebra (Definition I.6(a)) 𝔤^{c} = 𝔥 + i𝔮 (c-dual of 𝔤) 𝔤^{α} root space associated to the root α ∈ Δ (Theorem IV.1) 𝔥 1-eigenspace of τ (Definition I.6(a)) 𝔥̂ = 𝔤^{c} (1-eigenspace of τ̂)

H(C) edge of C (Definition VI.2(b)) Inn_{𝔤}(𝔟) group of inner automorphisms of 𝔤 generated by e^{ad 𝔟}, where 𝔟 ⊆ 𝔤

κ Cartan-Killing form of 𝔤 κ_{τ} = κ(·, τ(·)) (Proposition IV.7(vi))

lim C limit cone of C (Definition VI.2(b))

𝔫 maximal nilpotent ideal in 𝔤 𝔫_{𝔟}(𝔠) normalizer of 𝔠 in 𝔟 N_{B}(𝔠) normalizer of 𝔠 in B ⊆ Aut(𝔤)

𝔮 −1-eigenspace of τ (Definition I.6(a)) 𝔮̂ = i𝔤^{c} (−1-eigenspace of τ̂) 𝔭 maximal hyperbolic Lie triple system in 𝔮 (Definition I.7(c)) 𝔭̂ maximal hyperbolic Lie triple system in 𝔮̂

𝔮_{ell} elliptic elements in 𝔮 (Definiton I.7(a)) 𝔮_{hyp} hyperbolic elements in 𝔮 (Definition I.7(a)) 𝔯 radical of 𝔤 𝔯^{α} solvable part of 𝔤^{α} (Definition IV.4) 𝔰 Levi-complement in 𝔤 𝔰^{α} semisimple part of 𝔤^{α} (Definition IV.4) 𝔱 compactly embedded Cartan subalgebra of 𝔤^{c} (Proposition VII.10) θ Cartan-involution of 𝔰 𝔱_{𝔥} Cartan subalgebra of 𝔥^{0} (Proposition VII.10) τ̂ complex antilinear extension of τ to 𝔤_{ℂ} W Weyl group of (𝔤, τ, 𝔞) (Definition III.9) Ŵ Weyl group of (𝔤_{ℂ}, τ̂, 𝔞̂)

W_{max} maximal hyperbolic cone in 𝔮 (Chapter IX) Ŵ_{max} maximal hyperbolic cone in 𝔮̂ (Chapter X) 𝔷(𝔟) = 𝔷_{𝔟}(𝔟) (center of 𝔟) 𝔷_{𝔟}(𝔠) centralizer of 𝔠 in 𝔟 Z_{B}(𝔠) centralizer of 𝔠 in B ⊆ Aut(𝔤)

Conventions: Subscripts denote intesections, for instance 𝔯_{𝔮} = 𝔯 ∩ 𝔮 etc. and for a subspace 𝔟 ⊆ 𝔤 we write 𝔟^{0}: = 𝔷_{𝔟}(𝔞) and denote by 𝔟_{L} the subalgebra of 𝔤 generated by 𝔟.

I. Preliminaries on symmetric Lie algebras Invariant Levi complements

We start with some information that will be crucial to obtain suitable Levi decompositions of Lie groups.

Lemma I.1. Let V be a finite dimensional vector space over a field 𝕂 with char 𝕂 = 0 and G ⊆ Gl(V) an algebraic subgroup such that V is a semisimple G-module. Then G is reductive.

Proof. According to the Levi decomposition G = G_{u}⋊L, where L is reductive ([10, Th. VIII.4.3]), we only have to show that the unipotent radical G_{u} is reduced to {1}. Since V is a direct sum of simple modules, it suffices to show that G_{u} acts trivially on simple modules.

So let U ⊆ V be a simple submodule and put W := {v ∈ U: (∀g ∈ G_{u})g.v = v}. Then W is non-zero if V is non-zero because G_{u} is unipotent. Moreover W is invariant under G because G_{u} is normal. Hence W = U and the proof is complete.

Proposition I.2. Let 𝔤 be a finite dimensional Lie algebra over the field 𝕂 with char 𝕂 = 0. Let A ⊆ Aut(𝔤) be a subgroup such that 𝔤 is a semisimple A-module. Then there exists an A-invariant Levi complement in 𝔤.

Proof. First we recall the Levi decomposition G = G_{u} ⋊ L of the algebraic group G = Aut(𝔤), where G_{u} is the unipotent radical and L is a reductive subgroup. By passing to the Zariski closure of A, we may w.l.o.g. assume that A is algebraic. Then A is a reductive subgroup of G (Lemma I.1), hence conjugate to a subgroup of L ([10, Th. VIII.4.3]) and we therefore may assume that A ⊆ L. So it suffices to find an L-invariant Levi complement.

Let 𝔰 ⊆ 𝔤 be a fixed Levi complement. Then 𝔤 is a semisimple ad 𝔰module. Hence ad 𝔰 ⊆ der(𝔤) is conjugate to a subalgebra of 𝔩 := Ł(L) and we may w.l.o.g. assume that ad 𝔰 ⊆ 𝔩.

The subalgebra ad 𝔤 ⊆ der(𝔤) is an L-invariant ideal. Hence (ad 𝔤) ∩ 𝔩 is an L-invariant ideal in 𝔩 containing ad 𝔰. We conclude that (ad 𝔤) ∩ 𝔩 is a reductive subalgebra of ad 𝔤. Since ad 𝔰 is a Levi complement in ad 𝔤, it follows that ad 𝔰 = [(ad 𝔤) ∩ 𝔩, (ad 𝔤) ∩ 𝔩]. Thus ad 𝔰 is also an L-invariant ideal of 𝔩. Now ad^{−1}(ad 𝔰) = 𝔰 + 𝔷 must be L-invariant. Therefore 𝔰 = [𝔰 + 𝔷, 𝔰 + 𝔷] is L-invariant.

We obtain the following result as a corollary (cf. [11, App. 9.4]):

Corollary I.3. If 𝔤 is a real Lie algebra and K ⊆ Aut(𝔤) a compact group of automorphisms of 𝔤, then there exists a K -invariant Levi complement.

Corollary I.4. If τ is an involutive automorphism of the Lie algebra 𝔤 over a field of characteristic 0. Then there exists a τ -invariant Levi complement.

Invariant Cartan decompositions

Proposition I.5. Let 𝔤 be a semisimple real Lie algebra and U ⊆ Aut(𝔤) a compact subgroup. Then the following assertions hold:

(i) There exists a U -invariant Cartan decomposition 𝔤 = 𝔨 + 𝔭. The corresponding Cartan involution θ commutes with U.

(ii) If θ and θ′ are two Cartan involutions commuting with U, then there exists X ∈ 𝔤 with θ(X) = −X and [X, Ad(U)] = {0} such that θ′ = e^{ad X}θe^{− ad X}.

(iii) For each involutive automorphism τ of 𝔤 there exists a Cartan involution θ commuting with τ. If θ′ is another Cartan involution commuting with τ, then there exists X ∈ 𝔤 with θ(X) = −X and τ(X) = X such that θ′ = e^{ad X}θe^{− ad X}.

Proof. (i) Let 𝔤 = 𝔨 + 𝔭 be a Cartan decomposition of 𝔤 and θ the corresponding Cartan involution. Then Aut(𝔤) = Aut(𝔤)^{θ}e^{ad 𝔭} is a Cartan decomposition of the real Lie group Aut(𝔤) which is semisimple and, since it is an algebraic group over ℝ, it has at most finitely many connected components. Now M = Aut(𝔤)/ Aut(𝔤)^{θ} is the associated Riemannian symmetric space which in turn can be identified with the set of all Cartan decomposition of 𝔤. In view of [3, Th. VI.2.1], the group U has a fixed point in M, hence commutes with a Cartan involution and therefore leaves the corresponding Cartan decomposition invariant. (ii) In view of (i), we may w.l.o.g. assume that U ⊆ Aut(𝔤)^{θ}. Then the set of all Cartan involutions commuting with U can be written as e^{ad 𝔞}.θ, where 𝔞 = {X ∈ 𝔭: (∀γ ∈ U)γ.X = X}. This proves the assertion. (iii) This is a special case of (i) and (ii).

Symmetric Lie algebras

Definition I.6. (a) A symmetric Lie algebra is a pair (𝔤, τ), where τ is an involutive automorphism of 𝔤. A symmetric Lie group is defined analogously. Note that if (G, τ) is a symmetric Lie group, then (𝔤, dτ(1)) is a symmetric Lie algebra. If H ⊆ G^{τ} is an open subgroup, then G/H is called a symmetric space associated to the symmetric Lie algebra (𝔤, τ).

If (𝔤, τ) is a symmetric Lie algebra, then we write

𝔥 = 𝔤^{τ} = {X ∈ 𝔤: τ(X) = X} and 𝔮 = {X ∈ 𝔤: τ(X) = −X}

and note that 𝔤 = 𝔥+𝔮 is a direct vector space decomposition. For a τ -invariant subspace 𝔟 ⊆ 𝔤 we will always write 𝔟_{𝔥} := 𝔟 ∩ 𝔥 and 𝔟_{𝔮} := 𝔟 ∩ 𝔮, so that we have 𝔟 = 𝔟_{𝔥} ⊕ 𝔟_{𝔮}.

(b) For each symmetric Lie algebra (𝔤, τ) the subspace 𝔤^{c} := 𝔥 + i𝔮 of the complexification 𝔤_{ℂ} is also a symmetric Lie algebra with respect to the involution obtained by restricting the complex linear extension of τ.

(c) A symmetric Lie algebra (𝔤, τ) is called effective if 𝔥 does not contain any non-zero ideal of 𝔤. Note that 𝔥 = [𝔮, 𝔮] does not imply that (𝔤, τ) is effective because ([𝔥, 𝔥] + i𝔥, σ), where 𝔥 is a two-step nilpotent Lie algebra and σ is complex conjugation provides a counterexample.

(d) A symmetric Lie algebra (𝔤, τ) is said to be irreducible if each non-zero τ -invariant ideal of 𝔤 coincides with 𝔤.

Note that this implies in particular that (𝔤, τ) is effective. Since 𝔞 = 𝔞_{𝔥} + 𝔞_{𝔮} ↦ 𝔞^{c} := 𝔞_{𝔥} + i𝔞_{𝔮} defines a one-to-one correspondence between the τ - invariant ideals of 𝔤 and 𝔤^{c}, we see that (𝔤^{c}, τ) is irreducible if and only if (𝔤, τ) is irreducible.

There are three basic types of irreducible symmetric Lie algebras. If 𝔤 is not semisimple, then it must be abelian, and therefore 𝔤 = 𝔮 ≅ ℝ. If 𝔤 is semisimple, then either 𝔤 is simple or 𝔤 ≅ 𝔥 ⊕ 𝔥, where 𝔥 is simple and τ(X, Y) = (Y, X).

Definition I.7. Let (𝔤, τ) be a symmetric Lie algebra with the corresponding decomposition 𝔤 = 𝔥 + 𝔮.

(a) An element X ∈ 𝔤 is called hyperbolic if ad X is diagonalizable over ℝ. We write 𝔮_{hyp} for the set of all hyperbolic elements in 𝔮. An element X ∈ 𝔤 is called elliptic if ad X is semisimple with purely imaginary spectrum. We write 𝔮_{ell} for the set of elliptic elements in 𝔮. We note that if 𝔤^{c} = 𝔥 + i𝔮 is the dual symmetric Lie algebra, then i𝔮_{hyp} = (i𝔮)_{ell} and vice versa.

(b) An abelian subspace 𝔞 ⊆ 𝔮 is called maximal hyperbolic abelian, resp., maximal elliptic abelian if 𝔞 consists of hyperbolic, resp., elliptic elements and is maximal with respect to this property.

(c) A subspace 𝔞 ⊆ 𝔮 is called a Lie triple system if [𝔞, [𝔞, 𝔞]] ⊆ 𝔞. Note that this means that the space 𝔞_{L} := 𝔞 + [𝔞, 𝔞] is a subalgebra of 𝔤. A Lie triple system 𝔞 ⊆ 𝔮 will be called hyperbolic, resp., elliptic if it consists of hyperbolic, resp., elliptic elements.

(d) For a subalgebra 𝔞 of a Lie algebra 𝔤 we write Inn_{𝔤}(𝔞) := ⟨e^{ad 𝔞}⟩ for the group of inner automorphisms of 𝔤 generated by e^{ad 𝔞}.

(e) A subalgebra 𝔞 of a Lie algebra 𝔤 is said to be compactly embedded if 𝔞 consists of elliptic elements which is equivalent to the condition that the closure of Inn_{𝔤}(𝔞) is compact (cf. [4, 2.6]).

(f) A compactly embedded Cartan subalgebra 𝔱 of 𝔤 is a compactly embedded subalgebra which in addition is maximal abelian. Note that this implies in particular that 𝔱 is self-normalizing, hence a Cartan subalgebra of 𝔤. A toral Cartan subalgebra is a subalgebra consisting of hyperbolic elements which in addition is maximal abelian.

The following lemma clarifies the meaning of the effectiveness assumption for a symmetric Lie algebra (𝔤, τ).

Lemma I.8. The largest ideal of 𝔤 contained in 𝔥 coincides with the kernel of the representation ad_{𝔮} of 𝔥 on 𝔮.

Proof. Let 𝔧 ⊆ 𝔥 denote the largest ideal of 𝔤 contained in 𝔥. Then [𝔧, 𝔮] ⊆ 𝔧 ∩ 𝔮 ⊆ 𝔥 ∩ 𝔮 = {0}. On the other hand ker ad_{𝔮} ⊆ 𝔥 is an ideal of 𝔥 with [𝔮, ker ad_{𝔮}] = {0}. Therefore ker ad_{𝔮} is an ideal of 𝔤.

The following observation will be of crucial use in the remainder of this paper.

Lemma I.9. (i) If 𝔞, 𝔟 ⊆ 𝔮 are hyperbolic (elliptic) subspaces with [𝔞, 𝔟] = {0}, then 𝔞 + 𝔟 is hyperbolic (elliptic). (ii) Let s be a semisimple Lie algebra and ρ: 𝔰 → End(V) a finite dimensional representation. Then the following assertions hold:

(a) For each Cartan involution θ of 𝔰 there exists a scalar product on V such that ρ(θ.X) = −ρ(X)^{⊤} for all X ∈ 𝔰.

(b) If X ∈ 𝔰 is hyperbolic or elliptic, then the same holds for ρ(X).

Proof. (i) In view of the duality of 𝔤 and 𝔤^{c}, it suffices to prove the assertion for the hyperbolic case, where we have to show that for X ∈ 𝔞 and Y ∈ 𝔟 the sum is hyperbolic. But this follows from the fact that two diagonalizable operators that commute can be diagonalized simultaneously. (ii) We may w.l.o.g. assume that V is a complex 𝔰-module, so that we obtain an extension of ρ to a representation of the complexification 𝔰_{ℂ}.

(a) If 𝔰 = 𝔰_{𝔨} ⊕ 𝔰_{𝔭} is the Cartan decomposition corresponding to the Cartan involution θ, then 𝔲 := 𝔰_{𝔨}+i𝔰_{𝔭} is a compact real form of 𝔰_{ℂ}. Now Weyl’s unitary trick shows that there exists a scalar product on V for which the operators in ρ(𝔲) are skew-Hermitian. Then the operators in ρ(𝔰_{𝔨}) are skew-symmetric and those in ρ(𝔰_{𝔭}) are symmetric. This proves (a).

(b) Since X ∈ 𝔰_{ℂ} is hyperbolic if and only if iX is elliptic, it suffices to prove that ρ(X) is elliptic whenever X ∈ 𝔰_{ℂ} is so. Let X ∈ 𝔰_{ℂ} be elliptic. Then X is contained in a maximal compactly embedded subalgebra 𝔲 ⊆ 𝔰_{ℂ} which therefore is a compact real form, i.e., 𝔰_{ℂ} = 𝔲 + i𝔲 is a Cartan decomposition. Therefore (a) applies and we see that, with respect to a certain scalar product on V , the operators in ρ(𝔲) are skew-Hermitian, hence elliptic.

Lemma I.10. If (𝔤, τ) = (𝔥_{ℂ}, σ), where σ is complex conjugation, then 𝔤^{c} ≅ 𝔥 ⊕ 𝔥, where τ(X, Y) = (Y, X).

Proof. We identify 𝔤_{ℂ} = (𝔥_{ℂ})_{ℂ} with ℂ ⊗_{ℝ} 𝔥_{ℂ} = ℂ ⊗_{ℝ} 𝔤. We claim that 𝔤_{ℂ} ≅ 𝔥_{ℂ} ⊕ \overline{𝔥_{ℂ}}, where \overline{𝔥_{ℂ}} denotes the Lie algebra 𝔥_{ℂ} endowed with the opposite complex structure. In fact, we define two mappings

η_{±}: 𝔥_{ℂ} → 𝔤_{ℂ}, X ↦ \frac{1}{2}(1 ⊗ X ± i ⊗ iX).

Then

[η_{±}(X), η_{±}(Y)] = \frac{1}{4}[1 ⊗ X ± i ⊗ iX, 1 ⊗ Y ± i ⊗ iY]
= \frac{1}{4}(1 ⊗ [X, Y] ± i ⊗ i[X, Y] ± i ⊗ i[X, Y] − 1 ⊗ [iX, iY])
= \frac{1}{2}(1 ⊗ [X, Y] ± i ⊗ i[X, Y]) = η_{±}([X, Y])

shows that the mappings η_{±} are Lie algebra homomorphisms. It is clear that both are injective and that their images intersect in {0}. Moreover,

\begin{aligned}[η_{+}(X), η_{−}(Y)] & = \frac{1}{2}[1 ⊗ X + i ⊗ iX, 1 ⊗ Y − i ⊗ iY] \\ & = \frac{1}{2}(1 ⊗ [X, Y] − i ⊗ i[X, Y] + i ⊗ i[X, Y] + 1 ⊗ [iX, iY]) = 0. \end{aligned}

Hence

𝔤_{ℂ} ≅ η_{−}(𝔥_{ℂ}) ⊕ η_{+}(𝔥_{ℂ}) ≅ 𝔥_{ℂ} ⊕ \overline{𝔥_{ℂ}}

because η_{+} is complex antilinear and η_{−} is complex linear. It is clear that in this representation the complex conjugation σ_{𝔤} with respect to 𝔤 acts by σ_{𝔤}(X, Y) = (Y, X), the complex linear extension τ of the involution τ on 𝔥_{ℂ} by τ(X, Y) = (τ.Y, τ.X), and the complex conjugation σ = τ ∘ σ_{𝔤} with respect to 𝔤^{c} by σ(X, Y) = (τ.X, τ.Y). The fixed point set of this involution is

𝔤^{c} ≅ {(X, Y): X, Y ∈ 𝔥} = 𝔥 ⊕ 𝔥,

and the corresponding involution maps (X, Y) to (Y, X).

II. Maximal elliptic and hyperbolic Lie triple systems

In this section we will show that maximal elliptic and hyperbolic Lie triple systems are conjugate under the group Inn_{𝔤}(𝔥). From that we will conclude in particular that all maximal compact symmetric subspaces of a general symmetric space which contain a given point x are conjugate under the stabilizer of x.

In the following we call a symmetric Lie algebra (𝔤, τ) non-compactly Riemannian if 𝔤 is reductive, 𝔷(𝔤) ⊆ 𝔮, and τ |_{[𝔤,𝔤]} is a Cartan involution.

Proposition II.1. If 𝔮 consists of hyperbolic elements and 𝔥 = [𝔮, 𝔮], then (𝔤, τ) is non-compactly Riemannian.

Proof. First we recall that [𝔤, 𝔯] is a nilpotent ideal of 𝔤 which implies that

[𝔤, 𝔯] ∩ 𝔮 = [𝔤, 𝔯] ∩ 𝔮_{hyp} ⊆ 𝔷(𝔤).

Let 𝔤 = 𝔯 + 𝔰 be a τ -invariant Levi decomposition (Corollary I.4). Then 𝔮 = 𝔯_{𝔮} + 𝔰_{𝔮}. Let X ∈ 𝔯_{𝔮}. Then

(ad X)^{2}.𝔮 ⊆ 𝔯_{𝔮} ∩ [𝔤, 𝔯] ⊆ 𝔷(𝔤)

implies that (ad X)^{3}.𝔮 = {0} and therefore that (ad X)^{4} = 0. Since ad X is semisimple, it follows that ad X = 0, i.e., that X ∈ 𝔷(𝔤). Now we have 𝔥 = [𝔮, 𝔮] = [𝔯_{𝔮} + 𝔰_{𝔮}, 𝔯_{𝔮} + 𝔰_{𝔮}] = [𝔰_{𝔮}, 𝔰_{𝔮}] ⊆ 𝔰 and therefore 𝔯 = 𝔯_{𝔮} ⊆ 𝔷(𝔤). This means that 𝔤 is reductive and that 𝔰 is the commutator algebra of 𝔤.

Let θ be a Cartan-involution of 𝔰 commuting with τ and 𝔰 = 𝔨 + 𝔭 the corresponding Cartan decomposition. Then the fact that 𝔮 consists of hyperbolic elements means that 𝔮 = 𝔭 ∩ 𝔮, and finally 𝔥 = [𝔮, 𝔮] shows that (𝔤, τ) is in fact a Riemannian symmetric Lie algebra.

Corollary II.2. If 𝔮 consists of elliptic elements and 𝔥 = [𝔮, 𝔮], then 𝔤 is a compact Lie algebra and 𝔷(𝔤) ⊆ 𝔮.

Proof. Let 𝔤^{c} := 𝔥 + i𝔮 denote the dual symmetric Lie algebra. Then 𝔤^{c} satisfies the assumptions of Proposition II.1, hence is Riemannian symmetric. We write 𝔤^{c} = 𝔷(𝔤^{c}) ⊕ 𝔰^{c}, where 𝔰^{c} is the commutator algebra. Then 𝔰^{c} = 𝔰_{𝔥}^{c} + 𝔰_{𝔮}^{c} = 𝔨^{c} + 𝔭^{c} is a Cartan decomposition of 𝔰^{c} and hence

𝔤 = i𝔷(𝔤^{c}) ⊕ (𝔨^{c} + i𝔭^{c})

is a compact Lie algebra.

Corollary II.3. If 𝔞 ⊆ 𝔮 is an elliptic Lie triple system, then the subalgebra 𝔞_{L} = 𝔞 + [𝔞, 𝔞] is compactly embedded in 𝔤.

Proof. First we use Corollary II.2 to see that 𝔞_{L} is a compact Lie algebra with 𝔷(𝔞_{L}) ⊆ 𝔞. Then the fact that [𝔞_{L}, 𝔞_{L}] is compact and semisimple implies that it is compactly embedded, hence that 𝔞_{L} = 𝔷(𝔞_{L}) ⊕ [𝔞_{L}, 𝔞_{L}] is compactly embedded.

Next we will show that maximal elliptic Lie triple systems in 𝔮 are conjugate under the group Inn_{𝔤}(𝔥). The proof will be by induction over the dimension of 𝔤. Therefore we first collect some preparatory lemmas.

Lemma II.4. If 𝔥 = [𝔮, 𝔮], 𝔯 is the radical of 𝔤, 𝔤/𝔯 is a compact Lie algebra and 𝔞 ⊆ 𝔮 is a maximal elliptic Lie triple system, then the commutator algebra of 𝔞_{L} is a Levi complement in 𝔤.

Proof. First we choose a Levi complement 𝔰 ⊆ 𝔤 which is simultaneously invariant under τ and 𝔞_{L}. Such a Levi complement exists because the group ⟨e^{ad 𝔞_{L}}, τ⟩ ⊆ Aut(𝔤) has compact closure (Corollary I.3). Then the normalizer 𝔫_{𝔤}(𝔰) of 𝔰 in 𝔤 is a subalgebra of 𝔤 which contains 𝔰 as an ideal, hence 𝔫_{𝔤}(𝔰) = 𝔫_{𝔯}(𝔰) ⊕ 𝔰, because [𝔫_{𝔯}(𝔰), 𝔰] ⊆ 𝔯 ∩ 𝔰 = {0}. Now 𝔞_{L} ⊆ 𝔫_{𝔤}(𝔰) follows from our construction of 𝔰. Let p: 𝔫_{𝔤}(𝔰) → 𝔫_{𝔯}(𝔰) denote the canonical projection and note that this is a homomorphism of Lie algebras which commutes with τ because the invariance of 𝔰 and 𝔯 implies the invariance of 𝔫_{𝔯}(𝔰) under τ. Let X ∈ 𝔞_{L}. Then 0 = [p(X), X −p(X)] = [p(X), X] implies that p(X) = X +(p(X)−X) is elliptic because X − p(X) ∈ 𝔰 is elliptic. Therefore p(𝔞_{L}) is a subalgebra of the solvable Lie algebra 𝔫_{𝔯}(𝔰) consisting of elliptic elements, hence abelian. Now p(𝔞_{L}) ⊕ 𝔰 is compactly embedded and contains 𝔞_{L}, so that the maximality of 𝔞 implies that 𝔞 = 𝔰_{𝔮} ⊕ 𝔞_{𝔯} is adapted to the decomposition of 𝔫_{𝔤}(𝔰). Finally 𝔥 = [𝔮, 𝔮] and the surjectivity of the quotient homomorphism 𝔤 → 𝔰 yields [𝔞, 𝔞] = [𝔰_{𝔮}, 𝔰_{𝔮}] = 𝔰_{𝔥} which in turn implies that 𝔰 = [𝔞_{L}, 𝔞_{L}].

Lemma II.5. Let (𝔤, τ) be a symmetric Lie algebra such that 𝔤 = 𝔳 + 𝔨, where 𝔳 is a τ -invariant abelian ideal and 𝔨 is τ -invariant and compactly embedded. Then two maximal elliptic Lie triple system in 𝔮 are conjugate under Inn_{𝔤}(𝔥).

Proof. Since 𝔨 is compactly embedded, 𝔤 is a semisimple 𝔨-module and therefore 𝔤 = 𝔷_{𝔤}(𝔨) ⊕ [𝔨, 𝔤], where [𝔨, 𝔤] = [𝔨, 𝔳] + [𝔨, 𝔨] = [𝔤, 𝔤] follows from [𝔳, 𝔳] = {0}. Thus 𝔷_{𝔤}(𝔨) is a subalgebra complementary to [𝔤, 𝔤] and therefore central, i.e., 𝔷_{𝔤}(𝔨) = 𝔷(𝔤). So 𝔤 decomposes as a direct sum of the ideals 𝔤 = 𝔷(𝔤) ⊕ [𝔤, 𝔤]. Since both ideals are τ -invariant, the projections of 𝔨 and 𝔳 on [𝔤, 𝔤] still satisfy the assumptions of the lemma, and the assertion is trivial for abelian Lie algebras, it suffices to prove the assertion under the additional assumption that 𝔤 = [𝔤, 𝔤] and 𝔷_{𝔤}(𝔨) = {0}.

Since 𝔨 ∩ 𝔳 is a compactly embedded subalgebra which is contained in a nilpotent ideal, it must be central and therefore trivial. Thus 𝔤 = 𝔳 ⋊ 𝔨 is a semidirect sum. Let G = 𝔳 ⋊ K denote a simply connected group corresponding to 𝔤 and G̃ = G ⋊ {1, τ}. Then the natural affine action of G on 𝔳 defined by

(v, k).x := k.x + v

together with the restriction of τ to 𝔳 defines an action of G̃ on 𝔳 by affine mappings.

Let 𝔞 ⊆ 𝔮 be a maximal elliptic Lie triple system and 𝔞_{L} ⊆ 𝔤 the corresponding Lie algebra which is compactly embedded (Corollary II.3). Let 𝔨̃ ⊇ 𝔨 be maximal compactly embedded. Then 𝔨̃ = 𝔨⊕(𝔨̃∩𝔳) and 𝔨̃∩𝔳 ⊆ 𝔷(𝔤) = {0} follows as above. Thus 𝔨̃ = 𝔨 and 𝔨 is maximal compactly embedded in 𝔤. Therefore 𝔞_{L} is conjugate to a subalgebra of 𝔨 (cf. [4, 2.6]). From that we conclude that the closure U of the image of exp(𝔞_{L}) ⋊ {1, τ} in Aff(𝔳) is a compact group. Therefore U has a fixed point x ∈ 𝔳. Then τ(x) = x implies x ∈ 𝔳_{𝔥} and moreover

(−x, 0)U(x, 0).0 = {0}

entails that e^{− ad x}.𝔞_{L} ⊆ 𝔨, hence that e^{− ad x}.𝔞 = 𝔨_{𝔮} by maximality of 𝔞. This proves that any maximal elliptic Lie triple system in 𝔮 is conjugate to 𝔨_{𝔮} and therefore that two maximal elliptic Lie triple system are conjugate under e^{ad 𝔳_{𝔥}}.

Proposition II.6. Let

𝔞 ⊆ 𝔭 ⊆ 𝔮 and 𝔞 ⊆ 𝔞̃ ⊆ 𝔭̃ ⊆ 𝔮̃,

where 𝔭 ⊆ 𝔮 is a maximal hyperbolic Lie triple system, 𝔞 ⊆ 𝔭 is maximal abelian, 𝔮̃ ⊆ 𝔮 is a Lie triple system, 𝔭̃ ⊆ 𝔮̃ is maximal hyperbolic with respect to 𝔮̃_{L}, and 𝔞̃ is maximal abelian in 𝔭̃. Then the following assertions hold:

(i) If 𝔭 ⊆ 𝔭̃, then [𝔭̃_{L}, 𝔭̃_{L}] = [𝔭_{L}, 𝔭_{L}].

(ii) 𝔞 = (𝔞 ∩ 𝔷(𝔭̃)) ⊕ (𝔞 ∩ [𝔭̃, [𝔭̃, 𝔭̃]]), where 𝔞 ∩ 𝔷(𝔭̃) = 𝔷(𝔭̃) ∩ 𝔮_{hyp} and 𝔞 ∩ [𝔭̃, [𝔭̃, 𝔭̃]] is maximal abelian in [𝔭̃, [𝔭̃, 𝔭̃]]. The same assertions hold with “elliptic” instead of “hyperbolic”.

Proof. If (i) and (ii) hold, then the same assertions for elliptic Lie triple systems follow by applying (i) and (ii) to the dual symmetric Lie algebra 𝔤^{c}.

(i) Let 𝔰 := [𝔭̃_{L}, 𝔭̃_{L}] = [𝔭̃, 𝔭̃] + [𝔭̃, [𝔭̃, 𝔭̃]] denote the commutator algebra in 𝔭̃_{L} which is a semisimple Lie algebra invariant under τ (Proposition II.1). Then 𝔭̃_{L} = 𝔷 ⊕ 𝔰, where 𝔷 = 𝔷(𝔭̃) denotes the center of 𝔭̃.

Let p: 𝔭̃ → 𝔷 denote the projection along 𝔰_{𝔮}. For X ∈ 𝔭 we then have X = X_{z} +X_{s} according to the above decomposition. Now X_{s} is contained in the semisimple Lie algebra 𝔰, where it is hyperbolic, hence is hyperbolic in 𝔤 (Lemma I.9(i)(b)). Therefore X_{z} = X −X_{s} is hyperbolic because [X, X_{s}] = [X_{z}, X_{s}] = 0. This proves that p(𝔭)+𝔭 is a hyperbolic Lie triple system so that p(𝔭) ⊆ 𝔭 follows from the maximality of 𝔭, whence 𝔭 = (𝔭 ∩ 𝔷) ⊕ (𝔭 ∩ 𝔰).

Next the hyperbolicity of (𝔞 ∩ 𝔷) ⊕ 𝔰_{𝔮} and the maximality of 𝔭 yield 𝔭 = (𝔭 ∩ 𝔷) ⊕ 𝔰_{𝔮}. Hence 𝔭̃ ⊆ 𝔭 + 𝔷 and therefore [𝔭, 𝔭] = [𝔭̃, 𝔭̃] as well as [𝔭, [𝔭, 𝔭]] = [𝔭̃, [𝔭̃, 𝔭̃]]. This proves (i). (ii) Since 𝔞 ⊆ 𝔭̃, the maximality of 𝔞 and a similar argument as in (i) show that p(𝔞) ⊆ 𝔞, hence that 𝔞 = (𝔞 ∩ 𝔷) ⊕ (𝔞 ∩ 𝔰). The maximality of 𝔞 further implies that 𝔞 ∩ 𝔰 is maximal abelian in the semisimple hyperbolic Lie triple system 𝔰_{𝔮}. Furthermore 𝔷 ∩ 𝔮_{hyp} is a vector space containing 𝔞, so that 𝔞 = 𝔷 ∩ 𝔮_{hyp} follows from the maximality of 𝔞.

In the following we write κ(X, Y) = tr(ad X ad Y) for the Cartan-Killing form of 𝔤.

Lemma II.7. Let (𝔤, τ) be a semisimple symmetric Lie algebra. Then the following assertions hold:

(i) If V ⊆ 𝔮 is an 𝔥-invariant subspace and V^{⊥} = {X ∈ 𝔮: (∀Y ∈ V)κ(X, Y) = 0} is the orthogonal space with respect to the Cartan-Killing form, then [V, V^{⊥}] = {0}. Moreover, if the restriction of κ to V is non-degenerate, then V_{L} is an ideal of 𝔤.

(ii) If (𝔤, τ) is irreducible and 𝔮 is not irreducible as an 𝔥-module, then the following assertions hold:

(a) 𝔮 splits into two irreducible components 𝔮 = 𝔮^{+} ⊕ 𝔮^{−} such that θ(𝔮^{+}) = 𝔮^{−} holds for any Cartan involution θ commuting with τ.

(b) The submodules 𝔮^{±} are isotropic for the Cartan-Killing form and abelian subalgebras of 𝔤.

(c) The subalgebras 𝔥 + 𝔮^{±} of 𝔤 are maximal parabolic.

Proof. First we claim that the Cartan-Killing form is non-degenerate on 𝔥 and 𝔮. In fact, we choose a Cartan involution θ commuting with τ (Proposition I.5). Then 𝔥 and 𝔮 are θ-invariant subspaces of 𝔤. If E ⊆ 𝔤 is θ-invariant and E = E^{+} ⊕E^{−} is the θ-eigenspace decomposition, then the invariance of κ under θ implies that E^{+} and E^{−} are orthogonal. Moreover, κ is negative definite on E^{+} and positive definite on E^{−}, hence κ is non-degenerate on E. This applies in particular to 𝔥 and 𝔮.

(i) We have

κ([V^{⊥}, V], 𝔥) = κ(V^{⊥}, [V, 𝔥]) ⊆ κ(V^{⊥}, V) = {0}.

Since the restriction of κ to 𝔥 is non-degenerate, we conclude that [V, V^{⊥}] = {0}. It follows in particular that V_{L} is an 𝔥-invariant subalgebra of 𝔤 satisfying [V^{⊥}, V_{L}] = {0}. If the restriction of κ to V is non-degenerate, then 𝔮 = V ⊕V^{⊥}, showing that [𝔮, V_{L}] ⊆ V_{L}, hence that V_{L} is an ideal. (ii) (a), (b): If V ⊆ 𝔮 is an irreducible 𝔥-submodule which is not isotropic, then the restriction of the Cartan-Killing form κ to V is non-degenerate and so (i) implies that V_{L} is a τ -invariant ideal of 𝔤. Hence the irreducibility of (𝔤, τ) yields 𝔮 = V . This contradiction shows that each irreducible submodule of 𝔮 must be isotropic. Moreover, if 𝔮^{+} is such a submodule, the subspace 𝔮^{+} +θ(𝔮^{+}) is a non-degenerate 𝔥-submodule, hence coincides with 𝔮. To complete the proof, we only have to note that (𝔮^{+})^{⊥} = 𝔮^{+} follows from the fact that 𝔮^{+} is isotropic and of half the dimension of 𝔮.

(c) In view of (b), we have

[𝔮^{+}, 𝔮^{−}] ⊆ 𝔥, [𝔮^{+}, 𝔥] ⊆ 𝔮^{+} and [𝔮^{+}, 𝔮^{+}] = {0}.

Therefore 𝔤 is a nilpotent 𝔮^{+} -module and the subalgebra 𝔟 := 𝔥 + 𝔮^{+} is not reductive in 𝔤. Since it is a maximal subalgebra, it must be parabolic (cf. [1, Ch. 8, §10, Cor. 1 de Th. 2]).

Theorem II.8. Any two maximal elliptic Lie triple systems in 𝔮 are conjugate under Inn_{𝔤}(𝔥). Proof. We prove the assertion by induction over the dimension of 𝔤. If dim 𝔤 = 0, then the assertion is trivial.

Suppose that dim 𝔤 > 0. First we assume that 𝔤 is semisimple. In addition, we may assume that 𝔥 = [𝔮, 𝔮] because 𝔮_{L} = 𝔮 + [𝔮, 𝔮] is an ideal of 𝔤 which is complemented by an ideal contained in 𝔥 (Lemma II.7(i)). Hence an element X ∈ 𝔮 is elliptic in 𝔤 if and only if it is elliptic in 𝔮_{L}.

Let 𝔨_{𝔮} ⊆ 𝔮 be a maximal elliptic Lie triple system. Then (𝔨_{𝔮})_{L} is a compactly embedded subalgebra (Corollary II.3). Let U := \overline{⟨e^{ad 𝔨_{𝔮}}, τ⟩} ⊆ Aut(𝔤). Then U is a compact subgroup and Proposition I.5 implies that there exists a Cartan involution θ of 𝔤 commuting with U. This means that θ commutes with τ and that (𝔨_{𝔮})_{L} ⊆ 𝔨, where 𝔤 = 𝔨+𝔭 is the corresponding Cartan decomposition of 𝔤. Now the maximality of 𝔨_{𝔮} implies that 𝔨_{𝔮} = 𝔨∩𝔮. Let (𝔨_{𝔮})^{⊥} ⊆ 𝔮 denote the orthogonal subspace with respect to the Cartan-Killing form. Then (𝔨_{𝔮})^{⊥} = 𝔭_{𝔮} and therefore 𝔥 = [𝔮, 𝔮] implies that

(2.1)  𝔨 = 𝔨_{𝔮} + [𝔨_{𝔮}, 𝔨_{𝔮}] + [𝔭_{𝔮}, 𝔭_{𝔮}] = (𝔨_{𝔮})_{L} + [(𝔨_{𝔮})^{⊥}, (𝔨_{𝔮})^{⊥}].

This means that 𝔨 and therefore θ can be reconstructed from 𝔨_{𝔮} via (2.1). Now the fact that two Cartan involutions commuting with τ are conjugate under the group Inn_{𝔤}(𝔥) (Proposition I.5(iii)) implies that two maximal elliptic Lie triple systems in 𝔮 are conjugate under Inn_{𝔤}(𝔥).

Now we turn to the general case. If 𝔤 is not semisimple, then the radical 𝔯 of 𝔤 is non-zero, hence the maximal nilpotent ideal 𝔫 of 𝔤 is non-zero and therefore its center 𝔷_{𝔫} is also non-zero. The ideals 𝔯, 𝔫 and 𝔷_{𝔫} of 𝔤 are invariant under each automorphism of 𝔤, hence in particular under τ. Let 𝔤_{1} := 𝔤/𝔷_{𝔫} and τ_{1} the involution induced by τ on the quotient algebra 𝔤_{1} . We write π: 𝔤 → 𝔤_{1} for the quotient morphism. Then

π(𝔥) = 𝔥_{1} and π(𝔮) = 𝔮_{1}.

Let 𝔨, 𝔟 ⊆ 𝔮 be maximal elliptic Lie triple systems. Then the subspaces π(𝔨) and π(𝔟) of 𝔮_{1} are elliptic Lie triple systems, hence contained in maximal elliptic Lie triple systems 𝔨_{1} and 𝔟_{1} of 𝔮_{1} . In view of our induction hypothesis, there exists h_{1} ∈ Inn_{𝔤_{1}}(𝔥_{1}) with h_{1}.𝔨_{1} = 𝔟_{1} .

The surjective homomorphism π: 𝔤 → 𝔤_{1} induces a surjective homomorphism q: Inn 𝔤 → Inn 𝔤_{1} defined by

π ∘ γ = q(γ) ∘ π

for γ ∈ Inn 𝔤 and we conclude from π ∘ τ = τ_{1} ∘ π that q( Inn_{𝔤}(𝔥)) = Inn_{𝔤}(𝔥_{1}). Hence there exists h ∈ Inn_{𝔤}(𝔥) with q(h) = h_{1} .

Let 𝔨̃:= π^{−1}(𝔨_{1}) ∩ 𝔮 and 𝔟̃ := π^{−1}(𝔟_{1}) ∩ 𝔮 and note that these spaces are Lie triple systems. Then π(h.𝔨̃) = h_{1}.𝔨_{1} = 𝔟_{1} = π(𝔟̃) and hence h.𝔨̃ = 𝔟̃. After replacing 𝔨 by h.𝔨 we may therefore assume that 𝔨̃ = 𝔟̃.

We note that the fact that π intertwines τ and τ_{1} implies that τ(𝔨̃) = 𝔨̃. Let 𝔨′ ⊇ 𝔨 be a maximal elliptic Lie triple system in 𝔨̃_{𝔮} with respect to 𝔨̃_{L}. Then the fact that 𝔨̃_{L} is an abelian extension of the compact Lie algebra (𝔨_{1})_{L} implies that 𝔨̃_{L} is compact modulo its radical. Therefore Lemma II.4 shows that [𝔨′_{L}, 𝔨′_{L}] is a Levi complement in 𝔨̃_{L}. Moreover, the fact that 𝔨′_{L} is a compact Lie algebra implies that [𝔨′_{L}, 𝔨′_{L}] = [𝔨_{L}, 𝔨_{L}] because the elliptic version of Proposition II.6(i) applies with 𝔮̃:= 𝔭̃ := 𝔨′.

Let 𝔳 := 𝔷_{𝔫} ∩ 𝔨̃_{L} = (ker π) ∩ 𝔨̃_{L}. Then 𝔳 is an abelian ideal of 𝔨̃_{L} which is invariant under τ. Now π([𝔨_{L}, 𝔨_{L}]) = π([𝔟_{L}, 𝔟_{L}]) is the unique Levi subalgebra of (𝔨_{1})_{L}. Therefore [𝔟_{L}, 𝔟_{L}] ⊆ 𝔳 + [𝔨_{L}, 𝔨_{L}]. In view of Lemma II.5, we may w.l.o.g. assume that [𝔨_{L}, 𝔨_{L}] = [𝔟_{L}, 𝔟_{L}]. Then 𝔷(𝔨) and 𝔷(𝔟) are contained in the τ -invariant solvable subalgebra 𝔠 := 𝔷_{𝔨_{L}}([𝔨_{L}, 𝔨_{L}]) of 𝔤. The Lie algebra 𝔠 is an abelian extension of the abelian algebra 𝔷(𝔨_{1}), hence solvable. Let 𝔨_{𝔥}, resp., 𝔟_{𝔥} be Cartan subalgebras of 𝔠 containing 𝔷(𝔨), resp., 𝔷(𝔟) (cf. [1, Ch. 7]). Then there exists Z ∈ [𝔠, 𝔠] with e^{ad Z}.𝔨_{𝔥} = 𝔥_{𝔟} (cf. [1, Ch. 7]). The fact that π(𝔠) is abelian implies that [𝔠, 𝔠] ⊆ 𝔷_{𝔫} and in particular that (ad Z)^{2} = 0 for Z ∈ [𝔠, 𝔠]. Let Z = Z_{𝔥} + Z_{𝔮} according to the τ -eigenspace decomposition of 𝔷_{𝔫} and note that

e^{ad Z}.𝔷(𝔨) = (1 + ad Z_{𝔥} + ad Z_{𝔮}).𝔷(𝔨) ⊆ 𝔥_{𝔟} ⊆ 𝔷_{𝔤}(𝔟).

For X ∈ 𝔷(𝔨) and Y ∈ 𝔟 we therefore have

0 = [e^{ad Z}.X, Y] = \underbrace{[X + [Z_{𝔥}, X], Y ]}_{∈𝔥} + \underbrace{[[Z_{𝔮}, X], Y ]}_{∈𝔮} .

So both summands have to vanish and consequently e^{ad Z_{𝔥}}.𝔷(𝔨) ⊆ 𝔷_{𝔮}(𝔟). Now 𝔟+ e^{ad Z_{𝔥}}.𝔷(𝔨) = 𝔟 + e^{ad Z_{𝔥}}.𝔨 ist an elliptic Lie triple subsystem of 𝔮 and maximality implies that 𝔟 = e^{ad Z_{𝔥}}.𝔨. This proves that 𝔨 and 𝔟 are conjugate under Inn_{𝔤}(𝔥).

Corollary II.9. Two maximal hyperbolic Lie triple systems in 𝔮 are conjugate under Inn_{𝔤}(𝔥).

Proof. This follows by applying Theorem II.8 to the dual symmetric Lie algebra 𝔤^{c} = 𝔥 + i𝔮.

Let G/H be a symmetric space associated to the symmetric Lie algebra (𝔤, τ) and (G, τ) a corresponding connected symmetric group. The exponential function of G/H is defined by

Exp: 𝔮 → G/H, X ↦ exp(X)H.

An important consequence of the conjugacy of the maximal elliptic Lie triple systems in 𝔮 is the conjugacy of the maximal compact symmetric subspaces of an associated symmetric space G/H which will follow from the following auxiliary proposition.

Proposition II.10. If G/H is a symmetric space associated to the symmetric Lie algebra (𝔤, τ) and 𝔮 consists of elliptic elements, then

\{X ∈ 𝔮: \overline{Exp(ℝX)} is compact \}

is a Lie triple subsystem of 𝔮.

Proof. Since the analytic subgroup of G generated by exp(𝔮+[𝔮, 𝔮]) still acts transitively on G/H, we may w.l.o.g. assume that 𝔥 = [𝔮, 𝔮]. If 𝔮 is elliptic, then Corollary II.2 says that 𝔤 is a reductive Lie algebra and 𝔮 = 𝔷(𝔤) + 𝔨_{𝔮}, where 𝔨 = [𝔤, 𝔤] is the semisimple compact commutator algebra. Let 𝔱 ⊆ 𝔷 be the Lie algebra of a maximal torus T in Z(G). Then Exp(𝔱 + 𝔨_{𝔮}) ⊆ G/H is compact and for X ∈ 𝔮 the conditions X ∈ 𝔱 + 𝔨_{𝔮} and the relative compactness of Exp(ℝX) are equivalent. Now the assertion follows from the fact that 𝔱 + 𝔨_{𝔮} is a Lie triple system.

Theorem II.11. If G/H is a symmetric space associated to the symmetric Lie algebra (𝔤, τ) and 𝔨, 𝔟 ⊆ 𝔮 are maximal Lie triple systems with the property that Exp(𝔨) and Exp(𝔟) are compact subsets of G/H, then there exists h ∈ Inn_{𝔤}(𝔥) with h.𝔨 = 𝔟. If, in addition, 𝔮 is elliptic, then 𝔨 = 𝔟.

Proof. We may w.l.o.g. assume that 𝔥 = [𝔮, 𝔮]. Then 𝔨 and 𝔟 are elliptic Lie triple systems because the quadratic representation q: G/H → G, gH ↦ gτ(g)^{−1} maps the compact sets Exp 𝔨, resp. Exp 𝔟, onto exp(𝔨), resp. exp(𝔟), and we see that both are compact subsets of G. Therefore Theorem II.8 implies that we may w.l.o.g. assume that 𝔮 is elliptic. Then Proposition II.10 shows that

𝔠 := \{X ∈ 𝔮: \overline{Exp(ℝX)} is compact \}

is a Lie triple system in 𝔮 which contains 𝔨 and 𝔟. Therefore 𝔠 = 𝔨 = 𝔟 by maximality. This proves the second assertion and therefore also the first one.

III. Hyperbolic elements and their orbits

In this section we turn to the study of the set 𝔮_{hyp} of hyperbolic elements in 𝔮. First we will investigate the interior of the set 𝔮_{hyp} and derive useful characterizations of the elements in its interior. From that we will conclude that this set has interior points if and only if 𝔮 contains maximal hyperbolic abelian subspaces which are in addition maximal abelian, and this will pave the way to the root decompositions which will be discussed in Section IV.

Lemma III.1. Let X ∈ 𝔮_{hyp} , V ⊆ 𝔮 be a subspace containing X, and

Ψ: Inn_{𝔤}(𝔥) × V → 𝔮, (h, v) ↦ h.v.

Then dΨ(h, X) is surjective if and only if

𝔷_{𝔮}(X) ⊆ V + [X, 𝔥].

Proof. We have

dΨ(h, X).(dλ_{h}(1). ad Z, Y) = h.[Z, X] + h.Y = h.([Z, X] + Y).

Therefore the linear mapping dΨ(h, X): T_{h}(Inn_{𝔤}(𝔥)) × V → 𝔮 is surjective if and only if [X, 𝔥] + V = 𝔮. Since 𝔤 is a semisimple ℝX -module, we have 𝔤 = [X, 𝔤] ⊕ 𝔷_{𝔤}(X) and hence that 𝔮 = [X, 𝔥] + 𝔷_{𝔮}(X). Therefore [X, 𝔥] + V = 𝔮 is equivalent to 𝔷_{𝔮}(X) ⊆ V + [X, 𝔥].

Proposition III.2. For X ∈ 𝔮_{hyp} the following are equivalent:

(1) X ∈ int 𝔮_{hyp} .

(2) 𝔷_{𝔮}(X) is a hyperbolic Lie triple system. Proof. (1) “⇒” (2): From 𝔷_{𝔮}(X) = 𝔮 ∩ 𝔷_{𝔤}(X) and the fact that 𝔷_{𝔤}(X) is a subalgebra of 𝔤, it follows that 𝔷_{𝔮}(X) is a Lie triple system. Suppose that X ∈ int 𝔮_{hyp} and let Y ∈ 𝔷_{𝔮}(X). Then there exists a t > 0 with X + tY ∈ 𝔮_{hyp} . Now [X, X + tY] = 0 implies that tY = (tY + X) − X is hyperbolic, hence that Y is hyperbolic. This proves that 𝔷_{𝔮}(X) is a hyperbolic Lie triple system.

(2) “⇒” (1): First we note that Lemma III.1 implies that for

Ψ: Inn_{𝔤}(𝔥) × 𝔷_{𝔮}(X) → 𝔮, (h, v) ↦ h.v

the differential dΨ(1, X) is surjective. Therefore the implicit function theorem implies that X = Ψ(1, X) ∈ int Ψ( Inn_{𝔤}(𝔥), 𝔷_{𝔮}(X)) ⊆ int 𝔮_{hyp} .

Theorem III.3. Let 𝔤 = 𝔥+𝔮 be a symmetric Lie algebra. Then the following assertions hold:

(i) All maximal hyperbolic abelian subspaces 𝔞 ⊆ 𝔮 are conjugate under Inn_{𝔤}(𝔥).

(ii) If 𝔞 ⊆ 𝔮 is maximal hyperbolic abelian, then 𝔮_{hyp} = Inn_{𝔤}(𝔥).𝔞.

(iii) int 𝔮_{hyp} ≠ Ø if and only if 𝔞 is maximal abelian in 𝔮.

Proof. (i) Let 𝔞, 𝔟 ⊆ 𝔮 be maximal hyperbolic abelian subspaces and 𝔞̃, resp., 𝔟̃ maximal hyperbolic Lie triple systems in 𝔮 containing 𝔞, resp., 𝔟. Then 𝔞̃ and 𝔟̃ are conjugate under Inn_{𝔤}(𝔥) (Corollary II.9), hence we may w.l.o.g. assume that 𝔞̃ = 𝔟̃. Then 𝔞 and 𝔟 are nothing but maximal abelian subspaces of 𝔞̃.

According to Proposition II.1, 𝔞̃_{L} is a Riemannian symmetric Lie algebra with 𝔷(𝔞̃_{L}) ⊆ 𝔮. Therefore the conjugacy of 𝔞 and 𝔟 follows from [3, Lemma V.6.3]. (ii) Let X ∈ 𝔮_{hyp} and 𝔞̃ ⊆ 𝔮 be maximal hyperbolic abelian with X ∈ 𝔞̃. Then we use (i) to find h ∈ Inn_{𝔤}(𝔥) with h.𝔞̃ = 𝔞. Then h.X ∈ 𝔞 and therefore 𝔮_{hyp} = Inn_{𝔤}(𝔥).𝔞. (iii) We consider the mapping

Ψ: Inn_{𝔤}(𝔥) × 𝔞 → 𝔮, (h, Y) ↦ h.Y.

Then Lemma III.1 shows that the linear mapping dΨ(h, Y_{0}): T_{h}(Inn_{𝔤}(𝔥))×𝔞 → 𝔮 is surjective if and only if 𝔷_{𝔮}(Y_{0}) ⊆ 𝔞 + [Y_{0}, 𝔥]. Since 𝔮 = [Y_{0}, 𝔥] ⊕ 𝔷_{𝔮}(Y_{0}), this holds if and only if 𝔞 = 𝔷_{𝔮}(Y_{0}).

If 𝔞 is not maximal abelian in 𝔮, then 𝔷_{𝔮}(Y_{0}) is always strictly bigger than 𝔞. Therefore Sard’s Theorem implies that the image 𝔮_{hyp} of Ψ contains no interior points.

If, conversely, 𝔞 is maximal abelian, then 𝔷_{𝔮}(𝔞) = 𝔞 and there exists Y_{0} ∈ 𝔞 such that 𝔞 = 𝔷_{𝔮}(𝔞) = 𝔷_{𝔮}(Y_{0}). Then Proposition III.2 shows that Y_{0} ∈ int 𝔮_{hyp} and therefore that the latter set is non-empty.

Lemma III.4. Let 𝔭 ⊆ 𝔮 be maximal hyperbolic Lie triple system and 𝔞 ⊆ 𝔭 maximal abelian. Then 𝔞 is a maximal hyperbolic abelian subspace of 𝔮.

Proof. Let 𝔞̃ ⊇ 𝔞 be maximal hyperbolic abelian and 𝔭̃ ⊇ 𝔞̃ a maximal hyperbolic Lie triple system. Then there exists h ∈ Inn_{𝔤}(𝔥) with h.𝔭̃ = 𝔭 (Corollary II.9). Hence h.𝔞̃ is a maximal abelian subspace of 𝔭. Now the fact that the symmetric Lie algebra 𝔭_{L} is Riemannian entails that all maximal abelian subspaces of 𝔭 have the same dimension (cf. [3, Lemma V.6.3]). Therefore dim 𝔞̃ = dim 𝔞 implies 𝔞 = 𝔞̃, i.e., that 𝔞 is maximal hyperbolic abelian in 𝔮.

Since we know at this point that all maximal hyperbolic abelian subspaces of 𝔮 are conjugate, the following specific constructions of maximal hyperbolic Lie triple systems give some important additional information on the location of maximal hyperbolic Lie triple systems with respect to certain nice Levi decompositions.

Proposition III.5. Let 𝔯 denote the radical of 𝔤 and 𝔯 = 𝔯_{𝔥} + 𝔯_{𝔮} its τ - eigenspace decomposition. Let 𝔞_{𝔯} ⊆ 𝔯_{𝔮} be a subspace which is maximal with respect to the property of being abelian and hyperbolic with respect to 𝔤 and 𝔰 ⊆ 𝔤 a Levi complement which is invariant under τ and 𝔞_{𝔯}. Then the following assertions hold:

(i) For any maximal hyperbolic abelian subspace 𝔞_{𝔰} ⊆ 𝔰_{𝔮} the space 𝔞 := 𝔞_{𝔯} ⊕ 𝔞_{𝔰} is maximal hyperbolic.

(ii) For any maximal hyperbolic abelian subspace 𝔞 ⊆ 𝔮 the intersection 𝔞 ∩ 𝔯 is maximal hyperbolic in 𝔯_{𝔮} with respect to 𝔤. All these subspaces are conjugate under Inn_{𝔤}(𝔥).

(iii) [𝔞_{𝔯}, 𝔰] = {0}.

(iv) If θ is a Cartan involution of 𝔰 commuting with τ |_{𝔰} and 𝔰 = 𝔰_{𝔨} ⊕𝔰_{𝔭} the corresponding Cartan decomposition, then 𝔭 := 𝔞_{𝔯} ⊕ (𝔰_{𝔭})_{𝔮} is a maximal hyperbolic Lie triple system in 𝔮. Moreover 𝔞 ⊆ 𝔭 is maximal abelian if and only if 𝔞 = 𝔞_{𝔯} ⊕ 𝔞_{𝔰}, where 𝔞_{𝔰} ⊆ (𝔰_{𝔭})_{𝔮} is maximal abelian.

(v) The correspondence in (iv) sets up a one-to-one correspondence between Cartan involutions on the ideal (𝔰_{𝔮})_{L} of 𝔰 commuting with τ and maximal hyperbolic Lie triple systems in 𝔮 containing 𝔞_{𝔯}.

(vi) If 𝔫 is the nilradical of 𝔤, then 𝔯_{𝔥} ⊆ 𝔷_{𝔯_{𝔥}}(𝔞) + 𝔫_{𝔥}.

Proof. (i) First we note that 𝔤 = 𝔷_{𝔤}(𝔞_{𝔯}) + [𝔞_{𝔯}, 𝔤] ⊆ 𝔷_{𝔤}(𝔞_{𝔯}) + 𝔯 implies that 𝔷_{𝔤}(𝔞_{𝔯}) contains a Levi complement of 𝔤. Since 𝔷_{𝔤}(𝔞_{𝔯}) is in addition τ -invariant, we find a τ -invariant Levi complement 𝔰 ⊆ 𝔷_{𝔤}(𝔞_{𝔯}) (Corollary I.4). We pick a maximal hyperbolic abelian subspace 𝔞_{𝔰} ⊆ 𝔰_{𝔮} and set 𝔞 := 𝔞_{𝔯} + 𝔞_{𝔰}. Since both summands commute, this subspace is hyperbolic. Let 𝔞̃ ⊇ 𝔞 be maximal hyperbolic abelian and write π: 𝔤 → 𝔰 ≅ 𝔤/𝔯 for the canonical projection. Then π(𝔞̃) ⊆ 𝔰_{𝔮} is hyperbolic and because it contains 𝔞_{𝔰}, we even see that 𝔞_{𝔰} = π(𝔞̃). Thus 𝔞̃ ⊆ 𝔯 + 𝔞_{𝔰} and therefore 𝔞̃ = (𝔯 ∩ 𝔞̃) + 𝔞_{𝔰}. Then 𝔯 ∩ 𝔞̃ is hyperbolic in 𝔯_{𝔮} and therefore 𝔯 ∩ 𝔞̃ = 𝔞_{𝔯} follows from the maximality of 𝔞_{𝔯}. This proves that 𝔞̃ = 𝔞, i.e., 𝔞 is maximal hyperbolic abelian in 𝔮. (ii) These two assertions follow from (i) and the fact that all maximal hyperbolic abelian subspaces are conjugate under Inn_{𝔤}(𝔥) (Theorem III.3(i)). (iii) If 𝔰 ⊆ 𝔤 is an 𝔞_{𝔯} -invariant Levi complement, then [𝔞_{𝔯}, 𝔰] ⊆ 𝔯 ∩ 𝔰 = {0}. (iv) Since 𝔞_{𝔯} commutes with 𝔰, it is clear that 𝔭 is a hyperbolic Lie triple system. To see that it is maximal, let 𝔭̃ ⊇ 𝔭 be a hyperbolic Lie triple system. First we note that the elements in (𝔰_{𝔨})_{𝔮} are elliptic, so that (𝔰_{𝔭})_{𝔮} is maximal hyperbolic in 𝔰_{𝔮}. Hence projecting 𝔭̃ along 𝔯_{𝔮} into 𝔰_{𝔮} shows that 𝔭̃ ⊆ 𝔯_{𝔮} + (𝔰_{𝔭})_{𝔮} and thus

𝔭̃ = (𝔭̃ ∩ 𝔯_{𝔮}) ⊕ (𝔰_{𝔭})_{𝔮}.

Since the hyperbolic Lie triple system 𝔭̃∩𝔯_{𝔮} is contained in a solvable Lie algebra, it is abelian (cf. Proposition II.1), and so the maximality of 𝔞_{𝔯} gives 𝔭̃ ∩ 𝔯_{𝔮} = 𝔞_{𝔯}. This shows that 𝔭̃ = 𝔭, i.e., that 𝔭 is maximal.

For the second part of the assertion we first observe that 𝔞_{𝔯} ⊆ 𝔷(𝔭), so that any maximal abelian subspace 𝔞 of 𝔭 contains 𝔞_{𝔯}. Now the assertion is immediate.

(v) We only have to show that 𝔭 determines the Cartan involution on (𝔰_{𝔮})_{L} uniquely. The subalgebra 𝔭^{c} := i(𝔭 ∩ 𝔰) + [𝔭, 𝔭] of 𝔰^{c} = 𝔰_{𝔥} + i𝔰_{𝔮} is compactly embedded, so that Proposition I.5 implies the existence of a Cartan involution θ̃ of 𝔰^{c} commuting with τ such that i(𝔭 ∩ 𝔰) ⊆ 𝔰_{𝔨}^{c} and therefore i(𝔭 ∩ 𝔰) ⊆ i𝔰_{𝔮} ∩ 𝔰_{𝔨}^{c}. Then

𝔰_{𝔨} = (𝔰_{𝔥} ∩ 𝔰_{𝔨}^{c}) ⊕ (𝔰_{𝔮} ∩ i𝔰_{𝔭}^{c}) and 𝔰_{𝔭} = (𝔰_{𝔥} ∩ 𝔰_{𝔭}^{c}) ⊕ (𝔰_{𝔮} ∩ i𝔰_{𝔨}^{c})

defines a Cartan involution of 𝔰 with 𝔭 ∩ 𝔰 ⊆ 𝔰_{𝔮} ∩ 𝔰_{𝔭}. In view of the maximality of 𝔭, this implies that

𝔭 = 𝔞_{𝔯} ⊕ (𝔰_{𝔭})_{𝔮}.

Then (𝔰_{𝔨})_{𝔮} = 𝔰_{𝔮} ∩ (𝔭 ∩ 𝔰)^{⊥} and

[𝔰_{𝔮}, 𝔰_{𝔮}] ∩ 𝔰_{𝔨} = [(𝔰_{𝔭})_{𝔮}, (𝔰_{𝔭})_{𝔮}] + [(𝔰_{𝔨})_{𝔮}, (𝔰_{𝔨})_{𝔮}]

shows that 𝔭 determines the Cartan decomposition of (𝔰_{𝔮})_{L} uniquely. (vi) Since 𝔯 is a semisimple 𝔞-module, we have 𝔯 = 𝔷_{𝔯}(𝔞) ⊕ [𝔞, 𝔯]. From the invariance of this decomposition under τ we further conclude that 𝔯_{𝔥} = 𝔷_{𝔯_{𝔥}}(𝔞) ⊕ [𝔞, 𝔯_{𝔮}]. Hence the assertion follows from [𝔞, 𝔯] ⊆ [𝔤, 𝔯] ⊆ 𝔫.

In the following we simply write 𝔟^{0} instead of 𝔷_{𝔟}(𝔞) for the centralizer of 𝔞 in the subspace 𝔟 of 𝔤.

Lemma III.6. For a maximal hyperbolic Lie triple system 𝔭 ⊆ 𝔮 and a maximal abelian subspace 𝔞 ⊆ 𝔭 we have

N_{Inn_{𝔤}(𝔥)}(𝔞) = N_{Inn_{𝔤}([𝔭,𝔭])}(𝔞) Inn_{𝔤}(𝔥^{0})

and

Z_{Inn_{𝔤}(𝔥)}(𝔞) = Z_{Inn_{𝔤}([𝔭,𝔭])}(𝔞) Inn_{𝔤}(𝔥^{0}).

Proof. First we note that, in view of Proposition III.5 and the conjugacy of all maximal hyperbolic Lie triple systems, there exists a maximal hyperbolic abelian subspace 𝔞_{𝔯} ⊆ 𝔯_{𝔮} with respect to 𝔤, a τ -𝔞_{𝔯} -invariant Levi decomposition 𝔤 = 𝔯 ⋊ 𝔰, and a Cartan decomposition 𝔰 = 𝔰_{𝔨} ⊕ 𝔰_{𝔭} which is invariant under τ such that 𝔭 = 𝔞_{𝔯} ⊕ (𝔰_{𝔮} ∩ 𝔰_{𝔭}).

We first show that

(3.1)  N_{Inn_{𝔤}(𝔥)}(𝔞) = N_{Inn_{𝔤}(𝔰_{𝔥})}(𝔞) Inn_{𝔤}(𝔯_{𝔥}^{0}).

The inclusion “⊇” is obvious. For the converse let h ∈ N_{Inn_{𝔤}(𝔥)}(𝔞). In view of 𝔥 = 𝔯_{𝔥} + 𝔰_{𝔥} = 𝔫_{𝔥} + 𝔯_{𝔥}^{0} + 𝔰_{𝔥} (Proposition III.5(vi)), we can write h as h = snz, where s ∈ Inn_{𝔤}(𝔰_{𝔥}), n ∈ Inn_{𝔤}(𝔫_{𝔥}) and z ∈ Inn_{𝔤}(𝔯_{𝔥}^{0}).

We denote the projection onto 𝔰 along 𝔯 by p_{𝔰}: 𝔤 → 𝔰. Since p_{𝔰}(r.Y) = Y_{s} holds for every Y ∈ 𝔞 and r ∈ Inn_{𝔤}(𝔯), we see that s ∈ N_{Inn_{𝔤}(𝔰_{𝔥})}(𝔞_{𝔰}) ⊆ N_{Inn_{𝔤}(𝔰_{𝔥})}(𝔞) because [𝔞_{𝔯}, 𝔰] = {0}. Hence n ∈ N_{Inn_{𝔤}(𝔫_{𝔥})}(𝔞). The nilpotency of the subalgebra 𝔫_{𝔥} implies the existence of an element Y ∈ 𝔫_{𝔥} with n = e^{ad Y} , and further the nilpotency of ad Y entails that ad Y = log e^{ad Y} preserves 𝔞. Now we use the semisimplicity of the 𝔞-module 𝔤 to see that

𝔫_{𝔤}(𝔞) = 𝔷_{𝔤}(𝔞)

because any 𝔞-submodule 𝔪 of 𝔫_{𝔤}(𝔞) complementary to 𝔷_{𝔤}(𝔞) satisfies [𝔞, 𝔪] ⊆ 𝔞 ∩ 𝔪 = {0}. This proves that Y ∈ 𝔯_{𝔥}^{0} . Hence h ∈ s Inn_{𝔤}(𝔯_{𝔥}^{0}), which establishes (3.1).

Next we show that

(3.2)  N_{Inn_{𝔤}(𝔰_{𝔥})}(𝔞) = N_{Inn_{𝔤}([𝔭,𝔭])}(𝔞) Inn_{𝔤}(𝔰_{𝔥}^{0}).

Let 𝔰_{𝔥} = 𝔨_{𝔥} ⊕ 𝔭_{𝔥} the Cartan decomposition according to θ |_{𝔰_{𝔥}}. Applying [21, Lemma 1.1.3.7] with respect to θ, we see that

(3.3)  N_{Inn_{𝔤}(𝔰_{𝔥})}(𝔞) = N_{Inn_{𝔤}(𝔨_{𝔥})}(𝔞)e^{ad 𝔭_{𝔥}^{0}}.

Since (𝔭∩𝔰)_{L} = (𝔭∩𝔰)⊕[𝔭, 𝔭] is an ideal of the reductive subalgebra 𝔨_{𝔥}⊕(𝔰_{𝔮}∩𝔰_{𝔭}) = 𝔨_{𝔥} ⊕ (𝔭 ∩ 𝔰) (cf. Lemma II.7(i)), it follows in particular that [𝔭, 𝔭] is an ideal of 𝔨_{𝔥} so that 𝔨_{𝔥} = [𝔭, 𝔭] ⊕ [𝔭, 𝔭]^{⊥}, where [𝔭, 𝔭]^{⊥} ⊆ 𝔷_{𝔨_{𝔥}}(𝔭) ⊆ 𝔨_{𝔥}^{0} . We conclude that

(3.4)  N_{Inn_{𝔤}(𝔨_{𝔥})}(𝔞) = N_{Inn_{𝔤}([𝔭,𝔭])}(𝔞) Inn_{𝔤}(𝔨_{𝔥}^{0}).

Hence (3.2) follows from (3.3), and (3.4).

Combining (3.1) and (3.2) implies the first statement of the lemma. The second assertion is immediate from the first one.

Proposition III.7. Two maximal hyperbolic Lie triple systems in 𝔮 containing a maximal hyperbolic abelian subspace 𝔞 are conjugate under Inn_{𝔤}(𝔥^{0}).

Proof. Let 𝔭 and 𝔭′ be maximal hyperbolic Lie triple systems containing 𝔞. Let γ ∈ Inn_{𝔤}(𝔥) such that γ.𝔭′ = 𝔭 (Corollary II.8). Then 𝔞 and γ.𝔞 are maximal abelian subspaces in 𝔭, hence conjugate under Inn_{𝔤}([𝔭, 𝔭]) (Theorem III.3). Thus we may assume that γ.𝔞 = 𝔞, i.e., γ ∈ N_{Inn_{𝔤}(𝔥)}(𝔞).

Now Lemma III.6 applies and yields γ = ση, where σ ∈ Inn_{𝔤}([𝔭, 𝔭]) and η ∈ Inn_{𝔤}(𝔥^{0}). Obviously σ stabilizes 𝔭 and therefore η.𝔭′ = 𝔭, which proves the proposition.

For symmetric Lie algebras of the type (𝔥_{ℂ}, σ), where σ is complex conjugation, we call these of complex type, Proposition III.7 can be used to obtain a new proof of the following results which plays a crucial role for Lie algebras with compactly embedded Cartan subalgebras (cf. [4, 3.13]).

Corollary III.8. Let 𝔤 be a Lie algebra with compactly embedded Cartan subalgebra 𝔱. Then 𝔱 is contained in a unique maximal compactly embedded subalgebra 𝔨. Moreover, i𝔨 is a maximal hyperbolic Lie triple system in i𝔤 for the symmetric Lie algebra (𝔤_{ℂ}, σ), where σ denotes complex conjugation.

Proof. We consider the symmetric Lie algebra (𝔤_{ℂ}, σ) with σ complex conjugation. Then 𝔞 := i𝔱 is a maximal hyperbolic abelian subspace in 𝔮 = i𝔤. If 𝔨 ⊇ 𝔱 is a maximal compactly embedded subalgebra, then 𝔭 := i𝔨 is a hyperbolic Lie triple system in 𝔮 containing 𝔞. We claim that 𝔭 is maximal.

To see this, we only have show that one maximal hyperbolic Lie triple systems 𝔭′ in 𝔮 the subspace i𝔭′ of 𝔤 is a subalgebra. Then the conjugacy result in Corollary II.8 implies that this holds for all maximal hyperbolic Lie triple systems. So observe that 𝔞_{𝔯} := i(𝔱 ∩ 𝔯) is a maximal hyperbolic abelian subspace in i𝔯 with respect to 𝔤_{ℂ} because, according to Theorem III.3, all compactly embedded Cartan subalgebras of 𝔤 are conjugate. Let 𝔤_{ℂ} = 𝔯_{ℂ} ⋊ 𝔰_{ℂ} be an 𝔞_{𝔯} - σ -invariant Levi decomposition, i.e., 𝔯_{ℂ} and 𝔰_{ℂ} are complexifications of 𝔯 and 𝔰 for a Levi decomposition 𝔤 = 𝔯 ⋊ 𝔰 of 𝔤. We know from Proposition III.5 how to construct from this data a maximal hyperbolic subspace of 𝔮 by taking

𝔭′ := 𝔞_{𝔯} ⊕ ((𝔰_{ℂ})_{𝔭} ∩ i𝔰),

where 𝔰_{ℂ} = (𝔰_{ℂ})_{𝔨} ⊕(𝔰_{ℂ})_{𝔭} is a Cartan decomposition, i.e., 𝔲 := (𝔰_{ℂ})_{𝔨} is a compact real form of 𝔰_{ℂ} and (𝔰_{ℂ})_{𝔭} = i𝔲. Hence 𝔭′ = 𝔞_{𝔯} ⊕ i(𝔲 ∩ 𝔰) implies that i𝔭′ is a subalgebra of 𝔤 and therefore 𝔭 = i𝔨 is maximal.

Now the assertion follows from 𝔥^{0} = 𝔱 ⊆ 𝔨 and Proposition III.7.

Definition III.9. We define the Weyl group W of (𝔤, τ, 𝔞) by

W := N_{Inn_{𝔤}(𝔥)}(𝔞)/Z_{Inn_{𝔤}(𝔥)}(𝔞)

and recall from Lemma III.6 that

W ≅ N_{Inn_{𝔤}([𝔭,𝔭])}(𝔞)/Z_{Inn_{𝔤}([𝔭,𝔭])}(𝔞)

holds for any maximal hyperbolic Lie triple system 𝔭 containing 𝔞. This shows in particular that W is isomorphic to the Weyl group of the Riemannian symmetric Lie algebra 𝔭_{L} and that W is trivial whenever 𝔤 is solvable.

Next we apply the information we have on the normalizer of a maximal hyperbolic abelian subspace to get a nice description of the intersection of Inn_{𝔤}(𝔥)-orbits in 𝔮_{hyp} with 𝔞 (cf. also Theorem III.3).

Theorem III.10. Let 𝔭 be a maximal hyperbolic Lie triple system in 𝔮, 𝔞 ⊆ 𝔭 maximal abelian, and X ∈ 𝔞. Then the orbit O_{X} := Inn_{𝔤}(𝔥).X intersects 𝔞 in an orbit of the Weyl group W.

Proof. Let 𝔤^{X} = ker ad X denote the centralizer of X in 𝔤 and note that this subalgebra is τ -invariant and contains 𝔞. Let 𝔭^{X} := 𝔷_{𝔭}(X) and 𝔭̃ ⊇ 𝔭^{X} be a maximal hyperbolic Lie triple system in 𝔮^{X} := 𝔷_{𝔮}(X) with respect to 𝔮_{L}^{X}. Then Proposition II.6(ii) applied with 𝔮̃ = 𝔮^{X} yields

(3.5)  𝔞 = (𝔞 ∩ 𝔷(𝔭̃)) ⊕ (𝔞 ∩ [𝔭̃, [𝔭̃, 𝔭̃]]) and 𝔞 ∩ 𝔷(𝔭̃) = 𝔷(𝔭̃) ∩ 𝔮_{hyp},

where 𝔞 ∩ [𝔭̃, [𝔭̃, 𝔭̃]] is maximal abelian in [𝔭̃, [𝔭̃, 𝔭̃]].

Now let h ∈ Inn_{𝔤}(𝔥) with h.X ∈ 𝔞. Then 𝔟 := h^{−1}.𝔞 ⊆ 𝔷_{𝔮}(X) is a hyperbolic subspace. Hence there exists h_{1} ∈ ⟨e^{ad 𝔥^{X}} ⟩ with h_{1}.𝔟 ⊆ 𝔭̃ (Corollary II.9). Since hh_{1}^{−1}.X = h.X, we may therefore w.l.o.g. assume that 𝔟 ⊆ 𝔭̃. Now (3.5) applies to 𝔟 as well and asserts that

𝔟 ∩ 𝔷(𝔭̃) = 𝔷(𝔭̃) ∩ 𝔮_{hyp} = 𝔞 ∩ 𝔷(𝔭̃).

Let 𝔞_{0} := 𝔞 ∩ 𝔷(𝔭̃), put 𝔞_{1} := 𝔞 ∩ [𝔭̃, [𝔭̃, 𝔭̃]], and note that this is a maximal abelian subspace in [𝔭̃, [𝔭̃, 𝔭̃]]. Defining 𝔟_{1} similarly, we obtain

𝔞 = 𝔞_{0} ⊕ 𝔞_{1} and 𝔟 = 𝔞_{0} ⊕ 𝔟_{1},

where 𝔟_{1} is also maximal abelian in [𝔭̃, [𝔭̃, 𝔭̃]]. Next we use [3, Lemma V.6.3] to find h_{2} ∈ exp[𝔭̃, 𝔭̃] with h_{2}.𝔟_{1} = 𝔞_{1} . Replacing h by hh_{2}^{−1} , we may now w.l.o.g. assume that h.𝔞 = 𝔞. So we have shown that

O_{X} ∩ 𝔞 = N_{Inn_{𝔤}(𝔥)}(𝔞).X.

Finally we use Lemma III.6 to obtain

O_{X} ∩ 𝔞 = N_{Inn_{𝔤}([𝔭,𝔭])}(𝔞) Inn_{𝔤}(𝔥^{0}).X = N_{Inn_{𝔤}([𝔭,𝔭])}(𝔞).X = W.X.

For later reference we also record the following lemma.

Lemma III.11. Let (𝔤, τ) be a symmetric Lie algebra, 𝔧 ⊆ 𝔤 a τ -invariant ideal, and π: 𝔤 → 𝔤_{1}: = 𝔤/𝔧 the quotient map. Write 𝔟_{1} := π(𝔟) for a subspace 𝔟 ⊆ 𝔤. Then τ induces a symmetric structure τ_{1} on 𝔤_{1} and the eigenspace decomposition of 𝔤_{1} w.r.t. τ_{1} is given by 𝔤_{1} = 𝔥_{1} ⊕ 𝔮_{1} . Moreover, if 𝔭 ⊆ 𝔮 is a maximal hyperbolic Lie triple system and 𝔞 ⊆ 𝔭 is maximal abelian in 𝔭 and 𝔮, then 𝔭_{1} is a maximal hyperbolic Lie triple system in 𝔮_{1} , and 𝔞_{1} is maximal abelian in 𝔭_{1} and 𝔮_{1} . Proof. The existence of τ_{1} and the corresponding decomposition 𝔤_{1} = 𝔥_{1} ⊕𝔮_{1} is trivial.

Writing 𝔮 as 𝔮 = 𝔞 ⊕ [𝔞, 𝔥], we see that 𝔮_{1} = 𝔞_{1} ⊕ [𝔞_{1}, 𝔥_{1}] and therefore that 𝔞_{1} is maximal abelian in 𝔮_{1} because 𝔤_{1} is a semisimple 𝔞_{1} -module. The same argument proves that 𝔞_{1} is maximal abelian in 𝔭_{1} .

Therefore it only remains to show that 𝔭_{1} is a maximal hyperbolic Lie triple system. For that we may w.l.o.g. assume that 𝔭 = 𝔞_{𝔯} ⊕ 𝔭_{𝔰} is constructed as in Proposition III.5(iv) (cf. Theorem III.3), where 𝔰 is a τ -𝔞-invariant Levi complement, 𝔰 = 𝔰_{𝔨} ⊕ 𝔰_{𝔭} is a Cartan decomposition of 𝔰 commuting with τ, and 𝔭_{𝔰} = (𝔰_{𝔭})_{𝔮}. Then the ideal 𝔧 decomposes as 𝔧 = (𝔧 ∩ 𝔯) ⋊ (𝔧 ∩ 𝔰) (cf. [1, Ch. 1, §6, no. 8, Cor. 4]) and 𝔧 ∩ 𝔰 is invariant under θ because ideals of semisimple Lie algebras are invariant under Cartan involutions. Choosing a complementary ideal 𝔰_{0} for 𝔧 ∩ 𝔰 in 𝔰, we see that 𝔰_{1} = (𝔰_{𝔨})_{1} ⊕ (𝔰_{𝔭})_{1} is a Cartan decomposition of 𝔰_{1} invariant under τ_{1} , and that (𝔭_{𝔰})_{1} = π(𝔭_{𝔰}) = (𝔰_{𝔭})_{1} ∩ (𝔰_{𝔮})_{1} . Since 𝔭_{1} = (𝔞_{𝔯})_{1} ⊕ (𝔭_{𝔰})_{1} , in view of Proposition III.5(iv), it suffices to show that (𝔞_{𝔯})_{1} is maximal hyperbolic abelian in 𝔯 with respect to 𝔤_{1} . Since 𝔞_{1} is a maximal hyperbolic abelian subspace of 𝔮_{1} and (𝔞_{𝔯})_{1} = 𝔯_{1} ∩ 𝔞_{1} , this follows from Proposition III.5(ii).

Example III.12. Let 𝔥 be a 2-step nilpotent Lie algebra and (𝔤, τ) := (𝔥_{ℂ}, σ), where σ denotes complex conjugation. Then 𝔞 = i𝔷(𝔥) is a maximal hyperbolic abelian subspace in 𝔮 = i𝔥 which is not at the same time maximal abelian in 𝔮. The image of 𝔞 in 𝔤/𝔷(𝔤) is {0} but since 𝔤/𝔷(𝔤) is abelian, it is not maximally hyperbolic in 𝔮. This shows that the additional assumption that 𝔞 is also maximal abelian in 𝔮 is crucial in Lemma III.11.

IV. Root decompositions

In this section (𝔤, τ) denotes a symmetric Lie algebra. Here we analyze the root decomposition of 𝔤 with respect to an abelian hyperbolic subspace 𝔞 of 𝔮 which is, in addition, maximal abelian in 𝔮. The latter condition is important to ensure that 𝔞 is big enough so that the corresponding root decomposition carries significant information on the structure of the whole Lie algebra.

Theorem IV.1. Let 𝔞 ⊆ 𝔮 be a maximal hyperbolic abelian subspace which is maximal abelian in 𝔮. For a linear functional α ∈ 𝔞^{∗} we set

𝔤^{α} := {X ∈ 𝔤 : (∀Y ∈ 𝔞)[Y, X] = α(Y)X}

and

Δ := Δ(𝔤, 𝔞) := {α ∈ 𝔞^{∗} \ {0} : 𝔤^{α} ≠ {0}}.

Then the following assertions hold:

(i) 𝔤 = 𝔷_{𝔤}(𝔞) ⊕ ⨁_{α∈Δ} 𝔤^{α}.

(ii) τ(𝔤^{α}) = 𝔤^{−α}.

(iii) Let α ≠ 0, Z ∈ 𝔤^{α}, and write 𝔤(Z) = span{Z, τ(Z), [Z, τ(Z)]} for the τ -invariant subalgebra of 𝔤 generated by Z. Then [Z, τ(Z)] ∈ 𝔞, 𝔥(Z) = ℝ(Z + τ(X)) is one-dimensional, and there are four possibilities:

(SR) α([Z, τ(Z)]) > 0. Then 𝔤(Z) ≅ 𝔰𝔩(2, ℝ), 𝔥(Z) ≅ 𝔰𝔬(1, 1) and 𝔮(Z) is neither elliptic nor hyperbolic. The pair (𝔤(Z), 𝔥(Z)) is semi-Riemann-ian.

(R) α([Z, τ(Z)]) < 0. Then 𝔤(Z) ≅ 𝔰𝔩(2, ℝ), 𝔥(Z) ≅ 𝔰𝔬(2, ℝ) and 𝔮(Z) is

hyperbolic. The pair (𝔤(Z), 𝔥(Z)) is Riemannian. (N) α([Z, τ(Z)]) = 0 and [Z, τ(Z)] ≠ 0. Then 𝔤(Z) is isomorphic to

the three-dimensional Heisenberg algebra, and for every E ∈ 𝔞 with α(E) ≠ 0 the algebra 𝔤(Z, E) := 𝔤(Z)⋊ℝE is a four dimensional solvable symmetric Lie algebra.

(A) [Z, τ(Z)] = 0. Then 𝔤(Z) ≅ ℝ^{2} and for every E ∈ 𝔞 with α(E) ≠ {0} the algebra 𝔤(Z, E) = 𝔤(Z) ⋊ ℝE is a three dimensional solvable Lie algebra.

Proof. (i) The algebra ad 𝔞 ⊆ 𝔤𝔩(𝔤) is abelian and consists of diagonalizable elements. Hence this set permits a simultaneous diagonalization. This proves (i). (ii) For E ∈ 𝔞 and Z ∈ 𝔤^{α} we have

[E, τ(Z)] = τ([τ(E), Z]) = −α(E)τ(Z).

Therefore τ(Z) ∈ 𝔤^{−α}. (iii) Let H_{Z} := [Z, τ(Z)]. Then, in view of (ii), H_{Z} ∈ [𝔤^{α}, 𝔤^{−α}] ⊆ 𝔤^{0} = 𝔷_{𝔤}(𝔞) and

τ(H_{Z}) = τ([Z, τ(Z)]) = [τ(Z), Z] = −H_{Z}

it follows that H_{Z} ∈ 𝔷_{𝔤}(𝔞) ∩ 𝔮 = 𝔞. After rescaling Z, we may w.l.o.g. assume that α(H_{Z}) ∈ {2, 0, −2}. (SR) If α([Z, τ(Z)]) > 0, then α(H_{Z}) = 2 so that

[H_{Z}, Z] = 2Z, [H_{Z}, τ(Z)] = −2τ(Z), H_{Z} = [Z, τ(Z)]

implies that 𝔤(Z) ≅ 𝔰𝔩(2, ℝ), where 𝔥(Z) = ℝ(Z +τ(Z)) corresponds to 𝔰𝔬(1, 1). (R) If α([Z, τ(Z)]) < 0, then α(H_{Z}) = −2 so that

[H_{Z}, Z] = −2Z, [H_{Z}, τ(Z)] = 2τ(Z), H_{Z} = [Z, τ(Z)]

implies that 𝔤(Z) ≅ 𝔰𝔩(2, ℝ), where 𝔥(Z) = ℝ(Z + τ(Z)) corresponds to 𝔰𝔬(2). (N) If α([Z, τ(Z)]) = 0 and [Z, τ(Z)] ≠ 0, then H_{Z} is central in 𝔤(Z) and H_{Z} = [Z, τ(Z)] implies that 𝔤(Z) ≅ 𝔥_{1} , the three dimensional Heisenberg algebra. (A) If [Z, τ(Z)] = 0, then 𝔤(Z) ≅ ℝ^{2} .

The remaining assertions are easy consequence of these computations.

Definition IV.2. In the preceding theorem we have seen that for an element Z ∈ 𝔤^{α} there are four possibilities. In the following we will say that Z is of semi-Riemannian type (SR), Riemannian type (R), nilpotent type (N), and abelian type (A), whenever the corresponding case in Theorem IV.1 occurs.

Before we turn to the structure of the root decomposition we discuss some basic examples.

Example IV.3. (a) Let 𝔤 = 𝔤_{0} ⊕ 𝔤_{0} where 𝔤_{0} is a Lie algebra and τ is given by τ(X, Y) = (Y, X). Then we get

𝔥 = {(X, X) : X ∈ 𝔤_{0}} and 𝔮 = {(X, −X) : X ∈ 𝔤_{0}}.

We observe that an element (X, −X) ∈ 𝔮 is hyperbolic if and only if X is a hyperbolic element of the Lie algebra 𝔤_{0} . Therefore there exists a hyperbolic subspace 𝔞 ⊆ 𝔮 which is maximal abelian in 𝔮 if and only if 𝔤_{0} possesses a Cartan subalgebra 𝔞_{0} consisting of hyperbolic elements, i.e., if 𝔤_{0} has a toral Cartan subalgebra. If 𝔤_{0} is semisimple, this means that 𝔤_{0} is a normal real form of its complexification.

Applying Theorem III.3 to this situation, we see that all maximal abelian hyperbolic subspaces of a Lie algebra 𝔤_{0} are conjugate under inner automorphisms and in particular that two toral Cartan subalgebras are conjugate whenever such Cartan subalgebras exist.

Let 𝔞_{0} ⊆ 𝔤_{0} be a toral Cartan subalgebra. Then the subspace 𝔞 = {(X, −X) : X ∈ 𝔞_{0}} is hyperbolic and maximal abelian in 𝔮. If Δ_{0}: = Δ(𝔞_{0}, 𝔤_{0}) is the root system of 𝔤_{0} with respect to 𝔞_{0} , then the root spaces for 𝔞 are given by

𝔤^{α} := (𝔤_{0}^{α_{0}} × {0}) ⊕ ({0} × 𝔤_{0}^{−α_{0}}), α_{0} ∈ Δ_{0},

where α(X, −X) = α_{0}(X) is the corresponding root. Let Z = (Z_{α_{0}}, Z_{−α_{0}}) ∈ 𝔤^{α}. Then

[Z, τ(Z)] = ([Z_{α_{0}}, Z_{−α_{0}}], −[Z_{α_{0}}, Z_{−α_{0}}]) and α([Z, τ(Z)]) = α_{0}([Z_{α_{0}}, Z_{−α_{0}}]).

Suppose that 𝔤_{0} is semisimple with Cartan involution θ satisfying 𝔞_{0} ⊆ 𝔭. Then Theorem IV.1 implies that α_{0}([Z_{α_{0}}, θ(Z_{α_{0}})]) < 0. Thus, putting Z_{−α_{0}} = 0 or Z_{−α_{0}} = ±θ(Z_{α_{0}}), we obtain root vectors Z of the types (SR), (R) and (A) in each root space 𝔤^{α}.

(b) Let 𝔥 be a Lie algebra and (𝔤, τ) := (𝔥_{ℂ}, σ), where σ denotes complex conjugation. In this case 𝔮 = i𝔥. An element X ∈ 𝔮 is hyperbolic if and only if X is an elliptic element of the Lie algebra 𝔥. Therefore a hyperbolic subspace 𝔞 ⊆ 𝔮 is maximal abelian in 𝔮 if and only if i𝔞 is a compactly embedded Cartan subalgebra of 𝔥.

Using Theorem III.3 again, we regain the fact that all maximal abelian compactly embedded subalgebras of a Lie algebra 𝔥 are conjugate under inner automorphisms and in particular that two compactly embedded Cartan subalgebras are conjugate whenever they exist.

Let 𝔱 ⊆ 𝔥 be a compactly embedded Cartan subalgebra. Then the subspace 𝔞 := i𝔱 ⊆ 𝔮 is hyperbolic and maximal abelian in 𝔮 and the root systems Δ(𝔤, 𝔞) and Δ(𝔥_{ℂ}, 𝔱_{ℂ}) coincide. For Z ∈ 𝔤^{α} = 𝔥_{ℂ}^{α} we have [Z, τ(Z)] = [Z, \overline{Z}] so the four types from Theorem IV.1 correspond to the four types of root spaces considered in [5, Ch. 7].

Having introduced the general setup for the root decompositions, we now turn to their relations to well chosen Levi decompositions of 𝔤.

Definition IV.4. If 𝔤 = 𝔯 ⋊ 𝔰 is an 𝔞-τ -invariant Levi decomposition, then we say that a root α ∈ Δ is semisimple if 𝔰^{α} ≠ {0} and solvable if 𝔤^{α} ⊆ 𝔯. We write Δ_{s} for the set of semisimple roots and Δ_{r} for the set of solvable roots.

Lemma IV.5. For an 𝔞-τ -invariant Levi decomposition 𝔤 = 𝔯⋊𝔰 the following assertions hold:

(i) 𝔤^{α} = 𝔯^{α} ⊕ 𝔰^{α}, where 𝔯^{α} = 𝔤^{α} ∩ 𝔯 and 𝔰^{α} = 𝔤^{α} ∩ 𝔰.

(ii) Δ_{s}^{⊥} = 𝔞_{𝔯}.
(iii) 𝔤 ⊆ 𝔮_{L} + 𝔥^{0} .

Proof. (i) Since 𝔯 and 𝔰 are invariant under 𝔞, both subspaces of 𝔤 decompose according to the root decomposition of Theorem IV.1. Therefore each root space 𝔤^{α} can be written as 𝔤^{α} = (𝔤^{α} ∩ 𝔯) ⊕ (𝔤^{α} ∩ 𝔰), whence the assertion. (ii) If α ∈ Δ_{s}, then 𝔰^{α} ≠ {0}, so that [𝔞_{𝔯}, 𝔰^{α}] ⊆ 𝔯 ∩ 𝔰^{α} = {0} implies that α|_{𝔞_{𝔯}} = 0.

If, conversely, X ∈ Δ_{s}^{⊥}, then X ∈ 𝔷_{𝔞}(𝔰) ⊆ 𝔞_{𝔯}, because each element in 𝔷_{𝔤}(𝔰) is mapped by the natural homomorphism 𝔤 → 𝔤/𝔯 ≅ 𝔰 onto a central element of 𝔰, hence to 0. (iii) According to the root space decomposition, we have 𝔤 = 𝔥^{0} ⊕ 𝔞 ⊕ ⨁_{α∈Δ} 𝔤^{α}. Therefore 𝔞 ⊆ 𝔮 and 𝔤^{α} ⊆ 𝔮_{L} imply that 𝔤 ⊆ 𝔮_{L} + 𝔥^{0} .

We recall our notation κ for the Cartan-Killing form of 𝔤 and write m_{α} := dim 𝔤^{α} for the multiplicity of the root α ∈ Δ.

Definition IV.6. Let (𝔤, τ) be a symmetric Lie algebra and V a 𝔤-module carrying a bilinear form ϕ. The form ϕ is called τ -covariant, if

ϕ(X.v, w) = −ϕ(v, τ(X).w)

holds for all X ∈ 𝔤 and v, w ∈ V .

Proposition IV.7. If 𝔰 is τ -𝔞-invariant Levi complement and κ_{𝔞} denotes the restriction of the Cartan-Killing form to 𝔞, then the following assertions hold:

(i) κ_{𝔞}(X, Y) = ∑_{α∈Δ} m_{α}α(X)α(Y) and κ_{𝔞}(X, X) ≥ 0 for X, Y ∈ 𝔞.

(ii) 𝔞^{⊥_{κ𝔞}} = 𝔞_{𝔷} := 𝔷(𝔤) ∩ 𝔞, 𝔞_{𝔯}^{⊥_{κ𝔞}} = 𝔞_{𝔷} ⊕ 𝔞_{𝔰}, and 𝔞_{𝔰}^{⊥_{κ𝔞}} = 𝔞_{𝔯}.
(iii) If we define ϕ: 𝔞 → 𝔞^{∗} by ϕ(X)(Y) := κ(X, Y), then

ϕ(X) = ∑_{α∈Δ} m_{α}α(X)α,

ker ϕ = 𝔞_{𝔷}, and ϕ(𝔞) = span Δ. The prescription ⟨ϕ(X), ϕ(Y)⟩ := κ(X, Y) defines a scalar product on span Δ.

(iv) For each α ∈ Δ_{s} there exists a unique element A_{α} ∈ 𝔞_{𝔰} with ϕ(A_{α}) = α which, in addition, satisfies α(A_{α}) > 0.

(v) If Z ∈ 𝔤^{α}, then [Z, τ(Z)] ∈ 𝔞_{𝔷} for α ∈ Δ_{r}, and

[Z, τ(Z)] ∈ κ(Z, τ(Z))A_{α} + 𝔞_{𝔷}

for α ∈ Δ_{s}. If, in addition, Z ∈ 𝔰^{α}, then

[Z, τ(Z)] = κ(Z, τ(Z))A_{α}.

(vi) The form defined by κ_{τ} (X, Y) := κ(X, τ.Y) is a symmetric τ - covariant form on 𝔤. Moreover, on each root space 𝔤^{α} the degenerate subspace coincides with 𝔯^{α}, and an element X ∈ 𝔤^{α} is

(a) of type (SR) if κ_{τ} (X) > 0,

(b) of type (R) if κ_{τ} (X) < 0, and

(c) of type (N) or (A) if κ_{τ} (X) = 0.

Proof. (i) This is an immediate consequence of the definition of the Cartan-Killing form. (ii) That 𝔞^{⊥_{κ𝔞}} = 𝔷(𝔤) ∩ 𝔞 follows directly from the explicit formula in (i) and the observation that 𝔷(𝔤) ∩ 𝔞 = ⋂_{α∈Δ} ker α.

Next we recall from [1, Ch. 1, §5.5, Prop. 5(b)] that 𝔯 = [𝔤, 𝔤]^{⊥_{κ}}. From that we conclude that 𝔞_{𝔯} ⊆ 𝔞_{𝔰}^{⊥_{κ𝔞}} , so that the equality follows from the nondegeneracy of κ_{𝔞} on 𝔞_{𝔰}. Thus we get that

𝔞_{𝔯}^{⊥_{κ𝔞}} = (𝔞_{𝔰}^{⊥_{κ𝔞}})^{⊥_{κ𝔞}} = 𝔞_{𝔰} ⊕ 𝔞_{𝔷}.

(iii) The formula for ϕ follows by rewriting (i). From that we observe that ker ϕ = 𝔞_{𝔷} and ϕ(𝔞) = span Δ. In view of (i), it now is clear that ⟨ϕ(X), ϕ(Y)⟩ := κ(X, Y) defines a scalar product on span Δ. (iv) Let α ∈ Δ_{s} and X ∈ ϕ^{−1}(α). Then κ(X, 𝔞_{𝔯}) = α(𝔞_{𝔯}) = 0 (cf. Lemma IV.5) implies that X ∈ 𝔞_{𝔯}^{⊥_{κ𝔞}} = 𝔞_{𝔰} ⊕ 𝔞_{𝔷}. Writing X = X_{s} + X_{z} with X_{z} ∈ 𝔞_{𝔷} and X_{s} ∈ 𝔞_{𝔰}, we see that X_{s} ∈ 𝔞_{𝔰} ∩ ϕ^{−1}(α) ≠ Ø. Now the uniqueness of A_{α} follows from the injectivity of ϕ|_{𝔞_{𝔰}}. Finally we observe that

α(A_{α}) = κ(A_{α}, A_{α}) = ∑_{β∈Δ} β(A_{α})^{2} > 0.

(v) Let Z ∈ 𝔤^{α}. If α ∈ Δ_{r}, then 𝔤^{α} = 𝔯^{α} and therefore [Z, τ(Z)] ∈ 𝔞_{𝔯} ∩[𝔯, 𝔯] ⊆ 𝔞_{𝔷} because for each element X in [𝔯, 𝔯] the operator ad X is nilpotent, so that the simultaneous semisimplicity implies that it is central.

If α ∈ Δ_{s} and A_{α} is chosen as in (iv), then we have for Y ∈ 𝔞:

\begin{aligned}ϕ([Z, τ(Z)])(Y) & = κ([Z, τ(Z)], Y ) = κ(Z, [τ(Z), Y]) \\ & = α(Y)κ(Z, τ(Z)) = κ(Z, τ(Z))ϕ(A_{α})(Y) \end{aligned}

which entails that

[Z, τ(Z)] ∈ ϕ^{−1}(κ(Z, τ(Z))A_{α}) = κ(Z, τ(Z))A_{α} + 𝔞_{𝔷}.

If, in addition, Z ∈ 𝔰^{α}, then A_{α} ∈ 𝔞_{𝔰} entails [Z, τ(Z)] = κ(Z, τ(Z))A_{α}. (vi) First we note that the invariance of the Cartan-Killing form κ under τ and τ^{2} = id_{𝔤} imply that κ_{τ} (X, Y) := κ(X, τ.Y) defines a τ -covariant symmetric bilinear form on 𝔤.

We recall from the proof of (ii) that 𝔯 = [𝔤, 𝔤]^{⊥_{κ}} which, in view of 𝔤^{α} ⊆ [𝔤, 𝔤], shows that κ_{τ} (𝔯^{α}, 𝔤^{α}) = {0}. Since κ_{τ} is τ -covariant, ad 𝔞 acts by κ_{τ} -symmetric operators, hence κ_{τ} (𝔤^{α}, 𝔤^{β}) = {0} for α + β ≠ 0. For any 𝔞-τ -invariant Levi complement 𝔰 we have κ(𝔯, 𝔰) ⊆ κ(𝔯, [𝔤, 𝔤]) = {0} and hence 𝔰^{⊥} ∩ 𝔰 ⊆ 𝔤^{⊥} ∩ 𝔰 ⊆ 𝔯 ∩ 𝔰 = {0}. Thus κ_{τ} restricted to 𝔰 is non-degenerate and the orthogonality of the root spaces implies that κ_{τ} is non-degenerate on 𝔰^{α}.

Now let Z ∈ 𝔤^{α}. If α ∈ Δ_{𝔯}, then 𝔤^{α} ⊆ 𝔯, κ_{τ} vanishes on 𝔤^{α}, and since 𝔯 contains no semsisimple subalgebras, Z is of type (N) or (A). If α ∈ Δ_{s}, then (v) shows that

α([Z, τ.Z]) = α(A_{α})κ_{τ} (Z).

Therefore the assertion follows from α(A_{α}) > 0.

Proposition IV.8. Let 𝔤 = 𝔯 ⋊ 𝔰 be a τ -𝔞-invariant Levi decomposition and θ a Cartan involution of 𝔰 commuting with τ. Then the following assertions hold:

(i) The involution τ^{a} := θτ |_{𝔰} preserves the root spaces 𝔰^{α} of 𝔞 and the quadratic form κ_{τ} is positive definite on the −1-eigenspace and negative definite on the 1-eigenspace of τ^{a} in 𝔰^{α}.

(ii) Each element Z ∈ 𝔰^{α} is of type (SR), (R) or (A). If the types (SR) and (R) occur in 𝔰^{α}, then type (A) also occurs.

(iii) Each root space 𝔤^{α}, α ∈ Δ_{s} contains an element of type (SR) or (R).

(iv) In 𝔯^{α} only the types (N) and (A) may occur.

Proof. (i) Since 𝔞_{𝔯} commutes with 𝔰 and σ leaves the elements of 𝔞_{𝔰} pointwise fixed, it preserves the root spaces 𝔰^{α}. For Z ∈ 𝔰^{α} we write Z = Z_{+} +Z_{−} according to the σ -eigenspace decomposition. Note that the invariance of κ_{τ} under σ implies that the components are orthogonal with respect to κ_{τ}. Therefore

\begin{aligned}κ_{τ} (Z) & = κ(Z, τ(Z)) = κ(Z_{+} + Z_{−}, θ(Z_{+}) − θ(Z_{−})) \\ & = \underbrace{κ(Z_{+}, θ(Z_{+}))}_{≤0} − \underbrace{κ(Z_{−}, θ(Z_{−}))}_{≥0} \end{aligned}

follows from the fact that the form κ_{θ}(Z) := κ(Z, θ(Z)) is negative definite on 𝔰 which in turns follows from the fact that for θ.X = X the element X is elliptic and non-central, and for θ.X = −X the element X is hyperbolic and non-central (cf. Lemma I.9(ii)). (ii) Since for Z ∈ 𝔰^{α} we have [Z, τ(Z)] = κ(Z, τ(Z))A_{α}, the assertion follows from Proposition IV.7(vi) if we note that the occurence of type (R) and (SR) means that κ_{τ} is indefinite on 𝔰^{α} which implies the existence of isotropic vectors and therefore of elements of type (A). (iii), (iv) In view of Proposition IV.7(vi), the form κ_{τ} is non-zero on 𝔤^{α} if and only if α ∈ Δ_{s}. Therefore the assertions follow from Proposition IV.7(vi).

Lemma IV.9. If α ∈ Δ_{s} with 𝔤^{α} = 𝔰^{α} and κ_{τ} is definite on 𝔤^{α}, then 𝔤^{α} is an irreducible 𝔥^{0} -module. Proof. Let 0 ≠ Z ∈ 𝔤^{α} and note that, in view of Proposition IV.8, our assumption implies that the subalgebra spanned by Z, τ(Z) and [Z, τ(Z)] is isomorphic to 𝔰𝔩(2, ℝ). Thus 𝔰𝔩_{2} -theory, applied to the module ∑_{n∈ℤ} 𝔤^{nα} shows that ad Z: 𝔤^{0} → 𝔤^{α} is surjective. Therefore 𝔤^{α} = [Z, 𝔞] + [Z, 𝔥^{0}] = ℝZ + [Z, 𝔥^{0}]. From that we obtain that the 𝔥^{0} -submodule of 𝔤^{α} generated by Z coincides with 𝔤^{α}. Since Z ∈ 𝔤^{α} was arbitrary, the irreducibility follows.

Example IV.10. (a) Let 𝔤 = (𝔥_{1} ⊕ V) ⋊ ℝH, where 𝔥_{1} = span{P, Q, Z} is the three dimensional Heisenberg algebra with [P, Q] = Z ∈ 𝔷(𝔥_{1}) on which H acts as by

H.P = P, H.Q = −Q, and H.Z = 0,

and V = ℝP′ ⊕ ℝQ′ on which H acts in the same way. Let

𝔥 := ℝ(P + Q) ⊕ ℝ(P′ + Q′), and 𝔮 := ℝZ ⊕ ℝ(P − Q) ⊕ ℝ(P′ − Q′) ⊕ ℝH.

Since [𝔮, 𝔮] ⊆ 𝔥, [𝔮, 𝔥] ⊆ 𝔮 and [𝔥, 𝔥] ⊆ 𝔥, we obtain an involutive automorphism τ of 𝔤 by τ |_{𝔥} = id_{𝔥} and τ |_{𝔮} = − id_{𝔮}.

Now 𝔞 = 𝔷(𝔤)⊕ℝH is a hyperbolic subspace which is maximal abelian in 𝔮, and the root system is given by Δ = Δ_{r} = {±α}, where α(H) = 1. Moreover

𝔤^{α} = span{P, P′}, 𝔤^{−α} = span{Q, Q′},

and τ(P) = Q, τ(P′) = Q′. Hence type (N) as well as type (A) occurs in 𝔤^{α}.

(b) Let 𝔤 = 𝔥_{2} ⋊ ℝH, where 𝔥_{2} = span{P, Q, P′, Q′, Z} is the five dimensional Heisenberg algebra with [P, Q] = [P′, Q′] = Z ∈ 𝔷(𝔥_{2}). The action of H is defined by

H.P = P, H.Q = −Q, H.P′ = −P′, H.Q′ = Q′, and H.Z = 0.

We put

𝔥 := ℝ(P − P′) ⊕ ℝ(Q + Q′) and 𝔮 := ℝZ ⊕ ℝ(P + P′) ⊕ ℝ(Q − Q′) ⊕ ℝH.

As in the preceding example, the prescriptions τ |_{𝔥} = id_{𝔥} and τ |_{𝔮} = − id_{𝔮} define an involutive automorphism of 𝔤. We obtain Δ = {±α}, where α(H) = 1, and the root spaces are given by

𝔤^{α} = span{P, Q′} and 𝔤^{−α} = span{Q, P′}.

From τ(P) = −P′ and τ(Q) = Q′, we get

[λP + μQ′, τ(λP + μQ′)] = [λP + μQ′, −λP′ + μQ)] = 2λμZ.

Thus the set of type (A)-elements in 𝔤^{α} is not a subspace.

(c) We give an example of a symmetric simple Lie algebra in which the root spaces are all of mixed type. So let us take 𝔰 = 𝔰𝔬(3, 3) and τ conjugation by diag(−1, 1, 1, 1, 1, 1). Then 𝔞 = ℝ(E_{1,6} + E_{6,1}) is a hyperbolic subspace in 𝔮 which is also maximal abelian in 𝔮. Further, the root system is given by Δ = {±α}, where α(E_{1,6} + E_{6,1}) = 1. For j = 2, 3, we set

X_{j} = E_{1,j} − E_{j,1} + E_{6,j} + E_{j,6},

while for j = 4, 5, we put

X_{j} = E_{1,j} + E_{j,1} + E_{6,j} − E_{j,6}.

Then it is easy to check that 𝔰^{α} = span{X_{2}, X_{3}, X_{4}, X_{5}}, and another simple computation shows that the form κ_{τ} on 𝔰^{α} has signature (1, 1, −1, −1). Hence in 𝔰^{α} all types (SR), (R) and (A) occur.

This example also shows that the subspace 𝔞 might be contained in several maximal hyperbolic Lie triple systems. In fact, we have 𝔥^{0} ≅ 𝔰𝔬(2, 2) and [𝔥^{0}, (𝔰_{𝔭})_{𝔮}] ⊈ (𝔰_{𝔭})_{𝔮}, where the Cartan involution is given by θ(X) = −X^{⊤}. Therefore all the conjugates of (𝔰_{𝔭})_{𝔮} under the group Inn_{𝔰}(𝔥^{0}) are maximal hyperbolic Lie triple systems containing 𝔞 (cf. Proposition III.7). To make this more explicit, let E ⊆ 𝔰^{α} be a two-dimensional plane on which the scalar product κ_{τ} is negative definite. Then 𝔞 + {τ(X) − X: X ∈ E} is a maximal hyperbolic Lie triple system in 𝔮 containing 𝔞 (cf. Proposition IV.8). Since there exists a continuous family of such subspaces E, this (𝔰_{𝔭})_{𝔮} is not unique.

Example IV.11. Let 𝔰 = 𝔰𝔩(2, ℝ) with the basis

H = ( \begin{matrix}1 & 0 \\ 0 & −1 \end{matrix} ) , U = ( \begin{matrix}0 & 1 \\ −1 & 0 \end{matrix} ) , T = ( \begin{matrix}0 & 1 \\ 1 & 0 \end{matrix} )

with the commutator relations

[H, U] = 2T, [H, T] = 2U, and [U, T] = 2H.

We consider the involution on 𝔰 of type (SR) which corresponds to the decomposition

𝔰_{𝔥} = ℝT, and 𝔰_{𝔮} = ℝH + ℝU.

Let 𝔯 be the three dimensional 𝔰-module with the basis (H′, U′, T′) corresponding to the identification with 𝔰 endowed with the adjoint representation. We form the Lie algebra 𝔤 := 𝔯 ⋊ 𝔰 and put

𝔯_{𝔥} := span{H′, U′}, 𝔯_{𝔮} := ℝT′, 𝔥 := 𝔯_{𝔥} + 𝔰_{𝔥} and 𝔮 := 𝔯_{𝔮} + 𝔰_{𝔮}.

Then [𝔰_{𝔥}, 𝔯_{𝔥}] ⊆ 𝔯_{𝔥}, [𝔰_{𝔥}, 𝔯_{𝔮}] ⊆ 𝔯_{𝔮}, [𝔰_{𝔮}, 𝔯_{𝔮}] ⊆ 𝔯_{𝔥}, and [𝔰_{𝔮}, 𝔯_{𝔥}] ⊆ 𝔯_{𝔮} imply that τ |_{𝔥} = id_{𝔥} and τ |_{𝔮} := − id_{𝔮} defines an automorphism of 𝔤.

The subspace 𝔞 := ℝH ⊆ 𝔮 is maximal abelian, hyperbolic, and 𝔷_{𝔤}(𝔞) = 𝔞 + ℝH′. The corresponding roots are given by Δ = Δ_{s} = {±α}, α(H) = 2, and

𝔤^{α} = span{U + T, U′ + T′}, 𝔯^{α} = ℝ(U′ + T′), and 𝔰^{α} = ℝ(U + T).

Since the form κ_{τ} is positive definite on 𝔰^{α} and degenerate on 𝔯^{α} (cf. Proposition IV.7(vi)), the elements in 𝔯^{α} are of type (A) and the other elements are of type (SR).

The space 𝔮 = span{H, U, T′} is three dimensional, and to see the action of Inn_{𝔤}(𝔥) on this space, we note the the matrices of the elements of 𝔥 with respect to the basis (T′, H, U) are given by

ad_{𝔮} T = ( \begin{matrix}0 & 0 & 0 \\ 0 & 0 & −2 \\ 0 & −2 & 0 \end{matrix} ) , ad_{𝔮} H′ = ( \begin{matrix}0 & 0 & 2 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{matrix} ) ,

and

ad_{𝔮} U′ = ( \begin{matrix}0 & −2 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{matrix} ) .

From that for

W_{max} := ℝT′ + ℝ^{+}(H + U) + ℝ^{+}(H − U)

the set 𝔮_{hyp} = Inn_{𝔤}(𝔥).𝔞 is given by

{0} ∪ W_{max}^{0} ∪ −W_{max}^{0}.

Note that W_{max} ∩ 𝔞 = ℝ^{+}H. The orbits in W_{max}^{0} are surfaces of the type ℝT′ + e^{ad 𝔰_{𝔥}}λH, λ > 0. It follows in particular that their convex hulls always contain affine lines.

For the coadjoint action on 𝔮^{∗} we obtain the matrices

ad_{𝔮}^{∗} T = ( \begin{matrix}0 & 0 & 0 \\ 0 & 0 & −2 \\ 0 & −2 & 0 \end{matrix} ) , ad_{𝔮}^{∗} H′ = ( \begin{matrix}0 & 0 & 0 \\ 0 & 0 & 0 \\ 2 & 0 & 0 \end{matrix} ) ,

and

ad_{𝔮}^{∗} U′ = ( \begin{matrix}0 & 0 & 0 \\ −2 & 0 & 0 \\ 0 & 0 & 0 \end{matrix} ) .

Therefore the orbit of each functional not vanishing on T′ is a two-dimensional plane parallel to (T′)^{⊥}. In the plane (T′)^{⊥} the orbits meeting 𝔞^{∗} ≅ {T′, U}^{⊥} are hyperbolas. Therefore the only orbits in W_{max}^{⋆} which are closed and have a pointed convex hull are those through 𝔞^{∗}.

V. The Weyl group and quasihermitian symmetric Lie algebras

In this section (𝔤, τ) denotes a symmetric Lie algebra and 𝔞 ⊆ 𝔮 is a maximal hyperbolic abelian subspace which is maximal abelian in 𝔮.

Definition V.1. Let 𝔭 ⊇ 𝔞 be a maximal hyperbolic Lie triple system.

(a) We say that a root α ∈ Δ is compact if 𝔤^{α} ∩ 𝔭_{L} ≠ Ø and non-compact otherwise. We write Δ_{k} = Δ(𝔭_{L}, 𝔞), resp. Δ_{n}, for the set of compact, resp. non-compact, roots. Note that, in view of Proposition III.7, the set Δ_{k} does not depend on the choice of 𝔭 because all maximal hyperbolic Lie triple systems containing 𝔞 are conjugate under the group Inn_{𝔤}(𝔥_{0}) which preserves the root spaces.

If 𝔭 is related to a τ -𝔞-invariant Levi complement 𝔰 as in Proposition III.5(iv), then it is clear that

Δ_{k} ⊆ Δ_{s} and Δ_{r} ⊆ Δ_{n}.

The roots in Δ_{p} := Δ_{n} ∩ Δ_{s} are called the non-compact semisimple roots.

(b) The root system Δ is called split if all root vectors X ∈ 𝔤^{α}, α ∈ Δ_{k} are of Riemannian type (R).

(c) A positive system Δ^{+} is called 𝔭-adapted if Δ_{n}^{+} is invariant under the Weyl group.

(d) If C is a subset of the finite dimensional real vector space V , then we write cone(C) for the smallest closed convex cone containing C. For a positive system Δ_{n}^{+} of non-compact roots, we consider the following cones:

C_{min,p} := C_{min,p}(Δ_{n}^{+}) := cone({[X, τ(X)]: X ∈ 𝔤^{α}, α ∈ Δ_{p}^{+}}) ⊆ 𝔞,

C_{min,z} := C_{min,z}(Δ_{n}^{+}) := cone({[X, τ(X)]: X ∈ 𝔯^{α}, α ∈ Δ^{+}}) ⊆ 𝔞 ∩ 𝔫 = 𝔷(𝔤)_{𝔮},

C_{min} = C_{min}(Δ_{n}^{+}) = \overline{C_{min,p} + C_{min,z}} , and

C_{max} := C_{max}(Δ_{n}^{+}) = {X ∈ 𝔞: (∀α ∈ Δ_{n})α(X) ≥ 0}.

(e) For every α ∈ Δ_{s} we set α̌: = \frac{2A_{α}}{⟨α,α⟩} and accordingly Δ̌_{s}: = \{α̌: α ∈ Δ_{s}\}. Note that α(α̌) = 2 for all α ∈ Δ_{s} and, more generally, that

β(α̌) = \frac{2β(A_{α})}{⟨α, α⟩} = \frac{2ϕ(A_{β})(A_{α})}{⟨α, α⟩} = \frac{2κ(A_{β}, A_{α})}{⟨α, α⟩} = \frac{2⟨α, β⟩}{⟨α, α⟩}

for all β ∈ span Δ = ϕ(𝔞).

(f) We say that (𝔤, τ) has cone potential, if no non-zero Z ∈ 𝔤^{α}, α ∈ Δ is of type (A).

(g) We say that (𝔤, τ) has strict cone potential, if it has cone potential and for each β ∈ Δ there is an element α ∈ 𝔞_{𝔷}^{∗} such that

(5.1)  α([X, τ(X)]) > 0  for all 0 ≠ X ∈ 𝔯^{β}.

If, in addition, there exists a 𝔭-adapted positive system Δ^{+} such that (5.1) holds simultaneously for all β ∈ Δ^{+} , then we say that (𝔤, τ) has strong cone potential.

Proposition V.2. Let (𝔤, τ) be a symmetric Lie algebra, 𝔭 ⊆ 𝔮 a maximal hyperbolic Lie triple system and 𝔞 ⊆ 𝔭 maximal abelian. Then the following assertions hold:

(i) The Weyl group W is isomorphic to the finite group generated by the reflections

s_{β}: 𝔞 → 𝔞, X ↦ X − β(X)β̌

for β ∈ Δ_{k}.

(ii) The center of 𝔭 is given by the fixed point set of the Weyl group W, i.e.

𝔷(𝔭) = 𝔞^{W} = {X ∈ 𝔞: (∀α ∈ Δ_{k}) α(X) = 0}.

It follows in particular that this space is independent of the choice of 𝔭.

(iii) If W_{s} ⊆ Gl(𝔞) is the group generated by the reflections s_{α}, α ∈ Δ_{s}, then W_{s} is finite and W_{s}.Δ = Δ, where the action of W_{s} on 𝔞^{∗} is given by s_{β}(α) = α − α(β̌)β.

Proof. (i) In view of Definition III.9, this is a statement on Riemannian symmetric Lie algebras and therefore follows from [3, Cor. VII.2.1]. (ii) Since 𝔷(𝔭) ⊆ 𝔞 holds for any maximal abelian subspace 𝔞 in 𝔭, the assertion follows immediate from the definition of the compact roots and (i). (iii) Let 𝔞_{𝔯} = 𝔞 ∩ 𝔯, and 𝔰 a τ -𝔞_{𝔯} -invariant Levi complement. Then we may w.l.o.g. assume that 𝔞 = 𝔞_{𝔯} ⊕ 𝔞_{𝔰}, where 𝔞_{𝔰} = 𝔞 ∩ 𝔰_{𝔮} (cf. Proposition III.5(i),(ii)). Now 𝔞_{𝔯} = Δ_{s}^{⊥} (Lemma IV.5(ii)) shows that 𝔞_{𝔯} = 𝔞^{W_{s}} is the fixed point set of W_{s}.

Let α ∈ Δ, β ∈ Δ_{s}, and 0 ≠ Z_{β} ∈ 𝔤^{β} be an element of type (SR) or (R) (Proposition IV.7(vi)) and note that this implies that 𝔤(Z) ≅ 𝔰𝔩(2, ℝ). We consider the 𝔤(Z)-module

V := ∑_{n∈ℤ} 𝔤^{α+nβ}.

Then (α+nβ)(β̌) = α(β̌)+2n shows that the spaces 𝔤^{α+nβ} are the β̌-eigenspaces in V . Therefore the representations theory of 𝔰𝔩(2, ℝ) shows that whenever α − pβ and α + qβ are roots, the same holds for all n ∈ {−p, …, q}. Moreover, if p and q are maximal, then

α(β̌) − 2p = (α − pβ)(β̌) = −(α + qβ)(β̌) = −α(β̌) − 2q

([1, Ch. 8, §1, no. 3]). This proves that α(β̌) = p − q, and hence that

s_{β}.α = α − α(β̌)β = α + (q − p)β ∈ Δ.

Thus Δ is invariant under s_{β}, and therefore under W_{s}.

To see that W_{s} is finite, we note that Δ_{s}^{⊥} = 𝔞_{𝔯} implies that 𝔞_{𝔰}^{∗} = span Δ_{s}. Now the fact that W_{s} acts effectively on 𝔞_{𝔰} and the invariance of the finite set Δ_{s} imply that W_{s} is finite.

Corollary V.3. Let α ∈ Δ and β ∈ Δ_{s}. Then the β -string containing α has the form α + nβ, −p ≤ n ≤ q, where p, q ≥ 0. Furthermore p − q = α(β̌). Moreover, we have

(i) [𝔤^{β}, 𝔤^{α}] = 𝔤^{α+β} whenever α(β̌) ≥ −1.

(ii) [𝔤^{β}, 𝔤^{α}] ≠ {0} if and only if α + β ∈ Δ.
(iii) [𝔤^{β}, 𝔤^{α}] = 𝔤^{α+β} whenever α, β ∈ Δ_{s} and α ≠ −β.

Proof. The first part has been shown in the proof of Proposition V.2(iii). (i), (ii) This follows from the representation theory of 𝔰𝔩(2, ℝ) which shows that [Z, 𝔤^{α}] = 𝔤^{α+β} whenever α(β̌) ≥ −1 and [Z, 𝔤^{α}] ≠ {0} whenever α(β̌) ≤ −1 (cf. [1, Ch. 8, §1, no. 3]). (iii) (cf. [20]) Let α, β ∈ Δ_{s} with [𝔤^{α}, 𝔤^{β}] ≠ 𝔤^{α+β}. Then (i) implies that

\frac{2⟨α, β⟩}{⟨β, β⟩} = α(β̌) ≤ −2 and \frac{2⟨α, β⟩}{⟨α, α⟩} = β(α̌) ≤ −2.

Multiplication of both inequalities yields ⟨α, β⟩^{2} ≥ ⟨α, α⟩⟨β, β⟩, so that equality follows from the Cauchy-Schwarz inequality. We conclude that β = λα and obtain β(α̌) = 2λ ≤ −2 as well as α(β̌) = \frac{2}{λ} ≤ −2, hence that λ = −1, i.e., α = −β.

Proposition V.4. Let (𝔤, τ) be a symmetric Lie algebra, 𝔭 ⊆ 𝔮 a maximal hyperbolic Lie triple system and 𝔞 ⊆ 𝔭 maximal abelian. Then for a positive system Δ^{+} the following are equivalent:

(1) The system Δ^{+} is 𝔭-adapted.

(2) The cone C_{max} is W -invariant.

(3) C_{max}^{0} ∩ 𝔷(𝔭) ≠ Ø.

(4) (Δ_{n}^{+} + Δ_{k}) ∩ Δ ⊆ Δ_{n}^{+}.

(5) If

𝔪 = ⨁_{α∈Δ_{k}∪{0}} 𝔤^{α} and 𝔭^{+} = ⨁_{α∈Δ_{n}^{+}} 𝔤^{α},

then [𝔪, 𝔭^{+}] ⊆ 𝔭^{+} .

Proof. (1) ⇒ (2): It is clear that the W -invariance of Δ_{n}^{+} implies the W - invariance of C_{max} .

(2) ⇒ (3): If C_{max} is invariant under W and X ∈ C_{max}^{0} , then X_{0} := ∑_{γ∈W} γ.X is still contained in C_{max}^{0} and, in addition, W -invariant. Therefore each compact root vanishes on X_{0} and thus X ∈ 𝔷(𝔭) (Proposition V.2(ii)).

(3) ⇒ (4): Let X_{0} ∈ C_{max}^{0} ∩ 𝔷(𝔭). Then Δ_{n}^{+} = {α ∈ Δ: α(X) > 0} because all compact roots vanish on X. Hence α ∈ Δ_{n}^{+} , β ∈ Δ_{k} implies that (α + β)(X) = α(X) > 0, and therefore that α + β ∈ Δ_{n}^{+} whenever it is a root.

(4) ⇒ (5): This is an immediate consequence of [𝔤^{α}, 𝔤^{β}] ⊆ 𝔤^{α+β} and (4).

(5) ⇒ (1): Let α ∈ Δ_{k} and Z ∈ 𝔤^{α} be of type (R). Then 𝔤(Z) ⊆ 𝔪, and (5) implies that [𝔤(Z), 𝔭^{+}] ⊆ 𝔭^{+} . Thus 𝔭^{+} is a module of the subalgebra 𝔤(Z) ≅ 𝔰𝔩(2, ℝ), and the invariance of Δ_{n}^{+} , the corresponding set of weights, under the reflection s_{α} follows from 𝔰𝔩(2, ℝ)-representation theory. We conclude that W.Δ_{n}^{+} = Δ_{n}^{+} .

Definition V.5. Let (𝔤, τ) a symmetric Lie algebra and 𝔭 ⊆ 𝔮 a maximal hyperbolic Lie triple system.

(i) We call (𝔤, τ) quasihermitian, if 𝔷_{𝔮}(𝔷(𝔭)) = 𝔭, where 𝔷(𝔭) = {X ∈ 𝔭: [X, 𝔭] = {0}}. (ii) If (𝔤, τ) is semisimple and irreducible, then we call (𝔤, τ) (NCR) non-compactly Riemannian, if 𝔤 is non-compact and τ is a Cartan

involution. (NCC) non-compactly causal, if (𝔤, τ) is quasihermitian and 𝔷(𝔭) ≠ {0}. (CC) compactly causal, if (𝔤^{c}, τ) is (NCC). (CT) of Cayley type, if it is both (CC) and (NCC).

Note that (𝔤, τ) is quasihermitian if and only if it is (NCR) or (NCC). The three dimensional simple Lie algebra (𝔰, τ) of type (R) is (NCR) and if it is of type (SR), then it is (CT) (cf. Theorem IV.1).

Let (𝔤, τ) be a semisimple symmetric Lie algebra and θ a Cartan involution commuting with τ. Denote by 𝔤 = 𝔤_{𝔨} ⊕ 𝔤_{𝔭} the corresponding Cartan decomposition. The prescription τ^{a} = θ ∘ τ defines an involution and we call (𝔤, τ^{a}) an associated symmetric Lie algebra. Note that the eigenspace decomposition of 𝔤 according to τ^{a} is given by

𝔤 = 𝔥^{a} ⊕ 𝔮^{a}, where 𝔥^{a} = (𝔤_{𝔨})_{𝔥} ⊕ (𝔤_{𝔭})_{𝔮}, 𝔮^{a} = (𝔤_{𝔨})_{𝔮} ⊕ (𝔤_{𝔭})_{𝔥}.

Proposition V.6. If (𝔤, τ) is irreducible semisimple such that 𝔷(𝔭) ≠ {0}, then the following assertions hold:

(i) The symmetric Lie algebra (𝔤, τ) is one of the following two types:

(1) The c-dual 𝔤^{c} is simple and hermitian with 𝔷(𝔨^{c}) ⊆ 𝔮^{c}.

(2) The subalgebra 𝔥 is simple hermitian and (𝔤, τ) ≅ (𝔥_{ℂ}, σ), where σ denotes complex conjugation. It follows in particular that 𝔤 is simple. (ii) There exists an up to sign unique element H ∈ 𝔷(𝔭) with 𝔷(𝔭) = ℝH,

Spec(ad H) = {2, 0, −2} and 𝔷_{𝔮}(H) = 𝔭. It follows in particular that (𝔤, τ) is quasihermitian, i.e., (NCC) and that H defines a triangular decomposition of 𝔤:

𝔤 = 𝔤(ad H; −2) ⊕ 𝔤(ad H; 0) ⊕ 𝔤(ad H; 2).
(iii) The involution τ^{a} is given by τ^{a} = e^{i \frac{π}{2} ad H}.

Proof. (cf. [8, Lemma 1.3.5, Th. 1.3.8]) (i), (ii) First we choose a Cartan decomposition 𝔤 = 𝔤_{𝔨} ⊕ 𝔤_{𝔭} such that 𝔭 = (𝔤_{𝔭})_{𝔮} (Proposition III.5(iv)). Then the fact that 𝔭_{L} ⊆ 𝔥^{a} is an ideal (Lemma II.7(i)) implies that 𝔷(𝔭) ⊆ 𝔷(𝔥^{a}). On the other hand 𝔨^{c} := (𝔤_{𝔨})_{𝔥} ⊕ i(𝔤_{𝔭})_{𝔮} ⊆ 𝔤^{c} is a maximal compactly embedded subalgebra.

As already observed in Definition I.6(d), there are two possibilities:

(1) 𝔤^{c} is simple. Then the fact that 𝔷(𝔨^{c}) ≠ {0} implies that 𝔤^{c} is a hermitian Lie algebra with 𝔷(𝔨^{c}) ⊆ i𝔮.

(2) 𝔤^{c} ≅ 𝔥 ⊕ 𝔥 and 𝔥 is simple. Then 𝔨^{c} ≅ (𝔤_{𝔨})_{𝔥} ⊕ (𝔤_{𝔨})_{𝔥}, so that the same argument as above shows that 𝔥 is simple hermitian. Moreover 𝔤 ≅ (𝔥_{ℂ}, σ) (Lemma I.10), so that 𝔭 = i(𝔤_{𝔨})_{𝔥}.

If 𝔤 is not simple, then 𝔤 ≅ 𝔥 ⊕ 𝔥 and 𝔤^{c} ≅ 𝔥_{ℂ} (cf. Lemma I.11). Therefore the fact that a simple hermitian Lie algebra is never complex implies that 𝔤 is simple.

In both cases we see that 𝔷(𝔭) is one-dimensional. Since the spectrum of an element in the center of a maximal compactly embedded subalgebra of a hermitian simple Lie algebra consists of {0, ci, −ci} (cf. [3, Prop. VIII.6.2]), we can find H ∈ 𝔷(𝔭) with Spec(ad H) = {−2, 0, 2, }. Moreover, the fact that 𝔷_{𝔤^{c}}(𝔷(𝔨^{c})) = 𝔨^{c} implies that 𝔷_{𝔮}(H) = 𝔭. This completes the proof. (iii) This is an immediate consequence of (ii).

The preceding result can be sharpened significantly if we assume, in addition, that the Lie algebra under consideration is of Cayley type.

Lemma V.7. For a symmetric (CT) Lie algebra (𝔤, τ) the following assertions hold:

(i) There are elements T ∈ 𝔷(𝔥) ∩ 𝔤_{𝔭} and U ∈ 𝔷(𝔤_{𝔨})_{𝔮} such that

[H, U] = 2T,  [H, T] = 2U,  [U, T] = 2H.

(ii) The element T defines a triangular decomposition

𝔤 = 𝔤(ad T; −2) ⊕ 𝔤(ad T; 0) ⊕ 𝔤(ad T; 2),

and τ = e^{i \frac{π}{2} ad T} .

(iii) The Cartan involution is given by θ = e^{\frac{π}{2} ad U}.

Proof. (i) - (ii) [8, Th. 1.3.11]. (iii) [8, Lemma 1.2.1].

Lemma V.8. The symmetric Lie algebra (𝔤, τ) is quasihermitian if and only if there exists an element X ∈ 𝔷(𝔭) such that 𝔷_{𝔮}(X) = 𝔭.

Proof. “⇐”: This is trivial. “⇒”: Let X_{0} ∈ 𝔷(𝔭) such that no root in Δ(𝔤, 𝔷(𝔭)) vanishes on X_{0} . Then 𝔷_{𝔤}(X_{0}) = 𝔷_{𝔤}(𝔷(𝔭)) and the implication follows from the τ -invariance of both sides.

Proposition V.9. Let (𝔤, τ) be a quasihermitian symmetric Lie algebra, 𝔭 a maximal hyperbolic Lie triple system, and 𝔞 maximal abelian in 𝔭. Then the following assertions hold:

(i) The maximal hyperbolic abelian subspace 𝔞 is maximal abelian in 𝔮. (ii) The root system Δ(𝔤, 𝔞) is split. (iii) The maximal hyperbolic abelian subspace 𝔞 is contained in a unique

maximal hyperbolic Lie triple system 𝔭 which is given by

𝔭 = 𝔞 ⊕ ⨁_{α∈Δ_{k}}(1 − τ)(𝔤^{α}).

(iv) int 𝔮_{hyp} ∩ 𝔷(𝔭) = 𝔷(𝔭) \ ⋃_{α∈Δ_{n}} ker α.

(v) If 𝔰 := 𝔤/𝔯 is endowed with the inherited involution τ_{𝔰}, then (𝔰, τ_{𝔰}) is quasihermitian. This means that the irreducible constituents of (𝔰, τ) are either (NCR) or (NCC).

(vi) There exists a 𝔭-adapted positive system.

(vii) If Δ^{+} is a 𝔭-adapted positive system, then (Δ_{p}^{+} + Δ_{p}^{+}) ∩ Δ ⊆ Δ_{r}^{+} and C_{min,p} ⊆ (Δ_{p}^{+})^{⋆}.

(viii) The Lie algebra 𝔰 has cone potential.

Proof. (i) Since 𝔞 is maximal abelian in 𝔭 and (𝔤, τ) is quasihermitian, any abelian subspace of 𝔮 containing 𝔞 must be contained in 𝔷_{𝔮}(𝔷(𝔭)) = 𝔭, hence has to coincide with 𝔞. This proves that 𝔞 is maximal abelian in 𝔮. (ii) First we note that each root α ∈ Δ_{k} vanishes on 𝔷(𝔭) ⊆ 𝔞. Therefore 𝔤^{α} ⊆ 𝔷_{𝔤}(𝔷(𝔭)) and hence for each X ∈ 𝔤^{α} the element X − τ(X) is contained in 𝔷_{𝔮}(𝔷(𝔭)) = 𝔭. Now the 𝔞-invariance of 𝔭_{L} shows that the 𝔤^{α} -component X of this element is contained in 𝔭_{L}. Thus 𝔤^{α} = (𝔭_{L})^{α} and this is what we had to show. (iii) The proof of (ii) implies that for α ∈ Δ_{𝔨} we have 𝔤^{α} = (𝔭_{L})^{α}, hence that

𝔭 = 𝔞 ⊕ ⨁_{α∈Δ_{k}}(1 − τ)(𝔤^{α})

which shows that 𝔭 is unique. (iv) In view of (iii), an element X ∈ 𝔷(𝔭) is contained in int 𝔮_{hyp} if and only if 𝔷_{𝔮}(X) is not bigger than 𝔭 (cf. Proposition III.2), i.e., if and only if no noncompact root vanishes on X. This proves (iv).

(v) In view of Proposition III.5(iv), we may assume that 𝔰 is realized in 𝔤 as a τ -𝔞-invariant Levi complement and that 𝔭 has been constructed as 𝔭 = 𝔞_{𝔯} ⊕ 𝔭_{𝔰}, where 𝔭_{𝔰} = (𝔰_{𝔭})_{𝔮} with a Cartan decomposition 𝔰 = 𝔰_{𝔨} ⊕ 𝔰_{𝔭} which is invariant under τ_{𝔰}. Then the fact that 𝔞_{𝔯} commutes with 𝔰 (Proposition III.5(iii)) shows that 𝔷(𝔭) = 𝔞_{𝔯} ⊕ 𝔷(𝔭_{s}). Hence

𝔷_{𝔰_{𝔮}}(𝔷(𝔭_{𝔰})) = 𝔷_{𝔰_{𝔮}}(𝔷(𝔭)) ⊆ 𝔭 ∩ 𝔰_{𝔮} = 𝔭_{𝔰}

and so (𝔰, τ_{𝔰}) is quasihermitian.

If we decompose (𝔰, τ) into a direct sum of irreducible symmetric Lie algebras (𝔰_{j}, τ_{j}), it is clear that 𝔭 is adapted to this decomposition. Therefore all irreducible factors are quasihermitian, i.e. (NCR) or (NCC), if and only if (𝔰, τ) is quasihermitian.

(vi) Let X ∈ 𝔷(𝔭) with 𝔭 = 𝔷_{𝔮}(X) (Lemma V.8). Then (ii) shows that α(X) ≠ {0} holds for all α ∈ Δ_{n}^{+} . We choose X_{1} ∈ 𝔞 such that sgn α(X_{1}) = sgn α(X) for all α ∈ Δ_{n} and α(X_{1}) ≠ 0 for all α ∈ Δ_{k}. Then Δ^{+} = {α ∈ Δ: α(X_{1}) > 0} is a positive system, and Δ_{n}^{+} = {α ∈ Δ: α(X) > 0} is W -invariant because X is fixed by W. This proves that Δ^{+} is 𝔭-adapted. (vii) In view of (v), it suffices to prove (Δ_{p}^{+} + Δ_{p}^{+}) ∩ Δ = Ø for (NCC) algebras because this holds trivially for (NCR) algebras. Let H be as in Proposition V.6(ii) and Δ^{+} be a 𝔨-adapted positive system. Then Proposition V.4(3) shows that we may w.l.o.g. assume that H ∈ C_{max}^{0} . Then Δ_{p}^{+} = {α ∈ Δ: α(H) = 2}, and the assertion follows from the fact that 4 is no eigenvalue of ad H.

To see that C_{min,p} ⊆ (Δ_{p}^{+})^{⋆}, let α, β ∈ Δ_{p}^{+} . For Z ∈ 𝔤^{β} we have [Z, τ.Z] ∈ κ(Z, τ.Z)A_{β} + 𝔞_{𝔷} and hence α([Z, τ.Z]) = κ(Z, τ.Z)α(A_{β}). Since κ(Z, τ.Z) ≥ 0 by (ii) and Proposition IV.7(vi), we have to show that α(β̌) ≥ 0. Since α + β is no root in Δ(𝔰, 𝔞_{𝔰}), this follows from Corollary V.3. (viii) We have already seen in (v) that (𝔰, τ_{𝔰}) is quasihermitian, hence (ii) implies that Δ(𝔰, 𝔞_{𝔰}) is split. Now the assertion follows from the non-degeneracy of the forms κ_{τ} on the root spaces 𝔰^{α} (cf. Proposition IV.8).

Putting all these facts together, we arrive at the following characterization of the quasihermitian symmetric Lie algebras.

Proposition V.10. The symmetric Lie algebra (𝔤, τ) is quasihermitian if and only if there exists a maximal hyperbolic abelian subspace 𝔞 ⊆ 𝔮 such that

(1) 𝔞 is maximal abelian in 𝔮,

(2) Δ(𝔤, 𝔞) is split, and

(3) Δ(𝔤, 𝔞) contains a 𝔭-adapted positive system.

Proof. The necessity of (1)-(3) follows from Propositon V.9.

Assume that (1)-(3) are satisfied and choose a maximal hyperbolic Lie triple system 𝔭 containing 𝔞. Using Proposition V.4(3), we find X_{0} ∈ 𝔷(𝔭)∩C_{max}^{0} . Hence

𝔷_{𝔮}(X_{0}) = 𝔞 ⊕ ⨁_{α∈Δ_{k}}(1 − τ).𝔤^{α}

because 𝔞 is maximal abelian in 𝔮. To see that this implies that 𝔷_{𝔮}(X_{0}) = 𝔭, it now suffices to show that 𝔤^{α} ⊆ 𝔭_{L} holds for all α ∈ Δ_{k}.

For this we recall from Proposition III.5(iv) that we can realize 𝔭 as 𝔭 = 𝔞_{𝔯} ⊕ (𝔰_{𝔭} ∩ 𝔰_{𝔮}), where 𝔞_{𝔯} is maximal hyperbolic abelian in 𝔯_{𝔮} with respect to 𝔤, 𝔰 is an 𝔞_{𝔯} -τ -invariant Levi complement, and 𝔰 = 𝔰_{𝔨} + 𝔰_{𝔭} is a τ -invariant Cartan decomposition defined by the Cartan involution θ.

Let α ∈ Δ_{k} ⊆ Δ_{s} and recall that 𝔰^{α} is invariant under the involution σ := τθ of 𝔰, and that its −1-eigenspace consists of elements of type (SR) (Proposition IV.8). Now the fact that Δ is split shows that σ |_{𝔰^{α}} = id, hence that

𝔰^{α} ⊆ (1 − τ).𝔰^{α} + [𝔞, (1 − τ).𝔰^{α}] ⊆ 𝔭 + [𝔭, 𝔭] ⊆ 𝔭_{L}.

Moreover, in view of Lemma IV.5 and Proposition IV.8(iii), (2) implies that 𝔤^{α} = 𝔰^{α} and finally that 𝔤^{α} ⊆ 𝔭_{L}.

Example V.11. (a) Let 𝔰 = 𝔰𝔩(2, ℝ) with the basis from Example IV.11. We consider the Cartan involution on 𝔰 which corresponds to the decomposition

𝔰_{𝔥} = ℝU, and 𝔰_{𝔮} = ℝH + ℝT.

Let 𝔯 be the three dimensional 𝔰-module with the basis (H′, U′, T′) corresponding to the identification with 𝔰 endowed with the adjoint representation. We form the Lie algebra 𝔤 := 𝔯 ⋊ 𝔰 and put

𝔯_{𝔥} := span{H′, T′}, 𝔯_{𝔮} := ℝU′, 𝔥 := 𝔯_{𝔥} + 𝔰_{𝔥} and 𝔮 := 𝔯_{𝔮} + 𝔰_{𝔮}.

Then, as in Example IV.11, these prescriptions define an involutive automorphism τ of 𝔤.

The subspace 𝔭 := ℝH + ℝT ⊆ 𝔮 is a maximal hyperbolic Lie triple system, 𝔞 := ℝH ⊆ 𝔭 is maximal abelian, and 𝔷(𝔭) = {0}. Therefore 𝔷_{𝔮}(𝔷(𝔭)) = 𝔮 ≠ 𝔭 and we see that (𝔤, τ) is not quasihermitian. Nevertheless, the subspace 𝔞 is maximal abelian in 𝔮, W ≅ ℤ^{2} , Δ = Δ_{k}, and Δ_{n} = Ø, so that each positive system is 𝔭-adapted. Of course Δ is not split because the root spaces 𝔤^{α} = 𝔯^{α} ⊕ 𝔰^{α} are 2-dimensional, the form κ_{τ} is negative semidefinite with onedimensional degeneracy in 𝔯^{α}.

(b) Let 𝔰 be a compact Lie algebra and V a real 𝔰-module. Let further 𝔫 = V × V × ℝ denote the Heisenberg algebra with bracket

[(v, w, t), (v′, w′, t′)] := (0, 0, ⟨v, w′⟩ − ⟨v′, w⟩).

Let further H denote the operator on 𝔫 given by H.(v, w, t) = (v, −w, 0) and consider the Lie algebra 𝔤 := 𝔫 ⋊ (𝔰 ⊕ ℝH). Then

𝔥 = {(v, v, 0): v ∈ V } ⋊ 𝔰 and 𝔮 = {(v, −v, t): v ∈ V, t ∈ ℝ} ⋊ ℝH

defines an involution τ on 𝔤. The subspace 𝔞 := 𝔷 ⊕ ℝH with 𝔷 = {0} × {0} × ℝ is maximal hyperbolic and maximal abelian in 𝔮. Moreover, 𝔭 = 𝔞 is a maximal hyperbolic Lie triple system, and Δ = {±α} with α(H) = 1, where 𝔤^{α} = V × {0}. Therefore (𝔤, τ) is quasihermitian, effective with strong cone potential. Nevertheless 𝔥^{0} = 𝔰 is non-trivial.

(c) Let 𝔤_{0} be a split semisimple real Lie algebra and 𝔞_{0} ⊆ 𝔤_{0} a toral Cartan subalgebra. We consider 𝔤 = 𝔤_{0} ⊕ 𝔤_{0} with τ(X, Y) = (Y, X). Then 𝔮 = {(X, −X): X ∈ 𝔤_{0}} and if 𝔤_{0} = 𝔨_{0} + 𝔭_{0} is a Cartan decomposition, then 𝔭 := {(X, −X): X ∈ 𝔭_{0}} is a maximal hyperbolic Lie triple system in 𝔮. According to Example IV.3(a), all root spaces are of mixed type, hence Δ = Δ_{k}. We conclude in particular that all positive systems are 𝔭-adapted, but that Δ is not split.

VI. Convexity properties and invariant convex sets

In this section we come to the subject proper of this paper, the convexity properties of the action of Inn_{𝔤}(𝔥) on the space 𝔮. The key role will be played by those invariant convex subsets having a sufficiently large intersection with 𝔞. The main point of this section is that the existence of such invariant convex sets has significant consequences for the structure of the symmetric Lie algebra (𝔤, τ). In particular we will see that it implies that (𝔤, τ) is quasihermitian, and also that it has strong cone potential (cf. Definition V.1(g)).

Throughout this section we assume that 𝔞 ⊆ 𝔮 is a maximal hyperbolic abelian subspace which is in addition maximal abelian in 𝔮.

A key tool in everything that follows is the following lemma which gives precise information on the projections of orbits of elements in 𝔞 with respect to the action of one-parameter subgroups of Inn_{𝔤}(𝔥) coming from root vectors.

Lemma VI.1. For α ∈ Δ, X ∈ 𝔞, and Z ∈ 𝔤^{α} we have the following formulas:

ad (Z + τ(Z))^{m}(X) (i)

= \{ \begin{matrix}X & for m = 0 \\ −α(X)α(2[Z, τ(Z)])^{n}(Z − τ(Z)) & for m = 2n + 1 \\ −α(X)2α(2[Z, τ(Z)])^{n}[τ(Z), Z] & for m = 2(n + 1). \end{matrix}

(ii) If p: 𝔮 → 𝔞 is the projection along 𝔮 ∩ [𝔞, 𝔤], then

p(e^{ad(Z+τ(Z))}.X)

= cosh ( ad(Z + τ(Z))).X

=X − \{ \begin{matrix}α(X)[τ(Z), Z] & for α([Z, τ(Z)]) = 0 \\ α(X)^{\frac{cosh(\sqrt{2α([Z,τ(Z)])})−1}{α([Z,τ(Z)])}}[τ(Z), Z] & for α([Z, τ(Z)]) > 0 \\ α(X)^{\frac{cos(\sqrt{2α([τ(Z),Z])})−1}{α([Z,τ(Z)])}}[τ(Z), Z] & for α([Z, τ(Z)]) < 0 \end{matrix}

and

p(e^{ℝ ad(Z+τ(Z))}.X)
=p(e^{ℝ^{+} ad(Z+τ(Z))}.X)
=X + \{ \begin{matrix}ℝ^{+}α(X)[Z, τ(Z)] & for α([Z, τ(Z)]) = 0 \\ ℝ^{+}α(X)[Z, τ(Z)] & for α([Z, τ(Z)]) > 0 \\ [0, 2]\frac{α(X)}{α([τ(Z),Z])}[Z, τ(Z)] & for α([Z, τ(Z)]) < 0. \end{matrix}

Proof. (i) We prove the assertion by induction with respect to m. For m = 0 there is nothing to prove. Suppose that the assertion is true for m = 2n. If n = 0, then

ad(Z + τ(Z))^{m+1}(X) = [Z + τ(Z), X] = −α(X)(Z − τ(Z)).

If m = 2n > 0, then

ad(Z + τ(Z))^{m+1} = ad(Z + τ(Z))( − α(X)2α(2[Z, τ(Z)])^{n−1}[τ(Z), Z])
= −α(X)2α(2[Z, τ(Z)])^{n−1}[Z + τ(Z), [τ(Z), Z]]
= −α(X)α(2[Z, τ(Z)])^{n}(Z − τ(Z)),

and if m = 2n + 1, then

\begin{aligned}ad(Z + τ(Z))^{m+1}(X) & = ad(Z + τ(Z))( − α(X)α(2[Z, τ(Z)])^{n}(Z − τ(Z))) \\ & = −α(X)α(2[Z, τ(Z)])^{n}2[τ(Z), Z]. \end{aligned}

(ii) Using (i), we see that ad(Z+τ(Z))^{2n+1}.X ∈ [𝔞, 𝔤]∩𝔮 and ad(Z+τ(Z))^{2n}.X ∈ 𝔞. Thus

p(e^{ad(Z+τ(Z))}.X) = cosh ( ad(Z + τ(Z))).X.

Now the first formula for cosh ( ad(Z + τ(Z))).X follows from (i). For the remaining assertions we distinguish several cases. The case α([Z, τ(Z)]) = 0 is trivial. If α([Z, τ(Z)]) > 0, then we use the surjectivity of the function ℝ^{+} → ℝ^{+} , t ↦ cosh(ts) − 1 for s > 0, and for α([Z, τ(Z)]) < 0 we use cos(ℝ) − 1 = [−2, 0].

Definition VI.2. Let V be a finite dimensional real vector space and V^{∗} its dual space.

(a) For a subset C ⊆ V we define

B(C): = {α ∈ V^{∗}: inf α(C) > −∞} and C^{⋆} := {α ∈ V^{∗}: inf α(C) ≥ 0}.

Note that both are convex cones and that C^{⋆} is always closed, whereas B(C) need not be closed. One obtains an instructive example by taking the graph of the exponential function in V = ℝ^{2} .

(b) For a convex subset C ⊆ V we put

lim C: = {v ∈ V: v + C ⊆ C} and H(C) := lim(C) ∩ − lim(C).

We call lim C the limit cone of C. Note that lim C is always a convex cone which is closed if C is closed or open (cf. [19, Prop. III.1.5]). If C is open or closed, then the geometric meaning of H(C) is that c + H(C), c ∈ C are the maximal affine subspaces contained in C.

Proposition VI.3. For Y ∈ 𝔞 the following assertions hold:

(i) If X ∈ 𝔤^{α} is of type (SR), (A) or (N), then

Y + ℝ^{+}α(Y)[X, τ(X)] ⊆ conv(O_{Y}).

(ii) If X ∈ 𝔤^{α} is of type (SR) and 𝔤^{α} also contains elements of type (R),

then

Y + ℝα(Y)α̌ ⊆ conv(O_{Y}).

(iii) C_{Y} := cone (\{α(Y)[X, τ(X)]: X of type (SR), (N)\}) ⊆ lim \overline{conv(O_{Y})}.

Proof. (i) In view of Lemma VI.1(ii), this follows from the observation that for Z ∈ 𝔥 we have

p(e^{ad Z}.Y) = cosh(ad Z).Y = \frac{1}{2}(e^{ad Z}.Y + e^{− ad Z}.Y) ∈ conv(O_{Y}).

(ii) First we note that if 𝔤^{α} contains an element of type (R), then the same holds for 𝔰^{α} (cf. Proposition IV.7(v)). Hence the reflection in ker α defined by

s_{α}(X) := X − α(X)α̌

is an element of the Weyl group W. Now (i) and Proposition IV.7(vi) imply that Y + ℝ^{+}α(Y)α̌ ⊆ conv(O_{Y}) and therefore the invariance of O_{Y} under W yields

s_{α}(Y + ℝ^{+}α(Y)α̌) = s_{α}(Y) − ℝ^{+}α(Y)α̌ ⊆ conv(O_{Y}).

If α(Y) ≠ 0, then the convex hull of these two half-lines coincides with the line Y + ℝα̌, and this proves (ii). (iii) This is immediate from (i).

Next we introduce some notions that will be used to describe the effect of the existence of certain invariant convex sets in 𝔮 on the structure of the Lie algebra 𝔤.

Definition VI.4. Let (𝔤, τ) be a symmetric Lie algebra.

(a) We call a convex Inn_{𝔤}(𝔥)-invariant set C ⊆ 𝔮 hyperbolic (resp. elliptic), if it has non-empty interior and int C consists of hyperbolic (resp. elliptic) elements.

(b) A symmetric Lie algebra (𝔤, τ) is called admissible if there exists a closed convex generating hyperbolic invariant subset C ⊆ 𝔮 with H(C) = {0}.

In the following D^{0} for a subset D of 𝔞 means the relative interior of D with respect to 𝔞.

Lemma VI.5. (i) If C ⊆ 𝔮 is an invariant hyperbolic convex subset, then C^{0} = Inn_{𝔤}(𝔥).(C ∩ 𝔞)^{0} . (ii) If C ⊆ 𝔮 is an invariant subset such that C ∩ 𝔞 has interior points, then C has interior points.

Proof. (i) Let C_{𝔞} := C ∩𝔞. Then C^{0} ∩𝔞 ⊆ C_{𝔞}^{0} , so that, in view of C^{0} ⊆ 𝔮_{hyp} , Theorem III.3(ii) implies that C^{0} = Inn_{𝔤}(𝔥).(C^{0} ∩ 𝔞) ⊆ Inn_{𝔤}(𝔥).C_{𝔞}^{0} . It remains to show that C_{𝔞}^{0} ⊆ C^{0} ∩ 𝔞.

So let X ∈ C_{𝔞}^{0} and U ⊆ C_{𝔞} be an open convex subset. If Y ∈ U satisfies 𝔷_{𝔮}(Y) = 𝔞, i.e., if α(Y) ≠ 0 holds for all α ∈ Δ, then Lemma III.1 shows that the mapping

Ψ: Inn_{𝔤}(𝔥) × 𝔞 → 𝔮, (h, Z) ↦ h.Z

has surjective differential in (1, Y), hence that Ψ(Inn_{𝔤}(𝔥)×U) is a neighborhood of Y in 𝔮. Thus Y ∈ C^{0} . Since

U = conv({Y ∈ U: (∀α ∈ Δ)α(Y) ≠ 0}),

it follows that X ∈ C^{0} . (ii) If U ⊆ C ∩ 𝔞 is an open subset of 𝔞, then the argument above shows that Inn_{𝔤}(𝔥).U ⊆ C contains an interior point.

The following theorem contains some key observations relating invariant hyperbolic sets to the structure of the Lie algebra.

Theorem VI.6. Let (𝔤, τ) be a symmetric Lie algebra, 𝔭 ⊆ 𝔮 a maximal hyperbolic Lie triple system, and 𝔞 ⊆ 𝔭 maximal abelian. Suppose that there exists an invariant hyperbolic closed convex subset C ⊆ 𝔮. Then the following assertions hold:

(i) The hyperbolic subspace 𝔞 is maximal abelian in 𝔮.

(ii) The symmetric Lie algebra (𝔤, τ) is quasihermitian and there exists a 𝔭-adapted positive system Δ^{+} such that

C_{min} ⊆ lim(C ∩ 𝔞) and C ∩ 𝔞 ⊆ C_{max}.

In particular we have C_{min} ⊆ C_{max}.

(iii) If H(C) = {0}, then (𝔤, τ) has strong cone potential.

(iv) If, in addition, C ≠ 𝔮, then 𝔷(𝔭) ≠ {0}. Proof. (i) The existence of C implies that 𝔮_{hyp} has non-empty interior. Hence (i) follows from Theorem III.3(ii). (ii) Let C_{𝔞} := C ∩ 𝔞 and X ∈ C^{0} ∩ 𝔞. Then X_{0} := ∑_{γ∈W} γ.X ∈ C_{𝔞}^{0} is fixed under W. According to Proposition V.2(ii), we have X_{0} ∈ 𝔷(𝔭). In view of Lemma VI.5, X_{0} ∈ C^{0} ⊆ int 𝔮_{hyp} , so that Proposition III.2 implies that 𝔷_{𝔮}(X_{0}) is a hyperbolic Lie triple system. Clearly we have 𝔭 ⊆ 𝔷_{𝔮}(X_{0}), so that the maximality of 𝔭 implies that 𝔭 = 𝔷_{𝔮}(X_{0}), hence that (𝔤, τ) is quasihermitian.

We define a positive system of non-compact roots by

Δ_{n}^{+}: = {α ∈ Δ_{n} : α(X_{0}) > 0}.

Since no non-compact root vanishes on an element of C_{𝔞}^{0} (Lemma VI.5, Proposition V.9(iv)), we get C_{𝔞}^{0} ⊆ C_{max} and hence that C_{𝔞} ⊆ C_{max} . Finally Proposition VI.3(iii) implies that C_{min} ⊆ lim C_{𝔞}. (iii) First we show that (𝔤, τ) has cone potential. Let Z ∈ 𝔤^{α} be of type (A). We have to show that Z = 0. Let X ∈ C_{𝔞}^{0} . Now Lemma VI.1(i) shows that

e^{t ad (Z+τ(Z))}.X = X − tα(X)(Z − τ(Z)) ∈ C

for all t ∈ ℝ. Now H(C) = {0} implies that Z − τ(Z) = 0, hence that Z = 0, and this proves that (𝔤, τ) has cone potential.

Since lim(C_{𝔞}) is pointed and contains the cone C_{min} , this cone is also pointed. Thus (𝔤, τ) has strong cone potential because each functional α ∈ int lim(C_{𝔞})^{⋆} satisfies α([X, τ.X]) > 0 for all X ∈ 𝔤^{α}, α ∈ Δ^{+} . The 𝔭adaptedness of Δ^{+} follows from the choice of Δ_{n}^{+} which ensures that it is invariant under the Weyl group. (iv) If 𝔷(𝔭) = {0}, then the proof of (ii) shows that 0 ∈ C^{0} . If C is a cone, then this implies that C = 𝔮. This proves (iv).

Example VI.7. (a) Let (𝔰, τ) = (𝔰𝔩(2, ℝ) ⊕ 𝔰𝔩(2, ℝ), τ_{1} ⊕ τ_{2}), where τ_{1} = θ is of type (R), and τ_{2} of type (SR) is given by conjugation by T (cf. Example IV.11). Then (𝔰, τ) is quasihermitian and admits e^{ad 𝔥} -invariant pointed generating convex hyperbolic cones. An example is given by

C = {(tT + hH, h′H′ + u′U′) : 0 ≤ |u′| ≤ h′, 0 ≤ t^{2} + h^{2} ≤ (h′)^{2} − (u′)^{2}}.

Note that dim 𝔮 = 4 and dim 𝔭 = 3. The set C is invariant since the Lorentzian form given in coordinates by (h)^{′2} − (u′)^{2} − t^{2} − u^{2} is invariant under Inn_{𝔰}(𝔥). Moreover, C is a Lorentzian cone in 𝔮.

The preceding example shows that it is not necassary that the irreducible pieces of (𝔰, τ) are purely (NCC), which in the irreducible case can be shown to be equivalent to the existence of a hyperbolic invariant convex cone (cf. [8]). Below we will give a necessary and sufficient condition for symmetric Lie algebras to have this property.

VII. Symmetric Lie algebras with cone potential

In this chapter we exploit the implications of (strong) cone potential for the structure of a symmetric Lie algebra. We recall from Proposition V.9 that a quasihermitian semisimple symmetric Lie algebra always has cone potential. The following example shows that the converse is not true.

Example VII.1. Let 𝔤 = 𝔰𝔬(2, n), n ≥ 3 and τ given by conjugation with

I_{n,2} = diag(\underbrace{1, …, 1}_{n}, −1, −1).

Then 𝔥 ≅ 𝔰𝔬(2, n − 2) ⊕ 𝔰𝔬(2), 𝔞 = ℝX_{1} + ℝX_{2}, where X_{1} = E_{1,n} + E_{n,1} and X_{2} = E_{2,n−1} + E_{n−1,2} , is a hyperbolic subalgebra which is maximal abelian in 𝔮. The root system is given by

Δ = {±α_{1}, ±α_{2}, ±(α_{1} ± α_{2})},

where α_{i}(X_{j}) = δ_{ij}, i, j ∈ {1, 2}. It is easy to check that (𝔤, τ) has cone potential. In particular Δ_{k} = {±(α_{1} ± α_{2})} and Δ_{p} = {±α_{1}, ±α_{2}}. The Weyl group W is isomorphic to the Weyl group of the Riemannian symmetric Lie algebra 𝔰𝔬(2, 2) ≅ 𝔰𝔩(2, ℝ) ⊕ 𝔰𝔩(2, ℝ) and is generated by the two orthogonal reflections s_{α_{1}+α_{2}}, s_{α_{1}−α_{2}} . Hence (𝔤, τ) does not have any 𝔭-adapted positive system. This gives an example of a simple symmetric Lie algebra with cone potential which is not quasihermitian.

For any 𝔞-invariant subspace 𝔟 ⊆ 𝔤 we write 𝔟^{+} := [𝔞, 𝔟] for the effective part of 𝔟 and 𝔟^{0} for 𝔷_{𝔟}(𝔞). Note that 𝔟 = 𝔟^{0} ⊕ 𝔟^{+} .

Proposition VII.2. If (𝔤, τ) has cone potential and 𝔫 is the nilradical of 𝔤, then the following assertions hold:

(i) Every τ -invariant abelian ideal 𝔟 ⊆ 𝔤 is contained in 𝔫^{0} . Moreover, 𝔟_{𝔮} ⊆ 𝔷(𝔤) and 𝔟_{𝔥} ⊴ 𝔤.

(ii)[𝔫, 𝔫] ⊆ 𝔫^{0} . In particular [𝔯^{α}, 𝔯^{β}] = {0} for α ≠ −β.

(iii) If (𝔤, τ) is effective, then (a) [𝔫, 𝔫] ⊆ 𝔷(𝔤),

(b) 𝔷(𝔤) ⊆ 𝔮, and

(c) [𝔫_{𝔥}, 𝔫_{𝔥}] = {0}. (iv) The subspace [𝔞, 𝔥] of 𝔮 contains no non-zero 𝔥-submodule.

Proof. (i) (cf. [5, Lemma 7.14]) Since 𝔟 is an ideal, it is 𝔞-invariant. We claim that [𝔞, 𝔟] = {0}. Suppose that this is false and let 0 ≠ Z ∈ 𝔤^{α} ∩ 𝔟. Since 𝔟 was supposed to be τ -invariant, we obtain 𝔤(Z) ⊆ 𝔟. Since (𝔤, τ) has cone potential, 𝔤(Z) is not abelian, contradicting the assumption that 𝔟 is abelian. Now 𝔟 = 𝔟^{0} yields 𝔟_{𝔮} ⊆ 𝔮^{0} ∩ 𝔟 ⊆ 𝔞 ∩ 𝔟 ⊆ 𝔷(𝔤) because 𝔟 is a nilpotent ideal. Since 𝔫 + 𝔟 is a nilpotent ideal, we have 𝔟 ⊆ 𝔫.

To see that 𝔟_{𝔥} is an ideal, we first note that

[𝔟, 𝔤^{α}] ⊆ [𝔤^{0}, 𝔤^{α}] ∩ 𝔟 ⊆ 𝔤^{α} ∩ 𝔟 = {0}

implies [𝔟, [𝔞, 𝔤]] = {0}. Now the assertion follows from the fact that 𝔟_{𝔥} is an ideal in 𝔥 and from 𝔮 = 𝔞 ⊕ [𝔞, 𝔥]. (ii) (cf. [5, Th. 7.15]) We prove the assertion by induction on dim(𝔤). We denote by 𝔫^{m} the elements in the lower central series of 𝔫. Suppose that 𝔫^{m} ≠ {0} for some integer m ≥ 2. According to [5, Lemma 7.13], the ideal 𝔟 = 𝔫^{m−1} is abelian. Note that 𝔟 is τ -invariant, so that (i) applies. If 𝔟_{𝔥} ≠ {0}, we consider the symmetric Lie algebra 𝔤_{1} := 𝔤/𝔟_{𝔥}. Let π: 𝔤 → 𝔤_{1} denote the quotient homomorphism. In view of Lemma III.11, π(𝔞) is a maximal hyperbolic abelian subspace of 𝔮_{1} := π(𝔮). Moreover, 𝔤_{1} has cone potential because 𝔟_{𝔥} does not intersect 𝔞 and 𝔤 has cone potential. Hence induction applies and yields the assertion if 𝔟_{𝔥} ≠ {0}.

Thus, in view of (i), we may assume that 𝔟 = 𝔟_{𝔮} is central. But this contradicts 𝔫^{m} ≠ {0}. Hence 𝔫^{1} = [𝔫, 𝔫] is abelian. Again (i) applies and shows that [𝔫, 𝔫] ⊆ 𝔷_{𝔫}(𝔞). (iii) (a) We can use the same arguments as in the proof of (ii). Here 𝔟_{𝔥} = {0} since (𝔤, τ) is effective and 𝔟_{𝔥} is an ideal of 𝔤 (cf. Lemma I.8). Hence the argument above shows that [𝔫, 𝔫] is an abelian ideal. Thus effectivity and (i) give [𝔫, 𝔫] ⊆ 𝔷(𝔤).

(b) That 𝔷(𝔤) ⊆ 𝔮 follows from the fact that 𝔷(𝔤)_{𝔥} is an ideal of 𝔤 contained in 𝔥.

(c) In view of (a) and (b), we have [𝔫_{𝔥}, 𝔫_{𝔥}] ⊆ [𝔫, 𝔫] ∩ 𝔥 ⊆ 𝔷(𝔤)_{𝔥} = {0}. (iv) Let V ⊆ [𝔞, 𝔥] be an 𝔥-submodule. Then its orthogonal subspace V^{⊥_{κ}} ⊆ 𝔮 with respect to the Cartan-Killing form is an 𝔥-submodule containing 𝔞, hence contains 𝔮_{hyp} = Inn_{𝔤}(𝔥).𝔞, and therefore coincides with 𝔮. We conclude that V ⊆ 𝔮^{⊥_{κ}} and hence that V ⊆ 𝔤^{⊥_{κ}} ⊆ 𝔯.

If, on the other hand, 𝔰 is an 𝔞-τ -invariant Levi complement, then

[𝔞, 𝔥] = [𝔞, 𝔰_{𝔥} + 𝔯_{𝔥}] = \underbrace{[𝔞, 𝔰_{𝔥}]}_{⊆𝔰_{𝔮}} + \underbrace{[𝔞, 𝔯_{𝔥}]}_{⊆𝔯_{𝔮}} .

Therefore V ⊆ [𝔞, 𝔯_{𝔥}] = ∑_{α∈Δ}(1 − τ).𝔯^{α}. Let β ∈ Δ and X_{β} ∈ 𝔯^{β}. Then (ii) implies that

[(1 + τ).X_{β}, ∑_{α∈Δ}(1 − τ).𝔯^{α}] = [(1 + τ).X_{β}, (1 − τ).𝔯^{β}].

Moreover

[(1 + τ).X_{β}, (1 − τ).X_{β}] = −[X_{β}, τ.X_{β}] + [τ.X_{β}, X_{β}] = 2[τ.X_{β}, X_{β}] ∈ 𝔞_{𝔯} \ {0} whenever X_{β} ≠ {0}. For v ∈ V with v = ∑_{α∈Δ} v_{α}, v_{α} = (1−τ).X_{α} ∈ (1−τ).𝔯^{α} we now obtain

[(1 + τ).X_{α}, v] = 2[τ.X_{α}, X_{α}] ∈ V ∩ 𝔞_{𝔯}.

This proves that v_{α} = 0 for all α, hence v = 0 for all v ∈ V , and eventually V = {0}.

Example VII.3. The assertion of (ii) becomes false if (𝔤, τ) does not have cone potential. We consider the Lie algebra

𝔤 = span{T, X_{1}, Y_{1}, X_{2}, Y_{2}, X_{3}, Y_{3}}

with the non-zero brackets

[T, X_{k}] = kX_{k}, [T, Y_{k}] = −kY_{k}, [X_{1}, X_{2}] = X_{3}, [Y_{1}, Y_{2}] = Y_{3},

1 ≤ k ≤ 3. We define

𝔥 = ℝ(X_{1} + Y_{1}) ⊕ ℝ(X_{2} + Y_{2}) ⊕ ℝ(X_{3} + Y_{3}),
𝔮 = ℝT ⊕ ℝ(X_{1} − Y_{1}) ⊕ ℝ(X_{2} − Y_{2}) ⊕ ℝ(X_{3} − Y_{3}).

Since [𝔥, 𝔥] ⊆ 𝔥, [𝔥, 𝔮] ⊆ 𝔮 and [𝔥, 𝔥] ⊆ 𝔥, the prescriptions τ |_{𝔥} = id_{𝔥} and τ |_{𝔮} = − id_{𝔮} define an involution on 𝔤. Further on 𝔞 = ℝT is a maximal hyperbolic abelian subspace in 𝔮, which is maximal abelian in 𝔮. An easy calculation yields [𝔫, 𝔫] = ℝX_{3} ⊕ ℝY_{3}, hence [𝔫, 𝔫] ⊈ 𝔫^{0} .

Definition VII.4. Let (𝔤, τ) be a symmetric Lie algebra and ρ: 𝔤 → 𝔤𝔩(V) a representation. Then we call an operator A ∈ End(V) a τ -intertwiner if

A ∘ ρ(X) = ρ(τ.X) ∘ A

holds for all X ∈ 𝔤.

The significance of the notion of a τ -intertwiner becomes apparent in the following lemma.

Lemma VII.5. Let V be a module of the symmetric Lie algebra (𝔤, τ) and ϕ a τ -covariant bilinear form on V . Further let σ be a second involution on 𝔤 commuting with τ and A ∈ End(V) a σ -intertwiner. Then the following assertions hold:

(i) The form ϕ_{A} defined by ϕ_{A}(v, w) := ϕ(A.v, w) is τσ -covariant. (ii) If ϕ is symmetric (skew-symmetric), and A is ϕ-symmetric, then the

same holds for ϕ_{A}, and if A is ϕ-skew-symmetric, then ϕ_{A} is skew-symmetric (symmetric).

Proof. (i) For v, w ∈ V and X ∈ 𝔤 we have

\begin{aligned}ϕ_{A}(X.v, w) & = ϕ(AX.v, w) = −ϕ((σ.X)A.v, w) \\ & = −ϕ(A.v, (τσ.X).w) = −ϕ_{A}(v, (τσ.X).w). \end{aligned}

(ii) This is a simple computation.

Lemma VII.6. Let A be a convex set of symmetric operators on ℝ^{n} satisfying the condition

d := max{rank A: A ∈ A} < n.

Then there exists 0 ≠ v ∈ ℝ^{n} such that

⟨A.v, v⟩ = 0 for all A ∈ A.

Proof. (cf. [13, Prop. II.32]) Let A_{0} ∈ A be of maximal rank d and {v_{1}, …, v_{n}} an orthonormal basis of eigenvectors, i.e.

A_{0}.v_{i} = \{ \begin{matrix}α_{i}v_{i} & i ≤ d, \\ 0 & i ≥ d + 1. \end{matrix}

We claim : ⟨A.v_{d+1}, v_{d+1}⟩ = 0 for all A ∈ A.

Since all assumptions transfer to the subspace B := ⟨v_{1}, …, v_{d+1}⟩ we may assume that n = d + 1. Let 0 < μ < 1 and μ_{0} := \frac{1−μ}{μ}. For all A ∈ A we consider the expression

0 = det(μA+(1−μ)A_{0}) = μ^{d+1} det(A+μ_{0}A_{0}) = μ^{d+1}(μ_{0}^{d}α_{1}·. . .·α_{d}·b_{d+1}+p(μ_{0})),

where b_{d+1} = ⟨A.v_{d+1}, v_{d+1}⟩ and p is a polynomial of degree smaller than d − 1. The expression above is clearly analytic in μ_{0} and the identity theorem for analytic functions implies b_{d+1} = 0 since α_{1} · . . . · α_{d} ≠ 0.

In the following we identify 𝔮^{∗} with the subspace 𝔥^{⊥} in 𝔤^{∗} and 𝔞^{∗} with the subspace [𝔞, 𝔥]^{⊥} of 𝔮^{∗}.

Proposition VII.7. Let 𝔫 be the nilradical of a symmetric Lie algebra (𝔤, τ) with cone potential. To every α ∈ 𝔞_{𝔷}^{∗} we associate the skew symmetric bilinear form

ϕ^{α}: 𝔫 × 𝔫 → ℝ; ϕ^{α}(X, Y) := α([X, Y]).

Then the following assertions hold:

(i) The forms ϕ^{α} are 𝔤-invariant and τ -antiinvariant, i.e., ϕ^{α}(τ.X, τ.Y) = −ϕ^{α}(X, Y). Furthermore 𝔫^{β}⊥_{ϕ^{α}}𝔫^{γ} if β ≠ −γ.

(ii) The bilinear forms

ψ^{α}: 𝔫 × 𝔫 → ℝ, (X, Y) ↦ α([τ.X, Y])

are symmetric and τ -covariant.

(iii) There exists an element α ∈ 𝔞_{𝔷}^{∗} such that ψ^{α} is non-degenerate on 𝔫^{+} . Moreover, the set of these α is open and dense in 𝔞_{𝔷}^{∗}.

Proof. (i) By the Jacobi identity it is sufficient to show that

p_{𝔞}([Z, [X, Y]]) = 0 for Z ∈ 𝔤 and X, Y ∈ 𝔫.

But this is an immediate consequence of [𝔫, 𝔫] ⊆ 𝔤^{0} (cf. Proposition VII.2(ii)) which shows that

p_{𝔞}([𝔤, [𝔫, 𝔫]]) ⊆ p_{𝔞}([𝔤^{0}, [𝔫, 𝔫]]) ⊆ p_{𝔞}([𝔤^{0}, 𝔤^{0}]) = {0}

because 𝔤^{0} = 𝔥^{0} ⊕ 𝔞 and [𝔤^{0}, 𝔤^{0}] ⊆ 𝔥^{0} . The τ -antiinvariance of the forms follows immediately from τ^{∗}α = −α. The last assertion is a consequence of Proposition VII.2(ii). (ii) Since ϕ^{α} is τ -antiinvariant, τ is ϕ^{α} -skew-symmetric, so that the form ψ^{α} is symmetric. Moreover, the 𝔤-invariance of ϕ^{α} implies that ψ^{α} is τ -covariant (Lemma VII.5). (iii) First we prove the assertion for a single 𝔯^{β} instead of 𝔫^{+} . We choose a basis of 𝔯^{β} and thus identify it with ℝ^{n}. Under this identification the symmetric forms ψ^{α} |_{𝔯^{β}×𝔯^{β}} correspond to symmetric matrices on ℝ^{n}. Clearly, A := {ψ^{α} | 𝔯^{β}×𝔯^{β}: α ∈ 𝔞^{∗}} is a vector space of symmetric operators, hence convex. Then Lemma VII.6 shows that if (iii) is false, then there exists 0 ≠ X ∈ 𝔯^{β} with

ψ^{α}(X, X) = 0

for all α ∈ 𝔞_{𝔷}^{∗}, i.e., [X, τ(X)] = 0, contradicting the assumption that 𝔤 has cone potential.

We define

M_{β} := {α ∈ 𝔞_{𝔷}^{∗}: ψ^{α} |_{𝔯^{β}×𝔯^{β}} non-degenerate}.

The observation above shows that M_{β} is non-empty. Now we see that

M_{β} = {α ∈ 𝔞_{𝔷}^{∗}: det(ψ^{α} |_{𝔯^{β}×𝔯^{β}}) ≠ 0}.

Since it is non-empty, the fact that the determinant is a polynomial in the matrix entries shows that M_{β} is open and dense. Now M := ⋂_{β∈Δ_{r}} M_{β} is a finite intersection of open and dense sets, hence is open and dense. This proves (iii) because the elements of M correspond to non-degenerate symmetric forms on 𝔫^{+} .

Proposition VII.8. If (𝔤, τ) has cone potential and α ∈ Δ_{s}, then 𝔯^{α} = {0}.

Proof. Let 0 ≠ Z ∈ 𝔰^{α}. Then, in view of Proposition IV.8(ii), the fact that (𝔤, τ) has cone potential implies that Z is either of type (SR) or of type (R). Hence the corresponding subalgebra 𝔤(Z) (cf. Theorem IV.1(iii)) is isomorphic to 𝔰𝔩(2, ℝ).

We consider the space

V := ∑_{β∈ℝα} 𝔯^{β}.

Then the fact that 𝔯 is an ideal implies that [𝔰^{α}, 𝔯^{β}] ⊆ 𝔯^{β+α}, hence shows that V is invariant under 𝔤(Z). Suppose that 𝔯^{α} ≠ {0}. Then α(α̌) = 2 implies that there exists a simple 𝔤(Z)-submodule W ⊆ V of odd dimension intersecting 𝔯^{α} non-trivially. We write

W = ⊕_{j=−m}^{m}W^{j},

where the subspaces W^{j} := W ∩ 𝔯^{jα} are one-dimensional. Since 𝔤(Z) is τ - invariant, the subspace τ(W) of 𝔯 is also a 𝔤(Z)-submodule. Hence the irreducibility of W implies that either W ∩ τ(W) = {0} or W = τ(W).

Case 1: τ(W) = W. Let 0 ≠ X_{0} ∈ W^{1} = W ∩ 𝔯^{α}. Now cone potential and the τ -invariance of W imply the existence of an element β ∈ 𝔞^{∗} such that ϕ^{β} |_{W ×W} is non-degenerate (Proposition VII.7(iii)).

In view of Proposition VII.7(i), the skew-symmetric bilinear form ϕ^{β} | W ×W is invariant for 𝔤(Z). Since W is odd-dimensional, this contradicts the fact that an irreducible 𝔰𝔩(2, ℝ)-module has an invariant skew-symmetric bilinear form if and only if it is even dimensional (cf. [1, Ch. 8, §7, no. 5, Prop. 12]). Case 2: τ(W) ∩ W = {0}. Then we consider the 𝔤(Z)-submodule W̃ := W + τ(W) ⊆ [𝔤(Z), 𝔯] ⊆ 𝔫. Let 0 ≠ X_{0} ∈ W^{0} . Then W̃^{j} := W^{j} + τ(W^{−j}) ⊆ 𝔯^{jα} and therefore

X_{0} − τ(X_{0}) ∈ W̃^{0} ∩ 𝔮 ∩ 𝔫 ⊆ 𝔞 ∩ 𝔫 ⊆ 𝔷(𝔤).

We conclude that X_{0} − τ(X_{0}) and therefore X_{0} commutes with 𝔤(Z), contradicting the fact that W is a non-trivial simple 𝔤(Z)-module.

Corollary VII.9. If (𝔤, τ) has cone potential, then Δ is split, and

(7.1)  Δ_{r} = {α ∈ Δ: 𝔤^{α} ⊆ 𝔯} and Δ_{s} = {α ∈ Δ: 𝔤^{α} ⊆ 𝔰},

where 𝔰 in an 𝔞-invariant Levi complement.

Proof. Firstly Proposition VII.8 shows that (7.1) holds. Now this and the fact that 𝔤 has cone potential implies that Δ is split (cf. Proposition IV.8(ii)).

Proposition VII.10. Suppose that 𝔞 is maximal abelian in 𝔮 and that 𝔥^{0} is compactly embedded in 𝔤. If 𝔱_{𝔥} ⊆ 𝔥^{0} is a Cartan subalgebra, then 𝔱 := 𝔱_{𝔥} + i𝔞 is a compactly embedded Cartan subalgebra of 𝔤^{c} = 𝔥 + i𝔮.

Proof. From the fact that 𝔥^{0} is compactly embedded it follows in particular that 𝔱_{𝔥} is abelian, and hence that 𝔱 is abelian. It is clear that 𝔱 is compactly embedded in 𝔤^{c}. That it is maximal abelian follows from

𝔷_{𝔤^{c}}(𝔱) = 𝔷_{𝔤^{c}}(𝔱_{𝔥}) ∩ 𝔷_{𝔤^{c}}(i𝔞) = 𝔷_{𝔥^{0}}(𝔱_{𝔥}) ⊕ i𝔞 = 𝔱_{𝔥} ⊕ i𝔞.

Definition VII.11. Let 𝔤 be a Lie algebra and V a finite dimensional real 𝔤-module, where the module structure is defined by the representation ρ: 𝔤 → 𝔤𝔩(V). Then we say that V is of compact type if the group ⟨e^{ρ(𝔤)}⟩ has compact closure.

In this sense a subalgebra 𝔟 of 𝔤 is compactly embedded if and only if 𝔤 is a 𝔟-module of compact type.

Note that the class of modules of compact type is invariant under taking submodules, quotients, direct sums, tensor products etc.

Proposition VII.12. If the symmetric Lie algebra (𝔤, τ) has strict cone potential, then the following assertions hold:

(i) 𝔮_{L} and 𝔮 are 𝔥^{0} -modules of compact type.

(ii) If, in addition, (𝔤, τ) is effective, then 𝔥^{0} is compactly embedded and 𝔫_{𝔥}^{0} = {0}.

Proof. (i) First we show that each root space 𝔤^{α} is an 𝔥^{0} -module of compact type. In fact, if α ∈ Δ_{s}, then, in view of Proposition VII.8, the fact that 𝔤 has cone potential implies that the form κ_{τ} is definite on 𝔤^{α} (Proposition IV.7(vi)). Since this form is preserved by 𝔥^{0} , the module 𝔤^{α} is of compact type.

Now let α ∈ Δ_{r}. Then we use the strict cone potential to see that for an appropriate β ∈ 𝔞_{𝔷}^{∗} the form ψ^{β} on 𝔤^{α} is positive definite (cf. Proposition VII.7). Now the invariance of this form under 𝔥^{0} (Proposition VII.7(ii)) shows that 𝔤^{α} is a module of compact type.

Since the projection p_{𝔮}: 𝔤 → 𝔮, X ↦ \frac{1}{2}(X − τ(X)) is 𝔥-equivariant and

𝔮 = 𝔞 ⊕ ⨁_{α∈Δ}(1 − τ).𝔤^{α},

we conclude that 𝔮 is an 𝔥^{0} -module of compact type. Therefore 𝔮 ⊗ 𝔮 and hence [𝔮, 𝔮] ⊆ 𝔥 are 𝔥^{0} -modules of compact type, whence 𝔮_{L} = 𝔮 + [𝔮, 𝔮] is an 𝔥^{0} -module of compact type. (ii) If (𝔤, τ) is effective, then the representation ad_{𝔮} of 𝔥 is faithful. Thus we can embed 𝔥 via ad_{𝔮} in the 𝔥^{0} -module 𝔤𝔩(𝔮) of compact type. It follows that 𝔥^{0} is compactly embedded in 𝔥, and therefore that 𝔥^{0} is compactly embedded in 𝔤.

Since the subalgebra 𝔫_{𝔥}^{0} is on the one hand side is compactly embedded and on the other hand 𝔫 is a nilpotent ideal, so that all operators ad X, X ∈ 𝔫 are nilpotent, it follows that 𝔫_{𝔥}^{0} = 𝔷_{𝔥}. Now the effectiveness implies that 𝔷_{𝔥} = {0}.

Corollary VII.13. If (𝔤, τ) is an effective symmetric Lie algebra with strong cone potential and 𝔱_{𝔥} is a Cartan subalgebra of 𝔥^{0} , then 𝔱 := 𝔱_{𝔥} + 𝔞 is Cartan subalgebra of 𝔤 and 𝔱^{c} := 𝔱_{𝔥} + i𝔞 is a compactly embedded Cartan subalgebra of 𝔤^{c}. Proof. We only have to combine Proposition VII.10 with Proposition VII.12(ii).

Covariant forms on modules

Now we are going to describe the fine structure of quasihermitian Lie algebras with strong cone potential. But first we need some information about covariant forms and (NCC) Lie algebras.

Lemma VII.14. Let (𝔤, τ) be a semisimple symmetric Lie algebra, θ a Cartan involution commuting with τ, τ^{a} := θτ, and V a finite dimensional irreducible real 𝔤-module. Let further ϕ be a τ -covariant symmetric (skew-symmetric) bilinear form on V and assume that there exists a non-trivial ϕ-symmetric (skewsymmetric) τ^{a} -intertwiner. Then each τ -covariant symmetric bilinear form on V is a multiple of ϕ. Proof. We prove the assertion for the case where ϕ is symmetric. The other case can be proved in the same way. Let A be a non-zero ϕ-symmetric τ - intertwiner and ρ the representation of 𝔤 defining the module structure on V .

Then the form ϕ_{A} is symmetric and θ-covariant (Lemma VII.13). Using Weyl’s unitary trick, we also find a θ-covariant positive definite form ψ on V (Lemma I.9(ii)). Now we can write ϕ_{A} = ψ_{B} for a ψ -symmetric operator B and get

\begin{aligned}ψ(B.(X.v), w) = ψ_{B}(X.v, w) & = −ψ_{B}(v, θ(X).w) \\ & = −ψ(B.v, θ(X).w) = ψ(X.(B.v), w), \end{aligned}

so that the positive definiteness of ψ implies that ρ(X)B = Bρ(X) for all X ∈ 𝔤. Since B is ψ -symmetric, it is diagonalizable and all its eigenspaces are invariant under 𝔤. Therefore the irreducibility of V implies that B = λ1 with λ ≠ 0. So ϕ_{A} is a definite form.

If ϕ̃ is also τ -covariant, then ϕ̃_{A} = μϕ_{A}, and finally ϕ̃ = μϕ. This completes the proof.

Let ρ: 𝔤 → 𝔤𝔩(V) be a finite dimensional real representation of a (NCC) Lie algebra 𝔤. Then, according to Lemma I.9(ii), the element ρ(H) is diagonalizable. For every μ ∈ ℝ we denote by V_{μ} the corresponding eigenspace of ρ(H) and obtain the decomposition

V = ⨁_{μ∈ℝ} V_{μ}.

Proposition VII.15. Let (𝔤, τ) be a (NCC) symmetric Lie algebra and ρ: 𝔤 → 𝔤𝔩(V) a finite dimensional irreducible real representation. Then the following assertions hold:

(i) If λ ∈ ℝ is the maximal eigenvalue of ρ(H), then Spec ρ(H) ⊆ λ − 2ℕ_{0}.

(ii) There exists an, up to scalar multiple unique, non-trivial τ -covariant symmetric bilinear form ϕ on V . This form can be normalized in such a way that

ϕ|_{V_{λ−2n}} is \{ \begin{matrix}positive definite, & if n is even, \\ negative definite, & if n is odd. \end{matrix}

Proof. (i) From the triangular decomposition of 𝔤 (Proposition V.6(ii)) we deduce that

M := ⨁_{n∈ℕ_{0}} V_{λ−2n}

is a non-trivial submodule of V . Hence irreducibility yields M = V . (ii) By Weyl’s unitary trick we find a θ-covariant scalar product ψ on V (Lemma I.9(ii)). Then the operator ρ(H) − λ is ψ -symmetric with even eigenvalues. Therefore

A := e^{i \frac{π}{2} (ρ(H)−λ)}

is ψ -symmetric with A^{2} = 1. Moreover, Proposition V.6(iii) implies that A is a τ^{a} -intertwiner. Now the form ϕ := ψ_{A} is symmetric and τ -covariant (Lemma VII.5) and Lemma VII.14 shows that such a form is unique up to a scalar multiple. The statement on the signs of ϕ now follows from A.v = (−1)^{n}v for v ∈ V_{λ−2n}.

Suppose that (𝔤, τ) is (CT) and ρ: 𝔤 → 𝔤𝔩(V) is an irreducible finite dimensional real representation. Then Lemma V.7(i) and finite dimensional 𝔰𝔩(2, ℝ)-representation theory imply that Spec(T) ⊆ ℤ. In the same way as in Proposition VII.15 we obtain that either Spec (ρ(T)) ⊆ 2ℤ or Spec (ρ(T)) ⊆ 2ℤ + 1. In the first case we call V even, and we call it odd in the latter case.

Proposition VII.16. Let (𝔤, τ) be a (CT) symmetric Lie algebra and ρ: 𝔤 → 𝔤𝔩(V) an irreducible finite dimensional real representation. Then the following cases arise:

(1) If V is odd, then there exists a one-dimensional space of skew symmetric 𝔤-invariant forms on V .

(2) If V is even and 𝔻 := End_{𝔤}(V), then 𝔻 ≅ ℝ, ℂ, ℍ and in all cases the space of invariant symplectic forms is parametrized by 𝔻_{im} = {d ∈ 𝔻: \overline{d} = −d\} as follows. Let ψ be a θ-covariant scalar product on V . Then d^{⊤} = −\overline{d} holds for all d ∈ 𝔻 and (ψ_{d})_{C} with C = e^{\frac{π}{2} ρ(U)} is the corresponding 𝔤-invariant skew-symmetric form.

Proof. (1) Let V be an odd 𝔤-module and ϕ be a non-trivial τ -covariant form on V (Proposition VII.15(ii)). We extend ϕ to a hermitian form on V_{ℂ}

which is also denoted by ϕ and define B: = e^{i \frac{π}{2} (ρ(T)−1)} = ie^{i \frac{π}{2} ρ(T)} . Since ρ(T) is ϕ-skew-hermitian, iρ(T) is ϕ-hermitian, and therefore e^{i \frac{π}{2} ρ(T)} is ϕ-hermitian which implies that B is ϕ-skew-hermitian. Moreover B^{2} = 1, B leaves V invariant, and it is a τ -intertwiner, so that Lemma VII.5 shows that Ω := ϕ_{B} is a non-degenerate skew-symmetric 𝔤-invariant form on V . Furthermore the uniqueness up to scalar multiple follows from the uniqueness of ϕ because for each 𝔤-invariant skew-symmetric form Ω̃ the form Ω̃_{B} is symmetric and τ - covariant, where on the other side Ω_{B} = ϕ_{B^{2}} = ϕ.

(2) Now we assume that V is even and that ψ is a θ-covariant scalar product on V (cf. Lemma I.9(ii)). For d ∈ 𝔻 and X ∈ 𝔤 we then obtain

d^{⊤}ρ(X) = −(ρ(θ.X)d)^{⊤} = −(dρ(θ.X))^{⊤} = ρ(X)d^{⊤}

which shows that 𝔻 is invariant under taking transposes. It is clear that ℝ1 ⊆ 𝔻 consists of symmetric elements. Moreover, if 𝔻 ≅ ℂ, ℍ, then the compactness of the one-parameter subgroups e^{ℝd} for \overline{d} = −d implies that d cannot be symmetric. Hence ℝ1 = {d ∈ 𝔻: d^{⊤} = d}. Now the classification of the involutions on ℂ and ℍ implies that d^{⊤} = −\overline{d} for all d ∈ 𝔻. According to Lemma VII.5, the θ-covariant skew-symmetric forms on V are given by ψ_{d}, d ∈ 𝔻_{im} . Furthermore the operator C = e^{\frac{π}{2} ρ(U)} is a θ-intertwiner which is ψ -skew-symmetric. Hence the forms (ψ_{d})_{C} = (ψ_{C})_{d} are 𝔤-invariant and skewsymmetric, and reversing the construction, it follows that these are all such forms.

We note that in terms of the classification scheme for symplectic 𝔤modules described in [14], the preceding result means that whenever an even 𝔤module carries an invariant symplectic form, then it is of type ℂ_{II} or ℍ_{II}. These observations will be quite important for a classification of the quasihermitian symmetric Lie algebras with strong cone potential. To see how such Lie algebras may look like, we use the preceding proposition to construct an important class of examples.

Example VII.17. (a) Let (𝔰, τ) be a symmetric (CT) Lie algebra and V an irreducible odd real 𝔤-module. Further let Ω be a non-trivial skew symmetric invariant form on V , which exists by Proposition VII.16. We consider the Lie algebra 𝔥_{V}: = V ⊕ ℝ with the bracket

[(v, s), (w, t)]: = (0, Ω(v, w))

and put

𝔤: = 𝔥_{V} ⋊ 𝔰.

We extend τ to an involution on 𝔤 by setting τ |_{𝔷(𝔤)} = − id_{𝔷(𝔤)} and τ |_{V} = ie^{i \frac{π}{2} ρ(T)} , where ρ denotes the representation of 𝔰 on V . This turns 𝔤 into a symmetric Lie algebra (𝔤, τ). Since, in view of Proposition VII.15(iii), the τ - covariant form ϕ(v, w) := Ω(τ(v), w) is definite on the root spaces V^{α}, we see that (𝔤, τ) has cone potential. Furthermore Proposition VII.15(ii) shows that (𝔤, τ) has strong cone potential if and only if Spec (ρ(T)) = {1, −1}.

(b) (The symmetric Jacobi algebra) We consider the symmetric (CT) Lie algebra (𝔰𝔭(n, ℝ), τ), where τ is given by conjugation with I_{n,n}. Let V = ℝ^{2n} be the irreducible module for the standard representation of 𝔰𝔭(n, ℝ). Here the invariant skew symmetric form Ω on V is given by the matrix

( \begin{matrix}0 & I_{n} \\ −I_{n} & 0 \end{matrix} ) .

Moreover the involution on V is given by τ |_{V} = I_{n,n} and 𝔥_{V} is the (2n + 1)- dimensional Heisenberg algebra 𝔥_{n}. We call the symmetric Lie algebra given by

(𝔥𝔰𝔭(n, ℝ), τ) := (𝔥_{n} ⋊ 𝔰𝔭(n, ℝ), τ)

the symmetric Jacobi algebra. Note that Spec(T) = {−1, 1}, so that the Lie algebra (𝔥𝔰𝔭(n, ℝ), τ) has strong cone potential.

Theorem VII.18. (The Short String Theorem) Let (𝔤, τ) be a quasihermitian symmetric Lie algebra with strong cone potential, Δ^{+} be a 𝔭-adapted positive system, and α ∈ Δ_{r}, β ∈ Δ_{p}. Then the following assertions hold:

(i) The β -string through α has at most length 2 and contains at most one positive and one negative root. If α, β are positive, then α + β ∉ Δ, α − β ∈ Δ_{r}^{−} whenever it is a root, and ⟨α, β⟩ ≥ 0.

(ii) If

𝔭_{𝔰}^{±} = ⨁_{α∈Δ_{p}^{±}} 𝔤^{α}, 𝔭_{𝔯}^{±} = ⨁_{α∈Δ_{r}^{+}} 𝔤^{α}, and 𝔭^{±} = 𝔭_{𝔯}^{±} + 𝔭_{𝔰}^{±},

then [𝔭^{+}, 𝔭^{+}] = {0} and [𝔭_{𝔰}^{±}, 𝔭_{𝔯}^{∓}] ⊆ 𝔭_{𝔯}^{±}.

(iii) C_{min}(Δ^{+}) ⊆ C_{max}(Δ^{+}).

Proof. (i) Let 0 ≠ Z ∈ 𝔤^{β} and 𝔤(Z) the corresponding 3-dimensional symmetric Lie algebra of type (SR). Note that 𝔤(Z) is (NCC). Since (𝔤, τ) has strong cone potential, we find γ ∈ 𝔞_{𝔷}^{∗} such that ψ^{γ} is positive definite on the positive root spaces and negative definite on the negative root spaces.

We consider the 𝔤(Z)-submodule V := ∑_{n∈ℤ} 𝔯^{α+nβ} of 𝔯. If W ⊆ V is an irreducible submodule of maximal dimension, then W intersects each root space in V , and the restriction of ψ^{γ} to W is a non-degenerate τ -covariant symmetric bilinear form (Proposition VII.7(ii)). Hence Proposition VII.15(ii) implies that ψ^{γ} has alternating signs on the β̌-eigenspaces in W. This proves that V contains at most one positive and at most one negative root space, hence that the length of the β -string through α has at most length 2 and contains at most one positive and one negative root. From that the assertions of (i) are clear. (ii) First [𝔭_{𝔯}^{+}, 𝔭_{𝔯}^{+}] = {0} is a consequence of the cone potential of (𝔤, τ) (Proposition VII.2). Further the 𝔭-adaptedness of Δ and the semisimplicity of 𝔰 imply that [𝔭_{𝔰}^{+}, 𝔭_{𝔰}^{+}] = {0} (cf. Proposition V.9(vii)). Finally [𝔭_{𝔯}^{+}, 𝔭_{𝔰}^{+}] = {0} and [𝔭_{𝔯}^{∓}, 𝔭_{𝔰}^{±}] ⊆ 𝔭_{𝔯}^{±} follow from (i). (iii) The asserted statement is equivalent to

(7.2)  α([X_{β}, τ(X_{β})]) ≥ 0  for all α, β ∈ Δ_{n}^{+}, X_{β} ∈ 𝔤^{β} ,

If β ∈ Δ_{p}^{+} , then (7.2) means that α(β̌) ≥ 0. If this is false, then Corollary V.3 implies that α + β is a root. For α ∈ Δ_{r}^{+} this contradicts (i), and for α ∈ Δ_{p}^{+} this contradicts Proposition V.9(vii). If α ∈ Δ_{n}^{+} and β ∈ Δ_{r}^{+} , then [X_{β}, τ(X_{β})] ∈ 𝔷(𝔤) implies that α([X_{β}, τ(X_{β})]) = 0. This completes the proof.

VIII. Convexity theorems

The canonical extension of a symmetric Lie algebra

Let (𝔤, τ) be a symmetric Lie algebra. We think of (𝔤, τ) as sitting in the symmetric Lie algebra (𝔤_{ℂ}, τ̂), where τ̂ denotes the antilinear extension of τ, i.e., complex conjugation with respect to the real form 𝔤^{c} of 𝔤_{ℂ}. We also write \overline{X} := τ̂(X) for X ∈ 𝔤_{ℂ}. The inclusion (𝔤, τ) → (𝔤_{ℂ}, τ̂) is an embedding of symmetric Lie algebras. We call (𝔤_{ℂ}, τ̂) the canonical extension of (𝔤, τ) and 𝔮̂ := 𝔮 + i𝔥 the canonical extension of 𝔮. The complex linear extension of τ is again denoted by τ. Note that τ̂|_{𝔤} = τ.

For the remainder of this section we assume that (𝔤, τ) is quasihermitian.

Theorem VIII.1. (The Inheritage Theorem) Let (𝔤, τ) be a quasihermitian symmetric Lie algebra, 𝔞 ⊆ 𝔮 a maximal hyperbolic abelian subspace, Δ^{+} be a 𝔭-adapted positive system, and assume that 𝔥^{0} is compactly embedded in 𝔤. Then the following assertions hold:

(i) If 𝔱_{𝔥} ⊆ 𝔥_{0} is a Cartan subalgebra, then 𝔞̂ := 𝔞 + i𝔱_{𝔥} is a maximal hyperbolic abelian and maximal abelian subspace 𝔞̂ ⊆ 𝔮̂.

(ii) If 𝔭 is a maximal hyperbolic Lie triple system in 𝔮 containing 𝔞, then

𝔭̂ := 𝔭 + i[𝔭, 𝔭] + i𝔥^{0}

is a maximal hyperbolic Lie triple system in 𝔮̂, we have 𝔷(𝔭) ⊆ 𝔷(𝔭̂), and 𝔷𝔮̂(𝔷(𝔭)) = 𝔭̂. In particular (𝔤_{ℂ}, τ̂) is quasihermitian and 𝔭̂ is the unique maximal hyperbolic Lie triple system containing 𝔞̂.

(iii) There exists a 𝔭̂-adapted positive system Δ̂^{+} ⊆ Δ(𝔤_{ℂ}, 𝔞̂) which is compatible with Δ^{+} in the sense that

Δ̂_{n}^{+} |_{𝔞} = Δ_{n}^{+}, −τ(Δ_{n}^{+}) = Δ_{n}^{+}, and Δ̂_{k} = {α ∈ Δ̂: α|_{𝔞} ∈ Δ_{k} ∪ {0}}.

(iv) If Ŵ is the Weyl group of 𝔞̂, then there exists for each γ ∈ W a γ̂ ∈ Ŵ with γ̂ ∘ τ |_{𝔞} = τ |_{𝔞} ∘ γ and γ̂ |_{𝔞} = γ.

(v) If p_{𝔞}: 𝔞̂^{∗} → 𝔞^{∗} denotes the restriction map and Δ̂_{α} := p_{𝔞}^{−1}(α) for α ∈ Δ, then (𝔤^{α})_{ℂ} = ⨁_{β∈Δ̂_{α}} 𝔤_{ℂ}^{β}.

(vi) If Δ̂^{+} is compatible with Δ^{+} as in (iii) and Ĉ_{min} , resp. Ĉ_{max} , is the corresponding minimal, resp. maximal, cone, then

(a) Ĉ_{min} ∩ 𝔞 = p_{𝔞}(Ĉ_{min}) = C_{min}.

(b) Ĉ_{max} ∩ 𝔞 = p_{𝔞}(Ĉ_{max}) = C_{max}.

(c) C_{max}^{0} ⊆ Ĉ_{max}^{0} .

(d) p_{𝔞}( cone( ˇΔ̂_{k}^{+})) = cone(Δ̌_{k}^{+}). (vii) If, in addition, (𝔤, τ) is effective with strong cone potential, then (a) the symmetric Lie algebra (𝔤_{ℂ}, τ̂) has strong cone potential,

(b) Ĉ_{min} ⊆ Ĉ_{max}, and

(c) Ĉ_{min,z} = C_{min,z} .

Proof. (i) This follows from Proposition VII.10. (ii) To see that 𝔭̂ is a maximal hyperbolic Lie triple system in 𝔮̂, we show that 𝔨^{c} := i𝔭̂ = i𝔭 + [𝔭, 𝔭] + 𝔥^{0} is a maximal compactly embedded subalgebra of 𝔤^{c} (cf. Corollary III.8). To this end we may assume that 𝔭 is constructed as in Proposition III.5(iv) as 𝔭 = 𝔞_{𝔯} ⊕ 𝔭_{𝔰}, where 𝔞_{𝔯} = 𝔞 ∩ 𝔯, 𝔰 is a τ -𝔞-invariant Levi complement, 𝔰 = 𝔰_{𝔨} ⊕ 𝔰_{𝔭} is a τ -invariant Cartan decomposition, and 𝔭_{𝔰} = (𝔰_{𝔭})_{𝔮}.

First we consider the semisimple symmetric Lie algebra (𝔰, τ). It is a quasihermitian semisimple symmetric Lie algebra for which 𝔰_{𝔥}^{0} is compactly embedded. Now 𝔰_{𝔨}^{c} := (𝔰_{𝔨})_{𝔥} + i(𝔰_{𝔭})_{𝔮} is a maximal compactly embedded subalgebra of 𝔰^{c} = 𝔰_{𝔥} + i𝔰_{𝔮}. Since the subalgebra (𝔰_{𝔭})_{𝔮} + i[(𝔰_{𝔭})_{𝔮}, (𝔰_{𝔭})_{𝔮}] of 𝔰_{𝔨}^{c} is an ideal (Lemma II.7), we have (𝔰_{𝔨})_{𝔥} = 𝔰_{0} ⊕ [(𝔰_{𝔭})_{𝔮}, (𝔰_{𝔭})_{𝔮}], where 𝔰_{0} is an ideal of 𝔰_{𝔨}^{c}, hence contained in 𝔰_{𝔥}^{0} . This proves that 𝔰_{𝔨}^{c} ⊆ i𝔭̂_{𝔰}, and the maximality of 𝔰_{𝔨}^{c} gives equality.

Since 𝔰 is τ -𝔞-invariant, we have 𝔤^{0} = 𝔯^{0} ⋊ 𝔰^{0} and hence 𝔥^{0} = 𝔯_{𝔥}^{0} ⋊ 𝔰_{𝔥}^{0} . From [𝔯_{𝔥}^{0}, 𝔰^{α}] ⊆ 𝔰^{α} ∩ 𝔯 = {0} and 𝔤^{α} = 𝔰^{α} for all α ∈ Δ_{s} (Corollary VII.9), it follows that 𝔯_{𝔥}^{0} commutes with 𝔰_{𝔮} and therefore with 𝔭_{𝔰}. Hence

𝔭̂ = (𝔯_{𝔥}^{0} ⊕ 𝔞_{𝔯}) ⊕ (𝔭̂ ∩ 𝔰_{ℂ})

is a direct sum of Lie algebras. We have already seen above that (i𝔭̂) ∩ 𝔰^{c} is maximal compactly embedded in 𝔰^{c}. Let 𝔨̃ ⊇ 𝔨^{c} be a maximal compactly embedded subalgebra containing i𝔭̂. Projecting onto 𝔰^{c} along 𝔯^{c} and using the maximality of 𝔨^{c} ∩ 𝔰^{c} proves that 𝔨̃ ⊆ 𝔯^{c} + (𝔨^{c} ∩ 𝔰^{c}), hence that

𝔨̃ = (𝔨̃ ∩ 𝔯^{c}) ⊕ (𝔨^{c} ∩ 𝔰^{c}).

Now the compactness of the Lie algebra 𝔨̃ shows that 𝔨̃∩ 𝔯^{c} is central in 𝔨̃, hence contained in 𝔷_{𝔯^{c}}(i𝔞) = 𝔯_{𝔥}^{0} ⊕ i𝔞_{𝔯} = 𝔨^{c} ∩ 𝔯^{c}. We conclude that 𝔨̃ = 𝔨^{c} and hence that 𝔨^{c} is maximal compactly embedded in 𝔤^{c}.

It remains to show that 𝔭̂ = 𝔷𝔮̂(𝔷(𝔭)). The construction of 𝔭̂ shows that 𝔷(𝔭) ⊆ 𝔷(𝔭̂) because 𝔷(𝔭) ⊆ 𝔞 commutes with 𝔥^{0} . The converse inclusion will be proved in (iii) below. (iii) From the construction of 𝔭̂ in (ii) and the definition of the compact roots, it follows that

𝔭̂_{ℂ} = 𝔨_{ℂ}^{c} = (𝔤^{0})_{ℂ} ⊕ ⨁_{α∈Δ̂_{k}}(𝔤^{α})_{ℂ}.

Therefore

Δ̂_{k} = {β ∈ Δ̂: β |_{𝔞} ∈ Δ_{k} ∪ {0}} and consequently Δ̂_{n} = {β ∈ Δ̂: β |_{𝔞} ∈ Δ_{n}}.

This shows in particular that Δ̂_{k} = {β ∈ Δ̂: β |_{𝔷(𝔭)} = 0} and hence that 𝔷_{𝔤_{ℂ}}(𝔷(𝔭)) = 𝔨_{ℂ}^{c}. From that we conclude in particular that

𝔷𝔮̂(𝔷(𝔭)) = 𝔭̂

which completes the proof of (ii).

Now it is easy to find a compatible 𝔭̂-adapted positive system in Δ̂. We simply choose X_{0} ∈ C_{max}^{0} and note that no non-compact root in Δ̂_{n} vanishes on X_{0} . Then we pick X_{1} ∈ C_{max}^{0} near to X_{0} such that no root in Δ vanishes on X_{1} and, in addition, all non-compact roots which are positive on X_{0} are still positive on X_{1} . Then we pick X_{2} ∈ 𝔞̂ near to X_{1} such that no root in Δ̂ vanishes on X_{2} and, in addition, all roots which are positive on X_{1} are still positive on X_{2} . We put Δ̂^{+} := {α ∈ Δ̂: α(X_{2}) > 0} and obtain a positive 𝔭̂-adapted system satisfying all the requirements if we define Δ^{+} = {α ∈ Δ: α(X_{1}) > 0}. Note in particular that −τ leaves Δ̂_{n}^{+} invariant because Δ̂_{n}^{+} = {α ∈ Δ: α(X_{0}) > 0} and −τ(X_{0}) = X_{0} . (iv) Let γ ∈ W and γ̃ ∈ N_{Inn_{𝔤}([𝔭,𝔭])}(𝔞) be an element with γ = γ̃ |_{𝔞} (Lemma III.6). Then γ̃.𝔥^{0} = 𝔥^{0} , and since γ̃.𝔱_{𝔥} is another compactly embedded Cartan subalgebra of 𝔥^{0} , there exists an inner automorphism σ ∈ Inn_{𝔤}(𝔥^{0}) with σγ̃.𝔱_{𝔥} = 𝔱_{𝔥}. Then σγ̃ |_{𝔞} = 𝔞 and σγ̃ normalizes 𝔞̂ = 𝔞 + i𝔱_{𝔥}. Therefore γ̂ := σγ̃ |𝔞̂ ∈ Ŵ satisfies γ̂ |_{𝔞} = γ.

(v) This follows from the 𝔞̂-invariance of (𝔤^{α})_{ℂ}. (vi) We start by proving the first equality in (a) and (b). Note that the finite group L := {1, −τ} operates on 𝔞̂ with fixed point set 𝔞 and leaves the convex sets Ĉ_{min} and Ĉ_{max} invariant because it preserves Δ̂_{n}^{+} . Hence the assertion follows from ([5, Prop. 1.6]). Now we establish the statements concerning C_{max} . The second equality in (b) follows from Δ̂_{n}^{+} |_{𝔞} = Δ_{n}^{+} . Let X ∈ C_{max}^{0} , which means that α(X) > 0 for all α ∈ Δ_{n}^{+} . Then β(X) > 0 for all β ∈ Δ̂_{n}^{+} and thus X ∈ Ĉ_{max}^{0} . This proves (c).

Next we show that p_{𝔞}(Ĉ_{min}) ⊆ C_{min} . Let α ∈ Δ_{n}^{+} , β ∈ Δ̂ with β |_{𝔞} = α, and X_{β} ∈ 𝔤_{ℂ}^{β} ⊆ (𝔤^{α})_{ℂ}. Write X_{β} = Y_{β} + iZ_{β}, where Y_{β}, Z_{β} ∈ 𝔤^{α}. Then

[X_{β}, τ̂(X_{β})] = [Y_{β} + iZ_{β}, τ(Y_{β}) − iτ(Z_{β})]

= \underbrace{[Y_{β}, τ(Y_{β})] + [Z_{β}, τ(Z_{β})]}_{∈C_{min}} + \underbrace{i([Z_{β}, τ(Y_{β})] + [τ(Z_{β}), Y_{β}])}_{∈i𝔱_{𝔥}} . (8.1)

Hence p_{𝔞}([X_{β}, τ̂.X_{β}]) ∈ C_{min} , which yields p_{𝔞}(Ĉ_{min}) ⊆ C_{min} .

Now we prove the converse inclusion C_{min} ⊆ Ĉ_{min} . Let α ∈ Δ_{n}^{+} , X_{α} ∈ 𝔤^{α} and write X_{α} = ∑_{β∈Δ̂_{α}} X_{β} with X_{β} ∈ 𝔤_{ℂ}^{β} according to (v). We compute

[X_{α}, τ(X_{α})] = [X_{α}, τ̂(X_{α})] = ∑_{β∈Δ̂_{α}}[X_{β}, τ̂(X_{β})] + ∑_{β≠γ∈Δ̂_{α}}[X_{β}, τ̂(X_{γ})].

Since the last summand above is contained in [𝔞̂, 𝔤_{ℂ}], it vanishes, and so the desired inclusion follows.

To see that (d) holds, let α ∈ Δ̂_{k}^{+} , β = α |_{𝔞}, and X_{α} = Y_{β} + iZ_{β} ∈ 𝔤_{ℂ}^{α}, where Y_{β}, Z_{β} ∈ 𝔤^{β} and α̌ = [τ̂.X_{α}, X_{α}]. As above we obtain

α̌ = [τ̂.X_{α}, X_{α}] ∈ i𝔱_{𝔥} + [τ.Y_{β}, Y_{β}] + [τ.Z_{β}, Z_{β}] ⊆ i𝔱_{𝔥} + ℝ^{+}β̌.

This proves (d). (vii) First we prove (c). Let X_{β} ∈ 𝔯_{ℂ}^{β}. Then the second term of (8.1) vanishes since [𝔫, 𝔫] ⊆ 𝔷(𝔤)_{𝔮} (Proposition VII.2(iii)), and thus

(8.2)  [X_{β}, τ̂(X_{β})] = [Y_{β}, τ(Y_{β})] + [Z_{β}, τ(Z_{β})] ∈ C_{min}.

So Ĉ_{min,z} = C_{min,z} and (c) is established.

Next we show that (𝔤_{ℂ}, τ̂) has cone potential, i.e.,

(8.3)  [X_{β}, τ̂(X_{β})] ≠ 0  for 0 ≠ X_{β} ∈ 𝔤_{ℂ}^{β}, β ∈ Δ̂.

We have already seen that (𝔤_{ℂ}, τ̂) is quasihermitian, so that Δ̂ splits (Proposition V.9(ii)). Hence (8.3) follows from formula (8.1) and the pointedness of C_{min} . As Ĉ_{min,z} = C_{min,z} is pointed, the cone potential of (𝔤_{ℂ}, τ̂) implies that it has in fact strong cone potential. Now Ĉ_{min} ⊆ Ĉ_{max} follows from Theorem VII.18(iii).

Remark VIII.2. The structure of the root decomposition of 𝔤_{ℂ} with respect to 𝔞̂ is in general much simpler than that of the root decomposition of 𝔤 with respect to 𝔞. This is mainly due to the fact that here we always have that

Δ̂_{r} = {α ∈ Δ̂: 𝔤_{ℂ}^{α} ⊆ 𝔯_{ℂ}}, Δ̂_{s} = {α ∈ Δ̂: 𝔤_{ℂ}^{α} ⊆ 𝔰_{ℂ}},

and

Δ̂_{k} = {α ∈ Δ̂: 𝔤_{ℂ}^{α} ⊆ 𝔨_{ℂ}^{c}}

(cf. [13, Sect. II] or [19, Ch. V]). We also note that in this case there exists a 𝔭̂-adapted positive system if and only if 𝔷_{𝔤^{c}}(𝔷(𝔨^{c})) = 𝔨^{c} which means that the Lie algebra 𝔤^{c} is quasihermitian. It is instructive to note that here the conditions (1) and (2) of Proposition V.10 are always satisfied so that the condition that (𝔤_{ℂ}, τ̂), resp. the Lie algebra 𝔤^{c}, is quasihermitian reduces to the existence of a 𝔭̂-adapted, resp. 𝔨^{c} -adapted, positive system.

Reduced symmetric Lie algebras

For ω ∈ 𝔮^{∗} we write O_{ω}: = Inn_{𝔤}(𝔥)^{∗}.ω for the coadjoint orbit of ω in 𝔮^{∗}. The next concept will be a crucial tool in the following (cf. [17, Def. III.9] for the group case).

Definition VIII.3. We fix a 𝔭-adapted positive system Δ^{+} and consider C_{min}^{⋆} as a cone in 𝔞^{∗} ⊂ 𝔤^{∗}. Then the largest ideal 𝔟 = 𝔟(Δ_{n}^{+}) in (C_{min}^{⋆})^{⊥} = {X ∈ 𝔤 : ⟨X, C_{min}^{⋆}⟩ = {0}} is called the associated ideal of degeneracy. We note that 𝔟 is automatically τ -invariant because −τ(C_{min}) = C_{min} .

Lemma VIII.4. If (𝔤, τ) is quasihermitian, Δ^{+} a 𝔭-adapted positive system, and 𝔟 ⊆ 𝔤 a τ -invariant ideal, then the following assertions hold:

(i) If π: 𝔤 → 𝔤/𝔟 is the quotient homomorphism, then π(𝔞) is a maximal hyperbolic abelian and maximal abelian subspace of π(𝔮), Δ_{1}^{+} := {α ∈ Δ^{+}: 𝔤^{α} ≠ 𝔟^{α}} is a 𝔭-adapted positive system with respect to π(𝔞), Δ_{k,1} = Δ_{k} ∩ Δ_{1} is the corresponding system of compact roots, and Δ_{n,1} := Δ_{n} ∩ Δ_{1} the corresponding system of non-compact roots. The corresponding minimal cone C_{min,1} coincides with \overline{π(C_{min})}. If, in addition, 𝔟 = 𝔟(Δ_{n}^{+}) is the ideal of degeneracy, then

(ii) 𝔟 = H(C_{min}) ⊕ 𝔟_{𝔥}^{0} ⊕ ⨁_{α∈Δ} 𝔟^{α}, where 𝔟^{α} ⊆ 𝔯^{α} and X ∈ 𝔟^{α} implies that [X, τ(X)] ∈ H(C_{min}),

(iii) π(C_{min}) is closed and pointed, and the ideal of degeneracy of 𝔤/𝔟 with respect to Δ_{1}^{+} is trivial.

Proof. (i) We write 𝔠_{1} := π(𝔠) for a subspace 𝔠 ⊆ 𝔤 and τ_{1} for the involution induced by τ on 𝔤_{1} = 𝔤/𝔟. It follows from Lemma III.11 that 𝔞_{1} is a maximal hyperbolic abelian subspace and maximal abelian in 𝔮_{1} , and that 𝔭_{1} is a maximal hyperbolic Lie triple system in 𝔮_{1} . From the decomposition 𝔭 = 𝔷(𝔭) ⊕ [𝔭, [𝔭, 𝔭]] we obtain 𝔭_{1} = 𝔷(𝔭)_{1} ⊕ [𝔭_{1}, [𝔭_{1}, 𝔭_{1}]] so that 𝔷(𝔭)_{1} ⊆ 𝔷(𝔭_{1}) implies that 𝔷(𝔭_{1}) = 𝔷(𝔭)_{1} . The fact that (𝔤, τ) is quasihermitian implies that 𝔮 = 𝔭 ⊕ [𝔷(𝔭), 𝔥] which gives 𝔮_{1} = 𝔭_{1} ⊕ [𝔷(𝔭_{1}), 𝔥_{1}], and therefore 𝔷_{𝔮_{1}}(𝔷(𝔭_{1})) = 𝔭_{1} follows from the fact that 𝔤_{1} is a semisimple 𝔷(𝔤_{1})-module. This proves that (𝔤_{1}, τ_{1}) is quasihermitian. Now the assertions on the root system are consequences of 𝔭_{1} = π(𝔭), and the 𝔭_{1} -adaptedness of Δ_{1}^{+} follows from the 𝔭-adaptedness of Δ^{+} . The formula describing the minimal cone follows from π(𝔤^{α}) = 𝔤_{1}^{α} for α ∈ Δ_{1} and from π ∘ τ = τ_{1} ∘ π. (ii) Since 𝔟 is an ideal, it is in particular invariant under 𝔞, hence decomposes according to the root space decomposition, i.e., 𝔟 = 𝔟^{0} ⊕ ⨁_{α∈Δ} 𝔟^{α}.

According to the definition of 𝔟 we have 𝔟∩𝔞 ⊆ (C_{min}^{⋆})^{⊥} ∩𝔞 = H(C_{min}). To see the converse, it suffices to prove that H(C_{min}) ⊆ 𝔷(𝔤) whenever (𝔤, τ) is quasihermitian. In fact, in Proposition V.9(vii) we have seen that C_{min,p} ⊆ (Δ_{p}^{+})^{⋆}. Let 𝔫 ⊆ 𝔤 denote the maximal nilpotent ideal. Then 𝔫 is τ -invariant, 𝔫 ∩ 𝔞 = 𝔷(𝔤)_{𝔮}, and (𝔤/𝔫, τ_{𝔤/𝔫}) is reductive and quasihermitian (Proposition V.9(v)). Now Proposition V.9(vii) shows that that the corresponding cone C_{min,1} ⊆ 𝔞/(𝔞 ∩ 𝔫) is pointed because only the irreducible components of type (NCC) contribute to C_{min} . We conclude that H(C_{min}) ⊆ 𝔞 ∩ 𝔫 ⊆ 𝔷(𝔤) and this proves that 𝔟 ∩ 𝔞 = H(C_{min}).

For X ∈ 𝔟^{α} we have

[X, τ.X] ∈ 𝔟 ∩ 𝔞 = H(C_{min}) ⊆ 𝔷(𝔤).

Thus we conclude from Proposition IV.7(v) that the form κ_{τ} is degenerate of all root spaces 𝔟^{α}. Since 𝔟 = (𝔟 ∩ 𝔯) + (𝔟 ∩ 𝔰) holds for any Levi complement 𝔰 ([1, Ch. 1, §6, no. 8, Cor. 4]) and 𝔟 ∩ 𝔰 is an ideal of 𝔰, we conclude that 𝔟^{α} ⊆ 𝔯^{α}. (iii) In view of (i), we have C_{min,1} = \overline{π(C_{min})} and therefore

C_{min,1}^{⋆} = \overline{π(C_{min})}^{⋆} = \overline{C_{min} + (𝔟 ∩ 𝔞)}^{⋆} = C_{min}^{⋆},

where we identify 𝔞_{1}^{∗} with the subspace (𝔟 ∩ 𝔞)^{⊥} of 𝔞^{∗}. Similarly we identify 𝔤_{1}^{∗} with the subspace 𝔟^{⊥} ⊆ 𝔤^{∗}. Then the definition of 𝔟 shows that

𝔟 = ( Inn_{𝔤}(𝔤)^{∗}.C_{min}^{⋆})^{⊥}

which is equivalent to the fact that

𝔤_{1}^{∗} ≅ 𝔟^{⊥} = span ( Inn_{𝔤}(𝔤)^{∗}.C_{min}^{⋆}) = span ( Inn_{𝔤_{1}}(𝔤_{1}).C̃_{min}^{⋆}),

and this shows that the ideal of degeneracy in 𝔤_{1} is trivial, hence in particular that C_{min,1} is pointed.

That π(C_{min}) is closed follows from the fact that 𝔟 ∩ C_{min} = H(C_{min}) is a vector space which implies that 𝔟 + C_{min} is a closed convex cone in 𝔤 (cf. [5, Prop. 1.4]).

For every α ∈ Δ we define 𝔯_{(A)}^{α}: = {Z ∈ 𝔯^{α} : [Z, τ(Z)] = 0}. Note that in general 𝔯_{(A)}^{α} is not a vector space (cf. Example IV.10(b)).

Lemma VIII.5. If C_{min} is pointed, then (i) the set 𝔯_{(A)}^{α} is a vector space, [𝔯_{(A)}^{α}, 𝔤^{−α}] ⊆ 𝔫_{𝔥}^{0} , and

(ii) [𝔯_{(A)}^{α}, 𝔯_{(A)}^{β}] ⊆ 𝔯_{(A)}^{α+β} if β ≠ −α.
(iii) If, in addition, [𝔫, 𝔫] ⊆ 𝔷(𝔤), then [𝔯_{(A)}^{α}, 𝔤^{β}] ⊆ 𝔯_{(A)}^{α+β} if β ≠ −α.

Proof. (i) Let X ∈ 𝔯_{(A)}^{α} and Y ∈ 𝔤^{α}. We consider the expression

[X + Y, τ(X) + τ(Y)] = [Y, τ(Y)] + ([X, τ(Y)] + [Y, τ(X)]).

We see that the right hand side of this equation is linear in X. Since the left hand side is contained in C_{min} , it follows from the pointedness of C_{min} that the expression in brackets must vanish. We conclude that τ([X, τ(Y)]) = [X, τ(Y)], which establishes the second assertion because [𝔤, 𝔯] ⊆ 𝔫. Finally, the first statement drops out if we choose Y ∈ 𝔯_{(A)}^{α}.

(ii) For X ∈ 𝔯_{(A)}^{α} and Y ∈ 𝔯_{(A)}^{β} we have

\begin{aligned}[[X, Y], [τ(X), τ(Y)]] & = [[[X, Y], τ(X)], τ(Y)] + [τ(X), [[X, Y], τ(Y)]] \\ & ∈ [𝔤^{β}, 𝔯_{(A)}^{−β}] + [𝔯_{(A)}^{−α}, 𝔤^{α}] ⊆ 𝔫_{𝔥}^{0}. \end{aligned}

Therefore the assertion follows from [[X, Y], [τ(X), τ(Y)]] ∈ 𝔥 ∩ 𝔞 = {0}. (iii) For X ∈ 𝔯_{(A)}^{α} and Y ∈ 𝔤^{β} we now have

\begin{aligned}[[X, Y], [τ(X), τ(Y)]] & = [[[X, Y], τ(X)], τ(Y)] + [τ(X), [[X, Y], τ(Y)]] \\ & ∈ [𝔷(𝔤), 𝔤^{−β}] + [𝔯_{(A)}^{−α}, 𝔤^{α}] ⊆ 𝔫_{𝔥}^{0}, \end{aligned}

so we can argue as in (ii).

We call (𝔤, τ) or, more precisely, (𝔤, τ, Δ_{n}^{+}) reduced if the associated ideal of degeneracy is trivial. For every element in ω ∈ 𝔞^{∗} we denote by 𝔡(ω) the largest ideal in ker ω. Note that 𝔡(ω) is τ -invariant. We write π for the quotient homomorphism from 𝔤 onto 𝔤/𝔡(ω). We call the Lie algebra 𝔤(ω) = π(𝔤) the strictly reduced Lie algebra associated to ω. Finally ω ∈ 𝔞^{∗} is called strictly reduced if 𝔡(ω) = {0}. Note that the τ -invariance of 𝔡(ω) implies that the symmetric structure is inherited by 𝔤(ω) and that 𝔞(ω): = π(𝔞) is a maximal hyperbolic and maximal abelian subspace in 𝔮(ω): = π(𝔮) (Lemma III.11).

Proposition VIII.6. For a quasihermitian symmetric Lie algebra (𝔤, τ) the following assertions hold:

(i) If (𝔤, τ) is strictly reduced for an element ω ∈ C_{min}^{⋆}, then (𝔤, τ) is reduced.

(ii) If (𝔤, τ) is reduced, then it is effective and has strong cone potential. In particular, [𝔫, 𝔫] ⊆ 𝔷(𝔤), 𝔫_{𝔥}^{0} = {0} and 𝔞 can be extended to a Cartan subalgebra 𝔱 such that 𝔱^{c} ⊆ 𝔤^{c} is a compactly embedded Cartan subalgebra.

Proof. (i) This is clear because the ideal of degeneracy 𝔟 is contained in ker ω for all ω ∈ C_{min}^{⋆}. (ii) Obviously (𝔤, τ) is effective. Suppose that (𝔤, τ) has cone potential. Then the pointedness of C_{min} implies strong cone potential and the remaining assertions follow from Proposition VII.12 and Corollary VII.13.

It remains to show that (𝔤, τ) has cone potential. First we show that [𝔫, 𝔫] ⊆ 𝔷(𝔤). Let k be the length of 𝔫 and assume that k > 2. According to [5, Lemma 7.13], the ideal 𝔧 := 𝔫^{k−1} is abelian but not central because 𝔫^{k} ≠ {0}. Since 𝔧 is 𝔞-invariant, it decomposes under the root decomposition as

𝔧 = 𝔧^{0} ⊕ ⨁_{α∈Δ} 𝔧^{α}.

Note that 𝔧^{α} ⊆ 𝔯_{(A)}^{α} because 𝔧 is abelian. We claim that

𝔧′ = 𝔧_{𝔥}^{0} ⊕ ⨁_{α∈Δ} 𝔧^{α}

is a τ -invariant ideal of 𝔤. In fact, it is clear that 𝔧′ is invariant under 𝔤^{0} = 𝔞⊕𝔥^{0} , and the invariance under the root spaces 𝔤^{α} follows from Lemma VIII.5(i). From 𝔧′ ⊂ (C_{min}^{⋆})^{⊥} and the reducedness of (𝔤, τ, Δ_{n}^{+}) we now conclude that 𝔧′ = {0}, i.e., 𝔧 = 𝔧_{𝔮}^{0} = 𝔧 ∩ 𝔞 ⊆ 𝔷(𝔤). This contradiction shows that k ≤ 2 and [𝔫, 𝔫] ⊆ 𝔷(𝔫). If 𝔧 := [𝔫, 𝔫] is not contained in 𝔷(𝔤), we can argue as above and get a contradiction. Thus we have shown that [𝔫, 𝔫] ⊆ 𝔷(𝔤).

Let γ ∈ int C_{min}^{⋆} and note that such a functional exists because C_{min} is pointed which in turn follows from the assumption that (𝔤, τ) is reduced (cf. Lemma VIII.4). Now the form

ψ^{γ}: 𝔫 × 𝔫 → ℝ, (X, Y) ↦ γ([X, τ.Y])

is τ -covariant and semidefinite on each root space (cf. proof of Proposition VII.7). Therefore 𝔫^{⊥} := 𝔫^{⊥_{ψ^{γ}}} is an ideal of 𝔤. Moreover we have 𝔫^{⊥} ∩ 𝔫^{α} = 𝔫_{(A)}^{α}. On the other hand 𝔫^{0} = 𝔫_{𝔥}^{0} + 𝔫_{𝔮}^{0} = 𝔫_{𝔥}^{0} + 𝔷(𝔤)_{𝔮}, so that [𝔫^{0}, 𝔫^{0}] ⊆ 𝔫_{𝔥}^{0} ⊆ ker γ. We conclude that

𝔫^{⊥} = 𝔫^{0} + ∑_{α∈Δ} 𝔫_{(A)}^{α}

is an ideal of 𝔤. By the same argument as above, we see that 𝔫_{𝔥}^{0} + ∑_{α∈Δ} 𝔫_{(A)}^{α} is also an ideal, which has to vanish because (𝔤, τ, Δ_{n}^{+}) is reduced. Therefore 𝔫_{(A)}^{α} = {0} for all α ∈ Δ.

Let Z ∈ 𝔤^{α} be of type (A). Since Δ is split, Proposition IV.7(vi) implies that Z ∈ 𝔯_{(A)}^{α} = {0}. This proves that (𝔤, τ) has cone potential.

In the remainder of this subsection we give some information about the structure of C_{min} needed later on. We are in particular interested in the case where C_{min} coincides with the convex hull of the set {[X_{α}, τ(X_{α})]: X_{α} ∈ 𝔤^{α}, α ∈ Δ_{n}^{+}}. As has been shown in [17, Ex. III.8], this is not always the case. But, as the next lemma shows, the situation is rather well behaved if C_{min} is pointed.

Lemma VIII.7. If C_{min} is pointed, then

C_{min} = ∑_{𝔤^{α}=𝔰^{α}} ℝ^{+}α̌ + ∑_{𝔤^{α}≠𝔰^{α}} C_{α},

where C_{α}: = conv({[X_{α}, τ(X_{α})]: X_{α} ∈ 𝔤^{α}}). If B_{α} ⊆ 𝔤^{α} is a 0-neighborhood, then

conv({[X, τ(X)]: X ∈ B_{α}})

is a 0-neighborhood in C_{α}.

Proof. This can be proved in the same way as Lemma III.7 in [17]. One only has to replace the expressions i[X, \overline{X}] by [X, τ(X)].

The convexity theorems Theorem VIII.8. (Convexity Theorem, coadjoint version) Let (𝔤, τ) be a quasihermitian symmetric Lie algebra, Δ^{+} a 𝔭-adapted positive system and suppose that 𝔤 is strictly reduced for ω ∈ C_{min}^{⋆}. Then

p_{𝔞^{∗}}(O_{ω}) = conv(W.ω) + cone(Δ_{ω}^{+}) ⊆ conv(W.ω) + cone(Δ_{n}^{+}),

where Δ_{ω}^{+} = {α ∈ Δ_{n}^{+} : (∃γ ∈ W)(∃X_{α} ∈ 𝔤^{α}) ⟨ω, γ.[X_{α}, τ(X_{α})]⟩ < 0}.

Proof. To use [7, Cor. 5.25], we note that the assumption about the pointedness of C_{min}^{⋆} in this corollary is superfluous. According to Proposition VIII.6(ii), the assumptions of [7, Cor. 5.25] are satisfied and we get the asserted formula.

Corollary VIII.9. If (𝔤, τ) is quasihermitian and ω ∈ C_{min}^{⋆}, then

p_{𝔞^{∗}}(O_{ω}) ⊆ conv(W.ω) + cone(Δ_{n}^{+}).

Proof. By the isomorphism 𝔤(ω)^{∗} ≅ 𝔡(ω)^{⊥} ⊆ 𝔤^{∗} we realize the coadjoint orbit O_{ω} in 𝔤(ω)^{∗}. Under this identification we have Δ(ω) ⊆ Δ, where Δ(ω) denotes the root system w.r.t. 𝔞(ω) induced by Δ, 𝔤(ω) is quasihermitian, and Δ(ω)^{+} := Δ^{+} ∩Δ(ω) is 𝔭-adapted (Lemma VIII.4(i)). Now the inclusion follows from Theorem VIII.8.

Theorem VIII.10. (Convexity Theorem, adjoint version) Let Δ^{+} be a 𝔭adapted positive system with C_{min} ⊆ C_{max} . Then for X ∈ C_{max}^{0} and the adjoint orbit O_{X} = Inn_{𝔤}(𝔥).X the following formula holds:

\overline{conv (O_{X})} ∩ 𝔞 = p_{𝔞}(\overline{conv(O_{X})}) = conv(W.X) + C_{min}.

Proof. (cf. [17, Th. 3.12]) Since the cone

C := {X ∈ C_{max} : (∀α ∈ Δ_{k}^{+}) α(X) ≥ 0}

is a fundamental domain for the Weyl group action (cf. [19, Prop. III.2.7(i)]), we may assume that X ∈ C.

We will prove the chain of inclusions

\overline{conv (O_{X})} ∩ 𝔞 ⊆ p_{𝔞}(\overline{conv(O_{X})}) ⊆ conv(W.X) + C_{min} ⊆ \overline{conv (O_{X})} ∩ 𝔞,

where the first inclusion is trivial. Thus we start by proving the second inclusion.

Let ω ∈ C_{min}^{⋆} be such that ω(α̌) ≤ 0 for all α ∈ Δ_{k}^{+} . Then Corollary VIII.9 implies that

p_{𝔞^{∗}}(O_{ω}) ⊆ conv(W.ω) + cone(Δ_{n}^{+}).

Therefore

⟨ω, O_{X}⟩ = ⟨O_{ω}, X⟩ = ⟨p_{𝔞^{∗}}(O_{ω}), X⟩
⊆ conv(⟨W.ω, X⟩) + ℝ^{+} = conv(⟨ω, W.X⟩) + ℝ^{+}.

From W.X ⊆ X − cone(Δ̌_{k}^{+}) ([19, Prop. III.2.7(ii)]) we now get ⟨ω, O_{X}⟩ ⊆ ω(X) + ℝ^{+}. This means that

p_{𝔞}(O_{X}) − X ⊆ (C_{min}^{⋆} ∩ (−Δ̌_{k}^{+})^{⋆})^{⋆} = \overline{C_{min} − cone(Δ̌_{k}^{+})} = C_{min} − cone(Δ̌_{k}^{+}),

since C_{min} − cone(Δ̌_{k}^{+}) is closed (cf. [19, Lemma III.2.15]). Thus p_{𝔞}(O_{X}) ⊆ X + C_{min} − cone(Δ̌_{k}^{+}) and therefore the closedness of the right hand side yields

p_{𝔞}(\overline{conv(O_{X})}) ⊆ ⋂_{γ∈W} γ.(X + C_{min} − cone(Δ̌_{k}^{+})) = conv(W.X) + C_{min}

([19, Cor. III.2.10]).

To complete the proof, we show that conv(W.X)+C_{min} ⊆ \overline{conv(O_{X})}∩𝔞. Let α ∈ Δ_{n}^{+} and Y ∈ 𝔤^{α} be a non-zero element. Then Lemma VI.1 shows that

p_{𝔞}(e^{ℝ ad(Y +τ.Y)}.X) = X + α(X)ℝ^{+}[Y, τ.Y] = X + ℝ^{+}[Y, τ.Y]

because α(X) > 0. Let F := conv({[X_{α}, τ.X_{α}]: α ∈ Δ_{n}^{+}, X_{α} ∈ 𝔤^{α}}) and note that this is a dense subcone of C_{min} . Now by Proposition VI.3(iii) X + F ⊆ conv(O_{X}) and since O_{X} is W -invariant, we conclude that

conv(W.X) + F = conv (W.(X + F)) + F = conv (W.(X + F)) ⊆ conv(O_{X}) ∩ 𝔞.

As a consequence, we obtain conv(W.X)+C_{min} ⊆ \overline{conv(O_{X})}∩𝔞. This completes the proof.

IX. Existence of hyperbolic invariant convex cones

Up to this section we have only considered consequences of the existence of invariant hyperbolic convex sets in 𝔮. In this section we will use the convexity theorems of Section VIII to prove that hyperbolic invariant convex cones exist in 𝔮 if and only if (𝔤, τ) is quasihermitian and C_{min} ⊆ C_{max} holds for a 𝔭-adapted positive system Δ^{+} . The latter condition is crucial for the applicability of the convexity theorems and we have already seen in Theorem VI.6 that it is necessary. Since there are only finitely many possibilities for positive systems, this condition has the advantage that it can be checked quite easily by computing the cones C_{min} and C_{max} which usually is quite easy because it reduces to calculating the brackets [X, τ.X], X ∈ 𝔤^{α}.

Before we turn to the maximal cones, we need some preliminaries on invariant convex sets.

Invariant convex sets and exposed points

In this subsection V denotes a finite dimensional real vector space.

Lemma IX.1. Let F ⊆ V be a non-empty closed subset and C := \overline{conv(F)}. If H(C) = {0}, then there exists for each f ∈ B(C)^{0} an x ∈ F with f(x) = min f(C). Proof. If f ∈ B(C)^{0} , then f |_{C} is a proper function ([7, Cor. 1.13]). Since F ⊆ C is closed, the function f |_{F} is also proper. Now the fact that it is bounded from below implies that there exists an x ∈ F with f(x) = min f(F). Finally the assertion follows from f(C) ⊆ [min f(F), ∞[.

We recall that a point x in a closed convex set C ⊆ V is called exposed if there exists a linear functional f ∈ B(C) with {x} = {y ∈ C: f(y) = min f(C)}. We write Exp(C) for the set of exposed points of C. A convex function ϕ on a convex subset Ω of V is said to be strictly convex if it is not affine on any non-trivial line segment, i.e., if x ≠ y, x, y ∈ Ω and λ ∈]0, 1[ implies that

f(λx + (1 − λ)y) < λf(x) + (1 − λ)f(y).

Proposition IX.2. Let Ω ⊆ V be an open convex set, ϕ: Ω → ℝ be a strictly convex function with ϕ(x) → ∞ whenever x → x_{0} ∈ ∂Ω, and Ø ≠ F ⊆ Ω a closed subset of V such that ϕ|_{F} is constant. Then

Exp (\overline{conv(F)}) = F.

Proof. Since Ω is open, the function ϕ on Ω is continuous and the fact that it tends to infinity at the boundary implies that the sets Ω_{c} := {x ∈ C: ϕ(x) ≤ c} are closed in V . Let ϕ(F) = {c_{0}}. Then the convexity of ϕ implies that conv(F) ⊆ Ω_{c_{0}} , hence that

D := \overline{conv(F)} ⊆ Ω_{c_{0}} ⊆ Ω.

If x ∈ D, then x+H(D) ⊆ D. Therefore the convex function ϕ is bounded from above on this affine subspace and therefore constant. Now the strict convexity of ϕ implies that H(D) = {0}.

Let x ∈ Exp(D) and f ∈ B(D) with {x} = {y ∈ D: f(y) = min f(D)}. The closed convex set D contains the cone x + lim(D) with vertex x. Therefore f ∈ lim(D)^{⋆} and 0 is the unique minimum of f in lim(D). This proves that f ∈ int lim(D)^{⋆}, hence, by [7, Lemma 1.9], that f ∈ B(D)^{0} . Now Lemma IX.1 shows that there exists y ∈ F with f(y) = min f(D), hence that x = y ∈ F.

It remains to show that F ⊆ Exp(D). So let x ∈ F and f ∈ V^{∗} a subgradient of ϕ in x, i.e., ϕ(x) + f(y − x) ≤ ϕ(y) for all y ∈ Ω. For the existence of such functionals we refer to [19, Lemma III.3.16]. For y ∈ D \ {x} we then have

f(y) ≤ f(x) + ϕ(y) − ϕ(x) = f(x) + ϕ(y) − c_{0} ≤ f(x).

If f(y) = f(x), then ϕ(x) = ϕ(y), so that ϕ(x) = min ϕ(D) and the strict convexity of ϕ imply that x = y. Hence

{x} = {y ∈ D: −f(y) = min ( − f(D))}

shows that x ∈ Exp(D). This proves that Exp(D) = F.

Remark IX.3. Note that under the assumptions of Proposition IX.2 the set conv(F) is in general not closed. Let V = ℝ^{2} , Ω =]0, ∞[^{2} , ϕ(x, y) = \frac{1}{xy}, and F = \{(n, \frac{1}{n}): n ∈ ℕ\}. Then the points (1 + t, 1), t > 0 are contained in the closure of conv(F) but not in conv(F).

Now we draw a general conclusion from Proposition IX.2.

Theorem IX.4. Let H be a group and π: H → Sl(V) a representation with closed image. Furthermore let Ω ⊆ V be an open convex subset with H(\overline{Ω}) = \{0\} which is H -invariant. Then for all x ∈ Ω the orbit H.x is closed and

Exp (\overline{conv(H.x)}) = H.x.

Proof. Using [19, Th. III.5.4], we see that the characteristic function ϕ_{Ω} of Ω defined by

ϕ_{Ω}(x) := ∫_{B(Ω)} e^{−α(x)+inf α(Ω)}dμ_{V^{∗}}(α),

where μ_{V^{∗}} denotes Lebesgue measure on V^{∗}, has the following properties: (1) ϕ_{Ω} is invariant under H because π(H) ⊆ Sl(V).

(2) If x_{n} → x ∈ ∂Ω, then ϕ_{Ω}(x_{n}) → ∞.

(3) ϕ_{Ω} is strictly convex.

Next we show that the orbit H.x is closed. We consider the vector space V^{♯} := V × ℝ, the cone C := cone(Ω × {1}) and the action of H given by h.(v, t) := (h.v, t). Then C is a pointed closed convex cone in V^{♯} which is invariant under the action of H. Therefore the closedness of the orbit H.x follows from [5, Prop. 1.12].

Since the characteristic function ϕ_{Ω} is H -invariant, it is constant on H.x, and the assertion now follows from Proposition IX.2.

Hyperbolicity of the maximal cone

Throughout this section (𝔤, τ) denotes a quasihermitian symmetric Lie algebra, 𝔭 ⊆ 𝔮 a maximal hyperbolic Lie triple system, 𝔞 ⊆ 𝔭 a maximal abelian subspace, and Δ^{+} a 𝔭-adapted positive system.

We define

W_{max} := ⋂_{h∈Inn_{𝔤}(𝔥)} h.p_{𝔞}^{−1}(C_{max}) = {X ∈ 𝔮: p_{𝔞}(O_{X}) ⊆ C_{max}}.

Lemma IX.5. If C_{min} ⊆ C_{max} , then W_{max} is a generating closed convex invariant cone in 𝔮 with p_{𝔞}(W_{max}) = W_{max} ∩ 𝔞 = C_{max} , and 𝔫_{𝔮} ⊆ W_{max} .

Proof. As an intersection of closed convex cones, the cone W_{max} is closed and convex, and the invariance follows from the definition. Further it is clear that

W_{max} ∩ 𝔞 ⊆ p_{𝔞}(W_{max}) ⊆ C_{max}.

For X ∈ C_{max}^{0} the adjoint version of the Convexity Theorem (Theorem VIII.10) shows that

p_{𝔞}(O_{X}) ⊆ conv(W.X) + C_{min} ⊆ C_{max} + C_{min} ⊆ C_{max}

because C_{min} ⊆ C_{max} . Hence C_{max}^{0} ⊆ W_{max} ∩ 𝔞, and C_{max} ⊆ W_{max} ∩ 𝔞 follows from the closedness of W_{max} . This proves the stated equality. That W_{max} has interior points follows from Lemma VI.5(ii).

To see that 𝔫_{𝔮} ⊆ W_{max} , let X ∈ 𝔫_{𝔮}. Then O_{X} ⊆ 𝔫_{𝔮} and therefore p_{𝔞}(O_{X}) ⊆ p_{𝔞}(𝔫_{𝔮}) ⊆ 𝔫_{𝔮} ∩ 𝔞 ⊆ 𝔷(𝔤)_{𝔮} = H(C_{max}). This proves that X ∈ W_{max} , whence 𝔫_{𝔮} ⊆ W_{max} .

Now the main problem is to show that the cone W_{max} is in fact hyperbolic. Using that W_{max} contains 𝔫_{𝔮}, we will reduce this question to the case where (𝔤, τ) is irreducible. We start with this case.

Proposition IX.6. If (𝔤, τ) is irreducible and quasihermitian, then the cone W_{max} is hyperbolic.

Proof. According to the remark after Definition V.5, we have to consider two cases. If (𝔤, τ) is (NCR), then Δ_{n} = Ø, C_{max} = 𝔞 and therefore 𝔭 = 𝔮 = W_{max} , showing that W_{max} is hyperbolic.

Now we assume that (𝔤, τ) is (NCC). First we show that W_{max} is pointed. We know already that W_{max} is generating and different from 𝔮 because C_{max} ≠ 𝔞. If W_{max} is not pointed, then H(W_{max}) is a non-zero 𝔥-submodule of 𝔮 and Lemma II.7(ii)(b) implies that H(W_{max}) is isotropic with respect to the Cartan-Killing form κ. Since κ is positive definite on 𝔞 (Proposition IV.7(ii)), we conclude from H(C_{max}) ⊆ H(W_{max}) that C_{max} is pointed. Therefore p_{𝔞}(H(W_{max})) ⊆ H(C_{max}) = {0} shows that H(W_{max}) is an 𝔥-submodule of 𝔞^{⊥_{κ}} = [𝔞, 𝔥]. Eventually the fact that (𝔤, τ) has cone potential (Proposition V.9(viii)) and Proposition VII.2(iv) entail that H(W_{max}) = {0}, i.e., that W_{max} is pointed.

The fact that W_{max} is pointed and generating implies that the dual cone W_{max}^{⋆} ⊆ 𝔮^{∗} is also pointed and generating. Let θ be a Cartan involution of 𝔤 commuting with τ. Then 𝔥 is θ-invariant and therefore reductive. For X ∈ 𝔥 we therefore have

tr ad_{𝔮} X = tr ad X − tr ad_{𝔥} X = 0 − 0 = 0

and thus Inn_{𝔮}(𝔥)^{∗} ⊆ Sl(𝔮^{∗}). To see that Inn_{𝔮}(𝔥) ⊆ Gl(𝔮) is closed, we note that 𝔤 = 𝔮_{L} = 𝔮 + [𝔮, 𝔮], so that the non-closedness of Inn_{𝔮}(𝔥) would imply that Inn_{𝔤}(𝔥) is not closed. But Inn_{𝔤}(𝔤) = Aut(𝔤)_{0} is closed and therefore Inn_{𝔤}(𝔥) = Aut(𝔤)_{0}^{τ} is closed.

Now we can apply Theorem IX.4 with V = 𝔮^{∗}, H = Inn_{𝔤}(𝔥), and Ω = int W_{max}^{⋆}, and find that for each f ∈ int W_{max}^{⋆} the coadjoint orbit O_{f} = Inn_{𝔤}(𝔥)^{∗}.f is closed and satisfies

Exp (\overline{conv(O_{f})}) = O_{f}.

From the definition of W_{max} we also conclude that C_{max}^{⋆} ⊆ W_{max}^{⋆} ∩ 𝔞^{∗} because X ∈ W_{max} and f ∈ C_{max}^{⋆} implies that f(X) = ⟨f, p_{𝔞}(X)⟩ ≥ 0. Using the Inn_{𝔤}(𝔥)-equivariant isomorphism ψ: 𝔮 → 𝔮^{∗} given by ψ(X)(Y) = κ(X, Y) obtained by the Cartan-Killing form, we see that we can use Proposition III.2 to show that for each element f ∈ int C_{max}^{⋆} with 𝔷_{𝔮}(ψ^{−1}(f)) = 𝔞 we have f ∈ int W_{max}^{⋆}.

Let f ∈ int C_{max}^{⋆} be such an element and X′ ∈ W_{max}^{0} . Then Lemma IX.1 implies that there exists an element f′ ∈ O_{f} with

⟨f′, X′⟩ = min⟨\overline{conv(O_{f})}, X′⟩.

Write f′ = γ.f with γ ∈ Inn_{𝔤}(𝔥) and put X := γ^{−1}.X′. Then

⟨f, X⟩ = ⟨f′, X′⟩ = min⟨\overline{conv(O_{f})}, X⟩ = min⟨O_{f}, X⟩ = min⟨f, O_{X}⟩.

Therefore f([𝔥, X]) = {0} and thus

{0} = κ(ψ^{−1}(f), [𝔥, X]) = κ([X, ψ^{−1}(f)], 𝔥).

Since κ is non-degenerate on 𝔥, this means that [X, ψ^{−1}(f)] = {0}, i.e., X ∈ 𝔞 by the choice of f. This proves that X′ ∈ O_{X} is hyperbolic and therefore that W_{max}^{0} ⊆ 𝔮_{hyp} .

Corollary IX.7. If (𝔤, τ) is reductive and quasihermitian, then the cone W_{max} is hyperbolic.

Proof. The reductive symmetric Lie algebra (𝔤, τ) decomposes as a direct sum of quasihermitian irreducible symmetric Lie algebras (Proposition V.9(v)) and so do the cones C_{max} and W_{max} . Therefore the assertion follows from Proposition IX.6.

To pave the way from the reductive to the general case we will need the following result on solvable symmetric Lie algebras.

Proposition IX.8. Let (𝔯, τ) be a solvable symmetric Lie algebra, X_{1}, X_{2} ∈ 𝔯_{𝔮} regular elements and 𝔱_{j} := 𝔯^{0}(ad X_{j}) the corresponding Cartan subalgebras. Then 𝔱_{1} and 𝔱_{2} are conjugate under Inn_{𝔤}(𝔯_{𝔥}).

Proof. We prove the proposition by induction over the dimension of 𝔯. Let 𝔷 := 𝔷(𝔫) denote the center of the nilradical 𝔫 of 𝔯. If 𝔯 ≠ {0}, then 𝔷 is a non-zero abelian ideal which is invariant under τ. Let 𝔯_{1} := 𝔯/𝔷 and write π: 𝔯 → 𝔯_{1} for the quotient homomorphism. Then π(X_{j}) ∈ 𝔮_{1} := π(𝔯_{𝔮}) are regular elements in the Lie algebra 𝔤_{1} ([1, Ch. 7, §2, no. 2, Prop. 8]) and π(𝔱_{j}) = π(𝔯^{0}(ad X_{j})) = 𝔯_{1}^{0}(ad π(X_{j})). Hence our induction hypothesis implies the existence of γ ∈ Inn_{𝔯_{1}}(𝔯_{𝔥,1}) with γ.π(𝔱_{1}) = π(𝔱_{2}). Let γ̃ ∈ Inn_{𝔯}(𝔯_{𝔥}) with π ∘ γ̃ = γ ∘ π. Then π(γ̃.𝔱_{1}) = π(𝔱_{2}) shows that γ̃.𝔱_{1} ⊆ 𝔱_{2} + 𝔷. From now on we may therefore w.l.o.g. assume that 𝔱_{1} ⊆ 𝔱_{2} + 𝔷. Now 𝔱_{1} and 𝔱_{2} are τ -invariant Cartan subalgebras of the solvable Lie algebra 𝔱_{2} + 𝔷. Hence there exists X ∈ 𝔷 with e^{ad X}.𝔱_{1} = 𝔱_{2} ([1, Ch. 7, §3, no. 4, Th. 3]). The fact that the ideal 𝔷 is abelian implies that (ad X)^{2} = {0}. Therefore e^{ad X} = 1 + ad X and if we write X = X_{𝔥} + X_{𝔮} with X_{𝔥} ∈ 𝔷_{𝔥} and X_{𝔮} ∈ 𝔷_{𝔮}, we obtain

e^{ad X}.X_{1} = \underbrace{X_{1} + [X_{𝔥}, X_{1}]}_{∈𝔯_{𝔮}} + \underbrace{[X_{𝔮}, X_{1}]}_{∈𝔯_{𝔥}} ∈ e^{ad X}.𝔱_{1} = 𝔱_{2} = (𝔱_{2})_{𝔥} ⊕ (𝔱_{2})_{𝔮}.

We conclude that e^{ad X_{𝔥}}.X_{1} ∈ 𝔱_{2} and hence that 𝔱_{2} = 𝔯^{0}( ad(e^{ad X_{𝔥}}.X_{1})) = e^{ad X_{𝔥}}.𝔯^{0}(ad X_{1}) = e^{ad X_{𝔥}}.𝔱_{1}. Thus 𝔱_{1} and 𝔱_{2} are conjugate under Inn_{𝔯}(𝔯_{𝔥}).

Theorem IX.9. If (𝔤, τ) is a quasihermitian symmetric Lie algebra and Δ^{+} a 𝔭-adapted positive system such that C_{min} ⊆ C_{max} , then the cone W_{max} is hyperbolic.

Proof. Let 𝔫 denote the nilradical of 𝔤, 𝔤_{1} := 𝔤/𝔫, and π: 𝔤 → 𝔤_{1} the canonical quotient map. Then Lemma VIII.4(i) implies that (𝔤_{1}, τ_{1}) is quasihermitian with root system \{α ∈ Δ: 𝔤^{α} ≠ 𝔫^{α}\} = Δ_{s}. Moreover C_{min,1} = \overline{π(C_{min})} ⊆ C_{max,1} follows from C_{min} + (𝔞 ∩ 𝔫) ⊆ C_{max} ⊆ C_{max,1} . From C_{max} ⊆ C_{max,1} we also conclude that π(W_{max}) ⊆ W_{max,1} and hence that π(W_{max}^{0}) ⊆ W_{max,1}^{0} ⊆ 𝔮_{hyp,1} (Corollary IX.7).

Let X ∈ W_{max}^{0} . We have to show that X is hyperbolic, i.e., that it is conjugate to an element in 𝔞. The preceding observations show that there exists γ ∈ Inn_{𝔤_{1}}(𝔥_{1}) with γ.π(X) ∈ 𝔞_{1} . Choosing γ̃ ∈ Inn_{𝔤}(𝔥) with π ∘ γ̃ = γ ∘ π, we obtain π(γ̃.X) ∈ 𝔞_{1} , hence that γ̃.X ∈ π^{−1}(𝔞) ∩ 𝔮 = 𝔞 + 𝔫_{𝔮}, and we may w.l.o.g. assume that X ∈ 𝔞 + 𝔫_{𝔮}.

Let 𝔡 := 𝔫 + 𝔞. Then 𝔡 is τ -invariant so that (𝔡, τ |_{𝔡}) is a solvable symmetric Lie algebra. Since the fact that 𝔤 is quasihermitian implies that Δ is split (Proposition V.9(ii)), the compact roots do not contribute to 𝔫, and we obtain the root decomposition

𝔡 = 𝔫_{𝔥}^{0} ⊕ 𝔞 ⊕ ⨁_{α∈Δ_{n}} 𝔫^{α}.

An element Y ∈ 𝔡 is regular if and only if the dimension of 𝔡^{0}(ad Y) = ker(ad Y)_{s}, where (ad Y)_{s} denote the semisimple part in the Jordan decomposition of ad Y , is minimal. For Y ∈ 𝔞 and Z ∈ 𝔫 the fact that 𝔫 is a nilpotent ideal implies that the eigenvalues and their multiplicities for ( ad(Y + Z))_{s} are the same as for (ad Y)_{s} = ad Y . Hence Y + Z is regular if and only if Y is regular which in turn is equivalent to α(Y) ≠ 0 for all α ∈ Δ_{n} with 𝔫^{α} ≠ {0}.

Our element X ∈ 𝔞 + 𝔫_{𝔮} from above satisfies p_{𝔞}(X) ∈ C_{max}^{0} and X − p_{𝔞}(X) ∈ 𝔫. Hence the fact that α(p_{𝔞}(X)) > 0 for all α ∈ Δ_{n}^{+} and the preceding remark show that X and p_{𝔞}(X) are regular elements in 𝔡. Now Proposition IX.8 implies the existence of γ ∈ Inn_{𝔤}(𝔫_{𝔥}) with

γ.X ∈ 𝔡^{0}( ad p_{𝔞}(X)) ∩ 𝔮 = 𝔷_{𝔡_{𝔮}}(p_{𝔞}(X)) = 𝔷_{𝔡_{𝔮}}(𝔞) = 𝔞.

This completes the proof.

Corollary IX.10. If (𝔤, τ) is a symmetric Lie algebra, then 𝔮 contains hyperbolic invariant convex cones if and only if (𝔤, τ) is quasihermitian and there exists a 𝔭-adapted positive system Δ^{+} such that C_{min} ⊆ C_{max} .

If this condition is satisfied, then the cone W_{max} is hyperbolic and each invariant hyperbolic cone W ⊆ 𝔮 is contained in a unique cone W_{max} , where Δ_{n}^{+} is determined by W ∩ 𝔞 ⊆ C_{max} . Proof. The first part follows by combining Theorem IX.9 with Theorem VI.6(ii).

For the second part it only remains to apply Theorem VI.6(ii) to see that there exists a 𝔭-adapted positive system with W ∩ 𝔞 ⊆ C_{max} . Then W = \overline{Inn_{𝔤}(𝔥).(W ∩ 𝔞)} ⊆ \overline{Inn_{𝔤}(𝔥).C_{max}} ⊆ W_{max} and it is clear that the fact that W ∩ 𝔞 has interior points determines Δ_{n}^{+} and therefore W_{max} uniquely.

Problems IX. Is it true that for a solvable symmetric Lie algebra (𝔤, τ) all τ -invariant Cartan subalgebras 𝔱 are conjugate under Inn_{𝔤}(𝔥)?

X. Characterization of invariant convex hyperbolic sets

In the preceding section we have seen whenever (𝔤, τ) is a quasihermitian symmetric Lie algebra and Δ^{+} is a 𝔭-adapted positive system such that C_{min} ⊆ C_{max} , the cone W_{max} is hyperbolic. Throughout this section we will make these assumptions.

The characterization of invariant convex sets

Theorem X.1. For X ∈ W_{max}^{0} we have (i) \overline{conv(O_{X})} is contained in W_{max}^{0} .

(ii) \overline{conv(O_{X})} = \{Y ∈ W_{max}^{0}: p_{𝔞}(O_{Y}) ⊆ conv(W_{𝔨}.X) + C_{min}\}.

(iii) If the cone C_{min} is pointed or if C_{min} = conv{[X_{α}, τ.X_{α}]: α ∈ Δ_{n}^{+}, X_{α} ∈ 𝔤^{α}}, then conv(O_{X}) is closed. Proof. (cf. [17, Th. III.12]) (i) Let ϕ: W_{max}^{0} → ℝ^{+} denote the characteristic function of W_{max} , i.e.,

ϕ(X) = ∫_{W_{max}^{⋆}} e^{−α(X)} dμ(α),

where μ is the restriction of a Lebesgue measure in the subspace W_{max}^{⋆} − W_{max}^{⋆}. In view of [19, Th. III.5.4], this function has the following properties:

(1) ϕ(γ.x) = χ(γ)ϕ(x) for γ ∈ Inn_{𝔤}(𝔥), where χ(γ) = det_{𝔮/H(W_{max})}(γ)^{−1} .

(2) If x_{n} → x ∈ ∂W_{max} , then ϕ(x_{n}) → ∞.

(3) ϕ is a convex function.

To strengthen (1), we show that the character χ of Inn_{𝔤}(𝔥) is trivial. Since 𝔫_{𝔮} ⊆ H(W_{max}) (Lemma IX.5), this reduces the problem to the corresponding assertion on the Lie algebra 𝔤_{1} := 𝔤/𝔫 which is reductive. Then 𝔥_{1} ⊆ 𝔤_{1} is a reductive subalgebra and hence ad_{𝔮}(𝔥) ⊆ 𝔰𝔩(𝔮) follows as in the proof of Proposition IX.6. This proves that χ is trivial, hence that the function ϕ is invariant under Inn_{𝔤}(𝔥).

Now the convexity of ϕ implies that

conv(O_{X}) ⊆ {Y ∈ W_{max}^{0}: ϕ(Y) ≤ ϕ(X)}.

According to the continuity of ϕ and (2), the set {Y ∈ W_{max}^{0}: ϕ(Y) ≤ ϕ(X)} is closed in 𝔤, and therefore contains \overline{conv(O_{X})}. (ii) According to (i), both sets are invariant and contained in W_{max}^{0} . Hence it suffices to check that their intersection with 𝔞 is the same. This is a consequence of Theorem VIII.10. (iii) If C_{min} is pointed, then Lemma VIII.7 shows that

C_{min} = F := conv({[X, τ.X]: X ∈ 𝔤^{α}, α ∈ Δ_{n}^{+}}).

Suppose that F = C_{min} . Since O_{X} meets 𝔞, we may w.l.o.g. assume that X ∈ 𝔞. Then we have seen in the proof of Theorem VIII.10 that

\overline{conv(O_{X})} ∩ 𝔞 = conv(W.X) + C_{min} = conv(W.X) + F ⊆ conv(O_{X}).

In view of (i), this proves that conv(O_{X}) is closed.

Theorem X.2. (Characterization Theorem) Let (𝔤, τ) be a quasihermitian symmetric Lie algebra and Δ^{+} a 𝔭-adapted positive system with C_{min} ⊆ C_{max} .

(i) If C ⊆ W_{max}^{0} is an invariant subset, then (2)⇒(1) holds for the following statements:

(1) C is convex.

(2) C_{𝔞} := C ∩ 𝔞 is convex and C_{𝔞} + C_{min} ⊆ C_{𝔞}. Furthermore, if is either C is closed or open or if C_{min} is pointed, then

also (1)⇒(2).

(ii) If C_{𝔞} ⊆ C_{max}^{0} is a convex W -invariant subset satisfying C_{𝔞} +C_{min} ⊆ C_{𝔞}, then C: = Inn_{𝔤}(𝔥).C_{𝔞} is an invariant convex subset of W_{max}^{0} with C ∩ 𝔞 = p_{𝔞}(C) = C_{𝔞}.

Proof. (i) (cf. [17, Prop. III.14]) (2)⇒(1): Since C is invariant, the convex set C_{𝔞} is in particular invariant under the Weyl group W. Moreover, (2) and Theorem VIII.10 imply for each X ∈ C_{𝔞} that

p_{𝔞}(O_{X}) ⊆ C_{𝔞} + C_{min} ⊆ C_{𝔞}

and therefore that

p_{𝔞}(C) = p_{𝔞}( Inn_{𝔤}(𝔥).C_{𝔞}) ⊆ C_{𝔞}.

Since C_{𝔞} is convex, it even follows that

conv(C) ∩ 𝔞 ⊆ p_{𝔞}( conv(C)) = conv p_{𝔞}(C) ⊆ C_{𝔞}.

So the invariance of conv(C) ⊆ W_{max}^{0} and the hyperbolicity of W_{max} (Theorem IX.9) yield

conv(C) = Inn_{𝔤}(𝔥).( conv(C) ∩ 𝔞) ⊆ Inn_{𝔤}(𝔥).C_{𝔞} = C.

This means that C is convex.

(1) ⇒ (2): If C is convex, then C_{𝔞} is trivially convex. Let X ∈ C_{𝔞} ⊆ C. Then conv(O_{X}) ⊆ C and therefore X + C_{min} ⊆ \overline{conv(O_{X})} ∩ 𝔞 ⊆ \overline{C_{𝔞}}. Hence C_{min} ⊆ lim(\overline{C_{𝔞}}) follows from Theorem VIII.10. If either C is closed, open, or C_{min} is pointed, then Lemma VIII.7 and Theorem X.1 show that C_{𝔞}+C_{min} ⊆ C_{𝔞}, where we have used that if C is closed or open, then lim C = lim \overline{C} is closed. (ii) First we note that O_{X} ∩ 𝔞 = W.X (Theorem III.10) and the W -invariance of C_{𝔞} show that C ∩ 𝔞 = C_{𝔞}. Thus (i) implies that C is convex, and the proof of (i) implies that p_{𝔞}(C) ⊆ C_{𝔞}. Therefore C_{𝔞} ⊆ C ∩ 𝔞 ⊆ p_{𝔞}(C) ⊆ C_{𝔞} and the equality follows.

Theorem X.3. (Characterization and Reconstruction of Invariant Cones) Let (𝔤, τ) be a symmetric Lie algebra.

(i) There is a pointed generating invariant hyperbolic closed convex cone in 𝔮 if and only if

(1) (𝔤, τ) has strong cone potential,

(2) is quasihermitian, and

(3) 𝔷(𝔭) ≠ {0} holds for a maximal hyperbolic Lie triple system

𝔭 ⊆ 𝔮. (ii) Suppose that (𝔤, τ) is quasihermitian and Δ^{+} is a 𝔭-adapted positive

system with C_{min} ⊆ C_{max} . A closed convex subset C_{𝔞} ⊆ C_{max} with interior points arises as the trace C ∩ 𝔞 of a closed convex hyperbolic set C ⊆ 𝔮 if and only if C_{𝔞} is W -invariant and C_{min} ⊆ lim(C_{𝔞}). Suppose that (1)-(3) in (i) are satisfied.

(iii) A pointed generating closed convex cone C_{𝔞} ⊆ 𝔞 arises as the trace C ∩ 𝔞 of a pointed generating invariant closed convex hyperbolic cone C ⊆ 𝔮 if and only if C_{𝔞} is W -invariant and there exists a 𝔭-adapted positive system Δ^{+} such that C_{min} ⊆ C ⊆ C_{max}.

Proof. (i) The necessity of (1)-(3) follows from Theorem VI.6. To prove the sufficiency, let us assume that (1)-(3) are satisfied. We are going to construct a cone with the desired properties. Let Δ^{+} be a 𝔭-adapted positive system. According to Theorem VII.18(iii), we have C_{min} ⊆ C_{max} .

We claim that there exists a pointed generating W -invariant closed convex cone C_{𝔞} ⊆ 𝔞 satisfying C_{min} ⊆ C_{𝔞} ⊆ C_{max} .

Let 0 ≠ X ∈ 𝔷(𝔭) ∩ C_{max}^{0} (Proposition V.4(3)) and K ⊆ C_{max}^{0} a W - invariant compact convex neighborhood of X not containing 0. Then ℝ^{+}K is a pointed generating W -invariant closed convex cone in 𝔞. We put C_{𝔞} := C_{min} + ℝ^{+}K and show that C_{𝔞} satisfies all our requirements:

First we note that the cone

C_{min} ∩ −ℝ^{+}K ⊆ H(C_{max}) ∩ −ℝ^{+}K

is {0}. In fact, if C_{max} ≠ 𝔞 this follows from H(C_{max}) ∩ −ℝ^{+}K = {0}, and if C_{max} = 𝔞, then Δ_{n} = Ø, hence C_{min} = {0} and it is also clear. Now [5, Prop. 1.4] implies that C is closed with H(C_{𝔞}) = H(C_{min}) + H(ℝ^{+}K) = {0} + {0} = {0}. Since K has interior points, the same holds for C_{𝔞}, and the W -invariance of ℝ^{+}K implies the W -invariance of C_{𝔞}. Moreover, C_{min} ⊆ C_{𝔞} ⊆ C_{max} holds by construction. Now the assertion follows from (iii). (ii) If C ⊆ 𝔮 is an invariant hyperbolic convex set, then clearly C_{𝔞} = C ∩ 𝔞 is W -invariant, and C_{min} ⊆ lim(C_{𝔞}) follows from Theorem VI.6 because the fact that C_{𝔞} has interior points determines the cone C_{max} containing C_{𝔞} uniquely.

Suppose that these conditions are satisfied by C_{𝔞} ⊆ C_{max} and put C: = \overline{Inn_{𝔤}(𝔥).C_{𝔞}}. That C is generating follows from Lemma VI.5(ii), and since closures of convex sets are convex, we obtain from Theorem X.2 and C = \overline{Inn_{𝔤}(𝔥).C_{𝔞}^{0}} that C is hyperbolic and convex. Moreover,

C_{𝔞} ⊆ C ∩ 𝔞 ⊆ p_{𝔞}(C) ⊆ \overline{p_{𝔞}( Inn_{𝔤}(𝔥).C_{𝔞}^{0})} ⊆ \overline{C_{𝔞}^{0}} = C_{𝔞}

follows from Theorem X.2(ii). This proves that C_{𝔞} = C ∩ 𝔞. (iii) As noted in (i), the necessity of the condition follows from Theorem VI.6(ii). If, conversely, they are satisfied, then (ii) implies the existence of a hyperbolic closed convex invariant subset C ⊆ 𝔮 with C ∩ 𝔞 = C_{𝔞}. The hyperbolicity of C implies that Inn_{𝔤}(𝔥).C_{𝔞} is dense in C, hence that C is a cone. So it remains to show that C is pointed. As p_{𝔞}(H(C)) ⊆ H(C) ∩ C_{𝔞} = {0}, we conclude that H(C) ⊆ [𝔞, 𝔥]. Since, according to Proposition VII.2(iv), [𝔞, 𝔥] contains no non-zero 𝔥-submodule, H(C) = {0} follows.

Extension of convex invariant hyperbolic sets

In this section we deal with the problem of extending hyperbolic invariant convex sets in 𝔮 to hyperbolic invariant convex sets in 𝔮̂ = i𝔤^{c}. First we have a look at some crucial examples.

Example X.4. (a) We consider the five dimensional Heisenberg algebra

𝔥_{2} = ℝZ ⊕ ℝQ_{1} ⊕ ℝP_{1} ⊕ ℝQ_{2} ⊕ ℝP_{2}

with non-zero brackets [Q_{1}, P_{1}] = [Q_{2}, P_{2}] = Z. Let mot(2) = V ⋊ ℝU, where V ≅ ℝ^{2} and U as in Example IV.11, be the motion algebra of the euclidean plane. We define a homomorphism of 𝔩 = mot(2) ⊕ ℝT into der(𝔥_{2}) by

V.𝔥_{2} = {0}, U.Q_{1} = Q_{2}, U.Q_{2} = −Q_{1}, U.P_{1} = P_{2}, U.P_{2} = −P_{1}

and T.Q_{i} = P_{i}, T.P_{i} = Q_{i} for i ∈ {1, 2}. This turns 𝔤 = 𝔥_{2} ⋊ 𝔩 into a Lie algebra and

𝔥 = span{P_{1}, P_{2}} ⋊ mot(2) and 𝔮 = span{Z, Q_{1}, Q_{2}, T }

defines an involutive automorphism τ on 𝔤 with τ |_{𝔥} = id_{𝔥} and τ |_{𝔮} = − id_{𝔮}. A maximal hyperbolic abelian subspace which is also maximal abelian in 𝔮 is given by

𝔞 = ℝZ ⊕ ℝT.

We have Δ = Δ_{r} = {±α} with α(T) = 1 and 𝔤^{α} = ℝ(P_{1} + Q_{1}) + ℝ(P_{2} + Q_{2}). Therefore (𝔤, τ) has strong cone potential, 𝔭 = 𝔞 is a maximal hyperbolic Lie triple system, so that the fact that 𝔞 is maximal abelian in 𝔮 implies that (𝔤, τ) is quasihermitian. Therefore 𝔮 admits invariant hyperbolic cones (Theorem X.3) and 𝔞 extends to a maximal hyperbolic abelian and maximal abelian subspace 𝔞̂ ⊆ 𝔮̂ (Theorem VIII.1), namely,

𝔞̂ = ℝZ ⊕ ℝT ⊕ ℝiU.

Also (𝔤_{ℂ}, τ̂) is quasihermitian, but does not contain any pointed generating invariant hyperbolic cone since it has no cone potential (Theorem X.3). In particular, it is impossible to extend an invariant pointed generating hyperbolic convex cone C ⊆ 𝔮 to an invariant pointed generating hyperbolic convex cone in 𝔮̂. We also note that 𝔥^{0} ≅ 𝔪𝔬𝔱(2) is not compactly embedded, but it has a compactly embedded Cartan subalgebra.

This situation changes drastically if we factor out the ideal V ⊆ 𝔤 and obtain the effective Lie algebra (𝔤_{0}, τ_{0}): = (𝔤/V, τ̃), where τ̃ is the canonical involution induced on the quotient. Now any invariant pointed generating hyperbolic convex cone in 𝔮_{0} can be extended to an invariant pointed generating hyperbolic convex cone in 𝔮̂ (cf. Theorem X.7 below).

(b) (cf. [17, Ex. II.9(c)]) Let

𝔰_{1}: = 𝔰𝔩(2, ℝ): = span{H, T, U} and 𝔰_{2}: = 𝔰𝔩(2, ℝ): = span{H′, T′, U′}

as in the notation of Example IV.10. We denote by V_{j}, j ∈ {1, 2} the 2dimensional real 𝔰_{j} -module and let P = (1, 1), Q = (1, −1) ∈ V_{1} . Now we define

𝔤 = (V_{1} ⊗ V_{2}) ⋊ (𝔰_{1} ⊕ 𝔰_{2}).

and equip 𝔤 with an involution τ on 𝔤 via

𝔥 = (ℝP ⊗ V_{2}) ⋊ (ℝT ⊕ 𝔰_{2}) and 𝔮 = (ℝQ ⊗ V_{2}) ⊕ span{H, U}.

A maximal hyperbolic abelian and maximal abelian subspace 𝔞 in 𝔮 is given by 𝔞 = ℝH. Note that 𝔞 is a maximal hyperbolic Lie triple system and thus (𝔤, τ) is quasihermitian. We can extend 𝔞 to a maximal hyperbolic abelian subspace 𝔞̂ ⊆ 𝔮̂, which is also maximal abelian in 𝔮, by

𝔞̂ = ℝH ⊕ ℝiU′.

Since 𝔞̂ is a maximal hyperbolic Lie triple system in 𝔮̂ we see that (𝔤_{ℂ}, τ̂) is quasihermitian. Note also that (𝔤, τ) and (𝔤_{ℂ}, τ̂) are effective. Let α ∈ 𝔞^{∗} be given by α(H) = 1 and extend α to an element of 𝔞̂^{∗} by setting α(iU) = 0.

Further we define β ∈ 𝔞̂^{∗} by β(iU′) = 1 and β(H) = 0. Then the root systems are given by

Δ = \{\underbrace{±α}_{Δ_{r}}, \underbrace{±2α}_{Δ_{s}}\}  and  Δ̂ = \{\underbrace{±α ± β}_{Δ̂_{r}}, \underbrace{±2α, ±2β}_{Δ̂_{s}}\}.

Hence for any positive system Δ^{+} we always have C_{min} ⊆ C_{max} but, as it has already been shown in [17, Ex.II.9(c)], there is no positive system Δ̂^{+} such that Ĉ_{min} ⊆ Ĉ_{max} . By Theorem VI.6 this means that there are no generating invariant hyperbolic convex sets in 𝔮̂. In particular, we cannot extend any invariant generating hyperbolic convex set in 𝔮 to an invariant hyperbolic convex set in 𝔮̂.

The first example above shows that for the sake of extension theorems it is reasonable to assume that (𝔤, τ) is effective. Note that passing to the effective quotient algebra touches in no way all the properties of hyperbolic sets nor changes the root structure and the minimal and maximal cone.

From now on we assume that the subalgebra 𝔥^{0} is compactly embedded so that Theorem VIII.1 applies to (𝔤_{ℂ}, τ̂).

Proposition X.5. We consider the subspaces 𝔞 ⊆ 𝔞̂ and the corresponding Weyl groups W and Ŵ. Let further Δ_{k}^{+} and Δ̂_{k}^{+} be positive systems with Δ̂_{k}^{+} |_{𝔞} ⊆ Δ_{k}^{+}∪{0}, and 𝔞^{+} = (Δ_{k}^{+})^{⋆}, resp. 𝔞̂^{+} = (Δ̂_{k}^{+})^{⋆}, denote the corresponding fundamental domains for W, resp. Ŵ. Then the following assertions hold:

(i) p_{𝔞}(𝔞̂^{+}) = 𝔞̂^{+} ∩ 𝔞 = 𝔞^{+} .

(ii) If Ĉ ⊆ 𝔞̂ is a Ŵ -invariant set and Ĉ^{+} := Ĉ ∩ 𝔞^{+} is convex, then

p_{𝔞}(conv Ĉ) = conv (W.p_{𝔞}(Ĉ^{+})).

(iii) For X ∈ 𝔞̂^{+} we have p_{𝔞}(conv Ŵ.X) = conv (W.p_{𝔞}(X)).

(iv) Let C ⊆ 𝔞 be a W -invariant convex set. Then

Ĉ := {X ∈ 𝔞̂: p_{𝔞}(Ŵ.X) ⊆ C}

is a Ŵ -invariant convex subset of 𝔞̂ with Ĉ ∩ 𝔞 = p_{𝔞}(Ĉ) = C. If C has interior points, then the same holds for Ĉ.

Proof. (i) From the compatibility of the positive systems Δ_{k}^{+} and Δ̂_{k}^{+} it follows that 𝔞^{+} = 𝔞̂^{+} ∩ 𝔞 ⊆ p_{𝔞}(𝔞̂^{+}). Let X ∈ 𝔞̂^{+} . Then p_{𝔞}(X) = \frac{1}{2}(X − τ.X). If α ∈ Δ_{k}^{+} , then there exists α̃ ∈ Δ̂_{k} with α̃|_{𝔞} = α. Moreover α̃ must be positive because otherwise α̃ |_{𝔞} ∈ Δ_{k}^{−} ∪ {0}, and since −τ.α has the same restriction to 𝔞, this root must also be positive. We conclude that

2⟨α, p_{𝔞}(X)⟩ = ⟨α, X − τ.X⟩ = ⟨α − τ.α, X⟩ ≥ 0.

Hence p_{𝔞}(𝔞̂^{+}) ⊆ 𝔞^{+} , and (i) follows. (ii) Let D := conv Ĉ. Then D is a Ŵ -invariant convex set. Since for each element w ∈ W there exists ŵ ∈ Ŵ with ŵ |_{𝔞} = w and ŵ commutes with τ |𝔞̂ (Theorem VIII.1(iv)), the set p_{𝔞}(D) is convex and invariant under W. It follows in particular that “⊇” holds.

On the other hand [19, Prop. III.2.9] implies that D ⊆ Ĉ^{+} − cone( ˇΔ̂_{k}^{+})

so that p_{𝔞}( cone( ˇΔ̂_{k}^{+})) = cone(Δ̌_{k}^{+}) (Theorem VIII.1(vi)(d)) implies that

p_{𝔞}(D) ⊆ p_{𝔞}(Ĉ^{+}) − p_{𝔞}( cone( ˇΔ̂_{k}^{+})) = p_{𝔞}(Ĉ^{+}) − cone(Δ̌_{k}^{+}),

where (i) implies that p_{𝔞}(Ĉ^{+}) ⊆ 𝔞^{+} . Now the W -invariance of p_{𝔞}(D) shows that

p_{𝔞}(D) ⊆ ⋂_{w∈W} w.(p_{𝔞}(Ĉ^{+}) − cone(Δ̌_{k}^{+})) = conv (W.p_{𝔞}(Ĉ^{+}))

(cf. [19, Prop. III.2.9]). This completes the proof of (ii). (iii) This is a special case of (ii). (iv) The inclusions Ĉ ∩ 𝔞 ⊆ p_{𝔞}(Ĉ) ⊆ C are trivial consequences of the definition of Ĉ. Let X ∈ C. Then there exists X′ ∈ 𝔞^{+} with X ∈ W.X′. Now (iii) implies that

p_{𝔞}(Ŵ.X) = p_{𝔞}(Ŵ.X′) ⊆ conv(W.X′) ⊆ C.

Therefore C ⊆ Ĉ ∩ 𝔞, and thus C = Ĉ ∩ 𝔞 = p_{𝔞}(Ĉ).

If, in addition, C has interior points, then C^{0} contains a fixed point X_{0} for W, i.e., α(X) = 0 holds for all α ∈ Δ_{k}. Then we also have α(X) = 0 for all α ∈ Δ̂_{k}, i.e. X is fixed by Ŵ. Now it is clear that Ĉ contains a sufficiently small neighborhood of X, so that Ĉ has interior points.

Let G^{c} be a connected Lie group with Lie algebra 𝔤^{c} and suppose that τ integrates to an involution of G^{c}, which is also denoted by τ. Further we denote by H^{τ} the group of τ -fixed points and by H_{0}^{τ} its identity component. Let H ⊆ G^{c} be any subgroup of G such that

H_{0}^{τ} ⊆ H ⊆ H^{τ} .

For an element X ∈ 𝔮 we define O_{X}^{H} = Ad(H).X and O_{X}^{G^{c}} = Ad(G^{c}).X.

Theorem X.6. Suppose that 𝔥^{0} is compactly embedded, (𝔤, τ) is quasihermitian, Δ^{+} is a 𝔭-adapted positive system, and Δ̂^{+} a compatible 𝔭̂-adapted positive system satisfying Ĉ_{min} ⊆ Ĉ_{max} . Then for X ∈ C_{max}^{0} the following assertions hold:

(10.1) \overline{conv(O_{X}^{G^{c}})} ∩ 𝔞̂ = p𝔞̂(\overline{conv(O_{X}^{G^{c}})}) = conv(Ŵ.X) + Ĉ_{min}

(10.2) \overline{conv(O_{X}^{H})} ∩ 𝔞 = p_{𝔞}(\overline{conv(O_{X}^{H})}) = conv(W.X) + C_{min}.

In particular, p_{𝔞}(\overline{conv(O_{X}^{H})}) = p_{𝔞}(\overline{conv(O_{X})}) is independent of the choice of H. Moreover

p_{𝔞}(\overline{conv(O_{X}^{G^{c}})}) = \overline{conv(O_{X}^{G^{c}})} ∩ 𝔞 = \overline{conv(O_{X}^{H})} ∩ 𝔞

(10.3) = p_{𝔞}(\overline{conv(O_{X}^{H})}) = conv(W.X) + C_{min}.

Proof. It follows from Theorem VIII.1(vi)(c) that X ∈ Ĉ_{max}^{0} . Equation (10.2) for H = H_{0} is obtained from Theorem VIII.10. Since Ĉ_{min} ⊆ Ĉ_{max} by assumption, we can apply Theorem VIII.10 also to the canonical extension (𝔤_{ℂ}, τ̂) and obtain (10.1). Using Theorem VIII.1(vi)(a) and Proposition X.5(iii), we get

conv(W.X) + C_{min} = p_{𝔞}(\overline{conv(O_{X}^{H_{0}})}) ⊆ p_{𝔞}(\overline{conv(O_{X}^{H})})
⊆ p_{𝔞}(\overline{conv(O_{X}^{G^{c}})}) ⊆ p_{𝔞}( conv(Ŵ.X) + Ĉ_{min})
= p_{𝔞}( conv(Ŵ.X)) + p_{𝔞}(Ĉ_{min})
= conv(W.X) + C_{min}.

This proves (10.2) for arbitrary H and also (10.3).

Theorem X.7. (The Extension Theorem) We keep the assumptions from Theorem X.6 and write Ŵ_{max}: = \overline{Ad(G^{c}).Ĉ_{max}} for the maximal cone in 𝔮̂ = i𝔤^{c}.

(i) Every Inn_{𝔤}(𝔥)-invariant closed hyperbolic convex set C ⊆ W_{max} is H -invariant and can be extended by

Ĉ: = {X ∈ Ŵ_{max}: p_{𝔞}(O_{X}^{G^{c}}) ⊆ C ∩ 𝔞}

to a {G^{c}, −τ}-invariant hyperbolic convex subset of Ŵ_{max} . The extension Ĉ is maximal with respect to

(10.4)  Ĉ ∩ 𝔮 = p_{𝔮}(Ĉ) = C.

(ii) If, in addition, C is a pointed generating invariant hyperbolic cone and (𝔤, τ) is effective, then Ĉ is a pointed generating invariant hyperbolic cone.

Proof. (i) From Ĉ = Ŵ_{max} ∩ ⋂_{g∈G^{c}} g.p_{𝔞}^{−1}(C ∩ 𝔞) it follows that Ĉ is a closed convex {G^{c}, −τ}-invariant subset of Ŵ_{max} . Moreover (10.3) in Theorem X.6 implies that for X ∈ C^{0} ∩ 𝔞 we have

p_{𝔞}(O_{X}^{G^{c}}) ⊆ \overline{conv(O_{X}^{H_{0}})} ∩ 𝔞 ⊆ C ∩ 𝔞,

hence that C^{0} ∩ 𝔞 ⊆ Ĉ, and the hyperbolicity of C entails that C ⊆ Ĉ ∩ 𝔮. On the other hand the −τ -invariance entails Ĉ ∩ 𝔮 = p_{𝔮}(Ĉ), so it remains to show that Ĉ is hyperbolic and that Ĉ ∩ 𝔮 ⊆ C.

We put C_{𝔞} := C ∩ 𝔞 and Ĉ_{𝔞} := {X ∈ Ĉ_{max}: p_{𝔞}(Ŵ.X) ⊆ C_{𝔞}}. Then Proposition X.5(iv) shows that p_{𝔞}(Ĉ_{𝔞}) = Ĉ_{𝔞} ∩ 𝔞 = C_{𝔞} and that Ĉ_{𝔞} has interior points.

For X ∈ Ĉ_{𝔞}^{0} we now conclude with Theorem X.6 that

p_{𝔞}(O_{X}^{G^{c}}) = conv(W.X) + C_{min} ⊆ C_{𝔞} + C_{min} ⊆ C_{𝔞}.

This proves that Ĉ_{𝔞}^{0} and therefore that Ĉ_{𝔞} is contained in Ĉ ∩ 𝔞̂. The converse inclusion follows from Ŵ.X ⊆ O_{X}^{G^{c}}, and thus we obtain Ĉ ∩𝔞̂ = Ĉ_{𝔞}. This proves that Ĉ has interior points (Lemma VI.5(ii)), and so Ĉ is hyperbolic.

To see that Ĉ ∩ 𝔮 ⊆ C, we note that the set on the left hand side is hyperbolic because (Ĉ ∩ 𝔮)^{0} = Ĉ^{0} ∩ 𝔮 ⊆ Ŵ_{max}^{0} . On the other hand it is Inn_{𝔤}(𝔥)- invariant, so that it suffices to show that Ĉ ∩ 𝔞 ⊆ C_{𝔞} which trivially follows from the definition of Ĉ. This proves (10.4). Finally, as Ĉ is H -invariant and p_{𝔮} commutes with the action of H, we conclude that C is also H -invariant. (ii) Let C be a pointed generating invariant hyperbolic cone in 𝔮. Then Ĉ is a generating invariant hyperbolic cone in 𝔮̂, and it remains to show that Ĉ is pointed. Since C is pointed, p_{𝔮}(H(Ĉ)) = {0}, so that H(Ĉ) is a 𝔤^{c} -invariant subspace of i𝔥 ⊆ 𝔮̂. This means that iH(Ĉ) is an ideal of 𝔤 contained in 𝔥. Now the assumption that (𝔤, τ) is effective shows that H(Ĉ) = {0}, i.e., that Ĉ is pointed.

Remark X.8. In the preceding two theorems we have needed the assumptions that 𝔥^{0} is compactly embedded to apply Theorem VIII.1 and that Ĉ_{min} ⊆ Ĉ_{max} to apply the convexity theorems to 𝔮̂ itself. Moreover, to extend pointed cones in 𝔮 to pointed cones in 𝔮̂, the effectiveness was a crucial condition.

Suppose that (𝔤, τ) is semisimple and quasihermitian. Then (𝔤_{ℂ}, τ̂) also is semisimple and quasihermitian (Theorem VIII.1), and Proposition V.9(vii) shows that Ĉ_{min} ⊆ Ĉ_{max} is automatically satisfied. In this case the effectiveness is also not a severe restriction because each ideal is a direct summand.

Problems X. (1) Does the requirement that X ∈ 𝔮̂ satisfies p_{𝔞}(O_{X}^{G^{c}}) ⊆ C_{max} imply that X ∈ Ŵ_{max} ? In this case it would not be necessary in the Extension Theorem to take the intersection with Ŵ_{max} .

(2) Does the assumption that 𝔥^{0} is compactly embedded (cf. Theorem VIII.1) and C_{min} ⊆ C_{max} imply that Ĉ_{min} ⊆ Ĉ_{max} . Note that in Example X.4(b) the subalgebra 𝔥^{0} ≅ 𝔰𝔩(2, ℝ) is not compactly embedded.

References

[1] Bourbaki, N., Groupes et algèbres de Lie, Chapitres 1 – 8, Masson, Paris, 1990.

[2] Faraut, J., J. Hilgert, G. Ólafsson, Spherical functions on ordered symmetric spaces, Ann. Inst. Fourier 44(1994), 927–966.

[3] Helgason, S., “Differential geometry, Lie groups, and symmetric spaces”, Acad. Press, London, 1978.

[4] Hilgert, J., and K. H. Hofmann, Compactly embedded Cartan Algebras and Invariant Cones in Lie Algebras, Advances in Math. 75(1989), 168– 201.

[5] Hilgert, J., and K.-H. Neeb, “Lie semigroups and their Applications”, Lecture Notes in Math. 1552, Springer, 1993.

[6] —, Spherical functions on Ol’shanskiĭ spaces, Journal of Funct. Anal., to appear.

[7] Hilgert, J., K. - H. Neeb, and W. Plank, Symplectic convexity theorems and coadjoint orbits, Comp. Math. 94(1994), 129–180.

[8] Hilgert, J., Ólafsson, G., “Causal Symmetric Spaces, Geometry and Harmonic Analysis”, Acad. Press, to appear.

[9] Hilgert, J., Olafsson, G., and B. Ørsted, Hardy Spaces on Affine Symmetric Spaces, J. reine angew. Math. 415(1991), 189–218.

[10] Hochschild, G. P., “Basic Theory of Algebraic Groups and Lie Algebras”, Springer, Graduate Texts in Mathematics 75, 1981.

[11] Kobayashi, S., and K. Nomizu, “Foundations of Differential Geometry II”, Interscience Publishers, 1969.

[12] Krötz, B., “Plancherel-Formel für Hardy-Räume”, Diplomarbeit, Universität Erlangen, 1995.

[13] Neeb, K.–H., Invariant subsemigroups of Lie groups, Memoirs of the AMS 499, 1993.

[14] —, The classification of Lie algebras with invariant cones, Journal of Lie Theory 4:2(1994), 1–46.

[15] —, Holomorphic representation theory II, Acta math. 173:1(1994), 103– 133.

[16] —, Holomorphic representations of Ol’shanskiĭ semigroups, in ”Semigroups in Algebra, Geometry and Analysis”, K. H. Hofmann et al., eds., de Gruyter, 1995.

[17] —, Invariant convex sets and functions in Lie algebras, Preprint 12, Institut Mittag-Leffler, 1995-96.

[18] —, On the complex and convex geometry of Ol’shanskiĭ semigroups, Preprint 10, Institut Mittag-Leffler, 1995-96.

[19]  —, ”Holomorphy and Convexity in Lie Theory”, book in preparation.

[20] Neidhardt, C., Konvexität und projektive Einbettungen in der Riemannschen Geometrie, Dissertation, Universität Erlangen, 1996.

[21] Warner, G., “Harmonic analysis on semisimple Lie groups I”, Springer, Berlin, Heidelberg, New York, 1972.

B. Krötz, K. - H. Neeb Mathematisches Institut Universität Erlangen-Nürnberg Bismarckstr. 1\frac{1}{2} D-91054 Erlangen Germany

Received May 5, 1996 and in final form June 11, 1996