Seminar Sophus Lie 2 (1992) 135–151

Near-Cartan algebras and groups

Karl H. Hofmann

0. Preliminary results on nilpotent algebras

We let 𝔥 denote a real or complex nilpotent Lie algebra and V a finite dimensional complex 𝔥-module. For X ∈ 𝔥 let X_{V} = (v ↦ X·v): V → V . We set 𝔥_{ℂ} = 𝔥 if the ground field K is ℂ and let 𝔥_{ℂ} denote the complexification of 𝔥 if K = ℝ. Recall that a linear form λ: 𝔥_{ℂ} → ℂ is a weight if there is a nonzero v ∈ V such that for some natural number n we have (X_{V} − λ(X)·1_{V})^{n}v = 0 for all X ∈ 𝔥. The element v is called a weight-vector for λ, and V^{λ} is the set of all weight vectors for λ. We let V_{λ} = {v ∈ V : (∀X ∈ 𝔥) X·v = λ(X)·v}. Since V_{λ} ≠ {0} we have a 0 ≠ v ∈ V_{λ} so that λ([X, Y])·v = [X, Y]·v = X_{V} Y_{V} (v) − Y_{V} X_{V} (v) = λ(X)λ(Y)v − λ(Y)λ(X)v = 0. Hence λ([𝔥, 𝔥]) = {0}.

We let Λ denote the (finite!) set of nonzero weights and V = V^{0} ⊕ ⨁_{λ∈Λ} V^{λ} the weight decomposition of V . We set Λ^{⊥} = {X ∈ 𝔥 : (∀λ ∈ Λ)λ(X) = 0}.

Lemma 0.1. Λ^{⊥} = {X ∈ 𝔥 : (∃n)X_{V}^{n} = 0} = {X ∈ 𝔥 : Spec X_{V} = {0}} = ⋂_{λ∈Λ} ker λ|𝔥. In particular, Λ^{⊥} contains [𝔥, 𝔥] and thus is an ideal of 𝔥.

Proof. We have X ∈ Λ^{⊥} iff for all λ ∈ Λ and all v ∈ V^{λ} we have X_{V}^{n}(v) = 0 for some n iff X_{V}^{n} = 0 for some n since V = ⨁_{λ∈Λ} V^{λ}. This is the case iff Spec X_{V} = {0}. The remainder is immediate from the definition and the preceding remarks.

Suppose that α is an automorphism of 𝔥_{ℂ} and φ ∈ Hom(V, V) is such that φ(X·v) = α(X)·φ(v). If λ is a weight, then for v ∈ V^{λ} we have

\begin{aligned}0 = φ((X_{V} − λ(X)·1_{V})^{n}v) & = (α(X)_{V} − λ(X)·1_{V})^{n}φ(v) \\ & = (α(X)_{V} − (λ ∘ α^{−1})(α(X))·1_{V})^{n}φ(v). \end{aligned}

Thus λ ∘ α ∈ Λ and φ(V^{λ}) = V^{λ∘α^{−1}} . Every such α leaves Λ and thus Λ^{⊥} invariant and therefore induces an automorphism α_{Λ} of 𝔥_{ℂ}/Λ^{⊥} via α_{Λ}(X + Λ^{⊥}) = α(X) + Λ^{⊥}.

Lemma 0.2. If α and φ are as in the preceding paragraph, then the following statements are equivalent:

(1) (∀λ ∈ Λ) λ ∘ α = λ.

(2) α_{Λ} = 1 (on 𝔥_{ℂ}/Λ^{⊥}).

(3) α_{Λ} − 1 is nilpotent (on 𝔥_{ℂ}/Λ^{⊥}).
(4) 𝔥_{ℂ} = 𝔥_{ℂ}^{1}(α) + Λ^{⊥}.

Proof. (1)⇔(2): Let Γ denote the subgroup generated by α in Aut(𝔥_{ℂ}). The Γ-module 𝔥_{ℂ}/Λ^{⊥} is dual to the Γ-module span_{ℂ}Λ in 𝔥_{ℂ}^{∗}. Now (1) means that span_{ℂ}Λ is the trivial Γ-module which is the same as saying that 𝔥_{ℂ}/Λ^{⊥} is the trivial module which is (2).

(2)⇒(3) clear!

(3)⇒(2): We define the permutation α^{∗} of Λ by α^{∗}(λ) = λ ∘ α^{−1} . It extends to E = span_{ℂ}Λ. Since Γ^{∗} is a finite subgroup of Aut(E), the automorphism α^{∗} is semisimple on E. Since α_{Λ} − 1 is nilpotent by (3), then α^{∗}−1 is nilpotent on E. Since α^{∗}−1 is also semisimple, we conclude α^{∗}−1 = 0.

(3)⇔(4) is basic linear algebra.

Definition 0.3. A Cartan subalgebra 𝔥 of a Lie algebra 𝔤 is a nilpotent subalgebra which is its own normalizer.

Let 𝔤 denote a Lie algebra and 𝔤_{ℂ} its complexification if 𝔤 is real. Otherwise let 𝔤_{ℂ} = 𝔤. If 𝔥 is a Cartan algebra of 𝔤, then we can apply the preceding with V = 𝔤_{ℂ} and the adjoint action. In particular, Λ is now the set of all roots, including the zero-root. We observe that X ∈ 𝔥 ∩ Λ^{⊥} iff ad X is nilpotent. If 𝔫 is the nilradical of 𝔤, then \frac{𝔥+𝔫}{𝔫} is a Cartan algebra of the reductive algebra 𝔤/𝔫, and since ad ξ for ξ ∈ 𝔤/𝔫 is semisimple on 𝔤/𝔫 we conclude 𝔥 ∩ Λ^{⊥} ⊆ 𝔫. Thus 𝔥 ∩ Λ^{⊥} ⊆ 𝔥 ∩ 𝔫. Since X ∈ 𝔫 implies that ad X is nilpotent we have

(1)  𝔥 ∩ Λ^{⊥} = 𝔥 ∩ 𝔫.

Each automorphism α of 𝔤 extends uniquely to an automorphism of 𝔤_{ℂ} which we shall again denote by α. Thus we have Aut 𝔤 ⊆ Aut 𝔤_{ℂ}.

Definition 0.4. We define

Aut(𝔤, 𝔥) = {α ∈ Aut 𝔤 : α(𝔥) = 𝔥} ⊆ Aut 𝔤_{ℂ}.

The function π: Aut(𝔤, 𝔥) → S(Λ) into the group of all permutations of Λ given by π(α)(λ) = λ ∘ α^{−1} is a representation. We set

CAut(𝔤, 𝔥) = ker π, BW(𝔤, 𝔥) = im π.

We call BW(𝔤, 𝔥) the big Weyl group of 𝔤 w. r. t. 𝔥.

If V if a complex vector space and α an automorphism then we set V^{λ}(α) = {v ∈ V : (∃n) (α − λ·1_{V})^{n}(v) = 0}.

Proposition 0.5. For an element α ∈ Aut(𝔤, 𝔥) the following statements are equivalent:

(1) α ∈ CAut(𝔤, 𝔥).

(2) α_{Λ} = 1 (on 𝔥_{ℂ}/Λ^{⊥}).

(3) α_{Λ} − 1 is nilpotent (on 𝔥_{ℂ}/Λ^{⊥} ).

(4) 𝔥_{ℂ} ⊆ Λ^{⊥} + 𝔤_{ℂ}^{1}(α).

(5) 𝔥 ⊆ 𝔤^{1}(α) mod 𝔫 where 𝔫 is the nilradical of 𝔤. Proof. The equivalence of (1)–(4) follows from Lemma 0.2.

(4)⇒(5): We know 𝔥 ∩ Λ^{⊥} = 𝔥 ∩ 𝔫. Thus Λ^{⊥} ⊆ 𝔫_{ℂ}. From this and 𝔤_{ℂ}^{1}(α) ∩ 𝔤 = 𝔤^{1}(α) the assertion follows.

(5)⇒(3): If 𝔥_{ℂ} is contained in 𝔤_{ℂ}^{1}(α) + 𝔫_{ℂ}, then α − 1 is nilpotent on (𝔥_{ℂ} + 𝔫_{ℂ})/𝔫_{ℂ} ≅ 𝔥_{ℂ}/(𝔥_{ℂ} ∩ 𝔫_{ℂ}) = 𝔥_{ℂ}/Λ^{⊥}.

Lemma 0.6. The nilpotent group e^{ad 𝔥} is normal in Aut(𝔤, 𝔥) and is contained in CAut(𝔤, 𝔥). Proof. Trivially 𝔢^{ad X}𝔥 = 𝔥 for X ∈ 𝔥. If α ∈ Aut(𝔤, 𝔥) then X ∈ 𝔥 implies α(X) ∈ 𝔥, and thus αe^{ad X}α^{−1} = e^{α ad Xα^{−1}} = e^{ad α(X)} is in e^{ad 𝔥}. Further, e^{ad X}|𝔥 is unipotent for X ∈ 𝔥. Hence (e^{ad X})_{Λ} − 1 is nilpotent on 𝔥_{C} and thus (3) of 0.5 is satisfied. Thus e^{ad X} ∈ CAut(𝔤, 𝔥).

Lemma 0.7. For β ∈ CAut(𝔤, 𝔥) the following statements are equivalent: (a) β|(𝔥 ∩ 𝔫) is unipotent.

(b) 𝔥 ∩ 𝔫 ⊆ 𝔤^{1}(β).

(c) 𝔥 ⊆ 𝔤^{1}(β). Proof. The equivalence of (a) and (b) follows from the definitions also 𝔥 ⊆ 𝔤^{1}(β) + (𝔥 ∩ 𝔫). Thus (b) implies (c), and trivially, (c)⇒(b).

For the next lemma we need some preparation.

Lemma 0.8. (Karl-Hermann Neeb) Let α be a unipotent automorphism of a finite dimensional vector space V and ν an endomorphism with its additive Jordan decomposition ν = ν_{s} + ν_{n}. Let 𝔞 denote a subalgebra of 𝔤𝔩(V) which is nilpotent on V . We assume the following hypotheses:

(i) αν_{s} = ν_{s}α. (ii) α𝔞α^{−1} = 𝔞. (iii) ν_{n} ∈ 𝔞. Write β = αe^{ν} with its multiplicative Jordan decomposition

β = β_{s}β_{u}.

Then β_{s} = e^{ν_{s}} and β_{u} = αe^{ν_{n}}. Proof. From (i) we know that e_{s}^{ν} commutes with α, and since ν_{s} and ν_{n} commute anyhow. Hence e^{ν_{s}} commutes with αe^{ν_{n}} It therefore remains to show that αe^{ν_{n}} is unipotent. In the group Gl(V) we consider the subgroup U = ⟨e^{𝔞}, α⟩. The subgroups e^{𝔞} and ⟨α⟩ are unipotent, and by (ii) the former is normal in U. Hence U is unipotent by [4], Proposition 2.2 on p. 64. Since αe_{n}^{ν} ∈ U by (iii), the assertion follows.

Lemma 0.9. Suppose α ∈ CAut(𝔤, 𝔥) and let 𝔫 denote the nilradical of 𝔤. Then there is a zero neighborhood U such that for any regular X ∈ 𝔥 ∩ 𝔤^{1}(α) we have

(2)  𝔥 = 𝔤^{1}(αe^{ad X}) + (Λ^{⊥} ∩ 𝔥) = 𝔤^{1}(αe^{ad X}) + (𝔫 ∩ 𝔥).

Proof. (Karl-Hermann Neeb) The second equality in (2) is a consequence of (1). For any X ∈ 𝔥, we define the automorphism β = αe^{ad X} ∈ CAut(𝔤, 𝔥) and set 𝔪 = Λ^{⊥} ∩ 𝔥_{ℂ} = 𝔫_{ℂ} ∩ 𝔥_{ℂ}. By 0.5(4) and 0.6 we have 𝔥_{ℂ} ⊆ 𝔤_{ℂ}^{1}(β) + 𝔪. In particular, taking X = 0 we have 𝔥_{ℂ} = (𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ}) + 𝔪.

In order to obtain the reverse containment for a suitable X we must show 𝔤_{ℂ}^{1}(β) ⊆ 𝔥_{ℂ}.

We consider the generalized eigenspace decomposition

𝔤_{ℂ} = ⨁_{μ∈Spec(α)} 𝔤_{ℂ}^{μ}(α).

Since [𝔤_{ℂ}^{1}(α), 𝔤_{ℂ}^{μ}(α)] ⊆ 𝔤_{ℂ}^{μ}(α), all 𝔤_{ℂ}^{μ}(α) are 𝔤_{ℂ}^{1}(α)-modules. Thus, if we now take an arbitrary X ∈ 𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} and set β = αe^{ad X}, then

β(𝔤_{ℂ}^{μ}(α)) ⊆ 𝔤_{ℂ}^{μ}(α).

Thus

𝔤_{ℂ}^{1}(β) = ⨁_{μ∈Spec α} (𝔤_{ℂ}^{μ}(α))^{1}(β).

If μ ≠ 1 and if X is small enough, then 1 is not in the spectrum of β|(𝔤_{ℂ}^{μ}(α)) and thus (𝔤_{ℂ}^{μ}(α))^{1}(β) = {0}. Hence there is a zero neighborhood U in 𝔥 such that X ∈ U ∩ 𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} implies 𝔤_{ℂ}^{1}(β) ⊆ 𝔤_{ℂ}^{1}(α).

Now we consider the vector space

𝔞̃ = ℂ·X + (𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} ∩ 𝔫_{ℂ}).

Since X ∈ 𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} and 𝔫_{ℂ} is an ideal, 𝔞̃ is a subalgebra of 𝔥_{ℂ}. Note that ad(𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} ∩ 𝔫_{ℂ}) is nilpotent since 𝔫_{ℂ} is the nilradial. The nilpotent part

(ad 𝔞̃)_{n} = ℂ·(ad X)_{n} + (𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} ∩ 𝔫_{ℂ})

acts nilpotently on 𝔤_{ℂ}. We define V = 𝔤_{ℂ}^{1}(α) and set 𝔞 = (ad 𝔞̃)_{n}|V ⊆ 𝔤𝔩(V). We claim that 𝔞 is an algebra. Now for any derivation D of a Lie algebra and any of its elements x one has [D, ad x] = ad(Dx). But (ad X)_{n} is a derivation since Der(𝔤_{ℂ}) is scindable in 𝔤𝔩(𝔤_{ℂ}) as the Lie algebra of the algebraic group Aut(𝔤_{ℂ}). Hence [(ad X)_{n}, ad(𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} ∩ 𝔫_{ℂ})] = ad ((ad X)_{n}(ad(𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} ∩ 𝔫_{ℂ}))). But (ad X)_{s}|𝔥_{ℂ} = 0 since 𝔥_{ℂ} is nilpotent, and thus (ad X)_{n}|𝔥_{ℂ} = (ad X)|𝔥_{ℂ}. Thus (ad X)_{n}(ad(𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} ∩ 𝔫_{ℂ}) ⊆ ad (𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} ∩ 𝔫_{ℂ}) ⊆ 𝔞̃. This proves that 𝔞 is an algebra.

We shall verify the hypotheses of Lemma 0.8 with α|V in place of α and ad X|V in place of ν. For this purpose we have to check hypotheses (i), (ii) and (iii). Since X ∈ 𝔞̃, condition (iii) is satisfied. The operator (ad X)_{s} acts by scalar multiplication on each root space 𝔤_{ℂ}^{λ}, and α leaves each of them invariant because of α ∈ CAut(𝔤, 𝔥). Hence [α, (ad X)_{s}] = 0 and thus (i) follows. It remains to verify that Y = r·X + Z with Z ∈ 𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} ∩ 𝔫_{ℂ} implies

(∗)  (α|V)(ad Y)_{n}|V (α|V)^{−1} ∈ 𝔞.

Now α(ad Y)_{n}α^{−1} = (α(ad Y)α^{−1})_{n} = ( ad α(Y))_{n}. The element α(X) − X is contained in 𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} since 𝔤_{ℂ}^{1}(α) and 𝔥_{ℂ} are invariant under α. Further, α − id induces on 𝔥_{ℂ}/(𝔥_{ℂ} ∩ 𝔫_{ℂ}) the zero map by Proposition 0.5(2). Thus α(X) − X ∈ 𝔫_{ℂ}. Thus

α(X) − X ∈ 𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} ∩ 𝔫_{ℂ},

whence

( ad α(X))_{n} ∈ ( ad X + 𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} ∩ 𝔫_{ℂ})_{n},

and thus

(α|V)( ad(r·X))_{n}|V (α|V)^{−1} ∈ 𝔞.

Also α(Z) ∈ 𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} ∩ 𝔫_{ℂ} since 𝔤_{ℂ}^{1}(α), 𝔥_{ℂ} and 𝔫_{ℂ} are all invariant under α. Hence

(α|V)(ad Z)_{n}|V (α|V)^{−1} = ad (α(Z))|V ∈ 𝔞,

too. Thus (∗) is proven and condition (iii) of Lemma 0.8 is satisfied. Lemma 0.8 then applies and shows that on V = 𝔤_{ℂ}^{1}(α) the unipotent factor β_{u} of β is αe^{(ad X)_{n}} and the semisimple factor is e^{(ad X)_{s}}. Thus the generalized eigenspace decomposition of β|V is the eigenspace decomposition of e^{(ad X)_{s}}|V .

If we finally assume that X is regular, then

𝔤_{ℂ}^{1}(β) = 𝔤_{ℂ}^{1}(α) ∩ (𝔤_{ℂ})_{1}(e^{(ad X)_{s}}) = 𝔤_{ℂ}^{1}(α) ∩ 𝔥_{ℂ} ⊆ 𝔥_{ℂ}

which is what we had to show.

1. Definitions

We let H(𝔤) denote the set of all Cartan algebras.

The group Aut(𝔤) acts on H(𝔤). Let Inn(𝔤) = ⟨e^{ad X}⟩ denote the subgroup of inner automorphisms. If the ground field is algebraically closed, then all Cartan algebras are conjugate, i.e., Inn(𝔤) is transitive on H(𝔤). Over the reals, H(𝔤) decomposes into finitely many Inn(𝔤)-orbits. The Cartan algebras have the same dimension, called the rank rank 𝔤 of 𝔤.

An element X ∈ 𝔤 is called regular if the nilspace 𝔤^{0}(ad X) has the smallest possible dimension. If this is the case then 𝔤^{0}(ad X) is a Cartan algebra, and every Cartan algebra is so obtained. The set reg 𝔤 of regular elements is open dense in 𝔤, and from what we set it follows that

reg(𝔤) ⊆ ⋃ H(𝔤)

.

Cartan subgroups of a connected real or complex Lie group G are harder to define than Cartan algebras [8]. A necessary condition is that 𝔥 = L(H) is a Cartan algebra. Let Λ ⊆ 𝔥_{ℂ}^{∗} the set of nonzero roots on the complexification 𝔥_{ℂ} of 𝔥. Let N(𝔥) = {g ∈ G : Ad(g)𝔥 = 𝔥} denote the normalizer of H_{0} or, equivalently, 𝔥 in G. Then N(𝔥) acts on Λ on the right via (λ, g) ↦ λ ∘ Ad(g).

Definition 1.1.a. We set

C(𝔥) = {g ∈ N(𝔥) : λ ∘ Ad(g) = λ for all λ ∈ Λ}.

We say that a subgroup H of a connected real or complex Lie group G is a Cartan group if L(H) is a Cartan algebra and H = C(L(H)). We let H(G) denote the set of all Cartan groups.

One notes at once that C(𝔥) is normal in N(𝔥). Then N(𝔥)/C(𝔥) is a well defined discrete group

Definition 1.1.b. We say that the (discrete) group W(G, 𝔥) =^{def} N(𝔥)/C(𝔥) is the Weyl group of G with respect to 𝔥.

We note that Ad: N(H) → Aut(𝔥, 𝔤) is a homomorphism with kernel N(H) ∩ Z(G). We observe that

C(𝔥) = Ad^{−1} ( CAut(𝔤, 𝔥)).

As a consequence we record:

Proposition 1.2. (i) Let G be any Lie group and 𝔥 a Cartan algebra of 𝔤. For an element g ∈ N(𝔥) the following statments are equivalent:

(1) g ∈ C(𝔥).

(2) Ad(g)_{Λ} = 1 (on 𝔥_{ℂ}/Λ^{⊥}).
(3) Ad(g)_{Λ} − 1 is nilpotent (on 𝔥_{ℂ}/Λ^{⊥}).
(4) 𝔥_{ℂ} ⊆ Λ^{⊥} + 𝔤_{ℂ}^{1}( Ad(g)).
(5) 𝔥 ⊆ 𝔤^{1}( Ad(g)) + (𝔫 ∩ 𝔥) where 𝔫 is the nilradical of 𝔤.
(ii) For each a ∈ G we have aC(𝔥)a^{−1} = C( Ad(a)𝔥)).

Proof. (i) follows from the preceding remark and from Proposition 0.5.

(ii) If φ, ψ ∈ Aut(𝔤) then φ(𝔤^{1}(ψ)) = 𝔤^{1}(φψφ^{−1}). Now let φ = Ad(a) and ψ = Ad(g). Then Ad(a)𝔤^{1}( Ad(g)) = 𝔤^{1}( Ad(aga^{−1})). Now (5) above is equivalent to Ad(a)𝔥 ⊆ 𝔤^{1}( Ad(aga^{−1})) + (𝔫 ∩ Ad(a)𝔥) since 𝔫 is an ideal. Thus the assertion aC(𝔥)a^{−1} = C(Ad(a)𝔥) follows from (i).

Lemma 1.3. For each g ∈ G and each identity neighborhood W of G there is a neighborhood V of g such that for v ∈ V there is a w ∈ W such that 𝔤^{1}(v) ⊆ Ad(w)𝔤^{1}(Ad g) = 𝔤^{1}( Ad(wgw^{−1})).

Proof. [3], Ch. VII, §4, no 2, Prop.5.

In particular, dim 𝔤^{1}(v) ≤ dim 𝔤^{1}(g). An element g ∈ G is called regular if v ↦ dim 𝔤^{1}(Ad v): G → ℕ_{0} is constant on a neighborhood of 𝔤. The set Reg(G) of all regular elements is open and dense in G (see [3], Ch. VII, §1, no 4, Prop.1).

Let reg exp denote the set of all X ∈ 𝔤 such that d exp(X) is invertible. We have observed the following fact:

Lemma 1.4. Suppose that G is a real Lie group and X ∈ 𝔤. Then exp X ∈ Reg(G) if and only if X ∈ reg 𝔤 ∩ reg exp. If this holds, then 𝔤^{1}(Ad(exp X)) = 𝔤^{0}(ad X). Proof. See [6], Lemma 3.

In other words, we have

(3)

exp^{−1}(Reg G) = reg 𝔤 ∩ reg exp,
exp(reg 𝔤 ∩ reg exp) = Reg G ∩ exp 𝔤.

If G is connected, then for every regular g the set 𝔤^{1}(Ad g) is a Cartan algebra, and every Cartan algebra is so obtained (see [3], Ch. VII, §4, no 4, Prop.8). In particular, every regular element g is contained in N(𝔤^{1}(Ad(g))). By 1.2(5) we have in fact g ∈ C(𝔤^{1}(Ad(g))). Thus in an arbitrary Lie group G, if g ∈ G_{0} ∩ Reg G, then 𝔥 = 𝔤^{1}( Ad(g)) is a Cartan algebra and by the preceding we conclude that g ∈ C(𝔥). As a consequence, we record

Proposition 1.5. If G is a Lie group then

G_{0} ∩ Reg(G) ⊆ ⋃ H(G).

Proposition 1.6. (i) Let G a Lie group and 𝔥 a Cartan algebra of 𝔤. Then arbitrarily close to any point g ∈ C(𝔥) there is an element g′ ∈ C(𝔥) such that (with the nilradical 𝔫 of 𝔤)

𝔥 = 𝔤^{1}(Ad(g′)) + (𝔥 ∩ 𝔫).

(ii) If g ∈ G_{0} , then arbitrarily close to g there are regular points g′ ∈ C(𝔥) such that

𝔥 = 𝔤^{1}( Ad(g′)).

In particular, Reg(G) ∩ G_{0} ∩ C(𝔥) is open and dense in G_{0} ∩ C(𝔥). Proof. (i) Let g ∈ C(𝔥). Recall that Ad(g)e^{ad X} = Ad(g exp X). By Lemma 0.9 there are arbitrarily small elements X in 𝔥 such that 𝔥 = 𝔤^{1}(Ad(g)e^{ad X}) + (Λ^{⊥}∩𝔥) = 𝔤^{1}( Ad(g exp(X)))+(𝔥∩𝔫) with the nilradical 𝔫 of 𝔤 (see also 0.5(5)). Thus 𝔥 = 𝔤^{1}(Ad(g′)) + (𝔥 ∩ 𝔫) with g′ = g exp X ∈ C(𝔥) as close to g as we wish. In particular, since G_{0} is open in G, if g ∈ G_{0} then we may also take g′ ∈ G_{0} . Let us simplify notation by setting g = g′ and assume that

(4)  𝔥 = 𝔤^{1}(Ad(g)) + (𝔥 ∩ 𝔫).

By 1.4, given an identity neighborhood W in G, there is a neighorhood V of g such that for v ∈ V there is a w ∈ W such that 𝔤^{1}(wgw^{−1}) ⊇ 𝔤^{1}(v). Thus (4) implies

Ad(w)𝔥 = 𝔤^{1}(wgw^{−1}) + (Ad(w)(𝔥) ∩ 𝔫) ⊇ 𝔤^{1}(v) + (Ad(w)(𝔥) ∩ 𝔫)

In particular, dim 𝔤^{1}(v) ≤ rank 𝔤.

(ii) If g ∈ G_{0} , then dim 𝔤^{1}(Ad(v)) ≥ rank 𝔤 (see [3], Ch. VII, §4, no 4, Prop.8(i)). Hence v ↦ dim 𝔤^{1}( Ad(v)) is locally constant around g and thus g ∈ Reg(G). We have seen that 𝔥 = 𝔤^{1}( Ad(g)).

Example E1. (i) Let G = SO(3) and 𝔤 = 𝔰𝔬(3) = span{e_{j} : j ∈ ℤ/3ℤ} with [e_{j}, e_{j+1}] = e_{j+2} .

e_{0} = ( \begin{matrix}0 & 1 & 0 \\ −1 & 0 & 0 \\ 0 & 0 & 0 \end{matrix} ) , e_{1} = ( \begin{matrix}0 & 0 & −1 \\ 0 & 0 & 0 \\ 1 & 0 & 0 \end{matrix} ) , e_{2} = ( \begin{matrix}0 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & −1 & 0 \end{matrix} ) .

We consider 𝔥 = ℝ·e_{0} . Then

C(𝔥) = exp 𝔥 = \{( \begin{matrix}cos t & sin t & 0 \\ − sin t & cos t & 0 \\ 0 & 0 & 1 \end{matrix} ) : t ∈ ℝ\} .

N(𝔥) = C(𝔥) ∪ C(𝔥) ( \begin{matrix}1 & 0 & 0 \\ 0 & −1 & 0 \\ 0 & 0 & −1 \end{matrix} ) = C(𝔥) ∪ C(𝔥) exp(π·e_{2}).

Here N(𝔥) ⊆ Reg(G) = G \ {1}. Note W(G, 𝔥) ≅ ℤ(2). Also, if g = exp(π·e_{2}), then 𝔤^{1}( Ad(g)) = ℝ·e_{2} and 𝔥 ∩ 𝔤^{1}( Ad(g)) = {0}.

(ii) Let G = Sl(2, ℝ) and 𝔤 = 𝔰𝔩(2, ℝ), spanned by

h = ( \begin{matrix}1 & 0 \\ 0 & −1 \end{matrix} ) , p = ( \begin{matrix}0 & 1 \\ 0 & 0 \end{matrix} ) , q = ( \begin{matrix}0 & 0 \\ 1 & 0 \end{matrix} ) .

Then [h, p] = 2p, [h, q] = −2q, [p, q] = h. Set u = p − q = ( \begin{matrix}0 & 1 \\ −1 & 0 \end{matrix} ) and

h′ = p + q. Then [\frac{1}{2}·u, h] = h′, [\frac{1}{2}·u, h′] = −h. We take 𝔥 = ℝ·h. Then

C(𝔥) = exp 𝔥 ∪ − exp 𝔥 = \{± ( \begin{matrix}e^{t} & 0 \\ 0 & e^{−t} \end{matrix} ) : t ∈ ℝ\} .

N(𝔥) = C(𝔥) ∪ C(𝔥) exp \frac{π}{2} ·u.

We have N(𝔥)\\{1\} ⊆ Reg G∩exp 𝔤 and W(G, 𝔥) ≅ ℤ(2). Also, if g = exp(\frac{π}{2} ·u), then 𝔤^{1}( Ad(g)) = ℝ·u and 𝔥 ∩ 𝔤^{1}( Ad(g)) = {0}.

We construct an example which illustrates that the situation we consider is rather general in the absence of connectivity.

Example E2. Let 𝔥 be a nilpotent Lie algebra and ψ: 𝔥 → 𝔤𝔩(𝔳) a representation over ℂ. Then we construct on 𝔤 =^{def} 𝔥 ⊕ 𝔳 a bracket

[(X_{1}, Y_{1}), (X_{2}, Y_{2})] = ([X_{1}, X_{2}], ψ(X_{1})(Y_{2}) − ψ(X_{2})(Y_{1})),

making 𝔤 into a solvable Lie algebra with the abelian ideal 𝔳.

Let us assume that ψ(𝔥)(Y) = {0} implies Y = 0. Then 𝔥 is a Cartan subalgebra. We write

X·Y = ψ(X)(Y), X ∈ 𝔥, Y ∈ 𝔳.

Then for X ∈ 𝔥 we have (ad X)|𝔳 = (Y ↦ X·Y). In particular, if we set X_{ψ}(Y) = X·Y = ψ(X)(Y), then λ: 𝔥 → ℂ is a root iff the nilspace 𝔳^{λ} of X_{ψ} − λ(X)·1 is nonzero, i.e., if λ is a weight of ψ.

Now any automorphism in Aut(𝔤, 𝔥) is of the form α×β with α ∈ Aut(𝔥) and β ∈ Gl(𝔳) such that ψ(α(X))(β(Y)) = β(ψ(X)Y), i.e.,

Aut(𝔤, 𝔥) = {(α, β) ∈ Aut(𝔥) × Gl(𝔳) : ψ ∘ (α ⊕ β) = β ∘ ψ} = {(α, β) ∈ Aut(𝔥) × Gl(𝔳) : (∀(X, Y) ∈ 𝔥 ⊕ 𝔳) α(X)·β(Y) = β(X·Y)} .

We suppose now that an arbitrary nilpotent Lie algebra 𝔥 is given and that Λ is any finite set of nonzero vectors in (𝔥′)^{⊥} ⊆ 𝔥^{∗}. We set

Aut(Λ) = {φ ∈ Gl (span(Λ)) : φ(Λ) = Λ)}

and consider the finite orbit space J = Λ/ Aut(Λ) and the orbit map ρ: Λ → J. For each j ∈ J we fix an arbitrary nilpotent representation π_{j}: 𝔥 → 𝔤𝔩(𝔳_{j}) of 𝔥 (e.g., the zero representation!) and define 𝔳 = ⨁_{λ∈Λ} 𝔳_{ρ(λ)} . We set

ψ(X)(Y) = X· (⨁_{λ∈Λ} Y_{λ}) = (⨁_{λ∈Λ}(λ(X)·1 + π_{ρ(λ)}(X))(Y_{λ})) .

Let α ∈ Aut(𝔥) with α^{∗}|Λ ∈ Aut(Λ). Notice that for an abelian 𝔥, every element of Aut(Λ) is so obtained by the definition of Aut(Λ), and if span(Λ) = 𝔥^{∗}, then this representation is unique. Now we define β: 𝔳 → 𝔳 by

β (⨁_{λ∈Λ} Y_{λ}) = ⨁_{λ∈Λ} Y_{α^{∗}(λ)}.

Now

α(X)·β(Y) = ⨁_{λ∈Λ} (λ(α(X))·1 + π_{ρ(λ)}(X))(Y_{α^{∗}(λ)})

and

β(X·Y) = ⨁_{λ∈Λ} ((α^{∗}(λ)(X)·1 + π_{ρ(α^{∗}(λ))}(X))(Y_{α^{∗}(λ)}).

We see that α ⊕ β ∈ Aut(𝔤, 𝔥). In particular, if 𝔥 is abelian, then

BW(𝔤, 𝔥) = Aut(Λ).

If Λ is any finite generating set of a real Hilbert space 𝔥 with scalar product (·|·), we can take 𝔥 as an abelian Lie algebra with 𝔥^{∗} = 𝔥 (writing λ(X) = (λ|X)) and for π_{j} the one dimensional zero representation. Then we see that

every finite group Γ which is isomorphic to a group Aut(Λ) for a finite set Λ of vectors spanning a real vector space, can occur as a big Weyl group of a Lie algebra 𝔤 with respect to some Cartan subgroup 𝔥.

If G_{0} is the simply connected Lie group with Lie algebra 𝔤, then we may construct the semidirect product G_{0} ×_{γ} Aut(Λ) with the action γ induced by the action of Aut(Λ) on 𝔤 = 𝔥 ⊕ 𝔳. Thus Aut(Λ) is realized as the Weyl group of a Lie group G with respect to a Cartan algebra 𝔥 of 𝔤.

Every family (β_{λ})_{λ∈Λ} of intertwining operators β_{λ}: 𝔳_{ρ(λ)} → 𝔳_{ρ(λ)} for the representation π_{ρ(λ)} yields an automorphism

(X, Y) ↦ (X, ⨁_{λ∈Λ} β_{λ}(Y_{λ})) in CAut(𝔤, 𝔥).

In particular, if 𝔥 = ℂ = 𝔥^{∗} , Λ = {1}, π: 𝔥 → 𝔤𝔩(𝔳), π(z)(v) = z·v, then 𝔤 is almost abelian, Aut(Λ) = {1}, and

CAut(𝔤, 𝔥) = {1} × Gl(𝔳) ≅ Gl(𝔳).

For the definitive information on this issue see Neeb [9].

The space of closed subgroups. In [6] we considered for a real Lie group G with Lie algebra 𝔤 the compact Hausdorff spaces Σ(G) of all closed subgroups of G and Σ(𝔤) of all additive closed subgroups of 𝔤. We note H(𝔤) ⊆ Σ(𝔤) and H(G) ⊆ Σ(G).

Definition 1.8. A subalgebra 𝔥 of a real or complex Lie algebra 𝔤 is said to be a near-Cartan algebra if 𝔥 ∈ \overline{H(𝔤)}. A subgroup H of a real or complex Lie group is called a near-Cartan group of G if H ∈ \overline{H(G)}.

Proposition 1.9. Let 𝔤 be a a real or complex Lie algebra and 𝔥 a near-Cartan subalgebra. Then

(i) dim 𝔥 = rank(𝔤)

(ii) 𝔥 is nilpotent and its class of nilpotency is dominated by that of the Cartan algebras of 𝔤.

Proof. (i) The subspace in Σ(𝔤) of all vector spaces of dimension dim(rank 𝔤) is a compact manifold containing H(𝔤). Then \overline{H(𝔤)} is contained in this manifold.

(ii) It was proved in [6], Prop. 24 that the space of closed nilpotent subgroups of class ≤ k in Σ(G) is closed for all k.

Proposition 1.10. For each real or complex Lie algebra 𝔤 and each real or complex Lie group G we have

\begin{aligned}𝔤 & = ⋃ \overline{H(𝔤)}, \\ G & = ⋃ \overline{H(G)}. \end{aligned}

Proof. Let X ∈ 𝔤. There is a sequence X_{n} ∈ reg 𝔤 with X = lim X_{n}. The sequence 𝔤^{0}(ad X_{n}) ∈ H(𝔤) has a subsequence in the compact space \overline{H}(𝔤) converging to an element 𝔥. Then X ∈ 𝔥 by the definition of the topology of Σ(𝔤). The proof of the second part is similar.

Proposition 1.11. Each near-Cartan subgroup H of a Lie group G contains the center Z of G.

Proof. By definition, H = lim H_{n} in \overline{H(G)} with a sequence of Cartan subgroups H_{n}. Now Z ⊆ C(𝔥_{n}) = H_{n} for all n ∈ ℕ. The assertion Z ⊆ lim H_{n} = H follows.

It is very instructive to consider, in this context, the examples of the universal covering group of Sl(2, ℝ) and that of the group of motions of the euclidean plane. (For the latter example, see e.g. [6], Example 23.)

Lemma 1.12. Let f: G → G/Z the quotient homomorphism of a locally compact group G modulo a closed normal subgroup Z. Let Σ_{Z} denote the subspace in Σ(G) of all closed subgroups H of G with Z ⊆ H. Then the function

H ↦ H/Z : Σ_{Z}(G) → Σ(G/Z)

is a homeomorphism.

Proof. The function is clearly a bijection, and domain and range are compact Hausdorff spaces. It therefore suffices to establish its continuity. Let A_{n} be a net of closed subgroups of G converging to A. This means that for every compact subspace C of G and every identity neighborhood U in G eventually we have A_{n} ∩ C ⊆ A ∩ CU and A ∩ C ⊆ A_{n} ∩ CU (see [1], Chap. VIII, §5, n^{o} 6, p. 188). Now let K be a compact subspace of G/Z and V an identity neighborhood in G/Z. Then there is a compact subspace C of G such that K = f(C). We claim that f(A_{n}) ∩ K = f(A_{n}) ∩ f(C) = f(A_{n} ∩ C): indeed the last term is contained in the preceding, and if f(c) = f(a_{n}) with c ∈ C and a_{n} ∈ A_{n} then there is a z ∈ Z with c = a_{n}z. But Z ⊆ H implies a_{n}z ∈ A_{n} and so c ∈ A_{n} ∩ C. Let U = f^{−1}(V). Then by the same argument, f(A) ∩ KV = f(A ∩ CU). Hence f(A_{n}) ∩ K ⊆ f(A) ∩ KV iff f(A_{n} ∩ C) ⊆ f(A ∩ CU) and this is eventually the case by hypothesis on A_{n}. In the same vein we see that eventually f(A) ∩ K ⊆ f(A_{n}) ∩ KV . Thus lim f(A_{n}) = f(A) which we had to show.

If 𝔤 is a Lie algebra, then the group int(𝔤) = ⟨e^{ad 𝔤}⟩ of inner automorphisms is an analytic subgroup of Aut(𝔤) whose Lie algebra is ad 𝔤 ⊆ Der(𝔤). If one endows int(𝔤) with its intrinsic Lie group structure then it is isomorphic to G/Z where G is any of the connected Lie groups with L(G) ≅ 𝔤 and Z is its center. In a sense, therefore, the following proposition says that the Cartan subgroups of a connected Lie group are determined by its Lie algebra 𝔤 alone.

Proposition 1.13. The quotient homomorphism f: G → G/Z maps (i) H(G) bijectively onto H(G/Z), and (ii) \overline{H(G)} bijectively onto \overline{H(G/Z)}.

Proof. Every Cartan algebra 𝔥 in 𝔤 contains the center 𝔷 and 𝔥/𝔷 is a Cartan algebra of 𝔤/𝔷. Also, every Cartan algebra of 𝔤/𝔷 is of this form (see [3], Chap. VII, §2, n^{o} 2, p. 21). Let H = C(𝔥) be a Cartan group of G. This means that h ∈ H implies Ad(h) ∈ CAut(𝔤, 𝔥). Then Ad (f(h)) ∈ CAut(𝔤/𝔷, 𝔥/𝔷) as is readily verified. Conversely, if g ∈ G is such that Ad (f(g)) ∈ CAut(𝔤/𝔷, 𝔤/𝔷) then also Ad(g) ∈ CAut(𝔤, 𝔥), i.e., g ∈ C(𝔥) = H. This proves the first part of the assertion concerning (i).

Now we turn to part (ii). The quotient map f: G → G/Z induces a homeomorphism φ: Σ_{Z}(G) → Σ(G/Z), φ(H) = H/Z by Lemma 1.12. Now \overline{H(G)} ⊆ Σ_{Z} by Proposition 1.11. By the first part of the proof, φ maps H(G) bijectively onto H(G/Z). Then φ maps \overline{H(G)} homeomorphically onto \overline{H(G/Z)}.

2. More on near-Cartan groups

Definition 2.1. Let 𝔤 denote a real Lie algebra. A subalgebra 𝔪 is said to be a subalgebra of maximal rank if every Cartan subalgebra of 𝔪 is a Cartan algebra of 𝔤. The set of maximal rank subalgebras of 𝔤 is denoted by M(𝔤) ⊆ Σ(𝔤). A closed subgroup M of a Lie group with Lie algebra 𝔤 is called a subgroup of maximal rank if 𝔪 = L(M) is a maximal rank subalgebra of 𝔤. The set of maximal rank subgroups of G is denoted by M(G) ⊆ Σ(G).

Every Cartan algebra is a maximal rank subalgebra of 𝔤 and every Cartan subgroup of G is a maximal rank subgroup.

Lemma 2.2. Let B be any open neighborhood of 0 in the Lie algebra 𝔤 of a Lie group G such that exp |B: B → U is a diffeomorphism onto an open identity neighborhood of G. Let M be a maximal rank subgroup of G. Let S =^{def} (exp |B)^{−1}(U ∩ M). Then S = 𝔪 ∩ B.

Proof. (i) If X ∈ 𝔪 ∩ B, then exp X ∈ M ∩ U and thus X ∈ S by the definition of S. Thus 𝔪 ∩ B ⊆ S.

(ii) In order to show that S ⊆ B ∩ 𝔪 we must show that S ⊆ 𝔪. Since M is closed then S is closed in B. It therefore suffices to find a dense subset T ⊆ S with T ⊆ 𝔪. We let T = (exp |B)^{−1}(Reg M ∩ U). Since Reg(M) is dense in M then T is dense in S. Now let X ∈ T. We must show X ∈ 𝔪. Now m = exp X ∈ Reg(M) and 𝔪^{1}(Ad(m)|𝔪) is a Cartan algebra in 𝔪. Since M is a maximal rank subgroup, it is a Cartan algebra of 𝔤. Hence 𝔤^{1}( Ad(m)) ⊇ 𝔪^{1}(Ad(m)|𝔪) is a Cartan algebra of 𝔤, equality holds, and m is regular in G. Now Ad(m) = Ad(exp X) = e^{ad X}. It follows that (Ad(m) − 1)(X) = (∑_{j=1}^{∞} \frac{1}{j!}(ad X)^{j})X = 0. Thus X ∈ 𝔤^{1}(Ad(m)) = 𝔤^{1}(Ad(m)|𝔪) ⊆ 𝔪.

Proposition 2.3. Let H = lim H_{n} in \overline{M(G)} and suppose that H_{n} ∈ M(G). Then L(H) = lim L(H_{n}).

Proof. Let C denote an open, relatively compact convex symmetric Campbell-Hausdorff-neighborhood in 𝔤 contained in a neighborhood B as is Lemma 2.2. Then S_{n} =^{def} (exp |C)^{−1}(H_{n} ∩ exp C) converges to S =^{def} (exp |C)^{−1}(H ∩ exp C) in the space of closed subsets of C. If H_{n} ∈ M(G), then S_{n} = C ∩ L(H_{n}) by Lemma 2.2. Now dim H_{n} = dim L(H_{n}) and dim H = dim S. If (n(j))_{j∈ℕ} is any subsequence such that L(H_{N(j)}) converges to E in the compact space Σ_{L}(𝔤), then C ∩ L(H_{n(j)}) converges to C ∩ E on one hand and to S on the other. Hence S = C ∩ E. Thus E = ℝ·(C ∩ E) does not depend on the choice of the subsequence and E = lim L(H_{n}). Now E is a vector subspace of 𝔤 such that C ∩ E = (exp |C)^{−1}(H ∩ exp C). If we recall that for a closed subgroup H of G we have L(H) = {X ∈ 𝔤 : exp ℝ·X ⊆ H} we may conclude E = L(H).

We set Σ_{m}(G) = {H ∈ Σ(G) : dim H = m}.

Corollary 2.4. \overline{M(G) ∩ Σ_{m}(G)} ⊂ Σ_{m}(G). Proof. Consequence of Proposition 2.3.

Now we apply this with m = rank G. Since H(G) ⊆ M(G) ∩ Σ_{rank G}, Corollary 2.4 yields at once

Corollary 2.5. Every near-Cartan group has dimension rank(G).

More precisely:

Theorem 2.6. The Lie algebra 𝔥 = L(H) of a near-Cartan group H is a near-Cartan algebra. The analytic subgroup H = exp 𝔥 generated by a near-Cartan algebra 𝔥 is the identity component of a near-Cartan group. Proof. (i) Let H ∈ \overline{H(G)}. Then there is a sequence H_{n} of Cartan groups with H = lim H_{n}. Then L(H) = lim L(H_{n}) by Proposition 2.3. Since L(H_{n}) ∈ H(𝔤) we conclude L(H) ∈ \overline{H(𝔤)}.

(ii) Let 𝔥 ∈ \overline{H(𝔤)}. Then there is a sequence 𝔥_{n} of Cartan algebras with 𝔥 = lim 𝔥_{n}. The analytic groups H_{n} =^{def} exp 𝔥_{n} are closed (since 𝔥_{n} is an ideal in L(\overline{exp 𝔥_{n}}) and a Cartan algebra is its own normalizer!). Each H_{n} is the identity component of the Cartan group C(𝔥_{n}) which is a maximal rank group. Because of the compactness of \overline{H(G)} there is a sequence (n(j))_{j∈ℕ} such that H^{∗} = lim C(𝔥_{n(j)}) exists. By definition, H^{∗} is a near-Cartan group. The relation L(H^{∗}) = lim L(C(𝔥_{n(j)})) = lim 𝔥_{n(j)} = 𝔥 follows from Proposition 2.3.

Then H =^{def} (H^{∗})_{0} = exp 𝔥 follows.

We have seen in [6] that there may be sequences 𝔥_{n} of Cartan algebras such that (H, 𝔥) = lim(exp 𝔥_{n}, 𝔥_{n}) exists in \overline{H(G)} × \overline{H(𝔤)} such that H is not connected. In that example all Cartan groups were connected, but we have disconnected near-Cartan groups. In constrast with the situation of a Cartan group which is uniquely determined by its identity component, different near-Cartan groups may have the same identity component. This is the case in

Sl(2, ℝ). The algebra 𝔥 = ℝ· ( \begin{matrix}0 & 1 \\ 0 & 0 \end{matrix} ) is the Lie algebra of two different near-

Cartan groups H_{1} = exp 𝔥 = lim exp ℝ· ( \begin{matrix}0 & 1 \\ \frac{1}{n} & 0 \end{matrix} ) and H_{2} = \{1, −1\}H_{1} =

lim exp ℝ· ( \begin{matrix}0 & 1 \\ −\frac{1}{n} & 0 \end{matrix} ).

Further remarks. The function L: Σ(G) → Σ_{L}(𝔤) from the space of closed subgroups into the space of Lie subalgebras of 𝔤 which associates with a closed subgroup K of G its Lie algebra L(K) is not continuous as was observed in [6]. If G = ℝ^{2}/ℤ^{2} with 𝔤 = ℝ^{2} and the quotient map as exponential function, then 𝔥_{n} = ℝ·(\frac{1}{n}, 1) converges to 𝔥 = ℝ × \{0\} in Σ_{L}(𝔤). However, H_{n} = exp 𝔥_{n} converges to G ≠ exp 𝔥 in Σ(G).

However, in [6] we proved:

Lemma 2.7. Let G denote a Lie group and set D = {(H, 𝔥) ∈ Σ(G) × Σ_{L}(𝔤) : 𝔥 ⊆ L(H)}. Then D is closed in Σ(G) × Σ_{L}(𝔤).

Now we set ℕ_{G} = {0, 1, …, dim L(G)} and consider the function

dim: Σ(G) → ℕ_{G}, H ↦ dim H = dim L(H).

Lemma 2.8. The function dim: Σ(G) → ℕ_{G} is upper semicontinuous.

Proof. We claim that the lower graph {(H, n) : n ≤ dim L(H), H ∈ Σ(G), n ∈ ℕ_{G}} is closed in Σ(G) × ℕ_{G} by Lemma 2.3. Indeed, if (H, n) = lim(H_{k}, n_{k}), then for any subsequence (k(j))_{j∈ℕ} with 𝔥 = lim L(H_{k(j))} we have 𝔥 ⊆ L(H), i.e., lim dim L(H_{k(j)}) ≤ dim H. But n_{k(j)} ≤ dim L(H_{k(j)}). Hence n ≤ lim dim L(H_{k(j)}) ≤ dim H.

Lemma 2.9. Suppose H = lim H_{n} in Σ(G). If dim H = lim dim H_{n}, then L(H) = lim L(H_{n}).

Proof. This is Corollary 22 in [6].

Proposition 2.10. (i) The restrictions

L: Σ_{m}(G) → Σ_{L}(𝔤) and dim: Σ_{m}(G) → ℕ_{G}

are continuous.

(ii) The function

C: H(𝔤) → H(G)

is continuous.

Proof. (i) follows from Lemma 2.9 and 2.8.

(ii) If 𝔥 = lim 𝔥_{n} in H(𝔤), then from [6] we know that there is a sequence Y_{n} → 0 such that 𝔥_{n} = e^{ad Y_{n}}𝔥. If a_{n} = exp Y_{n} then a_{n} → 1 and C(𝔥_{n}) = C(Ad(a_{n})𝔥) = a_{n}C(𝔥)a_{n}^{−1} by 1.2(ii). Thus C(𝔥) = lim C(𝔥_{n}) as asserted.

Proposition 2.11. (i) Let 𝔥 be a near-Cartan algebra. Then the following are equivalent

(1) 𝔥 is a Cartan algebra

(2) 𝔥 ∩ reg 𝔤 ≠ Ø.

(3) \overline{𝔥 ∩ reg 𝔤} = 𝔥. (ii) Suppose that G is connected and let H be a near-Cartan group. Then

the following are equivalent:

(1) H is a Cartan group.

(2) H ∩ Reg(G) ≠ Ø.
(3) H ⊆ \overline{H ∩ Reg G}.

Proof. (i) If 𝔥 is a Cartan algebra, then the set of regular elements of 𝔤 is dense in 𝔥. So (1)⇒(3). Next (3)⇒(2) is trivial. Now we show (2)⇒(1): For any X ∈ 𝔥 we have 𝔥 ⊆ 𝔤^{0}(ad X) since 𝔥 is nilpotent by 1.9(ii). If now X is regular, then dim 𝔤^{1}(ad X) = rank 𝔤 = dim 𝔥 in view of 1.9(i). Thus 𝔥 = 𝔤^{1}(ad X) follows and so 𝔥 is a Cartan algebra.

(ii) If H is a Cartan group, then Proposition 1.6 proves (1)⇒(3). But (3)⇒(2) is trivial. Next we show that (2)⇒(1). By the definition of a near-Cartan group we have H = lim H_{n} with H_{n} = C(𝔥_{n}). If g ∈ H then g = lim g_{n} with g_{n} ∈ C(𝔥_{n}). We may assume that g_{n} is regular by 1.6. Then 𝔤^{1}( Ad(g_{n})) = 𝔥_{n}. By 1.3 there is a sequence w_{j} → 1 such that we have 𝔤^{1}( Ad(g_{n_{j}})) ⊆ 𝔤^{1}( Ad(w_{j}gw_{j}^{−1})) for a suitable sequence n_{j} of natural numbers going to infinity. Since the g_{n} are regular, 𝔤^{1}( Ad(g_{n_{j}})) = 𝔥_{n_{j}}.

Now suppose that g is regular. Then dim 𝔤^{1}( Ad(w_{j}gw_{j}^{−1})) = rank 𝔤 = dim 𝔥_{n_{j}}, and thus we have 𝔥_{n_{j}} = 𝔤^{1}( Ad(w_{j}gw_{j}^{−1})) = w_{j}𝔤^{1}( Ad(g))w_{j}^{−1} . Now 𝔥 = lim 𝔥_{n_{j}} on the one hand and 𝔤^{1}( Ad(g)) = lim w_{j}𝔤^{1}( Ad(g))w_{j}^{−1} on the other. Thus 𝔥 = 𝔤^{1}(Ad(g)) is a Cartan algebra and g ∈ C(𝔥) by 1.2(5). But now C(𝔥) = lim C(𝔥_{n}) = H by 2.10(ii). In particular, H is a Cartan group.

3. The exponential function and near-Cartan groups

Let exp: 𝔤 → G denote the exponential function of a Lie group. For X ∈ 𝔤 we choose the abbreviation

Ω(X) = Spec ad X ∩ 2πiℕ ⊆ ℂ.

We note that for X ∈ 𝔤 we have

ker d exp X = ⨁_{λ∈Ω(X)} ker ((ad X)^{2} + |λ|^{2}).

(See e.g. [5,6].) Then the set in which the exponential function is singular is

S(𝔤) = 𝔤 \ reg exp = {X ∈ 𝔤 : Ω(X) ≠ {0}}.

This set is invariant under Aut(𝔤). As an abbreviation we write sing(𝔤) = 𝔤 \ reg 𝔤.

Lemma 3.1. Let H be a near-Cartan group in a Lie group and assume X ∈ reg exp, exp X ∈ H. Then X ∈ 𝔥.

Proof. By the definition of a near-Cartan group we find a sequence of Cartan groups H_{n} converging to H in \overline{H(G)}. Since H_{n} ∩ Reg G is dense in G_{0} ∩ H_{n} by 2.11, and by the definition of the topology in Σ(G), we find a sequence of regular elements g_{n} ∈ G_{0} ∩ H_{n} converging to exp X. Since the exponential function is regular at X, there is an open neighborhood U of X in 𝔤 and an open neighborhood V of exp X in G such that the restriction exp |U: U → G corestricts to a diffeomorphism ε: U → V . We may assume that g_{n} ∈ V . Set X_{n} = ε^{−1}(g_{n}). Then lim X_{n} = lim ε^{−1}(g_{n}) = ε^{−1}(exp X) = ε^{−1}(ε(X)) = X. Also, exp X_{n} = ε(X_{n}) = ε(ε^{−1}(g_{n})) = g_{n}. Since g_{n} ∈ Reg G we know that X_{n} ∈ reg exp 𝔤 ∩ reg 𝔤 by 1.4. Then 𝔤^{0}(ad X_{n}) = 𝔤^{1}(Ad(g_{n})) = 𝔥_{n} by 1.4. Thus X_{n} ∈ 𝔥_{n}. From 2.10(i) we infer 𝔥 = lim 𝔥_{n}. But then X = lim X_{n} and X_{n} ∈ 𝔥_{n} show X ∈ 𝔥.

It is interesting to observe that, in the preceding proof, the hypothesis “exp is regular at X ” cannot be replaced by the weaker hypothesis “exp 𝔤 is a neighborhood of exp X ”. Using this hypothesis we could still find elements X_{n} ∈ reg exp ∩ reg 𝔤 ∩ 𝔥_{n} with exp X_{n} = g_{n}. But we could not conclude that X = lim X_{n}. The weaker conclusion that X′ = lim X_{n(j)} for some sequence n(j) → ∞ of natural numbers would suffice for the conclusion that we could find an X′ ∈ 𝔥 with exp X = exp X′. Thus the modification of the proof of 3.1 which we have just suggested yields the following obervation:

Remark 3.2. Assume that H is a Cartan subgroup and exp X ∈ H and assume that there is a bounded subset B in 𝔤 such that exp B is a neighborhood of exp X. Then there is an X′ ∈ 𝔥 with exp X = exp X′.

Lemma 3.3. Let G be a Lie group and H a near-Cartan group with Lie algebra 𝔥. If H is not a Cartan group, then exp^{−1} H ⊆ S(𝔤) ∪ sing(𝔤).

Proof. Suppose that exp X ∈ H. Since H is not a Cartan group, then exp X ∈ H cannot be regular by 2.11. Hence X ∉ reg(𝔤) ∩ reg exp by 1.4.

Theorem 3.4. Let G be a Lie group and H a near-Cartan group with Lie algebra 𝔥. Then

exp^{−1} H ⊆ 𝔥 ∪ S(𝔤).

If H is not a Cartan subgroup then

exp^{−1} H ⊆ (𝔥 ∪ S(𝔤)) ∩ ( sing(𝔤) ∪ S(𝔤)) = (𝔥 ∩ sing(𝔤)) ∪ S(𝔤).

Proof. This follows at once from Lemmas 3.1 and 3.3.

References

[1]  Bourbaki, N., “Intégration”, Chapitres 7 et 8, Hermann, Paris, 1963.

[2] Bourbaki, N., “Groupes et algèbres de Lie”, Chapitres 2 et 3, Hermann Paris, 1972.

[3] Bourbaki, N., “Groupes et algèbres de Lie”, Chapitres 7 et 8, Hermann Paris, 1975.

[4] Hochschild, G. P., “Basic Theory of Algebraic Groups and Lie Algebras”, Springer Verl. New York etc., 1981.

[5] Hofmann, K. H., A memo on the singularities of the exponential function, Seminar Notes 1990.

[6] —, A memo on the exponential function and regular points, Archiv der Mathematik 92 (1992), 24–37.

[7] Hofmann, K. H., and A. Mukherjea, On the density of the image of the exponential function, Math. Ann. 234 (1978), 263–273.

[8] Neeb, K.-H., A memo on Cartan subgroups and regular elements, Seminar Report Technische Hochschule Darmstadt 1992.

[9] —, Weyl groups of disconnected Lie groups, Seminar Sophus Lie 2 (1992), 153–157.

Fachbereich Mathematik Technische Hochschule Darmstadt Schlossgartenstr. 7 W-6101 Darmstadt, Germany hofmann@mathematik.th-darmstadt.de

Received July 10, 1992 and in final form October 5, 1992