Journal of Lie Theory Volume 11 (2001) 505–543 \begin{matrix}c \\ ◯ \end{matrix} 2001 Heldermann Verlag

Cartan-Decomposition Subgroups of SU(2,n)

Alessandra Iozzi and Dave Witte

Communicated by E. B. Vinberg

Abstract. We give explicit, practical conditions that determine whether or not a closed, connected subgroup H of G = SU(2, n) has the property that there exists a compact subset C of G with CHC = G. To do this, we fix a Cartan decomposition G = KA^{+}K of G, and then carry out an approximate calculation of (KHK) ∩ A^{+} for each closed, connected subgroup H of G. This generalizes the work of H. Oh and D. Witte for G = SO(2, n).

1 Introduction

Definition 1.1. [14, Defn. 1.2] Let H be a closed subgroup of a connected, simple, linear, real Lie group G. We say that H is a Cartan-decomposition subgroup of G if

• H is connected, and

• there is a compact subset C of G, such that CHC = G.

(Note that C is only assumed to be a subset of G; it need not be a subgroup.)

Example 1.2. The Cartan decomposition G = KAK shows that the maximal split torus A is a Cartan-decomposition subgroup of G.

It is known that G = KNK [9, Thm. 5.1], so the maximal unipotent subgroup N is also a Cartan-decomposition subgroup.

If ℝ-rank G = 0 (that is, if G is compact), then every (closed, connected) subgroup of G is a Cartan-decomposition subgroup.

If ℝ-rank G = 1, then it not difficult to see that every (closed, connected) noncompact subgroup of G is a Cartan-decomposition subgroup (cf. [5, Lem. 3.2]).

It is more difficult to characterize the Cartan-decomposition subgroups when ℝ-rank G = 2, but H. Oh and D. Witte [14] studied two examples in detail. Namely, they described all the Cartan-decomposition subgroups of SL(3, ℝ) and of SO(2, n), and they also explicitly described the closed, connected subgroups that are not Cartan-decomposition subgroups. Here, we obtain similar results for SU(2, n). Unfortunately, the results are rather complicated to state.

ISSN 0949–5932 / $2.50 \begin{matrix}c \\ ◯ \end{matrix} Heldermann Verlag Notation 1.3. Let G = SU(2, n) and fix an Iwasawa decomposition G = KAN and a corresponding Cartan decomposition G = KA^{+}K, where A^{+} is the (closed) positive Weyl chamber of A in which the roots occurring in the Lie algebra of N are positive. Thus, K is a maximal compact subgroup, A is the identity component of a maximal split torus, and N is a maximal unipotent subgroup.

To simplify, let us restrict our attention here to subgroups of N.

Theorem 1.4. (cf. 3.4) Let G = SU(2, n) and let H be a closed, connected subgroup of N. Then H is a Cartan-decomposition subgroup of G if and only if

1. H satisfies at least one of the eight conditions in Proposition 4.1; and

2. H satisfies at least one of the five conditions in Proposition 5.1.

Theorem 1.5. Let G = SU(2, n) and let H be a closed, connected, nontrivial subgroup of N. Then H is not a Cartan-decomposition subgroup of G if and only if H belongs to one of the eleven types of subgroups explicitly described in Theorem 6.1.

For subgroups H that are not contained in N, there is no loss of generality in assuming that H ⊂ AN (see 7.1), and that H satisfies the additional technical condition of being compatible with A (see 7.3). Under these assumptions, Theorem 7.4, Proposition 7.6, and Lemma 7.8, taken together, list the possibilities for H and, in each case, determine whether H is a Cartan-decomposition subgroup or not.

Our results require an effective method to determine whether a subgroup is a Cartan-decomposition subgroup or not. This is provided by the Cartan projection.

Definition 1.6. (Cartan projection) For each element g of G, the Cartan decomposition G = KA^{+}K implies that there is an element a of A^{+} with g ∈ KaK. In fact, the element a is unique, so there is a well-defined function

μ: G → A^{+} given by g ∈ K μ(g) K.

The function μ is continuous and proper (that is, the inverse image of any compact set is compact). Some properties of the Cartan projection are discussed in [1] and [7].

We have μ(H) = A^{+} if and only if KHK = G. This immediately implies that if μ(H) = A^{+}, then H is a Cartan-decomposition subgroup. Y. Benoist and T. Kobayashi proved the deeper statement that, in the general case, H is a Cartan-decomposition subgroup if and only if μ(H) comes within a bounded distance of every point in A^{+}.

Notation 1.7. For subsets U and V of A^{+}, we write U ≈ V if there is a compact subset C of A, such that U ⊂ V C and V ⊂ UC. This is an equivalence relation.

Theorem 1.8. (Benoist [1, Prop. 5.1], Kobayashi [8, Thm. 1.1]) A closed, connected subgroup H of G is a Cartan-decomposition subgroup if and only if μ(H) ≈ A^{+}.

Remark 1.9. We may consider SO(2, n) to be the subgroup of SU(2, n) consisting of the real matrices. Then, because A ⊂ SO(2, n)^{∘}, we see that SO(2, n)^{∘} is a Cartan-decomposition subgroup of SU(2, n). More generally, for any subgroup H of SO(2, n)^{∘}, we see that H is a Cartan-decomposition subgroup of SO(2, n)^{∘} if and only if H is a Cartan-decomposition subgroup of SU(2, n). (For example, this follows from the fact that the Cartan projection for SO(2, n)^{∘} is the restriction of the Cartan projection for SU(2, n).) Thus, our results generalize those theorems of H. Oh and D. Witte [14] that are directed toward SO(2, n).

Remark 1.10. One may define a preorder ≺ on the set of closed, connected subgroups of G by

H_{1} ≺ H_{2} if there is a compact subset C of G, such that H_{1} ⊂ CH_{2}C .

(So H is a Cartan-decomposition subgroup of G if and only if G ≺ H.) We see from [1, Prop. 5.1] that H_{1} ≺ H_{2} if and only if there is a compact subset C of A, such that μ(H_{1}) ⊂ μ(H_{2})C. Thus, it is of interest to calculate μ(H), for each subgroup H of G. Our results solve this problem: for each (closed, connected) subgroup H, we give an explicit subset U of A^{+}, such that μ(H) ≈ U. For the cases where μ(H) ≉ A^{+}, these results are summarized in Tables 1, 2, and 3 of Section 8, and the subset U is given in a standard form that makes it easy to determine whether H_{1} ≺ H_{2}. Thus, we determine the order structure of the relation ≺, and also determine precisely where each subgroup lies in this preorder.

The interest in Cartan-decomposition subgroups is largely due to the following basic observation that, to construct nicely behaved actions on homogeneous spaces, one must find subgroups that are not Cartan-decomposition subgroups. (See [7, §3] for some historical background on this result.)

Proposition 1.11. (Calabi-Markus phenomenon, cf. [10, pf. of Thm. A.1.2]) If H is a Cartan-decomposition subgroup of G, then no closed, noncompact subgroup of G acts properly on G/H.

H. Oh and D. Witte [15, 16] used this proposition as a starting point to study the existence of tessellations. (A homogeneous space G/H is said to have a tessellation if there is a discrete subgroup Γ of G, such that Γ acts properly on G/H, and Γ\G/H is compact.) In particular, when n is even, they determined exactly which homogeneous spaces SO(2, n)/H have a tessellation (under the assumption that H is connected). These results depend not only on the characterization of Cartan-decomposition subgroups, but also on the calculation of μ(H) for each subgroup H, and on the maximum possible dimension of subgroups with a given image under the Cartan projection. In [4] we use some of the results of the current paper to study tessellations of homogeneous spaces of SU(2, n).

Here is an outline of the paper. Section 2 describes the notation we use to specify elements of SU(2, n). Section 3 recalls some general results on Cartandecomposition subgroups, and defines a representation ρ. Section 4 determines whether H contains large elements with ∥ρ(h)∥ approximately equal to ∥h∥^{2}. Similarly, Section 5 determines whether H contains large elements with ∥ρ(h)∥ approximately equal to ∥h∥. By combining the calculations of the preceding two sections, Section 6 determines which subgroups of N are Cartan-decomposition subgroups. Then Section 7 determines which other subgroups of G are Cartandecomposition subgroups. Section 8 determines the maximum possible dimension of a subgroup of H with any given image under the Cartan projection.

Acknowledgments 1.12. This research was partially supported by a grant from the National Science Foundation (DMS-9801136). Much of the work was carried out during productive visits to the University of Bielefeld (Germany) and the Isaac Newton Institute for Mathematical Sciences (Cambridge, U.K.). We would like to thank the German-Israeli Foundation for Research and Development for financial support that made the visit to Bielefeld possible, and an anonymous referee for helpful comments. D.W. would also like to thank the mathematics department of the University of Maryland for its hospitality during the visit that initiated this project.

2 Explicit coordinates in SU(2, n)

Notation 2.1. We realize SU(2, n) as isometries of the indefinite Hermitian form

⟨v | w⟩ = v_{1}\overline{w_{n+2}} + v_{2}\overline{w_{n+1}} + ∑_{i=3}^{n} v_{i}\overline{w_{i}} + v_{n+1}\overline{w_{2}} + v_{n+2}\overline{w_{1}}

on ℂ^{n+2}. The virtue of this particular realization is that we may choose A to consist of the diagonal matrices in SU(2, n) that have nonnegative real entries, and N to consist of the upper-triangular matrices in SU(2, n) with only 1’s on the diagonal. Thus, the Lie algebra of AN is

𝔞 + 𝔫 = \{ (\begin{matrix}t_{1} & ϕ & x & η & ix \\ 0 & t_{2} & y & iy & −\overline{η} \\ 0 & 0 & 0 & −y^{†} & −x^{†} \\ 0 & 0 & 0 & −t_{2} & −\overline{ϕ} \\ 0 & 0 & 0 & 0 & −t_{1} \end{matrix}) | \begin{matrix}t_{1}, t_{2} ∈ ℝ, \\ ϕ, η ∈ ℂ, \\ x, y ∈ ℂ^{n−2}, \\ x, y ∈ ℝ \end{matrix} \} ,  (2.1)

where \overline{ϕ} or \overline{η} denotes the conjugate of a complex number ϕ or η, and x^{†} or y^{†} denotes the conjugate-transpose of a row vector x or y. Note that the first two rows of any element of 𝔞 + 𝔫 are sufficient to determine the entire matrix.

Notation 2.2. Because the exponential map is a diffeomorphism from 𝔫 to N, each element of N has a unique representation in the form exp u with u ∈ 𝔫. Thus, each element h of N determines corresponding values of ϕ, x, y, η, x and y (with t_{1} = t_{2} = 0). We write

ϕ_{h}, x_{h}, y_{h}, η_{h}, x_{h}, y_{h}

for these values.

Notation 2.3. We let α and β be the simple real roots of SU(2, n), defined by α(a) = a_{1}/a_{2} and β(a) = a_{2}, for an element a of A of the form

a = diag(a_{1}, a_{2}, 1, 1, …, 1, 1, a_{2}^{−1}, a_{1}^{−1}).

Thus, • the root space 𝔲_{α} is the ϕ-subspace in 𝔫,

• the root space 𝔲_{β} is the y-subspace in 𝔫,

• the root space 𝔲_{α+β} is the x-subspace in 𝔫,

• the root space 𝔲_{α+2β} is the η-subspace in 𝔫,

• the root space 𝔲_{2β} is the y-subspace in 𝔫, and

• the root space 𝔲_{2α+2β} is the x-subspace in 𝔫. Notation 2.4. For a given Lie algebra 𝔥 ⊂ 𝔫, we use 𝔷 to denote 𝔥 ∩ (𝔲_{α+2β} + 𝔲_{2α+2β} + 𝔲_{2β}). In other words,

𝔷 = { u ∈ 𝔥 | ϕ_{u} = 0 and x_{u} = y_{u} = 0 }.

(We remark that if ϕ_{u} = 0 for every u ∈ 𝔥, then [𝔥, 𝔥] ⊂ 𝔷 and 𝔷 is contained in the center of 𝔥.) Notation 2.5. For h ∈ SU(2, n), define

Δ(h) = det (\begin{matrix}h_{1,n+1} & h_{1,n+2} \\ h_{2,n+1} & h_{2,n+2} \end{matrix}) .

The following results collect some straightforward calculations that will be used repeatedly throughout the paper. Remark 2.6. For

u = (\begin{matrix}0 & ϕ & x & η & ix \\ 0 & 0 & y & iy & −\overline{η} \\ 0 & 0 & 0 & −y^{†} & −x^{†} \\ 0 & 0 & 0 & 0 & −\overline{ϕ} \\ 0 & 0 & 0 & 0 & 0 \end{matrix}) ∈ 𝔫 and h = exp u ∈ N,

we have

exp(u) = (\begin{matrix}1 & ϕ & x + \frac{1}{2}ϕy & \begin{matrix}η − \frac{1}{2}xy^{†} \\ + \frac{1}{2}iϕy − \frac{1}{6}ϕ|y|^{2} \end{matrix} & \begin{matrix}−\frac{1}{2}|x|^{2} − Re(ϕ\overline{η}) + \frac{1}{24}|ϕ|^{2}|y|^{2} \\ + i (x − \frac{1}{6}|ϕ|^{2}y + \frac{1}{3} Im(\overline{ϕ}xy^{†})) \end{matrix} \\ 0 & 1 & y & iy − \frac{1}{2}|y|^{2} & −\overline{η} − \frac{1}{2}yx^{†} − \frac{1}{2}i\overline{ϕ}y + \frac{1}{6} \overline{ϕ}|y|^{2} \\ 0 & 0 & Id & −y^{†} & −x^{†} + \frac{1}{2} \overline{ϕ}y^{†} \\ 0 & 0 & 0 & 1 & −\overline{ϕ} \\ 0 & 0 & 0 & 0 & 1 \end{matrix} )

and

Δ(h) = −|η|^{2} + xy − \frac{1}{4}|x|^{2}|y|^{2} + \frac{1}{4}|xy^{†}|^{2} − \frac{1}{6}|y|^{2} Re(η\overline{ϕ}) − \frac{1}{6}y Im(xy^{†}\overline{ϕ}) + \frac{1}{12}y^{2}|ϕ|^{2} − \frac{1}{144}|y|^{4}|ϕ|^{2} + i (\frac{1}{24}y|ϕ|^{2}|y|^{2} + Im(xy^{†}\overline{η}) + \frac{1}{2}x|y|^{2} + \frac{1}{2}y|x|^{2}) .

When ϕ = 0, these simplify to:

exp(u) = (\begin{matrix}1 & 0 & x & η − \frac{1}{2}xy^{†} & ix − \frac{1}{2}|x|^{2} \\ 0 & 1 & y & iy − \frac{1}{2}|y|^{2} & −\overline{η} − \frac{1}{2}yx^{†} \\ 0 & 0 & Id & −y^{†} & −x^{†} \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \end{matrix})

and

Δ(h) = \begin{matrix}−|η|^{2} + xy − \frac{1}{4}|x|^{2}|y|^{2} + \frac{1}{4}|xy^{†}|^{2} \\ + i (Im(xy^{†}\overline{η}) + \frac{1}{2}x|y|^{2} + \frac{1}{2}y|x|^{2}) . \end{matrix}

Similarly, when y = 0, we have

exp(u) = (\begin{matrix}1 & ϕ & x & η + \frac{1}{2}iϕy & \begin{matrix}−\frac{1}{2}|x|^{2} − Re(ϕ\overline{η}) \\ + i (x − \frac{1}{6}|ϕ|^{2}y) \end{matrix} \\ 0 & 1 & 0 & iy & −\overline{η} − \frac{1}{2}i\overline{ϕ}y \\ 0 & 0 & Id & 0 & −x^{†} \\ 0 & 0 & 0 & 1 & −\overline{ϕ} \\ 0 & 0 & 0 & 0 & 1 \end{matrix}) ,

and

Δ(h) = (xy + \frac{1}{12}|ϕ|^{2}y^{2} − |η|^{2}) + i (\frac{1}{2}|x|^{2}y) .  (2.2)

Remark 2.7. For

u = (\begin{matrix}0 & ϕ & x & η & ix \\ & 0 & y & iy & −\overline{η} \\ & & ⋯ \end{matrix}) and ũ = (\begin{matrix}0 & ϕ̃ & x̃ & η̃ & ix̃ \\ & 0 & ỹ & iỹ & −\overline{η̃} \\ & & ⋯ \end{matrix}) , (2.3)

we have

[u, ũ] = (\begin{matrix}0 & 0 & ϕỹ − ϕ̃y & −xỹ^{†} + x̃y^{†} + iϕỹ − iϕ̃y & −2i Im(xx̃^{†} + ϕ\overline{η̃} − ϕ̃\overline{η}) \\ & 0 & 0 & −2i Im(yỹ^{†}) & ỹx^{†} − yx̃^{†} + i\overline{ϕ}ỹ − i\overline{ϕ̃}y \\ & & & ⋯ \end{matrix}) ,

and

[[u, ũ], û] = (\begin{matrix}0 & 0 & 0 & −(ϕỹ − ϕ̃y)ŷ^{†} + 2iϕ̂ Im(yỹ^{†}) & ∗ \\ & 0 & 0 & 0 & ∗ \\ & & & ⋯ \end{matrix}) .  (2.4)

3 Preliminaries on Cartan-decomposition subgroups

Notation 3.1. We employ the usual Big Oh and little oh notation: for functions f_{1}, f_{2} on H, and a subset Z of H, we say f_{1} = O(f_{2}) for z ∈ Z if there is a constant C, such that, for all z ∈ Z with ∥z∥ > C, we have ∥f_{1}(z)∥ ≤ C∥f_{2}(z)∥. (The values of each f_{i} are assumed to belong to some finite-dimensional normed vector space, typically either ℂ or a space of complex matrices. Which particular norm is used does not matter, because all norms are equivalent up to a bounded factor.) We say f_{1} = o(f_{2}) for z ∈ Z if ∥f_{1}(z)∥/∥f_{2}(z)∥ → 0 as ∥z∥ → ∞. (For a sequence {h_{m}} in H, we may write h_{m} → ∞ if ∥h_{m}∥ → ∞.) Also, we write f_{1} ≍ f_{2} if f_{1} = O(f_{2}) and f_{2} = O(f_{1}).

Definition 3.2. Define ρ: SU(2, n) → GL(ℂ^{n+2} ∧ ℂ^{n+2}) by ρ(h) = h ∧ h, so ρ is the second exterior power of the standard representation of SU(2, n). Thus, we may define ∥ρ(h)∥ to be the maximum absolute value among the determinants of all the 2 × 2 submatrices of the matrix h.

We now introduce convenient notation for describing the image of a subgroup under the Cartan projection μ.

Notation 3.3. For functions f_{1}, f_{2} : ℝ^{+} → ℝ^{+}, and a subgroup H of SU(2, n), we write μ(H) ≈ [f_{1}(∥h∥), f_{2}(∥h∥)] if, for every sufficiently large C > 1, we have

μ(H) ≈ { a ∈ A^{+} | C^{−1}f_{1}(∥a∥) ≤ ∥ρ(a)∥ ≤ Cf_{2}(∥a∥) } .

(If f_{1} and f_{2} are monomials, or other very tame functions, then it does not matter which particular norm is used.)

We have A^{+} = { a ∈ A | a_{1,1} ≥ a_{2,2} ≥ 1 }, so, for a ∈ A^{+}, we have

∥a∥ = a_{1,1} ≤ a_{1,1} a_{2,2} = ∥ρ(a)∥ ≤ a_{1,1}^{2} = ∥a∥^{2}.

Thus A^{+} ≈ [∥h∥, ∥h∥^{2}], so, from Theorem 1.8, we see that H is a Cartandecomposition subgroup of G if and only if μ(H) ≈ [∥h∥, ∥h∥^{2}]. This observation, which is essentially due to Y. Benoist (in a much more general context, cf. [1, Lem. 2.4]), leads to the following result.

Proposition 3.4. (cf. [14, Prop. 3.14]) A closed, connected subgroup H of SU(2, n) is a Cartan-decomposition subgroup if and only if

1. there is a sequence {h_{m}} in H, such that h_{m} → ∞ as n → ∞, and ρ(h_{m}) ≍ ∥h_{m}∥^{2}; and

2. there is a sequence {h_{m}} in H, such that h_{m} → ∞ as n → ∞, and ρ(h_{m}) ≍ h_{m}.

Proof. (⇒) Fix k ∈ {1, 2}. There is a sequence a_{m} → ∞ in A^{+}, such that ∥ρ(a_{m})∥ ≍ ∥a_{m}∥^{k}. Also, because μ(H) ≈ A^{+} (see 1.8), there is a compact subset C of A, such that A^{+} ⊂ μ(H)C. Therefore, there is a sequence h_{m} → ∞ in H, such that a_{m} ∈ μ(h_{m})C. So

∥ρ(h_{m})∥ ≍ ∥ρ(μ(h_{m}))∥ ≍ ∥ρ(a_{m})∥ ≍ ∥a_{m}∥^{k} ≍ ∥μ(h_{m})∥^{k} ≍ ∥h_{m}∥^{k}.

(⇐) For simplicity, let us assume that, instead of only sequences, there are continuous curves h′_{1}(t) and h′_{2}(t), t ∈ [0, ∞), in H, such that h′_{k}(t) → ∞ as t → ∞, and, for each k = 1, 2, we have ρ_{2}(h′_{k}(t)) ≍ ∥h′_{k}(t)∥^{k}, for t ∈ [0, ∞). (In our applications of the proposition, this stronger assumption will be known to be true. Also, the proof of [14, Prop. 3.14] shows how to prove the existence of continuous curves from the existence of sequences, by using the theory of o-minimal structures [18, 19, 20].)

There is no loss of generality in assuming that H ⊂ AN (see 7.1). This implies that H is homeomorphic to some Euclidean space ℝ^{m} (with m ≥ 2), so it is easy to find a continuous and proper map Φ: [1, 2] × ℝ^{+} → H, such that Φ(k, t) = h′_{i}(t) for k = 1, 2 and for all t ∈ ℝ^{+}. Then an elementary homotopy argument shows that there is some C > 1, such that

{ a ∈ A^{+} | C∥a∥ ≤ ∥ρ(a)∥ ≤ ∥a∥^{2}/C } ⊂ μ[Φ([1, 2] × ℝ^{+})] ⊂ μ(H),

so μ(H) ≈ [∥h∥, ∥h∥^{2}]. Therefore H is a Cartan-decomposition subgroup of G.

The following result allows us to replace H by a conjugate subgroup whenever it is convenient.

Lemma 3.5. (cf. [1, Prop. 1.5], [8, Cor. 3.5]) Let H be any closed, connected subgroup of SU(2, n). For every g ∈ G, we have μ(g^{−1}Hg) ≈ μ(H).

In particular, H is a Cartan-decomposition subgroup if and only if g^{−1}Hg is a Cartan-decomposition subgroup.

4 When is the size of ρ(h) quadratic?

In this section, Proposition 4.1 is a list of types of subgroups of N that contain a sequence {h_{m}} with ρ(h_{m}) ≍ ∥h_{m}∥^{2}, and Proposition 4.3 is a list of types of subgroups of N that do not contain such a sequence. Then Proposition 4.4 shows that both lists are complete.

Proposition 4.1. Assume that G = SU(2, n). Let H be a closed, connected subgroup of N. There is a sequence h_{m} → ∞ in H with ρ(h_{m}) ≍ ∥h_{m}∥^{2} if either

1. there is an element u of 𝔥 with ϕ_{u} = 0, such that the vectors x_{u} and y_{u} are linearly independent over ℂ; or

2. there is an element z of 𝔷, such that |η_{z}|^{2} ≠ x_{z}y_{z}; or

3. there are elements u of 𝔥 and z of 𝔷, such that ϕ_{u} = 0, and x_{z}|y_{u}|^{2} + y_{z}|x_{u}|^{2} + 2 Im(x_{u}y_{u}^{†}\overline{η_{z}}) ≠ 0; or

4. there is an element u of 𝔥, such that ϕ_{u} ≠ 0, y_{u} = 0, y_{u} = 0, and |x_{u}|^{2} + 2 Re(ϕ_{u}\overline{η_{u}}) = 0; or

5. 𝔲_{2α+2β} ⊂ 𝔥 and there is an element u of 𝔥, such that ϕ_{u} ≠ 0, y_{u} ≠ 0, and y_{u} = 0; or 6. there are elements u and v of 𝔥, such that ϕ_{u} ≠ 0, y_{u} ≠ 0, ϕ_{v} = 0, y_{v} = 0,

x_{v} ≠ 0, y_{v} = 0, and x_{v}y_{u}^{†} = 0; or

7. 𝔲_{2α+2β} ⊂ 𝔷, and there are nonzero elements u and v of 𝔥, satisfying ϕ_{u} ≠ 0, y_{u} ≠ 0, ϕ_{v} = 0, y_{v} = 0, y_{v} ≠ 0, and x_{v}y_{u}^{†} = −iϕ_{u}y_{v}; or

8. dim 𝔥 = 3, 𝔷 = 𝔲_{2α+2β}, there exist u, v ∈ 𝔥 \ 𝔷, such that y_{u} ≠ 0, y_{v} = 0, y_{v} = 0, |x_{v}|^{2} + 2 Re(ϕ_{v}\overline{η_{v}}) > 0, and we have ϕ_{h} ≠ 0 for every h ∈ 𝔥 \ 𝔷.

Remark 4.2. In Conclusions (6) and (7), the restriction on x_{v}y_{u}^{†} is not necessary; it was included to avoid overlap with Conclusion (2). Namely, if x_{v}y_{u}^{†} ≠ −iϕ_{u}y_{v}, then [u, v] satisfies y = 0 and η ≠ 0, so Conclusion (2) holds. Also, it is not necessary to assume y_{v} ≠ 0 in Conclusion (7), because Conclusion (6) holds if y_{v} = 0 (and x_{v} ≠ 0). Thus, (6) and (7) may be replaced with the following:

(6*) there are elements u and v of 𝔥, such that ϕ_{u} ≠ 0, y_{u} ≠ 0, ϕ_{v} = 0, y_{v} = 0, x_{v} ≠ 0, and y_{v} = 0; or

(7*) 𝔲_{2α+2β} ⊂ 𝔷, and there are nonzero elements u and v of 𝔥, satisfying ϕ_{u} ≠ 0, y_{u} ≠ 0, ϕ_{v} = 0, y_{v} = 0, and x_{v} ≠ 0.

Proof. We separately consider each of the eight cases in the statement of the proposition.

(1) Let h^{t} = exp(tu). Replacing H by a conjugate under U_{α}, we may assume that x_{u} is orthogonal to y_{u}; that is, x_{u}y_{u}^{†} = 0. Then it is clear that ρ(h^{t}) ≍ Δ(h^{t}) ≍ t^{4} ≍ ∥h^{t}∥^{2}.

(2) Let h^{t} = exp(tz). We have h^{t} ≍ t and

Δ(h^{t}) = x_{tz}y_{tz} − |η_{tz}|^{2} = t^{2}(x_{z}y_{z} − |η_{z}|^{2}) ≍ t^{2}.

Therefore ρ(h^{t}) ≍ Δ(h^{t}) ≍ t^{2} ≍ ∥h^{t}∥^{2}.

(3) For any large t, let h = exp(tu+t^{2}z). Clearly, we have |x_{h}|+|y_{h}| = O(t) and |x_{h}| + |y_{h}| + |η_{h}| = O(t^{2}), so h = O(t^{2}).

We have

Im Δ(h^{t}) = t^{4}[\frac{1}{2}(2 Im(x_{u}y_{u}^{†}\overline{η_{z}} + x_{z}|y_{u}|^{2} + y_{z}|x_{u}|^{2})] + O(t^{3}) ≍ t^{4}.

Therefore, ρ(h^{t}) ≍ t^{4} ≍ ∥h^{t}∥^{2}.

(4) For any large t, let h = exp(tu). Then h_{1,n+2} = itx_{u}, so it is easy to see that h ≍ t. We have ρ(h) ≍ t^{2} ≍ ∥h∥^{2}.

(5) Replacing H by a conjugate (under a diagonal matrix), we may assume that ϕ_{u} = y_{u}. Then, by renormalizing, we may assume that ϕ_{u} = y_{u} = 1. Let z be the element of 𝔲_{2α+2β} with x_{z} = 1. By subtracting a multiple of z from u, we may assume x_{u} = 0. For any large t, let h = exp(6tu + 36t^{3}z), so h_{1,n+2} is real. We have

Re Δ(h) = (36t^{3})(6t) + \frac{1}{12}(6t)^{2}(6t)^{2} + O(t^{2}) ≍ t^{4},

so ρ(h) ≍ t^{4} ≍ ∥h∥^{2}.

(6) For each large t, let h be an element of exp(tu + ℝv), such that h_{1,n+2} is pure imaginary. (This exists because the sign of −\frac{1}{2}|x|^{2} is opposite that of \frac{1}{24}|ϕ|^{2}|y|^{2}.) We note that x_{h} ≍ t^{2} and |η_{h}| + |x_{h}| = O(t^{2}), but ϕ_{h} ≍ y_{h} ≍ t and |y_{h}| + |x_{h}y_{h}^{†}| = O(t). Thus h = O(t^{3}) and

ρ(h) ≍ Re Δ(h) = −\frac{1}{4}|x_{h}|^{2}|y_{h}|^{2} − \frac{1}{144}|y_{h}|^{4}|ϕ_{h}|^{2} + O(t^{5}) ≍ t^{6} ≍ ∥h∥^{2}.

(7) Because x_{v}y_{u}^{†} = −iϕ_{u}y_{v}, we have x_{v} ≠ 0, so, for any large t, we may choose h ∈ exp(tu + ℝv + 𝔲_{2α+2β}), such that h_{1,n+2} = 0. Thus ϕ_{h} ≍ y_{h} ≍ t, but x_{h} ≍ y_{h} ≍ t^{2} and |η_{h}| + |x_{h}| = O(t^{2}). Then (because h_{1,n+2} = 0) it is easy to verify that h = O(t^{3}). However

Im Δ(h) = \frac{1}{24}y_{h}|ϕ_{h}|^{2}|y_{h}|^{2} + \frac{1}{2}y_{h}|x_{h}|^{2} + O(t^{5}) ≍ t^{6}.

So ρ(h) ≍ ∥h∥^{2}.

(8) For any large t, choose s = O(1), such that Re(exp(su + tv)_{1,n+2}) = 0. (This is possible, because −\frac{1}{2}|x_{v}|^{2} − Re(ϕ_{v}\overline{η_{v}}) < 0.) Then we may choose h ∈ exp(su + tv + 𝔷), such that h_{1,n+2} = 0. Then ϕ_{h} ≍ t, |x_{h}| + |η_{h}| = O(t), and |y_{h}| + |y_{h}| = O(1), so we have ρ(h) ≍ t^{2} ≍ ∥h∥^{2}.

Proposition 4.3. Assume that G = SU(2, n). Let H be a closed, connected, nontrivial subgroup of N.

1. If dim 𝔥 = 1, 𝔥 = 𝔷, and we have |η_{h}|^{2} = x_{h}y_{h} for every h ∈ H, then ρ(h) ≍ h for every h ∈ H.

2. If ϕ_{h} = 0 and y_{h} = 0 for every h ∈ 𝔥, 𝔷 ⊂ 𝔲_{2α+2β}, and there is some u ∈ 𝔥, such that y_{u} ≠ 0, then μ(H) ≈ [∥h∥, ∥h∥^{3/2}], unless dim H = 1, in which case ρ(h) ≍ ∥h∥^{3/2} for every h ∈ H.

3. Suppose ϕ_{h} = 0 for every h ∈ 𝔥, and there is some λ ∈ ℂ, such that x_{h} = λy_{h} for every h ∈ H, and we have η_{z} = iλy_{z} and x_{z} = |λ|^{2}y_{z} for every z ∈ 𝔷.

(a) If there is some u ∈ 𝔥, such that x_{u} + |λ|^{2}y_{u} + 2 Im(λ\overline{η_{u}}) ≠ 0, then μ(H) ≈ [∥h∥, ∥h∥^{3/2}], unless dim H = 1, in which case ρ(h) ≍ ∥h∥^{3/2} for every h ∈ H.

(b) Otherwise, ρ(h) ≍ h for every h ∈ H.

4. If y_{h} = 0, y_{h} = 0, and |x_{h}|^{2} + 2 Re(ϕ_{h}\overline{η_{h}}) ≠ 0 for every h ∈ 𝔥 \ 𝔲_{2α+2β} (so 𝔷 ⊂ 𝔲_{2α+2β}), then ρ(h) ≍ h for every h ∈ H.

5. If 𝔷 = 0, there is some u ∈ 𝔥 and some nonzero ϕ_{0} ∈ ℂ, such that ϕ_{u} ≠ 0, and we have ϕ_{h} = ϕ_{0}y_{h} and y_{h} = 0, for every h ∈ 𝔥, then μ(H) ≈ [∥h∥, ∥h∥^{4/3}], unless dim H = 1, in which case, ρ(h) ≍ ∥h∥^{4/3} for every h ∈ H.

6. If dim 𝔥 ≤ 3, 𝔷 = 0, we have ϕ_{v} ≍ y_{v} and v = O(|ϕ_{v}| + |y_{v}|) for every v ∈ 𝔥, and there exists u ∈ 𝔥, such that ϕ_{u} ≠ 0, then ρ(h) ≍ ∥h∥^{3/2} for every h ∈ H.

7. If dim 𝔥 = 2, 𝔷 = 𝔲_{2α+2β}, and ϕ_{h} ≠ 0 and y_{h} ≠ 0 for every h ∈ 𝔥 \ 𝔷, then μ(H) ≈ [∥h∥, ∥h∥^{3/2}].

Proof. We separately consider each of the seven cases in the statement of the proposition.

(1) Because Δ(h) = 0 for every h ∈ H, it is clear that ρ(h) ≍ h for every h ∈ H.

(2) We have |η_{h}|+|y_{h}| = O(x_{h}), so h_{1,n+2} ≍ |x_{h}|^{2} +|x_{h}| and h_{i,j} = O(x_{h}) = O(|h_{1,n+2}|^{1/2}) whenever (i, j) ≠ (1, n + 2). Thus, ρ(h) = O(∥h∥^{3/2}).

We have ρ(exp(tu)) ≍ Im Δ(exp(tu)) ≍ t^{3} ≍ ∥ exp(tu)∥^{3/2}. If dim H > 1, then there is some nonzero v ∈ 𝔥, such that y_{v} = 0. Then, for h ∈ exp(ℝv), we have ρ(h) ≍ |x_{h}|^{2} + |x_{h}| ≍ h.

(3) Replacing H by a conjugate under U_{α}, we may assume that λ = 0, so x_{h} = 0 for every h ∈ H, and η_{z} = x_{z} = 0 for every z ∈ 𝔷 (which means 𝔷 ⊂ 𝔲_{2β}). Therefore, the Weyl reflection corresponding to the root α conjugates 𝔥 to a subalgebra either of type (2) or of type (4), depending on whether or not there is some u ∈ 𝔥, such that x_{u} + |λ|^{2}y_{u} + 2 Im(λ\overline{η_{u}}) ≠ 0.

(4) By assumption, the quadratic form |x|^{2} +2 Re(ϕ\overline{η}) is definite on 𝔥/𝔷, so |x|^{2} + |ϕ|^{2} + |η|^{2} = O(|x|^{2} + 2 Re(ϕ\overline{η})). Therefore, h_{i,j} = O(|h_{1,n+2}|^{1/2}) whenever (i, j) ≠ (1, n + 2). Furthermore, h_{i,j} = O(1) whenever i ≠ 1 and j ≠ n + 2. Thus, ρ(h) ≍ h.

(5) For any sequence {h_{m}} → ∞ in H, we write ϕ_{m}, x_{m}, y_{m}, y_{m}, η_{m}, x_{m} for ϕ_{h_{m}}, etc.

We have ϕ_{m} ≍ y_{m}. If x_{m} = O(|y_{m}|^{3/2}), then ρ(h_{m}) ≍ Re Δ(h_{m}) ≍ y_{m}^{4} ≍ ∥h_{m}∥^{4/3}. (This completes the proof if dim H = 1.) If |y_{m}|^{3/2} = o(x_{m}), then h_{m} ≍ h_{1,n+2} ≍ |x_{m}|^{2}, but h_{i,j} = O(|x_{m}| + y_{m}^{2}) = O(|x_{m}|^{4/3}) whenever (i, j) ≠ (1, n + 2), and h_{i,j} = O(y_{m}) = O(|x_{m}|^{2/3}) whenever i ≠ 1 and j ≠ n + 2. Therefore

ρ(h_{m}) = O(|x_{m}|^{2}|x_{m}|^{2/3} + |x_{m}|^{4/3}|x_{m}|^{4/3}) = O(|x_{m}|^{8/3}) = O(∥h_{m}∥^{4/3}).

If dim H > 1, then there is some (large) h ∈ H with y_{h} = 0 (and hence ϕ_{h} = 0). Thus ρ(h) ≍ |x_{h}|^{2} ≍ h.

(6) For any sequence {h_{m}} → ∞ in H, we show that ρ(h_{m}) ≍ Δ(h_{m}) ≍ ∥h_{m}∥^{3/2}. We write ϕ_{m}, x_{m}, y_{m}, y_{m}, η_{m}, x_{m} for ϕ_{h_{m}}, etc.

If y_{m} = o(ϕ_{m}^{2}), then h_{1,n+2} ≍ ϕ_{m}^{4}, but h_{i,j} = O(ϕ_{m}^{3}) whenever (i, j) ≠ (1, n + 2), and h_{i,j} = O(ϕ_{m}^{2}) whenever i ≠ 1 and j ≠ n + 2. Thus, ρ(h_{m}) ≍ Re Δ(h_{m}) ≍ ϕ_{m}^{6} ≍ ∥h_{m}∥^{3/2}.

We may now assume that ϕ_{m}^{2} = O(y_{m}). Thus, there is some v ∈ 𝔥, such that ϕ_{v} = 0 and y_{v} = 1. (Note that, because y_{v} ≍ ϕ_{v}, we have y_{v} = 0.) Because [u, v] ∈ 𝔷 = 0, we must have η_{[u,v]} = 0, so x_{v}y_{u}^{†} = −iϕ_{u}y_{v} ≠ 0. In particular, x_{v} ≠ 0, so x_{m} ≍ y_{m}.

We have h_{1,n+2} = O(|x_{m}|^{2}) = O(y_{m}^{2}), but h_{i,j} = O(|ϕ_{m}y_{m}| + |y_{m}|) = O(|y_{m}|^{3/2}) whenever (i, j) ≠ (1, n + 2), and h_{i,j} = O(y_{m}) whenever i ≠ 1 and j ≠ n + 2. Thus, h_{m} = O(y_{m}^{2}) and ρ(h_{m}) = O(y_{m}^{3}).

Furthermore, we have

Im Δ(h_{m}) = \frac{1}{24}y_{m}|ϕ_{m}|^{2}|y_{m}|^{2} + \frac{1}{2}y_{m}|x_{m}|^{2} + O(y_{m}^{2}ϕ_{m}) ≍ y_{m}^{3},

because y_{m}|x_{m}|^{2} ≍ y_{m}^{3}, and the terms \frac{1}{24}y_{m}|ϕ_{m}|^{2}|y_{m}|^{2} and \frac{1}{2}y_{m}|x_{m}|^{2} cannot cancel (since they both have the same sign as y_{m}). We conclude that ρ(h_{m}) ≍ Δ(h_{m}) ≍ y_{m}^{3}.

All that remains is to show y_{m}^{2} = O(h_{m}). If ϕ_{m}^{2} = o(y_{m}), then

Re h_{1,n+2} = −\frac{1}{2}|x|^{2} + O(ϕ_{m}^{2}y) ≍ y_{m}^{2},

as desired. If y_{m} = o(ϕ_{m}^{2}), then

Re h_{1,n+2} ≍ o(ϕ_{m}^{4}) + o(ϕ_{m}^{3}) + |ϕ_{m}^{4}| ≍ ϕ_{m}^{4},

so y_{m} = o(ϕ_{m}^{2}) = o(ϕ_{m}^{4}) = o(h_{m}), as desired. Thus, we may assume that y_{m} ≍ ϕ_{m}^{2}. Because x_{m} = y_{m}x_{v} + O(ϕ_{m}) and x_{v}y_{m}^{†} = −iϕ_{m}y_{v} = −iϕ_{m}, we have

Im(h_{1,n+2}) = O(y_{m}) − \frac{1}{6}|ϕ_{m}|^{2}y_{m} + [\frac{1}{3} Im(\overline{ϕ_{m}}(y_{m}x_{v})y_{m}^{†}) + O(ϕ_{m}^{3})]
= −\frac{1}{6}|ϕ_{m}|^{2}y_{m} − \frac{1}{3}|ϕ_{m}|^{2}y_{m} + O(ϕ_{m}^{3}) ≍ y_{m}^{2},

as desired.

(7) For z ∈ 𝔷, we have ρ(z) ≍ z. For u ∈ 𝔥 \ 𝔷 with y_{u} ≠ 0, we have ρ(exp(tu)) ≍ t^{6} ≍ ∥ exp(tu)∥^{3/2}. All that remains is to show ρ(h) = O(∥h∥^{3/2}) for every h ∈ H.

Note that ϕ_{h} ≍ y_{h}, and |x_{h}| + |η_{h}| + |y_{h}| = O(ϕ_{h}). If ϕ_{h} = O(1), then it is obvious that ρ(h) ≍ h. Thus, we may assume |ϕ_{h}| → ∞. Then, because Re h_{1,n+2} ≍ |ϕ_{h}|^{2}|y_{h}|^{2} ≍ ϕ_{h}^{4}, but h_{i,j} = O(ϕ_{h}|y_{h}|^{2}) = O(ϕ_{h}^{3}) whenever (i, j) ≠ (1, n + 2), and h_{i,j} = O(ϕ_{h}^{2}) whenever i ≠ 1 and j ≠ n + 2, we have

ρ(h) = O[|ϕ_{h}|^{4}|ϕ_{h}|^{2} + (|ϕ_{h}|^{3})^{2}] = O(|ϕ_{h}|^{6}) = O(|h_{1,n+2}|^{3/2}) = O(∥h∥^{3/2}).

Proposition 4.4. Assume that G = SU(2, n). Let H be a closed, connected, nontrivial subgroup of N.

1. There is a sequence h_{m} → ∞ in H with ρ(h_{m}) ≍ ∥h_{m}∥^{2} if and only if H is one of the subgroups described in Proposition 4.1.

2. There is not a sequence h_{m} → ∞ in H with ρ(h_{m}) ≍ ∥h_{m}∥^{2} if and only if H is one of the subgroups described in Proposition 4.3.

Proof. It suffices to show that H is described in either Proposition 4.1 or Proposition 4.3.

We may assume

|η_{z}|^{2} = x_{z}y_{z} for every z ∈ 𝔷  (4.1)

(otherwise, 4.1(2) holds). Because |η|^{2} − xy is a quadratic form of signature (3, 1) on 𝔲_{2β} + 𝔲_{α+2β} + 𝔲_{2α+2β}, then we must have dim 𝔷 ≤ 1. Thus, we may assume 𝔥 ≠ 𝔷 (otherwise 4.3(1) holds).

Case 1. Assume ϕ_{h} = 0 and y_{h} = 0 for every h ∈ H (and 𝔥 ≠ 𝔷). We may assume y_{z} = 0 for every z ∈ 𝔷, for, otherwise, 4.1(3) holds. Then, from Eq. (4.1), we have η_{z} = 0 for every z ∈ 𝔷. Thus, 𝔷 ⊂ 𝔲_{2α+2β}. We may assume y_{h} = 0 for every h ∈ H, for otherwise Conclusion 4.3(2) holds. We conclude that 4.3(4) holds.

Case 2. Assume ϕ_{h} = 0 for every h ∈ H, and there is some u ∈ 𝔥 with y_{u} ≠ 0. We may assume that x_{h} and y_{h} are linearly dependent over ℂ for every h ∈ H (otherwise 4.1(1) holds). In particular, there exists λ ∈ ℂ, such that x_{u} = λy_{u}.

Subcase 2.1. Assume 𝔷 = 0.

Subsubcase 2.1.1. Assume there exists v ∈ 𝔥, such that either x_{v} ∉ ℂy_{u} or y_{v} ∉ ℂy_{u}. We may assume there exists w ∈ 𝔥, such that x_{w} ≠ λy_{w} (otherwise 4.3(3) holds). Furthermore, by adding a small linear combination of u and v to w, we may assume that y_{w} ≠ 0 and that either x_{w} ∉ ℂy_{u} or y_{w} ∉ ℂy_{u}. Because x_{w} and y_{w} are linearly dependent, there exists λ_{1} (≠ λ) such that x_{w} = λ_{1}y_{w}. (Then note that we must have y_{w} ∉ ℂy_{u}.) Then

x_{u+w} = x_{u} + x_{w} = λy_{u} + λ_{1}y_{w} ∉ ℂ(y_{u} + y_{w}) = ℂy_{u+w}

(because λ ≠ λ_{1} and {y_{u}, y_{w}} is linearly independent over ℂ). This contradicts the fact that x_{u+w} and y_{u+w} are linearly dependent over ℂ.

Subsubcase 2.1.2. Assume x_{h}, y_{h} ∈ ℂy_{u}, for every h ∈ 𝔥. For each h ∈ 𝔥, there exist λ_{x}, λ_{y} ∈ ℂ, such that x_{h} = λ_{x}y_{u} and y_{h} = λ_{y}y_{u}. Because 𝔷 = 0, we must have y_{[h,u]} = 0, so Im(y_{h}y_{u}^{†}) = 0, which means that λ_{y} is real. We must also have η_{[h,u]} = 0, so

0 = −x_{h}y_{u}^{†} + x_{u}y_{h}^{†} = (−λ_{x} + λ\overline{λ_{y}})|y_{u}|^{2} = (−λ_{x} + λλ_{y})|y_{u}|^{2}.

Thus λ_{x} = λλ_{y}, so

x_{h} = λ_{x}y_{u} = λλ_{y}y_{u} = λy_{h}.

Therefore 4.3(3) holds.

Subcase 2.2. Assume 𝔷 ≠ 0. We show that either 4.1(2), 4.1(3) or 4.3(3) holds. Straightforward calculations show that conditions 4.1(2), 4.1(3) and 4.3(3) are invariant under conjugation by U_{α}, so we may assume that λ = 0; that is, x_{u} = 0. Thus, we may assume x_{z} = 0 for every z ∈ 𝔷, for, otherwise, 4.1(3) holds. Then we may assume η_{z} = 0 for every z ∈ 𝔷, for, otherwise, 4.1(2) holds; therefore 𝔷 = U_{2β}. We may now assume x_{h} = 0 for every h ∈ 𝔥, for, otherwise, 4.1(3) holds. Thus, 4.3(3) holds (with λ = 0).

Case 3. Assume there exists u ∈ 𝔥 with ϕ_{u} ≠ 0. We claim that 𝔷 ⊂ 𝔲_{2α+2β}. If not, then there is some z ∈ 𝔷, such that either η_{z} ≠ 0 or y_{z} ≠ 0. If y_{z} = 0, then |η_{z}|^{2} ≠ 0 = x_{z}y_{z}, so 4.1(2) holds. On the other hand, if y_{z} ≠ 0, then, letting z′ = [u, z], we have y_{z′} = 0 and η_{z′} ≠ 0, so 4.1(2) holds once again.

Subcase 3.1. Assume y_{h} = 0 for every h ∈ 𝔥. We may assume that there is some v ∈ 𝔥, such that y_{v} ≠ 0 (otherwise, either 4.1(4) or 4.3(4) holds). Then we may assume 𝔷 = 0 (otherwise, 4.1(5) holds).

We claim that 4.3(5) holds. If not, then there is some w ∈ 𝔥, such that ϕ_{w} ≠ 0 and y_{w} = 0. Then η_{[v,w]} ≠ 0, which contradicts the assumption that 𝔷 = 0.

Subcase 3.2. Assume there is some v ∈ 𝔥, such that y_{v} ≠ 0.

Subsubcase 3.2.1. Assume 𝔷 = 𝔲_{2α+2β}. Suppose, for the moment, that there exists w ∈ 𝔥 \ 𝔷 with ϕ_{w} = 0. We may assume that y_{w} = 0 (otherwise, 4.1(3) holds). Therefore x_{w} ≠ 0, so 4.1(7*) holds.

We may now assume that ϕ_{w} ≠ 0 for every w ∈ 𝔥 \ 𝔷. This implies that x, y, η, and y are functions of ϕ; in particular, dim 𝔥 ≤ 3. Also, because 𝔷 ≠ 0 and u, v ∉ 𝔷, we must have dim 𝔥 ≥ 2.

We claim dim 𝔥 = 2 (so 4.3(7) holds). If not, then dim 𝔥 = 3, so there exist u, w ∈ 𝔥, such that ϕ_{u} = 1 and ϕ_{w} = i. Because ϕ_{[u,w]} = 0, we must have [u, w] ∈ 𝔲_{2α+2β}. Therefore 0 = x_{[u,w]} = y_{w} − iy_{u}, so y_{w} = iy_{u}. Furthermore,

0 = y_{[u,w]} = −2i Im(y_{u}y_{w}^{†}) = −2i Im(−i|y_{u}|^{2}) = −2i|y_{u}|^{2},

so y_{u} = 0. Then y_{w} = iy_{u} is also 0. This implies y_{h} = 0 for every h ∈ H. This contradicts the fact that y_{v} ≠ 0.

Subsubcase 3.2.2. Assume 𝔷 = 0. Lemma 4.5 below implies that either 4.3(6) or 4.1(6*) holds.

Lemma 4.5. Let H be a closed, connected subgroup of N, such that 𝔷 = 0, and assume there exist u, v ∈ 𝔥, such that ϕ_{u} ≠ 0 and y_{v} ≠ 0. Then either H is described in 4.3(6) (and in 5.2(4), which is the same), or H is a Cartandecomposition subgroup (and is described in 4.1(6*) and 5.1(2)).

Proof. Let us begin by establishing that ϕ_{h} ≍ y_{h} for h ∈ 𝔥. If not, then we may assume either that y_{u} = 0 or that ϕ_{v} = 0. Then, because [[u, v], v] ∈ 𝔷 = 0, we see from Eq. (2.4) that

0 = −(ϕ_{u}y_{v} − ϕ_{v}y_{u})y_{v}^{†} + 2iϕ_{v} Im(y_{u}y_{v}^{†}) = −ϕ_{u}|y_{v}|^{2} − 0 + 0 ≠ 0.

This contradiction establishes the claim.

Case 1. Assume there is a nonzero w ∈ 𝔥, such that ϕ_{w} = 0 and y_{w} = 0. Note, from the preceding paragraph, that y_{w} = 0. Then, because 𝔷 = 0, we must have x_{w} ≠ 0. Therefore, 4.1(6*) and 5.1(2) hold, so μ(H) ≈ [∥h∥, ∥h∥^{2}], so H is a Cartan-decomposition subgroup.

Case 2. Assume there does not exist such an element w ∈ 𝔥. Then H is described in 4.3(6) and in 5.2(4).

5 When is the size of ρ(h) linear?

In this section, Proposition 5.1 is a list of types of subgroups of N that contain a sequence {h_{m}} with ρ(h_{m}) ≍ h_{m}, and Proposition 5.2 is a list of types of subgroups of N that do not contain such a sequence. Then Proposition 5.3 shows that both lists are complete.

Proposition 5.1. Assume that G = SU(2, n). Let H be a closed, connected subgroup of N. There is a sequence h_{m} → ∞ in H with ρ(h_{m}) ≍ h_{m} if either

1. there is a nonzero element z of 𝔷 with |η_{z}|^{2} = x_{z}y_{z}; or

2. there is an element u of 𝔥, such that ϕ_{u} = 0, dim_{ℂ}⟨x, y⟩ = 1, and

x_{u}|y_{u}|^{2} + y_{u}|x_{u}|^{2} + 2 Im(x_{u}y_{u}^{†}\overline{η_{u}}) = 0;

or 3. there is an element h of H with y_{h} = 0, y_{h} = 0 and |x_{h}|^{2} + 2 Re(ϕ_{h}\overline{η_{h}}) ≠ 0;

or

4. there are elements u of 𝔥 and z of 𝔷, such that ϕ_{u} ≠ 0, y_{u} = 0, y_{u} ≠ 0, η_{z} ≠ 0, and y_{z} = 0; or

5. there are nonzero elements u of 𝔥 and z of 𝔷, such that ϕ_{u} ≠ 0, y_{u} ≠ 0, y_{z} = 0, ϕ_{u}\overline{η_{z}} is real, and

x_{z}|y_{u}|^{2} − ϕ_{u}y_{u}\overline{η_{z}} + 2 Im(\overline{η_{z}}x_{u}y_{u}^{†}) = 0.

Proof. We separately consider each of the five cases in the statement of the proposition.

(1) From 4.3(1), we have ρ(h) ≍ h for all h ∈ exp(ℝz).

(2) Replacing H by a conjugate under ⟨U_{α}, U_{−α}⟩, we may assume that y_{u} = 0 (and x_{u} ≠ 0). Then, from the assumption of this case, we know that y_{u} is also 0. Therefore, 4.3(4) implies that ρ(h) ≍ h for all h ∈ exp(ℝu).

(3) From 4.3(4), we have ρ(h) ≍ h for all h ∈ exp(ℝu).

(4). For any large t, choose h ∈ exp(tu + 𝔷), such that x_{h}y_{h} + \frac{1}{12}|ϕ_{h}|^{2}y_{h}^{2} − |η_{h}|^{2} = 0. Note that η_{h} ≍ |ϕ_{h}y_{h}| ≍ t^{2}, so h ≍ Re h_{1,n+2} ≍ t^{3}, but h_{i,j} = O(t^{2}) whenever (i, j) ≠ (1, n + 2), and h_{i,j} = O(t) whenever i ∉ {1, 2} or j ∉ {n + 1, n + 2}. From the choice of h, we have

Δ(h) = 0 + i (\frac{1}{2}|x_{h}|^{2}y_{h}) = O(t^{3}) = O(h),

so it is not difficult to see that ρ(h) ≍ h.

(5) Replacing 𝔥 by a conjugate, we may assume u ∈ 𝔲_{α} + 𝔲_{β}. (First, conjugate by an element of U_{β} to make y_{u} = 0. Then conjugate by an element of U_{α} to make x_{u} orthogonal to y_{u}. Then conjugate by an element of U_{β} that centralizes y_{u}, to make x_{u} = 0. Then conjugate by an element of U_{α+β} to make η_{u} = 0. Then conjugate by an element of U_{α+2β} to make x_{u} = 0.) Then, by assumption, we must have x_{z} = 0, because y_{u} = 0 and x_{u} = 0.

Furthermore, replacing 𝔥 by a conjugate under a diagonal matrix (that belongs to G), we may assume that ϕ_{u} and y_{u} are real. Then η_{z} must also be real (because ϕ_{u}\overline{η_{z}} is real). Thus, we see that u, z ∈ 𝔰𝔬(2, n). So [14, Thm. 5.3(1)] implies that H is a Cartan-decomposition subgroup.

Proposition 5.2. Assume that G = SU(2, n). Let H be a closed, connected, nontrivial subgroup of N such that

|η_{z}|^{2} ≠ x_{z}y_{z}, for every nonzero z ∈ 𝔷.  (5.1)

1. If 𝔥 = 𝔷 (so dim H ≤ 3), then ρ(h) ≍ ∥h∥^{2} for every h ∈ H.

2. If ϕ_{h} = 0 and dim_{ℂ}⟨x_{u}, y_{u}⟩ ≠ 1 for every h ∈ 𝔥, then ρ(h) ≍ ∥h∥^{2} for every h ∈ H.

3. If ϕ_{h} = 0 for every h ∈ 𝔥, there exist nonzero u and v in 𝔥, such that dim_{ℂ}⟨x_{u}, y_{u}⟩ ≠ 1 and dim_{ℂ}⟨x_{v}, y_{v}⟩ = 1, and x_{v}|y_{v}|^{2}+y_{v}|x_{v}|^{2}+2 Im(x_{v}y_{v}^{†}\overline{η_{v}}) ≠ 0 for every such v ∈ 𝔥, then μ(H) ≈ [∥h∥^{3/2}, ∥h∥^{2}].

4. If dim 𝔥 ≤ 3, 𝔷 = 0, we have ϕ_{v} ≍ y_{v} and v = O(|ϕ_{v}| + |y_{v}|) for every v ∈ 𝔥, and there exists u ∈ 𝔥, such that ϕ_{u} ≠ 0, then ρ(h) ≍ ∥h∥^{3/2} for every h ∈ H.

5. If dim 𝔥 ≤ 2 and ϕ_{h} ≠ 0, y_{h} = 0, y_{h} = 0, and |x_{h}|^{2} + 2 Re(ϕ_{h}\overline{η_{h}}) = 0 for every nonzero h ∈ 𝔥, then ρ(h) ≍ ∥h∥^{2} for every h ∈ H.

6. If dim 𝔥 = 2 and there exist nonzero u ∈ 𝔥 and z ∈ 𝔷, such that ϕ_{u} ≠ 0, y_{u} ≠ 0, y_{z} = 0, ϕ_{u}\overline{η_{z}} is real, and x_{z}|y_{u}|^{2} − ϕ_{u}y_{u}\overline{η_{z}} + 2 Im(\overline{η_{z}}x_{u}y_{u}^{†}) ≠ 0, then μ(H) ≈ [∥h∥^{5/4}, ∥h∥^{2}].

7. If dim 𝔥 = 1, and we have ϕ_{h} = 0, dim_{ℂ}⟨x_{h}, y_{h}⟩ = 1, and

x_{h}|y_{h}|^{2} + y_{h}|x_{h}|^{2} + 2 Im(x_{h}y_{h}^{†}\overline{η_{h}}) ≠ 0

for every nonzero h ∈ 𝔥, then ρ(h) ≍ ∥h∥^{3/2} for every h ∈ H.

8. If dim 𝔥 = 1, and ϕ_{h} ≠ 0, y_{h} = 0, and y_{h} ≠ 0, for every nonzero h ∈ 𝔥, then ρ(h) ≍ ∥h∥^{4/3} for every h ∈ H.

Proof. We separately consider each of the eight cases in the statement of the proposition.

(1) From Eq. (5.1), we know that the quadratic form |η|^{2} −xy is anisotropic on 𝔷 = 𝔥, so

Δ(h) = |η_{h}|^{2} − x_{h}y_{h} ≍ |η_{h}|^{2} + x_{h}^{2} + y_{h}^{2} ≍ ∥h∥^{2}.

(2) Because dim_{ℂ}⟨x_{u}, y_{u}⟩ ≠ 1, we have

|x_{h}|^{2}|y_{h}|^{2} − |x_{h}y_{h}^{†}|^{2} ≍ |x_{h}|^{4} + |y_{h}|^{4},

so Lemma 5.4 implies ρ(h) ≍ ∥h∥^{2}.

(7) From either Proposition 4.3(2) or 4.3(3a) (depending on whether y_{h} is 0 or not), we have ρ(h) ≍ ∥h∥^{3/2} for every h ∈ H.

(3) From Lemma 5.4, we have ∥h∥^{3/2} = O(ρ(h)). From (2), we see that ρ(h) ≍ ∥h∥^{2} for h ∈ exp(ℝu). From (7), we see that ρ(h) ≍ ∥h∥^{3/2} for h ∈ exp(ℝv).

(4) See Proposition 4.3(6).

(5) Because Re h_{1,n+2} = 0, it is easy to see that ρ(h) ≍ ϕ_{h}^{2} ≍ ∥h∥^{2}.

(6) Replacing H by a conjugate, we may assume x_{u} = 0 and y_{u} = 0. Therefore, x_{h} = 0 and y_{h} = 0 for every h ∈ H. Thus

x_{z}|y_{u}|^{2} = x_{z}|y_{u}|^{2} − ϕ_{u}y_{u}\overline{η_{z}} + 2 Im(\overline{η_{z}}x_{u}y_{u}^{†}) ≠ 0,

so x_{z} ≠ 0. From Eq. (5.1), we know η_{z} ≠ 0.

We have ρ(tz) ≍ ∥tz∥^{2} (see 5.2(1)). Because ϕ_{u} is a real multiple of η_{z}, we may let h be a large element of H,

such that η_{h} = −|y_{h}|^{2}ϕ_{h}/12 + O(ϕ_{h}). (So y_{h} ≍ ϕ_{h} and x_{h} ≍ η_{h} ≍ ϕ_{h}^{3}.) Then

Δ(h) = (−|η_{h}|^{2} − \frac{1}{6}|y_{h}|^{2}η_{h}\overline{ϕ_{h}} − \frac{1}{144}|y_{h}|^{4}|ϕ_{h}|^{2}) + i (\frac{1}{2}x_{h}|y_{h}|^{2})
= O(ϕ_{h}^{4}) + i (\frac{1}{2}x_{h}|y_{h}|^{2}) ≍ ϕ_{h}^{5}.

It is clear that all other matrix entries of ρ(h) are O(ϕ_{h}^{5}). Thus, we have ρ(h) ≍ ϕ_{h}^{5} ≍ ∥h∥^{5/4}.

Now suppose there is a sequence h_{m} → ∞ in H with ρ(h_{m}) = o(∥h_{m}∥^{5/4}).

Case 1. Assume η_{m} = o(ϕ_{m}^{3}). We have h_{m} ≍ ϕ_{m}^{4}, so

ϕ_{m}^{6} ≍ Re Δ(h_{m}) = O(ρ(h_{m})) = o(∥h_{m}∥^{5/4}) = o(ϕ_{m}^{5}).

This is a contradiction.

Case 2. Assume ϕ_{m}^{3} = o(η_{m}). We have h_{m} ≍ Re h_{1,n+2} ≍ ϕ_{m}η_{m}, so

η_{m}^{2} ≍ Re Δ(h_{m}) = O(ρ(h_{m})) = o(∥h_{m}∥^{5/4}) = o(∥h_{m}∥^{3/2}) = o(|ϕ_{m}η_{m}|^{3/2}) = o(η_{m}^{2}).

This is a contradiction.

Case 3. Assume η_{m} ≍ ϕ_{m}^{3}. We have h_{m} = O(ϕ_{m}^{4}), so

ϕ_{m}^{5} ≍ x_{m}|y_{m}|^{2} ≍ Im Δ(h_{m}) = O(ρ(h_{m})) = o(∥h_{m}∥^{5/4}) = o(ϕ_{m}^{5}).

This is a contradiction.

(8) See Proposition 4.3(5).

Proposition 5.3. Assume that G = SU(2, n). Let H be a closed, connected, nontrivial subgroup of N.

1. There is a sequence h_{m} → ∞ in H with ρ(h_{m}) ≍ h_{m} if and only if H is one of the subgroups described in Proposition 5.1.

2. There is not a sequence h_{m} → ∞ in H with ρ(h_{m}) ≍ ∥h_{m}∥^{2} if and only if H is one of the subgroups described in Proposition 5.2.

Proof. It suffices to show that H is described in either Proposition 5.1 or Proposition 5.2.

We may assume (5.1) holds (otherwise, Conclusion 5.1(1) holds).

Case 1. Assume ϕ_{h} = 0 for every h ∈ H. We may assume there exists v ∈ 𝔥, such that dim_{ℂ}⟨x_{v}, y_{v}⟩ = 1 (otherwise 5.2(2) holds). Furthermore, we may assume x_{v}|y_{v}|^{2} +y_{v}|x_{v}|^{2} +2 Im(x_{v}y_{v}^{†}\overline{η_{v}}) ≠ 0 for every such v (otherwise 5.1(2) holds). Then we may assume dim_{ℂ}⟨x_{u}, y_{u}⟩ = 1 for every nonzero u ∈ 𝔥 (otherwise 5.2(3) holds).

The argument in Subsubcase 2.1.1 of the proof of Proposition 4.3 implies there exists λ ∈ ℂ, such that, for every h ∈ H, we have x_{h} = λy_{h} (or viceversa: for every h, we have y_{h} = λx_{h}). Thus, replacing H by a conjugate under ⟨U_{α}, U_{−α}⟩, we may assume x_{h} = 0 for every h ∈ H.

If dim H > 1, then there is some nonzero u ∈ 𝔥, such that x_{h} = 0. This contradicts the fact that x_{v}|y_{v}|^{2} + y_{v}|x_{v}|^{2} + 2 Im(x_{v}y_{v}^{†}\overline{η_{v}}) ≠ 0. Thus, we conclude that dim H = 1, so 5.2(7) holds.

Case 2. Assume the projection of 𝔥 to 𝔲_{α} is one-dimensional. Replacing H by a conjugate under A, we may assume ϕ_{h} is real for every h ∈ H. Fix some u ∈ 𝔥, such that ϕ_{u} ≠ 0.

We may assume that 𝔲_{2α+2β} ⊄ 𝔥 (otherwise Conclusion 5.1(1) holds). Therefore [𝔥, u] must be zero, so y_{z} = 0 and η_{z} is a nonzero real, for every nonzero z ∈ 𝔷. (This implies dim 𝔷 ≤ 1.)

Subcase 2.1. Assume y_{h} = 0 for every h ∈ H. We may assume Conclusion 5.1(2) does not hold.

We claim that 𝔥 = ℝu + 𝔷. Suppose not. Then there is some v ∈ 𝔥, such that ϕ_{v} = 0 and x_{v} ≠ 0. Because Conclusion 5.1(2) does not hold, we must have y_{v} ≠ 0. Then [[v, u], u] is a nonzero element of 𝔲_{2α+2β}. (This can be seen easily by replacing H with a conjugate, so that u ∈ 𝔲_{α}.) This contradicts our assumption that 𝔲_{2α+2β} ⊄ 𝔥.

If y_{u} ≠ 0, then either Conclusion 5.2(8) or 5.1(4) holds (depending on whether 𝔷 is 0 or not). If y_{u} ≠ 0, then 5.1(3) or 5.2(6) holds.

Subcase 2.2. Assume the projection of 𝔥 to 𝔲_{β} is nontrivial. Then we may assume y_{u} ≠ 0.

Subsubcase 2.2.1. Assume there are nonzero v ∈ 𝔥 and z ∈ 𝔷, such that ϕ_{v} = 0, y_{v} = 0, and x_{v} ≠ 0. We may assume that Conclusion 5.1(5) does not hold. Therefore, for every real t, we must have

0 ≠ x_{z}|y_{u}|^{2} − ϕ_{u}(y_{u} + ty_{v})\overline{η_{z}} + 2 Im(\overline{η_{z}}(x_{u} + tx_{v})y_{u}^{†})
= t[−ϕ_{u}y_{v}\overline{η_{z}} + 2 Im(\overline{η_{z}}x_{v}y_{u}^{†})] + constant.

Thus, the coefficient of t must vanish, which (using the fact that η_{z} is real and nonzero) means

0 = −ϕ_{u}y_{v} + 2 Im(x_{v}y_{u}^{†}).  (5.2)
We have [u, v] ∈ 𝔷, so η_{[u,v]} is real. Thus,
0 = Im η_{[u,v]} = Im(x_{v}y_{u}^{†} + iϕ_{u}y_{v}) = Im(x_{v}y_{u}^{†}) + ϕ_{u}y_{v}.

Comparing this with Eq. (5.2), we conclude that ϕ_{u}y_{v} = 0. Therefore y_{v} = 0, so Conclusion 5.1(2) holds (for the element v).

Subsubcase 2.2.2. Assume there do not exist nonzero v ∈ 𝔥 and z ∈ 𝔷, such that ϕ_{v} = 0, y_{v} = 0, and x_{v} ≠ 0. We must have

y_{w} = 0 for every w ∈ 𝔥, such that ϕ_{w} = 0.  (5.3)

(Otherwise, we obtain a contradiction by setting v = [u, w] and z = [[u, w], w].) We may assume

y_{v} ≠ 0 for every v ∈ 𝔥 such that ϕ_{v} = 0, y_{v} = 0, and x_{v} ≠ 0.  (5.4)

(Otherwise, Conclusion 5.1(2) holds.)

We claim dim 𝔥 ≤ 2. If not, then there exist linearly independent v, w ∈ 𝔥, such that ϕ_{v} = ϕ_{w} = 0. From (5.3), we know that y_{v} = y_{w} = 0. By replacing with a linear combination, we may assume y_{w} = 0. Then, from (5.4), we know that x_{w} = 0, so w ∈ 𝔷. Because 𝔷 is (at most) one-dimensional, but v and w are linearly independent, we know that v ∉ 𝔷, so x_{v} ≠ 0. This contradicts the assumption of this subsubcase.

We may now assume dim 𝔥 = 2 (otherwise Conclusion 5.2(5) holds). Choose a nonzero v ∈ 𝔥, such that ϕ_{v} = 0. If x_{v} ≠ 0, then Conclusion 5.2(5) holds. If x_{v} = 0, then v ∈ 𝔷, so either Conclusion 5.1(5) or 5.2(6) holds.

Case 3. Assume the projection of 𝔥 to 𝔲_{α} is two-dimensional. We may assume 𝔷 = 0 (otherwise, 𝔲_{2α+2β} ⊂ 𝔥, so Conclusion 5.1(1) holds). We may assume y_{h} = 0 for every h ∈ H (otherwise Lemma 4.5 implies that either 5.2(5) or 5.1(2) applies. Therefore [𝔥, 𝔥] ⊂ 𝔷 = 0, so 𝔥 is abelian.

Let u, v ∈ 𝔥 with ϕ_{u} = 1 and ϕ_{v} = i. Then

0 = η_{[u,v]} = iy_{v} + y_{u},

so y_{u} = y_{v} = 0. Then, for every w ∈ 𝔥, we have 0 = η_{[u,w]} = iy_{w}, so y_{w} = 0. We may assume

|x_{h}|^{2} + 2 Re(ϕ_{h}\overline{η_{h}}) = 0  (5.5)

for every h ∈ 𝔥 (otherwise Conclusion 5.1(3) holds). This implies dim 𝔥 = 2 (otherwise, there is some w ∈ 𝔥 such that ϕ_{w} = 0 and x_{w} ≠ 0, and then Eq. (5.5) does not hold for h = u + tw when t is sufficiently large). Thus, Conclusion 5.2(5) holds.

Lemma 5.4. Let H be a closed, connected, nontrivial subgroup of N. Assume ϕ_{h} = 0 for every h ∈ 𝔥, that (5.1) holds, and that x_{v}|y_{v}|^{2}+y_{v}|x_{v}|^{2}+2 Im(x_{v}y_{v}^{†}\overline{η_{v}}) ≠ 0 for every v ∈ 𝔥 such that dim_{ℂ}⟨x_{v}, y_{v}⟩ = 1. Then ∥h∥^{3/2} = O(Δ(h)) for every h ∈ H.

Furthermore, Δ(h) ≍ ∥h∥^{2} whenever |x_{h}|^{2}|y_{h}|^{2} − |x_{h}y_{h}^{†}|^{2} ≍ |x_{h}|^{4} + |y_{h}|^{4}.

Proof. We have h ≍ |x_{h}|^{2} + |y_{h}|^{2} + |x_{h}| + |y_{h}| + |η_{h}|. Also, from Eq. (5.1), we have |η_{z}|^{2} − x_{z}y_{z} ≍ (|x_{z}| + |y_{z}| + |η_{z}|)^{2} for every z ∈ 𝔷. Also, x_{v}|y_{v}|^{2} + y_{v}|x_{v}|^{2} + 2 Im(x_{v}y_{v}^{†}\overline{η_{v}}) ≍ |v|^{3} whenever dim_{ℂ}⟨x_{v}, y_{v}⟩ = 1.

Case 1. Assume |x_{h}|^{2}|y_{h}|^{2} − |x_{h}y_{h}^{†}|^{2} = o(|x_{h}|^{4} + |y_{h}|^{4}). Then there is some v ∈ 𝔥 such that v − log h = o(|x_{h}| + |y_{h}|) and |x_{v}|^{2}|y_{v}|^{2} − |x_{v}y_{v}^{†}|^{2} = 0. We have dim_{ℂ}⟨x_{v}, y_{v}⟩ = 1. Therefore

Im Δ(h) ≍ x_{h}|y_{h}|^{2} + y_{h}|x_{h}|^{2} + 2 Im(x_{h}y_{h}^{†}\overline{η_{h}})
= x_{v}|y_{v}|^{2} + y_{v}|x_{v}|^{2} + 2 Im(x_{v}y_{v}^{†}\overline{η_{v}})
+ o(|η_{h}|^{3} + |x_{h}|^{3} + |y_{h}|^{3} + |x_{h}|^{3} + |y_{h}|^{3})
≍ |v|^{3} + o(|η_{h}|^{3} + |x_{h}|^{3} + |y_{h}|^{3} + |x_{h}|^{3} + |y_{h}|^{3})
≍ |η_{h}|^{3} + |x_{h}|^{3} + |y_{h}|^{3} + |x_{v}|^{3} + |y_{v}|^{3}
≠ o(∥h∥^{3/2}).

Thus, ∥h∥^{3/2} = O(ρ(h)).

Case 2. Assume |x_{h}|^{2}|y_{h}|^{2} − |x_{h}y_{h}^{†}|^{2} ≍ |x_{h}|^{4} + |y_{h}|^{4}. We may assume Re Δ(h) = o(|x_{h}|^{4} + |y_{h}|^{4}) for otherwise it is clear that Re Δ(h) ≍ ∥h∥^{2}. (So we have ∥h∥ ≍ |η_{h}| + |x_{h}| + |y_{h}| ≍ |x_{h}|^{2} + |y_{h}|^{2}.) Thus, there is some z ∈ 𝔷, such that z − log h = o(log h) and

|η_{z}|^{2} − x_{z}y_{z} = −\frac{1}{4}(|x_{h}|^{2}|y_{h}|^{2} − |x_{h}y_{h}^{†}|^{2}) + o(|x_{h}|^{4} + |y_{h}|^{4}) < 0.

(This implies that x_{z} and y_{z} must have the same sign.) From (5.1), we conclude that |η_{z}|^{2} − x_{z}y_{z} < 0 for every z ∈ 𝔷. Thus, there is a constant ϵ < 1, such that |η_{z}| ≤ ϵ\sqrt{x_{z}y_{z}} for every z ∈ 𝔷. Then

| Im(x_{h}y_{h}^{†}\overline{η_{z}})| ≤ |η_{z}||x_{h}||y_{h}| ≤ \frac{ϵ}{2}|x_{z}|y_{h}|^{2} + y_{z}|x_{h}|^{2}|,

so

Im(x_{h}y_{h}^{†}\overline{η_{z}}) + \frac{1}{2}x_{z}|y_{h}|^{2} + \frac{1}{2}y_{z}|x_{h}|^{2} ≍ \frac{1}{2}x_{z}|y_{h}|^{2} + \frac{1}{2}y_{z}|x_{h}|^{2}.

Therefore

Im Δ(h) = Im(x_{h}y_{h}^{†}\overline{η_{h}}) + \frac{1}{2}x_{h}|y_{h}|^{2} + \frac{1}{2}y_{h}|x_{h}|^{2}
= Im(x_{h}y_{h}^{†}\overline{η_{z}}) + \frac{1}{2}x_{z}|y_{h}|^{2} + \frac{1}{2}y_{z}|x_{h}|^{2} + o((|x_{h}|^{2} + |y_{h}|^{2}) log h)
≍ \frac{1}{2}x_{z}|y_{h}|^{2} + \frac{1}{2}y_{z}|x_{h}|^{2}
≍ |x_{h}|^{4} + |y_{h}|^{4}
≍ ∥h∥^{2}.

6 Non-Cartan-decomposition subgroups contained in N

Theorem 6.1. Assume that G = SU(2, n). Here is a complete list of the closed, connected, nontrivial subgroups H of N, such that H is not a Cartandecomposition subgroup.

1. If dim 𝔥 = 1, 𝔥 = 𝔷, and we have |η_{h}|^{2} = x_{h}y_{h} for every h ∈ H, then ρ(h) ≍ h for every h ∈ H.

2. If ϕ_{h} = 0 and y_{h} = 0 for every h ∈ 𝔥, there is some u ∈ 𝔥, such that y_{u} ≠ 0, and 𝔷 ⊂ 𝔲_{2α+2β}, then μ(H) ≈ [∥h∥, ∥h∥^{3/2}], unless dim H = 1, in which case ρ(h) ≍ ∥h∥^{3/2} for every h ∈ H.

3. Suppose ϕ_{h} = 0 for every h ∈ 𝔥, and there is some λ ∈ ℂ, such that x_{h} = λy_{h} for every h ∈ H, and we have η_{z} = iλy_{z} and x_{z} = |λ|^{2}y_{z} for every z ∈ 𝔷.

(a) If there is some u ∈ 𝔥, such that x_{u} + |λ|^{2}y_{u} + 2 Im(λ\overline{η_{u}}) ≠ 0, then μ(H) ≈ [∥h∥, ∥h∥^{3/2}], unless dim H = 1, in which case ρ(h) ≍ ∥h∥^{3/2} for every h ∈ H.

(b) Otherwise, ρ(h) ≍ h for every h ∈ H.

4. If y_{h} = 0, y_{h} = 0, and |x_{h}|^{2} + 2 Re(ϕ_{h}\overline{η_{h}}) ≠ 0 for every h ∈ 𝔥 \ 𝔲_{2α+2β} (so 𝔷 ⊂ 𝔲_{2α+2β}), then ρ(h) ≍ h for every h ∈ H.

5. If 𝔷 = 0, there is some u ∈ 𝔥 and some nonzero ϕ_{0} ∈ ℂ, such that ϕ_{u} ≠ 0, and we have ϕ_{h} = ϕ_{0}y_{h} and y_{h} = 0, for every h ∈ 𝔥, then μ(H) ≈ [∥h∥, ∥h∥^{4/3}], unless dim H = 1, in which case, ρ(h) ≍ ∥h∥^{4/3} for every h ∈ H.

6. If ϕ_{h} = 0 and dim_{ℂ}⟨x_{u}, y_{u}⟩ ≠ 1 for every h ∈ 𝔥, and |η_{z}|^{2} ≠ x_{z}y_{z}, for every nonzero z ∈ 𝔷, then ρ(h) ≍ ∥h∥^{2} for every h ∈ H.

7. If ϕ_{h} = 0 for every h ∈ 𝔥, there exist nonzero u and v in 𝔥, such that dim_{ℂ}⟨x_{u}, y_{u}⟩ ≠ 1 and dim_{ℂ}⟨x_{v}, y_{v}⟩ = 1, and we have x_{v}|y_{v}|^{2} + y_{v}|x_{v}|^{2} + 2 Im(x_{v}y_{v}^{†}\overline{η_{v}}) ≠ 0 for every such v ∈ 𝔥, and |η_{z}|^{2} ≠ x_{z}y_{z}, for every nonzero z ∈ 𝔷 then μ(H) ≈ [∥h∥^{3/2}, ∥h∥^{2}].

8. If dim 𝔥 ≤ 3, 𝔷 = 0, we have ϕ_{v} ≍ y_{v} and v = O(|ϕ_{v}| + |y_{v}|) for every v ∈ 𝔥, and there exists u ∈ 𝔥, such that ϕ_{u} ≠ 0, then ρ(h) ≍ ∥h∥^{3/2} for every h ∈ H.

9. If dim 𝔥 = 2, 𝔷 = 𝔲_{2α+2β}, ϕ_{h} ≠ 0 and y_{h} ≠ 0 for every h ∈ 𝔥 \ 𝔷, then μ(H) ≈ [∥h∥, ∥h∥^{3/2}].

10. If dim 𝔥 ≤ 2 and ϕ_{h} ≠ 0, y_{h} = 0, y_{h} = 0, and |x_{h}|^{2} + 2 Re(ϕ_{h}\overline{η_{h}}) = 0 for every nonzero h ∈ 𝔥, then ρ(h) ≍ ∥h∥^{2} for every h ∈ H.

11. If dim 𝔥 = 2 and there exist nonzero u ∈ 𝔥 and z ∈ 𝔷, such that ϕ_{u} ≠ 0, y_{u} ≠ 0, y_{z} = 0, ϕ_{u}\overline{η_{z}} ≠ 0 is real, and x_{z}|y_{u}|^{2} − ϕ_{u}y_{u}\overline{η_{z}} + 2 Im(\overline{η_{z}}x_{u}y_{u}^{†}) ≠ 0, then μ(H) ≈ [∥h∥^{5/4}, ∥h∥^{2}].

Proof. The theorem is obtained by merging the statement of Proposition 4.3 with the statement of Proposition 5.2, and eliminating some redundancy (see 3.4). Specifically:

• 4.3(1) appears here as 6.1(1).

• 4.3(2) appears here as 6.1(2).

• 4.3(3) appears here as 6.1(3).

• 4.3(4) appears here as 6.1(4).

• 4.3(5) appears here as 6.1(5).

• 4.3(6) appears here as 6.1(8).

• 4.3(7) appears here as 6.1(9).

• 5.2(1) is a special case of 6.1(6).

• 5.2(2) appears here as 6.1(6).

• 5.2(3) appears here as 6.1(7).

• 5.2(4) appears here as 6.1(8).

• 5.2(5) appears here as 6.1(10).

• 5.2(6) appears here as 6.1(11).

• 5.2(7) is a special case of 6.1(3a) (with dim H = 1).

• 5.2(8) is a special case of 6.1(5) (with dim H = 1).

Corollary 6.2. Assume that G = SU(2, n). Here is a complete list of the closed, connected, nontrivial subgroups H of N, such that H is not a Cartandecomposition subgroup, and N_{A}(H) is nontrivial.

1. Suppose dim 𝔥 = 1, 𝔥 = 𝔷, and we have |η_{h}|^{2} = x_{h}y_{h} for every h ∈ H. (a) If 𝔥 = 𝔲_{2β} or 𝔥 = 𝔲_{2α+2β}, then N_{A}(H) = A.

(b) Otherwise, N_{A}(H) = ker(α).

2. Suppose ϕ_{h} = 0 and y_{h} = 0 for every h ∈ 𝔥, there is some u ∈ 𝔥, such that y_{u} ≠ 0, and 𝔷 ⊂ 𝔲_{2α+2β}. If 𝔥 = (𝔥 ∩ (𝔲_{α+β} + 𝔲_{2β})) + 𝔷, then N_{A}(H) = ker(α − β).

3. Suppose ϕ_{h} = 0 for every h ∈ 𝔥, and there is some nonzero λ ∈ ℂ, such that x_{h} = λy_{h} for every h ∈ H, and we have η_{z} = iλy_{z} and x_{z} = |λ|^{2}y_{z} for every z ∈ 𝔷. If 𝔥 = (𝔥 ∩ (𝔲_{β} + 𝔲_{α+β})) + 𝔷 ≠ 𝔷, then N_{A}(H) = ker(α).

4. Suppose ϕ_{h} = 0 and x_{h} = 0 for every h ∈ 𝔥, we have 𝔷 ⊂ 𝔲_{2β}, and 𝔥 ≠ 𝔷. (a) If 𝔥 = (𝔥 ∩ 𝔲_{β}) + 𝔷, then N_{A}(H) = A.

(b) Otherwise:

i. If 𝔥 = (𝔥 ∩ (𝔲_{β} + 𝔲_{α+2β})) + 𝔷, then N_{A}(H) = ker(α + β). ii. If 𝔥 = (𝔥 ∩ (𝔲_{β} + 𝔲_{2α+2β})) + 𝔷, then N_{A}(H) = ker(2α + β). iii. If 𝔷 = 0 and 𝔥 ⊂ 𝔲_{β} + 𝔲_{2β}, then N_{A}(H) = ker(β).

5. Suppose y_{h} = 0, y_{h} = 0, and |x_{h}|^{2} +2 Re(ϕ_{h}\overline{η_{h}}) ≠ 0 for every h ∈ 𝔥\𝔲_{2α+2β}. (a) If 𝔥 = (𝔥 ∩ 𝔲_{α+β}) + 𝔷, then N_{A}(H) = A.

(b) If 𝔥 ⊂ 𝔲_{α+β} +𝔲_{2α+2β}, but 𝔥 ≠ (𝔥∩𝔲_{α+β})+𝔷, then N_{A}(H) = ker(α+β).

(c) If 𝔥 = (𝔥 ∩ (𝔲_{α} + 𝔲_{α+β} + 𝔲_{α+2β})) + 𝔷, but 𝔥 ⊄ 𝔲_{α+β} + 𝔲_{2α+2β}, then

N_{A}(H) = ker(β).

6. Suppose 𝔷 = 0, there is some nonzero ϕ_{0} ∈ ℂ, such that ϕ_{h} = ϕ_{0}y_{h} and y_{h} = 0, for every h ∈ 𝔥, and there is some u ∈ 𝔥, such that ϕ_{u} ≠ 0. If 𝔥 = (𝔥 ∩ (𝔲_{α} + 𝔲_{2β})) + (𝔥 ∩ 𝔲_{α+β}), then N_{A}(H) = ker(α − 2β).

7. Suppose ϕ_{h} = 0 and dim_{ℂ}⟨x_{u}, y_{u}⟩ ≠ 1 for every h ∈ 𝔥, and |η_{z}|^{2} ≠ x_{z}y_{z}, for every nonzero z ∈ 𝔷.

(a) If 𝔥 ⊂ 𝔲_{α+2β}, then N_{A}(H) = A.

(b) If 𝔥 ⊄ 𝔲_{α+2β}, and 𝔥 = (𝔥 ∩ (𝔲_{β} + 𝔲_{α+β})) + 𝔷, then N_{A}(H) = ker(α).

8. Suppose ϕ_{h} = 0 for every h ∈ 𝔥, there exist nonzero u, v ∈ 𝔥, such that dim_{ℂ}⟨x_{u}, y_{u}⟩ ≠ 1 and dim_{ℂ}⟨x_{v}, y_{v}⟩ = 1, x_{v}|y_{v}|^{2} + y_{v}|x_{v}|^{2} + 2 Im(x_{v}y_{v}^{†}\overline{η_{v}}) ≠ 0 for every such v ∈ 𝔥, and |η_{z}|^{2} ≠ x_{z}y_{z}, for every nonzero z ∈ 𝔷.

(a) If 𝔥 = (𝔥 ∩ (𝔲_{α+β} + 𝔲_{2β})) + (𝔥 ∩ 𝔲_{α+2β}), then N_{A}(H) = ker(α − β) (and dim H ≤ 3).

(b) If 𝔥 = (𝔥 ∩ (𝔲_{β} + 𝔲_{2α+2β})) + (𝔥 ∩ 𝔲_{α+2β}), then N_{A}(H) = ker(2α + β) (and dim H ≤ 3).

9. Suppose dim 𝔥 ≤ 3, 𝔥 = (𝔥∩(𝔲_{α}+𝔲_{β}))+(𝔥∩(𝔲_{α+β} +𝔲_{2β})), 𝔥∩(𝔲_{α}+𝔲_{β}) ≠ 0, and we have ϕ_{h} ≍ y_{h} and x_{h} ≍ y_{h} for h ∈ 𝔥, then N_{A}(H) = ker(α − β).

10. Suppose dim 𝔥 = 2, 𝔷 = 𝔲_{2α+2β}, ϕ_{h} ≠ 0 and y_{h} ≠ 0 for every h ∈ 𝔥 \ 𝔷. If 𝔥 = (𝔥 ∩ (𝔲_{α} + 𝔲_{β})) + 𝔷, then N_{A}(H) = ker(α − β).

11. Suppose dim 𝔥 ≤ 2 and ϕ_{h} ≠ 0, y_{h} = 0, y_{h} = 0, and |x_{h}|^{2} + 2 Re(ϕ_{h}\overline{η_{h}}) = 0 for every nonzero h ∈ 𝔥.

(a) If 𝔥 ⊂ 𝔲_{α}, then N_{A}(H) = A.

(b) If 𝔥 ⊂ 𝔲_{α} + 𝔲_{α+β} + 𝔲_{α+2β}, but 𝔥 ⊄ 𝔲_{α}, then N_{A}(H) = ker(β).

(c) If 𝔥 ⊂ 𝔲_{α} + 𝔲_{2α+2β}, but 𝔥 ⊄ 𝔲_{α}, then N_{A}(H) = ker(α + 2β).

Proof. It is clear that each of the given subgroups is normalized by the indicated torus. We now show that the list is complete, and that no larger subtorus of A normalizes H.

Assume N_{A}(H) is nontrivial. We proceed in cases, determined by Theorem 6.1.

Case 1. Assume 6.1(1). We may assume 𝔥 is neither 𝔲_{2β} nor 𝔲_{2α+2β} (otherwise (1a) applies). Then, because |η_{u}|^{2} = x_{u}y_{u} for every u ∈ 𝔥, we see that η_{u} ≠ 0 for every nonzero u ∈ 𝔥. Thus, the projection of 𝔥 to 𝔲_{α+2β} is nontrivial. However, because |η_{u}|^{2} = x_{u}y_{u}, we have 𝔥∩𝔲_{α+2β} = 0. We know that 𝔥 ⊂ 𝔲_{α+2β}+𝔲_{2β}+𝔲_{2α+2β} (because 𝔥 = 𝔷), so, because each of 2β and 2α + 2β differs from α + 2β by α, we conclude that N_{A}(H) = ker(α), so (1b) applies.

Case 2. Assume 6.1(2). Let V be the projection of 𝔥 to 𝔲_{α+β} + 𝔲_{2β}. Because y_{u} ≠ 0, we know that V projects nontrivially to 𝔲_{2β}. However, because 𝔷 ⊂ 𝔲_{2α+2β}, we also know that V ∩ 𝔲_{2β} = 0. Therefore N_{A}(H) = ker(α − β). Then, because neither α + 2β nor 2α + 2β differs from α + β by a multiple of α − β, we conclude that 𝔥 = (𝔥 ∩ (𝔲_{α+β} + 𝔲_{2β})) + 𝔷, so (2) applies.

Case 3. Assume 6.1(3). We may assume 𝔥 ≠ 𝔷 (otherwise Case 1 applies).

Subcase 3.1. Assume λ ≠ 0. Because 𝔥 ≠ 𝔷, the projection of 𝔥 to 𝔲_{β} + 𝔲_{α+β} is nontrivial. However, because λ ≠ 0, this projection intersects neither 𝔲_{β} nor 𝔲_{α+β}. Therefore N_{A}(H) ⊂ ker(α). Then, because neither 2β, α+2β, nor 2α+2β differs from β by a multiple of α, we conclude that 𝔥 = (𝔥 ∩ (𝔲_{β} + 𝔲_{α+β})) + 𝔷, so (3) applies.

Subcase 3.2. Assume λ = 0. This means x_{u} = 0 for every u ∈ 𝔥, and 𝔷 ⊂ 𝔲_{2β}.

Because 𝔥 ≠ 𝔷, we know that 𝔥 projects nontrivially to 𝔲_{β}. Because 𝔷 ⊂ 𝔲_{2β}, we know that 𝔥 ∩ 𝔲_{α+2β} = 𝔥 ∩ 𝔲_{2α+2β} = 0. Thus, it is easy to see that if 𝔥 projects nontrivially to 𝔲_{α+2β} or 𝔲_{2α+2β} then either (4(b)i) or (4(b)ii) applies.

Thus, we may assume 𝔥 ⊂ 𝔲_{β} + 𝔲_{2β}. If 𝔷 ≠ 0, then 𝔥 = (𝔥 ∩ 𝔲_{β}) + 𝔲_{2β}, so (4a) applies. Otherwise, (4(b)iii) applies.

Case 4. Assume 6.1(4).

Subcase 4.1. Assume the projection of 𝔥 to 𝔲_{α} is trivial. Because

|x_{u}|^{2} = |x_{u}|^{2} + 2 Re(ϕ_{u}\overline{η_{u}}) ≠ 0

for every u ∈ 𝔥 \ 𝔲_{2α+2β}, we know that x_{u} ≠ 0 for every u ∈ 𝔥 \ 𝔲_{2α+2β}. Thus, if the projection of 𝔥 to 𝔲_{α+2β} is nontrivial, then N_{A}(H) = ker(β), and we see that (5c) applies. If not, then 𝔥 ⊂ 𝔲_{α+β} + 𝔲_{2α+2β}, so either (5a) or (5b) applies.

Subcase 4.2. Assume the projection of 𝔥 to 𝔲_{α} is nontrivial. Let V be the projection of 𝔥 to 𝔲_{α} + 𝔲_{α+β} + 𝔲_{α+2β}. Because |x_{u}|^{2} + 2 Re(ϕ_{u}\overline{η_{u}}) ≠ 0 for every u ∈ 𝔥 \ 𝔲_{2α+2β}, we know that V ∩ 𝔲_{α} = 0. Then, because α, α + β, and α + 2β all differ by multiples of β, we conclude that N_{A}(H) = ker(β). Therefore (5c) applies.

Case 5. Assume 6.1(5). Let V be the projection of 𝔥 to 𝔲_{α} + 𝔲_{2β}. Because ϕ_{h} = ϕ_{0}y_{h}, we see that V ∩ 𝔲_{α} = 0 and V ∩ 𝔲_{2β} = 0. Therefore N_{A}(H) = ker(α − 2β).

Because no other roots differ by a multiple of α − 2β (and 𝔷 = 0), we conclude that 𝔥 = (𝔥 ∩ (𝔲_{α} + 𝔲_{2β})) + (𝔥 ∩ 𝔲_{α+β}). Thus, (6) applies.

Case 6. Assume 6.1(6).

Subcase 6.1. Assume 𝔥 ≠ 𝔷. Let V be the projection of 𝔥 to 𝔲_{β} + 𝔲_{α+β}. From the assumption of this subcase, we know V ≠ 0. However, because dim_{ℂ}⟨x_{u}, y_{u}⟩ ≠ 1 for every u ∈ 𝔥, we know that V ∩ 𝔲_{β} = 0 and 𝔥 ∩ 𝔲_{α+β} = 0. Therefore N_{A}(H) = ker(α), so (7b) applies.

Subcase 6.2. Assume 𝔥 = 𝔷. We may assume 𝔥 ⊄ 𝔲_{α+2β} (otherwise (7a) applies). Therefore, 𝔥 projects nontrivially to 𝔲_{2β} + 𝔲_{2α+2β}. However, because |η_{z}|^{2} ≠ x_{z}y_{z}, for every nonzero z ∈ 𝔷, we know that V ∩𝔲_{2β} = 0 and V ∩𝔲_{2α+2β} = 0. Because 2β, α+2β, and 2α+2β all differ by multiples of α, we conclude that N_{A}(H) = ker(α), so (7b) applies.

Case 7. Assume 6.1(7).

Subcase 7.1. Assume N_{A}(H) = ker(α). Because α + β is the only root that differs from β by a multiple of α, we must have 𝔥 = (𝔥 ∩ (𝔲_{β} + 𝔲_{α+β})) + 𝔷. Thus, there is some w ∈ 𝔥, such that x_{w} = x_{v} and y_{w} = y_{v}, but the projection of w to 𝔲_{2β} + 𝔲_{α+2β} + 𝔲_{2α+2β} is zero. This contradicts the fact that x_{w}|y_{w}|^{2} + y_{w}|x_{w}|^{2} + 2 Im(x_{w}y_{w}^{†}\overline{η_{w}}) ≠ 0.

Subcase 7.2. Assume N_{A}(H) ≠ ker(α). Because 2β, α+2β, and 2α+2β all differ by multiples of α, we must have 𝔷 = (𝔷 ∩ 𝔲_{2β}) + (𝔷 ∩ 𝔲_{α+2β}) + (𝔷 ∩ 𝔲_{2α+2β}). Then, because |η_{z}|^{2} ≠ x_{z}y_{z} for every nonzero z ∈ 𝔷, we conclude that 𝔷 ⊂ 𝔲_{α+2β}.

Let V be the projection of 𝔥 to 𝔲_{β} + 𝔲_{α+β}. Because β and α + β differ by α, we know that V = (V ∩ 𝔲_{β}) + (V ∩ 𝔲_{α+β}).

Subsubcase 7.2.1. Assume x_{v} ≠ 0. Because V = (V ∩ 𝔲_{β}) + (V ∩ 𝔲_{α+β}), there is some w ∈ V , such that x_{w} ≠ 0 and y_{w} = 0. For every such w, because x_{w}|y_{w}|^{2} + y_{w}|x_{w}|^{2} + 2 Im(x_{w}y_{w}^{†}\overline{η_{w}}) ≠ 0, we know that y_{w} ≠ 0. Thus, we see that N_{A}(H) = ker((α + β) − 2β) = ker(α − β).

We know that 𝔥 ∩ 𝔲_{β} = 0, that 𝔥 projects trivially to 𝔲_{α}, and that α is the only root that differs from β by a multiple of α − β, so we conclude that y_{h} = 0 for every h ∈ H.

We now see that (8a) applies.

Subsubcase 7.2.2. Assume y_{v} ≠ 0. This is similar to the preceding subsubcase (indeed, they are conjugate under the Weyl reflection corresponding to the root α); we see that (8b) applies.

Case 8. Assume 6.1(8). By considering the projection of 𝔥 to 𝔲_{α} + 𝔲_{β}, and noting that ϕ_{h} ≍ y_{h} for every h ∈ H, we see that N_{A}(H) = ker(α − β). The only other pair of roots that differ by a multiple of α − β is {α + β, 2β}. Thus, we see that (9) applies.

Case 9. Assume 6.1(9). By considering the projection of 𝔥 to 𝔲_{α} + 𝔲_{β}, we see that N_{A}(H) = ker(α − β). Because ϕ_{u} ≠ 0 for every u ∈ 𝔥 \ 𝔲_{2α+2β}, but β is the only root that differs from α by a multiple of α − β, we conclude that 𝔥 projects trivially into every root space except 𝔲_{α}, 𝔲_{β}, and 𝔲_{2α+2β}. Thus (10) applies.

Case 10. Assume 6.1(10). We may assume 𝔥 ⊄ 𝔲_{α} (otherwise (11a) applies). Thus, there is some root σ ≠ α, such that the projection of 𝔥 to 𝔲_{σ} is nontrivial. However, because ϕ_{h} ≠ 0 for every nonzero h ∈ 𝔥, we know that 𝔥 ∩ 𝔲_{σ} = 0. Thus, N_{A}(H) = ker(α − σ).

Because y_{h} = 0 and y_{h} = 0 for every nonzero h ∈ 𝔥, we know that σ ≠ β and σ ≠ 2β. If σ = α + β or σ = α + 2β, we obtain (11b). If σ = 2α + 2β, we obtain (11c).

Case 11. Assume 6.1(11). Because ϕ_{u} ≠ 0 and y_{u} ≠ 0, we must have N_{A}(H) = ker(α − β). Then, because α + β does not differ from α by a multiple of α − β, we conclude that x_{u} = 0.

Because η_{z} ≠ 0, but no root differs from α + 2β by a multiple of α − β, we conclude that 𝔥∩𝔲_{α+2β} ≠ 0. Because 𝔷 is one-dimensional, this implies z ∈ 𝔲_{α+2β}, so x_{z} = 0.

Since x_{z} = 0 and x_{u} = 0, we conclude, from the inequality x_{z}|y_{u}|^{2} − ϕ_{u}y_{u}\overline{η_{z}} + 2 Im(\overline{η_{z}}x_{u}y_{u}^{†}) ≠ 0, that y_{u} ≠ 0. This is a contradiction, because 2β does not differ from α by a multiple of α − β, and 𝔥 ∩ 𝔲_{2β} = 0 (because, as shown above, 𝔷 ⊂ 𝔲_{α+2β}).

7 Subgroups that are not contained in N

Let H be a closed, connected subgroup of G that is not contained in N. In this section, we determine whether H is a Cartan-decomposition subgroup or not (and, if not, we calculate μ(H)).

Lemma 7.1 shows that we may assume H ⊂ AN, and then Lemma 7.3 shows that we may assume H satisfies the technical condition of being compatible with A. (Both of these lemmas are well known.) Furthermore, we may assume that H ∩N is not a Cartan-decomposition subgroup, and that A ⊄ H (otherwise, it is obvious that H is a Cartan-decomposition subgroup).

Theorem 7.4 describes μ(H) for every such subgroup that is a semidirect product (H ∩ A) ⋉ (H ∩ N); and Proposition 7.6 describes μ(H) for the other subgroups (except that the one-dimensional case appears in Lemma 7.8).

Lemma 7.1. [14, Lem. 2.9] Let H be a closed, connected subgroup of a connected, almost simple, linear, real Lie group G. There is a closed, connected subgroup H′ of G and a compact subgroup C of G, such that CH = CH′, and H′ is conjugate to a subgroup of AN.

Definition 7.2. Let us say that a subgroup H of AN is compatible with A if H ⊂ T UC_{N}(T), where T = A ∩ (HN), U = H ∩ N, and C_{N}(T) denotes the centralizer of T in N.

Lemma 7.3. [14, Lem. 2.3] If H is a closed, connected subgroup of AN, then H is conjugate, via an element of N, to a subgroup that is compatible with A.

Theorem 7.4. Assume that G = SU(2, n). Here is a list of every closed, connected, nontrivial subgroup H of AN, such that H is of the form H = T ⋉ U, where T is a one-dimensional subgroup of A, and U is a nontrivial subgroup of N that is not a Cartan-decomposition subgroup.

1. Suppose dim 𝔲 = 1, 𝔲 = 𝔷, and we have |η_{h}|^{2} = x_{h}y_{h} for every h ∈ U.

(a) If 𝔲 = 𝔲_{2β} or 𝔲 = 𝔲_{2α+2β}, then μ(H) is described in [14, Prop. 3.17 or Cor. 3.18].

(b) Otherwise, T = ker(α), and H is a Cartan-decomposition subgroup.

2. Suppose 𝔲 = (𝔲 ∩ (𝔲_{α+β} + 𝔲_{2β})) + 𝔷, 𝔷 ⊂ 𝔲_{2α+2β}, there is some v ∈ 𝔲, such that y_{v} ≠ 0, and T = ker(α − β). Then μ(H) ≈ [∥h∥, ∥h∥^{3/2}], unless dim H = 2, in which case ρ(h) ≍ ∥h∥^{3/2} for every h ∈ H.

3. Suppose 𝔲 = (𝔲 ∩ (𝔲_{β} + 𝔲_{α+β})) + 𝔷, T = ker(α), and there is some nonzero λ ∈ ℂ, such that we have x_{u} = λy_{u} for every u ∈ U, and we have η_{z} = iλy_{z} and x_{z} = |λ|^{2}y_{z} for every z ∈ 𝔷. Then H is a Cartan-decomposition subgroup.

4. Suppose ϕ_{u} = 0 and x_{u} = 0 for every u ∈ 𝔲, we have 𝔷 ⊂ 𝔲_{2β}, and 𝔲 ≠ 𝔷.

(a) If 𝔲 = (𝔲 ∩ 𝔲_{β}) + 𝔷, then μ(H) is described in [14, Prop. 3.17 or Cor. 3.18].

(b) Otherwise:

i. If 𝔲 = (𝔲 ∩ (𝔲_{β} + 𝔲_{α+2β})) + 𝔷, then T = ker(α + β), and ρ(h) ≍ h for every h ∈ H. ii. If 𝔲 = (𝔲 ∩ (𝔲_{β} + 𝔲_{2α+2β})) + 𝔷, then T = ker(2α + β), and μ(H) ≈ [∥h∥, ∥h∥^{3/2}], unless dim H = 2, in which case ρ(h) ≍ ∥h∥^{3/2} for every h ∈ H.

iii. If 𝔷 = 0 and 𝔲 ⊂ 𝔲_{β} + 𝔲_{2β}, then T = ker(β), and H is a Cartandecomposition subgroup.

5. Suppose y_{u} = 0, y_{u} = 0, and |x_{u}|^{2}+2 Re(ϕ_{u}\overline{η_{u}}) ≠ 0 for every u ∈ U \U_{2α+2β}.

(a) If 𝔲 = (𝔲 ∩ 𝔲_{α+β}) + 𝔷, then μ(H) is described in [14, Prop. 3.17 or Cor. 3.18].

(b) If 𝔲 ⊂ 𝔲_{α+β} + 𝔲_{2α+2β}, but 𝔲 ≠ (𝔲 ∩ 𝔲_{α+β}) + 𝔷, then T = ker(α + β), and H is a Cartan-decomposition subgroup.

(c) If 𝔲 = (𝔲 ∩ (𝔲_{α} + 𝔲_{α+β} + 𝔲_{α+2β})) + 𝔷, but 𝔲 ⊄ 𝔲_{α+β} + 𝔲_{2α+2β}, then T = ker(β), and ρ(h) ≍ h for every h ∈ H.

6. Suppose 𝔲 = (𝔲 ∩ (𝔲_{α} + 𝔲_{2β})) + (𝔲 ∩ 𝔲_{α+β}), T = ker(α − 2β), 𝔲 ⊄ 𝔲_{α+β}, and there is some nonzero ϕ_{0} ∈ ℂ, such that ϕ_{u} = ϕ_{0}y_{u} for every u ∈ U. Then μ(H) ≈ [∥h∥, ∥h∥^{4/3}], unless dim H = 2, in which case, ρ(h) ≍ ∥h∥^{4/3} for every h ∈ H.

7. Suppose ϕ_{u} = 0 and dim_{ℂ}⟨x_{u}, y_{u}⟩ ≠ 1 for every u ∈ U, and |η_{z}|^{2} ≠ x_{z}y_{z}, for every nonzero z ∈ 𝔷.

(a) If 𝔲 ⊂ 𝔲_{α+2β}, then μ(H) is described in [14, Prop. 3.17 or Cor. 3.18].

(b) If 𝔲 ⊄ 𝔲_{α+2β}, and 𝔲 = (𝔲 ∩ (𝔲_{β} + 𝔲_{α+β})) + 𝔷, then T = ker(α), and ρ(h) ≍ ∥h∥^{2} for every h ∈ H.

8. Suppose ϕ_{u} = 0 for every u ∈ U, there exist nonzero v_{1}, v_{2} ∈ 𝔲, such that dim_{ℂ}⟨x_{v_{1}}, y_{v_{1}}⟩ ≠ 1 and dim_{ℂ}⟨x_{v_{2}}, y_{v_{2}}⟩ = 1, and we have x_{v_{2}}|y_{v_{2}}|^{2} + y_{v_{2}}|x_{v_{2}}|^{2} + 2 Im(x_{v_{2}}y_{v_{2}}^{†}\overline{η_{v_{2}}}) ≠ 0 for every such v_{2} ∈ 𝔲, and |η_{z}|^{2} ≠ x_{z}y_{z}, for every nonzero z ∈ 𝔷.

(a) If 𝔲 = (𝔲 ∩ (𝔲_{α+β} + 𝔲_{2β})) + (𝔲 ∩ 𝔲_{α+2β}), then T = ker(α − β) and μ(H) ≈ [∥h∥^{3/2}, ∥h∥^{2}].

(b) If 𝔲 = (𝔲 ∩ (𝔲_{β} + 𝔲_{2α+2β})) + (𝔲 ∩ 𝔲_{α+2β}), then T = ker(2α + β) and μ(H) ≈ [∥h∥^{3/2}, ∥h∥^{2}].

9. Suppose dim 𝔲 ≤ 3, 𝔲 = (𝔲∩(𝔲_{α}+𝔲_{β}))+(𝔲∩(𝔲_{α+β} +𝔲_{2β})), 𝔲∩(𝔲_{α}+𝔲_{β}) ≠ 0, and we have ϕ_{u} ≍ y_{u} and x_{u} ≍ y_{u} for u ∈ U. Then T = ker(α − β), and ρ(h) ≍ ∥h∥^{3/2} for every h ∈ H.

10. Suppose dim 𝔲 = 2, 𝔷 = 𝔲_{2α+2β}, ϕ_{u} ≠ 0 and y_{u} ≠ 0 for every u ∈ U \ Z. If 𝔲 = (𝔲 ∩ (𝔲_{α} + 𝔲_{β})) + 𝔷, then T = ker(α − β), and μ(H) ≈ [∥h∥, ∥h∥^{3/2}].

11. Suppose dim 𝔲 ≤ 2 and ϕ_{u} ≠ 0, y_{u} = 0, y_{u} = 0, and |x_{u}|^{2} + 2 Re(ϕ_{u}\overline{η_{u}}) = 0 for every nontrivial u ∈ U.

(a) If 𝔲 ⊂ 𝔲_{α}, then μ(H) is described in [14, Prop. 3.17 or Cor. 3.18].

(b) If 𝔲 ⊂ 𝔲_{α} + 𝔲_{α+β} + 𝔲_{α+2β}, but 𝔲 ⊄ 𝔲_{α}, then T = ker(β), and H is a Cartan-decomposition subgroup.

(c) If 𝔲 ⊂ 𝔲_{α} +𝔲_{2α+2β}, but 𝔲 ⊄ 𝔲_{α}, then T = ker(α+2β), and ρ(h) ≍ ∥h∥^{2} for every h ∈ H.

Proof. For h ∈ H, we wish to approximately calculate ∥ρ(h)∥. We write h = au with a ∈ T and u ∈ U. Writing a = diag(a_{1}, a_{2}, …, a_{n+2}), we always assume either that a_{1} > 1 or that a_{1} = 1 and a_{2} ≥ 1 (perhaps replacing h with h^{−1}—because ∥ρ(h)∥ = ∥ρ(h^{−1})∥, this causes no harm).

Because T normalizes U, we know that U is a subgroup that is listed in Corollary 6.2, and we have T ⊂ N_{G}(U). This leads to the various cases listed in the statement of the theorem.

(1b) We have ρ(u) ≍ u for u ∈ U and ρ(a) ≍ ∥a∥^{2} for a ∈ T, so H is a Cartan-decomposition subgroup.

(2) We have |ϕ_{u}|+|y_{u}|+|η_{u}|+|x_{u}| = 0 and y_{u} = O(x_{u}), so u_{i,j} = O(1+|x_{u}|) whenever (i, j) ≠ (1, n + 2). Then, because a_{1} = a_{2}^{2}, we see that

u_{i,j} = O[a_{2}(1 + |x_{u}|)] = O(|h_{1,1}|^{1/2} + |h_{1,n+2}|^{1/2}) = O(∥h∥^{1/2})

whenever i > 1. Therefore ρ(h) = O(∥h∥^{3/2}). This completes the proof if dim H > 2 (that is, if dim U > 1).

If dim U = 1, then y_{u} ≍ x_{u} and x_{u} = 0. We have ∥h∥ = a_{1}(1 + |x_{u}|^{2}),

Δ(h) = a_{1}a_{2} [i (\frac{1}{2}|x_{u}|^{2}y_{u})] ≍ (a_{1}|x_{u}|^{2})^{3/2}

and

det (\begin{matrix}h_{1,1} & h_{1,2} \\ h_{2,1} & h_{2,2} \end{matrix}) = a_{1}a_{2} = a_{1}^{3/2}.

Thus, ∥h∥^{3/2} = O(ρ(h)). We conclude that ρ(h) ≍ ∥h∥^{3/2}.

(3) Replacing H by a conjugate under U_{α}, we may replace H with a similar subgroup H′ with λ = 0. Thus, H′ = T ⋉ U′ with U′ ⊂ U_{β}U_{2β}. Then [14, Prop. 3.17] implies H is a Cartan-decomposition subgroup.

(4(b)i) The Weyl reflection corresponding to the root α conjugates H to a subgroup of type (5c).

(4(b)ii) The Weyl reflection corresponding to the root α conjugates H to a subgroup of type (2).

(4(b)iii) [14, Prop. 3.17] implies H is a Cartan-decomposition subgroup. (5b) [14, Prop. 3.17] implies H is a Cartan-decomposition subgroup. (5c) We have

h_{i,j} = \begin{cases}O(1) & if i ≠ 1 and j ≠ n + 2 \\ O(a_{1}x) & if i = 1 and j ≠ n + 2 \\ O(x) & if i ≠ 1 and j = n + 2 \end{cases}

and h_{1,n+2} ≍ a_{1}(|x|^{2} + |x|). We conclude that ρ(h) ≍ h.

(6) From the proof of 4.3(5), we know that u ≍ u_{1,n+2}, that u_{i,j} = O(∥u∥^{3/2}) whenever (i, j) ≠ (1, n + 2), and that u_{i,j} = O(∥u∥^{1/3}) whenever i ≠ 1 and j ≠ n + 2. (In particular, h ≍ a_{1}(1 + u_{1,n+2}).) Furthermore, we have a_{1} = a_{2}^{3}. Therefore

ρ(h) ≍ a_{1}a_{2}ρ(u) ≍ a_{1}^{4/3}ρ(u).

The desired conclusion follows.

(7b) From Lemma 5.4, we know ∥u∥^{2} = O(1 + |Δ(u)|). Then, because

det (\begin{matrix}h_{1,1} & h_{1,2} \\ h_{2,1} & h_{2,2} \end{matrix}) = a_{1}a_{2} = a_{1}^{2}

and Δ(au) = a_{1}^{2}Δ(u), we have

∥h∥^{2} = O(a_{1}^{2}∥u∥^{2}) = O(a_{1}^{2} + |Δ(h)|) = O(ρ(h)).

(8) Assume (8a). (The other case, (8b), is conjugate to this one by the Weyl reflection corresponding to the root α.) From Lemma 5.4, we have ∥u∥^{3/2} = O(1 + |Δ(u)|). Then, because a_{1} = a_{2}^{2}, we have

∥h∥^{3/2} = a_{1}a_{2}∥u∥^{3/2} = O(a_{1}a_{2} + |Δ(h)|) = O(ρ(h)).

(9) From the proof of 4.3(6), we know ρ(u) ≍ 1 + Δ(u) ≍ ∥u∥^{3/2}. The proof is completed as in (8).

(10) Because ϕ_{u} ≍ y_{u}, it is easy to see that

h ≍ a_{1}(1 + |ϕ_{u}|^{2}|y_{u}|^{2} + |x_{u}|) ≍ a_{1}(1 + |ϕ_{u}|^{4} + |x_{u}|)

and

Δ(h) ≍ a_{1}a_{2}(|y_{u}|^{4}|ϕ_{u}|^{2} + |x_{u}||y_{u}|^{2}) ≍ a_{1}^{3/2}(|ϕ_{u}|^{6} + |x_{u}||ϕ_{u}|^{2}) = O(∥h∥^{3/2}).

Then it is not difficult to see that ρ(h) = O(∥h∥^{3/2}) for every h ∈ H. So μ(H) ≈ [∥h∥, ∥h∥^{3/2}].

(11b) We have ρ(a) ≍ a for a ∈ T and ρ(u) ≍ ∥u∥^{2} for u ∈ U, so H is a Cartan-decomposition subgroup.

(11c) H is conjugate (via an element of U_{α+2β}) to T ⋉ U_{α}. From [14, Prop. 3.18], we have ρ(h) ≍ ∥h∥^{2} for every h ∈ T ⋉ U_{α}. Therefore ρ(h) ≍ ∥h∥^{2} for every h ∈ H.

Lemma 7.5. [14, Lem. 2.4] Assume G = SU(2, n), and let H be a closed, connected subgroup of AN that is compatible with A. Then either

1. H = (H ∩ A) ⋉ (H ∩ N); or

2. there is a positive root ω, a nontrivial group homomorphism ψ: ker ω → U_{ω}U_{2ω}, and a closed, connected subgroup U of N, such that

(a) H = { aψ(a) | a ∈ ker ω }U;

(b) U ∩ ψ(ker ω) = e; and

(c) U is normalized by both ker ω and ψ(ker ω).

Proposition 7.6. Assume that G = SU(2, n). Let H be a closed, connected, nontrivial subgroup of AN, that is compatible with A, such that

• H ∩ N is not a Cartan-decomposition subgroup;

• H ≠ (H ∩ A)(H ∩ N); and

• dim H > 1.

Then there are positive roots ω and σ, and a one-dimensional subspace 𝔵 of (ker ω) + 𝔲_{ω} + 𝔲_{2ω}, such that 𝔥 = 𝔵 + (𝔥 ∩ 𝔫), 𝔥 ∩ 𝔫 ⊂ 𝔲_{σ} + 𝔲_{2σ}, and either:

1. ω = α, σ = α + β, and μ(H) ≈ [∥h∥, ∥h∥^{2}/(log ∥h∥)]; or

2. ω = α σ = α + 2β, and μ(H) ≈ [∥h∥^{2}/(log ∥h∥)^{2}, ∥h∥^{2}]; or

3. ω = β, σ = α + 2β, and μ(H) ≈ [∥h∥(log ∥h∥)^{r/2}, ∥h∥^{2}], where

r = \begin{cases}1 & if 𝔵 ⊂ 𝔲_{2β} \\ 2 & otherwise \end{cases}

or

4. ω = β, σ = α + β, and μ(H) ≈ [∥h∥, ∥h∥(log ∥h∥)^{r}], where r is defined as above; or

5. 𝔲 ∩ (𝔲_{ω} + 𝔲_{2ω}) ≠ 0, in which case H is a Cartan-decomposition subgroup.

Proof. We use the notation of Lemma 7.5: T = ker ω, U = H ∩ N, ψ: T → U_{ω}U_{2ω}, and H = {aψ(a)} ⋉ U.

We need only consider the cases in Corollary 6.2 for which H (now called U) is normalized by the kernel of some (reduced) positive root. Here is a list of them.

1. N_{A}(U) = ker(β): 6.2(4(b)iii), 6.2(5c), and 6.2(11b).

2. N_{A}(U) = ker(α + β): 6.2(4(b)i) and 6.2(5b).

3. N_{A}(U) = ker(α): 6.2(1b), 6.2(3), and 6.2(7b).

4. N_{A}(U) = ker(α + 2β): 6.2(11c).

5. N_{A}(U) = A: 6.2(1a), 6.2(4a), 6.2(5a), 6.2(7a), and 6.2(11a).

Note that in each of the cases with N_{A}(U) = A, there is a (reduced) positive root σ, such that 𝔲 ⊂ 𝔲_{σ} + 𝔲_{2σ}.

Case 1. Assume ω = β.

Subcase 1.1. Assume 6.2(4(b)iii). From (7.7), we know that H is a Cartandecomposition subgroup.

Subcase 1.2. Assume 6.2(5c). There is some u ∈ U, such that ϕ_{u} ≠ 0. Then, because ψ(T) ⊂ U_{β}U_{2β} normalizes U, we must have U ∩ U_{α+2β} ≠ e. This is a contradiction.

Subcase 1.3. Assume 6.2(11b). Let u ∈ 𝔲. Because U is normalized by ψ(T), there is some nonzero v ∈ 𝔲_{β} + 𝔲_{2β}, such that v normalizes 𝔲; thus, [u, v] ∈ 𝔲. Then, because ϕ_{[u,v]} = 0, but ϕ_{h} ≠ 0 for every nontrivial h ∈ U, we conclude that [u, v] = 0. However, ϕ_{u} ≠ 0, and either y_{v} ≠ 0 or y_{v} ≠ 0, so either x_{[u,v]} ≠ 0 or η_{[u,v]} ≠ 0. This is a contradiction.

Subcase 1.4. Assume N_{A}(U) = A. There is a positive root σ, such that 𝔲 ⊂ 𝔲_{σ} + 𝔲_{2σ}.

If σ = β, then, from (7.7), we know that H is a Cartan-decomposition subgroup.

Suppose σ = α + 2β. Clearly ∥h∥ ≍ a_{1}|η_{u}|. Also,

ρ(h) ≍ a_{1}|η_{u}|^{2} + a_{1}(log a_{1})^{r},

where r = 1 if ψ(T) ⊂ U_{2β} (i.e., if y_{h} = 0 for every h ∈ H) and r = 2 if ψ(T) ⊄ U_{2β}. The smallest value of ∥ρ(h)∥ relative to ∥h∥ is obtained by taking η_{u} ≍ (log a_{1})^{r/2}, resulting in ρ(h) ≍ ∥h∥(log ∥h∥)^{r/2}. Then, since ρ(u) ≍ ∥u∥^{2} for u ∈ U, we conclude that μ(H) ≈ [∥h∥(log ∥h∥)^{r/2}, ∥h∥^{2}].

Because U is normalized by the nontrivial subgroup ψ(T) of U_{β}U_{2β}, we know that σ ≠ α. Therefore, we may now assume σ = α + β. We show that μ(H) ≈ [∥h∥, ∥h∥(log ∥h∥)^{r}]. For u ∈ U, we have ρ(u) ≍ u. For a ∈ T, we have

ρ(aψ(a)) ≍ ∥a∥(log ∥a∥)^{r} ≍ ∥aψ(a)∥(log ∥aψ(a)∥)^{r}.

All that remains is to show that ρ(h) = O[∥h∥(log ∥h∥)^{r}] for every h ∈ H. Because ρ(au) ≍ au for every au ∈ T U (see [14, Cor. 3.18]) and ∥ψ(a)∥ ≍ ∥ψ(a)^{−1}∥ ≍ (log ∥h∥)^{r}, we have

ρ(h) = ρ(ψ(a))ρ(au) = O[∥ρ(ψ(a))∥∥ρ(au)∥]
= O[(log ∥a∥)^{r}∥au∥] = O[(log ∥h∥)^{r}∥h∥].

Case 2. Assume ω = α + β. The Weyl reflection corresponding to the root α conjugates each of 6.2(4(b)i) and 6.2(5b) to a subgroup with ω = β.

Thus, we may now assume N_{A}(U) = A. If σ ≠ α, then the Weyl reflection corresponding to the root α conjugates H to a subgroup with ω = β. If σ = α, then the Weyl reflection corresponding to the root β does not change ω, but conjugates H to a subgroup H_{1} with σ = α + 2β. Then (as we already observed) the Weyl reflection corresponding to the root α conjugates H_{1} to a subgroup with ω = β.

Case 3. Assume ω = α. Because U must be normalized by the nontrivial subgroup ψ(T) of U_{α}, we see that U cannot be of type 6.2(1b) or 6.2(3).

Subcase 3.1. Assume 6.2(7b). Because U must be normalized by the nontrivial subgroup ψ(T) of U_{α}, we see that y_{u} = 0 for every u ∈ U, so 𝔲 = 𝔷. Thus, again using the fact that U is normalized by ψ(T), we see that 𝔲 ⊂ 𝔲_{α+2β} + 𝔲_{2α+2β}, and the projection of 𝔲 to 𝔲_{α+2β} is one-dimensional. For every z ∈ 𝔲, we see that η_{z} ≠ 0 (because |η_{z}|^{2} ≠ x_{z}y_{z}). Thus, we conclude that dim 𝔲 = 1. Therefore H is conjugate under U_{α} to a subgroup of type 6.2(7a) (considered in Subsubcase 3.2.2 below).

Subcase 3.2. Assume N_{A}(U) = A. If σ = α, then (7.7) implies that H is a Cartandecomposition subgroup. Because U is normalized by the nontrivial subgroup ψ(T) of U_{α}, we know that σ ≠ β.

Subsubcase 3.2.1. Assume σ = α + β. We have

h = aψ(a)u = (\begin{matrix}a_{1} & a_{1}ϕ_{ψ(a)} & a_{1}x_{u} & 0 & −\frac{1}{2}a_{1}|x_{u}|^{2} + ia_{1}x_{u} \\ & a_{1} & 0 & 0 & 0 \\ & & ⋯ \end{matrix}) .

We have ∥h∥ ≍ a_{1} log a_{1} + a_{1}|x_{u}|^{2} + a_{1}|x_{u}| and, for i > 1, we have h_{i,j} = O(a_{1} + |x_{u}|). The largest value of ∥ρ(h)∥ relative to ∥h∥ is obtained by taking log a_{1} ≍ |x_{u}|^{2} (and x_{u} small), which yields ρ(h) ≍ a_{1}^{2} log a_{1} ≍ ∥h∥^{2}/ log ∥h∥. Because ρ(u) ≍ u for u ∈ U, we conclude that μ(H) ≈ [∥h∥, ∥h∥^{2}/ log ∥h∥].

Subsubcase 3.2.2. Assume σ = α + 2β. We have

h = aψ(a)u = (\begin{matrix}a_{1} & a_{1}ϕ_{ψ(a)} & 0 & a_{1}η_{u} & −a_{1}ϕ_{ψ(a)}\overline{η_{u}} \\ & a_{1} & 0 & 0 & −a_{1}\overline{η_{u}} \\ & & ⋯ \end{matrix}) .

We have h ≍ (1 + a_{1}∥ψ(a)∥)(1 + |η_{u}|) and ρ(h) ≍ a_{1}^{2}(1 + |η_{u}|^{2}) (note that

det (\begin{matrix}h_{1,2} & h_{1,n+2} \\ h_{2,2} & h_{2,n+2} \end{matrix}) = 0). The smallest value of ∥ρ(h)∥ relative to ∥h∥ is obtained

by taking η_{u} = O(1), which results in ρ(h) ≍ a_{1}^{2} ≍ ∥h∥^{2}/(log ∥h∥)^{2}. Because ρ(u) ≍ ∥u∥^{2} for u ∈ U, we conclude that μ(H) ≈ [∥h∥^{2}/(log ∥h∥)^{2}, ∥h∥^{2}].

Case 4. Assume ω = α + 2β. The Weyl reflection corresponding to the root β conjugates 6.2(11c) to a subgroup H′ with ω = α (of type 6.2(7b) with 𝔥′ = 𝔷′ ⊂ 𝔲_{α+2β} + 𝔲_{2α+2β}).

Thus, we may now assume N_{A}(U) = A. If σ ≠ β, then the Weyl reflection corresponding to the root β conjugates H to a subgroup with ω = α. Now assume σ = β. The Weyl reflection corresponding to the root α does not change ω, but conjugates H to a subgroup H_{1} with σ = α + 2β. Then (as we already observed) the Weyl reflection corresponding to the root β conjugates H_{1} to a subgroup with ω = α.

Lemma 7.7. Assume G is a connected, almost simple, linear, real Lie group of real rank two. Let H be a closed, connected, nontrivial subgroup of AN, such that H is compatible with A, and H ≠ (H ∩ A)(H ∩ N). We use the notation of [14, Lem. 2.4]: T = ker ω, H = T ⋉ U, ψ: T → U_{ω}U_{2ω}, and H = {aψ(a)} ⋉ U.

If 𝔲 ∩ (𝔲_{ω} + 𝔲_{2ω}) ≠ 0, then H is a Cartan-decomposition subgroup.

Proof. By passing to a subgroup of H, there is no harm in assuming 𝔲 ⊂ 𝔲_{ω} + 𝔲_{2ω}. We use the notation of the proof of [14, Prop. 3.17]. For each a ∈ T, clearly μ_{MA}(aψ(a)U) ⊃ μ_{MA}(aψ(a))A_{ω}^{+}, so μ_{MA}(H) ⊃ μ_{MA}({aψ(a)})A_{ω}^{+}. Beause μ_{MA}(T) = T is a line perpendicular to A_{ω}, and μ_{MA}(aψ(a)) is logarithmically close to this line, it is clear that μ_{MA}(aψ(a))A_{ω}^{+} contains all but a bounded subset of the region C. Therefore μ(H) contains all but a bounded subset of A^{+}, so H is a Cartan-decomposition subgroup.

reference Cartan projection maximum dimension
6.1(1) ρ(h) ≍ h 1
6.1(2) μ(H) ≈ [∥h∥, ∥h∥^{3/2}]  2n − 3
6.1(2)* ρ(h) ≍ ∥h∥^{3/2} 1
6.1(3a) μ(H) ≈ [∥h∥, ∥h∥^{3/2}]  2n − 3
6.1(3a)*  ρ(h) ≍ ∥h∥^{3/2}  1
6.1(3b)  ρ(h) ≍ h  2n − 3
6.1(4)  ρ(h) ≍ h  2n − 1
6.1(5) μ(H) ≈ [∥h∥, ∥h∥^{4/3}]  2n − 3
6.1(5)*  ρ(h) ≍ ∥h∥^{4/3}  1
6.1(6)  ρ(h) ≍ ∥h∥^{2}  \begin{cases}2n − 1 & n even \\ 2n − 3 & n odd \end{cases}
6.1(7) μ(H) ≈ [∥h∥^{3/2}, ∥h∥^{2}]  \begin{cases}n + 1 & n ≥ 4 \\ 3 & n = 3 \end{cases}
6.1(8)  ρ(h) ≍ ∥h∥^{3/2}  \begin{cases}3 & n ≥ 4 \\ 2 & n = 3 \end{cases}
6.1(9) μ(H) ≈ [∥h∥, ∥h∥^{3/2}]  2
6.1(10) ρ(h) ≍ ∥h∥^{2} 2
6.1(11) μ(H) ≈ [∥h∥^{5/4}, ∥h∥^{2}]  2

Table 1: The subgroups of N that are not Cartan-decomposition subgroups.

Lemma 7.8. (cf. [14, Prop. 3.16(3)]) Assume that G = SU(2, n), and let H be a nontrivial one-parameter subgroup of AN, such that H is compatible with A, but H ≠ (H ∩ A)(H ∩ N).

Then there is a ray R in A^{+}, a ray R′ in A that is perpendicular to R, and a positive number k, such that

μ(H) ≈ { rs | r ∈ R, s ∈ R′, ∥s∥ = (log ∥r∥)^{k} }.

8 Maximum dimensions of the subgroups

For convenience of reference, Tables 1, 2 and 3 list the (approximate) Cartan projection of each subgroup of AN that is not a Cartan-decomposition subgroup. The maximum possible dimension for a subgroup of each type is also listed. (These dimensions are used in applications to the existence of tessellations.)

Remark 8.1. Here are brief justifications of the dimensions listed in Tables 1, 2 and 3.

6.1(1) By assumption, we have dim H = 1.

6.1(2) Let p: 𝔥 → 𝔲_{α+β} be the natural projection. Then ker p = 𝔷 ⊂ 𝔲_{2α+2β}, so

dim 𝔥 ≤ (dim 𝔲_{α+β}) + (dim 𝔲_{2α+2β}) = 2(n − 2) + 1 = 2n − 3.
6.1(3) We may assume λ = 0. Then 𝔥 ⊂ 𝔲_{β} + 𝔲_{2β}. So
dim 𝔥 ≤ (dim 𝔲_{β}) + (dim 𝔲_{2β}) = 2(n − 2) + 1 = 2n − 3.

reference  Cartan projection maximum dimension
7.4(1a)  μ(H) ≈ [∥h∥, ∥h∥^{s}]  2
7.4(2)  μ(H) ≈ [∥h∥, ∥h∥^{3/2}]  2n − 2
7.4(2)*  ρ(h) ≍ ∥h∥^{3/2}  2
7.4(4a)  μ(H) ≈ [∥h∥, ∥h∥^{s}]  2n − 2
7.4(4(b)i)  ρ(h) ≍ h  2n − 2
7.4(4(b)ii) μ(H) ≈ [∥h∥, ∥h∥^{3/2}]  2n − 2
7.4(4(b)ii)*  ρ(h) ≍ ∥h∥^{3/2}  2
7.4(5a)  μ(H) ≈ [∥h∥, ∥h∥^{s}]  2n − 2
7.4(5c)  ρ(h) ≍ h  2n
7.4(6)  μ(H) ≈ [∥h∥, ∥h∥^{4/3}]  2n − 2
7.4(6)*  ρ(h) ≍ ∥h∥^{4/3}  2
7.4(7a)  μ(H) ≈ [∥h∥^{s}, ∥h∥^{2}]  3
7.4(7b)  ρ(h) ≍ ∥h∥^{2}  \begin{cases}2n & n even \\ 2n − 2 & n odd \end{cases}
7.4(8a) μ(H) ≈ [∥h∥^{3/2}, ∥h∥^{2}]  4
7.4(8b) μ(H) ≈ [∥h∥^{3/2}, ∥h∥^{2}]  4
7.4(9)  ρ(h) ≍ ∥h∥^{3/2}  \begin{cases}4 & n ≥ 4 \\ 3 & n = 3 \end{cases}
7.4(10)  μ(H) ≈ [∥h∥, ∥h∥^{3/2}]  3
7.4(11a)  μ(H) ≈ [∥h∥^{s}, ∥h∥^{2}]  3
7.4(11c)  ρ(h) ≍ ∥h∥^{2}  3

Table 2: The subgroups of AN that are not Cartan-decomposition subgroups, and are a nontrivial semidirect product T ⋉ U.

reference  Cartan projection  maximum dimension
7.6(1)  μ(H) ≈ [∥h∥, ∥h∥^{2}/(log ∥h∥)]  2n − 2
7.6(2)  μ(H) ≈ [∥h∥^{2}/(log ∥h∥)^{2}, ∥h∥^{2}]  2
7.6(3) (r = 1) μ(H) ≈ [∥h∥(log ∥h∥)^{1/2}, ∥h∥^{2}]  3
7.6(3) (r = 2) μ(H) ≈ [∥h∥(log ∥h∥), ∥h∥^{2}]  3
7.6(4) (r = 1) μ(H) ≈ [∥h∥, ∥h∥(log ∥h∥)]  2n − 2
7.6(4) (r = 2) μ(H) ≈ [∥h∥, ∥h∥(log ∥h∥)^{2}]  2n − 3
7.8  ρ(h) ≍ ∥h∥^{s}(log ∥h∥)^{±k}  1

Table 3: The subgroups of AN that are not Cartan-decomposition subgroups and are not a semidirect product of a torus and a unipotent subgroup.

It is easy to construct an algebra of this dimension, with or without an element u as described in (3a).

6.1(4) Let V be the projection of 𝔥 to 𝔲_{α} + 𝔲_{α+β} + 𝔲_{α+2β}. Because ϕ\overline{η} is a form of signature (2, 2) on 𝔲_{α} + 𝔲_{α+2β}, we know that dim(V ∩ (𝔲_{α} + 𝔲_{α+2β})) ≤ 2. Thus we have

dim 𝔥 ≤ dim V + dim 𝔲_{2α+2β} ≤ (dim 𝔲_{α+β} + 2) + dim 𝔲_{2α+2β}
= (2(n − 2) + 2) + 1 = 2n − 1.

6.1(5) Consider p: 𝔥 → 𝔲_{α}. Because 𝔷 = 0, we have dim ker p ≤ dim 𝔲_{α+β} = 2n − 4. Because p(𝔥) ⊂ ℝϕ_{0}, we have dim p(𝔥) ≤ 1. Thus, dim 𝔥 ≤ 2n − 3.

6.1(6) See Lemma 8.2 below. 6.1(7) See Lemma 8.3 below. 6.1(8) See Lemma 8.4 below. 6.1(9), 6.1(10), 6.1(11) are obvious from the statements. 7.4(1a) Because dim 𝔲 = 1, we have dim 𝔥 = dim 𝔱 + dim 𝔲 = 2. 7.4(2) The kernel of the projection from 𝔲 to 𝔲_{α+β} is 𝔷, so dim 𝔥 =

1 + dim U ≤ 1 + (1 + dim 𝔲_{α+β}) = 2n − 2.

7.4(4) dim 𝔥 = 1 + dim 𝔲 ≤ 1 + (dim 𝔲_{β} + dim 𝔷) = 2n − 2. 7.4(5a) dim 𝔥 ≤ dim 𝔱 + dim 𝔲_{α+β} + dim 𝔷 ≤ 1 + (2n − 4) + 1 = 2n − 2. 7.4(5c) Add 1 (the dimension of T) to the bound in 6.1(4). 7.4(6) Add 1 (the dimension of T) to the bound in 6.1(5). 7.4(7a) dim 𝔥 ≤ dim 𝔱 + dim 𝔲_{α+2β} = 1 + 2 = 3. 7.4(7b) Add 1 (the dimension of T) to the bound in 6.1(6). 7.4(8a) Because y_{u} ≠ 0 for every nonzero u ∈ 𝔲 ∩ (𝔲_{α+β} + 𝔲_{2β}), we have

dim(𝔲 ∩ (𝔲_{α+β} + 𝔲_{2β})) ≤ 1. Therefore dim 𝔥 ≤ dim 𝔱 + 1 + dim 𝔲_{α+2β} = 4.

7.4(8b) This is conjugate to 7.4(8a), via the Weyl reflection corresponding to the root α.

7.4(9) Add 1 (the dimension of T) to the bound in 6.1(8). (To achieve this bound for n ≥ 4, choose u, ũ ∈ 𝔲 ∩ (𝔲_{α} + 𝔲_{β}) in the proof of Lemma 8.4.

7.4(10) and 7.4(11) are obvious from the statements. 7.6(1) dim 𝔥 ≤ 1 + dim(𝔲_{α+β} + 𝔲_{2α+2β}) = 2n − 2. 7.6(2) Because ψ(T) normalizes (hence centralizes) U, the subgroup U

cannot be all of U_{α+2β}, so dim U ≤ 1. Therefore dim H = 1 + dim U ≤ 2. 7.6(3) dim 𝔥 ≤ 1 + dim 𝔲_{α+2β} = 3. 7.6(4) Because ψ(T) normalizes (hence centralizes) U, the projection of 𝔲

to 𝔲_{α+β} cannot be all of 𝔲_{α+β} if ψ(T) ⊄ U_{2β}, that is, if r = 2. Therefore dim U ≤ dim(𝔲_{α+β} +𝔲_{2α+2β})−(r−1) = 2n−2−r. Therefore dim 𝔥 = 1+dim U ≤ 2n−1−r.

Lemma 8.2. The maximum dimension of a subalgebra of type 6.1(6) is as stated in Table 1.

Proof. We begin by showing that dim 𝔥 ≤ 2n − 1 (cf. [16, Lem. 5.8]). Let V be the projection of 𝔥 to 𝔲_{β} + 𝔲_{α+β}. Because dim 𝔷 ≤ 3, we just need to show that dim V ≤ 2n − 4. Because V does not intersect 𝔲_{β} (or 𝔲_{α+β}, either, for that matter), and 𝔲_{β} has codimension 2n − 4 in 𝔲_{β} + 𝔲_{α+β}, this is immediate.

When n is even, there is a subgroup of dimension 2n − 1. (For example, the N subgroup of Sp(1, n/2). More general examples are constructed in [15, §4].)

Let us show that if n is odd, then dim H ≤ 2n−3. (Our proof is topological; we do not know an algebraic proof.) Suppose that dim H ≥ 2n − 2 (this will lead to a contradiction). Because dim 𝔷 ≤ 3, we have dim 𝔥/𝔷 ≥ 2n − 5. Thus, there is a (2n − 5)-dimensional real subspace X of ℂ^{n−2} and a real linear transformation T: X → ℂ^{n−2}, such that x and T x are linearly independent over ℂ, for every nonzero x ∈ X (cf. [16, Cor. 5.9]). Thus, if we define U: X → ℂ^{n−2} by Ux = ix; then x, T x, and Ux are linearly independent over ℝ, for every nonzero x ∈ X. Thus (writing n = 2k + 3): there is a (4k + 1)-dimensional real subspace X of ℝ^{4k+2} and real linear transformations T, U: X → ℝ^{4k+2}, such that x, T x, and Ux are linearly independent over ℝ, for every nonzero x ∈ X. There is no harm in assuming X = ℝ^{4k+1} (under its natural embedding in ℝ^{4k+2}).

Let E = (S^{4k}×ℝ^{4k+2})/∼, where (x, v) ∼ (−x, −v), and define a continuous map ζ: E → ℝP^{4k} by ζ(x, v) = [x], so (E, ζ) is a vector bundle over ℝP^{4k}. Then (E, ζ) ≅ τ ⊕ ϵ^{1} ⊕ γ_{4k}^{1} , where τ is the tangent bundle of ℝP^{4k}, ϵ^{1} is a trivial line bundle, and γ_{4k}^{1} is the canonical bundle of ℝP^{4k}. (To see this, note that the subbundle

{ (x, v) ∈ S^{4k} × ℝ^{4k+1} | v ⊥ x }/∼

is the total space of τ [13, pf. of Lem. 4.4, pp. 43–44], the subbundle

{ (x, v) ∈ S^{4k} × ℝ^{4k+1} | v ∈ ℝx }/∼

has the obvious section x ↦ (x, x), and the subbundle (S^{4k} × (0 × ℝ))/∼ is isomorphic to γ_{4k}^{1} via the bundle map (x, (0, t)) ↦ (x, tx).) Therefore, letting a be a generator of the cohomology ring H^{∗}(ℝP^{4k}; ℤ_{2}), we see that the total Stiefel-Whitney class of (E, ζ) is w = (1+a)^{4k+1}(1)(1+a) = (1+a)^{4k+2} [13, Eg. 2, p. 43, and Thm. 4.5, p. 45], so

w_{(4k+2)−3+1} = w_{4k} = (\begin{matrix}4k + 2 \\ 4k \end{matrix}) a^{4k} = (2k + 1)(4k + 1)a^{4k} ≠ 0

(because (2k + 1)(4k + 1) is odd). Therefore, there do not exist three pointwise linearly independent sections of (E, ζ) [13, Prop. 4, p. 39].

Any linear transformation Q: ℝ^{4k+1} → ℝ^{4k+2} induces a continuous function Q̂: S^{4k} → ℝ^{4k+2}, such that Q̂(−x) = −Q̂(x) for all x ∈ S^{4k}; that is, a section of (E, ζ). Thus, Id, T, and U each define a section of (E, ζ). Furthermore, these three sections are pointwise linearly independent, because x, T x, and Ux are linearly independent over ℝ, for every x ∈ S^{4k}. This contradicts the conclusion of the preceding paragraph.

Lemma 8.3. The maximum dimension of a subalgebra of type 6.1(7) is as stated in Table 1.

Proof. Replacing H by a conjugate under ⟨U_{α}, U_{−α}⟩, we may assume x_{v} = 0. Therefore x_{z} = 0 for every z ∈ 𝔷. (Thus, in particular, we have dim 𝔷 ≤ 2.)

For the projection p: 𝔥 → 𝔲_{α+β}, we have ker p = ℝv + 𝔷. (There cannot exist a linearly independent v′; otherwise, replacing v′ by some linear combination with v, we could assume x_{v′} = 0, which is impossible.) Thus, dim ker p ≤ 3.

Because x_{z} = 0 for every z ∈ 𝔷, p(𝔥) must be a totally isotropic subspace for the symplectic form Im(xx̃^{†}), so dim p(𝔥) ≤ n−2. Therefore dim 𝔥 ≤ (n−2)+3 = n + 1.

For n ≥ 4, here is an example that achieves this bound:

𝔥 = \{ (\begin{matrix}0 & 0 & x_{1} & x_{2} & ⋯ & x_{n−2} & η & ix \\ 0 & 0 & ix & x_{1} & ⋯ & x_{n−3} & ix_{n−2} & −\overline{η} \\ & & & ⋯ \end{matrix}) | \begin{matrix}x, x_{1}, …, x_{n−2} ∈ ℝ, \\ η ∈ ℂ \end{matrix} \} .

For v ∈ 𝔥, we claim that dim_{ℂ}⟨x_{v}, y_{v}⟩ = 1 only if either x_{v} = 0 or y_{v} = 0. (In either case, it is clear from the definition of 𝔥 that either x_{u}|y_{u}|^{2} ≠ 0 or y_{u}|x_{u}|^{2} ≠ 0, respectively.) Suppose dim_{ℂ}⟨x_{v}, y_{v}⟩ = 1, with x_{v} ≠ 0 and y_{v} ≠ 0. There is some nonzero λ ∈ ℂ, such that y_{v} = λx_{v}. We must have x_{1} ≠ 0. (Otherwise, let i ∈ {1, 2, …, n − 2} be minimal with x_{i} ≠ 0. Then x_{i−1} = y_{i} = λx_{i} ≠ 0, contradicting the minimality of i.) Because y_{1} = ix is pure imaginary, but x_{1} is real, we see that λ is pure imaginary. On the other hand, y_{2} = x_{1} is real (and nonzero), and x_{2} is also real, so λ is real. Because λ ≠ 0, this is a contradiction.

Now let n = 3, and suppose dim 𝔥 = 4. (This will lead to a contradiction.) Because equality is attained in the proof above, we must have dim p(𝔥) = n−2 = 1 and dim 𝔷 = 2. In particular, there exists w ∈ 𝔥 with x_{w} ≠ 0. For t ∈ ℝ, let w_{t} = w + tv. Then

x_{w_{t}}|y_{w_{t}}|^{2} +y_{w_{t}}|x_{w_{t}}|^{2} +2 Im(x_{w_{t}}y_{w_{t}}^{†}η_{w_{t}}) = t^{3}x_{v}|y_{v}|^{2} +O(t^{2}) → \begin{cases}+x_{v}∞ & as t → ∞ \\ −x_{v}∞ & as t → −∞. \end{cases}

Thus, this expression changes sign, so it must vanish for some t. This is a contradiction, because dim_{ℂ}⟨x_{w_{t}}, y_{w_{t}}⟩ = 1 for every t.

Lemma 8.4. The maximum dimension of a subalgebra of type 6.1(8) is as stated in Table 1.

Proof. For n ≥ 4, here is the construction of 3-dimensional subalgebras of 𝔫 of this type. Let ϕ = 1 and ϕ̃ = i. Choose y, ỹ, x, x̃ ∈ ℂ^{n−2}, η, η̃ ∈ ℂ, and x, x̃ ∈ ℝ, such that

|y|^{2} = |ỹ|^{2} = 3iyỹ^{†} ≠ 0.  (8.1)

Now, choose y, ỹ ∈ ℝ, such that

Im(ỹx^{†} − iyx^{†} + ỹx^{†} − yx̃^{†} + iỹ) = 0  (8.2)

and

Im(ỹx̃^{†} − iyx̃^{†} + iỹx^{†} − iyx̃^{†} + iy) = 0.  (8.3)

Define u, ũ as in Eq. (2.3), and let v = [u, ũ]. Then y_{v} ≠ 0 and x_{v} ≠ 0, but, from Eq. (8.1), Eq. (8.2) and Eq. (8.3), we have [v, u] = [v, ũ] = 0. Thus, we may let 𝔥 be the subalgebra generated by u and ũ. (So {u, ũ, v} is a basis of 𝔥 over ℝ.)

Note that, because |yỹ^{†}| = |y|^{2}/3 ≠ |y|^{2}, we know that y and ỹ must be linearly independent over ℂ. Thus, these 3-dimensional examples do not exist when n = 3.

References

[1] Benoist, Y., Actions propres sur les espaces homogènes réductifs, Ann. Math. 144 (1996), 315–347.

[2] Borel, A., Compact Clifford-Klein forms of symmetric spaces, Topology 2 (1963), 111–122.

[3] Helgason, S., “Differential Geometry, Lie Groups, and Symmetric Spaces”, Academic Press, New York, 1978.

[4] Iozzi, A., and D. Witte, Tessellations of homogeneous spaces of classical groups of real rank two, (in preparation).

[5] Kobayashi, T., On discontinuous groups acting on homogeneous spaces with non-compact isotropy groups, J. Geom. Physics 12 (1993), 133–144.

[6] —, Proper action on a homogeneous space of reductive type, Math. Ann. 285 (1989), 249–263.

[7] —, Discontinuous groups and Clifford-Klein forms of pseudo-Riemannian homogeneous manifolds, in: B. Ørsted and H. Schlichtkrull, eds., “Algebraic and Analytic Methods in Representation Theory”, Academic Press, New York, 1997, pp. 99–165.

[8] —, Criterion of proper actions on homogeneous spaces of reductive groups, J. Lie Th. 6 (1996), 147–163.

[9] Kostant, B., On convexity, the Weyl group, and the Iwasawa decomposition, Ann. Sc. ENS. 6 (1973), 413–455.

[10] Kulkarni, R., Proper actions and pseudo-Riemannian space forms, Adv. Math. 40 (1981), 10–51.

[11] Labourie, F., Quelques résultats récents sur les espaces localement homogènes compacts, in: P. de Bartolomeis, F. Tricerri and E. Vesentini, eds., “Manifolds and Geometry”, Symposia Mathematica, v. XXXVI, Cambridge U. Press, 1996.

[12] Margulis, G. A., Existence of compact quotients of homogeneous spaces, measurably proper actions, and decay of matrix coefficients, Bull. Soc. Math. France 125 (1997), 447–456.

[13] Milnor, J. W., and J. D. Stasheff, “Characteristic Classes”, Princeton U. Press, Princeton, 1974.

[14] Oh, H., and D. Witte, Cartan-decomposition subgroups of SO(2, n), Trans. Amer. Math. Soc. (to appear).

[15] —, New examples of compact Clifford-Klein forms of homogeneous spaces of SO(2, n), Internat. Math. Res. Not. 5 (2000), 235-251.

[16] —, Compact Clifford-Klein forms of homogeneous spaces of SO(2, n), Geometriae Dedicata (to appear).

[17] Raghunathan, M. S., “Discrete Subgroups of Lie Groups”, Springer-Verlag, New York, 1972.

[18] van den Dries, L. P. D., “Tame Topology and O-minimal Structures”, Cambridge U. Press, Cambridge, 1998.

[19] Wilkies, A., Model completeness results for expansions of the ordered field of real numbers by restricted Pfaffian functions and the exponential function, J. Amer. Math. Soc. 9 (1996), 1051–1094.

Iozzi and Witte 543

[20] —, O-Minimality, Documenta Mathematica, Extra Volume ICM I (1998), 457–460.

A. Iozzi Department of Mathematics University of Maryland College Park, MD 20910 USA and FIM ETH Zentrum CH–8092 Zürich Switzerland iozzi@math.ethz.ch Dave Witte Department of Mathematics Oklahoma State University Stillwater, OK 74078 USA dwitte@math.okstate.edu

Received July 27, 2000 and in final form February 2, 2001