Characters of Irreducible Representations

of Simple Lie Superalgebras

Vera Serganova

1991 Mathematics Subject Classification: 17A70, 17B70

Keywords and Phrases: Lie superalgebra, character, irreducible representation, cohomology

1. Introduction

Simple Lie superalgebras were classified in 1977 by V. Kac [6]. These superalgebras can be divided into three groups

1. Contragredient Lie superalgebras, i.e. Lie superalgebras which can be determined by a Cartan matrix. These superalgebras have an invariant symmetric form and Cartan involution. There are two families of such algebras sl (m|n) (factored by the center when m = n) and osp (m|2n). The Lie superalgebra osp (4|2) has a one-parameter deformation, called D (α). There are also two exceptional Lie superalgebras G_{3} and F_{4}.

2. New “strange” superalgebras Q (n) and P (n), the former consists of operators commuting with an odd nondegenerate operator, the latter consists of operators preserving a non-degenerate odd symmetric form.

3. Cartan type superalgebras W_{n}, S_{n}, S′_{n} and SH_{n}, i.e. superalgebra of vector fields on a supermanifold of pure odd dimension and its simple subalgebras. In [7] it was shown that all finite-dimensional irreducible representations of a

simple Lie superalgebra 𝔤 are enumerated by a highest weight λ ∈ 𝔥^{∗}, satisfying certain conditions of dominance, 𝔥 being a Cartan subalgebra.

The problem of finding the character of an irreducible highest weight module L_{λ} for a general dominant λ appeared to be unexpectedly difficult. It was solved first for Lie superalgebras W_{n} and S_{n} of Cartan type by J. Bernstein and D. Leites in [1] and for SH_{n} by A. Shapovalov [20]. They considered a module M_{λ} of tensor fields on a supermanifold of purely odd dimension and described completely its Jordan–Hölder series. Since ch M_{λ} and the multiplicity [M_{λ} : L_{μ}] are known, one can obtain ch L_{λ} by solving a simple system of linear equations.

For contragredient and strange Lie superalgebras the problem was solved in [7] in the particular case of a generic (typical) highest weight λ. The character is given by a nice Weyl type formula:

ch L_{λ} = D ∑_{w∈W} sgn w · e^{w(λ)}, D = \begin{matrix}∏ & α∈Δ_{1}^{+} & (e^{α/2} + e^{−α/2}) \\ ∏ & α∈Δ_{0}^{+} & (e^{α/2} − e^{−α/2}) \end{matrix}, (1.1)

where W is the Weyl group of 𝔤_{0}.

Documenta Mathematica · Extra Volume ICM 1998 · II · 583–593

This formula can be obtained from the Borel–Weil–Bott theorem for a flag supermanifold and a typical invertible sheaf O_{λ}. Geometry of flag supermanifolds was studied in 1980s by Yu. Manin and his pupils I. Penkov, I. Skornyakov and A. Voronov. It was shown by Penkov and Skornyakov in [16] that the invertible sheaf O_{λ} with a typical dominant weight λ satisfies the Borel–Weil–Bott theorem, namely H^{0}O_{λ} = L_{λ} and H^{i}O_{λ} = 0 for i > 0.

On the other hand, any invertible sheaf O_{λ} on G/B can be considered as a sheaf L_{λ} on the underlying flag manifold G_{0}/B_{0}. L_{λ} has a filtration by invertible sheaves O_{λ+ν} (G_{0}/B_{0}), where ν runs over the set of sums of odd negative roots ∑_{α_{i}∈Δ_{1}^{−}} α_{i}. Therefore one can write down the Euler characteristic of L_{λ}

E_{λ} = ∑_{i} (−1)^{i} ch H^{i}O_{λ} = ∑_{ν} ∑_{i} (−1)^{i} ch H_{G_{0}/B_{0}}^{i}O_{λ+ν}, (1.2)

using additivity of Euler characteristic. It happens to be exactly the Kac character formula (1.1).

If a highest weight λ is atypical the Borel–Weil–Bott theorem fails. Depending on the degree of atypicality of λ the 𝔤-module structure of H^{i}O_{λ} becomes more and more complicated. There were several conjectures and partial results about a character formula for a general dominant weight (see [2, 13, 22, 21, 9, 14, 4]). For the case 𝔤 = gl (m|n) two different formulae were conjectured in [17] and in [5]. The first conjecture was proven in [18]. The second one is believed to be equivalent to the first one.^{1} For 𝔤 = Q (n) the problem was solved in [15]. For 𝔤 = osp (m|2n) it is solved just recently, we announce the results here.

We use the same method to solve the problem for gl, osp and Q. Namely we calculate the “Euler-multiplicities” a_{λ,μ} = ∑ (−1)^{i} [H_{G/B}^{i}O_{λ} : L_{μ}] (in some cases we have to use a suitable parabolic subgroup P instead of Borel subgroup B). Since a lot of numbers a_{λ,μ} vanish, one can find ch L_{λ} from the system ∑_{μ} a_{λ,μ} ch L_{μ} = E_{λ} and (1.2). To calculate the coefficients a_{λ,μ}, we represent B as the end of a flag of parabolic subgroups G = P^{(1)} ⊃ P^{(2)} ⊃ ⋯ ⊃ P^{(n)} = B. Composing push down of sheaves, the coefficients a_{λ,μ} can be expressed in terms of similar coefficients a_{λ,μ}^{(j)} for the supermanifold P^{(j)}/P^{(j+1)}.

In fact cohomology of “dominant sheaves” on P^{(j)}/P^{(j+1)} can be completely described by using a certain analogue of translation and reflection functors. The space of dominant weights can be stratified by the degree of atypicality. A translation functor allows us to move inside a stratum (compare Theorem 2.5 ), while a reflection functor increases the degree of atypicality by one. This together with the fact that cohomology of irreducible vector bundles on P^{(j)}/P^{(j+1)} are almost semisimple P^{(j)}-modules (see Lemma 4.2) helps to write down the crucial recurrence relations on a_{λ,μ}^{(j)} (compare Theorems 4.1 and 4.3 ).

In this paper we outline the schematic of the character formulae for gl and osp omitting all the details needed for the proof only. We also omit the case of Q_{n} containing additional complications, since irreducible representations of 𝔥 are not 1-dimensional.

1Note that in [18] references to [5] and [22] were corrupted.

Documenta Mathematica · Extra Volume ICM 1998 · II · 583–593

Currently the problem of finding irreducible characters remains open for exceptional Lie superalgebras and P (n). While similar methods should work for exceptional superalgebras, in the case of P (n) it is unclear how to define translation and reflection functors: the center of the universal enveloping algebra of P (n) is too small and central characters do not separate blocks.

2. Dominant weights, central characters and blocks

Throughout this paper 𝔤 stands for one of the Lie superalgebras gl (m|n), osp (2m|2n) or osp (2m + 1|2n). The Lie superalgebra 𝔤 has a root decomposition

𝔤 = 𝔥 ⊕ ⨁_{α∈Δ} 𝔤_{α}.

The roots are called even or odd depending on the parity of the root space 𝔤_{α}. Denote the set of even (correspondingly odd) roots by Δ_{0} (correspondingly Δ_{1}). Clearly, Δ = Δ_{0} ∪ Δ_{1}. Many odd roots are isotropic, put Δ^{is} = {α | (α, α) = 0}, where (,) stands for the Killing form.

Describe the set of roots in the standard basis {δ_{1}, …, δ_{n}, ε_{1}, …, ε_{m}} of 𝔥^{∗}. Note that (δ_{i}, δ_{j}) = δ_{i,j}, (ε_{i}, ε_{j}) = −δ_{i,j}.

Let 𝔤 = gl (m|n). Then

Δ_{0} = {ε_{i} − ε_{j} | i, j = 1, …, m} ∪ {δ_{i} − δ_{j} | i, j = 1, …, n} ,

Δ_{1} = Δ^{is} = {± (ε_{i} − δ_{j}) | i = 1, …, m, j = 1, …, n} .
Let 𝔤 = osp (2m|2n). Then Δ_{1} = Δ^{is} is the same as for 𝔤 = gl (m|n),

Δ_{0} = {ε_{i} ± ε_{j} | i, j = 1, …, m, i ≠ j} ∪ {δ_{i} ± δ_{j} | i, j = 1, …, n} .

Let 𝔤 = osp (2m + 1|2n). Then

\begin{aligned}Δ_{0} = \{ε_{i} ± ε_{j} | i, j = 1, …, m, i ≠ j\} & ∪ \{±ε_{i} | i = 1, …, m\} \\ & ∪ \{δ_{i} ± δ_{j} | i, j = 1, …, n\} , \end{aligned}

Δ^{is} = {± (ε_{i} − δ_{j}) | i = 1, …, m, j = 1, …, n} .
Δ_{1} = Δ^{is} ∪ {±δ_{i} | i = 1, …, n} .

Fix a subdivision Δ = Δ^{+} ∪ Δ^{−} and a triangular decomposition 𝔤 = 𝔫^{−} ⊕ 𝔥 ⊕ 𝔫^{+}, defined by 𝔫^{±} = ⨁_{α∈Δ^{±}} 𝔤_{α}. A choice of Δ^{+} is not unique, here we fix it by enumerating simple roots in each case.

For 𝔤 = gl (m|n) choose simple roots σ_{1} = δ_{1} − δ_{2}, …, σ_{n−1} = δ_{n−1} − δ_{n}, σ_{n} = δ_{n} − ε_{1}, …, σ_{m+n−1} = ε_{m−1} − ε_{m}.

For 𝔤 = osp (2m|2n) choose simple roots σ_{1} = δ_{1} − δ_{2}, …, σ_{n−1} = δ_{n−1} − δ_{n}, σ_{n} = δ_{n} − ε_{1}, …, σ_{m+n−1} = ε_{m−1} − ε_{m}, σ_{m+n} = ε_{m−1} + ε_{m}.

For 𝔤 = osp (2m + 1|2n) choose simple roots σ_{1} = δ_{1} − δ_{2}, …, σ_{n−1} = δ_{n−1} − δ_{n}, σ_{n} = δ_{n} − ε_{1}, …, σ_{m+n−1} = ε_{m−1} − ε_{m}, σ_{m+n} = ε_{m}.

For an even root α put α^{∨} = α/ (α, α). We say that λ ∈ 𝔥^{∗} is integral if (λ, α^{∨}) ∈ ℤ for any α ∈ Δ_{0}. Denote the set of integral weights by Λ.

Denote by L_{λ} an irreducible module generated by highest vector v of weight λ − ρ, i.e., 𝔫^{+}v = 0, hv = ⟨λ − ρ, h⟩ v for h ∈ 𝔥, and

ρ = 1/2 ∑_{α∈Δ_{0}^{+}} α − 1/2 ∑_{α∈Δ_{1}^{+}} α.

Call a weight λ ∈ Λ dominant if dim L_{λ} < ∞. Denote the set of dominant weights by Λ^{+}. The conditions on λ to be dominant were first calculated in [6]. We reproduce them here in our notations.

Proposition 2.1. Let 𝔤 = gl (m|n), osp (2m|2n) or osp (2m + 1|2n). Let λ = a_{1}δ_{1} +⋯+a_{n}δ_{n} +b_{1}ε_{1} +⋯+b_{m}ε_{m} ∈ Λ. Then λ ∈ Λ^{+} iff the following conditions on a_{i} and b_{j} hold:

1. for gl (m|n) : a_{i} − a_{i+1}, b_{j} − b_{j+1} ∈ ℤ_{>0};

2. for osp (2m|2n) : a_{i}, b_{j} ∈ ℤ, a_{1} > a_{2} > ⋯ > a_{n} > −m, b_{1} > b_{2} > ⋯ > b_{m−1} > |b_{m}| and for each l ≤ 0, l ≥ a_{n}, b_{m+l} = −l;

3. for osp (2m + 1, 2n), a_{i} ∈ 1/2 + ℤ, b_{j} ∈ 1/2 + ℤ or ℤ, a_{1} > a_{2} > ⋯ > a_{n} ≥ 1/2 − m, b_{1} > b_{2} > ⋯ > b_{m} > 0 and for each l ≤ 0, a_{n} ≤ l − 1/2, b_{m+l} = 1/2 − l.

Remark 2.2. For osp type superalgebras one can not choose Δ^{+} in such a way that the set of simple roots for Δ_{0}^{+} is the subset of simple roots for Δ^{+}. That is why in this case the conditions on dominance with respect to 𝔤_{0} differ from the conditions on dominance with respect to 𝔤.

Let 𝔤 = osp (2m|2n) or osp (2m + 1|2n). By Proposition 2.1 if λ ∈ Λ^{+} and a_{r} > 0 ≥ a_{r+1}, one can find odd isotropic roots δ_{r+1} − ε_{i_{1}}, …, δ_{n} − ε_{i_{s}} orthogonal to λ. Call the set of such roots the tail of λ and denote it by T_{λ}. Call the number t_{λ} = s = n − r the tail length of λ. For 𝔤 = gl (m|n) put t_{λ} = 0.

Note also that in the case 𝔤 = osp (2m|2n) there is a symmetry of Dynkin diagram which induces an outer automorphism τ such that τ (σ_{m+n−1}) = σ_{m+n}. The automorphism τ acts on the set of dominant weights. In what follows we always assume that b_{m} ≥ 0. If b_{m} < 0 we can obtain all coefficients using the rule a_{τ(λ),τ(μ)} = a_{λ,μ}, a_{τ(λ),μ} = 0 if τ (μ) < μ, τ (λ) < λ.

Let F = F (𝔤) be the category formed by finite-dimensional 𝔤-modules. To describe the structure of F, consider the center Z (𝔤) of the universal enveloping algebra U (𝔤). Recall that a central character is a homomorphism χ: Z (𝔤) → ℂ. We say that a 𝔤-module M has a central character χ if for any z ∈ Z (𝔤) , x ∈ M there is N ∈ ℤ_{≥0} such that (z − χ (z) id)^{N} · x = 0. Clearly any finite-dimensional indecomposable 𝔤-module has some central character, and any finite-dimensional 𝔤-module decomposes into a direct sum of submodules with central characters.

We use a Harish–Chandra homomorphism HC: Z (𝔤) ↪ Pol (𝔥^{∗}). The construction of this homomorphism is the same as for semi-simple Lie algebras (see for example [3]). Thus, any λ ∈ 𝔥^{∗} defines a central character χ_{λ} by the rule χ_{λ} (z) = HC (z) (λ). Definition of HC immediately implies that an irreducible module L_{λ} has a central character χ_{λ}. A central character χ is dominant if χ = χ_{λ} at least for one λ ∈ Λ^{+}.

The following statement was first formulated in [8] and proved in [19] and [12].

Proposition 2.3. Let λ, μ ∈ Λ, W be the Weyl group of 𝔤_{0}. Then χ_{λ} = χ_{μ} iff there is a sequence of isotropic roots α_{1}, …, α_{s} ∈ Δ^{is} and w ∈ W such that μ = w (λ + α_{1} + ⋯ + α_{s}) and (λ + α_{1} + ⋯ + α_{i−1}, α_{i}) = 0 for i = 1, …, s.

Let F_{χ} = F_{χ} (𝔤) be the full subcategory of F consisting of modules with central character χ. Obviously, F = ⨁ F_{χ}, where the summation is taken over all dominant central characters χ. Different categories F_{χ} may be equivalent. They fall into one of 4 series as we will see in Theorem 2.6.

State more constructive condition for χ_{λ} = χ_{μ}. For λ ∈ Λ^{+} let A_{λ} = {α_{1}, …, α_{k}} be a maximal set of mutually orthogonal positive isotropic roots such that (λ, α_{i}) = 0 for i = 1, …, k. If 𝔤 = gl (m|n), then the set A_{λ} is uniquely defined. For osp-type 𝔤 we choose A_{λ} in such way that T_{λ} ⊆ A_{λ}. The number k = |A_{λ}| is called the degree of atypically of λ and is denoted by #λ. A weight λ ∈ Λ^{+} is typical if #λ = 0.

Let 𝔥̄_{λ}^{∗} be the subspace of 𝔥^{∗} generated by all basis vectors ε_{i}, δ_{j} orthogonal to the roots from A_{λ}. Denote by λ̄ the image of λ under the orthogonal projection 𝔥^{∗} → 𝔥̄_{λ}^{∗}. One can see that if #λ = #μ, then w \binom{}{𝔥̄_{λ}^{∗}} = 𝔥̄_{μ}^{∗} for some w ∈ W.

Proposition 2.4. Let λ, μ ∈ Λ^{+}. Then χ_{λ} = χ_{μ} iff #λ = #μ and w \binom{}{λ̄} = μ̄ for some w ∈ W.

By Proposition 2.4 one can correctly define #χ for a dominant central character χ by putting #χ =^{def} #λ for any dominant λ with χ_{λ} = χ. Moreover, one can define χ̄ as a typical central character for Lie superalgebra 𝔤̄, here 𝔤̄ is an appropriate subalgebra of 𝔤 isomorphic to gl (m − k|n − k), osp (2 (m − k) |2 (n − k)) or osp (2 (m − k) + 1|2 (n − k)) depending on the type of 𝔤, and k = #χ.

Theorem 2.5. A category F_{χ} is indecomposable for any dominant central character χ. If 𝔤 = gl (m|n) or osp (2m + 1|2n), then two categories F_{χ_{1}} and F_{χ_{2}} are equivalent iff #χ_{1} = #χ_{2}.

Let 𝔤 = osp (2m|2n) and τ̄ be the outer automorphism of 𝔤̄ induced by the symmetry of Dynkin diagram. Then two categories F_{χ_{1}} and F_{χ_{2}} are equivalent iff #χ_{1} = #χ_{2}, and for both i = 1, 2 either τ̄ \binom{}{χ̄_{i}} = χ̄_{i} or τ̄ \binom{}{χ̄_{i}} ≠ χ̄_{i}.

An indecomposable category F_{χ} is called a block of F.

Theorem 2.6. Let 𝔤 = gl (m|n), osp (2m + 1|2n) or osp (2m|2n). A block F_{χ}

with #χ = k is equivalent to F′_{χ_{ρ′}} =^{def} F_{χ_{ρ′}} (𝔤′), where 𝔤′ = gl (k|k) if 𝔤 = gl (m|n), 𝔤′ = osp (2k + 1|2k) if 𝔤 = osp (2m + 1|2n), 𝔤′ = osp (2k + 2|2k) if

𝔤 = osp (2m|2n) and τ̄ \binom{}{χ̄} = χ̄, 𝔤′ = osp (2k|2k) if 𝔤 = osp (2m|2n) and τ̄ \binom{}{χ̄} ≠ χ̄,

here ρ′ is the analogue of ρ for 𝔤′.

Let Φ: F_{χ} → F′_{χ_{ρ′}} be a functor establishing equivalence of categories of theorem 2.6. Then Φ sends irreducible objects to irreducible objects, describe the corresponding mapping of highest weights:

Proposition 2.7. Let L_{λ} ∈ Ob F_{χ} and L_{λ′} = ΦL_{λ}. Let λ = ∑_{i=1}^{n} a_{i}δ_{i} + ∑_{j=1}^{m} b_{j}ε_{j}.

1. If 𝔤 = gl (m|n), A_{λ} = {α_{1} = δ_{i_{1}} − ε_{j_{1}}, …, α_{k} = δ_{i_{k}} − ε_{j_{k}}}, then

λ′ = ∑_{p=1}^{k} a′_{p} (δ_{p} − ε_{k−p+1}) , a′_{p} = a_{i_{p}} + (ρ, α_{p}) ;

2. If 𝔤 = osp (2m|2n) or osp (2m + 1|2n), and

A_{λ} = {δ_{i_{1}} + ε_{j_{1}}, …, δ_{i_{r}} + ε_{j_{r}}, δ_{i_{r+1}} − ε_{j_{r+1}}, …, δ_{i_{k}} − ε_{j_{k}}} ,

then λ′ = ∑_{p=1}^{k} (a′_{p}δ_{p} + |a′_{s(p)}|ε_{p}), a′_{p} = a_{i_{p}} − x_{p} · sgn a_{i_{p}}, where

x_{p} = # {q, l | (λ̄, δ_{q}) < |a_{i_{p}}|, 0 ≠ − (λ̄, ε_{l}) < |a_{i_{p}}|} ,

and s is a permutation such that |a_{s(1)}| > ⋯ > |a_{s(k)}|. In particular t_{λ} = t_{λ′}.

Example 2.8. Let 𝔤 = gl (3|3), λ = 5δ_{1} + 3δ_{2} − δ_{3} + 4ε_{1} + 3ε_{2} + ε_{3}. Then A_{λ} = {δ_{3} − ε_{3}}, #λ = 1, 𝔤̄ ≃ gl (2|2), 𝔤′ ≃ gl (1|1), λ̄ = 5δ_{1} + 3δ_{2} + 3ε_{2} + ε_{3}, λ′ = −3δ_{1} + 3ε_{1}.

Example 2.9. Let 𝔤 = osp (8|6), λ = 4δ_{1} + δ_{2} − 3δ_{3} + 3ε_{1} + 2ε_{2} + ε_{3} + 0ε_{4}. Then A_{λ} = {δ_{2} + ε_{3}, δ_{3} − ε_{1}} , t_{λ} = 1, #λ = 2, 𝔤̄ ≃ osp (4|2), 𝔤′ ≃ osp (6|4), λ̄ = 4δ_{1} + 2ε_{2} + 0ε_{4}, λ′ = δ_{1} − 2δ_{2} + 2ε_{1} + ε_{2} + 0ε_{3}.

3. Borel–Weil–Bott theorem and geometric induction

Let 𝔟 = 𝔥 ⊕ 𝔫^{+} be a Borel subalgebra. For a parabolic subalgebra 𝔭 ⊇ 𝔟 denote by Δ_{𝔭} the set of roots α such that 𝔤_{±α} ⊆ 𝔭. Denote by L_{λ} (𝔭) an irreducible 𝔭module with highest weight λ, which of course remains irreducible after restriction to a reductive subalgebra 𝔤_{𝔭} = 𝔥 ⊕ ⨁_{α∈Δ_{𝔭}} 𝔤_{α}. Due to the geometric origin of the following argument we will need representations of the supergroup P corresponding to 𝔭. A flag supermanifold G/P is a compact homogeneous supermanifold with the underlying manifold G_{0}/P_{0} (for definitions see [11] or [10]). Any finite-dimensional P -module M induces a vector bundle O (M) on G/P with the fiber M over P. Let^{2}

Γ_{i}^{P} (M) = [H_{G/P}^{i} O (M^{∗})]^{∗} .

Since G/P is compact, Γ_{i}^{P} (M) is a finite-dimensional G-module. One can consider Γ_{i}^{P} as a derived functor from the category F (P) of finite dimensional P -modules to the category F.

Lemma 3.1. Let 𝔫_{𝔭} be the nilpotent ideal in 𝔭 such that 𝔭 = 𝔤_{𝔭} ⊕ 𝔫_{𝔭}. If X ∈ Ob F, then coinvariants X_{𝔫_{𝔭}} form a P -module, there is a natural inclusion X ↪ H_{G/P}^{0} O (X_{𝔫_{𝔭}}).

If M ∈ Ob F_{χ}, then Γ_{i}^{P} (M^{𝔫_{𝔭}}) ∈ Ob F_{χ}, and there is a natural projection Γ_{0}^{P} (M^{𝔫_{𝔭}}) → M. In particular when M = L_{λ} there is a natural projection Γ_{0}^{P} (L_{λ} (P)) → L_{λ}.

2We use doubled duality to avoid a problematic notion of antidominant weight.

Documenta Mathematica · Extra Volume ICM 1998 · II · 583–593

Lemma 3.2. Let M be a P -module and E^{P} (M) = ∑ (−1)^{i} ch Γ_{i}^{P} (M). Consider D from (1.1). Then

E^{P} (M) = D ∑_{w∈W} sgn w · w (\frac{e^{ρ} ch M}{∏_{α∈Δ_{𝔭}∩Δ_{1}^{+}} (1 + e^{−α})}) . (3.1)

The following theorem is a generalization of a result by Penkov–Skornyakov. We say that λ ∈ Λ^{+} is P -typical if A_{λ} ⊆ Δ_{𝔭}.

Theorem 3.3. Let λ be a P -typical dominant weight. Then Γ_{0}^{P} (L_{λ} (𝔭)) = L_{λ} and Γ_{i}^{P} (L_{λ} (𝔭)) = 0 for i > 0.

Generalizing this theorem, denote by [M : L_{μ}] the multiplicity of an irreducible module L_{μ} in a 𝔤-module M. Define the Kazhdan–Lusztig polynomials and coefficients by:

K_{λ,μ}^{P} (q) = ∑_{i=0}^{dim G/P} [Γ_{i}^{P} (L_{λ} (𝔭)) : L_{μ}] q^{i}, a_{λ,μ}^{P} = K_{λ,μ}^{P} (−1) .

Let E_{λ}^{P} =^{def} E^{P} (L_{λ} (𝔭)). Clearly

∑_{μ} a_{λ,μ}^{P} ch L_{μ} = E_{λ}^{P} . (3.2)

Proposition 3.4. Let 𝔭 be a parabolic subalgebra, λ ∈ Λ^{+} and T_{λ} ⊆ Δ_{𝔭}. Then K_{λ,λ}^{P} = 1, and K_{λ,μ}^{P} ≠ 0 implies μ ≤ λ, χ_{λ} = χ_{μ}.

Let 𝔤 = gl (m|n), then t_{λ} = 0. Put P = B. By Proposition 3.4 the ma-

trix (a_{λ,μ}^{B}) is unipotent, thus easy to invert. Let (b_{λ,μ}) = (a_{λ,μ}^{B})^{−1}. Then the

equations (3.2) imply ch L_{λ} = ∑_{μ≤λ} b_{λ,μ}E_{μ}^{B}.

If 𝔤 is an algebra of osp type then the matrix (a_{λ,μ}^{B}) is not invertible. Let

Λ_{s}^{+} = {λ ∈ Λ^{+} | t_{λ} = s} and 𝔭^{(r)} be the parabolic subalgebra such that Δ_{𝔭^{(r)}} is generated by the simple roots σ_{r}, …, σ_{n+m}. One can see that T_{λ} ⊆ Δ_{𝔭^{(r+1)}} for any

λ ∈ Λ_{n−r}^{+}. By Proposition 3.4 the matrix (a_{λ,μ}^{P^{(r+1)}})_{λ,μ∈Λ_{n−r}^{+}} is again unipotent,

thus easy to invert. Since μ ≤ λ implies t_{μ} ≥ t_{λ}, the equation

∑_{μ∈Λ_{n−r}^{+}} a_{λ,μ}^{P^{(r+1)}} ch L_{μ} = E_{λ}^{P^{(r+1)}} − ∑_{t_{ν}>t_{λ}} a_{λ,ν}^{P^{(r+1)}} ch L_{ν}.

This taken together with (3.1) expresses ch L_{λ} in terms of ch L_{μ} (𝔭^{(r+1)}) = ch L_{μ} (𝔤_{𝔭^{(r+1)}}), a_{μ,ν}^{P^{(r+1)}} for r = n − t_{λ}, μ, ν ∈ Λ^{+} with t_{μ} = t_{λ}, t_{ν} ≥ t_{λ}, and ch L_{ν′} for t_{ν′} > t_{λ}. If t_{λ} < n, rk 𝔤_{𝔭^{(r+1)}} < rk 𝔤, which gives a recurrence relation for ch L_{λ}.

What remains is the case t_{λ} = n. Then T_{λ} = A_{λ}, and λ is Q-typical for the parabolic subalgebra 𝔮 with Δ_{𝔮} generated by σ_{1}, …, σ_{m+n−1}. By theorem 3.3

ch L_{λ} = E_{λ}^{Q} = D ∑_{w∈W} sgn w · w (\frac{e^{ρ} ch L_{λ} (𝔮)}{Π_{α∈Δ_{𝔮}∩Δ_{1}^{+}} (1 + e^{−α})}) ...

Documenta Mathematica · Extra Volume ICM 1998 · II · 583–593

On the other hand, ch L_{λ} (𝔮) = ch L_{λ} (𝔤_{𝔮}), and 𝔤_{𝔮} is isomorphic to gl (m|n). Since the case 𝔤 = gl (m|n) is already covered, we can calculate ch L_{λ}.

These arguments reduce the calculation of ch L_{λ} to the calculation of the matrix (a_{μ,ν}^{P}). The next statement reduces the latter problem to the case of the most atypical central character.

Proposition 3.5. Let λ ∈ Λ^{+}, λ′ and 𝔤′ be as in theorem 2.6 and proposition 2.7. Let 𝔭 = 𝔟 if 𝔤 = gl (m|n), 𝔭 = 𝔭^{(r+1)} if 𝔤 = osp (2m|2n) or osp (2m + 1|2n) and t_{λ} = n − r. Let 𝔭′ be the analogous parabolic subalgebra in 𝔤′ determined by λ′. Then K_{λ,μ}^{P} = K_{λ′,μ′}^{P′}.

4. Calculation of coefficients a_{λ,μ} in F_{χ_{ρ}}

In this section we concentrate on calculation of coefficients a_{λ,μ}^{P}. By Proposition 3.5 it is sufficient to find these coefficients only for the most atypical block F_{χ_{ρ}} and 𝔤 = gl (k|k), osp (2k|2k), osp (2k + 2|2k) or osp (2k + 1|2k).

First, consider the case 𝔤 = gl (k|k). Here t_{λ} = 0 and P = B. Introduce a formal operator A in the Grothendieck ring of F_{χ_{ρ}} by the formula

A [L_{λ}] = ∑_{μ∈Λ} a_{λ,μ}^{P} [L_{μ}] .

Let 𝔭^{(i)} be the parabolic subalgebra such that Δ_{𝔭^{(i)}} is generated by the simple roots σ_{i}, …, σ_{2k−i}. Consider the flag of parabolic subalgebras 𝔤 = 𝔭^{(1)} ⊃ 𝔭^{(2)} ⊃ ⋯ ⊃ 𝔭^{(k)} ⊃ 𝔭^{(k+1)} = 𝔟. Note that P^{(i)}/P^{(i+1)} is isomorphic to the supermanifold of (1|1)-dimensional subspaces in ℂ^{i|i}. As before define derived functors

Γ_{j}^{(i)} : F (𝔭^{(i+1)}) → F (𝔭^{(i)}) , Γ_{j}^{(i)} (M) = [H_{P^{(i)/}P^{(i+1)}}^{j}O (M^{∗})]^{∗} ,

generating functions K^{(i)} and coefficients a^{(i)} by

K_{λ,μ}^{(i)} (q) = ∑_{j} [Γ_{j}^{(i)} (L_{λ} (𝔭^{(i+1)})) : L_{μ} (𝔭^{(i)})] q^{j}, a_{λ,μ}^{(i)} = K_{λ,μ}^{(i)} (−1) .

Define the operators A^{(i)} [L_{λ}] = ∑_{μ} a_{λ,μ}^{(i)} [L_{μ}]. Obviously,

A = A^{(1)} ∘ ⋯ ∘ A^{(k)}. (4.1)

Theorem 4.1 below gives recurrence relations for calculating polynomials K_{λ,μ}^{(i)}. It is the most difficult result in the paper. Before stating it let us recall that any λ ∈ Λ^{+} with χ_{λ} = χ_{ρ} can be written as a_{1}α_{1} +⋯+a_{k}α_{k} where α_{i} = δ_{i} −ε_{k+1−i} ∈ A_{λ} and a_{1} > a_{2} > ⋯ > a_{k}. For S (q) ∈ ℤ [q, q^{−1}] denote the polynomial part of S by S_{+}.

Theorem 4.1. Let 𝔤 = gl (k|k). Then the following recurrence relations hold:

1. K_{λ,λ}^{(i)} = 1;

2. if a_{i} > a_{i+1} + 1, then K_{λ,λ−α_{i}}^{(i)} = 1 and K_{λ,μ}^{(i)} = (q^{−1}K_{λ−α_{i},μ}^{(i)})_{+} for any

μ ≠ λ, λ − α;

3. if a_{i} = a_{i+1} + 1, then K_{λ,μ}^{(i)} = qK_{λ−α_{i},μ}^{(i+1)} for any μ ≠ λ, λ − α_{i}; 4. K_{λ,λ−α_{k}}^{(k)} = 1 and K_{λ,μ}^{(k)} = 0 for any μ ≠ λ, λ − α_{k}.

These relations uniquely determine the polynomials K_{λ,μ}^{(i)}.

Note that the proof of Theorem 4.1 unravelled the following beautiful geometric

Lemma 4.2. Let λ ∈ Λ^{+}. The cohomology Γ_{j}^{(i)} (L_{λ} (𝔭^{(i+1)})) for j > 0 and the kernel of the natural projection Γ_{0}^{(i)} (L_{λ} (𝔭^{(i+1)})) → L_{λ} (𝔭^{(i)}) are semisimple 𝔭^{(i)}- modules, and any irreducible component of ⨁_{j} Γ_{j}^{(i)} (L_{λ} (𝔭^{(i+1)})) occurs with multiplicity 1.

Consider the case 𝔤 = osp (2k + l|2k), where l = 0, 1 or 2, and χ_{λ} = χ_{μ}. Consider the flag of parabolic subalgebras 𝔤 = 𝔭^{(1)} ⊃ 𝔭^{(2)} ⊃ ⋯ ⊃ 𝔭^{(k+1)}, where Δ_{𝔭^{(i)}} is generated by the simple roots σ_{i}, …, σ_{k+[l/2]}. Note that P^{(i)}/P^{(i+1)} is isomorphic to the supermanifold of (1|0)-dimensional subspaces in ℂ^{2k−2i|2k+l}. Let r = k − t_{λ}. As it was explained in section 3, we are interested in calculating a_{λ,μ}^{P} = a_{λ,μ}^{P^{(r+1)}}. Using the same notations as for the case 𝔤 = gl (k|k) one can write A = A^{(1)} ∘ ⋯ ∘ A^{(r)}.

Next, we write recurrence relations for polynomials K_{λ,μ}^{(i)}. Write A_{λ} = {α_{1}, …, α_{k}}, where α_{i} = δ_{i} + ε_{j_{i}} for i ≤ r and α_{i} = δ_{i} − ε_{j_{i}} if i > r. We assume that b_{k} ≥ 0, see remark 2.2. By Proposition 2.7 λ can be written as λ = a_{1}α_{1} + ⋯ + a_{k}α_{k}, where a_{i} ∈ ℤ + l/2 and a_{1} > a_{2} > ⋯ > a_{r} > 0 ≥ a_{r+1} > ⋯ > a_{k}.

Theorem 4.3. Let 𝔤 = osp (2k + l|2k). Then the following recurrence relations hold:

1. K_{λ,λ}^{(i)} = 1;

2. if a_{i} > a_{i+1} + 1 and a_{i} ≠ 1 − a_{j} for any j > r, then K_{λ,λ−α_{i}}^{(i)} = 1 and

K_{λ,μ}^{(i)} = (q^{−1}K_{λ−α_{i},μ}^{(i)})_{+} for any μ ≠ λ, λ − α;

3. if a_{i} = 1 − a_{j} for some j > r, then K_{λ,λ−δ_{i}−δ_{j}}^{(i)} = 1 and K_{λ,μ}^{(i)} =

(q^{−1}K_{λ−δ_{i}−δ_{j},μ}^{(i)})_{+} for any μ ≠ λ, λ − δ_{i} − δ_{j};

4. if a_{i} = a_{i+1} + 1, then K_{λ,μ}^{(i)} = qK_{λ−δ_{i},μ+ε_{j_{i}}}^{(i+1)} for any μ ≠ λ;

5. if l = 1 and a_{r} = 1/2, then K_{λ,λ−δ_{r}}^{(r)} = q^{2s+1} and K_{λ,μ}^{(r)} = 0 for any μ ≠ λ, λ − δ_{r};

6. if l = 2 and a_{r} = 1, then K_{λ,λ−2δ_{r}}^{(r)} = q^{2s+1} and K_{λ,μ}^{(r)} = 0 for any μ ≠ λ, λ − 2δ_{r};

7. if l = 0, a_{r} = 1 and r ≠ k, then K_{λ,λ−δ_{r}−δ_{r+1}}^{(r)} = q^{2s} and K_{λ,μ}^{(r)} = 0 for any μ ≠ λ, λ − δ_{r} − δ_{r+1};

8. if l = 0, a_{k} = 1, then K_{λ,λ−δ_{k}−ε_{k}}^{(k)} = 1 and K_{λ,μ}^{(k)} = 0 for any μ ≠ λ, λ−δ_{k}−ε_{k}.

These relations uniquely determine the polynomials K_{λ,μ}^{(i)}.

Note that in the case of gl (m|n) and P = B one can calculate K_{λ,μ}^{P} basing on (4.1) and Theorem 4.1, since in this case it happens that K_{λ,μ}^{P} = a_{λ,μ}^{P}. In the cases P ≠ B or 𝔤 = osp formulae for K_{λ,μ}^{P} are not known.

References

[1] I. N. Bernshteĭn and D. A. Leĭtes, Invariant differential operators and irreducible representations of Lie superalgebras of vector fields, Serdica 7 (1981), no. 4, 320–334 (1982).

[2] J. Bernstein and D. Leites, A formula for the characters of the irreducible finite dimensional representations of lie superalgebras of series gl and sl, C. R. Acad. Bulgare Sci. 33 (1980), 1049–1051.

[3] J. Dixmier, Enveloping algebras, Graduate Studies in Mathematics, vol. 11, American Mathematical Society, Providence, RI, 1996, Revised reprint of the 1977 translation.

[4] A. Frumkin, The irreducible characters of the Lie superalgebras of type A(n, m) and filtrations of their Kac modules, Israel J. Math. 96 (1996), no. , part A, 267–279.

[5] J. W. B. Hughes, R. C. King, and J. Van der Jeugt, On the composition factors of Kac modules for the Lie superalgebras sl(m/n), J. Math. Phys. 33 (1992), no. 2, 470–491.

[6] V. G. Kac, Lie superalgebras, Adv. Math. 26 (1977), 8–96.

[7] , Representations of classical Lie superalgebras, Lecture Notes in Math. 676 (1978), 597–626.

[8] , Laplace operators of infinite–dimensional Lie algebras and theta functions, PNAS USA 81 (1984), 645–647.

[9] V. G. Kac and M. Wakimoto, Integrable highest weight modules over affine superalgebras and number theory, Progress in Math. 123 (1994), 415–456.

[10] Yu. Manin, Gauge field theory and complex geometry, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 289, Springer-Verlag, Berlin, 1988, Translated from the Russian by N. Koblitz and J. R. King.

[11] I. Penkov, Borel-Weil-Bott theory for classical lie supergroups, Contemporary Problems in Mathematics, vol. 32, VINITI, Moscow, 1988.

[12] , Generic representations of classical Lie superalgebras and their localization, Monatsh. Math. 118 (1994), no. 3-4, 267–313.

[13] I. Penkov and V. Serganova, Cohomology of G/P for classical complex Lie supergroups G and characters of some atypical G-modules, Ann. Inst. Fourier (Grenoble) 39 (1989), no. 4, 845–873.

[14] , Generic irreducible representations of finite-dimensional Lie superalgebras, Internat. J. Math. 5 (1994), no. 3, 389–419.

[15] , Characters of irreducible G-modules and cohomology of G/P for the Lie supergroup G = Q(N), J. Math. Sci. (New York) 84 (1997), no. 5, 1382– 1412, Algebraic geometry, 7.

[16] I. Penkov and I. Skornyakov, Cohomologie des D-modules tordus typiques sur les supervariétés de drapeaux, C. R. Acad. Sci. Paris Sér. I Math. 299 (1984), no. 20, 1005–1008.

[17] V. Serganova, Kazhdan-Lusztig polynomials for Lie superalgebra GL(m, n), Advances in Soviet Mathematics 16 (1993), 151–165.

[18] , Kazhdan-Lusztig polynomials and character formula for the Lie superalgebra 𝔤𝔩(m|n), Selecta Math. (N.S.) 2 (1996), no. 4, 607–651.

[19] A. Sergeev, Enveloping algebra centre for Lie superalgebras GL and Q, Ph.D. thesis, Moscow State University, Moscow, 1987.

[20] A. V. Shapovalov, Invariant differential operators and irreducible representations of finite-dimensional Hamiltonian and Poisson Lie superalgebras, Serdica 7 (1981), no. 4, 337–342 (1982).

[21] J. Van der Jeugt, Character formulae for the Lie superalgebra C(n), Comm. Algebra 19 (1991), no. 1, 199–222.

[22] J. Van der Jeugt, J. W. B. Hughes, R. C. King, and J. Thierry-Mieg, A character formula for singly atypical modules of the Lie superalgebra sl(m/n), Comm. Algebra 18 (1990), no. 10, 3453–3480.

Vera Serganova

Dept. of Mathematics

University of California at Berkeley Berkeley, CA 94720, USA serganov@@math.berkeley.edu 594

Documenta Mathematica · Extra Volume ICM 1998 · II ·