Andreotti–Mayer Loci and the Schottky Problem

Ciro Ciliberto, Gerard van der Geer

Received: July 4, 2007 Revised: May 27, 2008

Communicated by Thomas Peternell

Abstract. We prove a lower bound for the codimension of the Andreotti-Mayer locus N_{g,1} and show that the lower bound is reached only for the hyperelliptic locus in genus 4 and the Jacobian locus in genus 5. In relation with the intersection of the Andreotti-Mayer loci with the boundary of the moduli space A_{g} we study subvarieties of principally polarized abelian varieties (B, Ξ) parametrizing points b such that Ξ and the translate Ξ_{b} are tangentially degenerate along a variety of a given dimension.

2000 Mathematics Subject Classification: 14K10

Keywords and Phrases: Abelian variety, theta divisor, Andreotti-Mayer loci, Schottky problem.

1. Introduction

The Schottky problem asks for a characterization of Jacobian varieties among all principally polarized abelian varieties. In other words, it asks for a description of the Jacobian locus J_{g} in the moduli space A_{g} of all principally polarized abelian varieties of given dimension g. In the 1960’s Andreotti and Mayer (see [2]) pioneered an approach based on the fact that the Jacobian variety of a non-hyperelliptic (resp. hyperelliptic) curve of genus g ≥ 3 has a singular locus of dimension g − 4 (resp. g − 3). They introduced the loci N_{g,k} of principally polarized abelian varieties (X, Θ_{X}) of dimension g with a singular locus of Θ_{X} of dimension ≥ k and showed that J_{g} (resp. the hyperelliptic locus H_{g}) is an irreducible component of N_{g,g−4} (resp. N_{g,g−3}). However, in general there are more irreducible components of N_{g,g−4} so that the dimension of the singular locus of Θ_{X} does not suffice to characterize Jacobians or hyperelliptic Jacobians. The locus N_{g,0} of abelian varieties with a singular theta divisor has codimension 1 in A_{g} and in a beautiful paper (see [27]) Mumford calculated its class. But in general not much is known about these Andreotti-Mayer loci N_{g,k}. In particular, we do not even know their codimension. In this paper we give estimates for the codimension of these loci. These estimates are in general not sharp, but we think that the following conjecture gives the sharp bound.

Conjecture 1.1. If 1 ≤ k ≤ g − 3 and if N is an irreducible component of N_{g,k} whose general point corresponds to an abelian variety with endomorphism ring ℤ then codim_{A_{g}}(N) ≥ \binom{k+2}{2}. Moreover, equality holds if and only if one of the following happens:

(i) g = k + 3 and N = H_{g};

(ii) g = k + 4 and N = J_{g}.

We give some evidence for this conjecture by proving the case k = 1. In our approach we need to study the behaviour of the Andreotti-Mayer loci at the boundary of the compactified moduli space. A principally polarized (g − 1)- dimensional abelian variety (B, Ξ) parametrizes semi-abelian varieties that are extensions of B by the multiplicative group 𝔾_{m}. This means that B maps to a part of the boundary of the compactified moduli space Ã_{g} and we can intersect B with the Andreotti-Mayer loci. This motivates the definition of loci N_{k}(B, Ξ) ⊂ B for a principally polarized (g − 1)-dimensional abelian variety (B, Ξ). They are formed by the points b in B such that Ξ and its translate Ξ_{b} are ‘tangentially degenerate’ (see Section 11 below) along a subvariety of dimension k. These intrinsically defined subvarieties of an abelian variety are interesting in their own right and deserve further study. The conjecture above then leads to a boundary version that gives a new conjectural answer to the Schottky problem for simple abelian varieties.

Conjecture 1.2. Let k ∈ ℤ_{≥1}. Suppose that (B, Ξ) is a simple principally polarized abelian variety of dimension g not contained in N_{g,i} for all i ≥ k. Then there is an irreducible component Z of N_{k}(B, Ξ) with codim_{B}(Z) = k + 1 if and only if one of the following happens:

(i) either g ≥ 2, k = g − 2 and B is a hyperelliptic Jacobian, (ii) or g ≥ 3, k = g − 3 and B is a Jacobian.

In our approach we will use a special compactification Ã_{g} of A_{g} (see [29, 28, 5]). The points of the boundary ∂Ã_{g} = Ã_{g} − A_{g} correspond to suitable compactifications of g-dimensional semi-abelian varieties. We prove Conjecture 1.1 for k = 1 by intersecting with the boundary. For higher values of k the intersection with the boundary looks very complicated.

2. The universal theta divisor

Let π : X_{g} → A_{g} be the universal principally polarized abelian variety of relative dimension g over the moduli space A_{g} of principally polarized abelian varieties of dimension g over ℂ. In this paper we will work with orbifolds and we shall identify X_{g} (resp. A_{g}) with the orbifold Sp(2g, ℤ)⋉ℤ^{2g}\ℍ_{g} ×ℂ^{g} (resp. with Sp(2g, ℤ)\ℍ_{g}), where

ℍ_{g} = {(τ_{ij}) ∈ Mat(g × g, ℂ) : τ = τ^{t}, Im(τ) > 0} is the usual Siegel upper-half space of degree g. The τ_{ij} with 1 ≤ i ≤ j ≤ g are coordinates on ℍ_{g} and we let z_{1}, ..., z_{g} be coordinates on ℂ^{g}. The Riemann theta function ϑ(τ, z), given on ℍ_{g} × ℂ^{g} by

ϑ(τ, z) = ∑_{m∈ℤ^{g}} e^{πi[m^{t}τm+2m^{t}z]},

is a holomorphic function and its zero locus is an effective divisor Θ̃ on ℍ_{g} ×ℂ^{g} which descends to a divisor Θ on X_{g}. If the abelian variety X is a fibre of π, then we let Θ_{X} be the restriction of Θ to X. Note that since θ(τ, z) satisfies θ(τ, −z) = θ(τ, z), the divisor Θ_{X} is symmetric, i.e., ι^{∗}(Θ_{X}) = Θ_{X}, where ι = −1_{X} : X → X is multiplication by −1 on X. The divisor Θ_{X} defines the line bundle O_{X}(Θ_{X}), which yields the principal polarization on X. The isomorphism class of the pair (X, Θ_{X}) represents a point ζ of A_{g} and we will write ζ = (X, Θ_{X}). Similarly, it will be convenient to identify a point ξ of Θ with the isomorphism class of a representative triple (X, Θ_{X}, x), where ζ = (X, Θ_{X}) represents π(ξ) ∈ A_{g} and x ∈ Θ_{X}. The tangent space to X_{g} at a point ξ, with π(ξ) = ζ, will be identified with the tangent space T_{X,x}⊕T_{A_{g},ζ} ≅ T_{X,0}⊕Sym^{2}(T_{X,0}). If ξ = (X, Θ_{X}, x) corresponds to the Sp(2g, ℤ) ⋉ ℤ^{2g}-orbit of a point (τ_{0}, z_{0}) ∈ ℍ_{g} × ℂ^{g}, then the tangent space T_{X_{g},ξ} to X_{g} at ξ can be identified with the tangent space to ℍ_{g} × ℂ^{g} at (τ_{0}, z_{0}), which in turn is naturally isomorphic to ℂ^{g(g+1)/2+g}, with coordinates (a_{ij}, b_{ℓ}) for 1 ≤ i, j ≤ g and 1 ≤ ℓ ≤ g that satisfy a_{ij} = a_{ji}. We thus view the a_{ij}’s as coordinates on the tangent space to ℍ_{g} at τ_{0} and the b_{ℓ}’s as coordinates on the tangent space to X or its universal cover. An important remark is that by identifying the tangent space to A_{g} at ζ = (X, Θ_{X}) with Sym^{2}(T_{X,0}), we can view the projectivized tangent space ℙ(T_{A_{g},ζ}) ≅ ℙ(Sym^{2}(T_{X,0})) as the linear system of all quadrics in the dual of ℙ^{g−1} = ℙ(T_{X,0}). In particular, the matrix (a_{ij}) can be interpreted as the matrix defining a dual quadric in the space ℙ^{g−1} with homogeneous coordinates (b_{1} : …: b_{g}). Quite naturally, we will often use (z_{1} : …: z_{g}) for the homogeneous coordinates in ℙ^{g−1}. Recall that the Riemann theta function ϑ satisfies the heat equations

\frac{∂}{∂z_{i}} \frac{∂}{∂z_{j}} ϑ = 2π\sqrt{−1}(1 + δ_{ij})\frac{∂}{∂τ_{ij}} ϑ

for 1 ≤ i, j ≤ g, where δ_{ij} is the Kronecker delta. We shall abbreviate this equation as

∂_{i}∂_{j}ϑ = 2π\sqrt{−1}(1 + δ_{ij})∂_{τ_{ij}}ϑ,

where ∂_{j} means the partial derivative ∂/∂z_{j} and ∂_{τ_{ij}} the partial derivative ∂/∂τ_{ij}. One easily checks that also all derivatives of θ verify the heat equations. We refer to [39] for an algebraic interpretation of the heat equations in terms of deformation theory. If ξ = (X, Θ_{X}, x) ∈ Θ corresponds to the Sp(2g, ℤ) ⋉ ℤ^{2g}-orbit of a point (τ_{0}, z_{0}), then the Zariski tangent space T_{Θ,ξ} to Θ at ξ is the subspace of T_{X_{g},ξ} ≃ ℂ^{g(g+1)/2+g} defined, with the above conventions, by the linear equation

(1) ∑_{1≤i≤j≤g} \frac{1}{2π\sqrt{−1}(1 + δ_{ij})}a_{ij} ∂_{i}∂_{j}ϑ(τ_{0}, z_{0}) + ∑_{1≤ℓ≤g} b_{ℓ} ∂_{ℓ}ϑ(τ_{0}, z_{0}) = 0

in the variables (a_{ij}, b_{ℓ}), 1 ≤ i, j ≤ g, 1 ≤ ℓ ≤ g. As an immediate consequence we get the result (see [36], Lemma (1.2)):

Lemma 2.1. The point ξ = (X, Θ_{X}, x) is a singular point of Θ if and only if x is a point of multiplicity at least 3 for Θ_{X}.

3. The locus S_{g}

We begin by defining a suborbifold of Θ supported on the set of points where π_{|Θ} fails to be of maximal rank.

Definition 3.1. The closed suborbifold S_{g} of Θ is defined on the universal cover ℍ_{g} × ℂ^{g} by the g + 1 equations

(2)  ϑ(τ, z) = 0, ∂_{j}ϑ(τ, z) = 0, j = 1, …, g.

Lemma 2.1 implies that the support of S_{g} is the union of Sing(Θ) and of the set of smooth points of Θ where π_{|Θ} fails to be of maximal rank. Set-theoretically one has

S_{g} = {(X, Θ_{X}, x) ∈ Θ : x ∈ Sing(Θ_{X})}

and codim_{X_{g}}(S_{g}) ≤ g + 1. It turns out that every irreducible component of S_{g} has codimension g + 1 in X_{g} (see [8] and an unpublished preprint by Debarre [9]). We will come back to this later in §7 and §8. With the above identification, the Zariski tangent space to S_{g} at a given point (X, Θ_{X}, x) of X_{g}, corresponding to the Sp(2g, ℤ)-orbit of a point (τ_{0}, z_{0}) ∈ ℍ_{g} × ℂ^{g}, is given by the g + 1 equations

(3) \begin{aligned}∑_{1≤i≤j≤g} a_{ij}∂_{τ_{ij}}ϑ(τ_{0}, z_{0}) & = 0, \\ ∑_{1≤i≤j≤g} a_{ij}∂_{τ_{ij}}∂_{k}ϑ(τ_{0}, z_{0}) + ∑_{1≤ℓ≤g} b_{ℓ}∂_{ℓ}∂_{k}ϑ(τ_{0}, z_{0}) & = 0,  1 ≤ k ≤ g \end{aligned}

in the variables (a_{ij}, b_{ℓ}) with 1 ≤ i, j, ℓ ≤ g. We will use the following notation:

(a) q is the row vector of length g(g + 1)/2, given by (∂_{τ_{ij}}θ(τ_{0}, z_{0})), with lexicographically ordered entries;

(b) q_{k} is the row vector of length g(g + 1)/2, given by (∂_{τ_{ij}}∂_{k}θ(τ_{0}, z_{0})), with lexicographically ordered entries;

(c) M is the g × g-matrix (∂_{i}∂_{j}ϑ(τ_{0}, z_{0}))_{1≤i,j≤g}. Then we can rewrite the equations (3) as

(4)  a · q^{t} = 0, a · q_{k}^{t} + b · M_{k}^{t} = 0, (k = 1, …, g),

where a is the vector (a_{ij}) of length g(g + 1)/2, with lexicographically ordered entries, b is a vector in ℂ^{g} and M_{j} the j-th row of the matrix M.

In this setting, the equation (1) for the tangent space to T_{Θ,ξ} can be written as:

(5)  a · q^{t} + b · ∂ϑ(τ_{0}, z_{0})^{t} = 0

where ∂ denotes the gradient. Suppose now the point ξ = (X, Θ_{X}, x) in S_{g}, corresponding to (τ_{0}, z_{0}) ∈ ℍ_{g}×ℂ^{g} is not a point of Sing(Θ). By Lemma 2.1 the matrix M is not zero and therefore we can associate to ξ a quadric Q_{ξ} in the projective space ℙ(T_{X,x}) ≃ ℙ(T_{X,0}) ≃ ℙ^{g−1}, namely the one defined by the equation

b · M · b^{t} = 0.

Recall that b = (b_{1}, …, b_{g}) is a coordinate vector on T_{X,0} and therefore (b_{1} : …: b_{g}) are homogeneous coordinates on ℙ(T_{X,0}). We will say that Q_{ξ} is indeterminate if ξ ∈ Sing(Θ). The vector q naturally lives in Sym^{2}(T_{X,0})^{∨} and therefore, if q is not zero, the point [q] ∈ ℙ(Sym^{2}(T_{X,0})^{∨}) determines a quadric in ℙ^{g−1} = ℙ(T_{X,0}). The heat equations imply that this quadric coincides with Q_{ξ}. Consider the matrix defining the Zariski tangent space to S_{g} at a point ξ = (X, Θ_{X}, x). We denote by r := r_{ξ} the corank of the quadric Q_{ξ}, with the convention that r_{ξ} = g if ξ ∈ Sing(Θ), i.e., if Q_{ξ} is indeterminate. If we choose coordinates on ℂ^{g} such that the first r basis vectors generate the kernel of q then the shape of the matrix A of the system (3) is

(6)  A = ( q  0_{g} ; q_{1}  0_{g} ; ⋮ ; q_{r}  0_{g} ; ∗  B) ,

where q and q_{k} are as above and B is a (g − r) × g-matrix with the first r columns equal to zero and the remaining (g − r) × (g − r) matrix symmetric of maximal rank. Next, we characterize the smooth points ξ = (X, Θ_{X}, x) of S_{g}. Before stating the result, we need one more piece of notation. Given a non-zero vector b = (b_{1}, …, b_{g}) ∈ T_{X,0}, we set ∂_{b} = ∑_{ℓ=1}^{g} b_{ℓ}∂_{ℓ}. Define the matrix ∂_{b}M as the g × g-matrix (∂_{i}∂_{j}∂_{b}ϑ(τ_{0}, z_{0}))_{1≤i,j≤g}. Then define the quadric ∂_{b}Q_{ξ} = Q_{ξ,b} of ℙ(T_{X,0}) by the equation

z · ∂_{b}M · z^{t} = 0.

If z = e_{i} is the i-th vector of the standard basis, one writes ∂_{i}Q_{ξ} = Q_{ξ,i} instead of Q_{ξ,e_{i}} for i = 1, …, g. We will use similar notation for higher order derivatives or even for differential operators applied to a quadric.

Definition 3.2. We let Q_{ξ} be the linear system of quadrics in ℙ(T_{X,0}) spanned by Q_{ξ} and by all quadrics Q_{ξ,b} with b ∈ ker(Q_{ξ}).

Since Q_{ξ} has corank r, the system Q_{ξ} is spanned by r+1 elements and therefore dim(Q_{ξ}) ≤ r. This system may happen to be empty, but then Q_{ξ} is indeterminate, i.e., ξ lies in Sing(Θ). Sometimes we will use the lower suffix x instead of ξ to denote quadrics and linear systems, e.g. we will sometimes write Q_{x} instead of Q_{ξ}, etc. By the heat equations, the linear system Q_{ξ} is the image of the vector subspace of Sym^{2}(T_{X,0})^{∨} spanned by the vectors q, q_{1}, …, q_{r}.

Proposition 3.3. The subscheme S_{g} is smooth of codimension g + 1 in X_{g} at the point ξ = (X, Θ_{X}, x) of S_{g} if and only if the following conditions hold:

(i) ξ ∉ Sing(Θ), i.e., Q_{ξ} is not indeterminate and of corank r < g; (ii) the linear system Q_{ξ} has maximal dimension r; in particular, if b_{1}, …, b_{r} span the kernel of Q_{ξ}, then the r + 1 quadrics Q_{ξ}, Q_{ξ,b_{1}}, …, Q_{ξ,b_{r}} are linearly independent.

Proof. The subscheme S_{g} is smooth of codimension g + 1 in X_{g} at ξ if and only if the matrix A appearing in (6) has maximal rank g + 1. Since the submatrix B of A has rank g − r, the assertion follows. □

Corollary 3.4. If Q_{ξ} is a smooth quadric, then S_{g} is smooth at ξ = (X, Θ_{X}, x).

4. Quadrics and Cornormal Spaces

Next we study the differential of the restriction to S_{g} of the map π : X_{g} → A_{g} at a point ξ = (X, Θ_{X}, x) ∈ S_{g}. We are interested in the kernel and the image of dπ_{|S_{g},ξ}. We can view these spaces in terms of the geometry of ℙ^{g−1} = ℙ(T_{X,0}) as follows:

Π_{ξ} = ℙ(ker(dπ_{|S_{g},ξ})) ⊆ ℙ(T_{X,0})

is a linear subspace of ℙ(T_{X,0}) and

Σ_{ξ} = ℙ(Im(dπ_{|S_{g},ξ})^{⊥}) ⊆ ℙ(Sym^{2}(T_{X,0})^{∨})

is a linear system of quadrics in ℙ(T_{X,0}). The following proposition is the key to our approach; we use it to view the quadrics as elements of the conormal space to our loci in the moduli space.

Proposition 4.1. Let ξ = (X, Θ_{X}, x) be a point of S_{g}. Then:

(i) Π_{ξ} is the vertex of the quadric Q_{ξ}. In particular, if ξ is a singular point of Θ, then Π_{ξ} is the whole space ℙ(T_{X,0});

(ii) Σ_{ξ} contains the linear system Q_{ξ}.

Proof. The assertions follow from the shape of the matrix A in (6). □

This proposition tells us that, given a point ξ = (X, Θ_{X}, x) ∈ S_{g}, the map dπ_{|S_{g},ξ} is not injective if and only if the quadric Q_{ξ} is singular. The orbifold S_{g} is stratified by the corank of the matrix (∂_{i}∂_{j}θ).

Definition 4.2. For 0 ≤ k ≤ g we define S_{g,k} as the closed suborbifold of S_{g} defined by the equations on ℍ_{g} × ℂ^{g}

(7)  \begin{matrix}ϑ(τ, z) = 0, ∂_{j}ϑ(τ, z) = 0, (j = 1, …, g), \\ rk((∂_{i}∂_{j}ϑ(τ, z))_{1≤i,j≤g}) ≤ g − k. \end{matrix}

Geometrically this means that ξ ∈ S_{g,k} if and only if dim(Π_{ξ}) ≥ k − 1 or equivalently Q_{ξ} has corank at least k. We have the inclusions

S_{g} = S_{g,0} ⊇ S_{g,1} ⊇ … ⊇ S_{g,g} = S_{g} ∩ Sing(Θ)

and S_{g,1} is the locus where the map dπ_{|S_{g},ξ} is not injective. The loci S_{g,k} have been considered also in [16]. We have the following dimension estimate for the S_{g,k}.

Proposition 4.3. Let 1 ≤ k ≤ g − 1 and let Z be an irreducible component of S_{g,k} not contained in S_{g,k+1}. Then we have

codim_{S_{g}}(Z) ≤ \binom{k + 1}{2}.

Proof. Locally, in a neighborhood U in S_{g} of a point z of Z\S_{g,k+1} we have a morphism f : U → Q, where Q is the linear system of all quadrics in ℙ^{g−1}. The map f sends ξ = (X, Θ_{X}, x) ∈ U to Q_{ξ}. The scheme S_{g,k} is the pullback of the subscheme Q_{k} of Q formed by all quadrics of corank k. Since codim_{Q}(Q_{k}) = \binom{k+1}{2}, the assertion follows. □

Using the equations (7) it is possible to make a local analysis of the schemes S_{g,k}, e.g. it is possible to write down equations for their Zariski tangent spaces (see §6 for the case k = g). This is however not particularly illuminating, and we will not dwell on this here. It is useful to give an interpretation of the points ξ = (X, Θ_{X}, x) ∈ S_{g,k} in terms of singularities of the theta divisor Θ_{X}. Suppose that ξ is such that Sing(Θ_{X}) contains a subscheme isomorphic to Spec(ℂ[ϵ]/(ϵ^{2})) supported at x. This subscheme of X is given by a homomorphism

O_{X,x} → ℂ[ϵ]/(ϵ^{2}), f ↦ f(x) + Δ^{(1)}f(x) · ϵ,

where Δ^{(1)} is a non-zero differential operator of order ≤ 1, hence Δ^{(1)} = ∂_{b}, for some non-zero vector b ∈ ℂ^{g}. Then the condition Spec(ℂ[ϵ]/(ϵ^{2})) ⊂ Sing(Θ_{X}) is equivalent to saying that ϑ and ∂_{b}ϑ satisfy the equations

(8)  f(τ_{0}, z_{0}) = 0, ∂_{j}f(τ_{0}, z_{0}) = 0, 1 ≤ j ≤ g,

and this, in turn, is equivalent to the fact that the quadric Q_{ξ} is singular at the point [b]. More generally, we have the following proposition, which explains the nature of the points in S_{g,k} for k < g.

Proposition 4.4. Suppose that x ∈ Sing(Θ_{X}) does not lie on Sing(Θ). Then Sing(Θ_{X}) contains a scheme isomorphic to Spec(ℂ[ϵ_{1}, …, ϵ_{k}]/(ϵ_{i}ϵ_{j}: 1 ≤ i, j ≤ k < g)) supported at x if and only if the quadric Q_{ξ} has corank r ≥ k. Moreover, the Zariski tangent space to Sing(Θ_{X}) at x is the kernel space of Q_{ξ}.

Proof. With a suitable choice of coordinates in X, the condition that the scheme Spec(ℂ[ϵ_{1}, …, ϵ_{k}]/(ϵ_{i}ϵ_{j}: 1 ≤ i, j ≤ k < g)) is contained in Sing(Θ_{X}) is equivalent to the fact that the functions ϑ and ∂_{i}ϑ for i = 1, …, k satisfy (8). But this the same as saying that ∂_{i}∂_{j}ϑ(τ_{0}, z_{0}) is zero for i = 1, …, k, j = 1, …, g, and the vectors e_{i}, i = 1, …, k, belong to the kernel of Q_{ξ}. This settles the first assertion. The scheme Sing(Θ_{X}) is defined by the equations (2), where τ is now fixed and z is the variable. By differentiating, and using the same notation as above, we see that the equations for the Zariski tangent space to Sing(Θ_{X}) at x are ∑_{i=1}^{g} b_{i}∂_{i}∂_{j}ϑ(τ_{0}, z_{0}), j = 1, …, g i.e., b · M = 0, which proves the second assertion. □

5. Curvi-linear subschemes in the singular locus of theta

A 0-dimensional curvi-linear subscheme Spec(ℂ[t]/(t^{N+1})) ⊂ X of length N +1 supported at x is given by a homomorphism

(9)  δ : O_{X,x} → ℂ[t]/(t^{N+1}),  f ↦ ∑_{j=0}^{N} Δ^{(j)}f(x) · t^{j},

with Δ^{(j)} a differential operator of order ≤ j, j = 1, …, N, with Δ^{(N)} nonzero, and Δ^{(0)}(f) = f(x). The condition that the map δ is a homomorphism is equivalent to saying that

(10)  Δ^{(k)}(fg) = ∑_{r=0}^{k} Δ^{(r)} f · Δ^{(k−r)} g, k = 0, …, N

for any pair (f, g) of elements of O_{X,x}. Two such homomorphisms δ and δ′ define the same subscheme if and only if they differ by composition with a automorphism of ℂ[t]/(t^{N+1}).

Lemma 5.1. The map δ defined in (9) is a homomorphism if and only if there exist translation invariant vector fields D_{1}, …, D_{N} on X such that for every k = 1, …, N one has

(11)  Δ^{(k)} = ∑_{h_{1}+2h_{2}+...+kh_{k}=k>0} \frac{1}{h_{1}! ⋯ h_{k}!}D_{1}^{h_{1}} ⋯ D_{k}^{h_{k}}.

Moreover, two N-tuples of vector fields (D_{1}, …, D_{N}) and (D′_{1}, …, D′_{N}) determine the same 0-dimensional curvi-linear subscheme of X of length N + 1 supported at a given point x ∈ X if and only if there are constants c_{1}, …, c_{N}, with c_{1} ≠ 0, such that

D′_{i} = ∑_{j=1}^{i} c_{j}^{i−j+1}D_{j}, i = 1, …, N.

Proof. If the differential operators Δ^{(k)}, k = 1, …, N, are as in (11), one computes that (10) holds, hence δ is a homomorphism.

As for the converse, the assertion trivially holds for k = 1. So we proceed by induction on k. Write Δ^{(k)} = ∑_{i=1}^{k} D_{i}^{(k)}, where D_{i}^{(k)} is the homogeneous part of degree i, and write D_{k} instead of D_{1}^{(k)}. Using (10) one verifies that for every k = 1, …, N and every positive i ≤ k one has

iD_{i}^{(k)} = ∑_{j=1}^{k−i+1} D_{j}D_{i−1}^{(k−j)}.

Formula (11) follows by induction and easy combinatorics. To prove the final assertion, use the fact that an automorphism of ℂ[t]/(t^{N+1}) is determined by the image c_{1}t + c_{2}t^{2} + … + c_{N}t^{N} of t, where c_{1} ≠ 0. □

In formula (11) one has h_{k} ≤ 1. If Δ^{(1)} = D_{1} then Δ^{(2)} = \frac{1}{2}D_{1}^{2} + D_{2}, Δ^{(3)} = (1/3!)D_{1}^{3} + (1/2)D_{1}D_{2} + D_{3} etc. Each non-zero summand in (11) is of the form (1/h_{i_{1}}! ⋯ h_{i_{ℓ}}!)D_{i_{1}}^{h_{i_{1}}} ⋯ D_{i_{ℓ}}^{h_{i_{ℓ}}}, where 1 ≤ i_{1} < … < i_{ℓ} ≤ k, i_{1}h_{i_{1}} + … + i_{ℓ}h_{i_{ℓ}} = k and h_{i_{1}}, …, h_{i_{ℓ}} are positive integers. Thus formula (11) can be written as

(12)  Δ^{(k)} = ∑_{\{h_{i_{1}} ,...,h_{i_{ℓ}} \}} \frac{1}{h_{i_{1}}! ⋯ h_{i_{ℓ}}!}D_{i_{1}}^{h_{i_{1}}} ⋯ D_{i_{ℓ}}^{h_{i_{ℓ}}} ,

where the subscript {h_{i_{1}}, …, h_{i_{ℓ}}} means that the sum is taken over all ℓ-tuples of positive integers (h_{i_{1}}, …, h_{i_{ℓ}}) with 1 ≤ i_{1} < ⋯ < i_{ℓ} ≤ k and i_{1}h_{i_{1}} + ⋯ + i_{ℓ}h_{i_{ℓ}} = k.

Remark 5.2. Let x ∈ X correspond to the pair (τ_{0}, z_{0}). The differential operators Δ^{(k)}, k = 1, …, N, defined as in (11) or (12) have the following property: if f is a regular function such that Δ^{(i)} f satisfies (8) for all i = 0, …, k − 1, then one has Δ^{(k)}f(τ_{0}, z_{0}) = 0.

We want now to express the conditions in order that a 0-dimensional curvilinear subscheme of X of length N + 1 supported at a given point x ∈ X corresponding to the pair (τ_{0}, z_{0}) and determined by a given N-tuple of vector fields (D_{1}, …, D_{N}) lies in Sing(Θ_{X}). To do so, we keep the notation we introduced above. Let us write D_{i} = ∑_{ℓ=1}^{g} η_{iℓ}∂_{ℓ}, so that D_{i} corresponds to the vector η_{i} = (η_{i1}, …, η_{ig}). As before we denote by M the matrix (∂_{i}∂_{j}θ(τ_{0}, z_{0})).

Proposition 5.3. The 0-dimensional curvi-linear subscheme R of X of length N + 1, supported at the point x ∈ X corresponding to the pair (τ_{0}, z_{0}) and determined by the N-tuple of vector fields (D_{1}, …, D_{N}) lies in Sing(Θ_{X}) if and only if x ∈ Sing(Θ_{X}) and moreover for each k = 1, …, N one has

(13)  ∑_{\{h_{i_{1}} ,...,h_{i_{ℓ}} \}} \frac{1}{h_{i_{1}}! ⋯ h_{i_{ℓ}}!}η_{i_{ℓ}} · ∂_{η_{i_{1}}}^{h_{i_{1}}} ⋯ ∂_{η_{i_{ℓ}}}^{h_{i_{ℓ}} −1} M = 0,

where the sum is taken over all ℓ-tuples of positive integers (h_{i_{1}}, …, h_{i_{ℓ}}) with 1 ≤ i_{1} < ⋯ < i_{ℓ} ≤ k and i_{1}h_{i_{1}} + ⋯ + i_{ℓ}h_{i_{ℓ}} = k.

Proof. The scheme R is contained in Sing(Θ_{X}) if and only if one has

Δ^{(k)}θ(τ_{0}, z_{0}) = 0, ∂_{j}Δ^{(k)}θ(τ_{0}, z_{0}) = 0 k = 0, …, N, j = 1, …, g.

By Remark 5.2 this is equivalent to

θ(τ_{0}, z_{0}) = 0, ∂_{j}Δ^{(k)}θ(τ_{0}, z_{0}) = 0 k = 0, …, N, j = 1, …, g.

The assertion follows by the expression (12) of the operators Δ^{(k)}. □

For instance, consider the scheme R_{1}, supported at x ∈ Sing(Θ_{X}), corresponding to the vector field D_{1}. Then R_{1} is contained in Sing(Θ_{X}) if and only if

(14)  η_{1} · M = 0.

This agrees with Proposition 4.4. If R_{2} is the scheme supported at x and corresponding to the pair of vector fields (D_{1}, D_{2}), then R_{2} is contained in Sing(Θ_{X}) if and only if, besides (14) one has also

(15)  (1/2)η_{1} · ∂_{η_{1}}M + η_{2} · M = 0.

Next, consider the scheme R_{3} supported at x and corresponding to the triple of vector fields (D_{1}, D_{2}, D_{3}). Then R_{3} is contained in Sing(Θ_{X}) if and only if, besides (14) and (15) one has also

(16)  (1/3!)η_{1} · ∂_{η_{1}}^{2}M + (1/2)η_{2} · ∂_{η_{1}}M + η_{3} · M = 0

and so on. Observe that (13) can be written in more than one way. For example η_{2} · ∂_{η_{1}}M = η_{1} · ∂_{η_{2}}M so that (16) could also be written as

(1/3!)η_{1} · ∂_{η_{1}}^{2}M + (1/2)η_{1} · ∂_{η_{2}}M + η_{3} · M = 0.

So far we have been working in a fixed abelian variety X. One can remove this restriction by working on S_{g} and by letting the vector fields D_{1}, …, D_{N} vary with X, which means that we let the vectors η_{i} depend on the variables τ_{ij}. Then the equations (13) define a subscheme S_{g}(D) of Sing(Θ) which, as a set, is the locus of all points ξ = (X, Θ_{X}, x) ∈ S_{g} such that Sing(Θ_{X}) contains a curvi-linear scheme of length N + 1 supported at x, corresponding to the N-tuple of vector fields D = (D_{1}, …, D_{N}), computed on X. One can compute the Zariski tangent space to S_{g}(D) at a point ξ = (X, Θ_{X}, x) in the same way, and with the same notation, as in §3. This gives in general a complicated set of equations. However we indicate one case in which one can draw substantial information from such a computation. Consider indeed the case in which D_{1} = … = D_{N} ≠ 0, and call b the corresponding tangent vector to X at the origin, depending on the the variables τ_{ij}. In this case we use the notation D_{b,N} = (D_{1}, …, D_{N}) and we denote by R_{x,b,N} the corresponding curvi-linear scheme supported at x. For a given such D = (D_{1}, …, D_{N}), consider the linear system of quadrics

Σ_{ξ}(D) = ℙ(Im(dπ_{|S_{g}(D),ξ})^{⊥})

in ℙ(T_{X,0}). One has again an interpretation of these quadrics in terms of the normal space:

Proposition 5.4. In the above setting, the space Σ_{ξ}(D_{b,N}) contains the quadrics Q_{ξ}, ∂_{b}Q_{ξ}, …, ∂_{b}^{N}Q_{ξ}.

Proof. The equations (13) take now the form

\begin{matrix}θ(τ, z) = 0, ∂_{i}θ(τ, z) = 0, i = 1, …, g \\ b · M = b · ∂_{b}M = ⋯ = b · ∂_{b}^{N−1}M = 0. \end{matrix}

By differentiating the assertion immediately follows. □

6. Higher multiplicity points of the theta divisor

We now study the case of higher order singularities on the theta divisor. For a multi-index I = (i_{1}, …, i_{g}) with i_{1}, …, i_{g} non-negative integers we set z^{I} = z_{1}^{i_{1}} ⋯ z_{g}^{i_{g}} and denote by ∂_{I} the operator ∂_{1}^{i_{1}} ⋯ ∂_{g}^{i_{g}}. Moreover, we let |I| = ∑_{ℓ=1}^{g} i_{ℓ}, which is the length of I and equals the order of the operator ∂_{I}.

Definition 6.1. For a positive integer r we let S_{g}^{(r)} be the subscheme of X_{g} which is defined on ℍ_{g} × ℂ^{g} by the equations

(17)  ∂_{I}ϑ(τ, z) = 0,  |I| = 0, …, r − 1.

One has the chain of subschemes

… ⊆ S_{g}^{(r)} ⊆ … ⊆ S_{g}^{(3)} ⊆ S_{g}^{(2)} = S_{g} ⊂ S_{g}^{(1)} = Θ

and as a set S_{g}^{(r)} = {(X, Θ_{X}, x) ∈ Θ : x has multiplicity ≥ r for Θ_{X}}. One denotes by Sing^{(r)}(Θ_{X}) the subscheme of Sing(Θ_{X}) formed by all points of multiplicity at least r. One knows that S_{g}^{(r)} = ∅ as soon as r > g (see [37]). We can compute the Zariski tangent space to S_{g}^{(r)} at a point ξ = (X, Θ_{X}, x) in the same vein, and with the same notation, as in §3. Taking into account that θ and all its derivatives verify the heat equations, we find the equations by replacing in (3) the term θ(τ_{0}, z_{0}) by ∂_{I}θ(τ_{0}, z_{0}). As in §3, we wish to give some geometrical interpretation. For instance, we have the following lemma which partially extends Lemma 2.1 or 3.3.

Lemma 6.2. For every positive integer r the scheme S_{g}^{(r+2)} is contained in the singular locus of S_{g}^{(r)}.

Next we are interested in the differential of the restriction of the map π : X_{g} → A_{g} to S_{g}^{(r)} at a point ξ = (X, Θ_{X}, x) which does not belong to S_{g}^{(r+1)}. This means that Θ_{X} has a point of multiplicity exactly r at x. If we assume, as we may, that x is the origin of X , i.e. z_{0} = 0, then the Taylor expansion of θ has the form

ϑ = ∑_{i=r}^{∞} ϑ_{i},

where ϑ_{i} is a homogeneous polynomial of degree i in the variables z_{1}, …, z_{g} and

θ_{r} = ∑_{I=(i_{1},...,i_{g}),|I|=r} \frac{1}{i_{1}! ⋯ i_{g}!}∂_{I}θ(τ_{0}, z_{0})z^{I}

is not identically zero. The equation θ_{r} = 0 defines a hypersurface T C_{ξ} of degree r in ℙ^{g−1} = ℙ(T_{X,0}), which is the tangent cone to Θ_{X} at x. We will denote by Vert(T C_{ξ}) the vertex of T C_{ξ}, i.e., the subspace of ℙ^{g−1} which is the locus of points of multiplicity r of T C_{ξ}. Note that it may be empty. In case r = 2, the tangent cone T C_{ξ} is the quadric Q_{ξ} introduced in §3 and Vert(T C_{ξ}) is its vertex Π_{ξ}. More generally, for every s ≥ r, one can define the subscheme T C_{ξ}^{(s)} = T C_{x}^{(s)} of ℙ^{g−1} = ℙ(T_{X,0}) defined by the equations

θ_{r} = … = θ_{s} = 0,

which is called the asymptotic cone of order s to Θ_{X} at x. Fix a multi-index J = (j_{1}, …, j_{g}) of length r − 2. For any pair (h, k) with 1 ≤ h, k ≤ g, let J_{(h,k)} be the multi-index of length r obtained from J by first increasing by 1 the index j_{h} and then by 1 the index j_{k} (that is, by 2 if they coincide). Consider then the quadric Q_{ξ}^{J} in ℙ^{g−1} = ℙ(T_{X,0}) defined by the equation

q_{ξ}^{J}(z) := ∑_{1≤h,k≤g} ∂_{J_{(h,k)}}θ(τ_{0}, z_{0})z_{h}z_{k} = 0

with the usual convention that the quadric is indeterminate if the left-hand-side is identically zero. This is a polar quadric of T C_{ξ}, namely it is obtained from T C_{ξ} by iterated operations of polarization. Moreover all polar quadrics are in the span ⟨Q_{ξ}^{J}, |J| = r − 2⟩. We will denote by Q_{ξ}^{(r)} the span of all quadrics Q_{ξ}^{J}, |J| = r − 2 and ∂_{b}Q_{ξ}^{J}, |J| = r − 2, with equation

∑_{1≤i,j≤g} ∂_{b}∂_{J_{(h,k)}}θ(τ_{0}, z_{0})z_{i}z_{j} = 0,

for every non-zero vector b ∈ ℂ^{g} such that [b] ∈ Vert(T C_{ξ}). We are now interested in the kernel and the image of dπ_{|S_{g}^{(r)},ξ}. Equivalently we

may consider the linear system of quadrics Σ_{ξ}^{(r)} = ℙ(Im(dπ_{|S_{g}^{(r)},ξ})^{⊥}), and the

subspace Π_{ξ}^{(r)} = ℙ(ker(dπ_{|S_{g}^{(r)},ξ})) of ℙ(T_{X,0}). The following proposition partly extends Proposition 4.1 and 4.4 and its proof is similar.

Proposition 6.3. Let ξ = (X, Θ_{X}, x) be a point of S_{g}^{(r)}. Then:

(i) Π_{ξ}^{(r)} = Vert(T C_{ξ}). In particular, if ξ ∈ S_{g}^{(r+1)}, then Π_{ξ}^{(r)} is the whole

space ℙ(T_{X,0});
(ii) Σ_{ξ}^{(r)} contains the linear system Q_{ξ}^{(r)}.

Remark 6.4. As a consequence, just like in Proposition 4.4, one sees that for ξ = (X, Θ_{X}, x) the Zariski tangent space to Sing^{(r)}(Θ_{X}) at x is contained in Vert(T C_{ξ}).

As an application, we have:

Proposition 6.5. Let ξ = (X, Θ_{X}, x) be a point of a component Z of S_{g}^{(3)} such that T C_{ξ} is not a cone. Then dim(Q_{ξ}^{(3)}) = g − 1 and therefore the codimension of the image of Z in A_{g} is at least g.

Proof. Since T C_{ξ} is a not a cone its polar quadrics are linearly independent. □

The following example shows that the above bound is sharp for g = 5.

Example 6.6. Consider the locus C of intermediate Jacobians of cubic threefolds in A_{5}. Note that dim(A_{5}) = 15 and dim(C) = 10. These have (at least) an isolated triple point on their theta divisor whose tangent cone gives back the cubic threefold. The locus C is dominated by an irreducible component of S_{5}^{(3)} for which the estimate given in Proposition 6.5 is sharp. Cf. [6] where Casalaina-Martin proves that the locus of intermediate Jacobians of cubic threefolds is an irreducible component of the locus of principally polarized abelian varieties of dimension 5 with a point of multiplicity ≥ 3.

7. The Andreotti–Mayer loci

Andreotti and Mayer consider in A_{g} the algebraic sets of principally polarized abelian varieties X with a locus of singular points on Θ_{X} of dimension at least k. More generally, we are interested in the locus of principally polarized abelian varieties possessing a k-dimensional locus of singular points of multiplicity r on the theta divisor. To define these loci scheme-theoretically we consider the morphism π : X_{g} → A_{g} and the quasi-coherent sheaf on A_{g}

F_{k}^{(r)} = ⨁_{i=k}^{g−2} R^{i}π_{∗}O_{S_{g}^{(r)}}.

Definition 7.1. For integers k and r with 0 ≤ k ≤ g − 2 and 2 ≤ r ≤ g we define N_{g,k,r} as the support of F_{k}^{(r)}. We also set M_{g,k,r} = π^{−1}(N_{g,k,r}), a subscheme of both S_{g,k} and S_{g}^{(r)}. We write N_{g,k} and M_{g,k} for N_{g,k,2} and M_{g,k,2}.

The schemes N_{g,k} are the so-called Andreotti–Mayer loci in A_{g}, which were introduced in a somewhat different way in [2]. Note that N_{g,k,r} is locally defined by an annihilator ideal and so carries the structure of subscheme. Corollary 8.12 below and results by Debarre [11] (see §19) imply that the scheme structure at a general point of N_{g,0} defined above coincides with the one considered by Mumford in [27]. We now want to see that as a set N_{g,k,r} is the locus of points corresponding to (X, Θ_{X}) such that Sing(Θ_{X}) has an irreducible component of dimension ≥ k of points of multiplicity ≥ r for Θ_{X}.

Lemma 7.2. Let X be an abelian variety of dimension g and W ⊂ X an irreducible reduced subvariety of dimension n and let ω_{W} be its dualizing sheaf. Then H^{0}(W, ω_{W}) ≠ (0).

Proof. Let f : W′ → W be the normalization of W. We first claim the inequality h^{0}(W′, ω_{W′}) ≤ h^{0}(W, ω_{W}). To see this, note that by [24], p. 48 ff (see also [17], Exerc. 6.10, p. 239, 7.2, p. 249), there exists a map f_{∗}ω_{W′} → Hom(f_{∗}O_{W′}, ω_{W}), hence a map

H^{0}(W′, ω_{W′}) → H^{0}(W, Hom(f_{∗}O_{W′}, ω_{W})).

Now O_{W} → f_{∗}O_{W′} is an injection and therefore H^{0}(W, Hom(f_{∗}O_{W′}, ω_{W}) maps to H^{0}(W, Hom(O_{W} , ω_{W})) and we thus get a map H^{0}(W′, ω_{W′}) → H^{0}(W, ω_{W}) which is injective as one sees by looking at the smooth part of W. Let W̃ be a desingularization of W′. According to [22] we have h^{0}(W̃ , Ωn ; W̃) ≤ h^{0}(W′, ω_{W′}). Since W̃ maps to X we have h^{0}(W̃ , Ω1 ; W̃) ≥ n. If h^{0}(W̃ , Ωn ; W̃) were 0 then ∧^{n}H^{0}(W̃ , Ω1 ; W̃) → H^{0}(W̃ , Ωn ; W̃) would be the zero map contradicting the fact that W has dimension n. □

Corollary 7.3. We have (X, Θ_{X}) ∈ N_{g,k,r} if and only if dim(Sing^{(r)}(Θ_{X})) ≥ k.

Proof. By the previous lemma and Serre duality for a reduced irreducible subvariety W of dimension m in X it follows that H^{m}(W, O_{W}) ≠ (0) and we know H^{k}(W, O_{W}) = (0) for k > m. This implies the corollary. □

There are the inclusions

N_{g,k,r} ⊆ N_{g,k,r−1}, N_{g,k,r} ⊆ N_{g,k−1,r}.

If p = (n_{1}, …, n_{r}) with 1 ≤ n_{1} ≤ … ≤ n_{r} < g and n_{1} + … + n_{r} = g is a partition of g we write A_{g,p} for the suborbifold (or substack) of A_{g} corresponding to principally polarized abelian varieties that are a product of r principally polarized abelian varieties of dimensions n_{1}, …, n_{r}. We write r(p) = r for the length of the partition and write A_{g,[r]} for the suborbifold ∪_{r(p)=r}A_{g,p} of A_{g} corresponding to pairs (X, Θ_{X}) isomorphic as a polarized abelian variety to the product of r principally polarized abelian varieties. One has the stratification

A_{g,[g]} ⊂ A_{g,[g−1]} ⊂ ⋯ ⊂ A_{g,[2]}.

We will denote by:

i) Π_{g} = ∪_{r≥2}A_{g,r} = A_{g,[2]} the locus of decomposable principally polarized abelian varieties; ii) A_{g}^{(ns)} the locus of classes of non-simple abelian varieties, i.e., of principally polarized abelian varieties of dimension g which are isogenous to a product of abelian varieties of dimension smaller that g; iii) A_{g}^{End≠ℤ} the locus of classes of singular abelian varieties, i.e., of principally polarized abelian varieties whose endomorphism ring is larger than ℤ.

Remark 7.4. Note the inclusions Π_{g} ⊂ A_{g}^{(ns)} ⊂ A_{g}^{End≠ℤ}. The locus Π_{g} is reducible with irreducible components A_{g,p} with p running through the partitions g = (i, g −i) of g for 1 ≤ i ≤ g/2 and we have codim_{A_{g}}(A_{g,(i,g−i)}) = i(g −i). In contrast to this A_{g}^{End≠ℤ} and A_{g}^{(ns)} are the union of infinitely countably many irreducible closed subsets of A_{g} of codimension at least g − 1, the minimum codimension being achieved for families of abelian varieties that are isogenous to products of an elliptic curve with an abelian variety of dimension g − 1 (compare with [7]).

We recall a result from [20] and the main result from [12].

Theorem 7.5. For every integer r with 2 ≤ r ≤ g one has:

(i) N_{g,k,r} = ∅ if k > g − r;

(ii) N_{g,g−r,r} = A_{g,[r]}, i.e., (X, Θ_{X}) ∈ N_{g,g−r,r} is an r-fold product.

Hence, for every integer r such that 2 ≤ r ≤ g, one has the stratification

N_{g,0,r} ⊃ N_{g,1,r} ⊃ … ⊃ N_{g,k,r} ⊃ … ⊃ N_{g,g−r,r} = A_{g,[r]},

whereas for every integer k such that 0 ≤ k ≤ g − 2, one has the stratification

N_{g,k,2} ⊃ N_{g,k,3} ⊃ … ⊃ N_{g,k,r} ⊃ … ⊃ N_{g,k,g−k} = A_{g,[g−k]}.

8. Lower bounds for the codimension of Andreotti-Mayer loci

The results in the previous sections give information about the Zariski tangent spaces to these loci and this will allow us to prove bounds on the dimension of the Andreotti–Mayer loci, which is our main objective in this paper. We start with the results on tangent spaces. We need some notation.

Definition 8.1. Let ζ = (X, Θ_{X}) represent a point in N_{g,k,r}. By L_{g,k,r}(ζ) we denote the linear system of quadrics ℙ(T_{N_{g,k,r},ζ}^{⊥}), where T_{N_{g,k,r},ζ} is the Zariski tangent space and where we view ℙ(T_{A_{g,ζ}}) as a space of quadrics as in Section 2. As usual, we may drop the index r if r = 2 and write L_{g,k}(ζ) for L_{g,k,2}(ζ).

Notice that

dim_{ζ}(N_{g,k,r}) ≤ \binom{g + 1}{2} − dim(L_{g,k}(ζ)) + 1.

Definition 8.2. For ζ = (X, Θ_{X}) ∈ N_{g,k,r} we denote by Sing^{(k,r)}(Θ_{X}) the locally closed subset

Sing^{(k,r)}(Θ_{X}) = {x ∈ Sing(Θ_{X}) : dim_{x}(Sing^{(r)}(Θ_{X})) ≥ k}.

Moreover, we define Q_{ζ}^{(k,r)} to be the linear system of quadrics in ℙ^{g−1} = ℙ(T_{X,0})

spanned by the union of all linear systems Q_{ξ}^{(r)} with ξ = (X, Θ_{X}, x) and x ∈ Sing^{(k,r)}(Θ_{X}).

Propositions 4.1 and 6.3 imply the following basic tool for giving upper bounds on the dimension of the Andreotti–Mayer loci.

Proposition 8.3. Let N be an irreducible component of N_{g,k,r} with its reduced structure. If ζ = (X, Θ_{X}) is a general point of N then the projectivized conormal space to N at ζ, viewed as a subspace of ℙ(Sym^{2}(T_{X,0}^{∨})), contains the linear system Q_{ζ}^{(k,r)}.

Proof. Let M be an irreducible component of π^{∗}N in S_{g,k}^{(r)}. If ξ is smooth point of M then the image of the Zariski tangent space to M at ξ under dπ is orthogonal to Q_{ξ}^{(r)} for all x ∈ Sing^{k,r}(Θ_{X}). Since we work in characteristic 0 the map dπ is surjective on the tangent spaces for general points m ∈ M and π(m) ∈ N. Therefore the result follows from Propositions 4.1 and 6.3. □

We need a couple of preliminary results. First we state a well-known fact, which can be proved easily by a dimension count.

Lemma 8.4. Every hypersurface of degree d ≤ 2n−3 in ℙ^{n} with n ≥ 2 contains a line.

Next we prove the following:

Lemma 8.5. Let V in ℙ^{s} be a hypersurface of degree d ≥ 3. If all polar quadrics of V coincide, then V is a hyperplane H counted with multiplicity d, and the polar quadrics coincide with 2H.

Proof. If d = 3 the assertion follows from general properties of duality (see [41], p. 215) or from an easy calculation. If d > 3, then the result, applied to the cubic polars of V, tells us that all these cubic polars are equal to 3H, where H is a fixed hyperplane. This immediately implies the assertion. □

The next result has been announced in [8].

Theorem 8.6. Let g ≥ 4 and let N be an irreducible component of N_{g,k} not contained in N_{g,k+1}. Then:

(i) for every positive integer k ≤ g − 3, one has codim_{A_{g}}(N) ≥ k + 2, whereas codim_{A_{g}}(N) = g − 1 if k = g − 2;

(ii) if N is contained in N_{g,k,r} with r ≥ 3, then codim_{A_{g}}(N) ≥ k + 3; (iii) if g − 4 ≥ k ≥ g/3, then codim_{A_{g}}(N) ≥ k + 3.

Proof. By Theorem 7.5 and Remark 7.4, we may assume k < g − 2. By definition, there is some irreducible component M of π_{S_{g}}^{−1}(N) with dim(M) = dim(N)+k which dominates N via π. We can take a general point (X, Θ_{X}, z) ∈ M so that ζ = (X, Θ_{X}) is a general point in N. By Remark 7.4 we may assume X is simple. Let R be the unique k-dimensional component of π_{|S_{g}}^{−1}(ζ) containing (X, Θ_{X}, z). Its general point is of the form ξ = (X, Θ_{X}, x) with x the general point of the unique k-dimensional component of Sing(Θ_{X}) containing z, and x has multiplicity r on Θ_{X}. By abusing notation, we may still denote this component by R. Proposition 6.3 implies that the linear system of quadrics ℙ(T_{N,ζ}^{⊥}) contains all polar quadrics of T C_{ξ} with ξ = (X, Θ_{X}, x) ∈ R. Thus we have a rational map

ϕ : (ℙ^{g−1})^{r−2} × R −−→ Q_{ζ}^{(r)}

which sends the general point (b, ξ) := (b_{1}, …, b_{r−2}, ξ) to the polar quadric Q_{b,ξ} of T C_{ξ} with respect to b_{1}, …, b_{r−2}. It is useful to remark that the quadric Q_{b,ξ} is a cone with vertex containing the projectivized Zariski tangent space to R at x (see Remark 6.4).

Claim 8.7 (The finiteness property). For each b ∈ (ℙ^{g−1})^{r−2} the map ϕ restricted to {b} × R has finite fibres.

Proof of the claim. Suppose the assertion is not true. Then there is an irreducible curve Z ⊆ R such that, for ξ = (X, Θ_{X}, x) corresponding to the general point in Z, one has Q_{b,ξ} = Q. Set Π = Vert(Q), which is a proper subspace of ℙ^{g−1}. Consider the Gauss map

γ = γ_{Z} : Z −−→ ℙ^{g−1} = ℙ(T_{X,0})

which associates to a smooth point of Z its projectivized tangent space. Then Proposition 6.3 implies that γ(ξ) ∈ R ⊆ Vert(Q_{ξ}) = Π. Thus γ(Z) is degenerate in ℙ^{g−1} and this yields that X is non-simple, cf. [31]. This is a contradiction which proves the claim. □

Claim 8.7 implies that the image of the map ϕ has dimension at least k, hence codim_{A_{g}}(N_{g,k}) ≥ k + 1. To do better we need the following information.

Claim 8.8 (The non-degeneracy property). The image of the map ϕ does not contain any line.

Proof of the claim. Suppose the claim is false. Take a line L in the image of the map ϕ, and let L be the corresponding pencil of quadrics. By Proposition 6.3 and Remark 6.4, the general quadric in L has rank ρ ≤ g − k. Then part (i) of Segre’s Theorem 21.2 in §21 below implies that the Gauss image γ(Z) of any irreducible component of the curve Z = ϕ^{−1}(L) is degenerate. This again leads to a contradiction. This proves the claim. □

Claim 8.8 now implies that the image of ϕ spans a linear space of dimension at least k + 1, hence (i) follows. To prove part (ii) we now want to prove that dim(Q_{ζ}^{(r)}) > k + 1. Remember that the image of ϕ has dimension at least k by Claim 8.7. If the image has dimension at least k + 1, then by Claim 8.8 it cannot be a projective space and therefore dim(Q_{ζ}^{(r)}) > k + 1. So we can assume that the image has dimension k. Therefore each component of the fibre F_{Q} over a general point Q in the image has dimension (g − 1)(r − 2). Consider now the projection of F_{Q} to R. If the image is positive-dimensional then there is a curve Z in R such that the image of the Gauss map of Z is contained in the vertex of Q. Then X is non-simple, a contradiction (see the proof of Claim 8.7). Therefore the image of F_{Q} on R is constant, equal to a point ξ, hence F_{Q} = (ℙ^{g−1})^{r−2}×{ξ}. By Lemma 8.5 there is a hyperplane H_{ξ} such that T C_{ξ} = rH_{ξ}, and Q = 2H_{ξ}. Therefore we have a rational map

ψ : R −−→ ℙ^{g−1}^{∨}

sending ξ to H_{ξ}. We notice that the image of ϕ is then equal to the 2-Veronese image if the image of ψ. By Claim 8.7, the map ψ has finite fibres. By an argument as in Claim 8.8, we see that the image of ψ does not contain a line, hence is not a linear space. Thus it spans a space of dimension s ≥ k+1. Then its 2-Veronese image, which is the image of ϕ spans a space of dimension at least 2s ≥ 2k + 2 > k + 2. To prove part (iii), by part (ii), we can assume r = 2. It suffices to show that dim(Q_{ζ}) > k + 1, where Q_{ζ} := Q_{ζ}^{(2)}. Suppose instead that dim(Q_{ζ}) = k + 1 and set Σ = ϕ(R). We have to distinguish two cases:

(a) not every quadric in Q_{ζ} is singular;

(b) the general quadric in Q_{ζ} is singular, of rank g − ρ < g. In case (a), consider the discriminant Δ ⊂ Q_{ζ}, i.e. the scheme of singular quadrics in Q_{ζ}. This is a hypersurface of degree g, which, by Proposition 4.4, contains Σ with multiplicity at least k. Thus deg(Σ) ≤ g/k ≤ 3 and Σ contains some line, so that we have the corresponding pencil L of singular quadrics. By Claim 8.8, one arrives at a contradiction. Now we treat case (b). Let g − h be the rank of the general quadric in Σ. One has g − h ≤ g − k, hence k ≤ h. Moreover one has g − h ≤ g − ρ, i.e. ρ ≤ h. Suppose first ρ = h, hence ρ ≥ k. Let s be the dimension of the subspace

Π := ⋂_{Q∈Q_{ζ},rk(Q)=g−ρ} Vert(Q).

By applying part (iii) of Segre’s Theorem 21.2 to a general pencil contained in Q_{ζ}, we deduce that

(18)  3ρ ≤ g + 2s + 2.

Claim 8.9. One has s < r − k.

Proof. Suppose s ≥ ρ − k. If Π′ is a general subspace of Π of dimension ρ − k, then its intersection with ℙ(T_{R,x}), where x ∈ R is a general point, is not empty. Since ρ − k < g − k and X is simple, this is a contradiction (see [31], Lemma II.12). □

By (18) and Claim 8.9 we deduce that ρ + 2k ≤ g and therefore 3k ≤ g, a contradiction. Suppose now ρ < h. Then part (iv) of Segre’s Theorem 21.2 yields

deg(Σ) ≤ \frac{g − 2 − ρ}{h − ρ}.

The right-hand-side is an increasing function of ρ, thus deg(Σ) ≤ g − h − 1 ≤ 2k − 1 because g ≤ 3k ≤ 2k + h. By Lemma 8.4 the locus Σ contains a line, and we can conclude as in the proof of the non-degeneracy property 8.8.

□

The following corollary was proved independently by Debarre [9] and includes a basic result by Mumford [27].

Corollary 8.10. Let g ≥ 4. Then:

(i) every irreducible component S of S_{g} has codimension g+1 in X_{g}, hence S_{g} is locally a complete intersection in X_{g};

(ii) if ξ = (X, Θ_{X}, x) is a general point of S, then either Θ_{X} has isolated singularities or (X, Θ_{X}) is a product of an elliptic curve with a principally polarized abelian variety of dimension g − 1;

(iii) every irreducible component N of N_{g,0} has codimension 1 in A_{g}; (iv) if (X, Θ_{X}) ∈ N is a general element of an irreducible component N

of N_{g,0}, then Θ_{X} has isolated singularities. Moreover for every point x ∈ Sing(Θ_{X}), the quadric Q_{x} is smooth and independent of x.

Proof. Take an irreducible component S of S_{g} and let N be its image via the map π : X_{g} → A_{g}. Then there is a maximal integer k ≤ g − 2 such that N is contained in N_{g,k}. Suppose first that 1 ≤ k ≤ g − 3. Then by Theorem 8.6 the codimension of N in A_{g} is at least k + 2. This implies that the codimension of S in X_{g} is at least g + 2, which is impossible because S_{g} is locally defined in X_{g} by g + 1 equations. Then either k = 0 or k = g − 2. In the former case S maps to an irreducible component N of N_{g,0} which is a proper subvariety of codimension 1 in A_{g}. If k = g − 2, then by Theorem 7.5 the polarized abelian variety (X, Θ_{X}) is a product of principally polarized abelian varieties. The resulting abelian variety has a vanishing thetanull and so N is contained in the divisor of a modular form (a product of thetanulls, cf. e.g. [27], p. 370) which is a component of N_{g,0}. Assertions (i)–(iii) follow by a dimension count (see Remark 7.4). Part (iv) follows by Propositions 4.1 and 8.3. □

Finally, we show a basic property of S_{g}.

Theorem 8.11. The locus S_{g} is reduced.

Proof. The assertion is well-known for g ≤ 3, using the theory of curves. We may assume g ≥ 4. Let S be an irreducible component of S_{g}, let N = π(S) and k ≤ g − 2 the maximal integer such that N is contained in N_{g,k}. As in the proof of Corollary 8.10, one has either k = 0 or k = g − 2. Assume first k = 0, and let ξ = (X, Θ_{X}, x) ∈ S be a general point. We are going to prove that Sing(Θ_{X}) is reduced of dimension 0. First we prove that x has multiplicity 2 for Θ_{X}. Suppose this is not the case and x has multiplicity r ≥ 3. Since N_{g,0} has codimension 1 all polar quadrics of T C_{ξ} are the same quadric, say Q (see Proposition 6.3 and 8.3). By Lemma 8.5 the tangent cone T C_{ξ} is a hyperplane H with multiplicity r. Again by Proposition 6.3 all the derivatives of Q with respect to points b ∈ H coincide with Q. By Proposition 5.3 the scheme D_{b,2} supported at x is contained in Sing(Θ_{X}). By taking into account Proposition 5.4 and repeating the same argument we see that this subscheme can be indefinitely extended to a 1-dimensional subscheme, containing x and contained in Sing(Θ_{X}). This implies that the corresponding component of N_{g,0} is contained in N_{g,1}, which is not possible since the codimension of N_{g,1} is at least 3. If x has multiplicity 2 the same argument shows that the quadric Q_{ξ} is smooth. By Corollary 3.4, ξ is a smooth point of S_{g} and this proves the assertion. Suppose now that k = g − 2. Then by Theorem 7.5 N is contained in the locus of products A_{g,(1,g−1)}, and for dimension reasons it is equal to it and then the result follows from a local analysis with theta functions. □

The following corollary is due to Debarre ([11]).

Corollary 8.12. If (X, Θ_{X}) is a general point in a component of N_{g,0} then Θ_{X} has finitely many double points with the same tangent cone which is a smooth quadric.

9. A conjecture

As shown in [8], part (i) of Theorem 8.6 is sharp for k = 1 and g = 4 and 5. However, as indicated in [8], it is never sharp for k = 1 and g ≥ 6, or for k ≥ 2. In [8] we made the following conjecture, which is somehow a natural completion of Andreotti–Mayer’s viewpoint in [2] on the Schottky problem. Recall the Torelli morphism t_{g} : M_{g} → A_{g} which maps the isomorphism class of a curve C to the isomorphism class of its principally polarized Jacobian (J(C), Θ_{C}). As a map of orbifolds it is of degree 2 for g ≥ 3 since the general abelian variety has an automorphism group of order 2 and the general curve one of order 1. We denote by J_{g} the jacobian locus in A_{g}, i.e., the Zariski closure of t_{g}(M_{g}) in A_{g} and by H_{g} the hyperelliptic locus in A_{g}, that is, the Zariski closure in A_{g} of t_{g}(H_{g}), where H_{g} is the closed subset of M_{g} consisting of the isomorphism classes of the hyperelliptic curves. By Torelli’s theorem we have dim(J_{g}) = 3g − 3 and dim(H_{g}) = 2g − 1 for g ≥ 2.

Conjecture 9.1. If 1 ≤ k ≤ g − 3 and if N is an irreducible component of N_{g,k} not contained in A_{g}^{End≠ℤ}, then codim_{A_{g}}(N) ≥ \binom{k+2}{2}. Moreover, equality holds if and only if one of the following happens:

(i) g = k + 3 and N = H_{g};

(ii) g = k + 4 and N = J_{g}.

By work of Beauville [4] and Debarre [11] the conjecture is true for g = 4 and g = 5. Debarre [10, 11] gave examples of components of N_{g,k} for which the bound in Conjecture 9.1 for the codimension in A_{g} fails, but they are contained in A_{g}^{End≠ℤ}, since the corresponding abelian varieties are isogenous to products. Our main objective in this paper will be to prove the conjecture for k = 1.

Remark 9.2. The question about the dimension of the Andreotti–Mayer loci is related to the one about the loci S_{g,k} introduced in §3. Note that M_{g,k} = π^{∗}(N_{g,k}) is a subscheme of S_{g,k}. Let N be an irreducible component of N_{g,k} not contained in N_{g,k+1}, let M be the irreducible component of M_{g,k} dominating N and let Z be an irreducible component of S_{g,k} containing M. We now give a heuristic argument. Recalling Proposition 4.3, we can consider Z to be well-behaved if codim_{S_{g}}(Z) = \binom{k+1}{2}. Since M is contained in Z, one has also codim_{S_{g}}(M) ≥ \binom{k+1}{2} and therefore codim_{A_{g}}(N) ≥ \binom{k+2}{2}, which is the first assertion in Conjecture 9.1. In this setting, the equality holds if and only if Z is well-behaved and M = Z. On the other hand, since M is got from Z by imposing further restrictions, one could expect that M is, in general, strictly contained in Z and therefore that codim_{A_{g}}(N) > \binom{k+2}{2}. In this circle of ideas, it is natural to ask if J_{g} [resp. H_{g}] is dominated by a well-behaved component of S_{g,g−4} [resp. S_{g,g−3}]. This is clearly the case if g = 4. A second, related, question is whether J_{g} [resp. H_{g}] is contained in some irreducible subvariety of codimension c < \binom{g−2}{2} [resp. c < \binom{g−1}{2}] in A_{g}, whose general point corresponds to a principally polarized abelian variety (X, Θ_{X}) with Sing(Θ_{X}) containing a subscheme isomorphic to Spec(ℂ[ϵ_{1}, …, ϵ_{k}]/(ϵ_{i}ϵ_{j}: 1 ≤ i, j ≤ k)) with k = g − 4 [resp. k = g − 3]. One might be tempted to believe that an affirmative answer to the first question implies a negative answer to the second. This is not the case. Indeed H_{4} is contained in the locus of Jacobians of curves with an effective even theta characteristic. In this case S_{4,1} has two well-behaved irreducible components of dimension 8, one dominating H_{4} with fibres of dimension 1, the other one dominating J_{4} ∩ θ_{4,0}, where θ_{4,0} is the theta-null locus, see §19 and §10 below and [16]. These two components intersect along a 7-dimensional locus in S_{4} which dominates H_{4}. Note that it is not always the case that a component of S_{g,k} is well-behaved. For example, there is an irreducible component of S_{g,g−2}, the one dominating A_{g,(1,g−1)}, which is also an irreducible component of S_{g}.

10. An Example for Genus g = 4

In this section we illustrate the fact that the quadrics associated to the singularities of the theta divisor may provide more information than obtained above. The locus S_{4} in the universal family X_{4} consists of three irreducible components: i) one dominating θ_{0,4}, the locus of abelian varieties with a vanishing theta-null; we call it A; ii) one dominating the Jacobian locus J_{4}; we call it B; iii) one dominating A_{4,(3,1)}, the locus of products of an elliptic curve with a principally polarized abelian variety of dimension 3. The components A and B have codimension 5 in X_{4} and they intersect along a locus C of codimension 6. The image of C in A_{4} is the locus of Jacobians with a half-canonical g_{3}^{1}. The quadric associated to the singular point of order 2 of X in Θ_{X} has corank 1 at a general point. We refer to [16] for a characterization of the intersection θ_{0,4} ∩ J_{4}.

Proposition 10.1. If ξ = (X, Θ_{X}, x) is a point of C then Sing(Θ_{X}) contains a scheme of length 3 at x.

Proof. The scheme Sing(Θ_{X}) contains a scheme Spec(ℂ[ϵ]) at x, say corresponding to the tangent vector D_{1} ∈ ker q_{x}, cf. Section 5. In order that ξ be a singular point of S_{4}, the quadrics q and q_{1} associated to x and D_{1} (cf. equation (6) and Proposition 3.3) must be linearly dependent. By Proposition 5.3 this implies that Sing(Θ_{X}) contains a scheme of length 3 at x. □

Proposition 10.2. The two components θ_{0,4} and J_{4} of N_{4,0} in A_{4} are smooth at a non-empty open subset of their intersection and are tangent there.

Proof. Let ξ = (X, Θ_{X}, x) be a general point of C so that the singular point x of Θ_{X} defines a quadric q_{x} of rank 3. Set η = (X, Θ_{X}) ∈ A_{4}. From the equations (3) and (6), one deduces that the tangent cone to N_{4,0} at η is supported on q_{x}^{⊥}. Suppose, as we may by applying a suitable translation, that x is the origin 0 in X. Then θ_{0,4} is locally defined by the equation θ(τ, 0) in A_{4} (see [16], §2). Hence θ_{0,4} is smooth at η with tangent space q_{x}^{⊥}. Now the jacobian locus J_{4} is also smooth at η by the injectivity of the differential of Torelli’s morphism at non-hyperelliptic curves (see [15]) and the assertion follows. □

11. The Gauss map and tangencies of theta divisors

In this section we shall study the situation where a number of translates of the theta divisor of a principally polarized abelian variety are ‘tangentially degenerate,’ that is, are smooth but with linearly dependent tangent spaces or singular at prescribed points. Let (B, Ξ) be a polarized abelian variety of dimension g, where Ξ is an effective divisor on B. As usual we let B = ℂ^{g}/Λ, with Λ a 2g-dimensional lattice, and p : ℂ^{g} → B be the projection. So we have coordinates z = (z_{1}, …, z_{g}) in ℂ^{g} and therefore on B, and we can keep part of the conventions and the notation used so far. Let ξ = ξ(z) be the Riemann theta function whose divisor on ℂ^{g} descends to Ξ via p. If Ξ is reduced, then the Gauss map of (B, Ξ) is the morphism

γ = γ_{Ξ} : Ξ − Sing(Ξ) → ℙ(T_{B,0}^{∨}), x ↦ ℙ(t_{−x}(T_{Ξ,ξ})),

where t_{−x}(T_{Ξ,x}) is the tangent space to Ξ at ξ translated to the origin. If Ξ_{b} = t_{b}(Ξ) is the translate of Ξ by the point b ∈ B defined by the equation ξ(z − b) = 0 then for x ∈ Ξ − Sing(Ξ) the origin is a smooth point for Ξ_{−x} and γ(x) = ℙ(T_{Ξ_{−x},0}). As usual we have natural homogeneous coordinates (z_{1} : …: z_{g}) in ℙ(T_{B,0}) = ℙ^{g−1} and therefore dual coordinates in the dual projective space ℙ(T_{B,0}^{∨}) = ℙ^{g−1}^{∨}. Then the expression of γ in coordinates equals γ(p(z)) = (∂_{1}ξ(z) : …: ∂_{g}ξ(z)) with ∂_{i} = ∂/∂z_{i}. The following lemma is well-known.

Lemma 11.1. Let (B, Ξ) be a simple abelian variety of dimension g and Ξ an irreducible effective divisor on B. Then the map γ_{Ξ} has finite fibres. Moreover, for a smooth point x ∈ Ξ there are only finitely many b ∈ B such that Ξ_{b} is smooth at x and tangent to Ξ there.

Proof. Suppose γ_{Ξ} does not have finite fibres. Then there is an irreducible curve C of positive geometric genus contained in the smooth locus of Ξ which is contracted by γ_{Ξ} to a point of ℙ^{g−1}^{∨} corresponding to a hyperplane Π ⊂ ℙ^{g−1}. Then the image of the Gauss map γ_{C} of C lies in Π. This implies that C does not generate B and contradicts the fact that B is simple. As to the second statement, if it does not hold there exists an irreducible curve C such that for all b ∈ C the divisor Ξ_{b} is smooth at x and tangent to Ξ. Then the curve C′ = {x − b : b ∈ C} is contained in Ξ and contracted by γ_{Ξ}. This contradicts the fact that γ_{Ξ} has finite fibres as we just proved. □

Definition 11.2. Let h be a natural number with 1 ≤ h ≤ g − 1. The subscheme T_{h}(B, Ξ) of B × B^{h} is defined in ℂ^{g} × (ℂ^{g})^{h} with coordinates (z, u_{1}, …, u_{h}) by the equations

(19)  \begin{aligned}ξ(z) = 0, ξ(z − u_{1}) = 0, …, ξ(z − u_{h}) & = 0, \\ rk ( ∂_{1}ξ(z)  ⋯  ∂_{g}ξ(z) ; ∂_{1}ξ(z − u_{1}) ⋯ ∂_{g}ξ(z − u_{1}) ; ⋮ ; ∂_{1}ξ(z − u_{h}) ⋯ ∂_{g}ξ(z − u_{h})) & ≤ h. \end{aligned}

The projection to the first factor induces a morphism p_{1} : T_{h}(B, Ξ) → Ξ. Note that for i = 1, …, h the variety E_{i} of codimension h + 1 ≤ g in B × B^{h} defined by the equations

ξ(z) = 0, ξ(z − u_{j}) = 0 (j ≠ i), and u_{i} = 0,

is contained in T_{h}(B, Ξ). Moreover, the expected codimension of the irreducible components of T_{h}(B, Ξ) is g + 1. We will say that an irreducible component T of T_{h}(B, Ξ) is regular if:

(i) T ≠ E_{i} for i = 1, …, h; (ii) on a non-empty open subset of T all the rows of the matrix in (19) are

non-zero. In particular, if T is regular then p_{1}(T) ⊈ Sing(Ξ).

Proposition 11.3. If B is simple each regular component of T_{h}(B, Ξ) has dimension hg − 1.

Proof. Let us first assume h = 1 and denote a component of T_{h}(B, Ξ) by T. By composing p_{1} with the Gauss map γ = γ_{Ξ}, we obtain a rational map ϕ : T −−→ ℙ(T_{B,0}^{∨}). We shall prove that ϕ has finite fibres. Let v be a point in the image of ϕ coming from a point in the open subset as in Definition 11.2, (ii). By Lemma 11.1 there are only finitely many smooth points z_{1}, …, z_{a} ∈ Ξ such that γ(z_{i}) = v for i = 1, …, a. For each 1 ≤ i ≤ a we consider the theta divisor defined by ξ(z_{i} − u) = 0. Again by Lemma 11.1 there are only finitely many points u_{i1}, …, u_{iℓ_{i}} in it such that (z_{i}, u_{ij}) may give rise to a point on T. So ϕ has finite fibres. Thus it is dominant and T has dimension g − 1. Now we prove the assertion by induction on h. Consider the projection q : T → B × B^{h−1} by forgetting the last factor B. If the image T′ of T is contained in T_{h−1}(B, Ξ), then it is contained in a regular component of T_{h−1}(B, Ξ), hence by induction the codimension of T is at least g + 1, while we know from the equations that it is at most g + 1, and the assertion follows. Suppose T′ is not contained in T_{h−1}(B, Ξ). Let 𝕌(h − 1, g − 1) → 𝔾(h − 1, g − 1) be the universal bundle over the Grassmannian of (h − 1)-planes in ℙ^{g−1}. Then we define a rational map ψ : T −−→ 𝕌(h − 1, g − 1) in the following way. If (z, u_{1}, …, u_{h}) is a general point in T, then by the assumption that T′ ⊈ T_{h−1}(B, Ξ) the hyperplanes γ(z), γ(z − u_{1}), …, γ(z − u_{h−1}) are linearly independent and we define

ψ(z, u_{1}, …, u_{h}) = (⟨γ(z), γ(z − u_{1}), …, γ(z − u_{h−1})⟩, γ(z − u_{h})).

We claim that Lemma 11.1 implies that the general fibre of ψ has dimension ≤ h(h − 1). Indeed, in a fibre of ψ the points γ(z), γ(z − u_{1}), …, γ(z − u_{h−1}) vary in a (h − 1)-dimensional space giving at most h(h − 1) parameters and if γ(z), γ(z − u_{1}), …, γ(z − u_{h}) are fixed there are for (z, u_{1}, …, u_{h}) only finitely many possibilities. Thus dim(T) is bounded from above by

dim(𝕌(h − 1, g − 1)) + h(h − 1) = h(g − h) + (h − 1) + h(h − 1) = hg − 1,

and this proves the assertion. □

We will also consider the closed subscheme T_{h}^{0}(B, Ξ) of T_{h}(B, Ξ) which is defined in ℂ^{g} × (ℂ^{g})^{h} by the equations

ξ(z) = 0, ξ(z − u_{i}) = 0, i = 1, …, h

and

rk ( ∂_{1}ξ(z)  ⋯  ∂_{g}ξ(z) ; ∂_{1}ξ(z − u_{1}) ⋯ ∂_{g}ξ(z − u_{1}) ; ⋮ ; ∂_{1}ξ(z − u_{h}) ⋯ ∂_{g}ξ(z − u_{h})) = 0.

Finally we will consider the closed subset T_{h}(X, B) which is the union of T_{h}^{0}(X, B) and of all regular components of T_{h}(B, Ξ). Look at the projection

p = p_{2} : T (X, B) → B^{h}.

Definition 11.4. We define N_{0,h}(B, Ξ) to be the image of T_{h}(X, B) under the map p. More generally, for each integer k we define

N_{k,h}(B, Ξ) := \overline{\{u = (u_{1}, …, u_{h}) ∈ B^{h}: u_{1}, …, u_{h} ≠ 0, dim(p_{2}^{−1}(u)) ≥ k\}}.

Roughly speaking, N_{k,h}(B, Ξ) is the closure of the set of all (u_{1}, …, u_{h}) ∈ B^{h} such that Ξ contains an irreducible subvariety V of dimension n ≥ k and for all z ∈ V either:

(a) z, z−u_{1}, …, z−u_{h} are smooth points of Ξ and γ(z), γ(z−u_{1}), …, γ(z− u_{h}) are linearly dependent, or

(b) all the points z, z − u_{1}, …, z − u_{h} are contained in Sing(Ξ). In case (a) we say that the divisors Ξ, Ξ_{u_{1}}, …, Ξ_{u_{h}}, all passing through z with multiplicity one, are tangentially degenerate at z.

In case h = 1 we have only two divisors which are just tangent at z. We will drop the index h if h = 1. Thus, N_{0}(B, Ξ) is the set of all u such that Ξ and Ξ_{u} are either tangent, or both singular, somewhere. Note that, if Ξ is symmetric, i.e., if ξ(z) is even, as happens for the Riemann theta function, then N_{0}(B, Ξ) contains the divisorial component 2Ξ := {2ξ: ξ ∈ Ξ}, since γ(ξ) = γ(−ξ) for all smooth points ξ ∈ Ξ. One has the following result by Mumford (see [27], Proposition 3.2).

Theorem 11.5. If (B, Ξ) is a principally polarized abelian variety of dimension g with Ξ smooth, then N_{0}(B, Ξ) is a divisor on B algebraically equivalent to \frac{(g+2)!}{6} Ξ.

As we will see later, the following result is related to Conjecture 9.1.

Proposition 11.6. Suppose that (B, Ξ) is a simple principally polarized abelian variety of dimension g. Assume that (B, Ξ) ∉ N_{g,k}. Then for k ≥ 0 and 1 ≤ h ≤ g − 1 and for every irreducible component Z of N_{k,h}(B, Ξ) one has codim_{B^{h}}(Z) ≥ k + 1.

Proof. By the definition of N_{k,h}(B, Ξ) and by the fact that Sing(Ξ) has dimension < k an irreducible component of N_{k,h}(B, Ξ) can only be contained in the image of a regular component of T_{h}(B, Ξ). The assertion follows by Proposition 11.3 and the fact that the fibres of p_{2} on a regular component have dimension ≥ k. □

12. Properties of the loci N_{k,h}(B, Ξ)

We will prove now a more precise result in the spirit of Proposition 11.6.

Proposition 12.1. Suppose that (B, Ξ) is a simple, principally polarized abelian variety of dimension g. Assume that N_{k,h}(B, Ξ) is positive-dimensional for some h ≥ 1 and k ≥ 1. Then (B, Ξ) ∈ N_{g,k−1}.

Proof. We may assume (B, Ξ) ∉ N_{g,k}, otherwise there is nothing to prove. We may also assume h ≥ 1 is minimal under the hypothesis that N_{k,h}(B, Ξ) has positive dimension. Let (u_{1}, …, u_{h}) be a point in N_{k,h}(B, Ξ), so that Ξ_{0} := Ξ and Ξ_{i} := Ξ_{u_{i}}, i = 1, …, h, are tangentially degenerate along a non-empty subset V^{0} of an irreducible subvariety V of B of dimension k such that Ξ_{i} for i = 0, …, h are smooth at all points v ∈ V^{0}. For j = 0, …, h the element s_{j}^{(i)} := ∂_{i}ξ(z − u_{j}) (with the convention that u_{0} = 0) is a section of O(Ξ_{j}) when restricted to V since Ξ_{j} contains V. We know that for given j not all s_{j}^{(i)} are identically zero on V^{0}. Our assumptions on the tangential degenerateness and minimality tell us that there exist non-zero rational functions a_{j} such that we have a non-trivial relation

(20)  ∑_{j=0}^{h} a_{j}s_{j}^{(i)} = 0 for i = 1, …, g.

Suppose that the a_{j} are regular on all of a desingularization f : W → V of V. Then they are constant and the relation ∑ a_{j}s_{j}^{(i)} = 0 holds on the whole of W. By writing relation (20) in different patches which trivialize the involved line bundles, and comparing them, we see that, if the transition functions are not all proportional, then we can shorten the original relation by subtracting two of them. This would contradict the minimality assumption. Therefore, we have that all of the divisors f^{∗}(Ξ − Ξ_{u_{j}})_{|V} with j = 1, … h are linearly equivalent. Since u = (u_{1}, …, u_{h}) varies in a subvariety of positive dimension this implies that the map Pic^{0}(B) → Pic(W) has a positive-dimensional kernel, and this is impossible since B is simple. So there exists an index j such that the function a_{j} has poles on a divisor Z_{j} of W. Note now that a point which is non-singular for all the divisors Ξ_{j}, j = 1, …, h, is certainly not a pole for the functions a_{j}. Therefore, for each j = 1, …, h, there an ℓ depending of j such that Z_{j} is contained in the divisorial part of the scheme f^{∗}(Sing(Ξ_{ℓ})). Moreover, since Z_{j} moves in a linear system on W, it cannot be contracted by the birational morphism f. This proves that Ξ_{ℓ} is singular along a variety of dimension k − 1 contained in V, which proves the assertion. □

Corollary 12.2. Suppose that (B, Ξ) is a simple, principally polarized abelian variety of dimension g. Assume that (B, Ξ) ∉ N_{g,0}, that is, Ξ is smooth. Then every irreducible component Z of N_{0,h}(B, Ξ) is a divisor of B^{h}.

Proof. Each irreducible component Z of N_{0,h}(B, Ξ) is dominated by a regular component T of T_{h}(B, Ξ), which has dimension hg − 1 by Proposition 11.3. The map p : T → Z is finite by Corollary 11.1. The assertion follows. □

Remark 12.3. If we have two divisors Ξ_{0}, Ξ_{1} which are tangentially degenerate along an irreducible k-dimensional variety V whose general point is smooth for both Ξ_{0}, Ξ_{1}, then Ξ_{0}, Ξ_{1} are both singular along some (k−1)-dimensional variety contained in V. This can be easily proved by looking at the relation (20) in this case, and noting that the polar divisor Z_{j} is contained in f^{∗}(Sing(Ξ_{j})), j = 0, 1.

Remark 12.4. Suppose (B, Ξ) ∉ N_{g,0}. Then N_{0}(B, Ξ) is described by all differences of pairs of points of Ξ having the same Gauss image. Suppose (B, Ξ) ∈ N_{g,0} − N_{g,1} and assume {x, −x} = Sing(Ξ) have multiplicity 2 and the quadric Q_{x} = Q_{−x} is smooth. It may or may not be the case that b = −2x ∈ N_{1}(B, Ξ). In any case, we claim that N_{1}(B, Ξ)−{−2x} is contained in the set of all differences of points in γ_{Ξ}^{−1}(Q_{x}^{∗}) with x. Let us give a sketch of this assertion, which provides, in this case, a different argument for the proof of Proposition 11.6. If b ∈ N_{1}(B, Ξ) − {−2x}, there is a curve C ⊂ Ξ such that for t ∈ C general γ_{Ξ}(t) = γ_{Ξ}(t + b). Along C the divisors Ξ and Ξ_{b} are tangent, hence the curve contains x (see Remark 12.3). Note that the curve C is smooth at x. Indeed, locally at x, the divisor Ξ is a quadric cone of rank g − 1 in ℂ^{g} with vertex x, whereas Ξ_{b} is a hyperplane through x, and they can only be tangent along a line.

Thus it makes sense to consider the image of x for γ_{C}, which is a point on Q_{x}. The point x + b ∈ C + b ⊂ Ξ is smooth for Ξ and γ(x + b) is clearly tangent to Q_{x} at γ_{C}(x).

13. On N_{g−2} and N_{g−3} for Jacobians

The following result shows that the bound in Proposition 11.6 is sharp in the case h = 1. Recall that a curve C is called bielliptic if it is a double cover of an elliptic curve.

Proposition 13.1. Let C be a smooth, irreducible projective curve of genus g and let (J = J(C), Θ_{C}) be its principally polarized Jacobian. Then one has:

(i) {O_{C}(p − q) ∈ J : p, q ∈ C} ⊆ N_{g−3}(J, Θ_{C}); (ii) either C is bielliptic or the equality holds in (i); (iii) if C is hyperelliptic, then

{O_{C}(p − q) ∈ J : p, q ∈ C, h^{0}(C, O_{C}(p + q)) = 2} = N_{g−2}(J, Θ_{C}).

Proof. We begin with (i). We assume that C is not hyperelliptic; the hyperelliptic case is similar and can be left to the reader. We may identify C with its canonical image in ℙ^{g−1}. Moreover, we identify J with Pic^{g−1}(C) and Θ_{C} ⊂ Pic^{g−1}(C) with the set of effective divisor classes of degree g−1. Then the Gauss map γ_{C} := γ_{Θ_{C}} can be geometrically described as the map which sends the class of an effective divisor D of degree g − 1 such that h^{0}(C, O_{C}(D)) = 1 to the hyperplane in ℙ^{g−1} spanned by D (see [15], p. 360). Take two distinct points p_{1}, p_{2} on C. Then |ω_{C}(−p_{1} − p_{2})| is a linear series of degree 2g − 4 and dimension g − 3. For i = 1, 2 we let V_{i} be the subvariety of Θ_{C} which is the Zariski closure of the set of all divisor classes of the type E + p_{i}, where E is a divisor of degree g − 2 contained in some divisor of |ω_{C}(−p_{1} −p_{2})|. Clearly dim(V_{i}) = g −3, hence V_{i} is not contained in Sing(Θ_{C}) which is of dimension g−4. If u denotes the divisor class p_{2} −p_{1} then x ↦ x+u defines an isomorphism V_{1} ≅ V_{2}. Moreover, for x in an non-empty open subset of V_{1} we have γ_{C}(x) = γ_{C}(x + u). This proves (i). Conversely, assume there is a point u ∈ Pic^{0}(C)−{0}, and a pair of irreducible subvarieties V_{1}, V_{2} of Θ_{C} of dimension g − 3 such that x ↦ x + u gives a birational map from V_{1} to V_{2}. For x in a non-empty open subset U ⊂ V_{1} we have γ_{C}(x) = γ_{C}(x + u). If D and D′ are the effective divisors of degree g − 1 on C corresponding to x and x + u and if E is the greatest common divisor of D and D′ then by the geometric interpretation of the Gauss map γ_{C} there is an effective divisor F with deg(E) = deg(F) such that D + D′ − E + F ≡ K_{C}. Thus (2D − E) + F ≡ K_{C} − u. Consider the linear series |K_{C} − u|, which is a g_{2g−2}^{g−2}. If this linear series has a base point q, then there is a point p such that K_{C} − u − q ≡ K_{C} − p, i.e. D′ − D ≡ u ≡ p − q, proving (ii). So we may assume that |K_{C} − u| has no base point. If C is not bielliptic, then |K_{C} − u| determines a birational map of C to a curve in ℙ^{g−2}. On the other hand it contains the (g − 3)-dimensional family of divisors of the form (2D−E)+F, which are singular along the divisor D − E. This is only possible if deg(D − E) = 1, i.e., u ≡ D′ − D ≡ p − q, with p, q ∈ C. But in this case |K_{C} − u| has the base point q, a contradiction. This proves (ii). Assume now C is hyperelliptic. Let p_{1} + p_{2} be an effective divisor of the g_{2}^{1} on C with p_{1} ≠ p_{2}. Then |ω_{C}(−p_{1} − p_{2})| is a linear series of degree 2g − 4 and dimension g − 2. For i = 1, 2 we let V_{i} be the subvariety of Θ_{C} which is the Zariski closure of the set of all divisors classes of the type E + p_{i}, where E is a divisor of degree g − 2 contained in a divisor of |ω_{C}(−p_{1} − p_{2})|. The variety V_{i} has dimension g − 2 and is not contained in Sing(Θ_{C}), which is of dimension g − 3. The translation over u induces an isomorphism V_{1} ≅ V_{2} and for x in a non-empty subset U of V_{1} we have γ_{C}(x) = γ_{C}(x + u). Hence the left-hand-side in (iii) is contained in N_{g−2}(J, Θ_{C}). Finally, assume there is a point u ∈ Pic^{0}(C) − {0}, and a pair of irreducible subvarieties V, V′ of Θ_{C} of dimension g − 2 such that translation by u gives a rational map V −−→ V′ with γ_{C}(x) = γ_{C}(x + u) on a non-empty open subset of V. Let D, D′ be the effective divisors of degree g − 1 on C corresponding to x and x + u. As in the proof of part (ii), let E be the greatest common divisor of D and D′. In the present situation the linear series |K_{C} − u| of dimension g − 2 contains the (g − 2)-dimensional family of divisors of the form 2D − E, which are singular along the divisor D − E. This means that 2(D − E) is in the base locus of |K_{C} − u|. This is only possible if D = E + q for some point q ∈ C, and K_{C} − u − 2q ≡ (g − 2)(p + q), where p is conjugated to q under the hyperelliptic involution. In conclusion, we have u ≡ p − q and the equality in (iii) follows. □

Remark 13.2. The hypothesis that C is not bielliptic is essential in (ii) of Proposition 13.1. Let in fact C be a non-hyperelliptic bielliptic curve which is canonically embedded in ℙ^{g−1}. Let f : C → E be the bielliptic covering. One has f_{∗}O_{C} ≃ O_{E} ⊕ ξ^{∨}, with ξ^{⊕2} ≃ O_{E}(B), where B is the branch divisor of f. Let u ∈ Pic^{0}(E) − {0} be a general point, which we can consider as a nontrivial element in Pic^{0}(C) via the inclusion f^{∗} : Pic^{0}(E) → Pic^{0}(C) Note that f is ramified, hence f^{∗} is injective. We want to show that u ∈ N_{g−3}(J, Θ_{C}), proving that equality does not hold in (i) in this case. The canonical image of C is contained in a cone X with vertex a point v ∈ ℙ^{g−1} over the curve E embedded in ℙ^{g−2} as a curve of degree g − 1 via the linear system |ξ|. Let us consider the subvariety W of |ξ| consisting of all divisors M ∈ |ξ| such that there is a subdivisor p + q of M with p, q ∈ E and p − q ≡ u. It is easily seen that W is irreducible of dimension g − 3. Notice that for M general in W one has M = p + q + N, with N effective of degree g − 3. Therefore we may write K_{C} ≡ f^{∗}(M) ≡ f^{∗}(p) + f^{∗}(q) + F + F′, where F, F′ are disjoint, effective divisors of degree g − 3 which are exchanged by the bielliptic involution. We let V be the (g − 3)-dimensional subvariety of Θ_{C} described by all classes of divisors D of degree g − 1 on C of the form D = f^{∗}(p) + F, as M varies in W. Then D + u ≡ D′ := f^{∗}(q) + F and D and D′ span the same hyperplane through v in ℙ^{g−1}. Therefore, if x ∈ V is the point corresponding to D, one has γ_{C}(x) = γ_{C}(x + u). This proves that u ∈ N_{g−3}(J, Θ_{C}).

14. A boundary version of the Conjecture

We will now formulate a conjecture. As we will see later, it can be considered as a boundary version of Conjecture 9.1 (see also Proposition 11.6).

Conjecture 14.1. Suppose that (B, Ξ) is a simple principally polarized abelian variety of dimension g. Assume that (B, Ξ) ∉ N_{g,i} for all i ≥ k ≥ 1. Then there is an irreducible component Z of N_{k}(B, Ξ) with codim_{B}(Z) = k + 1 if and only if one of the following happens:

(i) either g ≥ 2, k = g − 2 and B is a hyperelliptic Jacobian, (ii) or g ≥ 3, k = g − 3 and B is a Jacobian.

One implication in this conjecture holds by Proposition 13.1. Note that the conjecture would give an answer for simple abelian varieties to the Schottky problem that asks for a characterization of Jacobian varieties among all principally polarized abelian varieties. For related interesting questions, see [30].

15. Semi-abelian Varieties of Torus Rank One

Let (B, Ξ) be a principally polarized abelian variety of dimension g − 1. The polarization Ξ gives rise to the isomorphism

ϕ_{Ξ} : B → B̂ = Pic^{0}(B), b → O_{B}(Ξ − Ξ_{b}).

and we shall identify B and B̂ via this isomorphism. Thus an element b ∈ B̂ ≅ B determines a line bundle L = L_{b} = O_{B}(Ξ − Ξ_{b}) with trivial first Chern class. We can associate to L a semi-abelian variety X = X_{B,b}, namely the 𝔾_{m}- bundle over B defined by L which is an algebraic group since it coincides with the theta group G_{b} := G(L) of L (cf. [25], p. 221). This gives the well-known equivalence between B̂ and Ext(B, 𝔾_{m}), the group of extension classes of B with 𝔾_{m} in the category of algebraic groups (see [35], p. 184). Both the line bundle L and the 𝔾_{m}-bundle X determine a ℙ^{1}-bundle ℙ = ℙ(L ⊕ O_{B}) over B with projection π : ℙ → B and two sections over B, say s_{0} and s_{∞} given by the projections L ⊕ O_{B} → L and L ⊕ O_{B} → O_{B}. If we set P_{0} = s_{0}(B) and P_{∞} = s_{∞}(B) then we can identify ℙ − P_{0} − P_{∞} with X. By [17], Proposition 2.6 on page 371 we have O_{ℙ}(1) ≅ O_{ℙ}(P_{0}). We can complete X by considering the non-normal variety \overline{X} = \overline{X}_{B,b} obtained by glueing P_{0} and P_{∞} by a translation over b ∈ B ≅ B̂. On ℙ we have the linear equivalence P_{0} −P_{∞} ≡ π^{−1}(Ξ−Ξ_{b}). We set E := P_{∞} +π^{−1}(Ξ) and put M_{ℙ} = O_{ℙ}(E). This line bundle restricts to O_{B}(Ξ) on P_{0} and to O_{B}(Ξ_{b}) on P_{∞}, and thus descends to a line bundle M = M\overline{X} on \overline{X}. We have π_{∗}(O_{ℙ}(E)) = O_{B}(Ξ) ⊕ O_{B}(Ξ_{b}) and H^{0}(ℙ, M_{ℙ}) is generated by two sections with divisors P_{∞} + π^{−1}(Ξ) and P_{0} + π^{−1}(Ξ_{b}). One concludes that H^{0}(\overline{X}, M) corresponds to the sections of M_{ℙ} such that translation over b carries its restriction to P_{0} to the restriction to P_{∞}. It follows that h^{0}(\overline{X}, M) = 1 with effective divisor \overline{Ξ}, which is called the generalized theta divisor on \overline{X}. Analytically we can describe a section of OX̄(\overline{Ξ}) on the universal cover ℂ×ℂ^{g−1} by a function

ξ(τ, z) + u ξ(τ, z − ω),

where ω ∈ ℂ^{g−1} represents b ∈ B = ℂ^{g−1}/ℤ^{g−1} + τℤ^{g−1}, ξ(τ, z) is Riemann’s theta function for B and u = exp(2πiζ) is the coordinate on ℂ^{∗}. This is called the generalized theta function of \overline{X}. Let D be the Weil divisor on \overline{X} that is the image of P_{0} (or, what is the same, of P_{∞}). We consider a locally free subsheaf T_{vert} of the tangent sheaf to \overline{X}, namely the dual of the sheaf Ω^{1}(log D) of rank g. If d ∈ D is a point such that on the normalization ℙ near the two preimages z_{1}, …, z_{g−1}, u and z_{1}+b, …, z_{g−1}+b, v are local coordinates such that u = 0 defines P_{0} (resp. v = 0 defines P_{∞}) with uv = 1 then T_{vert} is generated by ∂/∂z_{1}, …, ∂/∂z_{g−1}, u∂/∂u − v∂/∂v. Here z_{1}, …, z_{g−1} are coordinates on B. We interpret local sections of T_{vert} as derivations. In particular, if an effective Cartier divisor Y of \overline{X} has local equation f = 0, then for each local section ∂ of T_{vert}, the restriction to Y of ∂f is a local section of O\overline{X}(Y)/O\overline{X}. Then the subscheme Sing_{vert}(Ξ) of \overline{Ξ} is locally defined by the g equations

(21)  ∂_{i}f = 0 modulo f in O\overline{X}(\overline{Ξ})/O\overline{X}

with f = 0 a local equation of \overline{Ξ} and ∂_{i} local generators of T_{vert}. The equations for Sing_{vert}(\overline{Ξ}) on X are thus given by

\begin{matrix}ξ(τ, z) + u ξ(τ, z − ω) = & 0, \\ ξ(τ, z − ω) = & 0, \\ ∂_{i}ξ(τ, z) + u∂_{i}ξ(τ, z − ω) = & 0, (1 ≤ i ≤ g − 1). \end{matrix}

The points in Sing_{vert}(\overline{Ξ}) are of two sorts depending on whether they lie on the double locus D of \overline{X} or not. The singular points of Sing(\overline{Ξ}) on X = \overline{X} − D are the points (z, u), with u ≠ 0, which are zeros of ξ(τ, z) and ξ(τ, z − ω) and such that γ(τ, z) = −uγ(τ, z − ω). That is, geometrically, these correspond under the projection on B to the points on B where Ξ and Ξ_{b} are tangent to each other. To describe the singular points of Sing(\overline{Ξ}) on D, we consider the composition

ϕ: B ≅ P_{0} → ℙ⟶^{ν}\overline{X},

where ν is the normalisation. Then we have ϕ^{−1}(Sing_{vert}(\overline{Ξ})) = Sing(Ξ) and the same if we identify B with P_{∞}. Points of Sing_{vert}(\overline{Ξ}) determine again quadrics in ℙ^{g−1} as follows. Note that the projective space ℙ(T_{X,0}) contains a point P_{b} corresponding to the tangent space T_{𝔾_{m}} ⊂ T_{X} of the algebraic torus 𝔾_{m} at the origin. Recall that we write γ(τ, z) for the row vector

γ(τ, z) = (∂_{1}ξ, …, ∂_{g−1}ξ)(τ, z).

Then a singular point determines a, possibly indeterminate, quadric defined by the matrix

(22)  (\begin{matrix}0 & γ(τ, z − ω) \\ γ(τ, z − ω)^{t} & M \end{matrix})

with M the (g − 1) × (g − 1) matrix (∂/∂τ_{ij}ξ(τ, z) + u∂/∂τ_{ij}ξ(τ, z − ω)). Note that we have γ(τ, z) = −uγ(τ, z − ω). The quadric passes through the point P_{b}. For a point on D the quadric is a cone with vertex P_{b} over a quadric in ℙ^{g−2} given by M.

Remark 15.1. The above considerations show that a point in Sing_{vert}(\overline{Ξ}) has to be regarded as a point of multiplicity larger than 2 if the matrix (22) vanishes identically. This can happen only if z and z + ω are both singular for Ξ.

16. Standard Compactifications of Semi-Abelian Varieties

Let (B, Ξ) be a principally polarized abelian variety. We assume now that dim(B) = g − r with r ≥ 1 and extend the considerations of the previous section. The extensions of B by 𝔾_{m}^{r} are parametrized by Ext^{1}(B, 𝔾_{m}^{r}) ≅ B̂^{r}. To a point (b_{1}, …, b_{r}) ∈ B̂^{r} one associates the 𝔾_{m}^{r}-extension X = X_{b} obtained as the fibre product of theta groups G_{b_{1}} ×_{B} ⋯ ×_{B} G_{b_{r}}. One of the type of degenerations of abelian varieties that we shall encounter are special compactifications of semi-abelian varieties. We shall call them standard compactifications of torus rank r. Let b = (b_{1}, …, b_{r}) ∈ B̂^{r}. The algebraic group X = X_{b} sits in a ℙ^{1} ×_{B} ⋯ ×_{B} ℙ^{1}-bundle π : ℙ → B that is obtained as the fibre product over B of the ℙ^{1}-bundles P_{b_{i}} = ℙ(L_{b_{i}} ⊕O_{B}). The complement ℙ − X is a union of 2r divisors ∑_{i=1}^{r} Π_{0}^{(i)} + Π_{∞}^{(i)}, where Π_{0}^{(i)} (resp. Π_{∞}^{(i)}) is given by taking 0 (resp. ∞) in the i-th fibre coordinate, with projections π_{i,0}, π_{i,∞} to B. We now define a non-normal variety obtained from ℙ by glueing Π_{0}^{(i)} with Π_{∞}^{(i)} for i = 1, …, r. This identification depends on a r × r-matrix T = (t_{ij}) with entries from 𝔾_{m} such that t_{ii} = 1 and t_{ij} = t_{ji}^{−1}. Let s_{0}^{(i)} : B → P_{b_{i}} (resp. s_{∞}^{(i)}) be the zero-section (infinity section) of P_{b_{i}}. We glue the point

(β, x_{1}, …, x_{i−1}, s_{0}^{(i)}(β), x_{i+1}, …, x_{r})

on Π_{0}^{(i)} with the point

(β + b_{i}, t_{i,1}x_{1}, …, t_{i,i−1}x_{i−1}, s_{∞}^{(i)}(β), t_{i,i+1}x_{i+1}, …, t_{i,r}x_{r})

on Π_{∞}^{(i)}. We denote the resulting variety by \overline{X}. It depends on the parameters b ∈ B̂^{r} and t ∈ Mat(r × r, 𝔾_{m}). We have the linear equivalences Π_{0}^{(i)} − Π_{∞}^{(i)} ≡ π^{∗}(Ξ − Ξ_{b_{i}}). We set E = Π_{∞} + π^{∗}(Ξ) = ∑ Π_{∞}^{(i)} + π^{∗}(Ξ) and E_{i} = Π_{∞} − Π_{∞}^{(i)} and M := M_{ℙ} = O_{ℙ}(E). This line bundle restricts to O_{Π_{0}^{(i)}}(E_{i} +π_{i,0}^{∗}(Ξ)) on Π_{0}^{(i)} and to O_{Π_{∞}^{(i)}}(E_{i} +π_{i,∞}^{∗}(Ξ_{b_{i}})) on Π_{∞}^{(i)}. Thus, by the definition of the glueing, M descends to a line bundle \overline{M} := M\overline{X} on \overline{X}. We have

\begin{matrix}π_{∗}(M) = & ( ⊗_{i=1}^{r} (O_{B} ⊕ L_{i}^{−1})) ⊗ O_{B}(Ξ) \\ ≅ & ⊕_{k=1}^{r} ( ⊕_{0≤i_{1}<...<i_{k}≤r} O_{B}(Ξ_{b_{i_{1}} +...+b_{i_{k}}})). \end{matrix}

Hence we have h^{0}(ℙ, M) = 2^{r}. As in the preceding section one sees that only a 1-dimensional space of sections descends to sections of \overline{M} on \overline{X}. In terms of coordinates (ζ_{1}, …, ζ_{r}, z_{1}, …, z_{g−r}) on the universal cover of X, where (z_{1}, …, z_{g−r}) ∈ ℂ^{g−r} are coordinates on the universal cover of B, a non-zero section of \overline{M} is given by

∑_{I⊆{1,...,r}} u_{I} t_{I} ξ(τ, z − ω_{I}),

where I runs through the subsets of {1, …, r}, u_{I} = ∏_{i∈I} u_{i} with u_{i} = exp(2πζ_{i}), t_{I} = ∏_{i,j∈I,i<j} t_{ij}, b_{I} = ∑_{i∈I} b_{i} and ω_{I} ∈ ℂ^{g−r} represents b_{I} ∈ B. This is the generalized theta function of \overline{X}, whose zero locus is the generalized theta divisor \overline{Ξ} of \overline{X}. Next we look at the singular points of \overline{Ξ}. All points in \overline{Ξ}∩D are singular points of \overline{Ξ}. However, just as in the rank one case in the preceding section we will in general disregard these singularities of \overline{Ξ}, and we will only look at the so-called vertical singularities, which we are going to define now (cf. [27], §2). The locally free subsheaf T_{vert} of rank g of the tangent sheaf T\overline{X} is the dual of Ω^{1}(log D). Its pull back to ℙ is generated, in the (u, z)-coordinates, by the differential operators u_{i}∂/∂u_{i} − v_{i}∂/∂v_{i} with u_{i}v_{i} = 1 for i = 1, …, r and ∂_{j} = ∂/∂z_{j} with j = 1, …, g − r. We interpret local sections of T_{vert} as derivations as above and define the scheme Sing_{vert}(\overline{Ξ}) of vertical singular points of \overline{Ξ} as the subscheme of \overline{Ξ} defined by the equations (21) with f = 0 a local equation of \overline{Ξ} for all local sections ∂ ∈ T_{vert}. This is independent of the choice of a local equation.

Lemma 16.1. Let (\overline{X}, \overline{Ξ}) be a standard compactification of a semi-abelian variety X of torus rank r with abelian part (B, Ξ). If dim(Sing_{vert}(\overline{Ξ})) ≥ 1 then (B, Ξ) ∈ N_{g−r,0}.

Proof. The compactification \overline{X} is a stratified space and the (closed) strata are (standard) compactifications of semi-abelian extensions of (B, Ξ) of torus rank s with 0 ≤ s ≤ r. In view of the relations ∑_{I} u_{I}t_{I}∂_{i}ξ(z − ω_{I}) = 0 the vertical singularities of \overline{Ξ} correspond to points where 2^{s} translates of Ξ are tangentially degenerate. Therefore we have dim(N_{1,h}(B, Ξ)) ≥ 1 with h = 2^{s}. By Lemma 12.1 it follows that (B, Ξ) ∈ N_{g−r,0}. □

17. Semi-abelian varieties of torus rank two

In the compactification of the moduli space of principally polarized abelian varieties of dimension g we shall encounter two types of degenerations of torus rank 2. The first of these is a standard compactification introduced above and its normalization is a ℙ^{1} × ℙ^{1}-bundle over a principally polarized abelian variety of dimension g − 2. For such a standard compactification the equations for Sing_{vert}(Ξ) are given in terms of the (u, z)-coordinates by the system (we write t instead of t_{1,2}; note that t ≠ 0)

ξ(z) − tu_{1}u_{2}ξ(z − ω_{1} − ω_{2}) =0,
u_{1}ξ(z − ω_{1}) + tu_{1}u_{2}ξ(z − ω_{1} − ω_{2}) =0,
u_{2}ξ(z − ω_{2}) + tu_{1}u_{2}ξ(z − ω_{1} − ω_{2}) =0,
∂_{i}ξ(z) + u_{1}∂_{i}ξ(z − ω_{1}) + u_{2}∂_{i}ξ(z − ω_{2}) + tu_{1}u_{2}∂_{i}ξ(z − ω_{1} − ω_{2}) =0,
i = 1, …, g − 2.

From this and the analogous equations in the v-coordinates (with u_{i}v_{i} = 1) we see that the vertical singular points of \overline{Ξ} are essentially of three types:

(i) A point x ∈ D that is the image via φ : X → \overline{X} of a point in Π_{1,0}∩Π_{2,0} ≅ B, (i.e. in the (u, z) coordinates one has u_{1} = u_{2} = 0) is a vertical singular point of \overline{Ξ} if and only if it corresponds to a singular point of Ξ on Π_{1,0} ∩ Π_{2,0} ≅ B and to a singular point of Ξ_{b_{1}+b_{2}} on Π_{1,∞} ∩ Π_{2,∞} ≅ B. (ii) A point x ∈ D which is the image via φ of a point in Π_{j,0} but not of a point in Π_{3−j,0}, (i.e., in the (u, z) coordinates one has u_{i} = 0, u_{3−j} ≠ 0, for a j = 1, 2) is a vertical singular point of \overline{Ξ} if and only if

\begin{matrix}ξ(τ, z) = 0, ξ(τ, z − ω_{3−j}) = & 0, \\ ∂_{i}ξ(τ, z) + u_{3−j}∂_{i}ξ(τ, z − ω_{3−j}) = & 0, i = 1, …, g − 2. \end{matrix}

i.e. if and only if z and z − b are in Ξ and γ_{Ξ}(z) = γ_{Ξ}(z − b). (iii) A point x ∉ D, (i.e. in the (u, z) coordinates one has u_{1} ≠ 0 ≠ u_{2}) is a vertical singularity if and only if z is a singular point of the divisor H ∈ |2Ξ_{b_{1}+b_{2}}| defined by the equation

ξ(τ, z − ω_{1})ξ(τ, z − ω_{2}) = t ξ(τ, z)ξ(τ, z − ω_{1} − ω_{2}).

By the way, this occurs even in case (ii) above. Note also that, by the above equations, the existence of a vertical singularity implies that the theta divisors Ξ, Ξ_{b_{1}}, Ξ_{b_{2}} and Ξ_{b_{1}+b_{2}} are tangentially degenerate at some point x of B, i.e., z ∈ N_{0,3}(B, Ξ). We call of type (i), (ii) or (iii) the singular points of \overline{Ξ} according to whether cases (i), (ii) or case (iii) occur.

Remark 17.1. A point x in Sing_{vert}(\overline{Ξ}) again determines a quadric Q_{x} in ℙ^{g−1}. It is useful to remark that:

(a) in case (i) the quadric Q_{x} is a cone with vertex the line L_{b} := ⟨P_{b_{1}}, P_{b_{2}}⟩ given by the tangent space to the 𝔾_{m}-part over the quadric Q_{z} in ℙ^{g−3} which corresponds to the singular point z of Ξ;

(b) in case (ii), say we are at a point with u_{1} = 0, u_{2} ≠ 0. Then Q_{x} is a cone with vertex P_{b_{1}} over the quadric in the hyperplane u_{1} = 0 with matrix

(\begin{matrix}0 & −γ(τ, z)^{t} \\ −γ(τ, z) & M \end{matrix})

with γ(τ, z) = (∂_{1}ξ, …, ∂_{g−2}ξ)(τ, z) and the matrix M is given by

M = (∂/∂τ_{ij}ξ(τ, z) + u∂/∂τ_{ij}ξ(τ, z − ω))_{1≤i,j≤g−2}.

In §15 we saw that all rank 1 compactifications of 𝔾_{m}-extensions of a principally polarized abelian variety B form a compact family B̂. This is no longer the case in the higher rank case. This is where semi-abelian varieties of non-standard type come into the picture. This will depend on choices. It is good to see this in some detail in the rank 2 case. Given a principally polarized abelian variety (B, Ξ) of dimension g − 2, all standard rank 2 compactifications of (B, Ξ) are of the form (\overline{X}, \overline{Ξ}) with \overline{X} = \overline{X}_{B,b,t} with b = (b_{1}, b_{2}) ∈ B × B and t ∈ ℂ^{∗}. Thus the parameter space may be identified (up to dividing by automorphisms) with the total space of the Poincaré bundle P → B × B minus the 0-section P_{0}. It is then natural to compactify this by looking at the associated ℙ^{1}-bundle and by glueing on it the 0-section P_{0} with the infinity section P_{∞}. This in fact works and it is explained in [26], §7, and in [29]. We describe next the new objects that arise. We denote by L_{i} the line bundle associated to b_{i}, for i = 1, 2. We consider again the ℙ^{1} × ℙ^{1}-bundle P on B as in §16 and in the glueing operation described in §16, we let t = t_{12} tend to 0 (or equivalently to ∞). Letting t → ∞, one contracts Π_{1,0} and Π_{2,0} to the section A = Π_{1,∞} ∩ Π_{2,∞} ≅ B, and Π_{1,∞} and Π_{2,∞} to the section Δ = Π_{1,0} ∩ Π_{2,0} ≅ B. In order to properly describe the glueing process, we have first to blow up the two sections A and Δ in P. Let us do that. Let w : P̃ → P be the blow-up, on which we have the following divisors: α is the exceptional divisor over A and δ is the exceptional divisor over Δ; β, γ, ϵ, ζ are the proper transforms on P̃ of Π_{1,∞}, Π_{2,0}, Π_{1,0}, Π_{2,∞}, respectively. We will abuse notation and denote by the same letters the restrictions of these divisors on the general fibre Φ of P̃ over B, which is a ℙ^{1} × ℙ^{1} blown up at two points, hence a ℙ^{2} blown up at three points. Note that α, β, γ, δ, ϵ, ζ are ℙ^{1}-bundles over B and one has

(23)  \begin{matrix}α ≅ ℙ(L_{1}^{∨} ⊕ L_{2}^{∨}), & γ ≅ ℙ(L_{2} ⊕ O_{B}), & ϵ ≅ ℙ(L_{1} ⊕ O_{B}), \\ δ ≅ ℙ(L_{1} ⊕ L_{2}), & ζ ≅ ℙ((L_{1} ⊗ L_{2}) ⊕ L_{2}), & β ≅ ℙ((L_{1} ⊗ L_{2}) ⊕ L_{1}). \end{matrix}

At this point one could be tempted to suitably glue α with γ and ϵ and δ with β and ζ. This however, as (23) shows, does not work. The right construction is instead the following. One considers two ℙ^{2}-bundles ϕ_{i} : P_{i}^{♯} → B, i = 1, 2, associated to the vector bundles L_{1} ⊕ L_{2} and L_{1}^{∨} ⊕ L_{2}^{∨} on B, i.e.,

P_{1}^{♯} = ℙ(L_{1}^{∨} ⊕ L_{2}^{∨} ⊕ O_{B}), P_{2}^{♯} = ℙ(L_{1} ⊕ L_{2} ⊕ O_{B}).

There are three relevant ℙ^{1}-subbundles of the bundles P_{i}^{♯}, i = 1, 2, namely

(24)  \begin{matrix}ᾱ = ℙ(L_{1}^{∨} ⊕ L_{2}^{∨}), & γ̄ = ℙ(L_{2}^{∨} ⊕ O_{B}), & ϵ̄ = ℙ(L_{1}^{∨} ⊕ O_{B}) & in P_{1}^{♯} \\ δ̄ = ℙ(L_{1} ⊕ L_{2}), & ζ̄ = ℙ(L_{1} ⊕ O_{B}), & β̄ = ℙ(L_{2} ⊕ O_{B}) & in P_{2}^{♯}. \end{matrix}

As (23) and (24) show, we can glue P with P_{1}^{♯} and P_{2}^{♯} in such a way that α and δ are respectively glued to ᾱ and δ̄; ϵ is glued to ϵ̄ and β to β̄ with a shift by −b_{1}, and ζ to ζ̄ and γ to γ̄ with a shift by −b_{2}. The resulting variety is denoted by \overline{X} = \overline{X}_{B,b}. As usual, we will denote by D its singular locus. On P̃ we have the line bundle

M̃ = w^{∗}O_{P} (Π_{1,∞} + Π_{2,∞}) ⊗ OP̃(Ξ − α),

where we write L instead of w^{∗}(π^{∗}(L)) for a line bundle, or divisor, L on B. With similar notation, one has

(25)  \begin{aligned}M̃ ≅ OP̃(α + β + ζ + Ξ) & ≅ OP̃(δ + ϵ + ζ + Ξ_{b_{1}}) \\ & ≅ OP̃(δ + γ + β + Ξ_{b_{2}}). \end{aligned}

One has O_{P_{1}^{♯}}(1) = O_{P_{1}^{♯}}(ᾱ) and the following linear equivalences

(26)  ᾱ − γ̄ ≡ L_{2}^{∨}, ᾱ − ϵ̄ ≡ L_{1}^{∨}

where again we write L instead of ϕ_{i}^{∗}(L), i = 1, 2, for a line bundle, or divisor, L on B (see again [17], Proposition 2.6, p. 371). From (25) and (26) one deduces that M̃ glues with the line bundle M_{1}^{♯} = O_{P_{1}^{♯}}(ᾱ + Ξ) and with the line bundle

M_{2}^{♯} = O_{P_{2}^{♯}}(Ξ), to give a line bundle \overline{M} on \overline{X}. In the obvious coordinates (u, z) = ((u_{1}, u_{2}), (z_{1}, …, z_{g−2})), which can be considered as coordinates on P̃ − (α ∪ ⋯ ∪ ζ), the sections of M̃ can be expressed as

a ξ(τ, z) + a_{1}u_{1} ξ(τ, z − ω_{1}) + a_{2}u_{2} ξ(τ, z − ω_{2})

with a, a_{1}, a_{2} complex numbers. By taking into account the glueing conditions, one sees that only a 1-dimensional subspace V of H^{0}(P̃, M̃) gives rise to a space of sections of H^{0}(\overline{X}, \overline{M}); V is generated by

(27)  ξ(τ, z) + u_{1}ξ(τ, z − ω_{1}) + u_{2}ξ(τ, z − ω_{2}).

in the (u, z)-coordinates. Note that (27) is just obtained from the generalized theta function in 16 by letting t = t_{12} tend to 0. In conclusion, one has h^{0}(\overline{X}, \overline{M}) = 1, hence there is a unique effective divisor \overline{Ξ} = \overline{Ξ} which is the zero locus of a non-zero section of H^{0}(\overline{X}, \overline{M}). As in the standard case, we parametrize an open subset of Pic^{0}(\overline{Ξ}) with points in the union of P − ⋃_{i=1,2;h=1∞} Π_{i,h} with P_{1}^{♯} − (ᾱ ∪ γ̄ ∪ ϵ̄) and P_{1}^{♯} − (δ̄∪ ζ̄∪ β̄). We can define the vertical singularities of the divisor \overline{Ξ}, whose equations, in the (u, z) coordinates, take the form (28)

\begin{matrix}ξ(τ, z) = 0, u_{1}ξ(τ, z − ω_{1}) = 0, u_{2}ξ(τ, z − ω_{2}) = & 0, \\ ∂_{i}ξ(τ, z) + u_{1}∂_{i}ξ(τ, z − ω_{1}) + u_{2}∂_{i}ξ(τ, z − ω_{2}) = & 0, i = 1, …, g − 2. \end{matrix}

Again the vertical singular points of \overline{Ξ} are essentially of three types:

(i) Consider a point x ∈ D which is the image via φ : X → \overline{X} of a point in Π_{1,0} ∩ Π_{2,0} ≅ B, i.e. in the (u, z) coordinates one has u_{1} = u_{2} = 0. Then this is a vertical singular point of \overline{Ξ} if and only if it corresponds to a singular point of Ξ on Π_{1,0} ∩ Π_{2,0} ≅ B and to a singular point of Ξ_{b_{1}+b_{2}} on Π_{1,∞} ∩ Π_{2,∞} ≅ B.

(ii) Consider a point x ∈ D which is the image via φ of a point in Π_{i,0} but not of a point in Π_{3−i,0}, i.e. in the (u, z) coordinates one has u_{i} = 0, u_{3−i} ≠ 0, for an i = 1, 2. If i = 1, this is a vertical singular point of \overline{Ξ} if and only if

\begin{matrix}ξ(τ, z) = 0, ξ(τ, z − ω_{2}) = & 0, \\ ∂_{i}ξ(τ, z) + u_{2} ∂_{i}ξ(τ, z − ω_{2}) = & 0, i = 1, …, g − 2. \end{matrix}

i.e., if and only if z and z − b_{2} are in Ξ and γ_{Ξ}(z) = γ_{Ξ}(z − b_{2}). Thus points of this type correspond to points in N_{0}(B, Ξ). (iii) Consider a point x ∉ D, i.e., in the (u, z) coordinates one has u_{1} ≠ 0 ≠ u_{2}. Then equations (28) mean that z corresponds to a point in Ξ ∩ Ξ_{b_{1}} ∩ Ξ_{b_{2}} where Ξ, Ξ_{b_{1}}, Ξ_{b_{2}} are tangentially degenerate. In other words points of this type correspond to points in N_{0,2}(B, Ξ).

Remark 17.2. A point x in Sing_{vert}(\overline{Ξ}) determines a quadric Q_{x} in ℙ^{g−1}. Remark 17.1 is still valid here.

A variant of this second type of rank-2 degeneration is obtained as follows. Given a 𝔾_{m}^{2}-extension X = X_{b} of B determined by a point b = (b_{1}, b_{2}) ∈ B̂^{2} we consider two ℙ^{2}-bundles ℙ and ℙ′ over B:

ℙ = ℙ(O_{B} ⊕ L_{1} ⊕ L_{2})  and  ℙ′ = ℙ(L_{2} ⊕ L_{1} ⊕ (L_{1} ⊗ L_{2})),

where we write as before L_{i} for L_{b_{i}}. We can glue these over the common ℙ^{1}- subbundle ℙ(L_{1} ⊕ L_{2}). Then we glue the ℙ^{1}-subbundle ℙ(O_{B} ⊕ L_{1}) of ℙ with the ℙ^{1}-subbundle ℙ(L_{2} ⊕ (L_{1} ⊗ L_{2})) of ℙ′ via a shift over b_{2}. Similarly, we glue the ℙ^{1}-subbundle ℙ(O_{B} ⊕ L_{2}) of ℙ with the ℙ^{1}-subbundle ℙ(L_{1} ⊕ (L_{1} ⊗ L_{2})) of ℙ′ via a shift over b_{1}. In this way we obtain a non-normal variety over B. Both ℙ and ℙ′ come with a relatively ample bundle O_{ℙ}(1) and O_{ℙ′}(1). On ℙ we have the linear equivalences

Π_{1} + π^{∗}(Ξ_{b_{1}}) ≡ Π_{2} + π^{∗}(Ξ_{b_{2}}) ≡ Π_{3} + π^{∗}(Ξ_{b_{1}+b_{2}}).

with Π_{i} = ℙ(O_{B} ⊕ L_{i}) for i = 1, 2 and Π_{3} = ℙ(L_{1} ⊕ L_{2}). We let M be the line bundle O(Π_{3} + π^{∗}(Ξ_{b_{1}+b_{2}})) on ℙ and M′ the line bundle O(Π′_{3} + π^{∗}(Ξ_{b_{1}+b_{2}})) on ℙ′, where Π′_{3} is the bundle ℙ(L_{1} ⊕ L_{2}). This descends to a line bundle \overline{M} on \overline{X}. This line bundle has a 1-dimensional space of sections generated by

θ(τ, z) = ξ(τ, z) + u_{1} ξ(τ, z_{1} − ω_{1}) + u_{2} ξ(τ, z − ω_{2})

in suitable affine coordinates (u_{1}, u_{2}) on ℙ^{2}. Again the vertical singular points of \overline{Ξ} are essentially of three types:

(i) A point x ∈ D which is the image via φ : X → \overline{X} of a point in Π_{1} ∩ Π_{2} = ℙ(O_{B}) ≅ B is a vertical singularity if it corresponds to a singularity on Ξ, to a singularity on Ξ_{b_{1}} on Π_{1} ∩ Π_{3} and a singularity on Ξ_{b_{2}} on Π_{2} ∩ Π_{3}; (ii) A point x ∈ D which is the image via φ of a point on one Π_{3} but not of a point in Π_{1} or Π_{2} is a vertical singular point of \overline{Ξ} if and only if x ∈ Ξ_{b_{1}} ∩Ξ_{b_{2}} and γ_{Ξ_{b_{1}}} (x) = γ_{Ξ_{b_{2}}} (x). Thus points of this type correspond to points in N_{0}(B, Ξ). Something similar happens for the points on exactly one of Π_{1} or Π_{2}.

(iii) A point x ∉ D is a vertical singularity if x ∈ Ξ ∩ Ξ_{b_{1}} ∩ Ξ_{b_{2}} and Ξ, Ξ_{b_{1}}, Ξ_{b_{2}} are tangentially degenerate at x. In other words points of this type correspond to points in N_{0,2}(B, Ξ). We see that the compactification depends on a choice, but in both cases we can deal explicitly with the singularities of the theta divisors.

Remark 17.3. The variant just given corresponds to a tesselation of ℝ^{2} given by the lines x = a, y = b and x + y = c with a, b, c ∈ ℤ. To a triangle with integral vertices (n, m), (n+1, m) and (n, m+1) (resp. (n−1, m), (n, m−1) and (n, m)) we associate the ℙ^{2}-bundle ℙ(L_{nb_{1}+mb_{2}} ⊕ L_{(n+1)b_{1}+mb_{2}} ⊕ L_{nb_{1}+(m+1)b_{2}}) (resp. ℙ(L_{(n−1)b_{1}+mb_{2}} ⊕ L_{nb_{1}+(m−1)b_{2}} ⊕ L_{nb_{1}+mb_{2}}) and we glue the bundles belonging to adjacent triangles over the ℙ^{1}-bundle defined by the common edge. Then the generator τ_{1} (resp. τ_{2}) of ℤ^{2} acts by glueing the ℙ^{1}-bundle associated to an edge to the ℙ^{1}-bundle associated to the translate by x ↦ x+1 (resp. y ↦ y + 1) of this edge using a translation over b_{2} (resp. b_{1}). The quotient under ℤ^{2} is the non-normal variety we just constructed. Also the earlier compactifications thus correspond to tesselations (see [26], §7).

18. Compactification of A_{g}

In order to study the Andreotti-Mayer loci we need to compactify A_{g}. The moduli space A_{g} admits a ‘minimal’ compactification, the Satake compactification constructed by Satake and Baily-Borel in characteristic 0 and by Faltings-Chai over the integers (see [32], [13]). This compactification A_{g}^{∗} is an orbifold or stack which admits a stratification

A_{g}^{∗} = ⊔_{i=0}^{g}A_{i}

and the closure of A_{m} in A_{g}^{∗} is A_{m}^{∗} = ⊔_{i=0}^{m}A_{i}. This compactification is highly singular for g ≥ 2. Smooth compactifications can be constructed by the theory developed by Mumford and his co-workers in characteristic 0 and by Faltings-Chai in general. These compactifications depend on combinatorial data. We shall use the Voronoi compactification Ã_{g} = Ã_{g}^{Vor} as described by Namikawa, Nakamura and Alexeev (see [29, 28, 5]). This compactification is a smooth orbifold with a natural map q : Ã_{g} → A_{g}^{∗}. It has the stratification induced by that of A_{g}^{∗}: the stratum

A_{g}^{(r)} = q^{−1}(A_{g−r})

is called the stratum of quasi-abelian varieties of torus rank r. In §20 we also shall use another compactification, the perfect cone compactification, cf. [29, 34]. It also has a map to the Satake compactification denoted by q : Ã_{g}^{pc} → A_{g}^{∗} ; sometimes we shall denote Ã_{g}^{pc} simply by Ã_{g} in order to avoid introducing more notation. It has the following properties: i) the boundary is an irreducible ℚ-Cartier divisor, ii) the general point of the boundary is smooth, iii) the codimension of A_{g}^{(r)} = q^{−1}(A_{g−r}) equals r, iv) there is a dense open subset U of A_{g}^{(≤4)} and a family of compactified semi-abelian varieties X → U extending the universal compactified semi-abelian variety over A_{g}^{≤1} such that the standard compactifications of §16 form a dense open subset of A_{g}^{(r)}. We point out that in this case for r ≤ 4 the general fibre of q : A_{g}^{(r)} → A_{g−r} has dimension gr − r(r + 1)/2, thus dim(A_{g}^{(r))}) = g(g + 1)/2 − r. We define for our chosen compactification Ã_{g} the boundary as ∂Ã_{g} := A_{g}^{(≥1)}. Moreover we set A_{g}^{(≤r)} := Ã_{g} − A_{g}^{(≥r+1)}. For the Voronoi compactification the fibres of the map q : A_{g}^{(≤r)} → A_{g}^{∗} are well-behaved if r ≤ 4. Indeed, the points of A′_{g} := A_{g}^{(≤4)} correspond to socalled stable quasi-abelian varieties which are compactifications of semi-abelian varieties, which can be explicitely described (see [29, 28, 5] and §§15, 17 for torus ranks 1 and 2). Thus one can define the vertical singular locus and the Andreotti-Mayer loci on the partial compactification A′_{g} (see Remark 18.1 below). For higher torus rank the situation is more complicated. For instance the fibres of q : A_{g}^{(≥r)} → A_{g−r}^{∗} might be non-reduced. However we will not need r > 4 here. An alternative approach might be to use the idea of Alexeev and Nakamura (cf. [1], [28], [5]) who have constructed canonical limits for 1-dimensional families of abelian varieties equipped with principal theta divisors. The stable quasi-abelian varieties that occur in A′_{g} for torus rank 1 and 2 are exactly those described in Section 15 and 17. For torus rank 3 these are described by the tesselations of ℝ^{3} occuring on p. 188 of [28], cf. also Remark 17.3. The open stratum of A_{g}^{(3)} over A_{g−3} corresponds to the standard compactifications (see §16), obtained by glueing six ℙ^{3}-bundles over a (g −3)-dimensional abelian variety B generalizing the construction for torus rank 2 where two ℙ^{2}-bundles were glued. These closed strata correspond to degenerations of the matrix T of the glueing data on which the standard compactifications depend (see §16). For instance the codimension 3 stratum corresponds to the fact that in T two of the three elements above the main diagonal tend to zero (or to ∞).

Remark 18.1. As we remarked before, for stable quasi-abelian varieties corresponding to points (\overline{X}, \overline{Ξ}) of A′_{g} one can define the vertical singularities Sing_{vert}(\overline{Ξ}) using Ω^{1}(log D) as in the previous sections. One checks that for these compactifications the analogue of Lemma 16.1 still holds.

We will need the following result from [14].

Theorem 18.2. Let Z be an irreducible, closed subvariety of Ã_{g}. Then Z ∩∂Ã_{g} is not empty as soon as codimÃ_{g}(Z) < g.

19. The Andreotti–Mayer loci and the boundary

We are working with a fixed compactification Ã_{g} = Ã_{g}^{Vor} and, as indicated in §18 above, we can define the Andreotti–Mayer loci over the part A′_{g} = A_{g}^{(≤4)} of Ã_{g}. We have N_{g,k} as a subscheme of A_{g} and we define Ñ_{g,k} as the schematic closure. The support of Ñ_{g,k} contains the set of points corresponding to pairs (\overline{X}, \overline{Ξ}) such that Sing_{vert}(\overline{X}) has a component of dimension at least k (see [27]).

It is interesting to look at the case k = 0, which has been worked out by Mumford [27] and Debarre [11]. In this case Ñ_{g,0} is a divisor and by Theorem 18.2, every irreducible component N of this divisor intersects ∂Ã_{g}. Let M be an irreducible component of N ∩ ∂Ã_{g}, which has dimension \binom{g+1}{2} − 2. First of all, notice that M cannot be equal to A_{g}^{(≥2)}. This follows by the results in §17 and by Propositions 11.6 and 12.1. More generally, in the same way, one proves that M cannot contain A_{g}^{(r)} for any r = 2, 3 and 4. Hence M intersects A_{g}^{(1)} in a non-empty open set of M, i.e., the intersection with the boundary has points corresponding to semi-abelian varieties of torus rank 1. If M does not dominate A_{g−1} via the map q, then each fibre must have dimension g − 1. By Proposition 11.6 this implies that M dominates N_{0,g−1}. If M dominates A_{g−1} via q, the fibre of q_{|M} over a general point (B, Ξ) ∈ A_{g−1} is N_{0}(B, Ξ). Recall now that Debarre proves in [11] that N_{0,g} has two irreducible components, one of which is the so-called theta-null component θ_{0,g}: the general abelian variety (X, Θ_{X}) in θ_{0,g}, with Θ_{X} symmetric, is such that Θ_{X} has a unique double point which is a 2-torsion point of X lying on Θ_{X}. Let M_{0,g} be the other component. The general abelian variety (X, Θ_{X}) in M_{0,g}, with Θ_{X} symmetric, is such that Θ_{X} has exactly two double points x and −x. It is useful to recall that, by Corollary 8.12, at a general point of either one of these component of N_{g,0}, the tangent cone to the theta divisor at the singular points is a smooth quadric. The component θ_{0,g} intersects the boundary in two components, θ′_{0,g}, θ′′_{0,g}, one dominating θ_{0,g−1}, the other dominating A_{g−1} with fibre over the general point (B, Ξ) ∈ A_{g−1} given by the component 2Ξ of N_{0}(B, Ξ) (see §11). Also M_{0,g} intersects the boundary in two irreducible divisors M′_{0,g}, M′′_{0,g}. The former is irreducible and dominates M_{0,g−1}, the latter dominates A_{g−1} with fibre over the general point (B, Ξ) ∈ A_{g−1} given by the components of N_{0}(B, Ξ) different from 2Ξ. The main ingredient for Debarre’s proof of the irreducibility of M_{0,g} is a monodromy argument which implies that, if (B, Ξ) is a general principally polarized abelian variety of dimension g, then N_{0}(B, Ξ) consists of only two irreducible components.

Remark 19.1. Let (B, Θ_{B}) be a general element in θ_{0,g} and let (\overline{X}, \overline{Ξ}) be a semi-abelian variety of torus rank one with abelian part B. Then there are no points in Sing_{vert}(\overline{Ξ}) with multiplicity larger than 2. This follows from the fact that Θ_{B} has a unique singular point and by Remark 15.1.

We finish this section with the following result which will be useful later on. It uses the notion of asymptotic cone given in §6.

Proposition 19.2. One has:

(i) let g ≥ 3, let (B, Ξ) be a general point of θ_{0,g} and let x be the singular point of Ξ. Then the asymptotic cone T C_{ξ}^{(4)} is strictly contained in the quadric tangent cone Q_{x}; (ii) let g ≥ 4, let (B, Ξ) be a general point of M_{0,g} and let x, −x be the singular points of Ξ. Then the asymptotic cone T C_{ξ}^{(3)} is strictly contained in the quadric tangent cone Q_{x} = Q_{−x}.

Proof. Degenerate to the jacobian locus and apply the results from [18, 19]. □

20. The Conjecture for N_{1,g}

In this section we prove Conjecture 9.1 for k = 1. We consider an irreducible component N of Ñ_{g,1} which is of codimension 3. The first observation is that the assumption about the codimension of N implies that the generic principally polarized abelian variety is simple since by Remark 7.4 every irreducible component of A_{g}^{(ns)} has codimension ≥ g − 1 > 3 if we assume g ≥ 5.

Proposition 20.1. Let g ≥ 6 and let N be an irreducible component of Ñ_{g,1} which is of codimension 3 in Ã_{g}. Then N intersects the stratum A_{g}^{(1)}.

Proof. We begin by remarking that N cannot be complete in A_{g} in view of Theorem 18.2. Therefore N intersects ∂Ã_{g}. Here we shall use the perfect cone compactification, see §18. Since ∂Ã_{g} is a divisor in Ã_{g} it intersects N in codimension one. Let M be an irreducible component of N ∩ ∂Ã_{g}. It is our intention to prove that M has a non-empty intersection with A_{g}^{(1)}. Suppose that M ⊆ A_{g}^{(≥4)}. For dimension reasons we have M = A_{g}^{(≥4)}. Since we are using a compactification Ã_{g} such that the general point of A_{g}^{(4)} corresponds to a standard compactification (\overline{X}, \overline{Ξ}) of torus rank 4 with abelian part (B, Ξ) ∈ A_{g−4} we deduce from Lemma 16.1 and Remark 18.1 that if dim(Sing_{vert}(\overline{Ξ})) ≥ 1 then (B, Ξ) ∈ N_{g−4,0}. But for g ≥ 5 the locus N_{g−4,0} is a divisor in A_{g−4} and we obtain the inequality codim_{∂Ã_{g}}(M) ≥ codim_{∂Ã_{g}}(A_{g}^{(4)}) + 1 ≥ 5, a

contradiction. Therefore we can assume that M ∩ A_{g}^{(≤3)} ≠ ∅. Suppose that M ∩ A_{g}^{(3)} has codimension 1 in A_{g}^{(3)}. Then either M maps dominantly to Ã_{g−3} via the map q : A_{g}^{(3)} −−→ Ã_{g−3} and M intersects the general fibre F of q in a divisor, or M maps to a divisor in A_{g−3} under q with full fibres F contained in M. The former case is impossible by Proposition 12.1. In the latter case for a general (B, Ξ) in q(M) all the quasi-abelian varieties (\overline{X}, \overline{Ξ}) in the fibre F over (B, Ξ) must have a 1-dimensional vertical singular locus of \overline{Ξ}. Note that (\overline{X}, \overline{Ξ}) corresponds to a standard compactification as considered in §16. By the discussion given in §16 and by Proposition 11.6, we see that q(M) has to be contained in N_{g−3,1}, against Theorem 8.6. We thus conclude that M ∩A_{g}^{(≤2)} ≠ ∅. Suppose that M ∩ A_{g}^{(≥2)} ≠ ∅. Then M ∩ A_{g}^{(2)} has codimension 2 in A_{g}^{(2)}. As above we have that q(M) is contained in N_{g−2,0}. Suppose first q(M) is dense in a component of N_{g−2,0}. If (B, Ξ) is a general element of q(M), then M intersects the fibre of q : A_{g}^{(2)} −−→ Ã_{g−2} over (B, Ξ) in codimension one. This gives a contradiction by the analysis §17 and Proposition 11.6. Suppose that q(M) is not dense in a component of N_{g−2,0}. If (B, Ξ) is a general element of q(M), then M contains the full fibre of q : A_{g}^{(2)} → A_{g−2} over (B, Ξ). By taking into account the analysis §17 and Proposition 11.6, this implies q(M) contained in N_{g−2,1}, giving again a contradiction. This proves that M ∩ A_{g}^{(≤1)} ≠ ∅. □

From now on we use again the Voronoi compactification. Let g ≥ 4 and let N be an irreducible component of N_{g,1} of codimension 3 in A_{g}. As in the proof above we denote by M an irreducible component of the intersection of the closure of N in Ã_{g} with the boundary ∂Ã_{g}. According to Lemma 20.1 the morphism q : Ã_{g} → A_{g}^{∗} induces a rational map α : M −−→ Ã_{g−1}, whose image is not contained in ∂Ã_{g−1}.

Lemma 20.2. In the above setting, the Zariski closure of q(M) in Ã_{g−1} is:

(i) either an irreducible component N_{1} of Ñ_{g−1,1} of codimension 3 in Ã_{g−1}; (ii) or an irreducible component N_{0} of Ñ_{g−1,0} and in this case:

(a) if η = (B, Ξ) ∈ N_{0} is a general point, then the closure of q^{−1}(η) in B is an irreducible component of N_{1}(B, Ξ) of codimension 2 in B;

(b) if ξ = (\overline{X}, \overline{Ξ}) ∈ M is a general point, then Sing_{vert}(\overline{Ξ}) meets the singular locus D of \overline{X} in one or two points, whose associated quadric has corank 1.

Proof. If q(M) ⊆ N_{g−1,1}, then Theorem 8.6 implies 3 ≤ codim_{A_{g−1}}(q(M)) ≤ codim_{∂Ã_{g}}(M) = 3 and the closure of q(M) must an irreducible component of Ñ_{g−1,1}. If q(M) ⊈ N_{g−1,1} then by Proposition 12.1 we have that q(M) ⊆ N_{g−1,0} and the fibre q^{−1}(B, Ξ) ⊆ N_{1}(B, Ξ). By Proposition 11.6 we have codim_{B}(N_{1}(B, Ξ)) ≥ 2 and since N_{g−1,0} is a divisor in A_{g−1} we see that (iia) follows. The last statement (iib) follows from Remark 12.3, the analysis in Section 17, the description of N_{g,0} by Mumford and Debarre (see [27], [11], and §19) and Corollary 8.12. □

We are now ready for the proof of the conjecture for N_{g,1}.

Theorem 20.3. Let g ≥ 4. Then the codimension of an irreducible component N of N_{g,1} in A_{g} is at least 3 with equality if and only if:

(i) g = 4 and either N = H_{4} is the hyperelliptic locus or N = A_{4,(1,3)}; (ii) g = 5 and N = J_{5} is the jacobian locus.

Proof. By Theorem 8.6, the codimension of N is at least 3. Suppose that N has codimension 3. It is well-known that the assertion holds true for g = 4 and 5 (see [4], [10], [8]). We may thus assume g ≥ 6 and proceed by induction. Let ζ = (X, Θ_{X}) be a general point of N, so that X is simple (see Remark 7.4). Let S be a 1-dimensional component of Sing(Θ_{X}). We can assume that the class of S in X is a multiple mγ_{X} of the minimal class γ_{X} = Θ_{X}^{g−1}/(g − 1)! ∈ H^{2}(X, ℤ). If not so, then End(X) ≠ ℤ and this implies that codim_{A_{g}}(N) ≥ g − 1 (see Remark 7.4). By Theorem 8.6, the general point in S is a double point for Θ_{X}. We let R be the curve in S_{g} whose general point is ξ = (X, Θ_{X}, x), with x ∈ S a general point. Note that R is birationally equivalent to S. Let Q be the linear system of all quadrics in ℙ(T_{X,0}). One has the map

ϕ : ξ ∈ R −−→ Q_{ξ} ∈ Q.

As in the proof of Theorem 8.6, the map ϕ is not constant. Let Q_{R} be the span of the image of ϕ. As in the proof of Theorem 8.6, one has dim(Q_{R}) ≥ 2. By Proposition 8.3, Q_{R} is contained in the linear system N_{g,1}(ζ) (see §7 for the definition), which has dimension at most 2 since codim_{A_{g}}(N) = 3. Thus Q_{R} = N_{g,1}(ζ) has dimension 2. By Lemma 20.1, the closure of N in A_{g}^{(≤1)} has non-empty intersection with the boundary. As in the proof of Lemma 20.1, we let M be an irreducible component of the intersection of the closure of N in A_{g}^{(≤1)} with the boundary. Consider the rational map α : M −−→ A_{g−1} and the closure of the image α(M), for which we have the possibilities described in Lemma 20.2.

Claim 20.4. Possibility (i) in Lemma 20.2 does not occur.

Proof of the claim. By induction, one reduces to the case g = 6 and α(M) = J_{5}. Let (\overline{X}, \overline{Ξ}) ∈ M be a general point. Then (\overline{X}, \overline{Ξ}) is a general rank one extension of the Jacobian (J(C), Θ_{C}) of a general curve C of genus 5. Note that if x ∈ J(C) corresponds to the extension, then Θ_{C} and x + Θ_{C} are not tangentially degenerate (see [17], Thm. 10.8, p. 273). Then the analysis of §15 implies that the vertical singular locus S_{0} of \overline{Ξ} sits on the singular locus D ≅ J(C) of \overline{X} and it is isomorphic to S_{C} = Sing(Θ_{C}) with cohomology class Θ^{4}/12 (see [3]). Thus \overline{Ξ} · S_{0} = Θ_{C} · S_{C} = 10. Hence, if ζ = (X, Θ_{X}) is a general point of N, then Sing(Θ_{X}) is a curve S such that Θ_{X} · S = 10. On the other hand S is homologous to mγ_{X} and one has 10 = mΘ_{X} · γ_{X} = 6m, a contradiction. □

Claim 20.4 shows that only possibility (ii) in Lemma 20.2 can occur. In particular, by (iib) of Lemma 20.2, for ξ = (X, Θ_{X}, x) general in R, the quadric Q_{ξ} has corank 1. Let v_{ξ} ∈ ℙ^{g−1} be the vertex of Q_{ξ}. Remember that R is birational to S. Hence, by Proposition 4.4, the map

γ : ξ ∈ R −−→ v_{ξ} ∈ ℙ^{g−1}

can be regarded as the Gauss map γ_{S} of S.

Claim 20.5. If the general quadric in the linear system Q_{R} is singular, then for ξ = (X, Θ_{X}, x) general in R, the vertex v_{ξ} of Q_{ξ} is contained in the asymptotic cone T C_{ξ}^{(4)}.

Proof of the claim. Suppose the general quadric in Q_{R} is singular. Then the general quadric in Q_{R} has corank 1 (see Lemma 20.2, (iib)) and, by Bertini’s theorem, its vertex lies in the base locus of Q_{R}. In particular, for ξ = (X, Θ_{X}, x) general in R, the vertex v_{ξ} of the quadric Q_{ξ} lies in all the quadrics of Q_{R}. Choose a local parametrization x = x(t) of S around a general point of it, with t varying in a disc Δ. Then ξ(t) = (X, Θ_{X}, x(t)) ∈ R and we set Q_{ξ(t)} := Q_{t}, its equation being

∑_{ij} ∂_{i}∂_{j}θ(x(t))z_{i}z_{j} = 0,

where we set θ(z) := θ(τ, z) for the theta function of X. The main remark is that all the subsequent derivatives of Q_{t} with respect to t lie in Q_{R} and actually Q_{t} and its first two derivatives Q′_{t} and Q′′_{t} span Q_{R}, because dim(Q_{R}) = 2. Hence Bertini’s theorem implies that x′ := x′(s) sits on all these quadrics for t and s general in Δ. The equations of Q′_{t} and Q′′_{t} are respectively

∑_{ijh} ∂_{i}∂_{j}∂_{h}θ(x(t))x′_{h}(t)z_{i}z_{j} = 0

∑_{ijhk} ∂_{i}∂_{j}∂_{h}∂_{k}θ(x(t))x′_{h}(t)x′_{k}(t)z_{i}z_{j} + ∑_{ijh} ∂_{i}∂_{j}∂_{h}θ(x(t))x′′_{h}(t)z_{i}z_{j} = 0

Thus we have the relations

(29)

\begin{matrix}∑_{ij} ∂_{i}∂_{j}θ(x(t))x′_{i}(s)x′_{j}(t) = 0,  ∑_{ijh} ∂_{i}∂_{j}∂_{h}θ(x(t))x′_{h}(t)x′_{i}(s)x′_{j}(s) = 0 \\ ∑_{ijhk} ∂_{i}∂_{j}∂_{h}∂_{k}θ(x(t))x′_{h}(t)x′_{k}(t)x′_{i}(s)x′_{j}(s) +∑_{ijh} ∂_{i}∂_{j}∂_{h}θ(x(t))x′′_{h}(t)x′_{i}(s)x′_{j}(s) = 0 \end{matrix}

identically in s, t ∈ Δ. The first of these relations says that the tangent hyperplane to Q_{t} at x′(t) contains the vertex x′(s). From the second relation we have

(30)  ∑_{ijh} ∂_{i}∂_{j}∂_{h}θ(x(t))x′_{h}(t)x′_{i}(t)x′_{j}(t) = 0

which shows that v_{ξ} is contained in the asymptotic cone T C_{ξ}^{(3)}. By differentiating (30), one finds

∑_{ijhk} ∂_{i}∂_{j}∂_{h}∂_{k}θ(x(t))x′_{h}(t)x′_{k}(t)x′_{i}(t)x′_{j}(t) + 3 ∑_{ijh} ∂_{i}∂_{j}∂_{h}θ(x(t))x′′_{h}(t)x′_{i}(t)x′_{j}(t) = 0.

By comparing with (29) for s = t, we deduce that

∑_{ijhk} ∂_{i}∂_{j}∂_{h}∂_{k}θ(x(t))x′_{h}(t)x′_{k}(t)x′_{i}(t)x′_{j}(t) = 0

which proves that v_{ξ} ∈ T C_{ξ}^{(4)}. □

The crucial step in our proof is the following claim.

Claim 20.6. The general quadric in the linear system Q_{R} is non singular.

Proof of the claim. Suppose this is not the case. Again we consider an irreducible component M of N̄ ∩ ∂Ã′_{g} and let (\overline{X}, \overline{Ξ}) be a general point of M. By Claim 20.4 and Lemma 20.2, (\overline{X}, \overline{Ξ}) is a rank 1 extension of (B, Ξ) corresponding to a general point in a component of N_{g−1,0}, with extension datum b varying in a codimension 2 component of N_{1}(B, Ξ). We let S_{0} be the vertical singular locus of \overline{Ξ}. By the analysis of §15, this corresponds to a contact curve C := C_{b} of Ξ with Ξ_{b}, which contains the singular points of both Ξ and Ξ_{b} (see Remark 12.3). This means that we have a point on S_{0}, corresponding to a singular point x of Ξ, where the tangent cone is the cone over Q_{x} with vertex the point of ℙ^{g−1} corresponding to b (see S 15). Now we note that C is smooth at x. Indeed locally around x, the divisor Ξ looks like a quadric cone of corank 1 in ℙ^{g−2} and Ξ_{b} looks like a hyperplane, which touches it along a curve. This implies that C locally at x looks like a line along which a hyperplane touches a quadric cone of corank 1 (see Remark 12.4). Hence, the Gauss image x_{b} := γ_{C_{b}}(x) lies in Q_{x} and actually, by Claim 20.5, the point x_{b} lies in the asymptotic cone T C_{x}^{(4)}. The Gauss map γ_{Ξ} : Ξ −−→ ℙ^{g−2} of Ξ has an indeterminacy point at x and −x, which can be resolved by blowing up x and −x, since we may assume Ξ to be symmetric. Let p : Ξ̃ → Ξ be the blow-up and let γ̃_{Ξ} be the morphism which coincides with γ_{Ξ} ∘ p : Ξ̃ −−→ ℙ^{g−2} on an open subset. The exceptional divisor at x and −x is isomorphic to Q_{x} and γ̃_{Ξ}(x_{b}) is the tangent hyperplane to Q_{x} at x_{b}. The tangency property of Ξ and Ξ_{b} along C implies that the tangent hyperplane to Q_{x} at x_{b} coincides with γ_{Ξ_{b}}(x). Now we let b vary in a component Z of N_{1}(B, Ξ) of dimension g − 3, so that we have a rational map

f : Z −−→ Q_{x}, b ↦ x_{b}.

Note that Q_{x} also has dimension g − 3 and we claim that f has finite fibres, hence it is dominant. If not, we would have an irreducible curve Γ in Z such that for all b ∈ Γ, x_{b} stays fixed. But then for the general b ∈ Γ, the divisor Ξ_{b} has a fixed tangent hyperplane at x, a contradiction by Lemma 11.1. On the other hand, by Claim 20.5 and by part (i) of Proposition 19.2, one has that f cannot be dominant, a contradiction. □

By Lemma 20.6, the curve ϕ(R) := Σ is an irreducible component of the discriminant Δ ⊂ Q_{R} of singular quadrics in Q_{R} ≅ ℙ^{2}. Note that Δ has degree g in ℙ^{2}. The map ϕ has degree at least 2 since we may assume Θ_{X} to be symmetric, hence it factors through the multiplication by −1_{X} on X.

Claim 20.7. The map ϕ : R → Σ has degree 2.

Proof. Let d ≥ 2 be the degree of the map. Then for ξ = (X, Θ_{X}, x) general in R, we have distinct points ξ_{i} = (X, Θ_{X}, x_{i}) ∈ R, with ξ = ξ_{1}, such that all the quadrics Q_{ξ_{i}}, i = 1, …, d, coincide with the quadric Q = Q_{ξ}. If for i = 1, …, d we let η_{i} be the tangent vector to S at x_{i}, we have that η_{1} = ⋯ = η_{d} and ∂_{η_{i}}Q_{ξ_{i}}, for i = 1, …, d, coincide with the quadric Q_{1} = ∂_{η_{1}}Q, which is linearly independent from Q. The analysis we made in §3 shows that S_{g} is smooth at each of the points ξ_{i}, i = 1, …, d, and the tangent space there is determined by Q and Q_{η_{1}}. This implies that a general deformation of ξ = (X, Θ_{X}, x) inside S_{g} carries with it a deformation of each of the points ξ_{i} (i = 1, …, d) in S_{g}, because the involved quadrics are the same at these points. This yields that a general element of an irreducible component of N_{g,0} containing N has at least d singular points. By Debarre’s result in [11] (see §19), one has d = 2, proving our claim. □

Claim 20.8. The map ϕ is a morphism.

Proof. To prove the claim, it suffices to show that N is not contained in N_{g,0,r}, with r ≥ 3. In order to prove this, one verifies that, for a general point (\overline{X}, \overline{Ξ}) in a component M of the intersection of N with the boundary, there are no points of multiplicity r ≥ 3 in Sing_{vert}(\overline{Ξ}). Recall that, by Proposition 20.1, we may assume that (\overline{X}, \overline{Ξ}) is a semi-abelian variety of torus rank 1, with abelian part η = (B, Ξ). Moreover, by Claim 20.4, only case (ii) of Lemma 20.2 can occur. Therefore we may assume that η is either a general point of θ_{0,g−1} or a general point of M_{0,g−1} and the extension corresponds to a general point in an irreducible component of N_{1}(B, Ξ) which has dimension g − 3 > 0. By Remark 15.1 we see that no triple points can occur on Sing_{vert}(\overline{Ξ}). □

Note now that the morphism ϕ is defined on R ≅ S by sections of O_{S}(Θ_{X}), since the points of S verify the equations (2) and, if ξ = (X, Θ_{X}, x) ∈ R, the entries of the matrix of Q_{ξ} are ∂_{i}∂_{j}θ(τ, z), where z corresponds to x. We deduce from deg(Δ) = g and from Claim 20.7, that

(31)  S · Θ_{X} ≤ 2g.

As we assumed at the beginning of the proof, the class of S in X is a multiple mγ_{X} of the minimal class. In view of (31), we find m ≤ 2. The Matsusaka-Ran criterion [31] and a result of Welters [40] imply that (X, Θ_{X}) is either a Jacobian or a Prym variety or depends on less than 3g parameters. Since g ≥ 6 this is not possible in view of the dimensions. This ends the proof. □

Remark 20.9. A. Verra communicated to us an interesting example of an irreducible component M of codimension 6 of N_{6,1} contained in the Prym locus. We briefly sketch, without entering in any detail, its construction and properties. Let C be the normalization of a general curve of type (4, 4) on ℙ^{1} × ℙ^{1} with two nodes on a line of type (0, 1), so that C has genus g = 7. Let d, t be the linear series formed by pull-back divisors on C of the rulings of type (1, 0), (0, 1) respectively. Consider a non trivial, unramified double cover f : C̃ → C of C and let (P, Ξ) be the corresponding Prym variety. Then Ξ has a 1-dimensional unstable singular locus R (see [23]), homologically equivalent to twice the minimal class, described by all classes in Pic^{12}(C̃) of divisors of the form f^{∗}(d) + M, with f_{∗}(M) ∈ t. One proves that the map ϕ described in the proof of Theorem 20.3 sends R to a plane sextic of genus 7 which is tetragonally associated to C (see [23] for the tetragonal construction). The divisor Ξ has 24 further isolated singular points, which are pairwise exchanged by the multiplication by −1. One shows that the corresponding 12 tangent cones span the same linear system Q of dimension 2 spanned by ϕ(R). The linear system Q is the tangent space to the Prym locus P_{6} at (P, Ξ), which is therefore a smooth point for P_{6}. By contrast, M is a non-reduced component of N_{6,1} of codimension 6 such that the projectivized normal space Q at a general point has dimension 2 rather than 5. This shows that the hypotheses of Theorem 20.3 cannot be relaxed by assuming only that the projectivized normal space to M at a general point has dimension 2.

21. Appendix: A Result on Pencils of Quadrics

One of the ingredients of the proof Theorem 8.6 is a classical result of Corrado Segre from [33] on pencils of quadrics. First we recall the following:

Proposition 21.1. Let L be a pencil of quadrics in ℙ^{n} with n ≥ 1 whose general member is smooth. Then:

(i) the number of singular quadrics Q ∈ L is n+1, where each such quadric Q has to be counted with a suitable multiplicity μ(Q) ≥ n + 1 − rk(Q); (ii) for a singular quadric Q ∈ L one has μ(Q) ≥ 2 if and only if either rk(Q) < n or the singular point of Q is also a base point of L;

(iii) for a singular quadric Q ∈ L with rank n one has μ(Q) = 2 if and only if any other quadric Q′ ∈ L is smooth at p and the tangent hyperplane to Q′ at p is not tangent to Q along a line.

Proof. Consider the linear system Q_{n} of dimension n(n + 3)/2 all quadrics in ℙ^{n}. Inside Q_{n} we have the discriminant locus Δ_{n} of singular quadrics, which is a hypersurface of degree n + 1, defined by setting the determinant of a general quadric equal to zero. The differentiation rule for determinants implies that the locus Δ_{n,r} of quadrics of rank r < n + 1 has multiplicity n + 1 − r for Δ_{n}. By intersecting Δ_{n} with the line corresponding to L we have (i). As for assertion (ii), we may assume rk(Q) = n, so that Q has a unique double point p, which we may suppose to be the point (1, 0, …, 0). Thus the matrix of Q is of the form

(\begin{matrix}0 & 0_{n} \\ 0_{n}^{t} & A \end{matrix} )

where 0_{n} ∈ ℂ^{n} is the zero vector and A is a symmetric matrix of order n and maximal rank. Let Q′ be another quadric in L, with matrix

(\begin{matrix}β & b \\ b^{t} & B \end{matrix} )

with β ∈ ℂ, b = (b_{1}, …, b_{n}) ∈ ℂ^{n} and B is a symmetrix matrix of order n. By intersecting L with Δ_{n}, we find the equation

(32)  det (\begin{matrix}tβ & tb \\ tb^{t} & A + tB \end{matrix} ) = 0.

The constant term in the left-hand-side is 0. The coefficient of the linear term is

det (\begin{matrix}β & b \\ 0_{n}^{t} & A \end{matrix} ) = β det(A)

which proves (ii). Let us prove (iii). Suppose rk(Q) = n, so that Q has a unique double point p, which is a base point of L. Again we may suppose p is the point (1 : 0 : …: 0) and we can keep the above notation and continue the above analysis. The left-hand-side in (32) is

t^{2} det (\begin{matrix}0 & b \\ b^{t} & A + tB \end{matrix} ) = 0

hence the coefficient of the third order term is

(33)  det (\begin{matrix}0 & b \\ b^{t} & A \end{matrix} ) .

One has μ(Q) = 2 if and only if this determinant is not zero, hence b ≠ 0, which is equivalent to saying that all quadrics in the pencil different from Q are smooth at p. Note that there is a vector c = (c_{1}, …, c_{n}) ∈ ℂ^{n} such that b = c · A. Now the determinant in (33) vanishes if and only if c · b^{t} = 0, i.e. c · A · c^{t} = 0. This means that the line L joining p with (0 : c_{1} : …: c_{n}) sits on Q and that the tangent hyperplane to Q′ at p, which has equation b_{1}x_{1} + · + b_{n}x_{n} = 0, is tangent to Q along r. □

Next we prove Segre’s theorem.

Theorem 21.2. Let L be a linear pencil of singular quadrics in ℙ^{n} with n ≥ 2 whose general member Q has rank n + 1 − r, i.e. Vert(Q) ≅ ℙ^{r−1}. We assume that Vert(Q) is not constant when Q varies in L with rank n + 1 − r. Then:

(i) the Zariski closure

V_{L} = \overline{( ⋃_{Q∈L, rk(Q)=n+1−r} Vert(Q))}

is a variety of dimension r spanning a linear subspace Π of dimension m in ℙ^{n} with r ≤ m ≤ (n + r − 1)/2;

(ii) V_{L} is a variety of minimal degree m − r + 1 in Π; (iii) if

dim( ⋂_{Q∈L, rk(Q)=n+1−r} Vert(Q)) = s

then r ≤ (n + 2s + 3)/3; (iv) the number of quadrics Q ∈ L of rank rk(Q) < n + 1 − r is n + r −

2m − 1 ≤ n − r − 1, where each such quadric Q has to be counted with a suitable multiplicity ν(Q) ≥ n + 1 − r − rk(Q).

Proof. We start with the proof of part (i). Notice that, by iteratedly restricting to a general hyperplane, we can reduce to the case r = 1. In this case V_{L} is a rational curve which, by Bertini’s theorem, is contained in the base locus B of L. Let p, q be general points on it and let L be the line joining them. There is a quadric Q_{p} ∈ L with vertex at p. Hence Q_{p} contains L. Similarly there is a different quadric Q_{q} with vertex at q, and it also contains L. Since Q_{p} and Q_{q} span L, we see that L is contained in B, i.e., the secant variety to V_{L} is contained in B. Take now three general points p, q, r on V_{L}. Since the lines pq, pr, qr are contained in B also the plane spanned by p, q, r is contained in B. Continuing this way, we see that Π = ⟨V_{L}⟩ is contained in B. Since the general quadric in L has rank n (recall we are assuming r = 1 now), the maximal dimension of subspaces on it is n/2. Thus dim(Π) ≤ n/2 which proves part (i). Also for (ii) we can reduce ourselves to the case r = 1, in which we have to prove that V_{L} is a rational normal curve in Π = ⟨V_{L}⟩. Set dim(Π) = m. Let p ∈ V_{L} be a general point. The polar hyperplane π_{p} of p with respect to Q ∈ L does not depend on Q, since there is a quadric in L which is singular at p (see the proof of Proposition 21.1). Note that π_{p} has to contain all vertices of the quadrics in L, hence it contains Π. By the linearity of polarity, we have that polarity with respect to all quadrics in L is constant along Π and for a general point x ∈ Π, the polar hyperplane π_{x} with respect to all quadrics in L contains Π. Furthermore the linear system of hyperplanes P = {π_{x}}_{x∈Π} has dimension m − 1. Now, let p ∈ V_{L} be a general point and let Q_{p} be the unique quadric in L with a double point at p. We denote by Star(p) the ℙ^{m−1} of all lines in Π containing p. Let π ∈ P be a general hyperplane, which is tangent to Q_{p} along a line L containing p. Moreover L sits in Π, because this is the case if π = π_{q} with q another general point on V_{L}, in which case L is the line ⟨p, q⟩. Thus we have a linear map ϕ_{p} : P → Star(p), which is clearly injective and therefore an isomorphism. Fix now another general point q on V_{L}. The two maps ϕ_{p} and ϕ_{q} determine a linear isomorphism ϕ : Star(p) → Star(q). Note that L meets ϕ(L) if and only if L ∩ ϕ(L) is a point of V_{L}. This implies that V_{L} is a rational normal curve in Π, proving part (ii). Let us prove part (iii). It suffices to prove the assertion if s = −1. The variety V_{L} is swept out by a 1-dimensional family of projective spaces of dimension r − 1, i.e., the vertices of the quadrics in L. Under the assumption s = −1 no two of these vertices can intersect. Thus we must have 2(r − 1) < m. Using part (i), the assertion follows. Finally we come to part (iv). Let us restrict L to a general subspace Λ of dimension n − r. We get a pencil L̄ of quadrics in Λ whose general member is smooth. We get a singular quadric in L̄ when we intersect Λ with a quadric in L whose vertex intersects Λ. We claim that this is the only possibility for getting a singular quadric in L̄. Indeed, let Q ∈ L and suppose that its intersection Q̄ ∈ L̄ with Λ is singular at p ∈ Λ, but Q is not singular at p. Then Λ is tangent to Q at p and therefore also intersects the vertex of Q. In conclusion we have only two possibilities for getting singular quadrics in L̄:

(a) there is quadric of rank n + 1 − r in L whose vertex intersects Λ;

(b) there is quadric of rank n + 1 − h < n + 1 − r in L giving rise to a quadric of rank n − h in L̄.

Case (a) occurs as many times as the degree of V_{L}, that is, m − r + 1 times. According to part (ii) of Proposition 21.1, each quadric Q in case (a) contributes with multiplicity at least 2 in the counting of singular quadrics in L̄. We claim that, because of the generality of Λ, this multiplicity is exactly 2. To prove this, by part (iii) of Proposition 21.1, we will prove that for each quadric Q in case (a), with vertex p ∈ Λ and for any other quadric Q′ ∈ L, the tangent hyperplane π to Q′ at p is not tangent to Q along a line contained in Λ. To see this we can, by first cutting with a general subspace of dimension n − r + 1 through Λ, reduce ourselves to the case r = 1, in which V_{L} is a rational normal curve. Choose then a general point q ∈ V_{L} and let Q′ = Q_{q} be the unique quadric in L with a double point at q. The hyperplane π is tangent to Q_{q} along the line ⟨p, q⟩. This implies that π is tangent to Q only along the tangent line L_{p} to V_{L} at p (see the proof of part ii)). By the generality assumption, Λ is not tangent to V_{L} at p. Thus the assertion follows. As for quadrics in case (b), again by part (i) of Proposition 21.1, each such quadric contributes to the same count with multiplicity h − r. Since, by part (i) of Proposition 21.1, the number of singular quadrics in L̄, counted with appropriate multiplicity, is n − r + 1, the assertion follows. □

One has the following consequence:

Corollary 21.3. Let L be a linear pencil of quadrics in ℙ^{n} with n ≥ 2. Then the general member Q ∈ L has rank n + 1 − r (i.e., Vert(Q) ≅ ℙ^{r−1}) if and only if the base locus of L contains a linear subspace Π of dimension m with r ≤ m ≤ (n + r − 1)/2, along which all the quadrics in L have a common tangent subspace of dimension n + r − m − 1. In this case Π is the span of the variety V_{L}.

Proof. As usual it suffices to prove the assertion for r = 1. If the general quadric Q ∈ L has rank rk(Q) = n, the assertion follows from the proof of Theorem 21.2. The converse is trivial, since a smooth quadric in ℙ^{n} has a tangent subspace of dimension n − m − 1, and not larger, along a subspace of dimension m. □

These results imply the existence of canonical forms for pencils of singular quadrics, originally due to Weierstrass [38] and Kronecker [21]. This is explained in some detail in [33], §§20-25, and we will not dwell on this here. It would be desirable to have an extension of the results in this Appendix to higher-dimensional linear families of quadrics.

References

[1] V. Alexeev, I. Nakamura: On Mumford’s construction of degenerating abelian varieties. Tohoku Math. J. 51 (1999), 399–420.

[2] A. Andreotti, A.L. Mayer: On period relations for abelian integrals on algebraic curves. Ann. Sc. Norm. Pisa 3 (1967), p. 189–238.

[3] A. Arbarello, M. Cornalba, Ph. Griffiths, J. Harris: Geometry of Algebraic Curves, vol. I, Grundlehren der math. Wissenschaft, Springer Verlag, 267, 1985.

[4] A. Beauville: Prym varieties and the Schottky problem. Invent. Math. 41 (1977), p. 149–196.

[5] M. Brion: Compactification de l’espace des modules des variétés abéliennes principalement polarisées [d’après V. Alexeev] Séminaire Bourbaki 2005.

[6] S. Casalaina-Martin: Prym varieties and the Schottky problem for cubic threefolds. ArXiv: math.AG/0605666.

[7] C. Ciliberto, G. van der Geer: Subvarieties of the moduli space of curves parametrizing jacobians with non trivial endomorphisms. Amer. J. Math., 114 (1991), 551–570.

[8] C. Ciliberto, G. van der Geer: The moduli space of Abelian varieties and the singularities of the theta divisor. In: Yau, S.T. (ed.), Surveys in differential geometry. Papers dedicated to Atiyah, Bott, Hirzebruch and Singer. Somerville, MA: International Press. Surv. Differ. Geom., Suppl. J. Differ. Geom. 7, 61-81 (2000).

[9] O. Debarre: S_{g} est Cohen–Macaulay. Private communication, unpublished.

[10] O. Debarre: Sur les variétés abéliennes dont le diviseur thêta est singulier en codimension 3. Duke Math. J.,25 (1988), 221–273.

[11] O. Debarre: Le lieu des variétés abéliennes dont le diviseur thêta est singulier a deux composantes. Ann. Sci. École Norm. Sup., 25 (1992), p. 687–707.

[12] L. Ein, R. Lazarsfeld: Singularities of theta divisors, and the birational geometry of irregular varieties. J. Am. Math. Soc. 10 (1997), p. 243-258.

[13] G. Faltings, C. L. Chai: Degeneration of abelian varieties. Ergebnisse der Math., 3 Folge 22, Berlin, Springer–Verlag, 1990.

[14] G. van der Geer: Cycles on the moduli space of abelian varieties. In: Moduli of Curves and Abelian Varieties. C. Faber, E. Looijenga, Ed., Aspects of Math. 33 (1999) Vieweg.

[15] Ph. Griffiths, J. Harris: Principles of algebraic geometry, A. Wiley, 1978.

[16] S. Grushevsky, R. Salvati Manni: Jacobians with a vanishing theta–null

in genus 4. ArXiv: arXiv:math. AG/0505160.

[17] R. Hartshorne: Algebraic Geometry. Graduate texts in Math., 52, Springer Verlag, 1977.

[18] G. Kempf: A Torelli theorem for osculating cones to the theta divisor. International Centre for Theoretical Physics, Trieste; Course on Riemann Surfaces, 1987.

[19] G. Kempf, F. O. Schreier: A Torelli theorem for osculating cones to the theta divisor. Compositio Math., 67 (1988), 343–353.

[20] J. Kollár: Shafarevich maps and automorphic forms. Princeton Univ. Press, (1995).

[21] L. Kronecker: Ueber Schaaren von quadratischer Formen. Monatsber. Akad. Wiss. Berlin, (1868), 339–346.

[22] E. Kunz: Holomorphe Differentialformen auf algebraischen Varietäten mit Singularitäten. I. Manuscr. Math. 15 (1975), 91–108.

[23] H. Lange, Ch. Birkenhake: Complex Abelian Varieties. Grundlehren der math. Wiss., 302, Springer–Verlag, 1992.

[24] J. Lipman: Dualizing sheaves, differentials and residues on algebraic varieties. Astérisque 117, (1984).

[25] D. Mumford: Abelian Varieties. Studies in Mathematics, 5. Tata Institute of Fundamental Research, Bombay. Oxford etc.: Oxford University Press. XII, 279 p.

[26] D. Mumford: An analytic construction of degenerating abelian varieties. Comp. Math. 24 (1972), 239–272.

[27] D. Mumford: On the Kodaira dimension of the Siegel modular variety. In: Algebraic geometry–open problems (Ravello, 1982), 348–375, Lecture Notes in Math., 997, Springer, Berlin, 1983.

[28] I. Nakamura: On moduli of stable quasi-abelian varieties. Nagoya Math. J. 58 (1975), 149–214.

[29] Y. Namikawa: A new compactification of the Siegel space and degeneration of abelian varieties. Math. Ann., I, II, 221 (1976), 97–141, 201–241.

[30] G. Pareschi, M. Popa: Generic vanishing and minimal cohomology classes on abelian varieties. ArXiv: math. AG/0610166. Math. Ann. 340 (2008), p. 209–222.

[31] Z. Ran: On subvarieties of abelian varieties. Inventiones Math., 62 (1981), 459–479.

[32] I. Satake: On the compactification of the Siegel space. J. Indian math. Soc., 20 (1956), 259–281.

[33] C. Segre: Ricerche sui fasci di coni quadrici in uno spazio lineare qualunque. Atti della R. Accademia delle Scienze di Torino XIX (1883/4), p. 692-710.

[34] N. Shepherd-Barron: Perfect forms and the moduli space of abelian varieties. Invent. Math. 163 (2006), p. 25–45.

[35] J. P. Serre: Algebraic Groups and Class Fields. Graduate texts in Math 117, Springer–Verlag, 1988.

[36] R. Smith, R. Varley: Components of the locus of singular theta divisor of genus 5. Springer Lecture Notes in Math., Proceedings of the Conference in Algebraic Geometry, Sitges, 1983, 1124, (1985), 338–416.

[37] R. Smith, R. Varley: Multiplicity g points on theta divisors. Duke Math. J. 82 (1996), p. 319–326.

[38] K. Weierstrass: Zur Theorie der bilineaeren und quadratischen Formen. Monatsber. Akad. Wiss. Berlin, (1868), 310–338.

[39] G. Welters: Polarized abelian varieties and the Heat Equation. Compositio Math. 49 (1983), no. 2, 173–194.

[40] G. Welters: Curves of twice the minimal class on principally polarized abelian varieties. Indag. Math., 94 (1987), 87–109.

[41] F. Zak: Determinants of projective varieties., in Algebraic transformation groups and algebraic varieties, V. L. Popov (ed.), Encyclopedia of Math. Invariant theory and transformation groups III, Springer Verlag, 207–238 (2004).