On the Entropy of Curves

Michael Maurice Dodson University of York Heslington York YO10 5DD United Kingdom

                    mmd1@york.ac.uk

Michel Mendès France Université Bordeaux I 351 cours de la Libération 33405 Talence Cedex F-33405 France

        Michel.Mendes-France@math.u-bordeaux1.fr

To Jean-Paul Allouche for his 60th birthday

Abstract

Using geometric probability, we apply the formal definitions of Shannon entropy and Rényi’s generalization to study the complexity of planar curves of finite length within a convex set. The bounds for the Shannon and Rényi entropies depend on the arc length of the curve and on that of the boundary of the convex set; they involve a Gibbs distribution and a power law distribution, respectively. We also obtain explicit formulae for the two entropies and determine convex sets that maximize the entropy of curves.

1 Introduction

Planar curves range from a simple straight line segment or a circle to a complicated tangle. Their complexity has been analysed using both the thermodynamic concept of Shannon entropy [9, 10] and the geometrical concept of dimension [7, 8]. Examples can be found in

x 0 y ω  D ρ  (ω, ρ)

Figure 1: A straight line D determined by polar coordinates (ω, ρ)

various other domains [1, 2, 3, 4, 5, 6, 11, 12]. A brief overview of the fundamental ideas is given and the Shannon and the more general Shannon-Rényi entropies are defined. The question of bounds for these entropies is considered; the analysis of the general Rényi entropy turns out to be quite different from that of the Shannon entropy and involves a modified Hurwitz zeta function. A more precise viewpoint which gives exact formulae for the entropies is then developed.

2 Geometric probability

Given a plane curve Γ of finite length |Γ|, we recall the classical definition of the probability that a straight line D intersects Γ in exactly n points [9, 15]. In the plane, a straight line D that does not pass through the origin O is determined by the polar coordinates (ω, ρ), where ω, 0 ⩽ ω < 2π, is the angle the normal to D makes with the x-axis and ρ > 0 is the distance from O. Straight lines that pass through the origin are not counted since, as we will see, the family of such lines has measure 0.

In the (ω, ρ) plane, a point in the strip S := [0, 2π) × (0, ∞) represents a straight line not passing through the origin in the usual (x, y)-plane. The measure μ in the strip is defined by

dμ := dρ dω,

i.e., the usual Lebesgue measure. Now let K be a bounded convex set in the (x, y)-plane and let F (K) be the set of straight lines that intersect K. The μ-measure of F (K) is known to be equal to the length |∂K| of the boundary ∂K of K, i.e.,

μ(F (K)) = |∂K|.

It follows that the measure given by

dp = \frac{dμ}{|∂K|} = \frac{dρ dω}{|∂K|}

is a probability measure defined on F (K).

Suppose K_{0} ⊆ K is a convex subset of K. Then |∂K_{0}| ⩽ |∂K| and the probability that a straight line meeting K also meets K_{0} is

\frac{|∂K_{0}|}{|∂K|}.

Consequently the probability that a straight line meeting K misses K_{0} is

1 − \frac{|∂K_{0}|}{|∂K|}.  (1)

Consider a curve Γ_{0} of finite length lying in the convex set K. Denote by p_{n}(Γ_{0}, K) the probability that a straight line D meeting K intersects Γ_{0} in exactly n points. Observe that if K = K_{0} = con(Γ_{0}), the convex hull of Γ_{0}, then D must hit Γ_{0}, so that p_{0}(Γ_{0}, K_{0}) = 0.

By the definition of probability,

∑_{n=0}^{∞} p_{n}(Γ_{0}, K) = 1.  (2)

A classical result of H. Steinhaus [17] states that the expected number λ say of intersection points of Γ_{0} with random straight lines is

λ := ∑_{n=1}^{∞} n p_{n}(Γ_{0}, K) = \frac{2|Γ_{0}|}{|∂K|}.  (3)

The next moment ∑_{n=1}^{∞} n^{2} p_{n}(Γ_{0}, K) corresponds to the “energy” of the curve but is much less well-behaved and will not be considered.

3 Entropy

The celebrated Shannon entropy of thermodynamics and information theory has a generalization due to Rényi [13, 14]. Their definitions are used in conjunction with geometric probability to define formally notions of entropy for curves of finite length within a convex set K. Note that unlike the thermodynamic setting, there is no underlying mechanism or dynamic here that leads to the entropies increasing.

3.1 The Shannon entropy

The sequence (p_{n}(Γ_{0}, K): n = 0, 1, 2, …) of probabilities of the number of intersection points can be substituted into the formula for Shannon entropy. To be precise, the Shannon entropy h(Γ_{0}, K) of the curve Γ_{0} relative to a convex set K containing Γ_{0} is defined by the formula

h(Γ_{0}, K) = ∑_{n=0}^{∞} p_{n}(Γ_{0}, K) log \frac{1}{p_{n}(Γ_{0}, K)} = ∑_{n=0}^{∞} p_{n}(K) log \frac{1}{p_{n}(K)},  (4)

where as usual we put p log(1/p) = 0 if p = 0 and where for simplicity here and elsewhere whenever the argument is clear, we suppress Γ_{0} or K and write the probability

p_{n}(Γ_{0}, K) = p_{n}(K) or p_{n}.

Similarly we will write the Shannon entropy

h(Γ_{0}, K) = h(K) = h.

3.2 The Rényi entropy

Rényi [13, 14] has defined a form of entropy h^{(s)}, where s ⩾ 0, by

h^{(s)} := \frac{1}{1 − s} log ∑_{n=0}^{∞} p_{n}^{s},  (5)

which tends to the Shannon entropy h as s → 1:

h^{(1)} := lim_{s→1} h^{(s)} = lim_{s→1} (\frac{1}{1 − s} log ∑_{n=0}^{∞} p_{n}^{s}) = ∑_{n=0}^{∞} p_{n} log 1/p_{n} = h .  (6)

In addition it is readily verified that d h^{(s)}/ds = 0 at s = 1 and that h^{(s)} is minimal for s = 1. The Rényi entropy can also be applied formally to curves. We will write

h^{(s)}(Γ_{0}, K) = \frac{1}{1 − s} log ∑_{n=0}^{∞} p_{n}^{s}(Γ_{0}, K)  (7)

for the Rényi entropy for a curve Γ_{0} relative to a convex set K ⊇ Γ_{0} and, as with the Shannon entropy, we will write

h^{(s)}(Γ_{0}, K) = h^{(s)}(K) = h^{(s)}

when the meaning is clear.

The two series have finitely many terms for an algebraic curve, since p_{n} = 0 for all n larger than its degree, but they converge (absolutely) in general by the following results. A separate argument is needed for the two entropies. The Shannon entropy (s = 1) has been treated in this context [9, 10] and is now discussed for completeness.

3.3 A bound for the Shannon entropy of curves

Theorem 1. The Shannon entropy h(Γ_{0}, K) of a curve Γ_{0} of finite length lying in the convex set K satisfies

h(Γ_{0}, K) ⩽ log (\frac{2|Γ_{0}|}{|∂K|} + 1) + \frac{2|Γ_{0}|}{|∂K|} log (1 + \frac{|∂K|}{2|Γ_{0}|}) ⩽ log (\frac{2|Γ_{0}|}{|∂K|} + 1) + 1  (8)

and more precisely in the case K = K_{0} = con(Γ_{0}) (so that p_{0} = 0),

h(Γ_{0}, K_{0}) ⩽ log (\frac{2|Γ_{0}|}{|∂K_{0}|}) + (\frac{2|Γ_{0}|}{|∂K_{0}|} − 1) log (\frac{2|Γ_{0}|}{2|Γ_{0}| − |∂K_{0}|})
⩽ log (\frac{2|Γ_{0}|}{|∂K_{0}|}) + 1.  (9)

Proof. By (4), it suffices to bound the sum ∑_{n=k}^{∞} p_{n} log 1/p_{n}, where p_{n} ∈ [0, 1] and k = 0 in (8) and k = 1 in (9), subject to the two constraints (2) and (3), i.e.,

∑_{n=k}^{∞} p_{n} = 1

and

∑_{n=k}^{∞} n p_{n} = \frac{2|Γ_{0}|}{|∂K|} = λ,

where k = 0 or 1. This is done using Lagrange multipliers. Let α, β ∈ ℝ and consider

U = ∑_{n=k}^{∞} p̂_{n} log 1/p̂_{n} − α ∑_{n=k}^{∞} p̂_{n} − β ∑_{n=k}^{∞} np̂_{n}.

Then for each n = k, k + 1, …, ∂U/∂p̂_{n} = 0 implies

− log p̂_{n} − 1 − α − nβ = 0,

whence the ‘maximal’ probability distribution p̂_{n} is a negative exponential with constant factor. More precisely,

p̂_{n} = e^{−1−α−nβ} = Ce^{−βn},  (10)

where C = e^{−1−α} and p̂_{n} is a Gibbs distribution [16]. Recall that Gibbs measure is a probability measure of importance in thermodynamics: it is the unique measure that maximizes the entropy for a given expected energy. It underlies maximum entropy methods and related algorithms and its appearance here is accordingly natural.

The constant C = e^{−1−α} (or α) is determined from the power series

∑_{n=k}^{∞} p̂_{n} = C ∑_{n=k}^{∞} e^{−βn} = 1

which gives C = C(β) = e^{βk}(1 − e^{−β}). In statistical mechanics, C(β) is the reciprocal of the partition function and in physics β corresponds to the inverse temperature [16]. Moreover C ∑_{n=k}^{∞} n e^{−βn} = λ whence

λ = \frac{1}{1 − e^{−β}} (k − (k − 1) e^{−β}) = \frac{1}{e^{β} − 1} (k e^{β} − k + 1) .

Thus

e^{β} = \frac{λ − k + 1}{λ − k} and e^{β} − 1 = \frac{1}{λ − k} .

Hence the entropy h satisfies

h ⩽ C ∑_{n=k}^{∞} e^{−βn} log \frac{e^{βn}}{C} = H

say, so that

H = \frac{β}{1 − e^{−β}} (k − (k − 1)e^{−β}) + log \frac{e^{−kβ}}{1 − e^{−β}}
= \frac{β}{e^{β} − 1} (ke^{β} − k + 1) − β k + log \frac{e^{β}}{e^{β} − 1}
= log(λ − k + 1) + (λ − k) log \frac{λ − k + 1}{λ − k}
⩽ log(λ − k + 1) + 1.

Case 1 k = 0, corresponds to the inequality (8).

Case 2 k = 1 corresponds to the inequality (9).

Remark 2. The first inequality (8) in the theorem shows that the entropy vanishes as |∂K| increases to infinity, a fact we shall rediscover in §4.

Remark 3. In the above calculation,

λ = ∑_{n=k}^{∞} n p̂_{n} ⩾ k ∑_{n=k}^{∞} p̂_{n} = k.

Thus obviously if λ → k from above, then H = 0. Hence

Remark 4. In the case k = 1, if 2|Γ|/|∂K| → 1 from above, then h = 0; which is otherwise obvious since 2|Γ| = |∂K| implies Γ is a segment of straight line.

3.4 Bounds for the Rényi entropy of curves

Recall from equation (7) that the Rényi entropy h^{(s)}(Γ_{0}, K) of Γ_{0} relative to the convex set K ⊃ Γ_{0} is given by

h^{(s)}(Γ_{0}, K) := \frac{1}{1 − s} log ∑_{n=0}^{∞} p_{n}^{s}(Γ_{0}, K).

For simplicity, we let K = K_{0} = con(Γ_{0}), so that p_{0}(Γ_{0}, K_{0}) = 0, and denote the Rényi entropy h^{(s)}(Γ_{0}, K_{0}) of Γ_{0} relative to its convex hull K_{0} by h_{0}^{(s)}. Two bounds for the Rényi entropy h_{0}^{(s)} are obtained, one for the range 0 < s < 1 and the other for 1/2 < s < 1. Although the Rényi entropy coincides with Shannon entropy at s = 1, the bounds for the former may well be infinite at s = 1.

Two auxiliary functions are introduced. For each γ ∈ [0, ∞) and s ∈ [1/2, 1], we write

F_{s}(γ) = \frac{ζ_{1} (\frac{s}{1−s}; γ)}{ζ_{1} (\frac{1}{1−s}; γ)},

where ζ_{1}(u, c), c ⩾ 0, is the modified Hurwitz zeta function given by

ζ_{1}(u, c) := ∑_{k=1}^{∞} \frac{1}{(k + c)^{u}} = ∑_{k=0}^{∞} \frac{1}{(k + c)^{u}} − \frac{1}{c^{u}} = ζ(u, c) − \frac{1}{c^{u}} .

Note that ζ_{1}(u, 0) = ζ(u) and that F_{s}(0) > 1.

For convenience, the function F_{s} is simplified when 1/2 ⩽ s < 1 by putting u = 1/(1−s) ∈ [2, ∞), so that s/(1 − s) = u − 1 and

F_{s}(γ) = \frac{ζ_{1}(u − 1; γ)}{ζ_{1}(u; γ)} := G_{u}(γ) = G_{1/(1−s)}(γ).

Then for u > 2 and γ ⩾ 0,

(γ + 1)ζ_{1}(u; γ) = ∑_{k=1}^{∞} \frac{γ + 1}{(k + γ)^{u}} = ∑_{k=1}^{∞} (\frac{k + γ}{(k + γ)^{u}} − \frac{k − 1}{(k + γ)^{u}} )

< ∑_{k=1}^{∞} \frac{1}{(k + γ)^{u−1}} = ζ_{1}(u − 1; γ),

and G_{u}(γ) := ζ_{1}(u − 1; γ)/ζ_{1}(u; γ) > 1 + γ.

A more precise estimate is possible using the familiar inequality

∫_{1}^{∞} f(x)dx ⩽ ∑_{k=1}^{∞} f(k) ⩽ f(1) + ∫_{1}^{∞} f(x)dx  (11)

for a positive increasing integrable function f: ℝ → [0, ∞). It follows from (11) that

\frac{(1 + γ)^{−u+1}}{u − 1} < ζ_{1}(u, γ) := ∑_{k=1}^{∞} \frac{1}{(k + γ)^{u}} < \frac{1}{(1 + γ)^{u}} + \frac{1}{(u − 1)(1 + γ)^{u−1}} .  (12)

Hence for any fixed u > 2,

G_{u}(γ) = \frac{ζ_{1}(u − 1, γ)}{ζ_{1}(u, γ)} > \frac{1}{(u − 2)(1 + γ)^{u−2}} (\frac{1}{(1 + γ)^{u}} + \frac{1}{(u − 1)(1 + γ)^{u−1}} )^{−1}
> \frac{(u − 1)}{(u − 2)} \frac{(1 + γ)^{2}}{(u + γ)} .

Thus given any u > 2, for all sufficiently large γ, there exists an ε_{u} > 0 such that

G_{u}(γ) > (1 + ε_{u})(1 + γ).

It follows that given any real κ, ε_{u}(1 + γ) > κ for all sufficiently large γ. Hence for fixed u the inequality

G_{u}(γ) > γ + κ  (13)

holds for any κ and all sufficiently large γ. This lower bound for the function F_{s} = G_{u} is used to show that the equation F_{s}(γ) = γ + λ, where λ = 2|Γ_{0}|/|∂K_{0}| ∈ (0, ∞) (see (3)), is soluble when λ ⩾ F_{s}(0) = G_{u}(0), i.e., when λ ⩾ ζ(u − 1)/ζ(u) > 1.

Lemma 5. Suppose λ ⩾ G_{u}(0). Then there exists a unique γ_{0} ⩾ 0 such that

G_{u}(γ_{0}) = γ_{0} + λ.

Proof. Suppose without loss of generality that λ > G_{u}(0). By definition

G_{u}(0) − λ < 0.

But by equation (13), for fixed u and sufficiently large γ,

G_{u}(γ) > γ + λ

and by continuity, there exists a γ_{0} > 0 such that G_{u}(γ_{0}) = γ_{0} + λ.

The uniqueness of the root γ_{0} follows from considering the derivative ∂G_{u}(γ)/∂γ for γ > 0. The partial derivative ∂ζ_{1}(u; γ)/∂γ = −u ζ_{1}(u + 1; γ). Thus

\frac{∂G_{u}(γ)}{∂γ} = \frac{∂}{∂γ} \frac{ζ_{1}(u − 1, γ)}{ζ_{1}(u; γ)} = \frac{−(u − 1) ζ_{1}(u; γ)ζ_{1}(u; γ) + u ζ_{1}(u − 1; γ)ζ_{1}(u + 1; γ)}{ζ_{1}(u; γ)^{2}}
= u (\frac{ζ_{1}(u − 1; γ)ζ_{1}(u + 1; γ)}{ζ_{1}(u; γ)^{2}} − 1) + 1 > 1

if ζ_{1}(u; γ)^{2} < ζ_{1}(u − 1; γ)ζ_{1}(u + 1; γ). Now by the Cauchy-Schwarz inequality,

ζ_{1}(u; γ) = ∑_{k=1}^{∞} \frac{1}{(k + γ)^{(u−1)/2}} \frac{1}{(k + γ)^{(u+1)/2}}

⩽ (∑_{k=1}^{∞} \frac{1}{(k + γ)^{u−1}} )^{1/2} (∑_{k=1}^{∞} \frac{1}{(k + γ)^{u+1}} )^{1/2} = (ζ_{1}(u − 1; γ)ζ_{1}(u + 1; γ))^{1/2}

and ζ_{1}(u; γ)^{2} < ζ_{1}(u−1; γ)ζ_{1}(u+1; γ), since the inequality is readily seen to be strict. Hence ∂G_{u}(γ)/∂γ > 1, so that the graph of G_{u} crosses that of γ ↦ γ + λ no more than once.

Theorem 6. Let Γ_{0} be a curve of finite length with convex hull K_{0}. Then for 0 < s < 1,

h_{0}^{(s)} ⩽ \frac{1}{1 − s} log (\frac{s}{β(1 − s)}) + log ζ_{1} (\frac{1}{1 − s}; γ) ,  (14)

where β = β(s, 2|Γ_{0}|/|∂K_{0}|), γ = γ(s, 2|Γ_{0}|/|∂K_{0}|). Moreover, provided 1/2 < s < 1 and

\frac{2|Γ_{0}|}{|∂K_{0}|} > \frac{ζ(s/(1 − s))}{ζ(1/(1 − s))} > 1,  (15)

there exists a unique γ_{0} such that

h_{0}^{(s)} ⩽ \frac{1}{1 − s} log (\frac{2|Γ_{0}|}{|∂K_{0}|} + γ_{0}) + log ζ_{1} (\frac{1}{1 − s}; γ_{0}) .  (16)

Proof. As with Shannon entropy, we use Lagrange multipliers under the same two constraints (2) and (3) to find a bound for h_{0}^{(s)}. Consider

V = \frac{1}{1 − s} log ∑_{n=1}^{∞} p̂_{n}^{s} − α ∑_{n=1}^{∞} p̂_{n} − β ∑_{n=1}^{∞} np̂_{n}.

Then for each n = 1, 2, …,

\frac{∂V}{∂ p̂_{n}} = \frac{sp̂_{n}^{s−1}}{(1 − s) ∑_{n=1}^{∞} p̂_{n}^{s}} − α − βn.

Hence ∂V/∂p̂_{n} = 0 implies

p̂_{n}^{s−1} = β (\frac{1 − s}{s}) (∑_{n=1}^{∞} p̂_{n}^{s}) (γ + n),

where γ = α/β. Hence the ‘maximal’ distribution is given by

p̂_{n} = (\frac{s}{β(1 − s)})^{1/(1−s)} (∑_{n=1}^{∞} p̂_{n}^{s})^{−1/(1−s)} (\frac{1}{γ + n})^{1/(1−s)} .  (17)

We compute ∑_{n=1}^{∞} p̂_{n}^{s} in two ways. First using (2),

1 = ∑_{n=1}^{∞} p̂_{n} = (\frac{s}{β(1 − s)})^{1/(1−s)} (∑_{n=1}^{∞} p̂_{n}^{s})^{−1/(1−s)} ∑_{n=1}^{∞} (\frac{1}{γ + n})^{1/(1−s)} ,

whence

(∑_{n=1}^{∞} p̂_{n}^{s})^{1/(1−s)} = (\frac{s}{β(1 − s)})^{1/(1−s)} ζ_{1} (\frac{1}{1 − s}; γ) .

i.e.,

∑_{n=1}^{∞} p̂_{n}^{s} = (\frac{s}{β(1 − s)}) ζ_{1} (\frac{1}{1 − s}; γ)^{1−s} ,  (18)

where the right hand side converges for s ∈ (0, 1) and (14) follows.

Secondly, substitute (18) in (17) to get

p̂_{n} = ζ_{1} (\frac{1}{1 − s}; γ)^{−1} (\frac{1}{γ + n})^{1/(1−s)} ,

so that, by contrast with the Shannon entropy case (10), the distribution p̂_{n} obeys an inverse power law, with factor the reciprocal of a modified Hurwitz zeta function. Hence

∑_{n=1}^{∞} p̂_{n}^{s} = ζ_{1} (\frac{1}{1 − s}; γ)^{−s} ∑_{n=1}^{∞} (\frac{1}{γ + n})^{s/(1−s)} = \frac{ζ_{1} (\frac{s}{1−s}; γ)}{ζ_{1} (\frac{1}{1−s}; γ)^{s}} ,  (19)

which converges when 1/2 < s < 1 but diverges for s ⩽ 1/2.

To determine γ, use the constraint 2|Γ_{0}|/|∂K_{0}| = λ given by (3):

\frac{2|Γ_{0}|}{|∂K_{0}|} = λ = ∑_{n=1}^{∞} np̂_{n} = ζ_{1} (\frac{1}{1 − s}; γ)^{−1} ∑_{n=1}^{∞} n (\frac{1}{γ + n})^{1/(1−s)}
= ζ_{1} (\frac{1}{1 − s}; γ)^{−1} ∑_{n=1}^{∞} [\frac{γ + n}{(γ + n)^{1/(1−s)}} − \frac{γ}{(γ + n)^{1/(1−s)}} ]
= ζ_{1} (\frac{1}{1 − s}; γ)^{−1} (ζ_{1} (\frac{s}{1 − s}; γ) − γζ_{1} (\frac{1}{1 − s}; γ))
= \frac{ζ_{1} (\frac{s}{1−s}; γ)}{ζ_{1} (\frac{1}{1−s}; γ)} − γ.  (20)

By (15) and Lemma 5 with u = 1/(1−s), 1/2 < s < 1, there is a γ_{0} = γ_{0}(s, λ) satisfying (20), i.e., such that

G_{u}(γ_{0}) = F_{s}(γ_{0}) = \frac{ζ_{1} (\frac{s}{1−s}; γ_{0})}{ζ_{1} (\frac{1}{1−s}; γ_{0})} = λ + γ_{0}.

Now by (19),

h^{(s)} = \frac{1}{1 − s} log (∑_{n=1}^{∞} p_{n}^{s}) ⩽ \frac{1}{1 − s} log (∑_{n=1}^{∞} p̂_{n}^{s}) = ĥ^{(s)}

and for u = 1/(1 − s),

ĥ^{(s)} = \frac{1}{1 − s} log [\frac{ζ_{1} (\frac{s}{1−s}; γ_{0})}{ζ_{1} (\frac{1}{1−s}; γ_{0})}] + log ζ_{1} (\frac{1}{1 − s}; γ_{0})
= \frac{1}{1 − s} log(λ + γ_{0}) + log ζ_{1} (\frac{1}{1 − s}; γ_{0}) ,

giving (16).

The corresponding parameters β_{0} and α_{0} = β_{0} γ_{0}, are fixed by the equation

\frac{s}{β_{0}(1 − s)} = \frac{ζ_{1} (\frac{s}{1−s}; γ_{0})}{ζ_{1} (\frac{1}{1−s}; γ_{0})} = F_{s}(γ_{0}) = λ + γ_{0} ,

obtained by dividing (18) by (19) and using (20). This equation implies that as s → 1 from below, β_{0} → ∞ and therefore α_{0} → ∞, while F_{s}(γ_{0}) → ∞ as s → 1/2 from above, which implies that β_{0}, α_{0} → 0.

Remark 7. The bound for the Rényi entropy h^{(s)} is finite for 1/2 < s < 1. The equation F_{s}(γ_{0}) = γ_{0} + λ only holds if λ = 2|Γ_{0}|/|∂K_{0}| ⩾ F_{s}(0) = ζ(s/(s − 1))/ζ(1/(s − 1)) > 1. This and the values α, β do not have an obvious interpretation. Nor does the bound for the Shannon entropy appear to follow from the limit of the Rényi entropy case as s → 1.

4 Changing the viewpoint: the parameter t

In this section, we discuss the Shannon-Rényi entropy h^{(s)} of the curve Γ_{0} relative to a compact subset K containing K_{0}; this involves a parameter t. Consider the curve Γ_{0}, its convex hull K_{0} and a bounded convex set K ⊇ K_{0}. The set K can be thought of as a variable set that “increases” and within which the curve can increase its entropy.

4.1 The parameter t

Let t ⩾ 1 be the ratio of the lengths of the boundaries of K and K_{0}, i.e., let

t = \frac{|∂K|}{|∂K_{0}|} ∈ [1, ∞).

In this construction, the length |∂K| of the boundary of the convex set K increases.

Lemma 8. The probability p_{n}(Γ_{0}, K) that a straight line D meeting K intersects Γ_{0} in exactly n points is given by

p_{n}(Γ_{0}, K) = p_{n}(K) = \begin{cases}\frac{1}{t} p_{n}(K_{0}), & if n ⩾ 1; \\ 1 − \frac{1}{t}, & if n = 0. \end{cases}  (21)

Proof. If n ⩾ 1,

p_{n}(Γ_{0}, K) = \frac{μ\{D: card(D ∩ Γ_{0}) = n\}}{μ\{D: D ∩ K ≠ ∅\}}
= \frac{μ\{D: D ∩ K_{0} ≠ ∅\}}{μ\{D: D ∩ K ≠ ∅\}} \frac{μ\{D: card(D ∩ Γ_{0}) = n\}}{μ\{D: D ∩ K_{0} ≠ ∅\}}
= \frac{|∂K_{0}|}{|∂K|} \frac{μ\{D: card(D ∩ Γ_{0}) = n\}}{μ\{D: D ∩ K_{0} ≠ ∅\}} = \frac{1}{t} p_{n}(K_{0}, Γ_{0}).

Γ_{0} K_{0}  K

Figure 2: The curve Γ_{0} inside a bounded convex set K.

If n = 0,

p_{0}(Γ_{0}, K) = 1 − \frac{|∂K_{0}|}{|∂K|} = 1 − \frac{1}{t} ,

as in (1) in §2.

Let h_{0}^{(s)} := h^{(s)}(Γ_{0}, K_{0}), 0 < s ⩽ 1, be the Shannon-Rényi entropy of Γ_{0} with respect to its convex hull K_{0}. In the preceding section §3, h_{0}^{(s)} is written by h or h^{(1)} when s = 1 (Shannon entropy) and h^{(s)} when 0 < s < 1. We begin with the case s = 1.

4.2 The Shannon entropy case

The Shannon entropy h(Γ_{0}, K) = h of Γ_{0} relative to K is now determined in terms of the Shannon entropy h(Γ_{0}, K_{0}) = h_{0} of Γ_{0} relative to the convex hull K_{0} of Γ_{0} (h_{0} is finite by Theorem 1).

Theorem 9. Let h_{0} be the Shannon entropy of Γ_{0} with respect to K_{0}. The entropy h(Γ_{0}, K) of Γ_{0} with respect to K is given by

h(Γ_{0}, K) = \frac{h_{0}}{t} + log t − \frac{t − 1}{t} log(t − 1) = \frac{h_{0}}{t} + (−1 + \frac{1}{t}) log(1 − \frac{1}{t}) + \frac{log t}{t} → 0

as t → ∞. Moreover h(Γ_{0}, K) is maximal when t = t_{1} = 1 + e^{−h_{0}}, with value h_{1} = log(1 + e^{h_{0}}) > h_{0}.

Proof. Let h = h(K) be the entropy of Γ_{0} relative to K and let h_{0} = h(K_{0}) be the entropy

t 0  1 t_{1} = 1 + e^{−h_{0}} h_{0} h_{1}

Figure 3: The graph of h with maximum at t_{1}

of Γ_{0} relative to K_{0}. Then by (4) and Lemma 8 (and writing p_{n}(K) = p_{n}(Γ_{0}, K)),

h = ∑_{n=0}^{∞} p_{n}(K) log \frac{1}{p_{n}(K)}

= (1 − \frac{1}{t}) log (1 − \frac{1}{t})^{−1} + ∑_{n=1}^{∞} \frac{1}{t} p_{n}(K_{0}) log \frac{t}{p_{n}(K_{0})}

= (1 − \frac{1}{t}) log \frac{t}{t − 1} + \frac{1}{t} ∑_{n=1}^{∞} p_{n}(K_{0}) log t + \frac{1}{t} ∑_{n=1}^{∞} p_{n}(K_{0}) log \frac{1}{p_{n}(K_{0})}
= \frac{h_{0}}{t} + log t − \frac{t − 1}{t} log(t − 1).

We wish to find the maximum value of h when t ∈ (1, ∞). The derivative

\frac{dh}{dt} = −\frac{1}{t^{2}} (h_{0} + log(t − 1)) = 0

for t_{1} = |∂K_{1}|/|∂K_{0}| = 1 + e^{−h_{0}}, with entropy h a maximum at t = t_{1} and corresponding value

h_{1} := log(1 + e^{h_{0}}) > h_{0}.

Here K_{1} is any convex set whose boundary has length t_{1}|∂K_{0}| and which contains K_{0}.

4.3 The Rényi entropy case

For convenience h^{(s)}(Γ_{0}, K), the Rényi entropy with respect to the convex set K ⊃ Γ_{0} will be written h^{(s)}, p_{n}(Γ_{0}, K) will be written p_{n}(K) and h^{(s)}(Γ_{0}, K_{0}), the Rényi entropy with respect to the convex hull K_{0} of Γ_{0} will be written h_{0}^{(s)}. The dependence of h^{(s)} on the parameter t will be indicated where helpful; thus when t = 1, h^{(s)}(t) = h^{(s)}(1) = h_{0}^{(s)}. Note that by Theorem 6, the quantity h_{0}^{(s)} is finite for s ∈ (0, 1).

Theorem 10. The Rényi entropy h^{(s)}(Γ_{0}, K), 0 < s < 1, of Γ_{0} with respect to K is given by

h^{(s)}(Γ_{0}, K) = \frac{1}{1 − s} log ((1 − \frac{1}{t})^{s} + ∑_{n=1}^{∞} \frac{1}{t^{s}} p_{n}^{s}(K_{0}))

= \frac{1}{1 − s} log (1 + (t − 1)^{−s}e^{(1−s)h_{0}^{(s)}}) + \frac{s}{1 − s} log (1 − \frac{1}{t}) ,

so that h^{(s)} → 0 as t → ∞. Moreover h^{(s)}(Γ_{0}, K) is maximal for

t_{1} = 1 + (∑_{n=1}^{∞} p_{n}^{s}(K_{0}))^{−1/(1−s)} = 1 + e^{−h_{0}^{(s)}},

with value h^{(s)}(t_{1}) = log(1 + e^{h_{0}^{(s)}}).

Proof. Write h^{(s)}(Γ_{0}, K) = h^{(s)}. It follows from (21) that

h^{(s)} = h^{(s)}(t) = \frac{1}{1 − s} log ∑_{n=0}^{∞} p_{n}^{s}(K) = \frac{1}{1 − s} log ((1 − \frac{1}{t})^{s} + ∑_{n=1}^{∞} \frac{1}{t^{s}} p_{n}^{s}(K_{0}))

= \frac{s}{1 − s} log (1 − \frac{1}{t}) + \frac{1}{1 − s} log (1 + \frac{e^{(1−s)h_{0}^{(s)}}}{(t − 1)^{s}} ) ,

as claimed.

We now ask which t ⩾ 1 maximizes h^{(s)} = h^{(s)}(t). The derivative

\frac{dh^{(s)}}{dt} = \frac{s}{(1 − s) (∑_{n=0}^{∞} p_{n}^{s}(K_{0}))} ((1 − \frac{1}{t})^{s−1} \frac{1}{t^{2}} − \frac{1}{t^{s+1}} ∑_{n=1}^{∞} p_{n}^{s}(K_{0})) = 0

iff

(t − 1)^{s−1} = ∑_{n=1}^{∞} p_{n}^{s}(K_{0}) = e^{(1−s) h_{0}^{(s)}},

i.e., iff

t = 1 + (∑_{n=1}^{∞} p_{n}^{s}(K_{0}))^{−1/(1−s)} = 1 + e^{−h_{0}^{(s)}}.  (22)

The entropy h^{(s)}(t) is maximal at the value t_{1} given by (22), i.e.,

h^{(s)}(t_{1}) = \frac{1}{1 − s} log ∑_{n=1}^{∞} p_{n}^{s}(K_{0}) + log (1 + (∑_{n=1}^{∞} p_{n}^{s}(K_{0}))^{− \frac{1}{(1−s)}} )

= h_{0}^{(s)} + log (1 + e^{−h_{0}^{(s)}}) ,  (23)

whence h^{(s)}(t_{1}) = log (1 + e^{h_{0}^{(s)}}). The first term on the right hand side of (23) is the s-

entropy h_{0}^{(s)} = h^{(s)}(1) of Γ_{0} relative to its convex hull K_{0}, so that h^{(s)}(t_{1}) > h^{(s)}(1).

Remark 11. For a certain ‘dilution’ corresponding to |∂K|/|∂K_{0}| = t_{1}, the Rényi s-entropy attains a maximal value and then decreases to 0. Suppose now s → 1. Then it is easily seen that t_{1} → 1 + e^{−h_{0}} and h^{(s)}(t_{1}) → log(1 + e^{h_{0}}), which is consistent with the results above in §4.2.

If we agree to identify entropy and complexity, we see that the complexity of a curve depends on the point of observation. Seen from a certain distance, the curve increases its complexity, while seen from infinity the curves reduces to a point with entropy 0.

5 Acknowledgment

We are grateful to Chris Hughes for his help with the Hurwitz zeta function.

References

[1] J.-P. Allouche and L. Maillard-Teyssier, Inconstancy of finite and infinite sequences, Theoret. Comput. Sci. 412 (2011), 2268–2281.

[2] A. Balestrino, A. Caiti, and E. Crisostomi, Generalised entropy of curves for the analysis and classification of dynamical systems, Entropy 11 (2009), 249–270.

[3] A. Cauchy, Notes sur divers théorèmes relatifs à la rectification des courbes, et à la quadrature des surfaces, C. R. Acad. Sci. Paris 13 (1841), 1060–1063; Mém. Acad, Sci. Paris 22 (1850), 3–15.

[4] P. Cordier, M. Mendès France, J. Pailhous, and P. Bolon, Entropy as a global learning variable of the learning process, Human Movement Science 13 (1994), 745–763.

[5] F. Crémoux and A. Denis, Using the entropy of curves to segment a time or spatial series, Mathematical Geology 34 (2002), 899–914.

[6] M. W. Crofton, On the theory of local probability, applied to straight lines drawn at random in a plane; the metjods used being also extended to the proof of certain new theorems in the Integral Calculus, Phil. Trans. R. Soc. Lond. 158 (1868), 181–199.

[7] F. M. Dekking and M. Mendès France, Uniform distribution modulo one: a geometrical viewpoint, J. Reine Angew. Math., 329 (1981), 143–153.

[8] Y. Dupain, M. Mendès France, and C. Tricot, Dimensions des spirales, Bull. Soc. Math. France 111 (1983), 193–201.

[9] M. Mendès France, Chaos implies confusion, in M. M. Dodson and J. A. G. Vickers, eds., Number Theory and Dynamical Systems LMS Lecture Note Series, Vol. 134, Cambridge University Press, 1989, pp. 137–152.

[10] M. Mendès France, The Planck constant of a curve, in J. Belair and S. Dubuc, eds., Fractal Geometry and Analysis, NATO Adv. Sci. Inst. Ser. C Math. Phys. Sci., Vol. 346, Kluwer, 1991, pp. 325–366.

[11] M. Mendès France, Poincaré et les probabilités géométriques, in E. Charpentier, E. Ghys, and A. Lesne, eds., L’Héritage Scientifique de Poincaré, Editions Belin, 2006, pp. 316–330. English translation, Poincaré and geometric probability, in E. Charpentier, E. Ghys, and A. Lesne, eds., The Scientific Legacy of Poincaré, History of Mathematics, Amer. Math. Soc, London Math. Soc. 36 (2010), pp. 293-305.

[12] M. Mendès France and T. Tokieda, Rings on the water and their entropy, to appear in J. Austral. Math. Soc.

[13] A. Rényi, Wahrscheinlichkeitsrechnung. Mit einam Anhang über Informationstheorie, Hochschulbücher für Mathematik, no. 54, VEB Deutscher Verlag der Wissenschaften, Berlin, 1962.

[14] A. Rényi, Probability Theory, North-Holland, 1970.

[15] L. Santaló, Integral Geometry and Geometric Probability, 2nd ed., Cambridge University Press, 2006.

[16] E. Schrödinger, Statistical Thermodynamics, Cambridge University Press, 1952, reprinted by Dover 1989.

[17] L. Steinhaus, Length, shape and area, Colloquium Math. 3 (1954), 1–13.

2010 Mathematics Subject Classification: Primary 54C70; Secondary 52A22, 28D20, 11M35. Keywords: curves, geometric probability, entropy.

Received June 28 2012; revised version received August 31 2012. Published in Journal of Integer Sequences, March 2 2013.

Return to Journal of Integer Sequences home page.