MATEMATIQKI VESNIK 49 (1997), 93–97

UDK 519.21

originalni nauqni rad research paper

ON WEIGHTED PARAMETRIC INFORMATION MEASURE

Om Parkash

Abstract. Shannon’s measure of information plays a very important role for measuring diversity in plants and animals. But this measure does not deal with growth models other than exponential. Thus we need a parametric model which is suitable to all types of distributions. In this communication, one such measure depending upon two real parameters has been developed under a set of certain axioms and its particular cases have been studied.

1. Introduction

Let P = {(p_{1}, p_{2},…, p_{n}), 0 ⩽ p_{i} ⩽ 1, ∑_{i=1}^{n} p_{i} = 1} be a finite discrete probability distribution of a set of n events E = (E_{1}, E_{2},…, E_{n}) on the basis of an experiment whose predicted probability distribution is Q = {(q_{1}, q_{2},…, q_{n}), 0 ⩽ q_{i} ⩽ 1, ∑_{i=1}^{n} q_{i} = 1} and the utility distribution is U = {(u_{1}, u_{2},…, u_{n}), u_{i} ⩾ 0}.

In Information theory, the following measures are well known:

I_{n}(P) = − ∑_{i=1}^{n} p_{i} log p_{i},  (1.1)
I_{n}(P; U) = − ∑_{i=1}^{n} u_{i}p_{i} log p_{i},  (1.2)
I_{n}(P/Q) = ∑_{i=1}^{n} p_{i} log(p_{i}/q_{i})  (1.3)

and

I_{n}(P; Q) = − ∑_{i=1}^{n} p_{i} log q_{i}.  (1.4)

The information-theoretic measures (1.3) and (1.4), respectively, known as Kullback’s [3] relative information and Kerridge’s [2] inaccuracy are of great significance in statistical estimation and Physics. The measure (1.2) known as quantitative-qualitative measure of information introduced by Belis and Guiasu [1] has found deep applications in Theory of questionnaries.

AMS Subject Classification: 62 B 10, 94 A 17

The Shannon’s [8] measure (1.1) possesses very nice mathematical properties and is very useful in many fields. Specifically, in life sciences, we have diversity of animal and plant types which can be measured using Shannon’s model. The greater the value of this measure, greater will be the diversity.

Where Shannon’s model gives useful applications, it has a drawback. It always leads to exponential family of distributions. Since there are families of distributions other than exponential and there are laws of population growth other than exponential, we cannot confine ourselves to exponential families only and consequently, Shannon’s measure may not be much applicable. Moreover, there are many factors like Physiography, Topography, Biotic interference, Anthropogenic, Climatic, Edaphic etc. which affect the diversity in plants. Let α, β etc. represent such factors upon which the information regarding such a cybernetic system E_{i}/u_{i}/p_{i}/q_{i} depends. In this communication, we develop a generalized weighted information measure depending upon two real parameters α and β.

2. Weighted information measure of type (α, β)

Let the weighted measure of the type (α, β) be denoted by

I_{n}^{(α,β)}[p_{1}, p_{2},…, p_{n}; q_{1}, q_{2},…, q_{n}; u_{1}, u_{2},…, u_{n}].

To derive this measure, we assume that it satisfies the following axioms:

Axiom I. The weighted masure of information I_{n}^{(α,β)}[P; Q; U] is symmetric function of its arguments p_{i}’s, q_{i}’s and u_{i}’s.

Axiom II. The measure I_{n}^{(α,β)}[P; Q; U] is proportional to its utility u_{i}, that is, for each non-negative λ, the following holds

I_{n}^{(α,β)}[p_{i}; q_{i}; λu_{i}] = λI_{n}^{(α,β)}[p_{i}; q_{i}; u_{i}].

Axiom III. The measure I_{n}^{(α,β)}[P; Q; U] satisfies the branching property of the type (α, β) given by

I_{n+1}^{(α,β)}[p_{1}, p_{2},…, p_{i−1}, v_{i1}, v_{i2}, p_{i+1},…, p_{n}; q_{1}, q_{2},…, q_{i−1}, h_{i1}, h_{i2}, q_{i+1},…, q_{n}; u_{1}, u_{2},…, u_{i−1}, r_{i1}, r_{i2}, u_{i+1},…, u_{n}]

= I_{n}^{(α,β)}[P; Q; U] + p_{i}^{α}q_{i}^{β}I_{2}^{(α,β)} [\frac{v_{i1}}{p_{i}} , \frac{v_{i2}}{p_{i}}; \frac{h_{i1}}{q_{i}} , \frac{h_{i2}}{q_{i}}; r_{i1}, r_{i2}]

for every v_{i1} + v_{i2} = p_{i} > 0, h_{i1} + h_{i2} = q_{i} > 0 and \frac{r_{i1}v_{i1} + r_{i2}v_{i2}}{v_{i1} + v_{i2}} = u_{i} > 0; i = 1,

2, …, n; where α ≠ 1, β ≠ 0 are arbitrary parameters.

The following theorem gives the generalized form of the weighted measure of type (α, β):

Theorem 1. The quantitive-qualitative measure of type (α, β) satisfying axioms I to III together with the continuity of I_{n}^{(α,β)}[P; Q; U] determine the functions

On weighted parametric information measure 95

I_{n} as

I_{n}^{(α,β)}[P; Q; U] = C(α, β)[∑_{i=1}^{n} u_{i}p_{i}^{α}q_{i}^{β} − ∑_{i=1}^{n} u_{i}p_{i}], α ≠ 1, β ≠ 0, (2.1)

where C(α, β) (≠ 0) is a constant depending upon parameters α and β. Proof. Parkash and Singh [4] proved the following result: If

v_{ij} ⩾ 0, j = 1, 2,…, m_{i}, ∑_{j=1}^{m_{i}} v_{ij} = p_{i} > 0, ∑_{i=1}^{n} p_{i} = 1,

h_{ij} ⩾ 0, j = 1, 2,…, m_{i}, ∑_{j=1}^{m_{i}} h_{ij} = q_{i} > 0, ∑_{i=1}^{n} q_{i} = 1

and

r_{ij} ⩾ 0, j = 1, 2,…, m_{i}, \frac{∑_{j=1}^{m_{i}} r_{ij}v_{ij}}{∑_{j=1}^{m_{i}} v_{ij}} = u_{i} > 0,

for every i = 1, 2, …, n, then

I_{nm_{n}}[V; H; R] =

I_{n}[P; Q; U] + ∑_{i=1}^{n} p_{i}I_{m_{i}} [\frac{v_{i1}}{p_{i}} ,…, \frac{v_{im_{i}}}{p_{i}}; \frac{h_{i1}}{q_{i}} ,…, \frac{h_{im_{i}}}{q_{i}}; r_{i1},…, r_{im_{i}}] . (2.2)

With the help of Axiom III, the equation (2.2) takes the following form: I_{nm_{n}}^{(α,β)}[V; H; R] =

I_{n}^{(α,β)}[P; Q; U] + ∑_{i=1}^{n} p_{i}^{α}q_{i}^{β}I_{m_{i}}^{(α,β)} [\frac{v_{i1}}{p_{i}} ,…, \frac{v_{im_{i}}}{p_{i}}; \frac{h_{i1}}{q_{i}} ,…, \frac{h_{im_{i}}}{q_{i}}; r_{i1},…, r_{im_{i}}] .

(2.3)

Next, replacing m_{i} by m in (2.3) and substituting v_{ij} = 1/mn, h_{ij} = 1/rs, r_{ij} = 1 and p_{i} = 1/m, q_{i} = 1/r, u_{i} = 1, for every i = 1, 2, …, n; j = 1, 2, …, m where m, n, r, s are positive integers such that 1 ⩽ m ⩽ r, 1 ⩽ n ⩽ s, we get

F^{(α,β)}[mn; rs; 1] = F^{(α,β)}[m; r; 1] + (\frac{1}{m})^{α−1}(\frac{1}{r})^{β}F^{(α,β)}[n; s; 1]  (2.4)

where

F^{(α,β)}[n; s; 1] = I^{(α,β)} [\frac{1}{n},…, \frac{1}{n}; \frac{1}{s},…, \frac{1}{s}; 1,…, 1] .  (2.5)

Because of the symmetry of I_{n}^{(α,β)}[P; Q; 1], (2.4) can be written as

F^{(α,β)}[mn; rs; 1] = F^{(α,β)}[n; s; 1] + (\frac{1}{n})^{α−1}(\frac{1}{s})^{β}F^{(α,β)}[m; r; 1].  (2.6)

Equations (2.4) and (2.6) give

\frac{F^{(α,β)}[m; r; 1]}{[1 − (\frac{1}{m})^{α−1}(\frac{1}{r})^{β}]} = \frac{F^{(α,β)}[n; s; 1]}{[1 − (\frac{1}{n})^{α−1}(\frac{1}{s})^{β}]} = −C(α, β) (say).  (2.7)

Thus

F^{(α,β)}[m; r; 1] = C(α, β)[(\frac{1}{m})^{α−1}(\frac{1}{r})^{β} − 1].  (2.8)

Now we prove the result for rationals and the continuiuty of I_{n}^{(α,β)} proves the result for reals.

If m, r, r_{i} and t_{i} are positive integers such that ∑_{i=1}^{n} r_{i} = m, ∑_{i=1}^{n} t_{i} = r and if we put v_{ij} = 1/m, h_{ij} = 1/r, r_{ij} = 1 and p_{i} = r_{i}/m, q_{i} = t_{i}/r, u_{i} = 1, for every i = 1, 2, … n, then equation (2.3) becomes

I_{n}^{(α,β)}[P; Q; 1] = F^{(α,β)}[m; r; 1] − ∑_{i=1}^{n} p_{i}^{α}q_{i}^{β}F^{(α,β)}[r_{i}; t_{i}; 1].  (2.9)

Using (2.8), (2.9) gives

I_{n}^{(α,β)}[P; Q; 1] = C(α, β)[\{(\frac{1}{m})^{α−1}(\frac{1}{r})^{β} − 1\} − ∑_{i=1}^{n} p_{i}^{α}q_{i}^{β}\{(\frac{1}{r_{i}} )^{α−1}(\frac{1}{t_{i}} )^{β} − 1\}]

= C(α, β)[∑_{i=1}^{n} p_{i}^{α}q_{i}^{β} − 1].

Thus

I_{n}^{(α,β)}[P; Q; 1] = C(α, β)[∑_{i=1}^{n} p_{i}^{α}q_{i}^{β} − ∑_{i=1}^{n} p_{i}].  (2.10)

Next, setting u_{i} = 1 and λ = u_{i} in Axiom II, we get

I_{n}^{(α,β)}[P; Q; U] = C(α, β)[∑_{i=1}^{n} u_{i}p_{i}^{α}q_{i}^{β} − ∑_{i=1}^{n} u_{i}p_{i}],

which is (2.1).

Particular cases.

Case I. Equation (2.1) under the boundary conditions

I_{2}[P; P; U] = 0  (2.11)

and

I_{2}[1, 0; \frac{1}{2}, \frac{1}{2}; 1, 1] = 1  (2.12)

gives

I_{n}^{α}[P; Q; U] = (2^{α−1} − 1)^{−1}[∑_{i=1}^{n} u_{i}p_{i}^{α}q_{i}^{1−α} − ∑_{i=1}^{n} u_{i}p_{i}], α ≠ 1.  (2.13)

Taking α → 1, (2.13) gives

I_{n}[P; Q; U] = ∑_{i=1}^{n} u_{i}p_{i} log \frac{p_{i}}{q_{i}} ,

a result studied by Taneja and Ruteja [6]. Also on ignoring utility, that is, taking u_{i} = 1, (2.13) gives

I_{n}^{α}[P; Q] = (2^{α−1} − 1)^{−1}[∑_{i=1}^{n} p_{i}^{α}q_{i}^{1−α} − 1], α ≠ 1,

which is a result studied by Rathie and Kannappan [5] and it reduces to Kullback’s [3] relative information in the limiting case as a → 1.

Case II. Equation (2.1) under the boundary conditions

I_{3}[p_{1}, p_{2}, p_{3}; q_{1}, q_{2}, q_{2}; u_{1}, u_{2}, u_{3}] = I_{2}[p_{1}, p_{2} + p_{3}; q_{1}, q_{2}; u_{1}, \frac{u_{2}p_{2} + u_{3}p_{3}}{p_{2} + p_{3}} ] (2.14)

and

I_{2}[\frac{1}{2}, \frac{1}{2}; \frac{1}{2}, \frac{1}{2}; 1, 1] = 1  (2.15)

gives

I_{n}^{β}[P; Q; U] = (2^{−β} − 1)^{−1}[∑_{i=1}^{n} u_{i}p_{i}q_{i}^{β} − ∑_{i=1}^{n} u_{i}p_{i}], β ≠ 0.  (2.16)

Taking β → 0, I_{n}^{β}[P; Q; U] reduces to − ∑_{i=1}^{n} u_{i}p_{i} log q_{i}, a result studied by Taneja and Tuteja [7].

Further on ignoring utility, (2.16) gives

I_{n}^{β}[P; Q] = (2^{−β} − 1)^{−1}[∑_{i=1}^{n} p_{i}q_{i}^{β} − 1], β ≠ 0,

which reduces to Kerridge’s [2] inaccuracy in the limiting case as β → 0.

REFERENCES

[1] Belis, M. and Guiasu, S, A quantitative-qualitative measure of information in cybernetic systems, IEEE Trans. Information Theory 14 (1968), 593–594.

[2] Kerridge, D. F., Inaccuracy and inference, J. Royal Statist. Soc. Ser. B 23 (1961). 184–194.

[3] Kullback, S., Information Theory and Statistics, Wiley, New York 1959.

[4] Parkash, O. and Singh, R. S., Characterization of useful information theoretic measures, Kybernetika 23, 3 (1987), 245–251.

[5] Rathie, P. N. and Kannappan, Pl., A directed divergence function of type β, Information and Control 20 (1972), 38–45.

[6] Taneja, H. C. and Tuteja, R. K., Characterization of a quantitative-qualitative measure of relative information, Inf. Sci. 33 (1984), 217–222.

[7] Taneja, H. C. and Tuteja, R. K., Characterization of quantitative-qualitative measure of inaccuracy, Kybernetika 22 (1986), 393–402.

[8] Shannon, C. E., A mathematical theory of communication, Bell Syst. Tech. Jr. 27 (1948), 623–659.

(received 01.09.1996.)

Department of Mathematics, Guru Nanak Dev University, Amritsar–143005 India