Appendix A
Lagrange multipliers

A.1  A general introduction to Lagrange multipliers

The idea for the Lagrange multipliers was first put forward in the ninth of Lagrange's "Leçons sur le calcul des fonctions" in 1806. A function M = f₀(x_i) for i = 1,2,...N is to be maximised, subject to constraints g_j for which is known:

This is equivalent to reducing the dimension of the original system.

A new function L is constructed, adding the dimension lost through the constraints by incorporating Lagrange multipliers. Therefore, the constrained maximisation of function M is equivalent to the unconstrained maximisation of function L where

This function has N+K unknowns.

The maximisation is performed by differentiating L with respect to both parameters. In the maximum the differentials with respect to the appropriate l_k and x_k will be 0:

and

We now have N+K equations (equations 88 and 87, respectively) to solve the N+K unknowns of function L.

A.2  Lagrange multipliers in entropy maximisation

In the general case of maximising entropy, the following entropy formula may be used (this is equation 57):

The constraints may be denoted as follows:

for B possible system states. j = 1,M where M is the number of constraints. Since the constraints must always be satisfied, it follows from equation 90 that

where Dp_i is a small variation in p_i. In the maximum of S we have:

Now a multiplier l_j is introduced for each constraint. Multiplying equation 90 by l_j for each j and summation of the results leads to

When M £ B, only B-M parameters Dp_i are independent, whereas the parameters for which i = 1,M are dependent. This means the multipliers l_j may be obtained from

A.3  Lagrange multipliers in crystallography

In crystallography the entropy may be maximised in the case of general linear constraints of the form

for j = 1,M where M is the number of constraints. C_j = e^{2pi[h]_j·[r]} for every reciprocal lattice vector [h]_j, and c_j = U([h]_j) is the unitary structure factor for the reflection corresponding to [h]_j. p([r]) is the unknown optimal distribution of atoms and V the volume of the unit cell. The normalisation condition, or the origin term with j = 0, may also be regarded as a constraint:

The constrained maximisation of the entropy S(p,q) (see equation 89) is equivalent to the unconstrained maximisation of

where l_j are the Lagrange multipliers. q = q([r]) is the prior distribution.

Differentiation with respect to l_j returns the constraint equations 95, while differentiation with respect to p([r]) yields:

or

where p^ME is the Maximum Entropy posterior distribution. Solving directly for l₀ by integrating this equation over [r] and using the normalisation constraint ò_Vp^ME([r])d[r] = 1 gives l₀-1 = -lnZ where

Z is the partition function which describes the way in which the unit cell has been divided up in fractions B. Then

and

Back to the Table of Contents