The Maximum Principle: A First Look

7.2.1. The Hamiltonian and the Maximum Principle. By analogy with the La­grangian, a much more economical way of expressing Theorem 7.2 is to construct the Hamil­tonian:^{[17] [18]}

Since f and g are continuously differentiable, so is H.

Theorem 7.3. (Maximum Principle) Consider the problem of maximizing (7.1) sub­ject to (7.2) and (7.3), with f and g continuously differentiable. Suppose that this problem has an interior continuous solution y (t) ∈Inty (t) with corresponding path of state variable X (t). Then there exists a continuously differentiable function λ (t) such that the optimal con­trol y (t) and the corresponding path of the state variable X (t) satisfy the following necessary conditions: x (0) = xo,

and λ (tι) = 0, with the Hamiltonian H (t, x, y, λ) given by (7.12). Moreover, the Hamiltonian H (t,x,y,λ) also satisfies the Maximum Principle that

For notational simplicity, in equation (7.15), I wrote x (t) instead of x(t) (= ∂x(t) /∂t). The latter notation is rather cumbersome, and we will refrain from using it as long as the context makes it clear that x (t) stands for this expression.

Theorem 7.3 is a simplified version of the celebrated Maximum Principle of Pontryagin. The more general version of this Maximum Principle will be given below. For now, a couple of features are worth noting:

(1) As in the usual constrained maximization problems, we find the optimal solution by looking jointly for a set of “multipliers” λ (t) and the optimal path of the control and state variables, y (t) and x (t).

(2) Again as with the Lagrange multipliers in the usual constrained maximization prob­lems, the costate variable λ (t) is informative about the value of relaxing the con­straint (at time t). In particular, we will see that λ (t) is the value of an infinitesimal increase in x (t) at time t.

(3) With this interpretation, it makes sense that λ (tι) = 0 is part of the necessary conditions. After the planning horizon, there is no value to having more x. This is therefore the finite-horizon equivalent of the transversality condition we encountered in the previous section.

While Theorem 7.3 gives necessary conditions, as in regular optimization problems, these may not be sufficient. First, these conditions may correspond to stationary points rather than maxima. Second, they may identify a local rather than a global maximum. Sufficiency is again guaranteed by imposing concavity. The following theorem, first proved by Mangasarian, shows that concavity of the Hamiltonian ensures that conditions (7.13)-(7.15) are not only necessary but also sufficient for a maximum.

Theorem 7.4. (Mangasarian’s Sufficient Conditions) Consider the problem of maximizing (7.1) subject to (7.2) and (7.3), with f and g continuously differentiable. Define H (t,x,y, λ) as in (7.12), and suppose that an interior continuous solutionand

the corresponding path of state variable x(t) satisfy (7.13)-(7.15). Suppose also that given the resulting costate variable λ (t), H (t,x,y,λ) is jointly concave in (x,y) for all t ∈ [0,tι], then the y (t) and the corresponding x(t) achieve a global maximum of (7.1). Moreover, if H (t,x,y, λ) is strictly jointly concave in (x,y) for all t ∈ [0,tι], then the pair (x (t),y (t)) achieves the unique global maximum of (7.1).

The proof of Theorem 7.4 is similar to the proof of Theorem 7.5, which is provided below, and is therefore left as an exercise (see Exercise 7.7).

The condition that the Hamiltonian H (t,x,y, λ) should be concave is rather demanding. The following theorem, first derived by Arrow, weakens these conditions. Before stating this result, let us define the maximized Hamiltonian as

with H (t, x,y,λ) itself defined as in (7.12). Clearly, the necessary conditions for an inte­rior maximum in (7.16) is (7.13). Therefore, an interior pair of state and control variables (x (t),y(t)) satisfies (7.13)-(7.15), then

Theorem 7.5. (Arrow’s Sufficient Conditions) Considertheproblemofmaximizing (7.1) subject to (7.2) and (7.3), with f and g continuously differentiable. Define H (t,x,y, λ) as in (7.12), and suppose that an interior continuous solution y (t) ∈Inty (t) and the corre­sponding path of state variable X (t) satisfy (7.13)-(7.15). Given the resulting costate variable λ (t), define M (t, X, λ) as the maximized Hamiltonian as in (7.16). If M (t, X, λ) is concave in x for all t ∈ [0,tι], then y (t) and the corresponding X (t) achieve a global maximum of (7.1). Moreover, if M (t,X,λ) is strictly concave in x for all t ∈ [0,tι], then the pair achieves the unique global maximum of (7.1) and X (t) is uniquely defined.

Proof. Consider the pair of state and control variables (X (t),y(t)) that satisfy the necessary conditions (7.13)-(7.15) as well as (7.2) and (7.3). Consider also an arbitrary pair (x (t),y (t)) that satisfy (7.2) and (7.3) and define M (t,x,λ) ? maxy H (t,x,y,λ). Since f and g are differentiable, H and M are also differentiable in x. Denote the derivative of M with respect to x by M_x. Then concavity implies that where the first line follows by an Envelope Theorem type reasoning (since Hy = 0 from equation (7.13)), while the second line follows from (7.15).

(7.18)

270

Theorems 7.4 and 7.5 play an important role in the applications of optimal control. They ensure that a pair (x (t),y (t)) that satisfies the necessary conditions specified in Theorem 7.3 and the sufficiency conditions in either Theorem 7.4 or Theorem 7.5 is indeed an optimal solution. This is important, since without Theorem 7.4 and Theorem 7.5, Theorem 7.3 does not tell us that there exists an interior continuous solution, thus an admissible pair that satisfies the conditions of Theorem 7.3 may not constitute an optimal solution.

Unfortunately, however, both Theorem 7.4 and Theorem 7.5 are not straightforward to check since neither concavity nor convexity of the g (∙) function would guarantee the concavity of the Hamiltonian unless we know something about the sign of the costate variable λ (t). Nevertheless, in many economically interesting situations, we can ascertain that the costate variable λ (t) is everywhere positive. For example, a sufficient (but not necessary) condition for this would be(see Exercise 7.9). Below we will see that λ (t) is

related to the value of relaxing the constraint on the maximization problems, which also gives us another way of ascertaining that it is positive (or negative depending on the problem). Once we know that λ (t) is positive, checking Theorem 7.4 is straightforward, especially when f and g are concave functions.

7.2.2. Generalizations. The above theorems can be immediately generalized to the case in which the state variable and the controls are vectors rather than scalars, and also to the case in which there are other constraints. The constrained case requires constraint qualification conditions as in the standard finite-dimensional optimization case (see, e.g.,

Simon and Blume, 1994). These are slightly more messy to express, and since we will make no use of the constrained maximization problems in this book, we will not state these theorems.

The vector-valued theorems are direct generalizations of the ones presented above and are useful in growth models with multiple capital goods. In particular, let

Theorem 7.6. (Maximum Principle for Multivariate Problems) Consider the problem of maximizing (7.21) subject to (7.22) and (7.23), with f and g continuously differ­entiable, has an interior continuous solution y (t) ∈Inty (t) with corresponding path of state variable x (t). Let H (t, x, y, λ) be given by

Moreover, we have straightforward generalizations of the sufficiency conditions. The proofs of these theorems are very similar to those of Theorems 7.4 and 7.5 and are thus omitted.

Theorem 7.7. (Mangasarian's Sufficient Conditions) Consider the problem of maximizing (7.21) subject to (7.22) and (7.23), with f and g continuously differentiable. De­fine H (t, x, y, λ) as in (7.24), and suppose that an interior continuous solution and the corresponding path of state variablesatisfy (7.25)-(7.27). Suppose also that for the resulting costate variable λ (t), H (t, x, y, λ) is jointly concave in (x, y) for all t ∈ [0,tι],

272

then y (t) and the corresponding x (t) achieves a global maximum of (7.21).

maximum of (7.21).

Theorem 7.8. (Arrow's Sufficient Conditions) Consider the problem of maximiz­ing (7.21) subject to (7.22) and (7.23), with f and g continuously differentiable. Define H (t, x, y, λ) as in (7.24), and suppose that an interior continuous solution)

and the corresponding path of state variable X (t) satisfy (7.25)-(7.27). Suppose also that for the resulting costate variable λ (t), define M (t, x, λ) ? max_y(_t)_ej(_t) H (t, x, y, λ). If M (t, x, λ) is concave in x for all t ∈ [0,tι], then y (t) and the corresponding x (t) achieve a global maximum of (7.21). Moreover, if M (t, x, λ) is strictly concave in x, then the pair (x (t), y (t)) achieves the unique global maximum of (7.21).

The proofs of both of these Theorems are similar to that of Theorem 7.5 and are left to the reader.

7.2.3. Limitations. The limitations of what we have done so far are obvious. First, we have assumed that a continuous and interior solution to the optimal control problem exists. Second, and equally important for our purposes, we have so far looked at the finite horizon case, whereas analysis of growth models requires us to solve infinite horizon problems. To deal with both of these issues, we need to look at the more modern theory of optimal control. This is done in the next section.

7.3.