versión On-line ISSN 1669-9637
Rev. Unión Mat. Argent. v.49 n.1 Bahía Blanca ene./jun. 2008
† Grupo de Sistemas No Lineales, INTEC (UNL-CONICET) Güemes 3450, 3000 Santa Fe, Argentina
‡ Centro de Matemática Aplicada, Escuela de Ciencia y Tecnología, UNSAM, M. de Irigoyen 3100, 1650 San Martín, Pcia. de Buenos Aires, Argentina
Hamilton's canonical equations (HCEs) have played a central role in Mechanics after (i) their equivalence with the principle of least action, and (ii) the variational calculus leading to the Euler-Lagrange equation, were established and applied (see ). Also, since the foundational work of Pontryagin , HCEs have been at the core of modern optimal control theory. When the problem concerning an -dimensional control system and an additive cost objective is regular , i.e. when the Hamiltonian of the problem is smooth enough and can be uniquely optimized with respect to at a control value (depending on the remaining variables), then HCEs appear as a set of ordinary differential equations whose solutions are optimal state-costate time trajectories.
Concerning the infinite-horizon bilinear-quadratic regulator and change of set-point servo problems, there exists a recent attempt to find the missing initial condition for the costate variable, based on a state-dependent (generalized) algebraic Riccati equation (GARE) with solution which allows to integrate the HCEs on-line with the underlying control process . The same approach in a finite time-domain leads to a first-order partial differential equation (PDE) called ‘Generalized Differential Riccati Equation' (GDRE) (see , , ) for a time-state dependent matrix whose solution allows to calculate the missing initial costate and exhibits, for a limiting behavior (see ) similar to that of linear systems with the same cost, i.e.
where is the duration of each optimization process.
In the general nonlinear finite-horizon optimization set-up, allowing for a free final state, the cost penalty imposed on the final deviation generates a two-point boundary-value situation. This is often a rather difficult numerical problem to solve. However, in the linear-quadratic regulator (LQR) case there exist well-known methods (see for instance , ) to transform the boundary-value into a final-value problem, related to the differential Riccati equation (DRE). Motivated by the role of Riccati equations, nonlinear situations have been treated for general final penalties by Byrnes , who posed a quasilinear first-order vector PDE (also labelled generalized Riccati equation by the author) for the optimal costate "in feedback form", i.e. as a function of the ‘event' but with boundary conditions on both and . Its usefulness is still under discussion, since a discretization of the state-space is unavoidable for numerical treatment. The same question in the one-dimensional case and for a quadratic (in this paper it will always be has been extended to a whole -family of problems (see , ), generating two first-order, quasilinear, uncoupled PDEs with classical initial conditions, where the dependent variables are the missing boundary conditions and of the HCEs. This approach has been completely disjoint from Riccati equations, but more in the line of the early invariant-imbedding ideas introduced by Bellman , . Analogous ideas were retaken and reformulated for the multidimensional case, in the light of symplectic properties inherent to Hamiltonian dynamics. The resulting matrix and vector PDEs are under review  and will be just summarized here, together with still unpublished feedback expressions for the optimal control.
When the -minimal control is not explicitly known, then new but similar PDEs appear, involving also the final value of the optimal control. The discussion of these equations would exceed the scope of this paper (see however , ).
After the relevant mathematical objects associated with the finite-horizon control problem are presented in Section 2, then the immersion into a family of -processes is worked out in Section 3. Afterwards, in Section 4 the main PDEs for the missing boundary conditions are substantiated. A brief discussion on the potentiality for feedback control follows in Section 5, applications are then developed in Section 6, and finally the conclusions and perspectives are summarized.
In what follows only initialized, autonomous (for simplicity) control systems of the form
will be considered. The state moves into some region of and the admissible control strategies are the real, piecewise continuous functions of the time-domain into some open subset of . The right-hand side is assumed to be smooth enough as to guarantee existence and uniqueness of solutions to the dynamics' equation (2) in the range of interest. The (finite-horizon) quadratic final penalty optimization context will imply here that a cost functional like
has to be minimized on the set of admissible control trajectories, where is a nonnegative smooth function called ‘the Lagrangian' of the problem, and is a nonnegative-definite symmetric matrix called ‘the final penalty coefficient'. The ‘value function' can always be defined for such a problem, namely
and, if the problem has a unique solution, then this solution is called ‘the optimal control strategy'
which in turn will generate ‘the optimal state trajectory'
The Hamiltonian of such a problem is defined as usual,
where is called the ‘costate', ranging in -dimensional ‘phase-space'. Since is assumed regular, then there exists a unique -optimal control
‘Explicitly regular' Hamiltonian means that the function is known (not only its existence but also its explicit form) and that it is sufficiently smooth on its variables. The control Hamiltonian,
that is a -dimensional ODE for a (Hamiltonian) vector field ,
It is useful to remind also that the control Hamiltonian is constant along the optimal trajectories, since
The following notation for the missing boundary conditions will be used in this Section
(the notation and may eventually emphasize that the quantities refer to a particular -problem)
By assuming that the Hamiltonian vector field is at least , then the existence of a flow
is guaranteed, where are appropriate regions of The flow verifies
where is preferred to the usual to avoid confusions.
By calling to the -advance transformations associated with the flow, the following identities become clear
where denote the ‘components' of the flow over the state and costate subspaces, respectively. The first component of Eq. (18) reads
and similarly for the second component, in brief
The (phase-space) derivative of the -advance function will be needed in the sequel, so a special name is given to it and to its partitions
where and so on. Existence and uniqueness of solutions imply that the inverse of exists and verifies
where the following notation is adopted for submatrices:
Since the same is true for its inverse can be calculated in terms of the submatrices namely
Now by deriving the first component of Eq. (20) with respect to
which will be written (with ) simply as
and similarly, for the second component,
By repeating the procedure for the -derivatives, analogous equations are obtained, namely
and from their combination,
which means that there really are only two independent first-order vector PDEs for namely
Notice that only and are involved, although a pair of equivalent PDEs can be obtained involving and In the one-dimensional case these equations can be uncoupled, obtaining (see )
but for a more involved treatment is needed, as will be shown below.
For an -dimensional state space it will be convenient to assign a name to the combined variable Eq. (18) can be written
and by taking derivatives on both members with respect to i.e. and then interchanging the order of derivation, the ‘variational equation' (see for instance , page 299) is obtained, namely
or, by abusing notation (the symbol is reserved for )
with the initial condition
i.e. the following identity is established
or, in short, reserving the symbol for
then the following identities are obtained
where the new matrices take the form
Since for a process of zero duration, then in such case , and therefore the main matrix PDEs in Eq. (52) are subject to the initial conditions
Now, Eqns. (53, 54) still include the unknown final state inside the s so the (matrix) PDEs in Eq. (52) can not be solved alone. But, having found expressions for the partitions of in terms of the auxiliary matrices and their derivatives, (vector) Eqns. (37, 38) turn into
which become solvable, at least in principle, when coupled to the matrix PDEs for and subject to initial conditions
| (57) (58) |
In short, the problem requires to solve in parallel two matrix first-order PDEs for and another two vector first-order PDEs for , all meeting appropriate initial conditions. If instead of the remaining submatrices were chosen, then Eqns. (32, 35) take also a condensed form, namely
but can not be directly recuperated from them.
Concerning the existence and uniqueness of solutions to the coupled system of Eqns. (52, 56), there exist only local results (see , page 51), although the field of vector and matrix PDEs integration is in active development (see for instance ).
Let us denote as the optimal initial costate corresponding to a -problem with initial state Smooth dependence on initial conditions for ODEs , extensive to first-order quasilinear PDEs , guarantees smooth dependence of on Neither the matrix nor the vector PDEs developed in the previous section depend explicitly on The initial state only affects solutions through the conditions (57, 58). As a consequence, numerical software can sometimes handle this dependence "analytically", i.e. considering as a dummy variable when solving for So it will be assumed that is available for some appropriate open set containing the "expected perturbations" from the optimal state trajectory starting at the original fixed initial condition Under these assumptions it is clear that the optimal costate trajectory must also verify (analogously to the Dynamic Programming Principle for the value function)
Therefore, if at some intermediate time the measured (or observed) state is possibly different from the expected but still inside a new optimal control problem starting at as initial condition (and duration may be considered to cope with state perturbations, and it follows that the new optimal control can be expressed in feedback form as
where Therefore in this case the HCEs become a linear, time-constant dynamical system or vector field , whose flow verifies
The following identities are easily obtained
Therefore, Eqns. (52) for can be integrated alone, since they do not depend on Actually, from
it follows that no further equations are needed for Since is always invertible (see , p.371), then the missing boundary conditions result
where is in turn the numerical solution of the DRE, i.e. the final-value matrix ODE
Therefore, from Eq. (71), for each -problem the Riccati matrix should also verify
The method based on PDEs for missing boundary conditions avoid solving DRE for each particular -problem, and storing, necessarily as an approximation, the Riccati matrix for the values of chosen by the numerical integrator, possibly different from the time instants for which the control is constructed. Instead, the HCEs (62) can be integrated with initial conditions
and the optimal trajectories obtained for which allows to generate the optimal control at each time
As a side-product, an alternative formula for the Riccati matrix results:
6.2. Bilinear systems and quadratic costs. The bilinear-quadratic case (with will be used to illustrate the application of previous results to nonlinear systems. The dynamics and trajectory cost will be, respectively,
The -optimal control is readily obtained (see )
and then the control Hamiltonian takes the form
The HCEs are therefore
where the -dependent matrix is clearly a generalization of the defined for linear systems, and analogously for
allowing to write the vector field and its derivative in concise expressions, namely
where new generalizations of LQR matrices appear,
The matrix can be evaluated by looking to the final conditions, i.e.
In conclusion, the relevant objects read in this case
The following checking procedure can be performed over numerical solutions. It is known (see ) that the value function verifies, for the finite-horizon bilinear-quadratic problem,
for some matrix solution of a generalized Riccati differential equation (GDRE), actually a first-order PDE in the variables that in the one-dimensional case takes the form
It is also known (see , and  for the linear analogue) that for the solutions to GDRE are compatible with solutions to the generalized algebraic Riccati equation (GARE) arising in the infinite-horizon case, namely
via the limiting behavior
Numerical calculations performed for increasing time-spans (approximately) confirm the asymptotic result
The solutions to the PDEs established in the previous Sections allow to transform the classical boundary-value problem posed for Hamilton equations in dimensions, into an initial-value situation when the Hamiltonian is regular. This allows in turn to numerically integrate the original HCEs on-line with the control process, and to continuously construct the manipulated variable from the state and costate values provided by this integration, since the -optimal control function is known. The on-line accessibility to an accurate value for the optimal state is most valuable in practical situations, since physical states of nonlinear control systems are hardly available at all desired times. Sometimes even a feedback control form can be constructed from the solutions to the quasilinear PDEs.
The PDEs' method solves a whole family of problems, avoiding additional off-line calculations and burdensome storing of information for each particular situation, as in methods of the DRE or GDRE type. The numerical integration of the new PDEs is relatively simple when only scalar values for are admitted, which is enough in many practical situations. Also, solutions providing the missing boundary conditions can be checked in several ways, and eventually iterated until convergence before using them to start controlling in real time.
Having the values of for a wide range of parameter values may be helpful at the design stage. From one side, the values of can be reconsidered by the designer when acknowledging the final values of the state that will be obtained under present conditions. And if a change in the parameter values is decided, then it will not be necessary to perform additional calculations to manage the new situation. Besides, the value of is an accurate measure of the ‘marginal cost' of the process, i.e. it measures how much the optimal cost would change under perturbations, which can also influence the decision on adopting the final values.
Other than the possibility of integrating HCEs on-line with the real plant (and constructing the optimal control for the model in real time), or even the possibility of generating a feedback law as we shall see, the approach presented in this paper may also be useful when studying input-output -stability of control systems, since the trajectory cost
may be analyzed in this set-up for variable gain even for nonlinear observation functions
and nonlinear dynamics (see , ), provided the resulting Hamiltonian is regular. Therefore, these PDEs seem to provide a novel environment where to explore the balance ‘performance versus stability'.
Other aspects of this approach deserve research. For instance, the curves are potentially a safeguard against Hamiltonian systems' instabilities (their linearizations have eigenvalues with positive real parts, because those associated with are symmetric to those corresponding to ). Therefore, it will probably add to robustness to construct the control by imposing a bound on costates for instance impeding the costates to trespass the reverse curve starting from when a finite horizon of duration is being optimized.
 Arnold, V.I., Mathematical Methods of Classical Mechanics, Springer-Verlag, New York, 1978. [ Links ]
 Bellman, R., Kalaba, R., A Note on Hamilton's Equations and Invariant Imbedding. Quarterly of Applied Mathematics, XXI:166-168, 1963. [ Links ]
 Bergallo, M., Costanza, V., Neuman, C.E., La Ecuación Generalizada de Riccati en Derivadas Parciales. Aplicación al Control de Reacciones Electroquímicas. Mecánica Computacional, XXV:1565-1580; 2006. [ Links ]
 Bernhard P., Introducción a la Teoría de Control Óptimo, Cuaderno del Instituto de Matemática "Beppo Levi" No 4, Rosario, Argentina, 1972. [ Links ]
 Byrnes, C., On the Riccati Partial Differential Equation for Nonlinear Bolza and Lagrange Problems, Journal of Mathematical Systems, Estimation, and Control 8(1)1-54, 1998. [ Links ]
 Costanza, V., Optimal State-Feedback Regulation of the Hydrogen Evolution Reactions, Latin American Applied Research, 35(4)327-335, 2005. [ Links ]
 Costanza, V., Finding initial costates in finite-horizon nonlinear-quadratic optimal control problems, Optimal Control Applications & Methods, 29(3)225-242; 2008. [ Links ]
 Costanza, V., Spies, R.D., Missing Boundary Conditions for Hamilton Canonical Equations in Optimal Control Problems, submitted to SIAM Journal on Control and Optimization, 2008. [ Links ]
 Costanza, V., Neuman C.E., Optimal Control of Nonlinear Chemical Reactors via an Initial-Value Hamiltonian Problem, Optimal Control Applications & Methods, 27:41-60, 2006. [ Links ]
 Costanza, V., Neuman C.E., Partial Differential Equations for Missing Boundary Conditions in the Linear-Quadratic Optimal Control Problem, Proc. XII Reunión RPIC, published in CD format, Río Gallegos, Argentina, 2007; selected for eventual publication in Latin American Applied Research, presently under review. [ Links ]
 Costanza, V., Picó, M., Control Hamiltoniano de Sistemas No Lineales en Tiempo Real; Proc. XI Reunión RPIC, published in CD format, Río Cuarto, Argentina, 2005. [ Links ]
 Costanza, V., Rivadeneira, P.S., Finite-Horizon Dynamic Optimization of Nonlinear Systems in Real Time. To be published in Automatica, 2008, DOI information: 10.1016/j.automatica.2008.01.033. [ Links ]
 Costanza, V., Rivadeneira, P.S., Regular Hamiltonian Problems with Implicit H-Optimal Control, Proc. XII Reunión RPIC, published in CD format, Río Gallegos, Argentina, 2007. [ Links ]
 Costanza, V., Rivadeneira, P.S., Minimal-Power Control of Electrochemical Hydrogen Reactions, submitted to Optimal Control Applications & Methods, 2008; under review. [ Links ]
 Folland, G.B., Introduction to Partial Differential Equations, 2nd. edition, Princeton University Press, Princeton, U.S.A., 1995. [ Links ]
 Hirsch, M.W. and S. Smale, Differential Equations, Dynamical Systems, and Linear Algebra, Academic Press, New York, 1974. [ Links ]
 Isidori, A., Nonlinear Control Systems II, Springer-Verlag, London, 1999. [ Links ]
 Jacobson, N., Basic Algebra I, W.H. Freeman and Company, San Francisco, U.S.A., 1974. [ Links ]
 Kalman, R.E., Falb, P.L., Arbib, M.A., Topics in Mathematical System Theory. McGraw-Hill, New York, 1969. [ Links ]
 Katok, A. and B. Hasselblatt, Introduction to the Modern Theory of Dynamical Systems, Cambridge University Press, Cambridge, U.K., 1999. [ Links ]
 Khalil, H.K., Nonlinear Systems. Prentice-Hall, 3rd Edition, Upper Saddle River, New Jersey, U.S.A., 2002. [ Links ]
 Pontryagin, L.S., Boltyanskii, V.G., Gamkrelidze, R.V., Mischenko, E.F., The Mathematical Theory of Optimal Processes. Wiley, New York, 1962. [ Links ]
 Sage, A.P., White, C.C, Optimum Systems Control. Prentice-Hall, 2nd Edition, Englewood Cliffs, U.S.A., 1977. [ Links ]
 Sontag, E.D., Mathematical Control Theory, Springer, 2nd Edition, New York, 1998. [ Links ]
3000 Santa Fe, Argentina
Recibido: 10 de abril de 2008
Aceptado: 18 de abril de 2008