Optimization of Functionals and the Calculus of Variations
There are many settings in Economics, particularly in mechanism design, where one wants to fund a function that maximizes (or minimizes) some objective. It turns out this works similarly to regular calculus.
Consider the expected revenue of a mechanism designed to sell a good to one agent who’s value of the good is distributed uniformly with support . This expected revenue is known to be
where is a function that determines the probability of selling the good to the agent when her value is and is the virtual surplus. The above equation, , is a functional of . That is, it maps the function to a real number. Suppose that you wanted to find the revenue maximizing mechanism . Then, the problem you want to solve is
subject to . If you look at this problem closely, you will note that the optimal function must be
where the value at one half does not matter. This is because is multiplied by some “slope”. When the slope is positive, we want to be as large as possible. When the slope is negative, we want to be as small as possible. Indeed, the functional derivative of is
We will see that the functional derivative formalizes our intuition about this slope. It will allow us to solve optimization problems that are nonlinear.
On the reals
Optimization over function spaces is a natural extension of optimization over the reals. So, it’s important to remember exactly how optimization works on functions of real vectors. Consider the following function .
where . The directional derivative in the direction is
If is a minimum, there cannot be a direction, , such that . This would imply that there exists an such that . This contradicts the assumption that is a minimum. In addition, there cannot be a such that because his would imply .
Therefore, we know that for any , for all . This is only possible if which means and . So, the only local stationary point is at . However, stationarity is a necessary but not sufficient condition. We would need to test for convexity to ensure that this is a minimum. Though, it obviously is.
Functional optimization follows the same principles.
On continuous functions
Suppose there is a known continuous function that we want to approximate with another continuous function using least squares on the interval . That is, you want to minimize the following functional
Of course, the minimum is going to be at . We will solve this by finding the Gateaux differential of with respect to some direction, . This is defined in the same way as the directional derivative on the reals
So, the Gateaux differential of is
which is the inner product of the functional derivative,
and the direction, . Like in calculus on the reals, we need this to be zero for all directions. It turns out that, just like before, we only need to set the directional derivative to zero. This is due to the following lemma.
Fundamental lemma of calculus of variations If is continuous on and satisfies the equality
for all continuous functions such that , then for all .
This means that if and only if for all . So, as anticipated.