Lagrange multipliers

The formula

∇f = λ·∇g,   with   g(x) = 0

At a constrained optimum, the gradient of the objective is parallel to the gradient of the constraint — you cannot improve f without leaving the constraint surface. The scale factor between the two gradients is the multiplier λ.

L(x, λ) = f(x) − λ·g(x)

The Lagrangian. Set every partial derivative of L to zero — with respect to each x component and with respect to λ — and you recover both the parallel-gradient condition and the constraint itself. One function, one system of equations, no substitution gymnastics.

∇f = λ₁·∇g₁ + λ₂·∇g₂ + … + λₘ·∇gₘ

With m constraints, the objective's gradient must be a linear combination of the constraint gradients: n + m equations in n + m unknowns.

λ = ∂f*/∂c   for the constraint written as g(x) = c

The multiplier is not scratch paper. It is the sensitivity of the optimal value to the constraint level — how much the best achievable f moves if the requirement moves. Economists call it a shadow price. Its units are always (units of f)/(units of g).

Where you meet it

Weight review on a bracket redesign. Minimize mass subject to first-mode frequency at or above the environments spec. The multiplier that falls out of the solution is the exchange rate — grams of mass per hertz of margin — and it's the number that settles the argument when someone asks what one more hertz of requirement actually costs.
Stage-split trades in mission design. Maximizing total Δv over how propellant divides between stages, with gross liftoff mass fixed, is a textbook multiplier problem, and it's how the classic staging results get derived before anyone opens a trajectory tool.
Telemetry and comm power allocation. Distributing a fixed transmitter power budget across channels for maximum total throughput is solved by setting one multiplier on the power constraint; the optimality condition — equal marginal return per watt in every active channel — is the water-filling rule baked into modern modem standards.
Tank and vessel sizing at the CAD seat. Least skin area for a required volume, least weld length for a required footprint. The cylinder example below is the whole method in miniature, and it's a five-minute back-check on whatever the optimizer spit out.

How it works

The geometric picture is worth having before the algebra. Walk along the constraint surface watching the level curves of f. Wherever a level curve crosses the constraint transversally, you can slide along the constraint and improve. The walk only stalls where a level curve is tangent to the constraint — and tangency of curves means parallelism of gradients. That is the entire theorem. λ is the ratio the parallelism happens at.

Worked example, numbers checked. Closed cylindrical tank, volume fixed at V = 1 m³, minimize surface area A = 2πr² + 2πrh subject to πr²h = V. The Lagrange system is:

∂A/∂r = λ·∂V/∂r :   4πr + 2πh = λ·2πrh
∂A/∂h = λ·∂V/∂h :   2πr       = λ·πr²

The second equation gives λ = 2/r; substituting into the first forces h = 2r. Height equals diameter — the squat proportion every bulk storage tank approximates. For V = 1 m³: r ≈ 0.542 m, h ≈ 1.084 m, A ≈ 5.536 m². And the multiplier earns its keep: λ = 2/r ≈ 3.69 m²/m³, which matches the numerically computed derivative dA*/dV at the optimum to six digits. Every additional cubic meter of required capacity costs about 3.69 m² of steel, at the margin, before you re-solve anything.

Lagrange's own reading of λ was mechanical, and it is still the most useful one in this town: in a constrained mechanism, the multiplier is the constraint force. Remove the physical constraint, apply the force λ·∇g in its place, and the system doesn't notice. When a multiplier comes out enormous, the constraint is being fought hard — that's the requirement to challenge at the next review. When it comes out near zero, the constraint is nearly free and nobody should be trading anything against it.

What goes wrong, in the order people actually hit it:

Treating the Lagrangian as something to minimize. The constrained optimum is a saddle point of L, not a minimum — descending on L with an off-the-shelf minimizer walks away to −∞. Solve the stationarity system, or use a solver built for constrained problems.
Believing the candidates. The conditions are necessary, not sufficient. The system's solutions include constrained maxima, minima, and inflections, all unlabeled. Evaluate f at every candidate, or check the bordered Hessian. The cylinder above has one candidate and a brute-force sweep confirms it's the minimum; real problems are rarely that polite.
Constraint qualification. The theorem requires ∇g ≠ 0 at the solution (linearly independent constraint gradients when there are several) ^[7]. Minimize f = x subject to x³ = y² and the answer is plainly the cusp at the origin — where ∇g = (3x², −2y) = (0, 0) and no λ can make (1, 0) a multiple of the zero vector. The method returns nothing, and the optimum is real. Degenerate constraint geometry means checking those points by hand.
Equality thinking on inequality problems. Most engineering constraints are one-sided — stress below allowable, budget not exceeded. The generalization is the Karush–Kuhn–Tucker (KKT) conditions: multipliers on inequality constraints must be nonnegative, and λᵢ·gᵢ = 0 for each (complementary slackness — a constraint either binds or its multiplier vanishes). Every serious optimization code reports these multipliers; an active constraint with a large one is your design driver, printed out for free.
Ignoring the units. λ carries (units of f)/(units of g). If a reported multiplier's units don't read as a sensible exchange rate — m² per m³, kg per Hz, dB per watt — something upstream is mis-scaled, and the optimizer's convergence behavior usually says so too.

The deeper habit worth forming: the multiplier is frequently more valuable than the optimum. The optimal point tells you where to sit today. The multipliers tell you which requirement to renegotiate, and what the renegotiation is worth.

History

Joseph-Louis Lagrange, born in Turin in 1736, announced himself at nineteen by mailing Leonhard Euler a purely analytical attack on the calculus of variations — the correspondence of 1754–56 that produced his "δ-algorithm" and what we now call the Euler–Lagrange equations ^[1]^[2]. Euler was impressed enough to propose him for the Berlin Academy, and — with d'Alembert's lobbying — Lagrange succeeded Euler there in 1766; after Frederick II's death he accepted the Paris Academy's invitation in 1787 ^[1]^[2].

The multiplier idea surfaced in mechanics, not optimization. In Part One of the Mécanique analytique (1788) — the book Lagrange bragged contained not a single figure — he handled the statics of constrained systems by multiplying each constraint's differential by an undetermined quantity and folding it into the virtual-work balance, then read the multipliers physically: each one is the force the constraint exerts, so a constrained system can be swapped for a free one with the constraint forces applied ^[2]^[3]^[4]. That swap, he wrote, is where the spirit of the method lives ^[3].

Nine years later, in the Théorie des fonctions analytiques (1797), he lifted the trick out of mechanics and stated it as a general rule for maxima and minima of a function of several variables under equation constraints: add to the proposed function each constraint function multiplied by an indeterminate quantity, then treat all the variables as free ^[3]^[4]. That one sentence is the method as taught today.

The inequality-constrained generalization took another century and a half. William Karush worked out the necessary conditions in a 1939 master's thesis at the University of Chicago that went unpublished; Harold Kuhn and Albert Tucker rediscovered and published them in "Nonlinear Programming" at the Second Berkeley Symposium in 1951, and the conditions carried only their two names until Karush's priority was recognized — hence the modern KKT label ^[5]^[6].

Related tools

/tools/tsiolkovsky-delta-v — the stage-split optimization behind multi-stage Δv is a Lagrange multiplier problem
/tools/link-budget — power allocation against a fixed budget is where the water-filling multiplier shows up
/tools/tank-volume-vertical — the minimum-area cylinder worked above
/tools/pressure-vessel-hoop — stress-below-allowable is the inequality constraint in vessel weight optimization
/tools/vibration-natural-freq — the frequency floor in mass-minimization trades
/tools/euler-buckling — the classic binding constraint in slender-member weight reduction

Sources

Written by HE in our own words from the cited sources — engineering judgment included, your stamp still required. All entries →