Section 3b: The convex conjugate and the variational bound

I am working through Mathematical Foundations of Generative AI, Prof. Prathosh AP’s public lectures. The playlist is the spine. These notes are my deep dive on each section: a visual when the picture is the point, a detour when a prerequisite is doing real work, and every formula read aloud with a plain-words line right after it.

Previously: Section 3a: f-divergences.

Symbol cheat-sheet

Symbol	Say it as	In plain words
$x$	”x”	one data point, e.g. one image as a list of numbers
$\mathcal{X}$	”script X”	the set of all possible data points
$P_x(x)$	”P-x of x”	how likely real data is to land at the point $x$ (the true density)
$P_\theta(x)$	“P-theta of x”	how likely the model is to produce the point $x$
$f(u)$	“f of u”	a convex penalty function applied to the density ratio $u$
$f^*(t)$	“f-star of t”	the convex conjugate of $f$: the biggest vertical gap between the line $t \cdot u$ and the curve $f(u)$
$\operatorname{dom} f^*$	“domain of f-star”	the set of slopes $t$ where $f^*(t)$ is finite
$f'(u)$	”f-prime of u”	the slope (derivative) of $f$ at $u$
$T(x)$	“T of x”	the critic / auditor function: takes a point $x$ and outputs a score
$\mathcal{T}$	”script T”	the family of allowed critic functions (e.g. neural networks $T_w$)
$\sup$, $\max$	“soup”, “max”	the best (largest) value achievable
$\arg\min$	”arg-min”	the knob setting that makes something smallest
$\mathbb{E}_{x \sim p}[h(x)]$	”expected value of h, with x drawn from p”	the average value of $h(x)$ over random draws from $p$
$\widehat{D}_f$	“D-f hat”	our sample-average estimate of the true divergence $D_f$

---

Section 3b: The convex conjugate and the variational bound

This is the last part of Week 1, and it fixes the catch from Section 3a: we cannot compute $D_f$ directly because it needs the densities $P_x(x)$ and $P_\theta(x)$, and we only have samples. By the end of this section, $D_f$ is rewritten using only averages over samples. That rewritten form is the GAN objective in disguise.

1. Intuition first

A bowl-shaped curve can be rebuilt entirely from its tangent lines: draw enough of them, and their top edge traces the curve. Straight lines are simpler to work with than curves, so we replace $f$ with “the best of all straight lines.” When we do that inside the divergence, the unknown ratio disappears and we are left with two plain averages: one over real data, one over model data. Measuring the divergence becomes a search for the scoring function that best tells the two apart.

2. Real-world analogy

You hire an auditor: a function $T$ that looks at any single request and gives it a score.

• The auditor gains credit for scoring real requests high: $\mathbb{E}_{P_x}[T(x)]$.
• The auditor pays a penalty for scoring synthetic requests high: $\mathbb{E}_{P_\theta}[f^*(T(x))]$.

The net score needs only request samples from both sources, never a formula. A lazy auditor gets a low net score; the best possible auditor’s net score equals the true divergence. In a GAN, the discriminator is this auditor, and the generator tries to fool it.

3. Visual: the conjugate as tangent lines

The core picture is how a convex $f$ relates to its conjugate $f^*$. Here $f(u) = u \log u$ (the forward-KL $f$). Drag the slope $t$ and watch the tangent line slide along the curve.

Things to watch:

• Slide $t$: for each slope $t$, the dashed line $t \cdot u$ sits above the curve at some points. The coral gap between the dashed line and the curve is $f^*(t)$. Shift the dashed line down by that exact gap and you get the green tangent line.
• Turn on the envelope: the green tangent lines all sit on or below the purple curve, and their top edge traces the curve itself ($f^{**} = f$).

4. The formula, decoded

The target result:

$D_f(P_x \,\|\, P_\theta) \;\ge\; \sup_{T \in \mathcal{T}} \Big( \mathbb{E}_{x \sim P_x}\big[T(x)\big] \;-\; \mathbb{E}_{x \sim P_\theta}\big[f^*\big(T(x)\big)\big] \Big)$

Read aloud: “the f-divergence is at least the best value, over all scoring functions $T$ in the family $\mathcal{T}$, of: the average score $T$ gives to real data, minus the average of $f$-star of the score $T$ gives to model data.”

In plain words: the divergence is at least as large as the best auditor’s net score, which is “score real data high, score model data low.”

Prerequisites, each as a detour: $\sup$, derivatives for finding a maximum, the conjugate itself, choosing the best option at each point, and restricted searches.

Detour 1: sup vs max

$\sup$ (“supremum”) means the smallest value that nothing in the set exceeds. Example: the values $1 - \tfrac1n$ ($0, 0.5, 0.67, 0.75, \dots$) never reach 1, but their sup is 1.

In plain words: “the best achievable value, even if it is only approached.” For this course, read $\sup$ as $\max$.

Back to the main thread. Every $\sup$ below means “the best possible.”

Detour 2: Finding a maximum with the derivative

The derivative $h'(u)$ is the slope of $h$ at $u$. At the top of a smooth hill the ground is flat, so a maximum satisfies

$h'(u^*) = 0$

Read aloud: “h-prime at u-star equals zero.”

In plain words: the best point is where the slope becomes zero.

Two derivative rules we will use:

$\frac{d}{du}(t \cdot u) = t$

Read aloud: “the derivative with respect to $u$ of $t$ times $u$ equals $t$.”

In plain words: a straight line with slope $t$ has slope $t$ everywhere.

$\frac{d}{du}(u \ln u) = \ln u + 1$

Read aloud: “the derivative with respect to $u$ of $u \ln u$ equals $\ln u + 1$.”

In plain words: from the product rule, “slope of the first times the second, plus the first times the slope of the second,” which gives $1 \cdot \ln u + u \cdot \tfrac1u = \ln u + 1$.

Back to the main thread. We will use these two rules to compute $f^*$ for forward KL.

Detour 3: The convex conjugate

The convex conjugate is defined as

$f^*(t) = \sup_{u \in \operatorname{dom} f} \big( t\,u - f(u) \big)$

Read aloud: “f-star of $t$ is the largest value, over all allowed inputs $u$, of $t$ times $u$ minus $f$ of $u$.”

Piece by piece:

• $t$: a slope (a single number).
• $t\,u$: a straight line through the origin with that slope.
• $f(u)$: the curve.
• $t\,u - f(u)$: the vertical height of the line above the curve at $u$.
• $\operatorname{dom} f$: the inputs where $f$ is defined.

In plain words: $f^*(t)$ is the biggest vertical gap by which a line of slope $t$ rises above the curve $f(u)$. In the widget, it is the coral segment.

Computing it for $f(u) = u \ln u$:

Step 1, the gap we want to maximise:

$h(u) = t\,u - u \ln u$

In plain words: the height of the line above the curve at $u$.

Step 2, set the slope to zero (Detour 2):

$h'(u) = t - (\ln u + 1) = 0$

In plain words: the gap is largest where the line and the curve have equal slopes.

Step 3, solve for $u$:

$\ln u^* = t - 1 \;\;\Longrightarrow\;\; u^* = e^{t-1}$

In plain words: the best point is $e$ raised to $t - 1$. That is the “touch $u^*$” point in the widget.

Step 4, plug $u^*$ back in:

$f^*(t) = t\,e^{t-1} - e^{t-1}(t-1) = e^{t-1}$

In plain words: after the $t\,e^{t-1}$ terms cancel, the biggest gap is simply $e^{t-1}$.

Check at $t = 1$: $u^* = e^0 = 1$, and $f^*(1) = 1$. The widget shows the exact same numbers.

Two key properties of the conjugate: first, $f^*$ is also convex. Second, taking the conjugate twice returns the original function ($f^{**} = f$):

$f(u) = \sup_{t \in \operatorname{dom} f^*} \big( t\,u - f^*(t) \big)$

Read aloud: ”$f$ of $u$ equals the largest value, over all slopes $t$, of $t$ times $u$ minus $f$-star of $t$.”

In plain words: each $t\,u - f^*(t)$ is a tangent line (slope $t$, shifted down by the gap $f^*(t)$). The curve’s value at $u$ equals the highest tangent line at $u$. That is the envelope picture in the widget. Lower-semicontinuity from 3a is the technical condition that makes this exact.

Back to the main thread. This is the key move: it replaces the curved $f(\cdot)$ with straight lines in $u$. Since straight lines are linear, when $u$ is a ratio of densities, the denominator can be multiplied away.

Detour 4: The best choice at each point becomes a function

$\int \sup_t\, g(x, t)\, dx \;=\; \sup_{T} \int g\big(x, T(x)\big)\, dx$

Read aloud: “adding up the best value at each $x$ equals the best value, over all functions $T$, of adding up $g$ with $T(x)$ plugged in.”

In plain words: picking the best price for each product separately is the same as picking the best price table for the whole catalogue. The lookup table is the function $T(x)$.

Back to the main thread. This is where a single number $t$ becomes a function $T(x)$: the best slope depends on $x$.

Detour 5: A restricted search can only do worse

$\max_{\text{small set}} \;\le\; \max_{\text{everything}}$

Read aloud: “the best found in a smaller set is at most the best found in everything.”

In plain words: the best restaurant within 1 km is never better than the best in the whole city.

Back to the main thread. We can only search a limited family $\mathcal{T}$ (later, neural networks of a fixed shape), so the result becomes "$\ge$" instead of "$=$".

The derivation, step by step

Step 1, start from the definition (3a):

$D_f = \int P_\theta(x)\, f\!\left(\frac{P_x(x)}{P_\theta(x)}\right) dx$

In plain words: the model-weighted total mismatch penalty.

Step 2, replace $f$ with its tangent-line form (Detour 3), with $u = P_x(x)/P_\theta(x)$:

$D_f = \int P_\theta(x)\, \sup_t\!\left( t\,\frac{P_x(x)}{P_\theta(x)} - f^*(t) \right) dx$

What changed: $f$ became “the best tangent line.” In plain words: the same total, but each penalty is now computed as a highest straight line.

Step 3, move $P_\theta(x)$ inside the sup:

$D_f = \int \sup_t\Big( t\,P_x(x) - P_\theta(x)\, f^*(t) \Big)\, dx$

What changed: multiplying by a non-negative number $P_\theta(x) \ge 0$ does not change which $t$ is best, so $P_\theta(x)$ can move inside; then it cancels the denominator of the ratio. In plain words: the division by an unknown density is gone. This is the crucial step.

Step 4, let the best $t$ depend on $x$ (Detour 4):

$D_f = \sup_T \int \Big( T(x)\, P_x(x) - P_\theta(x)\, f^*\big(T(x)\big) \Big)\, dx$

What changed: “best number at each point” became “best function $T(x)$.” In plain words: we are now searching for an entire scoring rule $T$.

Step 5, search a limited family and split the integral (Detour 5):

$D_f \;\ge\; \sup_{T \in \mathcal{T}} \left( \int P_x(x)\, T(x)\, dx \;-\; \int P_\theta(x)\, f^*\big(T(x)\big)\, dx \right)$

What changed: “all functions” became “functions in $\mathcal{T}$ that we can actually search over,” which turns "$=$" into ”$\ge$.” In plain words: our best searchable auditor can only underestimate the divergence.

Step 6, recognise each integral as an expectation (3a, Detour 3):

$D_f \;\ge\; \sup_{T \in \mathcal{T}} \Big( \mathbb{E}_{x \sim P_x}\big[T(x)\big] - \mathbb{E}_{x \sim P_\theta}\big[f^*\big(T(x)\big)\big] \Big)$

What changed: “density times function, integrated” was renamed “average under that distribution.” In plain words: two averages, one over real data and one over model data. No density values needed.

One-sentence summary of the derivation: by writing the penalty as its best tangent line and letting the slope vary with $x$, the divergence becomes the best achievable “real-score minus model-penalty” over scoring functions, and that only needs samples.

Piece	Type	In plain words
$T(x)$	function $\mathcal{X} \to \operatorname{dom} f^*$	the auditor’s score for point $x$
$\mathcal{T}$	a family of functions	the auditors we are allowed to try
$\mathbb{E}_{x \sim P_x}[T(x)]$	a number	average score on real data
$\mathbb{E}_{x \sim P_\theta}[f^*(T(x))]$	a number	average penalty for scores on model data
$\sup_{T \in \mathcal{T}}$	—	use the best auditor in the family

Turning it into sample averages. Recall the Law of Large Numbers from Section 1:

$\frac{1}{n}\sum_{i=1}^{n} h(x_i) \;\approx\; \mathbb{E}_{x \sim p}[h(x)]$

In plain words: with enough random draws, the plain average of the samples is close to the true expected value.

Applying it to both terms gives our computable estimator:

$\widehat{D}_f \;\approx\; \frac{1}{n}\sum_{i=1}^{n} T(x_i) \;-\; \frac{1}{m}\sum_{j=1}^{m} f^*\big(T(g_\theta(z_j))\big)$

with $x_i$ from the dataset $D$ and $z_j \sim \mathcal{N}(0, I)$ (random noise vectors).

Read aloud: “the estimated divergence is approximately the average score over the $n$ real data points, minus the average of $f$-star of the scores over $m$ generated points.”

In plain words: score the real data, score the fake data, and take the difference of the averages. Only samples are needed.

Bonus fact (standard in the f-GAN literature): the bound is exact when

$T^*(x) = f'\!\left(\frac{P_x(x)}{P_\theta(x)}\right)$

Read aloud: “the best auditor’s score at $x$ is the slope of $f$ at the density ratio.”

In plain words: the perfect auditor is secretly measuring the real-to-model density ratio at each point.

5. Tiny worked example

Same coins as 3a: $P_x = (0.5, 0.5)$ and $P_\theta = (0.8, 0.2)$. Using forward KL, $f^*(t) = e^{t-1}$, and the true KL is $\approx 0.223$.

An auditor is two scores, $T = (T_1, T_2)$, one per outcome:

$\text{bound}(T) = \big(0.5\,T_1 + 0.5\,T_2\big) - \big(0.8\,e^{T_1 - 1} + 0.2\,e^{T_2 - 1}\big)$

Read aloud: “the bound is the real-weighted average score, minus the model-weighted average of $e$ to the power (score minus 1).”

In plain words: credit for scoring real outcomes, minus a penalty for scoring model outcomes.

Auditor $(T_1, T_2)$	Real-score average	Model-penalty average	Bound	vs 0.223
Lazy $(1, 1)$, same score for both	1.000	$0.8(1) + 0.2(1) = 1.000$	0.000	far below
Decent $(0.8, 1.5)$	1.150	$0.8(0.819) + 0.2(1.649) = 0.985$	0.165	closer
Best $(1 + \ln 0.625,\; 1 + \ln 2.5) = (0.530, 1.916)$	1.223	$0.8(0.625) + 0.2(2.5) = 1.000$	0.223	exact ✓

What the numbers show:

• Every auditor gives a lower bound; none overshoots $0.223$.
• Better auditors score outcome 2 higher. That is the outcome the model under-produces ($u_2 = 2.5$), so it “looks most real.”
• The best auditor uses $f'(u) = \ln u + 1$ (from Detour 2) at the true ratios, and recovers the KL divergence exactly.

Notes on the lecture slides

• The slides write $T : \mathcal{X} \to \operatorname{dom} f^*$: the auditor’s scores must lie where $f^*$ is defined. For KL that is every real number. Other choices of $f$ restrict the domain, which is why Week 2 adds an output activation $g_f$ to squash the network’s output into $\operatorname{dom} f^*$.
• Step 6 of the derivation matches the opening line of Week 2: $D_f \ge \max_T \mathbb{E}[T(x)] - \mathbb{E}[f^*(T(x))]$.

6. Connect back

Week 2 replaces the auditor with a neural network $T_w$ (weights $w$) and wraps a minimisation over the generator around the bound:

$\theta^*, w^* = \arg\min_\theta \max_w \Big( \mathbb{E}_{x \sim P_x}\big[T_w(x)\big] - \mathbb{E}_{z \sim \mathcal{N}(0, I)}\big[f^*\big(T_w(g_\theta(z))\big)\big] \Big)$

Read aloud: “find the generator weights that minimise, and the critic weights that maximise, the gap between the critic’s average score on real data and its average penalty on generated data.”

In plain words: the critic tries to measure the divergence as accurately as possible; the generator tries to shrink it. This is Variational Divergence Minimisation (VDM).

With the JS-like $f$, the objective becomes the classic GAN loss:

$\mathbb{E}_{x \sim P_x}\big[\log D_w(x)\big] + \mathbb{E}_{z}\big[\log\big(1 - D_w(g_\theta(z))\big)\big]$

Read aloud: “the average log of the discriminator’s real-probability on real data, plus the average log of one minus its real-probability on generated data.”

In plain words: the discriminator wants to output near 1 (“real”) for real data and near 0 (“fake”) for generated data. The generator wants to fool it.

Week 1 idea	Week 2 name
auditor $T(x)$	discriminator $D_w(x)$
$\sup_T$	discriminator training (gradient ascent on $w$)
minimising the bound over $\theta$	generator training (gradient descent on $\theta$)
tightness of the bound	how good the discriminator is

Quick check (optional)

1. In the worked example, why can no auditor give a bound above 0.223?
2. Which step in the derivation removed the unknown density ratio, and what did it do?
3. In $\widehat{D}_f$, where do the two sets of samples come from?