A moment generating function packs all of a distribution's moments, $E[X], E[X^{2}], \ldots$, into one function. It turns the hard convolution of independent variables into a simple product, and it is the main tool for proving the central limit theorem. The key fact is the exponential series from known series, and the moments are the expected values of functions you already compute.
Idea clave
The moment generating function of $X$ is $M_X(t) = E\left[e^{tX}\right]$. Its derivatives at 0 are the moments: $M'(0) = E[X]$, $M''(0) = E[X^{2}]$, and so on, so $\operatorname{Var}(X) = M''(0) - (M'(0))^{2}$. For independent $X$ and $Y$, $M_{X + Y}(t) = M_X(t)M_Y(t)$. Two variables with the same MGF (near $t = 0$) have the same distribution.
Por qué funciona
Expand: $e^{tX} = 1 + tX + \frac{t^{2}X^{2}}{2!} + \cdots$, so by linearity $$M(t) = 1 + tE[X] + \frac{t^{2}}{2!}E[X^{2}] + \frac{t^{3}}{3!}E[X^{3}] + \cdots,$$ and the $k$th derivative at $t = 0$ picks out $E[X^{k}]$. For independent $X$ and $Y$, $e^{t(X + Y)} = e^{tX}e^{tY}$ is a product of independent factors, whose expected value is the product of the expected values.
| Distribution | $M(t)$ | Mean | Variance |
|---|---|---|---|
| Bernoulli($p$) | $1 - p + pe^{t}$ | $p$ | $p(1 - p)$ |
| Binomial($n, p$) | $(1 - p + pe^{t})^{n}$ | $np$ | $np(1 - p)$ |
| Poisson($\lambda$) | $e^{\lambda(e^{t} - 1)}$ | $\lambda$ | $\lambda$ |
| Exponential (rate $\lambda$) | $\frac{\lambda}{\lambda - t}$, $t < \lambda$ | $\frac{1}{\lambda}$ | $\frac{1}{\lambda^{2}}$ |
| Normal $N(\mu, \sigma^{2})$ | $e^{\mu t + \sigma^{2}t^{2}/2}$ | $\mu$ | $\sigma^{2}$ |
Worked examples
Ejemplo resuelto: A discrete MGF
$M(t) = \frac{1}{4}e^{-t} + \frac{1}{2} + \frac{1}{4}e^{2t}$. Find $E[X]$ and $\operatorname{Var}(X)$.
- This $X$ is $-1$, 0 or 2 with probabilities $\frac{1}{4}, \frac{1}{2}, \frac{1}{4}$.
- $M'(t) = -\frac{1}{4}e^{-t} + \frac{1}{2}e^{2t}$, so $E[X] = -\frac{1}{4} + \frac{1}{2} = \frac{1}{4}$.
- $M''(t) = \frac{1}{4}e^{-t} + e^{2t}$, so $E[X^{2}] = \frac{5}{4}$ and $\operatorname{Var}(X) = \frac{5}{4} - \frac{1}{16} = \frac{19}{16}$.
Respuesta: $\frac{1}{4}$ and $\frac{19}{16}$
Ejemplo resuelto: The Poisson
Find the mean and variance from $M(t) = e^{\lambda(e^{t} - 1)}$.
- $M'(t) = \lambda e^{t}M(t)$, so $M'(0) = \lambda$.
- $M''(t) = \lambda e^{t}M(t) + \lambda e^{t}M'(t)$, so $M''(0) = \lambda + \lambda^{2}$.
- $\operatorname{Var}(X) = \lambda + \lambda^{2} - \lambda^{2} = \lambda$.
Respuesta: Both are $\lambda$.
Ejemplo resuelto: Recognizing a sum
$X$ and $Y$ are independent Poisson with means 2 and 3. Identify the distribution of $X + Y$.
- $$M_{X + Y}(t) = e^{2(e^{t} - 1)}e^{3(e^{t} - 1)} = e^{5(e^{t} - 1)}$$
- That is the Poisson MGF with $\lambda = 5$, and an MGF determines the distribution.
Respuesta: Poisson with mean 5, matching the convolution lesson with far less work.
Error común
Reading $M''(0)$ as the variance. It is $E[X^{2}]$, the second moment about 0; the variance needs $(M'(0))^{2}$ subtracted. Check with a constant $X = 3$: $M(t) = e^{3t}$, $M''(0) = 9$, but the variance is 0.