What are moment generating functions and how do I use them to find moments?
I am studying probability theory and I need to understand moment generating functions (MGFs). The MGF of a random variable is:
My questions:
- Why is it called a "moment generating" function? How do I extract moments from it?
- Find the MGF of and use it to find and .
- What is the domain of for which the MGF exists?
- How do MGFs help prove the sum of independent Poissons is Poisson?
- What is the relationship between MGFs and characteristic functions?
I want to see the computations step by step.
2 answers222 views
Loading comments...
2 Answers
PR
Prof. Robert FischerMay 17, 2026 Accepted**Why "Moment Generating":**
The $n$-th moment $E[X^n]$ is the $n$-th derivative of $M_X(t)$ evaluated at $t = 0$:
$$M_X^{(n)}(0) = E[X^n]$$
This follows from the Taylor expansion $e^{tX} = \sum_{n=0}^{\infty} \frac{(tX)^n}{n!}$, so $E[e^{tX}] = \sum_{n=0}^{\infty} \frac{E[X^n]}{n!} t^n$.
**MGF of Exponential($\lambda$):**
The PDF is $f_X(x) = \lambda e^{-\lambda x}$ for $x \geq 0$.
\begin{align*}
M_X(t) &= \int_0^{\infty} e^{tx} \lambda e^{-\lambda x} \, dx \\
&= \lambda \int_0^{\infty} e^{-(\lambda - t)x} \, dx \\
&= \frac{\lambda}{\lambda - t} \quad \text{for } t < \lambda
\end{align*}
**Finding moments:**
\begin{align*}
M_X'(t) &= \frac{\lambda}{(\lambda - t)^2}, \quad M_X'(0) = \frac{1}{\lambda} = E[X] \\
M_X''(t) &= \frac{2\lambda}{(\lambda - t)^3}, \quad M_X''(0) = \frac{2}{\lambda^2} = E[X^2]
\end{align*}
Thus $\text{Var}(X) = E[X^2] - (E[X])^2 = 2/\lambda^2 - 1/\lambda^2 = 1/\lambda^2$.
**Sum of Independent Poissons:**
If $X \sim \text{Poisson}(\lambda_1)$ and $Y \sim \text{Poisson}(\lambda_2)$ are independent, their MGFs are $M_X(t) = e^{\lambda_1(e^t - 1)}$ and $M_Y(t) = e^{\lambda_2(e^t - 1)}$.
By independence: $M_{X+Y}(t) = M_X(t) M_Y(t) = e^{(\lambda_1 + \lambda_2)(e^t - 1)}$, which is the MGF of Poisson($\lambda_1 + \lambda_2$). By the uniqueness theorem, $X+Y \sim \text{Poisson}(\lambda_1 + \lambda_2)$.
**Characteristic Functions:** $\varphi_X(t) = E[e^{itX}]$ — always exists for all real $t$ (unlike MGFs which may not exist for some distributions like Cauchy). This makes characteristic functions more general, but they require complex analysis.
Loading comments...
A **Moment Generating Function (MGF)** is an alternative framework to describe a probability distribution. It packs all the statistical moments ($\mathbb{E}[X], \mathbb{E}[X^2], \mathbb{E}[X^3], \dots$) of a random variable into a single differentiable function.
**1. Why is it called "Moment Generating"? How to extract moments?**
It is called "moment generating" because it acts as a generating function in combinatorics. If we take the Taylor series expansion of $e^{tX}$ around $t = 0$:
$$e^{tX} = 1 + tX + \frac{(tX)^2}{2!} + \frac{(tX)^3}{3!} + \dots = \sum_{n=0}^{\infty} \frac{t^n X^n}{n!}$$
Taking the expected value on both sides yields:
$$M_X(t) = \mathbb{E}[e^{tX}] = 1 + t\mathbb{E}[X] + \frac{t^2}{2!}\mathbb{E}[X^2] + \frac{t^3}{3!}\mathbb{E}[X^3] + \dots = \sum_{n=0}^{\infty} \frac{t^n}{n!}\mathbb{E}[X^n]$$
- To extract the $n$-th raw moment ($\mathbb{E}[X^n]$), you differentiate $M_X(t)$ exactly $n$ times with respect to $t$, and then evaluate the derivative at $t = 0$:
$$\mathbb{E}[X^n] = \left. \frac{d^n}{dt^n} M_X(t) \right|_{t=0}$$
**2. MGF of $X \sim \text{Exponential}(\lambda)$ and finding $\mathbb{E}[X]$ and $\mathbb{E}[X^2]$**
The Probability Density Function (PDF) of an exponential random variable is $f_X(x) = \lambda e^{-\lambda x}$ for $x \ge 0$.
**Step-by-Step Computation of the MGF:**
$$M_X(t) = \mathbb{E}[e^{tX}] = \int_{0}^{\infty} e^{tx} \left( \lambda e^{-\lambda x} \right) dx$$
$$M_X(t) = \lambda \int_{0}^{\infty} e^{(t - \lambda)x} dx = \lambda \left[ \frac{e^{(t - \lambda)x}}{t - \lambda} \right]_{0}^{\infty}$$
**Finding the Moments:**
- **First Derivative for $\mathbb{E}[X]$:**
$$M'_X(t) = \frac{d}{dt} \left[ \lambda(\lambda - t)^{-1} \right] = \lambda(-1)(\lambda - t)^{-2}(-1) = \frac{\lambda}{(\lambda - t)^2}$$
$$\mathbb{E}[X] = M'_X(0) = \frac{\lambda}{(\lambda - 0)^2} = \frac{1}{\lambda}$$
- **Second Derivative for $\mathbb{E}[X^2]$:**
$$M''_X(t) = \frac{d}{dt} \left[ \lambda(\lambda - t)^{-2} \right] = \lambda(-2)(\lambda - t)^{-3}(-1) = \frac{2\lambda}{(\lambda - t)^3}$$
$$\mathbb{E}[X^2] = M''_X(0) = \frac{2\lambda}{(\lambda - 0)^3} = \frac{2}{\lambda^2}$$
**3. What is the domain of $t$ for which the MGF exists?**
For the limit at infinity in the integral $\left[ \frac{e^{(t - \lambda)x}}{t - \lambda} \right]_{0}^{\infty}$ to converge to $0$, the exponent coefficient must be strictly negative:
$$t - \lambda < 0 \implies t < \lambda$$
- If $t \ge \lambda$, the integral diverges to infinity.
- Therefore, the domain of $t$ is **$(-\infty, \lambda)$**. An MGF is valid only if it exists in an open interval containing $t = 0$.
**4. Proving the Sum of Independent Poissons is Poisson**
MGFs possess a unique property: the MGF of the sum of independent random variables equals the product of their individual MGFs ($M_{X+Y}(t) = M_X(t) \cdot M_Y(t)$).
Let $X \sim \text{Poisson}(\lambda_1)$ and $Y \sim \text{Poisson}(\lambda_2)$ be independent. The MGF of a Poisson variable $X$ is $M_X(t) = e^{\lambda_1(e^t - 1)}$.
**Step-by-Step Proof:**
$$M_{X+Y}(t) = \mathbb{E}[e^{t(X+Y)}] = \mathbb{E}[e^{tX} \cdot e^{tY}] = \mathbb{E}[e^{tX}] \cdot \mathbb{E}[e^{tY}]$$
$$M_{X+Y}(t) = e^{\lambda_1(e^t - 1)} \cdot e^{\lambda_2(e^t - 1)} = e^{(\lambda_1 + \lambda_2)(e^t - 1)}$$
- Because $e^{(\lambda_1 + \lambda_2)(e^t - 1)}$ matches the structural form of a Poisson MGF, the **Uniqueness Theorem of MGFs** guarantees that $X + Y \sim \text{Poisson}(\lambda_1 + \lambda_2)$.
**5. Relationship between MGFs and Characteristic Functions**
The main drawback of the MGF is that it can diverge to infinity for heavy-tailed distributions (like the Cauchy distribution). The **Characteristic Function ($\phi_X(t)$)** resolves this issue by introducing the imaginary unit $i = \sqrt{-1}$:
$$\phi_X(t) = \mathbb{E}[e^{itX}]$$
- **Relationship:** $\phi_X(t) = M_X(it)$.
- Since $|e^{itX}| = 1$, the integral or sum for $\phi_X(t)$ is bounded and absolutely convergent. Therefore, the characteristic function **always exists** for every real number $t$ for any random variable.
Loading comments...