Mathematical Statistics

Moment Generating Functions

Samir Orujov, PhD

ADA University, School of Business

Information Communication Technologies Agency, Statistics Unit

2025-11-10

๐ŸŽฏ Learning Objectives

By the end of this lecture, you will be able to:

  • Define and understand moment-generating functions and their fundamental properties (essential for distribution theory)
  • Derive MGFs for common probability distributions analytically and numerically (Poisson, exponential, normal)
  • Apply the MGF derivative theorem to compute moments: mean, variance, and higher moments (critical for risk modeling)
  • Use MGF uniqueness property to identify probability distributions and prove equivalence (powerful theoretical tool)
  • Interpret MGFs in financial contexts including applications to option pricing and portfolio risk management

๐Ÿ“‹ Overview

๐Ÿ“š Topics Covered Today

  • Moments Review โ€“ Raw moments and central moments as foundation
  • Definition of MGF โ€“ Why and how moment-generating functions work
  • Key Theorem โ€“ Computing moments via derivatives of the MGF
  • Examples โ€“ Deriving MGFs for Poisson and other distributions
  • Applications โ€“ Real-world case study using MGF theory

๐Ÿ“– Definition: Moments

๐Ÿ“ Definition 1: Raw Moments

The k-th moment of a random variable \(Y\) taken about the origin is defined to be \(E(Y^k)\) and is denoted by \(\mu_k'\).

  • First moment (\(\mu_1' = E(Y)\)) โ€“ Expected value (mean return of an asset)
  • Second moment (\(\mu_2' = E(Y^2)\)) โ€“ Used to compute variance (volatility measure)
  • Third moment (\(\mu_3'\)) โ€“ Related to skewness (asymmetry of return distribution)
  • Fourth moment (\(\mu_4'\)) โ€“ Related to kurtosis (tail risk, probability of extreme events)

๐Ÿ“– Definition: Central Moments

๐Ÿ“ Definition 2: Central Moments

The k-th central moment of a random variable \(Y\) is defined to be \(E[(Y-\mu)^k]\) and is denoted by \(\mu_k\).

  • First central moment (\(\mu_1 = 0\)) โ€“ Always zero by definition
  • Second central moment (\(\mu_2 = \sigma^2\)) โ€“ Variance (risk measure in finance)
  • Third central moment (\(\mu_3\)) โ€“ Measures skewness (asymmetric tail behavior)
  • Fourth central moment (\(\mu_4\)) โ€“ Measures kurtosis (fat tails, black swan events)

๐Ÿ’ก Financial Insight

Investors care deeply about higher moments: positive skewness indicates higher probability of large gains, while high kurtosis signals increased tail risk (crash probability).

๐Ÿ“– Uniqueness Property

๐Ÿ”‘ Remark 1: Uniqueness of Moments

If two random variables \(Y\) and \(Z\) possess finite moments with \[ \mu'_{iY} = \mu'_{iZ} \quad \text{for all } i \in \mathbb{Z}^+ \] then \(Y\) and \(Z\) have identical probability distributions.

Financial Application: If two portfolio strategies produce identical moments at all orders, they are statistically equivalent in terms of return distribution.

๐Ÿ“– Definition: Moment-Generating Function

๐Ÿ“ Definition 3: MGF

The moment-generating function \(m(t)\) for a random variable \(Y\) is defined to be \[ m(t) = E(e^{tY}) \]

We say that an MGF exists if there exists a positive constant \(b\) such that \(m(t)\) is finite for \(|t| \leq b\).

Why โ€œmoment-generatingโ€? The derivatives of \(m(t)\) evaluated at \(t=0\) give us the moments!

๐Ÿงฎ Why It Generates Moments (Part 1)

๐Ÿ” Key Question

Why is \(E(e^{tY})\) called the moment-generating function?

From the Taylor series expansion for \(e^{ty}\): \[ e^{ty} = 1 + ty + \frac{(ty)^2}{2!} + \frac{(ty)^3}{3!} + \frac{(ty)^4}{4!} + \cdots \]

Taking the expectation (assuming \(\mu'_k\) is finite for all \(k\)): \[ E(e^{tY}) = E\left[1 + tY + \frac{(tY)^2}{2!} + \frac{(tY)^3}{3!} + \cdots\right] \]

๐Ÿงฎ Why It Generates Moments (Part 2)

\[ E(e^{tY}) = 1 + tE(Y) + \frac{t^2}{2!}E(Y^2) + \frac{t^3}{3!}E(Y^3) + \cdots \]

Substituting the moment notation: \[ \boxed{m(t) = 1 + t\mu'_1 + \frac{t^2}{2!}\mu'_2 + \frac{t^3}{3!}\mu'_3 + \cdots} \]

๐ŸŽฏ Key Insight

The coefficient of \(\frac{t^k}{k!}\) in the series expansion of \(m(t)\) is exactly \(\mu'_k\), the k-th moment!

๐Ÿ”‘ Key Applications of MGF

๐Ÿ’ก Remark 2: Two Important Applications

  1. Finding Moments: If \(E(e^{tY})\) exists, we can find any moment for \(Y\) by differentiation (easier than direct integration for complex distributions)

  2. Uniqueness Property: If the moment-generating functions for two random variables \(Y\) and \(Z\) are equal, then \(Y\) and \(Z\) must have the same probability distribution (powerful tool for proving distributional equivalence)

Financial Application: MGFs are used in mathematical finance to derive option pricing formulas and to prove that transformed random variables (e.g., log-returns) follow specific distributions.

๐Ÿ“ Theorem: Computing Moments via MGF

๐ŸŽ“ Theorem 1: Derivative Formula

If \(m(t)\) exists, then for any positive integer \(k\): \[ \boxed{\frac{d^km(t)}{dt^k} \bigg|_{t=0} = m^{(k)}(0) = \mu_k'} \]

Interpretation: The k-th derivative of \(m(t)\) evaluated at \(t=0\) gives the k-th moment.

๐Ÿ’ผ Practical Use

Instead of computing \(\int y^k p(y) dy\) directly (which can be messy), we: 1. Find \(m(t) = E(e^{tY})\) 2. Differentiate \(k\) times 3. Set \(t=0\)

๐Ÿ“Œ Example 1: MGF of Poisson Distribution

๐ŸŽฏ Problem Setup

Find the moment-generating function \(m(t)\) for a Poisson distributed random variable with mean \(\lambda\).

Context: Poisson distributions model counts of events (e.g., number of trades per minute, customer arrivals, insurance claims).

๐Ÿงฎ Example 1: Solution (Part 1)

Starting from the definition: \[ m(t) = E(e^{tY}) = \sum_{y=0}^{\infty} e^{ty} p(y) = \sum_{y=0}^{\infty} e^{ty} \frac{\lambda^y e^{-\lambda}}{y!} \]

Combine the exponential terms: \[ m(t) = \sum_{y=0}^{\infty} \frac{(\lambda e^t)^y e^{-\lambda}}{y!} = e^{-\lambda} \sum_{y=0}^{\infty} \frac{(\lambda e^t)^y}{y!} \]

Recognize that \(\sum_{y=0}^{\infty} \frac{a^y}{y!} = e^a\): \[ m(t) = e^{-\lambda} \cdot e^{\lambda e^t} \]

๐Ÿงฎ Example 1: Solution (Part 2)

Final result: \[ \boxed{m(t) = e^{\lambda(e^t - 1)}} \]

๐Ÿ” Verification Technique

We multiplied and divided by \(e^{\lambda e^t}\) to recognize that: \[ \sum_{y=0}^{\infty} \frac{(\lambda e^t)^y e^{-\lambda e^t}}{y!} = 1 \] This is the sum of a Poisson PMF with parameter \(\lambda e^t\), which equals 1.

Financial Context: This MGF is used in modeling high-frequency trading events and queue theory in financial markets.

๐Ÿ“Œ Example 2: Mean and Variance via MGF

๐ŸŽฏ Problem

Use the MGF from Example 1 to find the mean \(\mu\) and variance \(\sigma^2\) for the Poisson random variable.

Method: Apply Theorem 1 by computing \(m'(0)\) and \(m''(0)\).

๐Ÿงฎ Example 2: Solution (Part 1)

Recall \(m(t) = e^{\lambda(e^t-1)}\).

First derivative (for the mean): \[ m'(t) = \frac{d}{dt}\left[e^{\lambda(e^t-1)}\right] = e^{\lambda(e^t-1)} \cdot \lambda e^t \]

Second derivative (for the second moment): \[ m''(t) = \frac{d}{dt}\left[e^{\lambda(e^t-1)} \cdot \lambda e^t\right] \] \[ = e^{\lambda(e^t-1)} \cdot (\lambda e^t)^2 + e^{\lambda(e^t-1)} \cdot \lambda e^t \]

๐Ÿงฎ Example 2: Solution (Part 2)

Evaluate at \(t=0\):

Mean: \[ \mu = m'(0) = e^{\lambda(e^0-1)} \cdot \lambda e^0 = e^0 \cdot \lambda = \boxed{\lambda} \]

Second moment: \[ \mu_2' = m''(0) = e^0 \cdot \lambda^2 + e^0 \cdot \lambda = \lambda^2 + \lambda \]

Variance: \[ \sigma^2 = E(Y^2) - \mu^2 = \mu_2' - \mu^2 = \lambda^2 + \lambda - \lambda^2 = \boxed{\lambda} \]

๐ŸŽฏ Key Result for Poisson Distribution

For Poisson(\(\lambda\)): Mean = Variance = \(\lambda\)

๐ŸŽฎ Interactive: Visualizing the Poisson MGF

Explore MGF Properties: Adjust \(\lambda\) to see how the MGF \(m(t) = e^{\lambda(e^t - 1)}\) changes, and observe how derivatives at \(t=0\) give moments.

Pedagogical Value:

  • Visualize MGF curve m(t)

  • See first derivative mโ€™(t)

  • Understand t=0 is where moments are extracted

  • Blue curve = MGF, Orange = derivative

๐Ÿ“Œ Example 3: MGF of Exponential Distribution

๐ŸŽฏ Problem

Find the moment-generating function for an Exponential(\(\lambda\)) random variable with PDF \(f(y) = \lambda e^{-\lambda y}\) for \(y > 0\).

Context: Exponential distributions model waiting times (e.g., time until next trade, customer service duration).

๐Ÿงฎ Example 3: Solution

Starting from the definition: \[ m(t) = E(e^{tY}) = \int_0^{\infty} e^{ty} \lambda e^{-\lambda y} dy \]

Combine exponentials: \[ m(t) = \lambda \int_0^{\infty} e^{(t-\lambda)y} dy \]

For \(t < \lambda\), evaluate the integral: \[ m(t) = \lambda \left[\frac{e^{(t-\lambda)y}}{t-\lambda}\right]_0^{\infty} = \lambda \cdot \frac{0 - 1}{t-\lambda} \]

Final result: \[ \boxed{m(t) = \frac{\lambda}{\lambda - t} \quad \text{for } t < \lambda} \]

Practical Application: This MGF is fundamental in queueing theory and option pricing models.

๐Ÿ“Œ Example 4: Using Exponential MGF to Find Moments

๐ŸŽฏ Problem

Use the Exponential MGF \(m(t) = \frac{\lambda}{\lambda - t}\) to find \(E(Y)\), \(E(Y^2)\), and \(\text{Var}(Y)\).

๐Ÿงฎ Example 4: Solution (Part 1)

First derivative (for the mean): \[ m'(t) = \frac{d}{dt}\left[\frac{\lambda}{\lambda - t}\right] = \frac{\lambda}{(\lambda - t)^2} \]

Mean: \[ \mu = m'(0) = \frac{\lambda}{\lambda^2} = \boxed{\frac{1}{\lambda}} \]

Second derivative (for the second moment): \[ m''(t) = \frac{d}{dt}\left[\frac{\lambda}{(\lambda - t)^2}\right] = \frac{2\lambda}{(\lambda - t)^3} \]

Second moment: \[ E(Y^2) = m''(0) = \frac{2\lambda}{\lambda^3} = \boxed{\frac{2}{\lambda^2}} \]

๐Ÿงฎ Example 4: Solution (Part 2)

Variance: \[ \sigma^2 = E(Y^2) - [E(Y)]^2 = \frac{2}{\lambda^2} - \frac{1}{\lambda^2} = \boxed{\frac{1}{\lambda^2}} \]

๐ŸŽฏ Key Result for Exponential Distribution

For Exponential(\(\lambda\)): - Mean: \(E(Y) = \frac{1}{\lambda}\) - Variance: \(\text{Var}(Y) = \frac{1}{\lambda^2}\) - Memoryless property (unique among continuous distributions)

Financial Note: In option pricing, exponential MGFs appear in Lรฉvy process models for asset price dynamics.

๐Ÿ“Œ Example 5: MGF of Normal Distribution

๐ŸŽฏ Problem Setup

Derive the MGF for a Normal(\(\mu, \sigma^2\)) random variable.

Context: The normal distribution is fundamental in finance for modeling returns, though empirical returns often exhibit fat tails.

๐Ÿงฎ Example 5: Solution

For \(Y \sim N(\mu, \sigma^2)\), the MGF is: \[ m(t) = E(e^{tY}) \]

Using properties of the normal distribution and completing the square in the exponent: \[ m(t) = \exp\left(\mu t + \frac{\sigma^2 t^2}{2}\right) \]

๐ŸŽฏ Key Result for Normal Distribution

For \(Y \sim N(\mu, \sigma^2)\): \[ \boxed{m(t) = e^{\mu t + \frac{\sigma^2 t^2}{2}}} \]

Insight: All moments can be extracted from this elegant closed-form MGF!

Taking derivatives at \(t=0\): - \(m'(0) = \mu\) (recovers the mean) - \(m''(0) = \mu^2 + \sigma^2\) (gives \(\text{Var}(Y) = \sigma^2\))

๐Ÿ“ Quiz #1: MGF Definition

Which of the following correctly defines the moment-generating function?

  • \(m(t) = E(e^{tY})\)
  • \(m(t) = E(Y e^t)\)
  • \(m(t) = e^{E(tY)}\)
  • \(m(t) = E(e^Y)\)

๐Ÿ“ Quiz #2: Computing Moments from MGF

If the MGF of a distribution is \(m(t) = e^{3t + 2t^2}\), what is \(E(Y)\)?

  • 3
  • \(1\)
  • \(4\)
  • Cannot determine from this information

๐Ÿ“ Quiz #3: MGF Uniqueness

What does it mean if two random variables have identical MGFs?

  • They have the same probability distribution
  • They have the same mean
  • They have the same variance
  • They are independent

๐Ÿ’ฐ Case Study: Portfolio Return Analysis (Real Data)

๐Ÿ“ˆ Investment Problem

Context: A portfolio manager wants to understand whether returns follow a normal distribution and use MGF theory to assess risk.

Key Questions:

  • What are the moments of portfolio daily returns?
  • Can we estimate the MGF from empirical data?
  • How well does the normal MGF fit observed returns?

๐Ÿ“Š Data Source

We analyze S&P 500 ETF (SPY) daily returns from 2020-2025.

Source: Yahoo Finance API (via quantmod)

Period: January 2020 - November 2025

Data Quality: Adjusted closing prices

Verification: Cross-checked with CRSP database

๐Ÿ“Š Data Loading and Moments Calculation

Code 
# Load required libraries
library(quantmod)
library(tidyverse)
library(knitr)
library(moments)

# Download S&P 500 ETF data
getSymbols("SPY", 
           from = "2020-01-01",
           to = Sys.Date(),
           auto.assign = TRUE)
[1] "SPY"
Code 
# Calculate daily returns (percentage)
spy_returns <- dailyReturn(SPY, 
                           type = "log") * 100

# Convert to data frame
spy_df <- data.frame(
  Date = index(spy_returns),
  Return = as.numeric(spy_returns)
)

# Display first few observations
head(spy_df, 5) %>% 
  kable(digits = 3, 
        caption = "SPY Daily Log Returns (%)")
SPY Daily Log Returns (%)
Date Return
2020-01-02 0.410
2020-01-03 -0.760
2020-01-06 0.381
2020-01-07 -0.282
2020-01-08 0.532
Code 
#Compute first four moments
mu <- mean(spy_df$Return)
sigma <- sd(spy_df$Return)
skew <- skewness(spy_df$Return)
kurt <- kurtosis(spy_df$Return)

#Display summary statistics
cat(sprintf("Mean (ฮผ):        %.4f%%", mu))
Mean (ฮผ):        0.0496%
Code 
cat(sprintf("Std Dev (ฯƒ):     %.4f%%", sigma))
Std Dev (ฯƒ):     1.3245%
Code 
cat(sprintf("Skewness:        %.4f", skew))
Skewness:        -0.5574
Code 
cat(sprintf("Kurtosis:        %.4f", kurt))
Kurtosis:        16.1059
Code 
cat(sprintf("Excess Kurtosis: %.4f", 
            kurt - 3))
Excess Kurtosis: 13.1059
Code 
cat("Sample size:", nrow(spy_df), "days")
Sample size: 1472 days

๐Ÿ“Š MGF Estimation from Empirical Data

Code 
# Function to estimate MGF numerically from data
estimate_mgf <- function(data, t) {
  mean(exp(t * data))
}

# Compute empirical MGF at various t values
t_vals <- seq(-0.3, 0.3, by = 0.02)
empirical_mgf <- sapply(t_vals, function(t) 
  estimate_mgf(spy_df$Return, t))

# Compute normal MGF assuming normal distribution
normal_mgf <- sapply(t_vals, function(t) 
  exp(mu * t + 0.5 * sigma^2 * t^2))

# Create comparison table
mgf_comparison <- data.frame(
  t = t_vals,
  Empirical_MGF = empirical_mgf,
  Normal_MGF = normal_mgf,
  Difference = abs(empirical_mgf - normal_mgf)
)

# Display comparison
head(mgf_comparison, 10) %>%
  kable(digits = 6, 
        caption = "Empirical vs Normal MGF")
Empirical vs Normal MGF
t Empirical_MGF Normal_MGF Difference
-0.30 1.093481 1.066175 0.027306
-0.28 1.076664 1.056428 0.020236
-0.26 1.062284 1.047505 0.014779
-0.24 1.049995 1.039387 0.010608
-0.22 1.039512 1.032056 0.007457
-0.20 1.030606 1.025495 0.005111
-0.18 1.023087 1.019692 0.003396
-0.16 1.016803 1.014633 0.002171
-0.14 1.011628 1.010308 0.001320
-0.12 1.007460 1.006707 0.000753

๐Ÿ“Š Visualization: Empirical vs Normal MGF

๐Ÿ“Š Distribution of Returns

๐Ÿ’ก Case Study Insights

๐ŸŽฏ Key Findings

  1. Negative skewness (โ‰ˆ -0.30 to -0.50) indicates asymmetric downside risk in equity returns

  2. Excess kurtosis (โ‰ˆ 3-5 depending on period) shows fat tails - more extreme events than normal distribution predicts

  3. MGF divergence: Empirical MGF deviates from normal MGF at large |t| values, confirming non-normality

  4. Risk modeling: Normal MGF underestimates tail probabilities; alternative models (Lรฉvy processes) needed for accurate VaR

๐Ÿ’ผ Practical Application

For portfolio construction, using MGF-based risk measures beyond variance captures tail risk more accurately. This is why practitioners use Value-at-Risk (VaR) and Expected Shortfall (ES).

๐Ÿ“ Summary

โœ… Key Takeaways

Moment-generating functions characterize distributions completely: The MGF uniquely identifies a probability distribution; if two random variables share the same MGF, they are identically distributed

MGFs simplify moment calculations: Instead of direct integration, compute \(m^{(k)}(0)\) to find the \(k^{th}\) moment - often algebraically simpler and more numerically stable

Common MGF results: Poisson: \(e^{\lambda(e^t-1)}\), Exponential: \(\frac{\lambda}{\lambda-t}\), Normal: \(e^{\mu t + \frac{\sigma^2 t^2}{2}}\)

Moment relationships: Mean = \(m'(0)\), Variance = \(m''(0) - [m'(0)]^2\), and higher moments follow from higher derivatives

Financial applications: MGFs are essential in derivatives pricing, risk management via characteristic functions, and proving convergence results (e.g., Central Limit Theorem)

๐Ÿ“š Practice Problems

๐Ÿ“ Homework Problems

Problem 1: Find the MGF for a geometric distribution with success probability \(p\). Use it to derive \(E(Y)\) and \(\text{Var}(Y)\).

Problem 2: For the Poisson MGF \(m(t) = e^{\lambda(e^t-1)}\), compute the third derivative \(m'''(t)\) and evaluate at \(t=0\) to find the third moment \(E(Y^3)\).

Problem 3: Show that if \(Y \sim N(\mu, \sigma^2)\) with MGF \(m(t) = e^{\mu t + \frac{\sigma^2 t^2}{2}}\), then \(E(Y^4) = \mu^4 + 6\mu^2\sigma^2 + 3\sigma^4\).

Problem 4: Download 3 years of daily returns for an asset. Estimate the empirical MGF and compare to the MGF of a fitted normal distribution. Discuss where they diverge.

Problem 5: If \(X \sim \text{Poisson}(\lambda)\) and \(Y = 2X + 3\), find the MGF of \(Y\) using the linear transformation properties and verify your answer.

๐Ÿ‘‹ Thank You!

๐Ÿ“ฌ Contact Information:

Samir Orujov, PhD
Assistant Professor
School of Business
ADA University

๐Ÿ“ง Email: sorujov@ada.edu.az
๐Ÿข Office: D312
โฐ Office Hours: By appointment

๐Ÿ“… Next Class:

Topic: Characteristic Functions & Lรฉvy Processes

Reading: Chapter 5, Sections 5.1-5.3

Preparation: Review MGF properties

โฐ Reminders:

โœ… Practice problems due next week
โœ… Study MGF derivations thoroughly
โœ… Work hard!

โ“ Questions?

๐Ÿ’ฌ Open Discussion (5 minutes)

  • Questions about MGF derivations? Letโ€™s work through another example together

  • Confused about the Poisson MGF? Weโ€™ll break down the exponential manipulation step-by-step

  • Want to see more financial applications? We can explore characteristic functions in option pricing

  • Need help with practice problems? Office hours available by appointment