Mathematical Statistics

Bivariate and Multivariate Probability Distributions

Samir Orujov, PhD

ADA University, School of Business

Information Communication Technologies Agency, Statistics Unit

2026-09-24

🎯 Learning Objectives

By the end of this lecture, you will be able to:

• Explain why two marginal probabilities do not determine the probability of a joint event

• Construct a joint probability function from a sample space, and check it against Theorem 5.1

• Evaluate a joint distribution function \(F(y_1, y_2)\) from a table, including the rectangle rule of Theorem 5.2

• Compute a joint probability as a double integral of a joint density over a sketched region

• Extend the definitions from two random variables to \(n\)

🗺️ Where We Are

Wackerly §5.1–5.2 — the start of Chapter 5

Saturday closed Chapter 4: one continuous random variable at a time — its density, its expected values, its MGF, and Tchebysheff’s bound on how far it strays.

Every model so far has described one quantity. A loan book, a portfolio, a payment system is never one quantity.

Today the object becomes a pair \((Y_1, Y_2)\), and the question becomes how two random variables behave together. This chapter carries the rest of the course.

❓ The Question This Lecture Answers

Two SME borrowers of a Baku bank, both in construction, each default within the year with probability \(0.07\). What is the probability that both default?

• If they are unrelated: \(0.07 \times 0.07 = 0.0049\)
• If they always move together: \(0.07\)

A fourteen-fold range, and the two marginals cannot choose between them. The answer lives in the joint distribution, which §5.2 defines.

§5.1: every sample \((y_1, \ldots, y_n)\) is also such an intersection, \((Y_1 = y_1, \ldots, Y_n = y_n)\). We build the tools with \(n = 2\), then extend.

📐 Definition 5.1 and Theorem 5.1

Definition 5.1

Let \(Y_1\) and \(Y_2\) be discrete random variables. The joint (or bivariate) probability function for \(Y_1\) and \(Y_2\) is \[p(y_1, y_2) = P(Y_1 = y_1, Y_2 = y_2), \quad -\infty < y_1 < \infty,\ -\infty < y_2 < \infty.\]

Theorem 5.1

1. \(p(y_1, y_2) \ge 0\) for all \(y_1, y_2\).
2. \(\sum_{y_1, y_2} p(y_1, y_2) = 1\), summing over all pairs given nonzero probability.

🏦 Routing Two Loan Applications

Two applications are each sent at random, independently, to committee 1 (Corporate), 2 (SME) or 3 (Retail). \(Y_1\) = number sent to Corporate, \(Y_2\) = number to SME. All \(3 \times 3 = 9\) routings \(\{i, j\}\) are equally likely.

	\(y_1 = 0\)	\(y_1 = 1\)	\(y_1 = 2\)
\(y_2 = 0\)	\(1/9\)	\(2/9\)	\(1/9\)
\(y_2 = 1\)	\(2/9\)	\(2/9\)	\(0\)
\(y_2 = 2\)	\(1/9\)	\(0\)	\(0\)

\(p(1,1) = 2/9\): both \(\{1,2\}\) and \(\{2,1\}\) give it. The zeros say \(Y_1 + Y_2 \le 2\): the support is a region, not a rectangle.

📈 Joint Returns of Two Assets

Monthly returns (%) on a Baku-listed bank share, \(Y_1\), and an oil-sector bond fund, \(Y_2\):

	\(y_2 = -1\)	\(y_2 = 1\)	\(y_2 = 3\)
\(y_1 = -2\)	\(0.10\)	\(0.08\)	\(0.02\)
\(y_1 = 0\)	\(0.06\)	\(0.30\)	\(0.09\)
\(y_1 = 4\)	\(0.02\)	\(0.12\)	\(0.21\)

• Both gain: \(p(4,1) + p(4,3) = 0.33\). Both lose: \(p(-2,-1) = 0.10\).
• An equal-weight portfolio gains when \(y_1 + y_2 > 0\): six cells, total \(0.76\).

💻 The Table in R

p <- matrix(c(0.10, 0.08, 0.02,
              0.06, 0.30, 0.09,
              0.02, 0.12, 0.21), nrow = 3, byrow = TRUE,
            dimnames = list(y1 = c(-2, 0, 4), y2 = c(-1, 1, 3)))
y1 <- as.numeric(rownames(p)); y2 <- as.numeric(colnames(p))
Fjoint <- function(a, b) sum(p[y1 <= a, y2 <= b])     # Definition 5.2
c(min = min(p), total = sum(p),                        # Theorem 5.1
  both_gain = sum(p[y1 > 0, y2 > 0]),
  port_gain = sum(p[outer(y1, y2, "+") > 0]),
  F01 = Fjoint(0, 1), F4m1 = Fjoint(4, -1))

      min     total both_gain port_gain       F01      F4m1 
     0.02      1.00      0.33      0.76      0.54      0.18 

🖼️ Mass Sits on the Diagonal

The large masses run from bottom left to top right: the two assets tend to move together.

📐 Definition 5.2: The Joint CDF

Definition 5.2

For any random variables \(Y_1\) and \(Y_2\), the joint distribution function is \(F(y_1, y_2) = P(Y_1 \le y_1, Y_2 \le y_2)\). For discrete variables, \(F(y_1, y_2) = \sum_{t_1 \le y_1} \sum_{t_2 \le y_2} p(t_1, t_2)\).

On the returns table:

• \(F(0, 1) = 0.10 + 0.08 + 0.06 + 0.30 = 0.54\)
• \(F(0.5, 1.5) = 0.54\) as well — no new mass between the grid points
• \(F(-3, 5) = 0\) and \(F(5, 5) = 1\)

📐 Theorem 5.2: Properties of \(F\)

Theorem 5.2

1. \(F(-\infty, -\infty) = F(-\infty, y_2) = F(y_1, -\infty) = 0\).
2. \(F(\infty, \infty) = 1\).
3. If \(y_1^* \ge y_1\) and \(y_2^* \ge y_2\), then \(F(y_1^*, y_2^*) - F(y_1^*, y_2) - F(y_1, y_2^*) + F(y_1, y_2) \ge 0\).

Part 3 is \(P(y_1 < Y_1 \le y_1^*,\ y_2 < Y_2 \le y_2^*)\): the rectangle rule. On the returns table, \[P(-2 < Y_1 \le 4,\ -1 < Y_2 \le 3) = 1 - 0.18 - 0.20 + 0.10 = 0.72,\] which is the four cells \(0.30 + 0.09 + 0.12 + 0.21\).

📝 Quiz #1: Reading \(F\) From a Table

In the joint returns table, what is \(F(4, -1) = P(Y_1 \le 4, Y_2 \le -1)\)?

• \(0.18\)
• \(0.02\)
• \(0.10\)
• \(1\)

📐 Definition 5.3 and Theorem 5.3

Definition 5.3

If there is a nonnegative \(f(y_1, y_2)\) with \(F(y_1, y_2) = \int_{-\infty}^{y_1} \int_{-\infty}^{y_2} f(t_1, t_2)\, dt_2\, dt_1\) for all \(y_1, y_2\), then \(Y_1, Y_2\) are jointly continuous, and \(f\) is the joint probability density function.

Theorem 5.3

1. \(f(y_1, y_2) \ge 0\) for all \(y_1, y_2\).
2. \(\int_{-\infty}^{\infty} \int_{-\infty}^{\infty} f(y_1, y_2)\, dy_1\, dy_2 = 1\).

Probabilities are now volumes: \(P(a_1 \le Y_1 \le a_2,\ b_1 \le Y_2 \le b_2) = \int_{b_1}^{b_2} \int_{a_1}^{a_2} f(y_1, y_2)\, dy_1\, dy_2\).

⏱️ Two Settlement Times

An AZN/USD foreign-exchange trade settles in two legs. \(Y_1\) = hours until the first leg settles, \(Y_2\) = hours until the second. By construction \(Y_1 \le Y_2\). Suppose \[f(y_1, y_2) = \begin{cases} 2e^{-(y_1 + y_2)}, & 0 \le y_1 \le y_2 < \infty,\\ 0, & \text{elsewhere.} \end{cases}\]

Check Theorem 5.3. \(f \ge 0\), and, integrating \(y_1\) first over \(0 \le y_1 \le y_2\), \[\int_0^\infty 2e^{-y_2}\left(1 - e^{-y_2}\right) dy_2 = 2\left(1 - \tfrac{1}{2}\right) = 1.\]

✏️ Worked Example: Two Probabilities

Both legs settled within one hour. Only \(Y_2 \le 1\) bites, because \(Y_1 \le Y_2\): \[F(1, 1) = \int_0^1 \int_0^{y_2} 2e^{-(y_1 + y_2)}\, dy_1\, dy_2 = 2(1 - e^{-1}) - (1 - e^{-2}) = (1 - e^{-1})^2 = 0.3996\]

First leg within 30 minutes, second after an hour. The rectangle \(0 \le y_1 \le 0.5,\ y_2 > 1\) lies wholly inside the support: \[\int_0^{0.5} \int_1^{\infty} 2e^{-(y_1 + y_2)}\, dy_2\, dy_1 = 2e^{-1}\left(1 - e^{-0.5}\right) = 0.2895\]

⚠️ Settlement Risk: The Gap

While one leg has settled and the other has not, the bank is exposed to its counterparty. What is \(P(Y_2 - Y_1 > 1)\)?

Sketch the region first. It is the part of the support above the line \(y_2 = y_1 + 1\). For each \(y_1\), \(y_2\) runs from \(y_1 + 1\) to \(\infty\): \[\int_0^{\infty} \int_{y_1 + 1}^{\infty} 2e^{-(y_1 + y_2)}\, dy_2\, dy_1 = \int_0^{\infty} 2e^{-1}e^{-2y_1}\, dy_1 = e^{-1} = 0.3679\]

More than a third of trades leave the bank exposed for over an hour.

💻 Checking the Integrals by Simulation

# Two independent exponential(1) legs; the earlier is Y1, the later Y2.
# This pair has exactly the joint density 2exp(-(y1 + y2)) on 0 <= y1 <= y2.
n  <- 100000
a  <- rexp(n); b <- rexp(n)
Y1 <- pmin(a, b); Y2 <- pmax(a, b)
rbind(simulated = c(F_1_1 = mean(Y1 <= 1 & Y2 <= 1),
                    rect  = mean(Y1 <= 0.5 & Y2 > 1),
                    gap   = mean(Y2 - Y1 > 1)),
      exact     = c((1 - exp(-1))^2, 2 * exp(-1) * (1 - exp(-0.5)), exp(-1))) |>
  round(4)

           F_1_1   rect    gap
simulated 0.3981 0.2916 0.3689
exact     0.3996 0.2895 0.3679

A hundred thousand simulated trades land within \(0.003\) of all three integrals — the region and the limits were right.

🖼️ Where the Gap Lives

No point falls below the solid line \(y_2 = y_1\); the red points above the dashed line are the event.

🧠 Think-Pair-Share

Back to the two construction borrowers. \(Y_i = 1\) if borrower \(i\) defaults, \(0\) if not:

	\(y_2 = 0\)	\(y_2 = 1\)
\(y_1 = 0\)	\(0.88\)	\(0.05\)
\(y_1 = 1\)	\(0.05\)	\(c\)

Four minutes, in pairs: (1) find \(c\); (2) find the probability that at least one defaults; (3) compare \(P(\text{both})\) with \(0.07 \times 0.07\).

✅ Think-Pair-Share: Solution

1. Theorem 5.1: the entries sum to 1, so \(c = 1 - 0.88 - 0.05 - 0.05 = 0.02\). Each borrower still defaults with probability \(0.05 + 0.02 = 0.07\).

2. “At least one” fails only at \((0, 0)\): \(1 - p(0,0) = 1 - 0.88 = 0.12\).

3. \(P(Y_1 = 1, Y_2 = 1) = 0.02\), about four times the \(0.0049\) that unrelated borrowers would give.

Same marginals, very different risk of a double loss. That is the opening question answered: only the joint table could say.

📝 Quiz #2: Making a Density Valid

A credit line’s limit \(Y_1\) and the amount drawn \(Y_2\) (million AZN) have constant density \(f(y_1, y_2) = k\) on the triangle \(0 \le y_2 \le y_1 \le 2\). What is \(k\)?

• \(1/2\)
• \(1\)
• \(1/4\)
• \(2\)

📋 Key Formulas

	Statement
Definition 5.1	\(p(y_1, y_2) = P(Y_1 = y_1, Y_2 = y_2)\)
Theorem 5.1	\(p \ge 0\), \(\ \sum_{y_1, y_2} p(y_1, y_2) = 1\)
Definition 5.2	\(F(y_1, y_2) = P(Y_1 \le y_1, Y_2 \le y_2)\)
Theorem 5.2, part 3	\(P(y_1 < Y_1 \le y_1^*, y_2 < Y_2 \le y_2^*) = F(y_1^*, y_2^*) - F(y_1^*, y_2) - F(y_1, y_2^*) + F(y_1, y_2)\)
Theorem 5.3	\(f \ge 0\), \(\ \iint f(y_1, y_2)\, dy_1\, dy_2 = 1\)
\(n\) variables	\(F(y_1, \ldots, y_n) = \int_{-\infty}^{y_1} \cdots \int_{-\infty}^{y_n} f(t_1, \ldots, t_n)\, dt_n \cdots dt_1\)

📋 Summary

• Marginal probabilities alone cannot price a joint event; the joint distribution can

• A joint probability function is a table of nonnegative entries summing to 1

• \(F(y_1, y_2)\) accumulates mass below and to the left; differences of \(F\) give rectangles

• For jointly continuous variables, probabilities are volumes under \(f(y_1, y_2)\)

• Sketch the support and the event before writing the limits

• Everything extends to \((Y_1, \ldots, Y_n)\) — the setting of every sample

📚 Practice Problems

Wackerly, 7th edition

• Exercises at the end of §5.2 — start with 5.1, 5.3 and 5.4 (tables), then 5.5, 5.7, 5.9 and 5.11 (densities: sketch every region first)

• On the settlement model, find \(P(Y_1 + Y_2 < 2)\) and check it with the simulation code above

Week 12, Problem Set 1 is open now and closes Sunday 6 December at 23:59 on WeBWorK, covering §5.1–5.2.

Next class: 28 November — marginal and conditional probability distributions (Wackerly §5.3): how to recover each borrower’s 7% from the joint table, and what one default says about the other.

🙏 Thank You

Dr. Samir Orujov
📧 sorujov@ada.edu.az
🏢 Building D, Room D325
🕓 Office hours: Wednesday, 16:00 – 18:00

Slides and readings: sorujov.net/teaching

❓ Questions

• Given two marginal default probabilities of \(0.07\), what are the largest and smallest values \(P(\text{both default})\) can take?

• Why must a joint density’s support be sketched before the limits are written, when a single density’s never had to be?

• In the settlement model, is \(P(Y_2 - Y_1 > h)\) a shape worth knowing for every \(h\)? Guess its form from \(h = 1\).