Antimatroid, The

thoughts on computer science, electronics, mathematics

Expected Maximum and Minimum of Real-Valued Continuous Random Variables

Introduction

This is a quick paper exploring the expected maximum and minimum of real-valued continuous random variables for a project that I’m working on. This paper will be somewhat more formal than some of my previous writings, but should be an easy read beginning with some required definitions, problem statement, general solution and specific results for a small handful of continuous probability distributions.

Definitions

Definition (1) : Given the probability space, $(\Omega, \mathcal{F}, \mathbb{P})$, consisting of a set representing the sample space, $\Omega$, a $\text{Borel }\sigma \text{-algebra}$, $\mathcal{F}$, and a Lebesgue measure, $\mathbb{P}$, the following properties hold true:

1. Non-negativity: $\mathbb{P}(F) \ge 0 \quad \forall F \in \mathcal{F}$
2. Null empty set: $\mathbb{P}(\emptyset) = 0$
3. Countable additivity of disjoint sets $\displaystyle \mathbb{P}\left( \bigcup_{i=0}^{\infty} F_i \right) = \sum_{i=0}^{\infty} \mathbb{P}(F_i) \quad F_i \subset \mathcal{F}$

Definition (2) : Given a real-valued continuous random variable such that $X : \Omega \to \mathbb{R}$, the event the random variable takes on a fixed value, $x \in \mathbb{R}$, is the event $\lbrace \omega : X(\omega) = x \rbrace \in \mathcal{F}$ measured by the probability distribution function $f_X(x) = \mathbb{P}(X = x)$. Similarly, the event that the random variable takes on a range of values less than some fixed value, $x \in \mathbb{R}$, is the event $\lbrace \omega : X(\omega) \le x \rbrace \in \mathcal{F}$ measured by the cumulative distribution function $F_X(x) = \mathbb{P}(X \le x)$. By Definition, the following properties hold true:

1. $\displaystyle F_X(x) = \int_{-\infty}^{x} f_X(t) \, dt$
2. $\displaystyle \lim_{x \to \infty} F_X(x) = 1$
3. $\displaystyle 1 - \int_{-\infty}^{x} t f_X(t) \, dt = \int_{x}^{\infty} t f_X(t) \, dt$

Defintion (3) : Given a second real-valued continuous random variable, $Y : \Omega \to \mathbb{R}$, The joint event $\lbrace \omega : X(\omega) = x, Y(\omega) = y \rbrace \in \mathcal{F}$ $(x,y) \in \mathbb{R}^2$ will be measured by the joint probability distribution $f_{X, Y}(x,y) = \mathbb{P}(X = x, Y = y)$. If $X$ and $Y$ are statistically independent, then $f_{X,Y}(x,y) = f_X(x) f_Y(y)$.

Definition (4) : Given a real-valued continuous random variable, $X : \Omega \to \mathbb{R}$, the expected value is $\displaystyle \mathbb{E}(X) = \int_{-\infty}^{\infty} x f_X(x) \, dx$.

Definition (5) : (Law of the unconscious statistician) Given a real-valued continuous random variable, $X$, and a function, $g : \mathbb{R} \to \mathbb{R}$, then $g(X)$ is also a real-valued continuous random variable and its expected value is $\displaystyle \mathbb{E}(g(X)) = \int_{-\infty}^{\infty} g(x) f_X(x) \, dx$ provided the integral converges. Given two real-valued continuous random variables, $X, Y$, and a function, $g : \mathbb{R}^2 \to \mathbb{R}$, then $g(X, Y)$ is also a real-valued continuous random variable and its expected value is $\displaystyle \mathbb{E}(g(X,Y)) = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} g(x,y) f_{X,Y}(x,y) \, dx \, dy$. Under the independence assumption of Definition (3), the expected value becomes $\displaystyle \mathbb{E}(g(X,Y)) = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} g(x,y) f_X(x) f_Y(y) \, dx \, dy$.

Remark (1) : For the remainder of this paper, all real-valued continuous random variables will be assumed to be independent.

Problem Statement

Theorem (1) : Given two real-valued continuous random variables $X, Y \in \Omega \to \mathbb{R}$, then the expected value of the minimum of the two variables is $\mathbb{E} \left( \min{ \left( X, Y \right) } \right ) = \mathbb{E} \left( X \right ) + \mathbb{E} \left( Y \right ) - \mathbb{E} \left( \max{ \left( X, Y \right) } \right )$.

Lemma (1) : Given two real-valued continuous random variables $X, Y \in \Omega \to \mathbb{R}$, then the expected value of the maximum of the two variables is $\displaystyle \mathbb{E} \left( \max{ \left ( X, Y \right) } \right ) = \int_{-\infty}^{\infty} x f_X(x) F_Y(x) \, dx + \int_{-\infty}^{\infty} y f_Y(y) F_X(y) \, dy$

Proof of Lemma (1) :

$\displaystyle \mathbb{E} \left( \max{ \left ( X, Y \right) } \right ) = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \max{\left( x, y \right)} f_X(x) f_Y(y) \, dx \, dy$ (Definition (5))

$\displaystyle = \int_{-\infty}^{\infty} \int_{-\infty}^{x} x f_X(x) f_Y(y) \, dy \, dx + \int_{-\infty}^{\infty} \int_{-\infty}^{y} y f_X(x) f_Y(y) \, dx \, dy$ (Definition (1.iii))

$\displaystyle = \int_{-\infty}^{\infty} x f_X(x) \left ( \int_{-\infty}^{x} f_Y(y) \, dy \right ) \, dx + \int_{-\infty}^{\infty} y f_Y(y) \left ( \int_{-\infty}^{y} f_X(x) \, dx \right ) \, dy$ (Fubini’s theorem)

$\displaystyle = \int_{-\infty}^{\infty} x f_X(x) F_Y(x) \, dx + \int_{-\infty}^{\infty} y f_Y(y) F_X(y) \, dy \quad \square$ (Definition (2.i))

Proof of Theorem (1)

$\displaystyle \mathbb{E} \left( \min{ \left ( X, Y \right) } \right ) = \int_{-\infty}^{\infty} \int_{-\infty}^{\infty} \min{\left( x, y \right)} f_X(x) f_Y(y) \, dx \, dy$ (Definition (4))

$\displaystyle = \int_{-\infty}^{\infty} \int_{x}^{\infty} x f_X(x) f_Y(y) \, dy \, dx + \int_{-\infty}^{\infty} \int_{y}^{\infty} y f_X(x) f_Y(y) \, dx \, dy$ (Definition (1.iii))

$\displaystyle = \int_{-\infty}^{\infty} x f_X(x) \left ( \int_{x}^{\infty} f_Y(y) \, dy \right ) \, dx + \int_{-\infty}^{\infty} y f_Y(y) \left ( \int_{y}^{\infty} f_X(x) \, dx \right ) \, dy$ (Fubini’s theorem)

$\displaystyle = \int_{-\infty}^{\infty} x f_X(x) \left (1 - \int_{-\infty}^{x} f_Y(y) \, dy \right ) \, dx + \int_{-\infty}^{\infty} y f_Y(y) \left (1 - \int_{-\infty}^{y} f_X(x) \, dx \right ) \, dy$ (Definition (2.iii))

$\displaystyle = \int_{-\infty}^{\infty} x f_X(x) \left (1 - F_Y(x) \right ) \, dx + \int_{-\infty}^{\infty} y f_Y(y) \left( 1 - F_X(y) \right ) \, dy$ (Definition (2.i))

$\displaystyle = \int_{-\infty}^{\infty} x f_X(x) \, dx - \int_{-\infty}^{\infty} x f_X(x) F_Y(x) \, dx + \int_{-\infty}^{\infty} y f_Y(y) \, dy - \int_{-\infty}^{\infty} y f_Y(y) F_X(y) \, dy$

$\displaystyle = \int_{-\infty}^{\infty} x f_X(x) \, dx + \int_{-\infty}^{\infty} y f_Y(y) \, dy - \left ( \int_{-\infty}^{\infty} x f_X(x) F_Y(x) \, dx + \int_{-\infty}^{\infty} y f_Y(y) F_X(y) \, dy \right )$

$\displaystyle = \mathbb{E}(X) + \mathbb{E}(Y) - \mathbb{E} \left( \max{ \left( X, Y \right) } \right) \quad \blacksquare$ (Definition (4), Lemma (1))

Remark (2) : For real values $x, y \in \mathbb{R}$, $\min{\left(x,y\right)} = x + y - \max{ \left( x, y \right) }$.

Proof Remark (2) : If $x \ge y$, then $\min{\left(x,y\right)} = y$, otherwise $x$. If $x \ge y$, then $\max{\left(x,y\right)} = x$, otherwise $y$. If $x \ge y$, then $\min{\left(x,y\right)} = y + \left( x - \max{\left(x,y\right)} \right )$, otherwise, $\min{\left(x,y\right)} = x + \left( y - \max{\left(x,y\right)} \right )$. Therefore, $\min{\left(x,y\right)} = x + y - \max{\left(x,y\right)} \quad \square$

Worked Continuous Probability Distributions

The following section of this paper derives the expected value of the maximum of real-valued continuous random variables for the exponential distribution, normal distribution and continuous uniform distribution. The derivation of the expected value of the minimum of real-valued continuous random variables is omitted as it can be found by applying Theorem (1).

Exponential Distribution

Definition (6) : Given a real-valued continuous exponentially distributed random variable, $X \sim \text{Exp}(\alpha)$, with rate parameter, $\alpha > 0$, the probability density function is $\displaystyle f_X(x) = \alpha e^{-\alpha x}$ for all $x \ge 0$ and zero everywhere else.

Corollary (6.i) The cumulative distribution function of a real-valued continuous exponentially distributed random variable, $X \sim \text{Exp}(\alpha)$, is therefore $\displaystyle F_X(x) = 1 - e^{-\alpha x}$ for all $x \ge 0$ and zero everywhere else.

Proof of Corollary (6.i)

$\displaystyle F_X(x) = \int_{-\infty}^{x} f_x(t) \, dt = \int_{0}^{x} \alpha e^{-\alpha t} \, dt = -\frac{\alpha}{\alpha} e^{-\alpha t} \bigg|_{0}^{x} = 1 - e^{- \alpha x} \quad \square$

Corollary (6.ii) : The expected value of a real-valued continuous exponentially distributed random variable, $X \sim \text{Exp}(\alpha)$, is therefore $\displaystyle \frac{1}{\alpha}$.

Proof of Corollary (6.ii)

The expected value is $\mathbb{E}(X) = \frac{1}{\alpha}$ by Definition (4) and Lemma (2) $\square$.

Lemma (2) : Given real values $\alpha, \gamma \in \mathbb{R} \quad \gamma \neq 0$, then $\displaystyle \int_{0}^{\infty} \alpha x e^{-\gamma x} \, dx = \frac{\alpha}{\gamma^2}$.

Proof of Lemma (2) :

$\displaystyle \int_{0}^{\infty} \alpha x e^{-\gamma x} \, dx = - x \frac{\alpha}{\gamma} e^{-\alpha x} \bigg|_{0}^{\infty} + \int_{0}^{\infty} \frac{\alpha}{\gamma} e^{-\gamma x} \, dx = - x \frac{\alpha}{\gamma} e^{-\alpha x} - \frac{\alpha}{\gamma}^2 e^{-\alpha x} \bigg|_{0}^{\infty}$

$\displaystyle = \lim_{x \to \infty} \left( - x \frac{\alpha}{\gamma} e^{-\alpha x} - \frac{\alpha}{\gamma^2} e^{-\alpha x} \right ) - \left( - \frac{\alpha}{\gamma^2} \right) = \frac{\alpha}{\gamma^2} \quad \square$

Theorem (2) : The expected value of the maximum of the real-valued continuous exponentially distributed random variables $X \sim \text{Exp}(\alpha)$, $Y \sim \text{Exp}(\beta)$ is $\displaystyle \frac{1}{\alpha} + \frac{1}{\beta} - \frac{1}{\alpha + \beta}$.

Proof of Theorem (2) :

$\displaystyle \mathbb{E} \left ( \max{\left( X, Y \right)} \right ) = \int_{-\infty}^{\infty} x f_X(x) F_Y(x) \, dx + \int_{-\infty}^{\infty} y f_Y(y) F_X(y) \, dy$ (Lemma (1))

$\displaystyle = \int_{0}^{\infty} x \alpha e^{-\alpha x} \left( 1 - e^{-\beta x} \right ) \, dx + \int_{0}^{\infty} y \beta e^{-\beta y} \left( 1 - e^{-\alpha y} \right ) \, dy$ (Corollary (6.i))

$\displaystyle = \left( \int_{0}^{\infty} x \alpha e^{-\alpha x} \, dx \right )- \left( \int_{0}^{\infty} x \alpha e^{-(\alpha + \beta) x} \, dx \right ) + \left( \int_{0}^{\infty} y \beta e^{-\beta y} \, dy \right ) - \left( \int_{0}^{\infty} y \beta e^{-(\alpha + \beta) y} \, dy \right)$ (Integral linearity)

$\displaystyle = \frac{1}{\alpha} - \frac{\alpha}{(\alpha + \beta)^2} + \frac{1}{\beta} - \frac{\beta}{(\alpha + \beta)^2}$ (Lemma (2), Corollary (6.ii))

$\displaystyle = \frac{1}{\alpha} + \frac{1}{\beta} - \frac{1}{\alpha+\beta} \quad \blacksquare$

Normal Distribution

Definition (7) : The following Gaussian integral is the error function $\displaystyle \text{erf}(x) = \frac{2}{ \sqrt{\pi} } \int_{0}^{x} e^{ - u^2 } \, du$ for which the following properties hold true:

1. Odd function: $\displaystyle \text{erf}(-x) = -\text{erf}(x)$
2. Limiting behavior: $\displaystyle \lim_{x \to \infty} \text{erf}(x) = 1$

Definition (8) : Given a real-valued continuous normally distributed random variable , $X \sim \mathcal{N}(\mu, \sigma)$, with mean parameter, $\mu$. and standard deviation parameter, $\sigma \neq 0$, the probability density function is $\displaystyle f_X(x) = \frac{1}{\sigma \sqrt{2 \pi} } e^{ -\frac{1}{2} \left ( \frac{x - \mu}{\sigma} \right )^2 }$ for all values on the real line.

Corollary (8.i) : The cumulative distribution function of a real-valued continuous normally distributed random variable, $X \sim \mathcal{N}(\mu, \sigma)$, is therefore $\displaystyle F_X(x) = \frac{1}{2} \left (1 + \text{erf} \left ( \frac{x-\mu}{\sqrt{2} \sigma} \right ) \right )$.

Proof of Corollary (8.i) :

$\displaystyle F_X(x) = \int^{x}_{-\infty} \frac{1}{\sigma \sqrt{2 \pi} } e^{ - \left ( \frac{t - \mu}{\sqrt{2} \sigma} \right )^2 } \, dt$ (Definition (2.i))

$\displaystyle = \frac{1}{ \sqrt{\pi} } \int_{-\infty}^{\frac{x - \mu}{\sqrt{2} \sigma}} e^{ - u^2 } \, du$ (U-substitution with $\displaystyle u = \frac{t - \mu}{\sqrt{2} \sigma} \implies du = \frac{1}{\sqrt{2} \sigma} dt$)

$\displaystyle = \frac{1}{ \sqrt{\pi} } \int_{-\infty}^{ 0 } e^{ - u^2 } \, du + \frac{1}{ \sqrt{\pi} } \int_{0}^{ \frac{x - \mu}{\sqrt{2} \sigma} } e^{ - u^2 } \, du$ (Definition (2.iii))

$\displaystyle = - \frac{1}{ \sqrt{\pi} } \int_{0}^{-\infty} e^{ - u^2 } \, du + \frac{1}{ \sqrt{\pi} } \int_{0}^{ \frac{x - \mu}{\sqrt{2} \sigma} } e^{ - u^2 } \, du$ (Reverse limits of integration)

$\displaystyle = \frac{1}{2} \lim_{u \to \infty} - \text{erf}(-u) + \frac{1}{2} \text{erf} \left ( \frac{x - \mu}{\sqrt{2} \sigma} \right )$ (Definition (7))

$\displaystyle = \frac{1}{2} \lim_{u \to \infty} \text{erf}(u) + \frac{1}{2} \text{erf} \left ( \frac{x - \mu}{\sqrt{2} \sigma} \right )$ (Definition (7.i))

$\displaystyle = \frac{1}{2} + \frac{1}{2} \text{erf} \left ( \frac{x - \mu}{\sqrt{2} \sigma} \right )$ (Definition (7.ii))

$\displaystyle = \frac{1}{2} \left (1 + \text{erf} \left ( \frac{x-\mu}{\sqrt{2} \sigma} \right ) \right ) \quad \square$

Corollary (8.ii) : The expected value of a real-valued continuous normally distributed random variable, $X \sim \mathcal{N}(\mu, \sigma)$, is therefore $\mathbb{E}(X) = \mu$.

Proof of Corollary (8.ii) :

$\displaystyle \mathbb{E}(X) = \int_{-\infty}^{\infty} x f_X(x) \, dx = \int_{-\infty}^{\infty} x \frac{1}{\sigma \sqrt{2 \pi} } e^{ -\frac{1}{2} \left ( \frac{x - \mu}{\sigma} \right )^2 } \, dx$ (Definition (4))

$\displaystyle = \int_{-\infty}^{\infty} (\sqrt{2}\sigma u + \mu) \frac{1}{\sqrt{\pi} } e^{ - u^2 } \, du$ (U-substitution with $\displaystyle u = \frac{x-\mu}{\sqrt{2} \sigma} \implies du = \frac{1}{\sqrt{2}\sigma} dx$ $\sqrt{2}\sigma u + \mu = x$)

$\displaystyle = \frac{\sqrt{2}\sigma}{\sqrt{\pi}} \int_{-\infty}^{\infty} u e^{ - u^2 } \, du + \frac{\mu}{\sqrt{\pi}} \int_{-\infty}^{\infty} e^{ - u^2 } \, du$ (Integral linearity)

$\displaystyle = \frac{\sqrt{2}\sigma}{\sqrt{\pi}} \left( \int_{-\infty}^{0} u e^{ - u^2 } \, du + \int_{0}^{\infty} u e^{ - u^2 } \, du \right ) + \frac{\mu}{\sqrt{\pi}} \int_{-\infty}^{\infty} e^{ - u^2 } \, du$ (Definition (1.iii))

$\displaystyle = \frac{\sqrt{2}\sigma}{\sqrt{\pi}} \left( - \int_{0}^{\infty} u e^{ - u^2 } \, du + \int_{0}^{\infty} u e^{ - u^2 } \, du \right ) + 2 \frac{\mu}{\sqrt{\pi}} \int_{0}^{\infty} e^{ - u^2 } \, du$ ($u e^{-u^2}$ is odd, $e^{-u^2}$ is even)

$\displaystyle = \mu \frac{2}{\sqrt{\pi}} \left ( \frac{\sqrt{\pi}}{2} \lim_{x \to \infty} \text{erf}(x) \right ) = \mu \quad \square$ (Definition (7), Definition (7.ii))

Definition (9) : Given a real-valued continuous normally distributed random variable, $X \sim \mathcal{N}(0, 1)$, the probability distribution function will be denoted as standard normal probability distribution function, $\phi(x)$, and the cumulative distribution function as the standard normal cumulative distribution function, $\Phi(x)$. By definition, the following properties hold true:

1. Non-standard probability density function: If $X \sim \mathcal{N}(\mu, \sigma)$, then $\displaystyle f_X(x) = \frac{1}{\sigma} \phi \left( \frac{x - \mu}{\sigma} \right )$
2. Non-standard cumulative distribution function: If $X \sim \mathcal{N}(\mu, \sigma)$, then $\displaystyle F_X(x) = \Phi\left( \frac{x - \mu}{\sigma} \right )$
3. Complement: $\Phi(-x) = 1 - \Phi(x)$

Definition (10) : [PaRe96] Given $\phi(x)$ and $\Phi(x)$, the following integrals hold true:

1. $\displaystyle \int_{-\infty}^\infty x\Phi(a+bx)\phi(x) \, dx = \frac{b}{\sqrt{1+b^2}} \phi \left( \frac{a}{\sqrt{1+b^2}} \right )$
2. $\displaystyle \int_{-\infty}^\infty \Phi(a+bx)\phi(x) \, dx = \Phi \left ( \frac{a}{\sqrt{1+b^2}} \right )$

Theorem (3) : The expected value of the maximum of the real-valued continuous normally distributed random variables $X \sim \mathcal{N}(\mu, \sigma)$, $Y \sim \mathcal{N}(\nu, \tau)$ is $\displaystyle \sqrt{ \sigma^2 + \tau^2 } \phi \left( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2 }} \right ) + (\nu - \mu) \Phi \left ( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2}} \right ) + \mu$.

Lemma (3) : Given real-valued continuous normally distributed random variables $X \sim \mathcal{N}(\mu, \sigma)$, $Y \sim \mathcal{N}(\nu, \tau)$, $\displaystyle \int_{-\infty}^{\infty} y f_Y(y) F_X(y) \, dy = \frac{\tau^2}{\sqrt{\sigma^2 + \tau^2 }} \phi \left( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2 }} \right ) + \nu \Phi \left ( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2}} \right )$.

Proof of Lemma (3) :

$\displaystyle \int_{-\infty}^{\infty} y f_Y(y) F_X(y) \, dy = \int_{-\infty}^{\infty} y \frac{1}{\tau} \phi \left ( \frac{y-\nu}{\tau} \right ) \Phi \left ( \frac{y-\mu}{\sigma} \right ) \, dy$ (Definition (9.i), Definition (9.ii))

$\displaystyle = \int_{-\infty}^{\infty} (u \tau + \nu) \phi(u) \Phi \left ( \frac{u \tau + \nu -\mu}{\sigma} \right ) \, du$ (U-substitution with $\displaystyle u = \frac{y-\nu}{\tau} \implies du = \frac{1}{\tau} dy$, $y = u \tau + \nu$)

$\displaystyle = \tau \int_{-\infty}^{\infty} u \phi(u) \Phi \left ( \frac{u \tau + \nu -\mu}{\sigma} \right ) \, du + \nu \int_{-\infty}^{\infty} \phi(u) \Phi \left ( \frac{u \tau + \nu -\mu}{\sigma} \right ) \, du$ (Integral linearity)

$\displaystyle = \frac{\tau^2}{\sqrt{\sigma^2 + \tau^2 }} \phi \left( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2 }} \right ) + \nu \Phi \left ( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2}} \right ) \, \square$ (Definition (10.i), Definition (10.ii))

Proof of Theorem (3) :

$\displaystyle \mathbb{E} \left( \max{ \left ( X, Y \right) } \right ) = \int_{-\infty}^{\infty} x f_X(x) F_Y(x) \, dx + \int_{-\infty}^{\infty} y f_Y(y) F_X(y) \, dy$ (Lemma (1))

$\displaystyle = \int_{-\infty}^{\infty} x \frac{1}{\sigma} \phi \left ( \frac{x-\mu}{\sigma} \right ) \Phi \left ( \frac{x-\nu}{\tau} \right ) \, dy + \int_{-\infty}^{\infty} y \frac{1}{\tau} \phi \left ( \frac{y-\nu}{\tau} \right ) \Phi \left ( \frac{y-\mu}{\sigma} \right ) \, dy$ (Definition (11.i), Definition (11.ii))

$\displaystyle = \frac{\sigma^2}{\sqrt{\sigma^2 + \tau^2}} \phi \left( \frac{\mu - \nu}{\sqrt{\sigma^2 + \tau^2 }} \right ) + \mu \Phi \left ( \frac{\mu - \nu}{\sqrt{\sigma^2 + \tau^2}} \right ) + \frac{\tau^2}{\sqrt{\sigma^2 + \tau^2 }} \phi \left( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2 }} \right ) + \nu \Phi \left ( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2}} \right )$ (Lemma (3))

$\displaystyle = \frac{\sigma^2}{\sqrt{\sigma^2 + \tau^2}} \phi \left( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2 }} \right ) + \mu \left ( 1 - \Phi \left ( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2}} \right ) \right ) + \frac{\tau^2}{\sqrt{\sigma^2 + \tau^2 }} \phi \left( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2 }} \right ) + \nu \Phi \left ( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2}} \right )$ (Definition (9.iii))

$\displaystyle = \frac{\sigma^2}{\sqrt{\sigma^2 + \tau^2}} \phi \left( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2 }} \right ) - \mu \Phi \left ( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2}} \right ) + \mu + \frac{\tau^2}{\sqrt{\sigma^2 + \tau^2 }} \phi \left( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2 }} \right ) + \nu \Phi \left ( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2}} \right )$

$\displaystyle = \sqrt{ \sigma^2 + \tau^2 } \phi \left( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2 }} \right ) + (\nu - \mu) \Phi \left ( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2}} \right ) + \mu \quad \blacksquare$

Continuous Uniform Distribution

Definition (11) : Given a real-valued continuous uniformly distributed random variable, $X \sim U(a,b)$, with inclusive boundaries $a, b$ such that $a < b$, the probability density function is $\displaystyle f_X(x) = \frac{1}{b-a}$ for all $x \in [a, b]$ and zero everywhere else.

Corollary (11.i) : The cumulative distribution function of a real-valued continuous uniformly distributed random variable, $X \sim U(a,b)$, is therefore $\displaystyle F_X(x) = \begin{cases} 0 & x < a \\ \frac{x-a}{b-a} & x \in [a,b] \\ 1 & x > b \end{cases}$.

Proof of Corollary (11.i) :

$\displaystyle F_X(x) = \int_{-\infty}^{\infty} f_X(t) \, dt = \int_{a}^{b} \frac{1}{b-a} \, dt = \frac{1}{b-a} x \bigg|_{a}^{x} = \begin{cases} 0 & x < a \\ \frac{x-a}{b-a} & x \in [a,b] \\ 1 & x > b \end{cases}\quad \square$.

Corollary (11.ii) : The expected value of a real-valued continuous uniformly distributed random variable, $X \sim U(a,b)$, is therefore $\displaystyle \frac{a+b}{2}$.

Proof of Corollary (11.ii)

$\displaystyle \mathbb{E}(X) = \int_{-\infty}^{\infty} x f_X(x) \, dx = \int_{a}^{b} x \frac{1}{b-a} \, dx = \frac{x^2}{2(b-a)} \bigg|_{a}^{b} = \frac{ b^2 -a^2 }{ 2(b-a) } = \frac{b+a}{2} \quad \square$

Theorem (4) : The expected value of the maximum of real-valued continuous uniformly distributed random variables $X \sim U(a,b)$, $Y \sim U(c,d)$ is $\displaystyle \begin{cases} \frac{c+d}{2} & a < b \le c < d \\ \frac{4 (b^3 - c^3) - 3 (c + a) (b^2 - c^2) }{6(d-c)(b-a)} + \frac{d^2 - b^2}{2(d-c)} & a \le c < b \le d \\ \frac{4(b^3-c^3) - 3(c+a)(b^2-c^2)}{6(b-a)(d-c)} + \frac{b^2-d^2}{2(b-a)} & a \le c < d \le b \\ \frac{4 (b^3-a^3) - 3 (c + a) (b^2 - a^2) }{6(d-c)(b-a)} + \frac{d^2-b^2}{2(d-c)} & c \le a < b \le d\\ \frac{4 (d^3 - a^3) -3 (c+a) (d^2-a^2) }{6(b-a)(d-c)} + \frac{b^2-d^2}{2(b-a)} & c \le a < d \le b \\ \frac{a+b}{2} & c < d \le a < b \end{cases}$.

Proof of Theorem (4) :

$\displaystyle \mathbb{E} \left ( \max{ \left ( X, Y \right )} \right ) = \int_{-\infty}^{\infty} x f_X(x) F_Y(x) \, dx + \int_{-\infty}^{\infty} y f_Y(y) F_X(y) \, dy$ (Lemma (1))

$\displaystyle = \int_{a}^{b} x \frac{1}{b-a} \begin{cases} 0 & x < c \\ \frac{x - c}{d-c} & x \in [c,d] \\ 1 & \text{otherwise} \end{cases} \, dx + \int_{c}^{d} y \frac{1}{d-c} \begin{cases} 0 & y < a \\ \frac{y - a}{b-a} & y \in [a,b] \\ 1 & \text{otherwise} \end{cases} \, dy$

Case (1) : $a < b \le c < d$

$\displaystyle = \left ( \int_{a}^{b} x \frac{1}{b-a} 0 \, dx \right ) + \left ( \int_{c}^{d} y \frac{1}{d-c} 1 \, dy \right )$

$\displaystyle = \frac{c+d}{2} \quad \square$

Case (2) : $a \le c < b \le d$

$\displaystyle = \left ( \int_{a}^{c} x \frac{1}{b-a} 0 \, dx + \int_{c}^{b} x \frac{1}{b-a} \frac{x-c}{d-c} \, dx \right ) + \left ( \int_{c}^{b} y \frac{1}{d-c} \frac{y-a}{b-a} \, dy + \int_{b}^{d} y \frac{1}{d-c} 1 \, dy \right )$

$\displaystyle = \frac{2 x^3 - 3 c x^2}{6(b-a)(d-c)} \bigg|_{c}^{b} + \frac{2 y^3 - 3ay^2 }{6(d-c)(b-a)} \bigg|_{c}^{b} + \frac{y^2}{2(d-c)} \bigg|_{b}^{d}$

$\displaystyle = \frac{2 (b^3 - c^3) - 3 c (b^2 - c^2) }{6(b-a)(d-c)} + \frac{2 (b^3 - c^3) - 3 a (b^2 - c^2) }{6(d-c)(b-a)} + \frac{d^2 - b^2}{2(d-c)}$

$\displaystyle = \frac{4 (b^3 - c^3) - 3 (c + a) (b^2 - c^2) }{6(d-c)(b-a)} + \frac{d^2 - b^2}{2(d-c)} \quad \square$

Case (3) : $a \le c < d \le b$

$\displaystyle = \left ( \int_{a}^{c} x \frac{1}{b-a} 0 \, dx + \int_{c}^{b} x \frac{1}{b-a} \frac{x-c}{d-c} \, dx + \int_{d}^{b} x \frac{1}{b-a} 1 \, dx \right ) + \left ( \int_{c}^{d} y \frac{1}{d-c} \frac{y-a}{b-a} \, dy \right)$

$\displaystyle = \frac{2x^3 - 3cx^2}{6(b-a)(d-c)} \bigg|_{c}^{b} + \frac{x^2}{2(b-a)} \bigg|_{d}^{b} + \frac{2y^3 - 3ay^2}{6(b-a)(d-c)} \bigg|_{c}^{d}$

$\displaystyle = \frac{2(b^3-c^3) - 3c(b^2-c^2)}{6(b-a)(d-c)} + \frac{b^2-d^2}{2(b-a)} + \frac{2(d^3-c^3) - 3a(d^2-c^2)}{6(b-a)(d-c)}$

$\displaystyle = \frac{4(b^3-c^3) - 3(c+a)(b^2-c^2)}{6(b-a)(d-c)} + \frac{b^2-d^2}{2(b-a)} \quad \square$

Case (4) : $c \le a < b \le d$

$\displaystyle = \left( \int_{a}^{b} x \frac{1}{b-a} \frac{x-c}{d-c} \, dx \right ) + \left( \int_{c}^{a} y \frac{1}{d-c} 0 \, dy + \int_{a}^{b} y \frac{1}{d-c} \frac{y-a}{b-a} \, dy + \int_{b}^{d} y \frac{1}{d-c} 1 \, dy \right )$

$\displaystyle = \frac{2 x^3 - 3 c x^2 }{6(d-c)(b-a)} \bigg|_{a}^{b} + \frac{2 y^3 - 3 a y^2 }{6(d-c)(b-a)} \bigg|_{a}^{b} + \frac{y^2}{2(d-c)} \bigg|_{b}^{d}$

$\displaystyle = \frac{2 (b^3-a^3) - 3 c (b^2 - a^2) }{6(d-c)(b-a)} + \frac{2 (b^3-a^3) - 3 a (b^2 -a^2) }{6(d-c)(b-a)} + \frac{d^2-b^2}{2(d-c)}$

$\displaystyle = \frac{4 (b^3-a^3) - 3 (c + a) (b^2 - a^2) }{6(d-c)(b-a)} + \frac{d^2-b^2}{2(d-c)} \quad \square$

Case (5) : $c \le a < d \le b$

$\displaystyle = \left ( \int_{a}^{d} x \frac{1}{b-a} \frac{x-c}{d-c} \, dx + \int_{d}^{b} x \frac{1}{b-a} 1 \, dx \right ) + \left ( \int_{c}^{a} y \frac{1}{d-c} 0 \, dy + \int_{a}^{d} y \frac{1}{d-c} \frac{y-a}{b-a} \, dy \right )$

$\displaystyle = \frac{2 x^3 -3 c x^2}{6(b-a)(d-c)} \bigg|_{a}^{d} + \frac{x^2}{2(b-a)} \bigg|_{d}^{b} + \frac{2 y^3 -3 a y^2}{6(b-a)(d-c)} \bigg|_{a}^{d}$

$\displaystyle = \frac{2 (d^3 - a^3) -3 c (d^2-a^2) }{6(b-a)(d-c)} + \frac{b^2-d^2}{2(b-a)} + \frac{2 (d^3-a^3) -3 a (d^2-a^2)}{6(b-a)(d-c)}$

$\displaystyle = \frac{4 (d^3 - a^3) -3 (c+a) (d^2-a^2) }{6(b-a)(d-c)} + \frac{b^2-d^2}{2(b-a)} \quad \square$

Case (6) : $c < d \le a < b$

$\displaystyle = \left ( \int_{a}^{b} x \frac{1}{b-a} 1 \, dx \right ) + \left ( \int_{c}^{d} y \frac{1}{d-c} 0 \, dy \right )$

$\displaystyle = \frac{a+b}{2}$

$\displaystyle \therefore \mathbb{E} \left ( \max{\left ( X, Y \right )} \right ) = \begin{cases} \frac{c+d}{2} & a < b \le c < d \\ \frac{4 (b^3 - c^3) - 3 (c + a) (b^2 - c^2) }{6(d-c)(b-a)} + \frac{d^2 - b^2}{2(d-c)} & a \le c < b \le d \\ \frac{4(b^3-c^3) - 3(c+a)(b^2-c^2)}{6(b-a)(d-c)} + \frac{b^2-d^2}{2(b-a)} & a \le c < d \le b \\ \frac{4 (b^3-a^3) - 3 (c + a) (b^2 - a^2) }{6(d-c)(b-a)} + \frac{d^2-b^2}{2(d-c)} & c \le a < b \le d\\ \frac{4 (d^3 - a^3) -3 (c+a) (d^2-a^2) }{6(b-a)(d-c)} + \frac{b^2-d^2}{2(b-a)} & c \le a < d \le b \\ \frac{a+b}{2} & c < d \le a < b \end{cases} \quad \blacksquare$

Summary Table

The following summary table lists the expected value of the maximum of real-valued continuous random variables for the exponential distribution, normal distribution and continuous uniform distribution. The corresponding minimum can be obtained by Theorem (1).

Random Variables Maximum
$X \sim$ $Y \sim$
$\text{Exp}(\alpha)$ $\text{Exp}(\beta)$ $\displaystyle \frac{1}{\alpha} + \frac{1}{\beta} - \frac{1}{\alpha + \beta}$
$\mathcal{N}(\mu, \sigma)$ $\mathcal{N}(\nu, \tau)$ $\displaystyle \sqrt{ \sigma^2 + \tau^2 } \phi \left( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2 }} \right ) + (\nu - \mu) \Phi \left ( \frac{\nu - \mu}{\sqrt{\sigma^2 + \tau^2}} \right ) + \mu$
$\text{U}(a, b)$ $\text{U}(c, d)$ $\displaystyle \begin{cases} \frac{c+d}{2} & a < b \le c < d \\ \frac{4 (b^3 - c^3) - 3 (c + a) (b^2 - c^2) }{6(d-c)(b-a)} + \frac{d^2 - b^2}{2(d-c)} & a \le c < b \le d \\ \frac{4(b^3-c^3) - 3(c+a)(b^2-c^2)}{6(b-a)(d-c)} + \frac{b^2-d^2}{2(b-a)} & a \le c < d \le b \\ \frac{4 (b^3-a^3) - 3 (c + a) (b^2 - a^2) }{6(d-c)(b-a)} + \frac{d^2-b^2}{2(d-c)} & c \le a < b \le d\\ \frac{4 (d^3 - a^3) -3 (c+a) (d^2-a^2) }{6(b-a)(d-c)} + \frac{b^2-d^2}{2(b-a)} & c \le a < d \le b \\ \frac{a+b}{2} & c < d \le a < b \end{cases}$

References

[GrSt01] Grimmett, Geoffrey, and David Stirzaker. Probability and Random Processes. Oxford: Oxford UP, 2001. Print.

[PaRe96] Patel, Jagdish K., and Campbell B. Read. Handbook of the Normal Distribution. 2nd ed. New York: Marcel Dekker, 1996. Print.

Written by lewellen

2013-01-01 at 8:00 am

9 Responses

1. Can you generalize your result to the case of n R.V.’s?

T

2013-05-20 at 4:32 am

2. T, I haven’t taken the time to generalize these results, but one could use the techniques presented -or research similar techniques- to obtain further results.

lewellen

2013-05-21 at 5:26 pm

3. It appears that the proof of Theorem 1 is needless. You can use Remark(2) (which holds for r.v’s also), and its proof, and then apply directly the expected value operator which, due to its linearity, will give you the desired result.

2013-06-13 at 1:42 pm

4. Thanks for the insight Alecos. At the time I wrote this, it wasn’t immediately clear to me how I could formalize that leap from real valued variables to continuous real valued random variables. Consequently, I decided to approach the problem of determining the expected maximum (minimum resp.) using the Law of the Unconscious Statistician.

lewellen

2013-06-13 at 5:21 pm

5. We formally say “a random variable is a real-valued function”. But we never really treat it as a function, i.e. we don’t write “x is a random variable, x = g(z)”. We essentially deal with its range (and with other functions describing characteristics of its range -this is what a density and a distribution function does), not with the “function” itself as a mathematical object. So when we assume x>y, even if we say that x and y are “random variables, i.e. functions”, what we have written is translated as “the values x can take are always greater than the values y can take”, or more formally, min(x)>max(y). This is a statement in real-number space. Then max{x,y} = x (It would not be so straightforward to write x=g(z), y=h(d), g(z)>h(d) => max{g(z), h(d)} = g(z) , because this would be a statement in function space, not in real-number space). Expected values are integrals of functions that are integrated over the range of an r.v. – the r.v. as a function itself does not enter the picture.

2013-06-13 at 7:12 pm

6. Typo in theorem 1 E(X)+E(X)… should be E(X) + E(Y)…, no?

MichaelC

2015-03-26 at 7:58 pm

7. Certainly -but I never saw it until now. One of those instances where you read what you know to be correct, and not what is written…

2015-03-26 at 8:12 pm

8. MichaelC, thanks for spotting the typo. It’s been fixed.

lewellen

2015-03-26 at 8:20 pm

9. I didn’t say it in the comments but very nice work and helpful to me in a problem I’m working on. Thanks.

Sent from my iPhone

>

Michael COLACINO

2015-03-27 at 7:17 am