[Home]
Last updated on: Sun Aug 09 14:21:36 IST 2026

Convergence of random variables

$\newcommand{\v}[1]{\boldsymbol{#1}}$ $\newcommand{\io}{~~\mathrm{i.o.}}$ $\newcommand{\calF}{\mathcal{F}}$ $\newcommand{\rightarrowL}[1]{\stackrel{L_{#1}}{\longrightarrow}}$ $\newcommand{\rightarrowP}{\stackrel{P}{\longrightarrow}}$ $\newcommand{\rightarrowD}{\stackrel{d}{\longrightarrow}}$ $\newcommand{\rightarrowA}{\stackrel{a.s.}{\longrightarrow}}$ Convergence of random variables The main theme of probability theory is exploring statistical regularity. This phenomenon is all about the limiting behaviour of a sequence of random variables. So it is only natural that convergence of random variables occupies a central position in probability theory. What is surprising, however, is that a sequence random variables may converge in many different ways. So there is no single definition of a "limit of a sequence of random variables". In this page we shall define 4 different notions of convergence of a sequence of random variables:
  1. Almost sure convergence
  2. convergence in probability
  3. convergence in distribution
  4. $L_p$ convergence
Each captures one aspect of our intuitive idea about convergence of random variables.

1. Almost sure convergence

Since random variables are just functions (from a probability space $(\Omega,\calF,P)$ to ${\mathbb R}$), the very first definition of convergence that comes to our mind is pointwise convergence, i.e., we want
"$X_n\rightarrow X$"
to mean
"$\forall \omega\in \Omega~~X_n(\omega)\rightarrow X(\omega).$"
However, we also know that random variables are used only in probability computation, and so if we take some $A\subseteq \Omega$ such that $P(A)=0,$ then the behaviour of a random variable over $A$ "does not really matter". Hence instead of wanting to have $X_n(\omega)\rightarrow X(\omega)$ everywhere (i.e., $\forall \omega\in \Omega$) it is enough to have it almost everywhere (i.e., $\forall \omega\in$ a set of probability 1).

Definition: Almost sure convergence Let $(X_n),X$ be random variables all defined on some common probability space $(\Omega,\calF, P).$ Then we say that $(X_n)$ converges to $X$ almost surely (or almost everywhere) and write $X_n\rightarrowA X$ if $P(X_n\rightarrow X) = 1.$

EXAMPLE 1: Let our probability space be $Unif[0,1]$ with $\Omega = {\mathbb R}.$ Let $X_n(\omega) =\omega^n$, and let $X(\omega)\equiv 0.$ Then $X_n\rightarrowA X.$

However, if we replace $Unif(0,1)$ with $Unif(0,2),$ then the convergence breaks down. ■

Theorem Let $X_n,Y_n,X,Y$ be all random variables defined on a common probability space. Let $a,b$ be fixed numbers. If $X_n\rightarrowA X$ and $Y_n\rightarrowA Y,$ then
  1. $aX_n+b\rightarrowA aX+b$.
  2. $X_n\pm Y_n\rightarrowA X\pm Y.$
  3. $X_nY_n\rightarrowA XY.$
  4. $X_n/ Y_n\rightarrowA X/Y$ if $P(Y\neq 0)=1.$

Proof: Let $A=\{\omega~:~X_n(\omega)\rightarrow X(\omega)\}$ and $B=\{\omega~:~Y_n(\omega)\rightarrow Y(\omega)\}.$

Then $P(A)=1$ and $P(B)=1$ and hence $P(A\cap B)=1.$

Now apply the corresponding results from real analysis for each $\omega\in A$ (for 1) and for $\omega\in A\cap B$ (for 2 and 3).

For $4,$ let $C=\{\omega~:~Y(\omega)\neq 0\}.$ Then $P(A\cap B\cap C)=1.$ Now apply the corresponding result from real analysis for each $\omega\in C.$ [QED]

Possibly the most famous use of almost sure convergence in exploring statistical regularity is the following theorem.

1.1. Strong law of large numbers (SLLN)

SLLNLet $(X_n)$ be an iid sequence of random variables with $E(X_1) = \mu \in{\mathbb R}.$ Let $\bar X_n = \frac 1n(X_1+\cdots+X_n).$

Then $\bar X_n \rightarrowA \mu.$

Proof:Skipped.[QED]

This is essentially what we saw when we did the coin tossing example of statistical regularity. Though we have skipped the proof we have a project that explores some ideas related to the proof.

1.2. Problem set

EXERCISE 1: Let $N$ be an ${\mathbb N}$-valued random variable. Let $(X_n)$ be a sequence of random variables defined on the same probability space as follows. $$X_n(\omega) =\left\{\begin{array}{ll}1&\text{if }n\leq N(\omega)\\ 0&\text{otherwise.}\end{array}\right.. $$ Show that $X_n\rightarrowA 0.$

EXERCISE 2:  Let $(A_n)$ be a sequence of events in some probability space such that $P(A_n)=\frac{1}{n^2}.$ Show that $I_{A_n}\rightarrowA 0.$ Here $I_{A_n}$ is the indicator variable for $A_n.$

EXERCISE 3:  Let $(A_n)$ be a sequence of independent events in some probability space such that $P(A_n)=\frac 1n.$ Check if $I_{A_n}\rightarrowA 0.$

EXERCISE 4:  Let $(X_n)$ be iid $Bernoulli(p)$ for some $p\in (0,1).$ Let $Y_n = \max_{1\leq k\leq n} X_k.$ Show that $Y_n\rightarrowA 1$.

EXERCISE 5:  Let $(X_n)$ be iid $Unif[0,1].$ Let $Y_n = \max_{1\leq k\leq n} X_k.$ Show that $Y_n\rightarrowA 1$.

EXERCISE 6:  Let $(X_n)$ be iid $Poi(\lambda).$ Let $Y_n = \frac 1n\sum_1^n X_k^2.$ Show that $(Y_n)$ converges a.s. Find the limit.

EXERCISE 7:  Let $(X_n)$ be iid $Poi(\lambda).$ Let $Y_n = \frac 1n\sum_1^n (X_k-1)^2.$ Show that $(Y_n)$ converges a.s. Find the limit.

EXERCISE 8: Let $(X_n)$ be iid with finite variance $\sigma^2.$ Show that $$\frac{1}{n-1}\sum_1^n (X_k-\bar X_n)^2\rightarrowA \sigma^2.$$

EXERCISE 9: Let $(X_n)$ be iid $Unif(1,2).$ Let $G_n$ be the geometric mean of the first $n$ of them, i.e., $$G_n = \left(\prod_1^n X_k\right)^{\frac 1n}.$$ Show that $G_n$ converges a.s. Find the limit.

EXERCISE 10: Let $(X_n)$ be an independent sequence of random variables with $$P(X_n=n) = \frac{1}{n^2}\mbox{ and } P(X_n=0) = 1-\frac{1}{n^2}.$$ Show that $X_n\rightarrowA 0.$

2. Convergence in probability

This is historically the first notion of convergence of random variables that was defined mathematically.

Definition: Probability convergence Let $(X_n)$ be a sequence of random variables all defined on some common probability space $\Omega,\calF, P).$ Let $X$ be some random variable also defined on the same probability space. Then we say that $(X_n)$ converges to $X$ in probability and write "$X_n\rightarrowP X$ if $$\forall \epsilon>0~~P(|X_n-X|> \epsilon)\rightarrow 0.$$ It is also called weak convergence.
The condition that all the $X_n$'s should be defined on a common probability space may be relaxed if $X$ is a degenerate random variable.

Theorem Let $X_n,Y_n,X,Y$ be all random variables defined on a common probability space. Let $a,b$ be fixed numbers. If $X_n\rightarrowP X$ and $Y_n\rightarrowP Y,$ then
  1. $aX_n+b\rightarrowP aX+b$.
  2. $X_n\pm Y_n\rightarrowP X\pm Y.$
  3. $X_nY_n\rightarrowP XY.$
  4. $X_n/ Y_n\rightarrowP X/Y$ if $P(Y\neq 0)=1.$

2.1. Weak law of large numbers (WLLN)

The coin tossing example of statistical regularity had intrigued many people. The first ever proven result on statistical regularity was formulated and proved by Jacob Bernoulli "after having meditated on it for twenty years". The notion of convergence that he used was convergence in probability. The following theorem is a slightly general version of what he had proved after twenty years of work.

Weak Law of Large Numbers (WLLN) Let $X_1,X_2,...$ be a sequence of iid random variables with $E(X_1)=\mu$ and $V(X_1)=\sigma^2<\infty.$ Let, for $n\in{\mathbb N}$, $$ \overline X_n = \frac 1n\sum_{i=1}^n X_i. $$ Then $\bar X_n\rightarrowP \mu.$

Proof: To show $$\forall \epsilon > 0~~ P(|\overline X_n-\mu|> \epsilon) \rightarrow 0\mbox{ as } n\rightarrow \infty.$$ Take any $\epsilon > 0.$

Applying Chebyshev inequality to the random variable $\bar X_n$ we have $$P(|\overline X_n-\mu|> \epsilon) \leq \frac{V(\overline X_n)}{\epsilon^2}.$$ Now $$V(\overline X_n) = V\left(\frac 1n\sum_1^n X_i\right) = \frac{1}{n^2} \sum_1^n V(X_i),$$ since $X_i$'s are independent. Also each $V(X_i) = \sigma^2.$ Hence $$V(\overline X_n) = V\left(\frac 1n\sum_1^n X_i\right) = \frac{\sigma^2}{n}.$$ Hence we have $$P(|\overline X_n-\mu|> \epsilon) \leq \frac{\sigma^2}{n\epsilon^2}\rightarrow 0,$$ as $n\rightarrow \infty.$ [QED]

2.2. Problem set

EXERCISE 11: If $X_n\rightarrowP X,$ show that $-X_n\rightarrowP -X.$

EXERCISE 12: Let $X_n\rightarrowP X$ (random sequence) and $a_n\rightarrow a$ (fixed sequence). Show that $X_n+a_n\rightarrowP X+a.$

EXERCISE 13: Let $X_n\rightarrowP X$ (random sequence) and $a_n\rightarrow a$ (fixed sequence). Show that $a_nX_n\rightarrowP aX.$

EXERCISE 14: If $X_n\rightarrowP X,$ show that $|X_n|\rightarrowP |X|.$

EXERCISE 15: If $X_n\rightarrowP X,$ and $f:{\mathbb R}\rightarrow{\mathbb R}$ is a continuous function, then show that $f(X_n)\rightarrowP f(X).$

EXERCISE 16: Suppose $P(X_n=n)=\frac 1n$ and $P(X_n=0)=1-\frac 1n.$ Then must we have $X_n\rightarrowP 0$?

EXERCISE 17: If $E(X_n)=0$ and $V(X_n) = \left(\frac 1n\right)^{1/3},$ then show that $X_n\rightarrowP 0.$

EXERCISE 18: Let $(X_n)$ be an iid sequence of Cauchy random variables. Let $\bar X_n$ denote the mean of the first $n$ of them. Show that $\bar X_n$ does not converge in probability.

Hint:

If $X,Y$ are independent Cauchy and $\alpha\in (0,1)$ is fixed, then $\alpha X+(1-\alpha) Y$ again has Cauchy distribution.

3. $L_p$ convergence for $p\geq1$

When we think of a sequence $(X_n)$ of random variables converging to a random variable $X,$ we tend to believe that for large enough $n$ the random variable $X_n$ behaves "just like" $X.$ In particular, $E(X_n)$ should be close to $E(X)$ (assuming the expectations exist). However, that is not the general case. We shall see such counterexamples soon, but first let us learn about a new mode of convergence, where convergence of expectation (or higher moments) is required by the definition.

Definition: $L_p$ convergence Let $p\geq 1.$ Let $(X_n), X$ be random variables defined on some probability space. We say that $(X_n)$ converges to $X$ in $p$-th moment or in $L_p$ if $E\big(|X_n-X|^p\big)\rightarrow 0.$

We write this as $X_n\rightarrowL p X.$

A couple of points are in order here:
  1. Why are we putting the $p\geq 1$ condition?
  2. Would it not be more natural to write $E(X_n^p)\rightarrow E(X^p)$ instead of $E|X_n-X|^p?$
The answer to the first question lies in the following inequality:
Minkowski inequality Let $1\leq p < \infty.$ Let $X,Y$ have finite $p$-th moment. Then $$ E|X+Y|^p\leq E|X|^p+E|Y|^p. $$ This is a form of the triangle inequality.
This inequality makes $L_p$ convergence well-behaved for $p\geq 1.$ Unfortunately, the inequality does not hold in general for $p < 1.$

Now let us explore the second question using $p=1.$

EXAMPLE 2:  Let $(X_n), X $ be random variables on the same probability space. Consider the following two statements:

One of these is strictly stronger. Which one?

SOLUTION: The first one is strictly stronger. $$ |E(X_n)-E(X)| \leq |E(X_n-X)|\leq E|X_n-X|\rightarrow 0. $$ Consider $X_n\equiv (-1)^n$ and $X\equiv 1.$ ■

If you ever need $E(X_n)\rightarrow E(X)$ and $E(X_n^2)\rightarrow E(X^2),$ then you do not need to assume $L_1$ and $L_2$ convergence separately. $L_2$ automatically implies $L_1.$ Indeed, we have the following theorem.

Theorem Let $p > q \geq 1.$ Then if $X_n\rightarrowL p X$ we must also have $X_n\rightarrowL q X.$

Proof:Shall not prove in this course.[QED]

I mentioned earlier that Minkowski inequality keeps $L_p$ convergence well-behaved for $p\geq 1.$ The next theorem is one example.

Theorem Let $X_n,Y_n,X,Y$ be all random variables defined on a common probability space. Let $a,b$ be fixed numbers. Let $p\in[1,\infty).$ If $X_n\rightarrowL p X$ and $Y_n\rightarrowL p Y,$ then
  1. $aX_n+b\rightarrowL p aX+b$.
  2. $X_n\pm Y_n\rightarrowL p X\pm Y.$

Proof:The first property follows directly.

The second property follows from Minkowski inequality.[QED]

However, even if $X_n\rightarrowL p X$ and $Y_n\rightarrowL p Y$ and $X_nY_n, XY$ have finite $p$-th moments, we may have $X_nY_n\not\rightarrowL p XY.$

The first exercise below provides a counterexample.

3.1. Problem set

EXERCISE 19:  Let $U\sim Unif(0,1).$ Define $$X_n = Y_n = \left\{\begin{array}{ll}n^{1/(2p)}&\text{if }U\in\left(0,\frac 1n\right)\\ 0&\text{otherwise.}\end{array}\right..$$ Also, let $X=Y\equiv 0.$

Show that $E|X_nY_n|^p=1.$ Hence show that this provides a counterexample to the statement above.

EXERCISE 20: Let $P(X_n=n) = 1-P(X_n=0)=\frac{1}{n^2}.$ Determine for which values of $p\geq 1$ we have $X_n\rightarrowL p 0.$

EXERCISE 21: Show that if $(X_n)$ is an iid sequence with mean $\mu$ and variance $\sigma^2<\infty,$ then $\frac 1n\sum_1^n X_k\rightarrowL 2 \mu$.

EXERCISE 22: Construct $(X_n)$ such that $X_n\rightarrowL 1 X$ but $X_n\not\rightarrowL 2 X$.

4. Convergence in distribution

Definition: Convergence in distribution Let $(X_n), X$ be some random variables (not necessarily defined on the same probability space). Let $X_n$ have distribution function $F_n$ and $X $ have distribution function $F.$ We say $X_n$ converges to $X$ in distribution and write $X_n\rightarrowD X$ if for all continuity point $x$ of $F$ we have $F_n(x)\rightarrow F(x).$ Many books also write $F_n\Rightarrow F$ or $X_n\Rightarrow X.$

Since the behaviour of a random variable is captured completely by its distribution, this is quite a natural definition, except for the mention of "continuity points". Why not demand $\forall x\in {\mathbb R} ~~F_n(x)\rightarrow F(x)$ "? Why relax the requirement at the discontinuity points of $F?$
Theorem Let $X_n,X$ be all random variables defined on a common probability space. Let $a,b$ be fixed numbers. If $X_n\rightarrowD X$, then $aX_n+b\rightarrowL p aX+b$.

Proof:Direct.[QED]

4.1. Problem set

EXERCISE 23: Let $(a_n)$ be a fixed sequence with $a_n\rightarrow a.$ Let $X_n\equiv a_n$ and $X\equiv a.$ Then is it true that $X_n\rightarrowD X$?

EXERCISE 24: Let $X_n\sim Bern\left(\frac 1n\right).$ Then show that $X_n$ converge in distribution. Find the limit.

EXERCISE 25: Let $(X_n)$ be a sequence of random variable with distribution function $F_n,$ where

$$F_n(x) = \left\{\begin{array}{ll}0&\text{if }x < 0\\ x^n&\text{if }x\in[0,1)\\ 1&\text{otherwise.}\end{array}\right.. $$ Show that $\X_n\rightarrowD X$ for some $X.$ What is the distribution of $X$?

EXERCISE 27: If $X_n\sim Unif\left(\frac{\sqrt n-\sqrt 2}{2\sqrt n},\frac{\sqrt n+\sqrt 2}{2\sqrt n}\right),$ then does $(X_n)$ converge in distribution? If so, find the limiting distribution.

EXERCISE 28: If $X_n\sim Unif\left(n-\frac 1n,n+\frac 1n\right),$ then does $(X_n)$ converge in distribution? If so, find the limiting distribution.

EXERCISE 29: Let $U_1,...,U_n$ be iid $Unif(0,1)$ random variables. Let $X_n = \min_i U_i.$ Show that $X_n\rightarrowD 0.$

EXERCISE 30: $(X_n), X$ are discrete random variables taking values in $\{1,...,10\}.$ If pmf of $X_n$ converges pointwise to pmf of $X,$ then show that $X_n\rightarrowD X.$

5. Central Limit Theorem

The Central imit Theorem (CLT) is possibly the most famous theorem in probability theory and statistics. Originally stated and proved by Gauss, the theorem has many variants due to other mathematicians. Here we shall state the simplest version.

Central imit Theorem (CLT) Let $X_1,X_2,...$ be IID with $E(X_i) = \mu$ and $V(X_i) = \sigma^2 < \infty.$ Let $$\bar X_n = \frac{X_1+\cdots+X_n}{n}\mbox{ for } n\in{\mathbb N}.$$ Then the distribution of $\frac{\sqrt n(\bar X_n-\mu)}{\sigma}$ tends to $N(0,1)$ as $n\rightarrow \infty.$ More precisely, if $F_n(\cdot)$ denotes the CDF of $\frac{\sqrt n(\bar X_n-\mu)}{\sigma},$ and $\Phi(\cdot)$ denotes the $N(0,1)$ CDF, then $$\forall t\in{\mathbb R}~~F_n(t)\rightarrow \Phi(t) \mbox{ as } n\rightarrow \infty.$$

Proof:Skipped.[QED]

This theorem is a manifestation of statistical regularity. Whatever may the true distribution of the $X_i$'s be, if you average a large number of $X_i$'s you get close approximation to the normal distribution. This allows statistician to deal with averages of a large number of IID observations without knowing the true underlying distribution.

Let's look at a typical example.

EXAMPLE 3: If 40% of the population of a city supports a poll candidate, then what is the approximate probability that a random sample of 500 persons from the city will have at least 250 supporters?

SOLUTION: Here we think of the sampling procedure as 500 trials of the same random experiment: Pick a person at random from the population of the city.

We shall assume that the trials are iid. Now here we are introducing an approximation: the first membr of the sample was drawn from the entire population, but since we generally sample without replacement in such a scenario, the second member of the sample was drawn from a population of size one less than in the case of the first member. So the random experiment has actually changed, and they are not independent also. But since the population is large (much larger than 500), so we are ignoring both the non-identical and dependent nature and assuming iid.

We also have a random variable: $$X(\omega) = \left\{\begin{array}{ll}1 &\text{if }\omega\mbox{ supports the candiate}\\ 0&\text{otherwise.}\end{array}\right.$$ Here $\omega$ is the person sampled. Each trial gives rise to one copy of this random variable, so we have $X_1,...,X_{500}$ iid $Bernoulli(0.4).$ This $0.4$ came from the 40% given in the problem.

By CLT we have $$\frac{\sqrt n (\bar X_n-\mu)}{\sigma}\rightarrow N(0,1)$$ as $n\rightarrow \infty,$ where $\mu = E(X_i)$ and $\sigma^2 = V(X_i)< \infty.$ We shall write this as $$\bar X_n \stackrel{\bullet}{\sim} N\left(\mu,\frac{\sigma^2}{n}\right)$$ for large $n.$ Here $\stackrel\bullet\sim$ means "approximately distributed as".

In our case, $\mu = 0.40$, $\sigma^2 = 0.4(1-0.4) = 0.24$ and $n=500.$ So $$\bar X_{500} \stackrel{\bullet}{\sim} N\left(0.40,\frac{0.24}{500}\right),$$ or $$\sum_1^n X_i \stackrel{\bullet}{\sim} N(0.40\times 500,0.24\times 500)\equiv N(200, 120).$$ Nowe we can find the required probability as $$P(\sum_1^{500} X_i \geq 250) \approx 1-\Phi\left(\frac{250-200}{\sqrt{120}}\right).$$ This probability may be obtained by looking up standard $N(0,1)$ tables or using R as 
1-pnorm((250-200)/sqrt(120))
■

In this problem we knew the distribution of the $X_i$'s, but we never really made any use of it, except to compute $E(X_i)$ and $V(X_i).$

5.1. Problem set 1

::

EXERCISE 31: [rossdistrib10.png]

::

EXERCISE 32: [rossdistrib8.png]

[Hint]

For the second part, just drop the 6's. This means you are rolling a 5-faced die 800 times.

::

EXERCISE 33: [rossdistrib5.png]

EXERCISE 34: Let $X_n\rightarrowD N(0,1)$ and $Y_n\rightarrowP 5.$ Then what is the limiting distribution of $X_n+Y_n?$

EXERCISE 35: Let $X_n\rightarrowD X$ and $Y_n\rightarrowP Y.$ Show that $X_n+Y_n\rightarrowD X+Y$ need not hold.

EXERCISE 36: Let $X_n\rightarrowD N(0,1)$, $Y_n\rightarrowP 5$ and $Z_n\rightarrowP 4$ with $z_n > 0.$ Then what is the limiting distribution of $\frac{X_n+Y_n}{\sqrt {Z_n}}?$

EXERCISE 37: Suppose that $\sqrt n(X_n-\theta)\rightarrowD Z$ and $Y_n\rightarrowP a.$ Show that $\sqrt n(X_nY_n-a\theta)\rightarrowD aZ.$

EXERCISE 38: Let $X_n$ be asymptotically $N\left(\mu,\frac{\sigma^2}{n}\right).$ What is the asymptotic distribution of $\frac{X_n}{1+X_n}?$

EXERCISE 39: Let $T_n$ be a consistent estimator of $\theta,$ and let $S_n$ be a consistent estimator of $\sigma^2.$ Show that the Studentized statistic $\frac{T_n-\theta}{\sqrt{S_n}}$ has the the same asymptotic distribution as $\frac{T_n-\theta}{\sigma},$ whenever an asymptotic distribution exists.

EXERCISE 40: Let $X_n\rightarrowD X$ and $X_n+Y_n\rightarrowD X+1.$ Does this imply that $Y_n\rightarrowP 1?$

6. Relative strengths of different modes

We have learned about four different modes of convergence (counting all the $L_p$s as a single mode). It is surprising and somewhat disconcerting that these modes are not equivalent. The following diagram shows the relative strengths of the these modes.
Relative strengths
Whenever there is a blue arrow from one mode to another, the first mode implies the second mode. None of the other implications hold in general. Since $L_1$ is the weakest member of the $L_p$ family (for $p\geq 1$), we shall take $L_1$ as the representative of $L_p$ in the following discussion.

TheoremIf $X_n\rightarrowA X$ then $X_n\rightarrowP X.$

Proof: To show $\forall \epsilon>0~~P(|X_n-X|\geq \epsilon) \rightarrow 0$ as $n\rightarrow \infty.$

Take any $\epsilon>0.$

Let $A_n = \{|X_n-X|\geq \epsilon\}$.

Since $X_n\rightarrowA X,$ hence $P(A_n\io) = 0.$

Hence $P(A_n)\rightarrow 0.$ [QED]

TheoremIf $X_n\rightarrowL 1 X$ then $X_n\rightarrowP X.$

Proof: Take any $\epsilon>0.$ Then, by Markov inequality applied to $|X_n-X|$ (which is nonnegative thanks to the modulus), we have $$P(|X_n-X|>\epsilon)\leq \frac{E(|X_n-X|)}{\epsilon}\rightarrow 0.$$ [QED]

TheoremIf $X_n\rightarrowP X,$ then $X_n\rightarrowD X.$

Proof:

This proof requires the concept of liminf and limsup.
Let $X_n\rightarrowP X.$ To show $X_n\rightarrowD X,$ i.e., if $X_n$ and $X $ have distribution functions $F_n$ and $F,$ then
$\forall$ continuity point $x$ of $F$ we have $F_n(x)\rightarrow F(x).$

Take any continuity point $x$ of $F.$

Enough to show that $\forall \epsilon>0~~F(x-\epsilon) \leq \liminf F_n(x)\leq \limsup F_n(x) \leq F(x+\epsilon).$
[Because...]
The result will follow by taking limit as $\epsilon\downarrow 0.$
Now $$\begin{eqnarray*} F(x-\epsilon) & = & P(X\leq x-\epsilon)\\ & = & P(X\leq x-\epsilon ~\&~ |X_n-X|<\epsilon) + P(X\leq x-\epsilon ~\&~ |X_n-X|\geq\epsilon)\\ & \leq & P(X\leq x-\epsilon ~\&~ |X_n-X|<\epsilon) + P(|X_n-X|\geq\epsilon)\\ & \leq & P(X_n\leq x) + P(|X_n-X|\geq\epsilon)\\ & = & F_n(x) + P(|X_n-X|\geq\epsilon). \end{eqnarray*}$$ So $$F(x-\epsilon) - P(|X_n-X|\geq\epsilon) \leq F_n(x).$$

Again, $$\begin{eqnarray*} F_n(x) & = & P(X_n\leq x)\\ & = & P(X_n\leq x ~\&~ |X_n-X|< \epsilon) + P(X_n\leq x ~\&~ |X_n-X| \geq \epsilon)\\ & \leq & P(X_n\leq x ~\&~ |X_n-X|< \epsilon) + P(|X_n-X| \geq \epsilon)\\ & \leq & P(X\leq x + \epsilon) + P(|X_n-X| \geq \epsilon)\\ & = & F(x + \epsilon) + P(|X_n-X| \geq \epsilon)\\ \end{eqnarray*}$$ So $$F(x-\epsilon) - P(|X_n-X|\geq\epsilon) \leq F_n(x) \leq F(x + \epsilon) + P(|X_n-X| \geq \epsilon).$$ Taking limit as $n\rightarrow \infty,$ we get $$F(x-\epsilon) \leq \liminf F_n(x) \leq \limsup F_n(x) \leq F(x + \epsilon).$$ Hence the result. [QED]

Though convergence in distribution does not imply convergence in probability in general, but there is one important special ase where it does! This is given in the theorem below.

TheoremLet $(X_n)$ be a sequence of random variables (not necessarily defined on a common probability space). Let $c\in{\mathbb R}$ be fixed. If $X_n\rightarrowP c$ then $X_n\rightarrowD c.$

Proof: Step 1: Shall show $\forall x < c~~P(X_n\leq x)\rightarrow 0.$

Take any $x < c.$

Then, for $\epsilon = c-x > 0,$ we have $P(|X_n-c| \geq \epsilon) \rightarrow 0.$

So $P(X_n \leq c-\epsilon) \rightarrow 0$ or $P(X_n \leq x) \rightarrow 0.$

Step 2: Shall show $\forall x > c~~P(X_n\leq x)\rightarrow 1.$

Similar argument. [QED]

6.1. Counterexamples

The following example shows that almost sure convergence does not imply $L_p$ convergence, even when the $p$-th moments exist finitely for all the random variables involved.

EXAMPLE 4: We work with $Unif(0,1)$ probability space. Let $X_n(\omega) = \left\{\begin{array}{ll}n&\text{if }\omega\in\left(0,\frac 1n\right)\\ 0&\text{otherwise.}\end{array}\right.$.

Let $X\equiv 0.$

Then clearly $\forall\omega\in (0,1)~~X_n(\omega)\rightarrow X(\omega).$ Hence $X_n \rightarrowA X.$

But $\forall n\in{\mathbb N}~~E(X_n) = n\times\frac 1n = 1\not\rightarrow 0 = E(X).$ ■

Other counterexamples are in the problem set below.

6.2. Problem set

EXERCISE 41: Let $f_1,f_2,f_3,...:[0,1]\rightarrow{\mathbb R}$ be defined as indicator functions of

$[0,1],$

$\left[0, \frac 12 \right], \left[ \frac 12, 1 \right], $

$\left[0,\frac 13 \right], \left[ \frac 13, \frac 23 \right], \left[ \frac 13, 1 \right], $

etc. Let $U\sim Unif[0,1].$ Let $X_n = f_n(U).$ Let $X\equiv 0.$ Show that $X_n\rightarrowL 1 X$ but $X_n\not\rightarrowA X.$

EXERCISE 42: Consider $X_n, X$ as in the last exercise. Show that $X_n\rightarrowP X.$

EXERCISE 43: Among the examples discussed so far, there is one counterexample that shows that convergence is probability does not imply $L_1$ counterexample. What is that example?

EXERCISE 44: Convergence in distribution obviously cannot imply convergence in probability, because convergence in distribution does not even require the random variables to be defined on the same probability space. But even if all the random variables are defined on a common probability space, we can get a counterexample. Let $U\sim Unif(0,1).$ Let $X_n = (-1)^n U.$ Then show that $X_n\rightarrowD U,$ but $X_n\not\rightarrowP U.$