Cumulative Distribution Function

Let \(X\) be a random variable with probability density function (p.d.f.) given by \(\mathrm{f}_X(x) = \mathrm{P}(X = x)\).

The cumulative distribution function (c.d.f.) is defined by the function \(F_X(x) = \mathrm{P}(X \le x) = \sum_{k \le x} \mathrm{f}_X(k)\).

Remarks.

When dealing with empirical data, we speak about the histogram that records the frequency of occurrence of the data points. We also speak about the cumulative frequency associate to these data points.

There is a marked distinction between empirical data and values of a random variable: an empirical datum is a happened instance of the event when a random variable takes on a certain value. There is nothing so random about a piece of empirical data because they already occurred. Now, the random variable is what it says it is. Since the random event of \(X = x\) has not taken place yet, we can only say about its theoretical probability of occurrence.

One way of viewing probability is to take the so-called frequentist approach, i.e., roughly speaking, one needs to perform an random event many times to get the limiting value of its frequency of occurrence over the total number of outcomes – which is its probability of occurrence.

In the toss of a fair coin, the probability of obtaining a head is \(\frac{1}{2}\) does not imply that when the experiment of tossing a fair coin is performed twice then you must get at least one head.

We now develop the concept of theoretical average of a random variable using this relational comparison between empirical data and values of a random variable.

Empirical view

Suppose we throw an unbiased dice \(100\) times and record the outcomes:

The frequencies total up to \(100\), i.e., \(\sum f = 100\).

The mean, or the average, of these data is

\[\bar{x} = \frac{\sum_{i=1}^{n} f_i x_i}{\sum_{i=1}^{n} f_i} = \frac{\sum fx}{\sum f} = \sum x \left( \frac{f}{\sum f} \right).\]

We term the coefficient \(\frac{f}{\sum f}\) as the relative frequency of occurrence associated to \(x\).

If we repeat this experiment many times, then in the long run what is expected of this mean?

Theoretical view

Let \(X\) be the random variable of the score on a random toss of a fair die. Then

\[\mathrm{P}(X = x) = \frac{1}{6}, \quad x = 1, 2, 3, 4, 5, 6.\]

If we adopt the frequentist approach, we would have viewed \(\mathrm{P}(X = x)\) as the limiting value of the relative frequency \(\frac{f_x}{\sum f_x}\). Hinging on this comparison, it is sensible to define the theoretical average of the random variable \(X\) to be

\[\mu := \mathrm{E}(X) = \sum_{\text{all}\hspace{0.5em} x} x \mathrm{P}(X = x)\]

and we call \(E(X)\) the expectation (or expected value) or simply, mean, of the random variable \(X\).

We view the expected value to be the theoretical average of this random variable as nothing has occurred yet. In a run of a random experiment where some empirical data have been obtained we do not anticipate that the actual mean of the data to be equal to the theoretical mean of the random variable associated to it.

In general, if \(X\) is a random variable and \(\mathrm{g}(X)\) is a function of \(X\) then the expected value of \(\mathrm{g}(X)\) is defined to be

\[\mathrm{E}(\mathrm{g}(X)) = \sum_{\text{all}\hspace{0.5em} x} \mathrm{g}(x) \mathrm{P}(X = x)\]