Discrete Statistical Distributions

February 22nd, 2010 by Kristopher Reese

Statistical Distributions of data are an important aspect in both analyzing resulting data and in generating random numbers based on a specified distribution.  This post will discuss a handful of Statistical Distributions that are common in Discrete-Event Simulations.  It will discuss the Uniform Distribution, the Triangular Distribution, the Binomial/Bernoulli Distribution, the Poisson Distribution, and the Exponential Distribution.  The two most important aspects of these distributions for the purposes of this discussion are the Probability Mass Function and the Cumulative Distribution Function.  Each distributions PDF and CDF will be discussed further here.

Before getting started, below is a table of equations for each of the distributions for finding the mean, median, mode, and variance of a specific distribution:

Distribution Mean Median Mode Variance
Uniform \frac{a+b}{2} \frac{a+b}{2} N/A \frac{n^{2}-1}{12}
Triangular \frac{a+b+c}{3} \left\{\begin{matrix} a+\frac{\sqrt{(b-a)(c-a)}}{\sqrt{2}} & \text {for} \; c \geq \frac{a+b}{2} \\ \\ b-\frac{\sqrt{(b-a)(c-a)}}{\sqrt{2}} & \text {for} \; c \leq \frac{a+b}{2} \end{matrix}\right. c \frac{a^{2}+b^{2}+c^{2}-ab-ac-bc}{18}
Binomial np \left \lfloor np \right \rfloor \text {or} \left \lceil np \right \rceil \left \lfloor (n+1)p \right \rfloor \text {or} \left \lceil (n+1)p \right \rceil-1 np(1-p)
Exponential \frac{1}{\lambda} \frac{ln(2)}{\lambda} 0 \frac{1}{\lambda^{2}}
Poisson λ \approx \left \lfloor \lambda+\frac{1}{3}-\frac{0.02}{\lambda} \right \rfloor \left\{\begin{matrix} \left \lfloor \lambda \right \rfloor & \text {if}\; \lambda \notin \mathbb{N} \\ \lambda-1 & \text {if}\; \lambda \in \mathbb{N} \end{matrix}\right. λ



The Uniform Distribution exists in both the discrete and the continuous spaces.  For this discussion however, we will strictly discuss the Discrete version. In a discrete uniform distribution, we can generate the Probability Mass Function with the equation:

\text{pmf} \left\{\begin{matrix} \frac{1}{n} & \text {for}\; a \leq k \leq b \\ \\ 0 & \text {otherwise} \end{matrix}\right.

where:

a, b \in \mathbb{N} \\ k \in \left \{ a, a+1, \ldots , b-1, b \right \} \\ n = b - a + 1

Using the same parameters, we can calculate the Cumulative Distribution Function using the equation:

\text {cdf} \left\{\begin{matrix} 0 & \text {for} \; k < a \\ \frac{\left \lfloor k \right \rfloor - a + 1}{n} & \text {for} a \leq k \leq b\\ 1 & \text {for} \; k > b \end{matrix}\right.

Using the PDF and CDF we can plot a set of points onto graphs which would look similar to Figure 1a & 1b respectively (with the same parameters):

Discrete Uniform Probability Mass Function(a)
Discrete Uniform Cumulative Distribution Function(b)

A plot of the Discrete Uniform (a) Probability Mass Function and (b) Cumulative Distribution Function. a = 1, b = 5. Graph generated with MATLAB

The Triangular Distribution can be used in situations which a normal distribution might be needed, but when we want to restricted the distribution to a set of bounds.  A triangular distribution has three parameters a lower limit a, an upper limit b, and a mode c.  These parameters are defined as:

a \in \Re \\ b > a \\ a \leq c \leq b

With these parameters we can find the PMF and CDF to be:

\text {pmf} \left\{\begin{matrix} \frac{2(x-a)}{(b-a)(c-a)} & \text {for} \; a \leq x \leq c \\ \\ \frac{2(b-x)}{(b-a)(b-c)} & \text {for} \; c \leq x \leq b \end{matrix}\right. \\ \\ \\ \text {cdf} \left\{\begin{matrix} \frac{(x-a)^{2}}{(b-a)(c-a)} & \text {for} \; a \leq x \leq c \\ \\ 1-\frac{(b-x)^{2}}{(b-a)(b-c)} & \text {for} \; c \leq x \leq b \end{matrix}\right.

Plotting these functions, will result in graphs that appear similar to Figure 2.  Figure 2 images were taken from the wikipedia article.  These two images are distributed under the Creative Commons Attribution ShareAlike 3.0 License.

Triangular Distribution Probability Mass function(a)
Triangular Distribution Cumulative Distribution Function(b)

A plot of a Continuous Triangular Distribution (a) Probability Mass Function and (b) Cumulative Distribution Function.

Binomial Distribution is a distribution of the number of sequences of trials in an experiment which contains only two possible outcomes (success/failure, true/false, etc.).  This distribution is associated with Bernoulli Trials and when parameter n = 1, we can call this distribution a Bernoulli Distribution.  The Binomial Distribution has three parameters:

n \in \mathbb{N}\text { - number of trials} \\ p \in [0,1] \\ k \in \left \{ 0, \ldots, n \right \}

We can find the PMF and the CDF to be:

pmf = \binom{n}{k} p^{k}(1-p)^{n-k} \\\\ cdf = \sum_{i=0}^{\left \lfloor x \right \rfloor}\binom{n}{i}p^{i}(1-p)^{n-i}

we define \binom{n}{k} to be the binomial coefficient which we can define as:

\binom{n}{k} = \frac{n!}{k!(n-k)!}

The resulting PMF and CDF graphs might look like (with the same parameters) Figure 3a & Figure 3b respectively:

Discrete Binomial Probability Mass Function(a)
Discrete Binomial Cumulative Distribution Function(b)

A plot of the Discrete Binomial distribution (a) Probability Mass Function and (b) Cumulative Distribution Function. n = 5, p = 0.5

Exponential Distribution is a continuous probability distribution that is frequently used in Simulations.  Though it is considered a class of continuous distributions, discretizing the distribution does not take a lot of effort.  The exponential function is always bounded between [0,∞) and has one parameter:

\lambda = \Re > 0

Using this parameter, we can solve the PMF and CDF for any exponential distribution:

\text {pmf} = \lambda e^{-\lambda x} \\ \\ \text {cdf} = 1-e^{-\lambda x}

Plotting these functions will result in graphs that look like (with the same parameters):

Exponential Distribution Probability Mass Function(a)
Exponential Distribution Cumulative Distribution Function(b)

A plot of a Continuous Exponential Distribution (a) Probability Mass Function and (b) Cumulative Distribution Function. λ = 0.4

Poisson Distribution will be the last distribution discussed in this entry.  This distribution can be used to express a probability of a number of events occurring within a fixed period of time, assuming that these events occur with an average rate and the events are independently from other events.  The Poisson Distribution functions have two parameters:

\lambda = \mathbb{N} > 0 \\ k \in \mathbb{N} \geq 0

The PMF and CDF are, respectively:

\text {pmf} = \frac{\lambda^{k}}{k!}\;\cdot\;e^{-\lambda} \\ \\ \text {cdf} = e^{-\lambda}\sum_{i=0}^{k}\frac{\lambda^{i}}{i!}

It is interesting to note that as λ moves towards sufficiently large values, a normal distribution with a mean and variance of λ and a standard deviation of \sqrt{\lambda} will decently approximate the Poisson distribution.  If we plot the PDF and CDF for the Poisson Distribution, we get a graph that would look like (with the same parameters):

Poisson Distribution Probability Mass Function(a)
Poisson Distribution Cumulative Distribution Function(b)

A plot of the the Discrete Poisson Distribution (a) Probability Mass Function and (b) Cumulative Distribution Function. λ = 4

These are only a handful of possible distributions that could result from simulation outcomes, but these are probably the most frequently encountered distributions.  More distributions will be touched on in future posts.  If you have one you’d like me to discuss or go into more detail about, leave a comment!

Leave a Reply