kReese.net http://kreese.net Notebook of a PhD Computer Scientist Student. Mon, 22 Feb 2010 08:50:18 +0000 http://wordpress.org/?v=2.9.2 en hourly 1 Discrete Statistical Distributions http://kreese.net/index.php/2010/02/discrete-statistical-distributions/ http://kreese.net/index.php/2010/02/discrete-statistical-distributions/#comments Mon, 22 Feb 2010 08:50:18 +0000 Kristopher Reese http://kreese.net/?p=367 Statistical Distributions of data are an important aspect in both analyzing resulting data and in generating random numbers based on a specified distribution.  This post will discuss a handful of Statistical Distributions that are common in Discrete-Event Simulations.  It will discuss the Uniform Distribution, the Triangular Distribution, the Binomial/Bernoulli Distribution, the Poisson Distribution, and the Exponential Distribution.  The two most important aspects of these distributions for the purposes of this discussion are the Probability Mass Function and the Cumulative Distribution Function.  Each distributions PDF and CDF will be discussed further here.

Before getting started, below is a table of equations for each of the distributions for finding the mean, median, mode, and variance of a specific distribution:

Distribution Mean Median Mode Variance
Uniform \frac{a+b}{2} \frac{a+b}{2} N/A \frac{n^{2}-1}{12}
Triangular \frac{a+b+c}{3} \left\{\begin{matrix} a+\frac{\sqrt{(b-a)(c-a)}}{\sqrt{2}} & \text {for} \; c \geq \frac{a+b}{2} \\ \\ b-\frac{\sqrt{(b-a)(c-a)}}{\sqrt{2}} & \text {for} \; c \leq \frac{a+b}{2} \end{matrix}\right. c \frac{a^{2}+b^{2}+c^{2}-ab-ac-bc}{18}
Binomial np \left \lfloor np \right \rfloor \text {or} \left \lceil np \right \rceil \left \lfloor (n+1)p \right \rfloor \text {or} \left \lceil (n+1)p \right \rceil-1 np(1-p)
Exponential \frac{1}{\lambda} \frac{ln(2)}{\lambda} 0 \frac{1}{\lambda^{2}}
Poisson λ \approx \left \lfloor \lambda+\frac{1}{3}-\frac{0.02}{\lambda} \right \rfloor \left\{\begin{matrix} \left \lfloor \lambda \right \rfloor & \text {if}\; \lambda \notin \mathbb{N} \\ \lambda-1 & \text {if}\; \lambda \in \mathbb{N} \end{matrix}\right. λ



The Uniform Distribution exists in both the discrete and the continuous spaces.  For this discussion however, we will strictly discuss the Discrete version. In a discrete uniform distribution, we can generate the Probability Mass Function with the equation:

\text{pmf} \left\{\begin{matrix} \frac{1}{n} & \text {for}\; a \leq k \leq b \\ \\ 0 & \text {otherwise} \end{matrix}\right.

where:

a, b \in \mathbb{N} \\ k \in \left \{ a, a+1, \ldots , b-1, b \right \} \\ n = b - a + 1

Using the same parameters, we can calculate the Cumulative Distribution Function using the equation:

\text {cdf} \left\{\begin{matrix} 0 & \text {for} \; k < a \\ \frac{\left \lfloor k \right \rfloor - a + 1}{n} & \text {for} a \leq k \leq b\\ 1 & \text {for} \; k > b \end{matrix}\right.

Using the PDF and CDF we can plot a set of points onto graphs which would look similar to Figure 1a & 1b respectively (with the same parameters):

Discrete Uniform Probability Mass Function(a)
Discrete Uniform Cumulative Distribution Function(b)

A plot of the Discrete Uniform (a) Probability Mass Function and (b) Cumulative Distribution Function. a = 1, b = 5. Graph generated with MATLAB

The Triangular Distribution can be used in situations which a normal distribution might be needed, but when we want to restricted the distribution to a set of bounds.  A triangular distribution has three parameters a lower limit a, an upper limit b, and a mode c.  These parameters are defined as:

a \in \Re \\ b > a \\ a \leq c \leq b

With these parameters we can find the PMF and CDF to be:

\text {pmf} \left\{\begin{matrix} \frac{2(x-a)}{(b-a)(c-a)} & \text {for} \; a \leq x \leq c \\ \\ \frac{2(b-x)}{(b-a)(b-c)} & \text {for} \; c \leq x \leq b \end{matrix}\right. \\ \\ \\ \text {cdf} \left\{\begin{matrix} \frac{(x-a)^{2}}{(b-a)(c-a)} & \text {for} \; a \leq x \leq c \\ \\ 1-\frac{(b-x)^{2}}{(b-a)(b-c)} & \text {for} \; c \leq x \leq b \end{matrix}\right.

Plotting these functions, will result in graphs that appear similar to Figure 2.  Figure 2 images were taken from the wikipedia article.  These two images are distributed under the Creative Commons Attribution ShareAlike 3.0 License.

Triangular Distribution Probability Mass function(a)
Triangular Distribution Cumulative Distribution Function(b)

A plot of a Continuous Triangular Distribution (a) Probability Mass Function and (b) Cumulative Distribution Function.

Binomial Distribution is a distribution of the number of sequences of trials in an experiment which contains only two possible outcomes (success/failure, true/false, etc.).  This distribution is associated with Bernoulli Trials and when parameter n = 1, we can call this distribution a Bernoulli Distribution.  The Binomial Distribution has three parameters:

n \in \mathbb{N}\text { - number of trials} \\ p \in [0,1] \\ k \in \left \{ 0, \ldots, n \right \}

We can find the PMF and the CDF to be:

pmf = \binom{n}{k} p^{k}(1-p)^{n-k} \\\\ cdf = \sum_{i=0}^{\left \lfloor x \right \rfloor}\binom{n}{i}p^{i}(1-p)^{n-i}

we define \binom{n}{k} to be the binomial coefficient which we can define as:

\binom{n}{k} = \frac{n!}{k!(n-k)!}

The resulting PMF and CDF graphs might look like (with the same parameters) Figure 3a & Figure 3b respectively:

Discrete Binomial Probability Mass Function(a)
Discrete Binomial Cumulative Distribution Function(b)

A plot of the Discrete Binomial distribution (a) Probability Mass Function and (b) Cumulative Distribution Function. n = 5, p = 0.5

Exponential Distribution is a continuous probability distribution that is frequently used in Simulations.  Though it is considered a class of continuous distributions, discretizing the distribution does not take a lot of effort.  The exponential function is always bounded between [0,∞) and has one parameter:

\lambda = \Re > 0

Using this parameter, we can solve the PMF and CDF for any exponential distribution:

\text {pmf} = \lambda e^{-\lambda x} \\ \\ \text {cdf} = 1-e^{-\lambda x}

Plotting these functions will result in graphs that look like (with the same parameters):

Exponential Distribution Probability Mass Function(a)
Exponential Distribution Cumulative Distribution Function(b)

A plot of a Continuous Exponential Distribution (a) Probability Mass Function and (b) Cumulative Distribution Function. λ = 0.4

Poisson Distribution will be the last distribution discussed in this entry.  This distribution can be used to express a probability of a number of events occurring within a fixed period of time, assuming that these events occur with an average rate and the events are independently from other events.  The Poisson Distribution functions have two parameters:

\lambda = \mathbb{N} > 0 \\ k \in \mathbb{N} \geq 0

The PMF and CDF are, respectively:

\text {pmf} = \frac{\lambda^{k}}{k!}\;\cdot\;e^{-\lambda} \\ \\ \text {cdf} = e^{-\lambda}\sum_{i=0}^{k}\frac{\lambda^{i}}{i!}

It is interesting to note that as λ moves towards sufficiently large values, a normal distribution with a mean and variance of λ and a standard deviation of \sqrt{\lambda} will decently approximate the Poisson distribution.  If we plot the PDF and CDF for the Poisson Distribution, we get a graph that would look like (with the same parameters):

Poisson Distribution Probability Mass Function(a)
Poisson Distribution Cumulative Distribution Function(b)

A plot of the the Discrete Poisson Distribution (a) Probability Mass Function and (b) Cumulative Distribution Function. λ = 4

These are only a handful of possible distributions that could result from simulation outcomes, but these are probably the most frequently encountered distributions.  More distributions will be touched on in future posts.  If you have one you’d like me to discuss or go into more detail about, leave a comment!

]]> http://kreese.net/index.php/2010/02/discrete-statistical-distributions/feed/ 0 Analyzing Simulation Data http://kreese.net/index.php/2010/02/analyzing-simulation-data/ http://kreese.net/index.php/2010/02/analyzing-simulation-data/#comments Sun, 21 Feb 2010 10:57:07 +0000 Kristopher Reese http://kreese.net/?p=349 This post will discuss various mathematical formulae that are used in Event Based Simulation for very simple analysis of the data from the simulation.  For this post lets assume the following example simulation results:

Customer Inter-arrival Time Arrival Time Service Time Begin Service Wait Time End Service Time in System Server Idle
1 - 0 3.3 0 0 3.3 3.3 0
2 5.1 5.1 4.5 5.1 0 9.5 4.5 1.8
3 3.9 9.0 3.2 9.6 0.6 12.8 3.8 0
4 4.5 13.5 4.8 13.5 0 18.3 4.8 0.7
5 4.4 17.9 4.9 18.3 0.4 23.2 5.3 0
6 5.6 23.5 4.8 23.5 0 28.3 4.8 0.3
7 4.1 27.6 3.1 28.3 0.7 31.4 3.8 0
8 4.2 31.8 3.2 31.8 0 35 3.2 0.4

Since we have this data, we can do a very simple analysis of the resulting data.  For the purposes of this post, I will present the mean service time, the mean inter-arrival time, the mean wait time for all customers and for just those customers who waited, server utilization, and the mean number of customers in the system.

Mean Service Time. The following equation can be used for determining the mean service time of the customers.

\bar{\mathrm{st}} = \frac{\sum \text {Service Times}}{\text {Total Customers}}

For our given example, we get:

\bar{\mathrm{st}} = \frac{31.8}{8} = 3.975

Mean Inter-Arrival Time. The following equation can be used for determining the mean inter-arrival time of the customers.

\bar{\mathrm{ia}} = \frac{\sum \text {Inter-Arrival Time}}{\text {Total Customers}}

For our example, we would get:

\bar{\mathrm{ia}} = \frac{31.8}{7} = 4.54

Mean Wait Time. We have two instances of the mean wait time to consider.  The first is the mean wait time for all of the customers.  In this case, we can find the mean wait time with:

\text {Mean Wait Time for all} = \frac{\sum \text {Waiting Time}}{\text {Total Customers}}

For our example the mean wait time of all customers is:

\text {Mean Wait Time for all} = \frac{1.7}{8} = 0.21

In contrast, the mean wait time for only the customers who waited is:

\text {Mean Wait Time for waiters} = \frac{1.7}{3} = 0.57

Server Utilization. Server utilization is the percent of time in which the server was doing some form of work, or not idle.  For this, we want to find the mean server idle time and subtract this number from 1.  This will give us a value less than 1 which is the percent of utilization of the server.  In otherwords:

\text {Server Utilization} = 1 - \frac{\sum \text {server idle}}{\text {Total Time}}

For our example, we get:

\text {Server Utilization} = 1 - \frac{3.2}{35.0} = 1 - 0.091 = 0.908 \approx 91\%

Mean Number of Customers in System. Unlike the other equations, which are relatively trivial, the Mean number of Customers in the system is slightly trickier.  However, we can still compute this using a relatively simple equation.  First we want to define λ to be the mean interarrival rate and μ to be the mean service rate.  With this, we can find the Traffic Intensity:

\rho = \frac {\lambda}{\mu}

With ρ we can find the Mean number of Customers in the system with the following equation:

E[n]=\frac {\rho}{(1-\rho)}

Therefore for our example, we can define ρ as and then solve the Mean Number of Customers in the System:

\rho = \frac {4.54}{3.98} = 1.14

E[n]=\frac {1.14}{(1-1.14)} = \frac {1.14}{(0.14)} \approx 8

These equations are essential to know for simple statistical analysis of the data received by a simulation.  There are much more complex equations within queuing theory that require the knowledge of these equations.  So these are worthwhile equations to have memorized if you plan to do more complex statistical analysis of data.

]]> http://kreese.net/index.php/2010/02/analyzing-simulation-data/feed/ 0 Introduction to Simulations http://kreese.net/index.php/2010/02/introduction-to-simulations/ http://kreese.net/index.php/2010/02/introduction-to-simulations/#comments Sun, 21 Feb 2010 07:59:58 +0000 Kristopher Reese http://kreese.net/?p=345 Simulations are an important tool in computer scientists’ arsenal. It allows a scientist to statistically analyze a designed experiment with only a fraction of the cost of implementing a final system.

In simulations of systems, be it computer performance analysis or protein folding, there are two major classes of simulations: Discrete-Event Simulations and Continuous Simulations. A Continuous Simulation is a simulation in which the state variables changes continuously over the designated time that the simulation is run. In contrast, a Discrete-Event Simulation is a simulation in which the state variable of the system change only at discrete points in time.

The majority of the posts about Simulations on this site will discuss Discrete-Event Simulations, though many of the equations and statistical concepts could be used with analysis of Continuous Simulations with only minor changes. The rest of this post will outline the advantages and disadvantages of simulation, as well as a list of common mistakes in simulation.

Advantages/Disadvantages

Before listing some of the advantages and Disadvantages, we must first know when simulation is appropriate and when simulation is likely to not be appropriate to a situation.  Though Simulations are a powerful tool, in many situations a common sense solution can be reached or in some cases can often by solved analytically using formulae related to the model.  In some cases, we may run into a lack of data for the model, in which case the model may be difficult to simulate.  We may also run into a few rare cases of the cost/benefit of the simulation to hold no significant value.  However, simulations can often be applied to a multitude of situations including: The study of complex systems or subsystems, the knowledge that would be gained by the simulation holds significant value, to study the effects of changing inputs, to determine possible system requirements (for example, software), to train humans or intelligent machines, visualization of systems, and of course many other topic.  The following lists are shortened versions from [1]

Advantages. A simulation can lead to discovery of new models or systems.  Some of the advantages that using simulation is to:

  • Explore new options without disrupting the current operation of a system.
  • It has significantly lower cost in comparison to implementing a final system before testing.
  • We can simulate a larger amount of time in a shorter period using simulations, known as time-warping.
  • It allows us the study the interactions or importance of variables in the system
  • It allows us to study various “what if…” conditions
  • and, it helps scientists or engineers to develop a model that helps understand the system better.

Disadvatages. Despite the significant advantages, there are a handful of disadvantages that should be considered before implementing a simulation for your needs.

  • Implementing a simulation takes time and often this results in a company spending more money to hire the necessary designers for the simulation.
  • Simulation is an art.  The results of a simulation are highly variable, often based on the skill of the designer.
  • Analysis of simulation results can often be difficult unless programmed correctly.  This again relates to Simulation being an art.

Common Mistakes in Simulation

This section discusses some of the aspects that have to be overcome and what makes Simulation more of an art.  The list below is a simplified version of the list of common mistakes in [2]

  1. Inappropriate Level of Detail: Without the proper level of detail within the simulation could result in the failure of the simulation or inaccurate results in comparison to the model being tested.  However in order to do a proper analysis of the data for the model, we have to make some simplifications and assumptions.  The difficulty lies in capturing the proper level of detail while simplifying the model well enough to analyze the data received from the simulation.
  2. Improper Language:  A simulation can be designed in almost any programming language, however choosing the proper programming language has a significant impact on the development time of the model.  Many languages can cut development times significantly, and some can even make analysis of the data significantly easier for the developer.
  3. Unverified Models: Simulation models are often implemented with large programs.  The developer has to take special care to ensure that there are no bugs or programming errors which might invalidate the results of the Simulation.
  4. Invalid Models:  Incorrect of assumptions about the system could result in invalidated data even with a flawless program.
  5. Improperly Handled Initial Conditions:  Including the initial portion of the simulation could result in non-representative data of the system.  To avoid this, one should give the simulation the required burn-in period for the data to converge on data that is representative of the system.
  6. Too Short Simulations:  Often times, the Analyst will try to save time by running the system within a shorter period of time.  Related to item 5, this can result in data that is non-representative of the systems data.
  7. Poor Random-Number Generators: Random Numbers are an important aspect of simulations.  It is often better to use a well known generator, however in some cases, the generators may not work for your simulation.  Generating random numbers will be discussed in a future post.
  8. Improper Selection of Seeds: Seeding the random number generator is important to maintain independence among the streams in the simulation.  In many cases, using the same seed in the streams may result in correlation among each of the system processes which might result in non-representative data.

References

[1] J.Banks, J.Carson, B. Nelson, and D. Nicol Discrete-Event System Simulation New Jersey: Prentice Hall, 2005.

[2] R.Jain, The Art of Computer Systems Performance Analysis New York: Wiley, 1991.

]]> http://kreese.net/index.php/2010/02/introduction-to-simulations/feed/ 2 kReese.net in 2010 http://kreese.net/index.php/2010/01/kreese-net-in-2010/ http://kreese.net/index.php/2010/01/kreese-net-in-2010/#comments Mon, 18 Jan 2010 08:06:41 +0000 Kristopher Reese http://kreese.net/?p=101 After spending a majority of the last few months of 2009 working on updating my site to make it easier to update, I finally caved in and decided to use Wordpress instead of programming my own site. It took me a while to cave in simply because of the enjoyment I get from programming my site. But by using Wordpress, I saved time with coding so that I can focus on schoolwork and content on the site. I look forward to finally getting more time to write on the blog rather than having to program new information.

Using Wordpress has its advantages! Here’s a list of plugins that I’ve used on this version of the site:

  1. Akismet (Spam) – created by Matt Mullenweg
  2. Clean-contact (Contact Form) – created by Monkey Tree Labs
  3. Delete-Revision (Revision Management) – created by gohsy
  4. EG-Attachements (Document Library) – created by Emmanuel GEORJON
  5. Google XML Sitemaps (Site Map Generator) – created by Arne Brachhold
  6. ICS Calendar (Events Calendar) – created by Daniel Olfelt
  7. Multi-level Navigation Plugin (Main Navigation) – created by PixoPoint Web Development / Ryan Hellyer
  8. Recent Posts – created by Nick Momrik
  9. Sticky Menu (Secondary Navigation) – created by ericdes
  10. WP-Cache (Page Caching) – created by Ricardo Galli Granada
  11. WP-DBManger (Database Management) – created by Lester ‘GaMerZ’ Chan
  12. WP-Syntax (Code Syntax Highlighting) – created by Ryan McGeary
  13. WP Photo Album (Photo Albums) – created by Rubin J. Kaplan

Despite this, there are a few plugins that I had to create myself.  This includes plugins for editing a Curriculum Vitae and creating document repositories.  The theme itself was created by me specifically for kReese.net.  As these plugins and themes become more stable, I will link to these through my website.

]]> http://kreese.net/index.php/2010/01/kreese-net-in-2010/feed/ 2