kReese.net

Introduction to Simulations

Simulations are an important tool in computer scientists’ arsenal. It allows a scientist to statistically analyze a designed experiment with only a fraction of the cost of implementing a final system.

In simulations of systems, be it computer performance analysis or protein folding, there are two major classes of simulations: Discrete-Event Simulations and Continuous Simulations. A Continuous Simulation is a simulation in which the state variables changes continuously over the designated time that the simulation is run. In contrast, a Discrete-Event Simulation is a simulation in which the state variable of the system change only at discrete points in time.

The majority of the posts about Simulations on this site will discuss Discrete-Event Simulations, though many of the equations and statistical concepts could be used with analysis of Continuous Simulations with only minor changes. The rest of this post will outline the advantages and disadvantages of simulation, as well as a list of common mistakes in simulation.

Advantages/Disadvantages

Before listing some of the advantages and Disadvantages, we must first know when simulation is appropriate and when simulation is likely to not be appropriate to a situation.  Though Simulations are a powerful tool, in many situations a common sense solution can be reached or in some cases can often by solved analytically using formulae related to the model.  In some cases, we may run into a lack of data for the model, in which case the model may be difficult to simulate.  We may also run into a few rare cases of the cost/benefit of the simulation to hold no significant value.  However, simulations can often be applied to a multitude of situations including: The study of complex systems or subsystems, the knowledge that would be gained by the simulation holds significant value, to study the effects of changing inputs, to determine possible system requirements (for example, software), to train humans or intelligent machines, visualization of systems, and of course many other topic.  The following lists are shortened versions from [1]

Advantages. A simulation can lead to discovery of new models or systems.  Some of the advantages that using simulation is to:

• Explore new options without disrupting the current operation of a system.
• It has significantly lower cost in comparison to implementing a final system before testing.
• We can simulate a larger amount of time in a shorter period using simulations, known as time-warping.
• It allows us the study the interactions or importance of variables in the system
• It allows us to study various “what if…” conditions
• and, it helps scientists or engineers to develop a model that helps understand the system better.

Disadvatages. Despite the significant advantages, there are a handful of disadvantages that should be considered before implementing a simulation for your needs.

• Implementing a simulation takes time and often this results in a company spending more money to hire the necessary designers for the simulation.
• Simulation is an art.  The results of a simulation are highly variable, often based on the skill of the designer.
• Analysis of simulation results can often be difficult unless programmed correctly.  This again relates to Simulation being an art.

Common Mistakes in Simulation

This section discusses some of the aspects that have to be overcome and what makes Simulation more of an art.  The list below is a simplified version of the list of common mistakes in [2]

1. Inappropriate Level of Detail: Without the proper level of detail within the simulation could result in the failure of the simulation or inaccurate results in comparison to the model being tested.  However in order to do a proper analysis of the data for the model, we have to make some simplifications and assumptions.  The difficulty lies in capturing the proper level of detail while simplifying the model well enough to analyze the data received from the simulation.
2. Improper Language:  A simulation can be designed in almost any programming language, however choosing the proper programming language has a significant impact on the development time of the model.  Many languages can cut development times significantly, and some can even make analysis of the data significantly easier for the developer.
3. Unverified Models: Simulation models are often implemented with large programs.  The developer has to take special care to ensure that there are no bugs or programming errors which might invalidate the results of the Simulation.
4. Invalid Models:  Incorrect of assumptions about the system could result in invalidated data even with a flawless program.
5. Improperly Handled Initial Conditions:  Including the initial portion of the simulation could result in non-representative data of the system.  To avoid this, one should give the simulation the required burn-in period for the data to converge on data that is representative of the system.
6. Too Short Simulations:  Often times, the Analyst will try to save time by running the system within a shorter period of time.  Related to item 5, this can result in data that is non-representative of the systems data.
7. Poor Random-Number Generators: Random Numbers are an important aspect of simulations.  It is often better to use a well known generator, however in some cases, the generators may not work for your simulation.  Generating random numbers will be discussed in a future post.
8. Improper Selection of Seeds: Seeding the random number generator is important to maintain independence among the streams in the simulation.  In many cases, using the same seed in the streams may result in correlation among each of the system processes which might result in non-representative data.

References

[1] J.Banks, J.Carson, B. Nelson, and D. Nicol Discrete-Event System Simulation New Jersey: Prentice Hall, 2005.

[2] R.Jain, The Art of Computer Systems Performance Analysis New York: Wiley, 1991.