- This topic has 1 reply, 2 voices, and was last updated 7 years, 6 months ago by
.
-
Posts
-
Here is a question I recently received by e-mail from a team that is developing a server based website that uses ST-Sim as it’s engine
:
Yesterday during our project meeting we discussed how many iterations we should”allow” a user to do. Some wanted the full power of Monte Carlo ie 100 runs (?). This would take much time and compute power so users would have to be notified it would not be instantaneous.But talking with a colleague who has run STSMs for riparian systems, I found out that he only does 20 iterations max but does many runs with many management or disturbance choices. His main goal being to look at effectiveness of management strategies.
There are a few considerations when taking this question into account. First of all, if the goal of your website is to allow users to explore lots of alternatives through gaming, but not necessarily make robust inferences from the model results, then I agree that 20 realizations or less (5-20) is probably adequate. If however you would like to produce Monte Carlo confidence intervals (https://www.jstor.org/stable/2530926?seq=1#page_scan_tab_contents). The range of your confidence intervals that you can create will be limited by the number of realizations that you run. With only 20 realizations you can at most have 90% confidence intervals, with 40 you can have 95% confidence intervals, etc… Furthermore, the more simulations you run, the less likely there may be some bias in your estimates of the average and confidence intervals across realizations. This will vary from one model to another so you may want to do some sensitivity analysis around how your projections change with the number of iterations run. Finally, ST-Sim does have the ability to run model iterations or realizations in parallel. This means that if the server you are running on has enough memory and cores, you could run 100 realizations in close to the same amount of time that it takes to run a few in series.
- You must be logged in to reply to this topic.