SIMULATIONS ARE NOT SWISS ARMY KNIVES. DOES YOUR CLIENT KNOW? LET’S TAKE A BRIEF LOOK AT REASONABLE EXPECTATIONS OF STAKEHOLDERS THIS WEEK. AS USUAL, LET’S START WITH THE DEFINITION GIVEN BY DAVE STURROCK ON THE SIMIO BLOG AS A STARTING POINT:
Empty space, drag to resize
“Stakeholders must recognize the limitations of the technology and project constraints and not have unrealistic expectations. A project based on the assumption of long work hours is a project that has been poorly managed.”
Dave Sturrock, Simio blog
Empty space, drag to resize
There are two types of clients: those that don’t trust simulation at all and don’t see the point. And those that think it will solve all their problems. In this post, we will focus on the latter and explore how we can communicate the limits of simulation to them. But what are these limits, first of all?
Every simulation has limits. But does your client know that?
Empty space, drag to resize
Simulation is not a Swiss army knife tool. It may be used in many situations and is often better suited than analysis or intuition for decision support. However, not always:
- Very detailed models fail at long-term predictions as the uncertainty around results explodes.
- Low fidelity models should not be used for specific, short-term decisions.
- Simulating human behaviour is challenging. Be careful at every corner when founding your business decision on such models.
- If your decision can safely be made with traditional tools such as analysis or even intuition, there is no need to replicate the result with simulation.
Another limit of simulation that stakeholders need to be aware of is scope creep. I experience this almost every day: “Can you add XYZ as well, please?”; “Well, since you just did ABC, sure it would be easy to quickly add DEF, no?” and so on. Scope creep is real and will never go away. Clients have no means to estimate the workload involved. They see a useful tool and want to do more with it. They’d like to have a Swiss army knife, bringing us back to the first limit of simulation.
Another common misconception is estimating work effort. Say I implemented feature X into a simple model for the client and it took me one day. Three months later, the model has grown a lot. The client asks for a very similar feature like X and expects it to take one day and a bit. In truth, it will take me a week, easily. It is exponentially more expensive to add features to growing models.
So how would you educate your client about these limits and create “reasonable expectations”? The term “expectation management” has become very popular as it can safe you much trouble. Here are some key ingredients that help me every day:
- Have a clear design document and model description upfront. What does it do, what is excluded? Every scope-creep request can be judged against it.
- Make realistic work efforts. Be clear why simple tasks require a lot of work for complex models. Normally, referring to the extensive testing effort suffices.
- Check every aspect of your model against the question “Do we need to simulate this or are there better suited alternatives?” Be honest about requests that should be better handled without simulation.
