Mfcttrf

"All models are wrong, but some are useful"

I usually work on what one could call operational problems. There I usually do not have too many troubles figuring out the level of details needed for a model to provide value. However, when I happen to work on tactical/strategic problems I struggle more to figure out the appropriate level of detail.

To give some support for the discussion let’s consider this made up example:
Let say that you are working on a problem where you want to determine what is the appropriate mix of vehicles to transport handicapped people from their home to day-care center. You know that the demand varies every day but you want to decide which vehicles you need to buy to operate for the next 4 years.

Some vehicles have a fixed configuration while others can be reconfigured (some seats can be folded and you can accommodate a wheel chair in place of 2 regular seats for example). In an operational model you would definitely want to take this reconfiguration into account because it can make a difference between a “good” and a “bad” route. However when solving a more long term problem in which you only have a partial view of what your demand will look like in 2 months from now does it make sense to take this into account? Or is it a step too far which is just going to make the model more complex/slow without really adding values to the solution.

TL;DR
How do you determine when your model is detailed enough to be useful?
When is an excess of detail actually hurting the model?

Interesting! Here are some questions that may be helpful to think through:

• Will more detail substantially change the optimal solution?


• Am I including enough detail to answer the question I want to answer?


• If there are similar models for my problem, what do they include?


• What other details am I excluding? Could they have more of an effect that the dynamic I'm debating?


• Will a more precise answer be a more accurate answer? (E.g., if data for the extra details are wrong, including them may give a better solution to the wrong problem)


• What are my computational limitations? (E.g., if I don't have to solve the problem very often, maybe a longer solution time with a better answer is the way to go.)


• What level of detail does my client/collaborator need to "trust" the model?

Some downsides to excess detail include:

• Longer solution time/tractability


• Risk of obscuring key takeaways


• Bad data may lead astray

To figure out whether more detail may affect the solution, one option is to do sensitivity/scenario analysis on the simpler model. For the example you give, that might be running the model with different levels of demand to see how the solution changes. If it doesn't change much, that may indicate you don't need to let it vary.

In terms of being able to answer the right question, that sounds obvious but perhaps a good practice to double-check. For the example you give, if the goal is to figure out what mix of vehicles to buy, the model should probably include all of the vehicle types/configuration options.

Looking forward to other responses.

I think it's useful to think about timescale, which is related to, but not equivalent to, level of detail. In particular, I think it is usually better to start with a model that does not include multiple, very different, timescales.

Your hypothetical problem involving vehicle mix (strategic question, timescale = years) and seat configuration (operational question, timescale = days) is a great example of this. One would presumably want to start with a model that optimized one or the other, but not both.

But this is not a hard-and-fast rule, and it is worth some experimentation.
If the shorter-timescale decisions do not significantly affect the longer-timescale ones, they should not be included in the model. So, if the optimal vehicle mix changes significantly when you include seat configuration in the model, seat configuration should be included in the vehicle mix model. Otherwise, it should not. (Probably it will not change the mix problem, so it should not be included.)

Of course, it is always a tradeoff. As another example, facility location is a strategic problem. So it's worth asking whether we should include tactical decisions like inventory or operational decisions like routing into the facility location decision.

In the case of inventory, the inclusion of inventory changes the optimal facility locations, and moreover in at least some location–inventory models, the computational cost of adding inventory is relatively small. Therefore, it seems reasonable to include inventory in the facility location problem.

On the other hand, routing tends not to change the optimal facility locations much (I believe—someone might want to check me on that), and moreover, location–routing models are much harder to solve than straight facility location models, so the tradeoff argues for not including routing, in general.