Is ABM result fabircated?

Share & Showcase
1

Hi everyone, I’m a netlogo architect. Recently I built a business simulation model that is too complex to be perfectly understood and calibrated, it includes more than 50 elasticity coefficient and it took me lots of hours to calibrate it. Thats where I encountered the classical problems: is ABM result really effective (ethical) or its fabricated? This experience inspired me in some philosophical ways. I dont have enought time to talk about this in detail, so I will deliver it in very brief ways.

  1. When we are dealing with ABM modeling, we are starting from theories and experience, instead of the real world data. (This is just from the project perspective, the first thing you do is write a decision flow instead of run some regressions)
  2. The experience and theories we use, is all just hypothesis.
  3. ABM design is the art of combining different hypothesis to make a process that we can not perfectly caputre “come to life“.
  4. An legit model should be valid and accurate, we examnine the validity and calibrate the model by comparing it to the real life data (including intuition and statistical prove)
  5. So the starting point of ABM is hypothesis, and the goal is to achieve the minimum error comparing with real life scenarios.
  6. Here comes the first logic: We use hypothesis to build models, and then check the result.
  7. But the result of ABM is always under the manipullation of reearcher, this is a paradox of ABM, it is too subjective, in the end of the day we are changing the hypothesis and theories we use, in order to see the data.
  8. There comes the second logic: We use the result to build models, and see which hypothesis we can use.
  9. Thats why, ABM is not only a economic and statistical tool, when we are changing the model, we are not only giving answers to questions, we are also changing the questions according to answers (or, we are changing the way we perceive and deliver the questions)
  10. ABM is also about narratology and hermeneutics. We can even do critical narratological analysis to ABM.

I know these ideas are 99% spoken by other thinkers or it can be absurdly wrong. So I am very anxious waiting to see what you think about this topic.

2

There is a lot to unpack here, but here are some thoughts.

Working with ABM in scientific contexts starts from a hypothetico-deductive approach, usually with a conceptual model (such as a causal-loop diagram; here is an example), and then moves to an inductive approach after you start to calibrate and analyze the model. This was only possible with the advent of computer science. Because this approach unites deduction and induction, some authors prefer to say that this is a third way of doing science (Axelrod, 1997).

A conceputal ABM with 50 parameters is hardly useful in this context. This is because you are accumulating a lot of errors and trying to compensate for them with calibration (that is, when you are able to do this). This is not the same thing when working with empirical ABMs. See this article from Sun et al. (2016) and this interview with Joshua Epstein for more about this subject.

What a 50-parameter conceptual ABM looks like, (just for fun, :slightly_smiling_face:):

An ABM should be viewed like any other model. For example, when performing experiments using an animal model, that does not mean the results will translate to humans. As George Box said, all models are wrong, but some are useful (Box, 1979).

Considering the Popperian approach to science, you will always need to check these results against reality. ABMs never confirm your hypothesis; they can only corroborate it. As Popper wrote:

“Confirmations should count only if they are the result of risky predictions”

(Popper, 1969/2002)

ABMs can be a useful tool for corroboration only. This has special value for areas like History and Archaeology, which can only corroborate a narrative but never confirm it (again, from a Popperian perspective). In most cases, ABMs are useful for:

  • Development and corroboration of theories
  • Exploration of scenarios and leverage points
  • Prediction of future scenarios (for empirical models only)

But we never really confirm anything with ABMs.

So, is an ABM result fabricated?

In the sense of modeling in general, yes, just like any other model.

2 Likes
3

Thank you so much! Your podcast link really led me to know more about the “generative explanation“ philosophy! Your answer are very elegant and helpful!

1 Like