Why simulations mislead, and how to read one

A simulation will tell you almost anything if you let it. It is the most persuasive kind of evidence and among the least reliable, because it produces confident, detailed, visually compelling output whether or not it corresponds to anything. Knowing how they go wrong is most of knowing how to read one.

The first failure: tuning until it works

The commonest way a simulation misleads is not fraud and does not feel like error at the time. It is this: build a world, run it, see nothing interesting, adjust something, run it again, and repeat until something interesting happens. Then report the interesting thing.

Every individual adjustment is defensible. The problem is the loop. By the end, the world has been shaped by the outcome it eventually produced, and the claim that the result emerged from generic rules is much weaker than it sounds, because the rules were selected for producing it.

The defence is to fix the rules before running and treat any change that would alter how the world unfolds as the start of a new experiment rather than a patch to the current one. That is what a frozen ruleset means here, and why changing physics starts a new universe rather than modifying the running one.

Tuning toward a result compared with fixing the rules first Two workflows. In the first, a loop runs from rules to result and back to adjusting the rules, repeating until an interesting result appears, which is then reported. In the second, rules are frozen, a prediction is registered in advance, the run happens once, and whatever comes out is reported including nothing. Tuning toward a result Fixing the rules first rules run interesting? freeze rules register prediction run once adjust and repeat until yes report whatever happened, including nothing
The dashed loop is the whole problem. It is invisible in the write-up, because a tuned world and a principled one look identical once you are only shown the final rules.

The second failure: the picture is not the model

A simulation has two separate things that can be wrong, and they fail independently: what it computes, and what it shows you.

This project has produced a clean example of the second. Organisms were visibly standing in the ocean. The physics was right, and roughly two thirds of those cases were organisms genuinely on land, because material had been excavated and deposited to raise those cells above the water line. The renderer had painted the terrain once when the world loaded and never looked again, so it was showing everyone a picture of the world as it had been at birth.

Nobody could have caught that by checking the numbers, because the numbers were correct. Every test passed. The full account is in everything that has gone wrong, along with several others of the same shape, and the pattern across all of them is the same: the physics has been comparatively reliable and the story told about the physics repeatedly has not.

The third failure: the assumptions do the work

Every simulation contains decisions that are not results. How much energy an action costs. How fast a resource regrows. How wide a tolerance is. These are set by a person, and the output depends on them.

A model can therefore be internally flawless and still tell you nothing, if the interesting part of the answer was contained in the parameters rather than derived from them. The honest response is not to eliminate assumptions, which is impossible, but to state them, and to check whether a conclusion survives when they are varied.

The version of this worth watching for is a simulation that reproduces a known phenomenon and is presented as explaining it. Reproducing something is easy and proves very little, since many different mechanisms produce similar-looking output. Explaining it means showing that the mechanism proposed is the one doing the work, which requires removing it and demonstrating the result disappears.

How to read one, including this one

A short list of questions that will tell you most of what you need about any simulation result, including anything on this site.

Was the threshold set before the run? A criterion chosen afterwards will be chosen to be met. One earlier candidate here failed its own pre-registered survival test, coming in at roughly a quarter of the baseline population against a threshold of sixty per cent, and that counts as a result only because the number was fixed in advance.

Where are the runs that produced nothing? If every reported run is interesting, the interesting ones are being selected from a larger set you are not being shown.

Was the behaviour written into the rules? A sensory channel called "food" makes moving toward food unremarkable. Every channel here is a generic quantity and none names an object, which is checked rather than asserted, on how organisms perceive and act.

Can it be run again? A result that cannot be reproduced is an anecdote. Every run here is reproducible from its seed down to the byte, which is what allowed an extinct world to be re-run offline with instruments attached rather than guessed about.

Why publish the failure modes

A page listing the ways work like this deceives people is an odd thing for a project doing that work to publish. It is here because the alternative, asking to be trusted, is worth less. The safeguards this site runs on, frozen rulesets, reproducible seeds, registered thresholds and published null results, are all on methodology, and they exist because of the mistakes rather than in anticipation of them.

Keep going

Watch the universe