Models don’t matter

Models don’t matter. What matters are predictions. If someone has a model of something, but it doesn’t make any concrete predictions, it can’t be tested. It is therefore worthless.

It doesn’t matter if it’s a fancy model made by a bunch of smart people. It doesn’t matter if you run the model 10,000 times. It doesn’t matter if the output of the model makes nice graphs. It doesn’t matter if the person making the model made previous predictions correctly.

The scientific method, in large, is about figuring out how to test predictions that follow from models of reality. Developing a model is a preliminary step. Testing the model is much more important.

So, if a political pundit makes a mathematically explicit, complex-sounding model, but there are no concrete predictions it makes or ways it can be tested, that model is worthless.

Ever hear a political pundit who will make what sounds like a pretty straightforward prediction, but then hedges it with all sorts of qualifications that render it a non-prediction? These people are – let’s use a technical term – blowing smoke. As entertainment, I suppose, it’s fine. But anyone who actually wants to know what the outcome is going to be should stop listening to these people.

Similarly with data-based punditry. I think many people hear ‘data’ or ‘mathematical’ or ‘model’ and think there must be something important or right about what the pundit is saying. This is nonsense – it is a variation of ‘b.s. baffles brains’. The truth is, political punditry is filled with charlatans who don’t know what they’re talking about, but pretend they do (typically because they’re getting paid to pretend).

There are some things that are very difficult to predict – instead of pretending they have some kind of crystal ball (including giving very official sounding percentages to various outcomes, all of which are basically untestable), the people in question who are blowing smoke should just say ‘we don’t know’.

Leave a Comment