Maybe there are some biological bases to the fear of the unknown. Astrology, runes, religion, predictive models, we come up with all sorts of things to make the unknown a little bit more foreseeable. For millennia we have been trying to control our fates and it can be argued that our lives are now more predictable than ever. Life expectancy has increased sharply; most of the “developed world” has low criminality rates, some form of social security, insurance(s). And, like it or not, we are predictable. In a series of studies using mobility data (mostly from cell-phones, mostly without users informed consent), researchers have described boundaries of predictability from 60% to up to 95%1Some examples of papers describing attempts at anticipate our movements:
Understanding individual human mobility patterns
Approaching the Limit of Predictability in Human Mobility. The precise definition of this may vary but the idea is that by looking at past patterns, it is possible to predict with some accuracy what our next movement will be: we go from home to our kids’ school, to work, to the supermarket, home again. We tend to go to the same restaurants; once in a while we try a new one and ruin someone’s model.
Then, a pandemic comes along and the paradoxes emerge. If on the one hand, we are now even more predictable (we either stay home or severely reduce the number of possible whereabouts2Google published a mobility report, using mostly cell-phone location data, showing a strong reduction in movement in the past weeks: COVID-19 Community Mobility Reports ), on the other this generates a lot more uncertainty about the near future (we no longer make plans for a week’s or even a month’s time). Another one: in these times of uncertainty, people seem to need to be more in control than ever. It is as if our brains cannot accept the fact that we do not know what is going on, it is likely that we will not know for quite some time, and we have to play it by ear (the Portuguese expression in the title means to navigate without a map, using both strategic caution and limited knowledge).
Because we can’t accept how little we (can) know, we keep trying to improve our prediction systems and, thanks to human ingenuity, we have had moderate success in some of them. We have some accuracy in weather prediction and, pandemics aside, we can make plans for dinner next week. The difficulty is with the long and even medium term. So, we invented first wizards, then pundits, and now the armchair epidemiologist3This is a joke, inspired by this funny piece: Flatten the Curve of Armchair Epidemiology. These are typically male, often white, and tell us the future on TV. They use confidence as a cape and narrative as a sword. The problem is: they suck.
Over several decades, Philip Tetlock and his colleagues tried to measure how good the “best” pundits were and year after year they would ask these experts to make clear predictions (we will all die is not a good one), using a defined time frame (eventually, we will all die is still not good), and a numeric likelihood (100% chance that you will die in the next 10 minutes is a testable prediction). Then, they compared the predictions to reality and eventually found out that they were no better than a “gorilla throwing darts”4Apart from the book references, I find this piece quite good: Everybody’s an Expert. I am less of a believer on the second part of Tetlock’s work, on identifying “superforecasters” as it might be too dependent on their definition (they identify numerism more than prediction, but that’s a discussion for another time).. Not all of course, some got it right sometimes, but often in could be attributable to luck. For instance, imagine that everyone, for some odd reason, would start making predictive models of say, the number of infected with a certain virus over time. If there were 100 independent predictions, we could expect some of them to get it right, by pure luck. These would then be convinced that they had a special power and become pundits. Of course, for us to make sure it hadn’t been just luck, this person had to get it right more times, in other instances and still, in all honesty, we would still not be completely sure, because by random chance, some people just hit the jackpot more times than one would like.
But it gets even more interesting: many of us, even after being told we got it wrong, will not have our self-confidence affected5This is something that I hope to get back at in more details, but here’s a good intro: Why Facts Don’t Change Our Minds. We will come up with a posteriori explanations for why we didn’t get it exactly right, just ignore the critics, or find faults with their arguments. Sometimes this is harder to do, but even when talking about something as quantifiable as “number of infected” we can still claim that the problem is with how we count.
Niels Bohr, who was not famous for being stupid, is quoted as once saying that “making predictions is hard, especially concerning the future” and I find this sentence as further proof of his intelligence. So what would we need to make decent predictive models? 1) solid theory, such as the ones that physicists have and allow us to land robots on moving asteroids or 2) individual level /highly detailed data, that would allow us to simulate very complex dynamics. This is because if we, as a mass, are predictable, we also have our own individual pattern(s) and behave differently from others. Some top researchers have access to extremely detailed information and computational power that allow them to build very complex models6Here is one of the top researchers in the field describing some of their efforts to the NYTimes: Mapping the Social Network of Coronavirus but of course, having such fine grained information raises serious ethical and privacy concerns7Here is an academic example and an excellent NYTimes report making it very clear that cell phone location data cannot truly be anonymized., which will be the focus of the next post.
For the past weeks I have been asked to make predictions about what will happen regarding the current pandemic as I have worked on contagion and spreading mechanisms, and these are problems that I think about a lot. But all of our work has been on what we call “now-casting”, in opposition to “forecasting”. We can try to adjust the data in close to real-time and at least describe, as best we can, what is going on. This is terrible for one’s confidence as it requires accepting a lot of unknowns and is even worse for politics, as decision-makers need confident people and clear arguments. But I have come to believe that pretty much everyone who claims to understand and be able to predict such a dynamic system, apart from some general, time-independent trends, is most likely just overconfident. Unfortunately, we have no good way of convincing them they are just making more or less sophisticated guesses. The goods news are that “navegação à vista” is what got us to cross Cabo Bojador in the first place and we might as well continue to do it now, trying our best to eventually get a map.
