How To Analyze Black Swans

Black swans, we are told, are both rare and dramatic.  They exist but they are so uncommon that no one would predict that the next swan they would see would be a black one.  

The black swan has become a metaphor for the limits of the forecasting sciences.  At its best, it is a warning against overconfidence in intelligence analysis.  At its worst (and far too often it is at its worst), the black swan is an excuse for not having wrung every last bit of uncertainty out of an estimate before we make it.

One thing does seem clear, though:  We can have all the information and structured analytic techniques we want but we can’t do a damn thing in advance about true black swan events.  They are, by definition, unpredictable.

Or are they?

Imagine a single grain of sand falling on a table.

And then another.  And then another.  While it would take quite some time, eventually you would have… well… a pile of sand.

Now, imagine this pile of sand growing higher and higher as each single grain falls.  The grains balance precariously against each other, their uneven edges forming an unsteady network of weight and weaknesses, of strengths and stored energy, a network of near immeasurable complexity.  

Finally, the sandpile reaches a point where every time a single grain falls it triggers an avalanche.  The vast majority of times the avalanches are small, a few grains rolling down the side of the sandpile.  Occasionally, the avalanches are larger, a side of the pile collapsing, shearing off as if it had been cut away with a knife.

Every once in a while – every once in a long while – the falling of a single grain triggers a catastrophe and the entire pile collapses, spilling sand off the edge of the table and onto the floor.

The sandpile analogy is a classic in complexity science but I think it holds some deep lessons for intelligence analysts trying to understand black swan events.

Just as we cannot predict black swan events, we cannot predict which precise grain of sand will bring the whole sandpile down.

Yet, much of modern intelligence focuses almost exclusively on collecting and analyzing the grains of sand – the information stream that makes up all modern intelligence problems.  In essence, we spend millions, even billions, of dollars examining each grain, each piece of information, in detail, trying to figure out what it will likely do to the pile of sand, the crisis of the day.  We forecast modest changes, increased tensions, countless small avalanches and most of the time we are right (or right enough).

Yet, we still miss the fall of the Soviet Union in 1991, the Arab Spring of 2010, and the collapse of the sandpile that began with a single grain.

What can we do?  It seems as though intelligence analysts are locked in an intractable cycle, constant victim to the black swan.

What we can do is to move our focus away from the incessant drumbeat of events as they happen (i.e. the grains of sand) and re-focus our attention on the thing we can assess:  The sandpile. 

It turns out that “understanding the sandpile” is something that complexity scientists have been doing for quite some time.  We know more about it than you might think and what is known has real consequences for intelligence.

An example of a power law, or long-tail, distribution

In the first place, for the sandpile to exhibit this bizarre behavior where a single grain of sand can cause it to collapse or a single small incident can trigger a crisis, the pile has to be very steep.  Scientists call this being in the critical state or having a critical point.  More importantly, it is possible to know when a system such as our imaginary sandpile is in this critical state – the avalanches follow something called a “power law distribution”.

Remember how I described the avalanches earlier?  The vast majority were quite small, a few were of moderate size but only rarely, very rarely, did the pile completely collapse.  This is actually a pretty good description of a power law distribution.  

Lots of natural phenomena follow power laws.  Earthquakes are the best example.  There are many small earthquakes every day.  Every once in a while there is a moderate sized tremor but only rarely, fortunately, are there extremely large earthquakes.

The internet follows a power law (many websites with few links to them but only a few like Google or Amazon).  Wars, if we think about casualties, also follow a power law (There are a thousand Anglo-Zanzibar Wars or Wars of Julich Succession for every World War 2).  Even acts of terrorism follow a power law.

And the consequences of all this for intelligence analysts?  It fundamentally changes the question we ask ourselves.  It suggests we should focus less on the grains of sand and what impact they will have on the sandpile and spend more resources trying to understand the sandpile itself.

Consider the current crisis in Crimea.  It is tempting to watch each news report as it rolls in and to speculate on the effect of that piece of news on the crisis.  Will it make it worse or better?  And to what degree?  

But what of the sandpile?  Is the Crimean crisis in a critical state or not?  If it is, then it is also in a state where a black swan event could arise but the piece of news (i.e. particular grain of sand) that will cause it to appear is unpredictable.  If not, then perhaps there is more time (and maybe less to worry about).  

We may not be able to tell decisionmakers when the pile will collapse but we might be able to say that the sandpile is so carefully balanced that a single grain of sand will, eventually, cause it to collapse.  Efforts to alleviate the crisis, such as negotiated ceasefires and diplomatic talks, can be seen as ways of trying to take the system out of the critical state, of draining sand from the pile.

Modeling crises in this way puts a premium on context and not just collection.  What is more important is that senior decisionmakers know that this is what they need.  As then MG Michael Flynn noted in his 2010 report, Fixing Intel: 

"Ignorant of local economics and landowners, hazy about who the powerbrokers are and how they might be influenced, incurious about the correlations between various development projects and the levels of cooperation among villagers, and disengaged from people in the best position to find answers – whether aid workers or Afghan soldiers – U.S. intelligence officers and analysts can do little but shrug in response to high level decision-makers seeking the knowledge, analysis, and information they need to wage a successful counterinsurgency."

The bad news is that the science of complexity has not, to the best of my knowledge, been able to successfully model anything as complicated as a real-time political crisis.  That doesn't erase the value of the research so far, it only means that there is more research left to do.

In the meantime, analysts and decisionmakers should start to think more aggressively about what it really means to model the sandpile of real-world intelligence problems, comforted by the idea that there might finally be a useful way to analyze black swans.

Tweet