Fermi Questions: Creating Intelligence Without Collection

Collection is, for many, a fundamental part of and, in extreme cases, the essential purpose of, intelligence.  What would we be without all our drones and spies and sensors?

What if I told you that you can do intelligence without any collection at all?

You probably wouldn't believe me ... but ... you'd likely admit that the advantages would be substantial.  It would be blazingly fast - no waiting around for satellites to come into position or agents to report back.  It would be mindnumbingly safe - virtually no footprint, no assets to risk, no burn notices to issue.  It could reduce as much as 90% of the uncertainty in any given intelligence problem at essentially zero cost.

What is this prodigious procedure, this miracle methodology, this aspirational apex of analytic acumen?

Fermi questions.

Enrico Fermi was a mid-twentieth century physicist who created the first nuclear reactor.  He also taught physics at the University of Chicago.  He liked to ask his students questions like, "How many piano tuners are there in Chicago?"  

In the pre-internet days, this kind of question required a tedious trip through the phone book to determine the number.  Even today, using brute force to answer this question is not a trivial exercise.  Students almost always balked at the work involved.

Fermi's approach, however, was different.  He wasn't asking, "What is the most direct route to the answer to this problem?"  Instead he asked a slightly different and, for intelligence purposes, vastly more useful, question: "How close can I get to the answer with what I already know?"

So.  What did Fermi already know?  Well, the population of Chicago is about 3 million and from this he could immediately devise that there could be no more than 3 million piano tuners and that the minimum was none.  That may not sound particularly useful but just recognizing these facts already limits the problem in useful ways and points the way towards how to make the estimate better.

We know, for example, that the number of piano tuners has to be driven by the number of pianos in Chicago.  How many of those 3 million people have pianos?  Here we could tap into our own experience.  How many people do you know?  How many of them have pianos in their houses?

Some will say 1 in 10.  Some might say 1 in 100.  Even this wide range is very useful.  Not only does it narrow the problem significantly but also it highlights one way in which we could get a better estimate if we absolutely have to (i.e get a more exact number of people with pianos in their houses).  But we want to do this without collection so let's carry on!

With the average household being a shade under 4 people, we can estimate that there are about 750,000 households in Chicago.  We can further refine that to between 75,000 and 7500 pianos (depending on whether you thought 1 in 10 households had a piano or 1 in 100).

Oh, I know what you  are thinking!  What about all the non-household pianos - at schools and such - that you are conveniently leaving out.  I would say that my high end estimate of the number of pianos includes them and my low end estimate does not so they are in there somewhere.  It is a "good enough" answer for right now for me.  For you that might not be the case, however, so you can make your own estimates about what these numbers might be and put them into the mix.

Working about 250 days a year (weekends, vacation and holidays excluded) on about 2 pianos a day means that Chicago needs between 150 and 15 piano tuners.  

How many piano tuners are there really in Chicago?  Wolfram Alpha is one of the best search engines to use to answer these kinds of questions.  It permits users to ask natural language questions and then dips deeply into public databases to extract precise answers.  When asked, "How many piano tuners are there in Chicago?" this is what you get:

Note that Wolfram gives us the number of all musical instrument repairers and tuners - 290 as of 2009.  Certainly not all of them are piano tuners.  In fact, once you consider just how many instruments need to be professionally tuned besides pianos and you subtract the number of repairers of all kinds of instruments that do not tune pianos, you are lucky to have a third of these musical instrument repairers and tuners who actually can tune a piano.

More importantly a third of 290 falls comfortably within the 15-150 limits derived from our Fermi process.

Without leaving our chairs.

Intelligence without collection.

What if relying on Fermi questions results in really wrong answers?  First, I could say the same thing about any intelligence methodology.  Very few of them have been tested to see if they actually improve forecasting accuracy and all of them take time and resources to implement.  All of them can be wrong.  Here, at least, both the logic chain and the path to improving the estimate is obvious.

Second, I would ask, what level of precision do you actually need?  Norm Augustine, former CEO of Lockheed Martin used to say, "The last 10 percent of performance generates one-third of the cost and two-thirds of the problems."  Augustine was talking about airplanes but he could have just as well been speaking of intelligence analysis.  Getting ever more narrow estimates costs time and money.  Good enough is often - in fact, surprisingly often - good enough.


Third, it is unlikely to give you really wrong answers - say one or two orders of magnitude off.  This is one of the best benefits of going through the Fermi process.  It allows you to have a good sense of the range in which the right answer will likely fall.   For example, if, before you had done a Fermi analysis, someone came up to you and said that there are 100,000 piano tuners in Chicago, you might not question it.  A Fermi analysis, however, suggests that either something is really wrong with your logic or, more likely, that the person does not know what they are talking about.  Either way, the red flag is up and that might be just enough to prevent a disastrous mistake.

You can easily try this method yourself.  Pick a country that you know little about and try to estimate the size of its military based on just a few easily found facts such as population and GDP.  Once you have gone through the process, check your answer with an authoritative source such as Janes - oh! - and please do not hesitate to post your results in the comments!

By the way, I routinely use this method to get students to answer all sorts of interesting and seemingly intractable problems like the number of foreign government spies working within the US Intelligence Community.  The answer we get is usually right around 100 which always seems to surprise them.

Finally, if you are interested in integrating Fermi Problems into your tradecraft, there are lots of good resources available.  One of the best has been put together by the Science Olympiad, which actually holds a Fermi Problem competition each year.

Tweet