Do Intel Analysts Believe Info Is Good Just Because It's Secret?

(Note:  This post introduces a new author to Sources and Methods, Melonie Richey.  Mel and I will be working on a number of projects over the next year focused on the intersection of game-based learning, cognitive bias and intelligence analysis.  In her first post for SAM, she introduces us to a new form of bias...)

Heuristics (or "rules of thumb") and biases influence both decisions and the analysis on which they are based.

This is an inescapable fact.

Anchoring bias causes us to hyperfocus sometimes on irrelevant information, confirmation bias leads us to marry ourselves to the first probable conclusion we reach and bias blind spot allows us to do all this operating under the assumption that we, as trained analysts and generally educated people with inquisitive minds, are not biased at all.

These biases are extensively studied and appear in a myriad of interdisciplinary literature ranging from Wizards Under Uncertainty: Cognitive Biases, Threat Assessment, and Misjudgments in Policy Making (cognitive biases in terms of Harry Potter) to The Big Idea (Harvard Business Review) to The Mind’s Lie.

A recent paper entitled The Secrecy Heuristic - authors Mark Travers, Leaf Van Boven, and Charles Judd from the University of Colorado - presents research substantiating a new heuristic that likely affects both intelligence professionals and decisionmakers: The idea that we “infer quality from secrecy” when it comes to intelligence analysis.  In other words, we will give the same information more value just because it is secret.

This paper presents the three reasons that we fall victim to the The Secrecy Heuristic, and outlines the experimental evidence that validates the presence of this heuristic in information quality evaluation.

• “First, secret information is sometimes genuinely better information than public information, particularly in strategic contexts.” As the article elaborates, secret information is something valuable simply because it is secret (think about financial investors and stock market prices – the information is only valuable because not a lot of people are privy to it – this is where the whole concept of insider trading originates). The fact that secret information is sometimes better, leads us to associate that quality with all secret information.

• “Second, people may view secret information as being of higher quality than public information because of personal experience with their own and others’ secrets.” Think about gossip. In a social context, what is a secret? Usually, it is something personal or embarrassing that an individual doesn’t want everyone else to know, but which everyone else (everyone else defined as the immediate social network) would likely take interest in.  Again, this reinforces our bias in favor of secret information.

• “Finally, governments often behave, in foreign policy contexts, as though secret information is valuable and of high quality.” After all, we have multiple intelligence agencies with 100's of thousands of employees dedicated to secrecy; both maintaining it on our soil and revealing it on others. Uncovering secrets led to, as the article references, tracking down the leader of the Pan Am flight 103 bombing, the 1993 World Trade Center bombing and Osama bin Laden. It’s not hard to see why secrets are so …well, secret.  This, in turn, reinforces our bias in favor of secret information even when non-secret information is just as valid.

The argument in The Secrecy Heuristic is that “these factors may lead people to use informational secrecy as a cue to infer informational quality” and, lo and behold, they found just that.

In the first of three experiments, the researchers tested whether or not people weighed secret information more heavily than public information in hypothetical foreign policy recommendations. In experiments two and three, the researchers tested whether or not secret information is perceived as higher quality than public information, and in experiment three, how secrecy impacted how favorably the foreign policy recommendation was rated.

On aggregate, “secrecy led to higher judgments of information quality.” For example, on a scale of 1 to 11, the mean judgment of secret information quality was 7.46 where the mean judgment of quality for public information was only 6.93.

In short, secrecy does not necessarily equate to importance, relevance or reliability - we just think it does.

(H/T to Tammy G for pointing us towards this article!)

Tweet