Business and Intelligence Techniques: the Role of Competing Hypotheses

I get a lot of requests to discuss further the application of intelligence analysis to business, so today I’ll discuss the uses and limitations of a common analytic technique.

One tool that I teach at IE is the Analysis of Competing Hypotheses (ACH).  ACH is an analytic tool originally developed by Richards J. Heuer at the CIA, but it is remarkably useful in business as well.  ACH uses a deceptively simple framework to use ideas from the scientific method, cognitive psychology and decision analysis to overcome a common but immensely important bias:  the fact that we tend to perceive what we expect to perceive rather than what actually exists.  To illustrate this tendency, read the words in the three triangles below:

If you’re like most people, the phrases “written” in the triangles are familiar.  To find out what’s actually written in each triangle, refer to the bottom of this entry (or try the old proof-readers trick of reading them backwards).

The implications of this little exercise for business strategy are profound.  It clearly demonstrates that perception is an active rather than a passive process:  your senses help construct what you expect to perceive rather than simply record reality.  Why does this matter?  It matters because the cumulative effect of our tendency to notice only that which we expect to perceive leads us to systematically gather facts that confirm our existing views and to systematically overlook facts that conflict with – or even demolish – these views.  The situation is made even more grave by the fact that studies repeatedly demonstrate that more (and more unambiguous) information is required to change a hypothesis once it has been formed, so if you get started down the wrong track, you’re more likely to stay on it until you get a rude shock (A.K.A. a strategic surprise or disruption).  Note that these statements aren’t about “wishful thinking” – they’re about the role of expectation and incorrect hypotheses in collecting and using information.

If “Holding an incorrect hypothesis” sounds like an abstract threat to your business, remind yourself that every marketing plan, financial model, consulting assignment or due diligence exercise embodies at least one hypothesis and rests on numerous assumptions.  If your product doesn’t sell, or if your investment fails, it’s not the product or investment that’s wrong:  it’s you, your assumptions and your analysis.

Not every aspect of strategy can be decided with ACH – it says nothing about goals or personal preferences – but it beats many processes (or lack thereof) often used to decide between competing explanations or proposals.  ACH helps overcome the predilection to notice only “confirming” facts by insisting that we systematically search for evidence that disconfirms our views.  It begins by constructing a matrix with the Y axis labelled “Evidence”, and the X axis listing mutually exclusive (indeed, “competing”) hypotheses that would drive your decision; in later steps, you fill in the resulting matrix by noting which evidence in the Y axis disconfirms each hypothesis in the X axis.

To learn how to do ACH in full, try chapter eleven in Jones, or Chapter eight of Heuer’s classic The Psychology of Intelligence Analysis. Free software for employing ACH is also available from the Palo Alto Research Center (but it is utterly useless if you don’t thoroughly understand the ACH technique before using it.  In my view, automating the ACH process can also tempt users to think of it as a “Magic Answer Machine” instead of a tool for thinking.

Where specifically might you decide to use ACH?  One area is when performing due diligence, where it can be used to uncover, categorize and evaluate “non-spreadsheet risk”:  the warning signals about an acquisition or business partner that don’t appear in conventional financial databases.  If you’re working in so-called Emerging Markets, this is where much of your risk lies.  If you’re not familiar with the concept, a great introduction to non-spreadsheet risk recently appeared in an article by Fred Enochs of TD International.

Like any method, however, ACH has its problems and limitations (some of them elaborated by Tim van Gelder).   It is laborious to construct (though it provides a great decision audit trail, and is a wonderful way to collaborate with others, over time, on the analysis of a complex problem), involves numerous discrete judgments, and it is susceptible to deception.  Nevertheless, in my view ACH is an indispensible tool for anyone doing more than a cursory investigation of a business idea.  It should certainly be part of the intellectual toolkit of high school students, much less MBAs, MIFs and MIMs.

One of the central thrusts on my current work with Philippe, however, is a focus on what we call the Generation of Competing Hypotheses (GCH).  After a strategic surprise or major business disruption, many people are content to ascribe the surprise to a “Failure of imagination”, and leave it at that.  This mental shoulder-shrug is enabled not only by intellectual lassitude (often reinforced by a dose of bureaucratic self-interest), but also by the habit of both intelligence analysts and business strategists to conceive of their work as purely mental processes.  Indeed, the second sentence of Chapter One of Heuer’s magnum opus begins with “Intelligence analysis is fundamentally a mental process…”

In contrast, our point of departure when seeking to understand strategic surprise and disruption is that business strategy creation is, first and foremost, a social – not a purely mental – process.  Intelligence analysts and business strategy practitioners are not solitary chess players (with complete and perfect information spread before them).  Nor are they lone physicists, concerned purely with natural phenomena.  Most of all, no matter powerful and appealing Rodin’s sculpture may be, they are not pure, isolated, solitary Thinkers.

Instead (as I have pointed out before) analysts and strategists are people embedded in a social context, dealing largely in the realm of social rather than natural facts.  As a result, their identities and culture (both corporate and individual) have a profound effect on how they generate hypotheses and how they focus their analytical efforts (even when they apply methods like “divergent thinking”).  That means that the identities and culture of analysts and strategists have a huge impact on their ability not merely to analyze competing hypotheses, but especially to generate the competing hypotheses that form the core (literally the key “X” axis) of the ACH method!  Purely psychological approaches to this problem miss these, and as I explain in Constructing Cassandra, the implications of this insight are far-reaching.

Nine times out of ten, when I explain the idea of systematically exploring the GCH to clients or students, “Brainstorming” is mentioned as a possible solution.  It’s not.  In fact, brainstorming’s utility has been largely debunked (Chapter Five in the book 59 Seconds explores this fact further, and links brainstorming’s ineffectiveness to theories of social loafing).

In future entries, we’ll discuss more about how identity and culture affect the generation of hypotheses.  In the meantime, you can fight confirmation bias (and experiment with the power of  ACH in its simplest form), by always asking what Neustadt and May call “Alexander’s Question”:  When considering a hypothesis, always ask yourself, “What information could I learn from any source that would force me to change my mind?”  If you can imagine a killer fact, consider where you would find it.  You can also, of course, give a full-blown ACH matrix a try when you next undertake serious analysis.  If you understand ACH’s limitations, it’s a valuable tool.

Triangles Solution:  The articles “the” and “a” are written twice in each phrase; the unmodified phrases are so familiar most people overlook the extra articles when the phrases are presented in this manner.  This “overlooking” can be cumulative if one isn’t careful to test hypotheses for disconfirming evidence… This example is taken from The Psychology of Intelligence Analysis, page 8.

NB:  For very close readers, please not that the CIA Analyst who developed this technique really is named Richards not Richard.  Check the cover of his book!

PS If you find our perspective interesting, why not subscribe to our blog?