Scams, Shams, and (Body) Slams

While preparing for a presentation (and a little book stemming from it), and doing some color studies for sketches to accompany them, the news has continued to be very distracting as it is presumably for everyone. In the last 24 hours alone, from a journalist sent crashing to the floor allegedly “body slammed” by a person aspiring to elected office (or is he already in office?)–to confirmation from the CBO (Congressional Budget Office) that the health of our nation is going to take a huge body blow if the latest health care plan is passed–to disconcerting news about NATO (also “body slammed?”), it is tough to keep one’s eyes on the task at hand.  But perhaps the combination of these colliding impressions is good for something after all…

In sorting through older material, I came across the famous “boiling frog”–a metaphor, of course, for not noticing when there are gradual changes in your surroundings, until it is too late.  According to the metaphor, a frog in a pot of slowly heating water will not react quickly enough to save himself and will eventually die.  (This is literally not true; the frog will jump out if he can, apparently.  I myself have not tested it, but I respect scientists and experts and they have).

Image: Watercolor, gouache, and ink by Black Elephant Blog author (2014)

This is a week too in which we have heard the word “suborn” used in open testimony. It’s a useful word.  It seems related to another one rarely heard:  “inure”, which the dictionary defines as “becoming accustomed to something, especially something unpleasant.”  (Perhaps this is a good time to recommend a currently best-selling new little book, available on Amazon for less than $6:  On Tyranny:  Twenty Lessons From the Twentieth Century,” by Timothy Snyder, a professor of history at Yale.)

With so much coming at us almost hourly, it sometimes seems like the fate of the world is being decided right now.

Illustration: Color study, Watercolor, acrylic and gouache, “Where Do We Go From Here?” by Black Elephant Blog author (2017)

People are tired of being distracted by it but the most conscientious know that too much is at stake to turn away. Much as we might like to, we can’t tune out what is going on because it’s unfortunately true– the fate of the world is being decided right now.  And if we tune out, we will surely not be as fortunate as the sensitive frog who manages to escape the dangers of his warming world.

So, we must not become inured to the bruising pace of the news cycle.  It seems to me essential to find ways collectively to both deal with every incoming distraction and yet look beyond it to make sense in time of where we are going and might wish to go instead.

Momentous times indeed, but I have faith we will prove to be at least as smart as  frogs.  So back to the drawing board…

Monkeys Typing Shakespeare

Illustration: Watercolor, gouache, and ink by Black Elephant Blog author.

Around the world overnight, we rang in the New Year with expressions of wishes for good fortune for all in 2015!  While we are considering here the roots of surprise (even, for fun, a “zoology” of surprise), the start of a New Year is an auspicious (another good luck phrase) occasion on which to consider the chances (there we go again) of things going extraordinarily well or badly.  For this, I’ve been turning to a new book by an eminent British mathematician and statistician who, it seems to me, has done the reading public a great service by translating his insights into language we non-mathematicians can (usually) understand!  (I am composing a blog post to record what I am learning, and not to review the book.  I also have received emails from friends who are looking forward to learning more about this book and reading it themselves.)  Future posts will come back to the ideas presented in this book.

Extraordinarily improbable events occur every day, according to Dr. David Hand, author of The Improbability Principle: Why Coincidences, Miracles, and Rare Events Happen Every Day (2014). (You can watch Dr. Hand explaining the Improbability Principle at a 2014 meeting of the Royal Statistical Society in London at this YouTube video here.)  It is clear from the video that the statisticians see the present moment in history with the deluge of data (including “big” data) as opportune for members of their profession to intersect with the formulation of public policy.  And based on my reading of Hand’s new book so far, I’d have to agree!

Hand traces the history of the study of probability, noting early on in the book the work of Emile Borel, an eminent French mathematician (1871-1956), who held that “Events with a sufficiently small probability never occur.”  Borel cited, according to Hand, the classic example of monkeys who, randomly hitting the keys of a typewriter, happen by chance to produce the complete works of Shakespeare.  Borel explained:

“Such is the sort of event which, though its impossibility may not be rationally demonstrable, is, however, so unlikely that no sensible person will hesitate to declare it actually impossible.  If someone affirmed having observed such an event we would be sure that he is deceiving us or has himself been the victim of a fraud.”

As Hand observes, on first glance it seems like “Borel’s law” contradicts “the improbability principle” which is the subject of his book. The Improbability Principle asserts that “extremely improbable events are commonplace.”   But Borel is referring to “very small probabilities” on human scales, says Hand, and indeed Borel clarifies his “single law of chance” by noting that “at least, we must act, in all circumstances as if they were impossible.”

By contrast, the Improbability Principle explains why highly unlikely events keep on happening.  Hand says, “that is, not only are they not impossible, but we see such events again and again.”  Can both these assertions be right, he asks?  Hand maintains that we can resolve this apparent contradiction by considering different strands of the improbability principle, including the “law of truly large numbers,” the “law of near enough,” the “law of selections”, and others.

When one goes through this process and understands the strands, Hand writes, the principle “tells us that the universe is in fact constructed so that these coincidences are unavoidable:  the extraordinarily unlikely must happen; events of vanishingly small probability will occur.”

Why read such a book at all?  I’ll hazard (another term from the world of risk!) a guess:

In the view of this blogger, in our interdependent, highly-interconnected world, where dense networks create a world of connections that change the meaning of “human scale” relative at least to what Borel understood it to be nearly a century ago, understanding how “rare” events, coincidences, and extraordinarily unlikely events occur has become vital for human (and other forms of life’s) security.   But, grasping that uncertainty is inherent in reality and that, indeed, even in the present, we can only have an approximate understanding of reality, does not come naturally, for some reason.

Spoiler Alert:  The rest of the post will discuss the different types of probability as Hand presents them in his book.  Future posts will delve into other aspects presented in this work. ###

And there is a reason for this mismatch of expectations, explains Hand.  Rarely addressed in our usual day-to-day settings but deftly discussed in this book is the gradual move in the last century or so –in science at least–beyond reliance upon  “deterministic” principles long said to explain the behavior of natural systems.  These principles, it has been assumed until recently, adequately explained the underlying causes and effects of events and outcomes, at least since the natural laws of physics began to be investigated in the 17th century.  The early proponents of the concept of scientifically testing ideas were onto something revolutionary for the times!  But they were limited in their understanding by what the tools and techniques of the day enabled them to observe:  this influenced the types of questions they asked, of course, and led to overconfidence about mankind’s abilities to master nature’s mysteries within the bounds of existing knowledge.

The “Baconian Revolution” first introduced the idea of the scientific method, writes Hand.  This method held that the way to understand the natural world is to collect data, conduct experiments, take observations, and use these as test beds through which to evaluate proposed explanations for what’s going on.  Before that, stories and superstitions held sway.  “But explanations that have not been or cannot be tested have no real force…,”  according to Hand.  “They serve the purpose of reassuring or placating those who are unwilling or unable to make the effort to dig deeper but they don’t lead to understanding.”

The first scientists (“natural philosophers” as they were called then) sought to devise laws that describe how nature works.  Hand notes that these laws are “shorthand summaries” encapsulating “what observations shows about how the universe behaves.”  They are “abstractions,” he notes.  An example is Newton’s Second Law of Motion, which holds that the “acceleration of a body is proportion to the force acting on it.”  The power of such laws is behind humanity’s progress in science and technology, Hand observes.

For a long time and even as recently as the 1930s, scientists and philosophers such as Karl Popper, held that the “rule that extreme improbabilities have to be neglected…agrees with the demand for scientific inquiry.”  Those tiny chances of extraordinarily rare events had to be swept under the rug to allow progress, or so it was (and still is) thought.  In addition, the idea of things happening for which we have no explanation is an intensely uncomfortable one, Hand writes, as humans have an innate need to know why things happen and “to establish the causal connections, and to understand the rules that lie behind what we observe.”  This is a basic human need related to safety and security:  if there “are no causes…illnesses, accidents, and failures couldn’t be avoided.  We’d live in a constant state of fear, waiting the unpredictable disaster just around the corner.”

Over the centuries, it was impossible to miss the inexplicable coincidences and other extraordinarily unlikely events, creating fertile conditions for prophets and fortune tellers, writes Hand–people who tap into the notion that there is some “mysterious force or being behind what happens, often acting with malicious intent.”  These notions have led to different explanations for otherwise unexplained events, including superstitions, prophecies, gods, miracles, and parapsychological explanations, he writes.

Yet, the notion that there is any real causal relationship between, for instance, sighting black cats and falling down, stems from misperceiving patterns.  Hand explains that the goal of science is to distinguish between those patterns that do represent a “real underlying cause-and-effect relationship” and those that don’t.  “Patterns we spot but that are mere accidents, without any underlying cause, have often formed the basis of superstitions.  (Animals also demonstrate this development of “superstitions,” he notes.)  But:

“Even if one event follows another surprisingly often, it doesn’t necessarily mean that the first causes the second.  Statisticians have a sound-bite for this:  correlation does not imply causation…Although the aim of a prophecy is to remove uncertainty about the future, uncertainty in the form of randomness is frequently the mechanism used to generate prophecies.”

The deterministic laws that evolved from the 17th to the 20th centuries were “mathematical equations…that told us how natural objects would behave,” writes Hand.  “There was nothing in the universe that was uncertain or unpredictable, at least in principle, according to science.”  And the immense technological progress of mankind “built on those ideas showed that they were largely correct.”   Thus came into being the ubiquitous view of nature as “the clockwork universe”–a universe ticking along a well-defined path, Hand writes. Ignorance could be eradicated by science.

Later in the 20th century, however, science began to expose gaps that it could not explain.  A huge shift in perception began slowly to take hold at least on the margins of science:

“It seemed as if the universe was not deterministic after all, but that randomness and chance lay at its very foundations.”  Randomness and chance are entirely probable in this universe, Hand explains, and can be understood through the improbability principle which is formed upon the basic laws of probability.

Types of Probability There are different kinds, and definitions, of probability, according to Hand.  Informal definitions even reveal the multi-facetness of probability: both,  “the extent to which an event is likely to happen” and “the strength of belief than an event is likely to happen.”  And Hand tells us that both can be represented by the same mathematics:  probabilities are numbers lying in the range from 0 to 1 with 0 meaning impossible and 1 meaning certain.  There are many other definitions, but none captures “probability” in its entirety.  This is not really a problem, says, Hand, because it is very natural to need “multiple views of an object to understand it properly.”

The three most widely used interpretations of probability are the frequentist, subjective and classical interpretations:

The frequentist interpretation of probability is based on the tendency of physical systems to produce roughly constant relative frequencies when situations are repeated.  For example:  the tendency for a coin to come up heads about half the time it’s tossed, or the 4 (or any other) face to show on a die about one-sixth of the time.  As we learn from reading Hand, there is a lot to think about in the word “roughly” above!  Complete accuracy is impossible event when measuring, as frequentist probability does, properties of the “external world.”

Subjective probability is very different. Instead of representing an aspect of the external world, subjective probability is the confidence an individual has that an event will occur, explains Hand.  This relates to your beliefs, whether about the probability of a coin turning up heads in a coin flip or your beliefs about the person tossing the coin (who might have rigged the process).

“Instead of being a property of the external world, the subjective view has it that probability is an internal property of your mind. Each person will have their own subjective probability for each event.”  Hand notes that another eminent mathematician therefore claimed that probability did not exist because it is a “property of how we think about the world.” Nonetheless, Hand notes that various methods exist for measuring subjective probability, including asking people to bet on an outcome–knowing that the results will depend on what they think.

Only recently have we humans come to understand the significance of these fundamental different notions of probability, Hand says, with steps newly taken to distinguish between  epistemological probability and “aleatory” probability. (For more on this, it will be necessary to read the book!)

The classical interpretation of probability is based on options of symmetry, Hand writes, giving the example of how natural it is to think of probability as distributed equally across the six faces of a die. “This interpretation is very convenient for games of chance, based on symmetrical randomization tools such as dice and coins,” he writes.  But life is not like a die, he says:  “it’s less clear how we might apply classical probability to situations in normal life which lack such obvious symmetries.”

There are other interpretations of probability, which Hand goes on to introduce.  What’s key, and still to be addressed in this blog  in future posts on the roots of surprise, is to understand how probability is calculated in the case of interdependent events, such as make up the natural and manmade systems of our world.  What is the significance of these relatively recent discoveries of the inevitability of improbability?

Boiling Frog Syndrome

Image: Watercolor, gouache, and ink by Black Elephant Blog author

We come now to the metaphor of the “boiling frog syndrome” which has many metaphorical cousins, such as an “ostrich with its head in the sand.”  This metaphor refers to the notion (apparently untrue according to biologists who have tested it) that a frog will not jump out of gradually heating up water–and, sadly, dies.  (In reality, the experts say, if the frog can jump out it will.)  This expression is used to warn of our built-in tendencies to ignore or dismiss changes that are so gradual (or long-term, or complex, or multi-faceted) as to be nearly invisible or even incomprehensible.  As specialization drives experts, analysts, and planners into focusing on ever-narrower slices of knowledge, the thought goes, they lose an ability to see the whole.

Whatever our daily preoccupation, this fragmentation of our attention affects us all, in sometimes unnoticed ways, leading me to wonder if we might all have a bit of boiling frog syndrome.  If so, what do we do to counter it?  Who are those who manage to overcome this syndrome, how do they do it, and are we able to hear them?

As someone who has been spending a lot of time in the gym lately (to get ready for those New Year’s resolutions!), it is impossible to miss the many HDTV screens propped up overhead.  They beam down at all, apparently selected to represent a spectrum of political and entertainment tastes.  Seemingly, there ought to be something for everyone, as no less than twelve different channels are simultaneously broadcasting literally in our faces.

Those who want to can tune in to the channel of their choice, wearing headphones connected to the fitness machine they’re on to hear their chosen flavor of the news. While people pump their legs on this or that machine, with water bottles propped up on the machines, towels cast to the floor, and iPods attached to their spandex belts, horrific images of violence in cities and countries around the world flash up in front of us–simultaneously.  When I leave the gym, I feel refreshed usually from exercising but usually have a vague sense of unease from having just been enveloped by oversized TV screens purporting to tell me something.

Bursts of “news” with next-to-no context tell us little, despite the inordinate amount of time devoted by each channel to endlessly rehashing details of the day’s top headlines.  Although these stories are immensely important, it is difficult to hear about them in dueling soundbites, so I generally try to ignore them altogether while at the gym.

The interconnections and deeper underlying causes or trends are rarely examined, of course, in the “news” because they are not news.  Such phenomena develop slowly over time, like gradually heating water.  Occasionally a tipping point of some sort or other is reached:  a catastrophe occurs that focuses our attention…but only for the time it takes before another sudden eruption captures the news.  Beneath the surface so much else is going on…but it is difficult to make sense of it, and thus it doesn’t ever make the news.

Some recent works (such as here  and here)  suggest that we are applying the wrong models (and thus harbor ill-founded expectations about risks) to trying to understand the complex (increasingly interconnected and interdependent) systems that make up global society today.  Mankind’s very successes in developing more efficient supply chains, transportation systems, and ICT are creating new vulnerabilities, or “systemic risks,” which we generally are poorly equipped to detect, let alone understand.  These risks represent “highways for failure propagation” which can ultimately result in “man-made disasters,” say the authors.  Highly interconnected systems, such as the financial system, or food and energy markets, are complex systems that are difficult to predict and control.  If there is a mismatch of expectations, it might be difficult-to-impossible to see these risks in time–leading in turn to the “surprises” or “abrupt changes” that are the focus of this blog.  Would such risks be like the “boiling frog”  or a “black elephant?”  It seems that this may be a fitting point at which to consider what the originator of the “black elephant” concept intended…coming up in a future post!

Surprising Creatures

It wasn’t until I read Nassim Nicholas Taleb’s book, The Black Swan, that my naturalist sensibilities became attuned to proliferating species of surprise. They are everywhere! As the years tick by, I have come to think that understanding the origins of surprise are fundamental to solving the greatest challenges of our times–including those so-called “black elephants!” (This is the main reason I am devoting a whole blog to the subject.)

How we learn, create, and share knowledge…how innovations occur…what role collaboration and teamwork play in fomenting breakthrough thinking…how the subconscious mind works with our conscious selves…how sparks of serendipity ignite new possibilities… All such themes and more belong in an examination of the “black elephants” of our times. But why am I talking about black elephants when I started with “black swans?” Clearly our taxonomy for surprising creatures needs attention. So let’s get started!

Before all these metaphors entered our lexicon, most of us were familiar with the thought cloud images from cartoons, showing a bright lightbulb over someone’s head! The lightbulb signified a new idea!!! Archimedes in the bathtub shouting “Eureka”…that’s another visual image of surprise.

Image: Google Images

But in recent years, particularly since the Financial Crisis of 2008, we’ve seen a stampede of elephants and hippos, flights of swans and hummingbirds, and pots of boiling frogs cross our fields of vision. What in the world is going on? Are we more prone to be surprised these days? Surprising creatures are helping us to make sense of these developments: let’s begin with the “black swan.”

The Black Swan…

Unless you live in Australia, black swans are rare and, according to Taleb, in most of the world–before the discovery of Australia–the absence of black swans led to an unexamined assumption that all swans are white. Such unexamined assumptions are typical to all of us: having a cognitive framework, or mental map of how the world works, enables us to function. The downside (one of many) to how we go about making sense of things normally is that our knowledge is limited by what we have observed or experienced. The size of our ‘sample set’–or real-world experiences–influences our concepts of reality and possibilities for the future.

Taleb tells us that his metaphor for a “black swan” event comprises three attributes:

• It is an “outlier” in the sense that it “lies outside the realm of regular expectations.” Nothing in our experience has prepared us for this possibility.
• It carries an extreme impact.
• It was “predictable” but only in hindsight! (Taleb says, our human nature persuades us, after experiencing an outlying event with an extreme impact, that it was predictable.)

In Taleb’s view, the way our human brain is wired makes what we don’t know more important than what we do know:

“Black Swans can be caused and exacerbated by their being unexpected,” he writes.

Our concept of what is “normal” tends to rule out outliers and uncertainty.  But, more and more, what we’re learning, from the “law of large numbers” and other principles of improbability, is that what seems normal often is not!  So, how do we manage in a world of surprising creatures like the black swan? It turns out that rare events are behind most breakthroughs in human history…so understanding how we get locked into assumptions, and when we need to unlock our assumptions, seems critically important not only to business success but perhaps survival in all its meanings.

Coming up: I’ll look at what Steven Johnson, the author of so many great books on where good ideas come from, says about the “hummingbird effect” in his new book, How We Got To Now. Why does it matter to know how we got to now?