What are insights and how do they occur? Why do we need them and what keeps us from encountering more of them in our daily lives? After years of investigation, cognitive psychologist Gary Klein offers an explanation, drawing anecdotes from science, business, medicine, and law enforcement to show us why we miss things.

There was a recent story of two cops casually doing their rounds in New York City. While stopped at a light behind a brand new BMW, the younger cop observed the driver flick cigarette ash into his passenger seat, rather than out the window or into an ashtray. The police officer reasoned that the owner of a brand new sports car would never dump cigarette ash on his fresh upholstery. Neither would a friend who was borrowing the owner’s car. They ran the plate to see if the car had been stolen, and, as it turned out, it had. They pulled the driver over and arrested him. A flash of insight had paid off.

This story shows the remarkable display of creative rationality. It’s a refreshing counterweight to the mountains of writing about our species’ irrationality and biases. Among other things, it draws attention to something that’s missing from discussions about human decision-making and improving performance.

The simplest formula to improving performance is minimizing error and maximizing insights. The father of the positive psychology movement, Martin Seligman, described how to beat depression, but realized that there’s so much more to life than being “not depressed.” Ironically, positive psychology was still on the defensive, and had failed to give people anything to aim for other than emotional damage control. Seligman’s most recent book, Flourish, was an attempt to balance the equation.

It’s not just Seligman and positive psychology, but the majority of people and organizations who live defensively, playing not to lose instead of playing to win. But what do we lose when we try not to lose? If performance is about reducing errors and increasing insight, then we are missing half the equation. Fewer mistakes can only bring us to zero, but who lays their head down on the pillow and thinks, “What a great day! I didn’t mess up that much”?  

It makes sense that people and organizations would want to minimize mistakes. Mistakes are embarrassing and can be expensive to fix. But no one’s talking about the other side of performance improvement. The hunt for insights is an attempt to rectify that problem in decision-making talks. When we learn to find insights, we deepen our understanding of life and how to operate well within it.

What’s the anatomy of an insight? It usually begins with a moment of being pulled up short, a surge of ecstasy as a unique combination of ideas that rapidly coalesce, and a growing feeling of confidence. Other people might be privy to the same information, but, as far as you know, only you have put the pieces together in this particular combination. When all these things come together, you’ve probably stumbled upon an insight.

There are a number of potential pathways to insights. The main categories are creative desperation, connection, contradiction, coincidences, and curiosities.

Creative desperation leads us to insights when we’re trapped by assumptions. The famous Nine Dot Puzzle would be an example. The challenge is to connect all nine dots that are laid out on a 3x3 formation, but you have to do so in just four strokes and without lifting the pen. Most people don’t solve it because they are trapped by assumptions, most saliently, that they are forced to work within the confines of the nine dots. It’s only when a person extends lines beyond the 3x3 grid of dots that all the dots can be connected. Not once is it mentioned that the person must remain confined to the grid, and yet most people subconsciously take on that constraint. When we discover those assumptions through experience and experimentation, we achieve breakthrough, and it’s exciting to explore what’s on the other side of it.

Connection refers to the process of combining known information with new information, or pieces of known information that combine in an unexpected way. Michael Gottlieb was the first to announce the discovery of AIDS. In 1981, while doing a fellowship with Stanford University, he noticed a disconcerting pattern involving cases of young gay men with diseases that indicated dysfunctional immune systems. His insight was not a sudden development, but one built case by case as he encountered the perplexing collision of certain demographics and sets of symptoms. He wrote and presented about 50 papers during the 1980s about the condition and treated numerous high profile cases like Rock Hudson.

Insights through contradiction occur when observations come together to jarring, discordant effect. When a New York City cop deduces that the new sports car in front of him is a stolen vehicle because no owner would flick his cigarette ash into the plush leather of his passenger seat, he has experienced an insight through observing contradictory elements.

Another example of insight by contradiction would be Harry Markopolos, a financier and investigator who was on to Bernie Madoff as early as 1999 and hounded Madoff until Madoff turned himself in in 2008. From publicly available records, Markopolos immediately saw that there was no way that the poorly designed split-strike strategy could yield such incredible profits year after year. Something was wrong. Unfortunately, the Securities and Exchange Commission was not structured to catch scandals of such a grand scale, and Markopolos’ reports were ignored for almost a decade.

What interferes with insights, or even keeps us from seeing the obvious? Among the chief insight blockers are flawed beliefs, inadequate experience, passivity, and concrete reasoning.

At one level, flawed beliefs are inevitable, but they are especially persistent when they form the foundation for theories or best practices. Thomas Kuhn wrote about how scientists who are used to doing “normal science” in a particular paradigm are the most resistant to anomalies or new information that could challenge the status quo. Scientists will challenge the new methodology, the relevance of the data, find anything that could be seen as problematic, or, they make a tiny alteration to their theory without upsetting its foundations.

A basic lack of experience blocks us from encountering insights, too. This is more than just a question of knowledge. It’s a question of attunement. When everything looks novel, nothing is actually novel. It’s people familiar enough with a field who are most likely to spot irregularities.

Passivity also obstructs insights. The author gained insight into how disposition makes people receptive or unreceptive to insights by reviewing scores of case studies of light bulb moments. People who approached situations with a sense of openness and curiosity were more likely to develop insights. Those more concerned with “getting the job done” almost always failed to see what was sometimes right in front of them. An active attitude creates the persistence needed to see insights through.

A concrete reasoning style is another detriment to insight creation. Unfortunately, it’s also Western culture’s bread and butter. It gets grafted into our personality from a very young age. It makes people very good at tasks, but bad at seeing anything other than the task. Insights, by nature, point to ideas beyond the pale.

There are useful ways to hunt for insights. But in the interest of eliminating weak assumptions, it’s also valuable to point out insight hunting methods that won’t serve you. If you want to find insights:

-Don’t schedule a specific date and time for you or your group to sit down to capture an insight.

-Don’t put time limits on insights or link their exploration with evaluation. The pressure of appraisal hamstrings the insight hunter.

-Don’t expect insights to come from meaningless tasks about which people don’t care. Insights flow from areas of interest and themes that matter to you or an organization.

-Don’t expect to come up with insights (or expect others to) in unfamiliar fields. In a substantial majority of case studies, people use experience to identify contradictions and make connections.

-Don’t insist on thinking-out-loud protocol for meetings. Being forced to verbalize thoughts interferes with the process.

-Don’t rush things. Some insights develop over a long period of time.

A huge limitation of the insight research that most universities have conducted is that they often break many of these rules. Quality insights tend not to come under such contrived circumstances as a sterile laboratory. From a scientific standpoint, it makes sense to control variables, but it is precisely the controlling of variables that detracts from the organic nature of insight production. Insight researchers have become trapped in puzzles of their own making, and rather ironically, have difficulty gaining insight into insights.

Outside the laboratory, amidst the ebb and flow of everyday life, it is typically the actively curious, in a state of relaxation (or at least composure) that gain insights. Pressure and constraints usually shut down the hunt.

Insights are an important aspect of a meaningful life. In addition to adding zest and interest to your day, they also create opportunity for challenge, which the human brain craves. And when you explore a new insight, you might just discover something that makes life better for you or for others.

So insights are important, but how do we get more of them?

The Triple Path Model helps us think of ways to open the gate to insights. The three triggers that can lead us to insights are contradiction, connection, and creative desperation. Each pathway is different, but the end result of a good insight is always a change in mindset, worldview, feelings, or behavior--whether for you or for others.

The contradiction pathway involves turning feelings of bewilderment to curiosity. When people discover a contradiction, the common reaction is to get frustrated. When you discover one, you can help resolve the contradiction or create an unexpected opportunity to make the most of it.

You can use the Tilt! reflex to become curious about a situation and set a new trajectory for yourself or your projects. The Tilt! reflex means turning frustration over a contradiction into curiosity. It creates a learning posture. Dennis Klein became a success in Los Angeles show business, creating The Larry Sanders Show that aired on HBO in the 90s. When he was in college, he had a watershed moment when he moved to L.A. and ABC promised him a job as a page, a position that could lead to bigger opportunities in show business by networking with high-profile figures in the industry. When he didn’t hear back from ABC for a few weeks, he stewed in his annoyance before crossing them off and moving on. But then ABC did call and Dennis led with a barrage of complaints and accusations. The manager cut the onslaught short by simply asking, “Do you want the job or not?”

This was a Tilt! moment for Dennis. Growing up in the Bronx with a family entrenched in an ongoing blame game, Dennis realized in that moment that no one outside his family really cared whose fault it was. It didn’t matter whether he could justify his actions or put the blame on someone else. What mattered to the manager was whether he could do his job well.

The manager’s curt response revealed a contradiction. It showed a weak anchoring point in need of strengthening.

Then there are connection triggers, opportunities to build an entirely new anchor that holds several threads together in an unexpected way. This is far more likely when ideas are swirling around. These unexpected links can take you to new places, expanding your horizons in surprising ways. Martin Chalfie could not have known what would come out of a seminar on jellyfish. Expectations might have been pretty low. But when he heard that the gene for a fluorescent protein had been isolated, he realized that he might be able to use it to track substances and processes across transparent bodies of worms. The flash of illumination led to remarkable discoveries in science and new applications in the bioengineering industry.

The pathway to critical desperation begins with an entirely different mental posture than that of the contradiction and connection triggers. Creative desperation comes when there is some kind of mental brick wall that we find ourselves butting up against. There’s some erroneous assumption that bars us from breakthrough.

In an attempt to preempt this dilemma, some employers will have their employees list all their assumptions to figure out which ones are false or misleading. This is not only tiring and tedious but useless. It takes expertise to judge a situation and discern which presuppositions are weak. This means without experience, identifying those presuppositions is unlikely. Napoleon did not become a brilliant tactician by holding a meeting with his officers in which they systematically listed their driving assumptions. Experience is the divining rod we need to unearth weak assumptions.