Around the time that his cult-classic, drug-culture novel Naked Lunch was released, author William S. Burroughs was experimenting with a writing strategy that he called the cut-up technique. Burroughs would chop up random lines of text from a page and rearrange them to form new sentences, with the aim of freeing his mind and the minds of his readers from conventional, linear ways of thinking.

Beat Generation writers such as Burroughs sought to dismantle old belief systems and to encourage alternative ways of looking at the world. They celebrated intellectual exploration, engagement in art and music, unconventionality and deep spiritual questioning. Perhaps no artist captured this spirit more than Jack Kerouac, whose novels have become manifestos for adventure and nonconformity.

The revelations and methods of Burroughs, Kerouac and other Beat writers illuminated an important truth about creativity, which is now backed by scientific research: we need new and unusual experiences to think differently. In fact, cultivating a mind-set that is open and explorative might be the best thing we can do for our creative work. As Kerouac famously wrote, “The best teacher is experience.”

For not only artists but innovators of all stripes, novel experiences provide the crucial tissue of real-world material that can be spun into original work. Openness to experience—the drive for cognitive exploration of one's inner and outer worlds—is the single strongest and most consistent personality trait that predicts creative achievement.

Among the “big five” personality traits (openness to experience, conscientiousness, extraversion, agreeableness and neuroticism), openness to experience is absolutely essential to creativity. Those who are high in openness tend to be imaginative, curious, perceptive, creative, artistic, thoughtful and intellectual. They are driven to explore their own inner worlds of ideas, emotions, sensations, and fantasies and, outwardly, to constantly seek out and attempt to make meaning of new information in their environment.

Seeking truth and beauty

Openness as a personality trait hinges on engagement and exploration, but it is also more complex and multifaceted than that. Openness to experience comes in many forms, from a love of solving complex problems in math, science and technology, to a voracious love of learning, to an inclination to ask the big questions and seek a deeper meaning in life, to exhibiting intense emotional reactions to music and art. Visionary tech entrepreneurs, world travelers, spiritual seekers and original thinkers of all types tend to have highly open personalities.

Research conducted by one of us (Kaufman) for his doctoral dissertation suggests that there are at least three major forms of cognitive engagement making up the core of openness. Intellectual engagement is characterized by a searching for truth, a love of problem solving and a drive to engage with ideas, whereas affective engagement has to do with exploration of the full depths of human emotion and is associated with a preference for using gut feeling, emotions, empathy and compassion to make decisions. Finally, those who are high in aesthetic engagement exhibit a drive toward exploring fantasy and art and tend to experience emotional absorption in beauty. Kaufman found intellectual engagement to be associated with creative achievement in the sciences and affective engagement and aesthetic engagement to be linked with artistic creativity.

Kaufman's research led him and his colleagues to another fascinating observation about “open” personalities. The desire to learn and discover seemed to have significantly more bearing on creative accomplishments than cognitive ability did. He found that people with high levels of cognitive engagement with imagination, emotions and beauty were more likely to make significant artistic creative achievements than people who were only high in IQ or divergent thinking ability (the ability to explore many possible solutions to a problem). Intellectual engagement was sometimes even a better predictor of scientific creative achievement than IQ was.

Looking at creativity across the arts and sciences, Kaufman and his colleagues found that openness to experience was more highly correlated with total creative achievement than other factors that had been traditionally associated with creativity, such as IQ, divergent thinking and other personality traits. Together these findings suggest the drive for exploration, in its many forms, may be the single most important personal factor predicting creative achievement.

Indeed, openness to experience speaks to our desire and motivation to engage with ideas and emotions—to seek truth and beauty, newness and novelty—and the act of exploring often provides the raw material for great artistic and scientific innovations.

The dopamine drive

This engagement starts at the neurological level, with the way the brain reacts to unfamiliar situations and new information. What unites each individual form of openness to experience is an intense desire and motivation to seek new information that is rooted in the individual's neurophysiology and forms the very core of his or her personality.

The drive for exploration hinges on the functioning of dopamine, which is probably the most well known of all the brain's neurotransmitters. As you may know, dopamine plays a strong role in learning and motivation. Unfortunately, there are many misconceptions about dopamine, which is commonly seen as the “sex, drugs and rock 'n' roll” neurotransmitter. Despite many popular descriptions, dopamine is not necessarily associated with pleasure and satisfaction.

Instead dopamine's primary role is to make us want things. We get a huge surge of dopamine coursing through our brain at the possibility of a big payoff, but there is no guarantee that we will actually like or enjoy what we obtain. Psychologist Colin DeYoung of the University of Minnesota has explained that “the release of dopamine … increases motivation to explore and facilitates cognitive and behavioral processes useful in exploration.” DeYoung has called dopamine the “neuromodulator of exploration.”

At the broadest level, dopamine facilitates psychological plasticity, a tendency to explore and engage flexibly with new things, in both behavior and thinking. Plasticity leads us to engage with uncertainty—whether it is pondering a new app to meet a consumer demand or questioning the next step in our own life path—exploring the unknown and finding reward in seeking its positive potential. With plasticity comes enhanced cognitive and behavioral engagement and exploration and, frequently, a commitment to personal growth. Of course, there is no guarantee that our open engagement will yield a positive outcome. For most creative people, however, the engagement itself is enough if it provides fodder for innovation. Indeed, research shows that psychological plasticity is associated with high levels of idea generation, engagement with everyday creative activities and publicly recognized creative achievement.

Plasticity consists of a blend of both extraversion and openness to experience, and dopamine is a source of exploratory motivation. It is easy to see why this might be the case evolutionarily; the drive to explore, the ability to adapt to new environments and the ability to thrive in the face of uncertainty all provide important survival advantages.

Nevertheless, there are crucial differences between extraversion and openness to experience. Extraversion, the personality trait that is most strongly associated with high sensitivity to environmental rewards, manifests in qualities such as talkativeness, sociability, positive emotionality, assertiveness and excitement seeking. Extraverts tend to be more likely to explore and pursue more primal “appetitive” rewards such as chocolate, social attention, social status, sexual partners or drugs like cocaine. But dopamine, which is indeed important to extraversion, also has projections in the brain that are strongly linked to numerous other aspects of cognition. Individuals who are particularly open to experience get energized not merely through the possibility of appetitive rewards but through the possibility of discovering new information. It is the thrill of the knowledge chase that most excites them.

This motivation for cognitive exploration engages and energizes us while influencing our drive for creative expression. We see the quality play out again and again in different realms of the arts and sciences. After all, it is difficult to imagine any great creative achievement that wasn't sparked by the drive to explore some aspect of the human experience.

“Leaky” filters and messy minds

It is hardly a stretch to say that dopamine is the mother of invention. In addition to facilitating cognitive exploration, the neurotransmitter is associated with a number of processes that facilitate creativity, including dreaming. We know that both daydreaming and dreaming at night are invaluable tools to help us access deeper realms of creativity. People who are high in openness to experience report dreaming more often and having more vivid dreams than those who are less open, possibly because of their higher dopamine production.

One intriguing possibility is that dopamine surges into the right hemisphere of the brain support both openness to experience and dreaming. Dreaming inspires creative insights, and those who have more creative insights show more activation in the brain's right hemisphere. Among people who are high in openness, the brain's dopamine systems are working day and night to inspire creative insights.

Another important cognitive process associated with creativity is latent inhibition—a mechanism in the brain that “filters out” objects in our environment that we have seen many times before and therefore consider irrelevant to our current goals and needs. In 2003 psychologist Shelley Carson of Harvard University and her colleagues discovered that the university's eminent creative achievers were seven times more likely to have a reduced latent inhibition—meaning that they had a harder time filtering out seemingly irrelevant information and continued to notice familiar things.

But here's the thing: the information did turn out to be relevant! In related research, Kaufman found that those with a reduced latent inhibition had a greater faith in their intuitions, and their intuitions were, in fact, correct. Reduced latent inhibition speaks directly to the concept of a “messy mind,” often associated with creativity, because it reflects the tendency to tune in to greater amounts of information from our surroundings rather than automatically filtering and compartmentalizing.

The downside of this quality is that it might make creative people more prone to distraction than others. Researcher Darya Zabelina of Northwestern University found that people with a “leaky” sensory filter—meaning that their brain does not efficiently filter out irrelevant information from the environment—tend to be more creative than those with stronger sensory gating. Zabelina also observed that highly creative people are more sensitive tonoises in their environment—a clock ticking, a conversation in the distance—than less creative people. “Sensory information is leaking in,” Zabelina has explained. “The brain is processing more information than it is in a typical person.”

This brain quirk was a known characteristic of many eminent creators, including Charles Darwin, Franz Kafka and Marcel Proust, who each expressed a hypersensitivity to sound. Proust kept his blinds drawn and lined his bedroom with cork to filter out unwanted light and noise and wore earplugs while he wrote, whereas Kafka said that he needed the solitude not of a hermit but of a “dead man” to write.

And although it may sometimes be a hindrance to creative work, this distractibility also seems to be distinctly beneficial to creative thinking. Sensory hypersensitivity most likely contributes to creativity by widening the brain's scope of attention and allowing individuals to take note of more subtleties in their environment. Taking in a greater volume of information increases your chances of making new and unusual connections between distantly related pieces of information.

Genius or madness?

These findings have deep implications for the long-standing mental illness–creativity debate. Research has linked dopamine production with not only reduced latent inhibition and creativity but also mental illness. To be clear: mental illness is neither necessary nor sufficient for creativity. Nevertheless, there does seem to be a nuanced link between the two because having an extremely open mind makes flights of fancy more likely. In support of this idea, there appear to be variations in the expression of dopamine receptors in certain areas of the brain among both creative individuals and those with psychotic symptoms.

In 2010 neuroscientist Fredrik Ullén of the Karolinska Institute in Stockholm and his colleagues found that dopamine systems in healthy, highly creative adults are similar in certain ways to those found in the brains of people with schizophrenia. In both cases, they observed a lower density of dopamine D2 receptors in the thalamus—a brain area associated with sensory perception and motor function that also plays an important role in creative thought, suggesting one possible link between creativity and psychopathology.

Having fewer D2 receptors in the thalamus probably means that the brain is filtering less incoming stimuli, leading to a higher flow of information being transmitted from the thalamus to other parts of the brain. In individuals who are not also suffering from the damaging symptoms of mental illness, this flow can lead to an increase in creative thinking and may very well underlie several cognitive processes that determine creative achievement. “Thinking outside the box might be facilitated by having a somewhat less intact box,” Ullén and his colleagues said in the study.

An excess of dopamine may cause an influx of emotions, sensations and fantasy, so much so that it causes substantial disruption to functions also important for creativity, such as working memory, critical thinking and reflection. Too little dopamine, however, and there may be less motivation and inspiration to create.

Dopamine aside, research has suggested similarities in brain activations between highly creative thinkers and people who are prone to psychosis. In 2014 neuropsychologist Andreas Fink of the University of Graz in Austria and his colleagues found that people scoring high in schizotypy—a personality continuum ranging from normal levels of openness to experience and imagination to extreme manifestations of magical thinking, apophenia (perceiving patterns that do not really exist) and psychosis—showed similar difficulty deactivating or suppressing activity in the precuneus region of the brain, an area associated with self-consciousness, a sense of self and the retrieval of deeply personal memories.

In reality, all of us lie somewhere on the schizotypy spectrum, and the existence of schizotypal characteristics does not necessarily indicate schizophrenia. Psychologically healthy biological relatives of people with full-blown schizophrenia tend to have unusually creative jobs and hobbies, compared with the general population, according to a 2001 study by Saybrook University psychologist Ruth Richards and her colleagues. Similarly, Simon Kyaga and his co-workers at the Karolinska Institute reported in 2013 that among more than 1.2 million Swedes, the siblings of patients with autism and the first-degree relatives of patients with schizophrenia were significantly overrepresented in scientific and artistic occupations.

It is possible that relatives of people with mental illness inherit creativity-boosting traits while avoiding the aspects of the mental illness that are more debilitating. In support of this observation, researchers have found that schizotypal characteristics—particularly the “positive” ones, such as unusual perceptual experiences and impulsive nonconformity—are related to creative personal qualities—individualistic, insightful, eclectic, reflective, resourceful and unconventional—as well as everyday creative achievements.

Go with the flow

Schizotypy is related to so-called flow states of consciousness and absorption. Flow is the mental state of being completely present and fully absorbed in a task. When in a flow state, the creator and his or her world become one—outside distractions recede from consciousness, and the mind is fully open and attuned to the act of creating. This happens, for instance, when a playwright sits up all night crafting a new scene without realizing that the sun is rising or when a filmmaker spends hours in front of a computer editing a rough cut.

Flow is essential to the artist's experience. In a study of 100 artists in music, visual arts, theater and literature, researchers Barnaby Nelson and David Rawlings, both at the University of Melbourne in Australia, found that those who said they experienced more flow during the creative process were also higher in schizotypy and openness to experience. Nelson and Rawlings linked their findings to latent inhibition, arguing that a leaky sensory filter is a common thread running through schizotypy and openness to experience—and, perhaps surprisingly, flow and absorption. The failure to precategorize incoming information as irrelevant, which is experienced by individuals with reduced latent inhibition, can, the researchers wrote, result in “immediate experience not being as shaped or determined by preceding events.”

In other words, an exceptionally large amount of information, far more than for those with higher levels of latent inhibition, enters their field of awareness and is explored by their mind. As Nelson and Rawlings explained, “it is precisely this newness of appreciation and the associated sense of exploration and discovery, that stimulates the deep immersion in the creative process, which itself may trigger a shift in quality of experience, generally in terms of an intensification or heightening of experience.”

So what determines whether schizotypy goes the way of intense absorption and creative achievement or tips over into mental illness? This is where a number of other factors come into play. If mental illness is defined as extreme difficulty functioning effectively in the real world, then the complete inability to distinguish imagination from reality is surely going to increase the likelihood of mental illness. If, however, one has an overactive imagination but also has the ability to distinguish reality from imagination and can harness these capacities to flourish in daily life (with the help of things such as motivation, post-traumatic growth, resilience and a supportive environment), then that is far from mental illness.

Mental processes on the schizotypy spectrum may interact with protective mental qualities such as greater intellectual curiosity, improved working memory and cognitive flexibility. Indeed, in 2011 neuroscientist Hikaru Takeuchi of Tohoku University in Japan and his colleagues studied people with no history of neurological or psychiatric illness and found that the most creative thinkers among them were those who were able to simultaneously engage their executive attention in an effortful memory task and keep the imagination network in the brain active.

You never know—some of the most seemingly irrelevant or “crazy” ideas at one point may be just the ingredients for a brilliant insight or connection in a different context. It bears repeating: creativity is all about making new connections.

Adapted from Wired to Create: Unraveling the Mysteries of the Creative Mind, by Scott Barry Kaufman and Carolyn Gregoire. Available from Perigee, an imprint of Penguin Publishing Group, a division of Penguin Random House LLC. Copyright © 2015 by Scott Barry Kaufman and Carolyn Gregoire.