Twelve years ago, when I graduated college, I was well aware of the Silicon Valley hype machine, but I considered the salesmanship of private tech companies a world away from objective truths about human biology I had been taught in neuroscience classes. At the time, I saw the neuroscientist Henry Markram proclaim in a TED talk that he had figured out a way to simulate an entire human brain on supercomputers within 10 years. This computer-simulated organ would allow scientists to instantly and noninvasively test new treatments for disorders and diseases, moving us from research that depends on animal experimentation and delicate interventions on living people to an “in silico” approach to neuroscience.

My 22-year-old mind didn’t clock this as an overhyped proposal. Instead, it felt exciting and daring, the kind of moment that transforms a distant scientific pipe dream into a suddenly tangible goal and motivates funders and fellow researchers to think bigger. And so I began a 10-year documentary project  following Markram and his Blue Brain Project, with the start of the film coinciding with the beginning of an era of big neuroscience where the humming black boxes produced by Silicon Valley came to be seen as the great new hope for making sense of the black boxes between our ears.

My decade-long journey documenting Markram’s vision has no clear answers except perhaps one: that flashy presentations and sheer ambition are poor indicators of success when it comes to understanding the complex biological mechanisms of brains. Today, as we bear witness to a game of Pong being mind-controlled by a monkey as part of a typically bombastic demonstration by Elon Musk’s start-up Neuralink, there is more of a need than ever to unwind the cycles of hype in order to grapple with what the future of brain technology and neuroscience have in store for humanity.

Hype of these sorts often relies on a selective amnesia for the unfulfilled promises of the past so that enthusiasm around scientific and technological progress can be replenished anew. My own initial excitement for the promises of the Blue Brain Project lingers with me, but has been folded into a messier, more complicated experience. The criticisms and technological limitations I encountered over the decade following Markram’s TED talk have rendered each new neuro hype cycle into a potent reminder of those early enthusiasms and the danger of mishandling them.

My first shift in thinking occurred around three years into my time documenting the Blue Brain Project. Things weren’t going as planned: there were magnificent fly-through visualizations of the first square millimeter of simulated rat brain set to The Blue Danube available in a visitor’s screening room, but a definite lack of progress along the road map towards a human brain. Soon, there was talk of a necessary, larger endeavor known as the Human Brain Project that would require more money but would finally provide the resources necessary to achieve the goal. Proposals were submitted and the project won a billion euros in funding from the European Union, only to then be mired in controversy a year later after an open letter was signed by over 800 neuroscientists disagreeing with the core vision of how to simulate a human brain and objecting to the leadership style of its originator and director, Markram.

As scientific controversy and interpersonal fallout gradually made clear to me that the 10-year plan to recreate a human brain on a computer had perhaps been a pipe dream all along, I started to interview more critics of the endeavor, and began to examine in greater detail what it even meant to say you wanted to do such a thing. Princeton neuroscientist Sebastian Seung had posed a question to me during our interview that over time had come to vex me, pointing to scientific, ethical and moral pitfalls that this work was hurtling toward: “I would ask you this,” Seung had begun, turning the question onto my own time so far filming at the Blue Brain Project and the nature of the piece of simulated mouse brain that project researchers had rendered in dazzling visualizations. “They showed you a simulation of some neural activity inside this. Suppose it looked different; how would you know that that was wrong or right?” Sitting behind the camera, I replied, “Well, I wouldn't know.” Seung reiterated: “Right, how would anybody know what was a wrong activity pattern or right activity pattern?”

The problems arrive when one begins to interrogate what “right” would entail in this situation, for recreating a profoundly noisy biological system inside the circuits of a perfectly programmed machine seems to eventually reach a fundamental platform issue. Biology runs on a motor of unpredictable “mistakes”—known as mutations—that generate the variability seen across individuals in a species and interact with our environment to drive evolutionary change through natural selection. Neurons are also known to be noisy elements, generating action potentials that are far from perfectly predictable events. In computers, on the other hand, structural mistakes, known as “bugs,” are quickly fixed to make way for the perfect code for the task at hand.

Certainly, many elements of a human brain can be modeled, probed and have their generalities extracted, just as we’ve done for the human heart in order to create a device that could function in my body or yours, keeping us alive. But when it comes to building a full simulation of an individual human brain, which would “have a consciousness” and “would speak languages,” as Markram had told me in our first interview, how would the determinate system of software running on computers ever capture the truly unpredictable mistakes seen at every level of biological life, from mutations in our DNA to the activity at a synapse?

Though the scientists I interviewed over the years shared a range of perspectives on the thorny issues of noise and chaos in computer simulations of biology, it wasn’t until I conducted an interview with a more junior neuroscientist at the Blue Brain Project that I heard an answer that cut through the positivist public-relations gloss. Asked how one would ever find the “right” kind of variability in a simulation of a biological organism, she replied: “That's a good question, because the right kind, we can never know what's the right kind of variability.”

If we can never know the right kind of variability, it seems that what we’re really talking about in efforts to simulate biological structures on computers is a digital system that does exactly what its creators want it to do. Taking cues from AI, computational neuroscience is gradually leaving behind biological brains in search of perfect algorithms, which like its cousins in deep learning, may ultimately produce more black boxes that execute tasks but remain internally inscrutable.

In the halls of server farms and neuroscience labs intent on reproducing biological function on digital machines, ethical accountability for the identity of the simulation then becomes a central issue. Far from reaching an objective reconstruction of “the human brain,” simulations of neural activity will ultimately hold a mirror up to the biases of their creators. And when it comes to tech companies that believe understanding a system is not essential to manipulating it or reproducing a version of it for profit, what will that mean when the system in question is the human brain?

One of Markram’s motivations for wanting to accelerate neuroscience towards a full simulation of a human brain was the helplessness he felt when confronted by his son’s autism diagnosis. Indeed, for many researchers in the field, the intentions behind the work can be deeply personal and, at least in theory, widely beneficial—which is why I remain drawn to the capability for improving the human condition enabled by experimentation and scientific ambition. Yet as technology races ahead and a certain strain of technocratic salesmanship continues to command the collective human ear, the line between fiction and reality will continue to be blurred, leading to cycles of hype and disappointment that threaten long-term public confidence in science.

This is an opinion and analysis article.