New Issue: Orbital Catastrophe Ahead? Read Now

To Qualify as ‘Scientific,’ Evidence Has to Be Reproducible

We can never be sure if the interstellar object ‘Oumuamua, for example, was artificial—but we could be ready to answer that question for such a visitor in the future

Alien spaceships? We'll never know.

In various conversations with reporters leading to the publication of my new book, Extraterrestrial, I encountered difficulties getting across one essential aspect of the scientific process. Science relies on reproducibility of results. In order to believe a phenomenon, it must be possible to reproduce it as an outcome of similar circumstances. Stories about one-time miracles are the foundation of myths and could be believed by the public, but they do not stand up to the standards of science.

According to the biblical story of the binding of Isaac, Abraham heard the voice of God asking him to sacrifice his only son. Today, with a voice memo app on his cell phone, Abraham could have recorded the voice of God and convinced all of humanity to believe in the reality of his experience. But without a recording device, hearsay evidence is not scientifically sufficient.

Even when recording devices are used, one-time events like the “Wow!” or BLC1 radio signals, are of dubious scientific value because they cannot be distinguished from environmental noise or instrumental malfunction.


On supporting science journalism

If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.


One-time astronomical events like the big bang, however, are of great value since they allow reproducible scientific studies of their consequences. For example, conditions in the early universe resulted in the brightness patterns of the cosmic microwave background that seeded the formation of large-scale structure in the universe. These outcomes can be studied repeatedly, in detail, and used in concert with theoretical calculations to determine the constituents of the cosmic mass budget.

The situation gets more complicated with eyewitness testimonies of one-shot events. Recently, the Pentagon was asked by lawmakers to disclose all it knows about Unidentified Flying Objects (UFOs) by mid-2021. But this focus on past eyewitness reports is misguided. It would be prudent to progress forward with our finest instruments, rather than examine past reports. Instead of declassifying documents that reflect decades-old technologies used by witnesses with no scientific expertise, it would be far better to deploy state-of-the-art recording devices, such as camera or audio sensors, at the sites where the reports came from, and search for unusual signals. A scientific expedition focused on reproducing old reports would be more valuable to unraveling the mysteries behind them. A conversation I had recently with podcast host Joe Rogan about my new book triggered a grassroots initiative to fund such experiments. 

My long-term collaborator and friend from Princeton University, Ed Turner, has noted that the number of cameras around the world increased by several orders of magnitude over the last couple of decades. They include security cameras, phone cameras, car cameras, drone cameras, dedicated astronomical observatories and arrays of cameras installed on satellites and aircraft. This led to a vast increase in the numbers of images, often videos, of car and aircraft crashes, meteors, natural disasters, crimes and so on. 

A vivid example is provided by the many clear video and sound recordings of the superbolide Chelyabinsk meteor event over Russia in 2013, obtained by different types of cameras from different directions. At the same time, there had been no noticeable increase in the rate at which UFOs are recorded or in the quality of the recordings. “We had fuzzy, intriguing but not compelling, images and occasional videos way back in the 1960s at a time when no one had ever photographed a commercial airplane crash,” points out Ed, “but we don't have much better evidence now.”

Scientists view reproducibility as a virtue, but others in the public consider it as a limitation. Ed remembers from his early career days that the famous physicist Richard Feynman liked to remark on the limitations of science as a tool for understanding many of our experiences of the world, when talking to graduate students like Ed over beer on Friday afternoons at the Caltech faculty club bar. Perhaps reproducibility is one of them.

As discussed in my book, our inability to continue collecting data on `Oumuamua, which is now a million times fainter than it was during its close passage to the sun, limits our ability to make further progress in deciphering its nature. Our best path forward is therefore to use survey telescopes like Pan-STARRS or the forthcoming Vera C. Rubin Observatory to identify other weird interstellar objects as they approach us, and then deploy a small spacecraft equipped with camera along their predictable path and take a close-up photograph of them. As the saying goes, “a picture is worth a thousand words.” In this case, it would allow us to distinguish between a rock and a technological artifact. 

The experience would be similar to walking on the beach and finding mostly seashells and rocks that are naturally produced, but every now and then stumbling over a plastic bottle that implies the existence of a civilization out there. Whereas the traditional search for radio signals is equivalent to speaking on the phone with a living counterpart, finding a “message in a bottle” resembles receiving a letter in the mail, which could occur after the sender had died. It offers a much more powerful peek into the past existence of dead civilizations, and provides the space analog of an archaeological search for relics left behind by cultures that are not around anymore on Earth. 

Ultimately, the assembly of reproducible results by instruments is essential for separating the subjective impressions of humans from objective data gathering. This does not imply that Abraham did not hear the voice of God. Instead, it highlights the fact that he would have needed a recording device to make the Biblical report convincing beyond a reasonable scientific doubt. 

Avi Loeb is the head of the Galileo Project, director of the Institute for Theory and Computation at the Center for Astrophysics | Harvard & Smithsonian, founding director of Harvard University’s Black Hole Initiative, and the former chair of the Harvard astronomy department (2011-2020). He is a former member of the President’s Council of Advisors on Science and Technology and a former chair of the Board on Physics and Astronomy of the National Academies. He has published more than a thousand peer-reviewed papers and is the bestselling author of Extraterrestrial and Interstellar and a co-author of the textbooks Life in the Cosmos and The First Galaxies in the Universe.

More by Avi Loeb

Subscribe to Support Independent Journalism

Great science journalism requires human expertise, time, effort and creativity. And it costs money. That’s why I and the journalists here at Scientific American hope you’ll join our community.

When you subscribe, you are supporting staff and freelance journalists who are passionate about telling science stories that are true, important and compelling. Our editors and reporters are often experts in their fields, which means they understand the nuances of big discoveries and can untangle the breakthroughs from the hype. With a subscription, you are also supporting rigorous fact-checking to ensure the words we publish are precise and accurate. And you’re supporting original illustrations, graphics and photos that bring you closer to an advanced laboratory, an ice sheet in Antarctica or a space mission in orbit. You’re helping us craft other types of high-quality journalism as well: Our newsletters are carefully written, edited and curated by staffers you have or will come to know and love. Our Science Quickly podcast is based on original reporting, collaboration with editors and scientists and exacting production.

Subscriptions keep this engine running so we can continue to deliver thoughtful, rigorous and independent science journalism to you. In an era of viral misinformation, this work is crucial. If you value what we do, I hope you’ll consider joining us as a subscriber

Thank you,

Jeanna Bryner, Editor in Chief, Scientific American

Subscribe