Q&A With the Statistician Who Calculated the Odds That This Tomb Belonged to Jesus

This sidebar belongs to the feature story Special Report: Has James Cameron Found Jesus's Tomb or Is It Just a Statistical Error?

Christopher Mims: When I saw that there was a statistician involved in gathering evidence for the documentary The Lost Tomb of Jesus, I was very intrigued.

Andrey Feuerverger, professor of statistics and mathematics at the University of Toronto: When they asked me to run some numbers for them, that consultation came as a surprise. It's not in the usual run of things one gets to do - but I am a statistician and that's why they came to me.

I have to tell you that a statistician working with a subject matter expert, in this case biblical historical scholars, essentially is obliged to rely on assumptions that come from them. I take responsibility for the calculations in some sense, but not for the specific assumptions that go into it.

So Ideally what you would want is a telephone directory of all names that lived in that era. There is something akin to that--there is a compendium--I think it's called Lexicon of Jewish Names in Late Antiquity, compiled by Tal Ilan. (published in 2002)

That is one of the most scholarly works available. It has 2509 names of males and 317 names of females that lived not only in that era but some hundred or so years prior to that and roughly that amount of time after that (but this is just from memory). Tal ilan had put together every single known reference of a name to every person who lived in that time, so we have what we think are reasonably good data on the frequency of names. So we know for example that every version of Mary—mariam—happened 80 times out of the 317 women in the sample. For men there was no one name as popular but some had a high degree of popularity. For example even if you pick something like Yeshua, which is Jesus, 103 out of the 2509 names in Tal Ilan actually happen to be Yeshua in one form or another.

CM: So what was your methodology and your assumptions going into this?

AF: Let me tell you first what the methodology was and then we'll go into the specific assumptions.

We're not supposed to just calculate the probability of the specific thing that has been observed. Because the specific thing that has been observed, we know a priori has a low probability. The idea behind the calculation is to say "let's look at every other possible cluster of names that could have happened by just literally sampling at random from this so-called telephone directory. And then we would compare how unusual or surprising it is relative to the thing that we're looking at. We would try to in some implicit way make a list of all the possible things we could have seen that are at least as striking as what we have seen. Then what you do is you take all those points and add up their probabilities - and that's the thing that's your evidence value. That's the basic technology, but the assumptions that go into it--these are things that biblical scholars need to vet.

For example, one quite key assumption is the tremendous specificity of the name Mariemene e Mara. I'm not a scholar of biblical history so I'm not one that can vet that, and I'm sure it will be contested and fairly so. But if you look at all of the 80 Marys of all the 317 women to have lived then, none has a name that is so extraordinarily specific as Maremene e Mara.

So instead of assigning to that particular name a probability of 80 out of 317, based on the assumptions I was asked to work with, I used only 1 out of 317. In other words, I was assuming that out of every 80 Marys, only one would have a version of a name that is so highly specialized toward someone who might have been Mary Magdalene.

The other very unusual name that occurs in the cluster is the name Yose - the Hebrew spelling of it is important. Josef and Jose and all the versions of it - it's a very common name - there are 231 of them amongst the 2509 male names.

But Yose actually only appears three times in the Lexicon of Jewish Names. One thing you could argue, and I'm not saying I would argue it but I might work with it if I was asked to, is that it's a rather appropriate version of a name for one of the brothers of Jesus. If you look in the New Testament - one of the gospels names the brothers of Jesus, and that particular brother is named Yose - obviously a translation for some greek version of his hebrew name. The uncommonness of the name drives to some extent the sharpness of some of the numbers that we're actually seeing.

Then what you can try to do, and it may require approximations, you have to try to list all the possible things that oculd have happened and in each case assess how surprising it is - and then take all of those that are at least as surprising as what you've got, and then just add up all of their probabilities together to get a kind of a p value.

This p value basically speaks to one thing—it speaks to the likelihood that there would have been another family alive at that time whose tomb this could reasonably have been; whose cluster of names was such that this could have been their tomb.

In doing the math you have to take into account that there were more than one tomb that has been looked into. I think at least 100 tombs have been looked into. So we're seeing the best of 100 observations, and there are at least 1000 tomb sites that are actually known to be in existence, and there may in fact be more of them—I've been told that there could be as many as 2,000 tomb sites all together. So we have to factor in the fact that what we've seen is possibly the best observation out of 1000 or 2000.

Depending on how you do the calculation you're going to get some range of numbers, and every statistician will have a different range of numbers. And you're going to get numbers on the order of 1 in 100 to perhaps as much as 1 in 1000 - that being the odds against there being another family with that particular combination of names.

We're also taking into account the fact that many families could not possibly afford ossuarial burials - that was an expensive proposition. There are some quite good estimates out there on what the population of Israel was during that time, as well as good estimates on what the birth and death rates were. So one can estimate how many people died during the entire era in question. All of those things need to factor in to the statistical calculation.

It's not a secret that these assumptions are contestable. I tried to stay with things that vaguely seemed reasonable to me, but I'm not a biblical scholar - at the end of the day I went with specific assumptions and I try to make clear what those assumptions were. I've been clear with you that the extraordinariness of the Mariemene e Mara inscription gets factored into the calculation as a very rare name - other people might say this is not like this.

Truthfully, I think that the cluster of names is a very interesting one. It's somehow apt, there are a number of names in that cluster and each time you add names to a cluster it alters the probability values, so six names is not a bad number to work with. It is an impressive collection. It might have made life easier all around if the probability numbers came in much much lower than 1 in 100 or if it came in much higher than 1 in 1000. As with many of these things, one needs to say that further study is warranted. So biblical scholars will have to think carefully in order to vet the assumptions that are reasonable to a calculation of this nature.

I did permit the number one in 600 to be used in the film. I'm prepared to stand behind that but on the understanding that these numbers were calculated based on assumptions that I was asked to use. These assumptions don't seem unreasonable to me but I have to remember that I'm not a biblical scholar.

At the end of the day, what does one in 600 mean? That number is saying that amongst those people who could afford ossuarial burials, the odds that there being another family that appeared in an ossuary that was like that or even more convincing than that, the odds against that are in that order, one in 100, one in 600 or one in 1000, depending on what you allow in your assumptions.

With every assumption that was made an attempt was made to be reasonable. The two that I can't really say anything about because of the biblical scholarship part of it are the Mariemene e Mara -- how truly appropriate that is to Mary Magdalene, and how appropriate Yose is for one of the brothers of Jesus.

This is not a classical hypothesis testing problem. Here it's more a question of just probabilities of things. I was trying to follow, to mirror the paradigm of hypothesis testing. I was trying to mirror that paradigm, and I was trying to mirror the thinking of Sir Ronald Fischer. His thinking was to calculate probabilities that would allow him to say the kind of thing like: "either we have observed an event of singularly rare chance or the null hypothesis must be false" - that's the logical process behind hypothesis testing if you take it back to the days of Ronald Fischer.

CM: So are you going to publish these results in a journal?

AF: I'm trying to prepare an article for publication as a data set for statisticians it's an extraordinary data set. I think that statisticians can look at it and develop new statistical technologies, and it will interest statisticians considerably. Everyone is likely to have their own two cents on how to proceed, how to analyze the data, and there will be massive dialogue between statistical community and the historical community to see what's the right thing to be doing here.