Moiya is an expert in science communication as well as being a scientist herself, so today I want to ask her about the steps you have to go through to produce a scientific paper. How do you go from an idea to a published paper? Why should we trust these scientific publications more than other writing or news reports we find on the internet?
I’ve said on this show before not to trust articles that don’t cite their sources, but we’ve not yet talked about why we should trust those cited sources to begin with. So, thank you for being here, Moiya.
Thanks for inviting me on.
So, I get an idea, I write it down and then it’s published, right?
That’s the three step version, yeah. But there’s a lot more underneath the hood, otherwise it wouldn’t take five or six years to get a PhD, right? Starting from the beginning, people say that science starts with an idea, but it starts even before that. Because when I was starting grad school, one of the most intimidating things to me about the scientific process was coming up with an idea. Up until then, I had been doing research for a few years, but the research projects had always been handed to me and I’d never come up with my own idea or my own thesis for a research project.
You have to go through a lot of training to get to the point where you’re able to come up with your own idea or research project. It starts with an idea, but there’s a lot that goes into that part as well.
Could you maybe walk us through one of your recent projects?
Yeah, I’d love to. I actually just submitted for publication so this is the perfect time to talk about this. This idea actually wasn’t my own. One the faculty members in my department thought of this idea while he was in the shower. That’s where all the scientists do their best thinking!
So he thought about this idea and he brought it to me, and I thought it would be really exciting to explore. The idea that he had or the question that he wanted to answer was, ‘Are planets more likely to form around slow moving stars? Is there any sort of relationship between the speed of a star and whether or not it will host planets?’
I took that and ran with it. The thing I had to do next was figure out what type of data I was going to study to answer this question. As an astronomer, I have access to so much data. Observatories are taking something like petabytes or terabytes of data every year. So I have my choice of what type of data I want to use.
For this project, because I’m interested in planets and the speed of stars, I knew that I should probably use data from the Kepler mission which is responsible for finding most of the planets we’ve ever discovered and the Gaia mission which is responsible for giving us the most accurate and precise map of stars in the Milky Way Galaxy that we’ve ever seen before. So that was the next step: figuring out what type of data I wanted to use.
And then I had to run experiments. We talk a lot about the scientific method as scientists. It’s a good rule of thumb, it’s a good suggestion for how the scientific process goes, but it’s not exact. Sometimes you’re going to deviate from the scientific method based on what type of research you do.
As an astronomer, I can’t really run experiments with stars. I can’t go out and have a control group and manipulate different stars with different methods to see how it changes the outcome which is what most people think of science. Instead I have to do all my experiments on a computer. I had to take the data that I had and split it up into different populations or different groups.
So I have a group of stars that have planets and a group of stars that don’t have planets and I can compare their velocities because that’s the question I want to answer. That sounds really easy, but it took me a few months to do this because I wanted to make sure I was doing it right. Which brings me to the next step that I did and the next reason you should really trust scientific publications.
One of the things I spent a long time doing was talking to other experts in the field and asking them what I could have done wrong. I went to different faculty members in my department, I emailed postdoctoral fellows who work in this field, and I said, I have observed this thing, I have this result, what do you think about it? Can you think of anything that I might have done wrong? Can you think of anything that might explain this that I haven’t thought of?
So, I really did do a lot of work trying to prove myself wrong, which might seem like it goes against what science is supposed to do, but a good scientist wants to make sure that her work is accurate, so I spent a lot of time proving myself wrong and eventually was successful in proving myself wrong ... unfortunately, or fortunately depending on how you look at it.
So, I spent a couple of months looking at my results and thinking that planets are more likely to form around slow-moving stars or are at least more likely to survive around slow-moving stars. Then one day after long conversations with my advisor and with other experts in the field, I changed a couple of variables in my code and the result went away. This thing that I had been observing for months was actually just a result of some selection bias by the Kepler Telescope.
Can you tell us what a selection bias means in the instance of Kepler?
Yes, definitely. Kepler is an amazing telescope that’s not operating anymore. But for nine years it was staring out into space trying to find planets. And the way it did that was by measuring the amount of brightness that we get from different patches of the sky. If a planet passes in front of a star, then it will block some of that star’s light. Over time we would measure a dip in the amount of brightness we’re getting from a certain patch of sky.
But a telescope can only observe certain things or it will be limited by certain things. So Kepler can only observe stars that are bright enough for Kepler to see. So that’s a bias right there. Kepler is biased towards observing really bright stars.
It’s also going to be biased towards finding planets that are easy to see. Those will be planets that are close to their stars. Those will be planets that are bigger. It’s much easier to see a larger planet than it is to see a smaller planet. So that’s what I mean when I say selection bias. There are things that are easier for this telescope to see and so we are going to be biased towards seeing those types of things instead of things that are harder to observe.