Eventually findings that bear on treatment decisions and policies should come with a disclosure of any uncertainty that surrounds them. It is fully acceptable for patients and physicians to follow a treatment based on information that has, say, only a 1 percent chance of being correct. But we must be realistic about the odds.
This article was originally published with the title An Epidemic of False Claims.
Subscribe
Buy This Issue
Already a Digital subscriber? Sign-in Now
If your institution has site license access, enter here.
Already a Digital subscriber? Sign-in Now
If your institution has site license access, enter here.
See what we're tweeting about
27 Comments
Add CommentWhen an article starts out by blaming the public, it doesn't fill me with confidence in the analytical ability of the writer.
Reply | Report Abuse | Link to thisThe real cause, as the writer correctly then points out, is the fact that "Much research is conducted for reasons other than the pursuit of truth. Conflicts of interest abound, and they influence outcomes."
Most research is either conducted by corporate sponsored science researchers or in the pursuit of continuing grants based on performance. The pressure to publish and publish real and positive results is higher than at any time in history.
Some of the suggest 'solutions' offered are imho totally unworkable and undesirable. Emphasising "large international collaborations" would favour such large project work disproportionately and prejudice excellent and innovative work by small research groups in unexpected and new areas.
The second real problem here is the almost complete breakdown in the peer review system. Papers with unfounded claims and unsubstantiated positive results are being passed routinely for publication. The bar has been lowered continuously over the last 25 years and the criteria for publication has been diluted.
If a peer review system is to succeed and survive, then the chosen review peers should be supplied with all of the things suggested in this article even if they don't need to be included in the final published paper.. Evidence of replication, total numbers of analysis, details of experimental protocols, FULL data results, limitations and flaws. Failure to disclose should be a clear red flag fro publication.
Rather than blaming the public, the blame and responsibility is squarely on the shoulders of the Scientific community. If it wants to raise standards then it must implement higher standards! it is that simple.
Ultimately the peer review system as it stands is on it's last legs anyway. The future of published science has to be on the web, in a fully open and democratic forum. Critical review has to be opened up to the public and all other scientists, with full data disclosure, instead of the present system where only the well off can afford to buy scientific publications. Publishing was supposed to be the making available of Scientific work to the wider public ! Instead it has become a closed, narrow dissemination of Science to a tiny group of people.
Quality work will stand up to criticism. Shoddy work will quickly flounder. The elitist attitude of the establishment toward such democratising of Science must be jettisoned.
Our science is very, very bad. One need only "look under the hood" of practically any published study to find flaws in methodology, flaws in the use of statistical measures, flaws in using adequately "controlled" study groups, and, of course, the MOTHER OF ALL INVALID SCIENTIFIC INFERENCES: mistaking correlation for cause-and-effect. We humans have far too elevated a view of our true intelligence and objectivity. In reality we are strongly subject to delusion, and "seeing what we want to see." Many of the greatest scientific "discoveries" have been refuted (e.g. that saturated fat causes heart disease). Many other "discoveries" have had a large element of luck, if not arrived at totally by accident.
Reply | Report Abuse | Link to thisThere should be some kind of minimum standard required before studies are published. It would reduce the publishing by 90%, and increase the validity of the studies by many multiples. The reduction in the constant flow of flawed studies would, in turn, allow more time for peer review of what actually gets published, which would further increase the quality.
First, there is no evidence of "fraud," which would mean intentionally misleading studies. This was ruled out even in the case of the leaked emails, which simply showed the scientists were unprofessional. Their actual work was checked out and re-affirmed by independent, unbiased sources.
Reply | Report Abuse | Link to thisSecond, you CANNOT KNOW FOR SURE one way or the other about man-made global warming. You are speculating, based on incomplete information. Let's say there's a one-chance-in-a-million that current global warming theories are correct. Are you willing to make that bet? The downside is global human catastrophe beyond description, if not total extinction of the human race. Imagine if, for example, the growing season for 300 million Russians was substantially shortened or eliminated by a slight shift in weather patterns. Where will those people go? They certainly won't quietly starve to death. There would be World War beyond anything we have ever seen. And that is only one scenario among thousands.
As we sort out the data, we should be seriously reducing our footprint before we destroy the human habitat. We should be studying the whole subject intensely. "Global warming isn't 'innocent until proven guilty' beyond the shadow of a doubt. Quite the opposite is. That's just common sense.
(note: I agree with you on the biofuel fiasco.)
The problem is that we have neither "evidence based medicine" or "evidence based belief systems". In either case, our beliefs underly the choices we make and our choices always have consequences. Ergo, we should try to base what we believe on the best evidence possible. The problem is that the "science" of belief creation is well understood by advertising and marketing firms. This results in societies that believe and make choices where there is no or even contradictory evidence. The other day my pastor told me that if I give money to the church (i.e. him) it makes God happy and when God is happy he will do nice things for me....plus the GM salesperson told me that if I buy a Corvette beautiful women will be attracted to me...and the toothpaste that gives me blazingly white teeth...so many things to believe and so many people to benefit from what I believe.
Reply | Report Abuse | Link to thisThank you for speaking out about this.
Reply | Report Abuse | Link to thisI for one would like to see Scientific American have more to say about the "scientific method" and the intellectual integrity required to do good science.
I agree.
Reply | Report Abuse | Link to thisUnfortunately much as I love SciAm - it is very much a passive conduit for any and all Science reporting. As a reader of SciAm for decades it has always been clear that it is basically a financial and commercial venture who's goal is to be a conduit for Science research to the public. Hey there's nothing bad about being commercial, but it is important that this is kept in mind and a desire to hype research claims is an ever present tendency.
I would love SciAm to lead the way in a campaign against bad science and for more scepticism and rational thinking. More 'critiquing' of the science that it reports is desperately needed. More robust insisting on adherence to the "Scientific method".
I know it incites the believers but this includes more scepticism of the claims of the AGW establishment. At the heart of much of the AGW claims is a plethora of weak science such as the extrapolation of dendrochronology into the field of ancient climate analysis.
Unfortunately challenging bad science is not necessarily a big money maker so I don't actually expect SciAm to change it's policies any time soon.
I'd also like to see the educational system...say 7th and 8th grades...focus more on the _application_ of the scientific method and the importance of objectivity and integrity in science -- even when you don't like the results. While we may never be able to avoid honest mistakes or the fog of wishful thinking, our kids today seem to be suffering from a supreme lack of problem solving skills and the moral convictions required of real science.
Reply | Report Abuse | Link to thisI recently had the opportunity to review the science projects of the top 7th grade science students in our area. To say it was disappointing is an understatement. Conclusions were weakly tied to the observations and results, but miraculously seemed to support the child's hypothesis. These same kids will be the doctors and researchers who will be curing my ailments in the a few years...and I'm not feeling so good about it.
I think teaching kids to love the process of discovery is the key to making great scientist. Unfortunately, the focus in school is on getting an "A" at any cost.
I agree, but I believe the time to start science education is earlier.
Reply | Report Abuse | Link to thisI've found many elementary school children have a strong desire to learn more about science, a desire that goes largely unmet at school. It seems such a waste that schools don't make a better effort at teaching such an important subject that their students are motivated to learn.
While there are only so many hours in the classroom, and every hour is already filled with something, learning the scientific method is "strategically" important. Like learning to read, learning the scientific method opens up new avenues for children to make discoveries on their own. It gives them important tools to distinguish fact from fallacy in a world filled with individuals, industries and organization pedaling wu.
Remember, that even if these children don't become scientists, one day they will be making decisions in their daily work, voting, and sitting on juries. Wouldn't you rather have them know what good science is.
It would also help if reputable non-specialist science publications took a more skeptical attitude towards research. There have been numerous instances in recent years in which the substance of the research did not match the headline statement.
Reply | Report Abuse | Link to thisThis is barely forgiveable in mass media. In "scientific" publications it should simply not occur. "Some slight evidence that X influences Y" may not grab the attention of "X causes Y" but standards demand truth in headlining!
This is why a Med School professor told my freshman class 50 some years ago: Remember, for every pearl on the ocean floor, there is one ton of whale manure. Way too much of the latter makes it into print.
Reply | Report Abuse | Link to thisFirst mention is the vitamin e controversy, aha boys as I look at my ankles I find that maybe some of you are really idiots. That Canadian doctor whose name I forget wrote an article and book that was published by Rodale in OGF, decades ago. I am glad that I started using Vit. E! Many people with varicose veins like I have get ulcerated legs and ankles. I have not. When I can not afforsd the vitamin e, the varicose veins increase and my ankles itch and are blue with the multiple venous arteries. I don't know who did the study but they don't convince me. People who listened to my narrative and used the vitamin e always watching not to overdose so that they had heart palipatations had my experience. It enables me to avoid all sorts of medical treatments. It costs medicare nothing but makes my life better. I know people who find that milk is a problem, some of our bodies are genetically susceptible to different things. My white heritage gave me MS but also the ability to absorb milk products. I have learned even though I am a college grad, not to believe every negative test for "old wives tales". Age does that for us if we keep an open mind.
Reply | Report Abuse | Link to thisI agree with the comments by Howard that scientists need to take more responsibility for the review process. But reviewing papers is a sort of "community service" that we do in our spare time; so it's tough to ask people to do a better job of it. I don't have a good answer.
Reply | Report Abuse | Link to thisI disagree with the suggestion that publishing should be open to anybody who wants to publish anything. Shoddy work does not necessarily flounder quickly. Take a look at "Merchants of Doubt" by Oreskes & Conway. The tobacco industry kept the 'scientific' discussion of the dangers of cigarette smoke going long after the evidence was established. Same with acid rain and ozone holes. If you have enough people who want to believe a proposition they'll find a way to promote their views. The idea of peer review is that *knowledgeable* people weigh in on the validity of the research before it gets published.
-george
What to do for researchers from developing countries who looks to publications in highly impacted journal as to gold standard of study quality. Quality of researches round the world is not equal and if gold standard is undermined that means that no need to reach best quality in trials. Or journal should openly say for all authors that in some cases they do not publish article not because of bad design but by reason if low interest by sponsor
Reply | Report Abuse | Link to thisAs a high school science teacher (now retired) of more than a third of a century I think I know that the poor elementary teacher must prepare herself for so many different subjects that it is impossible to do a good job in all of them. The current emphasis is upon math and reading. I have talked to perfectly intellegent elementary teachers who routinely go through some route-learning sequence with their class as though science really operated that way. If the student can learn to spell "Hypothesis" he gets a good grade in science.
Reply | Report Abuse | Link to thisThink about it. If a college elementary teacher prep. program finds that a student does well in science, where is the student directed? Into secondary science teaching - thereby depriving the elementary students of the best early science education they can get.
In all of my years of high school teaching I met only one elementary teacher who actually taught hands on science to elementary students. I could always tell which of my freshmen came from his school.
If we want to improve elementary science teaching then we need to fund summer institutes for elementary teachers so that they are paid to get the education they need to modify their instructional methods. I was able to go to such institutes for secondary science teachers in the 60s. Two of them gave me the knowledge to start a high school marine biology class. Such institutes for elementary teachers would do much to give children an early start in their science education.
All readers miss the point that clinical studies, as a quite general rule, use far too small statistical samples - due to the ethical problem of experiment upon humans, and some other constraints.
Reply | Report Abuse | Link to thisClinical studies are like anecdote: undependable in themselves, but perhaps useful when a huge number of them can be included in metastudy base.
Peer Review:
A few journals have successfully strong peer review, and can be trusted. Most scientists know these.
The go-to Med journals are fewer, but a couple are good.
This issue is constantly addressed, and those who comment must make an effort to educate themselves on the ongoing observation and evaluation, BEFORE bewailing in generalities, science in general.
The article is comprehensive, but it is you who must follow up with your own research, commentors, as the literature is far too extensive to be addressed in an article remotely as brief as it is necessary for the author to write. Above is journalism, and references would fill pages.
Medicine is not science, do not confuse the two.
Medicine uses various sciences, and although clinical studies can be statistically evaluated, leading untrained to believe that it is itself science, it has the constraints I mentioned, along with others, such as conflict of interest, which has skewed conclusions toward those which benefit the funders of both clinical and lab findings.
There are far too many cases of fraud, exaggeration, and of failing to adjust for placebo effect. Due to our evolved human nature, many cases are not conscious fraud or exaggeration.
To reiterate:
Many many scientific journals have been created to serve corporate industrial interests.
Many consumer health publications thrive due to acceptance of fraudulent and nonscientific claims made by producers who support overexuberant conclusions. These producers are incorporated in order to protect their investors/owners from personal liability. Government bureaus were intended to oversee such industries, as were groups such as AMA.
Politics is the means by which "free market" confidence men and others who engage in fraud seek continued opportunity for personal gain.
You must seek the kind of society in which you would prefer to live, knowing that our very nature devalues strangers, discounts the future for present gain, and rationalizes unethical behavior in large anonymous interaction.
Physician, heal thyself. SciAm has been wearing their politics on their sleeves and in their reader's faces for decades, which is why I canceled my subscription long ago. Politics doesn't belong in science any more than creationism in a science class. It's a form of intellectual corruption of the same type as is being discussed in this article - and a precursor of it as well. E.G. http://tinyurl.com/4y98zx6
Reply | Report Abuse | Link to thisI agree that there is about 999,999 out of a million that man-made global warming isn't happening. For those odds, I would gladly not bet to hang an additional millstone around the neck of the US (and world) economy. That will cause way more innocent deaths than "global warming". And by the way, common sense would tell you that global warming would increase the growing season in Russia, if anything. Typical red herrings from the left....
Reply | Report Abuse | Link to thisIt is correct more or less that there is no evidence of fraud per se in the AGW movement. What is far more likely is that the sincerity of proponents of AGW has interfered with their honesty, so to speak. They have taken the catastrophic outcomes of computer modeling seriously, assuming the adequacy of our knowledge of the climate system is adequate to model. Perforce it is reasonable to be alarmed to, explain their alarm, and to insist something be done. The problem is that AGW models as posed don't model or track reality.
Reply | Report Abuse | Link to thisAssuming the physical properties of anthropic-generated CO2 are no different than those of natural CO2, there is no logical grounds to assume AGW computer model behaviour is any more than a mathematical artifact, and we should conclude that the models lacks any special empirical significance. Geological evidence indicates that CO2 has generally been many times higher in the past than it is at present - a near all-time low level. Thus, unless there is something inherently different about the physics of anthropic CO2, there is no creditable hazard posed by any foreseeable change in CO2 due to human activity. There many more serious anthropic threats to the environment than CO2 can ever be.
Unbelievable. SciAm - what were you thinking?
Reply | Report Abuse | Link to thisThis is the identical problem "climate science" has. You have Nailed It.
But you'll never admit to that, will you?
Kay 42 writes: "This is the identical problem "climate science" has."
Reply | Report Abuse | Link to thisYou are mistaken. The problem is that parody of science which is funded by for profit market actors - drug companies, oil/gas industries, etc. The parody they create looks like science and it is used as a marketing technique to government regulators and the buying public.
Science funded by not for profit actors ("pure research") is less susceptible to becoming a parody of science and is more likely to be actual science at work.
natural law, scientific methodology and practice - peer review, falsifiability and demonstrable fact tell us that human induced global warming is real.
You are mistaken. Your talking points are old and tiresome. If drug companies and oil/gas industries are corrupt the entities that suck at the teat of government funding are equally corrupt.
Reply | Report Abuse | Link to thisGrow up.
Anyone who is still hanging on to this human-induced global warming foolishness is not doing their homework. A lively and satisfying discussion took place months ago at Climate Etc. See “Climate story telling angst” and all the following comments. Then come back here and spout your tripe.
Reply | Report Abuse | Link to thisAs Kip Hansen wrote at judithcurry.com:
I must be shown the following:
1) The hypothesis, which must be falsifiable.
2) What experiment has been done and that it has been carefully laid out well enough to falsify your null hypothesis (and thus support your original hypothesis).
3) What you did exactly, all the nasty details, how you controlled for every possible confounding factor (or didn’t control for this one and that…and why not, and how that might affect your findings).
4) Your conclusion and how it follows from your data (and not your beliefs, feelings, hunches, or desire to please your funding agency or university tenure board).
Then, and only then, will I listen to your opinion about what it might mean.
kay 42 writes: "If drug companies and oil/gas industries are corrupt the entities that suck at the teat of government funding are equally"
Reply | Report Abuse | Link to thissound, but invalid. thank you for considering the possibility that drug, oil and gas industries may be corrupt. you might also consider that the tobacco lobby/industry may be corrupt as well - meaning they have a financial interest in scientificaly proving, disproving or inciting controversy on certain issues. the monetary size of the research conducted by these potentially corrupt actors is significantly larger than the monetary size of the research conducted by not for profit (pure science) actors.
Instead of reading what I wrote, you dragged in another tired example. Good job.
Reply | Report Abuse | Link to thisPhil Jones brought in $22,000,000 in just one year. It's in a spreadsheet in the climate emails. The BBC has 33 billion dollars invested in the future of global warming. It would be a safe estimate to say that trillions of dollars have been "invested" in this scam.
You are not interested in a discussion. You only believe in the evils of the free market system and the nirvana we would have if only the government controlled everything.
Good luck with that.
kay 42 writes: "You only believe in the evils of the free market system and the nirvana we would have if only the government controlled everything."
Reply | Report Abuse | Link to thisyou are mistaken in your belief in the existence of a free market. and in your belief on my view of government.
If you don't like the magazine and canceled your subscription, why are you still funding them by viewing their online articles and thus generating revenue via advertising? Put your money where your mouth is and stop financially supporting SciAm by never coming to their site. Those of us that like SciAm will gladly take your place.
Reply | Report Abuse | Link to thisQuailty comment, hit's nail nicely on head...
Reply | Report Abuse | Link to this