An Alarming Number of Clinical Trials Are Greenlit Based on Shockingly Poor Data
This is the research that is being discussed here
Preclinical efficacy studies in investigator brochures: Do they enable risk–benefit assessment?
Study questions animal data underlying many clinical trials
"So-called investigator brochures—the documents that researchers produce to convince regulatory agencies and ethical review boards that a proposed trial is worth the risk—are often lacking crucial information about the efficacy of the proposed therapy in animal models. As a result, it’s often impossible to tell how good the evidence is.
“This is incredibly alarming,” says Shai Silberberg, director of research quality at the National Institute of Neurological Disorders and Stroke in Bethesda, Maryland, who was not involved in the study. The work "shows that decision-makers for ethics related to clinical trials don't get the information they really need to evaluate those preclinical trials.”
Most animal research studies may not avoid key biases
"Known drug adverse events have been estimated to be between the 4th and 6th leading cause of death in the US (1). There are currently insufficient funds to perform the quantity and quality of studies necessary to ensure all drug adverse events are discovered in a timely fashion."
Why Scientific Studies Are So Often Wrong: The Streetlight Effect
This article that appeared in Discover Magazine explores many of the problems that arise in designing clinical trials, even when the researchers have the best of intentions. There can be problems recruiting enough appropriate participants, there are often limited ways that researchers can measure the variables and outcomes that they are looking at, many questions- such as the long-term safety of a medication- would require more time to investigate properly than research allows, so proxy indicators are used that can turn out to be very wrong. Sometimes the results of studies are generalized too broadly. Examples of some of these problems include;
"The streetlight effect is just one way that research measurements go wrong, and measurement mess-ups are just one of several ways that studies go wrong. Bad measurements are not even the biggest source of wrongness in scientific studies. That honor would have to go to “publication bias”—journals’ tendency to eagerly publish the small percentage of studies that produce exciting, surprising, breakthrough results. Of course, the most likely explanation for why one team of researchers comes up with surprising results while several other teams get less-publishable, boring results is that the one team screwed up somewhere. The pernicious effect of this phenomenon on the trustworthiness of study results has been documented at length in scientific journals themselves in several fields."
"Studies in the 1990s appeared to prove that hormone replacement therapy reduced the risk of heart disease by 50 percent. Then in 2002 a large study seemed to prove that the therapy increased the risk of heart disease by 29 percent. It turns out that a woman’s age affects her response to hormone replacement therapy, and the discrepancy arose because the first study looked at somewhat younger women than did the second. The data in both studies were credible; they just did not apply to all women."
"Yet another problem is that much of what scientists think they know about human health comes from animal studies. Unfortunately, three-quarters of the drugs that prove safe and effective in animals end up failing in early human trials, sometimes spectacularly. In 2006 the experimental leukemia drug TGN1412 was given to six volunteer human patients. All six of them quickly fell seriously ill with multiple organ trauma, even though the stuff had worked well on rabbits and monkeys at doses up to 500 times as large."
The Deadly Corruption of Clinical Trials
"The cleverest manipulation has been with the clinical trials themselves. For years, critics have charged that pharmaceutical companies massage trials to make their own drugs look better than they really are. One common tactic is to suppress unfavorable data. A notorious example came in the 1990s, when a Wyeth safety officer overwrote the company's computer files, erasing evidence indicating that its diet drug, fen-phen, caused valvular heart disease. A less risky strategy is simply not to publish potentially damaging trials. In 2004, the Canadian Medical Association Journal described a leaked document indicating that GlaxoSmithKline had deliberately hidden two studies from regulators showing that its antidepressant, Paxil (paroxetine), could increase the risk of suicide in children. "
"A 2006 study in The American Journal of Psychiatry, which looked at 32 head-to-head trials of atypicals, found that 90 percent of them came out positively for whichever company had designed and financed the trial. This startling result was not a matter of selective publication. The companies had simply designed the studies in a way that virtually ensured their own drugs would come out ahead—for instance, by dosing the competing drugs too low to be effective, or so high that they would produce damaging side effects. Much of this manipulation came from biased statistical analyses and rigged trial designs of such complexity that outside reviewers were unable to spot them. As Dr. Richard Smith, the former editor of the British Medical Journal, has pointed out, "The companies seem to get the results they want not by fiddling the results, which would be far too crude and possibly detectable by peer review, but rather by asking the 'right' questions."
Shocking Report from Medical Insiders
"A shocking admission by the editor of the world’s most respected medical journal, The Lancet, has been virtually ignored by the mainstream media. Dr. Richard Horton, Editor-in-chief of the Lancet recently published a statement declaring that a shocking amount of published research is unreliable at best, if not completely false, as in, fraudulent.
Horton declared, “Much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.”
To state the point in other words, Horton states bluntly that major pharmaceutical companies falsify or manipulate tests on the health, safety and effectiveness of their various drugs by taking samples too small to be statistically meaningful or hiring test labs or scientists where the lab or scientist has blatant conflicts of interest such as pleasing the drug company to get further grants. At least half of all such tests are worthless or worse he claims. As the drugs have a major effect on the health of millions of consumers, the manipulation amounts to criminal dereliction and malfeasance.
The drug industry-sponsored studies Horton refers to develop commercial drugs or vaccines to supposedly help people, used to train medical staff, to educate medical students and more."
NEJM editor: “No longer possible to believe much of clinical research published”
"Harvard Medical School’s Dr. Marcia Angell is the author of The Truth About the Drug Companies: How They Deceive Us and What to Do About It. But more to the point, she’s also the former Editor-in-Chief at the New England Journal of Medicine, arguably one of the most respected medical journals on earth. But after reading her article in the New York Review of Books called Drug Companies & Doctors: A Story of Corruption, one wonders if any medical journal on earth is worth anybody’s respect anymore.
“It is simply no longer possible to believe much of the clinical research that is published, or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as an editor of The New England Journal of Medicine.”
Payments by US pharmaceutical and medical device manufacturers to US medical journal editors: retrospective observational study
Lies, Damned Lies, and Medical Science
"Much of what medical researchers conclude in their studies is misleading, exaggerated, or flat-out wrong. So why are doctors—to a striking extent—still drawing upon misinformation in their everyday practice? Dr. John Ioannidis has spent his career challenging his peers by exposing their bad science."
Corrupting Science – Hayes, Syngenta and Atrazine
This article presents information that the chemical company Syngenta has been using unethical, illegal, and aggressive methods (including intimidation and threats) to try to discredit a scientist who has raised important questions about the herbicide Atrazine made by this company. Below is the article mentioned in this one with more information about the situation.
Why Almost Everything You Hear About Medicine Is Wrong
"But what if wrong answers aren’t the exception but the rule? More and more scholars who scrutinize health research are now making that claim. It isn’t just an individual study here and there that’s flawed, they charge. Instead, the very framework of medical investigation may be off-kilter, leading time and again to findings that are at best unproved and at worst dangerously wrong. The result is a system that leads patients and physicians astray—spurring often costly regimens that won’t help and may even harm you.
It’s a disturbing view, with huge implications for doctors, policymakers, and health-conscious consumers. And one of its foremost advocates, Dr. John P.A. Ioannidis, has just ascended to a new, prominent platform after years of crusading against the baseless health and medical claims. As the new chief of Stanford University’s Prevention Research Center, Ioannidis is cementing his role as one of medicine’s top mythbusters. “People are being hurt and even dying” because of false medical claims, he says: not quackery, but errors in medical research."
Study: Elite scientists can hold back science
"All of this is another example of how progress in science is confounded by human behavior. We see this in so many ways. Scientists lie about results. Or they discount insights derived from failures. Science is so obsessed with the rewards of solving complicated problems that it forgets about the simple ones. The field overwhelmingly is biased toward males (experiments have shown "John" gets more accolades than "Jennifer" with the identical résumé).
It's worth remembering: Science may be a noble discipline based on cold logic and rational observation; but humans are animals fueled by emotion and bias. As the NBER researchers conclude: "[T]he idiosyncratic stances of individual scientists can do much to alter, or at least delay, the course of scientific advance."
Drug research routinely suppressed, study authors find
By John Fauber of the Journal Sentinel
"Drug research, even from clinical trials sponsored by the federal government, routinely is suppressed, harming patients and increasing health care costs, according to new data highlighting an ethical controversy that continues to plague the field of medicine."
Medical Studies Involving Children Often Go Unpublished
NPR August 4, 2016
"The report, reviewing clinical trials started in 2008-10, finds that 19 percent of the studies that recruited children didn't run to completion. That was often because researchers weren't able to recruit as many volunteers as they needed to run the experiments. And of the 455 trials that were completed, the results from 30 percent weren't published.
The harm is we end up with scientific literature that only shows all the things that do work," she said. "It may falsely appear that certain interventions do work. So our literature may become biased and may not be representative of the true efficacy or safety of an intervention.
Ross considers this an ethical lapse. "When you do a clinical study and you're asking patients to participate and subject themselves to a risk, in order to inform science and generate knowledge, you have an ethical obligation to disseminate those results to the wider scientific community," he said."
Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis.
BMJ. 2012 Jan 3;344:d7292.
"Despite recent improvement in timely publication, fewer than half of trials funded by NIH are published in a peer reviewed biomedical journal indexed by Medline within 30 months of trial completion. Moreover, after a median of 51 months after trial completion, a third of trials remained unpublished."
Research Misconduct Identified by the US Food and Drug Administration
Out of Sight, Out of Mind, Out of the Peer-Reviewed Literature
JAMA Intern Med. Published online February 09, 2015
"Every year, the US Food and Drug Administration (FDA) inspects several hundred clinical sites performing biomedical research on human participants and occasionally finds evidence of substantial departures from good clinical practice and research misconduct. However, the FDA has no systematic method of communicating these findings to the scientific community, leaving open the possibility that research misconduct detected by a government agency goes unremarked in the peer-reviewed literature.
Fifty-seven published clinical trials were identified for which an FDA inspection of a trial site had found significant evidence of 1 or more of the following problems: falsification or submission of false information, 22 trials (39%); problems with adverse events reporting, 14 trials (25%); protocol violations, 42 trials (74%); inadequate or inaccurate recordkeeping, 35 trials (61%); failure to protect the safety of patients and/or issues with oversight or informed consent, 30 trials (53%); and violations not otherwise categorized, 20 trials (35%). Only 3 of the 78 publications (4%) that resulted from trials in which the FDA found significant violations mentioned the objectionable conditions or practices found during the inspection. No corrections, retractions, expressions of concern, or other comments acknowledging the key issues identified by the inspection were subsequently published.
When the FDA finds significant departures from good clinical practice, those findings are seldom reflected in the peer-reviewed literature, even when there is evidence of data fabrication or other forms of research misconduct."
This Reuters article has more discussion about this study.
Harvard Scientists warn about Epidemic of Side Effects due to Corruption
"A recent study about institutional corruption of pharmaceuticals and the myth of safe and effective drugs, warns that over the past 35 years, patients have suffered from a largely hidden epidemic of side effects from drugs that usually have few, offsetting benefits.
The study, published in the Journal of Law, Medicine and Ethics,* conducted by Light, Lexchin and Darrow, warns that the pharmaceutical industry has corrupted the practice of medicine through its influence over what drugs are developed, how they are tested, and how medical knowledge is created."
Medical Journals Are an Extension of the Marketing Arm of Pharmaceutical Companies
PLoS Med 2(5): e138.
“Journals have devolved into information laundering operations for the pharmaceutical industry”, wrote Richard Horton, editor of the Lancet, in March 2004. In the same year, Marcia Angell, former editor of the New England Journal of Medicine, lambasted the industry for becoming “primarily a marketing machine” and co-opting “every institution that might stand in its way”. Medical journals were conspicuously absent from her list of co-opted institutions, but she and Horton are not the only editors who have become increasingly queasy about the power and influence of the industry. Jerry Kassirer, another former editor of the New England Journal of Medicine, argues that the industry has deflected the moral compasses of many physicians, and the editors of PLoS Medicine have declared that they will not become “part of the cycle of dependency…between journals and the pharmaceutical industry."
A Medical Madoff: Anesthesiologist Faked Data in 21 Studies
Article appeared in Scientific American in March 10, 2009
"Over the past 12 years, anesthesiologist Scott Reuben revolutionized the way physicians provide pain relief to patients undergoing orthopedic surgery for everything from torn ligaments to worn-out hips. Now, the profession is in shambles after an investigation revealed that at least 21 of Reuben's papers were pure fiction, and that the pain drugs he touted in them may have slowed postoperative healing.
"We are talking about millions of patients worldwide, where postoperative pain management has been affected by the research findings of Dr. Reuben," says Steven Shafer, editor in chief of the journal Anesthesia & Analgesia, which published 10 of Reuben's fraudulent papers."
FDA official: “clinical trial system is broken”
BMJ 2013;347:f6980"The clinical trial system is broken and it’s getting worse, according to longstanding Food and Drug Administration investigator, Thomas Marciniak."
“Drug companies have turned into marketing machines. They’ve kind of lost sight of the fact that they’re actually doing something which involves your health,” Marciniak says. “You’ve got to take away the key components of the trials from drug companies.”
What's in Placebos: Who Knows? Analysis of Randomized, Controlled Trials
Ann Intern Med. 2010;153(8):532-535.
"Placebos were seldom described in randomized, controlled trials of pills or capsules. Because the nature of the placebo can influence trial outcomes, placebo formulation should be disclosed in reports of placebo-controlled trials."
Major publisher retracts 64 scientific papers in fake peer review outbreak
"Made-up identities assigned to fake e-mail addresses. Real identities stolen for fraudulent reviews. Study authors who write glowing reviews of their own research, then pass them off as an independent report. These are the tactics of peer review manipulators, an apparently growing problem in the world of academic publishing.
Peer review is supposed to be the pride of the rigorous academic publishing process. Journals get every paper reviewed and approved by experts in the field, ensuring that problematic research doesn’t make it to print.
But increasingly journals are finding out that those supposedly authoritative checks are being rigged.
In the latest episode of the fake peer review phenomenon, one of the world’s largest academic publishers, Springer, has retracted 64 articles from 10 of its journals after discovering that their reviews were linked to fake e-mail addresses. The announcement comes nine months after 43 studies were retracted by BioMed Central (one of Springer's imprints) for the same reason.
The blog Retraction Watch, which monitors and reports on retractions for fraud, plagiarism and other dubious practices in the academic publishing industry, says this latest announcement brings the tally of papers withdrawn for faked reviews up to 230 in the past three years. Those papers make up only a fraction of the hundreds of thousands of studies published each year, but they have the publishing world concerned."
The Peer Review Scam
Nature 515, 480–482(27 November 2014)
"In the past 2 years, journals have been forced to retract more than 110 papers in at least 6 instances of peer-review rigging. What all these cases had in common was that researchers exploited vulnerabilities in the publishers' computerized systems to dupe editors into accepting manuscripts, often by doing their own reviews. The cases involved publishing behemoths Elsevier, Springer, Taylor & Francis, SAGE and Wiley, as well as Informa"
Inappropriate manipulation of peer review
"In November last year, BioMed Central uncovered evidence of repeated and inappropriate attempts to manipulate the peer review process of several journals (see our original statement and update).
The apparent intention was to deceive Editors and positively influence the outcome of peer review by suggesting fabricated reviewers. Given the scale of the deception, we alerted COPE (the Committee on Publication Ethics) and a number of other publishers. COPE issued a statement providing further advice for publishers. PLOS also recently issued a statement here.
Although we originally found only a handful of affected published articles, a subsequent extensive and systematic search of all of our journals identified 43 articles that were published on the basis of reviews from fabricated reviewers."
Former Harvard professor Marc Hauser fabricated, manipulated data, US says
"Marc Hauser, a prolific scientist and popular psychology professor who last summer resigned from Harvard University, had fabricated data, manipulated results in multiple experiments, and described how studies were conducted in factually incorrect ways, according to the findings of a federal research oversight agency posted online Wednesday."
Prescription for a Healthy Journal
PLoS Med 1(1): e22.
Pharmaceutical industry sponsorship and research outcome and quality: systematic review
"Systematic bias favours products which are made by the company funding the research. Explanations include the selection of an inappropriate comparator to the product being investigated and publication bias."
Outcome reporting among drug trials registered in ClinicalTrials.gov.
Arch Intern Med. 1994 Jan 24;154(2):157-63."The manufacturer-associated NSAID is almost always reported as being equal or superior in efficacy and toxicity to the comparison drug. These claims of superiority, especially in regard to side effect profiles, are often not supported by trial data. These data raise concerns about selective publication or biased interpretation of results in manufacturer-associated trials."
Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials.
CMAJ. 2004 Feb 17;170(4):477-80."Industry-funded trials are more likely to be associated with statistically significant pro-industry findings, both in medical trials and surgical interventions."
HARLOT plc: an amalgamation of the world's two oldest professions
BMJ. 2003 Dec 20; 327(7429): 1442–1445.
This is a tongue-in-cheek discussion of how the process of clinical trials and publication can be misused and manipulated to promote certain outcomes.
"Through a series of blind, wholly owned subsidiaries, we can guarantee positive results for the manufacturers of dodgy drugs and devices who are seeking to increase their market shares, for health professional guilds who want to increase the demand for their unnecessary diagnostic and therapeutic services, and for local and national health departments who are seeking to implement irrational and self serving health policies."
Why Most Published Research Findings Are False
"There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of true to no relationships among the relationships probed in each scientific field. In this framework, a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance. Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true. Moreover, for many current scientific fields, claimed research findings may often be simply accurate measures of the prevailing bias. In this essay, I discuss the implications of these problems for the conduct and interpretation of research."
Essay of the Day: David Graeber on the degradation of research
This is an essay on the changing climate of research in the US and some other countries that began in the second half of the twentieth century, and how is has limited and undermined the development of radically new discoveries. While the author is speaking on research in general, including the social sciences and hard sciences such as physics, his observations apply just as much (if not more) to medical research. He makes the case that the way the system is set up, the incentives and demands that researchers face, makes it nearly impossible for new ways of doing things to be developed that are a radical departure from the current norms. This is not because funding has decreased, it has increased significantly, but has more to do with the encroachment of corporate styles of management and organization into the ways that research is conducted and funded.