Quacks & Pseudoscience

[Millsy_Yiddo]

Bit of an understatement that... Yes he's almost certainly done his homework.

And he partly answers your question about types of studies...



Many issues not even mentioned here by Goldacre even. Like the point about placebos not showing side effects and studies using "active placebos" mimicking the side effects of the actual drugs sometimes showing less or no drug efficacy.

Had a bit more of a read of the article, although I'd probably need to read it another 2 or 3 times to take in all the info.

He's left out some pretty key information on the TGN1412 study. I don't agree with his conclusion that had the results been published that the 2006 study would not have happened. I do see his point that FIM studies should dose one patient at a time though (although there are some ethical questions surrounding this).

I also take issue with this paragraph;

How does this happen? How do industry-sponsored trials almost always manage to get a positive result? It is, as far as anyone can be certain, a combination of factors. Sometimes trials are flawed by design. You can compare your new drug with something you know to be rubbish—an existing drug at an inadequate dose, perhaps, or a placebo sugar pill that does almost nothing. You can choose your patients very carefully, so they are more likely to get better on your treatment. You can peek at the results halfway through, and stop your trial early if they look good (which is—for interesting reasons we shall discuss—statistical poison). And so on.

He only presents one side of the argument then uses it to suggest that all Pharmaceutical companies are corrupt/bad/evil. The only solutions to industry passing studies that he provides here are unethical ones. He fails to mention that the Pharma industry pay very intelligent and experienced people lots of money to develop drugs which pass clinical trials. That is their job and it could well be a valid reason for more of their drugs passing studies.

None of the above suggestions he makes would get through FDA review and approval for an un-approved drug in the current climate. It just wouldn't happen.

I can understand his point around the non-publishing of data which doesn't show favourable results. However he explains that this is not only an issue in industry but also academia then seems to use it as a stick to beat the Pharma industry with. I'm also not surprised that results of a study looking into the efficacy of a certain drug for a certain condition which show the drug is not efficacious for that condition do not end up published. It put that down as an unfortunate symptom of being human. I can understand why he thinks they should be published and it's definitely something that should be looked at.

I feel like I should clarify that I'm not trying to defend the Pharma industry here. I've no affiliation to it other than it pays my bills. Just pointing out that in my experience I've not seen evidence of what he's saying, particularly regarding the unethical ways he claims Pharma companies get positive results.

 

[braineclipse]

Had a bit more of a read of the article, although I'd probably need to read it another 2 or 3 times to take in all the info.

He's left out some pretty key information on the TGN1412 study. I don't agree with his conclusion that had the results been published that the 2006 study would not have happened. I do see his point that FIM studies should dose one patient at a time though (although there are some ethical questions surrounding this).

I also take issue with this paragraph;

How does this happen? How do industry-sponsored trials almost always manage to get a positive result? It is, as far as anyone can be certain, a combination of factors. Sometimes trials are flawed by design. You can compare your new drug with something you know to be rubbish—an existing drug at an inadequate dose, perhaps, or a placebo sugar pill that does almost nothing. You can choose your patients very carefully, so they are more likely to get better on your treatment. You can peek at the results halfway through, and stop your trial early if they look good (which is—for interesting reasons we shall discuss—statistical poison). And so on.

He only presents one side of the argument then uses it to suggest that all Pharmaceutical companies are corrupt/bad/evil. The only solutions to industry passing studies that he provides here are unethical ones. He fails to mention that the Pharma industry pay very intelligent and experienced people lots of money to develop drugs which pass clinical trials. That is their job and it could well be a valid reason for more of their drugs passing studies.

None of the above suggestions he makes would get through FDA review and approval for an un-approved drug in the current climate. It just wouldn't happen.

I can understand his point around the non-publishing of data which doesn't show favourable results. However he explains that this is not only an issue in industry but also academia then seems to use it as a stick to beat the Pharma industry with. I'm also not surprised that results of a study looking into the efficacy of a certain drug for a certain condition which show the drug is not efficacious for that condition do not end up published. It put that down as an unfortunate symptom of being human. I can understand why he thinks they should be published and it's definitely something that should be looked at.

I feel like I should clarify that I'm not trying to defend the Pharma industry here. I've no affiliation to it other than it pays my bills. Just pointing out that in my experience I've not seen evidence of what he's saying, particularly regarding the unethical ways he claims Pharma companies get positive results.

It might be that pharma company employees are just smarter than those working independently. I struggle to see how that is more than one of many factors at best.

I don't think he suggests that all pharma companies are corrupt/bad/evil. He's saying that with the current regulations and industry practices the oversight isn't sufficient, there is evidence of corruption and wrongdoing, but it's extremely difficult to know the extent of it in the current environment. That is scary and clearly there are better solutions available to us as a society. He's not the only one raising these issues. He's not the only one that seems very qualified at evaluating evidence that's raising these issues. You saying you've not seen evidence of it doesn't carry much weight in this kind of a conversation, and honestly if you're scientifically minded your personal experience shouldn't carry much weight with yourself either.

Yes academia as a whole struggles with publication bias and some of the issues he raises. Academia as a whole is not regulated to the extent the pharma industry is, and expectations and consequences are often much smaller in other areas of academia. This is people's health, it's a cliche, but it's true. I think it's fair to beat the pharma industry with a stick over not living up to a higher standard than linguistics or anthropology.

I'll often defend the pharma industry, as I've done above. I don't think its' a question of good/evil, but it's a question of oversight and regulation of a massive industry that has a huge influence on people's health, life span and well being. Absolutely no doubt the pharma industry has done a lot of good for the world. I'm sure we all know many people that would be dead if not for the pharmaceutical and medical science revolution.

 

[Millsy_Yiddo]

It might be that pharma company employees are just smarter than those working independently. I struggle to see how that is more than one of many factors at best.

Not just smarter (they may or may not be), but often better equipped, with more experience, more resources and more support. Of course as you say this is just one of many factors, but he only mentions the unethical factors that may or may not be in play. I see this as a little unfair and biased.

I don't think he suggests that all pharma companies are corrupt/bad/evil. He's saying that with the current regulations and industry practices the oversight isn't sufficient, there is evidence of corruption and wrongdoing, but it's extremely difficult to know the extent of it in the current environment. That is scary and clearly there are better solutions available to us as a society. He's not the only one raising these issues. He's not the only one that seems very qualified at evaluating evidence that's raising these issues. You saying you've not seen evidence of it doesn't carry much weight in this kind of a conversation, and honestly if you're scientifically minded your personal experience shouldn't carry much weight with yourself either.

Absolutely, my experience doesn't carry any weight at all. Which is why I made the previous point that I'm not defending the Pharma industry. However, having sat in on many meetings and correspondence with FDA regarding clinical studies my opinion is that it would be very difficult to get approval from FDA for a drug following completion of a clinical study with a flawed design. The author seems to suggest that this is fairly common place in Big Pharma. My experience says otherwise and therefore my opinion reflects that experience.

Yes academia as a whole struggles with publication bias and some of the issues he raises. Academia as a whole is not regulated to the extent the pharma industry is, and expectations and consequences are often much smaller in other areas of academia. This is people's health, it's a cliche, but it's true. I think it's fair to beat the pharma industry with a stick over not living up to a higher standard than linguistics or anthropology..

When I refer to academia I mean in the pharmaceutical academia. When it comes to human safety the gold standard should be based on the risk to human health i.e it's just as important for a failed study carried out by an academic to be published as it is for the same Big Pharma study, if it reveals some safety concerns populations that may come into contact with the drug. So for me its an equal playing field for Pharmaceuticals at least.

I'll often defend the pharma industry, as I've done above. I don't think its' a question of good/evil, but it's a question of oversight and regulation of a massive industry that has a huge influence on people's health, life span and well being. Absolutely no doubt the pharma industry has done a lot of good for the world. I'm sure we all know many people that would be dead if not for the pharmaceutical and medical science revolution.

It has, and on the other side, like all big corporation based industries, it is not perfect either. Maybe someday I'll sit down and read the book properly and we can re-do this discussion

 

[braineclipse]

Not just smarter (they may or may not be), but often better equipped, with more experience, more resources and more support. Of course as you say this is just one of many factors, but he only mentions the unethical factors that may or may not be in play. I see this as a little unfair and biased.

I think it's fine when it's clear from the outset what his motives for this text are. He doesn't have to go out of his way to bring up positives.

Absolutely, my experience doesn't carry any weight at all. Which is why I made the previous point that I'm not defending the Pharma industry. However, having sat in on many meetings and correspondence with FDA regarding clinical studies my opinion is that it would be very difficult to get approval from FDA for a drug following completion of a clinical study with a flawed design. The author seems to suggest that this is fairly common place in Big Pharma. My experience says otherwise and therefore my opinion reflects that experience.

You did defend the pharma industry against the claims I made earlier. My claims were not nearly as well made or as thoroughly argued as this book chapter of course, but you did defend the industry (nothing wrong with that).

Would you say a clinical study using an inactive placebo is flawed if the drug has known and fairly common side effects? I mean the blinding both for the participant and the researcher can easily be breached and it will also likely impact the placebo effect...

When I refer to academia I mean in the pharmaceutical academia. When it comes to human safety the gold standard should be based on the risk to human health i.e it's just as important for a failed study carried out by an academic to be published as it is for the same Big Pharma study, if it reveals some safety concerns populations that may come into contact with the drug. So for me its an equal playing field for Pharmaceuticals at least.

I agree that academia (and others) should also be under pressure to make their research more public. Which Goldacre also states explicitly.

It has, and on the other side, like all big corporation based industries, it is not perfect either. Maybe someday I'll sit down and read the book properly and we can re-do this discussion

I think in the interest of intellectual honesty one should familiarize oneself with the critics of one's field. Particularly if it's someone respected making well reasoned arguments like in this case.

 

[Millsy_Yiddo]

I think it's fine when it's clear from the outset what his motives for this text are. He doesn't have to go out of his way to bring up positives..

I disagree. I think it's important in any critical text to at least mention the other side of the argument. Then come to a conclusion and explain why you sit on the other side. He's almost done what he accuses the Pharma industry of doing, only instead of only presenting the good things, he only presents the bad.

You did defend the pharma industry against the claims I made earlier. My claims were not nearly as well made or as thoroughly argued as this book chapter of course, but you did defend the industry (nothing wrong with that)...

I didn't articulate myself particularly well there. I did of course as you say defend the industry, what I meant was I didn't defend it because I have a particular affiliation to it. It pays my bills and no more, if I seen the things that Golacre suggests go on happening, I would have no issue saying so.

Would you say a clinical study using an inactive placebo is flawed if the drug has known and fairly common side effects? I mean the blinding both for the participant and the researcher can easily be breached and it will also likely impact the placebo effect...

As I've said previous. For a company to run a study like this, they submit their clinical protocol to FDA for review. Part of that review will include risk to de-blinding. If FDA feel that risk is too high they advise the company to alter the protocol to reduce that risk. If the company ignore FDA advice they will not get approval on their NDA and will likely be asked to complete further studies (or worst case receive a refusal to file) in order to satisfy FDA concerns.

I agree that academia (and others) should also be under pressure to make their research more public. Which Goldacre also states explicitly.

Agreed.

I think in the interest of intellectual honesty one should familiarize oneself with the critics of one's field. Particularly if it's someone respected making well reasoned arguments like in this case.

Honestly, at this point in my life I have many higher priorities and much less time than would allow me to do this properly. But as I said, if I ever find the time I will read the book.

 

[braineclipse]

I disagree. I think it's important in any critical text to at least mention the other side of the argument. Then come to a conclusion and explain why you sit on the other side. He's almost done what he accuses the Pharma industry of doing, only instead of only presenting the good things, he only presents the bad.

Yeah. We'll just have to disagree on this one. I don't think not presenting the other side of an argument in a book like this is comparable to not publishing study results... Not at all.

I didn't articulate myself particularly well there. I did of course as you say defend the industry, what I meant was I didn't defend it because I have a particular affiliation to it. It pays my bills and no more, if I seen the things that Golacre suggests go on happening, I would have no issue saying so.

Surprised you say working for the industry isn't a particular affiliation.

Would you expect the kind of thing Goldacre is suggesting to be visible from your position?

As I've said previous. For a company to run a study like this, they submit their clinical protocol to FDA for review. Part of that review will include risk to de-blinding. If FDA feel that risk is too high they advise the company to alter the protocol to reduce that risk. If the company ignore FDA advice they will not get approval on their NDA and will likely be asked to complete further studies (or worst case receive a refusal to file) in order to satisfy FDA concerns.

There will just about always be flaws and problems in studies like these as various risks and benefits will have to be balanced. I think it's a problem and close to self affirming to assume that if the FDA approves something there are no flaws in the study design. I just mentioned one, but others have been described.

In an ideal world the FDA would have enough power and resources to be the regulatory body you describe it as. I'm not convinced it is.

Honestly, at this point in my life I have many higher priorities and much less time than would allow me to do this properly. But as I said, if I ever find the time I will read the book.

At least you're aware that there are arguments out there you have not familiarized yourself with, made by serious people, arguments that can not be easily dismissed.

 

[Millsy_Yiddo]

Surprised you say working for the industry isn't a particular affiliation.

Would you expect the kind of thing Goldacre is suggesting to be visible from your position?.

I have access to clinical protocols, manufacturing batch records, INDs, CoAs, stability data, basically everything that is required for the release of a clinical batch (for both active drug and placebo). I have enough confidence in my own understanding to believe that if the setup of a clinical study was purposefully flawed to make positive results I would recognise that. As I said previous, I've never seen it.

There will just about always be flaws and problems in studies like these as various risks and benefits will have to be balanced. I think it's a problem and close to self affirming to assume that if the FDA approves something there are no flaws in the study design. I just mentioned one, but others have been described.

In an ideal world the FDA would have enough power and resources to be the regulatory body you describe it as. I'm not convinced it is

There are always some flaws. But there is a big difference between flaws and some of the quite frankly unethical practices that Goldacre claims go on. I have enough confidence in the FDA that they would pick up on these. They aren't perfect, but I honestly don't feel they are in the business of playing with patient safety or knowingly allowing drugs which don't actually work in the populations they are supposed to work on onto the market.

 

[braineclipse]

I have access to clinical protocols, manufacturing batch records, INDs, CoAs, stability data, basically everything that is required for the release of a clinical batch (for both active drug and placebo). I have enough confidence in my own understanding to believe that if the setup of a clinical study was purposefully flawed to make positive results I would recognise that. As I said previous, I've never seen it.

There are always some flaws. But there is a big difference between flaws and some of the quite frankly unethical practices that Goldacre claims go on. I have enough confidence in the FDA that they would pick up on these. They aren't perfect, but I honestly don't feel they are in the business of playing with patient safety or knowingly allowing drugs which don't actually work in the populations they are supposed to work on onto the market.

https://en.wikipedia.org/wiki/List_of_largest_pharmaceutical_settlements

This is the industry that's not in the business of playing with patient safety?

 

[Millsy_Yiddo]

https://en.wikipedia.org/wiki/List_of_largest_pharmaceutical_settlements

This is the industry that's not in the business of playing with patient safety?

Most of these are for off-label promotion which I'm sure you know is where sales reps promote their products for uses not approved by FDA. So we can rule any mal-practice by FDA out. It also occurs post approval, and as I keep saying, in order to get approval you have to submit clinical protocol etc etc. So I still don't see any evidence of the clinical mal-practices suggested by Goldacre here.

Sales is a completely different kettle of fish to R&D unfortunately. It is driven by money rather than science. I've heard of numbers upward of 50% of sales reps losing their jobs every quarter due to not hitting their sales targets, so they are put under incredible pressure. Unfortunately it does not surprise me that this list exists but yes, I absolutely agree that off label promotion and bribery are wrong.

 

[braineclipse]

Most of these are for off-label promotion which I'm sure you know is where sales reps promote their products for uses not approved by FDA. So we can rule any mal-practice by FDA out. It also occurs post approval, and as I keep saying, in order to get approval you have to submit clinical protocol etc etc. So I still don't see any evidence of the clinical mal-practices suggested by Goldacre here.

Sales is a completely different kettle of fish to R&D unfortunately. It is driven by money rather than science. I've heard of numbers upward of 50% of sales reps losing their jobs every quarter due to not hitting their sales targets, so they are put under incredible pressure. Unfortunately it does not surprise me that this list exists but yes, I absolutely agree that off label promotion and bribery are wrong.

I didn't bring up that list as a response about the FDA or as examples of what Goldacre is claiming (he provides his own examples), but rather about the intentions of these large companies.

Off label promotion is essentially selling the drug with no scientific basis. So you think these companies are willing to do that, but they're not in the business of playing with patient safety? That doesn't add up at all to me.

 

[Millsy_Yiddo]

I didn't bring up that list as a response about the FDA or as examples of what Goldacre is claiming (he provides his own examples), but rather about the intentions of these large companies.

Off label promotion is essentially selling the drug with no scientific basis. So you think these companies are willing to do that, but they're not in the business of playing with patient safety? That doesn't add up at all to me.

I'm saying if they were willing to play with patient safety through running purposefully flawed clinical studies they would not get that past FDA or IRB. I genuinely don't feel that there is a trend in the Pharma industry to try and 'fix' clinical study results in their favour using unethical methods. This is based on my experience and nothing more which as you've said previously is a drop in a massive ocean. You can take it on board or ignore it, entirely up to you.

I understand your point that from the outside it doesn't look like it adds up. However, I have no experience in sales so I can't comment on the basis of off label promotion. I wouldn't feel comfortable either defending or not defending an area of the industry that I have never worked in. If I had worked in sales I would give you my informed opinion, but I haven't.

 

[braineclipse]

I'm saying if they were willing to play with patient safety through running purposefully flawed clinical studies they would not get that past FDA or IRB. I genuinely don't feel that there is a trend in the Pharma industry to try and 'fix' clinical study results in their favour using unethical methods. This is based on my experience and nothing more which as you've said previously is a drop in a massive ocean. You can take it on board or ignore it, entirely up to you.

I understand your point that from the outside it doesn't look like it adds up. However, I have no experience in sales so I can't comment on the basis of off label promotion. I wouldn't feel comfortable either defending or not defending an area of the industry that I have never worked in. If I had worked in sales I would give you my informed opinion, but I haven't.

Your trust in the FDA is much greater than mine. Somewhat off topic, but this is an oversight organization that's not even managed to stamp down on homeopathy. I struggle to see how the FDA and politicians who give them instructions (and appoint some people to the FDA) aren't influenced by the vast sums of money the industry spends on lobbying.

 

[Millsy_Yiddo]

Your trust in the FDA is much greater than mine. Somewhat off topic, but this is an oversight organization that's not even managed to stamp down on homeopathy. I struggle to see how the FDA and politicians who give them instructions (and appoint some people to the FDA) aren't influenced by the vast sums of money the industry spends on lobbying.

I think to a large extent they dropped the ball on homeopathy as it had such a small impact until relatively recently when there was a pretty explosive growth in the industry. That's the FDA party line anyway, again I've no experience in homeopathic medicines either (thankfully) so can't really comment on how true or not true that is. As far as I can see the wheels are starting to turn now on tougher regulations.

The FDA definitely come under heavy pressure from government, Pharma companies, patient groups and various other places.

Historically though, in cases where there are doubts/concerns over a new drug, FDA are more often investigated by congress if they approve the drug than if they reject it. In an organisation very keen to maintain its reputation for protecting public health this makes the decision to approve controversial drugs potentially very damaging if found to be down to anything other than science.

Every API on the planet that is ingested in to a human being carries a risk. So the decision that a drug is safe for use is basically an informed opinion based off the information provided at the time of review. This is why the guidelines for what will be accepted in a clinical protocol are strict and also why FDA almost always ask for more information during review (I've never had an NDA submitted which is approved without lengthy requests for further information).

This decision is undertaken in an environment where FDA are constantly pressured to approve drugs more quickly (by government, Pharma companies etc etc), yet absolutely destroyed for approving a 'bad' drug too quickly. As you know, the damage from approving a bad drug which causes death or the likes is very very difficult, in fact almost impossible to undo.

Basically the FDA try to find a balance between approving drugs quickly enough to satisfy speed of approval pressure whilst avoiding approval of anything that might cause serious harm. In my opinion and experience with FDA they always put more weight on the cost of approving a 'bad' drug too quickly than they do on the benefit of reducing the pressure placed on them by approving a drug more quickly/without adequate info. In my experience, they pay pretty much no attention whatsoever to the requests of the industry in terms of speed of approval, even in cases where company has paid for expedited review.

 

[Bullet]

Good debate chaps, very interesting and informative.

I'd like to add a tangential point, not meaning to derail things and not really on the same exact topic. But anyway, it is easy to sit here and say the FDA should be more careful and should not approve unsafe drugs, yet if I was diagnosed with pancreatic cancer and started to slide quickly downhill, I'd be begging to be given some new unproven miracle drug as a last chance shot in the dark before flying off to Geneva.

I know that isn't what is being debated, just popped into my brain sac

 

[braineclipse]

Good debate chaps, very interesting and informative.

I'd like to add a tangential point, not meaning to derail things and not really on the same exact topic. But anyway, it is easy to sit here and say the FDA should be more careful and should not approve unsafe drugs, yet if I was diagnosed with pancreatic cancer and started to slide quickly downhill, I'd be begging to be given some new unproven miracle drug as a last chance shot in the dark before flying off to Geneva.

I know that isn't what is being debated, just popped into my brain sac

Absolutely. And the risk between approving a potentially dangerous or ineffective drug has to be weighted up against the damage of unnecessarily delaying the drug approval process and leaving people to suffer or die needlessly because of the delay.

A lot of similar and related risk-benefit calculations have to be made by ethics board, companies and the FDA in a process such as this. The process could perhaps be sped up by increasing the test group size and thus providing stronger evidence quicker, but this puts more test subjects at risk of suffering under unknown side effects.

One of the more interesting points on this, since this is the quacks and pseudoscience thread after all, are the flu vaccines. Vaccine deniers and others will claim that flu vaccines aren't "properly" tested for safety and efficacy, and to an extent that's true. With a new flu strand arriving every year there's simply not time to test, manufacture and distribute vaccines if a very high bar of safety is expected. But the overall benefit of the vaccine is still clearly positive on a societal/group level so it becomes a very strange moral argument to argue against the vaccines as the consequences of not having the vaccines would be much worse than developing the vaccines with the current safety standards.

 

[Bullet]

Unless the new vaccine made babies heads shrink. Someone told me today that a new treatment costs between $1.4billion and $4billion to bring to market. Crazy money. They have a 20 year patent but spend 4to 12 years of that gaining approval from the FDA. So only a few years to milk it before rivals appear

 

[Notmegov]

Absolutely. And the risk between approving a potentially dangerous or ineffective drug has to be weighted up against the damage of unnecessarily delaying the drug approval process and leaving people to suffer or die needlessly because of the delay.

A lot of similar and related risk-benefit calculations have to be made by ethics board, companies and the FDA in a process such as this. The process could perhaps be sped up by increasing the test group size and thus providing stronger evidence quicker, but this puts more test subjects at risk of suffering under unknown side effects.

One of the more interesting points on this, since this is the quacks and pseudoscience thread after all, are the flu vaccines. Vaccine deniers and others will claim that flu vaccines aren't "properly" tested for safety and efficacy, and to an extent that's true. With a new flu strand arriving every year there's simply not time to test, manufacture and distribute vaccines if a very high bar of safety is expected. But the overall benefit of the vaccine is still clearly positive on a societal/group level so it becomes a very strange moral argument to argue against the vaccines as the consequences of not having the vaccines would be much worse than developing the vaccines with the current safety standards.

And thus why the Flue vaccine for 2015 was a complete failure.

http://www.bbc.co.uk/news/health-33967825

 

[milo]

This is interesting

http://digest.bps.org.uk/2016/02/why-is-it-so-hard-to-persuade-people.html?m=1

 

[milo]

How To Have An Opinion Worth Hearing

http://quillette.com/2016/03/03/how-to-have-an-opinion-worth-hearing/

 

[milo]

Wikipedia founder calls alt-medicine practitioners “lunatic charlatans”

http://arstechnica.com/science/2014...lt-medicine-practitioners-lunatic-charlatans/