Mental Illness Treatment

Bias in Online Stories of Psych Med Side Effects

the inherent bias in online reports of side effects - are they generalizable?

You can find all kinds of stories online of people reporting negative experiences with different forms of psychiatric medication. However, there’s inherent bias in that kind of reporting of side effects, so you really can’t use that information to effectively generalize how safe or effective a treatment is. In this post, we’ll look at where that bias comes from, and why it’s not a great idea to base your treatment decisions on what people say online.

Let’s talk about the negative!

People who have positive effects from treatments are less likely to be motivated to write about them. Are you more likely to contact a company to give good, or even neutral, feedback, or rant about a problem? The motivation is far stronger to rant. If we’re considering people coming off of antidepressants, who’s going to bother writing about having no issues? It’s the people experiencing brain zaps who are going to be motivated to get loud. Those negative experiences are totally valid, but it gives a very skewed picture of what the average experience is.

With the stigma surrounding treatments like psych meds and electroconvulsive therapy (ECT), people are even less likely to be motivated to write about their positive experiences. After all, if they do, they’ll probably be met with comments that medications are toxic or ECT causes brain damage. That’s a lot to sign up for just to share the positive side of the coin. On the other hand, people who share negative experiences are likely to be met with a much more welcoming reception.

The problem with individual reports

Individual case reports are considered a very low level of scientific evidence, not because that experience isn’t valid, but because it’s very hard to tell what’s causing what when you’re looking at just one person. Consider the link anti-vaxxers believe exists between vaccines and autism. Autism often starts to show up around the age that kids are getting their childhood vaccines. That doesn’t mean that vaccines cause autism or have anything to do with autism, but you can’t know that based only on individual case reports.

Self-reports can also miss certain information that’s either unknown or not recognized as relevant. Someone might report gaining or losing weight since they started on a medication, but it may have nothing to do with the medication and everything to do with them being hypo/hyperthyroid. Or someone may have only been taking their medication sporadically to begin with, and as a result, they were seeing more side effects than therapeutic effects.

In general, people aren’t particularly good historians; it’s very easy to mix up sequences of events and durations of experiences. This is why I find bullet journalling so beneficial; I would never be able to give anywhere near as accurate an account of things as my bullet journal can.

Another natural tendency is to make attributions based on beliefs more than events, as we seek out things that affirm our beliefs (known as confirmation bias). Let’s say I start a new medication and I get a migraine a few days later. Whether or not I associate those two in my mind will have a lot to do with my beliefs about the likelihood of medications causing side effects. I don’t tend to be prone to side effects, so I probably wouldn’t attribute the migraine to the medication, but someone else might be absolutely certain that the migraine was caused by the medication. Which of us is more likely to be correct? You can’t tell that from our individual experiences.

The role of research

That’s where research comes in. Studies are designed to compensate for normal human failings (and there are a lot of those). In the case of the medication and the migraine, I’d want to see a blinded, placebo-controlled trial, where no one knows if they’re being given the med or the placebo. By comparing the number of headaches that come up in the treatment group vs. the placebo group, you’ll start to get a clearer picture of whether the drug and the migraine are linked or if it’s just coincidence. There’s really no way to know that without running that clinical trial.

Research data matters not because of some medical/scientific supremacy, but because solid data from a well-designed study can support making conclusions about the phenomenon being studied that weaker data just wouldn’t be enough to support.

Unfortunately, not all scientific papers are created equal. There’s a well-established anti-mainstream-psychiatry fringe, with Dr. Peter Breggin being one of the louder voices. The anti-psychiatry fringe has generated quite a few papers, and they reference one another and it all sounds fairly legitimate on the surface. Depending on exactly what you’re searching for, Dr. Google may well serve up one of these papers, and red flags aren’t necessarily going to go off for people who aren’t used to reading scientific papers.

Getting back to individual case reports, where they can be really important, and why they’re published in scientific journals, is in identifying problems that aren’t well-recognized, and perhaps tend to be on the rare side. If the same problem keeps cropping up, that’s a good indicator that someone needs to do design a study to look into the matter further.

The rare looks common

Let’s say side effect X occurs in less than 1% of people who take a medication Y. That doesn’t sounds very common. But if a lot of people are taking a medication, that adds up very quickly. The other 99% have no reason to go online and get loud about not experiencing side effect X, which they may not have even heard of. But let’s say most of that 1% of people who take drug Y and get side effect X are motivated to get online and talk about it. If you Google side effect X and drug Y, you’ll come across the people talking about having it. It sounds common, but it still occurs in less than 1% of people. If you have a serious illness, a 1% chance of side effect X may well be a risk you’re very willing to take.

That’s where research can give you more information. It looks at what side effects each study participant did and did not have. That’s how they arrive at the figure that it only occurs in less than 1% of people.

Individual experiences are valid, but…

Whether it’s medication or ECT or any other form of treatment, everyone’s experiences are equally valid, whether they’re positive or negative. Where it gets problematic is if people start to generalize those experiences. Even if there are multiple individual stories of similar side effects, those are still individual stories that give you no reliable information about the overall safety or effectiveness of a treatment. That’s not because the stories don’t matter; that’s just the wrong kind of evidence to use to draw that kind of conclusion.

Do you tend to draw conclusions about medications, etc., based on people’s stories you read online?

book cover: Psych Meds Made Simple by Ashley L. Peterson

Want to know more about psych meds and how they work? Psych Meds Made Simple is everything you didn’t realize you wanted to know about medications.

It’s published by MH@H Books and available on Amazon and Google Play.

30 thoughts on “Bias in Online Stories of Psych Med Side Effects”

  1. I am a researcher because I am paranoid about trying new medication. And to add more than one at a time….whoa that almost threw me over the edge. When I get a new medication I research everything I can find about it. My favourite website is drugs.com. You can put in all your medications, it tells you which ones will potentially react with each other and how and it breaks down the medication by potential side effects (which are common and which are rare) and all the different things its used for.

    I try to stay away from reviews though because I do agree that most of the people posting are only posting negatives (although I should have believed what I read about antipsychotics because they do make me gain tons of weight). If I read most of the comments I’d probably never even try half of the medications I’m on at the moment.

  2. Unfortunately, far too many studies are sponsored by the drug companies and sometimes even written by them. Reliable sources of information on both sides can be hard to come by.

    1. Peer-reviewed academic journals mostly contain academic rather than industry research. They require authors to list their affiliations and disclose any potential conflicts of interest.

  3. Dr Rashid Butner said that there are no such things as side effects. There are only effects. If something is effecting you, it’s an effect. I was on sertraline for a while and it wasn’t for me. I did however experience some of the listed side effects. If you drink water, when your bladder has too much to hold, you have to urinate. Is this a side effect? The pharmacist said to me that medication is prescribed on the balance of pros vs cons and if the pros clearly outweigh the cons then the prescription is given. In the case of a closer comparison, secondary or specialist opinions are asked. The only thing I have noticed is that there are just so many different medications for mental illness that it’s a lottery what works for you as there is still a long way to go for the illnesses being fully understood.

  4. My psychiatrist advised me to become an expert in my bipolar and read and research everything that I can, relatable or not, so I can understand. As I did medicinal Chemistry at university, I started by looking at the medication that is used for bipolar and it’s mode of action to work backwards to move forwards. The only problem is that the understanding of how valproate works is like my own head 99% of the time, cloudy 😂. I have found that both valproate and lithium are not actually a cure for bipolar but they are simply just a remedy. The use of antidepressants prior to the diagnosis of bipolar for patients who are initially diagnosed with depression, exasperates their mood swings which poses a question of what damage this is potentially doing? I’m uncovering more questions as I’m researching and getting deeper into the subject. I’m not going to be a ground breaker for mental illnesses but I am hoping that I can help. As I said, I have my degree so I am documenting everything that I’ve done with proposals and justifications for my conclusions. It’s my hope to be a link between the people that have it and the people who haven’t so maybe one day nobody has to suffer as I have 🤞

  5. In the UK we have the Yellow Card Scheme to report positive or negative effects from medication. For anyone who from the UK that hasn’t heard about it, it’s website is https://yellowcard.mhra.gov.uk and there is an app too. Do you have anything like this in America?

      1. I had 3 epileptic type seizures whilst on sertraline and tramadol which I’ve reported. It seems that it has been noted but at the time it was dismissed as a possibility. The more we can constructively post and put out there for people to see and acknowledge, the better it is.

  6. I always appreciate your posts about medication as it’s something I am personally unsure about.

    I’ve read articles stating that antidepressants don’t work much better than a placebo, but I don’t know much about the validity of the study of course. Either way I’m conflicted!

    1. There was a big meta-analysis done a couple years back that concluded that yes, antidepressants do work better than placebo.

      I think where a lot of confusion comes in is that in milder cases of depression, there’s less benefit from antidepressants, but the more severe the illness, the more likely it is that antidepressants will be beneficial.

      There’s also no way of predicting whether a given individual is going to benefit or not from a given medication. The only way to know that is to try it.

      1. That’s good to know, I’ll have a read of the post 🙂

        You’re right the only way to know is to try, I might finally relent as my anxiety is getting a bit out of hand and it’s affecting my mood. Thanks for the info.

      2. I wish my 1st psychiatrist assessed me properly. I was under-medicated dosage-wise and endured wasted years of clinical depressive episodes and chronic nightmares and other ptsd symptoms.

          1. Yep, I did try advocating for myself with regards to depression (i used the Phq 9) for a long time but apparently because I could smile and laugh and work, it wasn’t valid.

  7. I was considering taking medication for anxiety at one point. But when I read people’s testimonies online I got nervous and decided not to pursue medication as an option. You are spot on – people’s individual, negative experiences can totally influence whether someone seeks out medication or not.

    1. And unfortunately, science isn’t yet at a point to be able to accurately predict how any given individual is going to respond (positive or negative) to a given medication.

  8. I don’t go so much by what I read online, especially if it’s only personal accounts and not well-researched information. I do tend to trust perceptions of mine in real-life associations with people who tell me they’re on certain medications. One does tend to draw conclusions that way after a while, though perceptions are subjective.

Leave a Reply