Why does this study claim the pandemic had no lasting impact on mental health?

Whether you regularly keep up with medical news or simply spend a lot of time scrolling through social media, chances are you’ve heard about the controversial study published in The BMJ that claims the pandemic has had no major long-term mental health effects.

The study was published on March 8 and quickly came to the awareness of the general public through mainstream media coverage, most notably a BBC article and a tweet promoting it that featured the headline “Mental-health crisis from pandemic was minimal, study suggests”.

The study has faced enormous backlash. Twitter users have been posting or retweeting things they did to stave off isolation and fear during the height of the pandemic. Journalists and experts have been sounding off as well. For instance, sources like this Buzzfeed article (which also features some of those reaction tweets) have pointed out that the WHO has stated that the pandemic increased cases of anxiety and depression by 25%.

Even the BBC article seems skeptical. A majority of its contents are facts and statistics about the pandemic’s mental health impact, not the opposite -- maybe a quiet equivalent of those reactionary tweets?

Most, if not all, of us know that that pandemic was hard on mental health. Even if you made it through relatively unscathed, how did your children (or kids you might know) cope with lockdown and not being in physical contact with friends and teachers? How did people you know deal with not being able to visit loved ones who were hospitalized, or grieve those who passed away at a time when we weren’t allowed to hold traditional funerals?

It seems evident that the pandemic had a massive impact on the mental health of a significant portion of the population. So why does this study say the opposite?

The study was conducted by The DEPRESSD Project, a group of doctors and students mostly based in Canada. Their goal seems to be to study depression screenings. The articles on their website’s blog often focus on accuracy in reporting.

Their own study wasn’t a survey or poll, but rather an AI-aided search of research about mental health published from the start of the pandemic until now. The sources were then selected based on criteria such as whether or not there were follow-ups, with an end total of 134 sources being used for the study. The idea was that this would show how specific findings had changed (or not) over the course of the pandemic.

Interestingly, while DEPRESSED appears to advocate for data accuracy, their own report seems strangely incomplete. They admit, for instance, that their sources mainly come from high-income European and Asian countries. This leaves out an enormous swath of the population that was impacted by the pandemic.

Additionally, some experts have pointed out that the study didn’t include research on children. Young people are considered one of the groups whose mental health was most strongly impacted by the pandemic.

It’s almost impossible for most studies to cover every country and group around the world, so it’s understandable, in a way, that this study didn’t do that. And when you read it, the researchers are open about certain groups being excluded. So maybe it’s the fault of the press, taking a complex conclusion and turning it into a deceptively simple headline. But then again, you might ask, what’s the point of publishing these essentially incomplete findings, anyway? Or, at least, why not publish them with a clear title about the population that was really studied?

Another issue is whether or not AI might have been the best tool to use for determining mental health at all. AI can be an immensely helpful tool for researchers, allowing them, for instance, to search thousands or even millions of documents and records for key terms. But AI is not human and can’t understand things like irony and particular contexts. Is it possible that some sources included in this study might have been misunderstood or misinterpreted by AI?

What about the AI’s more recent findings? It’s not as “trendy” or normalized now to post content about the pandemic and its effects, even though so many of us are still experiencing them in so many different ways. So is it really possible to gauge mental health if it’s not being discussed as much?

It’s also possible that societal pressure, or in some cases, even government censorship, made real data and honest opinions unavailable.

For some experts, though, the misinterpretation may also be on us. In an editorial about the study’s findings, DEPRESSD’s Dr. Sarah Markham writes that in a way, all of those strange things we did during the height of the pandemic are signs of good coping skills and resilience.

She uses her editorial to call out previous studies for inaccuracy, writing:

It is regrettable but predictable that the media have generally reported the findings of poor quality studies as evidence that we are experiencing a universal mental health crisis.4 Sensationalising and exaggerating human suffering is not helpful to anyone and can be harmful—especially when combined with other forms of misinformation and bias that influence policy and other decision making. If we are to improve mental health and wellbeing we need accurate information to identify, care for, and treat those in need.

Markahm never specifically names any of these “poor quality” studies, and one wonders which ones she means, since so much research on the impact of COVID has been published -- many from reputable sources like the WHO.

And regardless of the lesson Markahm wants us to take from DEPRESSD’s study, personal experience certainly shows that the pandemic has changed most of our lives in a significant way.

A few days after DEPRESSD’s study was published,for instance, this one, from a team at Columbia University's Mailman School of Public Health was also released. It focuses on mental health issues caused by the pandemic among teens aged 13-18. You may not be surprised that its researchers found that the “COVID-19 pandemic has long-lasting effects on adolescent mental health and substance use.”

The COVID-19 pandemic was - and continues to be - at once a shared experience and very personal. Each of us were impacted by it in a myriad of ways. But as articles like Buzzfeed’s feature, as well as general reactions on social media, show, a significant number of us still don’t feel okay. Is this just misunderstood resiliency, as DEPRESSD’s researchers would have us believe, or something more?

Whatever the case, there is one good, indisputable thing: As the end of the Buzzfeed article reminds us, mental health resources (including many that are free) are fortunately available for many of us who need them.

Original image source

Contact Our Writer – Alysa Salzberg