6. How to Read Weight Science Critically

In most classrooms, Science is taught as a way to determine fact from fiction. However, evidence shows that Science just like any other series of claims made by humans is always going to be culturally situated. While this has been well understood outside of science for over 50 years, the myth of scientific objectivity continues today. This does not mean we cannot gain any useful information from scientific processes or professional scientists. However, just like any other idea that is put forth, we need to consider those claims critically within the context that makes it possible to think about them and believe them. To be able to understand and evaluate the usefulness of scientific claims, we will want to consider specific histories of different kinds of claims, patterns and debates within a subfield, and both the broad and specific cultural values surrounding the people and institutions that produce the claims. In the appendices there are materials for a deep dive into critical science literacy with examples including one on weight science.

Here we briefly describe how we can use one tool of critical science literacy – critically reading primary scientific articles – to evaluate weight science research on the “weight-loss” drug, Wegovy. We put weight-loss in quotations because Wegovy is simply a higher dose of a diabetes drug Ozempic in which the pharmaceutical company found that high doses could cause the side effect of weight loss. In 2023, Novo Nordisk, the pharmaceutical company that makes and sells Wegovy, claimed in a press statement that their drug reduced negative heart related effects such as heart attacks by 20% in overweight and ob*se adults in a recent study. The press release was taken up by major news organizations around the world despite the fact that the actual study had not been shared with the journalists or gone through a peer-review process to be published yet. The study was published months later; long after the initial press release had increased Novo Nordisk’s profit.

When we look at the study carefully, we find that while they claim a 20% difference between the group taking the drug and the placebo group, the difference found was 6.5% vs. 8% of cases of negative heart related outcomes between the two groups. A difference of 1.5% does not sound as impressive as 20%. Do the math to see how they manipulate the numbers to get a better headline (insert figure with calculation on this - see note from Ragen when editing). This is something that could have been pointed out and corrected by journalists but was not.

Further, by looking at the participants of the study, we see that it was a narrow group of people who they included, rather than ‘adults’ as Novo Nordisk claimed, the study included only people over 45 with existing cardiovascular disease and without Type 2 Diabetes. We always have to check the definitions used by researchers as it may not be the definition commonly used by everyday people. This is an extreme example of misleading statistics by redefinition of common terms, in this case the term “adult.” Questions should be raised about how and why a particular sample was chosen because if you break people into enough different groups, you will statistically find something that looks meaningful eventually in some group. While statistics is supposed to account for the difference between meaningful differences and random effects, if you do not include all of the different groupings that were tried in previous trials, the random effect can appear meaningful (develop a figure on this too).

When they published the weight loss results of the same trial group, it turned out that by the end of the trial that began with over 8,000 participants in each group, there were about 150 in each group still participating (see figure below – we will add highlighting to show relevant section; also see Ragen’s comment on clarifying the two studies from the same trial when editing).

Fig.1: Percentage change in mean body weight from baseline through week 208 for all patients in-trial and first on-treatment.

Source: https://www.nature.com/articles/s41591-024-02996-7

While this was visible when you closely looked at some of the published figures, the authors did not address this in the text and obscured it under a figure that ignored the huge change in the number of participants. This raises many questions about the final data and the missing data from those who left the study which is the vast majority of participants. Given Novo Nordisk's assertion that people have to stay on this medication for the rest of their lives in order to maintain weight loss, it is very important that only about 10% of participants managed to stay on the drug for even four years. While there is significant evidence showing that people regain weight after ceasing the drug, there is no research showing that even if people are able to stay on the drug across their life span that they would maintain the weightloss–that is a (profitable) gamble that Novo Nordisk is making with higher weight people’s lives. Even if some number of people do lose weight, at least temporarily, the long-term side effects of taking the drugs at the weight loss dose can be serious, even fatal, and are not well studied. Further, the use of these drugs for weight loss has led to shortages that have impacted the access to the drug for many people with Type 2 Diabetes, many of whom are either contraindicated for other drugs or could not achieve glycemic management with other drugs.

The analyses shared in this section on weight and health are the critical pieces needed for us to understand why this study was conducted in the first place. We should be asking questions like the following: 

• Why did the media report on the study as they did, and why it was eventually published in top journals? 

• Why do you think journalists would uncritically cover this news? 

• What cultural values do you think impacted the existence of this study in the first place and its ability to make news? 

• Who benefits from this kind of study? 

• Who might be harmed by this study and how? 

• Who funded the study and to what level were they involved in the design and analysis? 

• How many of the study authors had conflicts of interest?

For more detail on this see our critical science literacy lesson and/or read the following analyses by Ragen Chastain from before the study was published as well as deeper dives once the study was published and when more data from the study was published the following year.