Skip to content

Stanford’s junk science on organic food

October 3, 2012

We all make spelling errors.  Most of us do not put out press releases claiming scientific breakthroughs on the basis of a spelling error.

A team of researchers from Stanford published a meta-analysis of nutritional studies of organic food, reaching the conclusion that “There isn’t much difference between organic and conventional foods, if you’re an adult and making a decision based solely on your health.”

But it turns out that the study is badly flawed.  Perhaps the most serious flaw is the spelling error – the Stanford researchers do not know the difference between flavanols and flavonols.  They are distinct nutrients.  Kristen Brandt lead a Newcastle University study that also was a meta-analysis of the related literature reached exactly the opposite conclusion.  As Brandt poured over the Stanford study, she noted that Stanford researchers had confused the two – an error one might expect from a second-year undergraduate student.

Mark Bittman has a nice summary of the misleading way in which the Stanford study was reported:

If I may play with metaphor for a moment, the study was like declaring guns no more dangerous than baseball bats when it comes to blunt-object head injuries. It was the equivalent of comparing milk and Elmer’s glue on the basis of whiteness. It did, in short, miss the point. Even Crystal Smith-Spangler, a Stanford co-author, perfectly captured the narrowness of the study when she said: “some believe that organic food is always healthier and more nutritious. We were a little surprised that we didn’t find that.” That’s because they didn’t look — or even worse, they ignored.

In fact, the Stanford study — actually a meta-study, an analysis of more than 200 existing studies — does say that “consumption of organic foods may reduce exposure to pesticide residues and antibiotic-resistant bacteria.”

Since that’s largely why people eat organic foods, what’s the big deal? Especially if we refer to common definitions of “nutritious” and point out that, in general, nutritious food promotes health and good condition. How can something that reduces your exposure to pesticides and antibiotic-resistant bacteria not be “more nutritious” than food that doesn’t?

Because the study narrowly defines “nutritious” as containing more vitamins. Dr. Dena Bravata, the study’s senior author, conceded that there are other reasons why people opt for organic (the aforementioned pesticides and bacteria chief among them) but said that if the decision between buying organic or conventional food were based on nutrients, “there is not robust evidence to choose one or the other.” By which standard you can claim that, based on nutrients, Frosted Flakes are a better choice than an apple….

Like too many studies, the Stanford study dangerously isolates a finding from its larger context. It significantly plays down the disparity in pesticides (read Tom Philpott on this) and neglects to mention that 10,000 to 20,000 United States agricultural workers get a pesticide-poisoning diagnosis each year. And while the study concedes that “the risk for isolating bacteria resistant to three or more antibiotics was 33 percent higher among conventional chicken and pork than organic alternatives,” it apparently didn’t seek to explore how consuming antibiotic-resistant bacteria might be considered “non-nutritious.”

Ultimately this is, perhaps, symptomatic of a more serious symptom of in our society – a lack of even basic scientific literacy even among journalists and readers of the papers such as the New York Times.  If readers actually looked even at the abstract of the Stanford paper – and certainly at the body of the paper, the errors would become obvious. 

And what is our cultural response to this?  Well, science just isn’t very important.  Indeed, the New York Times published a serious op-ed by Andrew Hacker, a professor at City University of New York, saying that colleges should stop mandating that students know algebra.  He points to an internal study at his university: 

The City University of New York, where I have taught since 1971, found that 57 percent of its students didn’t pass its mandated algebra course. The depressing conclusion of a faculty report: “failing math at all levels affects retention more than any other academic factor.” A national sample of transcripts found mathematics had twice as many F’s and D’s compared as other subjects.

Huh?  So giving up on mathematics and science education (in the same way that American schools have largely given up on foreign language education) will make things better how? And what is his argument?

It’s not hard to understand why Caltech and M.I.T. want everyone to be proficient in mathematics. But it’s not easy to see why potential poets and philosophers face a lofty mathematics bar. Demanding algebra across the board actually skews a student body, not necessarily for the better.

How does one respond to such a sentiment?  Perhaps by noting that philosophers really need to know algebra (a valuable pre-requisite to studying formal logic, which itself is essential to understanding contemporary philosophy)?  By pointing out that a poet such as Wallace Stevens certainly benefited from algebra (in his day job at a Hartford insurance company)?  By arguing that according to the numbers (no pun intended), employment prospects for engineers may be just a wee bit better than employment prospects for poets?

No, I think all of those arguments pale before the central point – that life without any knowledge of formal (symbolic) reasoning is empty.  Algebra is an essential pre-requisite skill – to understanding basic algorithms, to understanding basic economics, to understanding basic symbolic logic, to understanding basic abstraction.  The idea of a variable and a parameter are basic. 

We could, if we wish, simply grant every 22 year-old in the land a certificate that claimed that the recipient was “highly educated.”  However, actually being educated is much better than having a piece of paper that claims one is educated.

What are the rewards for societal ignorance?  One is that we might take junk science seriously.  I say, let’s teach those Stanford doctors how to spell, and let’s teach high school students how to perform algebra.

6 Comments leave one →
  1. nzumel permalink
    October 3, 2012 11:08 am

    The argument that organic food is not better than conventional because it doesn’t have more nutrients has always irked me. Did anyone ever seriously think that organic apples have more vitamins than conventional apples? People choose organic food, as you say, because it has fewer toxins and antibiotic-resistant bacteria. The fact that researchers even bother to study the nutrient question smacks of someone trying to change the axioms out from under the reasoning system, so to speak. It’s shabby argumentation technique.

    The fact that I can make an axiom-related metaphor, and you can follow it, also sums up my argument that math (or at least mathematical/analytical/logical reasoning) needs to be taught in schools. But perhaps more effectively than it is now. I don’t follow current pedagogical theory, but I do remember the disaster that was New Math, when I was in grade school. Making math increasingly abstract makes it harder to grasp, and at the elementary school level, it needs to be concrete and practical, which it may not be, at the moment.

    Have you read this article from The Atlantic?

    A Staten Island school was able to improve student performance when they realized their students couldn’t write (even though they could read), and they were able to improve performance by teaching analytical writing skills in every class (except math, oddly enough). Of course there is the usual blow-back about how teaching classic high school essay form is more stultifying than encouraging creative writing technique, but there is certainly something to the argument that you have to be competent before you can be brilliant.

  2. October 3, 2012 1:51 pm

    Wow. Even popular (non-scientific) websites distinguish between flavanols and flavonols:

    “Not to be confused with “flavanols,” the type of antioxidant found in dark chocolate, flavonol antioxidants are found in fruits and veggies and some beverages.”


    “fresh cocoa beans are super-rich in the type of flavonoid called flavanols (not flavOnols) which are very strong antioxidants”

    Nina, as you so eloquently point out: “that I can make an axiom-related metaphor, and you can follow it, also sums up my argument that math (or at least mathematical/analytical/logical reasoning) needs to be taught in schools.” Your argument is as compelling as Hemant Mehta’s wonderful, “Why Algebra is Necessary: Rebutting Andrew Hacker.” He sums it up with this drawing (which one can click below to see):

    My favorite bit is where Mehta gives analogies also to other disciplines beyond algebra:

    A few more notes.

    Hacker writes this:

    What is needed is not textbook formulas but greater understanding of where various numbers come from, and what they actually convey…

    It’s true that mathematics requires mental exertion. But there’s no evidence that being able to prove (x² + y²)² = (x² – y²)² + (2xy)² leads to more credible political opinions or social analysis.

    Riiight. Reading Jane Eyre and studying the periodic table doesn’t help with any of that either. Should we kick English and Chemistry to the curb, too?

    Does anyone know what that formula is? Because it’s actually pretty amazing. It’s Euclid’s formula for generating Pythagorean triples (like 3,4,5 and 5,12,13) — it even resembles the Pythagorean Theorem, c² = a² + b² — and it can be very useful if you’re studying Geometry. But if you just think of it as a series of variables without any context, as Hacker does, it’s pretty meaningless.

  3. Francesco permalink
    October 3, 2012 3:54 pm

    … Especially if we refer to common definitions of “nutritious” and point out that, in general, nutritious food promotes health and good condition. How can something that reduces your exposure to pesticides and antibiotic-resistant bacteria not be “more nutritious” than food that doesn’t?

    I guess if the meaning of “nutritious” has changed to “good for you” then this makes sense. “Nutritious”, however, means “provides nourishment”, not “good for you”. And “nourishment” comes from things like protein, carbohydrates, vitamins, etc., that people eat food to get.

    And I’ve met many people, both on the street and in my university’s agriculture department, who think that organic food has more vitamins that non-organic food.

  4. October 4, 2012 4:18 pm

    Francesco, that is interesting. I remember before the term “organic” became common, and the term “pesticide free” was used more widely. The dictionary definition of nutritious includes both “providing nourishment” and “healthful,” and I think that Bittman is using “healthful.” Still, it seems to me that there is something wrong with equating “nutritious” with “nutrients”, because (again, using Bittman’s example) most people would not say that Frosted Flakes was more nutritious than an apple, although the former could be said to contain more nutrients (essentially a multi-vitamin supplement).

    Kurk, thanks for the great links. Very funny. I do think that there is lots of opportunity to revise high school math courses (do we really need a repeated year of algebra? is trigonometry essential in high school in an era where few do surveying?)

    Nina, so many perceptive points in your comment. I loved New Math myself, but you are right — the way it was presented was absurdly abstract for most children. Have you ever seen Carl Linderholm’s book Mathematics Made Difficult? You may want to look at it in the library sometime — it is absolutely hilarious.

  5. Clayton permalink
    October 5, 2012 12:13 am

    Where can I read Brandt’s criticism of the study? All you do is link to a HuffPo piece which doesn’t link to anything she seems to have actually said. I’ve been trying hard to find her original critique but I can’t. All I see are other people repeating it.

  6. nzumel permalink
    October 10, 2012 12:59 pm

    Theophrastus — Just peeked at this comment section again. Thanks for the pointer to the Linderholm book! I will have to see if the library has it. One of the reviews says the author makes a lot of Category Theory jokes; coincidentally, my husband has been brushing up on Category Theory (which means I get to hear about it over dinner), so I’m actually almost prepared to get the jokes!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: