29 April 2012 @ 03:20 pm
Excuse me, I'm a scientist  
So, I think about the xkcd comic sometimes when I read or hear comments from people that are so far removed from reality that I feel compelled to correct them:

Except this time, it's one of my dragonboat teammates, who posted a link to a high school science fair project about plants being watered with boiled and microwaved water on her Facebook feed. Long story short: plants watered with microwaved water die! We must stop using microwaves immediately!!! FOR THE CHILDREN!

I hope it's clear that from my point of view, this is a rubbish experiment based on faulty premise and its results are inconclusive at best. Not only that, it's also followed by a series of absolutely ridiculous "facts". So ridiculous, in fact, that I didn't think there was any way someone could take this article seriously. It soon became apparent that I was wrong.

I am tempted to post this Snopes link on this article in return, but ultimately I don't think I will since I don't think I'll really change her mind, and I don't want her to lose face, either.

This is the kind of scientific illiteracy that makes me really unhappy, because you don't really need to have studied science to see what's wrong with the experiment. It helps if you have, of course, since the scientific method becomes second nature after a while, but to get you started, here's what I flagged immediately:

1) Water was presumably boiled on the stovetop in a (metal) kettle. You can't use metal in a microwave, so what kind of container was used to heat the water in the microwave, and could that have had an effect on the results?
2) Why only the single trial? Reproducibility matters.
3) What were the heating/cooling times of the water? Could the water temperature when it was poured on the plants be a factor?

Note that none of these rely on any special scientific knowledge - just scientific common sense. (I'm also being quite generous here in assuming the experimenter actually wanted to test a hypothesis rather than prove one.) Extraordinary claims demand extraordinary data, and there's clearly none of that here.

(Side note: I don't like to use microwaves, but it has nothing to do with a fear that microwave radiation secretly mutates my food. Mostly it's because we didn't have a microwave growing up - my mother never liked them - but also because I don't like the texture of many microwaved foods.)

I am kind of unsure when 'science' became a dirty word.
( Post a new comment )
[identity profile] dustthouart.livejournal.com on April 29th, 2012 10:42 pm (UTC)
You know, I was thinking about the scientific method re: parenting advice. There's a book that just came out in the "French people are better at everything" vein called something like French Kids Aren't Picky. And the story is that a woman moved to France with her two picky eater children and after living in France for a year, they weren't picky anymore. Get me an advance! I will write a book about this! BESTSELLER TIME.

Like we would ever be satisfied with a scientific experiment with two subjects. But even the most "researched" books generally deal with only one doctor's practice. The biggest numbers I've seen are in the low hundreds.

There's a movement called "evidence based parenting" (this is a website I really like) but even it is limited by the studies that have been done. And it's really tricky to do studies for a lot of these things (discipline methods, sleep approaches, etc) because #1 ethical considerations and #2 controlling for other factors. So you're left in the dark a lot anyway.
(Reply) (Thread) (Link)
[identity profile] gemtiger.livejournal.com on April 30th, 2012 05:59 pm (UTC)
I think books like that are fine as long as they don't try to sell themselves as "Here's a way to get your picky children to try more things!" Human subjects (or "participants," as we're supposed to call them now) are always tricky to deal with for the reason you've pointed out: there are just way too many variables, and you'll never get as large a sample size as you'd like.

I love Oliver Sacks' books reflecting on his experiences as a neurologist and all the interesting patients he's had over the years, but I certainly don't read them as "this is how we should treat all people with these conditions."

(Tangentially, this is actually making me feel better about my own chemistry: I might be dealing on a milligram scale but that's still something like 10,000,000,000,000,000,000 molecules.)
(Reply) (Parent) (Link)
[identity profile] lusimeles.livejournal.com on April 30th, 2012 03:57 pm (UTC)
...I am always flabbergasted when science people don't understand the basic scientific method. Maybe this comes from having gotten a degree in philosophy or whatever, but the scientific method feels SO common sense/intuitive to me that I am just like DO YOU NOT THINK WHEN YOU SORT THOSE LAB SPECIMENS??? whenever I hear certain people try to interpret certain scientific studies. *le sigh*

People in ANY discipline need to remember to formulate the theory to fit the facts, not the facts to fit the theory...

(On the bright side, looks like the commenters are on your side judging by the response to that girl's blog post.)

Edited 2012-04-30 03:58 pm (UTC)
(Reply) (Thread) (Link)
[identity profile] gemtiger.livejournal.com on April 30th, 2012 06:15 pm (UTC)
I ought to clarify: she didn't write the post, she just posted a link to it saying "something to think about." (That Snopes link I found got posted to her Facebook page by someone else soon after I wrote this post anyway, so I think she sees the light now :D)

Microwaves are easy to pick on because they're not intuitive at all to understand. It's easy to see how gas and conventional stoves transfer heat, but microwaves are (sometimes literally) a black box.

I'm going to digress here and practice my 'science for the general public' skills on you and explain how microwaves work! Microwaves heat food by emitting microwave radiation. These electomagnetic waves cause water and fat molecules in the food to rotate and move around, in essence giving them energy. The motion of the molecules is what causes the heat. See? Simple and totally not scary.

People in ANY discipline need to remember to formulate the theory to fit the facts, not the facts to fit the theory...
Reminds me of a discussion I had with a labmate last summer over elemental analysis (which finds the percentage of each element in a sample... as each substance has a unique chemical composition, it's a way to check for purity). He didn't think it was that great a technique, since what happens in practice is:
1) You get the results from the first sample. They don't match.
2) You do some more purification on your sample. You re-submit.
3) Repeat until the results from your sample match theory.

His point was that you haven't really proved anything by the end, since you just keep running this analysis until you get the numbers you want. I see his point, but it's still hard to publish without this data, so I'm going to keep running them... :P
(Reply) (Parent) (Thread) (Link)
[identity profile] lusimeles.livejournal.com on April 30th, 2012 06:38 pm (UTC)
Oh, of course! Sorry, I didn't mean to sound like I was harping on your friend. I just see people posting links like that all the time as well, or going "LOOK AT THIS NUTRITIONAL STUDY THAT SAYS CHOCOLATE IS GOOD FOR YOU, EVERYBODY GO BUY ALLLLL THE CHOCOLATE NOW," and it really makes me facepalm. I do generally follow psychology/science/nutritional news, so I see a lot of people run away with what they seem to think are very definite interpretations of very questionable data, and the philosopher in me is just like D: D: D: because AHHH empirical vertifiability, damnit! Popper! Quine! *insert meaningless philosophical venting here*

I agree that microwaves are not particularly intuitive, yes, so thank you for explaining how they work! (I am now going to steal that knowledge and try to impress people at parties with it. SEE, NOT ALL HUMANITIES MAJORS = SCIENCE ILLITERATE. See what you have done here, Catherine...)

I see his point, but it's still hard to publish without this data, so I'm going to keep running them... :P

*laughs* That seems fair! I don't know - so long as you aren't falsifying the data, I think. I don't know much about elemental analysis or even chemistry, for that matter, but I have heard horror story after horror story from other disciplines...
(Reply) (Parent) (Link)