Archive for category science
Only 14 percent of biomedical results are wrong, after all–Is this comforting?
Posted by mark in science, Uncategorized on February 2, 2013
The Scientist reports here that new mathematical studies refute previous findings that most current published medical research findings are false due to small study sizes and bias. I suppose–considering the original assertion of “most” announced discoveries being wrong–we can literally live with a false positive rate of ‘only’ 14% for findings that relate ultimately to our well being. But the best advice is:
It is still important to report estimates and confidence intervals in addition to or instead of p-values when possible so that both statistical and scientific significance can be judged by readers.
– Leah R. Jager, Jeffrey T. Leek (“Empirical estimates suggest most published medical research is true”)
The amazing persistence of biased scientific results—Popeye’s spinach found fraudulent
Posted by mark in science, Uncategorized on December 5, 2012
I recently completed a series of webinars on using graphical diagnostics to deal with bad experimental data.* The first thing I focused on was avoidance of confirmation bias – hearing what you want to hear, for example in the persistence of the possibilities of cold fusion. See more cases of confirmation bias in this detailing by Peter Bowditch in Australasian Science.
I came across another interesting example of the persistence of wished-for results in a review** of Samuel Arbesman’s new book on The Half-Life of Facts. It turns out that spinach really does not delivery the amount of iron that my mother always believed would make it worth us eating this horrible food. She was a child of the 1930’s, at which time it was widely believed that the edible (?) plant contained 35 milligrams of iron, a tremendous concentration, per serving. However, the actual value is 3.5 mg—the chemist who first analyzed it misplaced the decimal point when transcribing the data from his notebook in 1870! In 1937 this error was finally corrected, but my mom never got the memo, unfortunately for me and my six younger siblings. ; )
*“Real-Life DOE” presentation, posted here
** The Scientific Blind Spot by David A. Shaywitz in the 11/19/12 issue of Wall Street Journal
A strange pink elephant — the Higgs boson
Posted by hank in science, Uncategorized on July 9, 2012
In our business we focus a lot of energy to convince experimenters they must conduct enough runs to develop the statistical power needed for detecting an effect of interest. What amazed me about the recent discovery of the Higgs boson is the sample size required to see this “strange pink elephant” as it’s described in the embedded explanatory video cartoon. The boffins of CERN took 40 million measurements per second for 20 years. These physics fellows cannot be topped for being persistent, tenacious, dogged and determined. Good for them and, I suppose, us.
“If the particle doesn’t exist, one in 3.5 million is the chance an experiment like the one announced would nevertheless come up with a result appearing to confirm it does exist.”
– Carl Bialik, ‘The Numbers Guy’ for Wall Street Journal explaining in his July 7-8 column the statistical meaning of CERN’s 5 sigma standard of certainty (see How to Be Sure You’ve Found a Higgs Boson).
Brain drizzling? Try linking instead.
Posted by mark in science, Uncategorized on May 13, 2012
Early on in my career as a chemical engineer working in R&D process development I came to the realization that many good ideas get shot down prematurely. Granted, many of these thoughts come out half baked, but in the proper environment they can lead to some very nourishing developments.
I always thought Osborn’s methods for brainstorming counteracted the quenching of creativity. However, although his approach certainly does generate a quantity of ideas, the end results never proved nearly as astounding as one might expect.
Earlier this year The New Yorker magazine debunked as Groupthink many of Osborn’s cherished tenets, such as allowing no criticism. The current thinking is that you gather a diverse group of bright people and then just let things fly with no holds barred. The fear of public humiliation forces the participants to think a bit before speaking. If they do get strong criticism, these creative people must regroup their thoughts and try again. The best idea(s) tend to win out.
Here is one minor variation on the “no rules” creativity session that I suggest: Ask that everyone come in with one idea to throw into the pot. Then let the fun begin!
Given Osborn’s rules are passé, where should you turn to next for catalyzing creativity? I recommend you consider Idea-Links. I have had the pleasure of picking the brain of the author, Jim Link—a very energizing fellow. Believe me, he really knows how to get people to think outside of the box.
“To raise new questions, new possibilities, to regard old problems from a new angle, requires creative imagination and marks real advance in science.”
– Albert Einstein
If you have other ideas on fostering creativity (or wish to criticize those already proposed), toss them into the ring. Do not be shy (nor sensitive).
Ivory towers of academia (& shiny ones in Vancouver)
Today’s Vancouver Sun suggests that a competitive university culture discourages sharing of knowledge, which then leads to the publication of many flawed and fraudulent studies. This is a rehash of issues I cited recently with the warning Beware of obvious answers and positive results. It would be great, albeit a bit boring, if journals published negative results from well-designed experiments with adequate power to see beneficial results. As my colleague Wayne Adams says
“Most of what you learn from an experiment is what NOT to do.”
PS. I took this picture Granville Island looking across to downtown Vancouver.
Marshmallows measure the speed of light (and get put to other good uses)
One of my favorite blogs, Flowing Data, provided me the heads-up on a great lecture by Adam Savage (the Mythbuster’s guy) called “Simple ideas lead to scientific discoveries”. I really enjoyed all of his stories, but especially the one on Hippolyte Fizeau’s measurement of the speed of light in 1849. Ingenious!
Coincidentally, my brother Paul forwarded me a detailing of how one can measure the speed of light with a tray of mini-marshmallows! Check it out at this Science Blog written by theoretical astrophysicist Ethan Siegel. This sharp-fingered fellow (if you view his blog you will see what I mean) goes on to tout a marshmallow-made diorama that ‘peeps’ recent claims of particles going faster than the speed of light.
That leads me to puffing up my daughter Emily, who achieved the “peeple’s choice” award in the Saint Paul Pioneer Press Ninth Annual Marshmallow Peeps Diorama Contest. She and two of her closest peeps produced The Mupeeps Take Minnepeepolis. It looks very much like the view out of my window from Stat-Ease headquarters east of downtown Minneapolis.
By the way, my favorite Muppets are Bunsen and Beaker. See them demo their invention of fireproof paper here. At the Muppets Lab one should always be prepared with fresh marshmallows on a stick. I advise going for two at a time. o——<8
Beware of obvious answers and positive results
Posted by mark in science, Uncategorized on March 4, 2012
“Most results, including those that appear in top-flight peer-reviewed journals, can’t be reproduced.”
This is a “dirty secret” revealed by the Wall Street Journal’s Gautam Naik in this December report. It cites statistics from Bayer that nearly two-thirds of published studies could not be replicated. Naik blames the complicated nature of experiments nowadays along with the “positive bias” researchers driven to produce results. Glenn Begley, vice president of research at Amgen, a biotechnology company, suggests that “academic scientists, like drug companies, should perform more experiments in a ‘blinded” manner to reduce any bias toward positive findings.”
Meanwhile, Duncan Watts, author of Everything is Obvious: *Once You Know the Answer says
“When you do the experiment properly [randomized and controlled], all the numbers go down.”
He’s speaking on the bias of marketing executives toward their own sensibilities, which often do not reflect those of the population being sold to. See what the Financial Times “undercover economist” Tim Harford says about this here. Unfortunately, in my experience, those (the analysts) who know better than to extrapolate from small, non-representative sample of opinions from the ‘powers-that-be’ (often n=1, that is—the Boss) get very little support for spending money to put these assertions to the test. Even though you know the top dogs might be barking up the wrong tree it’s easiest just to go along with the pack and press ahead. To do otherwise risks suffering a painful bite-back. Yes, I am a cynic.
Speed of light exceeded (astounding!)? Or was it measurement error?
This morning I read this NY Times news that European physicists measured neutrinos at 0.0025 percent above the speed of light. If so, it may be only a matter of time before you can send yourself a telegram to not do whatever you did that you’ve always regretted and, by the way, to please invest a thousand dollars in Microsoft, Facebook or the like (depending on the timing).
Years ago I visited Mount Wilson Observatory in California with my son Hank. See me pictured by their two domes that house 60 and 100 inch telescopes; respectively. This was the center for landmark experiments on the speed of light as detailed in this Wikipedia article. Obviously measurement error made this a very difficult.
Being a skeptic, and seeing that a similar experiment* found neutrinos whizzing about at the speed of light, but not beyond that, I was going to advise caution. However, Hank gave me the heads up to today’s xkcd cartoon (click the image to make it bigger and more readable). I think this guy has got a better idea.
*Done with a group at the Soudan Underground Laboratory here in Minnesota. They first did physics experiments there, in an abandoned iron mine, in 1980. I featured this in a retro young-adult techno/adventure/mystery/thriller called The Secret of the Wolf Ring (Amazon, Kindle Edition).
Eggs-plosion in the aftermath of Easter
Posted by mark in science, Uncategorized on April 30, 2011
I’ve become accustomed to hard-boiling an egg for my breakfast so I was pleased to see a surplus of Easter eggs after celebrating this holiday last Sunday. My dexterity for shelling eggs is not a-pealing (ha ha) so I decided to try an eggs-periment: Microwave an Easter egg just long enough to heat it up and loosen it up for eggs-traction. If I’d have quit after my first try of 15 seconds, I may’ve succeeded. But the egg just didn’t feel hot enough so I added more time. Kapow! The eggs-plosion left nothing more than a millimeter of shell intact. The uniformity of organic matter throughout the inner surface of the microwave oven was very interesting, I think – quite impressive.
Although I consider this to be a very successful experiment, it’s one that I don’t feel needs to be replicated. My colleagues at Stat-Ease have provided a number of suggestions for another eggs-periment such as boiling the Easter eggs in vinegar or baking them. This creates a looser shell, they say.
Feel free to provide other ideas, but keep in mind that disaster seems to lurk behind me whenever I try an experiment. My failures tend to be quite spectacular. But, on the other hand, that’s what makes experimentation so eggs-citing!
Kitchen pantry science – fun experiments to do at home
Several months ago I watched a neat demonstration by kitchen-pantry scientist Liz Heinecke on how to write secret messages. All you need are cranberries, water, baking soda and some paper as detailed here. Liz, a mother of three, provides many fun experiments (“simple recipes for real science”) to try at home. I think it’s a great way to get kids interested in science. However, be forewarned, she’s got a masters degree in bacteriology so some of her ideas might grow on you. ; )
This reminded me of a parent who worked as a microbiologist for the FDA. She did a show-and-tell for a Cub Scout den that I led 20 years ago. One of her items collected from FDA was a can of vichyssoise (leek-potato-onion soup traditionally served cold). It had been tested positive for botulism. We were told that if opened, this container of bacteria could sicken all of the inhabitants of New York City. After hearing this, I vowed to always boil canned soup.
Science can be as easy as baking. I want to encourage parents to open up their kitchen cabinets, stir up some science with their kids, and feed those hungry minds.
— Liz Heinecke, kitchen-pantry scientist