Archive for category Uncategorized
The rarest of birds—a reproducible result from a scientific study
Posted by mark in science, Uncategorized on February 3, 2014
In The New York Times new column Raw Data, science writer George Johnson laments experimenters
“ways of unknowingly smuggling one’s expectations into the results, like a message coaxed from a Ouija board.”
– Science Times, 1/21/14
This, of course, leads to irreproducible findings.
As a case in point, only 6 of 53 landmark papers about cancer found support in follow up studies, even with the help of the original scientists working in their own labs, according to an article in the Challenges in Irreproducible Research archive of Nature cited by Johnson.
That is discouraging but I am not surprised. I feel fairly sure that the any assertions of import get filtered very rigorously until only ones that reproduce reliably make it through.
The trick is to remain extremely skeptical of initial reports, especially those that get trumpeted and reverberate around the popular press and the internet. Evidently it is human nature to then presume that when an assertion is repeated often enough then it must be true, even though it has not yet been reproduced. Saying it’s so does not make it so.
Boo Yahoo for breaking bad on my MAD
Posted by mark in sports, Uncategorized on November 20, 2013
“Once is happenstance. Twice is coincidence. Three times is enemy action!”
— Ian Fleming
, Goldfinger
A simple, yet effective, measure of forecasting prowess is the mean absolute deviation (MAD). Yahoo’s computer projections for fantasy football do poorly on this measure. For example, one of my teams is thus far, through the first 11 weeks of this season, at 16 points MAD from an average projection of 70 per game. That’s an error exceeding 20%! But to make matters far worse, their forecast on this team is terribly biased. Given my indignation you can guess which way Yahoo has been erring (yes, I am a loser)—consistently over-estimating how points my players actually accumulate. Enough data has come in to make this statistically significant as indicated by the confidence interval on the margin of error (MOE) being below zero. Between my fantasy team and the Vikings it’s hard to say which is doing worse at underachieving. Thank goodness for the Minnesota Gopher gridioners exceeding all expectations. That is a ray of sunshine in a gloomy Fall for a football fanatic like me.
Quants and nerds bring science and reason to the dark fortress of superstition
Posted by mark in sports, Uncategorized on November 2, 2013
Alison Gopnik, The Wall Street Journal’s “Mind & Matter” columnist, goes a bit over the top today while paying homage to baseball’s statisticians. But one must be mindful that she teaches at U Cal Berkeley—less than 15 miles from the home field of the Oakland Athletics and “Moneyball” wizard Billy Beane. At the other end of the country the Boston Red Sox rule supreme in Major League Baseball in large part to calculations by their adviser Bill James—inventor of sabermetrics: the empirical analysis of baseball, especially statistics that measure in-game activity.
However, BoSox hero (one of many!) Jonny Gomes, who got a lot of disrespect for his measures—yet came through in the clutch, came back with this shot in an on-field interview with FOX Sports’ Ken Rosenthal after the clincher at Fenway Park:
“There’s a lot of sabermetrics, there’s a lot of numbers and stuff. The whole WAR stat. But when you go to playoffs, you want me to go to war with.”
WAR stands for Wins Above Replacement. The Red Sox led MLB on this statistic as shown here. Gomes only rated a bit over 1 on WAR. A “solid starter” should achieve a WAR of 2 or more according to this white paper by Boston’s Yawkey Report.
It’s hard to argue with success, but take that Jonny!
Little correlation between pay and how meaningful you find your work
Posted by mark in Consumer behavior, pop, Uncategorized, Wellness on September 1, 2013
The August 26 issue of Business Week features this chart on median salary versus job meaning developed by salary comparison site PayScale. See if your profession is listed and, if so, how your colleagues rated their work.
I find it interesting that one of the lowest paying jobs—water treatment plant operator—came in at 100 percent self-rating of high job meaning. On the other hand, a securities trader makes twice the pay but only 14 percent felt their work meant much.
Neurosurgeons come out tops on both counts—salary and meaningfulness. That takes brains getting into a position like this. ; )
One of the least-paying jobs listed by PayScale is gas station attendant—it is also, evidently, nearly completely meaningless. It seems that a person stuck with this work would do well by becoming a dog kennel worker: The pay is about the same but carers for canines rates their job at 64% on the meaningful scale. My pet Penny (pictured sharing water with my grandson) approves. : )
Kids & Science
Posted by mark in pop, science, Uncategorized on June 23, 2013
I am heartened to hear of great work being done by current and former colleagues to get K-12 kids involved in Science, Technology, Engineering, and Mathematics (STEM). For example, Columbia Academy, a middle school (grades 6-8) in Columbia Heights (just north of Minneapolis), held an Engineering and Science Fair last month where two of our consultants, Pat Whitcomb and Brooks Henderson, joined a score of other professional engineers who reviewed student projects. Winners will present their projects this summer at the University of Minnesota’s STEM Colloquium.
Also, I ran across a fellow I worked with at General Mills years ago who volunteers his time to teach middle-schoolers around the Twin Cities an appreciation for chemistry. He makes use of the American Chemical Society (ACS) “Kids & Chemistry” program, which offers complete instructions and worksheets for many great experiments at middle-school level. Follow this link to discover:
– Chemistry’s Rainbow: “Interpret color changes like a scientist as you create acid and base solutions, neutralize them, and observe a colorful chemical reaction.”
– Jiggle Gels: “Measure with purpose and cause exciting physical changes as you investigate the baby diaper polymer,* place a super-absorbing dinosaur toy in water, and make slime.”
– What’s New, CO2? “Combine chemicals and explore the invisible gas produced to discover how self-inflating balloons work.”
– Several other intriguing activities contributed by ACS members.
Kudos to all scientists, engineers, mathematician/statisticians who are engaging kids in STEM!
*(The super-slurpers invented by the diaper chemists really are quite amazing as I’ve learned from semi-quantitative measurements of weight before and after soakings by my grandson. Thank goodness! Check out this video by “Professor Bunsen”, which includes a trick to recover the liquids that I am not going to try.)
Who foresees the future better—the Hedgehog or the Fox?
Posted by mark in pop, Uncategorized on June 5, 2013
“The fox knows many things, but the hedgehog knows one big thing.”
– Archilochus
In the June 3rd issue of Chemical & Engineering News Frederick M. Peterson, a chemical engineer who went on to achieve a doctorate in economics, dissects what happens when “Scientists Tackle Finance.” He warns against the tendency of experts in one field being overly bullish about their ability to manage things outside of their specialty. These are the hedgehogs—people who make bold predictions and happen to be right long enough that they attract a strong following.
On the other hand, the foxes, who observe many things and adapt readily to differing situations, lack confidence about the chances of any particular path leading to success. They are seen as being weak and wish-washy, which is not very popular. Nevertheless, it may not be surprising that foxes do better than hedgehogs at forecasting, according to Peterson, who cites a seminal study by the School of Business at University of California, Berkeley.
The moral of this story is to be wary of anyone who expresses too much certainty about the shape of things to come. It does not pay to follow hedgehogs—they will ultimately go beyond their narrow limits of competence and roll into something very prickly.
Statistics provide a decisive advantage to taking one route to work versus another
Posted by mark in pop, Uncategorized on April 7, 2013
See the results graphed on an experiment I just completed to decide whether to commute straight in to Minneapolis on Minnesota Highway 36 or take US Interstate 694 –a speedier, but longer, freeway bypass. Notice that the least significant difference bars do not overlap, thus providing more than 95 percent confidence that the scales tilt to one way (36) being faster–to put it simply.
For each run into work I randomly chose one route or the other based on a recipe sheet produced by Design-Expert software and timed it with a stopwatch app on my smart phone. Then I entered the results in the software and it gave me the answer I wanted.
It appears that I can save the better part of a minute by not shooting around on 694. That is good to know!
George Box–a giant in the field of industrial design of experiments (DOE)
Posted by mark in design of experiments, Uncategorized on March 30, 2013
George Box passed away this week at 94. Having a rare combination of numerical and communication skills along with an abundance of common sense, this fellow made incredible contributions to the cause of industrial experimenters. For more about George, see this wonderful tribute by John Hunter.
My memorable stories about Box both relate to his way with words that cut directly to a point:
- In 1989 at the Annual Quality Congress in Toronto seeing him open his debate with competing guru Genichi Taguchi by throwing two words on an overhead projector–“Obscurity” and “Profundity”, and then after a dramatic pause, adding the not-equal sign between them. This caused Taguchi’s son Shin to leap up from the front row and defend his father. This cause the largest crowd I have ever seen at a technical conference to produce a collective gasp that one only rarely experiences.
- In 1996 at a DOE workshop in Madison, Wisconsin enjoying his comeback to a very irritating disciple of Taguchi who kept interrupting the lecture: “If you are going to do something, you may as well do it right.”
Lest this give the impression that Box was mean-spirited see this well-reasoned white paper that provides a fair balance of praise and criticism of Taguchi, who created a huge push forward for the cause of planned experimentation for quality improvement.
The body of work by George Box in his field is monumental. It provides the foundation for all that we do at Stat-Ease. Thank you George, may you rest in peace!
Minnesota the “1st Fastest Warming State” — Whoo hoo!
Posted by mark in Uncategorized on March 15, 2013
Check out this graphic by Climate Central. It shows Minnesota being the leader for winter warming from 1970 to 2012! Unfortunately, as usual, we do not do well when at the top and so a great deal of snow fell this winter and more is forecast for next week.
Given the equinox does not come until March 20 at 11:02 GMT, I remain hopeful for Spring. I am invested in warmer weather having bought tickets to opening day for the Minnesota Twins on April 1. Perhaps that was a bit foolish, but at least I will fit in with the theme for this day. Also, I predict that our baseball team will make it to the playoffs. Mark my words.
Only 14 percent of biomedical results are wrong, after all–Is this comforting?
Posted by mark in science, Uncategorized on February 2, 2013
The Scientist reports here that new mathematical studies refute previous findings that most current published medical research findings are false due to small study sizes and bias. I suppose–considering the original assertion of “most” announced discoveries being wrong–we can literally live with a false positive rate of ‘only’ 14% for findings that relate ultimately to our well being. But the best advice is:
It is still important to report estimates and confidence intervals in addition to or instead of p-values when possible so that both statistical and scientific significance can be judged by readers.
– Leah R. Jager, Jeffrey T. Leek (“Empirical estimates suggest most published medical research is true”)