Archive for category Uncategorized
Over-reacting to month-to-month economic statistics
Posted by mark in politics, Uncategorized on April 18, 2010
In his column this weekend the Numbers Guy at Wall Street Journal, Carl Bialik, notes* how uncertain the monthly statistics for unemployment and the like can be. For example, the Census Bureau reported that sales of new single-family homes fell to record low last month. However, if anyone (other than Bialik) read the fine print, they’d see that the upper end of 90 percent confidence interval estimates an increase in sales!
“Most of the month-to-month changes are not only nonsignificant in a statistical way, but they are often straddling zero, so you can’t even infer the direction of the change has been accurately represented.”
– Patrick O’Keefe, economic researcher
The uncertainty stems for the use of sampling as a cost-saving measure for government agencies and ultimately us taxpayers. For example, field representatives covering 19,000 geographical units throughout the U.S. only sample 1 out of 50 houses to see whether they’ve been sold.
The trouble with all this uncertainty in statistics is that it ruins all the drama of simply reporting the point estimate. ; )
*(See “It Is 90% Certain That Unemployment Rose. Or Fell.” and a related blog on “What We Don’t Know About the Economy” )
Misuse of statistics calls into question the credibility of science
Posted by mark in science, Uncategorized on March 28, 2010
The current issue of Science News features an indictment of statistics by writer Tom Siegfried. He pulls no punches with statements like this:
“…a mutant form of math has deflected science’s heart..”
“Science was seduced by statistics…”
“…widespread misuse of statistical methods makes science more like a crapshoot.”
“It’s science’s dirtiest secret: …testing hypotheses by statistical analysis stands on a flimsy foundation.”
“Even when performed correctly, statistical tests are widely misunderstood and frequently misinterpreted. As a result, countless conclusions in the scientific literature are erroneous…”
Draw your own conclusions on whether science fails to face the shortcomings of statistics by reading Siegried’s article Odds Are, It’s Wrong.
My take on all this is that the misleading results boil down to several primary mistakes:
- Confusing correlation with causation
- Extrapolating from the region of experimentation to unstudied areas
- Touting statistically significant results that have no practical importance
- Reporting insignificant results from studies that lack power to see differences that could be very important as a practical matter.
I do not think statistics itself should be blamed. A poor workman blames his tools.
Test and evaluation of the Great Panjandrum – a spectacular failure for weaponry
Posted by mark in history, science, Uncategorized on March 23, 2010
When time becomes available – mainly while I do cardio-exercise on my home elliptical, I’ve been watching a classic 26-episode BBC series on The World at War that my oldest son gave me. It’s extremely compelling – rated 9.7 out of 10 by over two thousand voters at the Internet Movie Database (IMDb).
This morning I watched the chronicle of D-Day. Being that I just returned from the Annual National Test & Evaluation Conference by the National Defense Industry Association (NDIA), it was interesting to see what the boffins of Britain invented to defeat the defenses put up along the Normandy beaches. Perhaps the most amazing device was the Great Panjandrum, a rocket-propelled cart, which according to this write-up for Wikipedia was developed by the Admiralty’s Directorate of Miscellaneous Weapons Development. (One wonders about the “miscellaneous” bit.) The clip I viewed on the spectacular failure of the Great Panjandrum can be seen (along with other incredibly-inept military devices for D-Day) in a video on British Secret Wartime Follies posted in this article by UK’s Daily Mail. Check it out!
PS. The Brits continue to come up with the most audacious inventions, such as this flame-throwing moped developed as a deterrent against derelict drivers competing for motorway lanes.
Evolutionary operation
Posted by mark in design of experiments, Uncategorized on March 7, 2010
Last December, after an outing by the Florida sea, I put out an alert about monster lobsters. This reminded me of an illustration by statistical gurus Box and Draper* of a manufacturing improvement method called evolutionary operation (EVOP), which calls for an ongoing series of two-level factorial designs that illuminate a path to more desirable conditions.
With the aid of Design-Expert® software, I reproduced in color the contour plot in Figure 1.3 from the book on EVOP by Box and Draper (see figure at the right). To illustrate the basic principle of evolution, Box and Draper supposed that a series of mutations induced variation in length of lobster claws as well as the pressure the creatures could apply. The contours display the percentage of lobsters at any given combination of length and pressure who survive long enough to reproduce. Naturally this species then evolves toward the optimum of these two attributes as I’ve shown in the middle graph (black and white contours with lobsters crawling all over them).
In this way, Box and Draper present the two key components of natural selection:
- Variation
- An environment that favors select variants.
The strategy of EVOP mimics this process for improvement, but in a controlled fashion. As illustrated here in the left-most plot, a two-level factorial,** with ranges restricted so as not to upset manufacturing, is run repeatedly – often enough to detect a significant improvement. In this case, three cycles suffices to power up the signal-to-noise ratio. This case illustrates a big manufacturing-yield improvement over the course of an EVOP. However, any number of system attributes can be accounted for via multiple-response optimization tools provided by Design-Expert or the like. This ensures that an EVOP will produce more desirable operating conditions overall for process efficiency and product quality.
It pays to pay attention to nature!
*Box, G. E. P. and N. R. Draper, Evolutionary Operation, Wiley New York, 1969. (Wiley Classics Library, paperback edition, 1998.)
**(We show designs with center points as a check for curvature.)
Beware of bugs bearing backpacks
Posted by mark in science, Uncategorized on March 3, 2010
I am attending a conference sponsored by the National Defense Industry Association (NDIA). They provided all of us participants a copy of the latest issue (March) of their publication National Defense. While wiling away the time listening to some long-winded higher-ups I paged through the magazine and admired the weaponry developed to keep our war-fighters supported to the max. However, on page 17 a very odd picture caught my eye – a cockroach carrying a radiation sensor on its back! A researcher at Texas A&M reports that these bugs are ideal for sweeping potentially contaminated areas, ideally in teams of twenty. They can be operated remotely via devices that stimulate their leg muscles.
There is one problem though: Cockroaches cannot crawl backward. One had better hope that none of the bad guys wear pointy-toed cowboy boots, because they will be ideal for killing the sensor-bearing bugs that become stuck in the corners.
Simplifying the witches brew in Shakespeare’s MacBeth
Posted by mark in Uncategorized on February 14, 2010
Last week I enjoyed an innovative performance of Shakespeare’s MacBeth at the Guthrie Theater in Minneapolis. The director, Joe Dowling, takes some liberties with the original production, such as dropping the warrior MacBeth on stage down a ninja rope and equipping him with a machine gun. However, mostly the gruesome killings that riddle this dark play are accomplished with old-fashioned daggers and swords.
Being a chemical engineer who abhors overly complicated recipes, it bothers me that the three weird sisters in Macbeth put so many ingredients into their witches brew. It would be very hard to scale up their potent product from bench-level kettle to massive manufacturing. By the way, thanks to this heads-up by Nigel Beale I cracked this coven’s confidential code on components; for example, eye of newt, which cannot be easily sourced, is really readily-available mustard seed. Nevertheless, I’ll bet that a good mixture screening experiment, followed-up by an in-depth formulation design, such as this one on a cell-culture medium, would reveal that only a few key ingredients might do the job for enabling clairvoyance or whatever a witch might be up to.
Keep it simple, I say. Or as Shakespeare advises more eloquently, “Brevity is the soul of wit.”
Apples and oranges comparison of diets?
Posted by mark in science, Uncategorized, Wellness on January 21, 2010
While exercising on my elliptical machine this morning watching ABC’s Good Morning America the show captured my attention with a report that Weight Watchers (“WW”) this week filed a lawsuit against one of its top competitors, Jenny Craig (“JC”). The dispute stems from a claim by JC that their clients lost, on average, over twice as much weight as those on the largest weight loss program. WW alleges that this claim is deceptive due it comparing a study by JC done this year versus one done by WW 10 years ago. According to this news release by Weight Watchers the complaint states that generally accepted standards of biomedical research require Jenny Craig to compare the two current offerings of both companies through a head-to-head randomized clinical trial.
“You can’t compare studies that were done in different locations at different times using different groups of people.”
– Louis Aronne, M.D, New York Presbyterian Hospital weight loss expert and author of Eat This, Not That
Although the judge has put a temporary restraining order against their offending ad, I wouldn’t rule out the JC claim prima facie. After all, as Smartmoney Magazine writer Angie Marek stated in her column on The Skinny on Big, Fat Diet Programs “the science on most of these plans is hardly conclusive, since most of the research has been paid for by the diet companies themselves.” In fact, I predict that this case will keep at least two statisticians fat and sassy as expert witnesses (one on each side of this tug-of-war).
Marketing researchers go curb mining for treasure in the trash
Posted by mark in Uncategorized on January 16, 2010
The January Twin Cities Business magazine reported “A Real Good Marketing Experiment” that I found intriguing. Minneapolis-based Blu Dot Design and Manufacturing researchers abandoned 25 of their Real Good chairs on New York City sidewalks. See the results in this very entertaining eight-minute film.
“Once in a while you find some really good stuff.”
– A ‘Punko’ (scrounger) in the Real Good chair experiment.
In our suburban neighborhood we put out furniture and tape a sign on it that says “Free.” Whenever I’ve done this, the item has disappeared. A few times I’ve spied a person carting it off, but most stuff goes off with the garbage, I guess.
I suggest that the next time Blu Dot tosses out chairs they do a proper design of experiment (DOE). The measure of success will be how long it takes for a particular unit to be scrounged. The experimental factors will be color and various design elements. Of course the chairs must be laid out according to a randomized plan.
Stat-Ease performed a very simple, but statistically powerful, experiment on the chairs it considered for purchase in a new training room. See the details in this lead article of the June 2002 Stat-Teaser newsletter.
Management Blog Carnival, Review 3 – Seth Godin blog
Posted by mark in Uncategorized on January 3, 2010
(Note: This blog is the last of three in a carnival organized by John Hunter.
I picked the Seth Godin blog to review because his name rang a bell, but I’d never read anything by him that I specifically recall. As it turns out, I really did not see much that interested me greatly for exactly the opposite reason expressed in this comment by “ariana10” to Godin’s blog of 8/22 (bulleted below):
“I relate to this blog because I am a journalism major at the University of Kansas and I can’t do math for the life of me.”
I am an engineering major who likes math (in moderation!) and I can do it for the life of me. However, I am also keen on marketing and business (MBA, U Minn., ’80) so, even though Godin is light on stats, I must admit that he’s got much to offer for those of us trying to make a living in this high-tech world. Here are a few Seth Godin blogs of 2009 that hit my hot buttons.
- 1/24 Good guys finish… Godin suggests that under the bright light of the internet being generous and fair in business dealings pays off now more than ever. I like that idea a lot.
“When your customer service policies delight rather than enrage, word of mouth more than pays your costs.”
- 8/22 Not so good at math demonstrates the confusion creating by using miles per gallon (mpg) as the metric for fuel efficiency. As I noted in my blog on how the Inverse transformation puts mileage comparisons on track / the best measure for fuel efficiency is gallons per ten-thousand miles.
- 8/28 Spare no expense! does get somewhat quantitative (finally a graph!) in discussing the tradeoffs of giving almost no personal service to a huge number of users (Google) versus a great deal of attention to the troublesome individuals who soak it up.
- 10/26 Dunbar’s Number isn’t just a number, it’s the law is where Godin, a people person, draws the line at 150 – the limit predicted by British anthropologist Robin Dunbar for stable social relationships. I think the number might be a lot less (a tenth?) for engineers than marketers. ; )
What sets Seth Godin’s blog apart from the others I’ve seen (admittedly a small sample) is the amount of original content laced with thought-provoking observations of how people interact and what turns them on or off. He’s a guy worth keeping an eye on, I think.
Management Blog Carnival, Review 2 – “Hexawise” by Justin Hunter
Posted by hank in design of experiments, Uncategorized on January 1, 2010
(Editor’s note: This blog is contributed by my son Hank – a programmer by profession. It’s the second of three in a carnival organized by John Hunter. -Mark)
Justin Hunter is the founder of Hexawise, a SaaS tool that aids in setting up tests for software using statistical methods. This also happens to be the subject of his blog – no doubt influenced in part by his father, William Hunter, author of the classic text Statistics for Experimenters. Justin started the blog mid-way through ’09, so the pickings are a little slim, but there is still plenty of good stuff.
Some highlights from 2009:
- 10/6 The Stackoverflow.com for Software Testers marks the release of a beta version of testing.stackexchange.com. This is a community driven Q and A site that uses the same technology as Stack Overflow, a popular site for coders looking for help. Hunter’s version is aimed at testers, and already has an impressive database of answers and discussion.
- 8/25 What Else Can Software Development and Testing Learn from Manufacturing? Don’t Forget Design of Experiments (DoE) links to a Tony Baer post comparing software development to the manufacturing industry. Hunter further focuses on the application of Design of Experiments, pointing out the extensive use of DoE in quality improvement initiatives in Toyota and Six Sigma. These initiatives have yet to really penetrate the software development industry, despite some high profile successes (Google’s Website Optimizer and Youtube are mentioned).
- 12/9 Defect Seen >10 Million Times and Still not Corrected has some interesting trivia about the grammatical error in Lands’ End – something I hadn’t even noticed, and apparently the company hadn’t either until it was too late. The real point of the post, however, is to point out another much more fixable grammatical error in Google’s Blogger software. If there is only 1 comment on a post, it still says “1 comments”, instead of dropping the s. A trivial defect, perhaps, but a very visible and easily fixed one. It reminds me of something Mark always says about taking a break from work to sweep the dirt off the shop floor. That is, you shouldn’t let the little inconsequential bugs pile up while you’re focused on the big ones.
On a lighter note, in Famous Quotes that Make Just as Much Sense When You Substitute PowerPoint for Power Justin linked to a post by Jerry Brito about substituting PowerPoint for Power in famous quotes, adding a few of his own. I’d also like to add:
Kirk: “Spock, where the hell’s the PowerPoint you promised?”
Spock: “One damn minute, Admiral.” –Star Trek IV