Posts Tagged design of experiments
Evolutionary operation
Posted by mark in Uncategorized, design of experiments on March 7th, 2010
Last December, after an outing by the Florida sea, I put out an alert about monster lobsters. This reminded me of an illustration by statistical gurus Box and Draper* of a manufacturing improvement method called evolutionary operation (EVOP), which calls for an ongoing series of two-level factorial designs that illuminate a path to more desirable conditions.
With the aid of Design-Expert® software, I reproduced in color the contour plot in Figure 1.3 from the book on EVOP by Box and Draper (see figure at the right). To illustrate the basic principle of evolution, Box and Draper supposed that a series of mutations induced variation in length of lobster claws as well as the pressure the creatures could apply. The contours display the percentage of lobsters at any given combination of length and pressure who survive long enough to reproduce. Naturally this species then evolves toward the optimum of these two attributes as I’ve shown in the middle graph (black and white contours with lobsters crawling all over them).
In this way, Box and Draper present the two key components of natural selection:
- Variation
- An environment that favors select variants.
The strategy of EVOP mimics this process for improvement, but in a controlled fashion. As illustrated here in the left-most plot, a two-level factorial,** with ranges restricted so as not to upset manufacturing, is run repeatedly – often enough to detect a significant improvement. In this case, three cycles suffices to power up the signal-to-noise ratio. This case illustrates a big manufacturing-yield improvement over the course of an EVOP. However, any number of system attributes can be accounted for via multiple-response optimization tools provided by Design-Expert or the like. This ensures that an EVOP will produce more desirable operating conditions overall for process efficiency and product quality.
It pays to pay attention to nature!
*Box, G. E. P. and N. R. Draper, Evolutionary Operation, Wiley New York, 1969. (Wiley Classics Library, paperback edition, 1998.)
**(We show designs with center points as a check for curvature.)
Skepticism versus cynicism about science experiments
Eric Felten’s latest “De Gustibus” column in Wall Street Journal reports New Episodes of Scientists Behaving Badly. It details various scandals, for example the retraction of a landmark publication linking autism to childhood vaccines. This creates a great deal of cynicism such as that expressed by this parent of a kid she helped on a science project:
“The experiments never turned out the way they were supposed to, and so we were always having to fudge the results so that the projects wouldn’t be screwy. I always felt guilty about that dishonesty, but now I feel like we were doing real science.”
Ouch!
Coincidentally, Stat-Ease received an email from someone who goes by the pen-name “The Pyrrhonist.” (I see a trend here: I need to work on a scholarly-sounding moniker.) While researching pyrrhonism, I came across this skeptical quote by a Greek named Carneades who set the stage for his countryman Pyrrho:
“Nothing can be known, not even this.”
That’s tough to get around!
I truly believe that some degree of skepticism is healthy, such as judicious use of the null hypothesis for assessing the outcome of experiments. However, it’s not good for experimenters to abandon all standards by succumbing to an attitude of scornful or jaded negativity, especially a general distrust of the integrity or professed motives of others – the definition of cynicism (according to the Free Dictionary).
So, be skeptical, but not cynical.
Gambling with the devil
Posted by mark in Basic stats & math, design of experiments on November 15th, 2009
In today’s “AskMarilyn” column by Marilyn vos Savant for Parade magazine she addresses a question about the game of Scrabble: Is it fair at the outset for one player to pick all seven letter-tiles rather than awaiting his turn to take one at a time? The fellow’s mother doesn’t like this. She claims that he might grab the valuable “X” before others have the chance. Follow the link for Marilyn’s answer to this issue of random (or not) sampling.
This week I did my day on DOE (design of experiments) for a biannual workshop on Lean Six Sigma sponsored by Ohio State University’s Fisher College of Business (blended with training by www.MoreSteam.com.) Early on I present a case study* on a training experiment done by a software publisher. The goal is to increase the productivity of programmers by sending them to workshop. The manager asks for volunteers from his staff of 30. Half agree to go. Upon their return from the class his annual performance rating, done subjectively on a ten-point scale, reveals a statistically significant increase due to the training. I ask you (the same as I ask my lean six sigma students): Is this fair?
“Designing an experiment is like gambling with the devil: only a random strategy can defeat all his betting systems.”
– RA Fisher
PS. I put my class to the test of whether they really “get” how to design and analyze a two-level factorial experiment by asking them to develop a long-flying and accurate paper helicopter. They use Design-Ease software, which lays out a randomized plan. However, the student tasked with dropping the ‘copters of one of the teams just grabbed all eight of their designs and jumped up the chair. I asked her if she planned to drop them all at once, or what. She told me that only one at a time would be flown – selected by intuition as the trials progressed. What an interesting sampling strategy!
PPS. Check out this paper “hella copter” developed for another statistics class (not mine).
*(Source: “Design of Experiments, A Powerful Analytical Tool” by Christopher Nachtsheim and Bradley Jones, Six Sigma Forum Magazine, August 2003.)
Awesome demonstration of design of experiments
Posted by mark in design of experiments on April 27th, 2009
Team Awesome
The engineering students at South Dakota School of Mines and Technology really do rock. Where else could one present a class on statistics until 8:30 pm on a Friday night and continue it less than 12 hours later – early on a Saturday morning?
Our workshop on design of experiments (DOE) finished with a spirited competition of paper helicopters.* The winner was Team Awesome: Kayla Rithmiller, MacKenzie Trask and Samantha Johnson (pictured from left to right). They scored highest on the basis of flight time and accuracy. You can see their ‘copter spinning to another precise landing in their confirmation run.
Congratulations to Team Awesome and all the SDSM&T students who devoted their free time to learning DOE and demonstrating this newly-gained knowledge via well-planned experiments on the helicopter exercise. I predict that they all will go far!
*See details on this DOE exercise in the September 2004 Stat-Teaser article on Playing with Paper Helicopters.