“I had many years that I was not so successful as a ballplayer, as it is a game of skill.”
— Casey Stengel (from testimony before United States Senate Anti-Trust and Monopoly Hearing, 1958)

Last week the University of Minnesota School of Statistics sponsored a talk titled “In-season prediction of batting averages: A field test of empirical Bayes and Bayes methodologies,” presented by Lawrence D. Brown from the Statistics Department of Wharton School at the University of Pennsylvania. My colleague Pat Whitcomb attended and told me about a few findings by Brown that baseball fanatics like me would find a bit surprising:
“The simplest prediction method that uses the first half [of a season’s] batting average …performs worse than simply ignoring individual performances and using the overall mean of batting averages as the predictor for all players.”*

Evidently these professional players perform at such a consistent level that the ones hitting at a higher than average rate up until the mid-season break tend to regress back to the mean the rest of the way, and vice-versa.

Of course, by looking at many years of past performance, one would gain some predictive powers. For example, in 1978, more than ten years into his Hall of Fame (HOF) career, Rod Carew batted .333 for the Minnesota Twins. He made it to the Major Leagues only a few years ahead of fellow Twin Rick Dempsey, who hit at an average of .259 in 1978. Carew finished up his 19-year playing career with a lifetime batting average (BA) of .328, whereas Dempsey hung on for an astounding 24 years with a BA of only .233! It would not require a sabermetrician to predict over any reasonable time frame a higher BA for a HOF ballplayer like Carew versus a dog (but lovable, durable and reliable defensively at catcher) such as Dempsey.

Brown also verifies this ‘no brainer’ for baseball fans: “The naıve prediction that uses the first month’s average to predict later performance is especially poor.” Dempsey demonstrated the converse of this caveat by batting .385 (5 for 13) for his Baltimore Oriole team in the 1983 World Series to earn the Most Valuable Player (MVP) award!

Statistical anomalies like this naturally occur due to the nature of such binomial events, where only two outcomes are possible: When a batter comes to the plate, he either gets a hit, or he does not (foregoing any credit for a walk or sacrifice). It is very tricky to characterize binomial events when very few occur, such as in any given Series of 4 to 7 games. However, as a rule-of-thumb the statistical umpires say that if np>10 (for example over 50 at-bats for a fellow hitting at an rate of 0.200), the normal approximation can be used for binomial distributions and the variance becomes approximately p(1-p)/n.** From this equation one can see that the bigger the n, that is – at-bats, the less the fraction (batting average) varies.

PS. I leave you with this paradoxical question: Is it possible for one player to hit for a higher batting average than another player during a given year, and to do so again the next year, but to have a lower BA when the two years are combined?

*Annals of Applied Statistics, Volume 2, Number 1 (2008), 113-152

**This Wikipedia entry on the binomial distribution says that “this approximation is a huge time-saver (exact calculations with large n are very onerous); historically, it was the first use of the normal distribution, introduced in Abraham de Moivre’s book The Doctrine of Chances in 1733.”

Evidently due to its concentration algorithmic ‘philic’ minds, Stat-Ease gets review copies of new technical tomes from the Society for Industrial and Applied Mathematics (SIAM). I once fancied myself as a ‘mathelete,’ but I learned differently after moving up from being the big fish in my small pond at high school to a miniscule minnow at a major university in the Midwestern USA. My mistake was skipping into an advanced calculus class populated by some of the country’s top talent – National Merit scholars like me. Very quickly I realized that my math skills only put me on the very bottom rung and that only by the very tip of one fingernail. What saved me was begging for mercy by the teacher who, luckily, was sick and tired of the smart-mouths in the class who really got it and made sure to flout their chops in math. Thus, when the newest SIAM publication arrives, I always look it over in wonder before quickly passing it along to our master’s statisticians and algorithmic programmers, who may understand its true value.

For example, the book this week is Functions of Matrices, Theory and Computation by Nicholas J. Higham, which “emphasizes Schur decomposition, block Parlett recurrence, and judicious use of Padé approximants.” That blew me away immediately, but I rifled through the pages anyways and found a few pages of interest on the history of matrix functions, which really are useful in our business of experiment design and statistical analysis. (Thank goodness for the power of computers to do the calculations!) Higham credits English-born James Joseph Sylvester as the inventor of the matrix (not to be confused with the famous movie trilogy!). Sylvester emigrated to the USA where he founded the American Journal of Mathematics in 1878, the self-proclaimed “oldest mathematics journal in the Western Hemisphere in continuous publication.”

What amazes me is that anyone can read such esoteric materials, but it’s good they do, because great advances are made possible by developments in math. For example, Higham points out that the first practical application of matrices led to the elimination of unwanted flutter in aircraft wings. (Galloping flutter, or wake vortex flutter, caused the spectacular failure of the Tacoma Narrows Bridge in 1940.*) This work was done by the Aerodynamics Department of England’s National Physical Laboratory (NPL) in the 1930s. In parallel, not far away in the UK, Ronald Fisher, the founder of modern-day applied statistics, was developing the core catalog of experimental design matrices that still remain in use today.

“Here I stand because of you, Mr. Anderson. Because of you, I’m no longer an Agent of this system. Because of you, I’ve changed. I’m unplugged.”
– Agent Smith (played by Hugo Weaving ) from The Matrix Reloaded (2003)

PS. Neither the quote nor the picture really have much to do with matrices, but they provide me some amusement. For example, I saw the second movie of the Matrix trilogy with my brother Paul, an techie type like me. We annoyed the exiting theater patrons greatly by regurgitating Agent Smith’s lines about “Mr. Anderson” this and “Mr. Anderson” that – all with gagging glee.

The picture exhibits a physical matrix – the screen window. I just inserted all the screens earlier this week when it seemed as if Spring had finally arrived in Minnesota. However, we citizens of this northern State were chagrined to see a coating of snow yesterday morning – over a foot in some parts. 🙁

*If you’d like to set up an experiment on flutter that requires only a hair blower and some other materials that can be procured from your local hardware store, see this posting on Aeroelasticity Phenomenon by Wright State’s College of Engineering and Computer Science (Dayton, Ohio).

That’s what meteorologist Edward Lorenz postulated in his 1972 paper on predictability of weather. Lorenz, who died last week at the age of 90, used this example to illustrate his “chaos theory,” which linked small changes in a system to large, unforeseen consequences. For more background on the life and accomplishments of the 1991 Kyoto prize winner for earth and planetary sciences, see this article by Thomas Maugh.

I am certain I heard of chaos theory well before the movie Jurassic Park, but who can forget the pessimistic views the scientific character Doctor Ian Malcolm, who cited Lorenz’s thories to predict the subsequent catastrophe of dinosauric proportions. This is a recurring theme of Jurassic Park author, Michael Crichton: Any complex system will inevitably break down due to the natural state of disorder, or entropy.

I fear that I shall always remain unclear on distinctions a fine as this – chaos vs entropy. Perhaps things may come into focus after I read this article on “Chaos, Complexity, and Entropy”— a physics talk for non-physicists by Michel Baranger of the Center for Theoretical Physics, Laboratory for Nuclear Science and Department of Physics at Massachusetts Institute of Technology.

It seems to me that Lorenz in his chaos theory considered Earth’s meteorology as a system that often becomes so tightly wound that it comes right to the brink of breaking down — so close that the tiniest disturbance, such as that caused by a benign Brazilian butterfly, can create a terrible upset. Being a chemical engineer, what comes to mind for me is a supersaturated solution of a salt that solidifies around the tiniest seed.

I only hope that I do not get twisted up in Earth’s chaotic meteorology — a very real possibility here at the northern end of the USA’s tornado alley. Maybe a minnow in the Amazon is wiggling a fin at this very moment! I’d better bunker down in the basement…

I collected three articles for my blog this week that all involve the decision to go left or right.

Last Sunday’s Parade magazine reported that Tom Dowdy, an engineer for UPS delivery, estimates a savings of 3 million gallons of gas per year by biasing delivery routes to right, rather than left, turns. The reduction in idling time reduced UPS truck emissions by 32,000 metric tons – the equivalent air pollution of 5300 cars.

This week’s “Ask Marilyn” column in Parade features an observation by Bob English of Lakeland, Florida, who avoided a head on collision thanks to time seemingly slowing down. Marilyn calls this phenomenon “extreme concentration” – a positive reaction to incredible stress. This happened to me some years ago. On a peaceful weekend morning with ideal driving conditions I took my daughter and niece up the Saint Croix Valley for a visit with my mother. Halfway there the one car we encountered on the 15 mile country route veered into directly at us. To me it felt like time stood still as I realized that we’d hit head on in just a second. I remember seeing that I had only a narrow shoulder on the right and realizing that we’d roll if I went any further that direction. Then I clearly recall looking beyond the oncoming driver, who must have dozed off on this sunny morning. There were no other cars coming down the road. I then decided to go around to the left of the opposing automobile – a very radical move. What I did not consider was the other driver waking up and moving out of my lane back to the correct side of the road. I made the move successfully in any case. However, as I learned later from a defensive driving course, the correct maneuver is to go right not matter what – even it means you will crash into a ditch – better that then a head on collision.

The last of the three articles I collected this week is by Shankar Vedantam of the Washington Post. He discusses the natural “action bias” of people who would do best by doing nothing. This causes investors to hold stocks as they peak and sell them after a big fall in price – not an optimal strategy! In another example of action bias, economist Ofer Axar compiled statistics on soccer goalies defending a penalty kick. He concluded that they would stop the most goals by standing still. However, over 90 percent of kicks were defended by diving left or right.

So, the next time you feel pressured into a decision one way or the other, consider the option of not doing anything just yet. However, if something bad will happen for sure by sitting still, I hope that you will benefit from a spell of extreme concentration and not the other typical reaction of people under extreme stress – a paralyzing ‘freeze.’

One year ago, Bill Hathaway, founder of Six Sigma web-based trainer MoreSteam.com, announced their development of a university campus in Second Life – a 3D virtual world where users can socialize, connect and create using voice and text chat. That’s pretty cool, but what intrigued me was the picture shown here that accompanied Bill’s e-mail. It depicts an avatar-hurling catapult called the Avapult.™ (Avatars are the in-world characters assumed by the participants, for example, I am known in Second Life as “Stat Mathy.” My moniker betrays my interests!)

It took me a while to work through some issues related to my Vista operating system, but the day came not long ago when I typed in “moresteam” and teleported to the island home of the Avapult. I watched as Bill’s avatar donned a Viking helmet (in homage to me being a Minnesota football fan), climbed on to this fearsome-looking engine of destruction and flung himself virtually over the cliff. Unfortunately, Bill’s character missed the target but came close enough to become ignited. The virtual-human missile, easily tracked by its trail of smoke, then plunged into a swamp, where it literally (figuratively?) found itself up to its behind with an alligator.

So, in addition to the normal engineering challenge of determining which Avapult factors are significant, students of MoreSteam.com’s virtual university will face real (?) world stressors that make it imperative to find the best combination of settings quickly. That underscores the need for multifactor design of experiments (DOE) such as those detailed in DOE Simplified, 2nd Edition, used as a reference for MoreSteam.com web-based Six Sigma training. As Bill says, “this will be help teams separated by distance to build rapport among members, especially in advance of a blended learning classroom session.”

Tuesday I travel to Columbus for my twice-a-year teach at Fisher College of Business at The Ohio State University, who team up with MoreSteam.com for blended training on Six Sigma aimed at executives seeking Black Belt (BB) status. This Spring’s bunch of BBs have been invited to take a shot at the Avapult. I will be interested to hear how this goes. The proof for me will be seen in how well the teams do at my semi-annual paper helicopter fly-off. In the past, when confronted with the task of putting DOE tools to task, some of my students, especially those who work in non-technical areas like personnel, seemed very unclear on the concepts. I feel sure that work on the Avapult will be very useful for education on design and analysis of experiments.

My son Hank, who assisted me in a trebuchet response surface method (RSM) experiment that I wrote up in RSM Simplified, is way ahead of me on the virtual world. He traveled to MoreSteam island on Second Life the other day and scoped out their Avapult. I volunteer my alter ego Stat Mathy as fodder for Hank’s designed experiments. However, I plan to first purchase chain-mail shorts as discouragement against the gator!

While cleaning out files this week, I found my long-forgotten notes from “Design of Experiments for Discovery, Improvement and Robustness” co-presented by DOE guru George Box in March of 1996 – over a dozen years ago. The first thing I noticed was the photo roster showing how huge my spectacles and others were in that era.* The ones I have now are so narrow I cannot see, but at least they are fashionable! (There is an added factor: In 1996 I did not need progressively-lensed bifocals, although they would have benefited from the goggly glasses of that time.)

However, even more enjoyable than the chuckle over obsolete fashions were Box’s timeless anecdotes, which I recorded religiously. For example, Professor Box mentioned how his father would comment on trivial differences: “A blind man would be glad to see it.” This drove home a point Box wanted to make on how being statistically significant did not necessarily lead to anything of practical importance. In fact, while working as a statistician at Imperial Chemical Industries (ICI), he banned the use of p-values by their industrial experimenters! (Box advocated the use of confidence intervals, instead.)

In my webinar last month on “10 Ways to Mess Up an Experiment & 8 Ways to Clean it Up”. I made this point (statistical significance versus practical importance) in a slide similar to that shown here. It accomplishes little to achieve a low p value for a change that is so small that it produces nothing of any practical importance. In today’s age of robotic experimentation this happens more-and-more often due to the large number of runs — in the hundreds or even thousands. On the other hand, plenty of experiments are still done in situations where runs are dear and not many can be performed. Then a big difference may be seen that fails the pre-ordained threshold level for p. In that case it often pays to investigate further.

“Even if the probability was 6% of not finding a crock of gold behind the next tree, wouldn’t you go and look?”

— Quote from “An appendix featuring quaquaversal quotes … that embellish key concepts and enliven the learning process” presented by George E. P. Box, J. Stuart Hunter, and the late William G. Hunter in the second edition of the classic book Statistics for Experimenters: Design, Innovation and Discovery.

*You have to see this web site, at least the goofy glasses shown on the rotating eye-catcher, on spectacles through the ages.

This is the motto of those fearless few, such as successful surgeons, who forge ahead with never a look back. As they progress, by-standers bullets just bounce off these never-doubters impervious armor. Assuming it’s true that such confidence (if not outright arrogance) is not misplaced, this a highly desirable trait for doctors, lawyers and other professionals (such as statistical consultants) that others rely upon for good advice on critical matters. Umpires and referees mustn’t ever waver in their calls, but as all fans would doubtless agree, bad judgments are made every game, especially against their home team. Also, consider the current race for the American Presidency – do you hear any candidates saying that they doubt the country can ever be put back on the proper path (right or left, depending on party)?

Trouble comes when an expert in a specific area cannot acknowledge incompetence in other endeavors. If you ever run into such a maddening individual who dismisses your greater experience in an area of primary interest, consider this premise of neurologist Robert A. Burton: Certainty arises out of involuntary brain mechanisms that function independently of reason, which I paraphrased from this author’s web page for the newly-published book On Being Certain.

Obviously ignorance is bliss in my case, because I am not the least bit interested in reading Burton’s book – it will undermine my confidence in the few things I really feel certain about. Ever since I saw someone at an American Statistical Association conference wearing a shirt proclaiming that “Statistics Means Never Having to Say You’re Certain” it’s been hard for me to develop 100 percent confidence in anything. To be told that I am manifesting an ephemeral mental state like anger or other emotions when I leave no doubt in pillars of certainty such as ‘2 plus 2 equals 4’ would be too much for me to bear.

Can Burton possibly be right? No way! I doubt it very much.