Data detectives keep science honest
Posted by mark in science, Uncategorized on October 6, 2023
An article in Wall Street Journal last week* drew my attention to a growing number of scientists who moonlight as data detectives sleuthing out fraudulent studies. Thanks to their work the number of faulty papers retracted increased from 119 in 2002 to 5,500 last year. These statistics come from Retraction Watch who provide a better, graphical, perspective on the increase based on percent retractions per annual science and engineering (SE) publication–not nearly as dramatic given the explosion in publications over the last 20 years, but still very alarming.
“If you take the sleuths out of the equation it’s very difficult to see how most of these retractions would have happened.”
Ivan Oransky, co-founder of Data Colada –a blog dedicated to investigative analysis and replication of academic research.
Coincidentally, I just received this new cartoon from Professor Nadeem Irfan Bukhari. (See my all-time favorite from him in the April 27, 2007 StatsMadeEasy blog Cartoon quantifies commitment issue.)
It depicts statistics as the proverbial camel allowed to put its nose in the tent occupied by science disciplines until it become completely entrenched.
Thank goodness for scientists like Nadeem who embrace statistical tools for design and analysis of experiments. And kudos to those who guard against faulty or outright fraudulent scientific publications.
*The Band of Debunkers Busting Bad Scientists, Nidhi Subbaraman, 9/24/23
Temperature combines badly with humidity to maximize misery
The Twin Cities tied its record high temperature yesterday at 97 degrees Fahrenheit. However, the winds blew strong with air at a dew point in the low 60s, which made the heat relatively tolerable. After spending most of August at our second home in southwest Florida (leaving there just as Hurricane Idalia hit), my wife and I got acclimated to a far more uncomfortable daily combination of heat and humidity.
Before departing for Minnesota, I set up a SensorPush to monitor temperature, humidity and dew point—the temperature at which air becomes saturated with water vapor. I want to be on guard for the air conditioning going out. If that happens in Florida homes, mold can grow. After experiencing this once (due to renters not running the A/C) and dealing with an expensive remediation, I am keen to prevent another episode.
Closely related to dew point is the wet-bulb temperature, which, as chemical engineer, I learned how to measure with a sling psychrometer. The wet-bulb-temperature can then be converted to relative humidity.
To prevent heat-related deaths in training camps, the US military developed a more sophisticated measure called the wet bulb globe temperature (WBGT). It provides a good measure for the advisability of being active in full sun. The Texas University Interscholastic League requires that outdoor practices be shut down if WGBT exceeds 92.
“As with all indices that integrate elements of the thermal environment, interpretation of the observed levels of WBGT requires careful evaluation of people’s activity, clothing, and many other factors, all of which can introduce large errors into any predictions of adverse effects.”
– Grahame M Budd, Wet-bulb globe temperature (WBGT)–its history and its limitations
Other measures use to gauge comfort are Heat Index and Feels Like Temperature (FLT). I like the FLT because it accounts for the benefits of evaporative cooling. For example, as I write this, the actual temperature is 95 degrees and the FLT is only slightly higher at 96.
I’m getting too hot and bothered with all these measurements to continue much longer, but here’s yet another approach used by AccuWeather—the RealFeel Temperature.
What really matters is how you feel and what can be done to avoid discomfort. For example, earlier this summer I went to our Minnesota’s Washington County Fair on a very hot day and stopped in at a beer garden for a cold brew. However, I soon realized that its hot tin roof radiated heat down to the picnic tables—overcoming any advantage to being in the shade.
Sometimes you can find no relief other than hunkering down in an air-conditioned area. How did we ever get by without it?
Experimenting to make spirits more enticing
Posted by mark in food science on August 15, 2023
Spirits are distilled alcoholic drinks that typically contain 40% alcohol by volume (ABV) or 80 “proof”. Until the Pandemic, I avoided spirits—preferring to imbibe less intoxicating beers and wines. However, during the Quarantine, I made it my mission to drink up a stock of tequila that my Mexican exchange student’s father Pepe sent me when his daughter told him about the terrible cold in Minnesota.
Down the hatch went Don Julio and the like over some months…and yet the quarantine dragged on. Tiring of tequila I pivoted to bourbon, starting with top-shelf Woodford Reserve and settling after serial pairwise testing on bottom-shelf Evan Williams. Why pay more when you cannot discern a difference?
Last week my research on spirits expanded to rye whiskey purchased after a tour of the Chattanooga Whiskey Experimental Distillery. See my guide Sam pictured with a measurement guide for a key variable—the degree of charring in the storage barrels.
The mash bill for my bottle is malted rye, yellow corn, caramel malted rye (providing a smoother taste) and chocolate malted rye (not sure what that is, but it sounds tasty).
It seems to me that multifactor design of experiments would be an ideal tool for contending with the many process, mixture and categorical inputs to the optimization of whiskey. Once upon a time I toured Dewars Aberfeldy distillery in central Scotland. It concluded with my first taste of whiskey—shockingly strong. However, what interested me most was a simulator that allowed visitors to vary inputs and see how the output rated for taste. Unfortunately, I only had time to do one factor at a time (OFAT) testing and desperate stabs at changing multiple inputs.
If the spirit moves you (pun intended), please contact me for help designing your experiments and tasting the results.
Diabolical werewolves test trust in team
Posted by mark in Communication, leadership on August 1, 2023
Tonight, there’s a full moon. Then on August 30 comes the second full one of the month—a rare blue moon. Thus, it’s especially appropriate to consider werewolves and, in particular, an online game where these lycanthropes (secretly designated) undermine trust and security within a group.
I played Werewolf a few times but never got very far due to the cutthroat “kill the newbie” strategy deployed by more experienced (and vicious!) players. My interest in the Werewolf game stems from it providing a good laboratory for studying social dynamics and teamwork. See, for example, this blog by a relationship expert about What Werewolf teaches us about Trust & Security. For scientists studying such interactions, the Idiap Wolf Corpus (sounds creepy!) offers a wealth of data in the form of audio-visual recordings of 15 games played by 4 groups of people.
A newly published study by a trio of industrial engineers* delves into the impact of playing Werewolf at a distance and what this revealed about teamwork when members participate only on a virtual basis. The researchers divided 30 students into 3 teams of ten comprised of two werewolves, seven villagers, and one seer. Their experiment varied the groups by leadership experience.
The sample size of this study was far too small to support any conclusions, in my opinion. I just thought it would be fun to put teams, such as a group of researchers tasked with developing a new product, to the test of Werewolf.
Devious!
Cue the howls as the full moon rises…
PS I do wonder how well teams do at a distance versus in person. My feeling based on a lot of experience as a chemical engineer leading plant-process-improvement projects is that it pays to get together in one room every several meetings. It would be interesting to see well-designed research on all virtual, all in-person or a mix of the two.
*Vera Setyanitami, Hilya Mudrika Arini and Nurul Lathifah, People’s Trust in a Virtual Project Team: Results of a Game Experiment, Jurnal Teknik Industri, Vol. 25, No. 1, June 2023.
Selecting the most readable font for maximum impact
Posted by mark in Communication, Graphics on July 7, 2023
It’s Comic Sans Day today. If not so widely mocked, this font would be favored for its legibility across all ages and abilities, according to my daughter Emily—an expert in graphic design.
My early knowledge of writing options consisted of printing or cursive. As I progressed through college my preference became printing, which though slower to produce than cursive, resulted in a far more legible output and appealed to my engineering sensibilities.
I kept on handwriting through my early career—relying on secretaries to do the typing. However, it wasn’t long before I went DIY by becoming an early adopter of computers—a Radio Shack TRS-80. Its word processing capabilities made it far easier to write—a huge breakthrough by enabling editing.
Eventually, after a lot of hunting and pecking, I upgraded to MS-DOS (Microsoft’s disk operating system) and invested in Maven Beacon Teaches Typing to gain the ‘touch’ of my keyboard.
Things got really interesting with the advent of graphical user interfaces and widely available True Type fonts. After some wild and wacky times making bad blends of too many fonts, I settled in on the Microsoft Word defaults of the classic (invented 1931) Times New Roman (serif—featuring tails and feet) for text and more modern (1982) Arial (sans serif) for headlines. A big issue then, but far less so now that “e” rules, was whether a document would be read in print or electronically (on screen).
Around these times, Stat-Ease shifted its training materials from transparencies for ‘overheads’ to Powerpoint for projection from a personal computer (PC). Unfortunately, projectors in those early days put out a very weak light. However, being well equipped with Stat-Ease software, I rose to the challenge by deploying an experiment design in-class to maximize screen readability via adjustments to fonts and other factors.
Nowadays, figuring that nearly all my writing will be read on screen, I go exclusively with the current Word default of Calibri—a font that being san serif provides a “small, but significant, advantage in response times” according to this study in the Journal of Cognitive Psychology.
It turns out that, not surprisingly, studies now show different fonts increase reading speed for different individuals.
Participants’ reading speeds (measured in words-per-minute (WPM)) increased by 35% when comparing fastest and slowest fonts without affecting reading comprehension.
Adobe scientists and others who authored “Towards Individuated Reading Experience”, ACM Transactions on Computer-Human Interaction, Volume 29, Issue 431, March 2022, Article No.: 38, pp 1–56.
Therefore, I envision that, aided by developments in artificial intelligence, our devices will keep track of how fast we read and adapt the fonts accordingly. Watch out: Like it or not, you may be subjected to Comic Sans.
PS Calibri fared well overall on average in Adobe’s experiment so that remains my favored font.
Hurricane season off to a hot start with great uncertainty ahead
Posted by mark in Nature, Uncategorized, Weather on June 30, 2023
After narrowly dodging Ian’s devastating blow last fall—predicted the day before landfall to hit just a few blocks from my southwest Florida winter home, I am keeping a close watch on this year’s storms.
Just prior to 2024 season on June 1, the National Oceanic and Atmospheric Administration (NOAA) predicted it would be near normal. The NOAA forecasters figure on the winds from the Pacific’s El Nino counteracting the storm inducing temperatures in the Atlantic.
A clash of the titans lies ahead as developing El Niño and notable warmth in the Tropical Atlantic go toe-to-toe.
Ryan Truchelut—the Weather Tiger’s Atlantic Hurricane Season Outlook for May 2023
The Weather Tiger (quoted above) calculated Florida landfall odds this year at slightly above 50/50 for at least 1 hurricane. That was before Tropical Storm Cindy formed behind Tropical Storm Bret in June, creating the first case of two storms in the tropical Atlantic in June since record keeping began in 1851–an alarmingly aggressive start to the season.
Based on these forecasts and the history of USA hurricanes, it seems certain to me that, before 2023 is over, our home will come into harm’s way. Therefore, I keep a close watch on NOAA’s graphical forecasts that display cones showing the probable track of the center of every tropical cyclone. These cones create a great deal of consternation and confusion due to difficulties comprehending probabilities, overly high expectations in the accuracy and precision of forecasting models, and other issues.
While admiring the continuing advancements in meteorology, including this year’s extension to 7 days for hurricane forecasts, I believe (but only half seriously) that if a weather forecast one-day ahead puts me at the bullseye of an oncoming storm, then it will be a miss. This worked for Hurricane Ian. But to hedge my bets, I greatly reinforced our home over the winter to resist wind, rain and flooding—bringing it all up to current hurricane codes and beyond.
Best be safe!
Masterful experiment delivers delectable chocolate chip cookies
Posted by mark in food science on June 27, 2023
There’s no better place to learn about design of experiments (DOE) than your own kitchen. Not being much of a cook or a baker, I do well by restricting my food science to microwave popcorn. Therefore, I happily agreed to help fellow DOE expert Greg Hutto advise his student Jessica Keel how to design an experiment on home-made chocolate chip cookies.
“Want to learn more in your own kitchen? Try making some cookies with different variations in ingredients. It’s a fantastic way to understand and help perfect your signature chocolate chip cookie.”
Danielle Bauer, The (Food) Science of Chocolate Chip Cookies
Optimizing cookies involves a tricky combination of mixture components and process factors. Furthermore, adhering to a gold standard for valid statistical studies—randomization—presents great difficulties. For each run in the combined design, the experimenter must mix one cookie according to the specified recipe and then bake it at the stated time and temperature. It’s much simpler to make a trayful of cookies with varying ingredients and bake them all at once. This can be accommodated by a specialized DOE called a split plot.*
Jessica took on a big challenge: Coming up with not one, but two chocolate chip recipes—soft-and-thick, versus thin-and-crispy. Starting from the specifications for Original Nestle Toll House Chocolate Chip Cookies, she used Design-Expert® software https://www.statease.com/software/design-expert/ to lay out an optimal, combined experiment-design based on a KCV model.** Jessica divided the runs into two blocks to spread it out over her Saturday-Sunday weekend. The experiment determined the effects of four recipe components—butter, granulated sugar, brown sugar, vanilla–baked while varying two hard-to-change process factors—temperature and time—in convenient groups (whole plots).
Jessica cleverly measured the density (by water displacement) and breaking strength (via the ‘penny test’ pictured) of each cookie before handing them over to her panel of tasters for sensory evaluation of taste, appearance and softness on a scale of 1 (worst) to 9 (best).
Focusing on taste alone, this combined mixture-process experiment led to a recipe—heavy on butter, vanilla free—that, when baked at the ideal conditions—325 deg F for 18 minutes—scores near perfect, as can be seen in the ternary contour plot produced by Design-Expert.
See Jessica’s full report for all the details . Then do your own optimal mixture-process experiment to ‘level up’ your homemade chocolate chip cookies. Yum!
*For details, see this tutorial from Stat-Ease that deploys a combined split-plot design to create a “rich and delicious” Lady Baltimore Cake .
**See my webinar on How to Unveil Breakthrough Synergisms Between Mixture and Process Variables.
Never ending quest for the perfect grind of coffee
Posted by mark in food science, pop on June 12, 2023
This graphic illustration from the National Coffee Association provides some amazing statistics in support of the claim that their beverage reigns supreme. I am doing more than my per ‘cupita’ (pun intended) of the nearly half a billion mugs of coffee that Americans drink every day.
Back before we all started working from home during the pandemic and kept on doing so afterwards, my son Hank (now VP of Software Development) and most of our Stat-Ease colleagues jived on java (the real stuff, not the coding language). He and our lead statistician Martin Bezener (now President) conducted a very sophisticated experiment on coffee-grinding, as reported by him in our September 2016 Stat-Teaser. Check out Hank’s dramatic video-detailing of the split-plot coffee experiment.
With the aid of Design-Expert® software’s powerful statistical tools, Martin discovered the secret for making delicious coffee: Use a burr, not a blade, grinder, and go for the finest granulation. Based on these findings, I upgraded my grinder to the highly-rated Baratza Encore, which works really well (though very noisy!).
However, a new study published this May in a Special Issue on Food Physics reveals an uneven extraction in coffee brewing. Evidently, “a complicated interplay between an initial imbalance in the porosities and permeabilities” creates “a cutoff point” where “grinding coffee more finely results in lower extraction.” Along the same lines, but with open content and some nice pictures and graphs to lighten up a lot of dense math (e.g., Navier-Stokes equations for fluid dynamics), see this earlier publication on Systematically Improving Espresso. It “strongly suggests that inhomogeneous flow is operative at fine grind settings, resulting in poor reproducibility and wasted raw material.”
So now that experiments show that finer may not always be better, the quest for the perfect grind continues!
2023 South Dakota Mines (SDM) paper helicopter flyoff
SDM Chemical Engineering Seniors Jarvie Arnold, Gregory Clark and Martin Gaffney, pictured left to right, ran away from the field with their “Team Helicopter?” flying machine.
With a lot of ingenuity and fine-tuning of the paper-helicopter design via a full two-level factorial, they achieved a flight time of 8.66 seconds from the balcony of the Chemical and Biological Engineering and Chemistry (CBEC) building. This nearly broke the all-time record of 8.94 seconds achieved by The Flaming Bagel Dogs in 2013.
Check out this awesome video from the 2011 flyoff and follow the link from there for background on the SDM paper-helicopter experiment, which I’ve been overseeing since 2004.
Kudos to Professor Dave Dixon for championing CBEC’s DOE class throughout the years. This elective rates well above any others in surveys of graduates, who say it was “immensely” helpful for their career advancement. I’m very thankful to be a contributor to this success story for teaching DOE at the college level.
Rock on SDM CBEC!
Hard-boiled Easter eggs not a-peeling
Posted by mark in pop, Uncategorized on April 17, 2023
One of my daughters provided me with half a dozen hard-boiled eggs left over from Easter. However, so far I have not cracked (pun intended) the process for peeling of their shells without also taking off much of the albumen (the white part on the outside). That leaves a very raggedy yolk with white bits hanging off. You could say that the yoke is on me (sorry, cannot resist). Even this supposedly “genius” hack (literally—using a spoon) failed miserably.
Unfortunately, my wife ate up the remaining stock before I could try more eggs-periments. She also experienced difficulties peeling them, which made me feel better (my dexterity falling fall short of hers). Should un-a-peeling eggs be encountered again, here are my other possible remedies:
- Shock them with an ice bath using my Waring Pro Wine cooler
- Soaking in vinegar per this procedure recommended by Gnarly Science blogger Dr C M Helm-Clark PhD
- Try an instant egg peeling device such as the well-reviewed Negg. (I am going to pass on the cheaper Eggstractor based on the hilarious difficulties encountered by Insider product testers.)
Any other ideas for easy and reliable egg peeling are welcome!