Pruient Photographs (not really)
I had a lot of feedback from interested readers in regards to Stupid Six Sigma Tricks #5, which I jokingly called “Pop Stars Without Clothing,” and which was about the frequent misuse of statistics in the Six Sigma world. Taking heart from this response to what might have been viewed as dusty old statistics, I decided forthrightly to call this Stupid Six Sigma Trick #3, Prurient Photographs. Oh, all right, it’s not really about that either. I thought I’d continue my riff on the bad statistics that I run across frequently, so this is really SSST #3, The Bride of Bad Stats.
Now remember, I’m just a lowly engineer who views statistics as a way to get things done efficiently and economically. I can appreciate the staggering beauty of a complex statistical analysis, but when it helps solve a problem or make some money for a company, that’s when I get really interested. So, the issues I’m going to bring up are ones that not only happen frequently with Black Belts, but also end up costing companies money either through missed opportunities or incorrect conclusions. This isn’t to say that these are the only statistical errors costing Six Sigma companies money—just that these are the only ones I want to write about now.
So let’s continue with what we were doing before. I’ll pose some questions for fun to see how Black Belts would answer, and you can assess the results.
Measurement level
1. You’re interested in finding out if a process change significantly affects the area of manufactured disks, since this in turn proportionally affects a critical output variable called spin friction. You gather the data in the form of disk diameters, figuring that since that’s what you actually measure in the process and since they’re continuous data and you use the same formula each time to get the area, you can save yourself some calculations that way. Can you use a t-test on those data to detect a process change? Why or why not?
Sample size
2. Calculating a sample size is for wusses. (True/False)
3. If you answered “False” for no. 2, then answer this: The correct sample size needed to detect a shift in the mean of 2 units from its historical average in a newly modified process, which is estimated to have a standard deviation of 3.2 is __________ at an alpha of 0.05 and beta of 0.10?
Here’s the story on this. Once upon a time, I was consulting at a heavy-industry manufacturing plant, on one of the few lunch breaks those taskmasters allowed me, when two employees walked up.
“So you know stats, right?” I nodded, mouth full.
“OK, so we want to change X in the process to save some money, but the cost of a shift in the process output would be very high.”
I swallowed.
“So we took a sample of 10 with the proposed process and did a t-test. The p-value was 0.20, so that means that there was no change in the process output, right? We can go ahead and make the change?”
“Uhmm, well no. No, not necessarily. What sample size did you calculate?”
“Sample size? We took ten samples.”
“Yeah, I got that. But what did you get for your sample size calculation?”
After a long blank look from both sides, I said, “Okaaay, let’s talk about your acceptable levels of alpha and beta then.”
After another pause, one guy cleverly said, “Ha ha! That’s all Greek to me!”
I sighed. “So how did you decide to take a sample size of 10 then?”
One guy spread his hands, which I took to mean that since he had 10 fingers, well, that was as good a number as any.
Too bad he didn’t take off his shoes to count.
Normality testing
4. What’s the best test for normality?
In SSST #5, I talked about the risks and costs of using an individuals chart with non-normal data. It turns out that a number of statistical tests assume normality so as to work as advertised. If normality isn’t a good assumption, then you leave the realm of alpha and beta error and have no idea what the probability of making the right conclusion is. The normal distribution is common in many processes, so these tests are still very useful to know and use, and if you can reasonably assume normality, you have an advantage that translates into more power to detect what you’re interested in detecting.
But not all processes follow a normal distribution. In some cases you don’t even want a normal distribution. Now, I suppose that you could just always use tests that don’t assume normality, but you would lose a lot of ability to detect changes that might save you money, so we don’t want to do that if we don’t have to. On the other hand, assuming normality for everything is bad because that too can lead you to the wrong conclusion, which in the business world we call “career limiting” or even a “résumé-adverse event.”
Now a quick note here: all those different probability distributions in statistics are just models of reality. No real process is constrained to produce only output that follows the normal distribution—it’s only an approximation, and it’s only a helpful approximation if it’s a useful approximation. These models give us a useful way to understand what is going on in a process, but they don’t reflect the truth of what’s going on. We don’t have to prove that a distribution is normal, we’re going to say it is until we disprove it with an appropriate statistical test. But we have to do the test.
Effect of SSST #3
Aaaargh! I still have a lot more statistical errors and no more space, but I hope you get the idea. As I tried to convince you in SSST #5, we’re asking our Black Belts to save us money and solve our problems, but my experience is that most of them haven’t been trained to recognize the common pitfalls above, or many others.
Businesses want a panacea for what ails ’em, and sometimes this is reflected in the shallow “Six Sigma for Dingbats” training Black Belts receive. Six Sigma Black Belts can pay you back many times the investment of training them, even at the amazing rates sometimes charged to do so. By training a cadre of expert problem solvers devoted to fixing problems permanently, even if you fix only 10 percent of your problems each year, after five years you’ve fixed about 40 percent of your problems so that they stay fixed. But if your Black Belts can’t recognize these issues correctly, they will have missed opportunities and implemented solutions that end up costing you more than the problems they were out there to fix.
Ever heard of a company that implemented Six Sigma and did not make a profit? I have.
Again, you don’t have to train statisticians to be Black Belts, but you need to train your best process experts in sound statistical thinking for them to be able to capture these potential savings.
But I could be wrong.