Appendix C — DAY 1 — DISCUSSION ON THE FIRST PARADOX

~ 16 min reading

In the “Springboard” article just mentioned on Day 1 page 8, as indeed in much other writing, I have largely followed the guidance of Dr Donald J Wheeler by using some language which is different from traditional terminology. Amongst living statisticians, I regard Don as the “world master” in understanding, using and teaching control charts in a way that is consistent with what Dr Shewhart gave us. Several years ago, Don became concerned that the word “control” was causing misunderstandings. Indeed, we shall show (and, more particularly, you will show) on Day 3 that attempts to “control” processes can often do more harm than good. So it was he that popularised the term “process behaviour chart” as an alternative to “control chart”—more of a mouthful but a far more appropriate description. Similarly, he preferred the term “process limits” rather than “control limits”. I used Don’s terms in the Springboard article, but I have already explained why I’ve mostly retained the traditional language during this course (see the small-print paragraph beginning at the bottom of Day 1 page 6).

Another great value of Don’s work is that, whereas it used to be the case that control charts were generally considered to require small samples of data to be available at each time-point represented on the chart, he popularised an excellent and simple method of how to construct control charts when only one value is obtainable at any particular time. Samples are often easily obtainable in manufacturing processes, but single values are all you can get with the majority of other types of process. Since most people only have “one-at-a-time” data available in their processes, that type of data is all I consider in the main material of this course, in the Springboard article, and in the case studies covered in both ST and EST. However, in the latter little books, I do also briefly describe charts that are suitable for “a-few-at-a-time” data. If you are interested in reading much more on control charts than is needed for this course then there is plenty in the “Optional Extras” section (file S) which was introduced on page 8 of the introductory “Welcome”.

In the extract from EST on Day 1 page 6, I referred to the control chart (process behaviour chart) as being “a remarkable statistical technique which, to all intents and purposes, was unknown at the time of the original edition”. It was actually the above point to which I was then referring. In contrast, versions of control charts suitable for “a-few-at-a-time” data had already been well-known for decades, although mainly in manufacturing areas. Versions for some very specific kinds of “one-at-a-time” data were equally well-known. We shall be using one of those when we get to the Red Beads Experiment on Day 2 since the data there turn out to be suitable for that version. But what was still largely unknown around 1980 was the version suitable for virtually any type of “one-at-a-time” data—i.e. the only kind of data that the large majority of people ever see or use! This is why Don’s contribution was, and is, so incredibly important.

If, either now or subsequently, you wish to obtain more substantial help and guidance on constructing and interpreting control charts than is included in my Springboard article or during this course (including the Optional Extras), I can do no better than point you to two of Dr Wheeler’s many fine books. These are firstly the superb and relatively slim introduction I have already mentioned on Day 1 page 5: Understanding Variation—the Key to Managing Chaos, and then the more substantial Making Sense of Data: SPC for the Service Sector.

But now let us return to the first paradox summarised on Day 1 page 7: the apparently strange matter of neither the all-important tool of the control chart nor its creator, Dr Shewhart, even being mentioned in many Statistics courses nor, I suspect, even being known to most teachers of Statistics. Why?

I am tackling the first paradox here rather than in the main text because I do need to refer to matters covered in the usual kind of Statistics courses. If you have “no previous knowledge” of Statistics then there will be several matters that I’ll mention here which will be unfamiliar to you. But don’t worry about that—I think you’ll still get the gist of what I’m saying!

The subject of Statistics has largely been developed by mathematicians: indeed it is sometimes regarded simply (and wrongly) as merely a branch of Mathematics. As you know, with regard to Statistics we are concentrating on “understanding variation”, and I believe that most Mathematical Statisticians would not object to that little phrase as an apt description of their subject. So what’s the difference?

Let’s consider how a mathematician tackles his problems. It might appear ambitious to try to describe that in a single sentence, but here’s my attempt! Mathematics essentially consists of virtually legalistic arguments: initially some definitions and terms and conditions are specified, and then everything that follows is deduced by pure logic in accordance with those terms and conditions. And that’s fine for arithmetic, algebra, trigonometry, calculus, etc.

Mathematics has, of course, contributed greatly to the development of Statistics. And, reflecting our interpretation of “Statistics” as “understanding variation”, Mathematics has indeed contributed greatly to understanding variation of many kinds, rather than just with our focus on process data, i.e. data from processes that are recorded over time. But, with the latter being our focus here, Mathematics turns out to be overly restrictive. Since a mathematical argument depends on its terms and conditions, what follows by logical argument will naturally be true under those terms and conditions. But not necessarily otherwise.

And there’s the difficulty. What kind of “terms and conditions” could possibly be specified that would apply in practice to data from processes of all the kinds that we might want to study (manufacturing processes, administrative processes, financial processes, service processes, management processes, sales processes, medical processes, etc, etc)? In particular, Mathematical Statisticians love to assume that their data are normally distributed: beautiful mathematics can be carried out if that’s included in the terms and conditions. And it is included in the terms and conditions for several statistical methods that are included in the conventional Statistics courses, e.g. t-tests, F-tests (including the tests used in Analysis of Variance and Experimental Design), the derivation of tests and confidence intervals in regression and correlation problems, etc. But the truth is that most (some would say all) real data from real processes are not normally distributed, maybe are not even approximately normally distributed—or maybe don’t even have a distribution at all! What can be done then? That’s a question which mathematicians tend to shy away from, because their beautiful mathematics just cannot be carried out in the absence of some such assumptions.

In our context, and with our special interest in processes being in, or at least heading toward, statistical control, we might try to narrow the breadth of the difficulty by proposing that the terms and conditions need relate only to the state of statistical control. And that is precisely what Shewhart initially did. I’ll let him explain in his own clearly autobiographical words what happened (this is from page 12 of his 1939 book: Statistical Method from the Viewpoint of Quality Control):

“Some of the earliest attempts to characterise a state of statistical control were inspired by the belief that there existed a special form of frequency function [nowadays usually called a probability density function] f and it was early argued that the normal law characterised such a state. When the normal law was found to be inadequate, then generalised functional forms were tried. Today, however, all hopes of finding a unique functional form f are blasted.”

He could hardly have expressed it more pointedly than that! So what’s the alternative?

I’ll prepare the ground by quoting verbatim from a magnificent presentation that Deming gave to an audience at the Palace of Versailles, France in 1989, transcribed in BDA Booklet A6: Profound Knowledge; here I am quoting from pages 3–4 of this booklet. (You will see further extracts from this presentation this afternoon and on Day 3.)

He was talking about the two obvious kinds of mistakes that can be made when analysing process data: i.e. concluding that a process is in statistical control when it isn’t, and vice-versa:

“So what shall we do? Anyone can set for himself a clean record from this hour henceforth on one of the two mistakes. But, if he does, he will achieve the maximum loss from the other kind of mistake. This is a very easy way of making decisions: attribute anything that happens to a special cause—or attribute anything that happens to common causes. (A lot of things are easy to do!) So you minimise the loss from one kind of error at the expense of maximising the loss from the other kind. Yes, you can always minimise one, but not both—not both.

Either kind of mistake causes loss. There is no way to avoid all of it—you can forget that! So we must resign ourselves to making both kinds of errors now and again—we hope not too often. We must aim to make them with minimum overall economic loss. It is all a matter of give and take. What shall we do? How shall we do it? Dr Shewhart helped us with these important questions, and this was a great contribution to man’s process of thought and ability to manage.

How can we aim for minimum economic loss? It is nothing to do with probabilities of the two kinds of mistakes. No, no, no, no; not at all. What we need is an operational definition of when to look for a special cause, and when not to. That is, a rule which guides us when to search in order to try to identify and remove a specific cause, and when not to. It is not a matter of probability. It is nothing at all to do with how many errors we make on average in 500 trials or 1,000 trials. No, no, no—it can’t be done that way. We need an operational definition of when to act, and which way to act. Shewhart provided us with a communicable operational definition: the control chart using 3σ-limits. Shewhart contrived and published the rules in 1924: 65 years ago. Nobody has done a better job since.”

After that final paragraph, those of you “in the know” will, I think, now begin to understand why “traditional” Mathematical Statisticians have tended to shy away from the control chart! But statisticians who are genuinely concerned with analysing real data from real processes owe Shewhart a considerable debt of gratitude in that he created and developed the control chart in a way which, truth to tell, does not depend on any such assumptions nor on “beautiful mathematics”!

Yet some people still try to claim that you must have normally distributed data in order to be able to use control charts—and many of them still insist that Shewhart said so!

Three quick notes here. Firstly, Deming’s use of the terms “common cause” and “special cause” will be described this afternoon. However, if you’d like to check on that now, see “bare bones” 3 and 4 on Day 1 page 32. Secondly, operational definitions will be studied on Day 11. And finally, “σ” (a Greek letter, pronounced “sigma”) is a symbol commonly used by statisticians to represent a measure of variation.

There will be much more about these matters, including the “3σ-limits” mentioned near the end of the above extract, during the Optional Extras. Here I’ll focus directly on the main point of contention as far as the orthodox Mathematical Statistician is concerned.

Shewhart’s general ideas leading to the construction of the control chart are not contentious. But the stage comes when he needs to propose in detail the criterion for judging the process to be in or out of statistical control. Bearing in mind some of what Deming said above, the nature of what follows may now be of little surprise (although it might have been before you read that extract from the Versailles presentation).

In effect, Shewhart was considering the tightness or the looseness of the criterion which should be used as the dividing-line between judging the process to be in statistical control or out of statistical control, and then acting on the basis of that judgment. On page 277 of his 1931 book he delivered his proposed criterion in a single short sentence of the form:

“Experience indicates that [his proposal] seems to be an acceptable economic value.”

No probabilities, no fancy theory, no assumptions about any particular probability distribution, etc—experience! Shewhart carried out a lot of experimentation with real data. Just like Deming in the Versailles presentation, Shewhart’s reference to “economic” implied reducing as far as practical the cost of the control chart giving the wrong advice: i.e. regarding the process as in statistical control when it isn’t and vice-versa. (Recall Deming’s discussion of the “two mistakes” on the previous page.) Shewhart’s extensive experimentation convinced him that his proposal (which we shall use on Days 2 and 3) seemed to do the trick. And as far as Deming was concerned more than half a century later, “Nobody has done a better job since.”

But, yet again, it was experience that was Shewhart’s deciding factor in deciding upon his proposal—not the “pure logic” of the mathematician’s usual way of solving problems. And I would suggest that most practical people would agree that, regarding practical problems, experience surely has a rather important part to play!

And so I think we now have several reasons to explain the paradox being considered. Why isn’t the control chart in most introductory (or even later) Statistics courses? First, we have the mistaken idea that the control chart is solely or primarily suitable just for manufacturing processes. But there are hosts of statistical methods used for studying manufacturing processes. So of course, if that were the prevailing idea, why should the control chart find a place in the standard courses on Statistics rather than any of the others? Awareness that the control chart is fundamentally important in studying processes of all kinds, especially the most important management processes, was effectively absent on this side of the world before Deming arrived on the scene in the 1980s. And, unfortunately, relatively few Mathematical Statisticians ever attended his four-day seminars. But, even if that awareness emerges, most mathematicians will still baulk at the idea of the crucial decision about the criterion to be used depending on experience rather than pure logic. That is not how most mathematicians think, and it is not how most mathematicians teach. So, even if its importance comes to be realised, it’s still not “real” Mathematics, so it has no place in their courses.

I know. I was educated as a mathematician. So I cringed at the very idea. I avoided teaching it. I was in print a long while ago describing Shewhart’s version of control charts as being “rough and ready”. At the time, of course, that was intended as a criticism. I now re-interpret the same description as praise.

You know that in 1985 I initially thought of Dr Deming as a statistician and a fine mathematician. Of course, I soon found out that he was much more than either of those. The same could surely be said of Dr Shewhart, and of Dr Wheeler, and of many others. But these three (and, I suggest, just a very few others) should be described as unusually wise members of that group. Let’s return to what we read from Shewhart near the bottom of page 3. His first thought about the nature of statistical control was that of a mathematician: i.e. maybe the relevant terms and conditions could include the Mathematical Statistician’s favourite of normality. But he found that, in practice, many real processes sadly do not fit that condition. So he continued as a mathematician would and tried more complex mathematical models. But they also turned out to be unsuitable. So eventually he concluded that Mathematics just could not provide a solution that would be appropriate in practice (rather than just in mathematical theory). And so he looked outside Mathematics.

And there is the special wisdom, shared by a few but (I believe) not by many Mathematical Statisticians. Their first port of call when tackling a new practical problem is indeed likely to be Mathematics. Their wisdom is to be able to accept, after due investigation, that Mathematics might not be able to provide the answer, and then to proceed accordingly.

That is wisdom which I most certainly did not have in 1985. So I shall be ever grateful that both Drs Deming and Wheeler proved to be such excellent and patient teachers and friends.

(Continue on Day 1 page 11.)

PAUSE FOR THOUGHT 1–c

My acquaintance was, in fact, also now a member of staff in another part of the same university. This was around the time that British universities were first being subjected to “quality assurance” initiatives, quality audits, etc. Having suffered from these, my acquaintance jumped to the conclusion that we in the Quality Unit were responsible for such things! If you, the reader, similarly have thoughts that the Deming philosophy might have any such connections, you will soon find the truth to be very different.

Why should a university teacher dislike “quality assurance”? Doesn’t he want to do a good job?

I believe that the large majority do indeed want to do a good job. But mostly they find that “quality assurance” hinders rather than helps. Yes, it can introduce some useful disciplines. But, to my friend and many others, the main effects are more and more paperwork, more and more “statistics”, judgment, fear, blame, “massaging the figures”, rules and regulations—which, the truth is, most teachers find obstruct the good job that they would like to do rather than aiding it. The comments about “performance indicators” on Day 7 will also be relevant here. What is needed is quality improvement, which turns out to be very different from “quality assurance”.

(Return to Day 1 page 21 or continue on Day 1 page 22.)

MAJOR ACTIVITY 1–f

Of course your behaviour would change—massively—depending on those different circumstances.

But you are still the same person. It’s differences in the system around you, and the effects of those differences on you, that change your behaviour and performance.

I venture to suggest that if just this one message coming from Dr Deming’s teaching were generally understood and appreciated then this world would be a better place. The emphasis would move away from attempting to improve performance by merely “doing things” to people toward concentrating instead on improving the system in which those people work and live. Please reread Day 1 page 41 which introduced this Major Activity—and see what you think.