Bush Approval: Trend at 32.6%

There have been several polls since my last update, and they have varied much more than usual. But the bottom line, with all the data, is a small decline in the trend estimate, and that well within the uncertainty of the trend.

The newly added polls are:

Fox: 10/23-24/07, Approve 35%, Disapprove 55%
LATimes/Bloomberg: 10/19-22/07, Approve 35%, Disapprove 60%
ARG: 10/18-21/07, Approve 25%, Disapprove 67%
CBS News: 10/12-16/07, Approve: 30, Disapprove 61%
CNN/ORC: 10/12-14/07, Approve 36%, Disapprove 61%
Zogby/Reuters: 10/10-14/07, Approve 25%, Disapprove 75%

That is quite a range from two at 25% to a top of 36%. Zogby and ARG are well below the trend estimate, while the two at 35% and one at 36% are only a bit above trend. (An older NPR poll at 38% is well above expectations, but as I explained in an earlier post this has something to do with the likely voter sample NPR favors, compared to the adult samples of most approval polls.)
























If we look at the residuals, NPR looks like a high outlier and Harris a low one. But Zogby/Reuters and ARG are even more extreme low outliers. Zogby uses the same 4 point job rating measure that Harris uses, so that partially explains their low value (a persistent question wording effect) but as the plot makes clear, the Zogby reading is even lower than we'd expect from that. The ARG reading is also extremely low, at the same 25% approval rating, and the points and labels are overwritten by each other.
























The curiosity is why two (three with Harris) outliers? Did opinion change and these caught it early? Apparently not, judging by other recent polls that are at or even a bit higher than the trend estimate of 32.6%.

In general I think it is a bad idea to seek a substantive explanation for outliers. The most reasonable story is simply "random variation" and we should leave it at that. There ARE some systematic elements, such as the question wording variation or sampling issues I pointed out, but I'm not inclined to say more. To do so becomes a post hoc search for what are most likely statistical phantoms. (Though when history keeps repeating itself we might look into house effects for a systematic effect, possibly due to question wording, sampling, or treatment of don't know responses.)

The bottom line is approval may have shifted down a tad, from 33.0% to 32.6%. BUT, one should consider the gray region around the trend line below. This gives you a good idea of the uncertainty in the trend estimate itself, after squeezing out as much random variation in the polls as possible (at least until next week! Stay tuned for that!). Clearly the change of .4 percentage points is not a clear indication of movement in approval. In fact, given the wide range of current polling, we have an unusually wide uncertainty about where approval actually is at the moment, with 32.6% being our best estimate, but an uncertain one.

We are in a period of relative stability in President Bush's approval rating but considerable polling variation. Waiting for the next "thing" to happen.

Bush Approval: Trend at 33.0%

President Bush's approval trend has been relatively flat in recent weeks. For the last month the estimated approval has held between 32% and 33% and currently stands at 33.0%, which includes Gallup's 32% approval from polling done 10/12-14/07.

There was a fairly sharp upturn that started in July, which has tapered off but not yet disappeared. Poll to poll fluctuations have pulled the trend estimate around a bit in the 32%-33% range but without clear evidence of significant change.

























The last six polls also demonstrate some persistent differences between polls. The NPR Poll, conducted by Public Opinion Strategies (R) and Greenberg Quinlan Rosner (D) has a persistent positive house effect, making NPR one of the higher measures of Bush approval. In the opposite direction, Harris has a persistently negative reading of approval. In the figure below, we see that both are outliers in the current approval trend.

But this is a good example of when being an "outlier" doesn't necessarily mean anything is wrong with the polling. NPR samples "likely voters" and we know from other analysis that samples of likely voters are generally more approving of President Bush. Likewise, Harris (along with Zogby) uses a four point approval scale, "Excellent", "Good", "Fair" and "Poor" while everyone else uses "Approve" or "Disapprove". Harris and Zogby (and me) arbitrarily map "excellent" and "good" into "approve", and "fair" and "poor" into "disapprove". But is that the right way to make different answers comparable? Do the first two of these choices mean exactly what "approve" means? No. This is just the best categorization possible for mapping these four response options into two. And one consequence is that Harris and Zogby tend to come in below the estimated trend.

The choices these polling organizations make to sample likely voters rather than adults, or to use a four choice approval measure, are not in any sense "wrong" decisions. But they do mean that the results these pollsters get will be somewhat different from what others, using different samples and different questions, will find. But the choices DO make these polls more likely to be "far" away from the trend which is based on all pollsters.

So why call them outliers? Simply because they are, in fact, well away from the trend and the distribution of everyone else. In some cases that might indicate a "bad" poll, a sample that for some reason is far from what we would normally expect, or perhaps a biased question wording or sequence of questions. But being an outlier can also result from making choices of sample or question or other procedures which are different from most. Different yes, not necessarily bad.

But different is surprising, nonetheless. My goal here is to put everyone's polling in perspective, and these two do deserve to be labeled "outliers" because they are in fact different. A normal reader of polls without benefit of the perspective we provide here, would reasonably ask how can NPR get Bush at 38% and Harris have him at 27% (and the trend estimator be at 33%?) That surprise factor is also part of what being an outlier means. Without some further explanation (sample, question wording) the results appear puzzling. An outlier may be understandable or explicable, as both of these appear to be. Or in may not. Either way outlier detection draws attention and asks for an explanation or a discounting of the finding.