Final Pennsylvania Updates

Clinton has increased her lead in the trend estimates over the course of the last polls to 6.6 points using the standard estimator, and to 8.4 points using the sensitive estimate. Last minute polls have given her bigger margins.

Now the key question is whether undecideds push her over a 10 point win, or whether increases in turnout by new "unlikely" voters raises Obama's total.


















Still a good bit of variation and some pollsters see a strong trend, others not so much.


















Pollster variation doesn't make a lot of difference in our trend estimates.

But remember, since the polls don't allocate undecided, both they and the trend estimates are leaving some 8 percent of voters on the table. They will go somewhere, and if they break disproportionately for Clinton you have a "huge win", while if they go overwhelmingly for Obama you have a nail biter or a dramatic come-from-behind win. In previous primaries, the "winner" has usually enjoyed a significant increase in support beyond what the last polls showed.

Pennsylvania Pollster Comparison

Judge for yourself.

Day Before Pennsylvania Sensitivity Update

The Pennsylvania race has turned slightly toward Clinton over the weekend, with her lead now at an even 6 points in our standard trend estimate. If you believe in taking more chances with random noise, the sensitive estimator has a 6.4 point Clinton lead.

In the rush of new polling over the weekend, it is also good to check how much any of them may be affecting our estimates.


















Dropping any single pollster makes only a bit of different to our estimates. The Clinton trend ranges from 48.5% to 49.6%, while Obama ranges from 42.6% to 43.5%. So dropping your least favorite pollster can, at most, account for the difference in a 5 point race and a 7 point one.

And note that we still have about 9 percent undecided. I wonder what they will do?

Pollster Effects on Pennsylvania Trends

There is a lot of interest in the differences among pollsters, and especially what effect they have on the perception of the race. Here at Pollster, the interest is specifically on the question of whether individual pollsters drive our results, and if so by how much.

So here is a quick look at those effects in Pennsylvania's Democratic primary.

The chart above shows the trend estimates that result from dropping each pollster in turn, and reestimating the trend without that pollster. This is a specific test of how much it matters whether we include a particular pollster or not.

Over the 15 pollsters we have represented in Pennsylvania, the estimates excluding each one for Clinton range from 46.6% to 48.3. The estimate with all pollsters included is 47.4% for Clinton.

For Obama, the estimates range from 41.2% to 42.5%, with the estimate for all pollsters at 42.0%.

The upshot is that pollsters do matter, but none drive the results by very much. A 1.7% range on Clinton and a 1.3% range on Obama for the trend estimate is very small compared to the range we see across the raw poll results. Another example of the greater stability of the trend estimators we use compared to the substantial variability across polls.

We can look at the effects of each pollster by comparing the trend estimate without that pollster to the estimate with them. The higher the effect, the more that pollster drives our trend estimate up for that candidate. Negative effects means the pollster drives our estimate down. Again, this is compared to the trend with and without the individual pollster.

The two charts below show these effects for Clinton and for Obama.


















For Clinton, SurveyUSA has the highest positive impact on our trend estimate, followed by Rasmussen. At the opposite end, PPP has the largest negative effect on our trend, with Zogby/Newsmax the next largest negative effect. Other pollsters are clustered rather closely around zero.

And it is important to note that even the four largest positive and negative effects are all less that 1 percentage point.


















For Obama's trend, Quinnipiac shows the largest positive effect, followed by Zogby and PPP with near identical effects. On the negative end, SurveyUSA has the most negative effect of Obama trend. Again, none of these effects is as much as one percentage point, and SurveyUSA's is less that half a percentage point. Other pollsters have less impact.

We can also look at the joint effects. These are the same as seen individually above but plotted against one another.


















Pollsters do matter, and outliers matter even more. But the net effect of any individual pollster on our trend estimates in Pennsylvania are modest, especially when viewed in comparison to the wide range of raw poll results for each candidate. Another advantage of combining information across polls rather than pick single polls we "like".

Pennsylvania Dem Sensitivity Comparison

Here is an update on the Pennsylvania Trend Estimates for both the standard estimator and the more sensitive one.

The difference is small: 0.3 for Clinton and 0.8 for Obama. But hey, it can be a game of inches. Either way it looks like about a 5 point margin right now.

(See earlier posts for details on the standard and sensitive estimate. The sensitive is about twice as responsive to the data, but tends to chase ghosts.)

Pennsylvania Dem Trend Sensitivity

Time for a look at the sensitivity of our trend estimators. ARG has a new Pennsylvania poll out showing a 20 point Clinton lead. But Susquehanna Polling has one completed three days earlier with a 3 point Clinton lead and Zogby has one on the same day with a 4 point Clinton lead. Did things shift that swiftly or do we have an outlier?

Our standard trend estimator is designed to resist outliers, and it manages to do so in this case (see the chart above.) With or without ARG (and with or without ARG AND the close Susquehanna poll) the trend estimates only change by 10ths of a percentage point. What does change slightly is the slope of the trend estimate between the solid line with all polls and the dashed and dotted lines without ARG and without ARG and Susquehanna. In fact, the changes are slight enough you need to squint to really see them.

It may surprise you that removing ARG makes Clinton's trend estimate go UP, when you would understandably expect it to go down. The reason is simply that she is trending down either with or without ARG. Removing ARG means that the latest poll for the trend estimate is 3 days earlier, hence higher up on the downward trend. Since ARG doesn't much affect the slope, removing it just "backs the trend up" by three days, making Clinton a tiny bit better off. The same happens in reverse for Obama-- without ARG he is a shade worse off (0.1 percentage points) because his trend is rising either way, so backing up 3 days hurts him a tiny bit. (Removing ARG and the close Susquehanna poll makes similarly modest changes to the trend estimate.)

So for our standard estimator, whether we include or exclude the latest ARG poll makes very little difference.

That is by design. At this point a single new poll that shows a big change should be regarded with caution. It MAY reflect a big shift over the weekend. But it could just as easily be a statistical fluke that will not be replicated in other new polls. Until there is more evidence one way or the other, the trend estimate is designed to not chase after a single poll far away from the other data. But if we get two or three more polls showing similar results, the trend estimator will then be convinced the shift is real, and will turn in the direction of the new data.

But what if we decided to be less cautious and more willing to respond to new polling? For that, we have an alternative estimator that is about twice as sensitive. This one will pick up new trends much more quickly, but will also be misled by a single outlier much more easily. Let's see what this more sensitive estimator thinks is going on.

















With the sensitive estimator, the ARG poll makes a HUGE difference. It shifts the difference in trends from a 4.3 point Clinton lead to a 12.6 point lead! That is really responsive. But it also demonstrates what a large difference a single poll can make when we crank up the sensitivity of the trend estimator.

What is more interesting with the sensitive estimator is that taking out ARG, we see a flattening of Obama's trend and even a tiny downturn, though also a continued decline for Clinton. If we also take out Susquehanna we find Obama continuing to rise but Clinton falling at a slower pace than without ARG alone. (You really have to squint to see the dotted line. Sorry about that.) Clinton leads by 4.2 without ARG AND without Susquehanna, by 4.3 without ARG alone.

So perhaps the sensitive estimator is showing us something new-- Obama may have flattened or even turned down a shade recently, but if so Clinton seems to have continued to decline as well. This also illustrates why we prefer our more conservative estimator. The sensitive version is just too dependent on individual polls at the end of the series. It provides a possible early hint of things to come, but is too unstable to count on. Our standard estimator may be a little slow to pick up a change of direction, but it seldom chases after random noise.

As for ARG's result, let's wait a day or two and see what other polls have to tell us and how our estimator responds.

For an animation of the Pennsylvania sensitivity analysis for a wide range of sensitivity, download this Acrobat file and use the arrow keys to control the animation speed.