(Click the graphs once or twice for a full size view.)
A large number of media outlets and blogs ran this AP story on congressional approval. USAToday carried it this way:
AP Poll: Congress' approval hits high point
Public approval for Congress is at its highest level in a year as Democrats mark 100 days in power and step up their confrontation with President Bush over his handling of the Iraq War, the issue that overshadows all others.. . .
The findings from an AP-Ipsos nationwide poll provide a snapshot of public sentiment in the days after the House and Senate triggered a series of veto threats from the president by passing separate bills that provide funds for the war, yet also call for the withdrawal of U.S. combat troops.
Overall approval for Congress is 40%. The survey shows Bush's approval ratings remain in the mid-30 percent range ...
Which is all well and good except the AP poll is quite high compared to other recent polls of congressional approval, including AP's previous polls.
Of the last six polls the three most recent have come in above the previous trend estimate of 30.8%: Gallup has it at 33% (up from their previous 28%), AP at 40% and the Hotline at 37%. (Gallup has done two of the last six, so appears twice in the figure below.) The effect of these new polls is to revise my estimate of congressional approval up to 33.1%. But that is still a good deal shy of the AP result at 40% and a bit south of Hotline at 37%.
With the addition of the three new polls, what had looked like a pretty steady downturn in congressional approval since January now looks pretty flat. For the Democrats, that's a definite improvement, but it certainly isn't the large upturn that the AP story (and its reverberations in the blogosphere) have taken it to be.
Notice in the figure above for the AP, that AP polling has generally tracked quite well with my trend estimate of congressional approval. Only this most recent poll is well above the trend, with all other AP polls hugging the blue line pretty tightly. So AP doesn't routinely produce high congressional approval numbers, just this time.
As always, one poll should not dominate our interpretation of political dynamics. In this case, it is possible that something dramatic has suddenly boosted congressional approval, and AP (with the Hotline a couple of days earlier) was just lucky enough to be in the field right after it happened. We can offer reasons why this may or may not be the case, though I prefer to just wait for more data to settle the case one way or another.
But what we can do is look at how the trend estimator varies, and how the polling varies around that trend. The plot below shows the blue trend line with a gray region representing the variation in that estimate from 30,000 "bootstrap" estimates. A bootstrap is a good way of estimating the variability of an estimator, especially as in this case when we don't wish to make strong assumptions about the statistical distribution of the variation. The gray region represents the range of estimates of congressional approval we might reasonably expect to see, given the randomness of the polls and of events that drive congressional approval. I've overlaid the actual poll results on top of this, in red, so it is clear how the estimator varies (the gray area) and how the polls vary (the red points) all around the estimated trend (the blue line).
So what do we see? Quite a bit of variation. Congressional approval is asked a good deal less often than is presidential approval, so with less data, the gray area is relatively wider than it is for presidential approval, especially early in the series when very few polls were available for the early 1990s. That's somewhat better now, and the range of the gray area is smaller now than at the beginning.
The other variation is the polls. They are distributed roughly equally above and below the trend line, and spread a little bit more than does the estimator's gray area.
And right at the end, you can see that the AP data point at 40% is at the extreme upper end of the range of plausible estimates.
This doesn't mean it is necessarily wrong, and it certainly doesn't mean the poll was flawed. ALL poll vary, and individual questions vary as well. It is perfectly normal to see this kind of range of variation, as looking at the spread of the other red data points clearly illustrates.
But before we write a headline about support for congress being at a new high point, it would be worth considering the evidence a bit more carefully. Judging from all the data we have in hand today, it would be rash to suggest that congressional approval is at 40%. A much more plausible estimate is where the trend estimator stands: 33.1%.
And that changes the interpretation of the story which plays on Congress at 40% vs Bush "in the mid-thirties". In fact, the best evidence as of tonight is that Bush is at 34.4% and Congress at 33.1%. Politically, that means neither has a clear edge in support over the other at the moment. (Caution: We should really look at approval of "the Democrats in Congress" if we want to directly address the question of which end of Pennsylvania Avenue enjoys more support in the coming struggle over the Iraq funding bill. And other questions too!)
One final caution is that the trend estimator is sensitive to the most recent polls. So these three new polls pulled the estimate up enough that what had previously looked like some modest decline since January now looks to be pretty flat. Is it? The graph below shows how sensitive the estimator is to each of the last 20 polls.
The results show that for much of the last 20 estimates, the trend has been moving down. That has only been arrested as the last three polls have been added. So any conclusion about whether congressional approval is declining, steady, or increasing, rests on the thin reed of just three polls. Before we reach bold conclusions about the trajectory of public support for Congress, we should take some strong caution pills, and look at each new poll with some healthy skepticism until it gives us reason to trust it.
