Updated with polls through 8/9/06 See bottom of the post for the latest estimates and confidence intervals. The graph above is also updated through the latest polls.
I've often posted on "house effects", the tendency of polling organizations to systematically vary in their results from one another. It seems time to consolodate that information into a single post, rather than as part of a post on something else. So here is a post that will become the standard spot for current estimates of house effects.
The graph above plots the house effects for presidential approval for all pollsters in 2005-2006.
The house effects measure how far polls by a firm are from what would be predicted based on trends over time for the average firm. In other words, the average house effect is set to zero by design. A poll that is consistently 2% below what would be predicted for the date the poll was conducted will have a house effect of -2.0%. This model accounts for trends in presidential approval over time and for events, such as Katrina, that shock approval up or down.
The confidence interval in the plot represents the margin of error for the estimated house effect. That margin is wider for organizations that do fewer polls and also reflects variability in results for the house. The very wide margins for CNN/ORC, LATimes and Annenberg are due to those polls conducting only a single poll during the 2005-06 period. CNN/ORC in particular will shrink the confidence interval as more polls accumulate. LATimes has now partnered with Bloomberg so I'm developing estimates of their house effects separately from the old LATimes estimate.
These estimates are specific to the presidential approval question as posed by each organization. We would get different house effects for different questions, such as party identification. I may add house effects estimates for some of those other items later.
House effects are usually pretty stable over time, but I'll update this as more polls accumulate.
Here are the estimated house effects as of polling completed by 4/23/06. (Note that since CNN/ORC only have one poll, the confidence interval is very wide and the estimate below will certainly change with more data.)
CNN/ORC -2.92
Quinnipiac -2.72
Pew -2.32
Newsweek -2.19
Harris -2.09
CBS/NYT -1.88
AP -1.86
Zogby -1.14
NBC/WSJ -0.90
LATimes -0.55
Marist 0.20
Time 0.42
Fox 0.72
Gallup 0.81
LATimesBloomberg 0.84
ABC/WP 1.30
Cook 1.48
Greenberg 1.82
Hotline 1.89
NPR 2.01
Annenberg 2.42
Battleground 4.65
For another look at house effects see the superb graphs and discussion by Robert Chung here and also here.
UPDATE 4/27/06: Here are the current estimates with confidence intervals, for polls completed through 4/24/06. There is a small shift for NBC/WSJ due to its new poll. That estimates shifts from -.90 to -.73. Note that these are 90% confidence intervals.
Update: 8/12/2006: I've updated the estimates of house effects to reflect new polling in 2006. The table of new estimates and the confidence intervals appears below.
Click here to go to Table of Contents
6 comments:
Out of curiousity why is Rasmussen not on the list? Is it folded into some other category or do they ask their questions in a manner that makes a direct comparison impossible?
Tlaloc asked about Rasmussen.
I have generally stayed away from using "robo-polls" which use automated recorded voices to ask questions. This is a method that is not widely accepted by establishing polling organizations and which poses some difficult questions of validity. While polls using human interviewers are certainly subject to their own problems, they do represent the industry norm.
That said, I've used SurveyUSA data which are also collected by automated surveys simply because they are the only monthly, 50 state, survey data available. I also have used them because SurveyUSA has been careful to analyze their survey responses and to post that analysis on their website. Rasmussen did not (last time I checked) provide that level of transparancy.
I now think that these polls need to be looked at more carefully, and I'm in the process of doing that, as is Mark Blumenthal at MysteryPollster.
Because Rasmussen IS widely read in the blogosphere, and included in RealClearPolitic's polling summary, perhaps it is time to add him to these data. See MysteryPollster for some comparisons of Rasmussen with other polls.
Great work, but can you give the estimated confidence interval or standard error in the table of averages?
I've added a table with both estimates and CI. If anyone knows how to include an HTML table in a Blogger post I'd appreciate the info. It always leaves a HUGE gap at the top when I try it, hence the .png file here. My HTML skills are quite limited.
"I have generally stayed away from using "robo-polls" which use automated recorded voices to ask questions."
Gotcha. Thank you for the explanation. I read mystery pollster often but a lot of the nuance I know I miss.
This paper deals with ARCH effects of the monthly presidential approval series from 1978 to 2000. It is shown that inflation is another short-term effect on presidential approval. While unemployment influences on the level of presidential approval, inflation has an effect on the variances of it. To take into account of different kinds of effects of unemployment and inflation, Multiplicative Heteroskedasticity AutoRegressive Model is proposed. Divided government provides ambiguous and conflicting information about which branch of government to hold accountable for government performance. The implication for presidents, who are easy targets of blame, is that they are less likely to be held accountable for government's failures during periods of divided government because the public has a plausible alternative for affixing responsibility the U.S. Congress. We analyze the relationship between public approval and presidential success in Congress using time-varying parameter regression methods. Cues from constituency, ideology, and party dominate congressional vote choice, so the effect of public approval of the president is typically marginal.
-----------------------------
Jenifer
homes for sale by owner
Post a Comment