(Click on the figure, and click again for best resolution.)
UPDATED: The graph has been updated to include the April 2006 data released today by SurveyUSA.
An article by Richard Morin in yesterday's Washington Post presented evidence that approval of President Bush has fallen considerably in "red" states. Jay Cost at RealClearPolitics has written a nice critique of the Morin article, pointing out that Morin is aggregating across states rather than looking at individual states and that his sample size is probably quite small in important cases. Both pieces are well worth reading.
The problem for Morin is that we want to know about state level approval but national polls are ill suited to such estimation. Morin uses only the latest Washington Post poll, divides the sample into residents of strong Bush, close Bush, close Kerry and strong Kerry states and presents his results for these categories of state. As Cost points out, some of these categories have few cases in a national sample, and the large margin of error that results makes his conclusions statistically shaky.
But there is an alternative: SurveyUSA has been doing monthly polling in each of the 50 states since May 2005. Each state has 600 respondents each month, making the small "n" problem go away. SurveyUSA uses a conventional sampling frame for random digit dialing, so their initial selection of numbers should be as good as any telephone poll. But SurveyUSA then uses a recorded voice to ask questions and respondents must enter responses by pushing the buttons on their phone. This method is viewed with deep skepticism by the conventional polling industry. To SurveyUSA's credit, they have published response rates on their website and provided analysis that addresses the concern that automated surveys produce such low response rates as to be unreliable. This isn't all there is to the problem of validating their methodology, but SurveyUSA has been more open about the issues, and has provided better data, than virtually all other survey houses, whether conventional or automated calling.
But the bottom line is that if you want to ask what has been happening in the states over the past year, SurveyUSA is the only game in town. The Morin and Cost pieces have pushed me to go with this post a little sooner than I had planned, and I think it is important that we examine the SurveyUSA data closely. However, the data have the advantage of consistent methodology over time and consistent sample size across states. They also pass the "inter-ocular" test-- the low approval states and the high approval states all "look right".
Read the figure from the lower-left to the upper-right to see approval from the lowest to highest. Look at the trends within and across states to see what has been happening over this time.
Some trends are stronger than others. Of the "red" states, the trend is bad news for President Bush in places like KY, NC, SC and NE. Others are quite stable: AL, MS, OK. The trend is down somewhat in more places than not, and that includes some but not all of the "red" states. I'll leave it as an exercise to the reader to count which states are down and which not.
Be careful of the scale: Nationally over this period, approval has only moved some 10-15%. The gray background grids are divided into 15% intervals, so that gives you a good sense of the scale of the change.
Click here to go to Table of Contents
Data: These data are provided by SurveyUSA on their website here. SurveyUSA deserves thanks for making this information widely available. The sponsors of the survey in the states also deserve acknowledgement, which follows below.
Alabama WKRG-TV
Alaska SurveyUSA
Arizona KPNX-TV
Arkansas KTHV-TV
California KABC-TV KPIX-TV KXTV-TV KGTV-TV
Colorado KUSA-TV
Connecticut WABC-TV
Delaware WCAU-TV
Florida WFOR-TV WPTV-TV WFLA-TV WTLV-TV WKRG-TV
Georgia WXIA-TV WTLV-TV
Hawaii KHON-TV
Idaho SurveyUSA
Illinois KSDK-TV
Indiana WXIN-TV WHAS-TV WCPO-TV
Iowa KAAL-TV
Kansas KWCH-TV
Kentucky WHAS-TV WCPO-TV
Louisiana SurveyUSA
Maine WCSH-TV WLBZ-TV
Maryland WMAR-TV WUSA-TV
Massachusetts WBZ-TV
Michigan WDIV-TV WZZM-TV
Minnesota KSTP-TV WDIO-TV KSAX-TV KAAL-TV
Mississippi SurveyUSA
Missouri KSDK-TV
Montana SurveyUSA
Nebraska SurveyUSA
Nevada KVBC-TV
New Hampshire WBZ-TV
New Jersey WABC-TV WCAU-TV
New Mexico KOB-TV
New York WABC-TV WNYT-TV WGRZ-TV WHEC-TV
North Carolina WTVD-TV WFMY-TV
North Dakota SurveyUSA
Ohio WKYC-TV WYTV-TV WCPO-TV
Oklahoma KFOR-TV
Oregon KATU-TV
Pennsylvania WCAU-TV
Rhode Island WLNE-TV
South Carolina WCSC-TV WLTX-TV
South Dakota SurveyUSA
Tennessee WBIR-TV
Texas WOAI-TV KEYE-TV
Utah KSL-TV
Vermont SurveyUSA
Virginia WUSA-TV WDBJ-TV
Washington KING-TV KATU-TV
West Virginia WUSA-TV
Wisconsin WDIO-TV
Wyoming SurveyUSA
Tuesday, April 18, 2006
Bush approval in the 50 states from SurveyUSA
Subscribe to:
Post Comments (Atom)
3 comments:
Charles, a real contribution here!
But a perhaps-naive question: Is a sample size of 600 in California "consistent" with a sample size of 600 in Wyoming?
Matt,
That's a good question, and I get it a lot so this is a good chance to answer it.
The margin of error for a "simple random sample" depends on three things: the sample size, the proportion of the population in the category of interest (e.g. "approve"), and the fraction of the population included in the sample. The three things matter in that order, with the fraction of the population being by far the least crucial for opinion surveys.
Here is the math.
The margin of error is
2 sqrt((1-f)p(1-p)/n)
where f is the fraction of the population included in the sample, p is the proportion in the category of interest ("approve"), and n is the sample size.
The "sampling fraction", f, is n/N, where N is the population size and n is the sample size.
So for Wyoming, f=600/506000=.001186.
For California, f=600/35894000=.000017
So that is a difference of two orders of magnitude. Quite a bit. But what matters for the formula above is 1-f. So for Wyoming we multiply p(1-p)/n by 1-.001186=.998814. For California 1-f is 1-.000017=.999983.
So there IS a difference, but both of these round to .999, which is pretty close to 1.
So for p=.5 and n=600 we get
WY:
2xsqrt(.998814x.5x.5/600)=.040801, or 4.0801%.
CA:
2xsqrt(.999983x.5x.5/600)=.040824 or 4.0824%.
So the sampling fraction only matters at the 3rd decimal point, which is far too small an effect to matter when the sampling error is on the order of 4%.
The sampling fraction does matter in other sampling problems where we might have a significant fraction of the population in the sample. But when we have such small populations, we often examine the entire population-- for example, we COULD estimate the proportion of women in the U.S. House from a sample of 200 members of congress, in which case f=200/435, so 1-f=.54, but we'd be more likely to find the sex of all 435 and not have to estimate at all.
The p(1-p) term matters, but also not a huge amount. For p=.5, p(1-p)=.25. But for p=.4, p(1-p)=.24, a quite small change. And even for p=.3, p(1-p)=.21, smaller but not by a lot. So for approval between 50% and 30%, the p(1-p) effect is larger than for sampling fraction but not all that large.
The sample size, n, makes the most difference. Here are some margin of errors for p=.5 and various sample sizes (while ignoring f for the reasons above):
n=200: 7.07%
n=400: 5.00%
n=600: 4.08%
n=800: 3.54%
n=1000: 3.16%
Thank you for the information and thank you for the headache reading all those numbers will cause me.
--Bucky
Post a Comment