While unpopular presidents see their party lose more seats at midterm elections, the relationship is so noisy that predictions are relatively worthless. What is more, the simple prediction of seat loss from approval leads to predictions for 2006 that are wildly out of line with most informed opinion. This doesn't mean we shouldn't look at the data or the election forecasting models but it does require that we mix our enthusiasm for prediction with some sober realism about the uncertainty in such forecasts.
President Bush's recent decline in the polls has led many to conclude that Republicans in congress are doomed next November. I've pointed out here that the president's approval ratings are (almost) unprecedented for a midterm election, and that we simply don't know what a president with an approval rating in the high 20s or low 30s would do to his party's Congressional fortunes. I stand by that, but at the same time we need to take a look at the historical record of approval, seat change and the noise in that relationship.
The figure above plots the change in House seats for the president's party in each midterm election since 1946. (I left 1946 out in the earlier post, but reader comments have convinced me to include it.) In all but two years the president's party loses seats. That is one of the most reliable regularities in American politics, or at least it was until 1998. Prior to that, 1934 was the last time a President's party gained seats in a midterm. Before that only 1902 saw a gain since 1860. So to have two midterm gains in a row raises some eyebrows about the possibility that House elections may have changed in some fundamental way in the last eight years. Or maybe they just happen to be two exceptional years and not harbingers of real change.
But there is a reliable relationship between approval and seat change. Each percentage point of approval gained or lost by a president predicts about 1 more seat held or lost by his party in the House. At least when considering all midterms since 1946. By that count, President Bush's decline in approval from 42% to 33% since January should cost his party 9 seats. That's the bad news for Republicans.
But there are two problems with such estimates. The first is that House districts have become much more uncompetitive than in the past, and most analysts believe that the redistricting for 2002 produced some of the most partisan, and hence "safe", seats in history. If that is so, then the past may be a poor predictor of the future. Based on history, a president at 36.5% approval in April (where President Bush was), should expect to lose 45 seats in the House. That is wildly out of line with the best informed opinion which says that the Democrats will be very lucky to gain the 15 seats needed to take control of the House. A loss of 45 seats is far beyond anyone's current expectations. So one issue is whether the past is much of a guide to the current House. Those who know it best think the current House is too well insulated from electoral tides for such predictions to prove valid.
The second problem is the size of the variation in seat change given approval. In 1994, a president with an April approval of 51% found his party losing 54 seats in November. Less popular presidents Carter and Reagan (yes, really) both had April approvals of 44% in 1978 and 1982 but lost only 15 and 25 seats respectively. The variation around the blue regression line in the figure makes clear that for any given approval level, your actual results may vary. A lot.
For President Bush, with an April approval of 36.5 and a predicted seat loss of 45, this uncertainty translates into a range of predicted losses from the ridiculous (-74) to the plausible (-17). That range of uncertainty is so large as to be politically useless for estimating practical consequences. If you read forecasting models, look to see if they provide a confidence interval for their predictions. Most talk about their point prediction (xx seats) and a few about their R-square or the fit within the sample. But very few advertise their confidence intervals for the out of sample forecast. For good reason. It is almost always huge.
What if we cherry pick our data? In the graph above, I see 1946, 1958, 1966 and 1994 don't seem to fit with the rest of the data. If I drop these, then a line through the remaining points fits better and produces a slightly more plausible prediction of -30 seats (with a confidence interval of -45 to -16.) But this grossly overstates my true confidence! How do I know that 2006 will not be like 1958 or 1966 or 1994? Why not take out 1954 which now doesn't look so close to the line? For every decision I make like this, I actually ADD uncertainty to my true confidence interval, not take it away.
So what conclusion should we reach from presidential approval in the months leading to a midterm election? We have a very good basis for concluding that less approval means more seats lost. But the best estimates we can manage, given that we only have 14 elections with approval data, are so imprecise that their political implications are basically worthless. I'd bet a lot (ok, I don't bet so that's a cheap line) that the Republican party will lose seats in the House. And that the lower the President's approval rating, the worse they will do. But to put any meaningful confidence around a single point prediction (say, -15 seats or more) requires much more work and a good deal more calculation than just looking at the history of approval and seat change.
A number of political scientists and a few economists have developed models to forecast election results. Those use more than just approval, and achieve forecasts somewhat more precise than what is possible with approval alone. I'll be writing about those models, and estimating some of my own in the next few weeks. But keep your eye on the confidence interval. The uncertainty is larger than Democrats would like right now, and that's the good news for Republicans.
And to my (former?) friends who do forecasting: show me your confidence interval and I'll show you mine. I'll be happy to be proved wrong with legitimate tight intervals.
Click here to go to Table of Contents
10 comments:
One problem is that the forecasters who say "there aren't that many seats in play," "the Democrats will be very lucky to win 15 seats" etc. seem to be assuming that the rules of the 1996-2004 House electons are still valid. In each of those elections, very few seats changed hands, and when they did it was usually a seat that had had a very close race two years earlier--or else an open seat, or else a redistricted seat.
That was actually true of most elections in the 1970s and 1980s, too. Not many seats changed hands in, for example, 1970 or 1976 or 1978 or 1986 or 1988--or for that matter 1990. The GOP did gain in 1992, but only modestly, and even this gain was largely the result of redistricting and the House bank scandal (also perhaps of Perot voters).
But in exceptional years like 1974 and 1994, the general rules didn't apply. Seats became "in play" that wouldn't have been so in normal years. It wasn't just Democrats who had a close call in 1992 that were in trouble in 1994, and it wasn't just Republicans who had a close call in 1972 that were in trouble in 1994. There were a number of genuine upsets of people who had easily beaten weak opponents for years.
(And by the way I am old enough to remember people saying after elections like 1976 and 1986 that, whatever may be the case in the Senate, in the House incumbents were almost invulnerable because of the advantages of incumbency combined with gerrymandering. Phil Burton didn't need any computers to gerrymander the House seats from California...)
I am *not* predicting that 2006 will necessarily be another 1974 (or 1994 in reverse). What I *am* saying is that such exceptional years make nonsense of normal projections--and that forecasters do not seem capable of knowing in advance when an election year will be "exceptional" in that sense. Yes, they sensed that the Democrats would make substantial gains in 1974 and the GOP in 1994, but I don't think they got the *extent* of the gains right in either case. So when forecasters argue "Whatever the generic congressional polls may say, whatever the presidential approval polls may suggest, the Democrats just *can't* gain seats in very substantial numbers" I would be a little skeptical of them.
The way I looked at it was that there are 2 periods, and one out-lier, Before 1970, and after, excluding the 1994 blip. If we are in the after 1970 period, then it looks to me as though the 30 seat loss is reasonable.
The only other numbers are that the mode is about 19 seats with 18 in the 1970-2002 time.
Thanks,
Another interesting plot, along with May/October approval scatter plot.
The chisquare for the best-fit straight line seems horrible (assuming 3% stat errors). So perhaps one could think about other variables that would show a better linear relationship. How about difference in popularity?
(B/wn Reps and Dems). Does that give a better fit line? Or is party popularity such a squishy concept that you would rather stick with a solid concept like presidential popularity?
Another statement one could make looking at this plot is that very popular presidents (55% and up) can limit loss to 10s (or even gain a few) and unpopular prresidents tend to lose 20 to 50 seats. In other words, a flat horizontal line from 0 to 55, and a straight line fit for 55% and up.
Still horrible chisquared (and I've introduced another variable!)
We really need a model here, then we can be proper Bayseans. I await Prof. Franklin's.
Since the Dems only need 15 seats to win back the House, it looks like every single prediction in the confidence interval is enough for the GOP to lose control.
Noted election forecaster John Murtha says the Democrats will gain 40-50 seats.
Anonymous, and others elsewhere, have pointed with optimism to the fact that the entire confidence interval lies below -15, taking that as evidence for a Democratic win. That's one view.
I would say that I find the prediction sufficiently far from what I perceive as reality that it leads me to question the value of the model.
I DO think that the example of 1994 is a sobering one-- the conventioal wisdom can be very wrong in the end. So it might be that this model knows more than those who look at the races one by one. But if I had to choose, I'd doubt the model more.
This is why I want to look at a number of models over the next few weeks. Let's see how they do and what the range of predictions is. That will give more of a handle on what our expectations should be, and how those change over time. (And partisans can go work on the campaigns of their choice to make the predictions come true-- or not!)
Charles
For what it's worth, I ran a multiple regression model that combined presidential approval with economic growth (percentage growth in per capita real disposable income, 2nd quarter previous year to 2nd quarter in election year). Projecting Bush's Labor Day approval rating to be 40 percent (seems somewhat generous) and RDI growth to be 5% (historically high, but consistent with current trends), I get the Republicans losing 27 seats. Of course, I have a standard error of the residual of around 15 points. This gives me a 72% chance of the Democrats taking the House.
I don't have any real reason to believe that we've entered a different "era" in midterm elections since 1998. However, one thing I've found hard to control for is the polarization of House districts. Districts have definitely polarized, particularly over the past decade, making it less likely that a given district will change hands in an election.
So I should probably fudge that 72% likelihood downward towards 50-50, but now I'm just guessing.
Smotus,
Nice-- thanks for the data analysis. I think the problem is whether you believe the intercepts or not. The slopes in various models look fairly plausible to me. But the intercepts are what drives the prediction to -27, for example. Now that may in fact be right in which case the close observers of individual races will be a bit embarassed in November. But the dependability of the estimate of the intercept is what bothers me the most about these models.
Charles
Charles,
Are you aware of the midterm forecasting contest, sponsored by the forecasting group?
The most efficient multiple regression estimation that I am aware of is on p 156 of Gary Jacobson's The Politics of Congressional Elections. This estimation includes not only presidential approval and economic growth (variables mentioned in previous posts), but also "exposure," a variable that controls for regression to the mean (i.e. if the Republicans are currently over-represented in the House, given historical levels of representation, then we expect a slight loss). The adjusted R^2 is at 0.70--nothing to spit at.
For my students' use, I've posted a House elections predictor on my site based on Jacobson's analysis. It doesn't display the confidence interval that you mention (for simplicity), though I may add that in later. But if you plug in Bush's current poll numbers, recent economic growth, and the other variables, it predicts roughly a 26 seat loss for the Republicans.
Post a Comment