Lessons from GOTV experiments

The following analysis presents broad findings from recent voter mobilization experiments. Lessons are categorized according to the level of uncertainty surrounding the conclusions. The star rating system we utilize was created by Donald P. Green and Alan S. Gerber in Get Out the Vote!: A Guide for Candidates and Campaigns (2004), and here it follows the same criteria as in their book:

*** A three-star rating indicates that the finding is based on experiments involving large numbers of voters and that the GOTV tactic has been implemented by different groups in a variety of sites.

** A two-star rating indicates that the finding is based on a small number of experiments. We have a reasonable level of confidence in the results, but the experiments have yet to be replicated across a variety of political, demographic, or geographic conditions.

* A one-star rating indicates that the finding is suggested by experimental evidence but not demonstrated conclusively in one or more studies.

Some recent research findings suggest…

*** Personalized methods and messages work better. Across a number of mobilization experiments, one consistent finding is that more personalized messages are more effective in mobilizing voters. When Donald Green and Alan Gerber put forth this generalized finding in Get Out the Vote!: A Guide for Candidates and Campaigns (2004), they contended that door-to-door canvassing was the most consistently effective and efficient method of voter mobilization, and they suggested that the success of canvassing could be attributed to the personal, face-to-face delivery of the GOTV messages. In recent years, this finding has proven to be robust. More recently, however, experiments of professional and volunteer phone calls (Nickerson 2006b; Arceneaux 2006; Arceneaux and Nickerson 2006) have suggested that personalized messages delivered in a conversational manner over the phone may be as effective (and cost-effective) as canvassing. Experiments testing impersonal GOTV methods, such as mass email (Nickerson 2006d; Stollwerk 2006) and robo calls (Green and Karlan 2006; Ramirez 2005), exhibit another robust finding: they are chronically ineffective and inefficient means of mobilizing voters. While it is apparent that methods other than door-to-door canvassing, such as volunteer and professional phone calls, may approach the level of effectiveness and cost-competitiveness of door-to-door canvassing, many results suggest that it is the dynamic interaction of authentic person-to-person contact that is most important in determining whether a method will successfully mobilize voters.

*** Mass email has yet to be proven effective in a single experiment. David Nickerson (2006d) presents strong evidence that mass email, as it has been used, has no positive or statistically significant effect on voter turnout. Pooled results from thirteen GOTV email experiments demonstrate no support for the claim that email mobilizes voters. Alissa Stollwerk’s (2006) large-scale partisan mass email experiment corroborates this null finding. Such results probably understate, however, the importance of email and other online fora as means to coordinate GOTV efforts. Results of Joel Middleton’s (2006) quasi-experimental analysis of MoveOn’s efforts in the 2004 presidential election, suggest that email and online organizing can be instrumental in organizing other successful mobilization methods even if mass email GOTV messages as a direct mobilizing tools have yet to be proven effective in any experiment.

** Social networks and interpersonal influence can be powerful mobilizing forces. The success of personally-delivered mobilization messages suggests that human interaction may be fundamental to the success of mobilization campaigns. Building upon this premise, recent voter research has begun to explore the effectiveness of mobilization carried out through interactions between members of voters’ own social networks. In his study of the contagion effects of canvassing, David Nickerson (2006e) argues that mobilization studies may underestimate the impact GOTV efforts on turnout, because they fail to account for the contagion effects of mobilization. Nickerson finds that the uncontacted resident of a canvassed two-member household votes at a higher rate than control group voters, meaning that the contacted member of the canvassed household communicates the GOTV message to the other member of the household. Alan Gerber, Mark Grebner, Donald Green and Christopher Larimer (2006) find that voters turn out at higher rates when they believe that the status of their participation will be made public within their neighborhoods. Joel Middleton (2006) employs quasi-experimental regression discontinuity analysis to estimate strong effects of highly personalized neighbor-to-neighbor mobilization in a competitive presidential election. The research on social networks and interpersonal influence, while under explored in the experimental literature, suggests that 1) personalized messages delivered by personal contacts may be more effective than personalized messages delivered by people outside a voter’s social network, and 2) social networks and interpersonal communication may effectively induce positive externalities of GOTV campaigns by proliferating mobilization messages once received to others in the network and, alternatively, by exerting strong social norms of participation within the network.

** The content of mobilization messages is not as important as the quality, timing and delivery of messages, which is not to say that message content does not matter at all. In 2000, when Alan Gerber and Donald Green published results of canvassing, phone calls and direct mail mobilization experiments, they noted that turnout did not vary significantly across messages of differing content (civic duty, close election and neighborhood solidarity). This early finding has proven to be robust across many subsequent experiments. David Nickerson (2006a) in comparing the impacts of professional and volunteer phone bank campaigns on turnout, finds little difference in mobilization effects associated with three different phone scripts (civil rights, terrorism and generational solidarity). Nickerson finds that the manner in which the messages are delivered (whether they are personal and conversational) is more important than the content of the messages themselves. Kevin Arceneaux and David Nickerson (2006) find no statistically significant difference in the effects of personally delivered phone and canvassing messages that are positive or negative in content. Neema Trivedi (2005) finds no statistically significant difference in mobilization effects among Indian-American voters who received direct mail pieces of differing content appealing to ethnic identity. Finally, an experiment conducted by Donald Green and Dean Karlan (2006) testing the effects of robotic calls on turnout, results in no statistically significant difference in turnout between the treatment group that received a GOTV message and the treatment group that received an election protection message. Two interesting exceptions that warrant further research may be found in David Nickerson’s (2006c) exploration of the impact on the timing of message delivery on turnout and Elizabeth Bennion’s (2005) Indiana student canvassing experiment during a competitive Congressional election. In a sub-experiment, Nickerson finds that messages that focused on comparisons of candidates were more effective than messages that focused on comparisons of parties in mobilizing turnout. Results from Bennion’s experiment suggest no difference between the effects of civic duty and close election messages among the entire sample; however, voters under the age of thirty were particularly motivated by the civic duty message.

** Volunteer and professional phone banks can both be effective (or ineffective). The debate about the effectiveness of phone bank campaigns on voter turnout originated from Donald P. Green and Alan S. Gerber’s (2000) findings that phone calls produced no positive substantively or statistically significant effects on voter mobilization. The original question revolved around whether or not phone calls could increase turnout at all. Subsequent studies have examined differential voter mobilization effects of partisan versus nonpartisan phone calls and volunteer versus professional phone banks. Up to this point, more experiments have found positive and significant effects of nonpartisan phone calls than partisan calls. However, there is not sufficient evidence to yet draw meaningful comparisons between the effectiveness of partisan and nonpartisan phone calls, particularly because only one experiment to date has done so in parallel (Panagopoulos 2006b); the results from which are only suggestive. John McNulty (2005) reports results from four phone bank mobilization experiments; while none of the experiments compares parallel partisan and nonpartisan message treatments, only the nonpartisan phone drive results in positive and significant. McNulty, therefore, contends that partisan phone drives may not be effective in increasing turnout. Green (2004, 2005) finds no evidence that partisan phone calls made by professional phone banks are effective in increasing turnout. Emily Cardy (2005) also finds that partisan phone calls do not increase turnout in the context of her phone and direct mail experiment. Findings from Costas Panagopoulos’ (2006b) experiment that entails the only direct parallel comparison between partisan and nonpartisan GOTV messages delivered by a professional phone bank, are suggestive of the conclusion that nonpartisan phone messages may be more effective than partisan messages in increasing turnout. Contrary to these conclusions, results from experiments by David Nickerson, Ryan Friedrichs and David King (2006) suggest that volunteer partisan phone calls are as effective at mobilizing voters as nonpartisan volunteer phone calls. The question of whether professional or volunteer phone banks are more effective in increasing turnout is equally as contested as the debate about partisan versus nonpartisan messages. As in the debate between partisan and nonpartisan content, to date, only one experiment has directly compared the effects of parallel professional and volunteer phone bank treatments on voter turnout (Nickerson 2006a). Results from several experiments have demonstrated the effectiveness of volunteer phone banks. As indicated above, Nickerson, Friedrichs and King (2006) find that partisan volunteer calls can increase turnout. Arceneaux and Nickerson (2006) find that personally-delivered phone messages by nonpartisan volunteers can be effective in mobilizing voters regardless of the positive or negative tone of the messages. Results from Janelle Wong’s (2005) experiment testing the effects of GOTV initiatives on voter turnout among Asian Americans and Ricardo Ramirez’s (2005) experiment examining effects of various mobilization methods on Latino voter turnout both demonstrate that nonpartisan volunteer phone banks can be effective in increasing mobilizing voters. As mentioned with regard to partisan or nonpartisan call content, Green (2005, 2006) and Cardy (2005) found that partisan phone calls placed by professional phone banks were not effective in increasing turnout. Nickerson (2006a), however, is the only scholar to have directly compared professional and volunteer phone banks in the same experiment. Nickerson’s findings suggests that whether or not callers are seasoned, paid professionals matter less than the quality and timing of the delivery of the messages. Mobilizing voters through phone calls may be a question of volunteer or professional oversight as well as strategically timing when the calls are placed in the course of the campaign. While a sufficient number of studies demonstrate that person-to-person phone calls can increase turnout, more research is needed to draw conclusions about the effectiveness of variations in the source, content, timing and quality of phone calls in mobilizing voters. At the present time, it is too early to predict with certainty which calling methods will consistently be most effective across varied electoral contexts and subject populations.

* Mass media has potential to be effective, but has been under measured as a mobilization tool. Radio, television and newspapers all share in common the fact that they are media capable of reaching a large number of voters at once. Very few randomized field experiments, however, have explored the effects of radio, television and newspaper ads on turnout. Donald Green and Lynn Vavreck (2006) present results from an experiment examining the effects of Rock the Vote television ads on turnout that suggest that such nonpartisan public service announcements are effective and cost-competitive methods ($14 per additional vote) for mobilizing turnout, particularly among the targeted age group of 18 to 24 year-old voters. Green and Panagopoulos (2006) found nonpartisan public service radio announcements in mayoral elections could efficiently mobilize voters ($10 per additional vote) and increase the competitiveness of elections. Panagopoulos (2006c) found that half or full-page newspaper ads are effective and cost competitive in turnout out voters ($5 per additional vote).

Visit our Research page to see information about ISPS studies, and the ISPS Data Archive for original data from field experiments.