What the Left Can Learn from Trump’s Analytics Team
Yes, the Clinton campaign struggled with messaging; that doesn’t mean we should throw away everything we’ve learned about how data can make campaigns more effective.
Last week, the RNC analytics team published a document called the RNC Testing Booklet, outlining almost 70 A/B tests that they ran on www.donaldjtrump.com. The booklet describes everything from tests on what color donation buttons people like to click to which photo of Trump’s face would get more people to RSVP to an event. In spite of some amateur mistakes there are some important lessons that progressives can take away from this booklet.
Campaign operations are typically closely held secrets (much like national security discussions on North Korea’s ballistic missile tests), so it might seem strange that the RNC would want to share this material publicly. I was curious, so I took one for the team and read through all 155 pages of the booklet, including 133 photographs of Trump’s stupid grin. Yes, I counted.
It quickly became clear that the progressive organizing community has a lot to learn from Trump’s analytics team. To be clear, the RNC Testing Booklet is poorly done—I’ve graded Statistics 101 exams in the past, and the work described in the booklet would not have received a passing grade.
These people are supposed to be the Republicans’ top testing wizards, but their tables are mislabeled, the data presented is inconsistent, and some of the numbers they report don’t actually mean anything. Many of their statistical tests report two different “confidence” percentages. Without going too far into the weeds, and leaving aside the fact that both numbers are calculated incorrectly, you should only have one confidence percentage. Ever. There is literally no reason why you would ever have two percentages. It’s not just a mistake in making the calculations—it’s a mistake in even understanding what calculations they should be making in the first place.
In addition to their statistics being wrong, the “upsell,” which they call their “key game-changing online strategy,” would maybe have been innovative on the 2004 Bush campaign. They might not be publishing their most interesting work, but none of the individual experiments that they describe are particularly innovative or exciting.
Nonetheless, the report shows that Trump’s analytics team is doing certain things right. The entire document appears to be an advertisement for their work geared towards demonstrating how analytics can make a campaign more effective. Even at the cost of potentially sharing sensitive information, the men (yes, it seems that their analytics team is all men) behind this document seem to recognize that effective analytics programs require buy-in from all parts of an organization, and that it is necessary to advertise the value of analytics to get this support. Even simple tests can yield big gains, and this write-up communicates that fact in a way that non-technical readers can understand.
This RNC document comes at a time when folks on the left are questioning the value of analytics and data-driven campaigning, and not enough of us are making a strong argument in favor of the benefits that data can provide. There have been rumblings on the left that many of the failings of the Clinton campaign can be attributed to too much attention paid to data, and too little to the story of the campaign or to the organizers on the ground. While some of that might be true, it ignores the fact that data is inherently woven into the storytelling and organizing of any campaign.
Without data, how do you know what type of donation button will bring in millions more dollars? How do you decide which voters to call when you don’t have the capacity to call everyone? How do you know if you should call 3 million people once or if you should call 1 million people three times each? Yes, the Clinton campaign struggled with messaging; that doesn’t mean we should throw away everything we’ve learned about how data can make campaigns more effective.
Given the failure of pollsters to predict Trump’s victory, it is reasonable to rethink the role of data in political campaigns, but it is also important for those of us doing analytics for progressive organizations and campaigns to clearly explain and advocate the importance of our work. Trump’s folks are doing this. It is time we learn something from them.
Andy Zack is the lead data scientist at ShareProgress, a social-good startup that helps progressive organizations use the power of data and technology to grow their base and win their campaigns.