Archives For November 2013

Announcing FEC Results

November 18, 2013

By Derek Willis

The Federal Election Commission isn’t the first place that reporters might expect to find election results. The agency is known for providing data about the financing of campaigns – who is giving and getting money – and for deciding how to enforce campaign finance laws.


But election results? There’s no direct link to the FEC’s section on congressional and presidential results from the site’s home page, but every two years the commission compiles certified results for primary and general elections from official sources, and then publishes spreadsheets and PDFs containing the data.

Today we’re announcing the release of FecResults, a Ruby gem for retrieving and parsing the data contained in those files, as well as a JSON API for that data.

OpenElections, like the FEC, collects official results data from state election officials, and we’re looking for as much detail as we can get. The FEC data, by contrast, is just concerned with the race-wide totals for House, Senate and presidential contests. So why would we make the FEC results data available?

One reason is that for many users, race-wide data for federal elections is useful enough, and some of the breakdowns that the FEC provides for presidential races are particularly useful (state-by-state electoral and popular vote totals, for example). Another reason is that we see this data as helping us fulfill the mission of OpenElections, which is not only to publish election results data but to make it easier to link to other kinds of information.

The FEC results data (2004 example) includes unique identifiers for most candidates within each election, which are assigned by the FEC for fundraising purposes. Most state election results sites don’t include this identifier, since it has little use for state authorities. Publishing the FEC data will enable us to use it as a reconciliation service so that we can connect state-produced data with the federal ID system, and thus with federal campaign finance data where possible. Having the certified totals is also a way to check our state data gathering processes, too.

Some technical details: the FecResults gem makes use of the RemoteTable library, which handles all kinds of tabular data in files on the Internet. Since the spreadsheet files aren’t always consistent from year to year, we needed to customize the process of grabbing and parsing the data into Ruby objects. A separate gem, FecResultsGenerator, publishes the data as static JSON files, which are then published via GitHub Pages.

MozFest 2013, Image: Sammy James Dodds

The Mozilla Festival in London seemed like the perfect venue to investigate the question of how well our data model would apply to elections outside the United States. It turns out that as complicated as American elections can seem, international elections have their own unique features.

At our session, we invited MozFest attendees to examine both our data specs and election results from countries around the world, in addition to their own knowledge of how such contests are designed and reported. Then we asked them to list differences between how we have modeled elections in the United States and those held in the rest of the world.

You can see some of the results in a collaboratively-edited document from the session, but one of the biggest differences is that elections outside the U.S. often center around political parties, not specific candidates. Modeling that could mean creating a first-class Party object to represent not just a party’s name but also its platform, candidates and number of seats it won in a parliamentary system.

Another factor that we had not given much thought to was the quality of the election. Although this isn’t an irrelevant topic for American elections (see: Florida, 2000), it can have much more immediate meaning in a country that is holding an election for the first time in years, or has a history of elections with significant flaws. Mohammed Haddad of Al Jazeera provided several examples in the session notes from his working experience. Many of the international election results also included the number of invalid ballots, too.

This wasn’t just an academic exercise, though: several of the recommendations would fit OpenElections well and could be adopted by the project. Among them are the addition of a MeasureResult object to handle election results for ballot initiatives and other non-candidate elections (currently we are only collecting candidate-specific results) and indicating the incumbency within results. As we work on election results from the United States, we’re trying to keep in mind how our model might be adapted and used in other nations, too.

Our thanks to the folks at Mozilla and all those who came to the session to talk election data. We’re grateful for your input and hope you keep following our progress.

By Sara Schnadt

Panelists Sara Schnadt, Dan Melton, Elise Hu, and Anthea Watson Strong.
Image: Phil Tenser via Twitter

OpenElections was invited to participate on a session about turning civic data into civic narrative at this year’s recent Online News Association Conference. The purpose of the session was to survey the current states of affairs of transparency in civic data sets, and how journalists are using them. Hosted by Anthea Watson Strong from Google’s civic data initiative, the panel included myself representing OpenElections and Census Reporter, Dan Melton of Granicus (and former CTO of Code for America), and Elise Hu, tech and culture reporter at NPR (and part of the team that founded the Texas Tribune).

Some of the key questions that came up in this open-discussion-based session were:

  • How has the amount of available civic data changed in the past ten years, and how well are journalists making use of it?

  • What is the current state of available data  – how much of it is clean and standardized -vs-  messy and inconsistent?

  • What does a robust civic data ecosystem look like, and what are it’s components?

  • How are newsrooms adopting open source values and best practices in software and app development as well as data cleaning and analysis?

  • What can algorithms do that journalists can’t? (bubble up patterns for lead gen)

  • What can journalists do that algorithms can’t? (identify and tell a good story)

  • How can we make data sets and their interfaces ‘sexy’ so that we entice more journalists and the public in general to become more data literate?

  • Since the internet ‘trends for entertainment’ how do we deliver substantive stories online alongside the more popular entertaining ones when getting to content online is so targeted?

  • Is crowd-sourcing data an effective strategy for journalists? When is it more and less effective and how much quality-control is necessary?

  • Is access to large amounts of civic and social data, and tools to hyper-personalize it, making the culture more or less civic-minded, more or less individualistic?

  • How do you tell compelling stories with data?

  • How can aesthetics help make data more relatable and articulate the dynamics within it?

  • Is data the chicken or the egg when creating a story (do you start with an idea and then find data to support it, or the other way around)?

Illustration: Graham Clark, ONA Student Newsroom, ONA Illustrated feature

There was lively discussion on all of these subjects between the panelists and the audience (which included many journalists with deep experience on the topic). There seemed to be a general consensus that this moment is more exciting than it is challenging, and that next steps include: moving beyond civic data transparency to consistently standardized and easily available data (where OpenElections comes in); widely available tools for exploring and analyzing data; and an increased data literacy among journalists and the general public (which can be facilitated by well-designed data visualizations and great data-driven storytelling).

To learn more, you can listen to the recorded session, and follow along with the slides.
(Also see credits and further links for my slides).