Archives For October 2013

An Improved Metadata API

October 23, 2013

derekBy Derek Willis

It’s not new, but it is improved. The Election Metadata API that we first released at the end of August has been revamped to make it easier to access specific objects and to provide more detail.

Rewritten using Tastypie, the API now provides a list of endpoints for individual states (for example, here is Maryland). That gives developers and users a roadmap for what kinds of information are available.

A common request could be for the elections in a given state. Continuing with Maryland, here are all the elections in the Old Line State contained in the OpenElections database. The responses include not only details about the elections but also about the source of the data (the organization in our parlance) and the state object itself.

While the initial release of the API provided the ability to filter elections by year, this updated version allows the use of specific dates to restrict the elections. For example, users can supply a date range in order to find elections held in Maryland in 2012. To drill down to only general elections, use the race_type filter. You can check out all of the possible filters at our GitHub repository.

For any request involving an object like election or organization or state, the response will include a meta object that includes the number of records returned.

The Metadata API is essentially dogfood for the project – it is mostly used to help other parts keep track of what we know about the availability and scope of election results. But if you find it useful, or have improvements you might like to see, let us know on GitHub, in our Google Group or on Twitter.

Eric Mill - Avatar 2

By Eric Mill

The New York State Board of Elections publishes a large set of election results in a central place. Naturally, they are all PDFs. But these are especially frustrating PDFs:

I spent a few months trying in vain to find the right staffer who would post the rest of the Excel files, or at least tell me that there were no Excel files to publish. The furthest I got was a one-line, 3-week-delayed email from someone telling me they’d see what they could get for me, with no further response for months.

And so, inspired by Sandra Fish’s successful Wyoming public records request, I filed my own FOIL request with the Board of Elections, using their handy web form. I called their Public Information office first before filing, to ask what they thought of the request, and they offered no opinion other than to just go ahead and file it.

In my request on August 26, I asked for “any and all election results, between 1994 and 2012, that the New York State Board of Elections has in spreadsheet form (where the file format is one of: .XLS, .XLSX, .TSV, or .CSV).” and pointed to the existing Excel files for 2010 as an example.

The Director of Public Information is legally obligated to respond within 5 business days, but by September 11, I still hadn’t heard. After an email and two calls over the next few days, they emailed me all of 12 Excel files on September 16.

The spreadsheets they sent me are somewhat mystifying.

It’s not clear why I got a mix of spreadsheets that are currently available and unavailable, or why these were the only ones sent. Could they really have thrown away all of their other 2008 general election spreadsheets? Do they never use spreadsheets for primary or special elections?

The format for primary and special election results the NY Board of Elections publishes are in different formats than general elections, so perhaps these are managed differently (though they are still suspiciously tabular). But all of their general election results from 2008 onwards use the same format, clearly generated in the same manner.

And what about the spreadsheets they already publish for 2006? The 2004 general elections use the same format as 2006, but no spreadsheets are available.

Whether it’s secrecy or simply disarray, the New York Board of Elections could be doing a much better job at serving the public. I’ve filed another, more detailed FOIL request, this time by mail, with the Board of Elections to try to get to the heart of how election information flows through the state of New York. It shouldn’t take this much work: election results are data. The Board should publish them that way.

Eric Mill
builds software, writes on tech policy, and helps manage the Sunlight Foundation’s international program. 
His projects at Sunlight include Congress API, its government search and alert system Scout, and a Congress app for Android phones. He also leads Sunlight’s involvement in many of the projects at of Sunlight, Eric runs a blog at, and works on numerous open source projects at Github.