Archives For December 2013

The Year in OpenElections

December 30, 2013

2013

As 2014 approaches, our team at OpenElections wanted to share an update on our progress and plans for the new year. Looking back, an anthropologist might peg as us a hunter-gatherer society. With the aid of dozens of volunteers, OpenElections in the last year has:

  1. Gathered metadata about the availability of election results from nearly every state (we still have a few left!).
  2. Created a system to store the metadata and a JSON API for using the data in our other systems.
  3. Held two metadata sprints where volunteers entered hundreds of records.
  4. Begun gathering election result data files for a set of initial states.
  5. Released an API for federal election results.
  6. Talked to a lot of people who care about election results.
  7. Tried to keep you informed of what we’ve been up to.

We’ve discovered communities that share our vision for elections, who are helping to make election data easier to access, developing new data standards, and crafting standardized data formats for the next generation of voting technology. We’re fortunate that the people and institutions that care about elections are a generous lot. Their collective efforts and shared wisdom (often directly on a phone call or over email) has helped us immensely, and we expect to continue collaborating with these generous folks in 2014.

In the new year we’ll be working hard to roll out standardized results that are easy to use, whether you’re a reporter, graphics designer, civic hacker, or just a curious citizen interested in playing with some election data (human-friendly data is the original goal of OpenElections, of course!)

We’re currently building out our data processing pipeline starting with Maryland, but have also started working on other states, including Florida, Ohio and West Virginia. We’re expecting to dive into many more states early in 2014, and we can use your help scraping files, loading results, and ensuring data quality (keep an eye out for a guide on Contributing Code). And of course, you don’t need to be a nerd to contribute; ping us at openelections@gmail.com and we’ll find something that suits your interests and talents from our holiday wishlist 🙂

Last — but certainly not least — we need to end the year with a huge thanks to the volunteers who spent hours gathering election metadata, writing code, and wrestling data files out of government agencies across the country. You are the ones making the OpenElections vision a reality, and we appreciate every minute you’ve spent in the last year. And for those folks who haven’t yet joined our volunteer ranks, yes, there will be more swag in 2014 😉

Happy New Year everyone!

Derek, Sara, Serdar

 

Screen Shot 2013-12-09 at 10.09.29 AM

OE : How did the US Elections Atlas come about?

DL: The project started as a paper hobby back in 1993 – exploring the fascinating geographic results by county for the three-way 1992 General Election for President. I colored by hand a photocopy of a U.S. County map using the results published in The World Almanac. This led to a desire to compare county data for other years and I soon was compiling election results. My desire for completeness and accuracy led me to find complete official results sets instead of using existing compiled sets published in books (these compilations typically omitted minor party candidates and often included errors).  In the late 1990s, I started putting these results online, creating uselectionatlas.org.

The site content continued to expand as I collected paper copies of official published results by county by contacting the many state election agencies (very few of these publications were available on line at the time) and by visiting multiple libraries and state archives. No official government source compiles election results by county and so there was a need to obtain results from each state separately – it was quite a time-intensive operation, and it continues to this day.

OE: What motivated you to start it?

DL: This was a combination of hobby, personal interest, and civic-mindedness – to help share this information as the budding web began to grow.

OE: How is it used now?

DL: The Atlas has grown into a multi-purpose site related to elections.  At its core it is a set of official elections data collected from state agencies, county agencies, and cities/towns gathered to improve the overall accuracy of nation-wide data.  These data sets are presented using maps, charts, and tables.  The offices covered have expanded beyond the initial focus on Presidential races and now include U.S. Senate and Gubernatorial races as well.  More interactive portions of the site have been added over the years, creating a community of election enthusiasts that engage in a discussion forum, election predictions, shared endorsements, mock elections, contributions of down-ballot election results, and more.  The site receives thousands of visits a day for many uses, in particular: research for individuals looking for specific election-related information; historical content is used by educators in their curricula; and members discuss politics, election results, future elections, and much more on the forum.

OE: Elections results archiving can vary wildly from state to state. What has your process been like for acquiring elections results?

DL: Very manual.  Older elections required finding where each state published the results – often in books or reports.  I have obtained most of these results as photocopies that I manually typed into spreadsheets.  More modern elections are now all available on the web – but each individual agency has had differing methods of publishing them, and their methods change every other year.  The compilation in these cases has been a manual download and transfer of these digital data into the Atlas databases.

OE: In your experience, have you noticed any discrepancies in results counts, and how have you addressed them with local and state board of elections officials?

DL: Yes, there are many instances of discrepancies – the most common is that the county data do not sum to the published totals (this happens less often in more modern elections due to the use of software to add up the votes).  Finding the error in these cases can be very difficult because very few states include horizontal reconciliation (a printed total vote by county or including the over votes and under votes to compare with total ballots cast). Another common error is transposing or scrambling candidate totals within an individual county.  Such an error can only be found by comparing the county published data to what the state published.  These are mostly found during the process of compiling precinct-level data. Other discrepancies occur when a county will amend its results after the state has published its certified results.  While some states amend their reports, most do not. When a discrepancy is found, I contact the county agency to determine the correct figure and include it within the Atlas.  If it is a modern election prior to certification, I will also contact the state election agency to notify them of the discovery.  For older elections, the agencies typically do not or cannot (by law) modify the reports.

OE: How do you think the work that we are doing with OpenElections might compliment your work?

DL: More eyes are always better – the more people  that are looking and compiling the election results, the more accurate and complete the tally can be.  Two sets of independent compilations provides for the opportunity to reconcile any differences for a more accurate data set.

OE: How might a project like OpenElections change your process?

DL: I will continue to compile results from official reports, but the existence of OpenElections will create an opportunity to provide a county-level check step to validate the correctness of the data.

OE: In an ideal world, what would the space of Elections Results management and transparency look like to you?

DL: Ideally, all election jurisdictions would report all of their results in a standard format:

  • All votes are allocated to a geographic precinct (no separate absentee, provisional, or early precincts county-wide).
  • No geographic precincts are split between any U.S. Congressional or state legislative districts.
  • All precinct-level data includes write-in, over-votes, under-votes, and total ballots cast as separate figures to provide horizontal reconciliation.
  • The results would be stored in a standard format that could be easily combined for all states.


David Leip
is Chief Electrical Engineer at LTX-Credence. He started the US Elections Atlas in 1993 as a graduate student at MIT, and it has evolved since then. More in David Leip, and the project, here.