Great new NYC Council API! A longtime goal, finally a reality through a combination of inside- and outside-gov’t #opengov hacking.
Building off the exciting announcement last week of a new API for NYC Council legislative info, here’s a look back at what the non-profit field has done over the past eleven years, and what we can still do. (Another version.)
Our non-profit startup PPF conceived of OpenCongress in 2004, as a political hack of the free culture movement, and later launched it publicly in 2007 as an #opengov transparency project.
With our work since then, it’s been encouraging to see many advances in accountable representative democracy moving-in to government itself. In other words, government is slowly adopting features to become more digitally-responsive and open to public comments. More ‘participatory’.
I’ve been on a personal mission for a while: anyone in the field weighing the return-on-investment of outside-gov’t non-profits and open-data co-ops should try to ballpark-capture this public value. When government takes in a best-practice from outside-government, it’s a huge level-up in scale of adoption.
It’s not all super-rosy, obviously. It’s true that civic tech non-profits generally do require charitable funding support while they ramp-up towards earned-revenue sustainability – but the widespread public benefits (of moving better UX into government digital agencies) are unique compared to much VC-backed for-profit gov-tech or commercial e-petition companies that prioritize revenue over lasting localization.
Over the past few years, non-profits and civic hackers who are really doing the work have lost the opportunity to work as full-time professionals on liberating public data and collaborating on open-source tools for advocates.
Much of the #opengov movement has unfortunately been demoted to volunteer status, and experienced community technologists had to move over to gov-tech companies for employment before cities (in many ways, #opengov’s most promising & agile partners) had benefits of access to a robust open-source Civic Commons for their residents.
But from 2007-2011, when fundraising leaders in the field were at liberty to explicitly support a full-time, non-profit approach to building tools outside of government, the benefits to U.S. governments were immense:
Before 2010, it was not possible to email the office of your members of Congress. Constituents were obligated to use webforms; issue groups were obligated to use a D.C. commercial vendor for delivering advocacy messages. Small, distributed online communities did not have a free & libre coordinated solution for publicly petitioning their members of Congress.
So in 2010, PPF created the open-source Contact-Congress system, launched publicly in 2011 as a free feature on “OpenCongress version 3”, for digital transmission of messages through to Congressional webforms, creating a publicly-shareable permalink to letters-to-reps for two-way constituent communication. Contact-Congress has for years been brilliantly maintained by the United States data co-op, and continues to be used to deliver millions of messages by EFF, Action Network clients, Fight For the Future, and any small online advocacy group that seeks a free technical solution. This project has since become an under-the-radar, permission-based API used by the U.S. House for intake of advocacy messages from select partners – hopefully someday, all of the House and over to the Senate. Keep in mind, the popular OpenCongress model of data aggregation & design towards real-world context, can still spread to your state house and city hall, for effective p2p local organizing.
Before 2010, design & search UX on the federal THOMAS site was relatively poor, and it was difficult to understand the status & context of a bill in the legislative process. OpenCongress paved the way for social wisdom & enhanced search surrounding bill search, and then added individual comments on sections of bill text and public comment forums for p2p organizing. From the open legislative data scraped by pioneering GovTrack, over a dozen for-profit startups downstream-consumed & sought to commercialize their lead-generation business model, for revenue through users tracking & comments on bills in Congress & hot issues.
Before 2010, state house reporters and issue groups and local watchdogs did not have a source for open-data for state government elected officials and legislation. The Sunlight Foundation’s groundbreaking OpenStates project made this possible over an API, federating state elected official data up to partners such as the Google Civic Information API, for state-legislator lookup. The Open Civic Data standard lives on as a community standard capturing events (such as committee hearings), documents (such as meeting agendas), and much more, towards the goal of placing arcane legislatures in accessible context. But Open Civic Data currently has no single full-time evangelist or project coordinator, despite its ongoing integrations with international gov data standards and major tech partners. It could employ several full-time equivalents for mass scale of re-use.
Before last week, it was not possible to access open data for NYC Council legislation, either in bulk or over an API, or to get email notifications of local legislative activity. Since 2015, our non-profit NYC Councilmatic project has scraped data nightly from NYC Legistar and standardized it as Open Civic Data, enabling thousands to look up their local Council members and track community issues with free email notifications (including custom keywords). On Thursday, under the great leadership of NYC Council Speaker Melissa Mark-Viverito, the Council announced a new free public API. This will make possible new local engagement apps and community-based alert services, towards timely & continual connection with city governments.
It’s nearly 2018, and there still exists no open government data for city-level elected officials: who represents you in your city Council? When major players such as Facebook develop civic features such as Town Halls, they license closed city data sets from commercial data sources. Other major players such as Google (which offers an awesome free service through its Civic Info API and more), Reddit, Pinterest, Ask, Tumblr, Flickr, LinkedIn, MeetUp – all don’t have county-,city-, or local-level open data either – simply because the civic-hacking non-profit funding dried up before moving down past state-government-level, all the ops infrastructure & volunteer community was there in place. This has meant a typical Pinterest user probably can’t easily get involved with a local cause and more-deeply-rooted peer community through their social media service. The field thankfully has had the Voting Information Project API for elections, or else that would be a huge missing area of U.S. infrastructure for civic engagement.
It’s nearly 2018, and there is no nationwide non-profit program for crowdsourcing questions ahead of midterm elections. The non-profit, non-partisan Open Debates project is making great strides in races, but our AskThem is already developed to offer this free public service in every district nationwide, with free embeddable widgets to spread public questions to regional media partners and political bloggers on the web pages & local groups they already visit – towards building greater trust in media. Our open infrastructure and proven question program stands ready for public dialogue through 2018 midterms.
It’s nearly 2018, and there is still no wide support for open data standards for constituent communication, so social media posts are still not count-able as e-petitions in various districts. The filter bubble problem continues, because public social media posts and digital campaigns don’t offer any hooks into existing organizing efforts. The advocacy groups with financial means use commercial lead-gen services to purchase promotions to email addresses and Facebook ads for target audiences, but these people never get the opportunity to meet and coordinate with each other, e.g. to plan district meetings with the much-needed VisitThem service. We keep sharing with like-minded friends on Facebook & being baffled at the other side’s intransigence on Twitter, when we’re not using the full potential of open tech for accountable and responsive representative democracy – flash polls, liquid feedback, public whip counts, escalating city issues to state legislatures and up the chain of government to federal chambers. This is the path not taken because the legacy civic tech companies have continued venture capital backing, government entities are often resistant to stronger public accountability exposure, and I’d argue because non-profits in the digital #opengov space haven’t succeeded in promoting the widespread benefits of outside-gov’t civic hacking.
Put broadly, the overall field has moved away from funding non-profit open-source #opengov projects, independent of government, to encourage for-profit gov-tech investment and moving into government digital services. The reasons for this are many, and the potential value is still admirably huge, but I’m afraid it doesn’t capture the full realistically potential value of civic hacking – bringing in those who don’t seek to wait for permission to build, say, an innovative new liquid feedback tool for accountable representative democracy. If the goal is people using & sharing tools, building real-world civic equity, taking civic actions with a fraction of the same activity on social media and visiting news sites, then outside-gov’t civic hacking delivers results as above, the kind that government is often unwilling or unable to do (until it isn’t, or is obligated to do so).
In the meantime, I believe that civic tech is not a big tent because its missions are so different in openness and lasting local & structural impacts. For example, in recent years, major well-known commercial civic-tech companies had the resources to liberate city government data in open standards via an API, but chose not to (in fact, kept the data closed, then shuttered its programs in the rep. democracy area). Commercial players had the ability to support open messaging standards to make public communications more analyzable and enable p2p local organizing, but they chose not to, because it threatened (not existentially, I’d argue) their business model. Commercial players had the ability to build public knowledge of the influence of money-in-politics on political issues at all levels of government, but many of them still rely on procurement contracts from government entities, so the story has yet to be fully told.
In the meantime, mostly-volunteer co-ops of practitioners remain ready to bring the benefits of open-data and open-source tools down to the city level – and our non-profit PPF stands ready – again – to run two unique programs through 2018, with charitable funding support:
AskThem – public proposal for questions-and-answers with 2018 midterm candidates in every congressional and state legislative district nationwide, using local media partners & free JavaScript widgets to mitigate the “filter bubble” of political info.
Councilmatic – free email notifications for city Council meeting agendas and more – currently in four major cities, available to roll out to dozens more as an open-data local resource.
For 2018, our non-profit’s ideal would be a goal of $450,000 annual budget – approx. $150k from two foundation funders, $100k from four major donors, $25k from open city data licensing and $25k from micro-donors before the midterms, supporting three full-time equivalents (PM, tech lead, community manager) and open API infrastructure. But we can still keep the lights on through 2018 with much less, as we continue to build towards non-profit sustainability – and continue to make state & local governments, where there exists so much opportunity, to be more accountable.
We’re still looking for champions & fundraising partners for 2018, at a very efficient operation level (virtually no overhead), with clear value from existing programs, and offering open project analytics for the civic tech community – get in touch for more detailed info, email: david at ppolitics.org.