One of the major benefits of AskThem.io being an open-source, open-data project is that our non-profit organization can share our findings back with the public commons. This impulse to liberate info about our process extends from the open data standards we’ve supported to our collaborations with community organizations (such as Participatory Budgeting in NYC). AskThem is a questions-and-answers platform with every U.S. elected official and any verified Twitter account (more about us). PPF works to build civic engagement on the open web for the public benefit.
We’re grateful that the Google Civic Innovation team (behind the Google Civic Info API) feels 
the same way. Their team, coordinated by Anthea Watson Strong, is not only doing some of the most valuable work in making government data available. They’re also interested in empirical studies of what works to increase civic engagement online, and making these findings publicly-available for dissemination. PPF received funding support from Google to run a number of open-analytics experiments, document our process and then publish them openly, and the following series of blog posts constitutes our findings. We’re excited to share them. Also, we’re proud to use the Google Civic Info API on AskThem as our primary data source for geolocation of elected officials (e.g. from our homepage), and I’m personally grateful that Anthea is one of ten advisors to our non-profit project.
Experiment writeups, questions & comments & requests for more info welcome:
- 1-Â testing homepage copy on “start petitions”
- 2- testing post-signature copy, “push for a public response”
- 3- testing streamlined homepage UI
- 4- testing homepage “people like you / politicians” copy
- 5- testing removing firstname & lastname fields from question pages (running now)
- 6- case study of signature conversion from allied org.
- 7 – open research on Q&A community best practices & comparable product UX
- 8 – split-test results of email subject headings around launch
PPF used three three tools for these experiments: Google Analytics, Optimizely, and Action Network (our mailing list solution). Optimizely was the most integral of these three – we don’t have any official relationship with their service, but I’m happy to vouch that it’s really valuable and user-friendly for split-testing of elements on pages. Knowing HTML isn’t even necessarily required (though I’d say it’s virtually required for tests across pages). You really do edit copy, buttons, and other elements right in the browser using their tools, largely drag & drop. Pretty cool. Tech background: AskThem is a Ruby on Rails web app with a MongoDB back-end, code & data on GitHub, contributions & remixes welcome. Our technical lead is Walter McGinnis, who helped with Optimizely setup; our community manager is Maryam Gunja, who helped with mailings & outreach.
I designed this series of experiments with the following overall goals for AskThem:
- increase our non-profit site community through signatures on question pages;
- increase shares of question pages for more signatures on social networks;
- and from the homepage, increase new question creation, for continually-new content.
The reason for these goals is that, by gathering a larger community of question-signers on our open-source platform, we can incentivize more elected officials and public figures to respond to questions on AskThem. Which in turn incentivizes users to ask & share more questions, and elected officials from city to state to federal levels of government can generate value by holding online public dialogue on AskThem. All of which would help AskThem move to greater degrees of non-profit sustainability.
The above-outlined broad goals were tested on different types of AskThem pages and e-mailings – the tests we devised weren’t as programming-intensive as they could be, or as extensive as a commercial e-petition site could support, but we did the best we could within our non-profit resources. (Speaking of, we’re actively seeking charitable funding support to keep operating through the Nov. 2014 election – please be in touch to support our non-profit work now.) More specifically, many of these experiments were implemented simply with Optimizely, as opposed to more-custom coding & design tests & tracking visitor funnels in Google Analytics – we’d welcome the opportunity to build on these tests. We’re open to partnerships and easy to reach. Sample sizes are important – the minimum standard, from our research and conversations with analytics practitioners, is A/B split test with a minimum of 100 conversions per competing page. So for example, if conversion of the original page is 4% and expected conversion of the alternate page is (say) 5%, a minimum of 2,500 observations per page would be advised. (The gold standard is for this to be accompanied by a one-sided test of original with a reliability of .95, but we didn’t test such for these experiments – Optimizely’s algorithms did a fine job determining winners between the split test variations, and that worked for us.)
I’m really pleased to publish our findings, discuss what worked and our next-round of pressing questions, and identify areas where we’re having the greatest impact. If more non-profits and for-profits joined us in liberating their analytics, we could increase the ability of the civic tech community to build lightweight tools for organizing and educating the public. With sharing of open data, you can have more powerful peer-to-peer platforms for connecting with other constituents who care about the same issues as you do; an open CRM & CMS for your members of the U.S. Congress; comprehensive open data on everyone who represents you, from local to federal levels of government; and much more. But in many ways, open-government movement is just getting started laying the foundation of infrastructure to build more valuable alerts & services & organizing tools on top of open government data. The Google Civic Info API is a terrific example – until its public launch last fall, there was no free API to return elected officials (down to the county level) for a five-digit U.S. zip code or residential street address. And while the Google API is making great progress in supporting municipal government data, there still does not exist a comprehensive API for nationwide elected official lookup down to city mayors & council members. I believe we’re on-track to making this available, however, in the U.S., and allies such as Code for America and Open Civic Data will push things forward.
Additional thoughts on what empirically works for civic engagement can be found in our non-profit advisors’ work: Micah Sifry’s forthcoming book, “The Big Disconnect: Why Technology Hasn’t Changed Politics (Yet)“; Anthea Watson Strong’s writing on her personal site and micropublishing stream; and Prof. Dave Karpf’s research & blog posts, “Shouting Loudly: building a healthy info ecosystem“; and Susannah Vila’s blog, including her recent report with Engine Room Advocacy,  “User Engagement Strategies for Open Data”; and our other six AskThem non-profit advisors, who can help us create a community ethos of good questions.
Shouts as well to some informal advisors, as friends: Nathan Woodhull of ControlShift Labs, for providing p2p organizing tools as a non-profit org.; our good friend Elana Berkowitz‘s research on  civic engagement in startups and internationally and everywhere; Matt Stempeck‘s research & consulting, coming out of MIT; and Hollie Russon Gillman of New America Foundation and GovLab and more, who generously took time to briefly advise these experiments and their goals. Thanks team.
From these revealing & crucial analytics experiments, my open-source development team and I now have empirical evidence on how we can enhance our non-profit platform. From here, I have other good revisions to connect visitors to their local elected officials – but we need charitable funding support to continue operating and move towards sustainability. Please support our non-profit work.
If you have any questions about our methodology or to see data in more detail, be in touch and I can accommodate, I’m email: david at ppolitics.org, @ppolitics. Thanks again to Anthea and Google for this support of our project. Questions & comments welcome, feel free to share, hope these are helpful in articulating some best-practices for open-source UI developers.