This is the writeup of AskThem Experiment 7, by Maryam Gunja, AskThem community manager – of our open-analytics experiments on AskThem, conducted with support from Google Civic Innovation. Background on this project.
Q+A Best Practices – product & community researchÂ
What is a good question? So asks Micah Sifry, a member of the AskThem Advisory Council. He goes on to answer:
A good question is one that presents its recipient with a problem that must be resolved.
It may raise uncomfortable facts, or highlight a contradiction, surface something that we haven’t thought of before, or merely demand that its subject explain him or herself on a topic they have avoided or would prefer to not address.
Good questions can crystallize something that is on a lot of people’s minds, often by personalizing the topic.
Good questions also are ones that take a topic that is “out there†but until that moment haven’t really been addressed by their subject.
Good questions insist on accountability, and good questioners insist on real answers, not obfuscations.
When we as individuals and as a society fail to ask good questions of those with power or those who act in our name, democracy falters.
In this, Micah has articulated well the public-benefit mission of AskThem: to create an open platform for good-faith engagement.
Here at AskThem, our goal is to facilitate a safe place for meaningful dialogue. We ask users to participate responsibly and display tolerance in Q+A exchanges.
Be advised that AskThem moderators will monitor Q+A activity and will remove any offensive, inflammatory, or explicit text. If you have any questions or concerns regarding acceptable Q+A practices, please feel free to email one of our moderators here.
We hope that you’ll use this guide to help formulate appropriate and effective questions to ensure that your ideas receives maximum exposure.
In general, the best essential questions center on major issues, problems, concerns, or interests relevant to the public interest. Good essential questions are open-ended, non-judgmental, and concise. Effective questions demonstrate good-faith engagement between legislators, constituents, and the general public.
Targeted City Elected Officials:
– Avoid asking city-service officials governmental procedural questions. AskThem provides the unique opportunity to generate deeper engagement with legislators though issue-driven discussion.
To: City Commissioner Vince Lago (Coral Gables, FL)
Less-Effective Questions:Â
Q: How do I get a permit for constructing a building?
Q: What is the city’s total budget?
More-Effective Questions:
Q: What long-term plan do you have for reducing opportunities for crime in our neighborhoods?
–How can the city systematically support existing proactive neighborhood approaches?
Q: What is the your opinion of the neighborhood schools model?
-The city council and the school board have discussed re-drawing school boundary lines in order to curb over-enrollment. Do you think the city should make exceptions to the model to consider offering specialized programs to cross-town students?
Targeted State Elected Officials:
-Remember to stick to the issues instead of asking personal and/or inappropriate questions.
To: Governor Cuomo (NY, NY)
Less-Effective Questions:
Q: What’s your favorite non-fiction work?
More-Effective Questions:
Q: How has the FreshConnect program impacted farmers since it was first instituted?
-You launched the FreshConnect program with the vision of connecting farmers with neighborhood based green markets in urban areas of New York.
Q: What are the challenges of running a convention center in New York?
– The proposed New York site, which will serve as the largest of its kind in the nation, will compete with other major convention venues in large cities. Those venues have suffered from a decline in a significant activity and trade show attendance as of late. How will this new center operate in response to this trend of diminishing profits?
Targeted Federal Elected Officials:
AskThem aims to foster non-partisan engagement. Refrain from asking abrasive or leading questions with the potential to polarize a wide spectrum of users.
To: Â Senator Bernie Sanders (VT)
Less-Effective Questions:
Q: Does the government retain far too much power to spy on innocent United States citizens without sufficient oversight?
More-Effective Questions:
Q: What impact did the community-trust housing authority have on the earlier part of your political career?
-Under your leadership, Burlington became the first city in the country to fund community-trust housing.
Q: Can you outline the provisions of the Older Americans Act?
-The senate committee recently advanced legislation by you to reauthorize the Older Americans Act.
Answering Questions- For Elected Officials:
As verified AskThem users, you are expected to be active, informative participants in all online discussions.
Remember that your answer won’t be viewed solely by the user who posed the question- your answer will be accessible to all of your constituents in addition to users who voted up the initial question. When necessary, foreground your answer with useful background information and context for those users unfamiliar with the premise of the initial question.
Case study – Brad Lander, AskThem Verified Responder
NYC Council Member Brad Lander has proved to be our most responsive verified responder and one of our most popular targets for our New York users. His participation is a terrific use case for AskThem and demonstrates the value of reciprocal dialogue-building tools for local impact.
From civic minded petitions to questions concerning policy agenda items- we’ve amassed crowd-sourced support for a wide range of dynamic asks submitted to Brad Lander.
Answered Questions
– Â In response to the affordable housing crisis, Lander outlines comprehensive policy steps and sister city legislation.
– Drawing off the success of Seattle’s Race & Social Justice Initiative, Lander responds to the push for a NYC initiative.
“Thanks to your question, I’ve put in a legislative drafting request to begin working with New York City Council lawyers to craft it.â€
We included our user’s feedback to Lander’s response to prompt further dialogue on the issue.
Government Processes
-After a participatory budgeting election in Lander’s district, user questions bubbled up in quick succession about PB resources and respective district participation.
-Users ask to tease out chain of command for city-wide legislation approval.
Hot-Button Issues
– In the midst of recent closures and proposed renovations, users demonstrate strong ties to local anchor institutions.
–Campaign to advance legislation on the operation of horse-drawn carriages in NYC.
Infrastructure/Public Safety
-On accessibility requirements and modernizing public transportation:
–Traffic concerns in local neighborhoods:
 AskThem – Product Comparisons & Research Notes
Quora
A. Basic structural model:
- Open to general public
- Ask/and or answer questions in more than 250,000 topics that are arranged within an overall topic ontology
- users with certain subject knowledge are tagged in specific categories, and alerted of questions specific to their domain
- Voting system allows for only the most popular content to be visible
B. User experience
- allows users to link their online Quora identities to their Facebook profiles
- Browse questions, ask or answer any question, “follow” or “promote” topics/ questions of your choice
- Viewable number of answers, comments, and # of people following the question
- Post content on topics of your choice via blogging platform
- Reviews feature gives users ability to include star as part of review
C. Ethos:
- Strangers interact with each other, in sharp contrast to other social networks
- Not just a Q+A platform, but a resource for reviews and recommendations from experts spanning a wide breadth of topics
D. Revenue model
- The site is not yet monetized and does not host any advertising (may be out of date – may have taken another round of startup capital funding)
E. Rules and Methodology
- Quora trusted users are entrusted with increased editing privileges
- Users accrue trust by building up a track record of good discretion on the site
- Quora offers users very little control on what appears in their feed
- Metrics
- 1,126,400 monthly users based on its panel estimates (433% annual growth)
Parliament Watch (Germany)
A. Basic structural model
- Users question members of parliament in a public environment (includes federal, european, 9 states, and 54 local govts)
- Participation does not require prior registration and all functions are available free of charge.
- Responses from elected officials are emailed to everyone who is following the question
- Leverages election cycles for revenue
B. User Experience
- Search voting record of their members of parliament
- Follow up on promises made by leaders through Q+A archive
- View extra earnings of members of parliament
C. Ethos:
- Champions citizen empowerment in government
- Denies cooperation with political parties or high profile party members or politicians
- Non-partisan communication between citizens and their elected representatives or candidates
- Motivates citizens to run as for public office
D. Revenue model
- Parliament Watch is financed through membership and one-off donations
- Media partnerships
- Extends to candidates during elections (candidate premium profiles for 200 euros allows posting of photo, cv, political goals, and calendar to profile)
E. Rules & methodology
- Moderators review every Q+A before posting, alerts user of a delay possible of several hours –> can appeal rejected questions
- Requirements to submit: full name, valid email address, under 2000 characters (for readability and accessibility)
- Must be a question, not a statement or expression of opinion directed at the elected official, not their staff
- Only one follow up question
- must reference sources of factual info/quotes
- No elected official/staffer to elected official/staffer communication
- Anonymous questions are not forwarded to elected official
- Advisory board oversees controversial questions or answers
- comprised of journalists, academics, lawyers, civil society leaders, political analysts
- safeguard of bipartisanship
F. Metrics
- 6,800 visitors daily, 143, 507+ questions, 115,921 answers or 80% response rate
- 80% of politicians participating actively
- in Germany offers a “report card†for engagement each year (http://blog.abgeordnetenwatch.de/2012/07/20/zeugnisnoten-fur-volksvertreter-abgeordnete-antworten-immer-haufiger/)
 Stack Exchange
A. Basic structural model:
- Network of 110 Q+A sites, community blogs and real-time chat spanning various topics
- Large network of individual communities, each servicing experts in specific fields.
- Curate high-quality questions and answers geared to each each community’s area of expertise.
- Free and open to all without registration, all users have access to full archive of content
B. User experience
- Users post questions and the community reviews, revises, and proposes answers to the question
- Answers are then ranked by the rest of the community, members vote for or against as they see fit
- Questions and answers can be edited by other members, Wikipedia-style.
C. Ethos:
- Unites individual communities of experts on specific topics
- Collaboration between open editing technology and feedback driven user ranking
- Large decrease in response time compared to other Q+A models
D. Revenue model
- Funded by Venture Capital, no advertising on site
- Launched “Stack Overflow†career website
E. Rules and Methodology
- To increase quality of answers, visitors are encouraged to vote on answers
- Users gain reputation points when others vote for their questions or answers
- Tags direct questions to experts from a specific field
- Periodically holdsdemocratic moderator elections.
- Metrics
- Approximately 35,000 registered users and 515,000 hosted questions (http://stanford.io/1bdDCVW)
Comparisons of previous local & national petition services
 Change.org
A. Basic structural model:
- Any individual or organization can post a petition
- Allows users to direct petitions to politicians or “decisionmakersâ€
- Decision maker is notified via email and has option to respond
- All petition signers are alerted when decision makers respond and receive alerts about ongoing petition activity
- Operates as a for-profit company by charging groups for sponsoring petitions that are matched to users who have similar interests
B. User experience
- Provides guidance on harnessing power of individual’s social networks
- Site offers an email address importer to invite user’s friends to join them on the campaigns
- Automatic posting to user’s own “FB wall†as a status update
- Users click on the “Share on FaceBook” button after signing a petition
C. Ethos:
- Propels “micro-movements”
- Sponsors petitions service users and advertisers
- Originally sponsored campaigns from organizations “fighting for the public good and the common values we hold dear—fairness, equality, and justice,”  but revised policy and now claim no ideological leaning.
D. Revenue model
- Hosts advertising from major nonprofits and political campaigns  and tracks profile data of all users
- Change.org sells promotions to organizations on a cost-per-email-address basis.
- When a user signs and/or submits a petition, they are greeted by a pop up displaying five related sponsored petitions
- Users can choose to “stay updated†of these campaigns and are added to opt-in mailing list
- 300 paying clients, including Sierra Club, Credo Wireless and Amnesty International, Lance Armstrong’s Livestrong Foundation, the Environmental Defense Fund, the Humane Society, the American Heart Association and Amnesty International.
E. Rules and Methodology
- Only advertisers explicitly identified as “hate groups” are barred from participating on the site
- Popular petitions are vetted and selected for home page, e-mail promotion and campaign support
- Metrics
- 50,000 users and 6,000 victories http://www.change.org/about/impact
- $15 million in annual revenue
 MoveOn.org
A. Basic structural model:
- Execute public education efforts and issue advocacy campaigns
- Petitions can be started by any person or organization, or non-MoveOn member.
- All members invited to participate in agenda-setting processes via daily surveys and membership votes on candidate endorsements and major issues
B. User experience
- Members are encouraged to contribute feedback daily about proposed actions
- If a petition receives 20 signatures, MoveOn staff will send it to a small test group of members. If the response is positive, the petition will be sent to a larger group
- If petition shows sizable growth, staffers work with petitioners on running the campaign
- Moveon support provides media training, event organizing and fundraising help.
C. Ethos:
- Harnesses energy of site’s activist base
- Focuses on promoting grassroots campaigns at the local level
- Consults members of group before every new campaign is unleashed.
D. Revenue model
- Operates as a not-for-profit and is funded by modest donations from more than 8 million members and grassroots fundraisers
E. Rules and Methodology
- Hosts progressive campaigns only
- Resolute in refusing ads or payment for petition hosting
- Metrics
- More than 8 million members
Comparisons with previous civic engagement tools for gov’t (Councilmatic, OpenCongress, Open States, mySociety, etc.) Â
CouncilMatic
A. Basic structural model:
- Functions as a primary source legislative tracker and lets anyone browse, search and subscribe to any piece of legislation published by a city.
- Philly.Councilmatic is scraped daily from the Philadelphia City Council’s legislation site
- Each piece of legislation provides a link to its source for reference
B. User experience
- Users can customize their subscription to legislation via email, RSS, or SMS to headlines that you desire
- Users can sign up for alerts on legislation of interest
- Users can comment on a bill and choose to specify whether their comment is in support or in opposition to the bill.
- Non-routine pieces of legislation are tagged for accessibility
C. Ethos:
- Promotes transparency and openness with local city councils
- Contextualizes non-routine legislation by drawing connections between similar bills
- Engages public in collaborating on legislation rhetoric
D. Revenue model
- open source application built by Code for America
E. Rules and Methodology
- Councilmatic tools change in accordance with respective city council procedures
- Scrapes legislative data from a local municipality (usually the City Clerk’s website)
- Metrics
- Currently operational in Philadelphia, Chicago, and Oakland
OpenCongress – created by PPF, operated solely from 2006-2013, now under Sunlight
A. Basic structural model:
- Free account allows users to identify their representatives in the House and Senate and track their respective their legislative activity
- Zip code search locates user’s representatives
- Equipped with profiles for every member of congress
- Users track the bills of interest, contact their, elected officials, and campaign contributions
B. User experience
- Users subscribe to congress members activity, news stories, and blog posts
- Users contribute comments on sponsored bills pages on House and Senate profile pages
- Built-in social network to connect users around a certain issue or location in groups
- Side-by-side comparisons of legislator voting history
- Review all roll call votes to date in the 113th U.S. Congress
C. Ethos:
- Combines official government information with news & blog coverage
- Nonpartisan source for legislative information
- Empower citizens to engage in day-to-day legislative activity
- User friendly transparency app
D. Revenue model
- OpenCongress is a free, open source and nonpartisan resource originally built from PPF and currently operated by The Sunlight Foundation
E. Rules and Methodology
- OpenCongress site code is availableon Github
- Bill, legislator, committee, vote, campaign finance and issue data are collected from official sources, including the Library of Congress, GovTrack, Congressional Research Service and the Sunlight Foundation.
- Metrics
- Surpassed 1 million visits per month,
- Active community of more than 150,000 registered users (http://bit.ly/1kj550s)
OpenStates
A. Basic structural model:
- Searchable legislative data for all 50 states, D.C. and Puerto Rico.
- Enter a U.S. address or select a state to research bills, review voting records, and contact elected officials
B. User experience
- Users subscribe to their state lawmakers’ actions
- Users explore voting records, committee assignments, and campaign finance records
- Users track status of bills, locate bill supporters and view supporting documents
C. Ethos:
- Only app to display comprehensive database of activities from all state capitols
- Draws connections between topics and legislative activity across different states
- Serves as a permanent archive of data that often disappears when a new congress is in session
D. Revenue model
- Open States is a project of theSunlight Foundation and receives charitable contributions.
E. Rules and Methodology
- Bill, legislator, committee and event data is collected from official sources, linked at the bottom of each legislator, bill, vote, committee or event page.
- Metrics
Analysis of prior efforts to promote “good questions†with elected officials
Pros/cons of different levels of moderation & UI decisions for finding elected officials by address
A. Strict admin moderation (usually by community manager)
Pros
- Useful to have experts in other fields guide site policy and policy enforcement
- Admin is highly sensitive to organizational values and will censor appropriately
- Admin will reframe submitted material that are cast as opinions instead of questions
- Admin protects integrity of those who are signed on to respond to the public’s questions
Cons
- Admin is a strain on org. resources
- Admin activity can conflict with progressive, participatory values of certain sites.
- Admin is disproportionately accountable for offensive material on the site
B. Self-organizing moderation
Pros
- Entrusts and rewards active users with more responsibility
- Appointed admins presumably have verifiable footprints on the site and would provide useful feedback of the user experience
- Active users appointed to overseeing a particular topic or community will have incentive to employ stricter discretion
Cons
- Appointed admins might field questions in service of personal biases instead of in alignment with the site employees
- Appointed admins require a lot of management as they will invariably rotate and work in limited capacities
C. Establish an advisory board for additional oversight
Pros
- Entrusts those with a vested interest and stake in the site with more responsibility
- Allows advisory board members to keep track of site progress
- Expands the diversity of the admin group
Cons
- Potential for conflicts of interest in curating content
- Potential to only screen for offensive questions instead of weighing other criteria attune to question length, redundancy, and topic
Admin tools needed for promoting & highlighting good questions & examples of responses with elected
- Ability to hide content, delete content, ban and unban users
- Timeline notifications about new comments?
- Profanity filter
- Moderation blocklist
“10 Questions Project†around U.S. Presidential race, why not broadly adopted?
- Only available densely populated states
- Driven by election cycles, does not encourage sustainable dialogue otherwise
Summary of “lessons learnedâ€
A. Guiding Principles
- Prioritize legibility and accessibility of app
- Emphasize the value of the action the user is taking (reward and thank)
- Media partners will significantly help legitimize platform
B. Technical Principles
- Use icons and issues with collapsible sub-categories more to funnel site visitors to ask exec & leg branch relevant questions (http://www.toronto.ca/311)
- Experiment with number of answer fields and character count cap to encourage electeds to give non-standardized responses
- Review views & interaction counts by issue
- Explore SEO optimization techniques
C. Maximize Adoption
- Enable user sharing on other social media sites before launch (WeThePeople) to cross-sell the platform
- Mobilize local blogs and local media
- Need to develop strategies to encourage politicians to offer non- standardized answers.
- Leverage election cycles
- Survey users to gather feedback and satisfaction ratings after their question is answered
From these revealing & crucial analytics experiments, my open-source development team and I now have empirical evidence on how we can enhance our non-profit platform. From here, I have other good ideas to connect visitors to their local elected officials – but we need charitable funding support to continue operating and move towards sustainability. Please support our non-profit work. Questions, comments: david at ppolitics.org.