This is the writeup of Experiment Two of our open-analytics experiments on AskThem, conducted with support from Google Civic Innovation. Background on this project.
This test focuses on the copy & message that a user receives after he or she has signed a question, in the post-signature modal that appears. This moment is key for AskThem’s success – for questions to spread through social networks, to gather signatures and be delivered for a public response. It builds our user base, demonstrates importance to the question recipient (whether or not he or she is a verified responder on AskThem), and circulates interesting questions and newsworthy asks.
As demonstrated by other petition sites, there’s no shortage of documentation that social networking sharing is vital to platform success. Tiny tweaks to the sharing message can mean percentage points difference in conversions on clicking the button and sharing on Facebook or Twitter or G+ or email, which of course (ideally) adds up.
Original copy: Support this question – and then on production, the text reads, “23 more signatures needed to deliver this question!” More straightforward message about task & threshold.
Variant copy: “Share this question and we’ll push for a public response” – slightly more-aggressive message about the offered outcome of getting to signature threshold.
… the experiment ran from Feb. 20th-27th, with the goal of tracking overall engagement in the modal window (any click, since we weren’t prioritizing just one button or social media service – though we could, which is an interesting side question). The results:
… showed that the “push for a public response” message received -6.7% engagement, and at close had only a 3% chance of beating original. So the message that more-aggressively articulated AskThem’s role in promoting public accountability didn’t result in more sharing in this instance. As always, more extensive empirical user surveys & testing could elucidate this result, but from this test we’ll take it as indication that promoting the positive virtues of online public dialogue means more clicks on sharing buttons.
Briefly, the post-signature modal is a really ripe area for testing design & messages – some thoughts on additional tests we’d like to run:
- Four buttons for sharing, as current, vs. one button to one social networking service at a time. Stands to reason this would be more effective, but as a new platform that already integrates with verified Twitter accounts, we wanted to offer Twitter sharing from every question. Our primary traffic referral is indeed from Facebook, as expected – bigger issues, there.
- More-local sharing copy, e.g., “Get 23 more people in Chicago to support this question for a response!”
- More results-oriented sharing copy, e.g., “A question to the Chicago city council was just answered last week – get 23 more signatures to deliver it this week.”
- More issue-based messaging, e.g., “Find 23 more people who care about immigration and we’ll deliver this question for a public response”.
From these revealing & crucial analytics experiments, my open-source development team and I now have empirical evidence on how we can enhance our non-profit platform. From here, I have other good revisions to connect visitors to their local elected officials – but we need charitable funding support to continue operating and move towards sustainability. Please support our non-profit work. Questions & comments welcome, david at ppolitics.org.