Join us at TNW Conference 2022 for insights into the future of tech →

The heart of tech

This article was published on February 29, 2016

    This man used a clever trick to find out what users really thought of his app

    This man used a clever trick to find out what users really thought of his app
    Martin Bryant
    Story by

    Martin Bryant

    Founder

    Martin Bryant is founder of Big Revolution, where he helps tech companies refine their proposition and positioning, and develops high-qualit Martin Bryant is founder of Big Revolution, where he helps tech companies refine their proposition and positioning, and develops high-quality, compelling content for them. He previously served in several roles at TNW, including Editor-in-Chief. He left the company in April 2016 for pastures new.

    When Ami Ben David wanted to find out what users liked and disliked about his ‘WhatsApp for businesses‘ app Xros, he used a smart trick – he pretended to be a bot.

    In a world where many people are used to engaging with Slack’s automated ‘Slackbot,’ he created ‘Xrosbot’ – a ‘personal assistant’ that could answer questions about how to use the app.

    Aware that users may be too polite or reserved to tell a human everything that they liked and disliked, Ben David was careful to write his interactions in a ‘bot-like,’ clear and straightforward manner. He even went so far as to write messages in an external app before pasting them into Xros, so people wouldn’t see the notification that he was typing.

    Ben David says that he came up with the idea when he read about how Facebook’s M was using humans operating in a bot-like way. “We were able to gather tons of verbal user input, much faster and in a more personal manner than if we had waited for people to approach us.”

    The month-long experiment gave Xros useful insights into how users saw the app. For example, users only talked about the workgroup-related features in the app, so the development team killed all other features to focus on what was clearly most important. Not only that, the very name ‘workgroups’ came from Xrosbot feedback. Before that Xros had called them ‘channels,’ a term that non-technical users in particular didn’t understand.

    XROSbot discussion

    The feedback also led to new features, like the ability to easily invite users to Xros workgroups from email and WhatsApp conversations, and most requested of all, voice calls.

    Ben David says that very few people realized that Xrosbot was in fact a human, and he always came clean if anyone directly challenged him on the point. While some users may feel cheated by Xrosbot’s deceit, it was a harmless hack and a good example of startup hustle in a competitive market.

    “It was a very successful hack. Non-scalable for sure – it creates a lot of noise for every signal – but very useful in the early days,” says Ben David.

    ➤ Xros