This man used a clever trick to find out what users really thought of his app

When Ami Ben David wanted to find out what users liked and disliked about his ‘WhatsApp for businesses‘ app Xros, he used a smart trick – he pretended to be a bot.

In a world where many people are used to engaging with Slack’s automated ‘Slackbot,’ he created ‘Xrosbot’ – a ‘personal assistant’ that could answer questions about how to use the app.

Aware that users may be too polite or reserved to tell a human everything that they liked and disliked, Ben David was careful to write his interactions in a ‘bot-like,’ clear and straightforward manner. He even went so far as to write messages in an external app before pasting them into Xros, so people wouldn’t see the notification that he was typing.

Ben David says that he came up with the idea when he read about how Facebook’s M was using humans operating in a bot-like way. “We were able to gather tons of verbal user input, much faster and in a more personal manner than if we had waited for people to approach us.”

The month-long experiment gave Xros useful insights into how users saw the app. For example, users only talked about the workgroup-related features in the app, so the development team killed all other features to focus on what was clearly most important. Not only that, the very name ‘workgroups’ came from Xrosbot feedback. Before that Xros had called them ‘channels,’ a term that non-technical users in particular didn’t understand.

The feedback also led to new features, like the ability to easily invite users to Xros workgroups from email and WhatsApp conversations, and most requested of all, voice calls.

Ben David says that very few people realized that Xrosbot was in fact a human, and he always came clean if anyone directly challenged him on the point. While some users may feel cheated by Xrosbot’s deceit, it was a harmless hack and a good example of startup hustle in a competitive market.

“It was a very successful hack. Non-scalable for sure – it creates a lot of noise for every signal – but very useful in the early days,” says Ben David.

➤ Xros