Engineer your data to get the answers that you need

After we added a couple hundred users to our mailing list, we decided to send them a survey with some very important questions about our product. I believe that asking before developing is crucial … test before launch. My intention was to target small bloggers who, in my opinion, will be our early adapters.

I began research and answered probably a few dozen surveys to learn what great companies are asking. After, I wrote 30-40 relevant questions and started to build our survey using Wufoo, a great survey building software. For me, it was about how to get the best looking survey out there. It doesn’t matter which software you use, but use one of them.

Building your own survey from scratch is a big no-no; think about data analysis. The biggest value this software gives you is the ability to monitor and learn from the answered questions.

Because Wufoo’s free version allows you to ask only 10 questions, I chose the best one and connected it to my MailChimp account. Another important thing was to create a very simple video, which explains generally what we thought of doing for small bloggers. (We had a really tough time deciding if that video will harm us because it is too simple.) To make a long story short, after a few days we had an online survey ready to be sent to our bloggers mailing list.

I sat for a few days waiting for hundreds of answers, who would enrich us with feedback on how to build the product better for our users. I sent it to all of the relevant LinkedIn groups, Tweeted it, Facebook shared it and the reaction I got was (from our MailChimp statistics):

You can imagine that even fewer actually answered our survey. Until now, I’m not sure exactly what I did wrong. Here are, nonetheless, a couple of things I learned in the process:

Researching Is Not Engaging: I spent long hours researching relevant bloggers whom I’d hoped would be our early adaptors. However, I neglected engaging them prior to the survey in order to get a better response. Researching is extremely important, but convincing bloggers and users you don’t already have a connection with to answer a survey is really tough. I tried, but apparently not hard enough, to make those early adaptors feel a part from our family. Perhaps engaging with a smaller group of bloggers would have produced better results.

The Look & Feel: We received much better results after we changed to our new design. When we first sent the survey, we had a 2 minute homepage; people went to our website before answering the survey. Apparently they didn’t like what they saw. It was something we knew could harm us, but we didn’t think that it would be so important.

Ask The Basics: As part of our A/B testing, we learned that there aren’t any good or bad questions, it’s just a question of relevancy. What I learned is that asking 50% of the questions on things you are not sure and 50% of the question on the features you are sure will rock, produces (for us) the best results. We were very surprised by the answers we got on the most basic features we are developing.

Engineer Your Data: I think it was the first time we got the users’ opinions on our thoughts. It was only a segment from our early adaptors (Yotpo’s main focus is on e-commerce and bloggers), but we got valuable data. This was the first time we had users’ data and now we need to decide what to do with it. I’m sure we didn’t get all the answers we wanted and the survey will be relevant for the next few months, but we are starting to create data. I learned that data management is crucial in order for us to learn and succeed as a company. I think I read it @OnStartups … Engineer your data. Bring your data management process to an engineering level.

I’m sure our survey is far from being perfect, but what are your thoughts?