Beta Testing in the Time of Corona
Seven Lessons from the private beta of Moja, a mobile app that enables family and friends to pool money simply, affordably and transparently.
Tarig Hilal, Gerard Mc Hugh, Will Gurney
Coronavirus has dealt a blow to businesses around the world and thousands of startups have been pummelled by a triple whammy of slowing investment, falling demand and restrictions on travel.
Not all startups have been equally affected. Indeed some are doing well; in industries like healthcare and robotics, companies like Kry, Hopin and Blue Ocean Robotics , are thriving.
The trick it seems is to 1. Have a value proposition that meets the challenges of the moment 2.Be fortunate enough to be in the right stage of the investment cycle and 3. Be determined to hustle.
It was in this spirit that in April we launched a beta version of Moja, a smartphone application that enables family and friends to pool money simply, affordably and transparently. The Corona pandemic prevented us from travelling, but the Moja value proposition was more relevant than ever, and we were determined to find a way to proceed.
After a bit of reflection we realised that we could do most of what we wanted to do remotely. The downside was that not all of the team would be on the ground with our customers, the upside was that it forced us to be more hands off with on-boarding, which gave us useful insights into customer acquisition.
The goal was to run a private beta with an 8 person pool. Customers would use Moja to pool money over a 16 day period during which we would monitor engagement and solicit their views on the product. The challenge was to recruit customers remotely and figure out the best way to communicate with them over the test period.
We had made a point during Moja’s prototyping and build phase of keeping a record of customers; and reached out to a user from our January testing phase and offered him a small incentive to recruit others. In terms of communications it quickly became clear that Whatsapp was the optimal tool.
Customers could provide qualitative feedback through WhatsApp messages and regular video calls and we were able to use our apps analytics to gather quantitative data on usage
Here are seven lessons that we learnt from the process:
1. Think carefully about how you will structure communications and set expectations
This would be good advice in any circumstances, but it is doubly so when you are communicating over distance and across cultures. We thought carefully about how to explain the Beta launch process to our customers and set clear expectations. This included drafting WhatsApp messages and reviewing them with our point person on the ground, mapping out our communications strategy over the two week period and identifying the optimal time in the day to talk to customers, whose work schedules looked very different to our own.
2. Be proactive about finding out customers’ preferred communications tools
For reasons of cost, language, cultural preferences or tech trends, your customers may not want to/be able to use the same communications tools as you. Don’t assume that they will point this out to you. In our experience, customers can be very polite, even shy and will often go along with whatever you suggest, meaning that you only find out that you have chosen the wrong tool when you have problems connecting !
We assumed, for example, that skype would be our customers preferred tool. As it turned out customers wanted to use Jitsi, a free video conferencing service popular in Kenya. So ask the question about communications platforms up front.
3. Plan for problems with mobile internet access.
For many customers, accessing good mobile internet service can be a challenge, data is expensive and coverage can be spotty. We learnt this the hard way, make sure you help with the cost of purchasing data. Also check with customers for the optimal time and place for a conversation, there are times in the day when coverage is better and sometimes a customer can plan to be at location where they have access to wifi.
4. Keep video conferencing to a minimum
We organised a series of video chats to discuss customers’ experiences but problems with internet connectivity meant that our attempts were marred by frequent disruptions. Moreover it quickly became clear that our customers were uncomfortable with the video format, it was unfamiliar and when combined with a language barrier (English is a second language for most of our customers) made for some awkward exchanges. Questions were often met with long silences and people struggled to stay focused.
5. Use messaging to solicit feedback
We found our customers to be extremely responsive on WhatsApp. At the beginning of the process we created a shared WhatsApp group with users, setting it up as our key communications channel. Customers liked being able to respond to questions in their own time and without the pressure of being on a conference call. The group doubled up as a customer support platform and at the end of the process we had a WhatsApp diary of our communications for analysis
6. Find ways to compensate for lost behavioural signals
For all of the wonders of modern communication, the technology falls far short of real world interaction, especially when you are trying to really listen, engage and understand. A lot of behavioural signals are simply lost and there are painfully numerous possibilities for distraction. So, keep video interviews short and make sure you are present and attentive; switch off notifications & turn off your phone; use whatsapp for follow-up questions. Set aside time in your day to read and respond thoughtfully to text messages. In addition we asked permission to record conversations which freed us from taking notes.
7. Make sure you don’t forget the niceties
Without the social cues of being in the same room it’s easy to cut straight to the chase. This is a mistake, when it comes to interviewing customers remotely it’s important to remember to put people at ease and a little polite conversation before you jump into questions goes a long way.
