Carrot Quest Case: HADI-cycles and how to increase project development with their help

Leonid Zverugo

wrote this on Дек 14, 2015

Ilya Krasinsky, AppCraft CEO, a team of a Carrot Quest service and IIDF representatives have carried out a workshop in order to increase unit-economy of 5 startups with the help of HADI-cycles.

Dmitry Soldatov of Carrot Quest has published an article on the basis of the workshop.

HADI-cycles is a methodology tool which helps to enhance startups in IIDF accelerator. HADI essence is simple. Almost any action affects a definite metric. If we can add changes to indices beforehand (to formulate hypotheses), we can control all the process of changes. You’ll be able to understand the way your actions affect the result and test your ideas quickly, getting rid of those that don’t work.

That’s the way a HADI-cycle looks like:

 

• Hypothesis. Write down hypotheses, changes (your actions) that can enhance indices of your economy. For example, “headline change will increase conversion into registration on landing by 5%”. Firstly, generate hypotheses on the indices that should be changed as soon as possible. It means that if there’s a problem with conversion into registration, it’s necessary to generate ideas that will directly affect this index.

• Action. Take several hypotheses and realize them at the beginning of each cycle. For example, rewrite headlines and add benefit for your clients there. The main formula is to do it quickly. If the change is beneficial, it’ll be easy to enhance and scale it. The most important in HADI is a quick check.

• Data. Then start gathering data on the indices affected by changes. In our example, it’s a conversion into registration. Selection (a quantity of visitors during testing) should be representative so that indices show real changes.

• Insights. Analyze whether a hypothesis has worked. If yes, then enhance and scale it.

 

But not all hypotheses will work. In order not to waste time on weak ones, you can use a special ranking algorithm. We’ll talk about it a little bit later.

Don’t check several hypotheses that affect one and the same metric simultaneously. You won’t be able to understand which one has worked and what exactly has changed an index. For example, you’ve changed a product description on your website and made a different payment system. The quantity of purchases decreased. Why? It will be difficult to find an answer to the question.

 

About HADI practice

5 startups took part in the workshop:

 

• Carrot Quest (a service for communication and user management on the basis of their communication)

• LPmotor (a service for quick landing creation)

• SkyparkCDN (CDN, a content delivery service, which speeds up content uploading)

• PlayKey (a cloud game service which allows to launch modern games on any device)

• Capsidea (a service for data visualization)

 

Several other teams also joined us in the process.

We analyzed landing pages of all startups and then they knew their problems and understood what to work on.

With the help of brainstorming, teams wrote down ideas and added to a special table.

Belief in effect shows how strongly a team believes in the hypothesis. The difficulty of realization means how many resources are needed in order to check the hypothesis (1 - easy, 5 - difficult).

Here is an example from one team:

Each team formulated 10 hypotheses.

In order to continue working, it’s necessary to arrange hypotheses according to their priority. There should be the most important ones at the beginning of the list, and the weakest ones should be abandoned in order not to waste time on checking them. If your belief in effect is low and it’s difficult to realize it, you should put off such a task.

Then, startups were testing their hypotheses during a week. For the sake of objectivity, we assigned a task: acquire 3000 visitors in a week. It will mean that conversion will increase or decrease by more than 1%. And the reason for it is our change.

For the majority of teams, it was the first experience of work with HADI-cycles. Someone checked 2-3 hypotheses, someone managed to check only 1. Some hypotheses worked well, others failed and led to a decrease in indices.

A week is an optimal period for short iterations, teams analyze their HADI-cycles in an accelerator once a week on a Saturday traction meeting. They also make up a plan on testing hypotheses for the following week and it helps them to focus. The main value for a startup is that each team checks several ideas quickly and see which ones work. A team also gets feedback, if result are shown to other teams in an accelerator or a similar workshop.

 

Examples of work with hypotheses from Carrot Quest

A little bit about Carrot Quest, so that you understand our specifics. Carrot Quest is a service for communication and user management on the basis of their behavior. It unites tools for internet marketing: eCRM, online chat, pop-up windows, newsletters and web analytics. Our target audience — SaaS-services, online shops and web projects.

Our task is to increase conversion into registration, that’s why we’ve built our hypotheses for this metric especially.

Here’s the work on one of hypotheses:

It was: “A personal approach to each visitor of our website. We’ve collected data on actions of 3,65 million users on 570 websites and increased conversion by 355% with the help of trigger emails and chat messages. Try for free.”

We decided to show that leaders of various market segments already use marketing automatization and personal communications in their projects. We found companies which surely automatize marketing and used their names in the headline.

It became: “Ikea already uses personal communications with its users in order to increase sales. Are you ready for it? We’ve collected data on actions of 3,65 million users on 918 websites and increased conversion by 355% with the help of trigger emails and chat messages. Use Carrot Quest - 14 days for free.”

We changed company names (Ikea, Kaspersky, Intel and so on). But conversion rate didn’t grow. So now we test other hypotheses.