Categories
Digital Analytics

Mobile Applications’ Analytics

The increasing penetration of smart phones along with better internet connectivity and reach has led to a growth in businesses looking at mobile apps to engage with their customers. Mobile apps enable marketers to reach out to potential customers at the right time, at the right place and if done correctly – with the right offer.

As with every opportunity, come a host of challenges – creating an engaging mobile application, driving app installs, ensuring engagement and regular app usage and estimating if the focus and expenses incurred are worth the investment. Collecting user-app interaction data is vital to mobile strategy success. Analytics can be used effectively to measure the performance of a mobile strategy and measure the pulse of the customers.

How to track mobile app analytics?

There are a host of tools out there, we prefer Mixpanel, Firebase and iOS App Analytics

Basic analytics for mobile applications

a. Download and Install statistics: Allows businesses to measure the growth of user base, types of devices, location and application ratings

b. Usage statistics: Measures the engagement levels, usefulness and the ability of the app to keep users engaged

c. Purchase Statistics: A direct measure of how many purchases, cart, abandonment etc. are being driven through the app

d. Crash and Application not responding (ANR) statistics: Indicates the performance of the application across devices and mobile operating systems

Mobile app data can provide businesses with a lot of information about the users and the app usage. These data, when converted into insights can help businesses design:

1. Ads to drive awareness and app downloads

2. Push notifications on searched/related products

3. Recommendation of new or unseen products

4. Personalized promotions

We present a few KPIs that should be tracked at the minimum to allow businesses generate actionable insights for better business outcomes:

1. Conversion* rate: Allows businesses to measure the fraction of users who are getting converted through the mobile app

*Conversion is defined as an intended outcome and not necessarily a signup or purchase

2. Traffic Heatmaps: Measures the traffic on each screen and the time spent by each user on a screen

3. Crash & ANR: Measures the crash and application not responding statistic

4. Users by App version: Provides statistic on number of users by version, indicating user lifecycle, engagement levels and few other proxies

5. Acquisition source: Helps a business monitor the source of downloads and the app download conversion rates

6. Retention: Allows businesses to measure the retention rates week on week, and can help distinguish seasonality, holiday behavior etc.

7. Demographics: Data on location, device, demographics and interests of the app user

8. Revenue: Revenue being generated by a mobile application

9. Event tracking: Data on automatically collected events and custom events, the number of times that event has been occurred and by how many user

Businesses should make use of data and mobile app analytics to generate insights about the mobile app usage. This can help them come up with new strategies to optimize the funnel for conversion and business growth.

Categories
Digital Analytics

How to perform A/B Tests for Digital Assets?

In our last blog, we presented the scope of A/B Testing, what it is and why digital platforms should focus on A/B Testing. In this blog, we will talk about how to perform A/B Tests.

How to perform an A/B Test?

A/B testing allows systematic way to find out what is working on a digital platform and what isn’t. Driving traffic to a digital platform is hard enough, and therefore providing the best digital experience to maximize the chances of conversion is of paramount importance. A/B Testing allows digital platforms to maximize conversions and identify the issues hindering conversions. It is important for digital platforms to create a structured and continuous A/B Testing plan, and not look at A/B Tests as a one-time activity. The steps in A/B Testing involves:

1. Research: Before initiating an A/B Test, it is important to create benchmarks i.e. how the platform is performing currently. Data points related to user visits, most visited pages, conversion goals of each page etc. The clicks and browsing behavior using standard heatmap tools can provide insights on time being spent on different sections of a page

2. Hypothesis Design: The next step then is to define a hypothesis aimed at increasing conversion. A sample hypothesis could be: A title having the product USP (unique selling point) leads to increased clicks on ‘View Details’ button

3. A/B Test Cases: The two versions of the webpage – based on the hypothesis in the previous step need to be created. Continuing with the previous example, the existing convention of product title is the ‘control’ and a page with title having the product USP would be the variation

4. Run Tests: Launch the test and let visitors generate sufficient data to help you arrive at statistically significant results.

There are primarily four types of testing: A/B Testing, Split Testing, Multivariate Testing and Multipage Testing. You need to identify the right test based on your experiment goals.

5. Analysis & Conclusion: Once sufficient data has been generated, analysis of results allows you to arrive at a data-driven conclusion i.e. which variation of the test is better. It might be possible that the test is inconclusive, in which case you would need to learn from the test and implement changes so that subsequent test(s) can provide clear winners

Mistakes to avoid while A/B Testing

1. Invalid or poor formulation of hypothesis: A/B Test case can only be created against a hypothesis you want to test. A poorly/wrongly formulated hypothesis will take you nowhere

2. Testing multiple elements in one A/B Test: A/B Test is mostly used to test one variation at a time – so that the differences can be measured. Too many variations may lead to ambiguous results

3. Not measuring statistical significance: The difference between two versions should be statistically significant to make the conclusions actionable

4. Unbalanced Traffic: Traffic should be balanced and not biased towards a certain type of visitors

5. Running a test for insufficient duration: Running an A/B Test for inappropriate time may result in insignificant conclusions

6. Accounting for external factors: A/B Tests should be avoided on days with higher traffic, holidays etc. to avoid skewed results

A/B Tests, if used well, can significantly improve the return on investment (RoI) of marketing campaigns. It helps the digital platform owners identify existing problems, address them and reach towards the desired conversion goals

Categories
Digital Analytics

Rank optimization with product detail, sales & search volume

The client was generating about 60% of their sales through Amazon. However, they were still thinking of SEO from Google perspective. And they were doing well on Google searches, having invested a lot of time and money to appear in top Google search results. They were not looking at Amazon searches differently – which resulted in less traffic and fewer sales. Amazon searches are focused on buying intent, and sales takes priority. Amazon SEO should therefore be viewed differently.

Amazon ranking begins with search behaviour

Users have two ways to buy on Amazon: directly from Amazon or from one of the retailers selling the product on Amazon. For Amazon SEO, sales is the single most important factor.

Amazon SEO provides higher importance to searchability of a product and sales

On page optimization covers product title, product information and images

Returned goods are rated as negative and lower the ranking

Multiple things need to be addressed to improve ranking on Amazon. The goal was to figure in Top 3 search results and stay there.

Amazon SEO different from Google SEO

The retailer had spent a lot of time and effort on Google SEO. However, they noticed that more than 60% of their sales was being generated through Amazon. And when they examined their search ranking performance on Amazon – the results were underwhelming. They realised that Amazon search ranking worked differently compared to Google, and required attention.

Amazon has a comprehensive system for categorizing its products. When a potential buyer searches for a product, the results could be displayed either in list view or gallery view. With the right keyword bidding, retailers can optimize the sponsored product listings for Amazon. Most buyers don’t look beyond the first three search results. As Amazon features a search engine, that means SEO becomes critical to success on Amazon.

Amazon is used ~4 times more than Google for product searches

Amazon SEO helped the retailer understand the differences between user intent on Google search vs. Amazon search.

The results shown on Amazon searches are driven by buyers’ past purchases, shopping preferences and other factors including:

a. Quality of product information that includes title of the product, product details and images

b. Seller performance – responding to user queries, responding to user reviews and comments are decisive factors

c. Product reviews and seller ratings are important factors in search ranking

SEO helps gain rankings and visibility on Amazon.

Categories
Digital Analytics

Purchase experience scorecard to improve CX and satisfaction

Purchase experience starts when a (potential) customer hears about a brand. In the case of our client, it starts with product search, discovery, purchase and finally the delivery experience. Our ecommerce client was focused on providing customer delight from end-to-end. They were not satisfied with NPS or CSAT scores, as they had extremely low response rates, and more often than not biased. They wanted a solution that would score for each transaction – even from same customer

Know about each purchase experience. Not just each customer.

Good purchase experience is the impression that a customer is left with on an ecommerce platform. With the implementation of the scorecard, our client was able to:An ecommerce platform uses a proprietary scorecard to measure customer buying experienceGood purchase experience is the impression that a customer is left with on an ecommerce platform. With the implementation of the scorecard, our client was able to:

Increase 1-month repeat purchase by ~14%

Improve 6-month repeat purchase by ~3%

21% reduction is service requests and 12% lower returns

The scorecard allowed the client to proactively monitor customer satisfaction and reach out to customers whose score went below a threshold

Beyond qualitative measures to score each purchase

The client was using Net Promoter Score (NPS) and Customer Satisfaction Scores (CSAT) to access customer satisfaction on their platform. They used to send out survey questionnaires to the purchasers as an automated communication. However, the client was not very happy with these measures on two accounts – the low response rates did not give them visibility on majority of purchases and the results were mostly biased towards either the highest or the lowest score.
They wanted a measure that would have a better coverage and not biased towards people who respond. Also, their inclination was towards a measure that was quantitative, cover all aspects of the purchase experience and provide them with a score that would reflect how the satisfaction score went up or down with every purchase.

More than a performance measurement metric

The scorecard was designed to measure multiple aspects of a purchase including ease of product search, time spent in finding the right product, purchase consideration time, ease of payment and finally, the delivery experience. With this, the client was able to:

a. Identity the experience at a transaction level, and monitor the drivers of the score

b. Identify customers whose score was below a certain threshold – and reach out to them proactively

c. Identify product discovery and delivery issues – and address them

The client is now able to score each transaction based on a feature set that is associated with the browsing and other behavioural data.

Categories
Digital Analytics

Algorithmic attribution of digital touchpoints in conversion

The telco client was running a digital customer acquisition campaign. They were advertising heavily across multiple platforms – social media, popular news websites and through agency adtech platform. They wanted to study the impact their ads were having across different digital channels, and how each touchpoint was contributing towards final conversion of the customer. They were also interested in the RoI from each channel and the optimal touchpoint frequency.

Identifying where credit is due

The client wanted to optimize their digital marketing budget using data-driven insights and using them to generate maximum return on investments (RoI)

More accurate distribution of weightage across all touchpoints

Identified cost per acquisition (CPA) and recommendations to reduce CPA

Optimal number of touchpoints for best conversion propensity

The study helped the telco identify leakage in digital campaigns, and optimize their future campaign strategy to minimize cost per acquisition (CPA) and optimize campaign spend.

Too much data. Scarcity of expertise.

The telco was using the rule based attribution models – First-Touch, Last- Touch and Time-Decay based models. All these models were giving more than 80% weightage to Google touchpoints. The client was aware that other touchpoints were contributing more. They also wanted a visibility on the return on investments and the optimal number of touchpoints to maximize conversions.
The biggest roadblock to implementing a more data-driven approach to attribution was the massive volume of data being generated. They lacked the expertise to unlock the insights in conversion data. The other challenge was dealing with a highly unbalanced dataset with less than 0.2% conversion rate. Our team provided the capability to combine our data handling and modeling expertise to unlock the strategic insights.

Beyond Last-touch, First-touch & other rule based attributions

Algorithmic attribution helped our client unlock the key marketing insights that were hidden in their data.

With algorithmic attribution (or multi touch attribution or MTA), the client was able to unlock key insights, including:

a. A more accurate estimation of credits across all touchpoints vis-a-vis rule based attribution models

b. Cost per acquisition (CPA) estimates and better understanding of the touchpoints that work best

c. Optimal number of touchpoints to maximize conversions, and potential campaign cost save

Multi Touch Attribution (MTA) de-duplicates user IDs across all channels, providing a more accurate, customer level view of the conversion journeys.

Categories
Digital Analytics

Next Best Action – right offer to the right customer at every touchpoint

The financial services marketplace was selling multiple products – credit cards, vehicle loan, personal loan, insurance from multiple financial services providers. They wanted a customer centric focus rather than a product centric view. This meant developing a solution that would allow them to offer products to each customer based on financial value, propensity to accept an offer and business priority, rather than single products being marketed through an expensive outbound campaign.

Optimized offers to customers, through the right channels

Using Next Best Action approach, the client was able to integrate multi-channel interaction with the customers to make offers valuable to both – the customers & to them.

Net uplift of ~18% on outbound campaigns (for the pilot)

Cost reduction of ~40% on outbound campaign cost

Consistent and consolidated communication across multiple channels

Making the customer as the central focal point optimized the campaign and offer strategies, generating better experience for the customers and a higher lifetime value

Customer focus instead of product focus

The client was using propensity modeling based cross sell approach for outbound campaigns. They were able to generate the usual benefits of cross sell i.e. higher uplift and lower spends. They were also able to identify factors that were driving subscription to a specific product. Cross selling was driving undesirable behaviour from a significant number of customers.

The client was looking for something more – a solution that would allow them to market the right product to each customer and integrate the multi-channel communication that the customers were having with them. We proposed a real-time decision engine to find the right product and integrated communication across channels – to make the right offer. With this approach, offers are now only made to customers when it adds value.

Manage multi-channel customer interactions

Next Best Action (NBA) is focused towards relevance and timeliness of push campaigns.

If done effectively, NBA implementation has the potential to boost multiple KPIs, including:

a. A better informed and updated communication stream

b. Making more relevant offers based on customer needs

c. Improved response rates and better campaign RoI

d. Marketing all products, and having a best offer for each customer

NBA drives smart, cost effective and value based reach out through the right communication channels.

Categories
Digital Analytics

Purchase Experience Scorecard – to boost customer experience and retention

Purchase experience starts when a (potential) customer hears about a brand. In the case of our client, it starts with product search, discovery, purchase and finally the delivery experience. Our ecommerce client was focused on providing customer delight from end-to-end. They were not satisfied with NPS or CSAT scores, as they had extremely low response rates, and more often than not biased. They wanted a solution that would score for each transaction – even from same customer

Know about each purchase experience. Not just each customer.

Good purchase experience is the impression that a customer is left with on an ecommerce platform. With the implementation of the scorecard, our client was able to:

Increase 1-month repeat purchase by ~14%

Improve 6-month repeat purchase by ~3%

21% reduction is service requests and 12% lower returns

The scorecard allowed the client to proactively monitor customer satisfaction and reach out to customers whose score went below a threshold

Beyond qualitative measures to score each purchase

The client was using Net Promoter Score (NPS) and Customer Satisfaction Scores (CSAT) to access customer satisfaction on their platform. They used to send out survey questionnaires to the purchasers as an automated communication. However, the client was not very happy with these measures on two accounts – the low response rates did not give them visibility on majority of purchases and the results were mostly biased towards either the highest or the lowest score.
They wanted a measure that would have a better coverage and not biased towards people who respond. Also, their inclination was towards a measure that was quantitative, cover all aspects of the purchase experience and provide them with a score that would reflect how the satisfaction score went up or down with every purchase.

More than a performance measurement metric

The scorecard was designed to measure multiple aspects of a purchase including ease of product search, time spent in finding the right product, purchase consideration time, ease of payment and finally, the delivery experience. With this, the client was able to:

a. Identity the experience at a transaction level, and monitor the drivers of the score

b. Identify customers whose score was below a certain threshold – and reach out to them proactively

c. Identify product discovery and delivery issues – and address them

The client is now able to score each transaction based on a feature set that is associated with the browsing and other behavioural data.

Categories
Digital Analytics

A/B Testing for digital conversion optimization

Every site visitor is a potential customer. Visits can help digital commerce businesses acquire new customers or engage existing ones. All businesses want visitors to take actions that can be categorized as a conversion (Here, conversion is defined as a desired outcome, and not necessarily a purchase made on the website/app). The better optimized the conversion funnel, the higher the chance for visitors to convert. And one of the most effective ways to optimize conversion funnel is through A/B Testing.

What is A/B Testing

A/B Testing is arguably the simplest, and most effective method to drive better conversion of site visitors. It is worthwhile to note that A/B Testing scope is limited by one’s imagination, and can be utilized for multiple other business problems e.g. Does 10% discount on a product drive better repeat customer volume or 20% cashback?

A/B Testing is a method of testing two variations of the same outcome against randomly selected user segments at the same time to compare which variant drives more conversion. As is obvious, the variation that drives more conversions is the preferred/better option.

Why A/B Test?

With A/B Testing, a digital commerce platform can discover answers to:

a. Visitor pain points: Visitors on a digital commerce website come for specific purpose. It may be to buy a product, compare products with another platform, know more about the product or to know more about the digital platform. Visitors might find difficulty in navigating a page, search results based on keywords entered or finding customer reviews. Not being able to achieve the purpose of the visit will lead to a bad user experience and a potentially dissatisfied visitor. Browsing data, heatmap analysis and A/B test results can help a digital platform identify the pain points and consequently address them

b. Optimize conversions: Every business is aware of how hard it is to bring a potential customer to their platform. A significant part of all the marketing effort is to acquire new customers. And once a prospect has been brought to the platform – businesses would want to convert most of these visitors. A/B Testing allows platforms to test the conversion rates and optimize the conversion propensity – thereby improving return on investment and better conversion funnel

c. Promotion Effectiveness: Every digital platform is jostling for buyer attention, and one way to drive short-term sales is through promotions. It is however more important to know which promotions work, and are able to drive conversions. A/B Testing can help digital platform optimize promotions at multiple levels – SKU, brand, category level. With the abundance of ‘% off’ tags and 24*7 running offers, it is very important to get the promotion effectiveness right – and A/B Testing can be the panacea

d. Webpage/App optimization: Digital platforms may not be achieving the desired conversions or the KPIs e.g. bounce rate is high, low session time. A/B Test enables platforms to make minor changes (one at a time) on the webpage and understand sections/layouts driving better outcomes. This helps in two ways:

1. Enables faster testing of hypotheses – and make minimal modifications on the existing website/app

2. Lets platform owners draw data driven evidence of the issues with current layout if the website redesign is desired

3. If a new feature is being planned, A/B Test allows to test the best way to introduce the feature – by testing different variations of the webpage.

Making changes based on A/B Test results can make the outcome of a change certain through statistically significant improvements.

We hope you have a better understanding of A/B Testing and the benefits that can be derived through them. In the next part of the blog, we will discuss how to perform A/B Tests and different types of A/B Tests.

Categories
Digital Analytics

Recommendation engine for a hyperlocal retailer

The hyperlocal retailer started their business with milk and egg delivery. Once they had onboarded significant number of customers, they expanded the items being sold to ~ 10,000 SKUs. Their biggest issue was that about 70% of their revenue was still being generated through milk and breakfast product offerings. They wanted to implement a solution that would help them in SKU visibility – that would eventually lead to a more diversified revenue stream.

Not just in-app recommendations

We suggested taking a leaf out of Amazon’s practice of not only in-app recommendations, but also through personalized communication with the customers – through emails and SMS

Improved unique SKU visibility per user by 14%

Repeat sale of non-milk SKUs increased from 3.2% to 5.8%

Average basket size improved from 1.53 to 1.61

Historical purchase data and few other data points associated with purchase history were used to develop a collaborative filtering based recommendation engine.

Multiple attributes. Lack of clarity

The hyperlocal retailer was inundated with purchase and browsing data. They were use association rules and market basket analysis to analyse the purchase basket and using it to reach out to the customers. This solution was useful in the shorter term when they were offerings limited number of SKUs. With addition of more than 10,000 SKUs, the biggest challenge for them was to make customer aware that they were selling more than milk and egg products.

Another unique element of their business model was the repeatability of purchase. They observed a clear trend of customers buying a few products at regular intervals e.g. a customer would buy 1 litre pack of a specific brand of olive oil every 3 weeks. They wanted the recommendation solution to incorporate product purchase cycle into the algorithm. We developed a constraint based recommendation engine solution for our client.

A wise business turns chance into good fortune!

Any business spends a lot on customer acquisition. And the cost of acquiring new customers is about 8x more than retaining an existing customer. Cross sell and Upsell are proven retention strategies that are used by offline businesses.

The issue with implementing a cross sell solution (similar to offline businesses) is the volume of products on display and the limited real estate for app based purchase platforms. Recommendation engine solves these problems for digital businesses by:

a. Improving SKU visibility

b. Suggesting products based on propensity of purchase [Cross Sell & Upsell]

c. Personalized SMS and email campaigns

These benefits make recommendation engine an indispensable solution for digital stores