Video Analytics

Detecting potential shoplifting incidents in advance using AI

The retail brand has been facing the issue of shoplifting at the retail store. They have an active CCTV based surveillance infrastructure in place throughout the shop floors. However, the process is manual and the security team looks into the footage to identify shoplifters after a theft has been reported. They are looking for a predictive solution that can flag potential shoplifting through behavioural analysis of the body language and general patterns in clothing to conceal identity.

Identifying shoplifter behaviour

The client wanted to test if behavioural analysis of shoplifters – body language, movement patterns in store, clothing and actions to conceal identity can flag potential shoplifting.

Identified three out of ten shoplifting incidences pre-emptively

A high number of false alarms leading to operational difficulties

Scope of expanding the application for shelf space monitoring

The study was able to identify a low percentage of shoplifting cases. The high cost involved outweighed the benefits of the solution.

Long training time. High video resolution

The wholesale retail brand was losing thousands in thefts every month. In most cases, the act was spotted through CCTV camera feeds after a shoplifting incident had been reported. The manual inputs required to monitor such incidents when they are happening is not practical given multiple camera feeds and general tendency of humans to lower their guard for isolated incidents.

They were looking for a solution that could help them overcome these issues of human error. The client turned to computer vision based monitoring of feeds from multiple CCTV cameras across the floors and develop an algorithm to flag suspicious activities based on body language, clothing patterns to hide identity or general demeanour of shoplifters. Their objective was to flag sufficient number of shoplifting incidents before they happen.

Visible benefits with high risk of misuse

The Deep CNN based behavioural monitoring and flagging of potential shoplifting incidences provided promising results in the pilot.

With limited data and training, the solution was able to identify three out of every ten potential shoplifting incidents correctly.

There were certain downsides to the approach:

a. A significant number of false alarms

b. High cost of running the application 24*7

c. Concerns around security as the CCTV feeds also captured the biometric features of shoppers

The client realised the benefits and the risks associated with deploying the solution on full scale.

Video Analytics

Video Analytics: Are the possibilities lesser than what is being promised?

Video Analytics or more precisely, Video Content Analytics haven’t quite matched the hype. Beside the well-founded concerns of privacy and security, there are functional and non-functional challenges to a successful video analytics solution deployment.

The promise of machine learning and artificial intelligence driven video analytics only if all issues and concerns are addressed – and knowing them is the first step towards finding solutions to those concerns. We discuss some of them in this post.

a. False Alarms

Any machine learning or artificial intelligence based algorithm has an inherent error – commonly classified as Type I and Type II errors.

Type I Error: An event of interest has not occurred but the algorithm think it has

Type II Error: An event of interest has occurred but the algorithm fails to detect it

These errors create nuisance and a general frustration for the agency where the solution is being tested or deployed. A feeling of half baked solution with no understanding of when to react to an alarm and when to let it go may seep-in in the longer term

b. Impact of ambient factors

The performance of a video analytics solution is highly dependent on feed quality. If there are extreme weather events – heavy rains, sudden dark clouds, gathering of large number of people, a traffic jam etc. – the performance of the video analytics solution may falter. And as these conditions are some of the extreme cases for which video analytics solution has been sought – the trust may completely evaporate

c. Unsuitable use cases

This problem occurs because of the hype and mis-selling associated with machine learning and artificial intelligence based video analytics solutions. While not a problem with the solution itself, but unachievable promises by the selling party may lead to disillusionment and negative publicity for the solutions

d. Machine vs. Human Interpretation

Video Analytics solutions are based on machine learning & artificial intelligence algorithms. Machines process and understand a feed differently than humans do. Infact, unless explicitly programmed, these algorithms cannot interact with (or read emotional response of) the users to find out if it’s response to a query was useful.

e. Cost

The cost of an effective and intelligent video analytics solution suffer with huge upfront deployment cost and high maintenance cost. The infrastructure required to process all the feed generated through cameras is not cheap – despite all the progress made on cheaper bandwidth and databases

f. Privacy concerns

More than 80% of world population doesn’t live in China. Citizens have a say on their privacy rights and can punish governments if they feel the State is snooping on them at will. And being a responsible company, we should point out here that these concerns are not unjustified. There is a high risk of the solution being misused. And our hypothesis is, if something can be misused – it will be. It is therefore important for the agency to look carefully into the possible misuses – and create ironclad checks and balances

Video analytics can be a gamechanger – if implemented to bring effective change. While we have not found any study establishing the benefits of video analytics solution bring about safer societies e.g. Reduction in crime rates on implementation of a city surveillance system, we strongly believe in the potential of video analytics.

To be successful, the requirements should be well defined and apart from specific challenges, the concerns raised above should be given a careful consideration.

Text Analytics

Data Analysis Chatbot Speeds Results for Pharma Co

Like many companies,this UK-based pharma organization had already invested in data analytics. But something was missing. They had the data, the people, and the tools … but no way to get information to users at speed. They needed something that would enable business users to get instant answers to their queries without involving the data analysis team.

Instant Data, Always Available

By integrating a Natural Language Generation (NLG) powered chatbot with their existing Some Image infrastructure, we enabled our client to:

Provide instant answers to users’ natural language questions

Empower business users to get their own information, when and as needed

Scale up their BI investments

This chatbot transformed how company data was consumed and utilized. And it was set up in just 15 weeks

Too Many Inputs, Not Enough Insights

Some Image Our client’s situation will be familiar to businesses that Have implemented data analysis but haven’t gotten all the benefits yet. They had multiple data sources, internal and external. They had Tableau and other data visualization tools. They had several dashboards for their business users. And they had a team of data analysts to help access specific insights. But there was still a bottleneck. As their system grew, so did the number of requests pouring into the data analysis team’s inbox. What the company needed now was clear: an easy, automated way to answer basic analysis questions and guide users to the appropriate dashboard. It would have to be simple enough for non-technical users, yet powerful enough to provide real-time insights. So that’s what they got.

A Chatbot That’s More Than A “Data Telle

Chatbots that simply find and relay data are becoming commonplace. But our clients needed more than just a “data teller” – a chatbot that merely passes on raw data. Our AI-enhanced chatbot assistant, ASK NAVIK, uses Natural Language Processing to understand users’ spoken queries ( e.g. “How did brand A do in the Central Europe market last year?”) It automatically finds and presents the information in a natural format (e.g. “Brand A sales grew 2% in the Central Europe market in 2019”). Now, there’s no waiting for the data analysis team to answer routine queries; they can spend their time more effectively. Users get the needed results in seconds, not days. And different types of data – such as sales, activity, and shipment information – are available to users whenever and wherever they need it!

Visualization & Reporting

Avoid being Data Fashionista. Focus on visualizations that matter!

Generating eye catching visualizations and charts is becoming easier by the day. With the data deluge, and the pervasion of data science, machine learning and artificial intelligence without an understanding of where the data is coming from or existence of bias in the way the data has been collected is leading most (if not all) of us being a victim of unconscious propaganda and digital activism. Add the need to grab attention in the content based marketing world – driven by obsession with search engine rankings and grabbing most views, likes and shares on professional networking platforms – we have the perfect recipe for disaster.

Having more data doesn’t make it easier to communicate. It makes it harder!

The root of all evil

In schools, we have language subjects and then we have math subjects. It is rare that anyone of us were taught how to combine these two. And that leaves us undercooked for one of the most important tasks for the digital world we live in today – making sense of all the data. Being able to process these data, visualize it and developing a compelling story on it will enable us to fulfill our desire of evidence based decision making. Being adept in Tableau, Power BI or R Shiny is not a differentiation – being a storyteller is.

Key considerations for compelling data storytelling

a. Setting the context: The first item on this list has nothing to do with visualizations. To communicate well, we should know who we are talking to. And why. If you were talking to a brother versus the dad – about a school game you won, you would choose different words, tone and details in the conversation, wouldn’t you? Same goes for a dashboard or a report. Is the dashboard for use by the sales team or for the Chief of Sales? And then, we should consider how is it being done – whether it’s a conference room presentation, just and email or a monthly report

b. Knowing how to show results: I have personally used more than 100 different visualizations in my 10+ years of professional work. If I were to analyze all the work I have produced, the results might follow the Pareto principle i.e. 80% of my work would be based on only 20% different visuals. Broadly speaking, we should know when to use tables, pie-chart family of visuals, bar-graph family of visuals, line graphs. It would also be valuable to know the purpose of showing a visual – is it a comparison across years or growth over the years that you want to highlight? What visuals to choose if the growth numbers are low in absolute terms (but might not be a poor achievement) vs. if the scale of growth is, say 20-40%? And then, you need to decide what details are you going to show?

It is obviously not possible to go through each type of visualization here, and that is not the intended purpose even. In most cases, you would have to choose between multiple suitable options. The general principle is – the simplest visual that captures all that you want to convey is your best choice

c. Design Thinking: The visualizations should be intuitive and simple to interpret and understand. Always remember, the purpose of the visualization is to communicate – and not showing off your ability to be able to create awesome looking graphs that divert attention, and lead the audience away. You should try to highlight the important stuff and eliminate distractions. The designs should be accessible, and not overcomplicated. And lastly, make your visuals aesthetically pleasing

d. Weaving the Story: Why do we love reading books? Because, it grabs our attention and creates an emotional connect. And after finishing it, you might also discuss about the book with your friends. And since we have been communicating with stories, in infancy and beyond, it is easy to leverage storytelling to communicate with your audience and create an emotional connect. One of the approaches could start with a plot for your data story, filling in the details in the middle part and call to action at the end. And in case you are wondering, who is the subject of the story – it has to be your audience

e. Putting it all together: Every data story should begin with the audience, and end with a story. And as we discussed above, the story should have key takeaways. It helps if you understand the purpose of your work i.e. why are you working on creating a dashboard or a report. And the intended use of the dashboard.

At Xtage Labs, we try to blur the line between numbers and insights. We see visualizations more as an art than science. And since dashboards, reports or just a power point presentations are the primary ways in which we deliver our final results, we understand the need for compelling visuals and communicating effectively.

Credits: I would like to acknowledge that the post is broadly a summarization of Cole Nussbaumer Knaflic’s work – Storytelling with Data

Image Analytics

Clustering of digital ads using automated tagging of images & videos

The media client wanted to create a generic solution to categorize all creatives used by different businesses. This would help them understand the brand positioning, target group and opportunities for their clients and competitors. OCR like capability was sought to extract text from the creatives. They were looking at a technology driven solution due to the explosion of digital marketing. It had become impossible for them to process digital ads manually.

Right tags for better results

To explain the overall messaging of a creative, a creative tagging system is very powerful, scalable and effective. We used attention representation to reflect the human visual system.

Reduced human effort of tagging creatives by 80%

Improved the number of creatives processed/day by 4x (further scalable)

Improved tagging with better description on qualitative validation of output

The number of parameters needed for the tagging algorithm is small with much lower computational complexity.

Lack of scalability & standardization

Display ads are required to be categorized for brand safety and sensitivity. Advertisers are required to mark their ads into several categories such as suggestive, violent or deceptive. As advertisers do not always have a clear understanding of these categories, creatives are manually labelled that can reduce the effectiveness of the ad and reach. Additionally, manual labelling is limited in terms of taxonomy.

A major issue with digital ads is multiple resolution and high variation of fonts, colours and layouts. Texts are integrated into the creative and any automated tagging exercise requires extracting both – the textual and the visual attributes from an image. We proposed using a recurrent neural network (RNN) based approach with attention representation to generate natural text tags.

Improved results with minimal human involvement

The solution we deployed was able to provide a more textual description of creatives, improving the tagging accuracy.

When stemming is applied on captions generated with the algorithm, the distinctiveness is better expressed as they are mapped to the same word even if the tense and form are different. The solution provided the media agency with following benefits:

a. Improve the efficiency of manual categorization by suggesting a list of possible tags from which editors can choose the best categories

b. Use the solution to backfill the categories of ads which classification was not completed

The features from the creatives is used as a channel of attributes in matching algorithms for ad-selection.

Image Analytics

Advanced Analytics with Satellite Images

Not so long ago, satellite images were available only to the government agencies and their analysts. Now, there are known sources for free and open source access to full satellite data. We also see a lot of private enterprises having interest in satellites for the following reasons:

1. Satellite & launch costs are decreasing

The infographic below shows the estimated cost comparisons in the context of recent SpaceX launch

2. Small launches are relatively much cheaper

3. The huge data infrastructure and bandwidth costs have gone down

4. The pervasion of image analytics based AI, reducing the entry barrier for data scientists & researchers

The above trends mean we have started seeing some out of the box use of satellite images to generate insights for businesses.

We list out some use cases for analytics/insights using satellite images:

a. Environmental Applications

1. Monitoring forest cover: Longitudinal data for a forest or reserved forest allows tracking of increase or decrease in forest cover, effects of urbanization and any illegal deforestation being carried out. It allows governments and green activists to raise awareness about issues that affect all of us

Image source: Earth Observatory, NASA

2. Water Scarcity Monitoring: Water is the most valuable natural resource. Less than 1% of water is available for drinking. With satellite images, we can now understand the processes driving the water cycle and the impact urbanization and climate change are having on the availability (rather, dwindling) of freshwater water sources across the globe.

Image source:

b. Economic activity estimations

Economic activity indicators are key to economic planning, investment decisions and knowing the growth prospects of one (economic) zone over the other. Night time illumination data, extracted from satellite imagery can provide access to this information in near real-time, rather than wait for the governments (and provincial agencies) to release those numbers at an aggregated level for a month or quarter, that too with a lag of upto six months.

A custom study for a PE firm clustering areas of economic activity based on NASA nighttime Illumination data

c. Estimate retailers’ sales

Using geospatial data, satellite images for a specific retail store (or a group of retail stores e.g. All Target stores) can be extracted and the traffic (vehicular as well as human) can be estimated using object detection algorithms. The use of satellite images to monitor own store performance as well that of competitor’s hands a competitive advantage to the decision makers who can think out of the box.

Sydney beach image by DigitalGlobe

d. Monitoring Port Activities

Ports and coastlines can be monitored to establish the baseline for the activities and measuring spike/reduction in traffic using satellite images. Given the proportion of trade between countries carried out through the seas, it becomes critical for businesses across sectors and organizations. This data can also be used for re-development of ports and associated facilities.

e. Cultivated area monitoring

Satellite images taken over farms allows businesses to estimate the area under cultivation for different crops. Depending on the business focus, this data can be used to estimate the expected yield for each crop, adverse weather conditions and timely alerts to mitigate risks, demand for seeds, pesticides etc. and supply chain planning. Satellite image analysis can help quantify uncertainty and manage food security more efficiently.

If you have business problem that requires satellite data or the solution can be made more efficient using satellite data as an alternate data source, please drop us a note. We can help you with the right open source to get the images from, recommend the spectral layer to look at and execute the analysis.

Text Analytics

Better candidate to job order matching using AI driven algorithms

Recruitment industry is primed for disruption due to expensive and inefficient processes towards recruitment. Early adopters are already reaping the benefits of machine learning and AI algorithms as they develop, and these developments are bound to change the recruitment industry as we know it.

As with any process that needs to make use of machine learning and AI, the availability of good quality data is the first step. Having latest resumes, correct contact information – email id and phone number, recruitment process data – candidates that were shortlisted (and rejected), called for interviews and final decision makes the machine learning algorithms achieve performance levels better than humans in some cases. It is important to note that these machine learning driven algorithms have a definitive impact on the hiring process, and organizations have started collecting the right data to be able to deploy these solutions.

Here, we discuss each stage of the recruitment process and how can a machine learning algorithm help companies generate efficiencies:

1. Job Description: Machine Learning algorithms can be trained to detect any bias in the language used in a Job Description. An example is – “We are proud of our success, and boast an impressive record” – which is skewed towards masculine words and deter women from applying for the position

2. Resume Bias: It has been proven through multiple studies that hiring managers have bias (conscious or unconscious) towards different names, and the perceived gender of the applicant. Machine learning algorithms can reduce bias by anonymizing information leading to bias

3. Shortlisting candidates from the pool of applicants: Most of the advertised positions generate hundreds and in some cases thousands of responses. It is therefore impossible for a team of recruiters to look at each resume with the level of detail required, and often some deserving candidates lose out for no fault of theirs. Machine learning can automate the screening process to generate a pool of suitable candidates as the first cut. Depending on the efficiency of the algorithm, the human component in resume screening can be eliminated altogether. Text Analysis based Natural Language Processing (NLP), Named Entity Recognition (NER), Ontology of technical skillset and Topic Modeling could be utilized to built an efficient screening algorithm that does not suffer from limitations of human efficiencies, ability to screen candidates across domains and soft skill assessment based on resume text.

4. Chat and Video based assessment: Instead of having all screened candidates go through the physical interview process, machine learning based conversational chatbots and video based assessment of candidates communication skills, confidence, body gestures etc. could be used to carry out a preliminary assessment of the candidate. Recruiters should deploy a solution in some form as machine learning & AI algorithms are only going to improve with time, and these resources could provide valuable training data for a solution deployment in the future

5. Candidate Matching: Once the assessment is done, a recruiter can generate the list of top matching applicants and proceed with the final recruitment process involving in-person assessments. Machine Learning algorithm for matching best applicants against the job order can be utilized. This has a potential to reduce time assessing the candidates and make decision making faster.

Use of machine learning & AI algorithms is generating efficiencies in the process and helps recruiters find the right candidate for the right job in lesser time and at a lower cost. As with any disruptive solution, recruiters may be skeptical of these technologies that promises to make their jobs easier. Recruiters need to be convinced that an application that automates one of their work tasks will do as good a job as they can. This is as much a change management challenge as a technical challenge to train and develop the right machine learning algorithm(s)

Digital Analytics

Next Best Action – right offer to the right customer at every touchpoint

The financial services marketplace was selling multiple products – credit cards, vehicle loan, personal loan, insurance from multiple financial services providers. They wanted a customer centric focus rather than a product centric view. This meant developing a solution that would allow them to offer products to each customer based on financial value, propensity to accept an offer and business priority, rather than single products being marketed through an expensive outbound campaign.

Optimized offers to customers, through the right channels

Using Next Best Action approach, the client was able to integrate multi-channel interaction with the customers to make offers valuable to both – the customers & to them.

Net uplift of ~18% on outbound campaigns (for the pilot)

Cost reduction of ~40% on outbound campaign cost

Consistent and consolidated communication across multiple channels

Making the customer as the central focal point optimized the campaign and offer strategies, generating better experience for the customers and a higher lifetime value

Customer focus instead of product focus

The client was using propensity modeling based cross sell approach for outbound campaigns. They were able to generate the usual benefits of cross sell i.e. higher uplift and lower spends. They were also able to identify factors that were driving subscription to a specific product. Cross selling was driving undesirable behaviour from a significant number of customers.

The client was looking for something more – a solution that would allow them to market the right product to each customer and integrate the multi-channel communication that the customers were having with them. We proposed a real-time decision engine to find the right product and integrated communication across channels – to make the right offer. With this approach, offers are now only made to customers when it adds value.

Manage multi-channel customer interactions

Next Best Action (NBA) is focused towards relevance and timeliness of push campaigns.

If done effectively, NBA implementation has the potential to boost multiple KPIs, including:

a. A better informed and updated communication stream

b. Making more relevant offers based on customer needs

c. Improved response rates and better campaign RoI

d. Marketing all products, and having a best offer for each customer

NBA drives smart, cost effective and value based reach out through the right communication channels.

Digital Analytics

Purchase Experience Scorecard – to boost customer experience and retention

Purchase experience starts when a (potential) customer hears about a brand. In the case of our client, it starts with product search, discovery, purchase and finally the delivery experience. Our ecommerce client was focused on providing customer delight from end-to-end. They were not satisfied with NPS or CSAT scores, as they had extremely low response rates, and more often than not biased. They wanted a solution that would score for each transaction – even from same customer

Know about each purchase experience. Not just each customer.

Good purchase experience is the impression that a customer is left with on an ecommerce platform. With the implementation of the scorecard, our client was able to:

Increase 1-month repeat purchase by ~14%

Improve 6-month repeat purchase by ~3%

21% reduction is service requests and 12% lower returns

The scorecard allowed the client to proactively monitor customer satisfaction and reach out to customers whose score went below a threshold

Beyond qualitative measures to score each purchase

The client was using Net Promoter Score (NPS) and Customer Satisfaction Scores (CSAT) to access customer satisfaction on their platform. They used to send out survey questionnaires to the purchasers as an automated communication. However, the client was not very happy with these measures on two accounts – the low response rates did not give them visibility on majority of purchases and the results were mostly biased towards either the highest or the lowest score.
They wanted a measure that would have a better coverage and not biased towards people who respond. Also, their inclination was towards a measure that was quantitative, cover all aspects of the purchase experience and provide them with a score that would reflect how the satisfaction score went up or down with every purchase.

More than a performance measurement metric

The scorecard was designed to measure multiple aspects of a purchase including ease of product search, time spent in finding the right product, purchase consideration time, ease of payment and finally, the delivery experience. With this, the client was able to:

a. Identity the experience at a transaction level, and monitor the drivers of the score

b. Identify customers whose score was below a certain threshold – and reach out to them proactively

c. Identify product discovery and delivery issues – and address them

The client is now able to score each transaction based on a feature set that is associated with the browsing and other behavioural data.

Digital Analytics

A/B Testing for digital conversion optimization

Every site visitor is a potential customer. Visits can help digital commerce businesses acquire new customers or engage existing ones. All businesses want visitors to take actions that can be categorized as a conversion (Here, conversion is defined as a desired outcome, and not necessarily a purchase made on the website/app). The better optimized the conversion funnel, the higher the chance for visitors to convert. And one of the most effective ways to optimize conversion funnel is through A/B Testing.

What is A/B Testing

A/B Testing is arguably the simplest, and most effective method to drive better conversion of site visitors. It is worthwhile to note that A/B Testing scope is limited by one’s imagination, and can be utilized for multiple other business problems e.g. Does 10% discount on a product drive better repeat customer volume or 20% cashback?

A/B Testing is a method of testing two variations of the same outcome against randomly selected user segments at the same time to compare which variant drives more conversion. As is obvious, the variation that drives more conversions is the preferred/better option.

Why A/B Test?

With A/B Testing, a digital commerce platform can discover answers to:

a. Visitor pain points: Visitors on a digital commerce website come for specific purpose. It may be to buy a product, compare products with another platform, know more about the product or to know more about the digital platform. Visitors might find difficulty in navigating a page, search results based on keywords entered or finding customer reviews. Not being able to achieve the purpose of the visit will lead to a bad user experience and a potentially dissatisfied visitor. Browsing data, heatmap analysis and A/B test results can help a digital platform identify the pain points and consequently address them

b. Optimize conversions: Every business is aware of how hard it is to bring a potential customer to their platform. A significant part of all the marketing effort is to acquire new customers. And once a prospect has been brought to the platform – businesses would want to convert most of these visitors. A/B Testing allows platforms to test the conversion rates and optimize the conversion propensity – thereby improving return on investment and better conversion funnel

c. Promotion Effectiveness: Every digital platform is jostling for buyer attention, and one way to drive short-term sales is through promotions. It is however more important to know which promotions work, and are able to drive conversions. A/B Testing can help digital platform optimize promotions at multiple levels – SKU, brand, category level. With the abundance of ‘% off’ tags and 24*7 running offers, it is very important to get the promotion effectiveness right – and A/B Testing can be the panacea

d. Webpage/App optimization: Digital platforms may not be achieving the desired conversions or the KPIs e.g. bounce rate is high, low session time. A/B Test enables platforms to make minor changes (one at a time) on the webpage and understand sections/layouts driving better outcomes. This helps in two ways:

1. Enables faster testing of hypotheses – and make minimal modifications on the existing website/app

2. Lets platform owners draw data driven evidence of the issues with current layout if the website redesign is desired

3. If a new feature is being planned, A/B Test allows to test the best way to introduce the feature – by testing different variations of the webpage.

Making changes based on A/B Test results can make the outcome of a change certain through statistically significant improvements.

We hope you have a better understanding of A/B Testing and the benefits that can be derived through them. In the next part of the blog, we will discuss how to perform A/B Tests and different types of A/B Tests.