Image Analytics

Object recognition in social posts to identify visitor activities

The tourism board was looking to invest in revised positioning and marketing campaign to attract more tourists. They wanted to understand the popular spots in the city and how tourists liked to spend their time while visiting the city. They wanted to look at the social media posts of visitors to get a better picture of tourist attractions, their activities across Facebook, Twitter, Instagram and Pinterest using an automated solution.

Classifying activities and places using image analytics

The board used survey based understanding of popular activities and spots. They were now looking at a more scalable solution that could provide them better insights.

Distinctiveness of tourist spots and activities mapping

Brand imaging by identifying uniqueness and personality of the city

Identifying the most popular spots and activities using image clustering

The study provided promising results in studying different tourism hotspots, the activities and the unique proposition of the city for tourism.

Non trivial, unclear requirements

The tourism board was not very clear and confident on the quality of results that an image analytics based solution could provide them. Looking at social media posts manually, they agreed that the information was useful if they could be aggregated. Another concern for them was the ability of the solution to provide them details at the granularity they were seeking.

One of the key challenges were skewed data with high number of images for 3 tourism hotspots. Other places did not seem to appear a lot in the shared images. And tourists would generally share images with popular activities only. Social media provides us with an amazing data source, and we are finding ways to utilize and understand the full potential. Our team proposed an image data-driven framework to help them answer all their questions using social data.

Persona of the city through social images

Deep CNN (D-CNN) allows image classification, object detection in the images and scene recognition in the image.

Compared to other computer vision algorithms, DCNN extracts high-level cognitive information from images, including visual concepts, therefore allowing better capture of cultural and historical styles.

One of the key challenges in the study was collecting enough images from distinct data sources. Social posts for a tourism location are generally biased towards the more popular places. It was therefore prudent to include multiple data sources, namely Facebook, Twitter, Instagram and Pinterest.

We analysed 18 spots and have been able to map out 12 activities. We were able to uncover the cultural, historical and religious uniqueness of the city.

Image Analytics

Automated meter reading (AMR) to drive efficiency

The utility service provider was looking at multiple solutions to drive efficiencies in their operational processes. They turned towards image analytics to assess if an automated meter reading solution would help them reduce the human resource requirement, increase accuracy and efficiency and solve their problem of unavailable customers (locked houses) and help them generate bills on time – improving the efficiency of the billing process and push customer satisfaction.

Machine learning based automation improves efficiency

The utility provider was looking at image analytics based automated meter reading (AMR) solution that could solve a lot of problems

An accuracy of 84% on untrained sample of meter images

Improved reading coverage of 4% in the pilot area

Improved payment compliance of 2.7% in the pilot area

One of the key challenges that emerged out of the pilot was no reading in a significant number of cases due to poor lighting in the meter location and unreachable meter locations.

Inefficient, human resource driven process.

As a regular practice, the analog meters are used to collect data for the energy consumed. The meters could have a number dial or more recently, digital displays are used. The utility provider’s employee visits each household with and notes down the reading at the end of every billing cycle. This process suffers from wastage of human labour, human error, manual process of reading the paper based notes every month to generate invoices.

This manual process is inefficient, time taking and cumbersome. Another big issue with the process is that it is not possible to take readings if no one is at home. This leads to delays in generating invoices, and in some cases clubbing bills for multiple months. The aim of the pilot was to showcase the current capabilities of machine learning based automated meter reading solution.

Latent benefits of Automated Meter Reading

Image analytics based automated meter reading solution allows customers to click the image of the meter through a web application. This process provides following benefits to the utility provider:

a. An automated solution generating efficiencies in the number of meters being read in each billing cycle

b. Enables timely and efficient generation of invoices, leading to a higher payment compliance

c. Freeing up human resources for other tasks and reducing human dependencies

The pilot though highlighted one of the key limitations of a machine learning based automated meter reading process. The best accuracy achieved was 84%, which was significantly lower than the manual process, which, despite all inefficiencies had an avg. error rate of ~2%.

Image Analytics

Clustering of digital ads using automated tagging of images & videos

The media client wanted to create a generic solution to categorize all creatives used by different businesses. This would help them understand the brand positioning, target group and opportunities for their clients and competitors. OCR like capability was sought to extract text from the creatives. They were looking at a technology driven solution due to the explosion of digital marketing. It had become impossible for them to process digital ads manually.

Right tags for better results

To explain the overall messaging of a creative, a creative tagging system is very powerful, scalable and effective. We used attention representation to reflect the human visual system.

Reduced human effort of tagging creatives by 80%

Improved the number of creatives processed/day by 4x (further scalable)

Improved tagging with better description on qualitative validation of output

The number of parameters needed for the tagging algorithm is small with much lower computational complexity.

Lack of scalability & standardization

Display ads are required to be categorized for brand safety and sensitivity. Advertisers are required to mark their ads into several categories such as suggestive, violent or deceptive. As advertisers do not always have a clear understanding of these categories, creatives are manually labelled that can reduce the effectiveness of the ad and reach. Additionally, manual labelling is limited in terms of taxonomy.

A major issue with digital ads is multiple resolution and high variation of fonts, colours and layouts. Texts are integrated into the creative and any automated tagging exercise requires extracting both – the textual and the visual attributes from an image. We proposed using a recurrent neural network (RNN) based approach with attention representation to generate natural text tags.

Improved results with minimal human involvement

The solution we deployed was able to provide a more textual description of creatives, improving the tagging accuracy.

When stemming is applied on captions generated with the algorithm, the distinctiveness is better expressed as they are mapped to the same word even if the tense and form are different. The solution provided the media agency with following benefits:

a. Improve the efficiency of manual categorization by suggesting a list of possible tags from which editors can choose the best categories

b. Use the solution to backfill the categories of ads which classification was not completed

The features from the creatives is used as a channel of attributes in matching algorithms for ad-selection.

Image Analytics

Advanced Analytics with Satellite Images

Not so long ago, satellite images were available only to the government agencies and their analysts. Now, there are known sources for free and open source access to full satellite data. We also see a lot of private enterprises having interest in satellites for the following reasons:

1. Satellite & launch costs are decreasing

The infographic below shows the estimated cost comparisons in the context of recent SpaceX launch

2. Small launches are relatively much cheaper

3. The huge data infrastructure and bandwidth costs have gone down

4. The pervasion of image analytics based AI, reducing the entry barrier for data scientists & researchers

The above trends mean we have started seeing some out of the box use of satellite images to generate insights for businesses.

We list out some use cases for analytics/insights using satellite images:

a. Environmental Applications

1. Monitoring forest cover: Longitudinal data for a forest or reserved forest allows tracking of increase or decrease in forest cover, effects of urbanization and any illegal deforestation being carried out. It allows governments and green activists to raise awareness about issues that affect all of us

Image source: Earth Observatory, NASA

2. Water Scarcity Monitoring: Water is the most valuable natural resource. Less than 1% of water is available for drinking. With satellite images, we can now understand the processes driving the water cycle and the impact urbanization and climate change are having on the availability (rather, dwindling) of freshwater water sources across the globe.

Image source:

b. Economic activity estimations

Economic activity indicators are key to economic planning, investment decisions and knowing the growth prospects of one (economic) zone over the other. Night time illumination data, extracted from satellite imagery can provide access to this information in near real-time, rather than wait for the governments (and provincial agencies) to release those numbers at an aggregated level for a month or quarter, that too with a lag of upto six months.

A custom study for a PE firm clustering areas of economic activity based on NASA nighttime Illumination data

c. Estimate retailers’ sales

Using geospatial data, satellite images for a specific retail store (or a group of retail stores e.g. All Target stores) can be extracted and the traffic (vehicular as well as human) can be estimated using object detection algorithms. The use of satellite images to monitor own store performance as well that of competitor’s hands a competitive advantage to the decision makers who can think out of the box.

Sydney beach image by DigitalGlobe

d. Monitoring Port Activities

Ports and coastlines can be monitored to establish the baseline for the activities and measuring spike/reduction in traffic using satellite images. Given the proportion of trade between countries carried out through the seas, it becomes critical for businesses across sectors and organizations. This data can also be used for re-development of ports and associated facilities.

e. Cultivated area monitoring

Satellite images taken over farms allows businesses to estimate the area under cultivation for different crops. Depending on the business focus, this data can be used to estimate the expected yield for each crop, adverse weather conditions and timely alerts to mitigate risks, demand for seeds, pesticides etc. and supply chain planning. Satellite image analysis can help quantify uncertainty and manage food security more efficiently.

If you have business problem that requires satellite data or the solution can be made more efficient using satellite data as an alternate data source, please drop us a note. We can help you with the right open source to get the images from, recommend the spectral layer to look at and execute the analysis.