Text Analytics

The right chatbot for your business

To achieve success with chatbots, businesses need to know what they are looking for and be aware of the options available to them. Given the AI hype, the ideal chatbot should be AI powered, able to converse with the users in free-flowing text format. But do you even need a complex AI bot for your business?

It is much better to introduce a simpler bot first, and evolve the bot from there. The bots can be classified based on the underlying conversation engine:

1. FAQ chatbots: The simplest form of chatbots. The backend is fed with standard questions that a user can ask, and the responses to those questions. These bots do not store conversation history and work on the principals of answer and forget i.e. it will not have the context of the conversation if the user asks a second, related question. The bot is hardcoded, uses no AI and can be deployed very quickly. This type of bot is ideal for answering frequent inquiries

2. Menu based chatbots: This is another simple form of a chatbot where you have defined a scenario tree and have built in responses within the bot, much like an automated IVR. The bot does not use any form of AI and simple to build. This type of bot is ideal for tasks that are straightforward, but involve multiple steps

3. Conversational Chatbots: These are most advanced bot, and are also referred to as virtual assistants. They are designed to be interactive, handle open ended and free flowing inputs from the users. If built right, these bots can learn the context and intent behind a query, and have the potential to minimize the need for human customer support staff. These bots use Machine Learning & AI algorithms at the backend, learn from the conversation data and improve query resolution success rate over time.

4. Voice based Chatbots: Voice based bots are an extension of conversational bot, utilizing the most natural form of conversations i.e. voice. The progress of voice based bot is driven by the progress of speech recognition and linguistic technologies within AI, which is an ongoing research area. The success also depends on the language of the geographic area a business operates in

No matter the industry you operate in, you could deploy a chatbot that can automate some part of your business. It is however important to know your requirement, and assess the return on investment (RoI) that you could generate from the solution. The advanced AI based bots take more time to develop, as well as require significant investment.

Benefits of deploying chatbots:

1. Reduce customer wait time

2. 24×7 availability

3. Standardized, quality controlled engagement with customers

4. No limitations on query volumes and highly scalable

5. Improved return on investment with reduced human resource training, hiring and retention

At Xtage Labs, we have developed a set of guidelines to help you choose the right chatbot solution:

1. Identify your business requirements

2. Build the bot across your simplest use cases

3. Measure the key metrics like customer satisfaction, resolution rate etc.

4. Measure the bot return on investment (RoI) and assess if an advanced conversational chatbot or voice based chatbot makes sense

Building the first chatbot for your business and testing how your customers respond does not require a huge investment or time.

Chatbots are here to stay, and an early start can provide you with the competitive advantage. It is however important to understand that a chatbot is like your website or a mobile app – and you will need to keep enhancing the customer experience through the chatbot for continued success.

Text Analytics

Automated matching of candidates to job orders using machine learning

The US based recruitment firm was struggling with low conversion rates on the submitted resumes. They had already identified a few reasons – limited competence of the recruiter to assess candidates on different skills including emerging tech, the slow process of manually skimming through thousand of resumes and difficulty of recruiters to identify and link different technologies in the job order. They were looking for an AI assisted solution that could address all these.

Better outcomes with lesser resources

The artificial intelligence (AI) algorithm is able to standardize the process, reducing the time to identify the right matches and enable recruiters to focus on the recruitment process itself.

The turn around time (TAT) for candidate matching process reduced by 90%

Accuracy on technical roles as high as 80%

Processed those jobs which were earlier set aside due to resource constraints

The solution was deployed as a platform to extract job orders from client links and then use the candidate database to find the matching candidates.

Manual and broken recruitment process

The recruitment process was very manual and broke. As a job order was published on a client site, an automated email would be sent to the registered mail. The person may or may not look at the job order immediately. Once the job order was received, the task would be assigned to a relevant team. Multiple recruiters would then work on the job order, after assessment of the job order details. This would often lead to shortlisting the same candidates.

The recruiters are not equipped to understand the job order requirements well. They are not conversant in different technologies to connect the dots and identify similar skills or progression of skills. Additionally, the technical details in the resumes is lost on recruiters. All these lead to an efficient process where recruiters would recommend candidates that are not fit for a role.

Machine Learning driven speed and accuracy

Artificial Intelligence based job order to candidate matching offers advantages of turn around time and better match results.

We used natural language based skill extraction from job orders, along with job requirements like location, travel required, domain expertise required.

The resumes were processed to extract skills like number of years on a specific skill or a role, the technology and domain expertise and other relevant details.

Finally, a scoring algorithm was developed to match job orders against all the candidate resumes to find the candidates best matching a job order.

A web based interface enabled recruiters to focus on prospects and assess soft skills and willingness to join.

Text Analytics

An insurer uses text analytics based automated categorization of claims

The insurer suffers from reputation of slow processing of claims. The issue is with manual steps that take time to process manually. They have identified one of the problems with the claims process – initial segregation of claims into specific categories and assigning them to the right team through a technology driven solution. They also want to explore how text analytics based processing of claims data can also help them with targeted investigations and better claims management.

Faster processing of claims

The client wanted to use text analytics based solution to identify the claim type and assign it to the right team automatically at the time of claim registration.

Greater than 90% accuracy in categorising claims

72% reduction in human error in assigning claims to the right team

Reduction of 80% time in assigning claims

The text analytics based automation of claims registration has reduced the dependence on human resources and improved operational efficiencies.

Unstructured data challenges

Insurance as a business line utilizes the services of a number of data experts – underwriters, actuaries etc. These experts have a good grasp of advanced analytics based solution design. However, as an industry, Insurance companies hardly make use of machine learning and artificial intelligence based solutions using the unstructured data sources that the insurers generate on a daily basis. There is a general inertia to keep doing what has been working.

The key challenge is the realisation of the value of unstructured data and the solutions they can enable. Claims processing is one such solution that requires extracting the information from unstructured text data to reduce the time taken to assign claims after registering them. This machine learning driven solution approach also has a potential to generate benefits in flagging claims and faster settlements.

A technology leap for the Insurer

The insurance client generates a lot of unstructured text data in the claims registration process. They wanted to utilize these data to bring operational efficiencies by machine learning based solution for claims segregation and assignment to the right team.

The added benefits of this solution are:

a. Reducing the claims settlement time – improving customer satisfaction and operational efficiency

b. Flagging of dubious claims and targeted investigations

c. Reduction in human errors in claims registration process, improving the resolution time

The machine learning based solution has brought multiple benefits to the insurer in the claims process.

Text Analytics

Processing of financial statements to identify debt & categorise expenses

The fintech startup client is looking to use text analytics based solution to solve one of their customer data processing issues. As a feature of the platform, they ask loan applicants to upload previous three months of bank statements and credit card statements. The operations team then manually looks at these statements for any running loans and the expense heads. The client needs an automated statement processing solution to extract running loan info and to segment expenses.

Technology driven efficiency

The client was looking for a text analysis based solution that could scale and reduce the time of manual processing of bank and credit card statements with a high degree of accuracy.

Reduced the turn around time (TAT) for document processing to 3 minutes

Applicants expenses report and product reco generated in less than 5 minutes

Data automatically pre-processed and mapped to customer data platform

The solution has reduced the processing time of documents, leading to speedier approval or rejection decisions. A list of recommended products also generated using expenses analysis.

Need for automated, reliable solution.

The fintech client is very open to data-driven solutions and creating efficient processes. Realising the manual effort required to process the documents, they understood that a technology driven solution is needed to scale up their operations and ensure fast loan decisions are being made. The automated solution should also aid them in collecting data automatically from the statements, ensuring good and consistent data is being collected.

One of the challenges in creating the automated solution was the lack of standardization in bank statements. Identification of whether it is a bank account statement or a credit card statement was another challenge. A product recommendations algorithm was desired, but because the fintech was a new player in the market, they did not have enough data to create a product recommendation engine.

Improved data quality for data-driven solutions

The client commissioned the engagement with three objectives in place. The first objective was to process the statement files and extract information on loans and segment the customer expenses.

The second objective was to create a standardized, consistent data collection system using named entity recognition (NER) algorithms in text analytics. These datapoints would then be used as inputs to develop a scorecard for loan approval.

The final objective was to segment customers based on expense categories and amount. This information would then be mapped onto product attributes to recommend products based on item-user collaborative filtering algorithm.

Developing the right ‘cold start’ strategy was the challenge owing to limited data on loan performance.

Text Analytics

Keyword trend prediction on social, news, eCommerce & search

The US based media agency aimed to bring the concept of high frequency share trading to the keyword bidding process. They wanted to identify the trending keywords based on user posts on social platforms, breaking news from news websites, products with high search volumes based on popular e-commerce platforms and web search trends on search engines. They would then identify and bid for keywords that are most likely to generate higher eyeballs for their client’s digital ads.

Real-time holds the key

The media agency wanted a (near) real-time platform to process the enormous amount of data and generate keyword association and topic modeling to present the trend results.

Predicted views in the next three hours using historical volumes

Able to predict eight out of twenty trending topics on twitter on an average

Developed a web based platform to combine prediction engine and reporting

The solution generated predictions with high degree of confidence between the predicted trends at T+1, T+2 and T+3 and historical chatter volume till time T.

High volume, high velocity challenge

The media client had conceptualised an innovative solution for serving it’s clients more effectively. The idea was to ride on the wave of popular keywords and place relevant ads so that the number of views are maximized. The scope of the data sources – social platforms, news websites, e-commerce platforms and web search was holistic, and covered almost everything to capture trends. The solution was conceived as a web based platform for ease of access.

There were multiple challenges in turning the idea to reality. The first challenge was the huge amount of data that was required to be processed in real-time. Another challenge was to process this data with text analytics driven algorithms that could process such high volume of data and generate the keywords, perform topic modeling and group similar keywords together – in (near) real-time.

Springboard for greater business outcomes

We designed a solution that could process more than 6GB of data per minute and extract the trend and topic insights with a lag of less than 5 minutes.

The solution was integrated on a platform, and the solution:

a. Predicted twitter trends at State level, capturing eight out of twenty keywords correctly, on an average

b. Twitter trends were captured at least 90 minutes before appearing in ‘Twitter Trends’

c. An accuracy of ~60% for ten defined categories of keywords

The media agency is now working on integrating the volume and bid price for digital ads.

Text Analytics

Data Analysis Chatbot Speeds Results for Pharma Co

Like many companies,this UK-based pharma organization had already invested in data analytics. But something was missing. They had the data, the people, and the tools … but no way to get information to users at speed. They needed something that would enable business users to get instant answers to their queries without involving the data analysis team.

Instant Data, Always Available

By integrating a Natural Language Generation (NLG) powered chatbot with their existing Some Image infrastructure, we enabled our client to:

Provide instant answers to users’ natural language questions

Empower business users to get their own information, when and as needed

Scale up their BI investments

This chatbot transformed how company data was consumed and utilized. And it was set up in just 15 weeks

Too Many Inputs, Not Enough Insights

Some Image Our client’s situation will be familiar to businesses that Have implemented data analysis but haven’t gotten all the benefits yet. They had multiple data sources, internal and external. They had Tableau and other data visualization tools. They had several dashboards for their business users. And they had a team of data analysts to help access specific insights. But there was still a bottleneck. As their system grew, so did the number of requests pouring into the data analysis team’s inbox. What the company needed now was clear: an easy, automated way to answer basic analysis questions and guide users to the appropriate dashboard. It would have to be simple enough for non-technical users, yet powerful enough to provide real-time insights. So that’s what they got.

A Chatbot That’s More Than A “Data Telle

Chatbots that simply find and relay data are becoming commonplace. But our clients needed more than just a “data teller” – a chatbot that merely passes on raw data. Our AI-enhanced chatbot assistant, ASK NAVIK, uses Natural Language Processing to understand users’ spoken queries ( e.g. “How did brand A do in the Central Europe market last year?”) It automatically finds and presents the information in a natural format (e.g. “Brand A sales grew 2% in the Central Europe market in 2019”). Now, there’s no waiting for the data analysis team to answer routine queries; they can spend their time more effectively. Users get the needed results in seconds, not days. And different types of data – such as sales, activity, and shipment information – are available to users whenever and wherever they need it!

Text Analytics

Better candidate to job order matching using AI driven algorithms

Recruitment industry is primed for disruption due to expensive and inefficient processes towards recruitment. Early adopters are already reaping the benefits of machine learning and AI algorithms as they develop, and these developments are bound to change the recruitment industry as we know it.

As with any process that needs to make use of machine learning and AI, the availability of good quality data is the first step. Having latest resumes, correct contact information – email id and phone number, recruitment process data – candidates that were shortlisted (and rejected), called for interviews and final decision makes the machine learning algorithms achieve performance levels better than humans in some cases. It is important to note that these machine learning driven algorithms have a definitive impact on the hiring process, and organizations have started collecting the right data to be able to deploy these solutions.

Here, we discuss each stage of the recruitment process and how can a machine learning algorithm help companies generate efficiencies:

1. Job Description: Machine Learning algorithms can be trained to detect any bias in the language used in a Job Description. An example is – “We are proud of our success, and boast an impressive record” – which is skewed towards masculine words and deter women from applying for the position

2. Resume Bias: It has been proven through multiple studies that hiring managers have bias (conscious or unconscious) towards different names, and the perceived gender of the applicant. Machine learning algorithms can reduce bias by anonymizing information leading to bias

3. Shortlisting candidates from the pool of applicants: Most of the advertised positions generate hundreds and in some cases thousands of responses. It is therefore impossible for a team of recruiters to look at each resume with the level of detail required, and often some deserving candidates lose out for no fault of theirs. Machine learning can automate the screening process to generate a pool of suitable candidates as the first cut. Depending on the efficiency of the algorithm, the human component in resume screening can be eliminated altogether. Text Analysis based Natural Language Processing (NLP), Named Entity Recognition (NER), Ontology of technical skillset and Topic Modeling could be utilized to built an efficient screening algorithm that does not suffer from limitations of human efficiencies, ability to screen candidates across domains and soft skill assessment based on resume text.

4. Chat and Video based assessment: Instead of having all screened candidates go through the physical interview process, machine learning based conversational chatbots and video based assessment of candidates communication skills, confidence, body gestures etc. could be used to carry out a preliminary assessment of the candidate. Recruiters should deploy a solution in some form as machine learning & AI algorithms are only going to improve with time, and these resources could provide valuable training data for a solution deployment in the future

5. Candidate Matching: Once the assessment is done, a recruiter can generate the list of top matching applicants and proceed with the final recruitment process involving in-person assessments. Machine Learning algorithm for matching best applicants against the job order can be utilized. This has a potential to reduce time assessing the candidates and make decision making faster.

Use of machine learning & AI algorithms is generating efficiencies in the process and helps recruiters find the right candidate for the right job in lesser time and at a lower cost. As with any disruptive solution, recruiters may be skeptical of these technologies that promises to make their jobs easier. Recruiters need to be convinced that an application that automates one of their work tasks will do as good a job as they can. This is as much a change management challenge as a technical challenge to train and develop the right machine learning algorithm(s)

Text Analytics

Mining relation between ratings & reviews for an eCommerce store

The ecommerce platform was amassing reviews but was not effectively using the information to know what the customers were saying. They wanted to use the information to understand the customers and identify problems on their platform. Going a step further, they wanted to understand the correlation between star ratings and the review content – to understand how customers were using the star ratings and generate actionable insights from the review data.

Actionable insights from reviews. At scale.

We developed a daily report that would collect reviews provided in the past 24 hours and generate insights for the strategy team

About 68% of 3 star ratings are associated with negative reviews

40% of 1 & 2 star ratings were negative feedback on delivery time

18% of 5 star rating were appreciative of the discounted offers on products

With the study, the ecommerce client is able to generate instant, actionable insights and intervene negative reviews before they can snowball into a brand crisis.

Unstructured text reviews. No standardized insights

The ecommerce client was using star rating to look at customer feedback. They were ignoring the information hidden in 4 and 5 star ratings, and looking at 1 and 2 star ratings, manually. Their assumption was that almost all negative reviews were associated with lowest 2 rating scale. They were also not making use of the reviews associated with promotions, delivery time and attributes.

Their biggest issue was lack of clarity on the insights that text analysis could provide, and if such a solution can be automated and is scalable. They had deployed a team to analyse the reviews manually, and were aware of the value the reviews provided. We proposed a web based platform utilizing natural language processing for automated extraction of insights from review data on a daily basis, beyond just sentiment of reviews.

Not just customer feedback. Measure of brand attributes

The client was initially looking at customer reviews to identify and address the negative reviews being received. They lacked a holistic analysis of review data to generate insights about their brand.

With our integrated approach of text analysis based insights from reviews, and correlating the star rating with review text, we are able to identify that:

a. A high proportion of 3 star ratings were negative

b. Every 2 in 5 reviews were on client brand attributes like value for money, trust and reliability

c. Operational issues like delivery and return were associated with certain geography & delivery clusters

The client is able to analyse reviews within a day now and pass on to the specific teams for actions

Text Analytics

AI driven marketing spend plan using performance data

The client is one of the Top 5 media agencies responsible for media buying and planning for their clients. Developing the media plan process was slow, manual and repetitive. Every time a new request came in, they would aggregate data from multiple internal sources, build the plan and share it with the client liaison. The whole process was notoriously slow. They were looking for a solution that could minimize their dependence on the data team and generate plans instantly.

Marketing spend plans delivered in minutes

The client wanted to serve their clients better, taking all the requests and necessary details through a conversational interface and minimize the time to serve

Average session time of 7 minutes on the bot

Average response time of 38 seconds to generate a marketing plan

Turn around Time (TAT) for generating marketing plans reduced by >80%

The conversational bot made the marketing planning process objective, automated and fast. The team can now have a greater focus on the strategy and campaign performance

Manual, slow and custom plans

The media agency had a defined process to generate media plans. Every time a new request from a client’s planning team came in, the entire process of registering the request, getting the right data and plan customization was triggered. Not only being slow, it created issues with manpower management, multiple back and forth with the clients and less than optimal turn around time for the customers.

Their biggest issue with creating an automated platform was the multiple versions of data stored across systems. Serving multiple clients through same systems was slowing down the data pull requests and impacting database performance. All these was leading to a manual, inefficient and utterly complex planning process. They were looking for a solution that was usable by non-technical users, allowing them to focus on strategy and performance.

Machine learning based marketing spend planner

The conversational chatbot automated the queries that were being sent to multiple databases – reducing variability with better optimization and faster results. The media agency was able to differentiate itself, with faster turn around time with natural language processing (NLP) based custom marketing plans.

The media agency is now able to:

a. Generate marketing plans within minutes leading to higher focus on strategy and execution

b. Standardize and create efficient process for the marketing planning process

We built a solution that the experience of interacting with a human media planner, blending in the benefits of a NLP driven conversational chatbot