Big Data Analytics

Big data is the new buzzword within the data warehousing and business analytics community.

According to TDWI recent report on BIG data, there are 3 Vs of big data – Volume which is multiple terabytes or over petabytes, Variety which is numbers, audio, video, text, streams , weblogs, Social media etc & velocity which is the speed with which it is collected.

Today, enterprises are exploring big data to discover facts they didn’t know before. This is an important task right now, because the recent economic recession forced deep changes into most businesses, especially those that depend on mass consumers. 

Using advanced analytics, businesses can study big data to understand the current state of the business and track customer behavior.

Here are few examples of Big Data to get the idea:

• Twitter produces over 90 million tweets per day
• Wal-Mart is logging one million transactions per hour
• Facebook creates over 30 billion pieces of content every day ranging from web links, news, blogs, photos etc.
• 72 hours of videos are added to Facebook every minute

Big Data Analytics usability - think about the possibilities of real-time location data with regard to promoting coupons or customized offers to consumers who pass by a retailer’s location, Insurance companies can analyze the data collected by electronic toll transponders to accurately determine a driver’s speed, location, and mileage – and adjust insurance rates accordingly.

Because it's early on, big-data technologies are still evolving and haven't yet reached the level of product maturity.

Discovery analytics against big data can be enabled by different types of analytic tools, including those based on SQL queries, data mining, statistical analysis, fact clustering, data visualization, natural language processing, text analytics, artificial intelligence, and so on.

Solutions getting most advantages by Big Data Analytics:

• Customer analytics – Churn, segmentation, Cross Sell & Behavior analytics
• Fraud detection
• Risk analytics
• Advanced data visualization

Today various technology platforms are becoming available for big data analytics – Hadoop-Mapreduce, Teradata, Greenplum, Kognitio.

Hadoop has become more popular amongst all the tools as it is open source with less total cost of ownership & allows combination of any form of data without needing to have any data types or schemas defined.  

With massively parallel processing using MapReduce functionality it gives power to get the results quickly.  It can scale up & out by adding new nodes. This also allowes fail safe mechanism and all time availability.

Big players like Google, Yahoo, Facebook, Linkedin  have already proved the Hadoop usability.

Analytical tools selection made easy

Almost every large organization today is jumping onto the analytics bandwagon.  Given the continued presence of economic pressures and cutthroat competition, all are keen to use analytical tools to maximize competitive advantage. Unfortunately, the field of analytics can be complicated and confusing, with an overabundance of terms to understand and myriad options to select from. Starting with text analytics and predictive analytics, the list goes on to social media analytics, data analytics, mobile analytics and possibly many more in the future. 

So, do you really need all of the analytics tools to get ahead of the pack, or will just one or two suffice? Let’s take a brief look at each of the options in order to decide........

check the post below :

http://searchbusinessintelligence.techtarget.in/news/1280098454/Analytical-tools-selection-made-easy

Cross/Up-Sell & Next Best Offer

 Today global markets are within reach of every organization & there is a lot of pressure in increasing the revenues and market share.  Business leaders demand to know how customers and prospects are most likely to transact with them in the near and long-term.  

The explosive growth of various mediums to reach customers has made marketing an easy but costly affair.

Typical sales & marketing analytics will involve the following:

• Customer Segmentation – Grouping of customers based on demographics data, buying behavior or some other patters which are further used in marketing
• Customer Acquisition – Finding new customers with qualified sales leads
• Cross-Sell / Up-sell – Existing customers present an opportunity to grow revenues through cross-selling and up-sell strategies. Banks can use cross-selling to push credit/debit cards, checking account, investment banking services to savings account holders. Credit Card Company can offer Gold or Platinum cards to Silver cardholders. Use of Social Media Analytics is very useful.
• Channel – Optimize channel performance for sales success by identifying the best leads and channel partners to pursue across channels.
• Churn / Loyalty – Define the right targeting criteria and messaging mix to improve customer retention and loyalty.

Let us look at next best offer in more detail.

Next-best offer is the personalization & optimization of cross/up-sell. It is the use of analytics to identify the products or services your specific customers are most likely to be interested in, for their next purchase.

Here is the typical process for next best offer: 

• Consolidate & create a holistic view of the customers with the vast and growing amounts of transactional, demographic data.
• Apply predictive analytics techniques to customer data & identify trends in purchase behaviors and product affinities.
• Segmentation
• Decision trees
• Neural networks
• Use product affinities/associations to recommend other products or services customers may be interested in.
• It’s very important to close the loop by applying customer responses to future targeting that addresses customer needs.

This coupled with social media analytics of competition offerings or speech analytics of call center data will make cross/up-sell & next best offers more fruitful since they receive critical and indispensable feedback about the issues that matter most to customers, product improvement, service requirements and spending patterns.

Predicting the next product or service, customers are most likely to be interested in, not only improves customer lifetime value but also supports profitable and long-term customer relationships. 

With increasing global competition, optimal customer experience is of prime importance  !!!

Customer churn and Retention

In any business, competitors are always looking to grab your customers, and many customers are on the lookout for a better deal. 

Customer attrition rates range from 7% to 40% annually in various industries. Slowing this customer "churn" rate by as little as 1% can add millions of dollars to any sizable company's bottom line. 

As it is already known customer acquisition is 4 to 5 times more expensive than to retain them, an effective customer retention strategy is crucial to a company's success. 

Marketing departments are traditionally focused on acquiring new customers than retaining existing ones. But after the economic recession, when finding the new customer is especially challenging, customer retention has become a major corporate priority.

While it is the fact that churn will always exist, ensuring that it is managed effectively is a key, to assure profit margins & sustainable business growth.

By predicting which customers are likely to leave, companies can reduce the rate of churn by offering customers new incentives or packages to stay. Apart from that, it gives the best strategy for them in terms of cost and effort by decreasing the total cost of retention and increasing the effectiveness of campaigns.

There are multiple data mining techniques which can be used for churn prediction.

Just to step back, Data mining is a process of extraction and analysis of patterns, relationships and useful information from massive databases. It usually involves four classes of tasks which revolves around classification, clustering, regression and association rules.

Here are the typical steps that are taken to address customer churn & retention:

• Define “Customer churn” as it varies from industry to industry
• Create a single 360-degree customer view
• Collect every Customer touch point data – billing data, transaction records, demographic details, Call center records, Credit history
• Understand the customer behavior
• Map the entire customer journey
• Profiling & segmenting them on various attributes
• Customer value analysis or Life Time Value
• Identify the customers with the highest chances to churn
• Predictive Churn model
• Typical techniques used are Regression, decision trees, survival analysis
• Discover what are major reasons for churn
• Product or Service-related issues
• Demographic constraints
• Better deals from competitors
• Setup targeted retention campaigns for high-value Customers who are likely to leave
• Customized promotions
• Next best offer strategy
• Location-based real-time offers
• Measure the campaign's effectiveness for continuous improvement
• What is the retention rate or how much Churn % has come down

Today social media analytics including Speech analytics is becoming a key aspect to analyze Customer sentiments which helps in finding out the reasons for customer churn. This involves capturing & analyzing unstructured data from customer touch points like customer support call notes, chats, email exchanges & scraping customer comments from Facebook, Twitter, blogs etc.

It is very important to go beyond just predicting customers who are likely to leave & identify the reasons for churn & effectively drive the targeted campaigns to retain those customers.

Predictive analytics software deployment: 4 pitfalls to watch out for

I have posted an article on SearchBusinessIntelligence.in about the 4 early signs to watch out while catching up on analytics initiatives.

you can read it at

http://searchbusinessintelligence.techtarget.in/tip/Predictive-analytics-software-deployment-4-pitfalls-to-watch-out-for

Fraud Detection & Prevention

 

The financial industry is facing the fiercest competition in current time after the economic meltdown. Banks are using all avenues to grow their customer base considering the survival aspect. This has lead to tremendous volume growth in banking accounts applications, credit card applications, and financial transactions.

Obviously, as a consequence, the number of fraudulent applications and transactions is also rapidly growing.

With Digital Transformation new payment channels like prepaid cards, e-payments & now mobile-payments, fresh opportunities for frauds are emerging.

Some of the industry research shows that:

• Credit card frauds losses are over 8 billion USD per year
• Insurance policyholders have to pay a higher premium up to 5%
• Total fraud Losses are estimated at over 30 billion USD per year

Frauds can be classified into various categories as below:

• Credit/Debit/Charge card fraud
• Check fraud
• Internet transaction/wire transfer fraud -
• Insurance or healthcare or warranty claim fraud – overpayments, false claims
• Subscription fraud – use of telecom services with false credentials
• Money laundering
• Identity theft or account takeover

Analytics approaches to detect & prevent Frauds:

• Combine historical fraud data with industry knowledge & external market data
• Create a proof of concept to test the history data to determine fraud cases
• If historical data is not available then anomaly detection or outlier detection is used
• Apply the statistical model for fraud detection
• Models are based on past spending patterns, demographic information
• Further text mining & link analysis for probable associations to find deeper frauds

Benefits:

• Increased number of identification of fraud cases
• Dollar savings from fraud prevention adds to the bottom line
• Protect the customer base from financial loss or identity theft
• Improvement in service helps to differentiate in the highly competitive market

How companies are using it:

• Financial institutions using it to identify frauds in leasing contracts
• Banks are using it to detect credit card, wire transfers, check frauds
• Insurers are using it to detect fraudulent claims to save the losses
• Healthcare provider can optimize the medical loss ratio by detecting claims frauds 

Today with help of Big Data platforms, companies can store all the historical data they have which can help in better fraud detection.

Credit Score Cards

 

Information Technology as an industry has grown up in leaps and bounds. You may not find any organization on the planet which does not have any IT involved.  This has given rise to a lot of jobs supporting the IT functions. Salaries have increased tremendously in IT compared to other business areas. The overall economy had gone up which increased the tendency of people to afford & buy more & more.

This has increased the usage of Credit in everyday life. “Buy now pay later” syndrome became common. Everyone started using the credit cards and also started availing credit or loans for big purchases like home, car etc.

Eventually, this resulted in many people avoiding or defaulting on the payments. This is where applying analytics for assessment of the risk of providing the credit came along and the birth of credit scoring.

Credit Risk is the risk of losing a bank or credit giving company will incur when Customer does not repay the mortgage, unsecured personal loan, auto loan, credit card amount, overdraft etc.

In the early days of lending businesses used to judge borrowers based on 5 Cs:

• The character of the applicant
• The capacity of the applicant to borrow
• Capital as backup
• Collateral as security for credit
• Conditions which were mostly external factors

Then Credit Scoring was introduced by Fair Isaac which is now commonly known as FICO score.

Credit Scoring in simple terms giving some numbers to customers based on certain parameters like age, earnings, accommodation type (owned or rented), expense history & payment history etc.

There are 3 types of Scorecards which are currently used:

Application Scorecard:  This is mainly used in scoring the customer's applications for credit. This tries to

predict the probability that the customer would become "bad". The score given to a customer is usually a three or four digit integer which is finally used to approve or reject the credit application of the customer. This is where you get messages from Banks that you have pre-approved loans or Credit cards.

Behavioral Scorecard: This is mainly used to identify or predict which of the existing customers are likely defaults on the payment so alternative measures can be taken to contact the customers & ensure that payments are received on time.

Collection Scorecards – This is mainly used to arrive at how much loss the company will incur, due to nonpayment from groups of Customers.

How businesses are using Credit Scorecards:

• Banks are using them to separate good borrowers from bad borrowers
• Financial institutions are using it to determine credit limits
• Early detection of high-risk account holders to reduce potential losses
• Improved debt collection
• Insurance companies are using it for the cost of insurance product for a Customer

Today applying analytics to the data to get such insights is of prime importance. 

Social Media Analytics

The Internet has taken the world by storm in the last few decades. Some of the surveys show that it has grown over 2000% in the last 10 years. Imagine how many people globally are online at any point in time. Being social animals, we like to talk about our feelings, good or bad, with our family & friends.   

Today people are using internet for communication more, than any other medium, This has given birth to social media sites like Facebook, Twitter, Orkut, myspace, digg & the list will go on. Imagine how much data is collected on these sites in one day.

So companies have started using this huge unstructured text data in these comments, sentiments of people for their product improvements, for Customer service improvements, for understanding what customers are talking about competitors. This is called Social Media Analytics.

In the recent example of Toyota recalls, they are closely watching these social sites to gauge what Customers are talking about their brand, quality & take necessary measures to correct their actions.

But social media analytics isn’t just about damage control, it can provide precise data to help you better understand your customers and discover new business opportunities.

Here are typical steps in implementing social media analytics.

• Collect the huge amount of unstructured data – comments, blogs, call center notes, twits from social sites
• Using statistical analysis & Natural Language processing (NLP) on texts & words to break up the information into good or bad
• Use categorization, classification & association methods for text processing
• Further, identify the categories on which these good or bad sentiments are applicable from the data
• Brand awareness, trust, loyalty
• Customer service
• Product improvements
• Competitor analysis
• Geographic locations
• Produce the results using visualization tools

Here is how various businesses are using Social Media Analytics:

• Manufacturing – using the warranty data combined with Customer complaints to improve products & reduce warranty costs
• Healthcare – find connections amongst the claims received to flag further frauds
• Banking & Finance – use the social network profiles data to improve credit calculations, identify reasons for Customer churn
• Insurance – claims cost prediction& fraud detection
• Telecom – improve the customer experience for new products introductions
• Hospitality & Travel – listen to guests comments to improve repeat customer rates

Social Media Analytics will continue to gain importance in near future !!

Use of Analytics in Business verticals.

In the last post, we saw how Analytics has become mainstream & how it is different from the Business Intelligence.

Let us see how businesses are using it for competitive advantage. Today businesses are more worried about survival than profitability. The very purpose of any business to exist is to be profitable & sustain it. This can happen only when there are good loyal & profitable Customers attached to the business.

Hence Customer analytics has become very prime importance in the current time and it is valid across all the business segments.

Customer Analytics:

• Customer Lifetime value – group Customers on high, medium, low value & take actions to increase revenue
• Customer Segmentation – the grouping of Customers based on demographics, or profitability, or lifetime value
• Customer Churn/attrition  - predict which Customers are likely to leave you & take suitable actions
• Customer Retention – identify most profitable Customers & then retain them
• Campaign management – selective campaigns based on segmentation or Customer’s likely behavior
• Cross-sell and Up-sell – increase the revenue proposing other products or high-end products

Apart from these, I am mentioning below some of the areas in business verticals, where Analytics is applied for foresight.

Banking & Financial Services Analytics:

• Anti-money laundering – identifying suspicious transactions to alert investigation officers
• Credit scoring – score the customer based on various parameters to arrive at a certain number and if that is above a threshold then approve the credit
• Credit Risk – predicting the risk involved due to nonpayment by borrowers in case of credit cards, loans etc
• Fraud detection & Prevention – predicting suspicious transactions which are likely to be fraud in all the transactions of the card, wire transfers, online transactions etc
• Price Optimization - Debt collection agency can predict the optimal price for the portfolio & forecast the probable recovery from defaulters

Insurance Analytics:

• Claims Fraud detection  - predicting the claims which are likely to be fraudulent
• Policy Lapse prediction – predict which are the policies that are going to lapse before completing the tenure
• Underwriting rate optimization – predicting the best price for the insurance products based on Customer profile
• Agent performance prediction – how agents are going to add revenues to the organization, improve customer satisfaction & retention
• Agent Lifetime Value – how best an agent is going to serve the organization throughout his/her tenure

Healthcare Analytics:

• Healthcare Claims Fraud detection  - predicting the claims which are likely to be fraud
• Financial recovery – predicting the payments from healthcare insurance payer which are overpayments to service providers
• Health plan analytics – allows organizations to compare & predict different benefits & risk options in terms of coverage & costs
• Condition Management – predict which of the people are likely to develop diseases like blood pressure, cholesterol etc

Retail Analytics:

• Discount or Price optimization – predict the optimal prices of discounts & normal prices of merchandizes  for today’s sensitive shopper
• Cross-Sell & Up-Sell – propose other products depending on various factors such as color, fashion, choice, location, earning patter & Customer buying behavior etc.
• Forecasting – based on demands from Customers predict how much stock is required to avoid stock-outs & excess inventory

Manufacturing Analytics:

• Predicting the parts failure – based on the history data predict  which of the mechanical parts are going to fail & when
• Issue detection – predicting the issues before they occur so preventive maintenance can be done on the parts
• Warranty Analytics – identify issues across the production period to reduce warranty costs
• Forecasting – based on demands from Customers predict how much stock is required to avoid stock-outs & excess inventory
• Inventory Optimization - to reduce inventory carrying costs & increase order fulfillment by predicting optimal inventory to be stored across warehouses

Text Analytics:

• Discover & extract meaningful patterns and relationships from the text collection from social media site such as Facebook, Twitter, Linked-in, Blogs, Call center scripts
• Understand Customer sentiments – positive & negative. Used for Product & Customer service improvements. Also for knowing what competition is good or bad at

Analytics is used in every area of life to get better insights into what is going to happen & what we can do so that the best outcome is expected !!!

So what is Business Analytics & its various components

In the first post, I talked about why Analytics is required more than ever now. In this post let us discuss, what is it all about & what are the typical components of Analytics.

Let us start with the definition of Analytics. There are multiple definitions available but as our focus is on Simplified-Analytics, I feel the one below will help you understand better.

Business Analytics is the use of statistical tools & technologies to:

• Find patterns in your data for further analysis e.g. product association
• Find out outliers from the huge data points e.g. fraud detection
• Identify relationships within the key data variables for further prediction e.g. next likely purchase from the Customer
• Provide insights as to what will happen next e.g. which of the Customers are leaving us
• Gain the competitive advantage.

So a more detailed comparison with Business Intelligence will help you understand better.

Business Intelligence	Business Analytics
What does it do?
Reports on what happened in the past or what is happening in now, in current time.	Investigate why it happened & predict what may happen in future.
How is it achieved?
Basic querying and reporting OLAP cubes, slice, and dice, drill-down Interactive display options – Dashboards, Scorecards, Charts, graphs, alerts	Applying statistical and mathematical techniques Identifying relationships between key data variables Reveal hidden patterns in data
What does your business gain?
Dashboards with “how are we doing” information Standard reports and preset KPIs Alert mechanisms when something goes wrong	Response to “what do we do next?” Proactive and planned solutions for unknown circumstances The ability to adapt and respond to changes and challenges

 

Now that you know the difference between BI & BA, let us discuss the typical components in Analytics.

There are 6 major components or categories in any analytics solution.

• Data Mining – Create models by uncovering previously unknown trends and patterns in vast amounts of data e.g. detect insurance claims frauds, Retail Market basket analysis.

         There are various statistical techniques through which data mining is achieved.

• Classification ( when we know on which variables to classify the data e.g. age, demographics)
• Regression
• Clustering ( when we don’t know on which factors to classify data)
• Associations & Sequencing Models
• Text Mining - Discover and extract meaningful patterns and relationships from text collections e.g. understand sentiments of Customers on social media sites like Twitter, Facebook, Blogs, Call center scripts etc. which are used to improve the Product or Customer service or understand how competitors are doing.
• Forecasting – Analyze & forecast processes that take place over the period of time e.g. predict seasonal energy demand using historical trends, Predict how many ice creams cones are required considering demand
• Predictive Analytics - Create, manage and deploy predictive scoring models e.g. Customer churn &  retention, Credit Scoring, predicting failure in shop floor machinery
• Optimization – Use of simulations techniques to identify scenarios which will produce best results e.g. Sale price optimization, identifying optimal Inventory for maximum fulfillment & avoid stock outs
• Visualization - Enhanced exploratory data analysis & output of modeling results with highly interactive statistical graphics

Hope this has helped you get the clarity on what is Analytics.

Next post I will cover how various business verticals are using Analytics.

Why Business Analytics is important for business more than ever NOW !!

As the global political & physical barriers are collapsing, all the global markets are getting opened for businesses, creating a fierce competition within companies to market their products, increase their revenues, grab most of the customers. Organization are not just worried about the profitability but also survival in this tsunami of Global Reach.

All along till now, Organizations were using business intelligence to get the information from the vast amount of data buried into various internal systems. But this is just information & decision makers had the reactive approach to deal with such situations e.g. North region is not meeting the revenue targets - so create some focused program to cover North region & so on.

This is where Business Analytics play !! It's not reporting of past data or what is happening now but giving organizations the forward look at the business.

It can answers questions like :

• how do we get more insights about our customers?
• what if we change the price or service of the products?
• what will be the impact on our customers?
• How do we target our most profitable customers?
• How do we detect frauds?
• which of the customers are likely to leave us in the future?

Business Analytics is an extremely important area going across all the business domains. If you know in advance, which of the customers are likely to leave you, you can take measures to hit only those customers with the right campaigns to retain them. there is no need for machine gun firing but a sniper is required in this case.

By looking at the future, Organizations can take proactive decisions and plan their business for maximum success.

So Business Intelligence can give you "Information" but Business Analytics gives you the "Knowledge" - TO ACT UPON !!!

In the next post, I will talk about What is Business Analytics & its various components.