Saturday 24 December 2011

Big Data Analytics

Big data is the new buzzword within the data warehousing and business analytics community.

According to TDWI recent report on BIG data, there are 3 Vs of big data – Volume which is multiple terabytes or over petabytes, Variety which is numbers, audio, video, text, streams , weblogs, Social media etc & velocity which is the speed with which it is collected.

Today, enterprises are exploring big data to discover facts they didn’t know before. This is an important task right now, because the recent economic recession forced deep changes into most businesses, especially those that depend on mass consumers. 

Using advanced analytics, businesses can study big data to understand the current state of the business and track customer behavior.

Here are few examples of Big Data to get the idea:
  • Twitter produces over 90 million tweets per day
  • Wal-Mart is logging one million transactions per hour
  • Facebook creates over 30 billion pieces of content every day ranging from web links, news, blogs, photos etc.
  • 72 hours of videos are added to Facebook every minute

Big Data Analytics usability - think about the possibilities of real-time location data with regard to promoting coupons or customized offers to consumers who pass by a retailer’s location, Insurance companies can analyze the data collected by electronic toll transponders to accurately determine a driver’s speed, location, and mileage – and adjust insurance rates accordingly.

Because it's early on, big-data technologies are still evolving and haven't yet reached the level of product maturity.

Discovery analytics against big data can be enabled by different types of analytic tools, including those based on SQL queries, data mining, statistical analysis, fact clustering, data visualization, natural language processing, text analytics, artificial intelligence, and so on.

Solutions getting most advantages by Big Data Analytics:

Today various technology platforms are becoming available for big data analytics – Hadoop-Mapreduce, Teradata, Greenplum, Kognitio.

Hadoop has become more popular amongst all the tools as it is open source with less total cost of ownership & allows combination of any form of data without needing to have any data types or schemas defined.  

With massively parallel processing using MapReduce functionality it gives power to get the results quickly.  It can scale up & out by adding new nodes. This also allowes fail safe mechanism and all time availability.

Big players like Google, Yahoo, Facebook, Linkedin  have already proved the Hadoop usability.


  1. It's really an interesting article. Thanks for this note!

  2. Thanks Sandeep for sharing this article. No wonder Big Data is so much in talks all around.

  3. Sandeep,

    Thanks for this article. It was a good read and lots to know on latest about Analytics.


  4. Thanks Sandeep for Sharing this article..

    While going through the article on question cames in my mind as a company uses Hadoop as an extract, transfer and load (ETL) engine but on other hand still other companies, such as T-Mobile have rejected going the route of Hadoop.
    They are happy to stay with Netezza as enabling data to be loaded 50% faster.

    Is it like we are still in a phase to decide that for which big data analysis tools,the company should go for?

    Waiting for your Reply...

    Ishleen Kohli

  5. Thanks Sandeep,

    Glad to stumble upon this blog of yours. It provides a good read definitely.

  6. Hi,

    I'm new to this post and also new to extensive world ofBusiness analytics. One thing i'm still confused with the conventional business analytics and Big data analytics. In broader perspective, i can see we can handle huge data, various type, and with relatively faster than earlier. The question with me is, those who recommend Big Data tech is already been in use of other DW tools earlier and that was working well since beginning. Earlier, it was said that SQL, Tera, SAS, SPSS and goes on..etc. and now big data.
    So i doubt, whether this hype is a marketing strategy?

    Waiting for your answer. !!!!

    Please dont take this question otherwise, because todays market is full of products and the company can adopt any strategy to develop their market.


    Vinit - India

  7. Vinit- all those old EDW still exists but with big data you can combine company internal structured data with external unstructured data.

  8. Having these Big Data Analytics results can allow business decision makers to make more informed decisions regarding their customers preferences regarding different products.

  9. Sandeep,
    Well described and greatly exposed the use case of Big Data and where analytical plays a huge role!

    Murale Narayanan

  10. I love what we've got here, love what you say and how you say it. You make it entertaining and you still manage to keep it smart. Thanks for sharing your knowledge !

  11. Great post to learn more things about big data. Measure your dos and does in social media presents and its help to improve your brand performance for feature. Visit us to know more…


360TotalSecurity WW