18 Big Data tools you need to know!!

In today’s digital transformation, big data has given organization an edge to analyze the customer behavior & hyper-personalize every interaction which results into cross-sell, improved customer experience and obviously more revenues.

The market for Big Data has grown up steadily as more and more enterprises have implemented a data-driven strategy. While Apache Hadoop is the most well-established tool for analyzing big data, there are thousands of big data tools out there. All of them promising to save you time, money and help you uncover never-before-seen business insights.

I have selected few to get you going….

Avro: It was developed by Doug Cutting & used for data serialization for encoding the schema of Hadoop files.

Cassandra: is a distributed and Open Source database. Designed to handle large amounts of distributed data across commodity servers while providing a highly available service. It is a NoSQL solution that was initially developed by Facebook. It is used by many organizations like Netflix, Cisco, Twitter.

Drill: An open source distributed system for performing interactive analysis on large-scale datasets. It is similar to Google’s Dremel, and is managed by Apache.

Elasticsearch: An open source search engine built on Apache Lucene. It is developed on Java, can power extremely fast searches that support your data discovery applications.

Flume: is a framework for populating Hadoop with data from web servers, application servers and mobile devices. It is the plumbing between sources and Hadoop.

HCatalog: is a centralized metadata management and sharing service for Apache Hadoop. It allows for a unified view of all data in Hadoop clusters and allows diverse tools, including Pig and Hive, to process any data elements without needing to know physically where in the cluster the data is stored.

Impala: provides fast, interactive SQL queries directly on your Apache Hadoop data stored in HDFS or HBase using the same metadata, SQL syntax (Hive SQL), ODBC driver and user interface (Hue Beeswax) as Apache Hive. This provides a familiar and unified platform for batch-oriented or real-time queries.

JSON: Many of today’s NoSQL databases store data in the JSON (JavaScript Object Notation) format that’s become popular with Web developers

Kafka: is a distributed publish-subscribe messaging system that offers a solution capable of handling all data flow activity and processing these data on a consumer website. This type of data (page views, searches, and other user actions) are a key ingredient in the current social web.

MongoDB: is a NoSQL database oriented to documents, developed under the open source concept. This comes with full index support and the flexibility to index any attribute and scale horizontally without affecting functionality.

Neo4j: is a graph database & boasts performance improvements of up to 1000x or more when in comparison with relational databases.

Oozie: is a workflow processing system that lets users define a series of jobs written in multiple languages – such as Map Reduce, Pig and Hive. It further intelligently links them to one another. Oozie allows users to specify dependancies.

Pig: is a Hadoop-based language developed by Yahoo. It is relatively easy to learn and is adept at very deep, very long data pipelines.

Storm: is a system of real-time distributed computing, open source and free.  Storm makes it easy to reliably process unstructured data flows in the field of real-time processing. Storm is fault-tolerant and works with nearly all programming languages, though typically Java is used. Descending from the Apache family, Storm is now owned by Twitter.

Tableau: is a data visualization tool with a primary focus on business intelligence. You can create maps, bar charts, scatter plots and more without the need for programming. They recently released a web connector that allows you to connect to a database or API thus giving you the ability to get live data in a visualization.

ZooKeeper: is a service that provides centralized configuration and open code name registration for large distributed systems. 

Everyday many more tools are getting added the big data technology stack and its extremely difficult to cope up with each and every tool. Select few which you can master and continue upgrading your knowledge.

Top 7 Virtual Reality Industry use cases

Today Digital Transformation has entered our life and we have been subconsciously using it in our day to day life. e.g. Smartphones, Smart cars, internet connected devices etc.

Virtual Reality technology has evolved dramatically in the past few years the costs of VR devices has gone down so it is all set to hit mainstream markets soon. While gaming applications like Pokemon Go have attracted most of the attention, there are many other use cases that could have a much larger impact on our lives.

Google Cardboard is a super low-cost headset ($15) to which a compatible, VR enabled mobile phone is attached to deliver the VR experience.

Other commercial product is Oculus Rift gear which has become extremely popular in gaming & business equally.

Here are some great VR use cases:

1.     VR for Tourism: do you want to sit on your couch and climb up the Eiffel tower? Or walk on the glass horse shoe at grand canyon? Wild Within is VR app available for experience of travel through rain forest in Canada. Travelers around the world are able to experience a helicopter flight around New York City or a boat ride around the Statue of Liberty.

2.     VR for Education: Over last decade eLearning had picked up very much. But it could not deliver hands on experience which is now possible with VR technology. Technicians can actually learn the real life examples and do their bit to solve the problems on the shop floor. Medical students can actually perform surgeries allowing them to make mistakes without any impact on actual patients.

3.     VR for Sales: Traditionally automakers have the showroom to show the cars to the customers and explain their features and sometimes a test drive is also possible. But customization of how the interior will look as per their choice was not possible which now can be done via VR.  Audi is experimenting this in London, where customer can configure their Audi with accessories as they want and drive virtually in real time.

4.     VR in Gaming: who does not know the excitement Pokemon Go had created and reached 50 million users in record time of 22 days.  Using AR/VR technology games have changed the life of seniors as well as teens. Game of Thrones has capitalized on VR and gone viral in various countries.

5.     VR in Designing: product designing is tedious task and changes to products based on the competition or customization is time consuming. This is where VR helps designers. They can now create the products easily, configure all the features and test them out. It is more popular in construction of buildings to see how the interior will look like.

6.     VR in Marketing: With Digital Marketing ads are becoming more intrusive. The best marketing campaigns use VR to create successful campaigns as users get completely immersed into the content, and create memorable experiences. Coca Cola created a virtual reality sleigh ride. New York times releases multiple immersive documentaries in their app. Finnair is showing their Airbus 350 via VR to attract more customers.

7.     VR in Sports coaching: The potential for VR in sports in endless. You get all the benefits of real-world interaction, but in a controlled environment. Showing is so much more effective than explaining, and experiencing something first-hand is that much more powerful again. Football, Cricket.

Virtual reality technology holds enormous potential to change the future for a number of fields, from medicine, business, and architecture to manufacturing. We are on the roller coaster ride !!

Internet of (Medical) things in Healthcare

Over the past few decades, we’ve gotten used to the Internet and cannot imagine our lives without it. Millennials and new age kids don’t even know what is life without being online.

With the disruption of Digital Transformation, Internet of Things have added lots of opportunities to business and consumers like us, equally.

IOT means connecting things, extracting data, storing, processing and analyzing in big data platforms and making decisions based on analytics. It helps in predicting certain outcomes thereby helping with taking preventive actions

The popularity of wearables, such as fitness trackers, blood glucose monitors and other connected medical devices, has taken healthcare by storm. Connected devices have become a prevalent phenomenon in the consumer space and have made their way into healthcare

Healthcare is fast adopting IoT & changing rapidly, as it reduces costs, boosts productivity, and improves quality. IoT can also boost patient engagement and satisfaction by allowing patients to spend more time interacting with their doctors.

There are a number of opportunities for the internet of things to make a difference in patients' lives. IoT-enabled devices capture and monitor relevant patient data and allow providers to gain insights without having to bring patients in for visits. Adding sensors to medicines or delivery mechanisms allows doctors to keep accurate track of whether patients are sticking to their treatment plan and avoid patient's readmission.

Patients are using these connected medical products to capture ECG readings, record medication levels, sense fall detection and act as telehealth units.

Diabetes self-management includes all sorts of gadgets and devices, which control glucose levels and remind patients to take their insulin dose. The newest wearables are even capable of delivering insulin on their own, according to health condition indicators. 

Remote patient monitoring is one of the most significant cost-reduction features of IoT in healthcare. Hospitals don’t have to worry about bed availability, and doctors or nurses can keep an eye on their patients remotely. At the same time, patients usually feel more relaxed at home and recover faster.

Smart beds are a convenient solution for patients who have trouble adjusting bed positions on their own. This kind of IoT tool can sense when the patient is trying to move on their own and it reacts by correcting the bed angle or adjusting pressure to make the person more comfortable. Additionally, this frees up nurses, who don’t have to be available all the time and can dedicate extra time to other duties. Many hospitals have already introduced smart beds in their rooms.

At Boston Medical Center, IoT is everyday life:

·       Newborn babies are given wristbands, allowing a wireless network to locate them at any time.

·       They have installed wireless sensors in refrigerators, freezers and laboratories to ensure that blood samples, medications and other materials are kept at the proper temperatures.

·       Hospital has more than 600 infusion pumps which are IoT enabled. BMC staff members can now dispense and change medications automatically through the wireless network, rather than having to physically touch each pump to load it up or make changes.

At Florida Hospital, when patients go in for surgery, they're tagged with real-time location system (RTLS) badges that track their progress through from the pre-op room to the surgical suite to the recovery unit so relatives can track the patients from outside.

Philips GoSafe can be worn as a pendant and it helps to detect and alert falls in elderly people

There are few challenges as well in implementing IoT:

·       Data security & lack of standard security policy

·       Hospital’s internal system integration with IoT data

·       Further changes and improvements in IoT hardware

The Internet of these Medical Things is a game-changer as future will be connected, integrated & secure healthcare industry 

Terminator or Iron Man – What will AI bring in future?

In the age of Digital Transformation, Artificial Intelligence has come a long way from Siri to driverless cars.

If you have used a GPS on Google Maps to navigate in your car, purchased a book recommended to you by  Amazon or watched a movie suggested to you by Netflix, then you have interacted with artificial intelligence.

Artificial Intelligence is the capability of a machine to imitate intelligent human behavior which relies on the processing and comparison of vast amounts of data in volumes with help of big data analytics, no human being could ever absorb.

Stephen Hawking, Elon Musk, Bill Gates have recently expressed concern in the media about the risks posed by AI.

According to them, AI will soon replace all kinds of manual tasks and make humans redundant. This could be true in some sense but still this is a far cry from the current maturity levels of AI, which is still at the stage of figuring out real-world use cases.

Today machines can carry out complex actions but without a mind or thinking for themselves. Smartphones are smart because they are responding to your specific inputs.

The world’s top tech companies are in a race to build the best AI and capture that massive market, which means the technology will get better fast, and come at us at faster speed. IBM is investing billions in its Watson, Apple improving Siri, Amazon is banking on Alexa;  Google, Facebook and Microsoft are devoting their research labs to AI and robotics.

Together, they will swirl into that roaring twister, blowing down the industries and businesses in its path.

Within maybe few years, AI will be better than humans at diagnosing medical images and converting speech to emotions. But it can also be stealing millions of records from a government agency to identify targets vulnerable to extortion.

Soon you’ll be able to contact an AI doctor on your smartphone, talk to it about your symptoms, use your camera to show it anything it wants to see and get a diagnosis that tells you to either take a couple of Tylenols or see a specialist.

In all the fairy tales we have seen so far, good almost always wins over evil.

This is what we have seen in the movies like I, Robot or Avengers: Age of Ultron.  But Will Smith or team of avengers does not know that till end of the story. That’s where we are now: face to face with the demon for the first time, doing everything we can to get through the scary plot alive.

Today many companies are using AI for improving their business:

·         Geico is using Watson based cognitive computing to learn the underwriting guidelines, read the risk submissions, and effectively help underwrite

·         Google Translate applies AI in not only translating words, but in understanding the meaning of sentences to provide a true translation.

·         IBM Watson is the most prominent example of AI based question answering via petabytes of data retrieval that helps in various areas like finance, healthcare & insurance.

As Humans we are programmed from childhood either by nurture or nature to do things the way we do. All the nine emotions we have learned since then are the inseparable part of our lives.

Currently we are in the control of the planet because we are smartest species compared to all the animals.

But when, and if machines learns to love or hate, work in peace or retaliate in anger, then it’s not too far that, with the ability to consume & digest the vast amount of data, they will become more smarter & start taking control of the planet.

Only then we will be able to know that AI is helping us like Iron Man's Jarvis or planning to eradicate us like Terminator!!