Sunday, 22 January 2017

See this simple introduction to Natural Language Processing (NLP)


Today, with Digitization of everything, 80 percent the data being created is unstructured. 

Audio, Video, our social footprints, the data generated from conversations between customer service reps, tons of legal document’s texts processed in financial sectors are examples of unstructured data stored in Big Data.

Organizations are turning to natural language processing (NLP) technology to derive understanding from the myriad of these unstructured data available online and in call-logs.

Natural language processing (NLP) is the ability of computers to understand human speech as it is spoken. NLP is a branch of artificial intelligence that has many important implications on the ways that computers and humans interact. Machine Learning has helped computers parse the ambiguity of human language.

Apache OpenNLP, Natural Language Toolkit(NLTK), Stanford NLP are various open source NLP libraries used in real world application below.

Here are multiple ways NLP is used today:

The most basic and well known application of NLP is Microsoft Word spell checking.

Text analysis, also known as sentiment analytics is a key use of NLP. Businesses are most concerned with comprehending how their customers feel emotionally adn use that data for betterment of their service.

Email filters are another important application of NLP. By analyzing the emails that flow through the servers, email providers can calculate the likelihood that an email is spam based its content by using Bayesian or Naive based spam filtering.

Call centers representatives engage with customers to hear list of specific complaints and problems. Mining this data for sentiment can lead to incredibly actionable intelligence that can be applied to product placement, messaging, design, or a range of other use cases.

Google and Bing and other search systems use NLP to extract terms from text to populate their indexes and to parse search queries.

Google Translate applies machine translation technologies in not only translating words, but in understanding the meaning of sentences to provide a true translation.

Many important decisions in financial markets use NLP by taking plain text announcements, and extracting the relevant info in a format that can be factored into algorithmic trading decisions. E.g. news of a merger between companies can have a big impact on trading decisions, and the speed at which the particulars of the merger, players, prices, who acquires who, can be incorporated into a trading algorithm can have profit implications in the millions of dollars.

Since the invention of the typewriter, the keyboard has been the king of human-computer interface. But today with voice recognition via virtual assistants, like Amazon’s Alexa, Google’s Now, Apple’s Siri and Microsoft’s Cortana respond to vocal prompts and do everything from finding a coffee shop to getting directions to our office and also tasks like turning on the lights in home, switching the heat on etc. depending on how digitized and wired-up our life is.

Question Answering - IBM Watson is the most prominent example of question answering via information retrieval that helps guide in various areas like healthcare, weather, insurance etc.

Therefore it is clear that Natural Language Processing takes a very important role in new machine human interfaces. It’s an essential tool for leading-edge analytics & is the near future.


Sunday, 15 January 2017

This is how Analytics is changing the game of Sports!!

Analytics and Big Data have disrupted many industries, and now they are on the edge of scoring major points in sports. Over the past few years, the world of sports has experienced an explosion in the use of analytics

Till few years back experience, gut feelings, and superstition have traditionally shaped the decision making process in sports.

It is first started with Oakland Athletics' General Manager, Billy Beane, who applied analytics for selecting right players. This was the first known use of statistics and data to make decisions in professional sports.

Today, every major professional sports team either has an analytics department or an analytics expert on staff.  From coaches and players to front offices and businesses, analytics can make a difference in scoring touchdowns, signing contracts or preventing injuries.

Big name organizations such as the Chicago Cubs, and Golden State Warriors are realizing that this is the future of sports and it is in their best interest to ride the wave while everyone else is trying to learn how to surf.

Golden State Warriors, have similarly used big data sets to help owners and coaches recruit players and execute game plans.

SportVu has six cameras installed in the NBA arenas to track the movements of every player on the court and the basketball 25 times per second. The data collected provides a plethora of innovative statistics based on speed, distance, player separation and ball possession to improve next games.

Adidas miCoach app works by having players attach a wearable device to their jerseys. Data from the device shows the coach who the top performers are and who needs rest. It also provides real-time stats on each player, such as speed, heart rate and acceleration.

Patriots developed a mobile app called Patriots Game Day Live, available to anyone attending a game at Gillette Stadium. With this app, they are trying to predict the wants and needs of fans, special content to be delivered, in-seat concession ordering and bathroom wait times.

FiveThirtyEight.com, provides details into more than just baseball coverage. It has over 20 journalists crunching numbers for fans to gain a better understanding of an upcoming game, series or season.

Motus’ new sleeves for tracking a pitcher’s throwing motion, measuring arm stress, speed and shoulder rotation. The advanced data generated from this increases a player’s health, performance and career. Experts can now predict with greater confidence if and when a pitcher with a certain throwing style will get injured.

In the recent Cricket world cup, every team had its own team of Data Analysts. They used various technologies like Cloud Platform and visualizations to predict scores, player performance, player profiles and more. Around 40 years’ worth of Cricket World Cup data is being mined to produce insights that enhances the viewer's experience. 

Analytics can advance the sports fans' experience as teams and ticket vendors compete with the at-home experience -- the better they know their fans, the better they can cater to them.

This collection of data is also used for internet ads, which can help with the expansion and growth of your organization through social media platforms or websites. 
  • What would be the most profitable food served at the concession stand?
  • What would be the best prices to sell game day tickets?
  • Determine which player on the team is the most productive?
  • Which players in the draft will become all-stars, and which ones will be considered role players?
  • Understand the fans behavior at the stadium via their app and push relevant information accordingly.
In this Digital age, Analytics are the present and future of professional sports. Any team that does not apply them to the fullest is at a competitive disadvantage.


Sunday, 8 January 2017

What are Microservices in Digital Transformation?

Today’s organizations are feeling the fear of becoming dinosaur every day. New disrupters are coming into your industry and turning everything upside down.

Customers are more demanding than ever and will abandon the service that is too slow to respond.  Everything is needed yesterday to make your customers happy.

Now, there is no time for organizations to implement huge enterprise applications which takes months and years. 

What they need is, more agile, smaller, hyper focused teams working together to innovate and provide customer value.

This is where Microservices have gain momentum and are becoming fast go-to solution for enterprises. They takes SOA a step further by breaking every component into effectively single-purpose applications.

Microservices, show a strategy for decomposing a large project, based on the functions, into smaller, more manageable pieces. While a monolithic app is One Big Program with many responsibilities, Microservice based apps are composed of several small programs, each with a single responsibility

Microservices are independently developed & deployable, small, modular services. Each component is developed separately, and the application is then simply the sum of its constituent components. Each service runs as a unique process and communicates with other components via a very lightweight methods like HTTP/Rest with Jason.

Unlike old single huge enterprise application which requires heavy maintenance, Microservices are easy to manage.

Here are few characteristics and advantages of Microservices:
  • Very small, targeted in scope and functionality
  • Gives developers the freedom to independently develop and deploy services
  • Loosely coupled & can communicate with other services on industry wide standards like HTTP and JSON
  • API based connectivity
  • Every service can be coded in different programming language
  • Easily deployable and disposable makes releases possible even multiple times a day
  • New Digital technology can be easily adopted for a service
  • Allows to change services as required by business, without a massive cost
  • Testing and releases easier for individual components
  • Better fault tolerance and scale up

There are some challenges as well, while using Microservices:
  • Incur a cost of the testing at system integration level
  • Need to configure monitoring and alerting and similar services for each microservice
  • Service calls to one another, so tracing the path and debugging can be difficult
  • Each service communicates through API/remote calls, which have more overhead
  • Each service generates a log, so there is no central log monitoring.
Netflix has great Microservice architecture that receives more than one billion calls every day, from more than 800 different types of devices, to its streaming-video API.

Nike, the athlete clothing and shoe giant & now digital brand is using Microservices in its apps to deliver extra ordinary customer experience.

Amazon, eBay are other great examples of Microservices architecture.

GE’s Predix - the industrial Internet platform is based on Microservices architecture.

So, if your IT organization is implementing a microservices architecture, here are some examples of an operating system (Linux, Ubuntu, CoreOS), container technology(Docker), a scheduler(Swarm, Kubernetes), and a monitoring tool(Prometheus).

The technical demands of digital transformation, all front/back-office systems that seamlessly coordinate customer experiences in a digital world is achieved by Microservices as the preferred architecture.

Microservices help close the gap between business and IT & are fundamental shift in how IT approaches software development and are absolutely essential in Digital Transformation.

LinkWithin

Related Posts Plugin for WordPress, Blogger...