Machine Learning Workflow

Machine Learning how to works in a step by step process with diagram:


Here Historic data is collected the entire data and used as the input for machine learning. This may however used indirectly it may need to be cleared and transformed etc. Some of the common tasks during feature engineering include handling missing values, handling outliers creating new features out of existing ones etc.

After feature engineering the data is split into two types are train data and data. The train data is used for training the machine learning model and data information. In other words, the machine learns from the training data. To build the model machine learning some algorithms are used.

Once the model is built it validated against test data. The concept of the test data is that it is real – time data. If the model performance on both train and test data satisfactory then the model completely.

Example:

Spam classification in E – mail conversations:

Historic data :

In emails msgs are received by user containing labels whether the user marked it as Spam or not

Feature Engineering :

Extracting information from past emails such as sender mail ID, sender IP, characteristics of email.

Train and Test data :

Some of the mails are kept for test, the remaining are used to train the machine learning model.

Machine Learning algorithm :

Supervised learning algorithms to learn pattern in train data. The patterns are a general representation of what makes an email spam or not spam

Machine learning Model :

The model built using the train data


Model Validation :

Test the machine learning model against the emails in the best data in order to check if the model is able to predict spam emails as spam correctly.

New data :

New email being sent to the user in real – time

Results :

If the models detects that this new email is spam, then the email is moved the “Spam” folder otherwise it is related in the user’s inbox.

 

 

 

The World of Machine Learning

When Machine Learning is Used:




Analysis on a given data by a human being has massive associated cost, time and effort as in the case of Express.

Human competence is absent for example If we want to navigate on space and we don’t have the expertise available we can make a machine lean and let it navigate on an unknown territory without any human

In Human competence cannot always be explained.

Example: Image processing, Self driven, Speech recognition

Amazon Go:

At present situation physical stores like amazon is setting up and eliminates the need for register while going through a common place shopping experience. Shoppers can pick items off the aisles as the would any supermarket without the need to go through the inconvenience of billing  the items at counters. All shoppers need to is swipe a card when they enter the store and pick up product via amazon wallet.

Translate Apps:

Now a days lot of built applications that help us communicate with people in any languages at all. Some applications like Google Translate have eliminated the need for memorizing common phrases from different languages and struggle of connecting with people from all over the world.

Self Driven Cars:




Present have lot of cars that can take anybody from one place to other place with no need for human interaction. These cars navigate independently, sense any other barrier in the path and manually accordingly. Slowing down, stopping and accelerating up on their own, self driven cars have the potential of reducing accidents. From tolerate  time for drivers reducing traffic, self driven cars have turned around the whole experience of travelling on roads.

Mahout in Machine Learning

Mahout:

Mahout is an open source by the Apache Software Foundation to implementations of all kinds of machine learning techniques with the goal of creating scalabe algorithms that are free to under the Apache license. In order to see the algorithms currently implemented in mahout type the following command in the terminal.



I)export MAHOUT_HOME = /Path

II)cd $MAHOUT_HOME

III)bin/mahout

All these can be accessed the $MAHOUT_HOME /bin/mahout command line driver. Each of these needs certain arguments as input to generate the output.

Mahout algorithms are Divided into 4 sections:

1.Collaborative filtering

2.Categorization

3.Clustering

4.Mahout utilities

1. Collaborative filtering:

Collaborative filtering is a machine learning technique used for generating recommendations. It uses information’s like ratings, user preference etc.

Collaborative filtering basically tow ways of generating recommendations.

I) User – Based : In this recommend items by finding similar users. Example if a user purchased a computer and second user has even purchased a computer along with other products then they are a suppose to be similar users and  the items purchased by the second user other than the computer recommended to the first user . It is like dynamic nature of users.

II) Item-Base: This item based recommendations calculate similarity between items and creates a similarity matrix from which recommendation are generated Mahout provides a set of components from our own recommendation engine.

Examples of Collaborative Filtering Algorithms:-

Distributed Item – based Collaborative Filtering

Non Distributed recommend



Collaborative Filtering using Matrix factorization.

Mahout Utilities :

In Mahout some algorithms , it helps in preparing content into formats for Mahout and are called MAHOUT UTILITIES.

Mahout Utilities mainly 3 categories

Creating Vectors from Text – In this utilities allow to produce Mahout Vector representations. There are mainly two utilities for converting a directory of text documents into Vector.

Creating text from Vectors – In this utilities allow to produce text from vectors.

Viewing Result – These utilities particularly is  used for viewing the clusters generating   by clustering algorithms.

 

 

 

Machine Learning

What is Machine Learning?

Machine Learning is a branch of artificial intelligence (AI) that focuses on the development of computer programs that can teach to grow and change exposed to new data. It is concerned with the design and development of algorithms that can take complex input data and can make intelligence decision based on the input data.

Types of Machine Learning:



1. Supervised Learning :

This type of machine learning is concerned with associating some undefined data document to some predefined label of the training data in order to predict the value of any valid input.  Common examples of supervised learning include classifying e-mail message as a spam. Mainly all classification algorithms. Example Mahout.

2.Unsupervised Learning :

Unsupervised learning type of machine learning is concerned with making sense out of complex hard to understand data by creating some similarity or some interesting patterns. No labels are associated with it. Mainly all clusterng algorithms in mahout for example.

3.Semi-Supervised Learning :

Semi -supervised learning is concerned with defining an undefined data document in the presence of both labeled and unlabeled data. It is a combination of supervised and unsupervised learning. The main aim of semi – supervised learning is to demonstrate how combining both labeled and unlabeled data can change the learning behavior.