Machine Learning Using WEKA

There are multiple gigs of data to deal with in our lives and this only goes

on increasing with each passing day. The gap between the data generated and

analyzed is also growing. So, you ought to look for techniques that make things

easier. Machine learning is one such technique that searches a very large

dataset of possible hypothesis to determine the best fit in the observed data

and any prior knowledge held by the learning system. Data Mining augments the

search and understanding of the electronically stored data.

What is WEKA?



Waikato Environment for Knowledge Analysis (WEKA), developed at the University
of Waikato, New Zealand, is a collection of machine learning algorithms with

data preprocessing tools to provide input to these algorithms. The tool was

developed in Java and runs on Linux as well as Windows. It can also be used to

develop and analyze new machine learning algorithms. It is open source software

and distributed under the terms of GNU General Public License. The input to the

Machine Learning algorithms is in the form of a relational table in the ARFF

format. Weka comes with an API documentation generated using Javadoc. More

details on Weka and its usage are available across a few chapters in the book

written by Ian H Witten and Eibe Frank, 'Data Mining: Practical Machine Learning

Tools and Techniques,' 2nd Edition, Morgan Kaufmann Series, San Francisco, 2005.

How it helps



Some key features of WEKA include:

Preprocess — Weka has file format converters for spreadsheets, C4.5

file formats and serialized instances. It can also open a URL and use HTTP to

download an ARFF file from the Web or open a database using JDBC, and retrieve

instances using SQL. It also provides a list of filters to delete specified

attributes from a dataset.

Classify — Weka trains and tests learning schemes that perform

classification or regression. The classifiers can be divided into Bayesian,

trees, rules, functions and lazy. It also builds a linear regression model and

allows the user to build their own classifiers interactively. It also provides

options for a number of meta learners.

Cluster — Weka shows the clusters and the number of instances in the

cluster. Thereafter it determines the majority class in each cluster and gives

the confusion matrix.

Associate — Weka contains three algorithms for determining

association rules-apriory, predictive apriory and filtered associators.

It has no methods for evaluating such rules.

Attribute Selection — Weka gives access to several methods for

attribute selection, which involves an attribute evaluator and a search method.

Attribute selection can be performed using the full training set or

cross-validation.

In the Preprocess tab, you can view attributes in the input file, properties of the selected attribute, and visualisation of class distribution for each attribute.	Building a Naïve Bayes Classifier with 10 fold cross-validation. The correctly classified instances can be viewed by right clicking on Classifier in Results Window.

Visualization - It displays a matrix of two-dimensional scatter plots

of each pair of attributes.

Preparing input



Major effort in the process of data mining/machine learning goes into the

preparation of input. In order to analyze data using Weka, you need to prepare

it in the Attribute Relation File Format (ARFF) and then load it in its

Explorer. Spreadsheets, Comma Separated Value (CSV) files and databases can be

converted to ARFF. In ARFF, there is an @relation tag, @attribute tag and @data

tag to represent the dataset name, attribute information and values

respectively.

Classifying data



Weka should preferably be used through a graphical user interface called
'Explorer' than the command-line interface. The other two interfaces are

'Knowledge Flow Interface,' which supports design configuration for streamed

data processing and 'Experimenter,' which helps users compare a variety of

learning techniques. In this example, we use an ARFF named age.arff which

contains a few selective words in the attribute and @data contains their number

of occurrences per 10,000 words in a blog dataset written by bloggers belonging

to various age groups.

1. Open the file you want to analyze using the Open file option in the

Preprocess tab in Weka explorer, ie open the age data file, age.arff.



2. Once the input file has been opened, all attributes in the input file are
shown in the Attributes Window. Properties of the selected attributes like

Attribute Name, Attribute Type, number of missing values, etc are displayed in

the 'Selected Attribute' window. Here, you can select attributes that you want

to include in working relations, eg age prediction.



3. Select the classifier algorithm in the Classify tab. In this example, we
selected Naïve Bayes with 10 fold Cross-Validation. Next, click on Start. The

result is displayed in the Classifier Output window as shown in figure on the

left.

Output of the Naïve Bayes Classifier in terms of errors, accuracy by class and confusion matrix, on Age dataset.	View of an ARFF dataset which consists of a list of instances, and the attribute values for each instance separated by commas.

Analyzing the result



The result displays the summary of the data set followed by the algorithm

used to analyze it. It also gives the predictive performance of the

machine-learning algorithm applied on the dataset. Thereafter the confusion

matrix displays the number of instances classified properly and those

misclassified. The classification error is displayed mentioning the mean

absolute error and the root mean squared error of the class probability

estimates.

Processing huge datasets



If the dataset is too huge, running to a few thousand attributes and a few

lakh records, it can happen that Weka runs into an 'OutOfMemory' exception. Most

Java virtual machines allocate a certain maximum amount of memory which is much

less than the amount of RAM to run Java programs. However, we can extend the

memory available for the virtual machine by setting appropriate options.

Alternately, Weka offers several filters for re-sampling a dataset and

generating a new dataset reduced in size. Besides, there are schemes that can be

trained in an incremental fashion, not just in batch mode unlike most

classifiers which require all the data before they can be trained. Such a

classifier will load the dataset incrementally and feed the data instance by

instance to the classifier.

Conclusion



It is difficult for a single machine learning tool to suite all data mining

requirements even as the universal learner is still a distant dream. In order to

obtain an accurate model of real datasets, the learning algorithm must match the

domain. Data mining is an experimental science and provides a workbench for data

preprocessing tools and machine learning algorithms. Weka helps in realizing the

goal of data mining, by predicting missing values and validating that the

predicted values are correct.

Abhinav Gupta & Sumit Goswami