You must be able to load your data before you can start modeling it.
In this post you will discover how you can load your CSV dataset in Weka. After reading this post, you will know:
- About the ARFF file format and how it is the default way to represent data in Weka.
- How to load a CSV file in the Weka Explorer and save it in ARFF format.
- How to load a CSV file in the ArffViewer tool and save it in ARFF format.
This tutorial assumes that you already have Weka installed.
How to Talk About Data in Weka
Machine learning algorithms are primarily designed to work with arrays of numbers.
This is called tabular or structured data because it is how data looks in a spreadsheet, comprised of rows and columns.
Weka has a specific computer science centric vocabulary when describing data:
- Instance: A row of data is called an instance, as in an instance or observation from the problem domain.
- Attribute: A column of data is called a feature or attribute, as in feature of the observation.
Each attribute can have a different type, for example:
- Real for numeric values like 1.2.
- Integer for numeric values without a fractional part like 5.
- Nominal for categorical data like “dog” and “cat”.
- String for lists of words, like this sentence.
On classification problems, the output variable must be nominal. For regression problems, the output variable must be real.
Need more help with Weka for Machine Learning?
Take my free 14-day email course and discover how to use the platform step-by-step.
Click to sign-up and also get a free PDF Ebook version of the course.
Data in Weka
Weka prefers to load data in the ARFF format.
ARFF is an acronym that stands for Attribute-Relation File Format. It is an extension of the CSV file format where a header is used that provides metadata about the data types in the columns.
For example, the first few lines of the classic iris flowers dataset in CSV format looks as follows:
The same file in ARFF format looks as follows:
You can see that directives start with the at symbol (@) and that there is one for the name of the dataset (e.g. @RELATION iris), there is a directive to define the name and datatype of each attribute (e.g. @ATTRIBUTE sepallength REAL) and there is a directive to indicate the start of the raw data (e.g. @DATA).
Lines in an ARFF file that start with a percentage symbol (%) indicate a comment.
Values in the raw data section that have a question mark symbol (?) indicate an unknown or missing value. The format supports numeric and categorical values as in the iris example above, but also supports dates and string values.
Depending on your installation of Weka, you may or may not have some default datasets in your Weka installation directory under the data/ subdirectory. These default datasets distributed with Weka are in the ARFF format and have the .arff file extension.
Load CSV Files in the ARFF-Viewer
Your data is not likely to be in ARFF format.
In fact, it is much more likely to be in Comma Separated Value (CSV) format. This is a simple format where data is laid out in a table of rows and columns and a comma is used to separate the values on a row. Quotes may also be used to surround values, especially if the data contains strings of text with spaces.
The CSV format is easily exported from Microsoft Excel, so once you can get your data into Excel, you can easily convert it to CSV format.
Weka provides a handy tool to load CSV files and save them in ARFF. You only need to do this once with your dataset.
Using the steps below you can convert your dataset from CSV format to ARFF format and use it with the Weka workbench. If you do not have a CSV file handy, you can use the iris flowers dataset. Download the file from the UCI Machine Learning repository (direct link) and save it to your current working directory as iris.csv.
1. Start the Weka chooser.
2. Open the ARFF-Viewer by clicking “Tools” in the menu and select “ArffViewer”.
3. You will be presented with an empty ARFF-Viewer window.
4. Open your CSV file in the ARFF-Viewer by clicking the “File” menu and select “Open”. Navigate to your current working directory. Change the “Files of Type:” filter to “CSV data files (*.csv)”. Select your file and click the “Open” button.
5. You should see a sample of your CSV file loaded into the ARFF-Viewer.
6. Save your dataset in ARFF format by clicking the “File” menu and selecting “Save as…”. Enter a filename with a .arff extension and click the “Save” button.
You can now load your saved .arff file directly into Weka.
Note, the ARFF-Viewer provides options for modifying your dataset before saving. For example you can change values, change the name of attributes and change their data types.
It is highly recommended that you specify the names of each attribute as this will help with analysis of your data later. Also, make sure that the data types of each attribute are correct.
Load CSV Files in the Weka Explorer
You can also load your CSV files directly in the Weka Explorer interface.
This is handy if you are in a hurry and want to quickly test out an idea.
This section shows you how you can load your CSV file in the Weka Explorer interface. You can use the iris dataset again, to practice if you do not have a CSV dataset to load.
1. Start the Weka GUI Chooser.
2. Launch the Weka Explorer by clicking the “Explorer” button.
3. Click the “Open file…” button.
4. Navigate to your current working directory. Change the “Files of Type” to “CSV data files (*.csv)”. Select your file and click the “Open” button.
You can work with the data directly. You can also save your dataset in ARFF format by clicking he “Save” button and typing a filename.
Use Excel for Other File Formats
If you have data in another format, load it in Microsoft Excel first.
It is common to get data in another format such as CSV using a different delimiter or fixed width fields. Excel has powerful tools for loading tabular data in a variety of formats. Use these tools and first load your data into Excel.
Once you have loaded your data into Excel, you can export it into CSV format. You can then work with it in Weka, either directly or by first converting it to ARFF format.
Resources
Below are some additional resources that you may find useful when working with CSV data in Weka.
Summary
In this post you discovered how to load your CSV data into Weka for machine learning.
Specifically, you learned:
- About the ARFF file format and how Weka uses it to represent datasets for machine learning.
- How to load your CSV data using ARFF-Viewer and save it into ARFF format.
- How to load your CSV data directly in the Weka Explorer and use it for modeling.
Do you have any questions about loading data in Weka or about this post? Ask your questions in the comments and I will do my best to answer.
No comments:
Post a Comment