Getting Started with Mulan

The typical usage scenario of Mulan would involve

• A machine learning reseacher performing an empirical evaluation of one or more multi-label learning algorithms, based on one or more multi-label datasets, and
• A machine learning practitioner building a multi-label model using a training dataset and then applying it to a new (unlabeled) dataset, in order to obtain predictions.

Requirements for running mulan

Each version of Mulan requires a different verion of Weka (the one included in the specific release) and a different version of Java to run. Additionally a specific version of JUnit is required to run tests. The following table shows the dependencies of each Mulan version:

Setting up an empirical evaluation experiment

We will here give an example of how to setup an experiment that empirically evaluates two multi-label learning algorithms on a multi-label dataset using cross-validation. Let's create a new Java class for the experiment, which we will call MulanExp1.java.

The first thing to do is load the multi-label dataset that will be used for the empirical evaluation. Mulan requires two text files for the specification of a multi-label dataset: a) an XML file specifying the names of the labels and optionally any hierarchical relationships among them, and b) an ARFF file specifying the actual data. The format of these two files is given here. In our example, we will allow the specification of the two filenames through command-line parameters to the experiment class:

String arffFilename = Utils.getOption("arff", args); String xmlFilename = Utils.getOption("xml", args);

Loading the data then can be done using the following code:

MultiLabelInstances dataset = new MultiLabelInstances(arffFilename, xmlFilename);

The next step is to create an instance from each of the two learners that we want to evaluate. We will create an instance of the RAkEL algorithm and an instance of the MLkNN algorithm. RAkEL is actually a meta algorithm and can accept any multi-label learner as a parameter, but is typically used in conjunction with the LabelPowerset algorithm. In turn LabelPowerset is a transformation-based algorithm and it accepts a single-label classifier as a parameter. We will use the C4.5 algorithm from Weka for this purpose (J48). MLkNN is an algorithm adaptation method that is based on kNN. The following code creates the two instances:

RAkEL learner1 = new RAkEL(new LabelPowerset(new J48())); MLkNN learner2 = new MLkNN();

We then declare an Evaluator object that handles empirical evaluations and a MultipleEvaluation object that stores the results:

Evaluator eval = new Evaluator(); MultipleEvaluation results;

To actually perform the evaluations we use the crossValidate method of the Evaluator class. This returns a MultipleEvaluation object, which we can print to see the results in terms of all applicable evaluation measures available in Mulan.

int numFolds = 10; results = eval.crossValidate(learner1, dataset, numFolds); System.out.println(results); results = eval.crossValidate(learner2, dataset, numFolds); System.out.println(results);

The whole code for MulanExp1.java is available here. For running the experiment, we will use the emotions data (emotions.xml and emotions.arff) that are available together with the Mulan distribution. Assuming the experiment's source file is in the same directory with emotions.arff, emotions.xml, weka.jar and mulan.jar from the distribution package, then to run this experiment in Windows you can type the following command:

javac -cp mulan.jar;weka.jar MulanExp1.java java -cp mulan.jar;weka.jar;. MulanExp1 -arff emotions.arff -xml emotions.xml

Since the classpath separator is platform dependent, replace ; with : if you are in Linux.

After a while, the results of this experiment appear on screen.

Obtaining predictions for an unlabeled dataset

We will here give an example of how to setup an experiment for obtaining the predictions of a trained model for a dataset with unlabeled instances. Let's create a new Java class for this experiment, which we will call MulanExp2.java.

The first thing to do is load the multi-label dataset that will be used for training the model. Mulan requires two text files for the specification of a multi-label dataset: a) an XML file specifying the names of the labels and optionally any hierarchical relationships among them, and b) an ARFF file specifying the actual data. The format of these two files is given here. In our example, we will allow the specification of the two filenames through command-line parameters to the experiment class:

String arffFilename = Utils.getOption("arff", args); String xmlFilename = Utils.getOption("xml", args);

Loading the data then can be done using the following code:

MultiLabelInstances dataset = new MultiLabelInstances(arffFilename, xmlFilename);

The next step is to create an instance of the learning algorithm that we want to train in order to build a model and obtain predictions. We will create an instance of the RAkEL algorithm. RAkEL is actually a meta algorithm and can accept any multi-label learner as a parameter, but is typically used in conjunction with the LabelPowerset algorithm. In turn LabelPowerset is a transformation-based algorithm and it accepts a single-label classifier as a parameter. We will use the C4.5 algorithm from Weka for this purpose (J48):

RAkEL model = new RAkEL(new LabelPowerset(new J48()));

The next step is to train the classifier using the data that we loaded:

model.build(dataset);

The next step is to load the unlabeled data instances. The dataset format (number of labels, number of features, order of attributes, etc) must conform to the format of the training data set based on which the model was built. The actual values of the labels don't matter, so they could be set to "0", "1" or the unknown value symbol of Weka, "?". This time the XML file is not necessary, so we will only obtain a filename for the ARFF file throught a command-line parameter:

String unlabeledFilename = Utils.getOption("unlabeled", args); FileReader reader = new FileReader(unlabeledFilename); Instances unlabeledData = new Instances(reader);

The next step is to perform a prediction for each instance in the data set:

int numInstances = unlabeledData.numInstances(); for (int instanceIndex=0; instanceIndex < numInstances; instanceIndex++) { Instance instance = unlabeledData.instance(instanceIndex); MultiLabelOutput output = model.makePrediction(instance); // do necessary operations with provided prediction output, here just print it out System.out.println(output); }

The learner returns an instance of the MultiLabelOutput class as the prediction result. Depending on the learner capabilities, the output may contain a bipartition of the labels into relevant and irrelevant and/or label confidences and/or label rankings as predicted for the given instance.

The whole code for MulanExp2.java is available here. For running the experiment, we will use the emotions data (emotions.xml and emotions.arff) that are available together with the Mulan distribution, using emotions.arff as both the training dataset as well as the unlabeled dataset. Assuming the experiment's source file is in the same directory with emotions.arff, emotions.xml, weka.jar and mulan.jar from the distribution package, then to run this experiment in Windows you can type the following command:

javac -cp mulan.jar;weka.jar MulanExp2.java java -cp mulan.jar;weka.jar;. MulanExp2 -arff emotions.arff -xml emotions.xml -unlabeled emotions.arff

Again, replace ; with : if you are in Linux.

After a while, the results of this experiment appear on screen.

More examples

More examples of the Mulan API usage can be found in the source files of Mulan, within the examples package. Some of them can be run by specifying command line parameters. The examples are illustrative, not trying to cover all usage scenarios and also are not designed as out of the box experiment 'launchers'.