Multiclass AUC measures.

• AUNU: AUC of each class against the rest, using the uniform class distribution. Computes the AUC treating a c-dimensional classifier as c two-dimensional 1-vs-rest classifiers, where classes are assumed to have uniform distribution, in order to have a measure which is independent of class distribution change (Fawcett 2001).

• AUNP: AUC of each class against the rest, using the a priori class distribution. Computes the AUC treating a c-dimensional classifier as c two-dimensional 1-vs-rest classifiers, taking into account the prior probability of each class (Fawcett 2001).

• AU1U: AUC of each class against each other, using the uniform class distribution. Computes something like the AUC of c(c - 1) binary classifiers (all possible pairwise combinations). See Hand (2001) for details.

• AU1P: AUC of each class against each other, using the a priori class distribution. Computes something like AUC of c(c - 1) binary classifiers while considering the a priori distribution of the classes as suggested in Ferri (2009). Note we deviate from the definition in Ferri (2009) by a factor of c. The person implementing this function and writing this very documentation right now cautions against using this measure because it is an imperfect generalization of AU1U.

mauc_aunu(truth, prob, na_value = NaN, ...)

mauc_aunp(truth, prob, na_value = NaN, ...)

mauc_au1u(truth, prob, na_value = NaN, ...)

mauc_au1p(truth, prob, na_value = NaN, ...)

## Arguments

truth :: factor() True (observed) labels. Must have the same levels and length as response. :: matrix() Matrix of predicted probabilities, each column is a vector of probabilities for a specific class label. Columns must be named with levels of truth. :: numeric(1) Value that should be returned if the measure is not defined for the input (as described in the note). Default is NaN. :: any Additional arguments. Currently ignored.

## Value

Performance value as numeric(1).

## Meta Information

• Type: "classif"

• Range: $$[0, 1]$$

• Minimize: FALSE

• Required prediction: prob

## References

Fawcett T (2001). “Using rule sets to maximize ROC performance.” In Proceedings 2001 IEEE international conference on data mining, 131--138. IEEE.

Ferri C, Hernández-Orallo J, Modroiu R (2009). “An experimental comparison of performance measures for classification.” Pattern Recognition Letters, 30(1), 27--38. doi: 10.1016/j.patrec.2008.08.010 .

Hand DJ, Till RJ (2001). “A simple generalisation of the area under the ROC curve for multiple class classification problems.” Machine learning, 45(2), 171--186.

Other Classification Measures: acc(), bacc(), ce(), logloss(), mbrier()
set.seed(1)
mauc_aunu(truth, prob)#> [1] 0.4047619