Computes the weighted balanced accuracy, suitable for imbalanced data sets. It is defined analogously to the definition in sklearn.

First, the sample weights \(w\) are normalized per class: $$ \hat{w}_i = \frac{w_i}{\sum_j 1(y_j = y_i) w_i}. $$ The balanced accuracy is calculated as $$ \frac{1}{\sum_i \hat{w}_i} \sum_i 1(r_i = t_i) \hat{w}_i. $$

bacc(truth, response, sample_weights = NULL, ...)

Arguments

truth

:: factor()
True (observed) labels. Must have the same levels and length as response.

response

:: factor()
Predicted response labels. Must have the same levels and length as truth.

sample_weights

:: numeric()
Non-negative sample weights. Must have the same levels and length as truth. Defaults to equal sample weights.

...

:: any
Additional arguments. Currently ignored.

Value

Performance value as numeric(1).

Meta Information

  • Type: "classif"

  • Range: \([0, 1]\)

  • Minimize: FALSE

  • Required prediction: response

References

Brodersen KH, Ong CS, Stephan KE, Buhmann JM (2010). “The Balanced Accuracy and Its Posterior Distribution.” In 2010 20th International Conference on Pattern Recognition. doi: 10.1109/icpr.2010.764 .

Guyon I, Bennett K, Cawley G, Escalante HJ, Escalera S, Ho TK, Macia N, Ray B, Saeed M, Statnikov A, Viegas E (2015). “Design of the 2015 ChaLearn AutoML challenge.” In 2015 International Joint Conference on Neural Networks (IJCNN). doi: 10.1109/ijcnn.2015.7280767 .

See also

Other Classification Measures: acc(), ce(), logloss(), mbrier()

Examples

set.seed(1) lvls = c("a", "b", "c") truth = factor(sample(lvls, 10, replace = TRUE), levels = lvls) response = factor(sample(lvls, 10, replace = TRUE), levels = lvls) bacc(truth, response)
#> [1] 0.2222222