Brier score for multi-class classification problems with $$r$$ labels defined as $$\frac{1}{n} \sum_{i=1}^n \sum_{j=1}^r (I_{ij} - p_{ij})^2.$$ $$I_{ij}$$ is 1 if observation $$i$$ has true label $$j$$, and 0 otherwise.

Note that there also is the more common definition of the Brier score for binary classification problems in bbrier().

mbrier(truth, prob, ...)

Arguments

truth :: factor() True (observed) labels. Must have the same levels and length as response. :: matrix() Matrix of predicted probabilities, each column is a vector of probabilities for a specific class label. Columns must be named with levels of truth. :: any Additional arguments. Currently ignored.

Value

Performance value as numeric(1).

Meta Information

• Type: "classif"

• Range: $$[0, 2]$$

• Minimize: TRUE

• Required prediction: prob

References

Brier GW (1950). “Verification of forecasts expressed in terms of probability.” Monthly Weather Review, 78(1), 1--3. doi: 10.1175/1520-0493(1950)078<0001:vofeit>2.0.co;2 .

Other Classification Measures: acc(), bacc(), ce(), logloss(), mauc_aunu()
set.seed(1)
mbrier(truth, prob)#> [1] 1.084326