data_format
and
data_formats
for Learner
, Task
,
and DataBackend
classes.partition()
function creates training, test
and validation sets now.Task$col_info
.Learner$predict
(#943).$internal_valid_task
can now be set to an
integer
vector.$predict_sets
(#1094). This is relevant for measures that only extract information
from the model of a learner (such as internal validation scores or AIC /
BIC)$divide()
methodTask$cbind()
now works with non-standard primary
keys for data.frames
(#961)."info"
instead of "debug"
(#972).regr.pinball
here and in
mlr3measures.mu_auc
here and in
mlr3measures.msr("regr.rsq")
.classif.debug
and
regr.debug
have new methods $importance()
and
$selected_features()
for testing, also in downstream
packages.default_fallback()
.$set_col_roles()
and $col_roles
.$encapsulate(method, fallback)
method. The
$fallback
field is read-only now and the encapsulate status
can be retrieved from the $encapsulation
field."primary_iters"
$obs_loss
. This is possible for
Prediction
, ResampleResult
and
BenchmarkResult
.Measure
s now also return a vector of
numerics.msr("classif.mcc")
."marshal"
property, which allows
learners to process models so they can be serialized. This happens
automatically during resample()
and
benchmark()
.default_values.Learner()
function.lgr
package.mlr_learners
respects
prototype arguments recently added in mlr3misc.resample()
.data.table
tests on mac.data_prototype
when resampling from
learner$state
to reduce memory consumption.data.table
and BLAS to
1 when running resample()
or benchmark()
in
parallel.resample()
and
benchmark()
by reducing the number of hashing
operations.HotstartStack
anymore
when the model is missing.hotstart_threshold
are not added to
the HotstartStack
anymore.learner$state$train_time
in hotstarted learners is
now only the time of the last training.HotstartStack
did not work with
column roles set in the task.design
of benchmark()
can now include
parameter settings.packageVersion()
.col_info
to allow adding new
methods for backends."mlr3.exec_chunk_bins"
option to split the
resampling iterations into a number of bins.data.table()
is now re-exported."try"
, which works similar to
"none"
but captures errorspaired
to benchmark_grid()
function, which can be used to create a benchmark design, where
resamplings have been instantiated on tasks.ResultData
for
as_resample_result()
converter.list
for
as_resample_result()
converter.print
method to make the output more readable.distr6
.GraphLearner
.as_prediction_classif()
for
data.frame()
input (#872).Learner
during
train for early stopping.mauc_aunu
,
mauc_aunp
, mauc_au1u
,
mauc_au1p
.classif.costs
does not require a
Task
anymore.as_task_unsupervised()
mlr_reflections
."mlr3.exec_random"
and
"mlr3.exec_chunk_size"
). These options are passed down to
the respective map functions in package future.apply
.head()
and tail()
methods for
Task
.label
,
i.e. Task
, TaskGenerator
,
Learner
, Resampling
, and
Measure
.as.data.table()
methods for objects of class
Dictonary
have been extended with additional columns.as_task_classif.formula()
and
as_task_regr.formula()
now remove additional atrributes
attached to the data which caused some some learners to break.$train()
and $predict()
methods of a Learner
. This
ensures that package loading errors are properly propagated and not
affected by encapsulation (#771)."evaluate"
(#763).as_task_classif()
and as_task_regr()
now
support the construction of tasks using the formula interface,
e.g. as_task_regr(mpg ~ ., data = mtcars)
(#761).default_values()
function to extract parameter
default values from Learner
objects."validation"
has been renamed to
"holdout"
. In the next release, mlr3
will
start switching to the now more common terms
"train"
/"validation"
instead of
"train"
/"test"
for the sets created during
resampling.ResampleResult
and BenchmarkResult
.resample()
and benchmark()
got a new
argument clone
to control which objects to clone before
performing computations.data.frame
to Task
in
as_task_classif()
and as_task_regr()
. A
warning is signaled if any column contains infinite values.(classif|regr|surv).xgboost
with hyperparameter
nrounds
updated) can now optionally store a stack of
trained learners to be used to hotstart their training. Note that this
feature is still somewhat experimental. See HotstartStack
and #719.sim.jaccard
(Jaccard Index) and sim.phi
(Phi
coefficient) (#690).predict_newdata()
now also supports
DataBackend
as input.install_pkgs()
to install required
packages. This generic works for all objects with a
packages
field as well as ResampleResult
and
BenchmarkResult
(#728).regr.debug
for debugging.Task
method $set_levels()
to control
how data with factor columns is returned, independent of the used
DataBackend
.NA
if prerequisite are not met
(#699). This allows to conveniently score your experiments with multiple
measures having different requirements.%
.Task$label()
. These will be used in visualizations in the
future.Task$add_strata()
.partition()
to split a task into a
training and test set.loglik()
for class
Learner
."aic"
and "bic"
to compute
the Akaike Information Criterion or the Bayesian Information Criterion,
respectively.ResamplingCustomCV
. Creates a
custom resampling split based on the levels of a user-provided factor
variable.encapsulate
for resample()
and benchmark()
to conveniently enable encapsulation and
also set the fallback learner to the featureless learner. This is simply
for convenience, configuring each learner individually is still possible
and allows a more fine-grained control (#634, #642).parallel_predict
for Learner
to
enable parallel predictions via the future backend. This currently is
only enabled while calling the $predict()
or
$predict_newdata
methods and is disabled during
resample()
and benchmark()
where you have
other means to parallelize.$data
in ResampleResult
and
BenchmarkResult
to simplify the API and avoid confusion.
The converter as.data.table()
can be used instead to access
the internal data.beta
.ordered
in
Task$data()
from TRUE
to
FALSE
.ResamplingRepeatedCV$folds()
(#643).uri
. This role be
split up into multiple roles by the mlr3keras
package.as.data.table.Resampling
method."row_id"
to "row_ids"
in
the as.data.table()
methods for
PredictionClassif
and PredictionRegr
(#547).as_prediction_classif()
and
as_prediction_regr()
to reverse the operation of
as.data.table.PredictionClassif()
and
as.data.table.PredictionRegr()
.learner$predict_newdata()
is not mandatory anymore
(#563).Task$data()
defaults to return only active rows and
columns, instead of asserting to only return rows and columns. As a
result, the $data()
method can now also be used to query
inactive rows and cols from the DataBackend
.uri
which is intended to
point to external resources, e.g. images on the file system.set_threads()
to control the number of
threads during calls to external packages. All objects will be migrated
to have threading disabled in their defaults to avoid conflicting
parallelization techniques (#605).mlr3.debug
: avoid calls to
future
in resample()
and
benchmark()
to improve the readability of tracebacks.mlr3.allow_utf8_names
: allow
non-ascii characters in column names in tasks.ResampleResult
and
BenchmarkResult
now optionally remove the DataBackend of
the Tasks in order to reduce file size and memory footprint after
serialization. To remove the backends from the containers, set
store_backends
to FALSE
in
resample()
or benchmark()
, respectively. Note
that this behavior will eventually will be the default for future
releases.Learner$predict_newdata()
now have row ids starting from 1
instead auto incremented row ids of the training task.as.data.table.DictionaryTasks
now returns an additional
column properties
.conditions
to
ResampleResult$score()
and
BenchmarkResult$score()
to allow to work with failing
learners more conveniently.Task
: $set_col_roles
and
$set_row_roles
as a replacement for the deprecated and less
flexible $set_col_role
and $set_row_role
.friedman.test.BenchmarkResult()
in
favor of the new mlr3benchmark
package.MeasureOOBError
now has set property
minimize
to TRUE
."featureless"
to tag learners
which can operate on featureless tasks.predict_sets
for returned [Prediction] objects.lgr
.NaN
for
BenchmarkResult
for resamplings with a single iteration
(#551).future
(mlr3tuning#270).ResampleResult
and BenchmarkResult
now
share a common interface to store the experiment results. Manual
construction is still possible with helper function
as_result_data()
ResamplingCV
and
ResamplingRepeatedCV
.classif.prauc
(area under precision-recall
curve).bibtex
.saveRDS()
or
serialize()
.ResampleResult
or
BenchmarkResult
are now de-duplicated for an optimized
serialization.breast_cancer
: all factor features are
now correctly stored as ordered factors.convert_task()
.breast_cancer
ResamplingLOO
for leave-one-out resampling."distr"
using the
distr6
package.ResamplingBootstrap
in combination with grouping
(#514).TaskGeneratorMoons
.keep_model
to learners
"classif.rpart"
and "regr.rpart"
."cassini"
,
"circle"
, "simplex"
, "spirals"
,
and "moons"
).plot()
method for most task generators.german_credit
(#514).future.apply
is now imported (instead of
suggested). This is necessary to ensure reproducibility: This way
exactly the same result is calculated, independent of the parallel
backend.Task$order
.classif.bbrier
(binary Brier score)
and classif.mbrier
(multi-class Brier score).ResamplingInsample
.TaskUnsupervised
.ResampleResult
s and
BenchmarkResult
s with c()
.Task$predict_newdata()
/Task$rbind()
(#423).Switched to new roxygen2
documentation format for R6
classes.
resample()
and benchmark()
now support
progress bars via the package progressr
.
Row ids now must be numeric. It was previously allowed to have
character row ids, but this lead to confusion and unnecessary code
bloat. Row identifiers (e.g., to be used in plots) can still be part of
the task, with row role "name"
.
Row names can now be queried with
Task$row_names
.
DataBackendMatrix
now supports to store an optional
(numeric) dense part.
Added new method $filter()
to filter
ResampleResult
s to a subset of iterations.
Removed deprecated character()
-> object
converters.
Empty test sets are now handled separately by learners (#421). An empty prediction object is returned for all learners.
The internal train and predict function of Learner
now should be implemented as private method: instead of public methods
train_internal
and predict_internal
, private
methods .train
and .predict
are now
encouraged.
It is now encouraged to move some internal methods from public to private:
Learner$train_internal
should now be private method
$.train
.Learner$predict_internal
should now be private method
$.predict
.Measure$score_internal
should now be private method
$.score
. The public methods will be deprecated in a future
release.Removed arguments from the constructor of measures
classif.debug
and classif.costs
. These can be
set directly by msr()
.
We have published an article about mlr3 in the Journal of Open
Source Software: https://joss.theoj.org/papers/10.21105/joss.01903. See
citation("mlr3")
for the citation info.
New method Learner$reset()
.
New method BenchmarkResult$filter()
.
Learners returned by BenchmarkResult$learners
are
reset to encourage the safer alternative
BenchmarkResult$score()
to access trained models.
Fix ordering of levels in
PredictionClassif$set_threshold()
(triggered an
assertion).
Switched from package Metrics
to package
mlr3measures
.
Measures can now calculate all scores using micro or macro averaging (#400).
Measures can now be configured to return a customizable
performance score (instead of NA
) in case the score cannot
be calculated.
Character columns are now treated differently from factor
columns. In the long term, character()
columns are supposed
to store text.
Fixed a bug triggered by integer grouping variables in
Task
(#396).
benchmark_grid()
now accepts instantiated
resamplings under certain conditions.
Task$set_col_roles()
and
Task$set_row_roles()
are now deprecated. Instead it is
recommended for now to work with the lists Task$col_roles
and Task$row_roles
directly.
Learner$predict_newdata()
now works without argument
task
if the learner has been fitted with
Learner$train()
(#375).
Names of column roles have been unified ("weights"
,
"label"
, "stratify"
and "groups"
have been renamed).
Replaced MeasureClassifF1
with
MeasureClassifFScore
and fixed a bug in the F1 performance
calculation (#353). Thanks to @001ben for reporting.
Stratification is now controlled via a task column role (was a
parameter of class Resampling
before).
Added a S3 predict()
method for class
Learner
to increase interoperability with other
packages.
Many objects now come with a $help()
which opens the
respective manual page.
It is now possible to predict and score results on the training
set or on both training and test set. Learners can be instructed to
predict on multiple sets by setting predict_sets
(default:
"test"
). Measures operate on all sets specified in their
field predict_sets
(default: "test"
).
ResampleResult$prediction
and
ResampleResult$predictions()
are now methods instead of
fields, and allow to extract predictions for different predict
sets.
ResampleResult$performance()
has been renamed to
ResampleResult$score()
for consistency.
BenchmarkResult$performance()
has been renamed to
BenchmarkResult$score()
for consistency.
Changed API for (internal) constructors accepting
paradox::ParamSet()
. Instead of passing the initial values
separately, the initial values must now be set directly in the
ParamSet
.
Deprecated support of automatically creating objects from
strings. Instead, mlr3
provides the following helper
functions intended to ease the creation of objects stored in
dictionaries: tsk()
, tgen()
,
lrn()
, rsmp()
, msr()
.
BenchmarkResult
now ensures that the stored
ResampleResult
s are in a persistent order. Thus,
ResampleResult
s can now be addressed by their position
instead of their hash.
New field
BenchmarkResult$n_resample_results
.
New field BenchmarkResult$hashes
.
New method Task$rename()
.
New S3 generic as_benchmark_result()
.
Renamed Generator
to
TaskGenerator
.
Removed the control object mlr_control()
.
Removed ResampleResult$combine()
.
Removed BenchmarkResult$best()
.