NEWS | R Documentation |
This release is primarily a version bump for the release of the paper in J. Stat. Software.
updated CITATION file.
We export set of about a dozen printing utility functions
which are used in our print
methods.
bootMer
now allows the use of re.form
.
fixed reordering bug in names of getME(.,"Ztlist")
(terms are reordered in decreasing order of the number of
levels of the grouping variable, but names were not being
reordered)
fixed issue with simulation when complex forms (such as nested random effects terms) are included in the model (Github #335)
explicit maxit
arguments for various functions
(refit
, mkGlmerDevfun
, ...)
terms
and formula
methods now have
random.only
options
getME
gains a glmer.nb.theta
option.
It is now (an S3) generic with an "merMod"
method in
lme4 and potentially other methods in dependent packages.
simulate
now works for glmer.nb
models
(Github #284: idea from @aosmith16)
prediction and simulation now work when random-effects
terms have data-dependent bases (e.g., poly(.)
or
ns(.)
terms) (Github #313, Edgar Gonzalez)
logLik
for glmer.nb
models now
includes the overdispersion parameter in the
parameter count (df
attribute)
lmList
handles offsets and weights better
lots of fixes to glmer.nb
(Github #176, #266, #287,
#318). Please note that glmer.nb is still somewhat
unstable/under construction.
import functions from base packages to pass CRAN checks
tweak to failing tests on Windows
getME
gains a "Tlist"
option
(returns a vector of template matrices from which the blocks of
Lambda
are generated)
hatvalues
method returns the diagonal of the hat
matrix of LMMs
nlminbwrap
convenience function allows use of
nlminb
without going through the optimx
package
as.data.frame.VarCorr.merMod
gains an order
option that allows the results to be sorted with variances first
and covariances last (default) or in lower-triangle order
allow more flexibility in scales
for
xyplot.thpr
method (John Maindonald)
models with only random effects of the form 1|f
have
better starting values for lmer
optimization (Gabor
Grothendieck)
glmer
now allows a logical vector as the response
for binomial models
anova
will now do (sequential) likelihood ratio
tests for two or more models including both merMod
and
glm
or lm
models (at present, only for GLMMs
fitted with the Laplace approximation)
deviance()
now returns the deviance, rather than half the
negative log-likelihood, for GLMMs fitted with Laplace
(the behaviour for LMMs and GLMMs fitted with nAGQ>1
has
not changed)
convergence warning and diagnostic test issues are now
reported in print
and summary
methods
update
now (attempts to) re-evaluate the original fit
in the environment of its formula (as is done with drop1
)
refit
of a nonlinear mixed model fit now throws an
error, but this will hopefully change in future releases (related
to bug fixes for Github #231)
lmList
now returns objects of class lmList4
,
to avoid overwriting lmList
methods from the recommended
nlme
package
names of random effects parameters in confint
changed (modified for consistency across methods);
oldNames=TRUE
(default) gives ".sig01"
-style names,
oldNames=FALSE
gives "sd_(Intercept)|Subject"
-style names
confint(.,method="Wald")
result now contains rows
for random effects parameters (values set to NA
)
as well as for fixed-effect parameters
simulate
and predict
now work more
consistently with different-length data, differing
factor levels, and NA
values (Github #153, #197, #246, #275)
refit
now works correctly for glmer
fits (Github #231)
fixed bug in family.merMod
; non-default links
were not retrieved correctly (Alessandro Moscatelli)
fixed bootMer
bug for type=="parametric"
,
use.u=TRUE
(Mark Lai)
gradient scaling for convergence checks now uses the Cholesky factor of the Hessian; while it is more correct, this will lead to some additional (probably false-positive) convergence warnings
As with lm()
, users now get an error for non-finite
(Inf
, NA
, or NaN
)
values in the response unless na.action
is set to
exclude or omit them (Github #310)
the nloptr
package is now imported;
a wrapper function (nloptwrap
) is provided so that
lmerControl(optimizer="nloptwrap")
is all that's necessary
to use nloptr
optimizers in the nonlinear optimization
stage (the default algorithm is NLopt's implementation
of BOBYQA: see ?nloptwrap
for examples)
preliminary implementation of checks for scaling
of model matrix columns (see check.scaleX
in
?lmerControl
)
beta
is now allowed as a synonym for fixef
when specifying starting parameters (Github #194)
the use of deviance
to return the REML criterion
is now deprecated; users should use REMLcrit()
instead
(Github #211)
changed the default value of check.nobs.vs.rankZ
to
"ignore"
(Github #214)
change gradient testing from absolute to relative
fix confint(.,method="boot")
to allow/work
properly with boot.type
values other than "perc"
(reported by Alan Zaslavsky)
allow plot()
to work when data are specified in a different
environment (reported by Dieter Menne)
predict
and simulate
work for matrix-valued
predictors (Github #201)
other simulate
bugs (Github #212)
predict
no longer warns spuriously when
original response was a factor (Github #205)
fix memory access issues (Github #200)
This version incorporates no changes in functionality, just modifications to testing and dependencies for CRAN/backward compatibility.
change drop1
example to prevent use of old/incompatible
pbkrtest
versions, for 2.15.3 compatibility
explicitly require(mlmRev)
for tests to prevent cyclic
dependency
bump RcppEigen
Imports: requirement from >0.3.1.2.3 to
>=0.3.2.0; Rcpp
dependency to >= 0.10.5
improved NA handling in simulate
and refit
made internal handling of weights
/offset
arguments slightly more robust (Github #191)
handle non-positive-definite estimated fixed effect
variance-covariance matrices slightly more generally/robustly
(fall back on RX approximation, with a warning,
if finite-difference Hessian
is non-PD; return NA
matrix if RX approximation is
also bad)
Added output specifying when Gauss-Hermite quadrature was used to fit the model, and specifying number of GHQ points (Github #190)
Models with prior weights returned an incorrect sigma and deviance (Github issue #155). The deviance bug was only a practical issue in model comparisons, not with inferences given a particular model. Both bugs are now fixed.
Profiling failed in some cases for models with vector random effects (Github issue #172)
Standard errors of fixed effects are now computed
from the approximate Hessian by default (see the
use.hessian
argument in vcov.merMod
); this
gives better (correct) answers when the estimates of
the random- and fixed-effect parameters are correlated
(Github #47)
The default optimizer for lmer
fits has been
switched from "Nelder_Mead" to "bobyqa" because we have
generally found the latter to be more reliable. To switch
back to the old behaviour,
use control=lmerControl(optimizer="Nelder_Mead")
.
Better handling of rank-deficient/overparameterized
fixed-effect model matrices; see check.rankX
option
to [g]lmerControl
. The default value is
"message+drop.cols", which automatically drops redundant
columns and issues a message (not a warning). (Github #144)
slight changes in convergence checking; tolerances can
be specified where appropriate, and some default tolerances
have changed (e.g., check.conv.grad
)
improved warning messages about rank-deficiency in X and Z etc. (warnings now try to indicate whether the unidentifiability is in the fixed- or random-effects part of the model)
predict
and simulate
now prefer
re.form
as the argument to specify which random effects
to condition on, but allow ReForm
, REForm
, or
REform
, giving a message (not a warning) that they are
deprecated (addresses Github #170)
small fixes for printing consistency in models with no fixed effects
we previously exported a fortify
function identical
to the one found in ggplot2
in order to be able to define a
fortify.merMod
S3 method without inducing a dependency on
ggplot2
. This has now been unexported to avoid masking
ggplot2
's own fortify
methods; if you want to
add diagnostic information to the results of a model, use
fortify.merMod
explicitly.
simulate.formula
now checks for names associated
with the theta
and beta
parameter vectors. If
missing, it prints a message (not a warning); otherwise, it
re-orders the parameter vectors to match the internal
representation.
preliminary implementation of a check.scaleX
argument
in [g]lmerControl
that warns about scaling if some columns
of the fixed-effect model matrix have large standard
deviations (relative to 1, or to each other)
The gradient and Hessian are now computed via finite
differencing after the nonlinear fit is done, and the results
are used for additional convergence tests. Control of the
behaviour is available through the check.conv.*
options
in [g]lmerControl
. Singular fits (fits with estimated
variances of zero or correlations of +/- 1) can also be tested for,
although the current default value of the check.conv.singular
option is "ignore"
; this may be changed to "warning"
in the future. The results are stored in @optinfo$derivs
.
(Github issue #120; based on code by Rune Christensen.)
The simulate
method will now work to generate
simulations "from scratch" by providing a model formula,
a data frame holding the predictor variables, and a list
containing the values of the model parameters:
see ?simulate.merMod
. (Github issue #115)
VarCorr.merMod
objects now have an as.data.frame
method, converting the list of matrices to a more
convenient form for reporting and post-processing. (Github issue #129)
results of fitted()
, predict()
,
and residuals()
now have
names in all cases (previously results were unnamed, or
named only when predicting from new data)
the anova
method now has a refit
argument
that controls whether objects of class lmerMod
should be
refitted with ML before producing the anova
table.
(Github issues #141, #165; contributed by Henrik Singmann.)
the print
method for VarCorr
objects
now has a formatter
argument for finer control
of standard deviation and variance formats
the optinfo
slot now stores slightly more
information, including the number of function evaluations
($feval
).
dotplot.ranef.mer
now adds titles to sub-plots by default,
like qqmath.ranef.mer
fitted
now respects na.action
settings (Github
issue #149)
confint(.,method="boot")
now works when there are
NA
values in the original data set (Github issue #158)
previously, the code stored the results (parameter values, residuals, etc.) based on the last set of parameters evaluated, rather than the optimal parameters. These were not always the same, but were almost always very close, but some previous results will change slightly (Github issue #166)
when using the default method="profile"
,
confint
now returns appropriate upper/lower bounds
(-1/1 for correlations, 0/Inf for standard deviations)
rather than NA
when appropriate
in a previous development version, ranef
returned
incorrect conditional variances (github issue #148). this is
now fixed
prediction now works when new data have fewer factor levels than are present in the original data (Github issue #143, reported by Rune Haubo)
the existence of a variable "new" in the global environment
would mess lme4
up: reported at
http://stackoverflow.com/questions/19801070/error-message-glmer-using-r-what-must-be-a-character-string-or-a-function
confint.merMod
and vcov.merMod
are
now exported, for downstream package-author convenience
the package now depends on Matrix >=1.1-0 and RcppEigen >=0.3.1.2.3
new rename.response
option for refit
(see BUG
FIXES section)
eliminated redundant messages about suppressed fixed-effect correlation matrices when p>20
most inverse-link functions are now bounded where
appropriate by .Machine$double.eps
, allowing fitting
of GLMMs with extreme parameter values
merMod
objects created with refit
did not
work with update
: optional
rename.response
option added to refit.merMod
, to allow
this (but the default is still FALSE
, for
back-compatibility) (reported by A. Kuznetsova)
fixed buglet preventing on-the-fly creation of index variables,
e.g. y~1+(1|rownames(data))
(reported by J. Dushoff)
predict
now works properly for glmer
models
with basis-creating terms (e.g. poly
, ns
)
step sizes determined from fixed effect coefficient standard
errors after first state of glmer
fitting are now bounded,
allowing some additional models to be fitted
refit()
now works, again, with lists of
length 1, so that e.g. refit(.,simulate(.))
works.
(Reported by Gustaf Granath)
getME(.,"ST")
was returning a list
containing the Cholesky factorizations that get repeated in
Lambda. But this was inconsistent with what ST
represents in
lme4.0
. This inconsistency has now been fixed and
getME(.,"ST")
is now consistent with the definition of the
ST
matrix in lme4.0
. See
https://github.com/lme4/lme4/issues/111
for more
detail. Thanks to Vince Dorie.
Corrected order of unpacking of standard
deviation/correlation components, which affected results
from confint(.,method="boot")
. (Reported by Reinhold
Kliegl)
fixed a copying bug that made refitML()
modify the original model
check.numobs.*
and check.numlev.*
in
(g)lmerControl
have been changed (from recent development
versions) to check.nobs.*
and
check.nlev.*
respectively, and the default values of
check.nlev.gtreq.5
and check.nobs.vs.rankZ
have been changed to "ignore"
and "warningSmall"
respectively
in (g)lmerControl
, arguments to the optimizer
should be passed as a list called optCtrl
, rather than
specified as additional (ungrouped) arguments
the postVar
argument to ranef
has been
changed to the (more sensible) condVar
("posterior variance"
was a misnomer, "conditional variance" – short for "variance of the
conditional mode" – is preferred)
the REform
argument to predict
has been changed
to ReForm
for consistency
the tnames
function, briefly exported, has been
unexported
getME(.,"cnms")
added
print
method for merMod
objects is now more
terse, and different from summary.merMod
the objective
method for the respMod
reference class now takes an optional sigma.sq
parameter
(defaulting to NULL
) to allow calculation of the
objective function with a residual variance different from
the profiled value (Vince Dorie)
Because the internal computational machinery has changed,
results from the newest version of lme4
will not be numerically
identical to those from previous versions. For reasonably well-
defined fits, they will be extremely close (within numerical
tolerances of 1e-4 or so), but for unstable or poorly-defined fits
the results may change, and very unstable fits may fail when they
(apparently) succeeded with previous versions. Similarly, some fits
may be slower with the new version, although on average the new
version should be faster and more stable. More numerical
tuning options are now available (see below); non-default settings
may restore the speed and/or ability to fit a particular model without
an error. If you notice significant or disturbing changes when fitting
a model with the new version of lme4
, please notify the maintainers.
VarCorr
returns its results in the same format as before (as a
list of variance-covariance matrices with correlation
and stddev
attributes, plus a sc
attribute giving the residual standard
deviation/scale parameter when appropriate), but prints them in a
different (nicer) way.
By default residuals
gives deviance (rather than Pearson)
residuals when applied to glmer
fits (a side effect of matching glm
behaviour more closely).
As another side effect of matching glm
behaviour, reported log-likelihoods from glmer
models
are no longer consistent with those from pre-1.0 lme4
,
but are consistent with glm
; see glmer
examples.
More use is made of S3 rather than S4 classes and methods: one
side effect is that the nlme
and lme4
packages are now much more
compatible; methods such as fixef
no longer conflict.
The internal optimizer has changed. [gn]lmer
now has an
optimizer
argument; "Nelder_Mead"
is the default for [n]lmer
,
while a combination of "bobyqa"
(an alternative derivative-free
method) and "Nelder_Mead"
is the default for glmer
. To use the
nlminb
optimizer as in the old version of lme4
, you can use
optimizer="optimx"
with control=list(method="nlminb")
(you will
need the optimx
package to be installed and loaded). See
lmerControl
for details.
Families in GLMMs are no longer restricted to built-in/hard-
coded families; any family described in family
, or following that
design, is usable (although there are some hard-coded families, which
will be faster).
[gn]lmer
now produces objects of class merMod
rather than
class mer
as before.
the structure of the Zt
(transposed random effect
design matrix) as returned by getME(.,"Zt")
, and the
corresponding order of the random effects vector
(getME(.,"u")
) have changed. To retrieve Zt
in the old format, use do.call(Matrix::rBind,getME(.,"Ztlist"))
.
the package checks input more thoroughly for
non-identifiable or otherwise problematic cases: see
lmerControl
for fine control of the test behaviour.
A general-purpose getME
accessor method allows
extraction of a wide variety of components of a mixed-model
fit. getME
also allows a vector of objects to be returned as
a list of mixed-model components. This has been backported to
be compatible with older versions of lme4
that still produce mer
objects rather than merMod
objects. However, backporting is incomplete;
some objects are only extractable in newer versions of lme4
.
Optimization information (convergence codes, warnings, etc.)
is now stored in an @optinfo
slot.
bootMer
provides a framework for obtaining parameter confidence
intervals by parametric bootstrapping.
plot.merMod
provides diagnostic plotting
methods similar to those from the nlme
package
(although missing augPred
).
A predict.merMod
method gives predictions;
it allows an effect-specific choice of conditional prediction or prediction at the
population level (i.e., with random effects set to zero).
Likelihood profiling for lmer
and glmer
results (see
link{profile-methods}
).
Confidence intervals by likelihood profiling (default),
parametric bootstrap, or Wald approximation (fixed effects only):
see confint.merMod
nAGQ=0
, an option to do fast (but inaccurate) fitting of GLMMs.
Using devFunOnly=TRUE
allows the user to extract a deviance
function for the model, allowing further diagnostics/customization of
model results.
The internal structure of [gn]lmer is now more modular, allowing
finer control of the different steps of argument checking; construction
of design matrices and data structures; parameter estimation; and construction
of the final merMod
object (see ?modular
).
the formula
, model.frame
, and terms
methods return full versions (including random effect terms and
input variables) by default, but a fixed.only
argument
allows access to the fixed effect submodel.
glmer.nb
provides an embryonic negative
binomial fitting capability.
Adaptive Gaussian quadrature (AGQ) is not available for multiple and/or non-scalar random effects.
Posterior variances of conditional models for non-scalar random effects.
Standard errors for predict.merMod
results.
Automatic MCMC sampling based on the fit turns out to be very difficult
to implement in a way that is really broadly reliable and robust; mcmcsamp
will not be implemented in the near future. See
pvalues
for alternatives.
"R-side" structures (within-block correlation and heteroscedasticity) are not on the current timetable.
In a development version, prior weights were not being used properly in the calculation of the residual standard deviation, but this has been fixed. Thanks to Simon Wood for pointing this out.
In a development version, the step-halving component of the penalized iteratively reweighted least squares algorithm was not working, but this is now fixed.
In a development version, square RZX
matrices would lead to a
pwrssUpdate did not converge in 30 iterations
error. This has been fixed
by adding an extra column of zeros to RZX
.
Previous versions of lme4
provided
the mcmcsamp
function, which efficiently generated
a Markov chain Monte Carlo sample from the posterior
distribution of the parameters, assuming flat (scaled
likelihood) priors. Due to difficulty in constructing a
version of mcmcsamp
that was reliable even in
cases where the estimated random effect variances were
near zero (e.g.
https://stat.ethz.ch/pipermail/r-sig-mixed-models/2009q4/003115.html),
mcmcsamp
has been withdrawn (or more precisely,
not updated to work with lme4
versions >=1.0).
Calling glmer
with the default gaussian
family
redirects to lmer
, but this is deprecated
(in the future glmer(...,family="gaussian")
may
fit a LMM using the penalized iteratively reweighted least squares
algorithm). Please call lmer
directly.
Calling lmer
with a family
argument redirects
to glmer
; this is deprecated. Please call glmer
directly.
The underlying algorithms and representations for all the mixed-effects models fit by this package have changed - for the better, we hope. The class "mer" is a common mixed-effects model representation for linear, generalized linear, nonlinear and generalized nonlinear mixed-effects models.
ECME iterations are no longer used at all, nor are analytic gradients. Components named 'niterEM', 'EMverbose', or 'gradient' can be included in the 'control' argument to lmer(), glmer() or nlmer() but have no effect.
PQL iterations are no longer used in glmer() and nlmer(). Only the Laplace approximation is currently available. AGQ, for certain classes of GLMMs or NLMMs, is being added.
The 'method' argument to lmer(), glmer() or nlmer() is deprecated. Use the 'REML = FALSE' in lmer() to obtain ML estimates. Selection of AGQ in glmer() and nlmer() will be controlled by the argument 'nAGQ', when completed.
The representation of mixed-effects models has been dramatically changed to allow for smooth evaluation of the objective as the variance-covariance matrices for the random effects approach singularity. Beta testers found this representation to be more robust and usually faster than previous versions of lme4.
The mcmcsamp function uses a new sampling method for the variance-covariance parameters that allows recovery from singularity. The update is not based on a sample from the Wishart distribution. It uses a redundant parameter representation and a linear least squares update.
CAUTION: Currently the results from mcmcsamp look peculiar and are probably incorrect. I hope it is just a matter of my omitting a scaling factor but I have seen patterns such as the parameter estimate for some variance-covariance parameters being the maximum value in the chain, which is highly unlikely.
The 'verbose' argument to lmer(), glmer() and nlmer() can be used instead of 'control = list(msVerbose = TRUE)'.